
Chris Wagner
Verified Expert in Engineering
Web Scraping Developer
Chris has been writing code for more than 20 years. He's spent the last 15 years on applications ranging from search and data aggregation (scraping) to online scheduling and email marketing. His preferred development environment is in a type-safe, functional language like Scala or Haskell, but he's no stranger to Ruby, JavaScript, Python, and PHP. Eagerly waiting for Bitcoin to change the world, Chris is particularly interested in crypto projects.
Portfolio
Experience
Availability
Preferred Environment
Git, Linux, Scala, Java, Ruby, Agile, Test-driven Development (TDD), Continuous Development (CD), IntelliJ, Python
The most amazing...
...thing I've developed recently is a search engine for things you can buy with bitcoin.
Work Experience
Lead Software Engineer
Go-UPC
- Developed an API for matching UPC and EAN numbers (i.e., barcodes) for the name, image, description, and other product information.
- Aggregated product information from over 15,000 unique websites by large-scale web crawling and a proprietary AI product-detection algorithm.
- Engineered a mechanism for locating products on the fly when the system could not find a matching product in its local database.
- Designed and co-developed a complex product distillation algorithm to select the most relevant and highest quality product information from various sources.
CTO
Spendabit
- Constructed a search engine with an index of millions of products from hundreds of independent data sources (online stores, mostly).
- Developed a proprietary scraping engine that identifies and imports products from arbitrary eCommerce websites. Some call it "AI"—just point and fire.
- Devised novel algorithms ("tricks") for improving search-result relevance beyond standard keyword matching methods.
- Managed a test suite covering hundreds of use cases and edge cases across search, scraping, and basic app functionality.
- Designed a responsive user interface with HTML5, CSS, Bootstrap, and a sprinkling of JavaScript.
- Designed and implemented an API to enable third-party integrations with the search back end.
Senior Software Engineer
Salon Lofts
- Supported and extended an online-scheduling system used by thousands of stylists across the United States.
- Dramatically expanded the application's test suite to include hundreds of "full-stack" test cases (tests running against a real web browser, interacting with the full application more or less as a real user would) and thousands of unit tests.
- Developed a "waiting list" system that attempts to autofill schedule openings by interacting with clients via SMS when an opening arises (using a bit of "natural language processing" to read client text messages, etc.).
- Built the interface and API integration to permit stylists to require a credit/debit card when booking an appointment.
- Implemented a feature to allow stylists to enforce cancellation policies (requiring clients to provide a credit card when booking).
Lead Developer
FreshAddress, LLC
- Supported a second-hand Rails application with limited documentation.
- Extended the Rails app to add new features ranging from enhanced search functionality to user interface improvements.
- Reverse-engineered many app components for lack of solid documentation.
Chief Programmer and Technologist
DownsizeDC.org & Zero Aggression Project
- Designed and implemented the "Educate the Powerful" System, a tool enabling constituents of the U.S. Congress to quickly and efficiently contact their representatives via automation of congressional web forms.
- Developed a polling and petitioning application for ZeroAggressionProject.org using a Scala and Scalatra microframework.
- Maintained several websites across several servers, including individual and shared test suites, to keep them ticking smoothly.
- Developed custom mailing-list management software (as a replacement for the rickety phpList open-source tool) used to send regular mailings to a 30,000-subscriber mailing list, coded in Scala and leveraging the JavaMail stack.
- Provided counsel on all of the organization's technical matters.
- Delivered technical support to the organization's user base and the team.
Senior Developer
Pubget
- Held one of two development roles in getting the Pubget bio-science search engine off the ground.
- Led the development of a powerful screen-scraping engine, enabling Pubget to aggregate articles from thousands of scientific journals and provide access to the latest journal articles days or weeks ahead of the then industry-standard PubMed.
- Leveraged a powerful Solr search server, distributed across many physical machines, to provide split-second search across the content of millions of scientific articles.
- Experienced a startup company going from the "out-of-the-garage" phase to becoming a "real" company with an office, proper staff, and the things that go along with that.
- Spearheaded the company's automated testing efforts (unit tests, etc.).
Lead Developer
SCRIP-SAFE
- Developed a Java/J2EE-based web application to enable educational institutions to securely exchange confidential documents (namely, student transcripts).
- Leveraged many agile (compared to other Java technologies, anyway) and open-source technologies, including Hibernate, Spring, and Lucene.
- Worked with non-technical people within the organization and at institutions to understand the requirements and needs of those in the industry.
Experience
Spendabit
https://spendabit.coSpendabit is a Scala-based application leveraging PostgreSQL on the back end for general data storage and search (using the PostgreSQL Full Text Search extensions) and several Java and Scala libraries (such as Slick for database access). It leverages state-of-the-art "scraping" (web crawling) technology to import, and keep up-to-date, the majority of its products.
DownsizeDC.org
https://downsizedc.org/The most significant component is the "Educate the Powerful" system that presents a straightforward user interface atop a powerful back end that automates the web-based contact forms of members of the US Congress, using a large set of rules and heuristics. It handles several hundred contact forms without the need for website-specific rules or coding and even supports captchas and other similar roadblocks.
(Unfortunately, the "Educate the Powerful" system is no longer live.)
Zero Aggression Project
The application presents potential respondents with a series of panels positing facts, questions, hypothetical scenarios, and general arguments regarding the particular campaign (for example, the drug prohibition or immigration campaign). Responses are recorded in a database and re-presented to users in a visual fashion.
The back end is written in Scala, leveraging the Scalatra micro-framework, with PostgreSQL as the data store. The front end makes use of Bootstrap and jQuery.
BitcoinChipin.com
BitcoinChipin.com is mostly PHP-based, with MySQL for the database and Bootstrap and jQuery for UI components. It uses the blockchain.info API for interacting with the bitcoin network (retrieving balances).
Skills
Languages
PHP, HTML5, Scala, Python, Java, Ruby, SQL, HTML, JavaScript, CSS
Paradigms
Agile Software Development, Test-driven Development (TDD), Unit Testing, Continuous Development (CD), Automated Testing
Platforms
Linux, JVM, Amazon Web Services (AWS), Buildkite
Other
HTTP, Agile Software Testing, Cryptocurrency, Web Scraping, Data Scraping, Software Development, Bitcoin, User Interface (UI), User Experience (UX), API Design, APIs, Search, Search Engines, Darcs, Scheduling, Credit Card Processing
Frameworks
Bootstrap, Scalatra, Ruby on Rails (RoR), Apache Struts, Hibernate, Spring, Play Framework, Flask, Django
Libraries/APIs
API Development, REST APIs, Apache Lucene, jQuery, Stripe
Tools
SBT, ScalaTest, IntelliJ IDEA, Git, IntelliJ, Apache Tomcat, RubyMine, RSpec, Mailchimp, Apache Solr, NGINX, Travis CI
Storage
MySQL, PostgreSQL, Amazon S3 (AWS S3), Elasticsearch, MongoDB, Sphinx Search Engine
Education
Bachelor of Engineering Degree in Computer Science
Ohio University - Athens, Ohio