Chris Wagner, Developer in Columbus, United States
Chris is available for hire
Hire Chris

Chris Wagner

Verified Expert  in Engineering

Web Scraping Developer

Location
Columbus, United States
Toptal Member Since
January 21, 2015

Chris has been writing code for more than 20 years. He's spent the last 15 years on applications ranging from search and data aggregation (scraping) to online scheduling and email marketing. His preferred development environment is in a type-safe, functional language like Scala or Haskell, but he's no stranger to Ruby, JavaScript, Python, and PHP. Eagerly waiting for Bitcoin to change the world, Chris is particularly interested in crypto projects.

Portfolio

Go-UPC
Scala, Java, JVM, PostgreSQL, API Design, NGINX, JavaScript, HTML, CSS...
Spendabit
PostgreSQL, Scalatra, Scala, API Development, Web Scraping, Data Scraping...
Salon Lofts
Amazon Web Services (AWS), Elasticsearch, MySQL, RSpec, Ruby on Rails (RoR)...

Experience

Availability

Full-time

Preferred Environment

Git, Linux, Scala, Java, Ruby, Agile, Test-driven Development (TDD), Continuous Development (CD), IntelliJ IDEA, Python

The most amazing...

...thing I've developed recently is a search engine for things you can buy with bitcoin.

Work Experience

Lead Software Engineer

2021 - PRESENT
Go-UPC
  • Developed an API for matching UPC and EAN numbers (i.e., barcodes) for the name, image, description, and other product information.
  • Aggregated product information from over 15,000 unique websites by large-scale web crawling and a proprietary AI product-detection algorithm.
  • Engineered a mechanism for locating products on the fly when the system could not find a matching product in its local database.
  • Designed and co-developed a complex product distillation algorithm to select the most relevant and highest quality product information from various sources.
Technologies: Scala, Java, JVM, PostgreSQL, API Design, NGINX, JavaScript, HTML, CSS, Test-driven Development (TDD), Git, API Development, Data Scraping, HTML5, SQL, HTTP, IntelliJ IDEA, User Interface (UI), User Experience (UX), Software Development, REST APIs, Web Scraping

CTO

2014 - 2020
Spendabit
  • Constructed a search engine with an index of millions of products from hundreds of independent data sources (online stores, mostly).
  • Developed a proprietary scraping engine that identifies and imports products from arbitrary eCommerce websites. Some call it "AI"—just point and fire.
  • Devised novel algorithms ("tricks") for improving search-result relevance beyond standard keyword matching methods.
  • Managed a test suite covering hundreds of use cases and edge cases across search, scraping, and basic app functionality.
  • Designed a responsive user interface with HTML5, CSS, Bootstrap, and a sprinkling of JavaScript.
  • Designed and implemented an API to enable third-party integrations with the search back end.
Technologies: PostgreSQL, Scalatra, Scala, API Development, Web Scraping, Data Scraping, Search, Java, JVM, Search Engines, JavaScript, HTML, CSS, Test-driven Development (TDD), Bitcoin, Cryptocurrency, ScalaTest, Git, API Design, HTML5, SQL, HTTP, IntelliJ IDEA, SBT, User Interface (UI), User Experience (UX), Software Development, REST APIs

Senior Software Engineer

2015 - 2019
Salon Lofts
  • Supported and extended an online-scheduling system used by thousands of stylists across the United States.
  • Dramatically expanded the application's test suite to include hundreds of "full-stack" test cases (tests running against a real web browser, interacting with the full application more or less as a real user would) and thousands of unit tests.
  • Developed a "waiting list" system that attempts to autofill schedule openings by interacting with clients via SMS when an opening arises (using a bit of "natural language processing" to read client text messages, etc.).
  • Built the interface and API integration to permit stylists to require a credit/debit card when booking an appointment.
  • Implemented a feature to allow stylists to enforce cancellation policies (requiring clients to provide a credit card when booking).
Technologies: Amazon Web Services (AWS), Elasticsearch, MySQL, RSpec, Ruby on Rails (RoR), Ruby, Darcs, Git, RubyMine, Buildkite, Travis CI, Scheduling, Credit Card Processing, Stripe, JavaScript, HTML, CSS, Test-driven Development (TDD), Amazon S3 (AWS S3), Software Development

Lead Developer

2014 - 2015
FreshAddress, LLC
  • Supported a second-hand Rails application with limited documentation.
  • Extended the Rails app to add new features ranging from enhanced search functionality to user interface improvements.
  • Reverse-engineered many app components for lack of solid documentation.
Technologies: Sphinx Search Engine, MySQL, Ruby on Rails (RoR), JavaScript, HTML, CSS, Git, Software Development

Chief Programmer and Technologist

2009 - 2014
DownsizeDC.org & Zero Aggression Project
  • Designed and implemented the "Educate the Powerful" System, a tool enabling constituents of the U.S. Congress to quickly and efficiently contact their representatives via automation of congressional web forms.
  • Developed a polling and petitioning application for ZeroAggressionProject.org using a Scala and Scalatra microframework.
  • Maintained several websites across several servers, including individual and shared test suites, to keep them ticking smoothly.
  • Developed custom mailing-list management software (as a replacement for the rickety phpList open-source tool) used to send regular mailings to a 30,000-subscriber mailing list, coded in Scala and leveraging the JavaMail stack.
  • Provided counsel on all of the organization's technical matters.
  • Delivered technical support to the organization's user base and the team.
Technologies: Linux, Apache Tomcat, MySQL, PostgreSQL, PHP, Java, Scala, JavaScript, HTML, CSS, SQL, IntelliJ IDEA, Software Development, Mailchimp

Senior Developer

2008 - 2009
Pubget
  • Held one of two development roles in getting the Pubget bio­-science search engine off the ground.
  • Led the development of a powerful screen-­scraping engine, enabling Pubget to aggregate articles from thousands of scientific journals and provide access to the latest journal articles days or weeks ahead of the then industry-­standard PubMed.
  • Leveraged a powerful Solr search server, distributed across many physical machines, to provide split-­second search across the content of millions of scientific articles.
  • Experienced a startup company going from the "out-of-the-garage" phase to becoming a "real" company with an office, proper staff, and the things that go along with that.
  • Spearheaded the company's automated testing efforts (unit tests, etc.).
Technologies: Apache Solr, MySQL, Ruby on Rails (RoR), JavaScript, HTML, CSS, Data Scraping, Software Development, MongoDB, Django, Web Scraping

Lead Developer

2005 - 2007
SCRIP-SAFE
  • Developed a Java/J2EE-­based web application to enable educational institutions to securely exchange confidential documents (namely, student transcripts).
  • Leveraged many agile (compared to other Java technologies, anyway) and open­-source technologies, including Hibernate, Spring, and Lucene.
  • Worked with non­-technical people within the organization and at institutions to understand the requirements and needs of those in the industry.
Technologies: Linux, Apache Lucene, Spring, Hibernate, Apache Struts, Java, JavaScript, HTML, CSS, Software Development

Spendabit

https://spendabit.co
Spendabit is a search engine for things you can buy with bitcoin. It searches across and aggregates products from thousands of merchants across hundreds of data sources (eCommerce websites, CSV data feeds, etc.).

Spendabit is a Scala-based application leveraging PostgreSQL on the back end for general data storage and search (using the PostgreSQL Full Text Search extensions) and several Java and Scala libraries (such as Slick for database access). It leverages state-of-the-art "scraping" (web crawling) technology to import, and keep up-to-date, the majority of its products.

DownsizeDC.org

https://downsizedc.org/
Downsizedc.org is an educational venue but also a platform for (primarily US-based) people to advocate the case for liberty on many issues.

The most significant component is the "Educate the Powerful" system that presents a straightforward user interface atop a powerful back end that automates the web-based contact forms of members of the US Congress, using a large set of rules and heuristics. It handles several hundred contact forms without the need for website-specific rules or coding and even supports captchas and other similar roadblocks.

(Unfortunately, the "Educate the Powerful" system is no longer live.)

Zero Aggression Project

The polling and petitioning application at Zeroaggressionproject.org aims to both recruit people to the position of the zero aggression principle (non-aggression principle) and offer a platform for recruitment and advocacy to existing adherents to the principle.

The application presents potential respondents with a series of panels positing facts, questions, hypothetical scenarios, and general arguments regarding the particular campaign (for example, the drug prohibition or immigration campaign). Responses are recorded in a database and re-presented to users in a visual fashion.

The back end is written in Scala, leveraging the Scalatra micro-framework, with PostgreSQL as the data store. The front end makes use of Bootstrap and jQuery.

BitcoinChipin.com

BitcoinChipin.com is a pet project but presents a clean, elegant user interface. It is a simple tool that provides users with a graphical widget, embeddable on their own website or blog, which they can use to encourage people to donate, via bitcoin, to a particular cause (e.g., "Help us feed the homeless").

BitcoinChipin.com is mostly PHP-based, with MySQL for the database and Bootstrap and jQuery for UI components. It uses the blockchain.info API for interacting with the bitcoin network (retrieving balances).

Languages

PHP, HTML5, Scala, Python, Java, Ruby, SQL, HTML, JavaScript, CSS

Paradigms

Agile Software Development, Test-driven Development (TDD), Unit Testing, Continuous Development (CD), Automated Testing

Platforms

Linux, JVM, Amazon Web Services (AWS), Buildkite

Other

HTTP, Agile Software Testing, Cryptocurrency, Web Scraping, Data Scraping, Software Development, Bitcoin, User Interface (UI), User Experience (UX), API Design, APIs, Search, Search Engines, Darcs, Scheduling, Credit Card Processing

Frameworks

Bootstrap, Scalatra, Ruby on Rails (RoR), Apache Struts, Hibernate, Spring, Play Framework, Flask, Django

Libraries/APIs

API Development, REST APIs, Apache Lucene, jQuery, Stripe

Tools

SBT, ScalaTest, Git, IntelliJ IDEA, Apache Tomcat, RubyMine, RSpec, Mailchimp, Apache Solr, NGINX, Travis CI

Storage

MySQL, PostgreSQL, Amazon S3 (AWS S3), Elasticsearch, MongoDB, Sphinx Search Engine

2001 - 2005

Bachelor of Engineering Degree in Computer Science

Ohio University - Athens, Ohio

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring