Artur Brugeman

Artur Brugeman

Chelyabinsk, Russia
Hire Artur
Scroll To View More
Artur Brugeman

Artur Brugeman

Chelyabinsk, Russia
Member since June 6, 2014
Artur is a software engineer with a proven ability to develop efficient, robust, and simple back-end solutions for dealing with large data sets. His current focus is on large-scale web crawling and processing of the crawled data.
Artur is now available for hire
Portfolio
  • MegaIndex.com
    C++11, Golang, MySQL, ZeroMQ, protobuf, libcurl, libcares
  • Rustoria.ru
    C, C++, Qt, OpenGL, NVIDIA Iray, NVIDIA Optix, SQLite, MySQL, PostgreSQL...
  • Papillon.ru
    C, C++, Linux, Qt, POSIX, multithreading, networking, ZeroMQ, PHP, XML, XSLT...
Experience
  • C++, 6 years
  • Webcrawling, 1 year
  • C, 7 years
  • Qt, 5 years
  • ZeroMQ, 3 years
  • Linux, 8 years
  • MySQL, 3 years
  • Go, 1 year
Chelyabinsk, Russia
Availability
Part-time
Preferred Environment
Linux, Emacs, GDB, Valgrind, Git
The most amazing...
...thing I've built is a web-crawler that scans billions of web pages daily.
Employment
  • Software Engineer and Architect
    MegaIndex.com
    2014 - PRESENT
    • Built a web-crawler that scans more than 3 billion pages daily, 100x improvement compared to previous system.
    • Implemented highly efficient HTML parser.
    • Built custom robust distributed data storage for inverse link graph, allowing for very high write/read throughput. Currently stores 200 billion unique links found on 30 billion crawled pages from 400 million websites.
    • Built an HTTP API to access link graph database.
    • Built a service to perform real-time ranked suggestions while users input their website url, performs up to 100K qps. Compact prefix arrays with pre-build result sets, multi-threaded reading and lock-free data structures for live updates.
    • Built a rating of web sites based on number of linking IP addresses, with filtering by user input. Compact suffix arrays for fast filtering, multi-threaded reading and lock-free data structures for live updates.
    • Re-built an API to serve reports on web site search engine rankings, implemented asynchronous MYSQL client library to perform concurrent requests to many servers.
    • Built a new storage for website search engine rankings, scaling up to 10 billion records with monthly history.
    • Built a service to aggregate various counters from web-crawler nodes, storing data on 10 billion domain names.
    • Built an HTTP API to access search engine rankings database.
    • All of the above were implemented with a focus on cost-efficiency: the HW costs to support all the mentioned services were negligible.
    Technologies: C++11, Golang, MySQL, ZeroMQ, protobuf, libcurl, libcares
  • Software Engineer
    Rustoria.ru
    2013 - 2014
    • Created a high quality rendering back-end for a 3D layer of a world map, using OpenGL, NVIDIA Iray, NVIDIA Optix.
    • Created a tool to preview 3D models and position them on a world map, using OpenGL and Qt.
    • Made supplementary tools to work with the OpenStreetMap data format (C++, XML).
    • Set up a 2D world map rendering back-end using Mapnik, pgSQL, mod_tile, Apache, and MapQuest styles.
    • Extended a PROJ.4 library to add support for isometric projection into Mapnik and more.
    Technologies: C, C++, Qt, OpenGL, NVIDIA Iray, NVIDIA Optix, SQLite, MySQL, PostgreSQL, Windows, Linux, OpenStreetMap, Mapnik, osm2pgsql, PROJ.4, XML, Apache
  • Software Engineer
    Papillon.ru
    2007 - 2013
    • Created a distributed cache for a proprietary NoSQL database to deliver data closer to processing nodes on big clusters (2013, ZeroMQ, C, C++).
    • Developed a distributed textual search engine for a proprietary NoSQL database, including highly scalable distributed sorting (2012-2013, ZeroMQ, ICU, C, C++).
    • Performed conversion of the national fingerprint database of Turkey from five different data formats. Set up a month-long project for automated processing of data on a 300 node cluster (2013, C, Oracle, MS SQL, XML, PHP).
    • Created a decision engine for the criminal division of the biometric passport system of Uzbekistan, integrating three biometric systems into a single solution (2011-2012, Oracle, C, distributed transactions).
    • Refactored a back-end component that aggregated fingerprint search results coming from cluster nodes. Reduced processing time by 10x. Documented the algorithms and wrote the code to be maintainable (2012, C).
    • Created a GUI application to manage data distribution between cluster nodes (2012, C++, Qt).
    • Created a GUI application to control AFIS search processes distributed between cluster nodes. (2012, C++, Qt).
    • Created a web application to train, test, and examine students studying the AFIS system (2010-2011, PHP, MySQL, Apache, JavaScript, jQuery).
    • Built a library to perform a multi-criterial evaluation of AFIS search results (2010, C).
    • Designed a database to perform multi-dimensional analysis of daily logs coming from hundreds of AFIS installations. Built an automatic process to import logs into the database (2009, MySQL, C).
    • Created an automated system to collect, extract, translate, and maintain documentation of a ~3M LOC codebase (2009, Doxygen, XSLT, XML, PHP, HTML, JavaScript, Apache).
    • Added support for customizable forms into a textual CRUD application. Refactored the code. Used static analysis to find and remove obsolete code paths (2008, C).
    • Created a viewer for various formats of biometric data, including NIST, EFTS, Papillon, and Interpol (2007, C++, Qt).
    Technologies: C, C++, Linux, Qt, POSIX, multithreading, networking, ZeroMQ, PHP, XML, XSLT, Doxygen, HTML, JavaScript, jQuery, ICU, Oracle, MySQL, MS SQL, BerkleyDB, AFIS, NIST, EFTS, image processing, fingerprints, face recognition, iris recognition, biometry
Experience
  • Sobnik (Development)

    Sobnik is an extension for Chrome browser that detects brokers on Russian real estate bulletin boards. It is actively used by thousands of users. Built using Chrome API, JavaScript, Golang, MongoDB, ZeroMQ, image processing.

Skills
  • Languages
    C++, C, JavaScript, SQL-99, PHP, Go
  • Misc
    Webcrawling
  • Frameworks
    Qt, AngularJS
  • Libraries/APIs
    SQLite, ZeroMQ
  • Tools
    Emacs, Valgrind, GDB
  • Platforms
    Linux, Windows
  • Storage
    MySQL, PostgreSQL, MongoDB, SQL Server, Oracle
  • Paradigms
    Concurrent Programming, Network programming
Education
  • Bachelor's degree in Information Technology
    South Ural State University - Chelyabinsk, Russia
    2003 - 2008
I really like this profile
Share it with others