Mikhail is available for hire
Hire MikhailMikhail Lyukmanov
Verified Expert in Engineering
Full-stack Developer
Location
Los Angeles, United States
Toptal Member Since
June 18, 2020
Mikhail is an ad-tech, data, and analytics professional with over 15 years of experience building and maintaining data lakes, data warehouses, DMPs, and ETL pipelines. He implements a simple minimalist approach to solving complex business and technical challenges. Mikhail is also a big fan of open-source software.
Portfolio
Amobee
Amazon Web Services (AWS), Git, Kubernetes, Scala, Python, Docker, PostgreSQL...
Freelance Software Development
Git, Apache Tomcat, Apache HTTP Server, Ajax, jQuery, C#, MySQL, PostgreSQL...
Adconion Media Group
Git, Apache Pig, Apache Hive, Python, Microsoft SQL Server, PostgreSQL, HBase...
Experience
Availability
Part-time
Preferred Environment
Amazon Web Services (AWS), Linux, Data Build Tool (dbt)
The most amazing...
...engineering thing I've done was to architect and build a scalable data management platform that hosted over 6 billion ad profiles for worldwide customers.
Work Experience
Vice President of Software Engineering, Data
2014 - PRESENT
Amobee
- Worked as head of the Amobee global data engineering team.
- Built world-class scalable data solutions for internal and third-party clients. For example, a scalable distributed DMP (data management platform) was built on an open-source stack that supported near real-time data processing. The solution was tested for up to 10 billion user profiles and terabytes of data streaming through the platform.
- Developed and maintained an existing data warehouse (PostgreSQL) and ETL pipelines (Hadoop, Spark, Scala, and MapReduce).
- Built and supported BI solutions (Microstrategy, Tableau, Metabase, Re:dash).
- Integrated AWS Cloud and developed apps (S3, Lambda, Redshift, and Kinesis).
Technologies: Amazon Web Services (AWS), Git, Kubernetes, Scala, Python, Docker, PostgreSQL, Spark, HBase, Cassandra, Hadoop, Big Data, Data Strategy, Technology Strategy & Architecture, Apache Airflow
Engineer
2005 - PRESENT
Freelance Software Development
- Developed web applications for various customers.
- Built an education planner for Transamerica Retirement Division, Ta-retirement.com/sip/shared/educationplanner/planner.aspx; used C#, responsive web design, jQuery, CSS, JSON, and iTextSharp.
- Developed a marketing planner for Transamerica; used C#, responsive web design, jQuery, CSS, and iTextSharp.
Technologies: Git, Apache Tomcat, Apache HTTP Server, Ajax, jQuery, C#, MySQL, PostgreSQL, Big Data, Technology Strategy & Architecture
Vice President of Data Engineering
2013 - 2014
Adconion Media Group
- Facilitated the successful implementation of the entire new internal business intelligence/data warehouse platform; it has more than 300+ worldwide users and is used in more than 15 countries.
Technologies: Git, Apache Pig, Apache Hive, Python, Microsoft SQL Server, PostgreSQL, HBase, Hadoop, Big Data, Data Strategy, Technology Strategy & Architecture
Senior Manager Data Warehouse and Business Intelligence
2010 - 2013
Adconion Media Group
- Created specs and helped with the hardware provisioning for a new database (PostgreSQL) and Microstrategy servers.
- Architected data marts and a BI domain model.
- Designed and provided lifecycle support of the existing enterprise data warehouse (PostgreSQL) and a number of reporting platforms (Jasper BI, Microstrategy, and Tableau).
- Handled the release management of the code—implementing best practices and coding standards.
Technologies: Cron, Bash, Python, PL/SQL, SQL, MySQL, PostgreSQL, Data Strategy, Technology Strategy & Architecture
Manager, Data Services
2006 - 2010
Edmunds.com
- Spearheaded the implementation and installment of a Hadoop cluster in Edmunds enterprise.
- Utilized Hadoop cluster and its MapReduce functionality which led to the doubling the performance of our daily data warehouse weblog loads.
Technologies: Apache Hive, Apache Pig, MapReduce, Hadoop, Informatica, PL/SQL, SQL, Oracle, Data Strategy, Technology Strategy & Architecture
Senior Data Architect
2006 - 2008
Edmunds.com
- Established from the ground up the development of a universal vehicle database with workflow management, a centralized dealer database, and a centralized geographic database.
- Evangelized and implemented a database release paradigm. It allowed us to keep track of all the changes as well as streamlined the release process.
Technologies: Scripting, Shell, Linux, Oracle Application Express (APEX), ETL, PL/SQL, Hadoop, Netezza, Informatica, Oracle
Senior Software Developer
2003 - 2005
Cymetrix
- Developed and administered the corporate database.
- Wrote up the financial reporting.
- Developed intranet applications (timesheets application, project management application).
- Implemented data mining.
Technologies: SQL Server Reporting Services (SSRS), ActiveReports, VorteXML, IIS, ASP.NET, C#.NET, SQL Server 2005, SQL Server 2000
Tech Lead Software Engineering
2001 - 2003
Arthur Andersen, LLP
- Designed and implemented Atlas (Assignment Tax calcuLAtion System): a multi-user web-based application based on MS SQL 2000 (later moved to Oracle 8i), IIS/ASP and with Microsoft Office XP web components. It was developed for Arthur Andersen and supports tax calculations for more than 84 countries.
Technologies: XML, DTS, PL/SQL, VBScript, JavaScript, ASP, Oracle8i, SQL Server 2000, Data Strategy
Experience
Content.ad Snowflake Data Pipelines
Using Snowflake and AWS DMS, I built and enabled real-time data pipelines connecting Kafka, AWS S3, AWS RDS, AWS DynamoDB, and Snowflake data warehouse.
It resulted in near-real-time CDC(change data capture) replication using snow pipes, streams, tasks, and other latest Snowflake technology.
Technologies: AWS, Python, Bash, Bitbucket, Snowflake
It resulted in near-real-time CDC(change data capture) replication using snow pipes, streams, tasks, and other latest Snowflake technology.
Technologies: AWS, Python, Bash, Bitbucket, Snowflake
Amobee DMP
A global distributed scalable application which included:
• a data lake
• a data warehouse
• API integrations with multiple third-party customers
• an analytical suite
Technologies: CDH (Cloudera Distributed Hadoop), Spark, Scala, Python, HBase, Oozie, Git, PostgreSQL, Cassandra, AWS EMR, AWS Lambda, AWS Redshift, AWS Gateway, AWS Kinesis, AWS S3, Glacier, Tableau, and Metabase
• a data lake
• a data warehouse
• API integrations with multiple third-party customers
• an analytical suite
Technologies: CDH (Cloudera Distributed Hadoop), Spark, Scala, Python, HBase, Oozie, Git, PostgreSQL, Cassandra, AWS EMR, AWS Lambda, AWS Redshift, AWS Gateway, AWS Kinesis, AWS S3, Glacier, Tableau, and Metabase
Amobee Email Data Warehouse
I developed a data warehouse which was built on AWS stack and it was designed to:
• Manage user profiles
• Manage email events (opens, clicks, unsubscribes)
• Run custom reporting requests
• Ensure data security, protection, and monitored access
Technologies: AWS Redshift, AWS EMR, AWS S3, AWS Lambda
• Manage user profiles
• Manage email events (opens, clicks, unsubscribes)
• Run custom reporting requests
• Ensure data security, protection, and monitored access
Technologies: AWS Redshift, AWS EMR, AWS S3, AWS Lambda
Adconion Data Warehouse
I developed a data warehouse built on the PostgreSQL platform. It was designed to store and manage ad campaigns, ad performance, financials, and other KPIs.
Technologies: Python, PostgreSQL, CHD (Cloudera Distributed Hadoop)
Technologies: Python, PostgreSQL, CHD (Cloudera Distributed Hadoop)
Adconion BI Suite
An analytics suite, built on the Microstrategy platform that was deployed to a worldwide audience—where over 300 users in 15 countries were using the application.
Technologies: Microstrategy, Java, PostgreSQL
Technologies: Microstrategy, Java, PostgreSQL
Transamerica Educational Planner
An application for the Transamerica retirement division. It was designed to help plan the retirement education sessions for Transamerica professionals.
Skills
Languages
Snowflake, Python, Bash Script, Bash, JavaScript, SQL, C#.NET, VBScript, XML, PHP, Scala, C#, T-SQL (Transact-SQL)
Tools
pgAdmin, Git, Amazon ElastiCache, Spree, Cron, ActiveReports, DTS, Apache HTTP Server, Apache Tomcat, Oracle Application Express (APEX), Shell, Oozie, AWS ELB, Apache Airflow
Paradigms
Agile Product Management, DevOps, REST, ETL, Automation, MapReduce
Platforms
AWS Lambda, Amazon Web Services (AWS), Amazon EC2, Docker, Kubernetes, Apache Pig, Oracle, Linux, Solidus, Databricks
Storage
Amazon S3 (AWS S3), HDFS, PostgreSQL, Amazon DynamoDB, Redshift, MySQL, Cassandra, Oracle PL/SQL, HBase, Apache Hive, PL/SQL, SQL Server 2000, SQL Server 2005, SQL Server Reporting Services (SSRS), Netezza, Microsoft SQL Server, Google Cloud
Other
Data Strategy, Technology Strategy & Architecture, Big Data, ELT, Document Management Systems (DMS), Web Development, AWS Auto Scaling, IT Management, Informatica, IIS, VorteXML, Oracle8i, Ajax, Scripting, Amazon API Gateway, Amazon Kinesis, Amazon Kinesis Data Firehose, Amazon Route 53, Analytics, Data Build Tool (dbt)
Frameworks
Hadoop, Spark, ASP.NET, ASP
Libraries/APIs
jQuery, React
Education
1991 - 1996
Master of Science Degree in Algorithms, Math, and Physics
Nizhny Novgorod State Technical University - Nizhny Novgorod, Russia