Senior Data Engineer2020 - 2021Arthur Intelligence
- Designed and developed novel data pipelines and operations to collect data from heterogeneous databases running on third-party Windows servers.
- Designed and built self-updating persistent data extractors for Windows.
- Designed and built ETL and aggregation pipeline framework in Airflow on Kubernetes (GKE).
- Planned and executed the overall DevOps and automation.
Senior Data Engineer2020 - 2021Precisely
Technologies: Big Data, ETL Tools, ETL, Mapping, GIS, Amazon S3 (AWS S3), Amazon Web Services (AWS), Java 8, PostGIS, PostgreSQL, Jenkins, Jenkins Pipeline, Groovy
- Developed and enhanced ETL pipelines for mapping data from a variety of vendors.
- Investigated and fixed customer-reported issues with address data.
- Stabilized the existing pipelines, added monitoring and tests, and facilitated the knowledge transfer to the new team.
CTO2004 - 2021Visual Units
Technologies: Celery, Debian, Linux, Django, Memcached, MongoDB, MySQL, C, Python, Java ME
- Made a full-fledged fleet management system from the ground up.
- Took the product through (so far) three successful major database migrations without downtime.
- Developed a product and infrastructure with less than an hour of downtime over the last three years, despite 24/7 traffic.
- Created tools to generate Excel reports for customers on demand.
- Developed a dead-reckoning system to track rail car speeds.
- Ended up buying out the company and its products.
Senior Data Engineer2019 - 2020BCG GAMMA
Technologies: Amazon Web Services (AWS), Amazon S3 (AWS S3), Spark, PySpark, Boto 3, Apache Airflow, EMR, Python
- Worked closely with the team of data scientists, enabling quick turnaround time for personalized offers and measurements at a chain of luxury retail stores.
- Designed and implemented ETL pipelines, CI/CD, and automation for provisioning EMR PySpark clusters on demand for a personalization project.
Senior Software Engineer2018 - 2019Pitney Bowes
Technologies: Technical Leadership, Amazon Web Services (AWS), ETL, Amazon Virtual Private Cloud (VPC), Amazon ECS (Amazon Elastic Container Service), Amazon EC2, Apache Airflow, Terraform, Python, Pipelines, Jenkins, Groovy
- Reviewed and audited a 35 kLOC ETL implementation for geographical data processing.
- Helped to design and implement a new platform for ETL processing and normalization of geographical data from a variety of vendors.
- Created new environments for a complex set of ETL pipelines on AWS, including securing and arranging access to new VPCs and existing VPNs.
- Created the new ETL pipeline for one vendor of geographical data, loading and transforming large amounts of data, testing the resulting output, and producing intermediate and redistributable datasets.
- Created common/reusable stages for automating the database (SQL Server) loading/offloading using S3, bulk copies, and compression on EC2.
Back-end/CloudHSM Developer2018 - 2018Sodlab
Technologies: Compliance, Amazon Web Services (AWS), PDF, PDFBox, Java 8, Dropwizard, Amazon EC2, Amazon CloudHSM
- Built HSM-backed digital signatures for PDF from the ground up.
- Set up automated provisioning of CloudHSM identities.
- Integrated digital signatures and visual elements in PDF design.
- Set up a management and transaction API for the digital signing of legal documents using Dropwizard.
- Implemented synchronous and asynchronous workflows for signing using single or two-factor authorization.
Senior Consultant2017 - 2018Meniga
Technologies: Leadership, Compliance, Technical Leadership, AWS Kinesis, Periscope, Redshift, PostgreSQL, AWS Systems Manager, Amazon ECS (Amazon Elastic Container Service), AWS Lambda, Python, Go
- Planned, designed, and implemented real-time streaming ETL solutions using an AWS stack, lowering information lag from six hours to one minute.
- Coached technical leadership. Provided feedback and evaluations on the tech stack and work methods.
- Implemented new serverless services in an existing microservice architecture in Python and Go.
- Supported the sales organization with improved data modeling and performance improvements in the OLAP (Redshift) data store.
- Planned and set up CI integration with AWS Codebuild.
- Developed tools for identifying and analyzing performance issues in the analytics pipeline.
- Involved in planning and management of daily team activities for development teams.
Senior Cloud Architect2017 - 2017QuantumBlack
Technologies: Compliance, SAML, Angular, Amazon CloudHSM, Beanstalk, Terraform, Ansible, PostgreSQL, ECS, Amazon EC2, Dropwizard, Java 8
- Built a secure data ingestion system for sensitive data.
- Used S3, KMS, and CloudHSM to secure data and keys.
- Built safe provisioning where operations need to know no secrets.
- Built a Dropwizard/Angular/PostgreSQL application from scratch, including design, implementation, operations, and documentation.
- Integrated with multiple third-party authentication providers using SAMLv2 SP- and IdP-initiated authentication.
Senior Developer2016 - 2017Assa Abloy Mobile Services
Technologies: Amazon Web Services (AWS), Linux Luna Client, Amazon CloudHSM, VPN, Relational Database Services (RDS), Amazon EC2, Ansible, Java 8
- Developed in Java 8 EE7 on JBoss/Wildfly and Dropwizard.
- Planned and implemented AWS migration and automation.
- Wrote custom Ansible modules for securely deploying secrets to AWS using Amazon KMS and S3.
- Fully automated deployments of CloudHSM clients.
- Planned and implemented zero-downtime deployments.
Tech Lead2015 - 2016DOOER
Technologies: Leadership, Apache Airflow, ETL, Technical Leadership, Amazon Web Services (AWS), Flask, Ansible, Node.js, Python
- Planned and helped implement a new platform for automated bookkeeping.
- Assisted in building a development team from scratch.
- Planned and executed automation of deployments and provisioning for the new platform.
- Planned and set up data pipelines for accounting events feeding into accounting software and ML solutions.
- Managed teams' day-to-day work and priorities in close cooperation with the CEO and the head of engineering.
Ops Consultant2015 - 2015Wondersign
Technologies: Amazon Web Services (AWS), Linux, PHP
- Helped migrate a product from physical servers to AWS.
- Did load tests and advised on provisioning.
- Together with others, performed final migration to the cloud.
Technical Lead2014 - 2015Registry of Scotland
Technologies: Leadership, Compliance, Web, Technical Leadership, Celery, RabbitMQ, RHEL, AngularJS, Flask, Python, MongoDB
- Worked as the technical lead of the Land Registry Act project.
- Worked to create modern web services for solicitors at the Registry of Scotland's Land Register.
- Deployed a fully redundant system of web applications, interfacing with new and legacy systems.
- Created a distributed logging and tracing system using Python logging and MongoDB.
- Supported releases and operations for an entirely new technology stack at the Registry of Scotland.
- Led the daily work of two development teams in cooperation with the Scrum Master, product owners, and other stakeholders.
Senior Developer2013 - 2014Ericsson
Technologies: Jython, Python, Java
- Built a new framework for non-functional testing of large distributed systems.
- Created data bridges between Java and Python.
- Enabled visualization of complex and heterogeneous test setups.
Consultant2005 - 2013Ericsson
Technologies: Technical Leadership, Provisioning Tools, Online Charging Systems (OCS), Telecommunications, Integrated Database Management Systems (IDMS), ETL, MySQL, MongoDB, C++, C, .NET, Python, Java EE, Java SE, Java ME
- Worked as the design team lead for a charging system node.
- Developed a custom business intelligence tool for site managers integrated with the customer's ERP system.
- Worked in a small team to create five complete and separate sales channels on a Django/AngularJS platform in six weeks.
- Led the modernization of a developer infrastructure, SCM tools, and source control for a customer site with 150 developers.
- Developed a non-functional test framework for telecom nodes from the ground up.
- Held courses and workshops in Python, Git, and Mercurial.