Andrew is available for hire

Andrew Kostan

Verified Expert in Engineering

Data Engineer and Developer

Nagoya, Aichi Prefecture, Japan

Toptal member since May 30, 2024

Expertise

SQL Python Agile Development Amazon S3 Postman Apache Airflow Hadoop AWS IAM Business Intelligence Development ETL TypeScript MongoDB MySQL

Bio

Andrew is a data and back-end engineer with a bachelor's degree in computer science. He has extensive experience building data lakes, ETL solutions, and data pipelines. Andrew is currently working in a back-end engineer role requiring TypeScript expertise, but he's also well-versed in managing web visualization tools, including Metabase and Superset.

Portfolio

FUTU-RE

TypeScript, Docker, RPC, PostgreSQL

Kasugai

VMware ESXi, Docker, TypeScript, ArangoDB, MikroTik, NGINX, IP Networks, GitHub...

AwanTunai

Python 3, Presto, Apache Airflow, AWS IAM, Metabase, Apache Superset, Superset...

Experience

Python 3 - 7 years
SQL - 6 years
Agile Software Development - 6 years
Presto - 4 years
Amazon S3 (AWS S3) - 3 years
Apache Airflow - 2 years
Hadoop - 2 years
Apache Superset - 1 year

Availability

Part-time

Preferred Environment

Python 3, SQL, AWS IAM, Metabase, Superset, ETL, TypeScript

The most amazing...

...jobs and projects I've undertaken involve data engineering, which has been my main focus area for the past seven years.

Work Experience

Software Engineer

2024 - PRESENT

FUTU-RE

Developed and designed back-end flows for web service applications.
Designed back-end RPC using TypeScript (Nx React) for web services.
Implemented PostgreSQL databases for back-end deployment.
Utilized Docker for test and production deployment.

Technologies: TypeScript, Docker, RPC, PostgreSQL

Software Engineer

2023 - 2023

Kasugai

Developed and designed back-end flows for web service applications.
Managed the infrastructure for back-end testing and production environments on VMware.
Implemented databases for back-end deployment, including MySQL and ArangoDB.
Designed back-end APIs for web services using Python.
Configured and managed NGINX web servers and Docker containers for high-performance web hosting, load balancing, containerization, and optimizing web application delivery.
Designed and implemented an efficient CI/CD pipeline using GitHub Actions, build, and deployment processes for software projects. Streamlined development workflows, ensured code quality, and facilitated rapid application delivery.

Technologies: VMware ESXi, Docker, TypeScript, ArangoDB, MikroTik, NGINX, IP Networks, GitHub, MinIO

Data Engineer

2018 - 2023

AwanTunai

Designed and implemented a data lake using the Presto query engine.
Created and designed an ETL data pipeline, leveraging MongoDB and MySQL.
Managed data processes and object data storage in systems, including Amazon S3.
Scraped data from websites using Selenium WebDriver.
Implemented data visualization and interpreted data analytics results using Superset and Metabase as a platform.
Managed the data process to object data storage like S3 and OSS.
Collaborated with multiple and diverse team members (product, business development, and finance) from other departments.
Participated in the audit data ISO training while being familiar with data security regulated by OJK.

Technologies: Python 3, Presto, Apache Airflow, AWS IAM, Metabase, Apache Superset, Superset, MongoDB, MySQL, Amazon S3 (AWS S3), Google BigQuery, Google Cloud Storage, Looker, Domo

Software Engineer

2017 - 2018

Adskom

Implemented ETL processes from raw data to data feed generation.
Managed the product development lifecycle using the Agile methodology.
Automated building, configuration, and deployment processes.
Maintained web applications using Superset for Adskom data visualization.

Technologies: Python 3, SQL, Hadoop, Agile

Experience

ETL Process and Data Lake for Analytics

This project involved designing and implementing a data lake using Presto, a distributed SQL query engine. Apache Airflow was incorporated into the workflow management system to handle the ETL process and deliver accurate data information. The final data was then effectively used by data science and business analysts.

Data Lake Design Using Presto

To easily access and combine our data feed with data science, we created and designed our data lake using Presto. The raw data comes from MongoDB (unstructured data) and MariaDB (structured data).

We connected the data from both using Presto and combined it with a data pipeline to build a data lake. It also includes other data on Object Storage, S3, and Google Cloud Storage.

Data Dashboard Using Apache Superset

Built for business purposes, this data dashboard allows users to check the business' health and more. It requires numerous queries and data preparations. The data source comes from ETL results that are refreshed daily. The real-time data comes from the raw data, which streams from the secondary database.

Education

2013 - 2017

Bachelor's Degree in Computer Science

Bina Nusantara University - Jakarta, Indonesia

Skills

Tools

AWS IAM, Apache Airflow, Postman, Superset, NGINX, GitHub, Looker, Domo

Languages

SQL, Python 3, TypeScript

Frameworks

Presto, Hadoop

Paradigms

Agile Software Development, ETL, Agile

Storage

Amazon S3 (AWS S3), MongoDB, MySQL, ArangoDB, Google Cloud Storage, PostgreSQL, MariaDB, Data Lakes

Platforms

Docker, Compute Instance

Other

Metabase, Apache Superset, Computer Science, VMware ESXi, MikroTik, IP Networks, MinIO, Google BigQuery, RPC, Node Clusters

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring