William is available for hire

William Y.

Verified Expert in Engineering

Data Engineer and Developer

Los Angeles, CA, United States

Toptal member since March 11, 2025

Expertise

Data Warehouse Data Engineering Data Science Data Analysis Data Migration SQL ETL Cloud Engineering BigQuery Python Apache Airflow JSON Tableau Development

Bio

William is a highly experienced and accomplished data and cloud solutions expert with a proven track record in designing, implementing, and managing robust data ecosystems. With extensive expertise in ETL/ELT processes and emerging cloud technologies, William has consistently delivered scalable, efficient, and high-performing data solutions. His deep knowledge of GCP and Snowflake spans the full stack, including data ingestion, transformation, storage, and analytics.

Portfolio

Ford Mobility

Google Cloud Platform (GCP), Python 3, Apache Airflow, Spark, MongoDB, BigQuery...

Macy's

Google Cloud Platform (GCP), BigQuery, Linux, Shell, Python 3, Apache Airflow...

CFIA

C#, Shell, Datastage, VB, Data Warehousing, Data Modeling, Data Engineering...

Experience

SQL - 16 years
Shell - 15 years
Datastage - 12 years
Google Cloud Platform (GCP) - 9 years
BigQuery - 9 years
Python 3 - 7 years
Apache Airflow - 6 years
Data Build Tool (dbt) - 5 years

Availability

Full-time

Preferred Environment

Linux, Windows, PyCharm, Python 3, Visual Studio Code (VS Code), Cloud

The most amazing...

...accomplishment was migrating a 40-petabyte on-premises database to GCP BigQuery, achieving improved performance and scalability.

Work Experience

Senior Data Engineer

2022 - 2025

Ford Mobility

Developed a Cloud Run service for MongoDB data ingestion and analysis from various file types, including XML, HTML, PDF, JSON, and more. Architected a data warehouse to support the service.
Designed and developed solutions for MongoDB migration.
Architected and ingested data into BigQuery data warehouses across different layers, transforming it as needed.

Technologies: Google Cloud Platform (GCP), Python 3, Apache Airflow, Spark, MongoDB, BigQuery, XML, JSON, HTML, Data Build Tool (dbt), Hadoop, SQL, Stored Procedure, Security, Data Architecture, Data Migration, Data Validation, Google Cloud Storage, Data Engineering, Cloud Key Management Service (KMS), Python, Large-scale Data Migration, Data Governance, Snowflake, Data Analysis, Looker, Data Analytics, Looker Studio, APIs, Azure SQL Data Warehouse, Microsoft Power BI

Staff Data Engineer

2016 - 2022

Macy's

Created new ETL/ELT jobs to replace the existing on-premises jobs for the migration of the database from on-premises to GCP.
Migrated a big data warehouse from on-premises to GCP with high efficiency.
Oversaw the optimization of existing jobs, improving process performance and overall project efficiency.
Developed new business requirements and implemented new services in the cloud.

Technologies: Google Cloud Platform (GCP), BigQuery, Linux, Shell, Python 3, Apache Airflow, Windows, Hadoop, IBM InfoSphere, IBM Db2, Oracle, Google Cloud SQL, AlloyDB, Cloud Run, PubSubJS, Dataform, Data Build Tool (dbt), Snowflake, Data Architecture, Data Migration, Data Validation, Google Cloud Storage, Data Engineering, Go, Python, Large-scale Data Migration, Data Governance, Data Analysis, Looker, Data Analytics, Looker Studio, Tableau, Azure

Senior Programmer Analyst

2009 - 2016

CFIA

Architected new applications for the import and export inspection department.
Designed and developed a new HR data warehouse using data from PeopleSoft.
Spearheaded a development team to complete a data warehouse application within a short timeframe.

Technologies: C#, Shell, Datastage, VB, Data Warehousing, Data Modeling, Data Engineering, Data Analysis, Data Analytics

Data Warehouse Developer

2007 - 2009

IBM

Developed big data warehouses for major banks and insurance companies, including HSBC.
Tracked and resolved issues reported directly by the client.
Optimized deployment tools for the data warehouse on a cluster of Linux and Windows servers.

Technologies: IBM Db2, Datastage, Shell, C, Cluster, Data Analysis, Data Analytics

Experience

MongoDB Migration

I spearheaded the migration of MongoDB from on-premises to GCP MongoDB Atlas, along with migrating the on-premises ingestion jobs that read new files from a shared Windows server drive and load them into MongoDB. I also created an analysis model with high-performance indexes in MongoDB. As the architect, lead designer, and developer, I oversaw the overall solution architecture and detailed design using Cloud Run jobs. I led a team of three developers to complete the project ahead of schedule, resulting in significant hardware cost savings for the client.

Teradata DB Migration

This project involved migrating the on-premises Teradata database to Google Cloud BigQuery, along with the existing on-premises ETL jobs developed using DataStage and Informatica. I handled the back end and helped design the strategy for the initial full-load migration of the database data and the incremental data migration. Additionally, I worked on the design of new ETL/ELT jobs to replace the existing ones.

Education

2005 - 2007

Master's Degree in Computer Science

McMaster University - Hamilton, Canada

Certifications

AUGUST 2023 - PRESENT

Google Cloud Certified Professional Architect

Google

JANUARY 2023 - PRESENT

Google Cloud Certified Professional Data Engineer

Google Cloud

Skills

Libraries/APIs

Cloud Key Management Service (KMS), PubSubJS

Tools

Apache Airflow, BigQuery, Shell, Looker, Tableau, dbt Cloud, Microsoft Power BI, PyCharm, Cluster

Languages

Python 3, SQL, Stored Procedure, Snowflake, Python, XML, HTML, C#, VB, C, Go

Paradigms

ETL

Platforms

Google Cloud Platform (GCP), Cloud Run, Azure SQL Data Warehouse, Azure, Linux, Windows, Visual Studio Code (VS Code), Oracle

Storage

JSON, Google Cloud SQL, Datastage, Data Validation, Google Cloud Storage, Databases, MongoDB, IBM Db2, NoSQL

Frameworks

Spark, Hadoop

Other

IBM InfoSphere, Data Warehousing, Data Modeling, ELT, Security, Data Architecture, Data Migration, Data Engineering, Large-scale Data Migration, Data Analysis, Data Analytics, Looker Studio, APIs, Cloud, Data Build Tool (dbt), AlloyDB, Cluster and Distribute Calculation, Dataform, Data Governance

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring