Senior Data Engineering Consultant2022 - PRESENTFreelance Client
Technologies: Amazon Web Services (AWS), Amazon Athena, AWS Lambda, AWS Glue, Redshift, AWS RDS
- Worked as a lead data migration consultant to migrate data from the on-premise Microsoft SQL Server data warehouse to AWS Redshift.
- Led the initial discovery phase with customers and designed an E2E approach to migrate data warehouse and business intelligence from on-premise SQL Server to AWS Redshift using various AWS technologies.
- Wrote Python code in AWS Lambda and Glue and created scheduled jobs using EventBridge.
Consultant2019 - PRESENTBuzzclan
Technologies: AWS Lambda, Amazon Athena, SQL, Pandas, Stock Market, Algorithmic Trading Analysis, Python, Interactive Brokers API, Stock Trading, Futures & Options
- Designed and developed an application for stock market data analysis, developed strategies to find bullish and bearish trends in stocks and ETFs and used many TA-Libs.
- Built the ETL pipelines to collect tick data (per day, four hours, and five seconds) and fundamentals for over 3,000 stocks, ETFs, futures, and SPACs using many APIs and web scrapers.
- Transformed data calculated technical indicators, such as EMA crossover, stochastic, RSI, CCI, AO, candles, and Fibonacci levels.
- Integrated Cathie Wood's ARK ETF daily trade data and developed views to quickly see ARK holdings and action trends.
- Made the entire dataset and research available through a website to help users make investment decisions.
- Developed a trading app to use real-time tick data (OHLCV) and place trades using IB-API on the Interactive Broker platform. Also developed and tested strategies using backtest.
Consultant2021 - 2022PennyMac
Technologies: Amazon Web Services (AWS), Amazon Athena, AWS Glue, AWS Lambda, Data Engineering, Linux, Snowflake, Python
- Led the implementation of a cloud data warehouse using AWS and Snowflake technologies.
- Built ETL pipelines for mortgage loan campaigns, advertising, and marketing from Google, Facebook, Salesforce, Blend, Leadspedia, and more.
- Fetched data using APIs and AppFlow to AWS S3, cleansed the data using Lambda, and pushed it to Snowflake using stored procedures and external tables.
Senior Vice President2011 - 2020Citi
Technologies: Data Warehousing, Data Analytics, Data Transformation, Reporting, Business Intelligence (BI), IT Management
- Managed ETL and BI teams (25 members) to re-engineer a legacy HR data warehouse and business intelligence platform.
- Delivered a new in-house human capital management data warehouse platform that consolidated all HR verticals under one system with centralized data security, fast time to market, and advanced analytics using MicroStrategy BI.
- Developed data migration strategy, measurable acceptance criteria, and implemented frameworks and tools to track progress toward go-live.
- Architected the data validation framework, which helped reduce the testing cycle from two weeks to one day.
- Designed the data reconciliation framework used to compare more than 400 batch feeds and resulted in $1.2 million in savings for two years.
Tech Lead2000 - 2011Verizon
Technologies: Oracle, Unix, C++, Shell Scripting, Perl, Workflow, Java 8
- Designed, developed, and managed the operations support system (OSS) to support Verizon's high-speed internet services.
- Wrote C++ code on Unix to process orders and provision services.
- Developed the Java code to take orders and manage inventory of telecom equipment.
- Built custom data migration (ETL) modules using Java, JDBC, SQL, and Oracle and migrated data from many companies Verizon bought.