Senior Data and Solution Architect
2019 - PRESENTPillar Base, Inc.- Developed an application to denormalize data from the relational database and stream it to an AWS DynamoDB single table.
- Worked on a proof of concept (POC) to assess the performance between Spark SQL, Presto DB, and Apache Drill with parquet files.
- Architected and developed integration solutions based on AWS Lambda, API Gateways, Cognito, WMS, Facebook Open Graph API, Google Ads API, BigCommerce, AWS DynamoDB, JavaScript, and TypeScript.
- Built a highly concurrent and fault-tolerant back-end engine that handles over one million requests using Rust. Worked on the POC to compare Rust concurrency with Go concurrency.
- Minted NFTs, integrated them with a website, and built a dApp.
Technologies: OpenAPI, BigCommerce API, TypeScript, Spark SQL, Amazon Cognito, Amazon DynamoDB, AWS Lambda, Serverless, Rust, DApps, Ethereum, Solidity, Web3.js, Solution Architecture, Go, QiskitGo Developer
2022 - 2022INSTRUXI LIMITED- Developed an integration layer between Go and the Ethereum contract.
- Integrated with Redis service for caching and speed optimization.
- Developed a Helm chart to enhance the deployment of Casbin-Mesh.
Technologies: Go, Kubernetes, Python, Redis, JSON Web Tokens (JWT), Helm, Web3.js, Blockchain, Casbin-MeshRust Smart Contract Developer
2022 - 2022PHI VENTURES Limited- Added features to the Rust smart contract and some tests for them.
- Rearchitected the front-end React application to use the back end to access the contract features.
- Built the back end and a deployment pipeline using GitHub actions.
Technologies: Rust, JavaScript, TypeScript, Solana, Node.js, Express.js, Web3.jsElasticsearch Expert
2021 - 2021Valence- Integrated Django with Elasticsearch. Built highly customizable indexes.
- Monitored and helped the team to deploy that solution.
- Analyzed the search result, adjusting the indexing parameters to update the priorities.
Technologies: Elasticsearch, Python 3, DjangoSenior eCommerce Integration Engineer
2020 - 2021Deliverr- Worked on a serverless project to develop a REST API to integrate with a different store management API.
- Developed REST services to integrate with BigCommerce, Google Shopping, and Facebook Shop.
- Used OAuth 2.0 and OIDC protocols to protect the API and manage the authentication and authorization with another eCommerce API.
Technologies: Serverless, AWS Lambda, Google Shopping, Facebook API, BigCommerce API, Koa, TypeScript, Node.jsSenior Scala Developer
2019 - 2020ReactiveCore- Architected and developed a solution to implement a worker thread pattern integrated with Apache Kafka.
- Rearchitected the microservices components to implement the service mesh (Istio) in Kubernetes over the Amazon EKS cluster.
- Built a real-time streaming solution on AWS using Kinesis and Glue. Developed a data model.
- Implemented Apache Spark streaming with Apache Kafka to support the data model.
- Integrated Apache Druid with Apache Kafka to implement time-series analytics on the Amazon EKS cluster.
- Worked on a POC and built a solution to ingest from Kafka, based on Rust concurrency.
Technologies: AWS Kinesis, React, Node.js, Istio, Kubernetes, Spark, Apache Kafka, AWS Glue, Amazon EKS, Go, Akka, Scala, RustSenior Data Engineer
2019 - 2019BCG GAMMA- Developed and optimized the Apache Airflow graph solution to support the data model.
- Packaged the solution using Docker and Docker Compose.
- Built a workflow in CircleCI to automate the building, packaging, and deploying processes.
- Developed a big data ETL solution on Azure Cloud based on DataFactory and integrated with Data Lake Analytics.
Technologies: Docker Compose, CircleCI, Docker, Apache Airflow, PythonSenior Solution Architect
2019 - 2019CN Railway- Built real-time and batch processing ETL solutions based on the DataFactory.
- Architected and developed an event-driven application using Node.js, React, and GraphQL using Serverless tech.
- Built and architected an integration solution using Apache Kafka on the AWS EKS cluster with Azure Power BI.
- Designed a solution based on React and Node.js and deployed the solution using Docker and Kubernetes.
- Developed and architected a solution to stream the data into the data warehouse on Azure.
- Engineered interactive dashboards using Power BI with a data warehouse.
Technologies: Kubernetes, Node.js, React, Apache Spark, Apache Kafka, AWS EMR, Python, Scala, Java, Solution Architecture, Data WarehousingBig Data Architect
2016 - 2019Morgan Stanley- Architected and developed a big data ETL solution based on AWS EMR and AWS Kinesis.
- Built a knowledge graph using a triple store database and Elasticsearch.
- Created a data streaming solution using S3, Lambda functions, AWS Kinesis data streaming, and AWS EMR service.
- Helped develop an NLP solution to enhance and enrich features in the market data reports as a data analyst.
- Built an interactive data UI based on React and React D3 integrated with Azure Power BI.
- Developed a search engine for market data integrated with Elasticsearch and Apache Solr.
- Worked on a POC to compare the latency between C++ and Go.
- Built a highly concurrent and fault-tolerant back-end engine that handles over one million requests using Rust.
- Worked on Nexus lifecycle and repos to securely manage the open source libraries.
Technologies: Deep Learning, AWS EMR, Machine Learning, Natural Language Processing (NLP), React, AWS Kinesis, Node.js, Python, Go, Java, Scala, RustBig Data Developer
2016 - 2016TD Bank- Developed a solution to three ETL big data projects using Apache Hive and Oozie.
- Worked as part of the team on extending Hadoop security functionality.
- Architected and developed a multi-threaded Java application to integrate a Linux local account with the Active directory, business requirement analysis, and sketching technical solution.
- Received “Act for Impact Award” for my work on a security project.
Technologies: Streaming, Spark, Hadoop, Go, Scala, Java