Data Engineer
2020 - 2020Aura- Provided data scientists and business analysts with reliable access to loan application and payment data by building an Airflow-orchestrated data pipeline between an RDS transactional database and a Snowflake data warehouse.
- Ensured reliable service by writing automated tests for data pipelines using Pytest and Tox.
- Increased team productivity by expanding documentation and writing Bash shell scripts to automate the installation of required tools and packages.
Technologies: Pytest, PostgreSQL, Apache Airflow, Snowflake, PythonData Engineer
2019 - 2020Insight Data Science- Assisted Google Ads users to find the most cost-effective options for their Google Ads (AdWords) purchases.
- Created an application that identifies new trending words within social media communities devoted to a specific topic.
- Provided a fast and resilient pipeline that ingests data from social media sites, processes the data with Spark to find trending topic-specific words, and stores the processed data in a PostgreSQL database that updates via Airflow DAG.
- Built an easy-to-use and informative Dash-based UI that delivers results from a database by converting user input into SQL queries to generate a list of possible words for Google Ads and informative plots about the words’ usage on Reddit.
Technologies: Amazon Web Services (AWS), Plotly, AWS, Python, PostgreSQL, SparkInstructor and Technology Committee Member
2010 - 2019Stanford University- Enabled online teachers to quantitatively track their students’ participation and use of class time.
- Developed a Python application that extracts student participation data from XML-log files and generates easily understandable reports and charts using Bokeh.
- Saved teachers approximately five hours per week by finding, testing, evaluating, and making recommendations about new software for learning management, grade book, video recording, and video playback.
- Increased new technology adoption rate by approximately 30% by giving talks, hosting workshops, and writing user guides for instructors and staff.
Technologies: Python