Steven Rouk

A dataset that used network device configuration files to determine the correct services that should be available to a customer. The project involved combining data from five different systems—several of which had significant data quality issues—into a single consumable dataset with 99%+ data quality accuracy.

As the sole developer on the project, I worked closely with end users to design the desired data schema, create data logic to meet business needs, and validate the accuracy of the dataset. As the requirements continually evolved, I brought together disparate teams at each step to reach a common understanding of the required data and data logic. In the end, we increased the data accuracy by 25%+ in a customer-facing production environment.

A nonprofit organization needed to set up a data and analytics infrastructure that ingested data from multiple sources (SendGrid, Bubble.io, and more). I transformed the data and output data marts on BigQuery, which are ready for analysis with Looker Studio.

I used Airbyte for the extract/load part of this process and dbt for the data transformations. Raw and transformed data were landed on BigQuery. Data visualizations and reports were built in Looker Studio.

Because of this new, automated analytics infrastructure, the team can now see metrics and data instantaneously rather than through the laborious manual approach they previously had to use.

A process I created to automatically determine the characteristics of a new dataset in a Snowflake database, including nulls, distinct values, correlation with other columns, trends over time, data types, and intersections with other datasets. This allowed data engineers to do in minutes what used to take days or weeks to accomplish. After the initial data profiling, data engineers could conduct additional ad hoc analysis as needed using Python, SQL, and Jupyter Notebooks. They used this analysis to answer questions that arose from the automated data profiling process.

An NLP data science project using arXiv.org research paper metadata to study the evolution of machine learning over the last 20 years. I pulled metadata for 1.6 million research papers, filtered the dataset down to 50,000 machine learning-focused research papers, and then used topic modeling to study shifts in the topics discussed over time. I used the Python scikit-learn package for topic modeling, Pandas for storing and manipulating data, Seaborn and Matplotlib for plotting, and custom text visualizations to demonstrate the topic modeling algorithm.

I worked with the Stanford Social Network: Reddit Hyperlink Network dataset made available through the Stanford Network Analysis Platform (SNAP). This dataset catalogs hyperlinks between subreddits over 2.5 years, from January 2014 through April 2017.

Key Activities
• Created a graph of the data using the NetworkX Python graph library.
• Experimented with creating my own graph data objects to load and traverse the graph.
• Analyzed connections (in-degree and out-degree), distinct networks (component analysis), sharing reciprocity, centrality, and PageRank.
• Experimented with ways to visualize large graph structures by randomly sampling neighbor nodes.