Data Engineering Developer in Adelaide, South Australia, Australia

I was the project owner of LUCI, a data analytics platform that was built to help users engage with analytics more quickly and naturally. I led the development team from the first line of code two years ago to our launch as a fully-featured SaaS solution earlier this year.

It is a single page application built using React for the front end, .NET Core, and PostgreSQL for the back end and Tableau as the visualization layer. The analytics layer is a combination of an S3 data lake, AWS Redshift, and Columnstore Microsoft SQL Server.

I was responsible for all aspects of our stack and responsible for choosing technology products and solutions. I managed our technology partnerships and start-up programs with Microsoft and AWS.

I wrote a simple web scrape app to regularly ping classifieds sites and notify me on my phone when a new ad matched my criteria. The notification included the title, location, price, ad URL, and first few lines of the description. I wanted to be able to disable a particular scrape and not be spammed by the same ad if it met different criteria multiple times. I also wanted to keep a searchable history of notifications that had been sent to me. I included a timestamp of the last scrape and check so I didn't hit the sites too often and my bot wouldn't get banned.

I used Angular and .NET Core (iis) with SQL Server to create a front and back end for data management. This allowed me to register a new URL to ping or scrape and manage it. I used the Pushbullet API with the Pushbullet iOS app to get notifications on my iPhone.

I scheduled Python scripts to do the following:
- connect to the SQL back end to pull the scraped URL
- beautiful soup to scrape and parse the site
- save scraped details back to the SQL database
- identify new ads and send them to the Pushbullet API which sent it to my phone

It works great and found excellent bargains.

ADA is the data analytics platform of HESTA, one of the largest pension funds in Australia. It was built to help analyze and improve member engagement by analyzing retention, churn, bench-marking, financial advice performance.

I architected and began the build of the Data Lake, dimensional model, dashboards, and extended the functionality of Tableau via an embedded wrapper (React) that overcame some of Tableau's limitations and enabled us to tell a more compelling data story.

The solution-focused on member engagement and management KPIs then evolved to provide analytics for churn rate, competition, financial adviser performance, call center workflows. Alteryx trialed for a time as an enterprise ETL tool.

The project gave me a platform to use and evolve my Agile approach to data modeling and analytics.

AWS Redshift + S3 + Parquet + Redshift Spectrum + Tableau + Confluence + Alteryx

As the head of technology at Laneway, I built a margin analytics application that utilizes econometrics principles such as price-volume-mix analysis to quantify strategic and management decisions.

I built the relational back end (SQL Server) and multidimensional calculation engine (Microsoft SSAS) that sits at the core of the solution and allows us to compare and contrast a portfolio/hierarchy against:

* prior performance (prior year/half/quarter/month adjusted for seasonality and year to date)
* industry benchmarks
* forecasts
* budgets
* any other object in the portfolio (e.g., compare business units and regions to each other)
* performance management (e.g., individual performance vs. an actual average performance by all staff for the period)
* impact of fx

Tableau dashboards act as the visualization layer, with an AWS API gateway and Lambdas sitting between Tableau and the SSAS cube.

This is another feature in an SSIS-driven export module that shards and parallelizes large export tasks.

This brand new startup pitched and won a proposal over two incumbents (including Deloitte) by suggesting a disruptive solution at Australia's largest pension fund (by member and funds under management) to deliver a member insurance analytics pilot. The purpose of the member insurance analytics pilot was to provide insights into claims management and optimize outcomes for members, aid in product design, increase retention, reduce lapse rates, and aid in insurer pricing negotiations.

I was the data engineer and architect and worked with my team and Australian Super to design and build the integration and data models using agile methodologies. I used my framework for rapidly developing analytics solutions for the first time at a large customer. It allowed us to iterate our solution at an incredible rate to deliver relevant deep insights. The high speed of the deployment and delivery significantly improved user engagement with data.

CSV + SQL Server + Tableau

Laneway was engaged to analyze a potential takeover target of PEP. Laneway received a cache of financial data in a locked-down environment and had to investigate profits and identify pricing and margin opportunities under very tight deadlines.

As the data architect, I integrated and modeled the data, which enabled the analysis to be completed in a very thorough manner.

AWS + CSV + SQL Server + Tableau

I developed a set of custom C#.Net libraries for relational databases that speed up geospatial computations and queries. The performance improvements are in the region of x10-1000 times faster and are cross-platform. I have seen great results in SQL Server, Oracle, and PostgreSQL.

I was very disappointed by the processing speed of geospatial tagging queries by relational database engines on a particular project. After researching the general approaches of the primary database engines, and leveraging my deep understanding of database principles, I saw a novel approach to the problem. That was in 2013 (for SQL Server 2008R2), my libraries are still many times faster than using the default functionality in 2017 on the latest version of SQL server 2016.

I built an accrual model and visualization for a national utility wholesaler as part of a large-scale project. The modeling tool allowed the wholesaler to bill retailers at more regular intervals than actual usage was available and allowed for long term forecasting and cash flow management. It maximized the cash on hand by accurately predicting customer consumption and growth.

The tool was built on a Microsoft stack, with SSIS used to integrate data from disparate Oracle, SQL Server and web data sources, SQL Server as the data warehouse, analysis services as the modeling tool, and Excel workbook dashboard report pack. The model performs complex calculations and takes prior usage, seasonality, actual, and forecast weather conditions (sunshine hours, wind speed, temperatures) as inputs to accrue the predicted income for the missing days of usage. The income normalizes over time as actual data becomes available.

The SA Ambulance Service was upgrading its call center and responder systems and required a new operational and management dashboard and reporting solution. The interesting part of this project was the limitations on only using existing technology due to being a public health/government institution, and the requirement for near real-time dashboards for the operational shift managers to allow them to monitor all aspects of incidents as they were happening and a history of all incidents for the eight-hour shift.

I was the data architect and data warehouse developer. I built a robust solution to quickly load data in as it was arriving, lightweight dashboards to render it quickly, and a framework to deal gracefully with deadlocks (complex SQL database concepts like partition switching, deadlock priority, handling lock escalation and schema locks).

SQL Server, SSIS, Reporting Services, SharePoint