Jakub Kaczanowski, Developer in Adelaide, South Australia, Australia
Jakub is available for hire
Hire Jakub

Jakub Kaczanowski

Verified Expert  in Engineering

Bio

Jakub started taming data nearly 20 years ago, building Access databases in the oil and gas sector. Since then, he's built data solutions for various Australian financial and government clients, co-founded a fintech startup, and freelanced for US and EU multinationals. An expert in BI, analytics, and data warehouse architecture and development, Jakub is much more than a technical resource; he has a sound understanding of the role of insightful data and its commercial application.

Portfolio

Department of Human Services SA
Microsoft Power BI, SSAS Tabular, DAX, Microsoft SQL Server, SSRS Reports, ETL...
REST Super
Data Build Tool (dbt), Redshift, ETL, ELT, SQL, Data Warehousing...
Stanford University
Data Engineering, Azure Data Factory, Databricks, Data Lakes, Data Pipelines...

Experience

  • Data Engineering - 16 years
  • Business Intelligence (BI) - 15 years
  • ETL - 15 years
  • SQL - 15 years
  • Dimensional Modeling - 12 years
  • Redshift - 5 years
  • Microsoft Power BI - 5 years
  • Tableau - 5 years

Availability

Part-time

Preferred Environment

Amazon Web Services (AWS), Tableau, Microsoft SQL Server, Visual Studio, Azure, Microsoft Power BI

The most amazing...

...thing I've built is a railway crossing monitoring tool that identifies unsafe crossings and proposes a safe train speed restriction based on data feeds

Work Experience

Business Intelligence Consultant

2023 - 2023
Department of Human Services SA
  • Worked directly with the director of finance to design and build salary and workforce planning models and reports in SSRS using agile methodology to iterate features into models and reports quickly.
  • Built the finance reports to combine data from the budgeting and HR systems to easily visualize historical and current budgeted positions and FTEs vs actual staffing, contracts, and payroll.
  • Architected, documented, and built the National Disability Insurance Scheme (NDIS) analysis platform on Microsoft SQL Server, Analysis Services, and Power BI.
  • Worked on NDIS analysis platform dashboards that enabled high-level actuals vs budget, plan utilization, and YoY change management reporting with data journey drill-down capabilities down to individual participant, plan, invoice, and provider levels.
  • Worked with the DHS equipment program team to architect, build, and deliver galaxy and star(s) schema data marts to enable fast, accurate, and easy-to-use analytics over equipment provision data.
  • Replaced existing manual reporting processes with an automated real-time solution that expanded on previous functionality to allow for deeper and more intuitive analysis.
  • Mentored and advised the team on the latest data analytics architecture patterns regarding cloud solutions (Databricks, Azure Synapse/Fabric, and dbt), data lakehouses, medallion architecture, virtual data marts, and ELT vs. ETL.
  • Mentored and advised the team on their journey from an on-premise environment to a cloud-based agile/DevOps-driven approach.
  • Migrated existing data projects and solutions into Azure DevOps.
Technologies: Microsoft Power BI, SSAS Tabular, DAX, Microsoft SQL Server, SSRS Reports, ETL, ELT, Data Warehousing, Data Warehouse Design, Dimensional Modeling, SQL, T-SQL (Transact-SQL), DevOps, SQL Server Analysis Services (SSAS), Business Intelligence (BI), Data Modeling, Cognos Analytics 11, Data Analysis, Data Analytics

Data Engineer

2022 - 2023
REST Super
  • Worked with the insurance data uplift team to provide advice, design, development, and technical guidance on implementing an insurance analytics data platform that utilizes ELT best practices optimized for cloud data warehouses.
  • Migrated and refactored existing Talend ETLs into modular dbt models, identifying and fixing bugs and issues.
  • Implemented automated data quality and deployment tests within the dbt test framework.
  • Design and build CI/CD and AD workflows within GitHub and dbt cloud to automate releases.
  • Mentored junior staff to uplift data engineering capabilities.
  • Showcased the demo solution and progress to clients and business managers.
Technologies: Data Build Tool (dbt), Redshift, ETL, ELT, SQL, Data Warehousing, Data Warehouse Design, GitHub, Dimensional Modeling, Slowly Changing Dimensions (SCD), DevOps, Data Analysis, Data Analytics

Data Warehouse Engineer

2022 - 2023
Stanford University
  • Worked with the Stanford Cancer Institute to design and build a modular data platform, conformed genomics data models, and a generic data framework using Azure, DBT, and Databricks.
  • Designed, developed, and deployed an Azure data lake and generic metadata-driven automated data factory pipelines to ingest public and private genomics data sources from various platforms (web, FTP, complex flat file, CDM, database).
  • Designed, developed, and deployed a Databricks data lakehouse (Unity Catalog) and Python libraries to ingest the data lake into the Unity Catalog. Incorporated automated execution into metadata-driven Azure pipelines.
  • Developed Python libraries to handle ingesting and converting complex genomics data types like VCF (via GLOW) and handle very uncommon big data (datasets with millions of columns) into Delta Lake.
  • Incorporated dbt (data build tool) to handle modeling data lakehouse raw data into useful data models. Automated CI/CD model deployment and execution via GitHub and Databricks workflow jobs.
  • Incorporated custom 'time travel' logic into dbt models, which allows researchers to retrieve data at a specific release, point in time, and current version to enable replicating research and experiments.
  • Automated data dictionary generation and deployment into confluence.
  • Worked closely with data scientists and research administrators to design enhanced data models to enable the TOBIAS (test of bias) application and other research projects.
  • Documented the entire solution within Confluence, including technical design documentation, solution design, deployment configuration, user onboarding, and developer onboarding.
  • Designed and developed Power BI operational dashboards to track the data platform, pipelines, and jobs.
Technologies: Data Engineering, Azure Data Factory, Databricks, Data Lakes, Data Pipelines, Data Warehousing, Data Warehouse Design, Python, SQL, Data Build Tool (dbt), Microsoft Power BI, Data Visualization, GitHub, CI/CD Pipelines, Azure Synapse, Microsoft Fabric, MDM, ETL, ELT, Azure, Azure SQL, Delta Lake, Dimensional Modeling, Slowly Changing Dimensions (SCD), ETL Tools, Data Analytics, Master Data Management (MDM)

Senior Data Engineer - Global Laboratory & Analytics Company

2022 - 2023
Tentamus Group GmbH - Main
  • Designed and developed a data mart to allow faster and deeper reporting for internal and external stakeholders by combining multiple data sources from the various international labs run by Tentamus.
  • Ran training sessions to train the data science team on the administration and management of the data platform.
  • Scaled platform by onboarding additional datasets from existing data sources and provisioning new data sources.
  • Provided additional support and feature enhancement for the data platform.
Technologies: SQL, Data Engineering, Data Warehousing, Data Mining, Python, Microsoft Power BI, ETL, Dataflow Programming, Data Analytics

Senior Data Consultant (Analytics Engineer)

2022 - 2022
Discovery Parks
  • Designed a data platform architectural strategy to enable deep analytics of customer data, marketing automation, and the attribution of online and offline activity, presented to and accepted by the CTO and senior managers.
  • Documented, designed, and developed a tactical platform based on my proposal using Azure Data Lakes, Synapse serverless, and virtual data marts to enable rapid development and prototyping of data models via dbt and visualization in Power BI.
  • Iterated the tactical model into an enterprise solution underpinned by a Databricks data lakehouse (delta lake) with dbt used to deploy the now more mature data models.
Technologies: Azure, Azure Synapse, Azure Data Lake, Azure Data Factory, Databricks, Data Build Tool (dbt), Microsoft Power BI, Python, Data Visualization, SQL, ELT, ETL, ETL Tools, Delta Lake, Data Lakehouse, Data Marts, Kimball Methodology, Azure SQL, Azure SQL Databases, Dimensional Modeling, Data Vaults, Data Analysis, Data Analytics

Data and Analytics Developer

2021 - 2022
Elders Rural
  • Provided expertise and advice on an Azure cloud PoC data lake implementation project to support greater analytics by integrating Elders Rural core financial data with that of other business entities acquired by Elders.
  • Built integration pipelines in just a few days to replace non-functional pipelines that had months of work.
  • Created a certificate-based JWT authentication to allow seamless integration to a data source.
  • Developed, designed, and implemented solutions using Azure Synapse (pipelines, data flows, serverless) and Databricks.
Technologies: Azure, Azure Synapse, Databricks, Azure Data Factory, Microsoft SQL Server, SQL, Database Design, ETL, ELT, Python, Data Engineering, Data Warehousing, Business Intelligence (BI), Business Intelligence Architecture, Data Warehouse Design, T-SQL (Transact-SQL), Azure SQL, Data Architecture, Data Integration, Data Marts, Azure SQL Databases, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics

Senior Data Engineer

2021 - 2022
Tentamus
  • Analyzed the company's existing data warehousing solution architecture, provided feedback, and wrote a proposal for a new solution and approach.
  • Designed and developed a PoC data mart based on the proposal to allow faster and deeper reporting for internal and external stakeholders by combining multiple data sources from the various international labs run by Tentamus.
  • Wrote and ran training sessions for the data science time on how to use and add new data sources to the data mart.
Technologies: Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL, ETL, ELT, Database Design, Data Engineering, Data Warehousing, Business Intelligence (BI), Database Architecture, Business Intelligence Architecture, Data Warehouse Design, T-SQL (Transact-SQL), Python, Data Architecture, Data Integration, Data Marts, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics

Data and Analytics Developer

2020 - 2021
Australian Rail Transport Corporation
  • Provided support in integrating and implementing ARTC's asset management support capabilities, such as improved work planning and evidence-driven proactive asset maintenance based on exceptions, trends, and predictions.
  • Designed, developed, and migration of integration data models using Azure Synapse and Parallel data warehouse, Azure Databricks, and Azure Data Factory.
  • Provided capability to the corporate services team to migrate a traditional ETL-driven data architecture to an Azure hybrid cloud solution.
Technologies: Data Build Tool (dbt), Microsoft SQL Server, Azure, Databricks, Azure Synapse, SQL, SQL Server Integration Services (SSIS), Tableau, Reports, Microsoft Power BI, Dashboards, Data Visualization, Database Design, Star Schema, Data Engineering, Data Warehousing, ETL, Business Intelligence (BI), Database Architecture, Dashboard Design, Business Intelligence Architecture, Data Warehouse Design, T-SQL (Transact-SQL), Power Pivot, Azure SQL, Data Architecture, Dashboard Development, Data Integration, SSAS Tabular, Embedded Business Intelligence, DAX, Data Marts, ELT, Azure SQL Databases, Business Intelligence (BI) Platforms, ETL Tools, Data Analysis, Data Analytics

Data and Analytics Developer and Architect

2019 - 2021
Aircraft Hardware West
  • Collaborated with senior management to architect and develop inventory management analytics to streamline analysis and forecasting of demand and stock levels to allow AHW to meet contractual SLAs and required stock on hand levels.
  • Loaded, integrated, and modeled disparate data sources and reference data using ELT and a relational data lake approach using Pentaho and SQL Server.
  • Designed and built a suite of dashboards in Power BI to visualize findings, forecasts, and historical trends and allow further what-if analysis.
  • Implemented statistical models to surface complex domain-specific attributes like safety stock and economic order quantity.
  • Designed, documented, and built dimensional star schema models to enable complex analysis.
Technologies: Microsoft Power BI, Microsoft SQL Server, Oracle, Data Visualization, Dashboards, Star Schema, ELT, ETL, Data Engineering, Data Warehousing, SQL, Business Intelligence (BI), Database Architecture, Dashboard Design, Business Intelligence Architecture, Data Warehouse Design, T-SQL (Transact-SQL), Power Pivot, Data Architecture, Dashboard Development, Data Integration, SSAS Tabular, Embedded Business Intelligence, DAX, Data Marts, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics

BI Developer and Architect

2018 - 2019
Department of Health (South Australia)
  • Completed a short-term contract within the system performance and service delivery team providing capability ETL design and development, solution architecture, and data warehouse architecture.
  • Acted as the architect and built a modular ELT framework using SSIS and SQL to productionize a Qlik Sense-based proof of concept solution.
  • Reverse-engineered Qlik script and migrated it into an ELT framework and the interface into the enterprise data warehouse.
Technologies: Qlik Sense, SQL Server Integration Services (SSIS), SQL, ETL, ELT, Data Engineering, Data Warehousing, Microsoft SQL Server, Business Intelligence (BI), T-SQL (Transact-SQL), Data Architecture, Data Integration, Data Marts, Azure SQL Databases, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics

Head of Technology, Architect, and Business Intelligence Consultant

2015 - 2019
Laneway Analytics
  • Served as the project owner for Luci, our data analytics platform that Laneway built to help users engage with analytics more quickly and naturally. Used Agile and Scrum.
  • Developed Luci, a SPA running on AWS with a React front end, .NET Core, PostgreSQL, and Tableau as the data visualization layer. The analytics component is built using S3, Redshift, Redshift Spectrum, and columnstore SQL Server.
  • Acted as the principal data architect at Laneway for our most important clients. Using my Agile approach to data integration and data modeling, I proved that a data warehouse could be functional within weeks, not months or years.
  • Developed a complex financial analysis tool utilizing an SQL Server OLAP cube as a calculation engine and Tableau as the visualization layer.
  • Delivered rapid architecture, design, and development of a number of tactical data marts built-in SQL server and Redshift for large clients in logistics, manufacturing, and finance. Integrated and modeled data with a logical view ELT-based approach.
Technologies: Amazon Web Services (AWS), .NET, Azure, Amazon S3 (AWS S3), Tableau, SSAS, SQL Server Reporting Services (SSRS), SQL Server Integration Services (SSIS), PostgreSQL, Redshift, Microsoft SQL Server, ETL, ELT, Databases, DataViz, Data Visualization, Dashboards, Star Schema, Data Engineering, Data Warehousing, SQL, Business Intelligence (BI), Database Architecture, SQL Server Analysis Services (SSAS), Microsoft Power BI, Dashboard Design, Business Intelligence Architecture, Data Warehouse Design, T-SQL (Transact-SQL), Azure SQL, Python, Data Architecture, Dashboard Development, Data Integration, Embedded Business Intelligence, MDX, Data Marts, Business Intelligence (BI) Platforms, ETL Tools, Data Analysis, Data Analytics

Senior Business Intelligence Consultant and Data Architect

2011 - 2015
Chamonix
  • Designed and developed enterprise data warehouse solutions that spanned the full Microsoft BI stack, including data store, integration, data models, and visualizations for numerous clients across the government, health, education, and utilities sector.
  • Mapped out and created numerous complex SSAS OLAP solutions utilizing multidimensional and tabular, including Power Pivot.
  • Planned and developed an SSIS-based modular ETL framework that integrated data from various RDBMS (Oracle and SQL Server), flat file, and web-based data sources. The framework was successfully deployed to numerous clients.
  • Outlined and developed a SQL Server data warehouse for a financial reporting and forecasting tool for a major utility provider in Australia.
  • Designed and developed a SQL Server data warehouse featuring a real-time, shift, and historical reporting dashboard solution for ambulance emergency response. Near-real-time data is visualized for the commanders at the incident response center.
  • Developed a suite of 30+ complex KPI, financial, and operational reports for a public utility company. Completed in under a month, allowing the project to go live on schedule after IBM had de-scoped the vital reporting assets.
  • Provided pre-sales and proof of concept and prototyping support to the sales team to win new business.
  • Worked closely with business stakeholders to analyze requirements and provide sound solutions.
  • Developed and deployed a number of analytics PoC solutions in Azure for client projects using Azure SQL and Power BI.
Technologies: Amazon Web Services (AWS), .NET, Azure, BIRT, Visual Basic for Applications (VBA), Power Pivot, Microsoft Excel, Microsoft Power BI, SSAS, SQL Server Integration Services (SSIS), Microsoft SQL Server, Business Intelligence (BI), Dashboards, Data Visualization, SQL, T-SQL (Transact-SQL), MDX, DAX, Data Modeling, Database Design, Star Schema, Data Engineering, Data Warehousing, ETL, Database Architecture, SQL Server Analysis Services (SSAS), Dashboard Design, Business Intelligence Architecture, Data Warehouse Design, SQL Server Reporting Services (SSRS), Azure SQL, Data Architecture, Dashboard Development, Data Integration, SSAS Tabular, Embedded Business Intelligence, Data Marts, ELT, Azure SQL Databases, Business Intelligence (BI) Platforms, ETL Tools, Data Analysis, Data Analytics

Business Intelligence Developer and Data Architect

2011 - 2011
Department of Business and Innovation (Victoria)
  • Created SSIS-driven ETL framework for reporting Australian Vocational Education and Training Management Information Statistical Standard (AVTEMISS) compliant training activity data to the National Centre for Vocational Education Research (NCVER).
  • Built metadata-driven ETL (SSIS) and reporting suite (SSRS) for data reconciliation reporting between source systems, data warehouse (SQL Server), and OLAP cubes (SSAS).
  • Developed lightweight optimized training activity analysis services multidimensional OLAP cube for faster reporting of commonly used measures for consumption via Excel reporting packs and self-service.
Technologies: Microsoft Excel, MDX, SSAS, SQL Server Integration Services (SSIS), T-SQL (Transact-SQL), Business Intelligence (BI), Data Engineering, Data Warehousing, Microsoft SQL Server, Data Visualization, SQL, ETL, SQL Server Analysis Services (SSAS), Dashboard Design, Data Warehouse Design, Data Architecture, Dashboard Development, Data Integration, Data Marts, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics

Analyst, Programmer, and Data Architect

2011 - 2011
Department of Education (Victoria)
  • Developed and deployed a centralized single source of truth of school-based reference data for application use.
  • Designed and built a multi-faceted data integration process, including change data capture, SQL replication from Oracle and old versions of SQL Server to a centralized SQL Server, and integration services.
  • Oversaw the analysis and architecture of the centralized SQL Server database, including all architectural documentation and code implementation, and consolidation of existing reference databases.
  • Implemented SQL Server schema-based security to lock down student reference data.
Technologies: CDC, SQL Server Integration Services (SSIS), T-SQL (Transact-SQL), Microsoft SQL Server, Databases, Database Design, SQL, ETL, Business Intelligence (BI), Data Integration, Business Intelligence (BI) Platforms, ETL Tools

Data Warehouse Developer

2010 - 2010
Link Group
  • Migrated superannuation customer data from an AS/400 source to SQL Server data warehouse.
  • Designed and developed a generic SSIS ETL metadata-driven framework for generating extracts and delivering via SFTP B2B link to AMP.
  • Fixed, enhanced, and created new dashboards and reports for the SQL Server Reporting Services report pack sent out to customers.
Technologies: SQL Server Integration Services (SSIS), T-SQL (Transact-SQL), Microsoft SQL Server, Reporting, Data Engineering, Data Warehousing, Data Visualization, SQL, ETL, Business Intelligence (BI), Database Architecture, Dashboard Design, Data Warehouse Design, SQL Server Reporting Services (SSRS), Data Architecture, Dashboard Development, Data Integration, Data Marts, Business Intelligence (BI) Platforms, ETL Tools

Database Administrator

2008 - 2010
Building and Plumbing Commission (Victoria)
  • Managed the SQL Server environment at the Building and Plumbing Commission (Victoria) as the DBA, including backup and restore, performance, and disaster recovery on a few occasions.
  • Identified timing issues with existing backup plans that had invalidated all point-in-time backups; created a new backup process and ran restore scenarios weekly.
  • Debugged, profiled, and queried optimization of database applications that used SQL Server as their data tier.
  • Wrote a database consolidation plan for management.
  • Advised management of risks, data costs, and feasibility of proposed third-party vendor applications in relation to SQL Server.
Technologies: T-SQL (Transact-SQL), SQL Server Integration Services (SSIS), Microsoft SQL Server, Databases, Database Administration (DBA), SQL, ETL, Database Architecture, SQL Server Reporting Services (SSRS), Data Integration, ETL Tools

Analyst and Programmer | Data Migration

2008 - 2008
National Australia Bank
  • Contributed to a large project to roll out a third-party solution to replace a legacy in-house product used for managing unit pricing for superannuation products.
  • Migrated historical data and implemented the new solution within the existing National Custodial Services (NCS) framework.
  • Updated the profile data in the Oracle database and wrote SQL migration scripts.
  • Used DataStage to create and orchestrate data migration processes.
Technologies: Datastage, Visual FoxPro, Oracle, SQL, ETL, Data Integration, ETL Tools

Senior Analyst and Programmer | Database

2007 - 2007
Department of Health (South Australia)
  • Developed a 3-tier enterprise-level community healthcare application using Visual FoxPro and SQL Server. I was involved in the entire lifecycle of the product, including front-end and database analysis, development, and optimization.
  • Designed and developed a scheduling module using Visual FoxPro and SQL Server.
  • Rewrote the address lookup when a patient presented at a hospital. The original screen had 30+ text boxes. The new version had a single text box and used custom fuzzy logic to create a ranked list of possible matches from an address lookup whitelist.
  • Performed database and query optimization using SQL Query profiling and index optimization.
Technologies: XML, SQL Server Reporting Services (SSRS), Crystal Reports, SQL Server Integration Services (SSIS), T-SQL (Transact-SQL), Microsoft SQL Server, Reports, Databases, Database Design, Query Optimization, SQL, ETL, Database Architecture, Data Integration, ETL Tools

Database Conversion Analyst

2007 - 2007
Department of Further Education, Employment, Science and Training (South Australia)
  • Served as the technical lead on data migration from legacy system to 3-tier distributed architecture.
  • Performed a gap analysis of before and after data models and created data mapping tables and complex SQL data migration procedures.
  • Created orchestration ETL process in SSIS to perform the data migration.
Technologies: T-SQL (Transact-SQL), SQL Server Integration Services (SSIS), Microsoft SQL Server, Data Migration, SQL, ETL, Data Integration, ETL Tools

Senior Analyst and Programmer | Data Migration

2006 - 2006
Department of Health (South Australia)
  • Acted as the technical lead on multi-site data migration from a legacy FoxPro system into a centrally hosted 3-tier enterprise-level solution.
  • Developed and executed the migration plans and tools.
  • Created a generic repeatable ETL process in DTS/SSIS to perform the data migration.
  • Oversaw the data mapping, validation, gap analysis, and cleansing using SQL Server.
  • Liaised with each hospital administration team to write test plans, validate trial migrations, and organize the production migrations.
  • Trained hospital staff on the use of the new system.
  • Rewrote and refactored Crystal Reports and SQL data exports.
Technologies: XML, SQL Server Reporting Services (SSRS), Crystal Reports, SQL Server Integration Services (SSIS), T-SQL (Transact-SQL), Microsoft SQL Server, Data Migration, Reporting, SQL, ETL, Database Architecture, Data Integration, ETL Tools

Programmer

2004 - 2006
Steadfast Australia
  • Developed and implemented a middleware solution that integrated orders on the web portal with the in-house FoxPro-based tracking system and external transport companies to generate pack orders and print shipping labels.
  • Created and supported a web-based portal for customer orders using PHP and MySQL.
  • Built financial and operational reporting solutions within TransLogix, the warehouse stock management system.
  • Developed end-to-end integration between the order entered on the web portal and the products picked for shipping. Created custom barcodes so workers could use handheld scanners to update workflow status without typing.
Technologies: Visual FoxPro, Microsoft Excel, SQL, MySQL, PHP, Dashboards, Reports, Data Integration

Database Developer

2004 - 2004
Origin Energy
  • Developed a MS Access-based application to create a searchable index of technical drawing of all assets at a number of power plants.
Technologies: Microsoft Access, SQL

LUCI (Laneway Analytics)

I was the project owner of LUCI, a data analytics platform that was built to help users engage with analytics more quickly and naturally. I led the development team from the first line of code two years ago to our launch as a fully-featured SaaS solution earlier this year.

It is a single page application built using React for the front end, .NET Core, and PostgreSQL for the back end and Tableau as the visualization layer. The analytics layer is a combination of an S3 data lake, AWS Redshift, and Columnstore Microsoft SQL Server.

I was responsible for all aspects of our stack and responsible for choosing technology products and solutions. I managed our technology partnerships and start-up programs with Microsoft and AWS.

Python-driven Web Classified Scraper

I wrote a simple web scrape app to regularly ping classifieds sites and notify me on my phone when a new ad matched my criteria. The notification included the title, location, price, ad URL, and first few lines of the description. I wanted to be able to disable a particular scrape and not be spammed by the same ad if it met different criteria multiple times. I also wanted to keep a searchable history of notifications that had been sent to me. I included a timestamp of the last scrape and check so I didn't hit the sites too often and my bot wouldn't get banned.

I used Angular and .NET Core (iis) with SQL Server to create a front and back end for data management. This allowed me to register a new URL to ping or scrape and manage it. I used the Pushbullet API with the Pushbullet iOS app to get notifications on my iPhone.

I scheduled Python scripts to do the following:
- connect to the SQL back end to pull the scraped URL
- beautiful soup to scrape and parse the site
- save scraped details back to the SQL database
- identify new ads and send them to the Pushbullet API which sent it to my phone

It works great and found excellent bargains.

ADA (HESTA/Laneway Analytics)

ADA is the data analytics platform of HESTA, one of the largest pension funds in Australia. It was built to help analyze and improve member engagement by analyzing retention, churn, bench-marking, financial advice performance.

I architected and began the build of the Data Lake, dimensional model, dashboards, and extended the functionality of Tableau via an embedded wrapper (React) that overcame some of Tableau's limitations and enabled us to tell a more compelling data story.

The solution-focused on member engagement and management KPIs then evolved to provide analytics for churn rate, competition, financial adviser performance, call center workflows. Alteryx trialed for a time as an enterprise ETL tool.

The project gave me a platform to use and evolve my Agile approach to data modeling and analytics.

AWS Redshift + S3 + Parquet + Redshift Spectrum + Tableau + Confluence + Alteryx

Gross Profit Analytics (Laneway Analytics)

As the head of technology at Laneway, I built a margin analytics application that utilizes econometrics principles such as price-volume-mix analysis to quantify strategic and management decisions.

I built the relational back end (SQL Server) and multidimensional calculation engine (Microsoft SSAS) that sits at the core of the solution and allows us to compare and contrast a portfolio/hierarchy against:

* prior performance (prior year/half/quarter/month adjusted for seasonality and year to date)
* industry benchmarks
* forecasts
* budgets
* any other object in the portfolio (e.g., compare business units and regions to each other)
* performance management (e.g., individual performance vs. an actual average performance by all staff for the period)
* impact of fx

Tableau dashboards act as the visualization layer, with an AWS API gateway and Lambdas sitting between Tableau and the SSAS cube.

This is another feature in an SSIS-driven export module that shards and parallelizes large export tasks.

Australian Super Member Insurance Analytics Pilot (Laneway Analytics)

This brand new startup pitched and won a proposal over two incumbents (including Deloitte) by suggesting a disruptive solution at Australia's largest pension fund (by member and funds under management) to deliver a member insurance analytics pilot. The purpose of the member insurance analytics pilot was to provide insights into claims management and optimize outcomes for members, aid in product design, increase retention, reduce lapse rates, and aid in insurer pricing negotiations.

I was the data engineer and architect and worked with my team and Australian Super to design and build the integration and data models using agile methodologies. I used my framework for rapidly developing analytics solutions for the first time at a large customer. It allowed us to iterate our solution at an incredible rate to deliver relevant deep insights. The high speed of the deployment and delivery significantly improved user engagement with data.

CSV + SQL Server + Tableau

Private Equity Partners - Whiteroom for Takeover Bid (Laneway Analyics)

Laneway was engaged to analyze a potential takeover target of PEP. Laneway received a cache of financial data in a locked-down environment and had to investigate profits and identify pricing and margin opportunities under very tight deadlines.

As the data architect, I integrated and modeled the data, which enabled the analysis to be completed in a very thorough manner.

AWS + CSV + SQL Server + Tableau

Simplify Spatial (Personal Project)

I developed a set of custom C#.Net libraries for relational databases that speed up geospatial computations and queries. The performance improvements are in the region of x10-1000 times faster and are cross-platform. I have seen great results in SQL Server, Oracle, and PostgreSQL.

I was very disappointed by the processing speed of geospatial tagging queries by relational database engines on a particular project. After researching the general approaches of the primary database engines, and leveraging my deep understanding of database principles, I saw a novel approach to the problem. That was in 2013 (for SQL Server 2008R2), my libraries are still many times faster than using the default functionality in 2017 on the latest version of SQL server 2016.

National Accruals Platform (APA/Chamonix)

I built an accrual model and visualization for a national utility wholesaler as part of a large-scale project. The modeling tool allowed the wholesaler to bill retailers at more regular intervals than actual usage was available and allowed for long term forecasting and cash flow management. It maximized the cash on hand by accurately predicting customer consumption and growth.

The tool was built on a Microsoft stack, with SSIS used to integrate data from disparate Oracle, SQL Server and web data sources, SQL Server as the data warehouse, analysis services as the modeling tool, and Excel workbook dashboard report pack. The model performs complex calculations and takes prior usage, seasonality, actual, and forecast weather conditions (sunshine hours, wind speed, temperatures) as inputs to accrue the predicted income for the missing days of usage. The income normalizes over time as actual data becomes available.

South Australian Ambulance Service Real Time Ops dashboards (Chamonix)

The SA Ambulance Service was upgrading its call center and responder systems and required a new operational and management dashboard and reporting solution. The interesting part of this project was the limitations on only using existing technology due to being a public health/government institution, and the requirement for near real-time dashboards for the operational shift managers to allow them to monitor all aspects of incidents as they were happening and a history of all incidents for the eight-hour shift.

I was the data architect and data warehouse developer. I built a robust solution to quickly load data in as it was arriving, lightweight dashboards to render it quickly, and a framework to deal gracefully with deadlocks (complex SQL database concepts like partition switching, deadlock priority, handling lock escalation and schema locks).

SQL Server, SSIS, Reporting Services, SharePoint
1999 - 2005

Bachelor's Degree in Computer Science

Flinders University - South Australia

SEPTEMBER 2013 - PRESENT

Exam 467: Designing Business Intelligence Solutions with Microsoft SQL Server

Microsoft

SEPTEMBER 2013 - PRESENT

Exam 466: Implementing Data Models and Reports with Microsoft SQL Server

Microsoft

Tools

DataViz, SSAS, Tableau, Microsoft Power BI, Microsoft Excel, Microsoft Visual Studio, Power Pivot, Pentaho Data Integration (Kettle), Visual Studio, Microsoft Access, Crystal Reports, Qlik Sense, Amazon Athena, GitHub, Cognos Analytics 11

Languages

T-SQL (Transact-SQL), SQL, MDX, Visual Basic for Applications (VBA), PHP, Visual FoxPro, XML, C#, Python

Frameworks

Data Lakehouse, .NET

Paradigms

Dimensional Modeling, OLAP, ETL, Business Intelligence (BI), Database Design, Kimball Methodology, Dataflow Programming, DevOps

Platforms

Windows, Azure, Databricks, Azure Synapse, BIRT, Pentaho, Oracle, Amazon Web Services (AWS), Alteryx, Microsoft Fabric

Storage

Microsoft SQL Server, Data Integration, SQL Server Analysis Services (SSAS), SQL Server Reporting Services (SSRS), SQL Server Integration Services (SSIS), SSAS Tabular, Database Architecture, Databases, Azure SQL Databases, Redshift, Azure SQL, Amazon S3 (AWS S3), Database Administration (DBA), Master Data Management (MDM), MySQL, Datastage, PostgreSQL, Data Lakes, Data Pipelines

Other

Data Engineering, Data Marts, Data Migration, Data Aggregation, Data Modeling, Business Intelligence Architecture, Data Warehousing, Data Warehouse Design, SSRS Reports, Software Development, Data Visualization, Data Architecture, Dashboard Design, Dashboard Development, CSV, Dashboards, Reports, Reporting, Query Optimization, Star Schema, ELT, Business Intelligence (BI) Platforms, ETL Tools, Data Analytics, Data Profiling, Data Analysis, Azure Data Lake, Azure Data Factory, Big Data, Real-time Business Intelligence, Embedded Business Intelligence, Multidimensional Expressions (MDX), Analytics, DAX, Data Build Tool (dbt), Delta Lake, CDC, Big Data Architecture, SSIS Custom Components, Data Vaults, Data Mining, CI/CD Pipelines, MDM, Slowly Changing Dimensions (SCD), Computer Science

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring