Dan Larson, Data Engineer and Developer in Dubai, United Arab Emirates
Dan Larson

Data Engineer and Developer in Dubai, United Arab Emirates

Member since September 13, 2022
Dan is a seasoned developer with deep experience in application and web development, data engineering, and leading the technical implementation of master data management (MDM) solutions. Passionate about mathematics, science, and all kinds of engineering, he loves finding ways to connect those fields with technology. Dan has excellent problem-solving skills and enjoys combining various technologies to produce results that exceed expectations and make people's jobs easier.
Dan is now available for hire

Portfolio

Experience

Location

Dubai, United Arab Emirates

Availability

Full-time

Preferred Environment

SQL Server Integration Services (SSIS), Profisee MDM, Azure Data Factory, Microsoft SQL Server, C#, ASP.NET

The most amazing...

...thing I've developed is a metadata repository with auto-generated data lineage, enabling users to search and see objects and data sources on a funny web tool.

Employment

  • MDM Technical Lead | Product Owner

    2021 - 2022
    US Financial Services Company
    • Led the migration of the enterprise Profisee master data management system to Azure hosting, organizing the MDM code repository and deployment strategy.
    • Created automated integrations between the company MDM system and Salesforce, Snowflake, and other data sources.
    • Developed workflows to automate and streamline the review of changes.
    • Oversaw data maintenance, provided audit support, and trained data stewards and data governance on usage and best practices.
    Technologies: Profisee MDM, Azure SQL Databases, ETL, Data Governance, Master Data Management (MDM), Salesforce, SQL, Snowflake, Data Engineering, Data Architecture, Data Modeling, Azure, APIs, Azure DevOps, Database Performance, Data Warehousing, Data Management, Data Pipelines, Azure Synapse, .NET, C#.NET, .NET Core, Windows PowerShell, T-SQL (Transact-SQL), Data Strategy, Data, Metrics, CSV, Dashboards, SQL Server DBA, Database Lifecycle Management (DLM), Microsoft DBA, Team Leadership, Windows Server, ServiceNow, Database Security, Project Management, Database Architecture
  • IM Engineer Technical Lead | MDM Architect

    2016 - 2021
    Global Food and Agriculture Company
    • Led the technical implementation of an enterprise MDM solution using Profisee tools. Developed a microservice and the application layer for master data integrations, including APIs, Azure Service Bus messaging, and workflows for data authoring.
    • Created data developer tools, including searchable interfaces for data lineage and job incident history, combining automation data from BMC Control-M with Cherwell ticket data, SSAS cube and tabular metadata, and SQL Server metadata.
    • Led the development efforts for a team working on a C# Web API and Angular intranet site. Led code reviews, created a deployment management process, and developed a method to crawl integration packages and database objects to display data lineage.
    • Collaborated with junior engineers on support and debugging issues, consulting on enterprise guidelines and best practices, and organizing Git repositories, including a database deployment system.
    • Provided development and production support for an organization with over 150 SQL Server instances, including writing and debugging SQL procedures and views and providing data for ad-hoc reporting. Developed tools to quickly debug priority failures.
    • Developed many data processes using SSIS and SQL objects and oversaw automation with BMC Control-M. Provided support for SSIS packages and SQL scripts from other developers as needed.
    Technologies: SQL Server Integration Services (SSIS), Microsoft SQL Server, C#, ETL, Profisee MDM, Data Quality, Data Lineage, Metadata, Automation, Control-M, Web API, AngularJS, Python, SQL, Data Engineering, Data Architecture, Data Modeling, Azure, APIs, Microsoft Excel, Excel 2016, Angular, CI/CD Pipelines, Azure DevOps, Database Performance, Data Warehousing, Star Schema, Data Management, Data Pipelines, Informatica, Integrated Database Management Systems (IDMS), .NET, C#.NET, .NET Core, Windows PowerShell, XML, T-SQL (Transact-SQL), Data Strategy, Data, Metrics, Big Data, Data Analytics, CSV, Dashboards, Data Visualization, SQL Server DBA, Database Lifecycle Management (DLM), Microsoft DBA, Team Leadership, Windows Server, Database Security, Project Management, Database Architecture
  • Senior Programmer and Analyst

    2015 - 2016
    Resource Data Inc. Consulting
    • Created a Web API and JavaScript package to simplify the creation of search pages. Custom classes enable the rapid creation of a full-function search grid with a predictive text search box and full-function record editor.
    • Developed new data import mechanisms using an HTML5 file API for page-side data and file processing with an asynchronous upload. It allows users to see the file parsing progress, interact with errors, and dynamically modify records before submission.
    • Led code reviews, mentored junior developers, and improved the database procedure designs and deployment strategy.
    Technologies: C#, ASP.NET, Entity Framework, JavaScript, Microsoft SQL Server, SQL, Data Modeling, APIs, Microsoft Excel, Macros, Excel 2010, Excel 2016, CI/CD Pipelines, Database Performance, Data Management, Data Pipelines, TIBCO, Integrated Database Management Systems (IDMS), .NET, C#.NET, .NET Core, Windows PowerShell, XML, T-SQL (Transact-SQL), Data, CSV, Data Visualization, SQL Server DBA, Database Lifecycle Management (DLM), Microsoft DBA, Team Leadership, Windows Server, Database Security, Project Management
  • Consulting Software Engineer

    2002 - 2015
    Freelance Clients
    • Created multiple ASP.NET websites, including a mobile-friendly sales quoting and CRM tool, customer access portal, service call tracking, installation status, and other reporting unavailable in the company's ERP system.
    • Developed a tracking system to monitor customer systems and create alerts for various system failures.
    • Implemented commission accounting systems that process millions of dollars of commissions annually using SQL Server databases and .NET applications, including Windows desktop and service applications and ASP.NET websites and web services.
    • Developed automated data feed processing for over 90% of transactions. Designed customizable reporting and auditing systems to allow staff to complete audits quickly.
    • Created a highly featured sales representatives website with features to review statements, search for detailed data, request status updates on cases, and track leads and appointments.
    • Authored a highly featured PDF generating system and reporting API to enable quick report generation.
    Technologies: C#, ASP.NET, Microsoft SQL Server, IIS, VB.NET, SQL, Data Modeling, APIs, Macros, Excel 2010, Data Warehousing, Data Management, COBOL, Integrated Database Management Systems (IDMS), .NET, C#.NET, XML, T-SQL (Transact-SQL), Data, CSV, Dashboards, Data Visualization, Microsoft DBA, Windows Server, Database Security, Project Management

Experience

  • SQL Messenger
    https://github.com/DJL-Consulting/SQL-Messenger

    The SQL Messenger project was meant to provide a simple mechanism for database monitoring. Users can write queries and add them to the control table along with the notification info targeting recipients and send criteria. It runs via a simple stored procedure, which allows different checks on different schedules defined as hourly, daily, or weekly.

    It comprises unlimited possibilities, including sending a list of orders that haven't been completed to the shipping department, a list of offline databases to the DBA team, or data quality check reports to data stewards. If users can write a query to identify a condition that needs attention, they can easily automate notifications to appropriate users with this tool.

  • Ballistic Calculator
    http://ballist.great-site.net/

    Ballistic Calculator is an easy-to-use calculator for exterior ballistics using hand-derived equations. It enables users to calculate external ballistic performance for any situation.

    Developed in React, the entirely client-side application allows users to enter information about their load, calculate results, and save favorites for later review.

    Mathematically, starting from the general-form equation for bullet deceleration due to air resistance—a = (A * V^m) / c—we can integrate twice to form equations for velocity and distance traveled, adjusting for initial muzzle velocity and gravity drop. Using a computer program to iterate, we can also calculate the amount of sight adjustment needed to compensate for drop at any given distance.

  • Data Lineage Library

    An application that allows users to find the table/view they're looking for quickly, see what objects are used to populate that data and where it flows downstream from there, and what automation jobs run the refresh function, when they're scheduled, and what happened the last time they failed. Exciting, isn't it? Users spanning ETL support to data analysts to business users saved tons of time using this helpful tool. Over 2-3 years of working closely with both Data Governance and business users, I took this project from the back-burner to something with more useful features, eventually adding all the tools necessary to have a one-stop metadata source.

    A SQL Server database system with automated 'crawlers' (C#, Python, and Powershell scripts) to populate database object dependency and lineage info from databases, integration packages, and additional metadata—automated job scheduling and status, job failure tickets, user notes—about those. A web application (C# WebAPI with an AngularJS front-end) allowed users to do a quick search, traverse the data lineage 'tree' view, and add supplemental metadata, such as business aliases, data domains, etc., to the auto-generated base.

Skills

  • Languages

    C#, Stored Procedure, SQL, C#.NET, T-SQL (Transact-SQL), JavaScript, Python, XML, VB.NET, Snowflake, COBOL
  • Frameworks

    ASP.NET, .NET, .NET Core, Windows PowerShell, AngularJS, Angular
  • Libraries/APIs

    Web API, Entity Framework, React
  • Tools

    Control-M, Microsoft Excel, Excel 2010, Excel 2016
  • Paradigms

    Database Design, ETL, Automation, Azure DevOps
  • Storage

    SQL Server Integration Services (SSIS), Microsoft SQL Server, Master Data Management (MDM), Data Pipelines, SQL Server DBA, Database Lifecycle Management (DLM), Microsoft DBA, Database Administration (DBA), Azure SQL Databases, Database Performance, Integrated Database Management Systems (IDMS), Database Security, Database Architecture
  • Other

    Profisee MDM, Algorithms, Software Engineering, Development, Data Modeling, Data Governance, Data Lineage, Metadata, Data Warehousing, Data Engineering, APIs, Macros, Data Management, Data, CSV, Azure Data Factory, Workflow, Data Analysis, Data Quality, IIS, Data Quality Analysis, Mathematics, Differential Equations, Numerical Methods, CI/CD Pipelines, Data Architecture, Star Schema, Data Strategy, Metrics, Dashboards, Data Visualization, Team Leadership, ServiceNow, Operating Systems, Architecture, Azure Synapse, Informatica, TIBCO, Big Data, Data Analytics
  • Platforms

    Azure, Windows Server, Salesforce
  • Industry Expertise

    Project Management

Education

  • Bachelor of Science Degree in Computer Science with a Minor in Mathematics
    1998 - 2003
    Texas Tech University - Lubbock, Texas, USA

Certifications

  • Master Data Architect
    JANUARY 2022 - PRESENT
    Profisee
  • Master Data Steward
    DECEMBER 2021 - PRESENT
    Profisee
  • Master Data Developer
    DECEMBER 2021 - PRESENT
    Profisee
  • Master Data System Administrator
    DECEMBER 2021 - PRESENT
    Profisee

To view more profiles

Join Toptal
Share it with others