Kiran Kumar Kolla, Database Engineer and Developer in Sydney, New South Wales, Australia
Kiran Kumar Kolla

Database Engineer and Developer in Sydney, New South Wales, Australia

Member since July 9, 2021
Kiran is a database engineer with over ten years of expertise implementing data lakes, data integrations, and data warehousing solutions. In addition to data engineering work with Cognizant and wireless with NBN Co, Kiran has enterprise experience in financial services from MetLife, ING Group, to Bank of America. He is highly motivated when challenges are thrown at him, as they bring out his best! Kiran loves to program and use logical thinking to solve complex problems efficiently.
Kiran is now available for hire

Portfolio

  • Data-Driven AI
    Azure, Azure Data Factory, Azure Synapse, Databricks, Microsoft Power Apps...
  • NBN Co
    Redshift, AWS Lambda, Alteryx, Tableau Desktop Pro, AWS S3, Apache Airflow...
  • MetLife
    Alteryx, Tableau, SQL, Dashboards, Dashboard Design, Regulatory Reporting...

Experience

Location

Sydney, New South Wales, Australia

Availability

Part-time

Preferred Environment

Azure SQL, Databricks, Azure Data Factory, Python 3, Data Warehousing, Apache Airflow, Tableau Desktop Pro, AWS, Spark, Microsoft Power Apps

The most amazing...

...thing I've implemented was an operation data lake to ingest real-time data from Open Data Hub APIs related to different transport modes.

Employment

  • Senior Data Engineer

    2020 - PRESENT
    Data-Driven AI
    • Developed Azure durable entity functions to ingest data from Open Data Hub APIs in case of changes in the response header. The action saved 8GB of storage space every day and reduced costs significantly.
    • Built an app in Microsoft Power Apps to fetch the resource usage costs and cost optimization recommendations from Azure subscription. The app helped users monitor resources and reduce costs.
    • Created a self-service API and Microsoft power app for business users to securely download historical GTFS data on-demand from a data lake without manual intervention. The action saved 40 hours of manual effort every month.
    • Built data pipelines using Azure Databricks (Pyspark) to process the ingested data into the Delta lake and the Synapse data warehouse.
    • Created streaming data pipelines using Databricks to ingest data from Event Hub and Cosmos DB.
    Technologies: Azure, Azure Data Factory, Azure Synapse, Databricks, Microsoft Power Apps, Microsoft Power Automate, Azure Functions, Azure Logic Apps, Azure Automation, Azure DevOps, Data Lakes, Data Engineering, Python, C#, Microsoft Power BI, PySpark, Azure IaaS, Data Pipelines, T-SQL, Pandas, Data Cleansing
  • Data Engineer

    2019 - 2020
    NBN Co
    • Developed automated ETL data pipelines using Spark on AWS EMR and Lambda to process the data, such as orders and services, from Comptel 7 and Maximo into an AWS S3 data lake.
    • Created event-based Lambda functions using Python to trigger the Glue crawlers and metadata tagging.
    • Developed data pipelines to transform order data from the UDS KPI model into business-friendly views that can easily be used for data analysis and reporting.
    Technologies: Redshift, AWS Lambda, Alteryx, Tableau Desktop Pro, AWS S3, Apache Airflow, ETL, AWS EMR, Spark, Data Lakes, Python, AWS, Tableau, Data Modeling, SQL, SQL Server Integration Services (SSIS), PySpark, T-SQL, Pandas, Data Visualization, Amazon Web Services (AWS), Data Cleansing
  • Data Analyst

    2018 - 2019
    MetLife
    • Created Alteryx workflows to extract data from multiple source systems and prepare the data for regulatory submission.
    • Prepared the data for regulatory reporting (including APRA, ASIC, and LCCC) and generated reconciliation dashboards.
    • Designed and developed Tableau functional dashboards based on business requirements.
    Technologies: Alteryx, Tableau, SQL, Dashboards, Dashboard Design, Regulatory Reporting, T-SQL, Data Visualization, Data Cleansing
  • Data Engineer

    2017 - 2018
    ING Group
    • Implemented Common Reporting Standard (CRS) workflows to generate customer and account extracts.
    • Developed an ETL framework using Informatica workflow, Windows PowerShell, and SQL to automate the batch scheduling process for the data warehouse.
    • Created the data model for the personal loans and credit card products to load the transactional data.
    Technologies: Informatica ETL, Data Warehouse Design, Python, Windows PowerShell, Data Warehousing, SQL, ETL, Dimensional Modeling, T-SQL
  • Senior Analyst

    2013 - 2017
    Bank of America
    • Implemented an Informatica ETL COE operations reporting solution to help managers generate reports on platform usage, the number of sessions executed, and the volume of data processed.
    • Automated the process of monthly metrics reports for application support and platform maintenance teams, using Excel VBA and Microsoft SQL Server, which saved a lot of manual effort.
    • Completed the process of upgrading all the Informatica ETL COE platforms to v10.2 by automating the process with Jenkins and shell scripting.
    Technologies: Informatica ETL, Shell Scripting, Tableau, ETL, Data Visualization
  • ETL Developer

    2008 - 2013
    Cognizant
    • Developed automatic data reconciliation workflows to compare the data with control files received from the source system.
    • Created Autosys workflows to set up job dependencies and schedules.
    • Created stored procedures using Oracle PL/SQL to perform the data validation.
    Technologies: Informatica ETL, Oracle, SQL Server Integration Services (SSIS), Autosys, Data Cleansing

Experience

  • Operational Data Lake for Transport

    An operational data lake for transport that ingests and processes real-time trip updates and vehicle position data from IoT devices. I developed the data lake, which forms the data foundation for many business use cases as part of multimodal performance reporting.

  • Web Scraping
    https://www.teamrankings.com

    A Python executable to extract the data from the TeamRanking website about on-the-day schedules and, for those teams, get the latest rankings for different categories. I created this using Pandas, openpyxl, and Beautiful Soup libraries to extract data from the NCAA website and generate an Excel file with all the teams and their strengths in different areas, which are color-coded.

  • Monitoring Application for Azure Cloud

    A Microsoft Power App to monitor resources and their consumption costs across multiple subscriptions. I developed the app and created functions to fetch the resource and cost information from Azure Subscriptions used by the Power App, which helps the cloud admin monitor costs and the resources created.

Skills

  • Languages

    Python 3, SQL, Python, T-SQL, C#
  • Frameworks

    Windows PowerShell, Spark, Azure Bot Framework, AWS EMR
  • Libraries/APIs

    Pandas, PySpark, Azure Cognitive Services, Beautiful Soup
  • Tools

    Azure Logic Apps, Informatica ETL, Tableau Desktop Pro, Tableau, Microsoft Power Apps, Autosys, Azure Automation, Apache Airflow, Azure Machine Learning, Microsoft Power BI
  • Paradigms

    ETL, Dimensional Modeling, Azure DevOps, DevOps, Business Intelligence (BI)
  • Platforms

    Databricks, Azure, Oracle, Azure Functions, Alteryx, Azure IaaS, Azure PaaS, AWS Lambda, Amazon Web Services (AWS)
  • Storage

    Azure SQL, Data Pipelines, Data Lakes, SQL Server Integration Services (SSIS), Redshift, AWS S3
  • Other

    Azure Data Factory, Data Warehousing, Azure Data Lake Analytics, Software Development, Data Warehouse Design, Shell Scripting, Data Engineering, Data Architecture, Data Cleansing, Azure Synapse, Microsoft Power Automate, Web Scraping, Dashboards, Dashboard Design, Data Modeling, Microsoft Data Transformation Services (now SSIS), Data Analyst, Data Visualization, Cloud Security, Cloud Services, Data Analytics, AWS, Regulatory Reporting, Internet of Things (IoT), Excel 365

Education

  • Bachelor's Degree in Computer Science
    2002 - 2006
    Jawaharlal Technological Universilty - Hyderabad, India

Certifications

  • Azure AI Fundamentals
    JULY 2021 - PRESENT
    Microsoft
  • Azure Data Engineer Associate
    DECEMBER 2020 - DECEMBER 2022
    Microsoft
  • Azure Fundamentals
    SEPTEMBER 2020 - PRESENT
    Microsoft
  • Tableau Desktop Certified Associate
    AUGUST 2018 - JANUARY 2021
    Tableau

To view more profiles

Join Toptal
Share it with others