
Naresh Reddy
Verified Expert in Engineering
Azure Data Factory (ADF) Developer
Hyderabad, India
Toptal member since June 18, 2020
Naresh is a data engineer and architect with more than nine years of experience in a wide range of technologies including, MSBI stack, Azure data services, AWS data services, ETL tools such as SSIS and Talend, and reporting tools including Power BI and Tableau. He has designed databases, warehouses and has architected ETLs for several clients. Naresh has worked with various fields including, health insurance, oral care, education, partner, and workforce management.
Portfolio
Experience
- Microsoft SQL Server - 5 years
- SQL Server Integration Services (SSIS) - 4 years
- Microsoft Power BI - 2 years
- Azure Data Factory (ADF) - 2 years
- Azure - 2 years
- Python - 1 year
- Redshift - 1 year
- Tableau - 1 year
Availability
Preferred Environment
Microsoft SQL Server, Azure, Azure Data Factory (ADF), Azure Data Lake, Databricks, Microsoft Power BI
The most amazing...
...thing I did was analyze and optimize the ETL execution time and maintenance cost of an ETL system on Azure production env from $3,000 to $1,000 per month.
Work Experience
Senior Data Engineer
ACS Solutions
- Designed and implemented the Batch framework to distribute the database load by moving the business logic from DB procs to the web servers. This cut the invoice generation time drastically and provided an opportunity to scale out an on-demand basis.
- Implemented HADR for the transactional databases using transactional replication.
- Created SQL jobs for automating database recovery strategy, which is done by creating jobs for database backups and restores.
- Produced the design and implemented a framework to automate database deployments and helped the client make continuous deployment a reality.
- Constructed and implemented a framework to verify the data quality in the system by running tests and sending the reports to the business stakeholders frequently.
- Built the scripts to send mail to the business stakeholders with the SQL job execution details. These jobs include business ETLs, test cases, backups, and other maintenance tasks.
Data Engineer
MAQ Software, LLC.
- Built the reporting database for a data mart and implemented the ETL to populate the database.
- Created and designed stored procedures in SQL server databases to implement business rules.
- Constructed and designed views that are exposed to the downstream systems and the reports. Row-level security is incorporated into these views.
- Implemented a sliding window partition to store the required snapshots of the transactional tables in the mart.
- Designed and implemented logic to purge historical or cold data and aggregated it to maintain minimal historical information. This improved performance and reduced the retrieval time of the most frequently fired queries on the latest or hot data.
- Created a BIML script to automate the SSIS package generation, which helped reduce the development cost of ETL.
Experience
Azure ETL and Reporting
As part of the project, we collected data generated by various departments. We moved it from multiple sources, e.g., Azure SQL DB, SharePoint lists, Cosmos DB APIs, and Blob storage, into Azure Data Lake Store using Azure Data Factory. The data was processed and transformed into dimensional model objects using Azure Databricks. We also pulled the data into Azure Analysis Services and created the KPIs needed for Power BI reporting.
Python ETL
I created a generic framework using Python and the Singer library to move data from any source to any target using CLI. I used Python to read data from various source API endpoints and staged the data into a centralized database for the data science team to consume.
Health Insurance
• Premium calculation for a given month
• Calculating the subsidies paid by the government considering the policy holder's economic slab
• Other miscellaneous charges or discounts provided by the insurance company, e.g., the discounts for newborns
• Accepting and distributing the cash paid across various billing periods
• Advancing pay through the date of a policy
• Generating a monthly invoice
BI Reporting
I coded for several stored procedures and enabled optimal load times of the BI reporting DB. Implemented sliding window partitioning for historical tables and achieved better query performance. Actively participated in all the design and architecture activities of the project. Helped five fresh graduates in the team to ramp up quickly and work on core project activities.
BI Reconciliation
Our team followed Agile software development methods, meaning daily status reports are sent to clients informing them of our progress. Daily code reviews and daily requirements calls are held to discuss client requirement changes.
Education
Bachelor's Degree in Computer Science and Engineering
Jawaharlal Nehru Technological University - Anantapur
Certifications
DP-200: Implementing an Azure Data Solution
Microsoft
Microsoft Certified Solution Expert - Data Platform
Microsoft
Skills
Libraries/APIs
PySpark
Tools
Talend ETL, Tableau, TFS, SSAS, Microsoft Power BI, Azure Key Vault, Amazon Elastic Container Service (ECS), AWS Batch, Amazon CloudWatch
Languages
Snowflake, SQL, JavaScript, C#, Python, HTML5, CSS3
Storage
SQL Server Integration Services (SSIS), Microsoft SQL Server, Azure SQL Databases, Redshift, SSAS Tabular, Greenplum, Amazon S3 (AWS S3), Google Cloud Storage
Frameworks
Selenium, AngularJS
Paradigms
Coded UI Tests, Azure DevOps
Platforms
Azure, Amazon Web Services (AWS), Windows, Linux, Databricks, AWS Lambda, Docker
Other
Azure Data Factory (ADF), Azure Analysis Services, Azure Data Lake, Data Analytics, Data, Azure Databricks, Google BigQuery, MSBI
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring