Kishore Kannapurakkaran
Verified Expert in Engineering
Data Integration Architect and Developer
Kishore is a data integration architect with over 15 years of experience designing and developing ETL and ELT pipelines and processes to implement data warehouses, data marts, and data lakes. He has extensive experience in Azure Synapse Analytics, IBM DataStage, Pentaho Data Integration, and database systems Azure SQL Data Warehouse, SQL Server, Oracle, and IBM Db2. Kishore also has vast experience developing and maintaining stored procedures and packages in SQL Server and Oracle.
Portfolio
Experience
Availability
Preferred Environment
Azure Synapse, Linux, Python, IBM InfoSphere (DataStage), Oracle, Azure SQL, Pentaho, Scribe Server, SQL Server 2016, Windows
The most amazing...
...thing I've developed is an ELT data pipeline architecture that leverages reusable generic data flows in Azure Synapse Analytics to load tables in an ODS.
Work Experience
Data Integration Architect
NYC Department of Education
- Designed and developed an ELT framework in Azure Synapse Analytics using data factory pipelines and data flows to extract data from various on-premise systems into Azure Data Lake, transform, and load it into the Azure SQL Data Warehouse system.
- Created views and materialized views in Oracle using complex SQL code to support various business reports. Developed stored procedure packages in Oracle and SQL Server to provide data for the school location, staff, student, and adult APIs.
- Developed batch jobs in Scribe Workbench to update the database objects in Microsoft Dynamics 365 CRM for an edtech portal and universal pre-K enrollment outreach program. Implemented a change capture process to improve the CRM upload performance.
- Collaborated with DOE vendors for requirements gathering, analysis, and development to create data feeds from SAS Output Delivery System (ODS) to the vendor.
- Designed data models using erwin Data Modeler for an operational data store containing data from multiple sources. Created a source to target mapping documents with detailed data transformation rules and business logic from source systems to ODS.
- Developed ETL jobs in IBM DataStage to process real-time student information like attendance and active directory using IBM MQ.
ETL Technical Lead
United Guaranty Corp
- Collaborated with business analysts and SAP functional analysts to understand the business requirements and created a source to target mapping specifications from the legacy transactional system (AS/400) to the SAP pre-processor files.
- Performed data profiling on source data (AS/400) to investigate special characters, nulls, and data anomalies using IBM QualityStage. Designed an error handling engine to perform data validations and error checks and manage and report data errors.
- Developed the Unix shell, i.e., Bash scripts for file validations, FTP files, file archiving, error processing, and ETL batch load automation.
ETL Technical Lead
NetApp
- Worked with solution architects to design the data model and the end-to-end ETL design and architecture. Prepared detailed design documents for end-to-end flow from source systems to staging and data warehouse.
- Led a team of developers in multiple data warehouse projects to code, test units, and deploy ETL jobs. Provided design recommendations and code reviews, helped with root cause analysis, and suggested performance improvements.
- Managed a database production support process addressing the resolution of data issues and enhancements.
- Prepared project plans and coordinated with cross-functional QA and UAT teams, administrators, and database administrators (DBAs) to ensure that the tickets were resolved on time.
DataStage ETL Developer
The Vanguard Group
- Created ETL job design templates and shared containers in IBM DataStage for loading Type 1 and 2, slowly changing dimensions and facts by leveraging a modular design concept. These reusable components saved ETL development and unit testing efforts.
- Fixed performance issues in batch jobs by optimizing SQL, implementing indexing and data partitioning strategies, and tuning ETL code. Removed duplicate data in Type 2 SCD tables, retaining only the relevant historical data.
- Designed and developed jobs to extract, transform, and load data into the data warehouse hub and various data marts from multiple sources like legacy Oracle databases, DB2 EnterpriseDB, and flat files from external vendors.
Experience
DaaP Data Lakehouse
Edtech CRM Project
SAP FS-CD
Pentaho ETL Conversion
AutoSupport (ASUP) Integration
Small Business Service Hub and Data Mart
Analytics Enablement Hub
• Reducing cost and time of delivery for the data warehouse development.
• Automating a decision science process and responding faster to new decision science requests.
• Using predictive models to drive the service model and show measurable results.
• Filling data gaps in the data warehouse to support strategic questions involving the web, call center, mail, and email.
• Improving the campaign targeting, selection, and measurement.
Skills
Languages
SQL, Python
Tools
IBM InfoSphere (DataStage), Scribe Server
Paradigms
ETL, Dimensional Modeling, ETL Implementation & Design
Platforms
Oracle, Oracle Database, Azure Synapse, Linux, Pentaho, Azure SQL Data Warehouse, Azure, Dedicated SQL Pool (formerly SQL DW), Windows
Storage
SQL Server 2016, Databases, Microsoft SQL Server, Azure SQL, Data Pipelines, PL/SQL, SQL Performance, Data Integration, DB2/400, Datastage
Other
Data Warehousing, ETL Tools, Data Cleansing, Programming, Data Migration, Azure Data Lake, Parquet, Data Engineering, Azure Data Factory, Computer Automation Design (CAD), Data Marts, erwin Data Modeler, Dynamics CRM 365, Change Data Capture, Unix Shell Scripting, Slowly Changing Dimensions (SCD), Performance Tuning
Education
Master's Degree in Engineering
University of Illinois Chicago - Chicago, Illinois, United States
Bachelor's Degree in Engineering
National Institute of Technology - Calicut, India
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring