Rakesh is available for hire

Rakesh Patel

Verified Expert in Engineering

Big Data Developer

London, United Kingdom

Toptal member since June 24, 2020

Expertise

Big Data Architecture Oracle Development Hadoop Spark Apache Kafka Amazon S3 Java Scala Apache

Bio

Rakesh has over two decades of data analysis, design, and development experience delivering data-centric solutions in the investment banking, insurance, and media sectors. He has thorough knowledge of data warehousing, transactional systems, and high-throughput processing, and he is a specialist in the fields of cloud computing, Hadoop, and big data processing.

Portfolio

Camelot Group

Apache Kafka, Java, Scala, Apache Spark, Amazon Athena, Amazon S3 (AWS S3)...

BGC Partners

Apache Thrift, Yarn, Elastic, Apache Airflow, Spark SQL, Spark, Cassandra...

Cognizant UK

Cassandra, Spark SQL, Spark Streaming, Apache Kafka, Synapse, Databricks, Azure...

Experience

Apache Kafka - 6 years
Big Data - 6 years
Hadoop - 6 years
Spark - 6 years
Spark SQL - 6 years
Amazon S3 (AWS S3) - 3 years
Amazon Elastic MapReduce (EMR) - 3 years
Kafka Streams - 1 year

Availability

Part-time

Preferred Environment

Amazon Web Services (AWS), Amazon CloudWatch, AWS Lambda, Amazon Simple Queue Service (SQS), Data Modeling, Oracle PL/SQL, Scala, Java, Amazon EC2, EMR, Amazon S3 (AWS S3), Apache Kafka, Apache Spark

The most amazing...

...thing I've created is a Swiss Army knife of utilities to quickly get projects started, with frameworks to automate the delivery of data engineering pipelines.

Work Experience

AWS/Big Data Architect

2019 - PRESENT

Camelot Group

Developed the Enterprise Data Lake on AWS: a serverless, event-driven framework that enables ingestion of data from any source in any format and a schema-less solution that dynamically accounts for changes in the source data.
Enabled a platform that supports the multiple communities of data scientists, sales, marketing, and business intelligence.
Took a legacy platform developed in Business Objects and Talend to create a code-free configuration-driven platform to enable the timely provision of data within the organization.
Developed strong links between the various technology areas of information security, enterprise architecture, and devops to ensure solution conformance.
Worked with external vendors and solution providers in respect to Delphix and Snowflake.

Technologies: Apache Kafka, Java, Scala, Apache Spark, Amazon Athena, Amazon S3 (AWS S3), Amazon EC2, EMR, AWS Glue, AWS Lambda, Amazon Simple Queue Service (SQS), API Gateways

Big Data Architect/Developer

2018 - 2019

BGC Partners

Developed a platform for capturing all regulatory related business data for internal data scientists and reporting teams.
Developed a schema-less solution enabling data of any schema or format to be ingested and transformed dynamically into a reportable format that handles schema evolution on the fly: totally configuration-driven without any code.
Installed and set up NFS Gateway, Samba, Apache Ranger, and Apache Airflow to better manage security, data access, and job orchestration.
Set up the Spark Graphite sinks to Prometheus to better monitor the Spark Streaming applications.
Upgraded the HDP Stack from 2.6.1 to 3.1, creating on-premise repos to manage updates and upgrades effectively.
Ensured business continuity during the move of physical servers to ensure the smooth stop and restarting of the big data technology stack.
Developed devops capability that automates product platform using Docker and Ansible to create tearable environments for Cassandra, Elastic Stack, HDP, and Airflow.
Retuned the existing Elastic Stack platform to enhance indexing capabilities along with Index Life Cycle Management.

Technologies: Apache Thrift, Yarn, Elastic, Apache Airflow, Spark SQL, Spark, Cassandra, Apache Kafka, Ansible, Docker

Principle Big Data Architect

2018 - 2018

Cognizant UK

Designed a Google Cloud-based solution for a major multimedia company to perform consumer-driven advertising based on consumer subscription and viewing patterns.
Built out an ETL solution on AWS for an energy trading firm to ingest public open data for real-time data visualization. The solution enabled the client to ingest and transform data using a bespoke framework that minimized coding.
Designed a baggage tracking system on Azure using Azure Databricks for a major international airport to allow the client to track, monitor performance, and manage their baggage handling processes.
Recruited new hires and upskilled graduates into the field of big data development.
Involved in the successful bidding of a number of public sector big data contracts.

Technologies: Cassandra, Spark SQL, Spark Streaming, Apache Kafka, Synapse, Databricks, Azure, Amazon S3 (AWS S3), Scala, Java

Big Data Developer

2017 - 2018

Barclays Bank

Built out a framework using Spark Streaming to perform real-time ELT data processing.
Optimized existing Spark applications running in Yarn.
Developed an operational data-lake on a Kerberos secured HBase Cluster.

Technologies: HBase, Apache ZooKeeper, Kerberos, Apache Kafka, Spark Streaming, Spark, Yarn, Hadoop, Cloudera

Lead Big Data Developer

2016 - 2017

UK Government Home Office

Oversaw the move to Graph databases to better model their data relationships.
Developed an AWS MI reporting system for both batch and real-time CEP style reporting.
Transitioned an existing developer base into the new technologies of big data computing using Apache Spark and Kafka.

Technologies: PostgreSQL, Cassandra, Kafka Streams, Spark Streaming, Apache Kafka, Spark, GRAPH, DataStax, Hadoop

Big Data/Hadoop Consultant for Finance IT

2014 - 2016

HSBC Bank

Delivered the future state architecture designed for high-throughput processing capabilities to deliver the next-generation of banking applications within Finance IT.
Developed an ETL solution based on HDFS, JSON, and Spark for building out the global reporting platform with 96 reports developed in under six months.
Oversaw the technical review of the full suite of emerging big data technologies with the aim of choosing the correct set of technologies that would provide financial and competitive advantage for the investment bank.

Technologies: Amazon Web Services (AWS), Ansible, Apache Hive, Spark Streaming, ELK (Elastic Stack), Flume, Apache Kafka, Akka, Scala, Java, Spark, HDFS, Hadoop

Database Performance Tuning specialist for Finance IT

2013 - 2014

HSBC Bank

Reduced SLA breaches to zero for the finance IT strategic accounting hub and regulatory reporting platform.
Increased system throughput to enable greater business growth. Reduced the daily batch run times by 80% and improved platform stability to allow the take-on of future business.
Reduced development and build times with the creation of build management frameworks used extensively across a number of finance IT systems.
Owned all the data management frameworks within the bank’s accounting platform. Reduced data volumes by defining several key data archiving processes.
Worked with business leaders to establish data controls to limit data growth while increasing business growth.
Enabled reduction in storage costs and further project costs by developing the enablers that have allowed for wholesale system migration and storage space reductions.
Mentored a global team across Brazil, India, and China to establish coding standards and build quality.
Automated build and delivery frameworks used globally within the finance IT department.
Performance tuned global finance systems, working quickly to identify issues to solve immediate emergencies that arise across any finance IT system.

Technologies: Optimization, SQL, XML, Ant Design, Java, Oracle 11g, Oracle 10g

Lead Oracle Developer for Cashflow/Collateral/Liquidity

2012 - 2013

Barclays Bank

Developed numerous data feed processors to allow the BA team to analyze data requirements.
Worked with the data architecture teams to deliver a physical database to manage cash flow and intra-day liquidity funding.
Reviewed and evaluated column-store databases to provision for real-time feed processing and reporting requirements.
Reviewed performance for the Spring integration framework to ensure volume capabilities are met.

Technologies: MongoDB, Kdb+, SAP HANA, Spring Batch, Spring Integration, JSON, XSLT, XML, Java, Oracle 10g, Oracle 11g

Oracle Lead Developer for Liquidity Risk/Treasury IT

2010 - 2012

Deutsche Bank

Performance tuned an ailing liquidity risk/treasury reporting system. Key technical and delivery lead taking on the task of managing and mentoring an offshore development team, with full ownership of system delivery.
Introduced rigorous coding standards, code delivery, build, and deployment processes to improve stability and allow the department to take on more business.
Re-tuned the 80+ feeds into the liquidity risk reporting system. Improved performance and system delivery.
Developed coding standards, performance metrics, release procedures that improved system stability during the monthly release cycles, code quality, and source control.
Reduced system outages and reporting delays to near zero.
Trained a mix of senior/junior offshore developers to ensure robust code delivery.
Built the core components that defined the foundation of the bank's treasury reporting system.

Technologies: XML, Oracle, Ant Design, Subversion (SVN), Java, Oracle 11g, Oracle 10g

Lead Oracle Developer for Finance Accounting

2009 - 2010

Barclays Bank

Owned and led the implantation, migration, and configuration management for the build, implementation and delivery of the database. Designed an automated build and deploy process using Java and ANT.
Developed the migration framework that ensures accountability within the bank to ensure the correct phased delivery of the system.
Worked with the key business users to ensure the smooth running of the project and ensuring the correct transformation of business requirements to technical solutions.
Developed the feeds to SAP and the various reconciliation systems within the bank.
Managed teams in Singapore, London, and New York to provide a 24/7 development center.
Worked with the bank's accountants and key business sponsors to ensure that all project expectations are fully met or exceeded.
Played a key role in the go-live of the system and the project's success.

Technologies: VB, Microsoft Excel, XSLT, XML, Perforce, SQL, Oracle, Java, Ant Design, Exadata, Oracle 11g

Lead Oracle Developer

2007 - 2009

NYSE Euronext

Enhanced the company's existing cash and derivatives trading platform, CONNECT. Performance tuned the database to handle the millions of transactions and messages that are expected of trading platforms.
Designed and developed the new pricing and settlement services system for pricing and settling contracts. This also involved updating the schema used by the existing trading system.
Developed the key messaging framework using Oracle AQ to communicate data changes between the database and all the components/services that use the database.
Worked with various business leads to develop test data and data loading frameworks to configure and test the new trading, pricing, and settlement engines.
Performance tuned key aspects of the database to ensure that the trading platform has optimal sub-second responses.
Developed trading platforms for Liffe, NYSE Metals, Tokyo Stock Exchange, and Tokyo Futures Exchange.

Technologies: Java, VB, Microsoft Excel, XSLT, XML, IBM Rational ClearCase, PL/SQL, Oracle, Oracle 10g

Lead Oracle Developer

2006 - 2007

HSBC Bank

Developed the bank's flagship data warehouse for asset servicing.
Developed cross-database tools to provide single and coherent views of data across Oracle, Sybase, and SQL Server databases. Used the tools for data migration, data reconciliation, and reporting.
Developed the bank standard messaging framework for persistent and reliable message delivery. Integrated messaging platforms via Oracle Messaging Gateway between MQ Series to Oracle AQ.
Mentored and empowered team members, performance-tuned other projects, designed and data-modeled other systems, and filled in the gaps as a Java developer when required.

Technologies: PL/SQL, Messaging, IBM MQ, Oracle, XSLT, XML, Sybase, Microsoft SQL Server, Oracle 11g, Oracle 10g

Lead Oracle Developer

2005 - 2006

Barclays Banlk

Re-architected the development infrastructure for legal and compliance IT.
Developed numerous feeds, based on different techniques, for loading data into the corporate data warehouse.
Redeveloped the existing scheduler system. Designed a new data model with improved scheduling processes and a web interface to control and support the scheduler.
Developed a generic reporting framework and data model to allow rapid production of user required reports. This provides tabular reports and highly customized reports with output to HTML, PDF, or Excel.
Developed a generic user administration web application that manages user application access. Integrated with a corporate LDAP server and application-specific authentication schemes.
Developed standards for application deployment, web development, Oracle performance tuning, PL/SQL coding standards, unit and regression testing, and system documentation.

Technologies: Ant Design, Messaging, XSLT, XML, Apache Struts, Java Servlets, PL/SQL, Perl, Unix, Data Modeling, Oracle 10g

Lead Oracle Developer

2001 - 2005

Deutsche Bank

Developed the global reporting systems for loan exposure management. Developed to ensure compliance to the BASEL II accord and provide key loan hedging decision-making systems to reduce the bank’s exposure to loans.
Delivered several back-end banking systems for loan exposure reporting.
Developed front/middle office ticketing systems used by controllers/traders to create new loans and for hedging of loans using credit derivative swaps, collateralized loan obligations, and loan sales.
Developed an active data warehouse that performs real-time refresh of the data warehouse for accurate reporting and hedge planning.
Managed building of an offshore development team based in Moscow. Mentored a support team based in New York, London, and Moscow.
Delivered numerous coding standards and development frameworks for reporting, feed management, processing/loading/transformation of flat files and XML data files, and general database development.

Technologies: Java, Performance Tuning, SQL, Unix, Oracle, Data Modeling, Actuate, Oracle8i

Lead Oracle Developer

2000 - 2001

Wanadoo France Telecom

Developed an internet content management tool aimed at delivering device independent content for Wanadoo, a France Telecom subsidiary. The tool is used to produce the Dutch Wanadoo website.
Designed and developed the database. In addition to normal duties of DBA, developed all stored procedures used by the Java middle tier for managing the system. Developed numerous database management scripts and performance monitoring views.
Developed Java utilities for JDBC connection management, LOB loading/retrieval of binary (images, streaming video, Flash) and character data, data caching in the middle tier and server side session tracking, and array handling for Java and Oracle.
Created web delivery frameworks using Java Servlets and JSP that take Oracle queries and transform the results using XSL to HTML/WML.

Technologies: Java, Oracle, SQL Loader, PL/SQL, Oracle8i

Oracle Developer

1999 - 2000

Deutsche Bank

Maintained the integrity of the data in the system. This includes managing all the counterparties, legal agreements, limits, exposures, and trade information.
Developed and managed feeds for CRMS with the bank's counterparty, risk engines, and trading systems. These include derivatives, FX, equity, money markets, MMJV, metals, etc.
Developed a business-focused reporting intranet site.
Performance-tuned the system. Enabled the corrective tracking of all data-related issues and reconciled the client-side application against the database.

Technologies: XML, Jakarta Server Pages (JSP), JDBC, Java, Unix Shell Scripting, PL/SQL, Oracle8i

Lead Datawarehouse Developer

1998 - 1999

General Electric Insurance Holdings

Designed and implemented of a number of data warehousing systems.
Developed, revised and extended the detailed operational processes and procedures necessary for warehouse/decision support type systems, e.g., load scripts, third-party software interfaces, SQL tuning, and backup processes.
Formed a set of best practices for database design and administration. This covers data modeling naming standards, code development standards, and standardization of processes for performing database administration.
Supported various development teams to aid in their development efforts against an Oracle database.

Technologies: Storage, Unix, Perl, PL/SQL, Oracle Discoverer

Full-stack Developer

1995 - 1998

Cambridge Technology Partners

Led the design and development of an annuities payment system data model.
Designed and developed a global CRM system for an asset management firm.
Developed a Java web based sales capture system for field sales teams.
Developed and supported the workflow and imaging system. Established the performance tuning framework for the application and the development of the front-end application.

Technologies: Unix, Visual Basic, Java, Perl, Sybase PowerDesigner, PL/SQL, SQL

Experience

Business Intelligence Data Lake for Camelot Group UK

http://www.camelotgroup.co.uk/

Developed the enterprise data lake using the best of bred technologies on AWS: a serverless, event-driven framework that enables ingestion of data from any source in any format. It is a schema-less solution that dynamically accounts for changes in the source data. I architected it using a real-time first approach that also enables batch processing.

Education

1989 - 1994

Masters of Engineering Degree in Electronic Engineering Design

King's College London - London

Certifications

AUGUST 1994 - PRESENT

Thomson Broadcast Award for Top Engineering Student

Thomson Broadcast

Skills

Libraries/APIs

Spark Streaming, Java Servlets, JDBC

Tools

Spark SQL, Amazon Elastic MapReduce (EMR), Amazon Simple Queue Service (SQS), Amazon CloudWatch, AWS Glue, Amazon Athena, Ansible, Apache Airflow, Elastic, Synapse, Cloudera, Apache ZooKeeper, DataStax, Flume, ELK (Elastic Stack), Subversion (SVN), Perforce, Microsoft Excel, IBM Rational ClearCase, IBM MQ, Sybase PowerDesigner, Kafka Streams

Frameworks

Hadoop, Spark, Apache Spark, Yarn, Apache Thrift, Akka, Ant Design, Spring Integration, Spring Batch, Apache Struts, Actuate, Jakarta Server Pages (JSP)

Platforms

Apache Kafka, Oracle, Amazon EC2, AWS Lambda, Docker, Azure, Databricks, SAP HANA, Unix, Amazon Web Services (AWS)

Storage

Amazon S3 (AWS S3), Oracle PL/SQL, Cassandra, HBase, PostgreSQL, HDFS, Apache Hive, Oracle 10g, Oracle 11g, JSON, Kdb+, MongoDB, Exadata, PL/SQL, Microsoft SQL Server, Sybase, SQL Loader

Languages

Java, Scala, XML, SQL, XSLT, VB, Perl, Visual Basic

Other

Big Data, EMR, Data Modeling, API Gateways, Kerberos, GRAPH, Optimization, Messaging, Oracle8i, Performance Tuning, Unix Shell Scripting, Oracle Discoverer, Storage

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring