Rakesh is available for hire
Hire RakeshRakesh Patel
Verified Expert in Engineering
Big Data Developer
Location
London, United Kingdom
Toptal Member Since
June 24, 2020
Rakesh has over two decades of data analysis, design, and development experience delivering data-centric solutions in the investment banking, insurance, and media sectors. He has thorough knowledge of data warehousing, transactional systems, and high-throughput processing, and he is a specialist in the fields of cloud computing, Hadoop, and big data processing.
Portfolio
Camelot Group
Apache Kafka, Java, Scala, Apache Spark, Amazon Athena, Amazon S3 (AWS S3)...
BGC Partners
Apache Thrift, YARN, Elastic, Apache Airflow, Spark SQL, Spark, Cassandra...
Cognizant UK
Cassandra, Spark SQL, Spark Streaming, Apache Kafka, Synapse, Databricks, Azure...
Experience
Availability
Part-time
Preferred Environment
Amazon Web Services (AWS), Amazon CloudWatch, AWS Lambda, Amazon Simple Queue Service (SQS), Data Modeling, Oracle PL/SQL, Scala, Java, Amazon EC2, EMR, Amazon S3 (AWS S3), Apache Kafka, Apache Spark
The most amazing...
...thing I've created is a Swiss Army knife of utilities to quickly get projects started, with frameworks to automate the delivery of data engineering pipelines.
Work Experience
AWS/Big Data Architect
2019 - PRESENT
Camelot Group
- Developed the Enterprise Data Lake on AWS: a serverless, event-driven framework that enables ingestion of data from any source in any format and a schema-less solution that dynamically accounts for changes in the source data.
- Enabled a platform that supports the multiple communities of data scientists, sales, marketing, and business intelligence.
- Took a legacy platform developed in Business Objects and Talend to create a code-free configuration-driven platform to enable the timely provision of data within the organization.
- Developed strong links between the various technology areas of information security, enterprise architecture, and devops to ensure solution conformance.
- Worked with external vendors and solution providers in respect to Delphix and Snowflake.
Technologies: Apache Kafka, Java, Scala, Apache Spark, Amazon Athena, Amazon S3 (AWS S3), Amazon EC2, EMR, AWS Glue, AWS Lambda, Amazon Simple Queue Service (SQS), API Gateways
Big Data Architect/Developer
2018 - 2019
BGC Partners
- Developed a platform for capturing all regulatory related business data for internal data scientists and reporting teams.
- Developed a schema-less solution enabling data of any schema or format to be ingested and transformed dynamically into a reportable format that handles schema evolution on the fly: totally configuration-driven without any code.
- Installed and set up NFS Gateway, Samba, Apache Ranger, and Apache Airflow to better manage security, data access, and job orchestration.
- Set up the Spark Graphite sinks to Prometheus to better monitor the Spark Streaming applications.
- Upgraded the HDP Stack from 2.6.1 to 3.1, creating on-premise repos to manage updates and upgrades effectively.
- Ensured business continuity during the move of physical servers to ensure the smooth stop and restarting of the big data technology stack.
- Developed devops capability that automates product platform using Docker and Ansible to create tearable environments for Cassandra, Elastic Stack, HDP, and Airflow.
- Retuned the existing Elastic Stack platform to enhance indexing capabilities along with Index Life Cycle Management.
Technologies: Apache Thrift, YARN, Elastic, Apache Airflow, Spark SQL, Spark, Cassandra, Apache Kafka, Ansible, Docker
Principle Big Data Architect
2018 - 2018
Cognizant UK
- Designed a Google Cloud-based solution for a major multimedia company to perform consumer-driven advertising based on consumer subscription and viewing patterns.
- Built out an ETL solution on AWS for an energy trading firm to ingest public open data for real-time data visualization. The solution enabled the client to ingest and transform data using a bespoke framework that minimized coding.
- Designed a baggage tracking system on Azure using Azure Databricks for a major international airport to allow the client to track, monitor performance, and manage their baggage handling processes.
- Recruited new hires and upskilled graduates into the field of big data development.
- Involved in the successful bidding of a number of public sector big data contracts.
Technologies: Cassandra, Spark SQL, Spark Streaming, Apache Kafka, Synapse, Databricks, Azure, Amazon S3 (AWS S3), Scala, Java
Big Data Developer
2017 - 2018
Barclays Bank
- Built out a framework using Spark Streaming to perform real-time ELT data processing.
- Optimized existing Spark applications running in Yarn.
- Developed an operational data-lake on a Kerberos secured HBase Cluster.
Technologies: HBase, Apache ZooKeeper, Kerberos, Apache Kafka, Spark Streaming, Spark, YARN, Hadoop, Cloudera
Lead Big Data Developer
2016 - 2017
UK Government Home Office
- Oversaw the move to Graph databases to better model their data relationships.
- Developed an AWS MI reporting system for both batch and real-time CEP style reporting.
- Transitioned an existing developer base into the new technologies of big data computing using Apache Spark and Kafka.
Technologies: PostgreSQL, Cassandra, Kafka Streams, Spark Streaming, Apache Kafka, Spark, GRAPH, DataStax, Hadoop
Big Data/Hadoop Consultant for Finance IT
2014 - 2016
HSBC Bank
- Delivered the future state architecture designed for high-throughput processing capabilities to deliver the next-generation of banking applications within Finance IT.
- Developed an ETL solution based on HDFS, JSON, and Spark for building out the global reporting platform with 96 reports developed in under six months.
- Oversaw the technical review of the full suite of emerging big data technologies with the aim of choosing the correct set of technologies that would provide financial and competitive advantage for the investment bank.
Technologies: Amazon Web Services (AWS), Ansible, Apache Hive, Spark Streaming, ELK (Elastic Stack), Flume, Apache Kafka, Akka, Scala, Java, Spark, HDFS, Hadoop
Database Performance Tuning specialist for Finance IT
2013 - 2014
HSBC Bank
- Reduced SLA breaches to zero for the finance IT strategic accounting hub and regulatory reporting platform.
- Increased system throughput to enable greater business growth. Reduced the daily batch run times by 80% and improved platform stability to allow the take-on of future business.
- Reduced development and build times with the creation of build management frameworks used extensively across a number of finance IT systems.
- Owned all the data management frameworks within the bank’s accounting platform. Reduced data volumes by defining several key data archiving processes.
- Worked with business leaders to establish data controls to limit data growth while increasing business growth.
- Enabled reduction in storage costs and further project costs by developing the enablers that have allowed for wholesale system migration and storage space reductions.
- Mentored a global team across Brazil, India, and China to establish coding standards and build quality.
- Automated build and delivery frameworks used globally within the finance IT department.
- Performance tuned global finance systems, working quickly to identify issues to solve immediate emergencies that arise across any finance IT system.
Technologies: Optimization, SQL, XML, Ant Design, Java, Oracle 11g, Oracle 10g
Lead Oracle Developer for Cashflow/Collateral/Liquidity
2012 - 2013
Barclays Bank
- Developed numerous data feed processors to allow the BA team to analyze data requirements.
- Worked with the data architecture teams to deliver a physical database to manage cash flow and intra-day liquidity funding.
- Reviewed and evaluated column-store databases to provision for real-time feed processing and reporting requirements.
- Reviewed performance for the Spring integration framework to ensure volume capabilities are met.
Technologies: MongoDB, Kdb+, SAP HANA, Spring Batch, Spring Integration, JSON, XSLT, XML, Java, Oracle 10g, Oracle 11g
Oracle Lead Developer for Liquidity Risk/Treasury IT
2010 - 2012
Deutsche Bank
- Performance tuned an ailing liquidity risk/treasury reporting system. Key technical and delivery lead taking on the task of managing and mentoring an offshore development team, with full ownership of system delivery.
- Introduced rigorous coding standards, code delivery, build, and deployment processes to improve stability and allow the department to take on more business.
- Re-tuned the 80+ feeds into the liquidity risk reporting system. Improved performance and system delivery.
- Developed coding standards, performance metrics, release procedures that improved system stability during the monthly release cycles, code quality, and source control.
- Reduced system outages and reporting delays to near zero.
- Trained a mix of senior/junior offshore developers to ensure robust code delivery.
- Built the core components that defined the foundation of the bank's treasury reporting system.
Technologies: XML, Oracle, Ant Design, Subversion (SVN), Java, Oracle 11g, Oracle 10g
Lead Oracle Developer for Finance Accounting
2009 - 2010
Barclays Bank
- Owned and led the implantation, migration, and configuration management for the build, implementation and delivery of the database. Designed an automated build and deploy process using Java and ANT.
- Developed the migration framework that ensures accountability within the bank to ensure the correct phased delivery of the system.
- Worked with the key business users to ensure the smooth running of the project and ensuring the correct transformation of business requirements to technical solutions.
- Developed the feeds to SAP and the various reconciliation systems within the bank.
- Managed teams in Singapore, London, and New York to provide a 24/7 development center.
- Worked with the bank's accountants and key business sponsors to ensure that all project expectations are fully met or exceeded.
- Played a key role in the go-live of the system and the project's success.
Technologies: VB, Microsoft Excel, XSLT, XML, Perforce, SQL, Oracle, Java, Ant Design, Exadata, Oracle 11g
Lead Oracle Developer
2007 - 2009
NYSE Euronext
- Enhanced the company's existing cash and derivatives trading platform, CONNECT. Performance tuned the database to handle the millions of transactions and messages that are expected of trading platforms.
- Designed and developed the new pricing and settlement services system for pricing and settling contracts. This also involved updating the schema used by the existing trading system.
- Developed the key messaging framework using Oracle AQ to communicate data changes between the database and all the components/services that use the database.
- Worked with various business leads to develop test data and data loading frameworks to configure and test the new trading, pricing, and settlement engines.
- Performance tuned key aspects of the database to ensure that the trading platform has optimal sub-second responses.
- Developed trading platforms for Liffe, NYSE Metals, Tokyo Stock Exchange, and Tokyo Futures Exchange.
Technologies: Java, VB, Microsoft Excel, XSLT, XML, IBM Rational ClearCase, PL/SQL, Oracle, Oracle 10g
Lead Oracle Developer
2006 - 2007
HSBC Bank
- Developed the bank's flagship data warehouse for asset servicing.
- Developed cross-database tools to provide single and coherent views of data across Oracle, Sybase, and SQL Server databases. Used the tools for data migration, data reconciliation, and reporting.
- Developed the bank standard messaging framework for persistent and reliable message delivery. Integrated messaging platforms via Oracle Messaging Gateway between MQ Series to Oracle AQ.
- Mentored and empowered team members, performance-tuned other projects, designed and data-modeled other systems, and filled in the gaps as a Java developer when required.
Technologies: PL/SQL, Messaging, IBM MQ, Oracle, XSLT, XML, Sybase, Microsoft SQL Server, Oracle 11g, Oracle 10g
Lead Oracle Developer
2005 - 2006
Barclays Banlk
- Re-architected the development infrastructure for legal and compliance IT.
- Developed numerous feeds, based on different techniques, for loading data into the corporate data warehouse.
- Redeveloped the existing scheduler system. Designed a new data model with improved scheduling processes and a web interface to control and support the scheduler.
- Developed a generic reporting framework and data model to allow rapid production of user required reports. This provides tabular reports and highly customized reports with output to HTML, PDF, or Excel.
- Developed a generic user administration web application that manages user application access. Integrated with a corporate LDAP server and application-specific authentication schemes.
- Developed standards for application deployment, web development, Oracle performance tuning, PL/SQL coding standards, unit and regression testing, and system documentation.
Technologies: Ant Design, Messaging, XSLT, XML, Apache Struts, Java Servlets, PL/SQL, Perl, Unix, Data Modeling, Oracle 10g
Lead Oracle Developer
2001 - 2005
Deutsche Bank
- Developed the global reporting systems for loan exposure management. Developed to ensure compliance to the BASEL II accord and provide key loan hedging decision-making systems to reduce the bank’s exposure to loans.
- Delivered several back-end banking systems for loan exposure reporting.
- Developed front/middle office ticketing systems used by controllers/traders to create new loans and for hedging of loans using credit derivative swaps, collateralized loan obligations, and loan sales.
- Developed an active data warehouse that performs real-time refresh of the data warehouse for accurate reporting and hedge planning.
- Managed building of an offshore development team based in Moscow. Mentored a support team based in New York, London, and Moscow.
- Delivered numerous coding standards and development frameworks for reporting, feed management, processing/loading/transformation of flat files and XML data files, and general database development.
Technologies: Java, Performance Tuning, SQL, Unix, Oracle, Data Modeling, Actuate, Oracle8i
Lead Oracle Developer
2000 - 2001
Wanadoo France Telecom
- Developed an internet content management tool aimed at delivering device independent content for Wanadoo, a France Telecom subsidiary. The tool is used to produce the Dutch Wanadoo website.
- Designed and developed the database. In addition to normal duties of DBA, developed all stored procedures used by the Java middle tier for managing the system. Developed numerous database management scripts and performance monitoring views.
- Developed Java utilities for JDBC connection management, LOB loading/retrieval of binary (images, streaming video, Flash) and character data, data caching in the middle tier and server side session tracking, and array handling for Java and Oracle.
- Created web delivery frameworks using Java Servlets and JSP that take Oracle queries and transform the results using XSL to HTML/WML.
Technologies: Java, Oracle, SQL Loader, PL/SQL, Oracle8i
Oracle Developer
1999 - 2000
Deutsche Bank
- Maintained the integrity of the data in the system. This includes managing all the counterparties, legal agreements, limits, exposures, and trade information.
- Developed and managed feeds for CRMS with the bank's counterparty, risk engines, and trading systems. These include derivatives, FX, equity, money markets, MMJV, metals, etc.
- Developed a business-focused reporting intranet site.
- Performance-tuned the system. Enabled the corrective tracking of all data-related issues and reconciled the client-side application against the database.
Technologies: XML, Jakarta Server Pages (JSP), JDBC, Java, Unix Shell Scripting, PL/SQL, Oracle8i
Lead Datawarehouse Developer
1998 - 1999
General Electric Insurance Holdings
- Designed and implemented of a number of data warehousing systems.
- Developed, revised and extended the detailed operational processes and procedures necessary for warehouse/decision support type systems, e.g., load scripts, third-party software interfaces, SQL tuning, and backup processes.
- Formed a set of best practices for database design and administration. This covers data modeling naming standards, code development standards, and standardization of processes for performing database administration.
- Supported various development teams to aid in their development efforts against an Oracle database.
Technologies: Storage, Unix, Perl, PL/SQL, Oracle Discoverer
Full-stack Developer
1995 - 1998
Cambridge Technology Partners
- Led the design and development of an annuities payment system data model.
- Designed and developed a global CRM system for an asset management firm.
- Developed a Java web based sales capture system for field sales teams.
- Developed and supported the workflow and imaging system. Established the performance tuning framework for the application and the development of the front-end application.
Technologies: Unix, Visual Basic, Java, Perl, Sybase PowerDesigner, PL/SQL, SQL
Experience
Business Intelligence Data Lake for Camelot Group UK
http://www.camelotgroup.co.uk/Developed the enterprise data lake using the best of bred technologies on AWS: a serverless, event-driven framework that enables ingestion of data from any source in any format. It is a schema-less solution that dynamically accounts for changes in the source data. I architected it using a real-time first approach that also enables batch processing.
Skills
Frameworks
Hadoop, Spark, Apache Spark, YARN, Apache Thrift, Akka, Ant Design, Spring Integration, Spring Batch, Apache Struts, Actuate, Jakarta Server Pages (JSP)
Tools
Spark SQL, Amazon Elastic MapReduce (EMR), Amazon Simple Queue Service (SQS), Amazon CloudWatch, AWS Glue, Amazon Athena, Ansible, Apache Airflow, Elastic, Synapse, Cloudera, Apache ZooKeeper, DataStax, Flume, ELK (Elastic Stack), Subversion (SVN), Perforce, Microsoft Excel, IBM Rational ClearCase, IBM MQ, Sybase PowerDesigner, Kafka Streams
Platforms
Apache Kafka, Oracle, Amazon EC2, AWS Lambda, Docker, Azure, Databricks, SAP HANA, Unix, Amazon Web Services (AWS)
Storage
Amazon S3 (AWS S3), Oracle PL/SQL, Cassandra, HBase, PostgreSQL, HDFS, Apache Hive, Oracle 10g, Oracle 11g, JSON, Kdb+, MongoDB, Exadata, PL/SQL, Microsoft SQL Server, Sybase, SQL Loader
Other
Big Data, EMR, Data Modeling, API Gateways, Kerberos, GRAPH, Optimization, Messaging, Oracle8i, Performance Tuning, Unix Shell Scripting, Oracle Discoverer, Storage
Languages
Java, Scala, XML, SQL, XSLT, VB, Perl, Visual Basic
Libraries/APIs
Spark Streaming, Java Servlets, JDBC
Education
1989 - 1994
Masters of Engineering Degree in Electronic Engineering Design
King's College London - London
Certifications
AUGUST 1994 - PRESENT
Thomson Broadcast Award for Top Engineering Student
Thomson Broadcast
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring