Rajeshwar Agrawal
Verified Expert in Engineering
Back-end Developer
Jabalpur, Madhya Pradesh, India
Toptal member since March 23, 2021
Rajeshwar is a staff-level software engineer and platform technical lead with 11+ years of experience turning ambiguous challenges in scale, reliability, and client delivery into back-end and platform systems. He leads technical direction while remaining hands-on across architecture, implementation, performance engineering, and production recovery. Rajeshwar's work creates leverage through shared platforms, engineering standards, open source contributions, mentoring, and cost-aware cloud design.
Portfolio
Experience
- Software Engineering - 11 years
- Back-end Development - 11 years
- Distributed Systems - 11 years
- Software Architecture - 11 years
- Cloud Architecture - 11 years
- Python - 11 years
- Platform Engineering - 8 years
- Technical Leadership - 5 years
Work Experience
Platform Tech Lead (Staff Software Engineer)
STEELEYE
- Owned technical direction and delivery for a multi-tenant financial surveillance platform supporting enterprise SaaS and private-cloud deployments from solution design through long-term production support.
- Defined an operating envelope covering 3 million trades per day, 350,000 concurrent surveillance workflows, 60 million trade queries within 20 hours, and 600,000 surveillance alerts, guiding load testing, capacity, and rollout decisions.
- Led modernization of Conductor from PostgreSQL toward Redis and Elasticsearch 8, covering architecture, workflow compatibility, reconciliation, deployment tooling, load testing, sizing, and production stabilization.
- Contributed 13 pull requests to Conductor OSS, with 10 merged, including Elasticsearch 8 persistence and improvements to sub-workflow reliability, recovery, and scalability.
- Designed an Elasticsearch sharding strategy that improved effective performance by 2.7x without added compute or storage. Schema and archival changes also cut Elasticsearch costs by 4x and archive-storage costs by 2x.
- Led cross-system production diagnosis across Conductor, Elasticsearch, Kafka, and workflow processing, tracing failures across component boundaries, and driving fixes through stabilization.
- Shaped platform roadmaps, engineering standards, ownership boundaries, and delivery priorities while mentoring engineers and partnering with product, sales, support, and infrastructure teams.
- Consolidated Python applications into a shared Pants monorepo, enabling more than 10 applications to launch within three months and reducing annual CI/CD costs from approximately $25,000 to $6,000.
- Designed FastAPI services and Prefect/Parquet pipelines serving financial quotes, trades, and tick data, reducing ingestion and storage costs by approximately 80%.
Senior Software Engineer
Motional
- Led architecture and delivery of an AWS data platform processing 20 TB of autonomous vehicle data per day from hundreds of test runs, using event-driven pipelines, AWS Step Functions, and Parquet storage.
- Designed a high-throughput Python streaming pipeline that converted non-streamable logs into timestamp-seekable Avro, achieving 3.1 Gbps encoding and compression throughput while enabling low-latency access.
- Built more than 25 PySpark, Pandas, and NumPy applications on Amazon EMR to transform ride data into Parquet and expose it through Athena, improving analytical access while reducing processing costs.
- Created a self-service data-application platform with an SDK, base Docker image, reusable framework, and EC2 autoscaling, enabling 10+ applications in three months versus an estimated two to three months per application.
Software Engineer
Works Applications
- Designed and delivered an order-fulfillment platform integrating order, cart, payment, and shipping services, meeting an SLA of 1,000 fulfilled orders per hour through REST microservices and autoscaling.
- Migrated a large Jakarta EE eCommerce monolith into Spring Boot REST microservices, helping establish clearer service boundaries and independently deployable back-end components.
- Designed Kafka-based asynchronous workflows and optimized Cassandra and MySQL data models for distributed order processing across multiple back-end services.
- Improved developer productivity by optimizing Jenkins CI/CD pipelines, tuning JVM parameters, and standardizing release artifact management through Artifactory.
Software Engineer Intern
Canon Marketing Japan
- Analyzed software projects using code metrics to establish a quantitative approach to software-quality assessment.
- Designed and implemented a Python-based model that combined weighted metrics into a single software-quality score.
- Received the computer science department’s best project award for my internship work.
Project History
Enterprise Financial Surveillance Platform at Scale
Conductor Workflow Orchestration Platform Modernization
Python Developer Platform and CI/CD Modernization
AI-assisted Guardrail for Production Infrastructure Changes
Elasticsearch Performance, Capacity, and Cost Optimization
Financial Market Data Services and Ingestion Platform
Self-service Developer Platform for Data Applications
Cloud Cost Governance Across Shared Platforms
Autonomous Vehicle Data Platform at 20 TB per Day
Event-driven Order Fulfillment System
Education
Bachelor's Degree in Computer Science
Indian Institute of Information Technology - Jabalpur, India
Skills
Libraries/APIs
REST APIs, PySpark, API Development, Pydantic, Back-end APIs, SQLAlchemy, Slack API, JDBC, Pandas, NumPy, Asyncio, Python Asyncio, Flask-Marshmallow, PyMySQL, Jenkins Pipeline, Refinitive API, OpenAI API, Claude API, OpenAPI, React
Tools
Git, Amazon Athena, Amazon Elastic MapReduce (EMR), Apache Maven, Kafka Streams, Pytest, GitHub, Claude, Codex, Cron, Amazon EKS, Confluence, Terraform, AWS Glue, AWS Step Functions, Amazon Elastic Container Service (ECS), Jira, Claude Code, Observability Tools, Jenkins, Apache Avro, Amazon CloudWatch, Amazon Redshift Spectrum, GitLab, GitLab CI/CD, Jupyter, IPython Notebook, Amazon Virtual Private Cloud (VPC), Shell, PyPI, AWS Deployment, AWS SDK, Prefect, Apache Airflow, Amazon Simple Queue Service (SQS), Celery, Sentry, Apache Iceberg, Apache Tomcat, Microsoft AI, Helm
Languages
Python, SQL, Java, C#, Python 2, Python 3, Java 8, XML, C#.NET, Bash, Batch, GraphQL, HTML, JavaScript, ECMAScript (ES6)
Frameworks
Hadoop, Spark, Spring Microservice, Spring Boot, .NET Core, Apache Spark, ASP.NET, Django, .NET, JUnit, OAuth 2, Flask, Spring, Django REST Framework, Hibernate, Spring MVC, Flux
Paradigms
MapReduce, Microservices, Microservices Architecture, ETL, DevOps, Object-oriented Design (OOD), Object-oriented Programming (OOP), Event-driven Architecture, Unit Testing, Serverless Architecture, Testing, Database Design, Asynchronous Programming, Automation, Event-driven Design (EDD), B2B, Refactoring, Distributed Computing, App Development, REST, Agile, Real-time Systems, Role-based Access Control (RBAC), Continuous Integration (CI), Continuous Delivery (CD), RESTful Development, Scrum, Management, Model Context Protocol (MCP), Load Testing
Platforms
Docker, Apache Kafka, Amazon Web Services (AWS), Amazon EC2, AWS Lambda, Linux, Java EE (Jakarta EE), Kubernetes, Apache Arrow, Cloud Run, Cloud Native, Ubuntu, Unix, Azure, Jupyter Notebook, MacOS, Amazon, Jakarta EE (Java EE or J2EE)
Storage
Amazon S3 (AWS S3), Relational Databases, PostgreSQL, Elasticsearch, Redis Cache, On-premise, Distributed Databases, JSON, Auto-scaling Cloud Infrastructure, Databases, Data Pipelines, MySQL, Redis, NoSQL, Cassandra, Apache Hive, Datadog, Redshift, Microsoft SQL Server, MongoDB, Spring Data JPA, Database Replication, Apache Parquet
Industry Expertise
Project Management
Other
Software Development, CI/CD Pipelines, Parquet, Big Data, Data Modeling, Data Engineering, Back-end, Distributed Systems, Software Engineering, APIs, eCommerce, Data Warehousing, Cloud, SDKs, Containers, System Design, Software Architecture, API Integration, Concurrency, Data, Software Implementation, RESTful Microservices, Multithreading, SOLID Principles, FastAPI, Architecture, Cloud Architecture, Data Management, Containerization, API Design, Debugging, Troubleshooting, Finance, Back-end Development, Monorepos, Platform Engineering, Code Review, Leadership, Feature Analysis, Streaming, Documentation, Pub/Sub, Fintech, Data Architecture, Command-line Interface (CLI), Communication, Data Infrastructure, AWS Cloud Architecture, SaaS, Solution Architecture, Systems Design, Technical Documentation, API Backwards Compatibility, System Architecture, Integration, Third-party Integration, Workflows, Technical Leadership, Cloud Infrastructure, Infrastructure, Infrastructure Architecture, Technical Architecture, Incident Response, Automated Deployment Scripts, Distributed Applications, Distributed Architecture, SDK Development, Cloud Storage, Data Migration, Event-driven Systems, Performance, Web Development, Integrations, Financial Services, AI-assisted Development, Anthropic, Data Extraction, Vibe Coding, Performance Optimization, Multi-tenant Architecture, DevOps Automation, DevOps Engineer, Infrastructure Automation, Metadata, Financial APIs, Team Management, Workflow, Orchestration, Data Orchestration, Pipelines, Third-party APIs, Large-scale Projects, Solution Design, Systems, ERP Systems, Performance Improvement, System Integration, Multi-tenant SaaS, Startups, Technical Writing, Back-end Performance, Data Warehouse Design, Relational Database Services (RDS), Scaling, Algorithms, Amazon RDS, Slackbot, Data Wrangling, Autoscaling, State Machines, Zstandard, ECS, SSH, Infrastructure as Code (IaC), Monitoring, Scalability, Enterprise Resource Planning (ERP), WebSockets, Lambda Functions, Message Queues, Artificial Intelligence (AI), Large Language Models (LLMs), AI Agents, Agentic AI, Multitenancy, Real-time Data, Error Logging, AWS Secrets Manager, Agentic Coding, Low Latency, Observability, Capacity Planning, Data Reconciliation, Enterprise Architecture, AI-generated Code, Database Optimization, Cloud Engineering, Site Reliability Engineering (SRE), Enterprise Integration, Middleware, SOC 2, Mentorship, Compliance, Financial Modeling, Email, API Connectors, Identity & Access Management (IAM), AWS ECS Fargate, SKUs, Retail Management, Microsoft Azure, Early-stage Startups, Search, Team Leadership & Supervision, Team Leadership, Solution Engineer, Enterprise, Data Compression, Serverless, Amazon API Gateway, Autonomous Navigation, Self-driving Cars, Messaging, Amazon Kinesis, Maps, Jupiter, Infrastructure Monitoring, Apache Cassandra, Networking, Deployment, Memory Management, Memory Mapped Files, Processing & Threading, Benchmarking, Memory Profiling, Software Integration, Big Data Architecture, Data Management Platforms, Conductor, Netflix OSS, Orkes, GitHub Actions, Software as a Service (SaaS), pantsbuild, Schemas, Amazon Glacier, ARM, Cost Control, LangChain, OpenAI, Medical Software, Metrics, Generative Artificial Intelligence (GenAI), OAuth, LLM Integration, AI Automation, AI Tools, Algorithmic Trading, Trading, Leadership & Coaching, Temporal, Azure AI Services, Terraform Cloud, Full-stack Development, Full-stack, Single Sign-on (SSO), Webhooks, OpenTelemetry, Workflow Orchestration, Open Source, Reliability Engineering, Large-scale Distributed Systems, Developer Experience (DX), Build Automation, Security Automation, Sharding, Prometheus, Batch File Processing, Access Control, Prompt Engineering, Caching, Polars, AI Engineering, Content Writing
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring