Sandesh Pawar

• Architected and delivered a distributed vector embedding pipeline processing 260+ million documents on Azure and Databricks.
• Designed a scalable ingestion framework for structured and unstructured data (XML/JSON/CSV).
• Built high-throughput embedding generation workflows using batch parallelization and optimized Spark jobs.
• Engineered a cost-efficient storage strategy for large-scale vector persistence and retrieval.
• Implemented similarity search optimization for low-latency semantic retrieval at production scale.
• Applied cluster tuning, partitioning, and workload optimization to reduce compute costs by ~40%.
• Integrated monitoring, logging, and failure recovery mechanisms for enterprise-grade reliability.
-Partnered with ML and product teams to productionize AI search capabilities.

IMPACT
• Generated 260+ million embeddings in production.
• Reduced semantic query latency by around 65%.
• Improved retrieval accuracy and relevance.
• Enabled AI-powered search across millions of records in seconds.

• Architected distributed ETL/ELT pipelines ingesting 260+ million patent XML files into Azure Data Lake using Databricks and Spark.
• Designed Medallion (Bronze/Silver/Gold) architecture enabling analytics, machine learning (ML), and vector search workloads.
• Parsed and normalized XML into structured Delta tables, producing 1.4+ billion claims records.
• Built schema-driven transformations with validation, incremental loads, and API-based ingestion.
• Implemented Liquid Clustering on publication_id to optimize pruning for interleaved large-scale tables.
• Enabled downstream embedding generation and AI-ready datasets for semantic search systems.

IMPACT
• 60% faster XML processing through distributed parallelization
• Around 70% query performance improvement on 1.4B billion-row claims table
• Around 35% reduction in compute/storage cost via optimization strategies

• Designed and built a scalable FastAPI back end powering a patent intelligence RAG platform.
• Implemented multi-stage retrieval: GTE (1024-dim embeddings) to Pinecone (k=6000) to PostgreSQL enrichment to Cohere Rerank v3.5.
• Developed parallel reranking (five workers × 300 docs per batch) to significantly improve semantic precision.
• Built conversational RAG with memory injection and dynamic model switching for 120,000+ token contexts.
• Engineered LLM-based search strategy generation using MapReduce summarization.
• Implemented automated answer evaluation and hallucination detection with iterative refinement.
• Added IPC-based K-Means and semantic clustering (UMAP and HDBSCAN) with LLM-generated summaries.
• Integrated Auth0 authentication, role-based authorization, and IP filtering.
• Enabled full observability with Datadog APM, OpenTelemetry tracing, and token and cost tracking.

IMPACT
• 3–6s P50 end-to-end latency, production-grade reliability, scalable to millions of vectors, significantly improved retrieval relevance and answer quality.

• Architected a multi-agent research system using LangGraph with three hierarchical graphs (Main to Supervisor to Researcher).
• Implemented subgraph orchestration, spawning up to 10 parallel researcher agents per workflow.
• Designed advanced state management with five Pydantic state schemas and custom reducers (override and additive).
• Built a hybrid RAG pipeline combining semantic search, SQL metadata filtering, and vector search (Pinecone and Cohere rerank v3.5).
• Implemented bounded parallelism with configurable concurrency to prevent DB/API exhaustion.
• Built context compression layer achieving 70–80% token reduction before final synthesis.
• Integrated robust error recovery (token limits, rate limits, and DB retries) with 95%+ recovery rates.
• Designed connection pooling architecture (3 DBs × 10 max connections × workers).
• Delivered end-to-end automated patent research reports with outputs and dynamic routing.

IMPACT
• 90% reduction in database I/O via two-stage retrieval
• 10× faster research vs sequential execution
• 70–80% context compression before synthesis
• 95%+ workflow completion rate
• 99.5%+ system reliability
• Supports 10–100 concurrent users
• Handles 10,00+ patents per research task

• Architected and deployed a Model Context Protocol (MCP) server enabling agentic patent search via FastMCP (tools, resources, and auth).
• Designed 5-stage RAG pipeline: Embed to Vector Search to Enrich to Rerank to Context Optimization.
• Implemented claim-level vector search (Pinecone) with patent-level aggregation and deduplication logic.
• Built dual-database enrichment layer (gold_patents and IFI) with batch ANY($1) queries eliminating N+1 patterns.
• Integrated Cohere rerank-v3.5 using full-content YAML docs for precision, then stripped heavy fields for efficiency.
• Engineered progressive enhancement pattern achieving 70–80% context window reduction.
• Implemented dual authentication (OAuth via WorkOS and API keys) with Stripe subscription middleware.
• Designed stateless, multi-worker architecture (Gunicorn and Uvicorn) with safe async connection pooling.
• Built metered billing system with Stripe usage events and async org resolution.

IMPACT
• 700ms–2.5s end-to-end multi-stage search latency
• 70–80% token reduction via progressive context optimization
• Eliminated N+1 queries through batch enrichment
• Horizontally scalable, stateless architecture
• Enterprise-ready auth, billing, and observability

• Designed an ROI analytics platform integrating marketing spend, sales, and product data across systems.
• Built a scalable reporting layer in Snowflake using Data Vault modeling.
• Developed dbt transformations for business logic standardization and version-controlled models.
• Orchestrated pipelines using Airflow DAGs with optimized dependency management.
• Reduced DAG runtime by around 50% via parallelization and task-level optimization.
• Automated reconciliation logic for net spend vs. revenue measurement.
• Enabled executive-level KPI dashboards and self-service analytics.

IMPACT
• Accelerated reporting delivery by 40%
• Reduced pipeline failures by >60%
• Enabled faster marketing ROI measurement across product portfolios
• Improved cost transparency and campaign optimization decisions

• Architected enterprise Azure data ecosystem using ADLS Gen2, Databricks, Event Hubs, SQL Server CDC, and Delta Lake.
• Built a real-time streaming ETL pipeline, reducing latency from 4–6 hours to under five minutes.
• Designed Medallion (Bronze, Silver, and Gold) lakehouse serving analytics, pricing, and ML use cases.
• Optimized partitioning, Z-ordering, and workload tuning to support high-concurrency BI queries.
• Implemented enterprise data governance with Informatica (lineage, metadata, and stewardship).
• Migrated MongoDB collections to Azure Cosmos DB and implemented multi-master configuration.
• Delivered executive dashboards in Power BI and Tableau for marketing, sales, and pricing teams.

IMPACT
• Reduced data latency by >90%, enabling near real-time pricing decisions.
• Improved reporting performance by 50–70%.
• Increased trusted dataset adoption across multiple business units.
• Enabled ML-ready curated datasets for AI initiatives.
• Reduced operational incidents through governance and monitoring.

• Led migration strategy from Azure Data Lake and Databricks to Microsoft Fabric Lakehouse.
• Assessed data workloads, performance bottlenecks, and cost structures.
• Re-architected Medallion layers into Fabric Lakehouse and Warehouse models.
• Migrated streaming and batch pipelines with minimal business disruption.
• Optimized storage, compute allocation, and workspace governance in Fabric.
• Established performance benchmarking framework pre- and post-migration.
• Ensured BI continuity across Power BI semantic models and downstream systems.

IMPACT
• Reduced total platform cost by 20–30% post-migration.
• Improved BI integration with the unified Fabric ecosystem.
• Reduced operational complexity across the analytics stack.
• Modernized enterprise data platform for AI-ready workloads.
• Improved query performance and developer productivity.

• Architected and implemented enterprise ETL pipelines converting OMOP CDM datasets into FHIR R4 resources for downstream clinical and research applications.
• Designed a USCDI-aligned common data model integrating OMOP and site-mediated EMR files across multiple health systems.
• Built a scalable data access layer to retrieve FHIR R4 datasets via 1upHealth APIs.
• Implemented Microsoft FHIR Converter service to transform STU3 to R4 using Liquid templates.
• Integrated with major EHR systems (Epic and Cerner) via SMART-on-FHIR APIs.
• Designed Snowflake and Databricks pipelines with optimized compute allocation and workload tuning.
• Standardized value set mappings and terminology alignment to support regulatory reporting and interoperability.

IMPACT
• Enabled seamless interoperability across multi-site healthcare systems
• Reduced data processing costs in Snowflake and Databricks by around 50%
• Accelerated FHIR resource generation for research and consent workflows
• Improved USCDI compliance readiness for clinical data exchange
• Increased reliability of cross-platform patient data normalization
• Enabled secure downstream data sharing via Snowflake consumption layer

• Created the relational database design for a claims-and-financial data warehouse. Using ETL packages, the data gets loaded into a central data warehouse.
• Designed different measure groups and dimensions.
• Implemented MDX scripts for a number of reports.
• Implemented an ad-hoc reporting solution with the help of SSAS for the finance DW.