Deep is available for hire
Hire DeepDeep Halder
Verified Expert in Engineering
Artificial Intelligence SDET Developer
Bangalore, India
Toptal member since September 16, 2026
Bio
Deep is a senior SDET with 8 years building automation frameworks for global products, twice from zero. His work spans web automation with Playwright and Selenium, API automation with REST Assured, Postman and gRPC, SQL validation, performance testing with k6, and CI/CD in Jenkins, GitLab and GitHub Actions. He also leads LLM and agent evaluation, building the eval frameworks and regression gates that catch AI quality problems before release.
Portfolio
Sentient
SQL, Playwright, Postman, CI/CD Pipelines, TypeScript, REST APIs, GitLab CI/CD...
Level AI
SQL, Metabase, Selenium, Playwright, Behavior-driven Development (BDD), Jenkins...
Motifworks
Selenium, Java, C#, SpecFlow, FlaUI, Appium, Quality Assurance (QA), Automation...
Experience
- SDET - 8 years
- Test Automation - 8 years
- QA Automation - 8 years
- API Testing - 7 years
- CI/CD Pipelines - 6 years
- Python - 5 years
- Playwright - 4 years
- LLM EVALUATION - 3 years
Preferred Environment
Playwright, Postman, Selenium, AI Agents, AI Evaluation, AI Testing, API Testing, QA Testing, Test Automation, Performance Testing
The most amazing...
...work I've done is automating non-deterministic AI testing with LLM-as-Judge evaluation, replacing manual validation with scalable automated quality checks.
Work Experience
Senior AI SDET & AI Evaluation Lead
2025 - PRESENT
Sentient
- Led core SDET/QA for the platform. Built end-to-end web automation from scratch in Playwright and API automation frameworks in REST Assured and Postman, integrated into CI/CD.
- Owned the evaluation intelligence layer for AGI-powered agents. Designed LLM-as-Judge pipelines scoring hallucination, reasoning correctness, tool-use accuracy, and prompt adherence.
- Built gold datasets and AI regression pipelines, and calibrated LLM judges against human reviewers to keep automated scores trustworthy—raising model/prompt release confidence by around 40% and cutting flaky validations by around 30%.
- Instrumented evaluation traces with Langfuse and analyzed conversation data in BigQuery/SQL to surface failure patterns and quantify the impact of each prompt/model release before rollout.
- Tested agentic and Crypto/Web3 workflows, including wallet integrations, transaction validation, and on-chain/off-chain data checks, with functional, regression, and backend-orchestration coverage.
- Built API automation covering REST and gRPC services, including proto contract validation, streaming and unary calls, status code handling across the error space, and deadline behaviour at the boundary.
- Solved authentication against Google OAuth and Cloudflare Turnstile with a Python session capture tool, letting the whole suite run authenticated without touching a login screen at runtime.
- Built the Playwright suite on the Page Object Model across dev, UAT and production, using stable test identities over brittle selectors so a UI change touches one page class instead of dozens.
Technologies: SQL, Playwright, Postman, CI/CD Pipelines, TypeScript, REST APIs, GitLab CI/CD, AI Testing, LLM EVALUATION, AI Evaluation, Test Automation, LLM Agents, AI Agents, QUALITY ENGINEERING, QA Testing, Grafana k6, QA Automation, Quality Assurance (QA), Automation, Integration Testing, Web Applications, Regression Testing, Artificial Intelligence (AI), Cloud, Test Automation Frameworks, E2E Testing, Performance Testing, Prompt Engineering, SDET, Claude Code, Web Content Accessibility Guidelines (WCAG), Manual Testing, REST Assured, Apache Maven, Communication, Attention to Detail, Code Review, Screeners, Written Communication, Team Leadership, API Testing, Interviewing, Team Management, Hallucination Detection, AWS IoT, PostgreSQL, Low Latency, Entity Extraction, Data Pipelines, Monitoring, ETL Testing, Database Testing, Observability, Penetration Testing, Black Box Testing, Cybersecurity, HTML, JavaScript, Mobile Apps, Mobile App Testing, SDKs, Flutter, Jira, Test Case Creation, Test Case Execution, Test Management, Functional Testing, Testing Services, Testing, Web Service Testing, Large Language Models (LLMs), Sentry, Automated Testing, Cypress, Agile, AI Tools, REST, JSON, DevOps, Docker, Manual QA, Localization Testing, Usability Testing, AI Chatbots
Founding QA – Conversational-AI Evaluation
2021 - 2025
Level AI
- Pioneered the QA function as the first hire. Built the quality and evaluation function from zero to one for a production voice/conversational-AI platform. Defined what "good" looked like via quality rubrics and metrics.
- Led voice AI evaluation—STT/ASR accuracy, intent detection, entity extraction, call summarization, latency, and agent-assist quality—including audio-layer issues not visible in transcripts.
- Analyzed production conversations in SQL/Metabase to identify failure patterns and feed prioritized fixes back to Product and Engineering, then measured the delta on quality metrics after each release.
- Defined the automation strategy, release gates, and quality processes. Built scalable UI+API automation frameworks (Selenium, Playwright, Rest Assured) with a BDD approach, lifting regression by 70% and integrating into CI/CD via Jenkins and Maven.
- Established the end-to-end test strategy across web, API, CRUD, performance (K6, JMeter), and back-end validation. Mentored SDETs and scaled automation execution.
Technologies: SQL, Metabase, Selenium, Playwright, Behavior-driven Development (BDD), Jenkins, Maven, Grafana k6, Apache JMeter, AI Evaluation, AI Agents, REST APIs, API Testing, QA Automation, Quality Assurance (QA), Automation, Integration Testing, Web Applications, Regression Testing, Test Automation, Artificial Intelligence (AI), Cloud, Test Automation Frameworks, E2E Testing, Performance Testing, Prompt Engineering, SDET, GitLab CI/CD, Claude Code, Web Content Accessibility Guidelines (WCAG), Manual Testing, TestNG, REST Assured, Apache Maven, Azure DevOps, Java, Communication, Attention to Detail, Code Review, Screeners, Written Communication, Interviewing, Team Management, Azure, AI Testing, Hallucination Detection, GCP, PostgreSQL, Low Latency, Entity Extraction, Data Pipelines, Monitoring, ETL Testing, Database Testing, Observability, Penetration Testing, Black Box Testing, HTML, JavaScript, Dart, Jira, Test Case Creation, Test Case Execution, Test Management, Functional Testing, Testing Services, Testing, Web Service Testing, Large Language Models (LLMs), Sentry, Automated Testing, Cypress, Agile, Google Cloud Platform (GCP), Team Leadership, AI Tools, REST, DevOps, JSON, Manual QA, Localization Testing, Usability Testing, AI Chatbots
SDET I
2020 - 2021
Motifworks
- Automated 300+ hybrid (web+desktop) scenarios with Selenium, Java/C#, SpecFlow, FlaUI, and Appium—cutting regression effort by around 70%.
- Developed a unified automation framework supporting web and desktop testing within a single repository, improving test maintainability.
- Integrated UI automation across web, desktop, and mobile workflows to expand automated regression coverage and accelerate releases.
- Built reusable automation components and test utilities that standardized execution across multiple application workflows.
Technologies: Selenium, Java, C#, SpecFlow, FlaUI, Appium, Quality Assurance (QA), Automation, Integration Testing, Web Applications, Regression Testing, Test Automation, Cloud, Test Automation Frameworks, E2E Testing, SDET, GitLab CI/CD, Manual Testing, TestNG, REST Assured, Apache Maven, Azure DevOps, Communication, Attention to Detail, Code Review, Screeners, Written Communication, API Testing, Interviewing, Azure, Penetration Testing, Black Box Testing, Jira, Test Case Creation, Test Case Execution, Test Management, Functional Testing, Testing Services, Testing, Web Service Testing, Large Language Models (LLMs), Automated Testing, Agile, QA Automation, JSON, Manual QA, Localization Testing, Usability Testing
QA Automation Engineer
2018 - 2020
Atos
- Developed automated regression suites using Selenium, Protractor, TypeScript, Cucumber, and BDD, expanding automated test coverage across critical application workflows.
- Built reusable page objects, test utilities, and BDD components to improve automation maintainability and accelerate the creation of new test scenarios.
- Automated critical end-to-end business workflows across web applications, reducing repetitive manual regression effort and improving release validation.
- Integrated automated test execution into the development workflow, enabling consistent regression validation across application releases and reducing dependency on manual testing.
- Implemented behavior-driven testing with Cucumber and TypeScript, enabling business scenarios to be translated into maintainable automated acceptance tests.
Technologies: Behavior-driven Development (BDD), Selenium, Protractor, TypeScript, Cucumber, Quality Assurance (QA), Automation, Integration Testing, Web Applications, Regression Testing, Test Automation, Test Automation Frameworks, E2E Testing, SDET, Manual Testing, TestNG, REST Assured, Java, Communication, Attention to Detail, Code Review, Written Communication, API Testing, ETL, Databricks, Black Box Testing, Jira, Test Case Creation, Test Case Execution, Test Management, Functional Testing, Testing Services, Testing, Web Service Testing, Large Language Models (LLMs), Automated Testing, Agile, QA Automation, JSON, Manual QA, Localization Testing, Usability Testing
Experience
Playwright UI Automation Framework with LLM as a Judge Evaluation
A Playwright and TypeScript UI automation framework covering four web apps across dev, UAT, production and a separate AI playground environment. Built on the Page Object Model, so every DOM interaction sits in a page class and specs only orchestrate business flows. A UI change touches one file instead of dozens, and locators use stable test identities rather than brittle CSS or text selectors.
The hardest part was authentication. The apps sit behind Google OAuth and Cloudflare Turnstile, which actively block automated browsers. I built a session capture tool in Python using undetected Chromium that logs in once and writes cookies and localStorage to a file the tests load before first navigation. Tests then run fully authenticated without ever touching a login screen.
The suite also scores AI chat responses using an LLM as a judge. Responses are non deterministic, so string matching is useless. The test drives the UI, captures the response, and GPT 4o scores it for accuracy, relevance, completeness and clarity against defined criteria. That moved a surface being checked by hand into normal CI regression.
Runs on GitHub Actions on every push and nightly, with Allure reporting and Slack notifications.
The hardest part was authentication. The apps sit behind Google OAuth and Cloudflare Turnstile, which actively block automated browsers. I built a session capture tool in Python using undetected Chromium that logs in once and writes cookies and localStorage to a file the tests load before first navigation. Tests then run fully authenticated without ever touching a login screen.
The suite also scores AI chat responses using an LLM as a judge. Responses are non deterministic, so string matching is useless. The test drives the UI, captures the response, and GPT 4o scores it for accuracy, relevance, completeness and clarity against defined criteria. That moved a surface being checked by hand into normal CI regression.
Runs on GitHub Actions on every push and nightly, with Allure reporting and Slack notifications.
LLM Evaluation Arena for Autonomous AI Agents
Designed and built an internal evaluation arena for comparing LLM and agent behavior across model versions, prompts, and tool configurations. The system runs pairwise head-to-head comparisons, scores responses against an LLM-as-judge rubric, and maintains Elo-style rankings, so model quality becomes a tracked metric rather than a subjective call.
I owned evaluation design end to end: rubric definition, gold dataset creation, judge-prompt calibration against human-labeled samples, and measuring judge-human agreement so the scores could actually be trusted. The arena is wired into CI as a regression gate—every model or prompt change runs against a fixed eval suite, and regressions in hallucination rate, reasoning validity, tool-call accuracy, or prompt adherence block the merge rather than surfacing in production.
I also built the reporting layer that renders per-dimension deltas between candidate and baseline models in a form both engineers and non-engineers can act on, moving the team from manual spot-checking to repeatable, automated evaluation.
I owned evaluation design end to end: rubric definition, gold dataset creation, judge-prompt calibration against human-labeled samples, and measuring judge-human agreement so the scores could actually be trusted. The arena is wired into CI as a regression gate—every model or prompt change runs against a fixed eval suite, and regressions in hallucination rate, reasoning validity, tool-call accuracy, or prompt adherence block the merge rather than surfacing in production.
I also built the reporting layer that renders per-dimension deltas between candidate and baseline models in a form both engineers and non-engineers can act on, moving the team from manual spot-checking to repeatable, automated evaluation.
BDD Test Automation Framework with Reflection Driven Generic Steps
A Maven and Java UI automation framework for a contact center QA and analytics platform, built on Selenium, Cucumber and TestNG with PicoContainer for dependency injection.
The design problem was step definition sprawl. Most BDD frameworks grow a new step method for every button on every page, so glue code outpaces coverage. I built generic Gherkin steps that use Java reflection to resolve actions against public field names on page objects. A step reading Click on "saveFooButton" Button Component maps to that field at runtime, so adding a test usually means adding one locator field, not a new step class.
Locators sit in custom component wrappers for buttons, inputs and KPI cards, initialised through a lazy element factory that re-finds elements on every access instead of caching them. That killed most stale element failures on a React UI that rerenders constantly.
RestAssured seeds data like rubrics and coaching sessions before a scenario, and hooks clean them up afterwards using ids held in scenario context. Config resolves per environment, so the same suite targets dev, stage or prod without code changes.
Coverage spans login, dashboards, analytics, conversation review, rubric building, coaching and calibration.
The design problem was step definition sprawl. Most BDD frameworks grow a new step method for every button on every page, so glue code outpaces coverage. I built generic Gherkin steps that use Java reflection to resolve actions against public field names on page objects. A step reading Click on "saveFooButton" Button Component maps to that field at runtime, so adding a test usually means adding one locator field, not a new step class.
Locators sit in custom component wrappers for buttons, inputs and KPI cards, initialised through a lazy element factory that re-finds elements on every access instead of caching them. That killed most stale element failures on a React UI that rerenders constantly.
RestAssured seeds data like rubrics and coaching sessions before a scenario, and hooks clean them up afterwards using ids held in scenario context. Config resolves per environment, so the same suite targets dev, stage or prod without code changes.
Coverage spans login, dashboards, analytics, conversation review, rubric building, coaching and calibration.
Education
2014 - 2018
Bachelor's Degree in Computer Science and Engineering
Visvesvaraya Technological University - Bangalore, India
Skills
Libraries/APIs
Playwright, FlaUI, REST APIs
Tools
Postman, Jenkins, Cucumber, REST Assured, Apache Maven, Jira, Allure Report, GitLab CI/CD, Pytest, Claude Code, Sentry, Grafana k6, Apache JMeter
Languages
SQL, C#, TypeScript, Python, Java, HTML, JavaScript, Dart
Frameworks
Selenium, SpecFlow, Appium, Protractor, TestNG, Flutter, Cypress
Paradigms
Behavior-driven Development (BDD), Test Automation, Automation, E2E Testing, Manual Testing, Azure DevOps, Penetration Testing, Functional Testing, Testing, Automated Testing, Page Object Model (POM), Dependency Injection, REST, Usability Testing, ETL, Agile, DevOps
Platforms
Databricks, Firebase, Azure, Google Cloud Platform (GCP), Docker, AWS IoT
Storage
JSON, PostgreSQL, Data Pipelines, Database Testing, MongoDB
Industry Expertise
Cybersecurity
Other
CI/CD Pipelines, Metabase, Maven, GCP, Hallucination Detection, Low Latency, Entity Extraction, Web Development, AI Testing, LLM EVALUATION, AI Evaluation, QUALITY ENGINEERING, QA Testing, API Testing, QA Automation, Quality Assurance (QA), Integration Testing, Regression Testing, Artificial Intelligence (AI), Test Automation Frameworks, Performance Testing, Prompt Engineering, SDET, Communication, Attention to Detail, Screeners, Written Communication, Team Leadership, Interviewing, Team Management, Black Box Testing, Test Case Creation, Test Case Execution, Test Management, Testing Services, Web Service Testing, Large Language Models (LLMs), OAuth, Test Automation Architecture, Reflection, AI Tools, Manual QA, Localization Testing, AI Chatbots, Natural Language Processing, Web Applications, Cloud, Web Content Accessibility Guidelines (WCAG), Code Review, Monitoring, ETL Testing, Observability, Mobile Apps, Mobile App Testing, SDKs, LLM evaluation, LLM Evaluation, LLM Agents, AI Agents, GitHub Actions
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring