Job Description: Senior Test Automation Lead – Playwright (AI/ML Focus)

Posted time May 27, 2025 Location Hyderabad Job type Full-time

Job Title: Senior Test Automation Lead – Playwright (AI/ML Focus)

Location: Hyderabad

Job Type: Full-Time

Company Overview:

  • GRhombus Technologies Pvt Ltd, a pioneer in Software Solutions – Especially on Test Automation, Cyber Security, Full Stack Development, DevOps, Salesforce, Performance Testing and Manual Testing.
  • GRhombus delivery centres are located in India at Hyderabad, Chennai, Bengaluru and Pune. In the Middle East, we are located in Dubai. Our partner offices are located in the USA and the Netherlands.

About the Role:

  • We are seeking a passionate and technically skilled Senior Test Automation Lead with deep experience in Playwright-based frameworks and a solid understanding of AI/ML-driven applications. In this role, you will lead the automation strategy and quality engineering practices for next-generation AI products that integrate large-scale machine learning models, data pipelines, and dynamic, intelligent UIs. You will define, architect, and implement scalable automation solutions across AI-enhanced features such as recommendation engines, conversational UIs, real-time analytics, and predictive workflows, ensuring both functional correctness and intelligent behavior consistency.

Key Responsibilities:

Test Automation Framework Design & Implementation

  •  Design and implement robust, modular, and extensible Playwright automation frameworks using TypeScript/JavaScript.
  •  Define automation design patterns and utilities that can handle complex AI-driven UI behaviors (e.g., dynamic content, personalization, chat interfaces).
  •  Implement abstraction layers for easy test data handling, reusable components, and multi-browser/platform execution.

AI/ML-Specific Testing Strategy

  • Partner with Data Scientists and ML Engineers to understand model behaviors, inference workflows, and output formats.
  • Develop strategies for testing non-deterministic model outputs (e.g., chat responses, classification labels) using tolerance ranges, confidence intervals, or golden datasets.
  • Design tests to validate ML integration points: REST/gRPC API calls, feature flags, model versioning, and output accuracy.
  • Include bias, fairness, and edge-case validations in test suites where applicable (e.g., fairness in recommendation engines or NLP sentiment analysis).

End-to-End Test Coverage

  1. Lead the implementation of end-to-end automation for: 
  •         Web interfaces (React, Angular, or other SPA frameworks)
  •         Backend services (REST, GraphQL, WebSockets)
  •         ML model integration endpoints (real-time inference APIs, batch pipelines)

2.Build test utilities for mocking, stubbing, and simulating AI inputs and datasets.·

CI/CD & Tooling Integration

  • Integrate automation suites into CI/CD pipelines using GitHub Actions, Jenkins, GitLab CI, or similar.
  •  Configure parallel execution, containerized test environments (e.g., Docker), and test artifact management.
  •  Establish real-time dashboards and historical reporting using tools like Allure, ReportPortal, TestRail, or custom Grafana integrations.

Quality Engineering & Leadership

  • Define KPIs and QA metrics for AI/ML product quality: functional accuracy, model regression rates, test coverage %, time-to-feedback, etc.
  •  Lead and mentor a team of automation and QA engineers across multiple projects.
  •  Act as the Quality Champion across the AI platform by influencing engineering, product, and data science teams on quality ownership and testing best practices.

Agile & Cross-Functional Collaboration

  • Work in Agile/Scrum teams; participate in backlog grooming, sprint planning, and retrospectives.
  • Collaborate across disciplines: Frontend, Backend, DevOps, MLOps, and Product Management to ensure complete testability.
  • Review feature specs, AI/ML model update notes, and data schemas for impact analysis.

Required Skills and Qualifications:

Technical Skills:

  • Strong hands-on expertise with Playwright (TypeScript/JavaScript).
  • Experience building custom automation frameworks and utilities from scratch.
  • Proficiency in testing AI/ML-integrated applications: inference endpoints, personalization engines, chatbots, or predictive dashboards.
  • Solid knowledge of HTTP protocols, API testing (Postman, Supertest, RestAssured).
  • Familiarity with MLOps and model lifecycle management (e.g., via MLflow, SageMaker, Vertex AI).
  • Experience in testing data pipelines (ETL, streaming, batch), synthetic data generation, and test data versioning.

Domain Knowledge:

  • Exposure to NLP, CV, recommendation engines, time-series forecasting, or tabular ML models.
  • Understanding of key ML metrics (precision, recall, F1-score, AUC), model drift, and concept drift.
  • Knowledge of bias/fairness auditing, especially in UI/UX contexts where AI decisions are shown to users.
  • Leadership & Communication:
  • Proven experience leading QA/Automation teams (4+ engineers).
  • Strong documentation, code review, and stakeholder communication skills.
  • Experience collaborating in Agile/SAFe environments with cross-functional teams.

 

Preferred Qualifications:

  • Experience with AI Explainability frameworks like LIME, SHAP, or What-If Tool.
  • Familiarity with Test Data Management platforms (e.g., Tonic.ai, Delphix) for ML training/inference data.
  • Background in performance and load testing for AI systems using tools like Locust, JMeter, or k6.
  • Experience with GraphQL, Kafka, or event-driven architecture testing.
  • QA Certifications (ISTQB, Certified Selenium Engineer) or cloud certifications (AWS, GCP, Azure).

Education:

  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related technical discipline.
  • Bonus for certifications or formal training in Machine Learning, Data Science, or MLOps.

Why Join Us?

  • At GRhombus, we are redefining quality assurance and software testing with cutting-edge methodologies and a commitment to innovation. As a test automation lead, you will play a pivotal role in shaping the future of automated testing, optimizing frameworks, and driving efficiency across our engineering ecosystem.
  • Be part of a workplace that values experimentation, learning, and professional growth.
  • Contribute to an organisation where your ideas drive innovation and make a tangible impact.