Meta 4 дня назад

QA Engineering Lead (AI)

163 000 – 224 000 $в год
  • Менло-Парк, Калифорния, Соединенные Штаты Америки
  • Опыт от 10 лет
PythonAIJavaC++PHPGen AIQuality AssuranceleadTest EngineeringMultimodalVoiceImagegenerative text

Описание

Responsibilities

  • Define and drive the technical quality strategy and roadmap for a major Gen AI initiative spanning multiple engineering teams, balancing product risk, delivery speed, and engineering investment
  • Lead and perform hands-on testing of consumer-facing and business-facing applications, APIs, and AI models, adapting quality priorities and test coverage in a fast-paced development environment
  • Architect and implement shared AI evaluation and test automation systems that support multiple models, products, and platforms, while directly solving the most challenging technical problems
  • Establish consistent evaluation methods, benchmark design, product-aligned weighted scoring frameworks, and quality standards for model behavior and end-to-end product experiences, working with research, engineering, and product partners
  • Lead the development of reliable evaluation signals, including calibration of human and automated judgments, analysis of measurement uncertainty, and investigation of gaps between offline evaluations and real-world outcomes
  • Identify systemic quality risks across models, data, applications, and infrastructure, and lead cross-team efforts to address their root causes
  • Drive quality planning for major launches and phased rollouts, aligning teams on success criteria, dependencies, monitoring, and regression response
  • Define metrics that connect model performance, product reliability, and user experience, and use them to guide quality priorities and investment decisions
  • Lead adoption of reusable evaluation frameworks, automation, and development practices that improve quality while reducing testing costs and time to feedback
  • Partner with engineering, infrastructure, research, product, and data teams to resolve technical trade-offs and influence product and system designs before quality issues emerge
  • Mentor QA engineers and technical leads, provide guidance, and raise the quality of engineering across the broader team

Requirements

  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • Currently has, or is in the process of obtaining, a Bachelor’s degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience. A degree must be completed prior to joining Meta
  • 10+ years of quality assurance, test engineering, or software engineering experience, including substantial experience in building quality engineering systems
  • 3+ years of experience in AI model testing or evaluation, including designing evaluation frameworks and establishing model quality standards
  • Experience successfully leading a major technical quality initiative across multiple teams, from problem definition and architectural design through implementation, rollout, and measurable outcomes
  • Experience architecting and implementing scalable testing, automation, or evaluation systems in Python, PHP, Java, C/C++, or an equivalent programming language
  • Experience solving complex quality and reliability problems across client applications, backend services, and distributed systems
  • Hands-on experience testing consumer-facing and business-facing applications and APIs, and evaluating AI models
  • Hands-on experience with risk-based test strategy, performance and stress testing, and the design and maintenance of client and server test automation
  • Experience evaluating the reliability and limitations of quality metrics and using data to guide technical decisions
  • Experience leading quality engineering in fast-paced development teams, adapting test strategies and priorities as requirements evolve
  • Experience setting technical direction, resolving ambiguous requirements and cross-team dependencies, and influencing stakeholders without direct authority
  • Experience communicating complex technical problems, quality risks, and investment trade-offs to technical and non-technical stakeholders
  • Experience mentoring engineers or technical leads and driving the adoption of quality engineering practices beyond a single project

Nice to have

  • Experience leading the evaluation strategy for generative text, image, voice, or multimodal products at scale
  • Experience designing shared evaluation infrastructure or testing frameworks and leading their adoption across multiple engineering teams
  • Experience designing evaluation systems that combine human review, model-based judgments, adversarial testing, and automated regression detection
  • Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
  • Demonstrated improvements in product reliability, evaluation coverage, regression prevention, or engineering efficiency across a major product initiative
  • Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
  • Experience leading cross-platform quality efforts within large engineering organizations and building consensus on technical standards and priorities
  • Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  • Experience applying frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance
  • Experience with sta

We offer

  • Base Salary Range: $163,000.00 - $224,000.00 per year

Hiring process

  • To apply for the "QA Engineering Lead (AI)" vacancy at Meta, please join or sign in to LinkedIn. (This information is derived from the application button text)