Meta 4 дня назад
QA Engineering Lead (AI)
163 000 – 224 000 $в год
- Менло-Парк, Калифорния, Соединенные Штаты Америки
- Опыт от 10 лет
PythonAIJavaC++PHPGen AIQuality AssuranceleadTest EngineeringMultimodalVoiceImagegenerative text
Описание
Responsibilities
- Define and drive the technical quality strategy and roadmap for a major Gen AI initiative spanning multiple engineering teams, balancing product risk, delivery speed, and engineering investment
- Lead and perform hands-on testing of consumer-facing and business-facing applications, APIs, and AI models, adapting quality priorities and test coverage in a fast-paced development environment
- Architect and implement shared AI evaluation and test automation systems that support multiple models, products, and platforms, while directly solving the most challenging technical problems
- Establish consistent evaluation methods, benchmark design, product-aligned weighted scoring frameworks, and quality standards for model behavior and end-to-end product experiences, working with research, engineering, and product partners
- Lead the development of reliable evaluation signals, including calibration of human and automated judgments, analysis of measurement uncertainty, and investigation of gaps between offline evaluations and real-world outcomes
- Identify systemic quality risks across models, data, applications, and infrastructure, and lead cross-team efforts to address their root causes
- Drive quality planning for major launches and phased rollouts, aligning teams on success criteria, dependencies, monitoring, and regression response
- Define metrics that connect model performance, product reliability, and user experience, and use them to guide quality priorities and investment decisions
- Lead adoption of reusable evaluation frameworks, automation, and development practices that improve quality while reducing testing costs and time to feedback
- Partner with engineering, infrastructure, research, product, and data teams to resolve technical trade-offs and influence product and system designs before quality issues emerge
- Mentor QA engineers and technical leads, provide guidance, and raise the quality of engineering across the broader team
Requirements
- Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
- Currently has, or is in the process of obtaining, a Bachelor’s degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience. A degree must be completed prior to joining Meta
- 10+ years of quality assurance, test engineering, or software engineering experience, including substantial experience in building quality engineering systems
- 3+ years of experience in AI model testing or evaluation, including designing evaluation frameworks and establishing model quality standards
- Experience successfully leading a major technical quality initiative across multiple teams, from problem definition and architectural design through implementation, rollout, and measurable outcomes
- Experience architecting and implementing scalable testing, automation, or evaluation systems in Python, PHP, Java, C/C++, or an equivalent programming language
- Experience solving complex quality and reliability problems across client applications, backend services, and distributed systems
- Hands-on experience testing consumer-facing and business-facing applications and APIs, and evaluating AI models
- Hands-on experience with risk-based test strategy, performance and stress testing, and the design and maintenance of client and server test automation
- Experience evaluating the reliability and limitations of quality metrics and using data to guide technical decisions
- Experience leading quality engineering in fast-paced development teams, adapting test strategies and priorities as requirements evolve
- Experience setting technical direction, resolving ambiguous requirements and cross-team dependencies, and influencing stakeholders without direct authority
- Experience communicating complex technical problems, quality risks, and investment trade-offs to technical and non-technical stakeholders
- Experience mentoring engineers or technical leads and driving the adoption of quality engineering practices beyond a single project
Nice to have
- Experience leading the evaluation strategy for generative text, image, voice, or multimodal products at scale
- Experience designing shared evaluation infrastructure or testing frameworks and leading their adoption across multiple engineering teams
- Experience designing evaluation systems that combine human review, model-based judgments, adversarial testing, and automated regression detection
- Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
- Demonstrated improvements in product reliability, evaluation coverage, regression prevention, or engineering efficiency across a major product initiative
- Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
- Experience leading cross-platform quality efforts within large engineering organizations and building consensus on technical standards and priorities
- Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
- Experience applying frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance
- Experience with sta
We offer
- Base Salary Range: $163,000.00 - $224,000.00 per year
Hiring process
- To apply for the "QA Engineering Lead (AI)" vacancy at Meta, please join or sign in to LinkedIn. (This information is derived from the application button text)