Staff Software Engineer, Gemini App, Horizontal Quality, DeepMind
- linkCopy link
- emailEmail a friend
Minimum qualifications:
- Bachelor’s degree or equivalent practical experience.
- 8 years of experience in software development.
- 5 years of experience leading technical strategy and architecting large-scale ML infrastructure (e.g., designing serving layers, model evaluation frameworks, or data processing pipelines).
- 5 years of experience testing, and launching software products.
- 3 years of experience with Generative AI, Large Language Models (LLMs), Machine Learning, and related frameworks.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
- Experience in building and scaling evaluation pipelines (e.g., RLHF, auto-evals, or side-by-side human evaluations) to measure helpfulness and accuracy.
- Proficiency in advanced prompting techniques and understanding how model fine-tuning, RL, or RAG impacts final response quality.
- Ability to use SQL, Python, or internal data tools to analyze user behavioral data and "pain points" to identify where the model is failing.
About the job
Our mission is to elevate the Gemini experience by perfectly aligning foundational model behaviors with high-quality data. We drive conversational excellence through thoughtful persona shaping, robust safety enforcement, and clear information architecture. By combining these efforts with rich online and offline signals, we deliver a product that is highly performant and effortlessly intuitive.
Artificial intelligence will be one of humanity’s most transformative inventions. At Google DeepMind, we are a pioneering AI lab with exceptional interdisciplinary teams focused on advancing AI development to solve complex global challenges and accelerate high-quality product innovation for billions of users. We use our technologies for widespread public benefit and scientific discovery, ensuring safety and ethics are always our highest priority.
Responsibilities
- Design, build, and maintain a highly scalable evaluation framework specifically tailored to measure the product-level quality of the Gemini App, moving beyond standard model-level benchmarks.
- Create, synthesize, and refine meaningful metrics that accurately capture the user experience.
- Be responsible for developing a holistic view of quality by combining newly engineered online metrics with deep offline evaluation data.
- Build a transparent, definitive ranking system for product-level quality. Use this system to benchmark the Gemini App against industry standards and clearly identify our competitive strengths and weaknesses.
- Act as a critical partner to Product Management and leadership by translating complex evaluation data into clear, strategic signals. Offer technical leadership on high-impact projects.
Information collected and processed as part of your Google Careers profile, and any job applications you choose to submit is subject to Google's Applicant and Candidate Privacy Policy.
Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents-to-be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google's EEO Policy, Know your rights: workplace discrimination is illegal, Belonging at Google, and How we hire.
If you have a need that requires accommodation, please let us know by completing our Accommodations for Applicants form.
Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.
To all recruitment agencies: Google does not accept agency resumes. Please do not forward resumes to our jobs alias, Google employees, or any other organization location. Google is not responsible for any fees related to unsolicited resumes.