Senior Technical Program Manager, Gemini Evals

About Deepmind

Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.

Job Summary

The role sits within Google DeepMind’s Gemini Evals TPM team. The team is responsible for designing and managing Gemini Evals suite to test model performance. The Gemini Evals team is responsible for developing new ways to test and measure model performance. This is an open-ended problem that requires close collaboration between research and engineering and applying the state of the art AI/ML to conversational problems. As a Technical Program Manager in the Gemini Eval team, you will be responsible for driving the development of new evals and the infrastructure to run them. This role requires a strong technical background in AI, excellent program management skills, and the ability to navigate the challenges of large-scale research and engineering efforts.

Key Responsibilities

• Lead complex and ambiguous programs focusing on execution and delivery.
• Partner with research and engineering leads to translate strategic goals into well-defined program plans, timelines, and success metrics.
• Lead and drive progress and improvements in the Evals space to enable Gemini to increase its operating velocity.
• Manage dependencies across workstreams, functions and orgs.
• Drive and lead engineering teams to identify, prioritize and track tasks to completion targets.
• Identify risks and corresponding mitigation plans, and put into motion the solutions.
• Communicate progress, risks and plans to leadership on a regular basis.
• Identify operational and process improvements and quickly set-up lightweight processes to fill gaps.
• Manage multiple, time-sensitive projects simultaneously.
• Partner closely with PM and Eng leads to understand and influence product direction and ensure execution matches.

Requirements

• Strong technical background, ideally a bachelor or master degree in computer science, data science or related fields.
• Strong biased towards action and looking for a start-up mindset and fast-paced environment and have an entrepreneurial drive.
• Excellent technical understanding and communication ability, with the ability to distil sophisticated technical ideas to their essence.
• Skilled at navigating, updating and defining complex processes.
• Strong and documented experience running large-scale projects.