AI Engineer/Scientist - Evaluation & Benchmarking

About us:

Axiomatic AI is building a new class of AI systems designed to reason with the rigor of the scientific method. By combining deep learning with formal logic and physics-based modeling, we create verifiable, interpretable AI systems that collaborate with and support human researchers in high-stakes scientific and engineering workflows. 

Our mission, 30×30, is to deliver a 30× improvement in the speed, accessibility, and cost of semiconductor and photonic hardware development by 2030. 

We aim to revolutionize hardware design and simulation in these industries and are building a team of highly motivated professionals to bring these innovations from research into commercial products.

Position overview: 

As an AI Engineer/Scientist specializing in Evaluation and Benchmarking, you will design, implement, and maintain scalable, reliable evaluation pipelines for advanced agentic AI systems. Your work will enable fast, reproducible, and rigorous measurement of model and agent performance across research and production settings.

You will bring strong engineering skills to build and maintain robust evaluation infrastructure end-to-end, and scientific rigor to define state-of-the-art evaluation metrics, interpret results critically, and continuously improve evaluation methodologies over time.

Your mission:

• Evaluation Infrastructure: Design, implement, and maintain reliable, scalable evaluation pipelines to measure model and agent performance efficiently and reproducibly.
• Benchmarking & Metrics: Develop automated benchmarks, performance metrics, and regression tests to enable systematic comparison and continuous performance tracking.
• Failure Analysis & Iteration: Analyze failure modes and evaluation blind spots, and iteratively improve tasks, metrics, and protocols to better capture real-world agent behavior.
• Experimentation Support: Partner with AI Scientists and Engineers to translate research goals into robust evaluation protocols and tooling that support rapid experimentation.
• Communication & Reporting: Clearly communicate evaluation results through concise reports, dashboards, and presentations, enabling both internal teams and external stakeholders to interpret and act on performance insights.
• Collaboration: Work closely with a cross-functional team of engineers and scientists, collaborating to solve challenging problems at the intersection of AI, physics, and engineering.

Key requirements:

• Master’s degree in Data Science, Computer Science, Information Technology, Artificial Intelligence, or a related field.
• 2+ years of experience in an AI, engineering, scientific, or technical setting.
• 1+ years of industry experience building AI infrastructure and applications.
• Strong software engineering skills, with experience building reliable, maintainable systems in production environments.
• Understanding of evaluation methodology and experimental design, with attention to reproducibility and statistical rigor.
• Ability to collaborate effectively within a multidisciplinary and multicultural environment.
• Curiosity and a proactive, solution-oriented mindset, with the ability to work autonomously and own the evaluation framework end-to-end.
• Excitement to work in a dynamic and fast-paced environment, and the ability to thrive in ambiguity.
• Personal interest in exploring recent AI developments and staying up to date with the state of the art.

Nice to have:

• Experience building or maintaining evaluation and benchmarking pipelines for ML/LLM systems.
• Familiarity with LLM/agent evaluation methods, including rubric-based scoring, judge models, or regression testing.
• Experience with experiment tracking and evaluation tooling such as Weights & Biases, LangSmith, or similar platforms.
• Research experience in AI/ML, demonstrated through publications, a thesis, or research projects.
• Evidence of strong technical initiative, such as open-source contributions, personal projects, or leading complex technical work.

What we offer:

• Competitive compensation
• Stock Options Plan: Empowering you to share in our success and growth.
• Cutting-Edge Tools: Access to state-of-the-art tools and collaborative opportunities with leading experts in artificial intelligence, physics, hardware and electronic design automation.
• Work-Life Balance: Flexible work arrangements in one of our offices with potential options for remote work.
• Professional Growth: Opportunities to attend industry conferences, present research findings, and engage with the global AI research community.
• Impact-Driven Culture: Join a passionate team focused on solving some of the most challenging problems at the intersection of AI and hardware.

Why join us? 

At Axiomatic_AI, you will be working on technology that drives innovation in AI for scientific and engineering applications in line with our 10X30 mission. This is your opportunity to contribute to the development of new AI architectures that can reason coherently and produce interpretable and verifiable solutions. Consequently, see those ideas commercialized into products that will shape the future of hardware and computing, while collaborating with a global team of engineers and AI specialists. We believe in pushing the boundaries of what is possible and continuously seek to redefine the intersection of AI, with focus on formal consistency. If you're ready to take your expertise in artificial intelligence and physics to the next level, we want to hear from you!

Worried about not meeting every qualification? Studies show that women and people of color are less likely to apply for jobs unless they meet every listed requirement. At Axiomatic-AI, we are dedicated to creating a diverse, inclusive, and authentic workplace. If this role excites you but your background doesn’t perfectly match every qualification, we still encourage you to apply. You could be the perfect fit for this position or another opportunity with us.