Senior Data Scientist (Agent Evaluation & Optimisation)

About Swap

Swap is the infrastructure behind modern agentic commerce. The only AI-native platform connecting backend operations with a forward-thinking storefront experience.

Built for brands that want to sell anything - anywhere, Swap centralizes global operations, powers intelligent workflows, and unlocks margin-protecting decisions with real-time data and capability. Our products span cross-border, tax, returns, demand planning, and our next-generation agentic storefront, giving merchants full transparency and the ability to act with confidence.

At Swap, we’re building a culture that values clarity, creativity, and shared ownership as we redefine how global commerce works.

About the Role

As a Senior Data Scientist, you’ll help build the intelligence layer behind Swap’s agentic storefront, making our AI agents feel helpful, consistent, and trustworthy. This is a hands-on role focused on agent components like memory, multimodal or VTO workflows, and our voice agent. You’ll work closely with product and engineering and have real ownership over what we build and how it evolves.

Key responsibilities

• Agent architecture & component design: Design and evolve the agent system and its components (memory, tool use, workflows), with clear interfaces and reliable behaviour. Partner with engineering to integrate retrieval and search capabilities into agent flows.
• Evaluation-first improvement: Build evaluation harnesses (offline and online) to measure quality, reliability, and task completion. Define rubrics, golden sets, regression tests, and lightweight dashboards. Run experiments (A/B tests where appropriate) and translate results into concrete product improvements.
• Memory systems (Python): Implement and iterate on memory. Capture relevant user and context signals, improve retrieval and selection, and evaluate usefulness and failure modes. Establish feedback loops to learn from real interactions and systematically reduce errors.
• Multimodality or VTO (Python): Iterate on visual and multimodal workflows using APIs. Evaluate outputs, identify failure patterns, and improve prompts, tools, and data. Build pragmatic test sets for multimodal quality and user-perceived outcomes.
• Voice agent experience (TypeScript and product sensibility): Help develop the voice agent. Evaluate the end-to-end experience, craft the persona, add tools, call APIs, and iterate. You don’t need to be a TypeScript expert, but you should be willing to work in it and learn fast.
• Prompting & tool reliability: Create and refine prompts, tool schemas, and structured outputs to reliably execute multi-step workflows. Add practical guardrails such as fallbacks, error handling, logging and tracing hooks, and regression checks.

What we would like to see

Must-haves

• 3+ years experience driving data-driven optimisation (GenAI products and or data science problems), with a track record of shipping improvements.
• Strong evaluation mindset. You have built measurement frameworks, test sets, or experimentation loops that make systems measurably better.
• Solid system and architecture thinking for agent-like products, including clear interfaces, iterative improvement, and pragmatic tradeoffs.
• Strong Python engineering, and comfortable integrating with APIs. You can build quick, reliable prototypes and harden them over time.
• Comfort working closely with product and engineering and owning outcomes end-to-end.

• Experience with voice systems, multimodal workflows, or building agent tools.
• Some TypeScript experience, or strong willingness to learn.
• E-commerce or marketplace familiarity.

Benefits:

• Competitive base salary
• Stock options in a high-growth startup
• Competitive PTO with public holidays additional
• Private Health
• Pension
• Wellness benefits
• Breakfast Mondays

Diversity & Equal Opportunities:

We embrace diversity and equality in a serious way. We are committed to building a team with a variety of backgrounds, skills, and views. The more inclusive we are, the better our work will be. Creating a culture of equality isn't just the right thing to do; it's also the smart thing