AI Quality Assurance Engineer, Consultant

Irembo is a technology company that designs and develops digital products to ease the accessibility of services in users’ everyday lives worldwide, starting with Rwanda. Our pioneer products, IremboGov and IremboPay, have enabled Rwandan citizens and foreigners to access and pay for over 150 public services online through our one-stop-shop e-governance and payment platforms. To date, we have facilitated over 30 million transactions through our platforms and have ambitious goals to scale our technology worldwide to enable more governments and institutions to serve their citizens better. More information is available on irembo.com.

Location: Kigali, Rwanda, or Remote. 

Duration: 10 months.

Terms of Reference - AI Quality Assurance Engineer, Consultant 

The Opportunity 

We are seeking a detail-oriented and experienced AI Quality Assurance (QA) Engineer to join our team on a 10-month contract basis. You will be responsible for ensuring the reliability, accuracy, and quality of our AI solutions, which are designed to assist citizens in applying for government services and feature voice-over-AI capabilities. This role requires hands-on testing and a good understanding of AI model evaluation.

Key Responsibilities: 

• Test Strategy & Planning: Develop and execute comprehensive test plans and test cases specifically for AI/ML components, covering functional, non-functional, integration, and performance testing.
• AI Model Validation: Design and implement methods to validate the accuracy, fairness, robustness, and performance of various AI/ML models, including those powering the Voice over AI and service application capabilities.
• Data Quality Assurance: Assess and ensure the quality, integrity, and coverage of the training, validation, and testing datasets used for the AI models.
• Voice over AI Testing: Focus on testing the end-to-end functionality and user experience of the Voice over AI feature, including testing for speech recognition accuracy, natural language understanding (NLU) precision, response relevance, and latency.
• Automation: Design, develop, and maintain automated testing scripts and frameworks for continuous integration and continuous deployment (CI/CD) pipelines, particularly for the software surrounding the AI models.
• Bug Identification and Reporting: Accurately identify, document, prioritize, and track defects and issues related to model performance or software implementation using defect tracking tools.
• Collaboration: Work closely with AI Engineers, Software Developers, and Product Managers to understand system requirements, communicate quality risks, and ensure timely resolution of issues.
• Compliance: Ensure the solution meets required quality standards, security protocols, and, where applicable, government accessibility and service application guidelines.

Qualifications: 

Required Skills & Experience

• Experience: Minimum of 3+ years of professional experience in Quality Assurance or Software Testing, with at least 1 year focused on testing AI/ML-driven products.
• Testing Expertise: Strong understanding of QA methodologies, tools, and processes, and experience with both manual and automated testing.
• Programming Skills: Proficiency in a programming language suitable for test automation.
• NLP/Speech Familiarity: Basic to intermediate understanding of Natural Language Processing (NLP), speech recognition, or machine learning concepts, especially those related to evaluating model output.
• Contract Focus: Ability to quickly integrate and deliver tangible results within a structured, fixed-term contract engagement.

Preferred Skills

• Experience with specific AI/ML testing frameworks or libraries (e.g., Deepchecks, Arize, or similar).
• Hands-on experience testing Voice-enabled applications (e.g., IVR, voice assistants).

Why Join This Project?

• High Impact: Your work will directly impact the lives of millions of Rwandans, ensuring no one is left behind in the digital transformation journey.
• Innovation: Work at the cutting edge of African AI, solving the challenge of digitization for low-resource languages.

Please note that the salary for this position is commensurate with experience and qualifications and will be discussed during the interview process. 

Application Deadline

• December 7, 2025

We are an equal opportunity employer and are committed to providing a positive interview experience for every candidate. We're on a mission to change our continent through technology and are committed to a diverse and inclusive workplace and strongly encourage applicants from all backgrounds, nationalities, and walks of life.

Our head office is based in Kigali, Rwanda.