Middle AI QA Engineer
We are looking for an AI QA Engineer to ruthlessly test our digital employees ability to listen, think, and speak in real-time. Any delay or logical glitch breaks the magic; your mission is to ensure it stays seamless.
About us
Mad Devs is a full-stack team specializing in developing and administering large-scale IT products across various industries. Many of our products are used in Europe, the USA, and Southeast Asia. We are passionate about tackling complex challenges and delivering innovative solutions that make a significant global impact
#LINKD
What will need to do:
- Voice UX Testing: Measure and optimize Latency (P95/P99). Verify Barge-In functionality: ensure the agent stops speaking immediately when interrupted, even in noisy environments
- Logic & Flow Validation: Read and understand NSL (Jinja2) scripts to identify potential logical loops or dead ends. Verify that the agent handles state transitions correctly (e.g., booking created -> confirmation sent)
- Hallucination Control: Detect when the AI drifts from its instructions or invents facts. Help build "Golden Datasets" of correct conversations for regression testing
- Integration Testing: Verify that the agent correctly triggers external tools (e.g., CRM updates) during a call. What happens if the API fails? Does the agent handle it gracefully?
Our expectations:
- 2+ years of hands-on experience in manual testing
- Ability to read code (Python, Jinja2) and work confidently with JSON data payloads to better understand test scenarios
- Understanding of STT (Speech-to-Text), TTS (Text-to-Speech), and VAD (Voice Activity Detection) concepts, including the importance of silence detection
- Strong exploratory testing mindset with the ability to intentionally stress and challenge AI agents (e.g., interruptions, rapid speech, topic switching, unclear pronunciation)
- Russian - Native or Fluent; English - Upper-Intermediate (B2+)
It will be a plus:
- Experience with LLM prompting
- Experience testing voice assistants (Alexa, Google Assistant)
Why Join Us
Be part of an innovative, remote-first team that values autonomy, clear communication, and building products that matter.
Employees benefits
Flexible working hours
Remote-first culture
Long-term projects
Salary in dollars
Professional communities
Onsite business trips
Training budget
Paid conferences
Hi, I'm Ekaterina.
Applying for jobs should be easy. I am here to simplify the hiring process and set you up for success. Feel free to send me a message via e-mail .