Job Description
World Wide - Remote
A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases.
Responsibilities
Create and execute
role-play–based evaluation scenarios
that simulate realistic customer service interactions across multiple domains, including:
Flight bookings and travel support
Financial services
Telecommunications and technical support
Contribute to the development of
diverse and representative datasets
used to assess conversational audio agents.
Evaluate model performance across a standardized set of qualitative and quantitative metrics.
Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow.
Evaluation Metrics
Model performance is assessed using a combination of conversational, technical, and audio-specific criteria, including but not limited to:
Task completion accuracy
and efficiency
Conversational naturalness
(tone, flow, and coherence)
Audio comprehension and response quality
Instruction adherence and contextual understanding
Basic computer programming literacy , including:
Understanding of
JSON structures
Familiarity with
functions and methods
Ability to reason about structured data and simple logic
Technical communication clarity
when handling support-style problem-solving
Technical & Equipment Requirements
Access to a
high-quality microphone
to ensure clean, reliable audio input during evaluations
Comfort working with structured prompts, evaluation rubrics, and technical guidelines
We offer a pay range of $8-to-$25 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.
#J-18808-Ljbffr
Ready to Apply?
Don't miss this opportunity! Apply now and join our team.
Job Details
Posted Date:
February 28, 2026
Job Type:
Education
Location:
Indonesia
Company:
Meridial Marketplace, by Invisible
Ready to Apply?
Don't miss this opportunity! Apply now and join our team.