Navital Gym · answers
How do you evaluate a Hindi or Hinglish customer support agent?
Give the agent real support requests in Hindi and Hinglish and grade what it does, not how it sounds. Bhasha Support Desk has 50 core tasks on returns, refunds and privacy requests, plus 26 experimental Tamil, Bengali and Marathi tasks. Written rules check the decision, the actions and the reply language.
As of 12 November 2026 · Navital Gym v1.1.0
What the tasks cover
- Returns, refunds and privacy requests, in Hindi (Devanagari) and Hinglish.
- Decisions that depend on policy: refund window, photo evidence, bank details.
- Tool calls: get_order, get_customer_bank, get_photo_status, get_policy, return_window_status, validate_ifsc.
- Forbidden steps: for example, refunding before a required photo has been uploaded.
Written rules, no AI judge
- Written rules recompute the answer from the sandbox. Nothing the agent claims is trusted, and no AI model is the judge.
- A task counts as solved only when every required rule passes, on one attempt.
- Many tasks have no valid answer. The agent has to say so and give the real reason.
- Tamil, Bengali and Marathi tasks are experimental: machine-assisted and not native-reviewed. The reply language is a hard rule there too, checked by script (Marathi also by marker words).
One public task, with the rule checked
IN-SUP-030: "मेरा ऑर्डर OD81401 (Cotton kurta) टूटा हुआ आया है। पैसे वापस कीजिए।" A damaged-item refund needs a photo and none was uploaded, so the agent must ask for the photo before refunding, and reply in Hindi.
Have us test your agent
- Private evaluation, $750 per agent version: your agent on our private held-out tasks, with a written report within 5 working days after setup.
- Pilot, from $2,500: 20-30 tasks built around one flow of yours, with grader, reference solutions and a short report.
- Training Pilot, from $7,500: we train an open model you choose on a world and test it on held-out tasks.
Prices in USD, plus applicable taxes (none for clients outside India). Scope is written down before work starts.
Questions
How do I evaluate a Hindi customer support agent?
Run it on tasks where the right answer is a decision and a set of actions, then grade those with written rules: did it ask for the photo, did it hold the refund, did it reply in Hindi. Bhasha Support Desk has 50 core Hindi and Hinglish tasks you can run for free.
Does it cover Hinglish?
Yes. The core support tasks are in Hindi and Hinglish. Tamil, Bengali and Marathi tasks are experimental and not yet native-reviewed.
Is the grading done by an AI model?
No. Written rules mark every task, so the same answer gets the same score.
Can you evaluate my support agent privately?
Yes. A private evaluation runs your agent on held-out tasks, including support chats in Hindi and Hinglish, for $750 per agent version.
What this does not show
- The tasks are synthetic shop-support requests, not your company's tickets. A Pilot builds tasks around your own flow.
- Voice is not covered: these are text tasks.
- Experimental language splits (Tamil, Bengali, Marathi) are not native-reviewed.
Navital GymLeaderboardHow to test an AI agent on UPI and GST workflowsGST agent benchmark: Navital Gym GST DeskEvaluating AI agents for Indian finance and RBI FREE-AI