Bhasha Support Desk
Status: Live · updated as runs finish
It answers an online shop’s customers in Hindi or Hinglish (plus an experimental, machine-assisted Tamil, Bengali and Marathi split) about returns, cancellations, damaged items and refunds, using tools that change order state.
Example request (illustrative, Hindi, not one of the test tasks). “मुझे जो हेडफ़ोन मिला है, वह टूटा हुआ है। मुझे पैसे वापस चाहिए।” The agent must follow the shop’s policy (a photo is needed first), act through the tools, and reply in Hindi.
The resulting order state, whether any forbidden action was taken (for example revealing another customer’s data), and, for Hindi and Hinglish, whether the reply is in the customer’s language, checked by script and marker words. The experimental Tamil, Bengali and Marathi split is graded on the order state and actions only.
The shop policy is invented for the environment and is not Indian consumer law. The language check is a heuristic and imperfect for code-mixed replies. The Hindi and Hinglish lines were drafted by the Navital Labs team with AI assistance; the Tamil, Bengali and Marathi lines are machine-assisted and not native-reviewed. Corrections are welcome.
git clone https://github.com/Navitallabs/navital-gym.git
cd navital-gym
pip install -e .
navital-gym run --env upi --agent oracleNavital GymWhat is an agent environment?Report a wrong rule
Rules last checked: 25 October 2026