Navital Gym · answers
How do you test an AI agent on UPI and GST workflows?
Give the agent real tasks in a sandbox and grade each one with written rules that recompute the answer. Navital Gym has 50 public UPI tasks and 50 public GST tasks (321 public tasks across six worlds), plus held-out tasks we have never published. You can run the public ones yourself with pip install navital-gym, or have us run your agent on the private set.
As of 12 November 2026 · Navital Gym v1.1.0
What the tasks cover
- UPI: pay and verify, failed-but-debited payments, limits, collect requests, wrong recipient, mandates, UPI Lite. 29 of the 50 tasks carry a forbidden step, and an attempt counts even if the simulated bank refuses it.
- GST: invoices, place of supply, input tax credit reconciliation, late filing, thresholds.
- Tool calls, not chat: 18 tools in UPI Desk, 5 in GST Desk. The agent submits a decision and the grader checks it.
- Impossible requests on purpose: the agent has to refuse, with the right reason.
Written rules, no AI judge
- Written rules recompute the answer from the sandbox. Nothing the agent claims is trusted, and no AI model is the judge.
- A task counts as solved only when every required rule passes, on one attempt.
- Many tasks have no valid answer. The agent has to say so and give the real reason.
- A reference solver must solve every public task, and scripted agents that game the format are checked before a world ships.
Related work
NPCI has published its own benchmark for banking agents, TauIndianBankBench. It covers banking conversations; ours covers UPI payment-desk and GST tasks.
Have us test your agent
- Private evaluation, $750 per agent version: your agent on our private held-out tasks, with a written report within 5 working days after setup.
- Pilot, from $2,500: 20-30 tasks built around one flow of yours, with grader, reference solutions and a short report.
- Training Pilot, from $7,500: we train an open model you choose on a world and test it on held-out tasks.
Prices in USD, plus applicable taxes (none for clients outside India). Scope is written down before work starts.
Questions
How do I test my AI agent on UPI workflows?
Install Navital Gym (pip install navital-gym), point it at any model that speaks the OpenAI chat API with tool calling, and run the UPI Desk tasks. Each task is graded by written rules, so the same answer always gets the same score. You do not need a GPU to run the tasks.
Can an AI agent be tested on GST invoices and returns?
Yes. GST Desk has 50 public tasks on invoices, place of supply, input tax credit, late filing and thresholds. The grader recomputes the tax from the documents and does not trust the agent's arithmetic.
Is the grading done by an AI model?
No. Every task is graded by named, deterministic rules. There is no judge model and no sampling noise in the grading.
Can you test my agent privately?
Yes. A private evaluation runs your agent on held-out tasks we have never published, for $750 per agent version, with a private written report.
What this does not show
- All data is synthetic. The rules are a simplified, documented version of Indian practice, not a statement of current law or bank rules.
- Public tasks are open, so a model may have seen them. The held-out tasks exist for that reason.
- Navital Gym is not affiliated with NPCI, GSTN or any bank.
Navital GymLeaderboardGST agent benchmark: Navital Gym GST DeskHow to evaluate Hindi and Hinglish customer support agentsEvaluating AI agents for Indian finance and RBI FREE-AI