Navital Gym · answers
Is there a benchmark for AI agents on GST tasks?
GST Desk, one of the six worlds in Navital Gym, is an open set of 50 public GST tasks for AI agents, with 30 more held out and never published. An agent looks up rates, reads the documents and submits the tax; written rules recompute the figures and mark the result. Code is Apache-2.0 and the task data is CC BY 4.0.
As of 12 November 2026 · Navital Gym v1.1.0
What GST Desk covers
- Invoices: 15 tasks. CGST, SGST and IGST from the documents.
- Input tax credit reconciliation: 10 tasks.
- Late filing: 10 tasks. Due dates, interest and fees.
- Place of supply: 8 tasks. Threshold: 7 tasks.
- Five tools: calendar_info, get_documents, get_policy, lookup_rate, validate_gstin.
Written rules, no AI judge
- Written rules recompute the answer from the sandbox. Nothing the agent claims is trusted, and no AI model is the judge.
- A task counts as solved only when every required rule passes, on one attempt.
- Many tasks have no valid answer. The agent has to say so and give the real reason.
- Rules are named per world (for example R-CGST, R-SGST, R-IGST, R-ITC-AMOUNT, R-INTEREST), so a failure points at the rule that failed.
Have us test your agent
- Private evaluation, $750 per agent version: your agent on our private held-out tasks, with a written report within 5 working days after setup.
- Pilot, from $2,500: 20-30 tasks built around one flow of yours, with grader, reference solutions and a short report.
- Training Pilot, from $7,500: we train an open model you choose on a world and test it on held-out tasks.
Prices in USD, plus applicable taxes (none for clients outside India). Scope is written down before work starts.
Questions
What does the GST agent benchmark measure?
Whether an agent gets Indian GST work right when it has to look things up and compute: tax split, input credit, late fees, place of supply. A task is solved only when every required rule passes.
How many GST tasks are there?
50 public tasks in GST Desk v1.1.0, plus 30 held-out tasks that are kept private. The public tasks are free to run.
How do I run it?
pip install navital-gym, then run the gst environment against any model that speaks the OpenAI chat API with tool calling. The code is on GitHub.
Does it use an AI judge?
No. The grader recomputes everything from the sandbox with written rules.
What this does not show
- GST rules are simplified and documented as of October 2026. This is a test for agents, not tax advice or a statement of current GST law.
- Not affiliated with GSTN or CBIC. All data is synthetic.
- Public tasks are open and models may have seen them.
Navital GymLeaderboardHow to test an AI agent on UPI and GST workflowsHow to evaluate Hindi and Hinglish customer support agentsEvaluating AI agents for Indian finance and RBI FREE-AI