Let AI take your exams

[Anand S]

At 2 pm IST today (Fri 12 Jun 2026), I'm running a workshop at Paradox, IITM - at DOMS 101.

You can join online at https://meet.google.com/cpt-faee-ucx and ask questions on chat.

Agenda:

You've been told AI can pass your exams. But what happens when you actually watch it try — live, on your questions, in real time?

This workshop starts with a collective experiment: we ask coding agents to solve real exams (including IITM exams) and see how it solves them.

What follows isn't a tutorial on prompting — it's an autopsy that reveals what your exams are actually testing, where AI confidently hallucinates, and what that means for what's worth learning.

You'll leave with a reframed understanding of your degree (the goal isn't answers, it's the ability to catch wrong ones) and a concrete study rituals that uses AI as a Socratic sparring partner rather than an answer machine.

Come with a question you got wrong recently — it's going to be useful.

Real agenda: An ask-me-anything session plus real-life experiments.

[Anand S]

At 2 pm IST today (Fri 12 Jun 2026), I conducted a workshop at Paradox, IITM - at DOMS 101.

My core message is: "AI can solve exams and help you learn. Delegate what AI can do. Learn what AI can't do instead."

Video: Watch video

My talks page for "Let AI take your exams" includes:

• The full story + transcript + audio
• How Codex solved a real exam, live
• My collection of AI-learning techniques - which was not covered in the workshop, but is a useful reference

Here are the takeaways from the workshop:

1. AI is more capable than you think — and getting smarter. Recalibrate constantly what it can and can't do. Note down what it can't, because that is precisely where your value lives.
2. Delegate first; learn the rest. Give everything to AI. Focus your learning on what it can't yet do — that's where the value will be. It's a moving filter; revisit it every quarter.
3. Always use the best model, turned up high. Reserve "fast and cheap" for the ~5% of moments you need a quick answer. And remember students get serious AI free via the GitHub Student Pack and Gemini.
4. Make the AI ask you for context. "If you need more information, ask me." You don't have to know what context it needs — push that burden back to the machine.
5. Beat hallucination with a maker-checker. Two independent models that must agree cut errors from 14% to under 4%. Tell the checker to "find the errors," not to grade.
6. Loop with feedback in verifiable environments. Point an agent at an exam, a codebase, anything that scores itself — let it try, submit, read the result, retry. This is the most powerful technique AI has.
7. Calibrate, don't just trust. Practise predicting whether AI will get something right — even on topics you don't know. Watch for base-rate traps and familiar problems with one changed premise.
8. Be lazy, productively. Don't read AI's 20-page output — train it to give you five words. Working well with AI is a management skill.
9. Learn from peers. Multiple people trying things is how you discover what works. Non-transactional relationships are the rising currency of the AI era.
10. Apply the scientific method to everything. Form a hypothesis, hunt for evidence, try to falsify yourself. And when a system blocks you unfairly — hack it, then publish what you learned.

Here are the takeaways from how Codex solved the exam:

1. An agent operates the environment; a chatbot answers the question. Codex read source, ran code, clicked Check, and looped on feedback. That's why it beat copy-paste — the exam was full of affordances a chatbot can't touch.
2. Verifiable environments favour AI. The more checkable the exam — validators, error strings, downloadable files, a live Check button — the more it helped the agent, not the student. "AI-proof" and "feedback-rich" are opposites.
3. Most failures were wording, not reasoning — and the source fixed them. The fix for a brittle validator was to read the validator. Pass the error message back to the agent and it converges.
4. Don't guess where attempts are limited. The network game punished early guesses. The recoverable mistakes had feedback; the costly ones didn't. Triage cheap-and-certain first.
5. The gap between 9 and 10 was a credential, not a brain. Same model, same skill. Anand's missing mark was an invalid token. In the AI era, "can it?" often means "does it have the keys?"
6. It cost about a coffee. ~$2–3 of tokens for the whole exam, ~96% cached. The real cost is the human judgment to know when the agent is plausibly, confidently wrong.

Original announcement