Testing AI Agents with Python: LLM Evaluation with DeepEval

About this course

This course contains the use of artificial intelligence.

Your AI agent says the refund is done. Status ok. Logs clean. Five unit tests green. It just sent $1,249.00 to the wrong customer, when $124.90 was owed to someone else. This course teaches you to catch that, and every failure like it, before it ships.

Testing AI agents is not like testing ordinary code. The same input can produce different words, a confident reply can hide a wrong tool call, and a passing test suite can miss the one thing that matters: what the agent actually did. In this hands-on course you build a complete AI agent testing and LLM evaluation harness in Python, step by step, on one realistic project.

Why this course is different

You work on one project from the first minute: a customer-support refund agent, 30 hand-labeled tickets, and a test harness you grow module by module. Every lab runs offline on your own machine. No API key, no cloud account, nothing to pay for. Every number on screen comes from a real run you can repeat, so your terminal matches the video.

Where a real model matters, the course ships recorded gpt-4o-mini answers (five runs of every ticket), so you can study real model behaviour without calling a model.

Every module ends with "your turn": one small test you write yourself, with the command that proves it.

What you will build and learnAI agent fundamentals for testers: tokens, context, temperature, the agent loop, tools, memory, and six ways agents fail (hallucination, wrong tool, wrong arguments, reasoning errors, goal drift, infinite loops).

Why traditional testing breaks for non-deterministic systems, and what to assert instead of exact text. LLM evaluation with pytest and DeepEval: test cases, golden datasets, metrics, thresholds, and reports.

Prices, discounts and availability are set by Udemy. We may earn a commission when you purchase through links on this site.