Evaluation, RAG and agents

Learn how AI engineers test, search and build agents with language models.

We offer interactive chapters where you can test while learning. The case studies, quizzes and Python labs use real situations and numbers. The labs run on your laptop and do not need an API key.

What's in a chapter

1

Case study

Each chapter starts with a short story about a team that had the problem, with their numbers.

2

The reading and a lab

A methodology with realistic numbers is presented, supported by a small Python lab. See the labs on GitHub

3

Notes for new grads and managers

Key points for new grads and founders are highlighted at the end of each chapter.

4

Quizzes

Each quiz describes a challenge and asks what you would do.

Available now

The AI Evaluation Engineer

Designing critical metrics and measurements for nondeterministic AI systems.

Coming soon

The AI Search / RAG Engineer

Designing systems retrieving and utilizing the right documents for the model.

Read Soon
Coming soon

The AI Agent Engineer

Building forward-looking, predictive and on-budget agents.

Read Soon

A course costs $29. The PDF book is emailed after the payment. The first chapter of every course is free, and you do not need an account to read it. You can also try the free lessons, which use real public data.

The RAG and Agents courses

Enter your email to get notified of releases.