Skip to content
A.B. Labs

AI / Machine Learning

Applied AI & LLM Engineering

Design, build, evaluate and ship applications powered by large language models — prompting, RAG, agents, fine-tuning, evaluation, safety and deployment.

Overview

Large language models have changed what software can do, but building reliable LLM applications takes real engineering. This course covers the full workflow: how these models behave, how to ground them in your own data with retrieval, how to let them use tools safely, how to measure whether any of it actually works, and how to run it in production within a budget.

The centre of gravity is evaluation. Anyone can demo a prompt; the difference between a demo and a product is a test suite, a cost model, a latency budget and a guardrail that holds when a user does something unexpected. You will build working AI applications in Python and FastAPI, break them deliberately, measure them properly and deploy them behind real observability.

Who it's for

  • Developers who want to build AI-powered products
  • Learners who have completed Python for AI & Machine Learning, or have equivalent experience
  • Engineers adding LLM features to existing systems
  • Technical leads who need to judge whether an AI feature is production-ready

Before you start

  • Working knowledge of Python
  • Familiarity with APIs, JSON and the command line
  • Comfort with Git and basic software engineering practice
  • Basic machine learning concepts are helpful but not required

Outcomes

What you'll learn.

  1. 01

    Explain how LLMs work, where they fail and why they hallucinate

  2. 02

    Choose between hosted and open models on capability, cost, latency and privacy

  3. 03

    Design effective, versioned, testable prompts and reliable structured outputs

  4. 04

    Build retrieval-augmented generation pipelines, and measure retrieval quality separately from generation

  5. 05

    Apply advanced retrieval techniques: hybrid search, re-ranking and query rewriting

  6. 06

    Create tool-using agents with clear boundaries and recovery paths

  7. 07

    Extract structured data from documents, images and PDFs

  8. 08

    Fine-tune a small model with LoRA, and judge when fine-tuning is worth it

  9. 09

    Build evaluation datasets and automated, model-graded and human review pipelines

  10. 10

    Implement guardrails against prompt injection, data leakage and unsafe output

  11. 11

    Deploy AI features behind streaming, cached, rate-limited production APIs

  12. 12

    Instrument an AI system with tracing, cost tracking and quality monitoring

Curriculum

Module by module.

  1. Module 01How LLMs Work
    • Tokens, embeddings and the transformer, intuitively
    • Pre-training, instruction tuning and preference alignment
    • Sampling: temperature, top-p and determinism
    • Capabilities, limits and characteristic failure modes
    • Why models hallucinate, and what actually reduces it
    • Context windows, cost and latency as engineering constraints
  2. Module 02Working with Models & APIs
    • Hosted APIs versus open-weight models you run yourself
    • Running local models with Ollama and vLLM
    • Streaming, batching and concurrency
    • Token accounting and building a cost model before you build the feature
    • Handling rate limits, timeouts and provider outages
    • Abstracting the provider so you are not locked in
  3. Module 03Prompt & Output Design
    • System prompts, instructions, examples and role design
    • Chain-of-thought, decomposition and self-consistency
    • Structured outputs: JSON mode, schemas and validation with Pydantic
    • Prompt versioning, regression testing and prompt registries
    • Handling ambiguity, refusals and partial answers
    • Context engineering: what to put in the window, and what to leave out
  4. Module 04Embeddings & Vector Search
    • Semantic similarity and how embeddings encode meaning
    • Choosing and benchmarking an embedding model
    • Chunking strategies, and why naive splitting fails
    • Vector databases: indexes, filters and metadata
    • Hybrid search: combining vectors with BM25 keyword search
    • Measuring retrieval with recall@k, MRR and nDCG
  5. Module 05Retrieval-Augmented Generation
    • End-to-end RAG pipeline design
    • Ingestion: parsing, cleaning and enriching source documents
    • Grounding, citations and answer attribution
    • Freshness, incremental indexing and deletion
    • Diagnosing RAG failures: retrieval, ranking or generation
    • Access control in retrieval — never returning what a user may not see
  6. Module 06Advanced Retrieval
    • Re-ranking with cross-encoders
    • Query rewriting, expansion and multi-query retrieval
    • Parent-document, hierarchical and contextual chunking
    • Knowledge graphs and structured retrieval
    • Routing between multiple sources and tools
    • Caching retrieval and generation, safely
  7. Module 07Tools & Agents
    • Function calling and tool schemas
    • Planning, multi-step workflows and loop termination
    • State, memory and conversation management
    • Multi-agent patterns, and their real costs
    • Sandboxing, permissions and human-in-the-loop approval
    • Failure recovery and retries
    • When an agent is the wrong answer and a workflow is the right one
  8. Module 08Document AI & Structured Extraction
    • Parsing PDFs, scans, tables and forms
    • OCR and layout-aware extraction
    • Multimodal models for images and documents
    • Schema-driven extraction with validation and repair
    • Confidence scoring and routing low-confidence cases to a human
    • Building an extraction pipeline that survives messy inputs
  9. Module 09Fine-Tuning & Adaptation
    • Prompting, RAG or fine-tuning: choosing correctly
    • Building and cleaning a training dataset
    • Parameter-efficient fine-tuning with LoRA and QLoRA
    • Running a fine-tune and reading the training curves
    • Distillation and small models for narrow tasks
    • Measuring whether the fine-tune actually helped
  10. Module 10Evaluation
    • Building an evaluation dataset from real usage
    • Deterministic checks, reference-based metrics and rubrics
    • LLM-as-judge: designing it, and its biases
    • Human review workflows and annotation quality
    • Regression testing AI features in CI
    • A/B testing and online evaluation
    • Reporting quality honestly to non-technical stakeholders
  11. Module 11Safety, Privacy & Guardrails
    • Prompt injection and indirect injection through retrieved content
    • Data leakage, PII handling and redaction
    • Input and output filtering, and moderation
    • Jailbreak testing and adversarial red-teaming
    • Tenancy, isolation and audit trails
    • Responsible AI: transparency, consent and documenting limitations
  12. Module 12Production & Deployment
    • Serving AI features with FastAPI and async Python
    • Streaming responses and perceived latency
    • Semantic caching, prompt caching and cost reduction
    • Rate limits, quotas, queues and graceful degradation
    • Tracing and observability with OpenTelemetry and LLM tracing tools
    • Monitoring quality, cost and drift in production
    • Containerizing and deploying the service with Docker
  13. Module 13Capstone & Career
    • Scoping an AI feature for a real problem, with success criteria up front
    • Build, evaluate and iterate against your own test suite
    • Deployment, documentation and a cost and latency budget
    • Presenting trade-offs, limitations and results to a non-technical audience
    • Portfolio and public writing for AI engineering roles
    • Keeping up: how to read model releases and papers without drowning

Projects

What you'll build.

  1. 01

    A document question-answering assistant built on RAG, with citations

  2. 02

    A structured extraction service that turns messy documents into validated JSON

  3. 03

    An automated evaluation suite that catches regressions in an LLM feature

  4. 04

    A capstone AI application deployed behind a FastAPI service, with tracing and a cost budget

We build this too

Machine learning models, LLM-powered applications and intelligent automation — designed around your data, evaluated rigorously and deployed into real products.

Keep learning

  • AI / Machine Learning

    Upcoming

    Python for AI & Machine Learning

    Learn Python, data analysis, statistics and core machine learning with real datasets — then take a model out of the notebook and serve it as an API.

    Level
    Beginner
    Duration
    14 Weeks
    • Python
    • NumPy
    • pandas
    • Matplotlib
    • scikit-learn
  • App Development

    Upcoming

    Modern App Development with Flutter

    Build and ship cross-platform mobile apps with Flutter and Dart — from your first widget to a tested, release-ready app on both stores.

    Level
    Beginner to Intermediate
    Duration
    14 Weeks
    • Flutter
    • Dart
    • BLoC
    • Firebase
    • Riverpod

FAQ

Course questions.

Who are the courses for?

Most courses start from the fundamentals and suit motivated beginners. More advanced courses list their prerequisites clearly on the course page, so you can choose the right starting point.

How are the courses taught?

Every course is project-based: you learn a concept, then apply it by building something real. Schedules, delivery format and fees are announced with each cohort — register your interest and we will share the details once they are confirmed.

Do I need prior experience?

Not for beginner-level courses. Each course page lists its level and prerequisites. If you are unsure which course fits, contact us and we will help you decide.

Will I build real projects?

Yes — projects are the core of every course. Each course page lists the projects you will build, ending with a capstone that brings everything together.

Do you offer training for teams?

If you are interested in training for a team or organization, get in touch and choose “Training”. Tell us about your team, its current skills and what you want to achieve.

Not sure where to start?

Let's find the right course.

Tell us about your background and goals, and we'll help you choose a starting point.