Series Editor Foreword xi
Preface xiii
Acknowledgments xvii
About the Author xix

Part I: Getting Started with Foundations of AI, LLMs, and Experimentation 1

Chapter 1: An Introduction to AI, LLMs, and Agents 3

Introduction 3


The Basics of Large Language Models 3


The Family Tree of LLM Tasks 10


Alignment 10


Prompt Engineering 12


Special LLM Features 17


LLM Workflows 25


AI Agents 25


Conclusion 28

Chapter 2: First Steps with LLM Workflows 31

Introduction 31


Case Study 1: Text-to-SQL Workflow 32


Conclusion 57

Chapter 3: AI Evaluation Plus Experimentation 59

Introduction 59


Evaluating and Experimenting with LLMs 59


Case Study 1, Revisited: The Text-to-SQL Workflow 61


Case Study 2: A "Simple" Summary Prompt 77


Conclusion 83

Part II: Moving the Needle with AI Agents, Workflows, and Multimodality 85

Chapter 4: First Steps with AI Agents and Multi-Agent Workloads 87

Introduction 87


Case Study 3: From RAG to Agents 88


When Should You Use Workflows Versus Agents? 104


Case Study 4: A (Nearly) End-to-End SDR 105


Evaluating Agents 118


Conclusion 121

Chapter 5: Enhancing Agents with Prompting, Workflows, and More Agents 123

Introduction 123


Case Study 5: Agents Complying with Policies Plus Synthetic Data Generation 124


Building Our Policy Bot Agent 127


Case Study 6: Deep Research Plus Content Generation Agentic Workflows 133


Multi-Agent Architectures 141


Case Study 4, Revisited: Adding a Supervisor Agent to Our SDR Team 148


Case Study 7: Agentic Tool Selection Performance 149


Conclusion 157

Chapter 6: Moving Beyond Natural Language: Multimodal and Coding AI 159

Introduction 159


Introduction to Multimodal AI 159


Case Study 8: Image Retrieval Pipelines 168


Case Study 9: Visual Q/A with Moondream 174


Case Study 10: Coding Agent with Image Generation, File Use, and Moondream 176


The Case for Any-to-Any Models 188


Conclusion 191

Part III: Optimizing Workloads with Fine-Tuning, Frameworks, and Reasoning LLMs 193

Chapter 7: Reasoning LLMs and Computer Use 195

Introduction 195


Seven Pillars of Intelligence 195


Case Study 11: Benchmarking Reasoning Models 198


Reasoning Models for ReAct Agents 210


Case Study 12: Computer Use 212


Conclusion 224

Chapter 8: Fine-Tuning AI for Calibrated Performance 225

Introduction 225


Case Study 13: Classification Versus Multiple Choice 227


Case Study 14: Domain Adaptation 245


Conclusion 258

Chapter 9: Optimizing AI Models for Production 261

Introduction 261


Model Compression 261


Case Study 15: Speculative Decoding with Qwen 269


Case Study 16: Voice Bot--Need for Speed 272


Case Study 17: Fine-Tuning Matryoshka Embeddings 277


Case Study
N
+ 1: What Comes Next? 284

Index 287