Series Editor Foreword xi
Preface xiii
Acknowledgments xvii
About the Author xix

Part I: Getting Started with Foundations of AI, LLMs, and Experimentation 1

Chapter 1: An Introduction to AI, LLMs, and Agents 3

Introduction 3

The Basics of Large Language Models 3

The Family Tree of LLM Tasks 10

Alignment 10

Prompt Engineering 12

Special LLM Features 17

LLM Workflows 25

AI Agents 25

Conclusion 28

Chapter 2: First Steps with LLM Workflows 31

Introduction 31

Case Study 1: Text-to-SQL Workflow 32

Conclusion 57

Chapter 3: AI Evaluation Plus Experimentation 59

Introduction 59

Evaluating and Experimenting with LLMs 59

Case Study 1, Revisited: The Text-to-SQL Workflow 61

Case Study 2: A "Simple" Summary Prompt 77

Conclusion 83

Part II: Moving the Needle with AI Agents, Workflows, and Multimodality 85

Chapter 4: First Steps with AI Agents and Multi-Agent Workloads 87

Introduction 87

Case Study 3: From RAG to Agents 88

When Should You Use Workflows Versus Agents? 104

Case Study 4: A (Nearly) End-to-End SDR 105

Evaluating Agents 118

Conclusion 121

Chapter 5: Enhancing Agents with Prompting, Workflows, and More Agents 123

Introduction 123

Case Study 5: Agents Complying with Policies Plus Synthetic Data Generation 124

Building Our Policy Bot Agent 127

Case Study 6: Deep Research Plus Content Generation Agentic Workflows 133

Multi-Agent Architectures 141

Case Study 4, Revisited: Adding a Supervisor Agent to Our SDR Team 148

Case Study 7: Agentic Tool Selection Performance 149

Conclusion 157

Chapter 6: Moving Beyond Natural Language: Multimodal and Coding AI 159

Introduction 159

Introduction to Multimodal AI 159

Case Study 8: Image Retrieval Pipelines 168

Case Study 9: Visual Q/A with Moondream 174

Case Study 10: Coding Agent with Image Generation, File Use, and Moondream 176

The Case for Any-to-Any Models 188

Conclusion 191

Part III: Optimizing Workloads with Fine-Tuning, Frameworks, and Reasoning LLMs 193

Chapter 7: Reasoning LLMs and Computer Use 195

Introduction 195

Seven Pillars of Intelligence 195

Case Study 11: Benchmarking Reasoning Models 198

Reasoning Models for ReAct Agents 210

Case Study 12: Computer Use 212

Conclusion 224

Chapter 8: Fine-Tuning AI for Calibrated Performance 225

Introduction 225

Case Study 13: Classification Versus Multiple Choice 227

Case Study 14: Domain Adaptation 245

Conclusion 258

Chapter 9: Optimizing AI Models for Production 261

Introduction 261

Model Compression 261

Case Study 15: Speculative Decoding with Qwen 269

Case Study 16: Voice Bot--Need for Speed 272

Case Study 17: Fine-Tuning Matryoshka Embeddings 277

Case Study N + 1: What Comes Next? 284

Index 287