DANIEL DUFFY, PhD, has BA (Mod), MSc and PhD degrees in pure, applied and numerical mathematics (University of Dublin, Trinity College) and he is active in promoting partial differential equations (PDE) and the Finite Difference Method (FDM) for applications in computational finance. He was responsible for the introduction of the Fractional Step (Soviet Splitting) method and the Alternating Direction Explicit (ADE) method in computational finance. He is the originator of the exponential fitting method for convection-dominated PDEs.

Preface xix

Who Should Read this Book? xxiii

Part A : Mathematical Foundation for One-Factor Problems

Chapter 1 : Real Analysis Foundations for this Book 3

1.1 Introduction and Objectives 3

1.2 Continuous Functions 4

1.2.1 Formal Definition of Continuity 5

1.2.2 An Example 6

1.2.3 Uniform Continuity 6

1.2.4 Classes of Discontinuous Functions 7

1.3 Differential Calculus 8

1.3.1 Taylor's Theorem 9

1.3.2 Big O and Little o Notation 10

1.4 Partial Derivatives 11

1.5 Functions and Implicit Forms 13

1.6 Metric Spaces and Cauchy Sequences 14

1.6.1 Metric Spaces 15

1.6.2 Cauchy Sequences 16

1.6.3 Lipschitz Continuous Functions 17

1.7 Summary and Conclusions 19

Chapter 2 : Ordinary Differential Equations (ODEs), Part 1 21

2.1 Introduction and Objectives 21

2.2 Background and Problem Statement 22

2.2.1 Qualitative Properties of the Solution and Maximum Principle 22

2.2.2 Rationale and Generalisations 24

2.3 Discretisation of Initial Value Problems: Fundamentals 25

2.3.1 Common Schemes 26

2.3.2 Discrete Maximum Principle 28

2.4 Special Schemes 29

2.4.1 Exponential Fitting 29

2.4.2 Scalar Non-Linear Problems and Predictor-Corrector Method 31

2.4.3 Extrapolation 31

2.5 Foundations of Discrete Time Approximations 32

2.6 Stiff ODEs 37

2.7 Intermezzo: Explicit Solutions 39

2.8 Summary and Conclusions 41

Chapter 3 : Ordinary Differential Equations (ODEs), Part 2 43

3.1 Introduction and Objectives 43

3.2 Existence and Uniqueness Results 43

3.2.1 An Example 45

3.3 Other Model Examples 45

3.3.1 Bernoulli ODE 45

3.3.2 Riccati ODE 46

3.3.3 Predator-Prey Models 47

3.3.4 Logistic Function 48

3.4 Existence Theorems for Stochastic Differential Equations (SDEs) 48

3.4.1 Stochastic Differential Equations (SDEs) 49

3.5 Numerical Methods for ODEs 51

3.5.1 Code Samples in Python 52

3.6 The Riccati Equation 55

3.6.1 Finite Difference Schemes 57

3.7 Matrix Differential Equations 59

3.7.1 Transition Rate Matrices and Continuous Time Markov Chains 61

3.8 Summary and Conclusions 62

Chapter 4 : An Introduction to Finite Dimensional Vector Spaces 63

4.1 Short Introduction and Objectives 63

4.1.1 Notation 64

4.2 What Is a Vector Space? 65

4.3 Subspaces 67

4.4 Linear Independence and Bases 68

4.5 Linear Transformations 69

4.5.1 Invariant Subspaces 70

4.5.2 Rank and Nullity 71

4.6 Summary and Conclusions 72

Chapter 5 : Guide to Matrix Theory and Numerical Linear Algebra 73

5.1 Introduction and Objectives 73

5.2 From Vector Spaces to Matrices 73

5.2.1 Sums and Scalar Products of Linear Transformations 73

5.3 Inner Product Spaces 74

5.3.1 Orthonormal Basis 75

5.4 From Vector Spaces to Matrices 76

5.4.1 Some Examples 76

5.5 Fundamental Matrix Properties 77

5.6 Essential Matrix Types 80

5.6.1 Nilpotent and Related Matrices 80

5.6.2 Normal Matrices 81

5.6.3 Unitary and Orthogonal Matrices 82

5.6.4 Positive Definite Matrices 82

5.6.5 Non-Negative Matrices 83

5.6.6 Irreducible Matrices 83

5.6.7 Other Kinds of Matrices 84

5.7 The Cayley Transform 84

5.8 Summary and Conclusions 86

Chapter 6 : Numerical Solutions of Boundary Value Problems 87

6.1 Introduction and Objectives 87

6.2 An Introduction to Numerical Linear Algebra 87

6.2.1 BLAS (Basic Linear Algebra Subprograms) 90

6.3 Direct Methods for Linear Systems 92

6.3.1 LU Decomposition 92

6.3.2 Cholesky Decomposition 94

6.4 Solving Tridiagonal Systems 94

6.4.1 Double Sweep Method 94

6.4.2 Thomas Algorithm 96

6.4.3 Block Tridiagonal Systems 97

6.5 Two-Point Boundary Value Problems 99

6.5.1 Finite Difference Approximation 100

6.5.2 Approximation of Boundary Conditions 102

6.6 Iterative Matrix Solvers 103

6.6.1 Iterative Methods 103

6.6.2 Jacobi Method 104

6.6.3 Gauss-Seidel Method 104

6.6.4 Successive Over-Relaxation (SOR) 105

6.6.5 Other Methods 105

6.7 Example: Iterative Solvers for Elliptic PDEs 106

6.8 Summary and Conclusions 107

Chapter 7 : Black-Scholes Finite Differences for the Impatient 109

7.1 Introduction and Objectives 109

7.2 The Black-Scholes Equation: Fully Implicit and Crank-Nicolson Methods 110

7.2.1 Fully Implicit Method 110

7.2.2 Crank-Nicolson Method 111

7.2.3 Final Remarks 114

7.3 The Black-Scholes Equation: Trinomial Method 115

7.3.1 Comparison with Other Methods 115

7.4 The Heat Equation and Alternating Direction Explicit (ADE) Method 120

7.4.1 Background and Motivation 120

7.5 ADE for Black-Scholes: Some Test Results 121

7.6 Summary and Conclusions 126

Part B : Mathematical Foundation for Two-Factor Problems

Chapter 8 : Classifying and Transforming Partial Differential Equations 129

8.1 Introduction and Objectives 129

8.2 Background and Problem Statement 129

8.3 Introduction to Elliptic Equations 130

8.3.1 What is an Elliptic Operator? 130

8.3.2 Total and Principal Symbols 131

8.3.3 The Adjoint Equation 132

8.3.4 Self-Adjoint Operators and Equations 133

8.3.5 Numerical Approximation of PDEs in Adjoint Form 134

8.3.6 Elliptic Equations with Non-Negative Characteristic Form 135

8.4 Classification of Second-Order Equations 135

8.4.1 Characteristics 136

8.4.2 Model Example 137

8.4.3 Test your Knowledge 138

8.5 Examples of Two-Factor Models from Computational Finance 139

8.5.1 Multi-Asset Options 139

8.5.2 Stochastic Dividend PDE 140

8.6 Summary and Conclusions 141

Chapter 9 : Transforming Partial Differential Equations to a Bounded Domain 143

9.1 Introduction and Objectives 143

9.2 The Domain in Which a PDE Is Defined: Preamble 143

9.2.1 Background and Specific Mappings 144

9.2.2 Initial Examples 146

9.3 Other Examples 147

9.4 Hotspots 148

9.5 What Happened to Domain Truncation? 148

9.6 Another Way to Remove Mixed Derivative Terms 149

9.7 Summary and Conclusions 151

Chapter 10 : Boundary Value Problems for Elliptic and Parabolic Partial Differential Equations 153

10.1 Introduction and Objectives 153

10.2 Notation and Prerequisites 154

10.3 The Laplace Equation 154

10.3.1 Harmonic Functions and the Cauchy-Riemann Equations 154

10.4 Properties of The Laplace Equation 156

10.4.1 Maximum-Minimum Principle for Laplace's Equation 158

10.5 Some Elliptic Boundary Value Problems 159

10.5.1 Some Motivating Examples 159

10.6 Extended Maximum-Minimum Principles 159

10.6.1 An Example 161

10.7 Summary and Conclusions 162

Chapter 11 : Fichera Theory, Energy Inequalities and Integral Relations 163

11.1 Introduction and Objectives 163

11.2 Background and Problem Statement 163

11.2.1 The 'Big Bang': Cauchy-Euler Equation 163

11.3 Well-Posed Problems and Energy Estimates 165

11.3.1 Time to Reflect: What Have We Achieved and What's Next? 167

11.4 The Fichera Theory: Overview 168

11.5 The Fichera Theory: The Core Business 168

11.6 The Fichera Theory: Further Examples and Applications 171

11.6.1 Cox-Ingersoll-Ross (CIR) 171

11.6.2 Heston Model Fundamenals 172

11.6.3 Heston Model by Fichera Theory 176

11.6.4 First-Order Hyperbolic PDE in One and Two Space Variables 177

11.7 Some Useful Theorems 178

11.7.1 Divergence (Gauss-Ostrogradsky) Theorem 179

11.7.2 Green's Theorem/Formula 180

11.7.3 Green's First and Second Identities 180

11.8 Summary and Conclusions 180

Chapter 12 : An Introduction to Time-Dependent Partial Differential Equations 181

12.1 Introduction and Objectives 181

12.2 Notation and Prerequisites 181

12.3 Preamble: Separation of Variables for the Heat Equation 182

12.4 Well-Posed Problems 184

12.4.1 Examples of an ill-posed Problem 185

12.4.2 The Importance of Proving that Problems Are Well-Posed 187

12.5 Variations on Initial Boundary Value Problem for the Heat Equation 188

12.5.1 Smoothness and Compatibility Conditions 188

12.6 Maximum-Minimum Principles for Parabolic PDEs 189

12.7 Parabolic Equations with Time-Dependent Boundaries 190

12.8 Uniqueness Theorems for Boundary Value Problems in Two Dimensions 192

12.8.1 Laplace Equation 192

12.8.2 Heat Equation 193

12.9 Summary and Conclusions 193

Chapter 13 : Stochastics Representations of PDEs and Applications 195

13.1 Introduction and Objectives 195

13.2 Background, Requirements and Problem Statement 196

13.3 An Overview of Stochastic Differential Equations (SDEs) 196

13.4 An Introduction to One-Dimensional Random Processes 196

13.5 An Introduction to the Numerical Approximation of SDEs 199

13.5.1 Euler-Maruyama Method 199

13.5.2 Milstein Method 201

13.5.3 Predictor-Corrector Method 201

13.5.4 Drift-Adjusted Predictor-Corrector Method 202

13.6 Path Evolution and Monte Carlo Option Pricing 203

13.6.1 Monte Carlo Option Pricing 204

13.6.2 Some C++ Code 205

13.7 Two-Factor Problems 209

13.7.1 Spread Options with Stochastic Volatility 209

13.7.2 Heston Stochastic Volatility Model 211

13.8 The Ito Formula 215

13.9 Stochastics Meets PDEs 215

13.9.1 A Statistics Refresher 215

13.9.2 The Feynman-Kac Formula 217

13.9.3 Kolmogorov Equations 218

13.9.4 Kolmogorov Forward (Fokker-Planck (FPE)) Equation 218

13.9.5 Multi-Dimensional Problems and Boundary Conditions 219

13.9.6 Kolmogorov Backward Equation (KBE) 220

13.10 First Exit-Time Problems 221

13.11 Summary and Conclusions 222

Part C : The Foundations of the Finite Difference Method (FDM)

Chapter 14 : Mathematical and Numerical Foundations of the Finite Difference Method, Part I 225

14.1 Introduction and Objectives 225

14.2 Notation and Prerequisites 226

14.3 What Is the Finite Difference Method, Really? 227

14.4 Fourier Analysis of Linear PDEs 227

14.4.1 Fourier Transform for Advection Equation 229

14.4.2 Fourier Transform for Diffusion Equation 230

14.5 Discrete Fourier Transform 232

14.5.1 Finite and Infinite Dimensional Sequences and Their Norms 232

14.5.2 Discrete Fourier Transform (DFT) 233

14.5.3 Discrete von Neumann Stability Criterion 235

14.5.4 Some More Examples 235

14.6 Theoretical Considerations 237

14.6.1 Consistency 237

14.6.2 Stability 238

14.6.3 Convergence 239

14.7 First-Order Partial Differential Equations 239

14.7.1 Why First-Order Equations are...