Richard J. Rossi, PhD, is Director of the Statistics Program and Co-Director of the Data Science Program at Montana Tech of The University of Montana, in Butte, MT. He acted as President of the Montana Chapter of the American Statistical Association in 2001 and as Associate Editor for Biometrics from 1997-2000. Dr. Rossi is a member of the American Mathematical Society, the Institute of Mathematical Statistics, and the American Statistical Association.

Preface xiii

Acknowledgments xvii

1 Probability 1

1.1 Sample Spaces, Events, and ;;-Algebras 1

Problems 7

1.2 Probability Axioms and Rules 9

Problems 14

1.3 Probability with Equally Likely Outcomes 16

Problems 18

1.4 Conditional Probability 19

Problems 25

1.5 Independence 28

Problems 31

1.6 Counting Methods 33

Problems 38

1.7 Case Study -The Birthday Problem 41

Problems 44

2 Random Variables and Random Vectors 45

2.1 Random Variables 45

2.1.1 Properties of Random Variables 46

Problems 50

2.2 Random Vectors 53

2.2.1 Properties of Random Vectors 53

Problems 60

2.3 Independent Random Variables 63

Problems 66

2.4 Transformations of Random Variables 68

2.4.1 Transformations of Discrete Random Variables 68

2.4.2 Transformations of Continuous Random Variables 69

2.4.3 Transformations of Continuous Bivariate Random Vectors 73

Problems 75

2.5 Expected Values for Random Variables 77

2.5.1 Expected Values and Moments of Random Variables 77

2.5.2 The Variance of a Random Variable 81

2.5.3 Moment Generating Functions 86

Problems 89

2.6 Expected Values for Random Vectors 94

2.6.1 Properties of Expectation with Random Vectors 96

2.6.2 Covariance and Correlation 99

2.6.3 Conditional Expectation and Variance 106

Problems 110

2.7 Sums of Random Variables 114

Problems 120

2.8 Case Study - HowMany Times Was the Coin Tossed? 123

2.8.1 The Probability Model 124

Problems 126

3 Probability Models 129

3.1 Discrete Probability Models 129

3.1.1 The Binomial Model 129

3.1.1.1 Binomial Setting 130

3.1.2 The HypergeometricModel 132

3.1.2.1 Hypergeometric Setting 132

3.1.3 The Poisson Model 134

3.1.4 The Negative BinomialModel 135

3.1.4.1 Negative Binomial Setting 135

3.1.5 The MultinomialModel 138

3.1.5.1 Multinomial Setting 139

Problems 140

3.2 Continuous Probability Models 147

3.2.1 The Uniform Model 147

3.2.2 The Gamma Model 149

3.2.3 The Normal Model 152

3.2.4 The Log-normal Model 155

3.2.5 The Beta Model 156

Problems 158

3.3 Important Distributional Relationships 163

3.3.1 Sums of Random Variables 163

3.3.2 The T and F Distributions 166

Problems 170

3.4 Case Study -The Central LimitTheorem 172

3.4.1 Convergence in Distribution 172

3.4.2 The Central LimitTheorem 173

Problems 176

4 Parametric Point Estimation 177

4.1 Statistics 177

4.1.1 Sampling Distributions 178

4.1.2 Unbiased Statistics and Estimators 179

4.1.3 Standard Error and Mean Squared Error 181

4.1.4 The Delta Method 186

Problems 186

4.2 Sufficient Statistics 190

4.2.1 Exponential Family Distributions 195

Problems 200

4.3 Minimum Variance Unbiased Estimators 203

4.3.1 Cramér-Rao Lower Bound 205

Problems 212

4.4 Case Study -The Order Statistics 214

Problems 219

5 Likelihood-based Estimation 223

5.1 Maximum Likelihood Estimation 226

5.1.1 Properties of MLEs 226

5.1.2 One-parameter Probability Models 228

5.1.3 Multiparameter Probability Models 235

Problems 240

5.2 Bayesian Estimation 247

5.2.1 The Bayesian Setting 247

5.2.2 Bayesian Estimators 250

Problems 255

5.3 Interval Estimation 258

5.3.1 Exact Confidence Intervals 259

5.3.2 Large Sample Confidence Intervals 264

5.3.3 Bayesian Credible Intervals 267

Problems 269

5.4 Case Study - Modeling Obsidian Rind Thicknesses 273

5.4.1 Finite Mixture Model 274

Problems 278

6 Hypothesis Testing 281

6.1 Components of a Hypothesis Test 282

Problems 286

6.2 Most Powerful Tests 288

Problems 293

6.3 Uniformly Most Powerful Tests 296

6.3.1 Uniformly Most Powerful Unbiased Tests 299

Problems 301

6.4 Generalized Likelihood Ratio Tests 305

Problems 311

6.5 Large Sample Tests 314

6.5.1 Large Sample Tests Based on the MLE 314

6.5.2 Score Tests 316

Problems 320

6.6 Case Study - Modeling Survival of the Titanic Passengers 323

6.6.1 Exploring the Data 324

6.6.2 Modeling the Probability of Survival 325

6.6.3 Analysis of the Fitted Survival Model 327

Problems 328

7 Generalized Linear Models 331

7.1 Generalized LinearModels 332

Problems 334

7.2 Fitting a Generalized LinearModel 336

7.2.1 Estimating ¿ ;; 336

7.2.2 Model Deviance 338

Problems 340

7.3 Hypothesis Testing in a Generalized Linear Model 341

7.3.1 Asymptotic Properties 341

7.3.2 Wald Tests and Confidence Intervals 342

7.3.3 Likelihood Ratio Tests 343

Problems 346

7.4 Generalized LinearModels for a Normal Response Variable 348

7.4.1 Estimation 349

7.4.2 Properties of the MLEs 353

7.4.3 Deviance 357

7.4.4 Hypothesis Testing 359

Problems 362

7.5 Generalized LinearModels for a Binomial Response Variable 365

7.5.1 Estimation 366

7.5.2 Properties of the MLEs 368

7.5.3 Deviance 370

7.5.4 Hypothesis Testing 371

Problems 373

7.6 Case Study - IDNAP Experimentwith Poisson Count Data 375

7.6.1 The Model 376

7.6.2 StatisticalMethods 376

7.6.3 Results of the First Experiment 379

Problems 381

References 383

A Probability Models 385

B DataSets 387

Problem Solutions 389

Index 413