Dr. Austen Rainer is a Senior Lecturer in the School of Computer Science at the University of Hertfordshire. He is an active member of the Centre for Empirical Software Process Research (CESPR) and teaches several courses?in the BSc(Hons) and MSc degree programs.

Dr. Per Runeson is leader of the Software Engineering Research Group at Lund University, as well as Research Director for EASE - Embedded Applications Software Engineering. He is a Senior Researcher at Sony Ericsson, where he fulfills his role as a verification advisor. His research focuses on efficient verification and validation processes.

Dr. Martin Höst is Director of the [...]. Programme in Computer Science and Engineering at Lund University and a member of the Software Engineering Research Group (SERG).

Dr. Björn Regnell is a professor of Software Engineering at Lund University's Department of Computer Science and Vice Dean of Research at the Faculty of Engineering, LTH. His research interests include market-driven software development, requirements engineering, software quality, software innovation, software product management, and empirical research methods.

PREFACE xv

ACKNOWLEDGMENTS xvii

PART I CASE STUDY METHODOLOGY

1 INTRODUCTION 3

1.1 What is a Case Study? 3

1.2 A Brief History of Case Studies in Software Engineering 5

1.3 Why a Book on Case Studies of Software Engineering? 6

1.4 Conclusion 9

2 BACKGROUND AND DEFINITION OF CONCEPTS 11

2.1 Introduction 11

2.2 Research Strategies 11

2.3 Characteristics of Research Strategies 13

2.3.1 Purpose 13

2.3.2 Control and Data 14

2.3.3 Triangulation 15

2.3.4 Replication 16

2.3.5 Inductive and Deductive Enquiries 16

2.4 What Makes a Good Case Study? 17

2.5 When is the Case Study Strategy Feasible? 19

2.6 Case Study Research Process 20

2.7 Conclusion 21

3 DESIGN OF THE CASE STUDY 23

3.1 Introduction 23

3.2 Elements of the Case Study Design 24

3.2.1 Rationale for the Study 24

3.2.2 Objective of the Study 24

3.2.3 Cases and Units of Analyses 26

3.2.4 Theoretical Framework 29

3.2.5 Research Questions 30

3.2.6 Propositions and Hypotheses 31

3.2.7 Concepts 32

3.2.8 Methods of Data Collection 32

3.2.9 Methods of Data Analysis 33

3.2.10 Case Selection 33

3.2.11 Selection of Data 35

3.2.12 Data Definition and Data Storage 36

3.2.13 Quality Control and Assurance 36

3.2.14 Maintaining the Case Study Protocol 37

3.2.15 Reporting and Disseminating the Case Study 38

3.3 Legal, Ethical, and Professional Issues 40

3.4 Conclusion 45

4 DATA COLLECTION 47

4.1 Introduction 47

4.2 Different Types of Data Source 47

4.2.1 Classification of Data Sources 47

4.2.2 Data Source Selection 49

4.3 Interviews 50

4.3.1 Planning Interviews 50

4.3.2 The Interview Session 52

4.3.3 Postinterview Activities 53

4.4 Focus groups 54

4.5 Observations 56

4.6 Archival Data 57

4.7 Metrics 58

4.8 Conclusion 60

5 DATA ANALYSIS AND INTERPRETATION 61

5.1 Introduction 61

5.2 Analysis of Data in Flexible Research 62

5.2.1 Introduction 62

5.2.2 Level of Formalism 64

5.2.3 Relation to Hypotheses 65

5.3 Process for Qualitative Data Analysis 65

5.3.1 Introduction 65

5.3.2 Steps in the Analysis 66

5.3.3 Techniques 68

5.3.4 Tool support 70

5.4 Validity 71

5.4.1 Construct Validity 71

5.4.2 Internal Validity 71

5.4.3 External Validity 71

5.4.4 Reliability 72

5.5 Improving Validity 72

5.6 Quantitative Data Analysis 74

5.7 Conclusion 76

6 REPORTING AND DISSEMINATION 77

6.1 Introduction 77

6.2 Why Report and Disseminate 78

6.3 The Audience for the Report 79

6.4 Aspects of the Case Study to Report and Disseminate 80

6.5 When to Report and Disseminate 81

6.6 Guidelines on Reporting 82

6.6.1 The Generic Content of an Academic Report 82

6.6.2 Reporting Recommendations from Evaluative Case Studies 84

6.6.3 Reporting to Stakeholders, Including Sponsor(s) 85

6.6.4 Reporting the Context of the Case Study 87

6.6.5 Reporting to Students 89

6.6.6 Ad Hoc and Impromptu Reporting 90

6.7 Formats and Structures for a Report 91

6.8 Where to Report 94

6.9 Ethics and Confidentiality 94

6.10 Conclusion 95

7 SCALING UP CASE STUDY RESEARCH TO REAL-WORLD SOFTWARE PRACTICE 97

7.1 Introduction 97

7.2 The Aims of Scaling up Case Studies 98

7.3 Dimensions of Scale 99

7.4 Longitudinal Case Studies 100

7.5 Multiple Case Studies 102

7.5.1 Multiple Cases and Replications 102

7.5.2 Selecting the Cases 104

7.6 Multiresearcher Case Studies 105

7.7 Conclusion 107

8 USING CASE STUDY RESEARCH 109

8.1 Introduction 109

8.2 Reading and Reviewing Case Studies 109

8.2.1 Development of Checklists 110

8.2.2 Checklists for Conducting Case Study Research 111

8.2.3 Checklists for Reading and Reviewing Case Studies 111

8.2.4 Development of Practice 111

8.3 Identifying and Synthesizing Use Case Research 111

8.3.1 Identifying Primary Studies 112

8.3.2 Synthesis of Evidence from Multiple Case Studies 113

8.3.3 Current State of Synthesis 117

8.4 The Economics of Case Study Research 118

8.4.1 Costs and Benefits of Evaluation Techniques 119

8.4.2 Evaluation of the DESMET Methodology 119

8.4.3 Frameworks for Organizing Methods of Evaluation 119

8.5 Specializing Case Study Research for Software Engineering 121

8.5.1 The Longitudinal Chronological Case Study Research Strategy 122

8.5.2 Controlled Case Studies 123

8.6 Case Studies and Software Process Improvement 123

8.7 Conclusion 125

PART II EXAMPLES OF CASE STUDIES

9 INTRODUCTION TO CASE STUDY EXAMPLES 129

9.1 Introduction 129

10 CASE STUDY OF EXTREME PROGRAMMING IN A STAGE-GATE CONTEXT 133

10.1 Introduction 133

10.1.1 Methodological Status 133

10.2 Case Study Design 134

10.2.1 Rationale 134

10.2.2 Objectives 134

10.2.3 Cases and Units of Analysis 135

10.2.4 Theoretical Frame of Reference 136

10.2.5 Research Questions 136

10.3 Planning 136

10.3.1 Methods of Data Collection 136

10.3.2 Selection of Data 137

10.3.3 Case Selection Strategy 137

10.3.4 Case Study Protocol 137

10.3.5 Ethical Considerations 137

10.4 Data Collection 139

10.5 Data Analysis 139

10.5.1 Threats to Validity 144

10.6 Reporting 144

10.6.1 Academics 144

10.6.2 Practitioners 144

10.7 Lessons Learned 146

11 TWO LONGITUDINAL CASE STUDIES OF SOFTWARE PROJECT MANAGEMENT 149

11.1 Introduction 149

11.2 Background to the Research Project 149

11.3 Case Study Design and Planning 150

11.3.1 Rationale 150

11.3.2 Objective 150

11.3.3 Definition of the Case 150

11.3.4 Units of Analyses 151

11.3.5 Theoretical Frame of Reference and Research Questions 151

11.3.6 Case Selection 151

11.3.7 Replication Strategy 152

11.3.8 Case Study Protocol 152

11.3.9 Quality Assurance, Validity, and Reliability 152

11.3.10 Legal, Ethical, and Professional Considerations 153

11.4 Data Collection 154

11.4.1 Sources of Data 154

11.5 Data Analysis 157

11.6 Reporting 159

11.6.1 Internal Reporting of Results 160

11.6.2 Dissemination of Artifacts 160

11.7 Lessons Learned 160

12 AN ITERATIVE CASE STUDY OF QUALITY MONITORING 163

12.1 Introduction 163

12.2 Case Study Design 164

12.2.1 Objectives 164

12.2.2 Cases and Units of Analysis 165

12.2.3 Theoretical Frame of Reference 165

12.2.4 Research Questions 165

12.3 Planning 165

12.3.1 Methods of Data Collection 165

12.3.2 Case Selection Strategy 167

12.3.3 Case Study Protocol 167

12.3.4 Ethical Considerations 167

12.3.5 Data Collection 168

12.3.6 Exploratory Study 168

12.3.7 Confirmatory Study 168

12.3.8 Explanatory Study 168

12.4 Data Analysis 169

12.5 Reporting 169

12.6 Lessons Learned 169

13 A CASE STUDY OF THE EVALUATION OF REQUIREMENTS MANAGEMENT TOOLS 171

13.1 Introduction 171

13.2 Design of the Case Study 172

13.2.1 Rationale 172

13.2.2 Objective 172

13.2.3 The Case and Its Context 173

13.2.4 The Units of Analyses 174

13.2.5 Theoretical Framework 175

13.2.6 Research Questions 175

13.2.7 Propositions, Concepts, and Measures 175

13.2.8 Case Study Protocol 175

13.2.9 Methods of Data Collection 176

13.2.10 Methods of Data Analysis 176

13.2.11 Case Selection Strategy 177

13.2.12 Data Selection Strategy 177

13.2.13 Replication Strategy 177

13.2.14 Quality Assurance, Validity, and Reliability 177

13.3 Data Collection 178

13.4 Data Analysis 179

13.5 Reporting and Dissemination 180

13.6 Lessons Learned 181

14 A LARGE-SCALE CASE STUDY OF REQUIREMENTS AND VERIFICATION ALIGNMENT 183

14.1 Introduction 183

14.2 Case Study Design 184

14.2.1 Rationale 184

14.2.2 Objectives 184

14.2.3 Cases and Units of Analysis 185

14.2.4 Theoretical Frame of Reference 186

14.2.5 Research Questions 187

14.3 Planning 188

14.3.1 Methods of Data Collection 189

14.3.2 Case Selection Strategy 190

14.3.3 Selection of Data 191

14.3.4 Case Study Protocol 191

14.3.5 Ethical Considerations 192

14.4 Data Collection 192

14.5 Data Analysis 193

14.6 Lessons Learned 195

14.6.1 Effort Estimation Lessons 195

14.6.2 Design and Planning Lessons 196

14.6.3 Data Collection Lessons 197

14.6.4 Data Analysis Lessons 198

14.6.5 Reporting Lessons 199

14.6.6 A General Lesson 199

EPILOGUE 201

Appendix A: CHECKLISTS FOR READING AND REVIEWING CASE STUDIES 203

A.1 Design of the Case Study 203

A.2 Data Collection 204

A.3 Data Analysis and Interpretation 204

A.4 Reporting and Dissemination 204

A.5 Reader's Checklist 205

Appendix B: EXAMPLE INTERVIEW INSTRUMENT (XP) 207

Appendix C: EXAMPLE INTERVIEW INSTRUMENT (REVV) 209

Appendix D: EXAMPLE OF A CODING GUIDE 213

D.1 Coding Instructions 213

D.2 Codes 214

D.2.1 High Level Codes:...