Description

Once you go through Ch. 9 (book files attached), please work on the following exercises from page 347. Do your best. I am looking for effort here, not perfection. These are the ones I’d like you to focus on below:(1) Exercise 9.1(2) Exercise 9.2(3) Exercise 9.4(4) Exercise 9.5(5) Exercise 9.6

Unformatted Attachment Preview

RESEARCH METHODS IN PRACTICE

THIRD EDITION

For my husband, Howard, thank you for everything, and to my father, who taught me analytical

thinking from an early age (DKR)

For Ada, Alina, and Lucia, my best buddies in life, and to my parents, who each in their own way

made me a researcher (GGVR)

Sara Miller McCune founded SAGE Publishing in 1965 to support

the dissemination of usable knowledge and educate a global

community. SAGE publishes more than 1000 journals and over

600 new books each year, spanning a wide range of subject areas.

Our growing selection of library products includes archives, data,

case studies and video. SAGE remains majority owned by our

founder and after her lifetime will become owned by a charitable

trust that secures the company’s continued independence.

Los Angeles | London | New Delhi | Singapore | Washington DC | Melbourne

RESEARCH METHODS IN PRACTICE

STRATEGIES FOR DESCRIPTION AND CAUSATION

THIRD EDITION

DAHLIA K. REMLER

BARUCH COLLEGE AND THE GRADUATE CENTER, CITY UNIVERSITY OF NEW YORK

GREGG G. VAN RYZIN

RUTGERS UNIVERSITY, NEWARK

FOR INFORMATION:

SAGE Publications, Inc.

2455 Teller Road

Thousand Oaks, California 91320

E-mail: [email protected]

SAGE Publications Ltd.

1 Oliver’s Yard

Copyright © 2022 by SAGE Publications, Inc.

All rights reserved. Except as permitted by U.S. copyright law, no part of

this work may be reproduced or distributed in any form or by any means, or

stored in a database or retrieval system, without permission in writing from

the publisher.

All third-party trademarks referenced or depicted herein are included

solely for the purpose of illustration and are the property of their

respective owners. Reference to these trademarks in no way indicates

any relationship with, or endorsement by, the trademark owner.

55 City Road

London EC1Y 1SP

United Kingdom

SAGE Publications India Pvt. Ltd.

B 1/I 1 Mohan Cooperative Industrial Area

Mathura Road, New Delhi 110 044

SPSS® is a registered trademark of International Business Machines

Corporation (“IBM®”).

Printed in the Canada.

Library of Congress Cataloging-in-Publication Data

India

Names: Remler, Dahlia K., author. | Van Ryzin, Gregg G. (Gregg Gerard),

author.

SAGE Publications Asia-Pacific Pte. Ltd.

Title: Research methods in practice : strategies for description and

causation / Dahlia K. Remler, Gregg G. Van Ryzin.

18 Cross Street #10-10/11/12

China Square Central

Singapore 048423

Description: Third edition. | London ; Washington, DC : SAGE

Publishing, [2022] | Includes bibliographical references and index.

Identifiers: LCCN 2021026184 | ISBN 978-1-5443-1842-4 (paperback)

| ISBN 978-1-5443-1840-0 (epub) | ISBN 978-1-5443-1841-7 (epub) |

ISBN 978-1-5443-1843-1 (pdf)

Subjects: LCSH: Research—Methodology—Study and teaching

(Graduate)

Classification: LCC LB2369 .R46 2022 | DDC 378.1/70281—dc23

LC record available at https://lccn.loc.gov/2021026184

This book is printed on acid-free paper.

Acquisitions Editor: Leah Fargotstein

Product Associate: Ivey Mellem

Production Editor: Rebecca Lee

Copy Editor: Karen E. Taylor

Typesetter: C&M Digitals (P) Ltd.

Cover Designer: Candice Harman

Marketing Manager: Victoria Velasquez

Cover Art: Critical Masses by Hanna Mandelbaum

(HannaMandelbaum.com | IG @hannajanecreates)

21 22 23 24 25 10 9 8 7 6 5 4 3 2 1

BRIEF CONTENTS

Preface

xxvi

Acknowledgments

xxxiii

About the Authors xxxvii

PART I

FOUNDATIONS

Chapter 1.

Chapter 2.

Chapter 3.

Research in the Real World

Theory, Models, and Research Questions

Qualitative Research

PART II

STRATEGIES FOR DESCRIPTION

Chapter 4.

Chapter 5.

Chapter 6.

Chapter 7.

Measurement

Sampling

Secondary Data

Surveys and Other Primary Data

2

26

62

104

154

202

234

PART III STATISTICAL TOOLS AND INTERPRETATIONS

Chapter 8.

Chapter 9.

Chapter 10.

Making Sense of the Numbers

Making Sense of Inferential Statistics

Making Sense of Multivariate Statistics

272

314

350

PART IV STRATEGIES FOR CAUSATION

Chapter 11.

Chapter 12.

Chapter 13.

Chapter 14.

Chapter 15.

Causation

Observational Studies

Using Regression to Estimate Causal Effects

Randomized Experiments

Natural and Quasi Experiments

390

424

458

492

542

PART V

CONTEXT AND COMMUNICATION

Chapter 16.

Chapter 17.

The Politics, Production, and Ethics of Research

How to Find, Review, and Present Research

Glossary

G-1

References

Index

I-1

R-1

584

616

DETAILED CONTENTS

Preface

xxvi

Acknowledgments

xxxiii

About the Authors xxxvii

PART I: FOUNDATIONS

Chapter 1. Research in the Real World

2

Learning Objectives

3

Do Methods Matter?

3

Good Evidence Comes From Well-Made Research

3

May the Best Methods Win

4

Research-Savvy People Rule

4

Research, Policy, and Practice

5

Analytics

5

Performance Measurement

6

Evaluation Research

7

Evidence-Based Policy and Programs

7

Evidence Can Mislead

7

Misleading Measurements

8

Misleading Samples

8

Misleading Correlations

9

What Is Research?

9

Secondary and Primary Research

9

It Comes in Various Shapes and Sizes

10

It’s Never Perfect

10

It’s Uncertain and Contingent

10

It Aims to Generalize

11

Bits and Pieces of a Puzzle

11

It Involves Competition and Criticism

12

It Can Be Quantitative, Qualitative, or a Mix of Both

12

It Can Be Applied or Basic

13

Descriptive and Causal Research

13

Description: What Is the World Like?

13

Causation: How Would the World Be Different If Something Changed?

14

Description of a Correlation Is Not Proof of Causation

14

Epistemology: Ways of Knowing

15

The Scientific Method

15

Are There Objective Truths in Social Science?

16

Induction and Deduction

17

Proof Requires Fresh Data

17

Approaching Research From Different Angles

18

Consuming Research

18

Commissioning Research

19

Conducting Research

19

Ethics of Research

20

Poisoned by New York’s Best Restaurants

20

History of Human Subjects Abuses in Research

21

Principles of Ethical Research Emerge

22

What Constitutes Informed Consent and Voluntary Participation?

23

Ethical Issues Depend on Research Form and Context

23

Conclusion: The Road Ahead

24

Exercises

25

Chapter 2. Theory, Models, and Research Questions

26

Learning Objectives

27

Community Policing Comes to Portland

27

Box 2.1: Broken Windows Theory and Fighting Crime in New York City

What Is a Theory?

28

28

Theories Tell Causal Stories

29

Theories Explain Variation

30

Theories Identify Key Variables

31

Theories Generate Testable Hypotheses

31

Theories (in Applied Research) Focus on Modifiable Variables

31

Theories Are Positive, Not Normative

32

Where Do Theories Come From?

32

What Is a Model?

Variables and Relationships

Box 2.2: Path Models, Causal Diagrams, and DAGs

33

34

34

Unit of Analysis, or Cases

35

Independent and Dependent Variables

36

Box 2.3: Equations as Models

Sign of a Relationship

Box 2.4: Relationship Signs for (Nominal) Categorical Variables

37

37

38

Patterns of Association: Correlation

39

Causal Mechanism

40

Box 2.5: What Went Wrong With Broken Windows

42

Logic Models: Mechanisms of Programs

42

Do Smaller Classes Help Kids Learn?

43

Naming Variables

45

What About Other Causes of the Outcome?

45

Usefulness of a Logic Model

46

Box 2.6: China Launches Nationwide AIDS Prevention Program

47

Tips for Creating a Logic Model

48

Multiple Mechanisms in a Logic Model

50

Box 2.7: Implementation-Oriented Logic Models With Inputs,

Activities, and Outputs

Alternative Perspectives on Theory in Social Research

51

52

Interpretivist Theory

52

Grand Theories

52

How to Find and Focus Research Questions

53

Applied Research Questions

53

Questions You Ideally Want to Answer, and Those

You Actually Can

54

Know If Your Question Is Descriptive or Causal

55

Make Your Question Positive, Not Normative

55

Generating Questions and Ideas

55

Conclusion: Theories Are Practical

Box 2.8: Critical Questions to Ask About Theory, Models,

and Research Questions

Box 2.9: Tips on Doing Your Own Research: Theory, Models,

and Research Questions

Exercises

Chapter 3. Qualitative Research

57

57

58

58

62

Learning Objectives

63

Fighting Malaria in Kenya

63

Theory, Causes, and Qualitative Research

What Is Qualitative Research?

65

65

Comparing Qualitative and Quantitative Research

66

Strengths of Qualitative Research

69

Existing Qualitative Data

70

Archival and Other Written Documents

71

Social Media

71

Visual Culture

72

Qualitative Interviews

72

Unstructured Interviews

72

Semistructured Interviews

73

Asking Truly Open-Ended Questions

74

The Power of Probes

75

Practical Considerations When Doing Interviews

76

Focus Groups

76

What Do People Think of Congestion Pricing?

77

Focus Group Selection and Composition

77

Why a Focus Group? Why Not Individual Interviews?

78

Moderating and Documenting a Focus Group

78

Virtual Focus Groups

79

Qualitative Observation

80

Participant Observation and Ethnography

81

Why Do the Homeless Refuse Help?

81

Levels on a Participation-Observation Continuum

82

Secret Shopping and Audit Studies

82

Case Study Research

83

Maryland’s Gun Violence Act

83

Selecting a Case to Study

84

Comparing Cases

84

Qualitative Data Analysis

85

Integration of Analysis and Data Gathering

85

Tools of Qualitative Analysis

86

Coding and Content Analysis

87

Qualitative Data Analysis Software

89

Analyzing Big (Qualitative) Data

90

The Qualitative-Quantitative Debate

91

A Brief History of the Debate

91

How Qualitative and Quantitative Approaches Overlap

92

A Qualitative-Quantitative Research Cycle

93

Mixed-Methods Research and Triangulation

95

Box 3.1: Transition Services for Incarcerated Youth: A Mixed

Methods Evaluation Study

Ethics in Qualitative Research

96

97

Presenting Qualitative Data

97

Can You Obtain Informed Consent?

97

Should You Intervene?

98

Should You Empower People?

98

Conclusion: Matching Methods to Questions

98

Box 3.2: Critical Questions to Ask About a Qualitative Study

99

Box 3.3: Tips on Doing Your Own Qualitative Research

100

Exercises

100

PART II: STRATEGIES FOR DESCRIPTION

Chapter 4. Measurement

104

Learning Objectives

105

The U.S. Poverty Measure

105

What Is Measurement?

105

Measurement in Qualitative Research

106

Performance Measurement and Analytics

106

Measurement: The Basic Model and a Road Map

107

Conceptualization

107

What Is Poverty?

108

Where to Look for Conceptualizations?

109

Latent Variables

110

Dimensions

110

Operationalization

How the U.S. Poverty Measure Was Operationalized

111

111

Instruments

112

Protocols and Personnel

112

Box 4.1: Operational Definition of Poverty in the United States

Proxies and Proxy Respondents

113

113

Indexes and Scales

114

Box 4.2: Rensis Likert

117

Validity

Is the U.S. Poverty Measure Valid?

Box 4.3: Using Items That Vary in Difficulty:

Item Response Theory

Face Validity

Content Validity

117

117

118

119

120

Box 4.4: Content Validity of Measured Race and Ethnicity

121

Box 4.5: Content Validity of Measured Gender

122

Valid for What Purpose?

122

Criterion-Related Validity

123

Self-Reported Drug Use: Is It Valid?

123

Does the Measure Predict Behavior?

124

Does the Measure Relate to Other Variables as Expected?

124

Limitations of Validity Studies

Box 4.6: Some Forms of Measurement Validity

Measurement Error

Bias

Box 4.7: Example of a Validity Study

Random Error—Noise

Bias and Noise in the U.S. Poverty Measure

126

127

127

127

128

129

129

Box 4.8: Bias, Bias Everywhere

130

Box 4.9: Classical Test Theory: What Is Seen and Unseen

132

Reliability

132

Why Reliability Matters

133

Many Ways to Tell If a Measure Is Reliable

134

Validity and Reliability: Contrasted and Compared

136

Box 4.10: Is It a Validity Problem or a Reliability Problem?

137

Validity and Reliability in Qualitative Research

137

Levels of Measurement

138

Quantitative Variables

Box 4.11: Unit/Level of Measurement/Analysis?

139

140

Categorical Variables

141

Turning Categorical Variables Into Quantitative Ones

143

Units of Analysis and Levels of Measurement

145

Measurement in the Real World: Trade-offs and Choices

146

What Will It Cost?

146

Is It Ethical?

146

How Will It Affect the Quality and Rate of Responding?

147

The Validity-Reliability Trade-off

147

Use an Established Measure or Invent a New One?

147

Gaming and Other Behavior Responses to Measurement

148

Multiple Dimensions—or Just One?

148

Conclusion: Measurement Matters

149

Box 4.12: Critical Questions to Ask About Measurement

149

Box 4.13: Tips on Doing Your Own Research: Measurement

150

Exercises

150

Chapter 5. Sampling

154

Learning Objectives

155

Gauging the Fallout From Hurricane Katrina

155

Generalizability

156

Population of Interest, Sampling, and Representativeness

156

Generalizing Beyond the Original Population of Interest

157

Are Relationships More Generalizable?

158

Replicating Research and Meta-Analysis

159

Generalizability of Qualitative Studies

160

Basic Sampling Concepts

161

Population, Sample, and Inference

161

Census Versus Sample

162

How to Select a Sample: Sampling Frames and Steps

163

Problems and Biases in Sampling

Box 5.1: Likely Voters Versus Actual Voters

164

165

Coverage Problems

166

Nonresponse Problems

167

When Do Coverage and Nonresponse Problems Cause Bias?

168

Box 5.2: Steps in Assessing Coverage and Nonresponse Bias

170

Ethics of Nonresponse

Box 5.3: Nonresponse Bias in 2020 U.S. Presidential

Election Polls?

Nonprobability Sampling

170

171

172

Voluntary Sampling

172

Convenience Sampling

173

Snowball Sampling

173

Quota Sampling

173

Online Sampling

174

Purposive Sampling and Qualitative Research

176

Random (Probability) Sampling

178

The Contribution of Random Sampling

178

Random Sampling Versus Randomized Experiments

179

Simple Random Sampling

179

Sampling Variability

180

Sampling Distributions, Standard Errors, and Confidence Intervals

Confidence Intervals (Margins of Error)

Box 5.4: Relationship Between Various Precision Measures

Sample Size and the Precision of Government Statistics

Determining How Large a Sample You Need

Box 5.5: What Is the True Sample Size?

180

182

183

184

185

187

Sampling in Practice

186

Systematic Sampling

186

Stratified Sampling

188

Disproportionate Sampling (Oversampling)

190

Poststratification Weighting

Box 5.6: An Evaluation of 2016 Election Polls in the U.S.

Sampling With Probabilities Proportional to Size

Cluster and Multistage Sampling

Box 5.7: Design Effects: Complex Survey Sampling Corrections

Random Digit Dialing Sampling

Sampling and Generalizability: A Summary

192

193

194

194

197

197

198

Box 5.8: Critical Questions to Ask About Sampling in Studies

198

Box 5.9: Tips on Doing Your Own Research: Sampling

199

Exercises

Chapter 6. Secondary Data

200

202

Learning Objectives

203

Tracking a Global Pandemic

203

Quantitative Data Forms and Structures

204

Quantitative Data Versus Quantitative Variables

204

Structures of Quantitative Data

205

Micro, Aggregate, and Multilevel Data

Box 6.1: Unit of Observation Versus Unit of Analysis

205

206

Time Dimension of Data

207

Metadata

210

Where Do Quantitative Data Come From?

210

Administrative Records

Adapting Administrative Data for Research

210

211

Box 6.2: How Organizations Can Make the Most of Their

Administrative Data

213

Vital Statistics, Crime Reports, and Unemployment Claims

213

Data for Purchase

214

Ethics of Administrative Record Data

214

Aggregate Data Tables

215

Where to Find Aggregate Tables

215

Aggregate Time-Series and Panel Data

217

Public Use Microdata

217

Know the Major Surveys in Your Field

217

Accessing and Analyzing Public Use Data

224

Data Archives

225

Ethics of Public Use Microdata

225

Secondary Qualitative Data

Ethics of Using Existing Qualitative Data

Big Data

226

227

227

Volume, Velocity, Variety—and a Lack of Structure

227

Analyzing Big Data

228

Ethics of Big Data

228

Linking Data

228

Some Limitations of Secondary Data

229

Does Data Availability Distort Research?

229

When to Collect Original Data?

230

Conclusion

230

Box 6.3: Critical Questions to Ask About Secondary Data

230

Box 6.4: Tips on Doing Your Own Research: Secondary Data

231

Exercises

Chapter 7. Surveys and Other Primary Data

231

234

Learning Objectives

235

Taking the Nation’s Economic Pulse

235

When Should You Do a Survey?

236

Do You Know Enough About the Topic?

236

Does the Information Exist Already in Another Source?

236

Can People Tell You What You Want to Know?

237

Will People Provide Truthful Answers?

237

Steps in the Survey Research Process

237

Clarify the Purpose

237

Identify the Population and Sampling Strategy

238

Develop a Questionnaire

238

Pretest Questionnaire and Survey Procedures

238

Recruit and Train Interviewers

239

Collect Data

239

Enter and Prepare Data for Analysis

240

Analyze Data and Present Findings

240

Modes of Survey Data Collection

240

Intercept Interview Surveys

240

Household Interview Surveys

241

Telephone Interview Surveys

242

Mail Self-Administered Surveys

244

Group Self-Administered Surveys

246

Online Surveys

247

Establishment (Business or Organization) Surveys

249

Panel or Longitudinal Surveys

249

Choosing or Mixing Modes

250

Crafting a Questionnaire

251

Starting Off

251

Closed-Ended Versus Open-Ended Questions

252

Box 7.1: Questionnaire Composed of

Open-Ended Questions

Question Order

Box 7.2: Comparing Opening Questions

253

253

254

Some Advice on Question Wording

256

Some Advice on Response Formats

258

Physical and Graphical Design

260

Put Yourself in Your Respondent’s Shoes

261

Ethics of Survey Research

261

Informed Consent

261

Pushing for a High Response Rate

262

Overburdening Respondents

262

Protecting Privacy and Confidentiality

262

Surveying Minors and Other Vulnerable Populations

263

Making Survey Data Available for Public Use

263

Other Ways to Collect Primary Data

264

Trained Observation

264

Scientific Instruments

266

Computer Code and Data Extraction Algorithms

267

Conclusion

267

Box 7.3: Critical Questions to Ask About Surveys and Other

Primary Data

268

Box 7.4: Tips on Doing Your Own Survey

268

Exercises

269

PART III: STATISTICAL TOOLS AND

INTERPRETATIONS

Chapter 8. Making Sense of the Numbers

272

Learning Objectives

273

“Last Weekend I Walked Eight”

273

Units, Rates, and Ratios

274

What Units?

274

Rates, or Why Counts Often Mislead

275

Box 8.1: Relevant Comparisons—Are Anesthesiologists

Prone to Addiction?

276

Percent Change and Percentage Point Change

276

The Strangeness of Percent Change on the Return Trip

277

Rates of Change

277

Odds

277

Prevalence and Incidence

278

Statistics Starting Point: Variables in a Data Set

279

Distributions

280

Distribution of a Categorical Variable

280

Distribution of a Quantitative Variable

281

Measures of Center: Mean and Median

Box 8.2: Mean: The Formula

When to Use Median? When to Use Mean?

Measures of Spread and Variation

Standard Deviation

Box 8.3: Standard Deviation: The Formula

283

284

284

285

285

286

Pay Attention to the Standard Deviation, Not Just the Mean

287

Standardized (z) Scores

287

Quantiles: Another Way to Measure Spread

288

Relationships Between Categorical Variables

289

Cross-Tabulation

289

Relative Risks and Odds Ratios: Another Way to Show

Relationships in Categorical Data

291

Adjusted and Standardized Rates: When to Use Them

293

Relationships Between Quantitative Variables: Scatterplots

and Correlation

293

Scatterplots

293

Correlation

295

Box 8.4: Correlation: The Formula

Relationships Between a Categorical and

a Quantitative Variable

Box 8.5: Which One Is the Dependent Variable? Which One Is

the Independent Variable?

Simple Regression: Best-Fit Straight Line

296

296

297

297

Box 8.6: Simple Regression: The Equations

299

Interpreting the Regression Coefficient (Slope)

299

Box 8.7: Steps for Interpreting a Regression Coefficient

300

Can a Regression Coefficient Be Interpreted as a Causal Effect?

300

Changes Versus Levels

301

R-Squared and Residuals: How Well Does the Line Fit the Data?

302

Practical Significance

303

Practical Significance Is a Matter of Judgment

304

Effect Size

304

Steps for Assessing the Practical Significance of Effects and Differences

305

Statistical Software

305

Spreadsheets

306

Statistical Packages: SAS, SPSS, Stata, and R

306

Specialized Modeling and Matrix Language Programs

306

Conclusion: Tools for Description and Causation

306

Box 8.8: Tips on Doing Your Own Research:

Descriptive Statistics

307

Exercises

Chapter 9. Making Sense of Inferential Statistics

308

314

Learning Objectives

315

But Is It Significant?

315

Statistical Inference: What’s It Good For?

316

The Sampling Distribution: Foundation of Statistical Inference

317

What a Sampling Distribution Looks Like

317

The Standard Error (of a Proportion)

319

The Standard Error (of a Mean)

320

Confidence Intervals

321

Univariate Statistics and Relationships Both Have Confidence Intervals

321

Confidence Intervals Reflect Only Some Sources of Error

323

Calculating a Confidence Interval (Margin of Error) for a Proportion

323

Calculating a Confidence Interval (Margin of Error) for a Mean

324

How Big Does the Sample Size Need to Be? Getting the Precision You Want

326

Significance Tests

328

Falsification and the Logic of Significance Testing

329

Running a Significance Test

330

p Values

330

Significance Tests for Simple Regression

332

Chi-Square Test of Cross-Tabs

334

Other Test Statistics

335

Statistical Significance, Practical Significance, and Power

336

Combinations of Statistical and Practical Significance

336

Box 9.1: Sources of Statistical Significance and of

Statistical Insignificance

339

Failing to Recognize a Difference: Type II Errors

339

Power

340

Multiple Comparison Corrections

341

Sample Size Calculations for Significance Tests

341

Adjusting Inference for Clustering and Other Complex Sampling

342

Issues and Extensions of Statistical Inference

343

Inference With a Nonprobability Sample: What Does It Mean?

343

Bootstrapping: Inference for Statistics With No

Standard Error Formulas

344

Equivalence and Noninferiority Tests

344

Bayesian Inference

345

Conclusion

Box 9.2: Tips on Doing Your Own Research:

Inferential Statistics

Exercises

Chapter 10. Making Sense of Multivariate Statistics

345

346

347

350

Learning Objectives

351

Multiple Regression: The Basics

351

Multiple Regression for Prediction

353

Box 10.1: How to Run a Multiple

Regression Using Software

354

Box 10.2: Steps for Predicting With Regression

354

The Danger (and Necessity) of Out-of-Sample Extrapolation

355

R-Squared and Adjusted R-Squared

355

All Else Held Constant: A Bit More Mathematics

356

Multicollinearity

356

Standardized Coefficients: The Relative Importance of

Independent Variables

358

Inference for Regression

359

Standard Error of the Coefficient

359

Confidence Intervals in Regression

360

Confidence Interval of a Predicted Value

361

Significance Testing in Regression

361

Influences on Inference in Multiple Regression

362

Categorical Independent Variables

363

Dummy Variables in Regression

363

Categorical Variables With More Than Two Possible Values

364

Box 10.3: Representing a Categorical Variable With More

Than Two Categories: Diabetes Example

Interpreting the Coefficient of a Dummy Variable

Analysis of Variance (ANOVA)

Box 10.4: Interpreting the Coefficient of a Dummy Variable

Interactions in Regression

365

366

366

367

368

How to Use and Interpret Interaction Variables

368

Interactions With Quantitative Variables

370

Always Include Both Main Effects

370

Functional Form and Transformations in Regression

370

How to Fit a Curved Relationship

371

How to Interpret Coefficients When a Variable Is Logged

371

The Value of Robustness and Transparency

373

Categorical Variables as Dependent Variables in Regression

373

Linear Probability Model

373

Logistic and Probit Regression

374

What If the Dependent Variable Has More Than Two Categories?

375

Beware of Unrealistic Underlying Assumptions

375

Which Statistical Methods Can I Use?

375

Other Multivariate Methods

376

Path Analysis

376

Factor Analysis

378

Structural Equation Modeling

379

Multilevel Models

380

Time Series and Forecasting

382

Panel Data Methods

383

Spatial Analysis

383

Limited Dependent Variables

383

Survival Analysis

384

Machine Learning

384

More Multivariate Methods Not Covered

385

Conclusion

Box 10.5: Tips on Doing Your Own Research: Multivariate Statistics

Exercises

386

386

387

PART IV: STRATEGIES FOR CAUSATION

Chapter 11. Causation

390

Learning Objectives

391

Family Dinners and Teenage Substance Abuse

391

Correlation Is Not Causation

Alternative Explanations of a Correlation

Box 11.1: Children Who Have Frequent Family Dinners Less

Likely to Use Marijuana, Tobacco, and Drink Alcohol

Reverse Causation

Box 11.2: Directed Acyclic Graphs (DAGs)

Common Causes

391

392

393

393

394

394

Bias From a Common Cause

395

Bias From Reverse Causation: Simultaneity Bias

396

Some More Correlations That May—or May Not—Imply Causation

396

Box 11.3: Get in the Habit of Thinking of Alternative Theories

397

Box 11.4: Recent Evidence on Family Dinners and Teen

Substance Abuse

399

Causal Mechanisms

399

Indirect and Direct Causal Effects

400

Context and Moderators

401

Arrows, Arrows Everywhere

401

Why Worry About the Correct Causal Model?

402

Box 11.5: How to Talk (or Write) About Causation

and Correlation

Evidence of Causation: Some Critical Clues

403

403

There Is a Correlation (Association)

404

The Cause Happens Before the Effect

404

The Correlation Appears in Many Different Contexts

404

Box 11.6: Prominent Epidemiologists Discuss

Replication and Causation

405

There Is a Plausible Mechanism

405

There Are No Plausible Alternative Explanations

406

Other Influences Are Accounted for in the Analysis

406

There Is Qualitative Evidence of a Mechanism

407

The Correlation Is Not Just a Chance Coincidence

408

Detective Work and Shoe Leather

408

Self-Selection and Endogeneity

Self-Selection

408

409

Endogeneity

409

Aggregation Bias and the Ecological Fallacy

410

The Counterfactual Definition of Causation

410

Potential Outcomes

411

If We Only Had a Time Machine

411

Can Really Good Prediction Replace Causal Inference?

412

Experimentation and Exogeneity: Making Things Happen

412

Can Exercise Cure Depression?

413

Why Experimentation Beats Passive Observation

413

Exogeneity: Intervening in the World

414

Box 11.7: Exogenous or Endogenous? It Depends on

the Dependent Variable

415

Box 11.8: The Meaning of Exogeneity and

Endogeneity in Structural Equation Modeling

416

Control: Holding Things Constant

Box 11.9: The Many Uses of the Term Control

417

417

Experimentation: The Basic Logic

418

Ethical Limitations of Experiments

418

Experimentation, Policy, and Practice

419

Conclusion: Tools to Probe Causation

419

Box 11.10: Critical Questions to Ask About Causation

420

Box 11.11: Tips on Doing Your Own Research: Causation

420

Exercises

421

Chapter 12. Observational Studies

424

Learning Objectives

425

Private Versus Public Schools

425

What Is an Observational Study?

426

The Gold Standard for Description—But Not

for Causal Estimation

426

Limitations of an Observational Study

427

Control Variables

427

How Control Variables Help Disentangle a Causal Effect

427

Why These Control Variables?

428

How Did Control Variables Change the Estimate

of a Causal Effect?

429

Matching

429

Individual-Level Matching

429

Aggregate Matching

431

Control Variables: An Empirical Example

431

Step 1: Speculate on Common Causes

432

Step 2: Look for Differences

433

Step 3: Stratify by Control Variables

433

Omitted Variable Bias

435

Box 12.1: Are the Police Fair, or Not? Layered Cross-Tabs for

Categorical Dependent Variables

Expanding the Choice of Control Variable

Box 12.2: Omitted Variables—and the Bias They Cause—

by Any Other Name

How to Choose Control Variables

436

437

438

440

What’s Driving the Independent Variable?

440

Causal Diagrams as a Guide for Choosing

Control Variables

442

Beware of Using Intervening Variables as Controls

443

Colliders: Bias in Causal Estimates From

Sample Selection

444

Empirical Approach to Choosing Controls

446

Causes That Can Be Ignored—Without Bias

446

Unmeasured Variables, Proxies, and Data

447

Choosing Good Control Variables Depends

on Your Question

447

Various Purposes of Control Variables

448

Owning Your Interest in Causation—

But Not Overclaiming

448

Epidemiological Approaches to Observational Studies

449

Prospective and Retrospective Cohort Studies

449

Case-Control Studies

451

Conclusion: Observational Studies in Perspective

453

Box 12.3: Critical Questions to Ask About Observational Studies

With Control Variables

453

Box 12.4: Tips on Doing Your Own Research: Observational

Studies With Control Variables

454

Exercises

454

Chapter 13. Using Regression to Estimate

Causal Effects

458

Learning Objectives

459

Cigarette Taxes and Smoking

459

From Stratification to Multiple Regression

460

Using More Than One (or Two) Control Variables

460

Control Variables That Are Quantitative

460

From Description to Causation: The Education-Earnings

Link Reconsidered

461

Multiple Regression: Brief Overview and Interpretation

463

How Multiple Regression Is Like Stratification: An Illustration

463

Specification: The Choice of Control Variables

465

Does Greenery Affect Birth Outcomes?

467

Step 1: Speculate on Common Causes

468

Step 2: Examine the Relationship Between the Independent

Variable of Interest and Potential Common Causes

469

Step 3: Implement Control Variables Through Multiple Regression

470

Interpreting Multiple Regression Coefficients as Effects of Interest

471

Practical Significance: Is the Effect Big Enough to Care About?

472

Further Topics in Regression for Estimating Causal Effects

473

How Controls and Omitted Variables Influence Estimated Effects

473

Box 13.1: Predicting the Direction of Omitted Variables Bias

474

Interactions, Functional Forms, and Categorical Dependent Variables

475

A Focus on One Causal Effect or Many

476

When Is Low R-Squared a Problem?

476

Software Doesn’t Know the Difference, But You Should

477

Control Variables With Exogenous Independent Variables:

The Gender Earnings Gap

477

Box 13.2: The Life Expectancy of Taxi Drivers: Prediction Versus Causation

478

The Gender Earnings Gap

478

The Gender Earnings Gap Depends on What Is Held Constant

479

Can Control Variables Analyses Show Discrimination?

Box 13.3: Tips on Interpreting Results Tables

Other Multivariate Techniques for Observational Studies

482

483

483

Propensity Score Matching

483

Machine Learning

485

Conclusion: A Widely Used Strategy, With Drawbacks

486

Box 13.4: Critical Questions to Ask About Studies That

Use Multiple Regression to Estimate Causal Effects

486

Box 13.5: Tips on Doing Your Own Research: Multiple Regression

to Estimate Causal Effects

486

Exercises

Chapter 14. Randomized Experiments

487

492

Learning Objectives

493

Time Limits on Welfare

493

Florida’s Family Transition Program: A Randomized Experiment

494

Random Assignment: Creating Statistical Equivalence

Random Assignment in Practice

495

496

Box 14.1: True Randomization or Haphazard?

497

Statistical Equivalence: A Look at the Data

497

Why Random Assignment Is Better Than Matching or Control Variables

499

Findings: What Happened in Pensacola

499

Box 14.2: Nobel Prize for Experimental Approach to

Alleviating Global Poverty

The Logic of Randomized Experiments: Exogeneity Revisited

Statistical Significance of an Experimental Result

The Settings of Randomized Experiments

501

502

503

504

Lab Experiments

505

Field Experiments

505

Box 14.3: Practical Difficulties in a Field Experiment

About Online Education

506

Box 14.4: Social and Policy Research Organizations

That Specialize in Randomized Field Experiments

507

Survey Experiments

507

Generalizability of Randomized Experiments

509

Random Assignment Versus Random Sampling

509

Volunteers and Generalizability

511

The Ideal Study: Random Sampling, Then Random Assignment

511

Box 14.5: Time-Sharing Experiments for

the Social Sciences (TESS)

Limited Settings

Generalizability of the Treatment

Box 14.6: The RAND Health Insurance Experiment

512

513

513

514

Support Factors and Causal Cakes

515

Generalizability in the Long Run

516

Variations on the Design of Experiments

517

Cluster Randomization

517

Arms in an Experiment

518

Levels of a Treatment: Probing a Dose-Response Relationship

519

Factors in an Experiment: Probing Interactions

520

Within-Subjects (Crossover) Experiments

521

Matching and Stratifying (or Blocking)

521

Artifacts in Experiments

The Hawthorne Effect and Demand Characteristics

Placebo Effect and Blinding

Box 14.7: The Perry Preschool Study

522

522

522

524

Contamination

524

Demoralization and Rivalry

524

Noncompliance and Attrition

525

Analysis of Randomized Experiments

525

Balancing, Control Variables, and Covariates

526

Sample Size and Minimum Detectable Effects

527

Heterogeneous Treatment Effects

528

Preregistration of Analysis

528

Intent to Treat and Treatment of the Treated Analyses

529

Box 14.8: The Moving to Opportunity Demonstration

Ethics of Randomized Experiments

531

532

Something for Everyone: The Principle of Beneficence

532

Informed Con