Description
Once you go through Ch. 9 (book files attached), please work on the following exercises from page 347. Do your best. I am looking for effort here, not perfection. These are the ones I’d like you to focus on below:(1) Exercise 9.1(2) Exercise 9.2(3) Exercise 9.4(4) Exercise 9.5(5) Exercise 9.6
Unformatted Attachment Preview
RESEARCH METHODS IN PRACTICE
THIRD EDITION
For my husband, Howard, thank you for everything, and to my father, who taught me analytical
thinking from an early age (DKR)
For Ada, Alina, and Lucia, my best buddies in life, and to my parents, who each in their own way
made me a researcher (GGVR)
Sara Miller McCune founded SAGE Publishing in 1965 to support
the dissemination of usable knowledge and educate a global
community. SAGE publishes more than 1000 journals and over
600 new books each year, spanning a wide range of subject areas.
Our growing selection of library products includes archives, data,
case studies and video. SAGE remains majority owned by our
founder and after her lifetime will become owned by a charitable
trust that secures the company’s continued independence.
Los Angeles | London | New Delhi | Singapore | Washington DC | Melbourne
RESEARCH METHODS IN PRACTICE
STRATEGIES FOR DESCRIPTION AND CAUSATION
THIRD EDITION
DAHLIA K. REMLER
BARUCH COLLEGE AND THE GRADUATE CENTER, CITY UNIVERSITY OF NEW YORK
GREGG G. VAN RYZIN
RUTGERS UNIVERSITY, NEWARK
FOR INFORMATION:
SAGE Publications, Inc.
2455 Teller Road
Thousand Oaks, California 91320
E-mail: [email protected]
SAGE Publications Ltd.
1 Oliver’s Yard
Copyright © 2022 by SAGE Publications, Inc.
All rights reserved. Except as permitted by U.S. copyright law, no part of
this work may be reproduced or distributed in any form or by any means, or
stored in a database or retrieval system, without permission in writing from
the publisher.
All third-party trademarks referenced or depicted herein are included
solely for the purpose of illustration and are the property of their
respective owners. Reference to these trademarks in no way indicates
any relationship with, or endorsement by, the trademark owner.
55 City Road
London EC1Y 1SP
United Kingdom
SAGE Publications India Pvt. Ltd.
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road, New Delhi 110 044
SPSS® is a registered trademark of International Business Machines
Corporation (“IBM®”).
Printed in the Canada.
Library of Congress Cataloging-in-Publication Data
India
Names: Remler, Dahlia K., author. | Van Ryzin, Gregg G. (Gregg Gerard),
author.
SAGE Publications Asia-Pacific Pte. Ltd.
Title: Research methods in practice : strategies for description and
causation / Dahlia K. Remler, Gregg G. Van Ryzin.
18 Cross Street #10-10/11/12
China Square Central
Singapore 048423
Description: Third edition. | London ; Washington, DC : SAGE
Publishing, [2022] | Includes bibliographical references and index.
Identifiers: LCCN 2021026184 | ISBN 978-1-5443-1842-4 (paperback)
| ISBN 978-1-5443-1840-0 (epub) | ISBN 978-1-5443-1841-7 (epub) |
ISBN 978-1-5443-1843-1 (pdf)
Subjects: LCSH: Research—Methodology—Study and teaching
(Graduate)
Classification: LCC LB2369 .R46 2022 | DDC 378.1/70281—dc23
LC record available at https://lccn.loc.gov/2021026184
This book is printed on acid-free paper.
Acquisitions Editor: Leah Fargotstein
Product Associate: Ivey Mellem
Production Editor: Rebecca Lee
Copy Editor: Karen E. Taylor
Typesetter: C&M Digitals (P) Ltd.
Cover Designer: Candice Harman
Marketing Manager: Victoria Velasquez
Cover Art: Critical Masses by Hanna Mandelbaum
(HannaMandelbaum.com | IG @hannajanecreates)
21 22 23 24 25 10 9 8 7 6 5 4 3 2 1
BRIEF CONTENTS
Preface
xxvi
Acknowledgments
xxxiii
About the Authors xxxvii
PART I
FOUNDATIONS
Chapter 1.
Chapter 2.
Chapter 3.
Research in the Real World
Theory, Models, and Research Questions
Qualitative Research
PART II
STRATEGIES FOR DESCRIPTION
Chapter 4.
Chapter 5.
Chapter 6.
Chapter 7.
Measurement
Sampling
Secondary Data
Surveys and Other Primary Data
2
26
62
104
154
202
234
PART III STATISTICAL TOOLS AND INTERPRETATIONS
Chapter 8.
Chapter 9.
Chapter 10.
Making Sense of the Numbers
Making Sense of Inferential Statistics
Making Sense of Multivariate Statistics
272
314
350
PART IV STRATEGIES FOR CAUSATION
Chapter 11.
Chapter 12.
Chapter 13.
Chapter 14.
Chapter 15.
Causation
Observational Studies
Using Regression to Estimate Causal Effects
Randomized Experiments
Natural and Quasi Experiments
390
424
458
492
542
PART V
CONTEXT AND COMMUNICATION
Chapter 16.
Chapter 17.
The Politics, Production, and Ethics of Research
How to Find, Review, and Present Research
Glossary
G-1
References
Index
I-1
R-1
584
616
DETAILED CONTENTS
Preface
xxvi
Acknowledgments
xxxiii
About the Authors xxxvii
PART I: FOUNDATIONS
Chapter 1. Research in the Real World
2
Learning Objectives
3
Do Methods Matter?
3
Good Evidence Comes From Well-Made Research
3
May the Best Methods Win
4
Research-Savvy People Rule
4
Research, Policy, and Practice
5
Analytics
5
Performance Measurement
6
Evaluation Research
7
Evidence-Based Policy and Programs
7
Evidence Can Mislead
7
Misleading Measurements
8
Misleading Samples
8
Misleading Correlations
9
What Is Research?
9
Secondary and Primary Research
9
It Comes in Various Shapes and Sizes
10
It’s Never Perfect
10
It’s Uncertain and Contingent
10
It Aims to Generalize
11
Bits and Pieces of a Puzzle
11
It Involves Competition and Criticism
12
It Can Be Quantitative, Qualitative, or a Mix of Both
12
It Can Be Applied or Basic
13
Descriptive and Causal Research
13
Description: What Is the World Like?
13
Causation: How Would the World Be Different If Something Changed?
14
Description of a Correlation Is Not Proof of Causation
14
Epistemology: Ways of Knowing
15
The Scientific Method
15
Are There Objective Truths in Social Science?
16
Induction and Deduction
17
Proof Requires Fresh Data
17
Approaching Research From Different Angles
18
Consuming Research
18
Commissioning Research
19
Conducting Research
19
Ethics of Research
20
Poisoned by New York’s Best Restaurants
20
History of Human Subjects Abuses in Research
21
Principles of Ethical Research Emerge
22
What Constitutes Informed Consent and Voluntary Participation?
23
Ethical Issues Depend on Research Form and Context
23
Conclusion: The Road Ahead
24
Exercises
25
Chapter 2. Theory, Models, and Research Questions
26
Learning Objectives
27
Community Policing Comes to Portland
27
Box 2.1: Broken Windows Theory and Fighting Crime in New York City
What Is a Theory?
28
28
Theories Tell Causal Stories
29
Theories Explain Variation
30
Theories Identify Key Variables
31
Theories Generate Testable Hypotheses
31
Theories (in Applied Research) Focus on Modifiable Variables
31
Theories Are Positive, Not Normative
32
Where Do Theories Come From?
32
What Is a Model?
Variables and Relationships
Box 2.2: Path Models, Causal Diagrams, and DAGs
33
34
34
Unit of Analysis, or Cases
35
Independent and Dependent Variables
36
Box 2.3: Equations as Models
Sign of a Relationship
Box 2.4: Relationship Signs for (Nominal) Categorical Variables
37
37
38
Patterns of Association: Correlation
39
Causal Mechanism
40
Box 2.5: What Went Wrong With Broken Windows
42
Logic Models: Mechanisms of Programs
42
Do Smaller Classes Help Kids Learn?
43
Naming Variables
45
What About Other Causes of the Outcome?
45
Usefulness of a Logic Model
46
Box 2.6: China Launches Nationwide AIDS Prevention Program
47
Tips for Creating a Logic Model
48
Multiple Mechanisms in a Logic Model
50
Box 2.7: Implementation-Oriented Logic Models With Inputs,
Activities, and Outputs
Alternative Perspectives on Theory in Social Research
51
52
Interpretivist Theory
52
Grand Theories
52
How to Find and Focus Research Questions
53
Applied Research Questions
53
Questions You Ideally Want to Answer, and Those
You Actually Can
54
Know If Your Question Is Descriptive or Causal
55
Make Your Question Positive, Not Normative
55
Generating Questions and Ideas
55
Conclusion: Theories Are Practical
Box 2.8: Critical Questions to Ask About Theory, Models,
and Research Questions
Box 2.9: Tips on Doing Your Own Research: Theory, Models,
and Research Questions
Exercises
Chapter 3. Qualitative Research
57
57
58
58
62
Learning Objectives
63
Fighting Malaria in Kenya
63
Theory, Causes, and Qualitative Research
What Is Qualitative Research?
65
65
Comparing Qualitative and Quantitative Research
66
Strengths of Qualitative Research
69
Existing Qualitative Data
70
Archival and Other Written Documents
71
Social Media
71
Visual Culture
72
Qualitative Interviews
72
Unstructured Interviews
72
Semistructured Interviews
73
Asking Truly Open-Ended Questions
74
The Power of Probes
75
Practical Considerations When Doing Interviews
76
Focus Groups
76
What Do People Think of Congestion Pricing?
77
Focus Group Selection and Composition
77
Why a Focus Group? Why Not Individual Interviews?
78
Moderating and Documenting a Focus Group
78
Virtual Focus Groups
79
Qualitative Observation
80
Participant Observation and Ethnography
81
Why Do the Homeless Refuse Help?
81
Levels on a Participation-Observation Continuum
82
Secret Shopping and Audit Studies
82
Case Study Research
83
Maryland’s Gun Violence Act
83
Selecting a Case to Study
84
Comparing Cases
84
Qualitative Data Analysis
85
Integration of Analysis and Data Gathering
85
Tools of Qualitative Analysis
86
Coding and Content Analysis
87
Qualitative Data Analysis Software
89
Analyzing Big (Qualitative) Data
90
The Qualitative-Quantitative Debate
91
A Brief History of the Debate
91
How Qualitative and Quantitative Approaches Overlap
92
A Qualitative-Quantitative Research Cycle
93
Mixed-Methods Research and Triangulation
95
Box 3.1: Transition Services for Incarcerated Youth: A Mixed
Methods Evaluation Study
Ethics in Qualitative Research
96
97
Presenting Qualitative Data
97
Can You Obtain Informed Consent?
97
Should You Intervene?
98
Should You Empower People?
98
Conclusion: Matching Methods to Questions
98
Box 3.2: Critical Questions to Ask About a Qualitative Study
99
Box 3.3: Tips on Doing Your Own Qualitative Research
100
Exercises
100
PART II: STRATEGIES FOR DESCRIPTION
Chapter 4. Measurement
104
Learning Objectives
105
The U.S. Poverty Measure
105
What Is Measurement?
105
Measurement in Qualitative Research
106
Performance Measurement and Analytics
106
Measurement: The Basic Model and a Road Map
107
Conceptualization
107
What Is Poverty?
108
Where to Look for Conceptualizations?
109
Latent Variables
110
Dimensions
110
Operationalization
How the U.S. Poverty Measure Was Operationalized
111
111
Instruments
112
Protocols and Personnel
112
Box 4.1: Operational Definition of Poverty in the United States
Proxies and Proxy Respondents
113
113
Indexes and Scales
114
Box 4.2: Rensis Likert
117
Validity
Is the U.S. Poverty Measure Valid?
Box 4.3: Using Items That Vary in Difficulty:
Item Response Theory
Face Validity
Content Validity
117
117
118
119
120
Box 4.4: Content Validity of Measured Race and Ethnicity
121
Box 4.5: Content Validity of Measured Gender
122
Valid for What Purpose?
122
Criterion-Related Validity
123
Self-Reported Drug Use: Is It Valid?
123
Does the Measure Predict Behavior?
124
Does the Measure Relate to Other Variables as Expected?
124
Limitations of Validity Studies
Box 4.6: Some Forms of Measurement Validity
Measurement Error
Bias
Box 4.7: Example of a Validity Study
Random Error—Noise
Bias and Noise in the U.S. Poverty Measure
126
127
127
127
128
129
129
Box 4.8: Bias, Bias Everywhere
130
Box 4.9: Classical Test Theory: What Is Seen and Unseen
132
Reliability
132
Why Reliability Matters
133
Many Ways to Tell If a Measure Is Reliable
134
Validity and Reliability: Contrasted and Compared
136
Box 4.10: Is It a Validity Problem or a Reliability Problem?
137
Validity and Reliability in Qualitative Research
137
Levels of Measurement
138
Quantitative Variables
Box 4.11: Unit/Level of Measurement/Analysis?
139
140
Categorical Variables
141
Turning Categorical Variables Into Quantitative Ones
143
Units of Analysis and Levels of Measurement
145
Measurement in the Real World: Trade-offs and Choices
146
What Will It Cost?
146
Is It Ethical?
146
How Will It Affect the Quality and Rate of Responding?
147
The Validity-Reliability Trade-off
147
Use an Established Measure or Invent a New One?
147
Gaming and Other Behavior Responses to Measurement
148
Multiple Dimensions—or Just One?
148
Conclusion: Measurement Matters
149
Box 4.12: Critical Questions to Ask About Measurement
149
Box 4.13: Tips on Doing Your Own Research: Measurement
150
Exercises
150
Chapter 5. Sampling
154
Learning Objectives
155
Gauging the Fallout From Hurricane Katrina
155
Generalizability
156
Population of Interest, Sampling, and Representativeness
156
Generalizing Beyond the Original Population of Interest
157
Are Relationships More Generalizable?
158
Replicating Research and Meta-Analysis
159
Generalizability of Qualitative Studies
160
Basic Sampling Concepts
161
Population, Sample, and Inference
161
Census Versus Sample
162
How to Select a Sample: Sampling Frames and Steps
163
Problems and Biases in Sampling
Box 5.1: Likely Voters Versus Actual Voters
164
165
Coverage Problems
166
Nonresponse Problems
167
When Do Coverage and Nonresponse Problems Cause Bias?
168
Box 5.2: Steps in Assessing Coverage and Nonresponse Bias
170
Ethics of Nonresponse
Box 5.3: Nonresponse Bias in 2020 U.S. Presidential
Election Polls?
Nonprobability Sampling
170
171
172
Voluntary Sampling
172
Convenience Sampling
173
Snowball Sampling
173
Quota Sampling
173
Online Sampling
174
Purposive Sampling and Qualitative Research
176
Random (Probability) Sampling
178
The Contribution of Random Sampling
178
Random Sampling Versus Randomized Experiments
179
Simple Random Sampling
179
Sampling Variability
180
Sampling Distributions, Standard Errors, and Confidence Intervals
Confidence Intervals (Margins of Error)
Box 5.4: Relationship Between Various Precision Measures
Sample Size and the Precision of Government Statistics
Determining How Large a Sample You Need
Box 5.5: What Is the True Sample Size?
180
182
183
184
185
187
Sampling in Practice
186
Systematic Sampling
186
Stratified Sampling
188
Disproportionate Sampling (Oversampling)
190
Poststratification Weighting
Box 5.6: An Evaluation of 2016 Election Polls in the U.S.
Sampling With Probabilities Proportional to Size
Cluster and Multistage Sampling
Box 5.7: Design Effects: Complex Survey Sampling Corrections
Random Digit Dialing Sampling
Sampling and Generalizability: A Summary
192
193
194
194
197
197
198
Box 5.8: Critical Questions to Ask About Sampling in Studies
198
Box 5.9: Tips on Doing Your Own Research: Sampling
199
Exercises
Chapter 6. Secondary Data
200
202
Learning Objectives
203
Tracking a Global Pandemic
203
Quantitative Data Forms and Structures
204
Quantitative Data Versus Quantitative Variables
204
Structures of Quantitative Data
205
Micro, Aggregate, and Multilevel Data
Box 6.1: Unit of Observation Versus Unit of Analysis
205
206
Time Dimension of Data
207
Metadata
210
Where Do Quantitative Data Come From?
210
Administrative Records
Adapting Administrative Data for Research
210
211
Box 6.2: How Organizations Can Make the Most of Their
Administrative Data
213
Vital Statistics, Crime Reports, and Unemployment Claims
213
Data for Purchase
214
Ethics of Administrative Record Data
214
Aggregate Data Tables
215
Where to Find Aggregate Tables
215
Aggregate Time-Series and Panel Data
217
Public Use Microdata
217
Know the Major Surveys in Your Field
217
Accessing and Analyzing Public Use Data
224
Data Archives
225
Ethics of Public Use Microdata
225
Secondary Qualitative Data
Ethics of Using Existing Qualitative Data
Big Data
226
227
227
Volume, Velocity, Variety—and a Lack of Structure
227
Analyzing Big Data
228
Ethics of Big Data
228
Linking Data
228
Some Limitations of Secondary Data
229
Does Data Availability Distort Research?
229
When to Collect Original Data?
230
Conclusion
230
Box 6.3: Critical Questions to Ask About Secondary Data
230
Box 6.4: Tips on Doing Your Own Research: Secondary Data
231
Exercises
Chapter 7. Surveys and Other Primary Data
231
234
Learning Objectives
235
Taking the Nation’s Economic Pulse
235
When Should You Do a Survey?
236
Do You Know Enough About the Topic?
236
Does the Information Exist Already in Another Source?
236
Can People Tell You What You Want to Know?
237
Will People Provide Truthful Answers?
237
Steps in the Survey Research Process
237
Clarify the Purpose
237
Identify the Population and Sampling Strategy
238
Develop a Questionnaire
238
Pretest Questionnaire and Survey Procedures
238
Recruit and Train Interviewers
239
Collect Data
239
Enter and Prepare Data for Analysis
240
Analyze Data and Present Findings
240
Modes of Survey Data Collection
240
Intercept Interview Surveys
240
Household Interview Surveys
241
Telephone Interview Surveys
242
Mail Self-Administered Surveys
244
Group Self-Administered Surveys
246
Online Surveys
247
Establishment (Business or Organization) Surveys
249
Panel or Longitudinal Surveys
249
Choosing or Mixing Modes
250
Crafting a Questionnaire
251
Starting Off
251
Closed-Ended Versus Open-Ended Questions
252
Box 7.1: Questionnaire Composed of
Open-Ended Questions
Question Order
Box 7.2: Comparing Opening Questions
253
253
254
Some Advice on Question Wording
256
Some Advice on Response Formats
258
Physical and Graphical Design
260
Put Yourself in Your Respondent’s Shoes
261
Ethics of Survey Research
261
Informed Consent
261
Pushing for a High Response Rate
262
Overburdening Respondents
262
Protecting Privacy and Confidentiality
262
Surveying Minors and Other Vulnerable Populations
263
Making Survey Data Available for Public Use
263
Other Ways to Collect Primary Data
264
Trained Observation
264
Scientific Instruments
266
Computer Code and Data Extraction Algorithms
267
Conclusion
267
Box 7.3: Critical Questions to Ask About Surveys and Other
Primary Data
268
Box 7.4: Tips on Doing Your Own Survey
268
Exercises
269
PART III: STATISTICAL TOOLS AND
INTERPRETATIONS
Chapter 8. Making Sense of the Numbers
272
Learning Objectives
273
“Last Weekend I Walked Eight”
273
Units, Rates, and Ratios
274
What Units?
274
Rates, or Why Counts Often Mislead
275
Box 8.1: Relevant Comparisons—Are Anesthesiologists
Prone to Addiction?
276
Percent Change and Percentage Point Change
276
The Strangeness of Percent Change on the Return Trip
277
Rates of Change
277
Odds
277
Prevalence and Incidence
278
Statistics Starting Point: Variables in a Data Set
279
Distributions
280
Distribution of a Categorical Variable
280
Distribution of a Quantitative Variable
281
Measures of Center: Mean and Median
Box 8.2: Mean: The Formula
When to Use Median? When to Use Mean?
Measures of Spread and Variation
Standard Deviation
Box 8.3: Standard Deviation: The Formula
283
284
284
285
285
286
Pay Attention to the Standard Deviation, Not Just the Mean
287
Standardized (z) Scores
287
Quantiles: Another Way to Measure Spread
288
Relationships Between Categorical Variables
289
Cross-Tabulation
289
Relative Risks and Odds Ratios: Another Way to Show
Relationships in Categorical Data
291
Adjusted and Standardized Rates: When to Use Them
293
Relationships Between Quantitative Variables: Scatterplots
and Correlation
293
Scatterplots
293
Correlation
295
Box 8.4: Correlation: The Formula
Relationships Between a Categorical and
a Quantitative Variable
Box 8.5: Which One Is the Dependent Variable? Which One Is
the Independent Variable?
Simple Regression: Best-Fit Straight Line
296
296
297
297
Box 8.6: Simple Regression: The Equations
299
Interpreting the Regression Coefficient (Slope)
299
Box 8.7: Steps for Interpreting a Regression Coefficient
300
Can a Regression Coefficient Be Interpreted as a Causal Effect?
300
Changes Versus Levels
301
R-Squared and Residuals: How Well Does the Line Fit the Data?
302
Practical Significance
303
Practical Significance Is a Matter of Judgment
304
Effect Size
304
Steps for Assessing the Practical Significance of Effects and Differences
305
Statistical Software
305
Spreadsheets
306
Statistical Packages: SAS, SPSS, Stata, and R
306
Specialized Modeling and Matrix Language Programs
306
Conclusion: Tools for Description and Causation
306
Box 8.8: Tips on Doing Your Own Research:
Descriptive Statistics
307
Exercises
Chapter 9. Making Sense of Inferential Statistics
308
314
Learning Objectives
315
But Is It Significant?
315
Statistical Inference: What’s It Good For?
316
The Sampling Distribution: Foundation of Statistical Inference
317
What a Sampling Distribution Looks Like
317
The Standard Error (of a Proportion)
319
The Standard Error (of a Mean)
320
Confidence Intervals
321
Univariate Statistics and Relationships Both Have Confidence Intervals
321
Confidence Intervals Reflect Only Some Sources of Error
323
Calculating a Confidence Interval (Margin of Error) for a Proportion
323
Calculating a Confidence Interval (Margin of Error) for a Mean
324
How Big Does the Sample Size Need to Be? Getting the Precision You Want
326
Significance Tests
328
Falsification and the Logic of Significance Testing
329
Running a Significance Test
330
p Values
330
Significance Tests for Simple Regression
332
Chi-Square Test of Cross-Tabs
334
Other Test Statistics
335
Statistical Significance, Practical Significance, and Power
336
Combinations of Statistical and Practical Significance
336
Box 9.1: Sources of Statistical Significance and of
Statistical Insignificance
339
Failing to Recognize a Difference: Type II Errors
339
Power
340
Multiple Comparison Corrections
341
Sample Size Calculations for Significance Tests
341
Adjusting Inference for Clustering and Other Complex Sampling
342
Issues and Extensions of Statistical Inference
343
Inference With a Nonprobability Sample: What Does It Mean?
343
Bootstrapping: Inference for Statistics With No
Standard Error Formulas
344
Equivalence and Noninferiority Tests
344
Bayesian Inference
345
Conclusion
Box 9.2: Tips on Doing Your Own Research:
Inferential Statistics
Exercises
Chapter 10. Making Sense of Multivariate Statistics
345
346
347
350
Learning Objectives
351
Multiple Regression: The Basics
351
Multiple Regression for Prediction
353
Box 10.1: How to Run a Multiple
Regression Using Software
354
Box 10.2: Steps for Predicting With Regression
354
The Danger (and Necessity) of Out-of-Sample Extrapolation
355
R-Squared and Adjusted R-Squared
355
All Else Held Constant: A Bit More Mathematics
356
Multicollinearity
356
Standardized Coefficients: The Relative Importance of
Independent Variables
358
Inference for Regression
359
Standard Error of the Coefficient
359
Confidence Intervals in Regression
360
Confidence Interval of a Predicted Value
361
Significance Testing in Regression
361
Influences on Inference in Multiple Regression
362
Categorical Independent Variables
363
Dummy Variables in Regression
363
Categorical Variables With More Than Two Possible Values
364
Box 10.3: Representing a Categorical Variable With More
Than Two Categories: Diabetes Example
Interpreting the Coefficient of a Dummy Variable
Analysis of Variance (ANOVA)
Box 10.4: Interpreting the Coefficient of a Dummy Variable
Interactions in Regression
365
366
366
367
368
How to Use and Interpret Interaction Variables
368
Interactions With Quantitative Variables
370
Always Include Both Main Effects
370
Functional Form and Transformations in Regression
370
How to Fit a Curved Relationship
371
How to Interpret Coefficients When a Variable Is Logged
371
The Value of Robustness and Transparency
373
Categorical Variables as Dependent Variables in Regression
373
Linear Probability Model
373
Logistic and Probit Regression
374
What If the Dependent Variable Has More Than Two Categories?
375
Beware of Unrealistic Underlying Assumptions
375
Which Statistical Methods Can I Use?
375
Other Multivariate Methods
376
Path Analysis
376
Factor Analysis
378
Structural Equation Modeling
379
Multilevel Models
380
Time Series and Forecasting
382
Panel Data Methods
383
Spatial Analysis
383
Limited Dependent Variables
383
Survival Analysis
384
Machine Learning
384
More Multivariate Methods Not Covered
385
Conclusion
Box 10.5: Tips on Doing Your Own Research: Multivariate Statistics
Exercises
386
386
387
PART IV: STRATEGIES FOR CAUSATION
Chapter 11. Causation
390
Learning Objectives
391
Family Dinners and Teenage Substance Abuse
391
Correlation Is Not Causation
Alternative Explanations of a Correlation
Box 11.1: Children Who Have Frequent Family Dinners Less
Likely to Use Marijuana, Tobacco, and Drink Alcohol
Reverse Causation
Box 11.2: Directed Acyclic Graphs (DAGs)
Common Causes
391
392
393
393
394
394
Bias From a Common Cause
395
Bias From Reverse Causation: Simultaneity Bias
396
Some More Correlations That May—or May Not—Imply Causation
396
Box 11.3: Get in the Habit of Thinking of Alternative Theories
397
Box 11.4: Recent Evidence on Family Dinners and Teen
Substance Abuse
399
Causal Mechanisms
399
Indirect and Direct Causal Effects
400
Context and Moderators
401
Arrows, Arrows Everywhere
401
Why Worry About the Correct Causal Model?
402
Box 11.5: How to Talk (or Write) About Causation
and Correlation
Evidence of Causation: Some Critical Clues
403
403
There Is a Correlation (Association)
404
The Cause Happens Before the Effect
404
The Correlation Appears in Many Different Contexts
404
Box 11.6: Prominent Epidemiologists Discuss
Replication and Causation
405
There Is a Plausible Mechanism
405
There Are No Plausible Alternative Explanations
406
Other Influences Are Accounted for in the Analysis
406
There Is Qualitative Evidence of a Mechanism
407
The Correlation Is Not Just a Chance Coincidence
408
Detective Work and Shoe Leather
408
Self-Selection and Endogeneity
Self-Selection
408
409
Endogeneity
409
Aggregation Bias and the Ecological Fallacy
410
The Counterfactual Definition of Causation
410
Potential Outcomes
411
If We Only Had a Time Machine
411
Can Really Good Prediction Replace Causal Inference?
412
Experimentation and Exogeneity: Making Things Happen
412
Can Exercise Cure Depression?
413
Why Experimentation Beats Passive Observation
413
Exogeneity: Intervening in the World
414
Box 11.7: Exogenous or Endogenous? It Depends on
the Dependent Variable
415
Box 11.8: The Meaning of Exogeneity and
Endogeneity in Structural Equation Modeling
416
Control: Holding Things Constant
Box 11.9: The Many Uses of the Term Control
417
417
Experimentation: The Basic Logic
418
Ethical Limitations of Experiments
418
Experimentation, Policy, and Practice
419
Conclusion: Tools to Probe Causation
419
Box 11.10: Critical Questions to Ask About Causation
420
Box 11.11: Tips on Doing Your Own Research: Causation
420
Exercises
421
Chapter 12. Observational Studies
424
Learning Objectives
425
Private Versus Public Schools
425
What Is an Observational Study?
426
The Gold Standard for Description—But Not
for Causal Estimation
426
Limitations of an Observational Study
427
Control Variables
427
How Control Variables Help Disentangle a Causal Effect
427
Why These Control Variables?
428
How Did Control Variables Change the Estimate
of a Causal Effect?
429
Matching
429
Individual-Level Matching
429
Aggregate Matching
431
Control Variables: An Empirical Example
431
Step 1: Speculate on Common Causes
432
Step 2: Look for Differences
433
Step 3: Stratify by Control Variables
433
Omitted Variable Bias
435
Box 12.1: Are the Police Fair, or Not? Layered Cross-Tabs for
Categorical Dependent Variables
Expanding the Choice of Control Variable
Box 12.2: Omitted Variables—and the Bias They Cause—
by Any Other Name
How to Choose Control Variables
436
437
438
440
What’s Driving the Independent Variable?
440
Causal Diagrams as a Guide for Choosing
Control Variables
442
Beware of Using Intervening Variables as Controls
443
Colliders: Bias in Causal Estimates From
Sample Selection
444
Empirical Approach to Choosing Controls
446
Causes That Can Be Ignored—Without Bias
446
Unmeasured Variables, Proxies, and Data
447
Choosing Good Control Variables Depends
on Your Question
447
Various Purposes of Control Variables
448
Owning Your Interest in Causation—
But Not Overclaiming
448
Epidemiological Approaches to Observational Studies
449
Prospective and Retrospective Cohort Studies
449
Case-Control Studies
451
Conclusion: Observational Studies in Perspective
453
Box 12.3: Critical Questions to Ask About Observational Studies
With Control Variables
453
Box 12.4: Tips on Doing Your Own Research: Observational
Studies With Control Variables
454
Exercises
454
Chapter 13. Using Regression to Estimate
Causal Effects
458
Learning Objectives
459
Cigarette Taxes and Smoking
459
From Stratification to Multiple Regression
460
Using More Than One (or Two) Control Variables
460
Control Variables That Are Quantitative
460
From Description to Causation: The Education-Earnings
Link Reconsidered
461
Multiple Regression: Brief Overview and Interpretation
463
How Multiple Regression Is Like Stratification: An Illustration
463
Specification: The Choice of Control Variables
465
Does Greenery Affect Birth Outcomes?
467
Step 1: Speculate on Common Causes
468
Step 2: Examine the Relationship Between the Independent
Variable of Interest and Potential Common Causes
469
Step 3: Implement Control Variables Through Multiple Regression
470
Interpreting Multiple Regression Coefficients as Effects of Interest
471
Practical Significance: Is the Effect Big Enough to Care About?
472
Further Topics in Regression for Estimating Causal Effects
473
How Controls and Omitted Variables Influence Estimated Effects
473
Box 13.1: Predicting the Direction of Omitted Variables Bias
474
Interactions, Functional Forms, and Categorical Dependent Variables
475
A Focus on One Causal Effect or Many
476
When Is Low R-Squared a Problem?
476
Software Doesn’t Know the Difference, But You Should
477
Control Variables With Exogenous Independent Variables:
The Gender Earnings Gap
477
Box 13.2: The Life Expectancy of Taxi Drivers: Prediction Versus Causation
478
The Gender Earnings Gap
478
The Gender Earnings Gap Depends on What Is Held Constant
479
Can Control Variables Analyses Show Discrimination?
Box 13.3: Tips on Interpreting Results Tables
Other Multivariate Techniques for Observational Studies
482
483
483
Propensity Score Matching
483
Machine Learning
485
Conclusion: A Widely Used Strategy, With Drawbacks
486
Box 13.4: Critical Questions to Ask About Studies That
Use Multiple Regression to Estimate Causal Effects
486
Box 13.5: Tips on Doing Your Own Research: Multiple Regression
to Estimate Causal Effects
486
Exercises
Chapter 14. Randomized Experiments
487
492
Learning Objectives
493
Time Limits on Welfare
493
Florida’s Family Transition Program: A Randomized Experiment
494
Random Assignment: Creating Statistical Equivalence
Random Assignment in Practice
495
496
Box 14.1: True Randomization or Haphazard?
497
Statistical Equivalence: A Look at the Data
497
Why Random Assignment Is Better Than Matching or Control Variables
499
Findings: What Happened in Pensacola
499
Box 14.2: Nobel Prize for Experimental Approach to
Alleviating Global Poverty
The Logic of Randomized Experiments: Exogeneity Revisited
Statistical Significance of an Experimental Result
The Settings of Randomized Experiments
501
502
503
504
Lab Experiments
505
Field Experiments
505
Box 14.3: Practical Difficulties in a Field Experiment
About Online Education
506
Box 14.4: Social and Policy Research Organizations
That Specialize in Randomized Field Experiments
507
Survey Experiments
507
Generalizability of Randomized Experiments
509
Random Assignment Versus Random Sampling
509
Volunteers and Generalizability
511
The Ideal Study: Random Sampling, Then Random Assignment
511
Box 14.5: Time-Sharing Experiments for
the Social Sciences (TESS)
Limited Settings
Generalizability of the Treatment
Box 14.6: The RAND Health Insurance Experiment
512
513
513
514
Support Factors and Causal Cakes
515
Generalizability in the Long Run
516
Variations on the Design of Experiments
517
Cluster Randomization
517
Arms in an Experiment
518
Levels of a Treatment: Probing a Dose-Response Relationship
519
Factors in an Experiment: Probing Interactions
520
Within-Subjects (Crossover) Experiments
521
Matching and Stratifying (or Blocking)
521
Artifacts in Experiments
The Hawthorne Effect and Demand Characteristics
Placebo Effect and Blinding
Box 14.7: The Perry Preschool Study
522
522
522
524
Contamination
524
Demoralization and Rivalry
524
Noncompliance and Attrition
525
Analysis of Randomized Experiments
525
Balancing, Control Variables, and Covariates
526
Sample Size and Minimum Detectable Effects
527
Heterogeneous Treatment Effects
528
Preregistration of Analysis
528
Intent to Treat and Treatment of the Treated Analyses
529
Box 14.8: The Moving to Opportunity Demonstration
Ethics of Randomized Experiments
531
532
Something for Everyone: The Principle of Beneficence
532
Informed Con
