Example of Propensity Score Matching
This example shows how to use the Causal Treatment personality in the Fit Model platform to analyze the effect of union membership on household income with propensity score matching. Data for this example are simulated to represent an observational study where employment information is collected from each subject. It is assumed that union membership was not assigned randomly.
1. Select Help > Sample Data Folder and open Minimum Wage Treatment.jmp.
2. Select Analyze > Fit Model.
3. Select Household Income from the Select Columns list and click Y.
4. From the Personality list, select Causal Treatment.
5. Select Union Member from the Select Columns list and click Treatment.
6. Select the Treatment Model tab.
7. Select Worker Age, Industry Type, Years Education, and Work Experience and click Add.
8. Click Run.
Figure 14.14 Fit Model Launch Window That Shows Treatment Model Tab
The Model 1 report appears, but does not show the propensity score matching results. The Model Launch Control Panel must be used to set the parameters for propensity score matching and to produce a report with the corresponding results.
9. Open the Advanced Models section in the Model Launch Control Panel.
10. Select Nearest Neighbors as the Propensity Score Matching Method.
11. Select Propensity Score as the Matching Statistic.
12. Click Go.
Figure 14.15 Model Launch Control Panel with Matching Parameters Set
A data table called Matched.jmp appears. This data table adds Original Row number, Match ID, Distance, and Propensity Score columns to the original data table. The table is arranged so that untreated observations appear directly below their treated matches. Rows with untreated observations show the distances in propensity scores from the treated counterparts. This table has 1,812 rows. The Number of Matches per Observation was set to 1, meaning that each treated observation was matched to one untreated observation. There were 907 treated observations in the original data set, so only one treated observation was left unmatched. This was because no untreated observations were within the caliper distance from this treated observation. You might also notice repeated observations in the data table, given the Match with Replacement parameter was selected.
Since each treated observation is matched to an untreated observation and many untreated observations are excluded from the analysis, only the average treatment effect on the treated (and not the average treatment effect) can be estimated using this data set.
13. Minimize the Matched.jmp window.
In the report window, a Model 2 report is displayed. To focus on this report, remove the Model 1 report.
14. Click the Model 1 red triangle and select Remove Fit.
Now only the Model 2 report is shown. Scroll to the Propensity Score Matching Summary section. The Propensity Score Distribution by Treatment plot shows the distribution of propensity scores as well as which observations were matched or unmatched for each treatment level. The red dot in the treatment group is the previously mentioned observation that was not matched. You can see that it has the lowest propensity score in the treated group.
Figure 14.16 Propensity Score Distributions by Treatment Level Plot
The Analysis for Matched Data section shows the results of the propensity score matching model. The estimate of the average treatment effect on the treated is the estimated coefficient of the fixed effect of the union membership variable in the parameter estimates table. With this modeling technique, the ATET estimate is $964.65 with a 95% confidence interval of -$431.95 to $2,361.00. Since the 95% confidence interval covers $0, under the proper causal assumptions, you conclude that there is not evidence for an effect of union membership on household income.
Figure 14.17 Propensity Score Matching Effect Estimates
You can compare this estimate to the one produced by IPWR. Moving to the Analysis of Causal Effects section, you see that the ATET estimate is -$1017.00 with a 95% confidence interval of -$2,197.00 to $163.61. (The IPWR method uses 0 as the treatment reference level while the matching method uses 1.) Under the proper causal assumptions, you conclude again that there is not evidence for an effect of union membership on household income. If matching is successful, the ATET estimates from both estimation methods should typically be close to one another.