The Ultimate Balancing Act: Creating Balanced Groups Using Propensity Score
-----
Multi-group cohort studies are common in clinical and health psychology. In these studies, participants are members of two (or more) groups–those exposed to some potential risk factor and those not exposed–then the groups are compared on outcomes. Because participants are not randomly assigned to groups, causal inferences are limited in these designs. Some previous attempts have been made, particularly with retrospective cohort studies, to balance groups on baseline characteristics; however, these methods do not work well for prospective studies with sequential enrollment, as characteristics of all eligible participants must be known prior to enrollment. Montoya proposes and evaluates a propensity score-based method of balancing groups on baseline characteristics during sequential enrollment, allowing for balanced groups even with incomplete information about eligible participants at the time of enrollment. The method uses iteratively estimated propensity score matching with holdout groups, allowing for case-by-case decisions to be made regarding recommendation for enrollment in the study. The proposed method is comprised of four modular steps repeated at regular intervals throughout the recruitment process: 1) Estimate propensity scores using all eligible individuals, 2) Match participants who are previously enrolled or newly eligible based on propensity scores, 3) Create groups based on percentiles of propensity scores, 4) Retain a fixed number of unmatched individuals in each propensity group, 5) Enroll all matched or retained individuals, decline all individuals unmatched or not retained. This method was used for recruitment in a study examining cardiovascular outcomes in individuals with posttraumatic stress disorder (PTSD) and trauma-exposed controls without PTSD. Montoya analyzed the data from this study to demonstrate the method and its real-world performance. Performance of the matching method was excellent in this case, where a test of the unmatched sample shows strong imbalance (p < .001), but the matched sample shows close balance (p = .99). The method can also account for dropout and other issues in data collection. Future research includes implementation in R, Monte Carlo simulations, and additional developments of matching methods.
-----