The main goal of this study was to illustrate and provide some direction for dealing with the complexities of propensity score matching within different multilevel contexts. Special attention is given to how procedures typically applied in a non-hierarchical setting may be modified to properly reduce the expected bias in the estimated treatment effect of a high school-level intervention on college-level outcomes. In particular, students self-selected into a high school level intervention and the outcome of interest was observed within the context of the college or university in which students subsequently enrolled. A simulation was prepared to delineate the relevant issues and to demonstrate the effects of ignoring a hierarchical nesting of subjects. An empirical example of an evaluation of a large-scale, national educational program is given.
Academic intensity or academic rigor of students’ high school curriculum is positively related to several college outcomes including the avoidance of remediation and graduation attainment (Adelman, 1999, 2006; Adelman, Daniel, & Berkovits, 2003). However, research on academic rigor has been limited possibly due to the difficulty in obtaining a quantitative measure applicable across schools and districts. This study is an attempt to create an index of academic rigor using self-reported course work data that would assist in providing information on the academic preparation of over one million graduating high school seniors each year The current study uses the SAT® Questionnaire (SAT-Q) that students complete when registering for the SAT exam to construct an academic rigor index (ARI). The SAT-Q asks students detailed questions on English, math, science, social science/history, and foreign/classical language course work completed during high school. The relationship between course participation and first-year GPA (FYGPA) was investigated using approximately 68,000 SAT takers students who fully completed the SAT-Q and attended one of the 110 four-year colleges and universities participating in an SAT validity study. Based on this data, the ARI was constructed on a 0-25 scale equally weighted between each of the five subject areas. Once the ARI was constructed a series of analyses were conducted to assess the relationship between the index and other concurrent measures of high school performance (HSGPA and SAT scores) and between the index and measures of college performance (enrollment, grades, and retention). The results indicated that students who took more rigorous courses in high school attained better grades, achieved higher SAT scores, and were more likely to enroll in college. Moreover, these students were also more likely to matriculate to a four-year college, attain higher college grades, and be retained to their second year.
Presented at the Advanced Placement Annual Conference (APAC) in Orlando, FL in July 2012. This presentation reviews concepts central to achieving equitable AP access and success for all willing and academically prepared students. We analyze trends in participation and performance by race/ethnicity from the AP Report to the Nation and consider the ethics of data in order to understand challenges to interpretation as well as behavioral consequences of various metrics. The intended audience is AP Coordinators, school administrators, and those who recognize the importance of building equity from the ground up.
Presented at the College Board National Forum, October 26, 2011. Choosing a college major is challenging enough, without stopping to consider the impact it has on a student’s college experience and career choice. To provide support during this major decision, participants in this session will develop strategies to facilitate students in making an appropriate major choice. Based on research and recommendations from high school and college educators, participants will discuss interventions and best practices. Participants will focus on factors related to major choice and retention (with an emphasis on STEM majors), and will investigate some noteworthy findings related to students with undeclared majors.
This Info To Go publication summarizes the research report: Is AP Exam Participation and Performance Related to Choice of College Major? The full report can be viewed by clicking herePrevious research has found a positive relationship between AP participation and performance with various college outcomes. Building on this work, the current study investigated the relationship between AP participation and performance with choice of college major. Specifically, this study examined whether students who take an AP Exam in a certain content domain are more likely to major in that domain than students who did not take an AP Exam in that area, controlling for relevant student characteristics. Results reveal a positive relationship between AP participation and majoring in a related field in college. This was true across all content areas examined; however, the effect was stronger for some areas than others. For example, there was a strong link between taking an AP Exam in computer science and majoring in computer sciences in college. Alternatively, taking an AP Exam in humanities or social sciences was not as strong of an indicator of a student subsequently majoring in that respective area in college. Additionally, students’ AP scores in that content domain were also related to their likelihood of majoring in that domain. Results showed that students who took no AP Exams were more likely to be undeclared.
The College Board offers fee reductions to students based on eligibility for free and reducedprice lunch in an attempt to introduce the benefits of AP® Exam participation to students most at risk in the education system. This report examined college outcomes of low-SES students with a focused investigation comparing students who took an AP Exam and received a fee reduction to students who took no AP Exams. Students were classified as low-SES if they reported that their annual household income was $30,000 or less. The results indicated that students who completed an AP Exam using a College Board–issued fee reduction had higher four-year college-going rates, retention rates, and first-year grade point average (FYGPA) than did their non-AP peers. Additionally, these results held generally even when the data were disaggregated by demographic variables (gender, ethnicity, parental income, or parental education) or by academic variables (high school GPA or SAT® score). "
The purpose of this study was to examine the effects of Advanced Placement (AP) exam participation and performance on college grades for courses taken in the same subject area as students’ AP Exam(s). Students’ first-year college subject area grade point averages (SGPAs) were examined in nine subject areas: mathematics, computer science, engineering, natural science, social science, history, English, world language, and art and music. Using cross-classified multilevel modeling for each subject area separately, and controlling for gender, racial or ethnic identity, socioeconomic status and prior academic ability, as average AP Exam score in each subject area increased, expected SGPA increased.
Presented at AERA in Denver, CO in April 2010. In today’s education climate, an enormous amount of pressure has been placed on states, school districts, and programs to produce graduates who are prepared to successfully enter, persist through, and graduate from the nation’s universities and colleges. However, much of this research has been limited to one school or has focused on publicly available variables, thus limiting the analyses’ generalizability and relevance. This study investigates the power of various student-level background and academic variables as well as school-level social and academic characteristics in the prediction of college enrollment, persistence, and graduation, demonstrating that characteristics at both levels play a role in the likelihood of reaching these goals. By creating this prediction model, researchers hope to further the field’s understanding of the impact that a student’s high school experience has on his or her likelihood of successfully navigating through college immediately following high school graduation.
Presented at AERA in Denver, CO in April 2010. This paper discusses issues related to the design and psychometric work for mixed-format tests—tests containing both multiple-choice (MC) and constructed-response (CR) items. The issues of validity, fairness, reliability and score consistency can be addressed but for mixed-format tests there are many decisions to be made and no examination or examination program faces exactly the same choices. This paper raised some issues and used the AP experience to illustrate the types of questions that need to be addressed and outlined the strong test development processes that need to be in place to address the issues.
The purpose of the research was to compare the college performance of three groups of AP students who took the AP Exam and either earned course credit, did not earn course credit, or earned course credit but elected to take the entry-level college course to three groups of Non-AP student groups matched on SAT scores and high school rank in 10 AP subject areas. In addition, the performance of the AP groups was also compared to matched groups of students who were concurrently enrolled in a college course in the same subject area as the AP students. Students’ records for four entering classes (1998–2001) at the University of Texas at Austin were analyzed. The results showed AP students who earn course credit consistently outperform their matched Non-AP group on most of the college outcome measures.