Presented at the Knowledge Is Power Program (KIPP) School Summit in San Antonio, TX in July 2008. This presentation explores the participation and performance of African American students in the SAT and AP programs and use data to inform ways to better support and encourage achievment and college success.
The main goal of this study was to illustrate and provide some direction for dealing with the complexities of propensity score matching within different multilevel contexts. Special attention is given to how procedures typically applied in a non-hierarchical setting may be modified to properly reduce the expected bias in the estimated treatment effect of a high school-level intervention on college-level outcomes. In particular, students self-selected into a high school level intervention and the outcome of interest was observed within the context of the college or university in which students subsequently enrolled. A simulation was prepared to delineate the relevant issues and to demonstrate the effects of ignoring a hierarchical nesting of subjects. An empirical example of an evaluation of a large-scale, national educational program is given.
Presented at the Advanced Placement Annual Conference (APAC) in Orlando, FL in July 2012. This presentation reviews concepts central to achieving equitable AP access and success for all willing and academically prepared students. We analyze trends in participation and performance by race/ethnicity from the AP Report to the Nation and consider the ethics of data in order to understand challenges to interpretation as well as behavioral consequences of various metrics. The intended audience is AP Coordinators, school administrators, and those who recognize the importance of building equity from the ground up.
Presented at the College Board National Forum, October 26, 2011. Choosing a college major is challenging enough, without stopping to consider the impact it has on a student’s college experience and career choice. To provide support during this major decision, participants in this session will develop strategies to facilitate students in making an appropriate major choice. Based on research and recommendations from high school and college educators, participants will discuss interventions and best practices. Participants will focus on factors related to major choice and retention (with an emphasis on STEM majors), and will investigate some noteworthy findings related to students with undeclared majors.
The purpose of this study was to examine the effects of Advanced Placement (AP) exam participation and performance on college grades for courses taken in the same subject area as students’ AP Exam(s). Students’ first-year college subject area grade point averages (SGPAs) were examined in nine subject areas: mathematics, computer science, engineering, natural science, social science, history, English, world language, and art and music. Using cross-classified multilevel modeling for each subject area separately, and controlling for gender, racial or ethnic identity, socioeconomic status and prior academic ability, as average AP Exam score in each subject area increased, expected SGPA increased.
Presented at AERA in Denver, CO in April 2010. This paper discusses issues related to the design and psychometric work for mixed-format tests—tests containing both multiple-choice (MC) and constructed-response (CR) items. The issues of validity, fairness, reliability and score consistency can be addressed but for mixed-format tests there are many decisions to be made and no examination or examination program faces exactly the same choices. This paper raised some issues and used the AP experience to illustrate the types of questions that need to be addressed and outlined the strong test development processes that need to be in place to address the issues.
The purpose of the research was to compare the college performance of three groups of AP students who took the AP Exam and either earned course credit, did not earn course credit, or earned course credit but elected to take the entry-level college course to three groups of Non-AP student groups matched on SAT scores and high school rank in 10 AP subject areas. In addition, the performance of the AP groups was also compared to matched groups of students who were concurrently enrolled in a college course in the same subject area as the AP students. Students’ records for four entering classes (1998–2001) at the University of Texas at Austin were analyzed. The results showed AP students who earn course credit consistently outperform their matched Non-AP group on most of the college outcome measures.
This study focused on the relationship between students’ performance in AP English Language, Biology, Calculus, and U.S. History, and their subsequent college success. For each AP Exam studied, students were divided into three groups according to their AP Exam performance (no AP Exam taken, score of 1 or 2, and a score of 3 or higher). Subsequent college success was measured by students’ first-year college grade point average (FYGPA), retention to the second year, and institutional selectivity. Results indicated that, even after controlling for students’ SAT scores and high school grade point average as measures of prior academic performance, students with an AP score of 3 or higher outperformed the other two groups. Additionally, students with an AP score of 1 or 2 tended to outperform students with no AP scores except in terms of FYGPA.
Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation discusses a methodology for directly connecting evidence-centered assessment design (ECD) to score interpretation and use through the development of Achievement level descriptors.
Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes the methodology that was used with subject-matter experts (SMEs) to articulate the content and skills important in the domain, and then the iterative processes that were used to articulate the claims and evidence to represent the targets of instruction for AP courses, and by extension, the targets of measurement for the AP exams.