This study examined the validity of the SAT for predicting performance in first-year English and mathematics courses. Results reveal a significant positive relationship between SAT scores and course grades, with slightly higher correlations for mathematics courses compared to English courses. Correlations were estimated by student characteristics (gender, ethnicity, and best language), institutional characteristics (size, selectivity, and control, i.e., private or public), and course content (e.g., calculus, algebra). The findings suggest that performance on the SAT is predictive of performance in specific college courses. Furthermore, stronger relationships were found between test scores and grades when the content of the two were aligned (such as the SAT mathematics section and mathematics course grades, or the SAT writing section and English course grades).
Presented at the Knowledge Is Power Program (KIPP) School Summit in San Antonio, TX in July 2008. This presentation explores the participation and performance of African American students in the SAT and AP programs and use data to inform ways to better support and encourage achievment and college success.
The main goal of this study was to illustrate and provide some direction for dealing with the complexities of propensity score matching within different multilevel contexts. Special attention is given to how procedures typically applied in a non-hierarchical setting may be modified to properly reduce the expected bias in the estimated treatment effect of a high school-level intervention on college-level outcomes. In particular, students self-selected into a high school level intervention and the outcome of interest was observed within the context of the college or university in which students subsequently enrolled. A simulation was prepared to delineate the relevant issues and to demonstrate the effects of ignoring a hierarchical nesting of subjects. An empirical example of an evaluation of a large-scale, national educational program is given.
The current study will explore the validity and potential of using the SAT, in conjunction with HSGPA, to arrive at a predicted FYGPA to improve student retention at four-year postsecondary institutions. Specifically, this study examined whether college students who did not perform as expected (observed FYGPA minus predicted FYGPA) were more likely to leave their institution. Results showed that both under- and over-performing students were more likely to leave college as compared to their academically similar peers who performed as expected. Recommendations for institutions to incorporate this information as part of a cost-effective and efficient detection tool to identify students that may be at risk for not completing their degrees and to help improve institutional retention rates are provided.
This presentation reviews the methodology for setting college and career benchmarks on national assessments. The presentation provides a logical argument for establishing performance level descriptors at the outset of the development and design of college and career ready assessments. The NCME presentation is based on a paper commissioned by PARCC to provide a validation argument and recommendations on establishing PLDs for college and career ready assessments. "
Academic intensity or academic rigor of students’ high school curriculum is positively related to several college outcomes including the avoidance of remediation and graduation attainment (Adelman, 1999, 2006; Adelman, Daniel, & Berkovits, 2003). However, research on academic rigor has been limited possibly due to the difficulty in obtaining a quantitative measure applicable across schools and districts. This study is an attempt to create an index of academic rigor using self-reported course work data that would assist in providing information on the academic preparation of over one million graduating high school seniors each year The current study uses the SAT® Questionnaire (SAT-Q) that students complete when registering for the SAT exam to construct an academic rigor index (ARI). The SAT-Q asks students detailed questions on English, math, science, social science/history, and foreign/classical language course work completed during high school. The relationship between course participation and first-year GPA (FYGPA) was investigated using approximately 68,000 SAT takers students who fully completed the SAT-Q and attended one of the 110 four-year colleges and universities participating in an SAT validity study. Based on this data, the ARI was constructed on a 0-25 scale equally weighted between each of the five subject areas. Once the ARI was constructed a series of analyses were conducted to assess the relationship between the index and other concurrent measures of high school performance (HSGPA and SAT scores) and between the index and measures of college performance (enrollment, grades, and retention). The results indicated that students who took more rigorous courses in high school attained better grades, achieved higher SAT scores, and were more likely to enroll in college. Moreover, these students were also more likely to matriculate to a four-year college, attain higher college grades, and be retained to their second year.
Wayne Camara delivered the 2011 Pearson Distinguished Lecture at the University of Texas at Austin. "Defining and measuring college and career readiness: Establishing validation evidence to support the use of new assessments" addressed how definitions of college and career readiness should be based on the desired outcomes or criteria such as college success and career success. The presentation is based on a paper commissioned by PARCC to provide a validation argument for state consortia involved in developing college and career ready assessments. "
This is an invited presentation before the US Department of Education and NCES in February 2O12 on prevention of cheating on computer based tests. The presentation focuses on the major unique threats to data integrity and test scores for computer administered tests. "
In the fall of 2009, the College Board conducted a curriculum survey to gather information on the curricula and institutional practices of high schools and colleges in the United States. The primary objective of the survey was to collect data on the knowledge and skills, or topics, taught in high school classrooms and assess the importance of these topics for institutions of higher education. The College Board periodically reviews the state of K–12 and college curricula as a standard part of the SAT test development process (see example in Milewski, Johnsen, Glazer & Kutota, 2005). The results of the curriculum survey will allow the College Board to validate and ensure that the topics measured on the SAT and SAT Subject Tests™ reflect what is being taught in the nation’s high schools and what college professors consider to be required for college success Curriculum surveys were completed by more than 5,000 high school and college teachers in English language arts (ELA) and mathematics. Surveys were also distributed in biology, chemistry, physics, and history, and the results of these surveys will be reported in later reports. Each survey covered the topics assessed on the SAT, SAT Subject Tests, and the College Board Standards for College Success™ and also inquired about various aspects of course curricula, including the use of assessments. This report presents the results of the English and mathematics surveys, with a focus on the alignment between the SAT and high school and college curricula. The report briefly introduces the SAT, discusses the method used to implement the survey, and concludes with a summary of the results of the ELA and mathematics surveys.
Over the past four decades, there has been incremental growth in computer-based testing (CBT) as a viable alternative to paper-and-pencil testing. However, the transition to CBT is neither easy nor inexpensive. As Drasgow, Luecht, and Bennett (2006) noted, many design engineering, test development, operations/logistics, and psychometric changes are required to develop a successful operational program. Early research on CBT almost exclusively focused on theoretical issues such as improving measurement efficiency by achieving adequate levels of test score reliability using as few items as possible. However, it was soon evident that practical issues — such as ensuring content representation, making sure all examinees have sufficient time to complete the test, implementation of new item types, and controlling the degree to which items were exposed to examinees — needed to be addressed, too. In the past few years, research on CBT has focused on developing models that achieve desired levels of measurement efficiency while simultaneously satisfying other important goals, such as minimizing item exposure and maintaining content validity. In addition, there has been a growing awareness among practitioners that basic CBT research using small samples or simulation studies needs to be vetted using cost-benefit analysis, as well as engineering design and implementation criteria to ensure that feasibility, scalability, and efficiency are evaluated in more concrete ways than by merely reporting a reduction of error variances for theoretical examinee scores (Luecht, 2005a, 2005b). "