Higher Education Brief: Classic Learning Test Concordance with the SAT

July 13, 2023

College Board has received questions from our higher education members about the legislation passed in Florida related to the Classic Learning Test (CLT). We’ve put together this fact sheet to address inquiries specifically related to the concordance published by CLT

Background 

This spring, Florida passed legislation designating the Classic Learning Test as an acceptable standardized test to be used for qualification for state scholarships, and the Florida Board of Governors is expected to approve the use of the CLT for admissions at Florida state institutions.

Executive summary

The College Board recognizes the role that the Classic Learning Test and other assessments can play in a diverse educational landscape.

In April 2023, Classic Learning Initiatives, LLC, published a paper that includes a concordance between the Classic Learning Test and the SAT—The Concordance Relationship Between the Classic Learning Test (CLT) and the Scholastic Aptitude Test (SAT). While the paper makes statements about comparability between the SAT and CLT, the study does not meet industry standards (as developed jointly by the American Educational Research Association, National Council on Measurement in Education, and the American Psychological Association) for an acceptable concordance.

College Board was not involved in the study. Without addressing several methodological and comparability issues, we cannot validate the interpretation and use of concorded CLT-to-SAT scores for high-stakes decisions, like admissions and scholarship awards, based on the published study.

We welcome the opportunity to meet with both ACT and CLT to evaluate the process of conducting a joint and valid concordance study. The results of that study would produce critical information on whether the CLT and SAT are comparable. 

Why valid and accurate test concordance matters 

When scores from different tests are being used to consider students for scholarships or college admission, the scores must be able to be reliably compared to each other to ensure that students taking different assessments aren't advantaged or disadvantaged. A well-formulated concordance links two different tests and allows scores on different exams to be used in a comparable way. College Board and ACT have completed concordance studies together for years using technically sound best practices.  

The concordance study published by CLT does not meet psychometric and industry standards on multiple dimensions. 

Core methodological issues with the published CLT study 

The study does not use a representative sample of test takers.  

  • The study uses 4 different samples totaling about 5,000 students. The report does not cite the demographics of the participants or controls used to ensure representation of all test takers. For comparison, the most recent concordance between the ACT and SAT included 589,753 students, and the sample was nationally representative.
     

  • In addition, the study did not control or minimize the time between when the CLT and the SAT were taken—students could have had multiple years of learning between the 2 tests, which negates the valid comparability of scores.  

The SAT score data were not verified by College Board.  

  • The study indicates students largely self-reported SAT scores, and over 20% of scores used in the study were different than the scores colleges had on record. 
     

  • For the most recent concordance study between ACT and SAT, ACT and College Board provided official, college reportable scores obtained in official, secure administrations of the exams.  

Comparability between the SAT and CLT

A core requirement for a valid and reliable concordance between two tests is that they measure similar content at similar levels of rigor. Analysis of available information about the development and design of the CLT raises questions about the comparability between the two assessments.  

  • The SAT is an achievement test that measures the knowledge and skills students learn in high school that are needed for college and career success. The CLT is aligned to a “classical curriculum.” When 2 tests are constructed around different curricula and standards, a rigorous joint study must be performed to determine if there can be concordance.
     

  • The SAT is a proven, valid predictor of college performance, based on years of published and accessible research and data. CLT has not published evidence of validity or predictiveness of college performance.
     

  • Our preliminary analysis of the skills and knowledge tested on the CLT indicates that the CLT and SAT do not test math at the same grade level. In reviewing a published CLT practice test, we found that 25% of questions were below high school grade level. For example, statistics concepts are not tested on the CLT. As a result, scoring very well on the CLT may not necessarily indicate an equally high score on the SAT, which is aligned to high school grade levels represented in state standards.

Appendix: Comparison of concordance study methodologies

 

ACT-SAT (2018) 

CLT (April 2023) 

Participants 

NCAA, ACT, SAT 

CLT 

Sample size 

589,753 

4,375 for total score; 1,550 for section scores. 

Testing population 

Targeted to all high school students; SAT cohort usually of 1.5M–2.2M students 

Targeted to homeschooled students and students enrolled in private schools and religious charter schools. Between 2016 and 2023, CLT reported 32,615 scores to 24,362 unique students. 

Sample representativeness 

Representative of SAT/ACT test takers, which is generally representative of high schoolers nationally. 

No demographics reported.  

Official scores  

  

ACT and College Board provided official, college reportable scores that were obtained in official, secure administrations of the exams. 

Official scores were not used and most students self-reported their SAT score. For students with scores at a partner college, >20% did not have SAT scores that matched the students’ self-reported scores.  

Shared content map/standards   

Content of both tests selected to assess the most commonly tested HS standards that are predictive of success in first-year college coursework. 

No alignment published. Mission of organization is focused on testing classical curriculum.  

Subgroup evaluations 

Subgroup concordance evaluated and matches total group. 

None reported. 

Time between assessments and testing order 

Sample controlled to minimize for time between assessments and to account for the order assessments were taken. 

No controls cited. 

Joint guidance on use of tables 

CB-ACT publication includes lists of potential uses and limitations.  

No joint statement or guidance on when appropriate to use.