Research

The table below contains basic descriptive statistics for the five most frequently administered ACAT disciplines. The statistics were calculated for the overall raw scores obtained over the 6-year period coinciding with the current comparison groups. Only the scores obtained from graduating senior majors, the target population, were included.

The raw scores, number correct divided by number of items, were used to measure the underlying parameters of the items. The actual ACAT scores are based on a weighted item scale translated to a standard score.

The data indicate that, for four of the five disciplines, the raw total scores over the most recent 6-year period approximate normal distributions. This finding is essential to allowing the data to be transformed to standard scores. The data also support the need for the extensive revision to the ACAT in Political Science that is scheduled to begin during 2007-08.

A frequently used measure of reliability when neither test-retest nor split-half techniques can be applied is odd-even. This divides the items into two groups and calculates the coefficient of reliability assuming that if the two groups are measuring the same domain, the scores should be strongly correlated.

The following table lists the calculated coefficients for odd-even reliability by individual content area for each of the ACAT disciplines. The raw scores, number correct divided by number of items, were used for the coefficients to insure the underlying reliability of the items themselves. The actual ACAT scores are based on a weighted item scale.

Because the content areas appearing on each version of the ACAT vary from campus to campus, the reliability coefficients are calculated at the level of the content areas rather than for the test as a whole.

Selected Research Bibliography

	Golden, A. An approach to "In-House" test development for major field assessment. In K. McGuinness (Ed.), Legislative action & assessment: Reason & reality (pp. 167-176). Washington, DC: American Association of State Colleges and Universities, (1986). (ERIC Document Reproduction Service No. ED 293 387).
	Cone, Al. Low Tech/High Touch Criterion-Based Learning. Psychological Reports, 1988, 63, 203-207
	Crain, R. An examination of the construct and context validity of the Area Concentration Achievement Test. Unpublished Master's thesis. Austin Peay State University, Clarksville, TN, 1989.
	Golden, A. Project for Area Concentration Achievement Testing. Washington, DC: AASCU/ERIC Model Programs Inventory Project, (1989). (ERIC Document Reproduction Service No. ED 306 863).
	A National Survey of Requirements for the Psychology Major (1990)
	Golden, A. PACAT: A national project in cooperative major field assessment. In R. McCormick (Ed.), The 1990 Montclair assessment conference: Strategies and prospects for the decade (pp. 80-83). Upper Montclair, NJ: Montclair State College, (1991).
	Golden, A. and Squire, D. The third alternative. Assessment Update, 1991, 3 (2), 10-11.
	Smith, M., Draper, G., & Bradley, J. Assessment Instruments in the Major: humanities, natural sciences and social sciences. Knoxville: Clearinghouse for higher education assessment instruments, 1993.
	Gender Differences in Psychology Content Area Mastery Nationwide by Graduating Seniors (1996)
	Female "flight" to postgraduate education (1997)
	Gender Differences in Research Methods Across Disciplines (1997)
	Fleming, S. and Golden, A. Higher education: Some gaps for women close while other widen. In Gender and race on the campus and in the school: Beyond affirmative action. Washington, DC: American Association of University Women, 1997.
	Kidda, M., & Golden, A. Using senior exit examinations to inform curriculum change in an academic department (Ed., Gender and Race on the Campus and in the School: Beyond Affirmative Action. Washington, DC: American Association of University Women, 1997.
	Significant Psychology Score Increases from Pretest to Post-test Across the Major (1998)

Statistical Evaluations of the ACATs

Selected Research Bibliography