The most important factor that
we have found in obtaining student cooperation
is communication. When students understand what
is expected of them and why, and when
everyone knows well in advance that the
assessment will take place, scores will tend to
more equitably reflect student achievement.
Faculty in participating departments
voluntarily submit items which they believe to
be appropriate for graduating senior majors and
which reflect mastery of the content base of the
discipline. In this way, faculty have a direct
continuing voice in the content of the
instrument. The resulting information has
greater credibility and is more likely to lead
to meaningful application.
The project provides material to interested
faculty to assist with item preparation. The
ACATs use a random sample of the items submitted
in each area. The selection is further refined
to include only those items with acceptable
psychometric parameters.
Item writing is not required but is an
important aspect of departmental participation
in the assessment process. Departments assist
with test revision and improvement by
identifying weaknesses in the version they use
and submitting items designed to correct them.
ACATs typically are revised every 2-3 years and
new items are continuously tested to determine
their usefulness.
The ACAT is in use at 4 year institutions
nationwide. Participating departments cover a
broad range, some graduating fewer than 5 majors
a year and others graduating several hundred.
Both public and private institutions are
participants. A representative list of recent
participants may be viewed by clicking here.
A limited number of ACATs (agriculture, biology, general
business, criminal justice) are calibrated for use at 2 year institutions.
There are several ways to do this. The first
is to model the test after the requirements of
your major. This will provide the best
assessment of the degree to which students are
meeting the content mastery goals implicit in
your requirements. Another approach is to test
the broadest range of areas which can be
included. Often departments will be able to
identify content areas which, while not
constituting separate courses, are nevertheless
being acquired in the major. Psychology,
Biology, History, Business, and Criminal Justice
programs must choose the number of areas
which will appear on their ACAT (curriculum) and
specify what those areas will be (content
areas).
Finally, a pragmatic approach dictates the
length of the test by the manner in which it
will be administered. A test to be given during
a single class period, for example, may only
contain 4 (48 minutes) or 5 (60 minutes) areas.
In all cases, the instrument should be
structured to provide the kind of information
most likely to be used by the department. To
request a sample or to order tests, please
specify the discipline, curriculum, and list the
specific areas to be included. For assistance
with making your selection, please call us toll-free
(866-680-2228) or email us.
Yes. If the answer is left blank, there is no
penalty. An incorrect answer, however, will
result in a penalty which is weighted
differently according to the difficulty level of
the item. The penalty is greatest for extremely
easy items while it constitutes only about 10%
of the value of an extremely difficult item.
This weighting impacts much more greatly on
those students who answer incorrectly items that
the majority of students taking the ACAT answer
correctly. An incorrect answer to an item that
is often missed has far less impact.
In general, we recommend that students
attempt to answer questions if they are
reasonably sure of the answer or can reduce the
number of alternatives to 3 or less.
Beginning in January 2008, ACATs are scored weekly. All answer sheets received
and ACAT Online sessions completed prior to the close of business on Thursday will be
included in score reports released the following Monday. Special scoring
which does not conform to this schedule can be
accomplished upon request for a fee.
Departments may elect to change their test
configuration only once during any testing year
(June-May). Because of the way that different
versions of the test are identified
electronically for scoring, the computers cannot
differentiate among ACAT versions used by the
same department during a single testing year.
Departments changing a test configuration should
clearly indicate this on their purchase order
and provide a complete list of areas to appear
on the new test including areas not being
changed. A department
wishing to change its test configuration during
the testing year but after administering it at
least once during the same year will be charged
an administrative fee for managing the
change. Please refer to the table of special fees
for the current amount.
Yes, we can accommodate departments wishing
to use multiple versions of the ACAT. For
example, one institution has two different
tracks in their psychology major and uses two
different tests, one for each track. However,
because doing this requires the assignment of
multiple institution codes to a single
department, we must be notified in advance
that a department wishes to exercise this
option. The department should also provide 2 or
3 word descriptions of the student groupings
that will be used to identify their institution
codes in the administration manual.
The standard scores provide a means by which
to compare your students' performance to that of
the overall 5 year national sample. A standard
score has a fixed mid-point or average, in this
case 500. That does not change from one year to
the next. The standard scores also have a fixed
standard deviation of 100 points. The standard
deviation allows us to determine percentiles.
For example, 68% of the students in the national
sample will score within 1 standard deviation of
the average. Put another way, 68% of the scores
on the test should fall between 400 and 600.
Each content area has its own 6-year
reference group consisting of all students
taking that particular content area subtest
regardless of the overall test configuration
used. The ACAT assumes that the content taught
in a specific area is more or less independent
of the required breadth of the curriculum
offered. In some disciplines, you will notice
that the reference group size is different for
each content area. This is why.
There is also an overall standard score and
percentile on the report. This is the measure of
the overall performance of your students
compared to all students in the 6-year sample
group taking a test with the same number of
content areas. This score can be interpreted as
a comparison to departments with similar
curricular requirements for breadth. Nothing is
perfect, however, and the applicability of this score depends to some
extent on the degree to which a department's
choice of test length and content areas actually
matches their curriculum.
Some departments want to know how well their students are
doing as a percent correct, much like the score that would result from taking
an in-class exam. Care should be taken in interpreting these
scores, however. The ACAT assumes that 70%-80%
of the students in a department have learned
70%-80% of the material presented on the test.
This will produce "ideal" scores of
between 49% and 64% correct. While these might
be considered low on an in-class exam designed
to meet the goals of an individual course, they
are well suited to a nationally administered
instrument trying to assess much broader goals.
The raw scores are weighted on an item by
item basis and equated for difficulty across
different forms of the test before begin
converted to standard scores. Therefore, it is
not possible to derive one from the other given
only the information contained within the score
report.
First of all, look at the top of the report.
If is says that it is an interim report, it will
contain only the scores for the most recent
group of students for whom you have submitted
answer sheets. The final report mailed out
during the first half of June should contain
scores for all students tested during the
June-May testing year.
Some students fail to provide adequate
information on their answer forms. Students must
provide their names and a student ID number. Failing to provide a proper booklet
number or test form number may also prevent
scoring. In these cases, you should receive a
"Scoring Exceptions Report" with your
score report. This will list all answer forms
which could not be scored and the reason why
they could not be scored.
Finally, if none of the above applies to your
situation, please call us toll-free
(866-680-2228)
and have ready the student's name and ID number used for the test
administration. Students often incorrectly mark
their discipline and/or institution codes and we
will need to search for them manually.
No. The ACAT as a pretest is designed to
assess only an overall cohort. It would be
inappropriate to use the ACAT, designed for
curriculum evaluation, as a "gateway"
for a major. There is also a strong possibility
of introducing an inadvertent bias towards
students who perform poorly on the ACAT. For
these reasons, we do not provide individual
scores for pretests. Otherwise, the same score
report as that for graduating seniors is sent to
departments.
