Research for CAT

Syndicate content
Data Quality for the Helicopter Maintenance Phase of the Marine Corps Job Performance Measurement Project
January 1, 1992
All large-scale data collection efforts must contend with the issue of data quality. This research memorandum examines the quality of data collected for the helicopter maintenance portion of the Marine Corps Job Performance Measurement project and describes measures taken to minimize the effect of questionable or missing cases.
Read More | Download Report
Uniqueness of a Psychomotor Construct to ASVAB
January 1, 1992
Computerized administration of the Armed Services Vocational Aptitude Battery (ASVAB) potentially offers several benefits over the current paper-and-pencil administration. One benefit is the possible expansion of predictor tests beyond the dominant mathematical and verbal orientation of the ASVAB. This research memorandum first examines the comparability of measurement for the computerized and paper-and-pencil tests. Second, several new tests that measure psychomotor skills (hand-eye coordination and manual dexterity) are examined for their ability to expand the range of dimensions currently assessed by the ASVAB, as well as to improve the prediction of performance in mechanical maintenance jobs.
Read More | Download Report
An Analysis of CAT-ASVAB Scores in the Marine Corps JPM Data
September 1, 1991
The Defense Department developed a Computerized Adaptive Testing (CAT) version of the Armed Services Vocational Aptitude Battery (ASVAB). During the Mechanical Maintenance phase of the Marine Corps Job Performance Measurement (JPM) project, CAT-ASVAB was administered to over 1,400 Marines in Automotive Repair and Helicopter Repair occupations. The scores of these Marines were analyzed to assess the reliability of CAT-ASVAB, the potential effects of test item compromise, and how the use of computers has affected the nature of speed tests. This research memorandum presents the results of the analysis.
Read More | Download Report
Approximations for Three Statistical Functions
April 1, 1990
Statistical work connected with the computerized adaptive testing version of the Armed Services Vocational Aptitude Battery often involves the density and cumulative functions of the normal distribution, and the logistic function. In long, iterative calculations, computation of these functions can be time consuming. This research memorandum presents piecewise linear and cubic approximations for the normal cumulative distribution function, the logistic function, and the normal density function. The approximations are as accurate as library functions, and much faster in computing speed.
Read More | Download Report
Calculating the Performance Gain Due to Improved Predictive Validity
April 1, 1990
Composites of test scores are used in the selection and classification of enlisted personnel. If a selection composite is replaced by a new composite with higher predictive validity, mean performance of the recruits increases. Formulas for calculating the performance gain require simplifying assumptions. Using an example, this research memorandum shows that the formulas are very sensitive to violations of these assumptions and hence are not dependable enough for use in operational decisions.
Read More | Download Report
Some Issues in the ECAT and CAT-ASVAB Projects
February 1, 1990
The Department of Defense is developing a computerized adaptive-testing version of the Armed Services Vocational Aptitude Battery (ASVAB). This or some other version of the ASVAB may be enhanced by the addition of new, computerized subtests. A cost/benefit analysis has estimated a benefit of $450 million per year as the result of such enhancement. This paper questions the operational relevance of any such estimate. The paper describes ways in which the validation study of new tests needs to be expanded. It also discusses pros and cons of adaptive testing.
Read More | Download Report
Two Consequences of Improving a Test Battery
October 1, 1988
Replacement of a paper-pencil test battery with a computerized adaptive version is likely to increase reliabilities of the subtests. This leads to an increase in the variances of composite scores, and to lower mean scores for subgroups whose average scores are already below those of the general population. These results are illustrated with a computer simulation.
Read More | Download Report
Estimating Reliabilities of CAT-ASVAB Subtests
January 1, 1988
An experimental computerized adaptive testing (CAT) version of the Armed Services Vocational Aptitude Battery (ASVAB) has been developed and administered, and a new version is under preparation. It is important that each CAT-ASVAB subtest be at least as reliable as its paper-pencil counterpart. This report presents two methods for estimating subtest reliabilities of the CAT version of the ASVAB, and illustrates them using data from the experimental version. These methods can be used with later versions.
Read More | Download Report
Dimensionality of the Math Knowledge Item Pool for the Accelerated CAT-ASVAB Project
August 1, 1987
The theory underlying computerized adaptive tests assumes that all items for a given subtest measure a single dimension. This assumption was examined for the math knowledge items in the item pool developed for the Armed Services Vocational Aptitude Battery. Departures from the assumption were found to be minor.
Read More | Download Report
Properties of Some Bayesian Scoring Procedures for Computerized Adaptive Tests
August 1, 1987
The computerized adaptive version of the Armed Services Vocational Aptitude Battery will use a Bayesian procedure for computing test scores. Properties of three common Bayesian procedures are examined in this research memorandum. The results show that the procedures are almost equally reliable and that reliability drops if item parameters change from paper-pencil to computerized administration.
Read More | Download Report