Research for Personnel Classification Tests

The Joint-Service Performance Measurement Project was initiated in 1981 at the request of Congress to validate the selection test used by the services against measures of job performance. The charter for the project called for an annual report to Congress describing the progress of each service. An additional requirement was for each service to outline its research effort to link its job performance data to the determination of enlistment standards. The material presented in this research memorandum formed the basis for the Marine Corps submission to the sixth annual report.

To make informed decisions concerning the allocation and management of military personnel, policymakers must have information that reflects the competence of their personnel -- the extent to which individuals can perform their job requirements. The Marine Corps Job Performance Measurement Project is a significant research effort to produce measures of job proficiency that will allow for such an absolute interpretation of test scores. This research contribution describes a test construction technique that results in the estimation of competency scores. Although the distribution of competency scores has implicit meaning with respect to a domain of job requirements, policymakers often have little knowledge of whether a particular score is acceptable or not. The interpretation of competency scores can be enhanced by establishing interpretative anchors for various performance levels of the competency score distribution. These interpretative anchors assist policymakers by providing a concrete reference against which to base their decisions and are essential in establishing the linkage of aptitude and job performance in determining enlistment standards.

The computerized adaptive version of the Armed Services Vocational Aptitude Battery will use a Bayesian procedure for computing test scores. Properties of three common Bayesian procedures are examined in this research memorandum. The results show that the procedures are almost equally reliable and that reliability drops if item parameters change from paper-pencil to computerized administration.

The selection of test content and the construction of job performance tests has not followed any specific methodology. For determining qualification standards on the Armed Services Vocational Aptitude Battery (ASVAB), a more absolute interpretation is required of the performance test scores. This report proposes that a domain-referenced approach is necessary for the linkage of job performance and aptitude in determining standards on the ASVAB. This absolute interpretation of test scores, called competency-based measurement, is based upon two critical requirements of the test construction process: detailed specification of the job requirements domain and appropriate sampling of test content from that domain.