The Effect of Credentialing on Sailor Advancement, Retention, and Unemployment

The Navy began its Credentialing Opportunities On-Line (COOL) program in FY07. COOL offers sailors the opportunity to earn credentials—such as licenses, certifications, or memberships—related to their occupations or collateral duties. The program pays for application, exam, and membership fees for credentials that have been mapped to at least 80 percent of the duties specified in the sailor’s rating or critical skill Job Duty Task Analysis (JDTA). In addition to having the relevant rating or skill, a sailor must have at least one year remaining on his or her contract at the time of application for the credential and must satisfy any other credential requirements, such as experience and education.

The Assistant Secretary of the Navy for Manpower and Reserve Affairs (ASN (M&RA)) asked CNA to determine the extent to which COOL enhances sailor advancement and retention and reduces the probability that a sailor requires unemployment compensation on separation from the Navy.

Participation in COOL

Our COOL data include 30,478 individuals and 72,885 applications, covering the period between August 30, 2007, and April 21, 2014. Of these, 8,170 sailors participated at least once involuntarily as part of the Information Assurance Workforce. We do not include these sailors in our analyses. Six ratings represent over 80 percent of applicants and 76 percent of all applications:

• Culinary Specialist (CS)
• Cryptologic Technician (Collections) (CTR)
• Electronics Technician (non-Nuclear Field) (ET)
• Hospital Corpsman (HM)
• Information Systems Technician (IT)/IT (Submarine) (ITS)
• Master-at-Arms (MA)

No other rating represents at least 2 percent of the total applicants. Early on, sailors in the CS and MA rating represented the largest percentage of participants, but since FY11 IT/ITS sailors have dominated COOL applications. The large increase in IT/ITS applications results from COOL exams being administered during IT A-School beginning in December 2010. We have evidence that other sailors are taking COOL exams during training, such as HMs in certain C-Schools.

When we control for relevant factors, we find statistically significant differences in participation based on gender (women are 5 percent less likely to participate, all else equal) and race (Asian/Pacific Islanders are 23 percent and blacks are 9 percent more likely to participate than otherwise similar Caucasians). The greatest difference, however, is based on education; sailors with associate degrees are 37 percent more likely to participate in COOL than sailors with high school diplomas, all else equal. This is the first indication we found that sailors who participate in COOL may be more motivated or more able sailors.

Advancement

We expect that the primary way that participation in COOL would affect the speed of advancement is the extent to which it helps sailors perform better on the occupational component of their advancement exams. We found that sailors who successfully participate in COOL—where success is defined as passing all COOL exams—before their first E4 advancement exam outperform their peers on the occupational component of the advancement exam, even when we control for typical measures of ability, such as Armed Forces Qualification Test and education.

We conclude that participation in COOL on its own does not directly benefit sailors in terms of advancement, however. Sailors who passed all of their COOL exams had no greater improvement in occupational exam scores on consecutive E5, E6, and E7 advancement exams than their peers who did not participate; if the participation itself improved sailors’ knowledge related to their ratings, we would expect participants to have a greater improvement than their peers who did not participate, all else equal.

These results are additional evidence that successful participants in COOL are likely more motivated and/or more able than their peers who do not participate, especially relative to those who participate but do not pass all of their COOL exams.

Retention

We find that retention initiatives and other factors that existed throughout the period of COOL make it impossible to correctly estimate the true effect of COOL participation on retention. We do not have a sufficient number of sailors in any rating who participated consistently across all zones and over the entire COOL experience to control for all of the relevant factors that affect retention. This is especially important because of the unusual circumstances under which COOL has operated—a period of historically high unemployment, which resulted in an increase in the proportion of sailors wishing to reenlist at the same time that the Navy was downsizing and restricting reenlistments. We have some indication, however, that sailors in the CS and MA ratings who successfully participated in COOL had higher Zone A retention during the era when Perform To Serve (PTS) imposed the greatest reenlistment limits, all else equal, but these results may be more a reflection of which sailors the Navy allowed to reenlist rather than of sailor preferences, and they may not be robust in a more steady-state retention environment.

Unemployment

For sailors who separated while in Zone A, we find that, all else equal, those who passed all of their COOL exams were 6.5 percentage points less likely to receive Unemployment Compensation for Ex-Servicemembers (UCX) than their peers who failed one or more COOL exams, and sailors who failed one or more of their COOL exams were 8.4 percentage points more likely to receive UCX than sailors who did not participate in COOL. This latter difference persists for sailors who separated in Zone B; those who failed at least one COOL exam were 10.0 percentage points more likely to receive UCX than their similar peers who did not participate in COOL. Clearly, COOL is not responsible for reducing their ability to find employment; we conclude that this is another indication that unsuccessful COOL participants differ in some unobservable traits that make them less able or less motivated than their peers.

We also looked at the UCX experience of sailors in the MA rating, the only rating with a sufficient number of participants in COOL who separated in each zone. For these sailors who separated in Zone B, successful COOL participants were 6.3 percentage points less likely than nonparticipants to collect UCX, all else equal.

We found that participation in COOL helps to lower total UCX payments for some sailors, especially those who separated in Zones C and E. Using the average cost per voluntary participant of about $590, the cost of COOL almost completely offsets the cost of UCX for participants in Zone E, and the net savings to the Navy for sailors in Zone C is over $800.

Zone A sailors who failed COOL exams received more UCX than their peers who did not participate, but we conclude that this is further evidence that these sailors are likely less able or less motivated than their peers and hence may have more difficulty in finding civilian employment.

Recommendations

Overall, we find some evidence that successful COOL participation is associated with less UCX, and it may have beneficial retention effects during periods of more typical unemployment, and when the Navy is not downsizing. However, our findings must be considered to be preliminary for a number of reasons. First, we do not have complete information regarding whether sailors actually earned credentials, only that they applied, and, in most but not all cases, whether they passed a particular exam. The effect of COOL on retention could depend on this outcome; they may reenlist in order to complete all of the exams necessary for one or more credentials but separate on receipt of the desired credentials. Therefore, our first and most important recommendation is that the Navy should revise the COOL data collection protocol so that it indicates whether the sailor was awarded the credential and the date of the award.

We conclude that COOL could serve as a signal for sailor motivation and ability, especially for sailors who pass all of their COOL exams. This could be useful in identifying sailors of the highest quality for purposes of evaluations, Selection Board consideration, and so on. The signal may be twofold; sailors who participate could be more motivated to seek out the credential for a particular purpose (selfimprovement, career enhancement, civilian opportunities), and they may be more motivated and able to do well on measures of competency. The less voluntary that participation is, the more muted the first source of the signal becomes.

Based on our finding that some sailors who pass all of their COOL exams are less likely to require UCX, the Navy may want to consider encouraging sailors who are denied reenlistment in-rate to pursue a COOL credential before separating. For these sailors, the Navy would need to waive the requirement of a minimum of one year remaining on their contract in order to participate in COOL.

Finally, we conclude that more accurate measures of the effect of COOL on advancement, retention, and unemployment will not be possible until COOL has been in existence longer and under more normal conditions. It is not clear how sailors learn about COOL, and participants so far have been “early adopters”; as such, they may exhibit different behaviors than those who participate in COOL in the future. It is also not clear what the long-term effects of PTS (now called C-Way) or mid-career retention boards will be on sailor retention behavior and their motivation for participating in COOL. Therefore, we recommend that the Navy revisit the effectiveness of COOL in a few years.