CNA Case Studies: An Inside Look at Great Analysis

A Deep Dive into Data

For the first several years of his career at CNA, Ronald Filadelfo helped the Navy find the most feared denizens of the deep at that time: Soviet submarines. Three decades later, the Navy still calls on Filadelfo for scientific analysis when it worries about what lurks below the surface. But Filadelfo no longer specializes in anti-submarine warfare. And he and the Navy aim to protect — not attack — the objects of his many studies: whales.

CNA’s deep dive into the issue of marine mammals and sonar began in 2005, when Vice Admiral J. Dan McCarthy asked the research organization to bring objective data to a subject awash in anecdotal evidence. Both the Navy and the public had become concerned about recent mass strandings of beaked whales that had coincided with naval sonar operations overseas. Filadelfo was impressed that McCarthy wanted the kind of brutally objective analysis that was CNA’s heritage, not a whitewash. "If I'm hurting whales, I want to know it," he told the researchers.

It may seem odd that the vice admiral chose a research organization without a single marine biologist on staff, a group better known for its history of operations research on weapons tactics. But it was a strategic calculation. "He was pretty sharp," says Filadelfo. "He realized it was fundamentally a mathematical study — classic operations research."

Filadelfo and his colleagues homed in on the Mediterranean Sea, and spent months digging through naval archives to identify the dates and locations of every exercise that would have used sonar over a 43 year period. He also plotted every mass stranding of beaked whales in the sea over that same period. With these two sets of data, he performed what mathematicians call "bootstrap" and "Poisson" analyses. Starting with the data on mass strandings during the times with no naval sonar activity — the background level — the analyst extrapolated to predict the "natural" likelihood of finding a mass stranding on the sonar days.

Filadelfo could see that in the absence of military activity, there was a high probability of finding at most a single mass stranding during all of those sonar periods combined. To find more than two strandings would be highly unlikely. But beaked whales had beached themselves five times during those sonar-use periods. The unhappy news was that the correlation between sonar activity and mass strandings in the Mediterranean was very high.

The Navy not only accepted the findings — and used them with the intention of planning safer exercises — they gave Filadelfo permission to take the results public. He published the study in the journal Aquatic Mammals with coauthors from Woods Hole Oceanographic Institution and Harvard Medical School. "That study really got us a name in this business," says Filadelfo.

The Navy also came back for more. Filadelfo, now the leader of CNA’s Energy and Environment team, has participated in more than 20 marine mammal research projects for the Navy. Some followed a similar methodology to the Mediterranean study, but with very different results. In several locations, he found no correlation at all between sonar activity and beached whales.

For example, when the Navy faced a lawsuit for endangering whales in Southern California, Filadelfo was asked to look for correlations there. That study required a more advanced mathematical model, because no mass strandings had been recorded, despite heavy use of sonar near California Navy bases. Filadelfo had to correlate using individual washed-up whales. He could find no evidence that those whale strandings were associated in space and time with Navy sonar use.

"So of course the million-dollar question is: Why do you see it in some places, and why do you not see it in others?" ponders Filadelfo. Though further research has not taken him to a conclusive answer, it suggests that underwater topography and the relative positions of whales and naval ships may hold an explanation. The analyst, who holds a Ph.D. in physical oceanography, says that where the underwater topography is steep, and whales forage close to shore, ships operating just seaward of whales might endanger them. "You ping the sonar, and you could scare them and run them up on the beach, stranding them," he explains. In other locations, where marine mammals and ships are farther offshore, the animals have more options for moving out of the way. "CNA’s research has been critical in getting us to the point that we could have an intelligent dialogue on sonar," says Vice Admiral McCarthy, now retired from the Navy.

Many of CNA’s marine mammal studies have addressed the Navy’s desire to meet regulations protecting whales and dolphins. Some have borne an uncanny resemblance to the origins of CNA, helping the Navy find German U-boats in World War II. "In this work I've gone back to perhaps the most famous CNA report of all, ‘Search and Screening,’" says Filadelfo. "It's one of the bibles of military operations analysis from the glory days of our history." Models of search created for that war are directly relevant in part because U-boats were largely found visually, just as spotters on Navy vessels today search for whales and dolphins. The decibel levels of sonars are adjusted according to their distance from marine mammals.

In recent years, Filadelfo has frequently worked with a colleague he calls "the god of agent-based modeling at CNA," Andrew Ilachinski, to develop more sophisticated models of ship-whale interactions. Those models of search helped inform a study that examined, among other questions, how many patrol boats would be required to effectively determine that no marine mammals were present before exploding underwater mines in anti-mine exercises. They found that three boats were always sufficient for spotting whales and dolphins, while one boat could never cover enough water, and two boats might or might not, depending upon the size of the charge and other variables.

Ilachinski and Filadelfo are just beginning to explore the possibilities of a new multi-agent model Ilachinski has dubbed "Whale Sim." By mimicking the decisions taken by animals individually and within pods, it promises to add a new level of realism to future evaluations. Filadelfo says that when a Whale Sim demo of whales moving away from ships was shown to officials at the National Oceanic and Atmospheric Administration, "they were blown away by it."

This body of work is not just a scientific and mathematical challenge for Filadelfo; it’s a passion. The analyst goes diving in his spare time, and marvels at his underwater encounters with dolphins. "Ocean environmental protection is one thing I care about," he says. "Doing this marine mammal work has been a breath of fresh air for me."