This is an archived article that was published on sltrib.com in 2013, and information in the article may be outdated. It is provided only for personal research purposes and may not be reprinted.
After airport security blunders enabled the 9/11 terrorists to kill nearly 3,000 people, Congress created the Transportation Security Administration to protect against attack in U.S. skies. To help analyze its progress, TSA says it decided to track how many potential weapons travelers surrender at each U.S. airport.
But The Salt Lake Tribune found huge holes in that TSA data that raise questions about the information's value in charting the effectiveness of airport security.
For example, TSA stopped counting confiscated firearms for three years. In 2008, it quit counting eight of 13 major categories of banned items, including knives, ammunition and box cutters (the weapon used by 9/11 terrorists).
The hit-and-miss data show TSA does not know how many total prohibited items are surrendered, so it cannot accurately track rates of such activity nationwide or at specific airports.
"If TSA is continually changing the parameters of how they test, truly they have no idea of how they are doing," said Douglas Laird, head of an aviation-security consulting firm. He is the former security director of Northwest Airlines and a former Secret Service agent.
But TSA says the data variations are not a big deal.
"TSA continues to be responsive to the threats that it knows are out there," said TSA spokeswoman Lorie Dankers. "Data collection has been fluid over time" as different threats required changes in prohibited items and focus — leading to frequent changes in data collection.
—
TSA has no explanation • At its creation, TSA decided, in its own words, to track surrendered items in a database "as a tool through which TSA captures and analyzes daily operational information to achieve performance goals."
The record for items surrendered came in 2005, when the agency recorded collecting 15.7 million potential weapons nationwide. That amounts to one for every 46 passengers that year — enough to provide a banned item to everyone on a fleet of nearly 38,000 full Boeing 747s.
But in 2010, the last year for which TSA data were released, records show TSA collected a mere 109,270 banned items — a 99 percent decrease from five years earlier and a record low. That amounts to one item for about every 6,700 passengers.
The vast difference comes not from a dramatic increase in travelers leaving banned items at home but mostly from changes in what TSA counted.
For example, in 2005, TSA was in the middle of a three-year period — from July 26, 2004, to June 29, 2007 — when it did not include firearms in its data. The agency confirmed that it left firearms out of the count during this period, but it did not say why. Dankers said in an interview that she also had no explanation.
An even bigger reason for variations in the data is that, beginning in 2008, TSA discontinued counting most types of banned items — including sharp objects, knives and blades, tools, ammunition and gunpowder, replica weapons, dangerous objects, clubs, bats and bludgeons, box cutters and lighters.
"It became very labor intensive to document these items," Dankers said. "Over the course of time, not every prohibited item was logged for the sake of efficiency. ... TSA screens between 1.7 [million] and 1.8 million people per day. We're dealing with large numbers of people at 450 federalized airports nationwide."
TSA decided it made sense to track only the "most significant items," she said, including guns, explosives, fireworks and "flammables/irritants."
—
Changing lists • Some items also disappeared from tracking because they were once banned in the wake of 9/11 but later allowed. Others were added to the banned list because of emerging threats, but they were not always counted.
One example of an item dropped from prohibited lists are cigarette lighters. In 2005, while they were still prohibited, TSA recorded collecting 9.3 million of them — about two-thirds of all items surrendered that year. The next year it seized 11.5 million of them. By 2010, that dropped to zero.
An example of an item added to the banned list based on an emerging risk assessment are liquids — which were prohibited abruptly on Aug. 10, 2006, when intelligence showed imminent threats of terrorists aiming to blow up aircraft traveling between Great Britain and the United States.
Eventually, TSA allowed small amounts of liquids that fit in 3.4-ounce containers placed in a 1-quart clear, zip-top bag, with one bag allowed per passenger.
But TSA never tracked in its database how many oversize liquids were surrendered — which Dankers says is one of the most commonly barred items now seen.
"The prohibited-items list was implemented, evaluated and adjusted as necessary," Dankers said, as focus and threats changed.
"We look at those items that pose a threat," Dankers said. "We look at a metric to determine at what rate people are bringing those through a checkpoint. We continue to evaluate that. It is a dynamic process."
—
Stopping terrorists: "It's impossible" • Frequent changes in what is counted render any year-to-year comparison meaningless.
Some security experts say such holes and variances make the database a poor tool to gauge TSA performance.
"It says that they're very sloppy about figuring out if what they're doing is any good," said Bruce Schneier, a security-technology consultant and author.
Laird, the former security director for Northwest Airlines, says it shows TSA keeps changing its mind about what is important to track, and that makes it difficult to measure progress.
Both experts say that even if the data were more consistently collected, that information still might not be a great way to measure air-travel security.
"Think about it this way: The goal of the TSA is to prevent terrorist plots on airplanes. The way to do that is to stop terrorists with viable plots from getting on airplanes. They can't do that; it's impossible," Schneier said. "There's no comprehensive lists of terrorists, and there's no comprehensive list of terrorist plots."
So, he added, "Instead, they invent another goal and hope it's close to the real goal. The new goal is to prevent a list of banned items from getting on airplanes. … Even if they ban 100 percent of guns and bombs, is there any evidence that's a worthwhile goal? I have no idea and neither do they."
Laird said "99.9999" percent of what is surrendered at airports is not from terrorists "but from people who forgot what was in their luggage" and likely doesn't say much about discouraging terrorists.
Ultimately, passenger checkpoints are just one of many layers of TSA security, said Dankers. "We have air marshals, we have canine teams, we have inspectors, we have intelligence gathering," and even "behavior detection" officers who watch body language for anything amiss — and items surrendered at airports are just one part of a mosaic of protection measures.
Mark Lewis, deputy federal security director for Utah, said maybe the best measurement "is when nothing bad happens" — and no terrorists have killed anyone on U.S. planes since 9/11. "Regardless of the reporting, the fact that we are focused on the real threat allows us to keep the traveling public safe."
—
Big numbers at small airports • One surprise from the data is that small airports have much higher rates of surrendered items per boardings than do large airports during the entire 2002-10 period.
Small general aviation airports with some passenger service had 143.67 prohibited items per 10,000 boardings during that time, double the rate for large hub airports. Similarly, non-hub airports primarily serving passenger flights had a rate of 149.3 items, compared with 131.97 for small hubs and 73.8 for large hubs.
Dankers didn't want to venture an explanation for the pattern of much lower rates of confiscation at larger, busier airports.
Asked whether higher rates at smaller airports could be a result of less harried, more thorough screenings, Dankers said: "The standard at every one of the federalized airports is the same. Our officers undergo the same training at a very small airport or one of the largest airports in the country."
Among all airports nationally, Vernal Regional Airport had the second-highest rate of surrendered items in the period: 515.6 per 10,000 boardings. The only airport with a higher rate was Moses Lake, Wash., at 553.53.
Kelly Harvey, manager of the Vernal airport, which offers daily flights to Denver, said he has no idea why it has such a high rate. But a factor might be that passengers at such small airports fly less often and may not be as familiar with what is banned in baggage.
He said he receives a lot of calls "about what they can and cannot bring. So I think it's just a matter of the passenger knowing what is and is not prohibited. It seems like it is always evolving and changing almost on a daily basis, so it's hard to keep up with."
Other small airports in Utah also have high rates: Wendover, 288.14 items per 10,000 boardings; St. George, 206.24; Moab, 202.5; and Cedar City, 150.28. (Scheduled passenger service at Provo and Ogden airports had not begun in the period for which data were provided.)
Though the rates are high, the actual numbers of potential weapons are small. Items surrendered at all those small Utah airports combined in the period totaled 19,306 — a figure dwarfed by Salt Lake City International Airport's haul of 452,590.
Salt Lake City's surrender rate was 53.4 items per 10,000 boardings — 20 percent lower than the average among the nation's 28 large hub airports. (New York's John F. Kennedy International was the highest at 128.99, and Charlotte/Douglas International Airport in North Carolina was the lowest at 26.46).
Items surrendered in Salt Lake City included: 66 firearms; 149,134 lighters; 143,535 knives with blades shorter than 3 inches; 7,676 knives with blades longer than 3 inches; 103,900 "sharp objects"; 29,821 tools; 10,321 flammables/irritants; 3,421 ammunition and gunpowder; 1,452 box cutters; 1,017 "dangerous objects"; 708 clubs, bats and bludgeons; 604 fireworks; and 83 explosives.
ldavidson@sltrib.com TSA provided data two years after request
In October 2010, The Salt Lake Tribune made a Freedom of Information Act request for TSA data about numbers of prohibited items, by type, surrendered at security checkpoints at each U.S. airport. Almost exactly two years later, the TSA provided data for 2005-2010. While The Tribune requested data in electronic format, TSA provided paper printouts, which required entering it by hand into computers for analysis. The newspaper combined this information with data for 2002-04, which was obtained earlier.