>
Officials involved in the Nevada Test Site project on radiation detection made the papers this morning in regard to the inadequacy of their testing, and this afternoon’s release of the full GAO analysis makes them look even worse. The GAO has now released its report on “next generation radiation testing” online. From the summary: “Based on our analysis of DNDO’s test plan, the test results, and discussions with experts from four national laboratories, we are concerned that DNDO’s tests were not an objective and rigorous assessment of the ASPs’ capabilities. Our concerns with the DNDO’s test methods include the following:”
(1) DNDO used biased test methods that enhanced the performance of the ASPs. Specifically, DNDO conducted numerous preliminary runs of almost all of the materials, and combinations of materials, that were used in the formal tests and then allowed ASP contractors to collect test data and adjust their systems to identify these materials. It is highly unlikely that such favorable circumstances would present themselves under real world conditions. Translation: The Department tested the materials, gave the results to the contractors who calibrated their systems to match the data, and then the contractors replicated the DNDO test results.
(2) “DNDO’s NTS tests were not designed to test the limitations of the ASPs’ detection capabilities–a critical oversight in DNDO’s original test plan. DNDO did not use a sufficient amount of the type of materials that would mask or hide dangerous sources and that ASPs would likely encounter at ports of entry. DOE and national laboratory officials raised these concerns to DNDO in November 2006. However, DNDO officials rejected their suggestion of including additional and more challenging masking materials because, according to DNDO, there would not be sufficient time to obtain them based on the deadline imposed by obtaining Secretarial Certification by June 26. 2007. By not collaborating with DOE until late in the test planning process, DNDO missed an important opportunity to procure a broader, more representative set of well-vetted and characterized masking materials.” Translation: The DNDO tests left the “answers out in the open,” giving the contractors a better chance of detecting what had already been detected, rather leading the dog toward the bones.
(3) “DNDO did not objectively test the performance of handheld detectors because they did not use a critical CBP standard operating procedure that is fundamental to this equipment’s performance in the field. Because of concerns raised that DNDO did not sufficiently test the limitations of ASPs, DNDO is attempting to compensate for weaknesses in the original test plan by conducting additional studies–essentially computer simulations. While DNDO, CBP, and DOE have now reached an agreement to wait and see whether the results of these studies will provide useful data regarding the ASPs’ capabilities, in our view and those of other experts, computer simulations are not as good as actual testing with nuclear and masking materials.” Translation: Handheld detectors may have worked if “standard operating procedures” were ignored, and instead of applying the standards the agencies settled for computer simulations in lieu of actual operational testing.
