Feb 13, 2008

Munger: Was Los Alamos weak link on SNS?

By Frank Munger

The Spallation Neutron Source is one of the world's leading centers for materials research, offering unprecedented levels of neutrons to do experiments. It also has become a symbol of Oak Ridge National Laboratory's success in the 21st century.

The $1.4 billion federal project was completed ahead of schedule and within its budget, a pretty rare feat in government circles these days, and ORNL has leveraged that to its advantage in many ways.

For instance, the project management of SNS is a primary reason Oak Ridge was chosen to host the U.S. involvement in the International Thermonuclear Experimental Reactor. It also was why Oak Ridge was chosen for the first of five Department of Energy-sponsored nanoscience centers, and the SNS played at least a supporting role in ORNL's winning of other projects, ranging from high-performance computing to development of biofuels.

For those for may have forgotten this little tidbit: The Spallation Neutron Source was developed as a partnership of six national laboratories. Besides Oak Ridge, the others were Argonne, Brookhaven, Lawrence Berkeley, Los Alamos and Thomas Jefferson. ORNL, of course, was the host site, and after the work was done, the lab took over as owner and manager.

If there had ever been a grand opening for a grand project, which there wasn't, each of the labs undoubtedly would have been thanked for their contributions.

This week, however, Los Alamos got just the opposite. The New Mexico lab got a spanking.

In a broad-based report, the Government Accountability Office blasted Los Alamos on everything from problems in securing classified data to protection of workers to weaknesses in project management. The SNS came into the conversation on the latter point.

The GAO used some of the work on the SNS to document Los Alamos problems with project management. Los Alamos was responsible for development of the linear accelerator and work on low-level radio-frequency systems.

The report said fabrication problems with the linear accelerator in 2002 resulted in a cost impact of $8 million that had to be overcome with the use of project contingency funds and "offsets." It also noted that ORNL had to take over work on the radio-frequency control system because of Los Alamos design problems and used a system already developed at another lab, Lawrence Berkeley.

"The former ORNL Spallation Neutron Source manager, who is now the laboratory director, told us that problems with these two projects led by LANL could have significantly delayed the overall project," the GAO report said.

The report, of course, was referring to Thom Mason, and I talked with Mason by telephone on Monday and asked about the GAO report. He said the report's descriptions of the SNS issues were accurate, except for a reference to "leaky tubing," which he indicated was probably a reference to the drift-tube linac.

But he was not inclined to call Los Alamos the weak sister on the SNS team.

"We had problems in a number of areas, including the linac, and that's sort of normal in projects, and that's why you have contingency and why you build float into the schedule," he said, noting that the project team was always working on a more aggressive schedule than the one committed to Congress.

Mason said it's true that, if the Los Alamos problems had not been fixed, they could have affected the entire project's success.

"But they were fixed," he said. "The way I look at it, the measure of success in a project is not the absence of problems, because that's not realistic, but how the problems are addressed."

Each of the SNS partners encountered significant issues during the development and construction period, including Oak Ridge, Mason said.

"At some point in time, everyone came under the gun," he said. "The most important thing is we were able to overcome these difficulties."

Senior writer Frank Munger may be reached at 342-6329. His telephone number is 865-342-6329. More information is available on Munger's blog, "Atomic City Underground," at http://blogs.knoxnews.com/knx/munger/


Anonymous said...

ORNL and SNL, both Lockmart run labs, are thriving.

LANL and LLNL, both run by UC/Bechtel are dieing.

Neither LANS nor LLNS have a clue how to bring back morale and help their labs thrive once again. Nor do I think they really care.

LANL/LLNL corporate partners want to get their juicy "for-profit" fees and the LLC executives want to be well compensated. Beyond that, they could care less what happens as long as there are no safety or security infractions to effect the performance fee. When you look around both LANL and LLNL and see what's going on it's hard to remain in denial about the situation.

Anonymous said...

Hard perhaps, but not impossible.

Anonymous said...

Munger got fed a line of shit by Tom Mason. Yes, it is true that LANL really screwed up the linear accelerator and low-level RF control systems. And while there were cost overruns on these two items, the other work that LANL did had cost underruns to the extent that the actual total cost at LANL was less than the budget and funds were returned to ORNL.

BUT, not to worry, all of the LANL managers who oversaw that work have now been promoted, at least to Group Leader if not to Division Director. As usual, at LANL the rule is "screw up, move up!"

But, as far as the project being completed ahead of schedule and under budget, that is baloney. Yes, the project met its milestones. BUT, the project laid timebombs in the process and continues to do so.
The spallation neutron target can only operate at the specified power for a few weeks before it has to be replaced, a two-month effort. The linear accelerator has serious beam loss problems that lead to activation of the items in the accelerator tunnel.

Of the major items of the project, the largest cost overrun (by percentage) was the facility and that was managed by ORNL. The worst technical issue still unresolved is the operational lifetime of the neutron target which was built by ORNL.

And, LANL was not the only lab that had problems. The superconducting cryomodules provided by the Jefferson Laboratory will ALL have to be totally reworked in order to meet their performance specifications.

The basic philosophy of the project was to meet schedule milestones and minimize installed cost without any regard to the later consequences. The entire system was designed and built with no consideration of the reliability and maintainability appropriate for a national user facility.

While the SNS has significant promise, it is far from achieving its design goals.

ORNL and DOE have been complicit in declaring success when both know that is far from the truth.

Anonymous said...

Post this where you wish.

Here's (http://dotearth.blogs.nytimes.com/2008/02/13/federal-lab-says-it-can-harvest-fuel-from-air/)a LANL concept that could have spectacular consequences. I hope that it isn't the usual idiocy like the plasma gasoline engine.

Anonymous said...

"The basic philosophy of the project was to meet schedule milestones and minimize installed cost without any regard to the later consequences. The entire system was designed and built with no consideration of the reliability and maintainability appropriate for a national user facility."

Good post, 12:03 PM

But, this is just the way government-funded science works in the U.S.A. It was explained to me a long time ago that this is a basic distinguishing feature between the U.S. and Europe. The Europeans try to get everything correct before they build something. The Americans throw something together that more or less works and then straighten it out as they use it. When/if reliability and maintainability become important enough to the national users, they should/will apply the pressure to get the funds needed. (If there is anything left after Iraq.) The designers and builders can only do so much with limited time and resources.

"ORNL and DOE have been complicit in declaring success when both know that is far from the truth." That's generally a given, but it would be good to have some more details on this thread, just for the record.

The UC/Bechtel - Lockmart comparison of the first post makes no sense on this issue. SNS started long before Bechtel was a gleam in the the beltway's eye.