LANL leadership rates low in first-year evaluation

By ROGER SNODGRASS, Los Alamos Monitor Editor

Editors note: On Jan. 17, 2008, the Monitor filed a Freedom of Information Act request for the FY 2007 Performance Evaluation Report of Los Alamos National Security, LLC, Management and Operation of the Los Alamos National Laboratory. The National Nuclear Security Administration and Los Alamos National Laboratory declined to make public the key document for assessing the first year of a new contract. After the formal request, the document was made available and forms the basis for the following story.

One version of an old joke asks, “How do you go about watching an 800-pound gorilla?”

The answer: “Very carefully.”

But the same answer more seriously seems to apply equally well to the question of how the National Nuclear Security Administration goes about evaluating the job performance of a $2-billion-a-year national nuclear weapons laboratory.

Very carefully, in more than 180 pages, the report represents the essential annual assessment of the contractor under the terms of the contract. It is used to establish the bottom line, the amount of performance fee the contractors will be paid, based on how well they have met the challenges of the job.

The total fee available was $73,280,000, with $21,984,004 of that a fixed fee, regardless of performance. The remainder, $51,295,996, was available in incentives, of which LANL earned $36,224,982 or 71 percent of the available fee and $58,208,986 in total.

Roger Snyder, acting deputy site manager in the local NNSA office that supervises the laboratory, was involved in the evaluation.

“The department went into the contract with a larger fee than we have ever had, the largest fee in all the NNSA sites,” he said in an interview Friday. “We went into that as a demanding customer, knowing that we needed improvements and that it was a challenge that would have been hard to meet.”

In the future, the performance incentives will also provide a criterion for deciding whether there will be an automatic contract extension.

“The contract extension provision is not available in the first year,” Snyder said. “It would be an unfair bar at this point to get to the satisfaction level.”

Incentives

The evaluation categories consisted of 13 performance-based incentives (PBI), 12 of which were largely objective, as the document points out. Overall, there were 170 milestones with specific performance measures, which, if performed correctly, qualified for a full share of the available incentive fee. Most of the milestones or metrics or deliverables were pass/fail; if accomplished the reward is earned; if not, only a partial amount is earned, or none at all.

Among the objective measures, LANL’s percentage of fee was highest in the areas of the weapons programs (98 percent); weapons quality assurance (100 percent) and threat reduction (100 percent). In a category shared with other sites across the complex, all the sites were awarded 90 percent of fee for such things as coordinating interdependent projects within the nuclear weapons complex and sharing best practices.

The lowest percentages were earned in project management (73 percent), environmental operations and programs (58 percent), safety and health (53 percent), and facilities management (44 percent).

In the area of contractor assurance, the laboratory’s performance in identifying, self-assessing and responding to problems on their own, earned 49 percent of fee, despite “impressive progress,” against challenging goals, according to the report.

In the area of safeguards and security, LANL was awarded 89 percent of fee, despite a nationally publicized cybersecurity incident that occurred during the first month of the new management and led to Congressional hearings and numerous audits and investigations. The performance-related assessment of this event was shifted to the overall management category, officials said, because it was an emergent issue without a finite response and thus a more subjective judgment.

Management

The final measure, “Management Integration and Effectiveness,” is considered a subjective measure. In this category, with the largest amount of fee at stake, LANS received its lowest rating, 35 percent of the available fee.

Snyder compared the difference between the objective and the subjective measure to building a sidewalk.

“If you build the sidewalk, you can stand on it and you can measure it,” he said, describing the objective measurement.

“But how well did you do? Some of that is in the eye of the beholder,” Snyder said.

A further indication of how subjective this category might be can be found in the report.

Laboratory Director Michael Anastasio’s self-assessment stated that “significant progress” had been made.

“Focusing on results that we have delivered and the improvements that we have made, in spite of numerous obstacles, I can only assess our overall performance over the past year as highly effective,” he stated.

This statement is followed in the report by the Los Alamos Site Office evaluation which credits LANS for having recruited “significant talent in the form of key personnel” to help address “longstanding weaknesses” at the laboratory. The weaknesses included “the integration of mission planning and execution with safe, secure and environmentally sound operations.”

LASO’s summary credited “significant efforts,” to improve, but also found important shortcomings.

“As the initial year of performance under the performance-based contract, LANS was expected to integrate the organization, eliminate stovepipes, establish clear roles and responsibilities and develop and implement a framework for integrated planning and policy development and execution,” the summary stated. “They have been only partially successful in their efforts.”

On Thursday, Anastasio was asked in a telephone interview if he considered the 35-percent rating for management integration and effectiveness to be a failing grade.

“I don’t think of these as grades at all,” he said. “The government is offering to pay a significantly large fee to us in LANS to make significant changes and significant improvements. We all knew that it would be a challenging job that would take many years to accomplish.”

Snyder agreed that the numbers of the performance measures could not easily be converted into a final grade, because the performance was based on improvement.

“If this set of challenges were presented to the former contractor, the fee would have been lower,” he said. “We’d be surprised if a firm got an A-plus in the first year, because change is hard.”

“I feel it was a fair assessment on how well we did meeting the objectives that were laid out for us,” Anastasio concluded. “We did not get it all done. I don’t agree that’s a grade by which to judge the laboratory.”

[Download the FY 2007 Performance Evaluation Report here.]