Contents | September 2006 | Volume 1, Issue 5
The Evaluation Experience
Over 60 agency staff have put in thousands of collective hours on the Data Center Services Request for Offer (RFO) evaluation. These evaluators all have hectic, full-time jobs and many competing priorities. Yet, they have given generously of their time to make sure agencies’ interests are incorporated into the RFO evaluation and future plans for data center services. DIR and the state are indebted to the evaluators, without whom a truly statewide RFO would simply not be possible.
The evaluation includes agency executives, information technology (IT) management, and technical, finance, legal, and human resources (HR) experts.
Each team focuses on its area of subject matter expertise and reviews the RFO responses based on its evaluation criteria. The advisors, a panel of agency executives, review the team’s results and provide a vital business perspective on the contract.
The teams have gone through a rigorous and structured evaluation process. Many team members helped build the RFO requirements in February and all participated in several days of training in May. All teams received basic training and each specialty team received training on their assigned exhibits. Since the training, team members have reviewed the RFO responses on their own and evaluated them together. The groups will convene again after BAFO (Best and Final Offer) responses are received. Their final report will be delivered to the state CTO and the agency executive advisors on November 1.
DIR had an opportunity to speak with the evaluators about their experiences. Here are some themes from the discussions.
A monumental task. Some evaluators “didn’t quite know what [they] were getting into,” when they first joined the evaluation team. Many wondered how 27 agencies could work together effectively on a project of the scale and importance of data center services for Texas. As one commented, “I thought we would be overwhelmed. With 27 people, all going in different directions, how will they [DIR] pull this off?” Other evaluators, with experience in similar large procurements, found themselves on familiar ground. Several evaluators commented that the process and rigor were in “lock step” with Texas procurement laws and their personal experiences. They felt the detailed process and organization were particularly necessary with such an important and large procurement.
A balancing act. RFO evaluation team members have participated in data gathering, individual reviews, clarification sessions, and numerous other meetings. At the same time, all evaluators have important responsibilities at their agencies. Balancing two “full-time jobs” has been a challenge for evaluators, particularly those from small agencies. Most evaluators, however, commented that while the level of involvement has been intense, it was also a necessary investment to fully represent their agencies. As one noted, “as the customer, it is critical for our agency to be as deeply involved as possible.”
Clarification sessions. Most evaluators found the face-to face meetings with service provider teams during the clarification sessions to be a “highly valuable” part of the evaluation process. Evaluators commented on the importance of meeting the proposed delivery teams, seeing how the teams worked together, and having an opportunity to clarify assumptions from the written responses. Overall, the sessions received high marks, as they provided a useful way to better understand proposed solutions though interactive questions and answers with the service provider teams.
Learning from each other. One side benefit of participating in the procurement has been the interaction among the 27 agencies. The evaluators represent agencies of different sizes, with different missions, funding streams, and even IT organizational structures. Some agencies have extensive experience outsourcing technology services and others do not. Many evaluators have participated in or even led significant procurements, while such involvement is new for others. With so many participants with diverse experiences, the procurement has been an opportunity for evaluators to learn from each other. As one said, "I think everyone walked away with new ideas. You look at things differently after hearing everyone's opinions."
External perspective. The evaluators appreciated the methods and extensive experience the procurement advisors, Technology Partners International (TPI), brought to the data center services process. The TPI consultants' personal experiences with these contracts and their ability to help the 27 agencies through a complex process were both noted as vital to the procurement's success. DIR thanks all of the evaluators for their time, their insights, and their commitment to the data center services project.
Base Case 101
The RFO Attachment 4-C.1 contains a critical element of the data center services project—the financial base case. The financial base case captures current costs incurred by the agencies in support of data center services and calculates future costs over the life of the contract (seven years). It is used in the financial evaluation of the RFO responses and to complete the business case analysis. The seven-year forecast is based on standard economic modeling factors, including historical productivity gains and price improvements in technology.
Costs captured in the financial base case include items like employee salaries and benefits, hardware and software leases, disaster recovery contracts, and other items like postage and printing. The total base-year spend of approximately $145.2 million was derived from the 27 agencies’ self-reported budgets for supplier- assumed in-scope costs for fiscal years 2006–07. The base case figures represent a point in time and are being adjusted based on due diligence findings and the FTE validation (see “Data Gathering Update,” below, for details).
The base case is different from the business case. The business case compares the total cost of outsourcing data center services with current costs. The total cost of outsourcing includes the service provider’s base charges, costs retained by the agencies, start-up fees, governance costs, and “pass-through” expenses (like postage) for which the state will reimburse the service provider at cost. The business case cannot be calculated yet because the contract has not yet been negotiated.
For more information, see RFO Exhibit 4, Attachment 4-C.1 (Financial Base Case) and Attachment 4-C.2 (Financial Base Case Assumptions).
Data Gathering Update
When it did not seem possible to get any busier, data gathering activity accelerated in August and September. This additional effort has yielded new information that is critical to the RFO process. The major activities were:
FTE Validation. This project consisted of interviews with IT managers and staff from the 27 agencies. PA Consulting, the firm hired to complete the validation, and DIR met with the agencies to review the allocation of positions to in-scope and out-of-scope functions. The process resulted in changes to the number of affected employees in several agencies, with some agencies increasing affected employees and some decreasing affected employees. Additionally, many positions were reclassified from one out-of-scope area (for example, miscellaneous) to another out-of-scope area (for example, help desk). All agencies have received their results from PA Consulting.
Due Diligence. This process included a number of visits from the service providers to the agencies and the collection of third-party contracts. Hundreds of new contracts were received and processed for the Data Room (see “Data Room,” in the March, 2006, Data Center Download). These contracts are now available for review by the service providers. The service providers also had an opportunity to meet with many agency staff to discuss operations and unique agency requirements. Due diligence has concluded.
Change Management Training
DIR has held eight change management training courses for the data center services project. Originally for supervisors, the new training program is open to interested affected employees, supervisors, and trainers. By popular demand, the training has been extended with several new workshops. Additionally, if an agency has six or more interested participants, DIR can schedule a special workshop for that agency.
Each workshop is three and one-half hours long and helps participants understand the data center services project and its effect on employees and agencies. These interactive workshops include a presentation and group discussions about the current status of the DCS project, RFO human resources requirements, and how to address issues related to the changes from the data center consolidation. On their training evaluation forms, participants wrote that they found the sessions to be valuable in helping them understand the DCS project activities and requirements, dispel rumors about the project, and encourage communication throughout IT. Anyone interested in or affected by the data center consolidation is encouraged to attend an upcoming workshop.
DIR is offering workshops at the following dates and times:
- October 9, 1:00-4:30 pm, TxDOT Riverside Campus Bldg. 200, Classroom E
- October 10, 8:30 am-noon, TxDOT Riverside Campus, Bldg. 200, Classroom C
- October 13, 8:30 am-noon, TxDOT Riverside Campus, Bldg. 200, Classroom E
To register for a workshop:
- Request supervisory approval for attending.
- Draft an e-mail with your name and the requested training date.
- Send e-mail to Joelyn Weeks.
- Copy your supervisor on the e-mail.
- To schedule a session for six or more participants from an agency, e-mail Joelyn Weeks
The TxDOT Riverside Campus is located at 200 East Riverside Drive, Austin, Texas 78704. Visitor parking is available in front of the building.
