The 2006 LTAP/TTAP Performance Report provides highlighted information and data analysis on the combined efforts of the 58 LTAP/TTAP Centers across the United States and Puerto Rico toward improving skills and increasing knowledge for the surface transportation workforce and its decision makers. The Report focuses on a national perspective as LTAP/TTAP works to foster a safe, efficient, and environmentally sound surface transportation system.

The 2006 Performance Report includes:

• A Background section on why and how performance data is collected under the LTAP/TTAP Strategic Plan;
• A 2006 Program Overview section that provides analysis of the PAR and CAR data as well as information on the performance of NLTAPA and the FHWA LTAP Clearinghouse; and,
• A Conclusion that broadly outlines action items and next steps from the national perspective.

Background

Previous editions of this Report have summarized the continued emphasis on planning for strategy and performance, improving accountability and demonstrating results across the Federal government. Through the Program Assessment Rating Tool process, the Office of Management and Budget is reviewing all Federal programs with an eye towards terminating outdated and ineffective programs and modernizing others. To date, fully one quarter of the programs reviewed have been categorized as "Not Performing" with either an "Ineffective" or "Results Not Demonstrated" rating.

As a small Federal program competing for limited resources, LTAP/TTAP must be able to articulate its successes. To accurately and effectively measure whether the national Program is both "doing things right" and "doing the right things," each year FHWA collects performance data from the 58 LTAP/TTAP Centers on these national efforts. At a minimum, FHWA must collect basic business information – how many courses, how many participants etc. – that justifies the expenditure of taxpayer dollars. However, the performance data collected by FHWA also allows national Program leaders to prove that the Program provides direct and lasting value to the local transportation community far in excess of the financial and human resources it expends.

FHWA continues to increase efforts to move the program forward and sustain its success in this new, more competitive environment. Three primary considerations are shared by FHWA and the LTAP/TTAP community for the national program:

• Meeting the national goals for the transportation community
• Insuring that taxpayer dollars are used in an appropriate manner
• Producing results that impact LTAP/TTAP customers and clients

Data Collection and Analysis

Information regarding the operations and management of the Program is provided in two distinct reporting tools. The Program Assessment Report (PAR) represents the raw quantitative and numerical data on how Centers are delivering program services. The Center Assessment Report (CAR) captures the qualitative judgment of Center directors and staff as to how they manage their day-to-day operations. The knowledge gleaned from both reports helps FHWA exercise effective stewardship of the national LTAP/TTAP, better share information on how individual Centers achieved success and overcame challenges, and provide targeted and focused assistance to particular issues raised.

Performance Reports present an overview and analysis from a national perspective of the information provided in the individual PARs and CARs. The PAR data is reported as aggregate national performance numbers across the four focus areas in the LTAP/TTAP Strategic Plan – Safety, Infrastructure Management, Workforce Development and Organizational Excellence. The CAR responses, however, are presented without direct attribution to any particular Center. FHWA uses the CARs to highlight national trends, successes and pressure points for the continued attention of the LTAP/TTAP community. This is meant to preserve the honesty, impartiality and confidentiality of CAR submissions, and insure that Program leaders are able to rely on Centers to provide useful and critical feedback in an environment of trust. FHWA also uses information in both the PAR and the CAR to follow up with individual Centers on the particular issues raised in their reports.

The analysis of the data in both reports includes the cumulative work of the LTAP/TTAP community, and the PAR numbers and the CAR context are considered together. Neither stands alone as an indicator of successes, challenges or areas for further review or assistance. Furthermore, changes in PAR numbers – whether positive or negative – are not indications of success or failure, nor do they tell the entire story of how the LTAP/TTAP is delivered, managed and operated.

2006 Program Overview

In their third year of PAR and CAR reporting, the response rate of LTAP/TTAP Centers maintained an excellent overall response rate. PAR data for 57 of the 58 Centers are represented in this report, while 52 of 58 CARs were submitted. This represents the commitment of the Centers to participate fully in a national assessment of program performance, and underscores the importance of developing information that effectively evaluates program activity at the national level. The national data set from the 2006 calendar year can be found on the National PAR form in Appendix I at the end of this report.

The 2006 calendar year represents the second full year of data for all reporting Centers. For this reason, comparisons with the data from the previous year are not necessarily valid and should be avoided. In the first year of data collection in 2004, only some Centers were able to capture data for the entire calendar year, while others collected data for only part of the year. Following the collection of the 2007 PARs and CARs in January 2008, FHWA expects to have enough valid data to engage in output- and trend analysis for the national LTAP/TTAP. As ever, FHWA will continue to partner with the Program community to analyze, review and improve the data collection process, as well as make better efforts to share best practices gleaned from the reports.

2006 PAR Analysis

The PAR data collected in 2006 continues to confirm the value that the LTAP/TTAP provides our customers and stakeholders. During 2006, Centers conducted 4,978 training courses for the local transportation community. Those courses were attended by 136,808 local roads workers augmenting their skills and improving their on-the-job knowledge in a total of total 901,829 participant contact hours of classroom time. LTAP/TTAP outreach efforts to share information and best practices across the local community also continue at a remarkable pace; last year, the total circulation of all Program newsletters reached 461,377. These overall program numbers were comparable to those reported in 2005 – slightly fewer courses and contact hours, for example, but more participants.

The focus areas in the LTAP/TTAP Strategic Plan represent a specific part of Program operations where LTAP/TTAP works to make a difference for partners and customers. Overall, Centers reported an emphasis on the Safety focus area, where they provide needed training in both highway safety and worker/workplace safety. Of these training sessions that LTAP/TTAP provided in 2006, nearly half – 2,364 out of 4,978 sessions – included highway or worker/workplace safety content, and these courses and workshops reached over 57,000 participants. In the Infrastructure Management focus area, LTAP/TTAP provided 1,974 sessions also reaching around 57,000 participants. To further Workforce Development in local transportation agencies, Centers conducted over 1,000 training events for an audience of just over 28,000 participants.

One important role that Centers fulfill for the local agency community is the augmenting their awareness of emerging technologies, best practices and other information from throughout the local roads community through the quarterly Center newsletters. The 132 issues circulated during 2006 contained an average of five articles on Safety topics, two on Workforce Development, and three on Infrastructure Management. Each Center also distributed additional media on LTAP/TTAP areas of interest, such as videos, printed articles and other training materials. In 2006, Centers distributed over 200,000 publications, nearly 8500 CDs and almost 3000 DVDs – predominantly to local agencies, but also to a significant number of state and private-sector partners and customers.

Additionally, 39 Centers reported that they implemented some form of Kirkpatrick Level II evaluation of their safety course offerings and materials in 2006, up from 36 in 2005 and 28 in 2004. These efforts to determine either the level of knowledge, skills and abilities or proficiency in a particular skill or capability ranged from pre- and post-tests to on-the-job simulations to group exercises. Centers reported using the results obtained improve teaching materials, evaluate instructors and gauge effectiveness of courses offered. LTAP/TTAP Centers will expand the use of these evaluation techniques to improve the level of instruction across all focus areas.

Differences in the needs of the customers serviced by the Centers in the program were apparent. For LTAP Centers, nearly half of training sessions provided – 2,364 – fell under the Safety focus area; for TTAP Centers, however, over half of their courses – 173 of 307 – covered topics in the Infrastructure Management focus area. While safety remains important across the local roads community, this may reflect a focus of TTAPs on the need for capacity building in tribal areas. Furthermore, while the overwhelming majority of participants came from the LTAP/TTAP local agency constituency, small but significant numbers of state DOT and private sector participants were recorded, indicating a possible opportunity to meet the needs of the contractor force working for those agencies.

Centers continue to achieve these impressive results with limited financial and human resources. The average Center staff is a little over three-and-a-half full-time employee equivalents, or FTEs. The impact of these small staffing levels is often exacerbated two factors; the fact that many staff are only working for LTAP/TTAP as a percentage of their personal FTE load to their state DOT or university, and the continued reliance on transient student assistants by many university-based Centers. Each Center received only $140,000 of Federal funds in 2006 as the basis for a 100% matching requirement that results in a minimum operating budget of $280,000. That total investment amount, forecast to remain constant through the term of SAFETEA-LU, provides an outstanding value to the taxpayer given the training and technology transfer impact of LTAP/TTAP.

2006 CAR Analysis

Each CAR contains mission critical information on how Center directors and staff manage their internal and external operations. The general results from the 2006 CARs did not differ from those received in 2005 and 2004, and focused on the recurring themes of funding, staffing and instructors.

Financial constraints remain the most serious challenge facing LTAP/TTAP Centers. Now that SAFETEA-LU is in place after numerous delays and continuing resolutions, the primary issue identified by Centers is the need to work more closely with some states and university program partners to minimize the financial impact of providing needed support services. This creates an unequal distribution of resources where similar Centers have markedly different financial situations, limiting their ability to deliver needed products, training and services. Centers are also focused on the need for adequate funding levels to maintain, sustain and grow the program beyond the $11 million per year level. Many Centers reported making concerted efforts to secure sources of additional funding from public and private sources to augment their ability to provide services, a growing concept throughout the Program community. One critical lesson for LTAP/TTAP is that planning for these fiscal challenges is a reality and not an exceptional situation.

Another continuing concern for Centers is maintaining adequate levels of Center staff and qualified instructors. While the average number of full-time equivalent staff (FTEs) across the national Program improved in 2006 to 3. 7 per Center from 3.4 in 2005, this rise tends to reflect more accurate data reporting in this field. Still, the responses indicate that staffing limitations prevent Centers from engaging in or completing mission-critical tasks such as Program outreach, coordination and reporting. Responses were also more focused on the need for qualified instructors to teach mission-critical LTAP offerings; some Centers report that they must restrict critically needed course offerings for the lack of an instructor in their state, while others continue to accept less experienced trainers to meet demand. There has been little change in perception of these challenges among Centers since 2004.

Conclusion

Collecting performance data is both a prerequisite of effective program management as well as a requirement of all Federal programs. Through the yearly review and analysis of Center reporting, FHWA is able to both indicate the general performance levels of the LTAP/TTAP as well as identify areas of need or national issues that require the attention of Program leadership.

The single most important factor that insures the presentation of the full picture of LTAP/TTAP efforts, however, is the submission from all 58 Centers of all available data through the PAR and CAR. Anything less than total Center participation in the data collection and submission process hinders the ability of all Centers to amplify their successes and truly the define the effectiveness of the program. The on-time submission rate for PARs and CARs dropped in 2006, and there continue to be a small minority of Centers who do not remit reports at all. FHWA, however, has committed to reaching a 100% submission rate, and will be reviewing all available options to insure that all Centers comply with mandatory submission of both reports by January 31st each year.

FHWA continues to work to identify those national programs and initiatives that have the greatest impact or address the priority needs of Centers to insure Program dollars are spent wisely and effectively. Each and every national spending item within the national LTAP/TTAP is under review and will be evaluated alongside the needs assessment nearing completion under the NLTAPA Products and Services work group. Those that do not provide maximum value to Centers or align with their most pressing needs will be curtailed or eliminated.

One final emerging issue stems from the role the TTAP Centers play in their unique and challenging constituencies. The CARs in particular have highlighted a need for a greater concentration of effort at the Federal level to manage expectations and responsibilities between the various departments and agencies working on Native American issues. FHWA is providing a new focus to respond to tribal needs by working over 2006 and into 2007 to improve relations with BIA and Federal Lands, and also plans to prioritize significant outreach and management efforts for tribal areas. The effect on the national TTAP and the overall program will be assessed in the coming year, as FHWA plans an additional effort to categorize and amplify the unique strategic and business elements of the TTAPs within current strategic and business planning.

For 2006, then, there again can be no real assessment of the quantitative PAR data. However, the Program has now amassed a significant baseline of performance data that will allow for a more in-depth comparative analysis for the 2007 data set in 2008, where it will be possible to identify trends and draw more accurate conclusions regarding national efforts. Even without the benefit of verifiable quantitative data, FHWA believes that LTAP/TTAP provides a valuable and important service with impressive reach and depth for a minimal Federal investment.

Appendix I: National LTAP/TTAP Program Assessment Report

Click here to view the 2006 reporting period