NIH seeks high efficiency and accountability in its research portfolio performance management.  Mandated by the U.S. Government Performance Results Act legislation, NIH developed a systematic approach of collecting performance information to articulate program goals and provide information on performance monitoring.  Under GPRA, NIH has one program – Research.

As the nation’s major biomedical research agency, NIH manages a vast and complex research portfolio.  The size of the research portfolio created the need for a systematic data collection IT tool for gathering evidence on performance relative to program objectives, effectiveness, and efficiency.  NIH also recognized the need for a centralized automated portal system that could harvest the vast performance information and make it available to stakeholders and decision-makers in a timely manner and in ways that can aid communication and planning.  Such objectives guided SAB in the development of NIH’s first Program Performance Monitoring System (PPMS), a centralized secure online performance storage and monitoring system for collecting, organizing, and analyzing performance data.

PPMS is the collaborative knowledge portal for NIH performance information.  The system provides an anthology of performance information that can be used to develop and support organizational annual planning and assessing budget-performance integration.  PPMS serves as NIH’s program performance portfolio management and evaluation tool.  The system was built to link to relevant performance resources, and was designed not to duplicate existing systems.  PPMS consist of a Website (http://nihperformance.nih.gov) and the online performance reporting Visual Performance Suite (VPS) database application. The Website links to relevant NIH systemic assessments resources to provide sufficient access to general performance information at NIH.  VPS, the key feature of the Website, provides a Web-enabled centralized performance reporting database that is used to collect, store, and report GPRA and PART performance data for NIH projects.  NIH decision-makers are able to use the data for planning and performance monitoring.

Website 

The NIH performance monitoring Website enables the general public access to the centralized collection of NIH performance, database links, and publicly released NIH-specific performance reports and news.  The site organizes data and provides information relevant to the GPRA, PART, PAR and other program performance activities across NIH.  The PPMS Website is comprised of two Web-based servers: one is an unsecured public gateway to NIH performance monitoring information, and the other is a highly secured intranet server for authorized NIH users who participate in performance activities across the agency.  The sensitive and critical nature of the information stored within, processed by, or transmitted by some portions of the Website was a major concern in risk management.  This concern led to the construction of a security plan that would ensure the identification and implementation of management, technical, environmental, and operational controls commensurate with the sensitivity of the information and criticality of the application.  The PPMS Webpage has been designed to accommodate the following three user roles:

1. Shadow Users – also known as public users – is designed for users outside the NIH firewall. Users are only allowed access to non-sensitive content.
2. Standard Users – view designed specifically for individuals within the NIH firewall.  Users have access to the public view and the VPS tool set, and are permitted to perform data entry and generate reports within the VPS environment.
3. Special User – User role is specifically for NIH Senior Management to view graphics and reports related to NIH performance data.

The public server contains a virtual warehouse that stores numerous NIH performance data sources.  Public users who are not authenticated to be data users within the reporting system can browse through a centralized collection of published NIH-specific performance reports and NIH-wide database links.  The site contains federal and agency-wide program performance information that allows users to ascertain the relative effectiveness of NIH performance on its research programs.

The secure site contains a performance library that includes numerous sources such as press releases and narratives of scientific advances.  The system enables users to search within NIH and across specific performance indicators. For example, if a user selects “AIDS” as a search topic in the PubMed search engine on the Website and “NIH press releases” as the source, they can obtain all NIH press releases related to AIDS.  The Website also provides links to the external performance and budget data reporting VPS system.  It uses a single sign-on role-based entry access to enter the VPS site.  When a user within the NIH system firewalls navigates to the PPMS Website, the security system examines user criteria and grants access to the appropriate information based on appropriate authority rules.  The single access log-in feature provides a useful and convenient way for all agency users to have instant access to all NIH applications by entering user login and password only once.  On the secure site, standard users have the capability of entering performance data online directly into the VPS system.  The users are able to communicate strategically and use the automated data collection tool to make performance data transparent to collaborators.  The special users have the capability to view performance graphic based on data entered in the system and generate analytic reports to evaluate the programs.  The content of the secure Webpage are summarized in Table 1.

The National Institutes of Health (NIH), a part of the U.S. Department of Health and Human Services (HHS), is the primary federal agency for supporting and conducting biomedical research.  Composed of 27 Institutes and Centers (ICs), NIH conducts and supports basic, clinical, and translational medical research, and investigates the causes, treatments, and cures for both common and rare diseases.  NIH’s research portfolio is extensive and complex, and presents many challenges to program evaluation and portfolio management.  Portfolio management is critical to NIH’s efforts to sustain and to advance the excellence and relevance of its research programs in order to be dynamic and responsive to new medical information, emerging scientific opportunities, and public health needs.

U.S. government programs are under increasing pressure to become more transparent with its program performance and finances.  Under this agenda, all government agencies are held accountable for meeting agency performance goals and annual targets.  To make sure programs achieve this objective annually, government initiatives, such as the ones listed below have been established to capture performance information and to evaluate programs across all government agencies in the United States.

• Government Performance Results Act (GPRA) – Federal statute mandating performance-budget information reporting.
• Program Assessment Rating Tool (PART) – A systematic method of assessing the performance of program activities across the federal government.  PART is also described as a diagnostic tool used to improve program performance.
• Performance and Accountability Report (PAR) Act – PAR implements a process of program review and evaluation to determine the strengths and weaknesses of federal programs with a particular focus on the results produced by individual programs.

To fulfill the above requirements, NIH has a set of core criteria in planning and assessing organizational performance goals:

• Time (short, mid-range and long-term)
• Scientific Risk (low, medium and high)
• Trans-NIH approach (across Institutes, Offices and Centers)
• Balanced scientific research portfolio (intramural and extramural science activities)
• Representative sampling (across diseases, disorders, and management issues)
• Annual performance target and budget monitoring

To comply with government performance-based program reporting requirements, NIH has developed a systematic approach of collecting performance and budget information across ICs to articulate program goals and provide information on performance monitoring.  The NIH Systemic Assessments Branch (SAB) of the Office of Portfolio Analysis and Strategic Initiatives (OPASI) under the Office of the Director addresses government performance mandates and liaises with ICs to collect, plan, measure and evaluate NIH performance information.

The former process of reporting performance information at NIH required extensive communication between the Planning and Evaluation (P&E) community and SAB on a regular basis with monthly meetings, working groups, and numerous exchanges of email.  SAB program staff and IC officers followed a series of manual tasks and activities to gather, review, consolidate and publish paper-based reporting materials to meet performance reporting requirements.  To prepare for the creation of reports, annual data were reviewed, revised, annotated when necessary, and published in a series of paper-based reports communicated through a series of emails.  Such a process, when multiplied over the various Institutes, Centers, and Offices at NIH, represents a substantial loss in productivity and a high risk of error in the performance reporting.  It was difficult to maintain the currency of the information. The email-based communication and the reliance on Microsoft Word and Excel for reporting were not efficient and did not meet the performance reporting needs of NIH.  The implementation of the centralized PPMS application allows for distributed content management, providing a more useful tool for knowledge management of performance information, while allowing performance goal leads and contributors a single point of access to a centralized repository of needed information.