Skip to the main content.

5 min read

Industry Evolution: Ensuring Program Effectiveness

In the preceding editions of this EWN Educational Series, EWN has reviewed PHMSA’s proposals for a new definition of a covered task, changes to span-of-control limits, new training requirements, new evaluator selection and training requirements, a new management of change program and expanded recordkeeping requirements.

These and other proposed requirements in the pending OQ NPRM are interdependent with PHMSA’s proposed requirement for an annual program effectiveness review. The program effectiveness review will require companies to maintain a very active role in their Operator Qualification program and the management of records, internal audits, and performance in the field.


The current OQ Rule does not directly address requirements for a routine program effectiveness review. However, shortly after the original OQ Rule went into effect, PHMSA issued guidance on individual and program performance monitoring and improvement via their responses to Frequently Asked Questions found at

Current PHMSA Guidance

5.1 Are operators required to continuously monitor the performance of individuals qualified to perform covered tasks?

While operators are not required to continuously monitor the performance of individuals qualified to perform covered tasks, the rule does require operators to: (a) evaluate an individual if the operator has reason to believe that the individual’s performance of a covered task contributed to an incident (as defined in Part 191) or accident (as defined in Part 195); and (b) evaluate an individual if the operator has reason to believe that the individual is no longer qualified to perform a covered task. The operator should document in its OQ Program how it intends to satisfy these requirements.

6.1 What continuing process of performance monitoring and improvement is expected of operators?

Given that the OQ rule is largely a performance rule with a limited set of specific prescriptive requirements, operators are expected to monitor the effectiveness of implementation of their programs, and to seek out opportunities for improving that performance. Improvements can be identified from sources such as: (a) internal innovation; (b) practices of other operators; (c) practices developed by industry consortia that are generally recognized as effective; or (d) provided in the forthcoming consensus standard, B31Q.

The pending OQ NPRM makes the direct shift from “performance based” requirements and guidance to a very “prescriptive” set of minimum mandatory regulations. The new demands related to program effectiveness include requiring companies to have a written process to measure their qualification program effectiveness and to conduct a program effectiveness review once each calendar year, but not more than every 15 months.

The PHMSA proposed program effectiveness requirement in 49 CFR 192.807 and 195.507 is intended to ensure that operators regularly complete a review of the effectiveness of their OQ program, and implement improvements. The review includes ensuring that procedures that have been amended have been captured in the necessary portions of the OQ program, that changes are communicated to affected personnel, and that the program is being enforced and administered in such a way as to support the core intent of the OQ Rule, which is:

To ensure a qualified workforce on jurisdictional pipelines, AND THEREBY reduce pipeline incidents and/or accidents caused by human error.

PHMSA also proposed to add requirements in 192.809 and 195.509 for the retention of records that are normally reviewed during the inspection of OQ programs and which are necessary to provide a thorough overview of an OQ program. The additional records would include records that document each evaluatorsΓÇÖ performance and impact on program effectiveness.

PHMSA’s Specific Proposal for Program Effectiveness Reviews

49 CFR 192.807 & 195.507ΓÇô Program Effectiveness
(a) General. The qualification program must include a written process to measure the program’s effectiveness. An effective program minimizes human error caused by an individual’s lack of knowledge, skills and abilities (KSAs) to perform covered tasks. An operator must conduct the program effectiveness review once each calendar year not to exceed 15 months.
(b) Process. The process to measure program effectiveness must:
(1) Evaluate if the qualification program is being implemented and executed as written; and
(2) Establish provisions to amend the program to include any changes necessary to address the findings of the program effectiveness review.
(c) Measures. The operator must develop program measures to determine the effectiveness of the qualification program. The operator must, at a minimum, include and use the following measures to evaluate the effectiveness of the program.
(1) Number of occurrences caused by any individual whose performance of a covered task(s) adversely affected the safety or integrity of the pipeline due to any of the following deficiencies:
(i) Evaluation was not conducted properly;
(ii) KSAs for the specific covered task(s) were not adequately determined;
(iii) Training was not adequate for the specific covered task(s);
(iv) Change made to a covered task or the KSAs was not adequately evaluated for necessary changes to training or evaluation
(v) Change to a covered task(s) or the KSAs was not adequately communicated;
(vi) Individual failed to recognize an AOC (task specific or non-task specific) which occurs anywhere on the system;
(vii) Individual failed to take the appropriate action following the recognition of an AOC (task specific or non-task specific) that occurs anywhere on the system;
(viii) Individual was not qualified;
(ix) Nonqualified individual was not being directed and observed by a qualified individual;
(x) Individual did not follow approved procedures and/or use approved equipment;
(xi) Span of control was not followed;
(xii) Evaluator or training did not follow program or meet requirements; or
(xiii) The qualified individual supervised more than one covered task at the time.

The potential resource requirements for these annual program effectiveness reviews was noted by the industry during the comment period for the OQ NPRM. At the PAC advisory session on June 2, 2016, PHMSA responded to industry feedback regarding the new management of change program and program effectiveness review requirements. Specifically, PHMSA stated the following:

  • As to operator size limit, a management of change process is critical for all regulated operators to have, regardless of size, so that changes made in such things as operator processes, procedures, and equipment are properly captured in the necessary portions of the OQ program.
  • As to the program effectiveness review period should be extended OQ requirements can achieve measurable outcomes in a much shorter time period.

Recommendation from Joint Session of GPAC and LPAC:

The PHMSA proposed rule changes were unanimously accepted without changes.


Due to the nature of the minimum review requirements, an annual OQ program effectiveness review is not an activity that can just be completed at a set time each year. Rather, it is an ongoing monitoring of the program and personnel. The prescribed review process requires each company to identify and review occurrences caused by any individual whose performance of a covered task(s) adversely affected the safety or integrity of the pipeline. As such, companies will need to establish processes for reviewing, inspecting, auditing and reporting various elements of personnel performance, and program administration and enforcement. The key to this process will be capturing accurate and reliable data at the various points of training, evaluation, and the performance of activities in the field. As such, program effectiveness reviews may become a significant activity for many companies.
Based on PHMSA’s comments during the joint GPAC and LPAC session, and the PAC’s unanimous approval of PHMSA recommendations, the program effectiveness review requirements are anticipated to become a Final Rule as written and without an extended implementation period.

Helpful Resources

Although PHMSA did not include the ASME B31Q-2014 Standard on Pipeline Personnel Qualification by reference in the pending OQ NPRM, it was mentioned and some of the proposed requirements are drawn directly from the B31Q Standard. As such, the ASME B31Q Standard provides useful information to assist companies in understanding and implementing many of the proposed PHMSA requirements. Some of the specific resources that will be benefit companies, include:

  • ASME B31Q Section 8.1 Evaluation Process (including information on evaluator selection) and related section 5.3.1. on Selecting SMEs.
  • ASME B31Q Section 11. Program Effectiveness.
  • ASME B31Q Section 12. Communicating the Qualification Program and Managing Program Changes.
  • ASME B31Q Section 13. Documentation Requirements (which includes some information relevant to program effectiveness).

The 2016 version of the ASME B31Q Standard is expected to be published later this year.

Special Note

Although timing for the issuance of Final Rules on these regulatory updates is unknown, it is anticipated that PHMSA will issue a Final Rules on these topics later this year. One factor that may influence the timing is the recent enactment of the PIPES Act of 2016, which includes an emphasis for PHMSA to accelerate the numerous rulemakings outstanding from the 2011 pipeline safety bill. Under the new PIPES Act, PHMSA is required to provide a report to Congress before the end of October 2016 on the status of statutory directives, including the status of each mandate, reasons for its incompletion, and estimated completion date. The Congressional reporting requirement is anticipated to create some urgency for PHMSA to complete several pending NPRMs, which would include the OQ rule updates.

It’s important to remember that PHMSA is required by law to hold advisory committee meetings with the GPAC and LPAC groups. PHMSA is not bound by the recommendations that are agreed to by the PACs, but will consider those recommendations as they develop a Final Rule.