Validation

“Market Acceptance of Performance Information”

  • Independent performance validation of technologies and projects based on customer-driven benchmarks and evaluation protocols.
  • Development of technology specific test plans and performance verification protocols.
  • Assessments of technology and project performance based on feedback from technology buyers and early adopters.
  • High-level credibility through a consortium of recognized performance testing organizations.

3rd Party Performance Measurement and Verification Procedure

GLOBE Performance Solutions provides third-party performance measurement and verification services based on the proposed international protocol for environmental technology verification currently being considered by the International Working Group on Environmental Technology Verification.

The third-party performance measurement and verification procedure used by GLOBE Performance Solutions is summarized below.

Outline of Performance Measurement and Verification Requirements

Step 1 – Technology screening and application

  • Technology pre-screening
  • Formal application
  • Stakeholder input

Step 2 – Performance testing

  • Independent, third-party testing
  • Technology-specific test plan
  • Performance testing operating conditions
  • Sample size
  • Analytical laboratory protocols
  • Chain of custody
  • Quality assurance
  • Performance testing report

Step 3 – Performance verification

  • Independent, third-party verification
  • Verification based on an internationally accepted protocol
  • Determination of 95% statistical confidence
  • Reporting

3rd Party Performance Measurement and Verification Procedure

Description Performance Measurement and Verification Procedure
1. Step 1 – Technology screening and application
1.1 Pre-screening to determine eligibility

Verification of a technology performance claim requires the proponent to provide reliable, high quality data on the technology. For a technology to be eligible for verification, the performance claim must be specific and unambiguous. The claim must clearly specify the minimum and maximum performance that is achievable with the technology. The performance claim must also clearly specify the applicable operating conditions and be measurable using acceptable test procedures and analytical techniques. The data must be collected by an independent, third-party test agent and analyzed by an independent, accredited laboratory.

Pre-screening checklist:

  • Alignment with priorities
  • Environmental benefits
  • Commercial readiness
  • Potential for market replication
  • Requirements for performance testing and verification – i.e., suitability of test plans (or need to develop), credibility of available data (or need for additional testing)
  • Level of effort and cost to client

1.2 Formal application for verification

Before confidential information is provided to GLOBE Performance Solutions and its associated testing and verification organizations, potential conflicts of interest must be resolved and a confidentiality agreement must be signed. The testing and verification organizations may be selected from the list of GLOBE Performance Solutions Consortium members. In some cases other testing and verification organizations may be selected based on the requirements for specific technology expertise.

1.3 Stakeholder input and guidance

Stakeholder engagement and involvement are basic requirements for ensuring that local needs and related social impacts are adequately addressed in the determination of relevant performance parameters for performance verification.

2. Phase 2 – Performance testing

2.1 Independent, third-party testing organization

Testing and preparation of the test report is conducted by an independent testing organization. Specific high-quality data is required to determine the performance of the technology. The testing organization reviews the proposed performance parameters and any available technology-specific test results to determine whether additional data is required. The data related to the technology must be of sufficient quality and quantity to permit statistical analysis of the data at a 95% confidence level.

2.2 Technology-specific test plan

The testing organization must be independent from the technology proponent (e.g., vendor). The test plan used to generate the performance data should be assessed against the test design required for the verification. The testing procedure must include a quality management plan to ensure compliance with the test data quality requirements. An audit of the test agent and analytical laboratory that generated the data may be required to show compliance with quality management system requirements.

2.3 Performance testing operating conditions

A verified claim is only valid if the technology is operated within the specified operating conditions. Data collected to support the technology performance claim must be collected within appropriate ranges of key operating parameters. Operating conditions during the testing must be monitored and documented to ensure that they are within the ranges defined in the experimental design. Information must be provided on the actual key operating variables during the test and the methodology used to monitor and document these conditions.

2.4 Sample size

The type of sample collected must be appropriate to generate data that can be used to verify the performance of the technology. Samples must be collected after sufficient time has elapsed to allow the process to stabilize, and to ensure that the samples are representative of the typical process characteristics at the sampling locations. The number of samples must be sufficient such that when the data supporting the claim is statistically analyzed, the claim can be accepted. The number of data required in any data set to be statistically acceptable will depend on the variability in the data and the nature of the technology and its application. It is the responsibility of the applicant to ensure that a sufficiently large number of data points are provided, with a sufficiently small variability, to be statistically acceptable. Acceptable sampling procedures or protocols must be used to collect and preserve samples including field and reagent blanks. Standard reference materials are required. All procedures and protocols must be applied by competent testing organization.

2.5 Analytical laboratory protocols

The laboratory that analyzes the samples must be independent from the applicant, and accredited (under ISO 17025) by a recognized certification agency for the specific parameters being analyzed. In Canada, acceptable laboratory accreditation to the applicable technical testing scope must be earned from the Canadian Association of Environmental Analytical Laboratories (CAEAL), the Standards Council of Canada (SCC), or a signatory to the International Laboratory Accreditation Cooperation.

2.6 Chain of custody

Chain of custody refers to the ability to trace the possession and handling of the sample from the time of collection through analysis and final deposition to ensure the integrity of the process from sample collection to data reporting. Chain-of-custody reporting is used in routine control of sample flow, and in litigation if required. Acceptable chain of custody for sample analysis may include sample labels, sample seals, sample submission sheets, sample receipt log and assignment for analysis.

2.7 Quality assurance

Testing under actual operating conditions is required following an established test plan and prescribed methodologies. The instruments used to measure the operating parameters must be calibrated regularly to ensure that the instruments are providing accurate, reliable readings and these calibrations must be documented. Acceptable quality assurance/quality control (QA/QC) procedures must be used for sample collection and analysis. For sampling, QA/QC may include the use of blanks, spiked samples and replicate samples as well as inter-laboratory studies and use of standard reference materials.

2.8 Performance testing report

A test report is produced highlighting the test plan, methods used, results achieved, observations, and conclusions. The data must demonstrate that the technology has a measurable effect on the specific parameter(s) identified in the performance claim.

3. Phase 3 – Assessment and verification

3.1 Independent verification organization

Verification and preparation of the verification report is conducted by an independent verification organization. Verification of the technology requires specific and high-quality data to determine the performance of the technology.

3.2 Verification based on an internationally accepted GVP

A general verification protocol (GVP) is used by the independent verification organization that is responsible for reviewing and verifying data and information that support the technology performance claim. The GVP contains an overview of the verification process, instructions for review of the technology, and a series of checklists that provide guidance to ensure that a comprehensive examination is undertaken. The GVP guides the verification organization through the process of determining the mandatory criteria (i.e., scientific soundness, absence of conflict of interest, environmental benefits) and requires an expression of the limitations of the technology, as well as validation of the actual performance data.

3.3 Determination of 95% statistical confidence

The verification organization reviews the proposed performance parameters and the technology-specific test results to determine whether the data meets the objectives of the verification process. The data related to the technology must be of sufficient quality and quantity to permit statistical analysis of the data at a 95% confidence level. Statistical analysis worksheets are used to support this determination.

3.4 Reporting

The client receives a certificate of verification, a verification fact sheet and a verification report. The certificate does not constitute a guarantee of performance.

Consortium Members

Centre des technologies de l’eau
www.cteau.com
Centre for Advancement of Trenchless Technologies
www.catt.ca
Centre for Alternative Wastewater Treatment
www.cawt.ca
Centre National en Electrochimie et en Technologies Environnementales
www.cnete.qc.ca
Edmonton Waste Management Centre of Excellence
www.ewmce.com
FPInnovations
www.fpinnovations.ca
Good Harbour Laboratories
www.goodharbourlabs.com
The Living City CAMPUS
www.thelivingcitycampus.com
Prairie Agricultural Machinery Institute
www.pami.ca
Process Research ORTECH
www.processortech.com