Transitioning to New Recreational Fishing Survey Designs

NOAA Fisheries’ Marine Recreational Information Program follows an ongoing cycle of continuous improvement: Existing methods are evaluated, potential improvements are tested and reviewed, and new or improved recreational fishing survey designs are put in place. For data derived from new or improved methods to support fisheries science and management, these methods must be sound, and the catch and effort estimates they produce must be incorporated into the existing time series with minimal disruption.

NOAA Fisheries fosters continued methodological improvement while maintaining the integrity of our long-term recreational catch and effort estimates through its established policy and procedures for reviewing, certifying, and transitioning to new or improved survey designs. These policies and procedures are summarized on this webpage and described in more detail in the following documents:

• Policy Directive 04-114: Implementing Recreational Fishery Catch and Effort Survey Design Changes (PDF, 4 pages)
• Procedural Directive 04-114-01: Guidance and Procedures for the Transition Process for Modification of Recreational Fishing Catch and Effort Methods (PDF, 5 pages)
• Procedural Directive 04-114-02: Guidance and Procedures for the MRIP Certification Process (PDF, 9 pages)

Survey sponsors may contact the Marine Recreational Information Program's Program Management Team for more information.

Certification

NOAA Fisheries’ process for reviewing the statistical rigor of a recreational fishing survey design is known as certification. To earn certification, survey designs and estimation methods must be documented, adhere to applicable standards, and earn validation by a peer review. The steps of the certification process are described in more detail in Procedural Directive 04-114-02: Guidance and Procedures for the MRIP Certification Process (PDF, 9 pages).

The certification of a new or improved survey design means:

• A peer review has found the survey design to be a statistically valid approach to meet survey objectives and provide key estimates.
• The program is eligible to receive financial support from NOAA Fisheries. For surveys that pre-date MRIP and our certification process and are already receiving NOAA Fisheries funding, becoming certified allows them to remain a priority to receive continued funding.
• Certification does not guarantee implementation or use in federal fisheries management. Decisions as to how the data are used are made by NOAA Fisheries and partners, independent of the certification process.

Documentation

Survey designs and estimation methods must be documented and pilot tested before the certification process begins. Following certification, survey sponsors must submit annual reports to confirm the design is being implemented as originally described. These documentation requirements are the same documentation requirements described in Recreational Fishing Survey and Data Standards 1, 2, 3, and 5.2 (Survey Concepts and Justification, Survey Design, Data Quality, and Annual Reporting).

Documentation submitted by survey sponsors can be found in the MRIP Reports Database.

Peer Review

Certified survey designs and estimation methods must meet applicable standards and earn validation by a peer review. Independent reviewers, who are experts in the field of survey statistics, must consider the following criteria in conducting their comprehensive evaluation:

• Sample survey designs follow formal probability sampling protocols, with known inclusion probabilities at all sampling stages.
• Estimation methods appropriately weight sample data to account for the sampling design. Both point and variance estimates are produced.
• Methods are in place to measure and/or correct for potential biases due to under-coverage, non-response, and/or response errors.
• The sensitivity of the accuracy of the survey to potential sampling and non-sampling errors is understood, and measures to evaluate, reduce, and/or limit that sensitivity are described.
• The sensitivity of the survey design to potential implementation errors is documented, and measures to evaluate, reduce, and/or limit that sensitivity are described.
• New survey design and/or estimation methods are compared to the design and/or methods they will replace, as well as any other certified survey components currently used to estimate the same population parameters. The relative statistical validity and efficiency of each are described.
• The survey design and/or estimation methods are collecting data and/or producing information that meet science and management needs.

Approval

Certified survey designs and estimation methods must receive a recommendation for approval from the Marine Recreational Information Program’s Executive Steering Committee and final approval from NOAA Fisheries’ Director of Scientific Programs. These decisions are informed by the outcome of the peer review.

Transition

Because stock assessments require consistent, long-term time series of recreational catch, new recreational fishing surveys cannot be incorporated into the cycle of science and management until differences between old and new estimates are examined and accounted for. For this reason, NOAA Fisheries requires the sponsors of any new or improved survey design or estimation method to develop and execute a Transition Plan that describes how estimates produced by the new design will be incorporated into the existing, or legacy, time series.

Fully transitioning to a new survey design may require benchmarking and/or statistical calibration.

• Benchmarking allows different survey designs to be implemented side-by-side. The resulting estimates are examined, differences are measured, and potential drivers of differences are identified and evaluated.
• Calibration accounts for the sources of variation that may be contributing to consistent differences between old and new survey estimates. It can be used to convert historical estimates to the “currency” of the new survey design, thereby preserving the continuity of the historical time series.

The Marine Recreational Information Program’s Transition Team was established to facilitate the transition process. The steps of the transition process, as well as the recommended sections of a Transition Plan, are described in more detail in Procedural Directive 04-114-01 (PDF, 5 pages).

Certified Survey Designs

The Marine Recreational Information Program continues to prioritize the certification of existing survey designs, as well as the certification of and transition planning for new survey designs. Because this is a time- and resource-intensive process, it’s not possible to certify all of the methods currently in use at once. However, NOAA Fisheries can support the continued use of a non-certified, “legacy” survey design, as long as:

• The estimates produced by such a survey design have been used in a peer-reviewed application, such as a federal stock assessment.
• The need to continue using such a survey design has been expressed in an MRIP Regional Implementation Plan.
• A plan to certify such a survey design is in place and is being followed.

The program documents its ongoing progress toward certification and transition planning in its annual implementation plan updates. Additional documentation for the surveys listed below can be found in the MRIP Reports Database. These anticipated timelines are subject to change.

MRIP-Administered Survey Designs

State and Regional Survey Designs

More Information

• Recreational Fishing Survey and Data Standards
• Statistical Calibration Overview