Additional Quantitative Fit-testing Protocols for the Respiratory Protection Standard
Federal Register: January 27, 2010 (Volume 75, Number 17)
Proposed Rules
Page 4323-4331
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
DOCID:fr27ja10-28
DEPARTMENT OF LABOR
Occupational Safety and Health Administration 29 CFR Part 1910
Docket No. OSHA-2007-0007
RIN 1218-AC39
Additional Quantitative Fit-testing Protocols for the Respiratory
Protection Standard
AGENCY: Occupational Safety and Health Administration (OSHA), Labor.
ACTION: Proposed rule; withdrawal.
SUMMARY: After thoroughly reviewing the comments and other information available in the record for the proposed rulemaking, OSHA concludes that the revised PortaCount[supreg] quantitative fit-testing protocols are not sufficiently accurate or reliable to include among the quantitative fit tests listed in Part II of Appendix A of its
Respiratory Protection Standard. Therefore, OSHA is withdrawing the proposed rule without prejudice, and is inviting resubmission of the revised protocols after developers of the protocols address the issues described in this notice.
DATES: The proposed rulemaking is withdrawn as of January 27, 2010.
FOR FURTHER INFORMATION CONTACT: General information and press inquiries: Contact Ms. Jennifer Ashley, Office of Communications, Room
N-3647, OSHA, U.S. Department of Labor, 200 Constitution Avenue, NW.,
Washington, DC 20210; telephone (202) 693-1999.
Technical inquiries: Contact Mr. John E. Steelnack, Directorate of
Standards and Guidance, Room N-3718, OSHA, U.S. Department of Labor, 200 Constitution Avenue, NW., Washington, DC 20210; telephone: (202) 693-2289; facsimile: (202) 693-1678.
Copies of this notice: Electronic copies of this Federal Register notice, as well as news releases and other relevant documents, are available at OSHA's Web page at http://www.osha.gov.
SUPPLEMENTARY INFORMATION:
-
Background
Appendix A of OSHA's Respiratory Protection Standard at 29 CFR 1010.134 currently includes three quantitative fit-testing protocols using the following challenge agents: a non-hazardous generated aerosol such as corn oil, polyethylene glycol 400, di-2-ethyl hexyl sebacate, or sodium chloride; ambient aerosol; and controlled negative pressure.
Appendix A of the Respiratory Protection Standard also specifies the procedure for adding new fit-testing protocols to the standard. The criteria for determining whether OSHA must publish a fit-testing protocol for notice-and-comment rulemaking under Section 6(b)(7) of the
Occupational Safety and Health Act of 1970 (29 U.S.C. 655) include: (1)
A test report prepared by an independent government research laboratory
(e.g., Lawrence Livermore National Laboratory, Los Alamos National
Laboratory, the National Institute for Standards and Technology) stating that the laboratory tested the protocol and found it to be accurate and reliable; or (2) an article published in a peer-reviewed industrial-hygiene journal describing the protocol and explaining how the test data support the protocol's accuracy and reliability. Using this procedure, OSHA added one fit-testing protocol (i.e., the controlled negative pressure REDON quantitative fit- testing protocol) to Appendix A of
Page 4324
its Respiratory Protection Standard (see 69 FR 46986). OSHA also published on December 26, 2007, a Notice of Proposed Rulemaking requesting public comment on an abbreviated Bitrex[supreg] qualitative fit-testing protocol (see 72 FR 72971). Subsequently, OSHA withdrew, without prejudice, this fit-testing protocol from the rulemaking process, and invited the developers of the protocol to conduct further research addressing issues described in the withdrawal notice (see 74
FR 30250).
-
Summary and Explanation of the Withdrawal Notice
-
Introduction
In a letter submitting two new quantitative fit-testing protocols for review under the provisions of Appendix A of OSHA's Respiratory
Protection Standard (Ex. OSHA-2007-0007-0001), Mr. Jeff Weed of TSI,
Inc., included a copy of a peer-reviewed article from an industrial- hygiene journal describing the accuracy and reliability of these proposed protocols (Ex. OSHA-2007-0007-0002).\1\ The submission letter also included instructions that described in detail the equipment and procedures required to administer the proposed protocols. According to this description, the proposed protocols are variations of the existing ambient-aerosol condensation-nuclei-counter quantitative fit-testing protocol developed by TSI, Inc., in the 1980s, commonly referred to as the PortaCount[supreg] quantitative fit-testing protocol (hereafter,
``the standard PortaCount[supreg] QNFT protocol''). OSHA included the standard PortaCount[supreg] QNFT protocol in Appendix A of its final
Respiratory Protection Standard. (For consistency, OSHA will refer to the two proposed protocols as ``revised PortaCount[supreg] quantitative fit-testing protocols 1 and 2'' (i.e., ``revised PortaCount[supreg]
QNFT protocols 1 and 2'').
\1\ This letter and the accompanying article describe three fit- testing protocols, but Mr. Weed of TSI Inc., in a subsequent telephone call to OSHA staff, requested that the Agency include only two of them in the proposed rulemaking.
The proposed protocols use the same fit-testing requirements and instrumentation specified for the standard PortaCount[supreg] QNFT protocol in paragraphs (a) and (b) of Part I.C.3 of Appendix A of the
Respiratory Protection Standard, with the following exceptions:
Revised PortaCount[supreg] QNFT protocol 1 reduces the duration of the eight fit-testing exercises from 60 seconds to 30 seconds; and
Revised PortaCount[supreg] QNFT protocol 2 eliminates two of the eight fit-testing exercises, with each of the remaining six exercises having a duration of 40 seconds; in addition, this proposed protocol increases the current minimum pass-fail fit-testing criterion
(i.e., reference fit factors) from a fit factor of 100 to 200 for half masks, and from 500 to 1000 for full facepieces.
Peer-reviewed industrial-hygiene journal article. The peer-reviewed article submitted by TSI, Inc., entitled `'Evaluation of Three New Fit
Test Protocols for Use With the TSI PortaCount[supreg],'' appeared in the Fall/Winter 2005 issue of the Journal of the International Society for Respiratory Protection (Ex. OSHA-2007-0007-0003). The article describes a study that determined whether performing the proposed protocols yields fit-testing results similar to results obtained with the standard PortaCount[supreg] QNFT protocol (hereafter referred to as
``the Study'').\2\
\2\ The standard PortaCount[supreg] QNFT protocol was the criterion measure or ``gold standard.''
Test subjects and respirator selection. The Study involved 30 test subjects who performed 140 fit tests while wearing elastomeric half- mask and full-facepiece respirators equipped with P100 filters. The test subjects selected respirators from among 24 models, with some test subjects using more than one model during fit testing. Respirator fit varied across the test subjects, with 60 of 140 fit factors below 100, and 91 of 140 fit factors less than 500, as determined by the standard
PortaCount[supreg] QNFT protocol. Poor respirator fit resulted from improper respirator selection by the test subjects themselves, or from assigning respirators to test subjects that were either too small or too large. Test subjects could adjust the respirator for comfort, but they did not perform user seal checks.
Procedures. In conducting the Study, the authors followed the recommendations for evaluating new fit-testing protocols specified by
Annex A2 (``Criteria for Evaluating Fit Tests Methods'') of ANSI
Z88.10-2001 (``Respirator Fit-testing Methods''). Specially designed testing software allowed for the calculation of fit factors every 10 seconds during the in-mask sampling periods without disturbing the facepiece (i.e., at 10-, 20-, and 30-second intervals for comparison with the 40-second in-mask sampling intervals determined using the standard PortaCount[supreg] QNFT protocol). The authors used a TSI
PortaCount[supreg] Plus Model 8020[supreg] quantitative fit-test system to assess respirator fit; the system used a TSI-supplied sampling adaptor, or fixed probes provided by the respirator manufacturer, to collect samples inside the respirators. The sampling point inside the respirator was between the nose and the mouth. During sampling, the test subjects performed the exercises listed in Part I.A.14 of Appendix
A of OSHA's Respiratory Protection Standard, which include: initial normal breathing, deep breathing, turning the head side to side, moving the head up and down, reading a passage, grimace, bending over, and final normal breathing. The TSI PortaCount[supreg] Plus fit-testing instrument performed particle counts on samples collected during the
Study. Table 1 provides the exercise and sampling parameters for each of the protocols used in the Study.
Table 1
In-Mask sampling
Number of
Duration of each duration for each
Protocol
exercises
exercise
exercise
(seconds)
(seconds) \1\
Standard PortaCount[supreg] QNFT Protocol..............
8
60
40
Revised PortaCount[supreg] QNFT Protocol 1.............
8
30
10
Revised PortaCount[supreg] QNFT Protocol 2.............
\2\ 6
40
20
\1\ Does not include 20 seconds for each exercise to collect ambient-air samples and to purge the in-mask and ambient-air sampling tubes.
\2\ This protocol eliminated the initial normal-breathing exercise and the deep-breathing exercise.
Page 4325
Results. The Study results describe the performance of the two revised PortaCount[supreg] QNFT protocols in relation to the reference fit factors (RFFs) that the proposed protocols designate as pass-fail criteria for half-mask respirators (100 and 200 for protocols 1 and 2, respectively) and full-facepiece respirators (500 and 1000 for protocols 1 and 2, respectively). However, OSHA could not evaluate the results for each type of respirator separately because the analyses performed in the Study grouped fit-testing results from half-mask respirators with fit-testing results from full-facepiece respirators.
In this regard, Table III of the Study showed 69 fit tests for half- mask respirators and 71 fit tests for full-facepiece respirators, for a total of 140 fit tests. However, the results in Table III of the Study also list 140 fit tests for RFFs 100, and another 140 fit tests for RFFs 500, when the number of fit tests for each set of RFFs should be 69 and 71, respectively (i.e., 69 fit tests for
RFFs 100, with these RFFs to be applicable to half-mask respirators, and 71 fit tests for RFFs 500, with these RFFs to be applicable to full-facepiece respirators).\3\
\3\ RFFs > 100 include RFFs > 200, which were to be applicable to half-mask respirators, while RFFs > 500 include RFFs > 1000, which were to be applicable to full-facepiece respirators.
Using the standard PortaCount[supreg] QNFT protocol as the criterion measure, the Study described the fit-testing results obtained with the revised PortaCount[supreg] QNFT protocols using the following statistics: test sensitivity; predictive value of a pass; test specificity; predictive value of a fail; and the kappa statistic. These statistics derive from the variables defined by ANSI Z88.10-2001, in which: A = false positives (passed the fit test with a fit factor =
RFF); C = true negatives (failed the fit test with a fit factor = RFF); Po
= observed proportion of the two fit tests that are concordant; and Pe
= expected proportion of the two fit tests expected to be concordant when the two tests are statistically independent. Using these variables, ANSI Z88.10-2001 specifies the formula and recommended value
(``RV'') for each statistic as follows: Test sensitivity = C/(A + C),
RV >= 0.95; predictive value of a pass = B/(A + B), RV >= 0.95; test specificity = B/(B + D), RV > 0.50; predictive value of a fail = C/(C +
D), RV > 0.50; and the kappa statistic = (Po-Pe)/(1-Pe). The following tables list the values of these descriptive statistics for revised
PortaCount[supreg] QNFT protocols 1 (at RFFs of 100 and 500) and 2 (at
RFFs of 200 and 1000).
Table 2--Descriptive Statistics for RFFs of 100 and 200
Revised
Revised
PortaCount[supreg] PortaCount[supreg]
Statistics
ANSI Requirement
QNFT Protocol 1
QNFT Protocol 2
RFF = 100
RFF = 200
Sensitivity..........................................
>=0.95
\1\ 0.91
1.00
Predictive Value of a Pass...........................
>=0.95
\2\ 0.94
1.00
Specificity..........................................
>0.50
0.99
0.81
Predictive Value of a Fail...........................
>0.50
0.98
0.79
Kappa Statistic......................................
>0.70
0.91
0.78
\1\ = Fail; \2\ = Borderline fail.
Table 3--Descriptive Statistics for RFFs of 500 and 1000
Revised
Revised
PortaCount[supreg] PortaCount[supreg]
Statistics
ANSI Requirement
QNFT Protocol 1
QNFT Protocol 2
RFF = 500
RFF = 1000
Sensitivity..........................................
>=0.95
0.97
1.00
Predictive Value of a Pass...........................
>=0.95
\1\ 0.94
1.00
Specificity..........................................
>0.5
0.98
0.84
Predictive Value of a Fail...........................
>0.50
0.99
0.92
Kappa Statistic......................................
>0.70
0.94
0.87
\1\ = Borderline fail.
For a RFF of 100, revised PortaCount[supreg] QNFT protocol 1 failed to meet the sensitivity value specified by ANSI Z88.10-2001, and, consistent with this failure, the value for the predictive value-of-a- pass statistic was marginal. However, for a RFF of 500, the sensitivity value for this proposed protocol exceeded the ANSI requirement, although the predictive value-of-a-pass statistic was again slightly below the ANSI specification. The failure of protocol 1 to achieve the sensitivity value specified by ANSI Z88.10-2001 at a RFF of 100 indicates that the proposed protocol is susceptible to alpha, or false positive, error--i.e., it would pass some half masks that would function below a fit factor of 100 when tested with the protocol used as the criterion measure (i.e., the standard PortaCount[supreg] QNFT protocol). This failure to meet the sensitivity value specified by ANSI
Z88.10-2001 raises a question of whether revised PortaCount[supreg]
QNFT protocol 1 is as protective as the standard PortaCount[supreg]
QNFT protocol. For protocol 1, the authors reported values well above the values established by the ANSI standard for the three remaining statistics, including specificity, predictive value of a fail, and the kappa statistic. However, the grouping of results for half-mask and full-facepiece respirators brings the applicability of these statistics into question.
For PortaCount[supreg] QNFT protocol 2, the sensitivity values for both RFFs were well in excess of the sensitivity value specified by the
ANSI standard. The sensitivity values for this proposed protocol indicate that it identified 100% of the poorly fitting half-mask and full-facepiece respirators. In addition, this
Page 4326
proposed protocol performed well above the values listed in the ANSI standard for the four remaining variables, including predictive value of a pass, specificity, predictive value of a fail, and the kappa statistic. Consistent with the sensitivity values derived for this proposed protocol, these four values indicate that the proposed protocol accurately determined whether respirators achieved, or failed to achieve, RFFs of 200 and 1000. Nonetheless, as mentioned above, the grouping of results for half-mask and full-facepiece respirators brings the applicability of these statistics into question.
In discussing the results for revised PortaCount[supreg] QNFT protocol 2, the authors asserted that excluding the two least strenuous fit-testing exercises (i.e., the initial normal-breathing exercise and the deep-breathing exercise) from this proposed protocol was a conservative approach in that the proposed protocol was more likely than protocols consisting of eight fit-testing exercises to detect respirator leakage (i.e., using data from less strenuous fit-testing exercises inappropriately inflates the overall fit factor for respirators, thereby increasing alpha error). Another conservative approach used by this proposed protocol was raising the RFFs for half masks from a fit factor of 100 to 200, and, for full-facepiece respirators, from 500 to 1000. While this approach may have enhanced the sensitivity of the proposed protocol, it may also increase beta
(false-negative) error; beta error would increase the number of repeated tests and, consequently, the total testing time required by some employees to identify a respirator having an acceptable fit.
-
Decision To Publish the Two Protocols for Notice-and-Comment
Rulemaking
OSHA reviewed the information submitted by TSI, Inc., in support of these proposed protocols to determine whether the protocols met the criteria for determining whether OSHA must publish new fit-testing protocols for notice-and-comment rulemaking established by the Agency in Part II of Appendix A of its Respiratory Protection Standard. The
Agency concluded that the proposed protocols warranted notice-and- comment rulemaking under Section 6(b)(7) of the Act (29 U.S.C. 655), and initiated rulemaking to determine whether to approve these proposed protocols for inclusion in Part I of Appendix A of its Respiratory
Protection Standard. OSHA published the proposal in the Federal
Register on January 21, 2009 (see 74 FR 3526).
-
Issues Raised for Public Comment
In the Federal Register notice announcing the proposal, OSHA invited comments, information, and data from the public regarding the accuracy and reliability of the proposed protocols, effectiveness of the protocols in detecting respirator leakage, and the usefulness of the protocols in selecting respirators that will protect employees from airborne contaminants in the workplace. Specifically, the Agency invited public comment on the following issues:
Were the studies described in the peer-reviewed journal article well controlled, and conducted according to accepted experimental design practices and principles?
Were the results of the studies described in this article properly, fully, and fairly presented and interpreted?
Will the proposed protocols generate reproducible fit- testing results?
Will the proposed protocols reliably identify respirators with unacceptable fit as effectively as the quantitative fit-testing protocols, including the standard PortaCount[supreg] QNFT protocol, already listed in Part I.C.3 of Appendix A of the Respiratory
Protection Standard?
Is the test-sensitivity value of 0.91 obtained for half masks by revised PortaCount[supreg] QNFT protocol 1 acceptable in view of the test-sensitivity value of 0.95 required by ANSI Z88.10-2001; if not, would it be appropriate for OSHA to limit application of revised
PortaCount[supreg] QNFT protocol 1 to full-facepiece respirators?
The Study evaluating the proposed protocols involved only elastomeric half-mask and full-facepiece respirators. Accordingly, is it appropriate to apply the results of the Study to other types of respirators (e.g., filtering-facepiece respirators)?
-
Summary of the Public Comments Received
Twenty-six commenters submitted responses to the proposal. The following paragraphs in this section address the responses made to each of the six issues described previously. 1. Were the studies described in the peer-reviewed journal article well controlled, and conducted according to accepted experimental design practices and principles?
In addressing this issue, the National Institute of Occupational
Safety and Health (NIOSH) stated:
The Study
does not provide sufficient detail about the study design and protocol to enable a complete assessment of how well it was controlled and conducted. The description in the article does indicate that design and principles met acceptable practices.
However, the study design did not include filtering-facepiece respirators (FFR), nor sufficient fit test trials for half-mask respirators or full facepiece respirators to provide data that would allow independent assessment of the performance of the proposed revised protocols for either facepiece type. To fully assess the acceptability of the new protocols for applicability to half-mask respirators (including filtering-facepiece respirators) and full facepiece respirators, each facepiece type needs to be evaluated separately. The data analyses reported in the peer-reviewed journal article grouped fit test results for the half-mask and full facepiece respirators to obtain the minimum number for paired data sets required by ANSI Z88.10-2001, Annex A2. (See Ex. OSHA-2007- 0007-0016.1.)
James S. Johnson (Ex. OSHA-2007-0007-0023.1) and Ching-tsen Bien
(Ex. OSHA-2007-0007-0017.1) both disapproved of the Study's experimental design practices and principles, and specifically criticized the grouping of results for half-mask and full-facepiece respirators. OSHA agrees that grouping results for half-mask and full- facepiece respirators in analyzing RFFs is a major limitation of this study (see, also, the discussion of this issue in paragraph D.2 of this section).
Similar to NIOSH, Ching-tsen Bien questioned the number of fit-test trials performed in the Study. Mr. Bien stated: ``The ANSI Z88.10-2001 requires a minimum of 100-paired tests. The proposed protocol only contains 69-paired tests for the half-mask, and 71-paired test sets for the full facepiece. It failed to meet this requirement.'' In addition, the American Federation of Labor and Congress of Industrial
Organizations (AFL-CIO) criticized the Study for using only 30 participants to generate fit-test data (Ex. OSHA-2007-0007-0015).
In response to the assertion that the Study did not consist of as many fit tests as required under ANSI Z88.10-2001, OSHA emphasizes that it has not adopted the criteria in ANSI Z88.10-2001 as absolute requirements for new fit-testing protocols. Nonetheless, as NIOSH and
Mr. Bien note, it appears that the Study did not consist of a sufficient number of fit tests to establish the respirator-specific performance of the proposed protocols. In response to the AFL-CIO, OSHA notes that researchers should, ideally, validate fit-testing protocols on a large number of study participants to account for variability across the population of employees who use the respirators. However,
OSHA believes the total number of study participants is less
Page 4327
important than the total number of fit tests the participants perform.
NIOSH also criticized the calculation of fit factors for the proposed protocols that used subsets of measurements taken during a standard PortaCount[supreg] fit-test (Ex. OSHA-2007-0007-0016.1). In its comment, NIOSH stated:
For the results of the fit test using shortened exercises to be similar to the reference protocol, the fit of the respirator must not change significantly over time for each fit test exercise. The data are inadequate to demonstrate reproducible fit-testing results for either proposed protocol. Therefore, any subsequent assessment of conformance or non-conformance with the ANSI Z88.10-2001 acceptance criteria cannot be presumed to be valid. Further investigation is required to compare potential changes in fit across the proposed 30- and 40-second exercise intervals in the reference protocol * * *. No information is provided in either the peer- reviewed journal article or application to OSHA that demonstrates the proposed shortened exercise times would encompass the most challenging aspects of each exercise. At a minimum, the frequency and consistency of leaks during each exercise, as well as the magnitude and type of those leaks (e.g. start of exercise, end of exercise, throughout exercise period) need to be identified and analyzed.
Clifton D. Crutchfield (Ex. OSHA-2007-0007-0019.1) and NIOSH (Ex.
OSHA-2007-0007-0016.1) also questioned the assertion by the Study's authors that removal of the initial normal-breathing exercise and the deep-breathing exercises from revised PortaCount[supreg] QNFT protocol 2 results in a conservative fit test. Dr. Crutchfield cited a number of studies to support the proposition that the normal-breathing exercise fit factor is among the lowest of the exercise fit factors, and that its elimination would produce a higher, less conservative, overall fit factor.
The Agency believes that researchers cannot evaluate validly the effects of shortened exercises on respirator fit using subsets of sampling data from a standard, full-length respirator fit test because respirator fit may vary during an exercise. Additionally, OSHA believes that Dr. Crutchfield raised important questions about the removal of the normal-breathing and deep-breathing exercises that the Study's limited data presentation does not fully rebut (see item D.2 of this section).
The Department of Defense (DOD) commented that the Study design was appropriate, but deviated from the ANSI protocol in that user seal checks were not conducted (Ex. OSHA-2007-0007-0021.1). DOD stated:
The DOD views user seal checks to be a necessary element in any respirator program and user seal checks should have been conducted even if the test subject was identified as testing a poorly fitting facepiece. User seal checks are required for performing fit-testing by the OSHA Respirator Standard and by ANSI Z88.10-2001.
In response to this comment, OSHA notes that some study participants used respirators that were too small or too large to ensure that a number of poor respirator fits occurred. This procedure induced poor facepiece-to-face seals, which caused the respirators to leak. These leaks, in turn, provided data for use in determining how effectively the revised PortaCount[supreg] QNFT protocols detected such leaks. Therefore, although the Study did not present a rationale for excluding seal checks, OSHA concludes that the Study needed leakage data to determine the efficacy of the revised PortaCount[supreg] QNFT protocols, which justified the omission.
Ching-tsen Bien (Ex. OSHA-2007-0007-0017.1) and Larry Janssen (Ex.
OSHA-2007-0007-0018.1) recommended that the authors of the Study validate the revised PortaCount[supreg] QNFT protocols using a generated-aerosol procedure in a test chamber. In this regard, Mr. Bien commented:
The PortaCount[supreg] is a field instrument but not a research instrument. For a validation study, the testing should be performed inside a test chamber with a uniform and constant stable concentration. The fit test results should be reported continuously, rather than at selected time intervals. The PortaCount[supreg] utilizes the ambient air as a test agent and the test results may be affected by a change in air particle concentration.
Similarly, Clifton D. Crutchfield wrote (Ex. OSHA-2007-0007-0019.1) that the use of the standard OSHA PortaCount[supreg] protocol as a reference measure for new protocols ``presents a real quandary because the sensitivity of the standard PortaCount protocol has itself not been established.''
In response to these criticisms regarding the use of the standard
PortaCount[supreg] protocol as a reference measure, OSHA notes that none of the existing fit-testing procedures, including generated- aerosol methods, has been validated as a reference tool. In the absence of a fully validated reference test, OSHA requires that new QNFT protocols be evaluated against accepted QNFT methods. Thus, the Agency allows QNFT protocols to be tested against ambient-aerosol protocols, and ANSI Z88.10-2001 provides guidelines for evaluating new QNFT protocols against any of the currently accepted QNFT procedures.
In summary, the commenters raised a number of valid concerns regarding the methodology used in the Study. The Agency concludes that the Study did not implement accepted experimental design practices to the extent necessary to include the revised PortaCount[supreg] QNFT protocols to Appendix A of the Respiratory Protection Standard. 2. Were the results of the studies described in this article properly, fully, and fairly presented and interpreted?
NIOSH (Ex. OSHA-2007-0007-0016.1), James S. Johnson (Ex. OSHA-2007- 0007-0023.1), and Ching-tsen Bien (Ex. OSHA-2007-0007-0017.1) criticized the failure to differentiate clearly the results for half- mask and full-facepiece respirators. Mr. Bien stated:
The purpose of this study should be the comparison between the revised PortaCount and the regular PortaCount methods. Both half- mask and full-facepiece elastomeric respirators were selected for this study. There should be two sets of data, one for each type of mask, since the passing criterion is different for each type of respirator. For each type of respirator, there should be two sets of data; one set for the 60-second exercise, and one set for shorter time or less exercises. Only one set of data is presented in the paper and it combines the half-mask and full-facepiece data.
Similarly, James S. Johnson commented:
Half-mask and full face piece respirators are normally considered two different types of air purifying respirators with different fitting, design and performance properties. The combination of these types of respirators into one set of data for analysis and conclusions doesn't appropriately recognize their performance differences.
OSHA believes that the Study failed to properly differentiate the fit-testing results for half-mask and full-facepiece respirators.
Although OSHA previously approved the controlled negative pressure
(CNP) REDON fit-testing protocol based in part on a study that mixed fit-testing results for half-mask and full-facepiece respirators (Ex. 2-2, Docket No. H-049C), the Agency finds the largely undifferentiated results from the revised PortaCount[supreg] QNFT protocols to be more problematic than the CNP REDON results. In the final rule on the CNP
REDON protocol, OSHA explained that ``[w]hile the Agency agrees that *
* * combining results for different respirator types may lead to inconsistent results with large statistical variations, the peer- reviewed studies showed that large statistical variations did not occur.'' In contrast to the studies submitted for the CNP REDON protocol, the study for the revised PortaCount[supreg] QNFT protocols does not present results in sufficient detail to allow OSHA to examine the variation in fit-testing results. Moreover, while two peer-
Page 4328
reviewed journal articles supported the CNP REDON protocol, the article describing the Study is the sole publication supporting the revised
PortaCount[supreg] QNFT protocols. Therefore, OSHA believes that the failure to differentiate fit-testing results for half-mask and full- facepiece respirators obscures interpretation of the Study's statistics because (1) evaluating the variability of the test results for this study is impossible, and (2) the limited data presentation does not support the revised PortaCount[supreg] QNFT protocols.
NIOSH (Ex. OSHA-2007-0007-0016.1) and Ching-tsen Bien (Ex. OSHA- 2007-0007-0017.1) noted that the Study failed to present clearly a number of important data. For both protocols, NIOSH noted that the
Study provided ``[i]nsufficient detail and data concerning application of the recommended ANSI acceptance criteria for the number of tests performed and the distribution of good and poor fitting respirators in the test population.'' With regard to revised PortaCount[supreg] protocol 2, NIOSH cited a ``lack of detail, data and discussion of performance in relation to the unique acceptable fit factors of 200 for a half-mask and 1000 for a full facepiece respirator.'' Mr. Bien noted that the Study did not follow the ANSI Z88.10-2001 recommendation that investigators present a table containing information on respirator make, model, size, individuals tested, and the results of the new test and fit factors for the reference test. Mr. Bien also observed that
``except for Figure 1 in the paper, the test data is not presented.''
OSHA agrees that the Study did not present a sufficient level of detail regarding individual fit-testing results, the types of respirators selected, and the distribution of respirator fits in the test population. Although the Study provided a histogram showing the distribution of RFFs, these data are difficult to interpret in the absence of information about which fit factors derive from half-mask versus full-facepiece respirators. 3. Will the proposed protocols generate reproducible fit-testing results?
Several commenters, including Ching-tsen Bien (Ex. OSHA-2007-0007- 0017.1), Clifton D. Crutchfield (Ex. OSHA-2007-0007-0019.1), and NIOSH
(Ex. OSHA-2007-0007-0016.1) noted that the data presented in the Study do not facilitate an evaluation of reproducibility. Mr. Bien stated,
``[s]ince the individual test data is not presented in the paper, there is no information to determine the data reproducibility.'' While similarly noting the absence of data describing the variability of fit- testing results in the Study, Dr. Crutchfield drew OSHA's attention to the results of a study by Sreenath et al. (2001). Examining the results of this study, Dr. Crutchfield noted that data from 10-second mask samples had a larger standard deviation than the data from 60-second mask samples.
NIOSH (Ex. OSHA-2007-0007-0016.1) also questioned the reproducibility of the fit-testing results from the revised
PortaCount[supreg] QNFT protocols. Because revised PortaCount[supreg]
QNFT protocol 1 did not meet the ANSI Z88.10-2001 acceptance criteria for sensitivity and predictive value of a pass, NIOSH concluded that protocol 1 would have ``a diminished likelihood of achieving reproducible fit-testing results when compared to the established method.'' With regard to revised PortaCount[supreg] QNFT protocol 2,
NIOSH stated:
The results of the Protocol 2 evaluation are insufficient to conclude that reproducible fit-testing results could be achieved using this protocol. The article does not describe whether each paired set represents the fit factors for a half mask or full facepiece respirator. It appears that some full facepiece respirator paired sets failed to meet the acceptable fit factor at 500. Thus, they were grouped with paired sets of data and treated as meeting the acceptable fit factor of 100, normally used for half mask respirators. These paired sets were also included in the data for failing to meet the required fit factor of 500, normally used for full facepiece respirators.
OSHA believes that NIOSH's comments regarding test sensitivity and the predictive value of a pass address the accuracy, rather than the reproducibility, of the fit-test results. An evaluation of reproducibility would require information concerning the variability of the fit-testing results, which, as noted above, the Study did not provide. However, OSHA agrees that the reproducibility of the data is further obscured by the failure to differentiate clearly the fit- testing results for both half-mask and full-facepiece respirators.
James S. Johnson wrote (Ex. OSHA-2007-0007-0023.1) that
``additional experimental work is needed to determine if the reported results are reproducible when obtained from a representative set of workers following the required manufacturer user instructions and using a user seal check.'' While additional information about the characteristics of the Study participants would allow OSHA to evaluate whether these participants were representative of employees who use the respirators, the Agency finds no evidence that the participants were unrepresentative of the employee population. In addition, while strict compliance with manufacturer instructions may improve fit-test performance, the commenter provided no data indicating that poor compliance with these instructions biased the Study results. Finally, as discussed above (see item D.1 of this section), OSHA determined that omitting seal checks was necessary to determine the efficacy of the revised PortaCount[supreg] QNFT protocols.
Jeff Weed (Ex. OSHA-2007-0007-0014.1) expressed confidence in the reproducibility of the test results from revised PortaCount[supreg]
QNFT protocols 1 and 2, and described the revised exercises as ``long enough to ensure that face leaks are accurately detected.'' Mr. Weed also asserted that the Study ``proved that shortened measurement yields the same result as the longer measurement.'' However, OSHA believes that Mr. Weed failed to address the issue of the reproducibility of the fit-testing results because he did not adequately explain the deficiencies in the data presentation identified elsewhere in this section.
Several commenters, including DOD (Ex. OSHA 2007-0007-0021.1) and
James Johnson (Ex. OSHA-2007-0007-0023.1) recommended that OSHA require additional validation testing before accepting revised
PortaCount[supreg] QNFT protocol 1 or 2, implying that the results were not reproducible.
In summary, the Study did not establish the reproducibility of test results for the revised PortaCount[supreg] QNFT protocols. The Study did not present test results or statistics describing the variability of the results of protocols 1 and 2. Moreover, because of the previously discussed flaws in the data analysis, a meaningful evaluation of the reproducibility of the results is not possible. 4. Will the proposed protocols reliably identify respirators with unacceptable fit as effectively as the quantitative fit-testing protocols, including the standard PortaCount[supreg] QNFT protocol, already listed in Part I.C.3 of Appendix A of the Respiratory
Protection Standard?
Jeff Weed (Ex. OSHA-2007-0007-0014.1) asserted that the revised
PortaCount[supreg] QNFT protocols would perform as well as any of the
QNFT methods, and that the differences between the reference methods and the proposed protocols ``can be easily explained in terms of the limited number of test subjects and instrument variability.'' OSHA believes that any fit-testing protocol based on a study that involved significant instrument
Page 4329
variability and small sample size, as well as a flawed data analysis and an inadequate data presentation, is of questionable validity and utility.
In the view of NIOSH (Ex. OSHA-2007-0007-0016.1), DOD (Ex. OSHA- 2007-0007-0021.1), and Clifton D. Crutchfield (Ex. OSHA-2007-0007- 0019.1), the failure of revised PortaCount[supreg] QNFT protocol 1 to meet the ANSI Z88.10-2001 criteria demonstrates that this protocol will not identify respirators with unacceptable fit as effectively as the accepted QNFT protocols. Because revised PortaCount[supreg] QNFT protocol 2 met the ANSI Z88.10-2001 criteria, DOD concluded that protocol 2 would identify respirators with unacceptable fit as reliably as accepted QNFT methods. In contrast to this view, NIOSH found that
``[u]ncertain data treatment * * * prevent[s] answering the question of whether revised PortaCount[supreg] QNFT protocol 2 will reliably identify respirators with unacceptable fit as effectively as [accepted
QNFT] protocols,'' and ``[t]he report of the test-sensitivity [of this protocol] having surpassed ANSI criteria does not resolve uncertainty.'' Similarly, Ching-tsen Bien (Ex. OSHA-2007-0007-0017.1) wrote that ``[s]ince the individual test data is not available, it is not possible to determine whether the proposed test protocols would reliably identify respirators with unacceptable fit as effectively as the regular quantitative fit-testing protocols.''
OSHA agrees with NIOSH and Mr. Bien that the flawed data analysis and inadequate presentation of fit-testing results (see item D.2 of this section) prevents the Agency from thoroughly evaluating whether either of the proposed protocols would reliably identify respirators with unacceptable fit as effectively as accepted quantitative fit- testing protocols. However, the test-sensitivity value reported for revised PortaCount[supreg] QNFT protocol 1 indicates that this protocol would not identify respirators with unacceptable fit as reliably as accepted quantitative fit-testing protocols.
Clifton D. Crutchfield questioned whether doubling the RFFs for revised PortaCount[supreg] QNFT protocol 2 is sufficient to compensate for the protocol's potential deficiency of test sensitivity, and asserted that Sreenath et al. (2001) multiplied the conventional RFFs by fourteen to ensure the sensitivity of a new protocol that relied on a 20-second in-mask sampling period (Ex. OSHA-2007-0007-0019.1). OSHA agrees that the Study did not discuss adequately the implications of doubling the RFFs. As noted in section A above, increasing the sensitivity of a protocol by raising the RFFs may increase beta (false- negative) error, which would increase the number of repeated tests and, consequently, total testing time. Although the Study reported sensitivity and specificity values for revised PortaCount[supreg] QNFT protocol 2 that exceeded the ANSI criteria, the Study's flawed data analysis and inadequate data presentation bring into question the validity of these values.
In conclusion, OSHA believes that the Study did not analyze or present the fit-testing results in a manner that demonstrates that the proposed protocols would reliably identify respirators with unacceptable fit as effectively as accepted quantitative fit-testing protocols. 5. Is the test-sensitivity value of 0.91 obtained for half masks by revised PortaCount[supreg] QNFT protocol 1 acceptable in view of the test-sensitivity value of 0.95 required by ANSI Z88.10-2001; if not, would it be appropriate for OSHA to limit application of revised
PortaCount[supreg] QNFT protocol 1 to full-facepiece respirators? \4\
\4\ See discussion of grouping fit-testing results for half-mask and full-facepiece respirators under section II.A (``Introduction'') of this notice. Accordingly, commenters generally responded to this issue as though the fit tests comprising RFFs 100 consisted of fit tests for both half-mask and full-facepiece respirators, not just fit tests for half-mask respirators.
Many commenters, including Clifton D. Crutchfield (Ex. OSHA-2007- 0007-0019.1), David Spelce (Ex. OSHA-2007-0007-0013.1), NIOSH (Ex.
OSHA-2007-0007-0016.1), James Johnson (Ex. OSHA-2007-0007-0023.1), DOD
(Ex. OSHA-2007-0007-0021.1), AFL-CIO (Ex. OSHA-2007-0007-0015), and
Ching-tsen Bien (Ex. OSHA-2007-0007-0017.1) expressed the opinion that the test-sensitivity value of 0.91 is unacceptable, and that it would be inappropriate to accept revised PortaCount[supreg] QNFT protocol 1 for use with half-mask or full-facepiece respirators. Dr. Crutchfield noted that ``[t]he test-sensitivity value of 0.95 was the only test statistic designated by ANSI in its Fit Test Methods standard as a criterion value that `shall' be met when accepting new fit test methods.'' NIOSH stated:
The results reported in the peer-reviewed journal article for either reference fit factor (RFF) of protocol 1 do not meet the full criteria of the Annex A2 evaluation standard against which they are to be judged. As such, it would not be appropriate to accept the application of revised PortaCount[supreg] QNFT protocol 1 to either half-mask or full-facepiece respirators.
Larry Janssen (Ex. OSHA-2007-0007-0018.1) and Jeff Weed (Ex. OSHA- 2007-0007-0014.1) commented that the test-sensitivity value of 0.91 is acceptable despite the ANSI criterion sensitivity value of 0.95. In explaining this position, Mr. Janssen stated that instrument variability is approximately 5% of the true value, and asserted that the variability of facepiece-to-face seal leakage in the
Study would increase this variability by at least another 5%. Assuming an overall variability of at least 10%, he questioned whether it is meaningful to calculate sensitivity values to two decimal places. In addition, Mr. Janssen cited a study (Janssen, L.L., et al., 2002) that found that none of the three currently accepted quantitative fit- testing protocols met the ANSI sensitivity criterion of 0.95, noting that ``it would be inappropriate for OSHA to hold new fit tests to a higher standard than the currently accepted fit tests can meet.''
Recognizing that the variability described by Mr. Janssen introduces error into fit-testing measurement, OSHA does not believe that increasing this error further by adopting a sensitivity value of 0.91 would improve employee protection.
OSHA believes that the ANSI Z88.10-2001 standard represents the consensus of the industrial-hygiene community regarding the criteria to use in assessing fit-testing protocols. The majority of the comments to the proposal indicated that the industrial-hygiene community generally supports using the ANSI standard for this purpose. Thus, despite Mr.
Janssen's assertion of an inevitable 10% variability in any fit-testing protocol, and regardless of whether the accepted fit-testing protocols achieve the ANSI criteria, OSHA believes that the ANSI criteria are meaningful measures of performance for new fit-testing protocols, although it does not treat the ANSI criterion for test sensitivity as an absolute requirement for new fit-testing protocols. In considering the test-sensitivity value for the Abbreviated Bitrex Qualitative Fit-
Testing (ABQLFT) protocol, OSHA projected the annual number of employees with improperly fitting respirators who would pass the proposed ABQLFT protocol, which achieved a test-sensitivity value of 0.92, and compared this estimate with the projected number of false- positives expected if the ABQLT protocol achieved the ANSI sensitivity criterion of 0.95. OSHA deemed the excess number of false positives at the test-sensitivity of 0.92 to be unacceptable. (See 74 FR 30250, 30254.) However, OSHA could not make this
Page 4330
determination for revised PortaCount[supreg] QNFT Protocol 1 because the Study did not present adequate fit-testing results to do so.
Nonetheless, the frequency of ambient-aerosol fit testing (see NIOSH-
BLS survey, Ex. 6-3, Docket No. H-049C) indicates that, compared to a fit-testing protocol having a test sensitivity at the ANSI criterion of 0.95, substantially more employees would receive false-positive fit- testing results using revised PortaCount[supreg] QNFT protocol 1. Thus,
OSHA concludes that the test-sensitivity value of 0.91 achieved by revised PortaCount[supreg] QNFT protocol 1 is too low to include this protocol in Appendix A of its final Respiratory Protection Standard.
Jeff Weed recommended that the high test-sensitivity value obtained by revised PortaCount[supreg] QNFT protocol 1 at the RFF of 500 justifies the protocol's acceptance at the RFF of 100 (Ex. OSHA-2007- 0007-0014.1). In this regard, Mr. Weed commented, ``The fact that the testing near 500 had better results than the near 100 results is indicative of the inherent limitations of this type of study including variability of face seal leaks, the instrumentation, and the statistical sample size (number of people).'' Mr. Weed also compared revised PortaCount[supreg] QNFT protocol 1 to the previously proposed
ABQLFT protocol, which also failed to meet the ANSI criterion for test specificity. Mr. Weed stated, ``Any decision by OSHA to reject a protocol based on the ANSI criteria must be applied equally.''
OSHA does not believe that the test-sensitivity value that the
Study reported at the RFF of 500 justifies acceptance of revised
PortaCount[supreg] QNFT protocol 1. Mr. Weed cites variability due to face leaks, instrumentation, and small sample size as possible explanations for an erroneous test-sensitivity result at the RFF of 100. However, OSHA believes that the inconsistency of the test- sensitivity values at RFFs of 100 and 500 raises doubt about both of these values. In addition, as discussed above (see item D.4 of this section), OSHA concluded that instrument variability or a small sample size does not justify acceptance of a protocol with flawed data analyses and inadequate data presentation, particularly when OSHA determined that the ANSI criterion for test sensitivity, although not an absolute requirement for new fit-testing protocols, is reasonable.
Finally, OSHA does not treat the ANSI criteria for test sensitivity as absolute requirements for new fit-testing protocols. Therefore, OSHA would not base a decision to reject a protocol with inadequate test- sensitivity solely on the ANSI criteria. In conclusion, OSHA finds that including revised PortaCount[supreg] QNFT protocol 1 in Appendix A of its final Respiratory Protection Standard is unwarranted because this protocol would allow a substantially larger number of employees to use improperly fitting respirators than would be the case for a protocol that achieves the 0.95 test-sensitivity criterion specified by ANSI
Z88.10-2001. 6. The Study evaluating the proposed protocols involved only elastomeric half-mask and full-facepiece respirators. Accordingly, is it appropriate to apply the results of the Study to other types of respirators (e.g., filtering-facepiece respirators)?
Jeff Weed (Ex. OSHA-2007-0007-0014.1) and Larry Janssen (Ex. OSHA- 2007-0007-0018.1) provided comments in favor of applying the Study results to untested respirator types. In support of this view, Mr.
Janssen wrote, ``There are no data that suggest a measured amount of faceseal leakage for a Class 100 FFR would be somehow different that the same amount of leakage measured on elastomeric facepieces with
Class 100 filters.'' Elaborating on this point, Mr. Weed stated:
Leaks are leaks. An instrument used for QNFT does not ``know'' what type of respirator is attached to the end of the sample tube.
The instrument cannot know the path taken by a particle found in the breathing zone of a respirator. Particles are either present, or not present. As far as the instrument is concerned, there is no difference between leaks in an elastomeric face seal vs. the seal of a filtering-facepiece. The McKay study was conducted with a target fit factors of 100 and 500, which qualifies the application of the resulting protocols for fit-testing any respirator at those values.
NIOSH (Ex. OSHA-2007-0007-0016.1), DOD (Ex. OSHA-2007-0007-0021.1),
AFL-CIO (Ex. OSHA-2007-0007-0015), and Ching-tsen Bien (Ex. OSHA-2007- 0007-0017.1) discouraged application of the Study results to respirator types not tested in the Study. NIOSH stated that it is ``unaware of any studies or data demonstrating that all respirator types perform similarly when being subjected to a fit test,'' and, ``It is inappropriate to conclude that a test result applies to more than just those types of respirators that were tested.'' Similarly, DOD stated:
I t is not appropriate to apply the study results to other types of respirators. * * * There are many types and styles of NIOSH approved filtering-facepiece respirators. There is also ongoing controversy about fit testing, efficacy and actual protection afforded by filtering facepiece respirators given the variation in styles within the class. * * * Any change to current QNFT protocols that allow filtering facepiece respirators (as a class) to be included should be based on actual fit testing data per ANSI Z88.10- 2001 or the current edition.
Larry Janssen asserted that Class 100 filtering-facepiece respirators are the only filtering-facepiece respirators that would be appropriate for fit-testing using the revised PortaCount[supreg] QNFT protocols (Ex. OSHA-2007-0007-0018.1). Clifton D. Crutchfield questioned whether any filtering-facepiece respirators can be effectively fit tested with the PortaCount[supreg] N-95 Companion using the proposed protocols (Ex. OSHA-2007-0007-0019.1). Dr. Crutchfield stated, ``The [N-95] Companion can * * * report fit factors only up to 200. This obviously precludes the use of Revised PortaCount[supreg]
Protocol 2.'' Dr. Crutchfield also noted that revised
PortaCount[supreg] QNFT protocol 1 has an in-mask sampling time of 10 seconds, which ``allows sampling only about 2 breaths per exercise in order to determine an in-mask concentration for that exercise.'' In the absence of data demonstrating that the PortaCount[supreg] N-95
Companion can effectively measure respirator leakage in ten seconds,
Dr. Crutchfield remarked that ``allowing such fit-testing to occur would be neither justified nor prudent.''
OSHA does not believe that it is appropriate to apply the fit- testing results to types of respirators not tested in the Study. While
Mr. Janssen emphasizes the absence of data demonstrating that fit- testing protocols perform differently on different respirator types,
OSHA views this lack of information on the consistency of fit-test performance as a reason to avoid generalizing from the results of the
Study. Accordingly, OSHA believes that it would be prudent to validate new fit-test protocols using filtering-facepiece respirators because filtering-facepiece respirators are the most commonly used respirator.
(See Table 30, NIOSH-BLS survey, Ex. 6-3, Docket No. H-049C.)
However, as Dr. Crutchfield and Mr. Janssen note, a question remains as to whether filtering-facepiece respirators can be effectively fit tested using the revised PortaCount[supreg] QNFT protocols. In view of the considerable uncertainty as to the consistency of fit-test protocol performance on different respirator types, OSHA concludes that the Study did not establish that the revised
PortaCount[supreg] QNFT protocols will accurately determine fit for N95 filtering-facepiece respirators.
-
Conclusions
Based on a complete and thorough review of the rulemaking record,
OSHA concludes that:
Page 4331
-
The Study was not conducted according to accepted experimental design practices and principles. 2. The Study did not properly or fully describe the fit-testing results. 3. The Study did not establish the reproducibility of the results generated by the revised PortaCount[supreg] QNFT protocols. 4. The Study did not demonstrate that the revised
PortaCount[supreg] QNFT protocols will identify respirators with unacceptable fit as effectively as the quantitative fit-testing protocols already listed in Part I.C.3 of Appendix A of OSHA's
Respiratory Protection Standard. 5. The reported test-sensitivity value of 0.91 indicates that revised PortaCount[supreg] QNFT protocol 1 would allow a substantial number of employees to pass fit tests with improperly fitting respirators compared to a protocol that achieves the 0.95 sensitivity value that ANSI Z88.10-2001 lists as a criterion measure for new fit- testing protocols. 6. The Study did not demonstrate that the revised
PortaCount[supreg] QNFT protocols will accurately determine fit for filtering-facepiece respirators.
Additional validation testing of, or revisions to, the revised
PortaCount[supreg] QNFT protocols may provide new data that demonstrate the accuracy and reproducibility of the fit-testing results generated by these protocols. OSHA would evaluate any new data and supporting documentation received, and, if appropriate, would submit it to the public for notice and comment. If the revised protocols are to apply to filtering-facepiece respirators, then the resubmission must include appropriate fit-testing results for these respirators.
List of Subjects in 29 CFR Part 1910
Fit testing, Hazardous substances, Health, Occupational safety and health, Respirators, Toxic substances.
Authority and Signature
David Michaels, PhD, MPH, Assistant Secretary of Labor for
Occupational Safety and Health, U.S. Department of Labor, 200
Constitution Avenue, NW., Washington, DC 20210, directed the preparation of this notice. Accordingly, the Agency issues this notice under the following authorities: Section 4, 6(b), 8(c), and 8(g) of the
Occupational Safety and Health Act of 1970 (29 U.S.C. 653, 655 657);
Section 3704 of the Contract Work Hours and Safety Standards Act (40
U.S.C. 3701 et seq.); Section 41 of the Longshore and Harbor Worker's
Compensation Act (33 U.S.C. 941); Secretary of Labor's Order No. 5-2007
(72 FR 31160); and 29 CFR part 1911.
Signed at Washington, DC, on January 22, 2010.
David Michaels,
Assistant Secretary of Labor for Occupational Safety and Health.
FR Doc. 2010-1656 Filed 1-26-10; 8:45 am
BILLING CODE 4510-26-P
-
-
