Abbreviated Bitrex Qualitative Fit-Testing Protocol; Withdrawal
Federal Register: June 25, 2009 (Volume 74, Number 121)
Proposed Rules
Page 30250-30256
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
DOCID:fr25jn09-20
DEPARTMENT OF LABOR
Occupational Safety and Health Administration 29 CFR Part 1910
Docket No. OSHA-2007-0006
RIN 1218-AC29
Abbreviated Bitrex[supreg] Qualitative Fit-Testing Protocol
AGENCY: Occupational Safety and Health Administration (OSHA); Labor.
ACTION: Proposed rule; withdrawal.
SUMMARY: After thoroughly reviewing the comments and other information available in the record for the proposed rulemaking, OSHA decided that the abbreviated Bitrex[supreg] qualitative fit test is not sufficiently accurate to include among the qualitative fits tests listed in Part II of Appendix A of its Respiratory Protection Standard. Therefore, OSHA is withdrawing the proposed rule without prejudice, and is inviting resubmission of the proposed fit test after conducting further research to improve the accuracy of the protocol.
DATES: Effective June 25, 2009, the proposed rule published December 26, 2007 (72 FR 72971) is withdrawn.
FOR FURTHER INFORMATION CONTACT: General information and press inquiries: Contact Ms. Jennifer Ashley, Office of Communications, Room
N-3647, OSHA, U.S. Department of Labor, 200 Constitution Avenue, NW.,
Washington, DC 20210; telephone (202) 693-1999.
Technical inquiries: Contact Mr. John E. Steelnack, Directorate of
Standards and Guidance, Room N-3718, OSHA, U.S. Department of Labor, 200 Constitution Avenue, NW., Washington, DC 20210; telephone: (202) 693-2289; facsimile: (202) 693-1678. Electronic copies of this Federal
Register notice, as well as news releases and other relevant documents, are available at OSHA's Web page at http://www.osha.gov.
SUPPLEMENTARY INFORMATION:
-
Background
Part I to Appendix A of OSHA's Respiratory Protection Standard at 29 CFR 1910.134 currently includes four qualitative fit-testing protocols using the following challenge agents: Isoamyl acetate; saccharin-solution aerosol; Bitrex[supreg] (denatonium benzoate) aerosol in solution; and irritant smoke (stannic chloride). Part II to
Appendix A specifies the procedure by which OSHA determines whether to propose adding a new fit-testing protocol to the Respiratory Protection
Standard. The criteria used in making this determination include: (1) A test report prepared by an independent government research laboratory
(e.g., Lawrence Livermore National Laboratory, Los Alamos National
Laboratory, the National Institute for Standards and Technology) stating that the laboratory tested the protocol and found it to be accurate and reliable; or (2) an article published in a peer-reviewed industrial-hygiene journal describing the protocol and explaining how the test data support the protocol's accuracy and reliability. If a fit-testing protocol meets one of these criteria, OSHA must initiate notice-and-comment rulemaking on the proposed fit-testing protocol under Section 6(b)(7) of the Occupational Safety and Health Act of 1970
-
Summary and Explanation of the Withdrawal Notice
A. Introduction
In the letter submitting the abbreviated Bitrex[supreg] qualitative fit-testing (ABQLFT) protocol for review under the provisions of
Appendix A of OSHA's Respiratory Protection Standard (Ex. OSHA-2007- 0006-0002), Dr. Michael L. Runge of the 3M Company included a copy of a peer-reviewed article from an industrial-hygiene journal describing the accuracy and reliability of the ABQLFT protocol (Ex. OSHA-2007-0006- 0003). This article also described in detail the equipment and procedures required to administer the ABQLFT protocol. According to this description, the protocol is a variation of the existing
Bitrex[supreg] qualitative fit-testing protocol developed by the 3M
Company in the early 1990s, which OSHA approved for inclusion in the final Respiratory Protection Standard. The ABQLFT protocol uses the same fit-testing requirements and instrumentation specified for the existing Bitrex[supreg] qualitative fit-testing protocol in paragraphs
(a) and (b) of Part I.B.4 of Appendix A of the Respiratory Protection
Standard, with the following two exceptions:
Exercise times are reduced from 60 seconds to 15 seconds; and
The ABQLFT protocol is used only with test subjects who can taste the Bitrex[supreg] screening solution within the first 10 squeezes of the nebulizer bulb (referred to as ``Level 1 sensitivity'').
The peer-reviewed article submitted by the 3M Company describing the study conducted on the ABQLFT, entitled ``Development of an
Abbreviated Qualitative Fit Test Using Bitter Aerosol,'' appeared in the Fall/Winter 2003 issue of the Journal of the International Society for Respiratory Protection (hereafter, ``the ABQLFT study'' or ``the study''; Ex. OSHA-2007-0006-0003). The authors of the study were T.J.
Nelson of NIHS, Inc., and L.L. Janssen, M.D. Luinenburg, and H.E.
Mullins of the 3M Company; the 3M Company supported the study. The study described by the article determined whether performing a fit test involving seven exercises lasting 15 seconds each while exposed to
Bitrex[supreg] solution aerosol yielded fit-testing results similar to results obtained with a generated-aerosol (i.e., corn oil) quantitative fit test (GAQNFT) using one-minute exercises (i.e., the GAQNFT was the criterion measure or ``gold standard'').
Page 30251
The study involved 43 experienced respirator users, 20 females and 23 males. The test subjects followed the existing Bitrex[supreg] qualitative fit-testing protocol in Appendix A of OSHA's Respiratory
Protection Standard except that they performed each of the fit-testing exercises for 15 seconds (instead of 60 seconds) while wearing a NIOSH- certified elastomeric half-mask respirator equipped with P100 filters.
The authors selected the best fitting respirator for each test subject from among four models, each available in three sizes; some test subjects used more than one model during fit testing. In addition, the authors induced poor respirator fits by assigning a respirator to test subjects that was one or two sizes too small or too large as determined by the Los Alamos National Laboratory panel-grid size and observation of the test subjects' facial characteristics. Test subjects could adjust the respirator facepiece for comfort, but they did not perform user seal checks.
In conducting the study, the authors used the recommendations for evaluating new fit-test methods specified by Annex A2 of ANSI Z88.10- 2001, including sequencing the ABQLFT and GAQNFT in random order without disturbing facepiece fit. The authors used fit-test sample adaptors or respirators with fixed probes to collect samples inside the respirator. The sample point inside the respirator was located between the nose and the mouth. For both fit tests, the authors had the test subjects perform seven of the eight exercises listed in Part I.A.14 of
Appendix A of OSHA's Respiratory Protection Standard, which included:
Normal breathing, deep breathing, turning the head side to side, moving the head up and down, reading a passage, bending over, and normal breathing.\1\ For the GAQNFT, the authors performed particle counts at one-second intervals inside a test chamber for 15-30 seconds before and after fit testing, and inside the respirator for the 60-second duration of each exercise.
\1\ The test subjects did not perform the grimace exercise.
The 43 test subjects used in the study had Level 1 sensitivity to
Bitrex[supreg] because they were able to taste the Bitrex[supreg] aerosol within 10 squeezes of the nebulizer bulb. Subjects having Level 2 or 3 sensitivity to Bitrex[supreg] were excluded from further participation in the study because the nebulizer could not be replenished for additional taste testing within the 15 seconds allotted to perform each fit-testing exercise. After the test subjects passed a
Bitrex[supreg] sensitivity-screening test, the authors administered the
ABQLFT using the procedures and techniques specified for the existing
Bitrex[supreg] qualitative fit-testing protocol in Part I.B.14 of
Appendix A of OSHA's Respiratory Protection Standard, and determined the fit factor using the particle count for the 15-second duration of each exercise.
The authors required a fit factor of 100 to pass a fit test, which served as the basis for determining the following statistics for the
ABQLFT: Test sensitivity; predictive value of a pass; test specificity; and predictive value of a fail. In calculating these statistics, the authors adopted the variables defined by ANSI Z88.10-2001, in which: A
= false positives (passed the fit test with a fit factor = 100); C = true negatives (failed the fit test with a fit factor = 100). Using these variables, ANSI Z88.10-2001 specifies the formula and recommended value
(RV) for each statistic as follows: Test sensitivity = C/(A + C), RV >= 0.95; predictive value of a pass = B/(A + B), RV >= 0.95; test specificity = B/(B + D), RV > 0.50; and predictive value of a fail = C/
(C + D), RV > 0.50.
Using the GAQNFT as the criterion measure, the variables for the
ABQLFT had the following values: A = 4; B = 95; C = 48; and D = 20. The statistics calculated for the ABQLFT from these values were: Test sensitivity = 0.92; predictive value of a pass = 0.96; test specificity
= 0.83; and predictive value of a fail = 0.71. Therefore, every statistic for the ABQLFT, except test sensitivity, attained a value in excess of the ANSI Z88.10-2001 recommended value.
The test-sensitivity value of 0.92 for the ABQLFT fell below the
ANSI recommended value of 0.95. The authors state that this slight difference represents a single false positive value for the ABQLFT
(i.e., failed the GAQNFT but passed the ABQLFT). However, an additional peer-reviewed article submitted by Dr. Runge of the 3M Company suggests an alternative approach to examining these test-sensitivity values (see
Ex. OSHA-2007-0006-0004). This article, entitled ``Recommendations for the Acceptance Criteria for New Fit Test Methods'' and published in the
Spring/Summer 2004 issue of the Journal of the International Society for Respiratory Protection, describes an analytical study conducted by
T. J. Nelson of NIHS, Inc. and H. Mullins of the 3M Company, and supported by the 3M Company. In this study, the authors performed a binary logistic-regression analysis on pass-fail fit-testing data from published studies involving two quantitative, and two qualitative, fit tests. The authors justify using the binary logistic-regression analysis for this purpose as follows:
When a simple sensitivity test is used to describe a new test, the result can be affected by the distribution of the data. In several cases using the theoretical distributions described in this paper, the outcome of a sensitivity test for the Bitrex and Ambient
Particle Counter fit tests could have failed to meet the ANSI Z88.10 sensitivity requirement. The method used to determine acceptability should be independent of specific data collected. (See Ex. OSHA- 2007-0006-0004, p. 8.)
The results of the binary logistic-regression analysis performed on the ABQLFT data showed that the ABQLFT had a 0.20 probability of passing a respirator user with a fit factor of 50 and a 0.33 probability of passing a respirator user with a fit factor of 100.
Figure 3 of the article compares the binary logistic-regression analysis results of test-sensitivity values obtained for a popular quantitative fit test and the existing 60-second Bitrex[supreg] qualitative fit test. The authors conclude that the analysis demonstrates that the distribution of fit-testing data affected the test-sensitivity values derived using the ANSI Z88.10-2001 test- sensitivity calculations. Based on this analysis, the authors assert that ``a sensitivity calculation may not be the best indicator of fit test method performance. The binary logistic regression analysis shows that the result of the 15 second exercise time test is very similar to the ambient aerosol and 60 second bitter aerosol tests'' (Ex. OSHA- 2007-0006-0003, p. 108). In summarizing the results, the authors state that ``[t]he 15 second bitter aerosol protocol sufficiently screens for adequate respirator fit in subjects with Level 1 Bitrex taste sensitivity.''
After carefully reviewing the peer-reviewed articles submitted in support of the ABQLFT, OSHA determined that the protocol met the second criterion specified in Appendix A of the Respiratory Protection
Standard, and then developed a proposal to add a new fit-testing protocol to the standard. OSHA published the proposal in the Federal
Register on December 26, 2007 (see 72 FR 72971).
B. Issues Raised for Public Comment
In the Federal Register notice announcing the proposal, OSHA invited comments and data from the public regarding the accuracy and reliability of the proposed ABQLFT protocol, its effectiveness in detecting respirator leakage, and its usefulness in selecting respirators that will protect employees
Page 30252
from airborne contaminants in the workplace. Specifically, the Agency invited public comment on the following issues:
Were the studies described in the submitted articles well controlled, and conducted according to accepted experimental design practices and principles?
Were the results of the studies described in the submitted articles properly, fully, and fairly presented and interpreted?
Will the proposed ABQLFT protocol generate reproducible fit-testing results, and what additional experiments or analyses of existing data are necessary to answer this question?
Will the proposed ABQLFT protocol reliably identify respirators with unacceptable fit as effectively as the qualitative fit-testing protocols, including the existing Bitrex[supreg] qualitative fit-testing protocol, already listed in Part I.B of
Appendix A of the Respiratory Protection Standard?
What is the significance of the test-sensitivity value of 0.92 obtained for the ABQLFT relative to the test-sensitivity value of 0.95 recommended by ANSI Z88.10-2001, and does the authors' assertion that ``a sensitivity calculation may not be the best indicator of fit test method performance'' adequately account for the lower test- sensitivity value?
What is the significance of limiting the ABQLT to respirator users who demonstrate Level 1 sensitivity to Bitrex[supreg]?
C. Summary of the Public Comments Received
Twenty-two commenters submitted responses to the proposal. The following paragraphs in this section address the responses made to each of the six issues described previously, as well as additional issues addressed by the commenters themselves. 1. Were the studies described in the submitted articles well controlled, and conducted according to accepted experimental design practices and principles? In addressing this issue, NIOSH stated:
The primary journal article cited, Development of an Abbreviated
Qualitative Fit Test Using Bitter Aerosol by Nelson et al. [2003], does not provide sufficient detail about the study design and protocol to enable a complete assessment of how well it was controlled and conducted. The description in the article does indicate that design and principles met acceptable practices. (See
Ex. OSHA-2007-0006-0026.)
Jeff Weed asserted that the study did not exclude from the statistical analysis the fit factors used to determine the reference-method fit factors within one standard deviation of the required fit factor, a determination required under ANSI Z88.10-2001 (Ex. OSHA-2007-0006- 0020.1).
Generally, the NIOSH comment appears to support the design practices and principles used in the study, and did not elaborate on what additional detail would ``enable a complete assessment of how well
the study
was controlled and conducted.'' Jeff Weed's comment appears to be mistaken because page 104 of the article describing the study
(see Ex. OSHA-2007-0006-0003) states that the ``[f]ive fit factors within one standard deviation of the required fit factor of 100 (86 to 114) were excluded from the data analysis as recommended by Z88.10.''
Therefore, OSHA concludes that the study was well controlled, and conducted according to accepted experimental design practices and principles. 2. Were the results of the studies described in the submitted articles properly, fully, and fairly presented and interpreted? NIOSH made the following comments regarding this issue:
NIOSH is concerned that the interpretation of the study results does not appropriately represent the performance of the fit testing protocol. The authors correctly stated that a shortened bitter aerosol fit test method relies on two assumptions: (1) Fit does not significantly change during an exercise and (2) people being tested will respond to the bitter taste of Bitrex[supreg] in the shorter time period. The results of the study support the second assumption, i.e., the test subjects classified with Level 1 sensitivity responded to the bitter taste of Bitrex[supreg] in the shorter time period. However, the study results do not provide convincing evidence to support the first assumption. * * *
The consistency of the respirator's fit throughout each of seven exercises is important in the assessment of the performance of the
ABQLFT fit test protocol. The fit factor assigned for each ABQLFT exercise in the study is based on a 15-second increment, in contrast to a 60-second increment for each of the same exercises performed in quantitative fit test (GAQNFT) protocol. Change in fit during an exercise suggests that the fit at the start of the next 60-second exercise in the GAQNFT is more likely to differ from the fit at the start of the corresponding 15-second exercise period of the ABQLFT.
There is no indication that the authors considered the significance of the noted changes in fit on the accuracy of the assigned fit factors. (See Ex. OSHA-2007-0006-0026.)
Pages 104, 105, and 107 of the article describing the study (see Ex.
OSHA-2007-0006-0003) addressed NIOSH's concerns about the variability of respirator fit for the 15-second and 60-second exercise periods, at least for the GAQNFT. Page 104 of the article states that the correlation between fit factors assessed for the two exercise periods was highly significant, with r = 0.97, while the text and figure on page 108 of the article note that variability was low for fit factors less than 100 and over 6,000. These results demonstrate convincingly that respirator fit factors, especially for fit factors in the range of interest (i.e., having values at and below 100), were reasonably consistent and stable across the 15-second and 60-second exercise periods.
Jeff Weed commented (see Ex. OSHA-2007-0006-0020.1) that the study did not report a Kappa value, which ANSI Z88.10-2001 defines as the
``statistic (K) used to calculate some degree of agreement between two fit tests''; the ANSI standard recommends a minimum Kappa value greater than 0.70. Based on the equation for the Kappa statistic provided in
Annex A2 of the ANSI standard, Mr. Weed calculated the Kappa value for the study data as 0.69, which corresponds closely to our calculation of 0.70, rounded from a figure of 0.69565. OSHA concludes, that the Kappa value calculated from the study data indicates an acceptable degree of agreement between the two fit tests used in the study, and conforms satisfactorily with the value recommended by the ANSI standard. 3. Will the proposed ABQLFT protocol generate reproducible fit- testing results, and what additional experiments or analyses of existing data are necessary to answer this question? NIOSH questioned the reproducibility of the fit-testing results, stating:
Based on review of Nelson et al. [2003] and Nelson and Mullins
2004
, NIOSH concludes that the evidence is inadequate to demonstrate reproducible fit testing results. Further investigation is required to compare potential changes in fit across the proposed 15-second exercise intervals in the ABQLFT protocol and the standard 60 second exercise intervals in the GAQNFT protocol. At a minimum, the frequency and consistency of leaks during each exercise, as well as the magnitude and type of those leaks (e.g. start of exercise, end of exercise, throughout exercise period) need to be identified and analyzed. (See Ex. OSHA-2007-0006-0026.)
OSHA addressed NIOSH's concern regarding the variability of respirator fit for the 15-second and 60-second exercise periods above (see item
C.2 of this section).
Jeff Weed questioned whether employers could reproduce the results of the ABQLFT study in the workplace, stating:
When qualitative fit test (QLFT) methods such as the ABQLFT are performed in a laboratory by researchers, the results are
Page 30253
reasonably reproducible. Researchers are keenly aware of the potential mistakes that cause variability, such as the manner in which the nebulizer bulb is squeezed (e.g. fully vs. partly, with the palm vs. the fingers, slowly vs. quickly). The way the nebulizer is used has a significant affect on the mass of agent that is injected into the fit test hood. Unfortunately, studies such as the one by Nelson do not take the practicality of the fit test method into account, when implemented by lay-persons. (See Ex. OSHA-2007- 0006-0020.1.)
The authors of the ABQLFT study mention on page 103 of the article describing the study (see Ex. OSHA-2007-0006-0003) that ``[t]he bitter aerosol fit test followed the procedure outlined in the OSHA respirator standard, except that a 15 second exercise period was used.'' Section
B.4 of Part I in Appendix A of that standard describes in elaborate detail how to administer properly the Bitrex[supreg] solution aerosol using the nebulizer bulb. OSHA holds that this description of the procedure is adequate, and that employers are responsible for complying fully with the procedure as described in OSHA's Respiratory Protection
Standard. In addition, Mr. Weed's comment appears to be speculative in that he provided no evidence to support it.
Ching-tsen Bien mentioned that ``[t]here is only one repeated test on the same test subject with a standard deviation of 14'' (Ex. OSHA- 2007-0006-0017.1). In a response to Mr. Bien, Robert A. Weber of 3M stated (see Ex. OSHA-2007-0006-0021.1) that Mr. Bien's comment describes the requirement specified in Annex A2 of ANSI Z88.10-2001.
Mr. Weber quotes this requirement from Annex A2 as follows: ``One standard deviation for the reference method can be approximated by identifying a subject having a fit factor near the required fit factor and making measurements on this subject during a single mask donning to determine system reproducibility.'' OSHA believes that Mr. Weber's response appropriately addresses Mr. Bien's concern. 4. Will the proposed ABQLFT protocol reliably identify respirators with unacceptable fit as effectively as the qualitative fit-testing protocols, including the existing Bitrex[supreg] qualitative fit- testing protocol, already listed in Part I.B of Appendix A of the
Respiratory Protection Standard? Pete Stafford of the Building and
Construction Trades Department, AFL-CIO, questioned whether the 15- second exercise periods prescribed by the proposed ABQLFT protocol were sufficient to challenge the face-to-facepiece seal, stating:
In the abbreviated protocol, normal and deep breathing exercises would only allow four to five breaths in 15 seconds. Side to side and up and down exercises might only allow one cycle of each in 15 seconds. The talking exercise would be difficult to accomplish, as the rainbow passage presents a variety of facial expressions, and could not be completed in the 15 second time frame.'' (See Ex. OSHA- 2007-0006-0024.)
NIOSH argued that with the aerosol concentration replenished only once every 30 seconds, the exercise occurring during the first 15 seconds of this 30-second period would be near the maximum aerosol concentration, while the exercise occurring during the last 15-second period would be near the minimum concentration that occurs after filtration removes much of the aerosol from the hood. NIOSH further noted:
* * * [T]he 60-second exercise duration in the OSHA-accepted
Bitrex[supreg] protocol would be conducted through two complete 30- second concentration-cycles, whereas the 15-second exercises of the
ABQLFT were conducted through only half of one. While the variation in the aerosol concentration during this procedure has not been documented, the fact that the replenishing amount is half the quantity to establish the appropriate test challenge (for a fit factor of at least 100) suggests that variability could significantly affect the results. In addition, the variability in subjects' ability to taste Bitrex[supreg] at reduced concentrations, and the impact on the pass/fail results, needs to be determined and analyzed. (See Ex. OSHA-2007-0006-0024.)
OSHA finds that the comments submitted by both Pete Stafford and
NIOSH did not adequately consider the effects the alleged deficiencies should have on the results of the ABQLFT study. Failure to adequately challenge the facepiece-to-face seal, and low levels of aerosol present during an exercise, should increase the number of false positives, but the study data show no such effect. Therefore, absent any supporting data or analyses, OSHA considers these comments to be speculative.
A number of commenters stated that the proposed protocol would not reliably assess proper fit for filtering-facepiece respirators because the authors did not include these respirators in the study design. In this regard, NIOSH noted that ``the submitted study did not include any filtering facepiece respirators. This type of respirator is commonly used and likely to be evaluated by the ABQLFT protocol. NIOSH encourages evaluation of filtering facepiece respirators before acceptance of the ABQLFT protocol'' (Ex. OSHA-2007-0006-0026). Ching- tsen Bien asserted that ``the validation testing should be performed on a variety of shapes of N-95 filtering facepieces to ensure that this method would reject inadequate fits for respirators of this type'' (Ex.
OSHA-2007-0006-0017.2).
OSHA received additional comments on this issue from Timothy
Roberts, who stated, ``Another major concern is that the primary article [Nelson, 2003] did not include filtering facepiece respirators as part of the tests. Filtering facepiece respirators are often tested with the Bitrex qualitative protocol and therefore, the data may not be representative of the adequacy of the ABQLFT proposal for this class of respirators'' (Ex. OSHA-2007-0006-0022). James S. Johnson recommended further testing of filtering facepieces using the proposed ABQLFT protocol, noting, ``A similar study (Article 1) needs to be done with filtering facepiece respirators to demonstrate acceptable performance is achieved with this type of half mask respirator'' (Ex. OSHA-2007- 0006-0028).
Robert Weber of 3M addressed the issue of testing filtering- facepiece respirators in his comments (see Ex. OSHA-2007-0006-0021.1), stating, ``It is not possible to use N95 filtering facepieces to validate a fit test with submicrometer particle QNFT,'' adding that
``[i]t is an evaluation of facepiece[-]to-face seal only; filter penetration is not included. While filter penetration of submicrometer particles through N95 filters is small, it is not zero.'' Mr. Weber concludes, ``The use of N95 [filtering-facepiece respirators] would therefore skew the data by increasing [false-negative] error, i.e. rejecting adequate fits.''
Contrary to Mr. Weber's comments, OSHA finds that testing N95 filtering-facepiece respirators as recommended by the other commenters is not validation testing, but instead is testing that would demonstrate that the proposed ABQLFT protocol performs adequately with
N95 filtering-facepiece respirators, even when filter penetration increases false-negative error. Therefore, OSHA could not approve using the proposed ABQLFT protocol for fit testing filtering-facepiece respirators absent appropriate results demonstrating that the proposed protocol adequately determines fit for these respirators. 5. What is the significance of the test-sensitivity value of 0.92 obtained for the ABQLFT relative to the test-sensitivity value of 0.95 recommended by ANSI Z88.10-2001, and does the authors' assertion that
``a sensitivity calculation may not be the best indicator of fit test method performance'' adequately account for the lower test-sensitivity
Page 30254
value? In addressing the first part of this issue (i.e., the significance of the test-sensitivity value of 0.92), Jeff Weed stated,
``[I]t should be noted that of the 5 ANSI criteria, test sensitivity is the only one that ANSI states `shall' be met. The others carry the
`should' qualifier. In ANSI parlance (paragraph 1.3), the word `shall' implies a mandatory provision, and `should' is used for advisory provisions'' (Ex. OSHA-2007-0006-0020.1). Similarly, Bill Kajola of the
AFL-CIO stated recommended that OSHA withdraw the proposed rule because
``the most important ANSI criterion for approving a new test method has not been achieved,'' and that ``[t]he research paper used by 3M in support of its application for approval (Ex. OSHA-2007-0006-0003) acknowledges the failure of the 15 second Bitrex fit test protocol to achieve the ANSI test sensitivity of 0.95 or greater, a consensus criteria established by the respiratory protection community'' (Ex.
OSHA-2007-0006-0019.1). Timothy Roberts, Mark Haskew, and Ching-tsen
Bien stated that failure to achieve the ANSI test-sensitivity criterion was sufficient justification for OSHA not to adopt the ABQLFT (see Exs.
OSHA-2007-0006-0022, -0023, and 0017.2, respectively). NIOSH believed that the reduced sensitivity-test value demonstrated that the proposed
ABQLFT protocol was defective, stating, ``A sufficient number of subjects met fit testing requirements using the ABQLFT protocol and failed using the GAQNFT protocol,'' and that ``[t]he sensitivity test is a critical criterion to ensure the rejection of inadequately fitting respirators'' (Ex. OSHA-2007-0006-0026). NIOSH concluded that
``[b]ecause the observed value of 0.92 is below the ANSI criterion of 0.95, NIOSH considers the value unacceptable.''
In the article describing the ABQLFT study (see Ex. OSHA-2007-0006- 0003), the authors state that ``[a]dvisory criteria for evaluating new fit test methods outlined in Annex A2 to ANSI Standard Z88.10-2001 were used. * * *'' Therefore, the authors adopted the ANSI standard as the method by which to evaluate the results of the study, including the test-sensitivity criterion which, as stated above by Mr. Weed, is the only criterion in the ANSI standard that is mandatory. OSHA believes adopting the ANSI standard is appropriate because that standard represents the consensus of the industrial-hygiene community regarding the criteria to use in assessing fit-testing protocols. The comments described in the previous paragraph clearly demonstrate that the industrial-hygiene community generally supports using the ANSI standard for this purpose.
In comments submitted to the record, Robert Weber of 3M noted that
``there is little significance to the test sensitivity of 0.92 versus a criterion of 0.95'' (Ex. OSHA-2007-0006-0021.1). On page 108 of the article describing the ABQLFT study (see Ex. OSHA-2007-0006-0003), the authors observe that ``[t]he difference between a sensitivity of 0.92 and a value greater than 0.95 in this comparison is one fit test where a person with a generated fit factor less than 100 passed the bitter aerosol fit test.'' Based on Table 1 in this article, the 0.95 criterion would permit three false-positive test subjects out of 167 subjects tested (i.e., 0.018% of the total subjects tested), while the obtained value of 0.92 resulted in four false-positive test subjects
(i.e., 0.024% of the subjects tested).
In the NIOSH-Bureau of Labor Statistics survey of respirator use cited in the proposal (NIOSH-BLS survey; Ex. 6-3, Docket H-049C), 282,000 establishments in the United States required respirator use, and these establishments fit tested about 3.3 million employees each year. According to the NIOSH-BLS survey, 18,938 (0.067%) of these establishments used the existing Bitrex[supreg] qualitative fit-testing protocol.\2\ Assuming that these establishments would substitute the proposed ABQLFT protocol for the existing Bitrex[supreg] qualitative fit-testing protocol, and that the distribution of employees across size classes for these establishments is representative of the establishments as a whole,\3\ then 221,100 employees would receive the proposed ABQLFT protocol annually (i.e., 0.067% x 3.3 million employees).
\2\ The proposal cited a figure of ``approximately 25,000 establishments,'' but this figure is for the original Controlled
Negative Pressure quantitative fit-testing protocol specified by
OSHA when it first published the Respiratory Protection Standard in 1998, not for the existing Bitrex[supreg] qualitative fit-testing protocol.
\3\ The term ``size classes'' refers to the number of employees in the establishments; the NIOSH-BLS survey designates these classes as follows: 1-10 employees; 11-19 employees; 11-49 employees; 50-249 employees; 250-999 employees; and 1,000 and more employees. A cursory review of the size-class distribution in the NIOSH-BLS survey shows that 0.088% of the total number of establishments have 1,000 or more employees, while 0.094% of establishments administering the existing Bitrex[supreg] qualitative fit-testing protocol have 1,000 or more employees; this comparison indicates that the distribution of size classes for the latter establishments is similar to the distribution of size classes for the establishments as a whole.
Under the 0.95 sensitivity-test criterion value for the ANSI
Z88.10-2001 standard, about 3,980 employees with improperly fitting respirators would pass the proposed ABQLFT protocol each year (i.e., a 0.018% false-positive rate x 221,100 total employees tested), while the 0.92 sensitivity-test value obtained for the proposed protocol would result in about 5,306 employees passing the test with improperly fitting respirators (i.e., a 0.024% false-positive rate x 221,100 total employees tested). OSHA believes that the 3,980 employees with false- positive values that would result from using the sensitivity-test criterion from the ANSI standard are too high; therefore, adding 1,326 employees each year to this already excessive figure is unacceptable.
Contrary to the previously cited statement made by Mr. Weber from 3M,
OSHA finds that the significance between test sensitivity values of 0.92 and 0.95, when viewed in practical terms, is highly significant because an additional 1,326 employees would not have adequate respiratory protection in the workplace. OSHA believes that the contribution of ANSI Z88.10-2001 to the process of evaluating proposed respirator fit-testing protocols is to provide procedures that OSHA can use in determining the practical effects of errors that result from the administration of these proposed protocols. Therefore, based on this analysis involving the sensitivity-test criterion from the ANSI standard, OSHA concludes that it cannot include the proposed ABQLFT protocol among the qualitative fit tests currently listed in Part I.B of Appendix A of its Respiratory Protection Standard.
Regarding the second part of this issue (i.e., that the sensitivity calculations may not be the best indicator of fit-test performance), the authors of the study recommended using binary logistic-regression analysis to determine sensitivity of the proposed protocol instead of the test-sensitivity criterion specified by ANSI Z88.10-2001. Every comment submitted to the record opposed this recommendation. For example, NIOSH stated:
A second cited journal article [Nelson and Mullins 2004] examined the treatment of data from previously reported studies, including the 2003 Nelson study, by use of a new method of data analysis. A more thorough evaluation of the method of data analysis should be undertaken to ensure the studies used to validate the new method include an appropriate range of fit factors and respirator designs.
* * * * *
The argument by the study authors that ``the method used to determine acceptability should be independent of specific data collected'' is not convincing. A sufficient number of subjects met fit testing requirements using the ABQLFT protocol
Page 30255
and failed using the GAQNFT protocol. These results were determined to be below the ANSI Z88.10-2001 recommended criteria of 0.95 for the test-sensitivity value. Recalculating test sensitivity
(proportion of failed reference method fit tests that also failed the new fit-test method) via alternative statistical techniques, or questioning the validity of the sensitivity calculation as an appropriate indicator of fit-test method performance to rationalize a positive conclusion, is a questionable response to the study outcome. The sensitivity test is a critical criterion to ensure the rejection of inadequately fitting respirators. Because the observed value of 0.92 is below the ANSI criterion of 0.95, NIOSH considers the value unacceptable. If the method of data analysis is changed, the new method needs to be thoroughly evaluated before challenging the standard criterion. (See Ex. OSHA-2007-0006-0026.)
Bill Kajola of the AFL-CIO recommended that OSHA not sanction the binary logistic-regression analysis as an alternate method for analyzing the study results, stating, ``There is no data or confirmation to suggest that a `binary logistic regression analysis' is an appropriate and adequate means to evaluate a new fit test method''
(OSHA-2007-0006-0019.1). James S. Johnson believed it was premature to use binary logistic-regression analysis to analyze the study data, asserting that ``[t]he proposed change is too significant to be based on one study that has to have additional mathematical analysis and assumptions proposed to pass the ANSI Z88.10 requirements'' (Ex. OSHA- 2007-0006-0028). Daniel K. Shipp of the International Safety Equipment
Association commented that binary logistic-regression analysis ``be validated by an additional source'' (Ex. OSHA-2007-0006-0027).
As noted earlier, none of the comments submitted to the record supported using binary logistic-regression analysis to interpret the study results. These comments clearly indicate that this analytic technique is currently inappropriate for use in determining the sensitivity of fit-testing protocols. OSHA agrees with these comments, and believes that the technique requires additional validation before it will be acceptable for this purpose. 6. What is the significance of limiting the ABQLT to respirator users who demonstrate Level 1 sensitivity to Bitrex[supreg]? Few commenters responded to this issue. NIOSH observed that information about ``the number or percentage of subjects in [the] study who did not meet Level 1 sensitivity to Bitrex[supreg]'' was not available in the article describing the study (see Ex. OSHA-2007-0006-0003), and, therefore, ``NIOSH is unable to estimate the proportion of workers in the population who demonstrate Level 1 sensitivity to Bitrex[supreg]''
(Ex. OSHA-2007-0006-0026). As a result, NIOSH found that ``the utility of the proposed ABQLFT protocol can not be determined at this time.''
James S. Johnson commented that determining Level 1 sensitivity is a restriction that ``adds another level of complexity to the test protocol'' (Ex. OSHA-2007-0006-0028). Ching-tsen Bien believed that using Level 1 sensitivity for screening purposes ``does not prevent some test conductors who ignore this limitation and use the ABQLFT method to fit test any worker, and it may result in the selection of
the
wrong respirator for workers with Levels 2 or 3 sensitivity * *
*'' (Ex. OSHA-2007-0006-0017.1). None of these comments challenged the validity or accuracy of the Level 1 sensitivity procedure; accordingly,
OSHA concludes that the ABQLFT study used the procedure appropriately, and that it accurately screened the test subjects for sensitivity to
Bitrex[supreg]. 7. Miscellaneous issues addressed by the comments. Several commenters objected that the test subjects in the ABQLFT study did not perform seal checks while using the respirators. For example, James S.
Johnson stated that ``[t]he exclusion of the users seal check may bias the data and this isn't representative of how this procedure is normally done'' (Ex. OSHA-2007-0006-0028). In response to the commenters, OSHA notes that the test subjects in the study used respirators that were one or two sizes too small or too large to ensure that a number of poor respirator fits occurred. This procedure induced poor facepiece-to-face seals, which caused the respirators to leak.
These leaks, in turn, provided data for use in determining how effectively the proposed ABQLFT protocol detected such leaks. The authors of the ABQLFT study explained the absence of seal checks as follows: ``Experience in this laboratory has shown that people who participate in fit tests on a frequent basis and who are allowed to perform user seal checks can adjust most respirators to fit well enough to pass a fit test (Janssen, 2002). For this reason, the subjects were instructed to adjust the facepiece until comfortable but were not permitted to perform a user seal check'' (Ex. OSHA-2007-0006-0003).
Therefore, OSHA concludes that removing seal checks from the study was necessary to obtain leakage data for use in determining the effectiveness of the proposed ABQLFT protocol.
D. Conclusions
Based on a complete and thorough review of the rulemaking record,
OSHA concludes that: 1. The study was well controlled, and conducted according to accepted experimental design practices and principles. 2. The authors of the studies described in the submitted articles presented the the results properly, fully, and fairly in the context of the ANSI Z88.10-2001 consensus standard. 3. The results generated by the proposed protocol provided reproducible fit-testing results, and the experiments and analyses were adequate for this purpose. 4. The results for the proposed protocol were reliable, but OSHA can reach no conclusion regarding how the proposed protocol compares to other qualitative fit-testing protocols because the study did not make these comparisons. Additionally, the study did not demonstrate that the proposed protocol accurately determined fit for N95 filtering-facepiece respirators; therefore, OSHA could not approve the proposed protocol for fit testing this class of respirators. 5. The test-sensitivity value of 0.92 would increase substantially the number of employees who would pass the proposed protocol with improperly fitting respirators, thereby making the proposed protocol unacceptable for listing in Part I.B. of Appendix A of OSHA's
Respiratory Protection Standard. In addition, using binary logistic- regression analysis as a substitute for the sensitivity-test criterion in ANSI Z88.10-2001 is premature because the analysis requires additional validation. 6. The results indicate that limiting the proposed protocol to test subjects who demonstrated Level 1 sensitivity to Bitrex[supreg] was appropriate. 7. To ensure adequate respirator leakage, the study justifiably omitted seal checks from the experimental procedures.
Additional validation testing of, or revisions to, the proposed
ABQLFT protocol may provide new results for the protocol that meet or exceed the sensitivity-test criterion established by the ANSI Z88.10 consensus standard. After submitting these new results and supporting documentation to OSHA, OSHA would evaluate this information and, if appropriate, would submit it to the public for notice and comment. If the revised protocol is to apply to filtering-facepiece respirators, then the resubmission should include testing on these respirators demonstrating that the
Page 30256
revised protocol accurately identifies poor fit among test subjects who use them.
List of Subjects in 29 CFR Part 1910
Hazardous substances, Health, Occupational safety and health, Toxic substances.
Authority and Signature
Jordan Barab, Acting Assistant Secretary of Labor for Occupational
Safety and Health, U.S. Department of Labor, 200 Constitution Avenue,
NW., Washington, DC 20210, directed the preparation of this notice.
Accordingly, the Agency issues this notice under the following authorities: Sections 4, 6(b), 8(c), and 8(g) of the Occupational
Safety and Health Act of 1970 (29 U.S.C. 653, 655, 657); Section 3704 of the Contract Work Hours and Safety Standards Act (40 U.S.C. 3701 et seq.); Section 41 of the Longshore and Harbor Worker's Compensation Act
(33 U.S.C. 941); Secretary of Labor's Order No. 5-2007 (72 FR 31160); and 29 CFR part 1911.
Signed at Washington, DC, on June 22, 2009.
Jordan Barab,
Acting Assistant Secretary of Labor for Occupational Safety and Health.
FR Doc. E9-14979 Filed 6-24-09; 8:45 am
BILLING CODE 4510-26-P
