Adoption of Recommendations

As Enacted ( Federal Register)

Cited authorities 30

Cited in

Published date	22 January 2021
Citation	86 FR 6612
Record Number	2021-01273
Section	Notices
Court	Administrative Conference Of The United States

This section of the FEDERAL REGISTER

contains documents other than rules or

proposed rules that are applicable to the

public. Notices of hearings and investigations,

committee meetings, agency decisions and

rulings, delegations of authority, filing of

petitions and applications and agency

statements of organization and functions are

examples of documents appearing in this

section.

Notices Federal Register

6612

Vol. 86, No. 13

Friday, January 22, 2021

ADMINISTRATIVE CONFERENCE OF

THE UNITED STATES

Adoption of Recommendations

AGENCY

: Administrative Conference of

the United States.

ACTION

: Notice.

SUMMARY

: The Administrative

Conference of the United States adopted

six recommendations and one official

statement at its virtual Seventy-third

Plenary Session. The appended

recommendations address: (a) Rules on

Rulemakings; (b) Protected Materials in

Public Rulemaking Dockets; (c) Agency

Appellate Systems; (d) Government

Contract Bid Protests Before Agencies;

(e) Publication of Policies Governing

Agency Adjudicators; and (f) Agency

Litigation Webpages. The official

statement addresses Agency use of

Artificial Intelligence.

FOR FURTHER INFORMATION CONTACT

: For

Recommendations 2020–1 and 2020–2,

Todd Rubin; for Recommendation

2020–3, Gavin Young; for

Recommendations 2020–4 and 2020–6,

and Statement #20, Mark Thomson; and

for Recommendation 2020–5, Leigh

Anne Schriever. For each of these

actions the address and telephone

number are: Administrative Conference

of the United States, Suite 706 South,

1120 20th Street NW, Washington, DC

20036; Telephone 202–480–2080.

SUPPLEMENTARY INFORMATION

: The

Administrative Conference Act, 5 U.S.C.

591–596, established the Administrative

Conference of the United States. The

Conference studies the efficiency,

adequacy, and fairness of the

administrative procedures used by

Federal agencies and makes

recommendations to agencies, the

President, Congress, and the Judicial

Conference of the United States for

procedural improvements (5 U.S.C.

594(1)). For further information about

the Conference and its activities, see

www.acus.gov. At its virtual Seventy-

third Plenary Session on December 16–

17, 2020, the Assembly of the

Conference adopted six

recommendations and one official

statement.

Recommendation 2020–1, Rules on

Rulemakings. This recommendation

encourages agencies to consider issuing

rules governing their rulemaking

procedures. It identifies subjects that

agencies should consider addressing in

their rules on rulemakings—without

prescribing any particular procedures—

and it urges agencies to solicit public

input on these rules and make them

publicly available.

Recommendation 2020–2, Protected

Materials in Public Rulemaking Dockets.

This recommendation offers agencies

best practices for protecting sensitive

personal and confidential commercial

information in public rulemaking

dockets. It identifies, in particular, best

practices for agencies to use when

redacting, summarizing, and aggregating

comments that contain such

information. It also encourages agencies

to provide public notices that

discourage commenters from submitting

such information in the first place.

Recommendation 2020–3, Agency

Appellate Systems. This

recommendation offers agencies best

practices to improve administrative

review of hearing-level adjudicative

decisions with respect to case selection,

decision-making process and

procedures, management oversight, and

public disclosure and transparency. In

doing so, it encourages agencies to

identify the objectives of such review

and structure their appellate systems to

serve those objectives.

Recommendation 2020–4,

Government Contract Bid Protests

Before Agencies. This recommendation

suggests improvements to the

procedures governing agency-level

procurement contract disputes—

commonly called bid protests—under

the Federal Acquisition Regulation and

agency-specific regulations to make

those procedures more simple,

transparent, and predictable. It urges

agencies to clarify what types of

decisions can be the subjects of agency-

level bid protests, what processes and

deadlines will govern such protests, and

who in the agency will decide such

protests; make it easier for protesters to

get information about the decisions they

protest; and publish more data on

agency-level protests.

Recommendation 2020–5, Publication

of Policies Governing Agency

Adjudicators. This recommendation

encourages agencies to disclose policies

governing the appointment and

oversight of adjudicators that bear on

their impartiality and constitutional

status. It offers best practices on how to

provide descriptions of, and access to,

such policies on agency websites.

Recommendation 2020–6, Agency

Litigation Webpages. This

recommendation offers agencies best

practices for making their federal court

filings and relevant court opinions

available to the public on their websites,

with particular emphasis on materials

from litigation dealing with agency

regulatory programs. It provides

guidance on the types of litigation

materials that will be of greatest interest

to the public and on how agencies can

disseminate the materials in a way that

makes them easy to find.

Statement #20, Agency Use of

Artificial Intelligence. This statement

identifies issues agencies should

consider when adopting, revamping,

establishing policies and practices

governing, and regularly monitoring

artificial intelligence systems. Among

the topics it addresses are transparency,

harmful biases, technical capacity,

procurement, privacy, security,

decisional authority, and oversight.

The Appendix below sets forth the

full texts of these six recommendations

and the official statement. The

Conference will transmit the

recommendations and statement to

affected agencies, Congress, and the

Judicial Conference of the United States,

as appropriate. The recommendations

and statement are not binding, so the

entities to which they are addressed will

make decisions on their

implementation.

The Conference based these

recommendations and the statement on

research reports that are posted at:

https://www.acus.gov/meetings-and-

events/plenary-meeting/73rd-plenary-

session. Committee-proposed drafts of

the recommendations and statement,

and public comments received in

advance of the plenary session, are also

available using the same link.

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00001 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6613

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

This Recommendation does not address

rulemakings subject to the formal hearing

requirements of the Administrative Procedure Act.

See 5 U.S.C. 556–57.

Cf. Admin. Conf. of the U.S., Recommendation

2019–1, Agency Guidance Through Interpretive

Rules, 84 FR 38927 (Aug. 8, 2019); Admin. Conf.

of the U.S., Recommendation 2017–5, Agency

Guidance Through Policy Statements, 82 FR 61734

(Dec. 29, 2017).

See, e.g., 2 U.S.C. 1534 (Unfunded Mandates

Reform Act); 5 U.S.C. 609 (Regulatory Flexibility

Act); Exec. Order No. 13,175, Consultation and

Coordination with Indian Tribal Governments, 65

FR 67249 (Nov. 11, 2000).

See Admin. Conf. of the U.S., Recommendation

92–1, The Procedural and Practice Rule Exemption

from the APA Notice-and-Comment Rulemaking

Requirements, 57 FR 30102 (July 8, 1992); see also

Recommendation 2019–1, supra note 2;

Recommendation 2017–5, supra note 2.

See, e.g., Health Ins. Ass’n of Am. v. Shalala, 23

F.3d 412, 423 (D.C. Cir. 1994) (stating that

‘‘publication in the Code of Federal Regulations, or

its absence’’ is only ‘‘a snippet of evidence of

agency intent’’ that the published pronouncement

be given binding effect).

Some rules on rulemakings include a statement

that they do not create any substantive or

procedural rights or benefits. This Recommendation

does not address whether such disclaimers should

be included or what legal effect they may have on

judicial review. These questions cannot be

answered in isolation from the broader question of

when a rule on rulemakings is judicially

enforceable.

See Admin. Conf. of the U.S., Recommendation

2015–1, Promoting Accuracy and Transparency in

the Unified Agenda, 80 FR 36757 (June 26, 2015).

Dated: January 14, 2021.

Shawne C. McGibbon,

General Counsel.

Appendix—Recommendations and

Statement of the Administrative

Conference of the United States

Administrative Conference Recommendation

2020–1

Rules on Rulemakings

Adopted December 16, 2020

Numerous agencies have promulgated

rules setting forth the policies and

procedures they will follow when conducting

informal rulemakings under 5 U.S.C. 553.

The rules can cover a variety of practices,

including processes for initiating and seeking

public input on new rules, coordinating with

the Office of Management and Budget and

other agencies as a rule is being formulated,

and obtaining approval from agency

leadership before a proposed rule is issued or

finalized. Agencies refer to these rules by

different names. This Recommendation calls

them ‘‘rules on rulemakings.’’

Rules on rulemakings vary—in terms of the

particular matters they address, their scope

and comprehensiveness, and other

characteristics—but they share several

common features. First, they authoritatively

reflect the agency’s position as to what

procedures it will observe when adopting

new rules. By ‘‘authoritative,’’ this

Recommendation means that a rule on

rulemakings sets forth the procedures that

agency officials responsible for drafting and

finalizing new rules will follow in at least

most cases within the rule on rulemakings’

scope, though it may contemplate the

possibility that agency leadership could

authorize an alternative set of procedures.

Second, rules on rulemakings do more than

simply summarize or explain rulemaking

requirements of the Administrative

Procedure Act and other statutes, although

they often serve an explanatory function at

the same time that they set forth the

procedures the agencies will follow in

conducting rulemakings. Rules on

rulemakings set forth additional

commitments by an agency concerning how

it will conduct rulemakings. And third,

agencies disseminate rules on rulemakings

publicly rather than only internally. They

appear on agency websites and are often

published not only in the Federal Register

but also in the Code of Federal Regulations

(CFR).

Rules on rulemakings can serve at least

four important objectives. First, they promote

efficiency by ensuring that both agency

officials and those outside the agency know

where to go to find the agency’s rulemaking

policies. Second, they promote predictability

by informing the public that the agency will

follow particular procedures, thereby

allowing the public to plan their

participation in the rulemaking process

accordingly. Third, they promote

accountability by ensuring that agency

leadership has approved the policies and

procedures the agency will follow. And they

can also provide accountability in connection

with individual rulemakings by creating an

internal approval process by which agency

leadership reviews proposed and final rules.

Finally, they promote transparency by

affording the public access to the agency’s

internal procedures pertaining to its

rulemaking process.

In promulgating a rule on rulemakings, an

agency may wish to solicit public input to

inform the rule’s development, even if such

a rule is subject to 5 U.S.C. 553’s exemption

from notice-and-comment procedures as a

rule of procedure, general statement of

policy, or otherwise. In soliciting public

input, agencies may wish to use mechanisms

that facilitate more robust participation,

including by underrepresented

communities.

As the Administrative

Conference has acknowledged in past

recommendations, public comment can both

provide valuable input from the public and

enhance public acceptance of an agency’s

rules.

An agency may also wish to publish its

rule on rulemakings in the CFR. Doing so can

enhance transparency and facilitate

accountability. Importantly, publishing a rule

on rulemakings in the CFR does not, by itself,

make the rule on rulemakings judicially

enforceable.

This Recommendation does not seek to

resolve whether, when, or on what legal

bases a court might enforce a rule on

rulemakings against an agency.

Recommendation

1. Agencies should consider promulgating

rules on rulemakings setting forth the

policies and procedures they will follow in

informal rulemaking under 5 U.S.C. 553.

2. In issuing rules on rulemakings, agencies

should consider including provisions

addressing the following topics (which

reflect topics frequently covered in existing

rules on rulemakings):

a. Procedures prior to the issuance of a

notice of proposed rulemaking;

b. Procedures connected with the notice-

and-comment process;

c. Procedures connected with the

presidential review process, if applicable;

d. Procedures for handling post-comment

period communications;

e. Internal approval procedures for issuing

and finalizing rules; and

f. Procedures for reassessing existing rules.

The appendix gives examples of particular

subtopics agencies may wish to consider

under each of these topics.

3. Agencies should make rules on

rulemakings available in a prominent, easy-

to-find place on the portion of their websites

dealing with rulemaking matters.

Additionally, agencies should consider

publishing them in the Federal Register and

the Code of Federal Regulations. When

posting rules on rulemakings on their

websites, agencies should use techniques like

linked tabs, pull-down menus, indexing,

tagging, and sorting tables to ensure that

relevant documents are easily findable.

Agencies should also design their search

engines to allow people to easily identify

relevant documents.

4. In addition to issuing rules on

rulemakings, agencies should consider

explaining in accessible language how the

rulemaking process works in order to educate

the public. Such explanations might be

integrated within a rule on rulemakings or

might be contained in separate explanatory

documents (e.g., documents identifying

frequently asked questions). When providing

such explanations, an agency should, to the

extent practicable, distinguish between

procedures it intends to follow and material

provided purely by way of background.

5. Agencies should consider a broad range

of means of seeking public input on rules on

rulemakings, even if the Administrative

Procedure Act does not require it.

6. Agencies should consider the extent to

which procedures required by a rule on

rulemakings should be made internally

waivable and, if so, by whom. For example,

they might consider drafting a rule on

rulemakings in a way that allows high-level

agency officials to permit other officials to

use alternative procedures.

Appendix

Non-Exhaustive List of Topics for Agencies

To Consider Including Within Their Rules

on Rulemakings

(a) Procedures Prior to the Issuance of a

Notice of Proposed Rulemaking

Subtopic Examples:

(1) Regulatory planning;

(2) Issuing advance notices of proposed

rulemaking and obtaining feedback from

members of the public using means other

than the notice-and-comment process, such

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00002 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6614

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

See Admin. Conf. of the U.S., Recommendation

2018–7, Public Engagement in Rulemaking, 84 FR

2146 (Feb. 6, 2019).

See Admin. Conf. of the U.S., Recommendation

2014–6, Petitions for Rulemaking, 79 FR 75117

(Dec. 17, 2014).

See Admin. Conf. of the U.S., Recommendation

2012–1, Regulatory Analysis Requirements, 77 FR

47801 (Aug. 10, 2012).

See Admin. Conf. of the U.S., Recommendation

2017–3, Plain Language in Regulatory Drafting, 82

FR 61728 (Dec. 29, 2017).

See Admin. Conf. of the U.S., Recommendation

2018–2, Severability in Agency Rulemaking, 83 FR

30685 (June 29, 2018).

See Admin. Conf. of the U.S., Recommendation

2017–2, Negotiated Rulemaking and Other Options

for Public Engagement, 82 FR 31040 (July 5, 2017).

See Admin. Conf. of the U.S., Recommendation

2018–6, Improving Access to Regulations.gov’s

Rulemaking Dockets, 84 FR 2143 (Feb. 6, 2019).

See Admin. Conf. of the U.S., Recommendation

2011–2, Rulemaking Comments, 76 FR 48791 (Aug.

9, 2011).

See Admin. Conf. of the U.S., Recommendation

2014–4, ‘‘Ex Parte’’ Communications in Informal

Rulemaking, 79 FR 35993 (June 25, 2014).

See Admin. Conf. of the U.S., Recommendation

2011–5, Incorporation by Reference, 77 FR 2257

(Jan. 17, 2012).

See Admin. Conf. of the U.S., Recommendation

2013–5, Social Media in Rulemaking, 78 FR 76269

(Dec. 17, 2013).

See Recommendation 2018–7, supra note 8.

See Admin. Conf. of the U.S., Recommendation

2020–2, Protected Materials in Public Rulemaking

Dockets, 86 FR (approved Dec. 16, 2020); Admin.

Conf. of the U.S., Recommendation 2011–1, Legal

Considerations in e-Rulemaking, 76 FR 48789 (Aug.

9, 2011).

See Recommendation 92–1, supra note 4.

See Admin. Conf. of the U.S., Recommendation

95–4, Procedures for Noncontroversial and

Expedited Rulemakings, 60 FR 43108 (Aug. 18,

1995).

See Admin. Conf. of the U.S., Recommendation

2011–6, International Regulatory Cooperation, 77

FR 2259 (Jan. 17, 2012).

See Recommendation 2011–2, supra note 15.

See id.

See Admin. Conf. of the U.S., Recommendation

2019–5, Agency Economists, 84 FR 71349 (Dec. 27,

2019).

See Admin. Conf. of the U.S., Recommendation

2017–7, Regulatory Waivers and Exemptions, 82 FR

61742 (Dec. 29, 2017).

See Admin. Conf. of the U.S., Recommendation

2014–5, Retrospective Review of Agency Rules, 79

FR 75114 (Dec. 17, 2014).

See Admin. Conf. of the U.S., Recommendation

2013–4, Administrative Record in Informal

Rulemaking, 78 FR 41358 (July 10, 2013).

See Admin. Conf. of the U.S., Recommendation

2013–6, Remand Without Vacatur, 78 FR 76272

(Dec. 17, 2013).

The public rulemaking docket is distinguished

from ‘‘the administrative record for judicial

review,’’ which is intended to provide courts with

a record for evaluating challenges to the rule, and

the ‘‘rulemaking record,’’ which means all

comments and materials submitted to agencies

during comment periods and any other materials

agencies considered during the course of the

rulemaking. See Admin. Conf. of the U.S.,

Recommendation 2013–4, The Administrative

Record in Informal Rulemaking, 78 FR 41358 (July

10, 2013).

Recommendation 2011–1, Legal Considerations

in e-Rulemaking, advises agencies to allow

submitters to flag confidential information,

including trade secrets, and advises agencies to

devise procedures for reviewing and handling such

information. Admin. Conf. of the U.S.,

Recommendation 2011–1, Legal Considerations in

e-Rulemaking, ¶1, 76 FR 48789, 48790 (Aug. 9,

2011). Recommendation 2013–4, supra note 1, ¶11,

advises agencies to develop guidance on managing

and segregating protected information, such as

confidential commercial information and sensitive

personal information, while disclosing non-

protected materials; see also Admin. Conf. of the

U.S., Recommendation 89–7, Federal Regulation of

Biotechnology, 54 FR 53494 (Dec. 29, 1988); Admin.

Conf. of the U.S., Recommendation 82–1,

Exemption (b)(4) of the Freedom of Information Act,

47 FR 30702 (July 15, 1982); Admin. Conf. of the

U.S., Recommendation 80–6, Intragovernmental

Communications in Informal Rulemaking

Proceedings, 45 FR 86408 (Dec. 31, 1980).

as requests for information and focus

groups;

(3) Accepting, reviewing, and responding

to petitions for rulemaking;

(4) Considering options besides

rulemaking;

(5) Performing ex ante regulatory analyses

(e.g., benefit-cost analysis and regulatory

flexibility analysis);

(6) Using plain language in regulatory

drafting;

(7) Preparing for potential judicial review

of rulemakings, including deciding whether

to make any of the provisions of a rule

severable;

(8) Conducting negotiated rulemaking;

and

(9) Establishing an effective date for rules.

(b) Procedures Connected With the Notice-

and-Comment Process

Subtopic Examples:

(1) Materials to be published on

Regulations.gov with the notice;

(2) Minimum comment periods to be

allowed;

(3) Policies on ex parte contacts;

(4) Handling external merits

communications not filed as comments;

(5) Incorporating standards by reference;

(6) Using social media to engage the public

in rulemaking;

(7) Obtaining feedback from American

Indian tribes, other historically

underrepresented or under-resourced groups,

and state and local governments;

(8) Posting, analyzing, and responding to

public comments, including comments that

may contain confidential commercial

information, protected personal information,

or other kinds of sensitive submissions;

(9) Waiving or invoking of Administrative

Procedure Act exemptions to notice and

comment;

and

(10) Using interim final rules or direct final

rules.

Presidential Review Process, if Applicable

Subtopic Examples:

(1) Interacting with the Office of

Information and Regulatory Affairs, the

Office of the Federal Register, the Regulatory

Information Service Center, the Small

Business Administration’s Office of

Advocacy, and other offices with

government-wide rulemaking

responsibilities;

(2) Participating in the interagency review

process; and

(3) Procedures related to international

regulatory cooperation.

(d) Procedures for Handling Post-Comment

Period Communications

Subtopic Examples:

(1) Provisions pertaining to reply

comments

and

(2) Handling late-filed comments.

(e) Internal Approval Procedures for Issuing

and Finalizing Rules

Subtopic Examples:

(1) Procedures for submitting rules to

offices with legal, economic, and other

responsibilities within the agency for

review

and

(2) Procedures for submitting rules to the

relevant agency official for final approval.

(f) Procedures for Reassessing Existing Rules

Subtopic Examples:

(1) Issuing regulatory waivers and

exemptions;

(2) Engaging in retrospective review of

rules;

(3) Maintaining and preserving rulemaking

records, including transparency of such

records and the handling of confidential

commercial information, protected personal

information, or other kinds of sensitive

information contained therein;

and

(4) Handling rules that have been vacated

or remanded without vacatur.

Administrative Conference Recommendation

2020–2

Protected Materials in Public Rulemaking

Dockets

Adopted December 16, 2020

As part of the rulemaking process, agencies

create public rulemaking dockets, which

consist of all rulemaking materials agencies

have: (1) Proactively published online or (2)

made available for public inspection in a

reading room. Public rulemaking dockets

include materials agencies generate

themselves and comments agencies receive

from the public. Their purpose is to provide

the public with the information that

informed agencies’ rulemakings.

The Administrative Conference has issued

several recommendations to help agencies

balance the competing considerations of

transparency and confidentiality in managing

their public rulemaking dockets.

This

project builds on these recommendations.

The scope of the Recommendation is

limited to personal information and

confidential commercial information that

agencies have decided to withhold from their

public rulemaking dockets, which this

Recommendation calls ‘‘protected material.’’

The Recommendation specifies how agencies

should consider handling protected material.

For purposes of this Recommendation,

personal information is information about an

individual including his or her education,

financial transactions, medical history,

criminal or employment history, or similarly

sensitive information, and that contains his

or her name, or the identifying number,

symbol, or other identifying particular

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00003 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6615

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

See Privacy Act of 1974 §3, 5 U.S.C. 552a(a)(4).

See Food Mktg. Inst. v. Argus Leader Media, 139

S. Ct. 2356, 2363 (2019); see also Exec. Order No.

12,600, Predisclosure Notification Procedures for

Confidential Commercial Information, 52 FR 23781

(June 23, 1987).

See Christopher Yoo, Protected Materials in

Public Rulemaking Dockets 24 (Nov. 24, 2020)

(report to the Admin. Conf. of the U.S.), https://

www.acus.gov/report/final-report-protected-

materials-public-rulemaking-dockets.

5 U.S.C. 553(c).

Portland Cement Ass’n v. Ruckelshaus, 486 F.2d

375, 393 (D.C. Cir. 1973). In addition to these public

transparency requirements, there are a number of

federal record-retention requirements of which

agencies should be aware. See, e.g., 44 U.S.C. 3301.

5 U.S.C. 552a(b).

18 U.S.C. 1905.

5 U.S.C. 552a(b)(2).

See CNA Fin. Corp. v. Donovan, 830 F.2d 1132,

1137–43 (D.C. Cir. 1987).

See Food Mktg. Inst., 139 S. Ct. at 2361.

Permitting the submission of anonymous and

pseudonymous comments is one way that some

agencies attempt to reduce the privacy risks

commenters face when submitting protected

material. Issues regarding the submission of

anonymous and pseudonymous comments are

being considered in an ongoing project of the

Administrative Conference titled Mass, Computer-

Generated, and Fraudulent Comments and are

beyond the scope of this Recommendation.

assigned to the individual.

Confidential

commercial information is commercial

information that is customarily kept private,

or at least closely held, by the person or

business providing it.

Other types of

information, such as national security

information and copyrighted materials, are

beyond the Recommendation’s scope. The

Recommendation is also limited to

addressing procedures for protecting

materials that agencies decide warrant

protection. It is not intended to define the

universe of protected materials. In particular,

the Recommendation does not address any

issue that may arise if agencies choose to rely

on protected material in explaining their

rulemakings, whether in notices of proposed

rulemaking, regulatory impact analyses, or

otherwise.

Agencies accept public comments for their

public rulemaking dockets primarily through

Regulations.gov, their own websites, and

email. Regulations.gov and many agency

websites that accept comments expressly

notify the public that agencies may publish

the information submitted in public

comments.

When people submit comments

to agencies, however, agencies typically do

not immediately publish the comments.

Instead, agencies generally take time to

screen comments before publishing them.

Most agencies perform at least some kind of

screening during this period.

For all agencies, whether to withhold or

disclose protected material is governed by

various laws: Some mandate disclosure,

some mandate withholding, and some leave

agencies with substantial discretion in

deciding whether to disclose. Although a full

description of those laws is beyond the scope

of this Recommendation, a brief overview of

at least some of this body of law helps to

identify the issues agencies face.

The Administrative Procedure Act requires

agencies to ‘‘give interested persons an

opportunity to participate in rulemaking

through submission of written data, views, or

arguments.’’

The United States Court of

Appeals for the D.C. Circuit has interpreted

this provision to ordinarily require that

agencies make publicly available the critical

information—including studies, data, and

methodologies—underlying proposed rules.

The Privacy Act and the Trade Secrets Act

place limits on the disclosure norm

discussed above. Generally, the Privacy Act

prevents agencies from disclosing any

information about a person, such as medical

records, educational background, and

employment history, contained in agencies’

systems of records, without that person’s

written consent.

The Trade Secrets Act

generally prevents agencies from disclosing

trade secrets and other kinds of confidential

commercial information, such as corporate

losses and profits.

Both the Privacy Act and the Trade Secrets

Act have exceptions. For the Privacy Act, the

main exception relevant to this

Recommendation is for information required

to be released under the Freedom of

Information Act (FOIA).

The Trade Secrets

Act only has one exception, which covers

any materials authorized to be disclosed by

statute (including FOIA) or regulation.

Whether a particular piece of personal or

confidential commercial information meets

one of these exceptions often involves a

complex determination that depends upon

the exact type of information at issue and its

contemplated use, and agencies must

determine the applicability of the exceptions

on a case-by-case basis. For example,

whether FOIA authorizes disclosure of

confidential commercial information may

turn in part on whether agencies in receipt

of the information assured submitters that the

information would be withheld from the

public.

If agencies offer assurances that

they will not disclose confidential

commercial information, agencies and

submitters may rely on those assurances as

a defense against compelled disclosure under

FOIA. In many cases, agencies assure

companies that they will not disclose such

information in order to encourage companies

to submit it.

Particular cases are governed by specific

requirements of law, not broad categorical

labels. But agencies often consider certain

categories of personal information and

confidential commercial information to be

protected material (e.g., trade secrets, social

security numbers, bank account numbers,

passport numbers, addresses, email

addresses, medical information, and

information concerning a person’s finances).

There are many ways protected material

may arrive at the agency in a rulemaking. A

person might submit his or her own

information, intentionally or unintentionally,

and then ask the agency not to disclose it. A

third party might submit another person’s

information, with or without that person’s

knowledge. A company might submit a

document containing its own confidential

commercial information, intentionally or

unintentionally, with or without the agency’s

prior assurance of protection. Or a company

might submit another company’s or person’s

information. Depending on the information

in question and the manner in which it was

submitted, there may be issues of waiver of

statutory protection. Such questions, like all

questions regarding the substance of the laws

governing protected material, are beyond this

Recommendation’s scope, but they illustrate

the various considerations that agencies and

the public often face in the submission and

handling of such material.

This Recommendation proposes steps

agencies can take to withhold protected

materials from their public rulemaking

dockets while still providing the public with

the information upon which agencies relied

in formulating proposed rules.

Recommendation

Recommendations for All Agencies

1. To reduce the risk that agencies will

inadvertently disclose protected material,

agencies should describe what kinds of

personal and confidential commercial

information qualify as protected material and

should clearly notify the public about their

treatment of protected material. An agency’s

notifications should:

a. Inform members of the public that

comments are generally subject to public

disclosure, except when disclosure is limited

by law;

b. Inform members of the public whether

the agency offers assurances of protection

from disclosure for their confidential

commercial information and, if so, how to

identify such information for the agency;

c. Provide guidance to the public

concerning the submission of protected

material that pertains to third parties,

including instructions that the disclosure of

some protected material may be prohibited

by law;

d. Advise members of the public to review

their comments for the material identified

above in (c) and, if they find such material,

to remove any such material that is not

essential to the comment;

e. Inform members of the public that they

may request, during the period between

when a comment is received and when it is

made public, that protected material they

inadvertently submitted be withheld from the

public rulemaking docket;

f. Inform members of the public that they

may request, after the agency has published

any comment, that protected material

pertaining to themselves or to their

dependents within the comment be removed

from the public rulemaking docket; and

g. Inform members of the public that the

agency reserves the right to redact or

aggregate any part of a comment if the agency

determines that it constitutes protected

material, or may withhold a comment in its

entirety if it determines that redaction or

aggregation would insufficiently prevent the

disclosure of this material.

2. Agencies should include the

notifications described in Paragraph 1, or a

link to those notifications, in at least the

following places:

a. Within the rulemaking documents on

which agencies request comments, such as a

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00004 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6616

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

There is no universally accepted definition of

‘‘artificial intelligence,’’ and the rapid state of

evolution in the field, as well as the proliferation

of use cases, makes coalescing around any such

definition difficult. See, e.g., John S. McCain

National Defense Authorization Act for Fiscal Year

2019, Public Law 115–232, 238(g), 132 Stat. 1636,

1697–98 (2018) (using one definition of AI); Nat’l

Inst. of Standards & Tech., U.S. Leadership in AI:

A Plan for Federal Engagement in Developing

Technical Standards and Related Tools 7–8 (Aug.

9, 2019) (offering a different definition of AI).

Generally speaking, AI systems tend to have

characteristics such as the ability to learn to solve

complex problems, make predictions, or undertake

tasks that heretofore have relied on human decision

making or intervention. There are many illustrative

examples of AI that can help frame the issue for the

purpose of this Statement. They include, but are not

limited to, AI assistants, computer vision systems,

biomedical research, unmanned vehicle systems,

advanced game-playing software, and facial

recognition systems as well as application of AI in

both information technology and operational

technology.

See 5 U.S.C. 591.

David Freeman Engstrom, Daniel E. Ho,

Catherine M. Sharkey, & Mariano-Florentino

Cue

´llar, Government by Algorithm: Artificial

Intelligence in Federal Administrative Agencies

(Feb. 2020) (report to the Admin. Conf. of the U.S.),

https://www.acus.gov/report/government-algorithm-

artificial-intelligence-federal-administrative-

agencies; Cary Coglianese, A Framework for

Governmental Use of Machine Learning (Dec. 8,

2020) (report to the Admin. Conf. of the U.S.),

https://www.acus.gov/report/framework-

governmental-use-machine-learning-final-report.

notice of proposed rulemaking or an advance

notice of proposed rulemaking;

b. On agencies’ own comment submission

forms, if agencies have them;

c. Within any automatic emails that

agencies send acknowledging receipt of a

comment;

d. On any part of agencies’ websites that

describe their rulemaking process or within

any rules on rulemakings they may have, as

described in Recommendation 2020–1, Rules

on Rulemakings; and

e. Within any notices of public meetings

pertaining to a rule.

3. The General Services Administration’s

eRulemaking Program Management Office

should work with agencies that participate in

Regulations.gov to include or refer to the

notifications described in Paragraph 1 within

any automated emails Regulations.gov sends

acknowledging receipt of a comment.

4. If a submitter notifies an agency that the

submitter inadvertently included protected

material in the submitter’s comment, the

agency should act as promptly as possible to

determine whether such material warrants

withholding from the public rulemaking

docket and, if so, withhold it from the public

rulemaking docket, or, if already disclosed,

remove it from the public rulemaking docket.

If an agency determines that such material

does not qualify as protected, it should

promptly notify the submitter of this finding

with a brief statement of reasons.

5. Agencies should allow third parties to

request that protected material pertaining to

themselves or a dependent be removed from

the public rulemaking docket. Agencies

should review such requests and, upon

determining that the material subject to the

request qualifies as protected material,

should remove it from the public rulemaking

docket as promptly as possible. If an agency

determines that the material does not qualify

as protected, it should promptly notify the

requestor of this finding with a brief

statement of reasons.

Recommendations for Agencies That Screen

Comments for Protected Material Before

Publication in the Public Rulemaking Docket

6. Agencies that screen comments for

protected material before publication in the

public rulemaking docket, either as required

by law or as a matter of discretion, should

redact the protected material and publish the

rest of the comment. Redaction should be

thorough enough to prevent the public from

discerning the redacted material, but not so

broad as to prevent the public from viewing

non-protected material.

7. If redaction is not feasible within a

comment, agencies should consider

presenting the data in a summarized form.

8. If redaction is not feasible across

multiple, similar comments, agencies should

consider presenting any related information

in an aggregated form. Agencies should work

with data science experts and others in

relevant disciplines to ensure that

aggregation is thorough enough to prevent

someone from disaggregating the

information.

9. If the approaches identified in

Paragraphs 6–8 would still permit a member

of the public to identify protected material,

agencies should withhold the comment in its

entirety. When doing so, they should

describe the withheld material for the public

in as much detail as possible without

compromising its confidentiality.

10. When deciding whether and how to

redact, aggregate, or withhold protected

material, agencies should explore using

artificial intelligence-based tools to aid in

identifying protected material. Agencies

should consult with private sector experts

and technology-focused agencies, such as the

General Services Administration’s

Technology Transformation Service and the

Office of Management and Budget’s United

States Digital Service, to determine which

tools are most appropriate and how they can

best be deployed given the agencies’

resources.

Recommendations for Agencies That Offer

Assurances of Protection From Disclosure of

Confidential Commercial Information

11. Agencies that offer assurances of

protection from disclosure of confidential

commercial information should decide how

they will offer such assurances. Agencies can

choose to inform submitters, directly upon

submission, that they will withhold

confidential commercial information from

the public rulemaking docket; post a general

notice informing submitters that confidential

commercial information will be withheld

from the public rulemaking docket; or both.

12. Such agencies should adopt policies to

help them identify such information.

Agencies should consider including the

following, either in tandem or as alternatives,

as part of their policies, including within any

rules on rulemakings they may have, as

described in Recommendation 2020–1, Rules

on Rulemakings:

a. Instructing submitters to identify clearly

that the document contains confidential

commercial information;

b. Instructing submitters to flag the

particular text within the document that

constitutes confidential commercial

information; and

c. Instructing submitters to submit both

redacted and unredacted versions of a

comment that contains confidential

commercial information.

Administrative Conference Statement #20

Agency Use of Artificial Intelligence

Adopted December 16, 2020

Artificial intelligence (AI) techniques are

changing how government agencies do their

work.

Advances in AI hold out the promise

of lowering the cost of completing

government tasks and improving the quality,

consistency, and predictability of agencies’

decisions. But agencies’ uses of AI also raise

concerns about the full or partial

displacement of human decision making and

discretion.

Consistent with its statutory mission to

promote efficiency, participation, and

fairness in administrative processes,

the

Administrative Conference offers this

Statement to identify issues agencies should

consider when adopting or modifying AI

systems and developing practices and

procedures for their use and regular

monitoring. The Statement draws on a pair

of reports commissioned by the

Administrative Conference,

as well as the

input of AI experts from government,

academia, and the private sector (some ACUS

members) provided at meetings of the ad hoc

committee of the Administrative Conference

that proposed this Statement.

The issues addressed in this Statement

implicate matters involving law, policy,

finances, human resources, and technology.

To minimize the risk of unforeseen problems

involving an AI system, agencies should,

throughout an AI system’s lifespan, solicit

input about the system from the offices that

oversee these matters. Agencies should also

keep in mind the need for public trust in

their practices and procedures for use and

regular monitoring of AI technologies.

1. Transparency

Agencies’ efforts to ensure transparency in

connection with their AI systems can serve

many valuable goals. When agencies set up

processes to ensure transparency in their AI

systems, they should consider publicly

identifying the processes’ goals and the

rationales behind them. For example,

agencies might prioritize transparency in the

service of legitimizing its AI systems,

facilitating internal or external review of its

AI-based decision making, or coordinating its

AI-based activities. Different AI systems are

likely to satisfy some transparency goals

more than others. When possible, agencies

should use metrics to measure the

performance of their AI-transparency

processes.

In setting transparency goals, agencies

should consider to whom they should be

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00005 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6617

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

While the term bias has a technical, statistical

meaning, the Administrative Conference here uses

the term more generally, to refer to common or

systematic errors in decision making.

5 U.S.C. 3371–76.

Agencies may also obtain AI systems that are

embedded in commercial products. The

considerations applicable to such embedded AI

systems should reflect the fact that agencies may

have less control over their design and

development.

Within the General Services Administration, for

example, the office called 18F routinely partners

with government agencies to help them build and

buy technologies. Similarly, the United States

Digital Service (which is within the Executive

Office of the President) has a staff of technologists

whose job is to help agencies build better

technological tools. While the two entities have

different approaches—18F acts more like an

information intermediary and the Digital Service

serves as an alternative source for information

technology contracts—both could aid agencies with

obtaining, developing, and using different AI

techniques.

See, e.g., Paperwork Reduction Act, 44 U.S.C.

3501–20.

See, e.g., 5 U.S.C. 552a(e), (g), & (p); 44 U.S.C.

3501 note.

transparent. For instance, depending on the

nature of their operations, agencies might

prioritize transparency to the public, courts,

Congress, or their own officials.

The appropriate level or nature of

transparency and interpretability in agencies’

AI systems will also depend on context. In

some contexts, such as adjudication, reason-

giving requirements may call for a higher

degree of transparency and interpretability

from agencies regarding how their AI systems

function. In other contexts, such as

enforcement, agencies’ legitimate interests in

preventing gaming or adversarial learning by

regulated parties could militate against

providing too much information (or specific

types of information) to the public about AI

systems’ processes. In every context, agencies

should consider whether particular laws or

policies governing disclosure of information

apply.

In selecting and using AI techniques,

agencies should be cognizant of the degree to

which a particular AI system can be made

transparent to appropriate people and

entities, including the general public. There

may be tradeoffs between explainability and

accuracy in AI systems, so that transparency

and interpretability might sometimes weigh

in favor of choosing simpler AI models. The

appropriate balance between explainability

and accuracy will depend on the specific

context, including agencies’ circumstances

and priorities.

The proprietary nature of some AI systems

may also affect the extent to which they can

be made transparent. When agencies’ AI

systems rely on proprietary technologies or

algorithms the agencies do not own, the

agencies and the public may have limited

access to the information about the AI

techniques. Agencies should strive to

anticipate such circumstances and address

them appropriately, such as by working with

outside providers to ensure they will be able

to share sufficient information about such a

system. Agencies should not enter into

contracts to use proprietary AI systems

unless they are confident that actors both

internal and external to the agencies will

have adequate access to information about

the systems.

2. Harmful Bias

At their best, AI systems can help agencies

identify and reduce the impact of harmful

biases.

Yet they can also unintentionally

create or exacerbate those biases by encoding

and deploying them at scale. In deciding

whether and how to deploy an AI system,

agencies should carefully evaluate the

harmful biases that might result from the use

of the AI system as well as the biases that

might result from alternative systems (such

as an incumbent system that the AI system

would augment or replace). Because different

types of bias pose different types of harms,

the outcome of the evaluation will depend on

agencies’ unique circumstances and priorities

and the consequences posed by those harms

in those contexts.

AI systems can be biased because of their

reliance on data reflecting historical human

biases or because of their designs. Biases in

AI systems can increase over time through

feedback. That can occur, for example, if the

use of a biased AI system leads to systematic

errors in categorizations, which are then

reflected in the data set or data environment

the system uses to make future predictions.

Agencies should be mindful of the

interdependence of the models, metrics, and

data that underpin AI systems.

Identifying harmful biases in AI systems

can pose challenges. To identify and mitigate

biases, agencies should, to the extent

practical, consider whether other data or

methods are available. Agencies should

periodically examine and refresh AI

algorithms and other protocols to ensure that

they remain sufficiently current and reflect

new information and circumstances relevant

to the functions they perform.

Data science techniques for identifying and

mitigating harmful biases in AI systems are

developing. Agencies should stay up to date

on developments in the field of AI,

particularly on algorithmic fairness; establish

processes to ensure that personnel that reflect

various disciplines and relevant perspectives

are able to inspect AI systems and their

decisions for indications of harmful bias; test

AI systems in environments resembling the

ones in which they will be used; and make

use of internal and external processes for

evaluating the risks of harmful bias in AI

systems and for identifying such bias.

3. Technical Capacity

AI systems can help agencies conserve

resources, but they can also require

substantial investments of human and

financial capital. Agencies should carefully

evaluate the short- and long-term costs and

benefits of an AI system before committing

significant resources to it. Agencies should

also ensure they have access to the technical

expertise required to make informed

decisions about the type of AI systems they

require; how to integrate those systems into

their operations; and how to oversee,

maintain, and update those systems.

Given the data science field’s ongoing and

rapid development, agencies should consider

cultivating an AI-ready workforce, including

through recruitment and training efforts that

emphasize AI skills. When agency personnel

lack the skills to develop, procure, or

maintain AI systems that meet agencies’

needs, agencies should consider other means

of expanding their technical expertise,

including by relying on tools such as the

Intergovernmental Personnel Act,

prize

competitions, cooperative research and

development agreements with private

institutions or universities, and consultation

with external technical advisors and subject-

matter experts.

4. Obtaining AI Systems

Decisions about whether to obtain an AI

system can involve important trade-offs.

Obtaining AI systems from external sources

might allow agencies to acquire more

sophisticated tools than they could design on

their own, access those tools sooner, and save

some of the up-front costs associated with

developing the technical capacity needed to

design AI systems.

Creating AI tools within

agencies, by contrast, might yield tools that

are better tailored to the agencies’ particular

tasks and policy goals. Creating AI systems

within agencies can also facilitate

development of internal technical capability,

which can yield benefits over the lifetime of

the AI systems and in other technological

tasks the agencies may confront.

Certain government offices are available to

help agencies with decisions and actions

related to technology.

Agencies should make

appropriate use of these resources when

obtaining an AI system. Agencies should also

consider the cost and availability of the

technical support necessary to ensure that an

AI system can be maintained and updated in

a manner consistent with its expected life

cycle and service mission.

5. Data

AI systems require data, often in vast

quantities. Agencies should consider whether

they have, or can obtain, data that

appropriately reflect conditions similar to the

ones the agencies’ AI systems will address in

practice; whether the agencies have the

resources to render the data into a format that

can be used by the agencies’ AI systems; and

how the agencies will maintain the data and

link them to their AI systems without

compromising security or privacy. Agencies

should also review and consider statutes and

regulations that impact their uses of AI as a

potential collector and consumer of data.

6. Privacy

Agencies have a responsibility to protect

privacy with respect to personally

identifiable information in AI systems. In a

narrow sense, this responsibility demands

that agencies comply with requirements

related to, for instance, transparency, due

process, accountability, and information

quality and integrity established by the

Privacy Act of 1974, Section 208 of the

E-Government Act of 2002, and other

applicable laws and policies.

More broadly,

agencies should recognize and appropriately

manage privacy risks posed by an AI system.

Agencies should consider privacy risks

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00006 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6618

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

See Nat’l Inst. of Standards & Tech. Special

Publication SP–800–37 revision 2, Risk

Management Framework for Information Systems

and Organizations: A System Lifecycle Approach

for Security and Privacy (Dec. 2018); Office of

Mgmt. & Budget, Exec. Off. of the President,

Circular A–130, Managing Information as a

Strategic Resource (July 28, 2016); see also Nat’l

Inst. of Standards & Tech., NIST Privacy

Framework: A Tool for Improving Privacy Through

Enterprise Risk Management, Version 1.0 (Jan. 16,

2020).

See supra note 10; see also Office of Mgmt. &

Budget, Exec. Off. of the President, M–21–06,

Guidance for Regulation of Artificial Intelligence

Applications (Nov. 17, 2020); Nat’l Inst. for

Standards & Tech., Framework for Improving

Critical Infrastructure Cybersecurity (Apr. 16, 2018).

Cf. Admin. Conf. of the U.S., Recommendation

2018–3, Electronic Case Management in Federal

Administrative Adjudication, 83 FR 30,686 (June

29, 2018) (suggesting, in the context of case

management systems, that agencies consider

implementing electronic systems only when they

conclude that doing so would lead to benefits

without impairing either the objective ‘‘fairness’’ of

the proceedings or the subjective ‘‘satisfaction’’ of

those participating in those proceedings).

Courts would analyze such challenges under

the three-part balancing framework from Mathews

v. Eldridge, 424 U.S. 319, 335 (1976).

See 5 U.S.C. 706(2)(A). Courts would likely

review such challenges under the standard set forth

in Motor Vehicle Manufacturers Ass’n v. State Farm

Mutual Automobile Insurance Co., 463 U.S. 29, 43

(1983).

See 5 U.S.C. 553(b)–(c).

See Office of Mgmt. & Budget, Circular A–130,

supra note 10; Office of Mgmt. & Budget, Exec.

Office of the President, Circular A–123,

Management’s Responsibilities for Enterprise Risk

Management and Internal Control (July 15, 2016).

Admin. Conf. of the U.S., Recommendation

2016–4, Evidentiary Hearings Not Required by the

Administrative Procedure Act, 81 FR 94,314 (Dec.

23, 2016).

Recommendation 2016–4 addressed agency

adjudications in which an evidentiary hearing,

though not governed by the formal hearing

provisions of the Administrative Procedure Act

throughout the entire life cycle of an AI

system from development to retirement and

assess those risks, as well as associated

controls, on an ongoing basis. In designing

and deploying AI systems, agencies should

consider using relevant privacy risk

management frameworks developed through

open, multi-stakeholder processes.

7. Security

Agencies should consider the possibility

that AI systems might be hacked,

manipulated, fooled, evaded, or misled,

including through manipulation of training

data and exploitation of model sensitivities.

Agencies must ensure not only that their data

are secure, but also that their AI systems are

trained on those data in a secure manner,

make forecasts based on those data in a

secure way, and otherwise operate in a

secure manner. Agencies should regularly

consider and evaluate the safety and security

of AI systems, including resilience to

vulnerabilities, manipulation, and other

malicious exploitation. In designing and

deploying AI systems, agencies should

consider using relevant government guidance

or voluntary consensus standards and

frameworks developed through open, multi-

stakeholder processes.

8. Decisional Authority

Agencies should be mindful that most AI

systems will involve human beings in a range

of capacities—as operators, customers,

overseers, policymakers, or interested

members of the public. Human factors may

sometimes undercut the value of using AI

systems to make certain determinations.

There is a risk, for example, that human

operators will devolve too much

responsibility to AI systems and fail to detect

cases in which the AI systems yield

inaccurate or unreliable determinations. That

risk may be acceptable in some settings—

such as when the AI system has recently

been shown to perform significantly better

than alternatives—but unacceptable in

others.

Similarly, if agency personnel come to rely

reflexively on algorithmic results in

exercising discretionary powers, use of an AI

system could have the practical effect of

curbing the exercise of agencies’ discretion or

shifting it from the person who is supposed

to be exercising it to the system’s designer.

Agencies should beware of such potential

shifts of practical authority and take steps to

ensure that appropriate officials have the

knowledge and power to be accountable for

decisions made or aided by AI techniques.

Finally, there may be some circumstances

in which, for reasons wholly apart from

decisional accuracy, agencies may wish to

have decisions be made without reliance on

AI techniques, even if the law does not

require it. In some contexts, accuracy and

fairness may not be the only relevant values

at stake. In making decisions about their AI

systems, agencies may wish to consider

whether people will perceive the systems as

unfair, inhumane, or otherwise

unsatisfactory.

9. Oversight

It is essential that agencies’ AI systems be

subject to appropriate and regular oversight

throughout their lifespans. There are two

general categories of oversight: External and

internal. Agencies’ mechanisms of internal

oversight will be shaped by the demands of

external oversight. Agencies should be

cognizant of both forms of oversight in

making decisions about their AI systems.

External oversight of agencies’ uses of AI

systems can come from a variety of

government sources, including inspectors

general, externally facing ombuds, the

Government Accountability Office, and

Congress. In addition, because agencies’ uses

of AI systems might lead to litigation in a

number of circumstances, courts can also

play an important role in external oversight.

Those affected by an agency’s use of an AI

system might, for example, allege that use of

the system violates their right to procedural

due process.

Or they might allege that the

AI system’s determination violated the

Administrative Procedure Act (APA) because

it was arbitrary and capricious.

When an AI

system narrows the discretion of agency

personnel, or fixes or alters the legal rights

and obligations of people subject to the

agency’s action, affected people or entities

might also sue on the ground that the AI

system is a legislative rule adopted in

violation of the APA’s requirement that

legislative rules go through the notice-and-

comment process.

Agencies should

consider these different forms of potential

external oversight as they are making and

documenting decisions and the underlying

processes for these AI systems.

Agencies should also develop their own

internal evaluation and oversight

mechanisms for their AI systems, both for

initial approval of an AI system and for

regular oversight of the system, taking into

account their system-level risk management,

authorization to operate, regular monitoring

responsibilities, and their broader enterprise

risk management responsibilities.

Successful internal oversight requires

advance and ongoing planning and

consultation with the various offices in an

agency that will be affected by the agency’s

use of an AI system, including its legal,

policy, financial, human resources,

internally-facing ombuds, and technology

offices. Agencies’ oversight plans should

address how the agencies will pay for their

oversight mechanisms and how they will

respond to what they learn from their

oversight.

Agencies should establish a protocol for

regularly evaluating AI systems throughout

the systems’ lifespans. That is particularly

true if a system or the circumstances in

which it is deployed are liable to change over

time. In these instances, review and

explanation of the system’s functioning at

one stage of development or use may become

outdated due to changes in the system’s

underlying models. To enable that type of

oversight, agencies should monitor and keep

track of the data being used by their AI

systems, as well as how the systems use

those data. Agencies may also wish to secure

input from members of the public or private

evaluators to improve the likelihood that

they will identify defects in their AI systems.

To make their oversight systems more

effective, agencies should clearly define goals

for their AI systems. The relevant question

for oversight purposes will often be whether

the AI system outperforms alternatives,

which may require agencies to benchmark

their systems against the status quo or some

hypothetical state of affairs.

Finally, AI systems can affect how

agencies’ staffs do their jobs, particularly as

agency personnel grow to trust and rely on

the systems. In addition to evaluating and

overseeing their AI systems, agencies should

pay close attention to how agency personnel

interact with those systems.

Administrative Conference Recommendation

2020–3

Agency Appellate Systems

Adopted December 16, 2020

In Recommendation 2016–4,

the

Administrative Conference offered best

practices for evidentiary hearings in

administrative adjudications. Paragraph 26

recommended that agencies provide for

‘‘higher-level review’’ (or ‘‘agency appellate

review’’) of the decisions of hearing-level

adjudicators.

This Recommendation offers

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00007 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6619

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

(APA) (5 U.S.C. 554, 556–57), is required by statute,

regulation, or executive order. Those adjudications,

which are often as formal as APA adjudications in

practice, far outnumber so-called APA

adjudications. Although Recommendation 2016–4

addresses only non-APA adjudications, most of its

best practices are as applicable to APA

adjudications as non-APA adjudications. Some

such practices, in fact, are modeled on the APA’s

formal hearing provisions.

Admin. Conf. of the U.S., Recommendation 83–

3, Agency Structures for Review of Decisions of

Presiding Officers Under the Administrative

Procedure Act, 48 FR 57,461 (Dec. 30, 1983);

Admin. Conf. of the U.S., Recommendation 68–6,

Delegation of Final Decisional Authority Subject to

Discretionary Review by the Agency, 38 FR 19,783

(July 23, 1973). Both recommendations concerned

only the review of decisions in proceedings

governed by the formal hearing provisions of the

APA. Their principles, though, are not so confined.

Christopher J. Walker & Matthew Lee Wiener,

Agency Appellate Systems (Dec. 14, 2020) (report

to the Admin. Conf. of the U.S.), https://

www.acus.gov/report/final-report-agency-appellate-

systems.

best practices for such review. The

Administrative Conference intends this

Recommendation to cover appellate review

of decisions resulting from (1) hearings

governed by the formal hearing provisions of

the Administrative Procedure Act (APA) and

(2) evidentiary hearings that are not governed

by those provisions but are required by

statute, regulation, or executive order.

Agencies may also decide to apply this

Recommendation to appellate review of

decisions arising from other hearings,

depending on their level of formality.

Appellate review of hearing-level decisions

can be structured in numerous ways. Two

structures are most common. In the first,

litigants appeal directly to the agency head,

which may be a multi-member board or

commission. In the second, litigants appeal

to an appellate adjudicator or group of

adjudicators—often styled as a board or

council—sitting below the agency head. The

appellate decision may be the agency’s final

action or may be subject to further appeal

within the agency (usually to the agency

head).

The Administrative Conference has twice

before addressed agency appellate review. In

Recommendations 68–6 and 83–3, it

provided guidance to agencies when

establishing new, and reviewing existing,

organizational structures of appellate

review.

Both recommendations focused on

the selection of ‘‘delegates’’—individual

adjudicators, review boards composed of

multiple adjudicators, or panels composed of

members of a multi-member agency—to

exercise appellate review authority vested in

agency heads (including boards and

commissions). Recommendation 83–3 also

addressed when agencies should consider

providing appellate review as a matter of

right and when as a matter of discretion, and,

in the case of the latter, under what criteria.

With the exception of the appropriate

standard for granting review, this

Recommendation’s focus lies elsewhere. It

addresses, and offers best practices with

respect to, the following subjects: First, an

agency’s identification of the purpose or

objective served by its appellate review;

second, its selection of cases for appellate

review, when review is not required by

statute; third, its procedures for review;

fourth, its appellate decision-making

processes; fifth, its management,

administration, and bureaucratic oversight of

its appellate system; and sixth, its public

disclosure of information about its appellate

system.

Most importantly, this Recommendation

begins by suggesting that agencies identify,

and publicly disclose, the purpose(s) or

objective(s) of their appellate systems.

Appellate systems may have different

purposes, and any given appellate system

may have multiple purposes. Purposes or

objectives can include the correction of

errors, inter-decisional consistency of

decisions, policymaking, political

accountability, management of the hearing-

level adjudicative system, organizational

effectiveness and systemic awareness, and

the reduction of litigation in federal courts.

The identification of purpose is important

both because it dictates (or should dictate)

how an agency administers its appellate

system—including what cases it hears and

under what standards of review it decides

them—and provides a standard against

which an agency’s performance can be

evaluated.

This Recommendation proceeds from the

recognition that agency appellate systems

vary enormously—as to their purposes or

objectives, governing substantive law, size,

and resources—and that what may be a best

practice for one system may not always be

the best practice for another. In offering the

best practices that follow, moreover, the

Administrative Conference recognizes that

(1) an agency’s procedural choices may

sometimes be constrained by statute and (2)

available resources and personnel policies

may dictate an agency’s decision as to

whether and how to implement the best

practices that follow. The Administrative

Conference makes this Recommendation

subject to these important qualifications.

Recommendation

Objectives of Appellate Review

1. Agencies should identify the objective(s)

of appellate review; disclose those objectives

in procedural regulations; and design rules

and processes, especially for scope and

standard of review, to serve them.

Procedures for Appellate Review

2. Agencies should promulgate and publish

procedural regulations governing agency

appellate review in the Federal Register and

codify them in the Code of Federal

Regulations. These regulations should cover

all significant procedural matters pertaining

to agency appellate review, including but not

limited to the following:

a. The objectives of the agency’s appellate

review system;

b. The timing and procedures for initiating

review, including any available interlocutory

review;

c. The standards for granting review, if

review is discretionary;

d. The standards for permitting

participation by interested persons and

amici;

e. The standard of review;

f. The allowable and required submissions

by litigants and their required form and

contents;

g. The procedures and criteria for

designating decisions as precedential and the

legal effect of such designations;

h. The record on review and the

opportunity, if any, to submit new evidence;

i. The availability of oral argument or other

form of oral presentation;

j. The standards of and procedures for

reconsideration and reopening, if available;

k. Any administrative or issue exhaustion

requirements that must be satisfied before

seeking agency appellate or judicial review,

including whether agency appellate review is

a mandatory prerequisite to judicial review;

l. Openness of proceedings to the public

and availability of video or audio streaming

or recording;

m. In the case of multi-member appellate

boards, councils, and similar entities, the

authority to assign decision-making authority

to fewer than all members (e.g., panels); and

n. Whether seeking agency appellate

review automatically stays the effectiveness

of the appealed agency action until the

appeal is resolved (which may be necessary

for appellate review to be mandatory, see 5

U.S.C. 704), and, if not, how a party seeking

agency appellate review may request such a

stay and the standards for deciding whether

to grant it.

3. Agencies should include in the

procedural regulations governing their

appellate programs: (a) A brief statement or

explanation of each program’s review

authority, structure, and decision-making

components; and (b) for each provision based

on a statutory source, an accompanying

citation to that source.

4. When revising existing or adopting new

appellate rules, agencies should consider the

appellate rules (Rules 400–450) in the

Administrative Conference’s Model Rules of

Agency Adjudication (rev. 2018).

5. When materially revising existing or

adopting new appellate rules, agencies

should use notice-and-comment procedures

or other mechanisms for soliciting public

input, notwithstanding the procedural rules

exemption of 5 U.S.C. 553(b)(A), unless the

costs clearly outweigh the benefits of doing

so.

Case Selection for Appellate Review

6. Based on the agency-specific objectives

of appellate review, agencies should decide

whether the granting of review should be

mandatory or discretionary (assuming they

have statutory authority to decide); if

discretionary, the criteria for granting review

should track the objectives of the appellate

system, and they should be published in the

procedural regulations.

7. Agencies should consider implementing

procedures for sua sponte appellate review of

non-appealed hearing-level decisions, as well

as for the referral of cases or issues by

hearing-level adjudicators to the appellate

entity for interlocutory review.

Appellate Decision-Making Processes and

Decisions

8. Whenever possible, agencies should

consider maintaining electronic case

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00008 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6620

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

See Federal Acquisition Regulation, 48 CFR ch.

1; see also Competition in Contracting Act of 1984,

Public Law 98–369, div. B, tit. VII, 98 Stat. 494,

942–85 (codified, as amended, in various parts of

the U.S. Code); Federal Acquisition Streamlining

Act of 1994, Public Law 103–355, 108 Stat. 3243;

Federal Acquisition Reform Act of 1996, Public Law

104–106, 110 Stat. 186 (later renamed the Clinger-

Cohen Act of 1996); Exec. Order No. 12,979, Agency

Procurement Protests, 60 FR 55,171 (Oct. 25, 1995).

See 48 CFR ch. 1.

See Admin. Conf. of the U.S., Recommendation

95–5, Government Contract Bid Protests, 60 FR

43,108, 43,113 (Aug. 18, 1995).

See 4 CFR 21.0(a)(1) (defining ‘‘interested party’’

for purposes of bid protest proceedings before the

Government Accountability Office); 48 CFR 33.101

(defining ‘‘interested party’’ for purposes of bid

protest proceedings before procuring agencies);

CliniComp Int’l, Inc. v. United States, 904 F.3d

1353, 1358 (Fed. Cir. 2018) (defining ‘‘interested

party’’ for purposes of 28 U.S.C. 1491(b), which

covers actions in the Court of Federal Claims).

There are some instances in which Congress has

restricted the ability to file a protest, regardless of

whether a vendor is an ‘‘interested party.’’ See, e.g.,

41 U.S.C. 4106(f) (limiting the ability to protest the

issuance or proposed issuance of a task or delivery

order); 48 CFR 16.505(a)(10) (same).

See 48 CFR 33.103.

See 31 U.S.C. 3552(a), 3553(a). For civilian

agencies, GAO has exclusive jurisdiction over

protests of task and delivery orders in excess of $10

million, unless the protest is on the grounds that

management systems that ensure that hearing

records are easily accessible to appellate

adjudicators. Such systems may include the

capability for electronic filing.

9. Although the randomized assignment of

cases to appellate adjudicators is typically an

appropriate docketing method for an agency

appellate system, agencies should consider

the potential benefits of sorting and grouping

appeals on the appellate docket, such as

reduced case processing times and more

efficient use of adjudicators’, staff attorneys’,

and law clerks’ skills and time. Criteria for

sorting and grouping cases may include the

size of a case’s record, complexity of a case’s

issues, subject matter of a case, and similarity

of a case’s legal issues to those of other

pending cases.

10. Consistent with the objectives of the

agency’s appellate system and in light of the

costs of time and resources, agencies should

consider adopting an appellate model of

judicial review in which the standard of

review is not de novo with respect to

findings of fact and application of law to

facts. For similar reasons, many agencies

should consider limiting the introduction of

new evidence on appeal that is not already

in the administrative record from the

hearing-level adjudication.

11. Taking agency resources into account,

agencies should emphasize concision,

readability, and plain language in their

appellate decisions and explore the use of

decision templates, summary dispositions,

and other quality-improving measures.

12. Agencies should establish clear criteria

and processes for identifying and selecting

appellate decisions as precedential,

especially for appellate systems with

objectives of policymaking or inter-decisional

consistency.

13. Agencies should assess the value of

oral argument and amicus participation in

their appellate system based on the agencies’

identified objectives for appellate review and

should establish rules governing both.

Criteria that may favor oral argument and

amicus participation include issues of high

public interest; issues of concern beyond the

parties to the case; specialized or technical

matters; and a novel or substantial question

of law, policy, or discretion.

Administration, Management, and

Bureaucratic Oversight

14. Agency appellate systems should

promptly transmit their precedential

decisions to all appellate program

adjudicators and, directly or through hearing-

level programs, to hearing-level adjudicators

(as appropriate). Appellate programs should

include in their transmittals, when feasible,

brief summaries of the decision.

15. Agencies should notify their

adjudicators of significant federal court

decisions reviewing the agencies’ decisions

and, when providing notice, explain the

significance of those decisions to the

program. As appropriate, agencies should

notify adjudicators if the agency will not

acquiesce in a particular decision of the

federal courts of appeals.

16. Agencies in which decision making

relies extensively on their own precedential

decisions should consider preparing or

having prepared indexes and digests—with

annotations and comments, as appropriate—

to identify those decisions and their

significance.

17. As appropriate, agency appellate

systems should communicate with agency

rule-writers and other agency policymakers—

and institutionalize communication

mechanisms—to address whether recurring

issues in their decisions should be addressed

by rule rather than precedential case-by-case

adjudication.

18. The Office of the Chairman of the

Administrative Conference should provide

for, as authorized by 5 U.S.C. 594(2), the

‘‘interchange among administrative agencies

of information potentially useful in

improving’’ agency appellate systems. The

subjects of interchange might include

electronic case management systems,

procedural innovations, quality-assurance

reviews, and common management

problems.

Public Disclosure and Transparency

19. Agencies should disclose on their

websites any rules (sometimes styled as

‘‘orders’’), and statutes authorizing such

rules, by which an agency head has delegated

review authority to appellate adjudicators.

20. Regardless of whether the Government

in the Sunshine Act (5 U.S.C. 552b) governs

their appellate review system, agencies

should consider announcing, livestreaming,

and maintaining video recordings on their

websites of appellate proceedings (including

oral argument) that present significant legal

and policy issues likely to be of interest to

regulated parties and other members of the

public. Brief explanations of the issues to be

addressed by oral argument may usefully be

included in website notices of oral argument.

21. Agencies should include on their

websites brief and accessibly written

explanations as to how their internal

decision-making processes work and, as

appropriate, include links to explanatory

documents appropriate for public disclosure.

Specific subjects that agencies should

consider addressing include: The process of

assigning cases to adjudicators (when fewer

than all of the programs’ adjudicators

participate in a case), the role of staff, and the

order in which cases are decided.

22. When posting decisions on their

websites, agencies should distinguish

between precedential and non-precedential

decisions. Agencies should also include a

brief explanation of the difference.

23. When posting decisions on their

websites, agencies should consider

including, as much as practicable, brief

summaries of precedential decisions and, for

precedential decisions at least, citations to

court decisions reviewing them.

24. Agencies should include on their

websites any digests and indexes of decisions

they maintain. It may be appropriate to

remove material exempt from disclosure

under the Freedom of Information Act or

other laws.

25. Agencies should affirmatively solicit

feedback concerning the functioning of their

appellate systems and provide a means for

doing so on their websites.

Administrative Conference Recommendation

2020–4

Government Contract Bid Protests Before

Agencies

Adopted December 17, 2020

Federal law establishes policies and

procedures governing how federal executive

agencies procure supplies and services.

The

primary source of these policies and

procedures is the Federal Acquisition

Regulation (FAR),

which applies to all

executive-agency acquisitions of supplies

and services with appropriated funds by and

for the use of the federal government, unless

expressly excluded. Other relevant policies

and procedures are found in federal statutes

and agencies’ own procurement rules.

If a vendor believes a federal executive

agency has not complied with the law or the

terms of a solicitation, it may file what is

called a bid protest—that is, a written

objection to a government agency’s conduct

in acquiring supplies and services for its

direct use or benefit.

Responding to bid

protests can require agencies to reevaluate

their procurement processes and, sometimes,

make improvements. That, in turn, results in

more competitive, fairer, and more

transparent procurement processes,

benefiting vendors, agencies, and ultimately

the public.

To file a bid protest, an actual or

prospective vendor must show that it is an

‘‘interested party’’—meaning that its direct

economic interest would be adversely

affected by the award of, or failure to award,

the contract in question

—and that it

suffered prejudice because of an error in the

procurement process. Ordinarily, vendors

who meet those requirements may file bid

protests in any of three forums: (1) The

procuring agency,

(2) the Government

Accountability Office (GAO),

or (3) the

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00009 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6621

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

the order increases the scope, period, or maximum

value of the contract. See 41 U.S.C. 4106(f); 48 CFR

16.505(a)(10).

See 28 U.S.C. 1491(b).

See Admin. Conf. of the U.S., Info. Interchange

Bull. No. 007, Agency Bid Protests (June 2020),

https://www.acus.gov/fact-sheet/iib-007-agency-bid-

protests.

See Exec. Order. No. 12,979, Agency

Procurement Protests, 60 FR 55,171, 55,171 (Oct.

25, 1995).

See Christopher Yukins, Stepping Stones to

Reform: Making Agency-Level Bid Protests Effective

for Agencies and Bidders by Building on Best

Practices from Across the Federal Government 12–

13 (May 1, 2020) (report to the Admin. Conf. of the

U.S.), https://www.acus.gov/report/agency-level-

bid-protests-final-report.

Id. at 23.

Id. at 13.

48 CFR 33.103(g).

41 U.S.C. 7101–09.

See id. §7103(f)(1)–(2).

See id. §605(c)(5).

Yukins, supra note 10, at 39.

48 CFR 33.103(g) (italics added).

4 CFR 21.3(c)–(d); 48 CFR 33.104(a).

Yukins, supra note 10, at 31.

See 4 CFR 21.0(e), 21.2.

See Yukins, supra note 10, at 13–14, 18–19.

See id. at 23.

48 CFR 33.103(f). Under certain circumstances,

the agency can override the regulatory stay for

agency-level protests. See 48 CFR 33.103(f)(1), (f)(3).

31 U.S.C. 3553(c)(1), (d)(3). Under certain

circumstances, the agency can override the

statutory stay for protests to GAO. See 31 U.S.C.

3553(c)–(d); 48 CFR 33.104(b)–(c).

United States Court of Federal Claims

(COFC),

and depending on where the protest

is initiated, may be able to file protests in

series. For example, a protest may be filed

first at the agency, then (if unsuccessful at

the agency) at GAO, and then (if again

unsuccessful) at COFC.

The procedural tools

available in a given forum, along with other

strategic and cost considerations, typically

drive vendors’ decisions about where to file

their bid protests.

Bid protests filed with procuring agencies

are commonly referred to as agency-level

protests. Agency-level protests have

important benefits for the public, contractors,

procuring agencies, and COFC and GAO. By

‘‘provid[ing] for inexpensive, informal,

procedurally simple, and expeditious

resolution of protests,’’

agency-level protest

mechanisms allow small businesses (among

other vendors) to affordably contest agencies’

procurement decisions. They also give

procuring agencies the chance to review and

improve their own procurement practices.

And they funnel some protests away from

COFC and GAO, reducing the likelihood that

the number of protests will overwhelm those

institutions.

Vendors, however, seldom file agency-level

protests. Although there is little data on the

number of agency-level protests filed each

year, available evidence suggests that

substantially more protests are filed with

COFC and GAO each year than with

procuring agencies.

There are several

reasons why vendors may forego agency-level

protests. Those reasons implicate the themes

of transparency, predictability, and

accountability.

First, some vendors report shying away

from agency-level protests because they

perceive the agency as unlikely to change its

decision.

Sometimes, for instance, the

official responsible for soliciting or awarding

a procurement contract is also responsible for

handling any agency-level protests that are

filed regarding the procurement. This

perception of a pre-judgment by the agency

may cause some vendors to file their protests

at GAO or COFC, rather than at the agency

level.

Second, some vendors report that they

view agency-level protest processes as

opaque.

Agencies do not publish or provide

comprehensive data on their bid protest

decisions. And the FAR and agency-specific

bid protest rules establish few hard-and-fast

requirements for the process. For example,

although the FAR states that ‘‘[a]gencies shall

make their best efforts to resolve agency

protests within 35 days after [an agency-level

protest] is filed,’’

that language is hortatory

and does not establish any binding deadlines

for agency decisions. Nothing in the FAR

does. The failure to provide for any binding

deadlines distinguishes the FAR from other

federal procurement statutes, such as the

Contract Disputes Act,

which sets or

requires contracting officers to set firm

deadlines for deciding most claims

and

provides that the passage of the deadline for

a claim means the claim is deemed denied.

Third, some vendors report being

dissuaded by their inability to compel

production of the procurement record as part

of an agency-level protest.

The FAR gives

disappointed offerors the right to an agency

debriefing—a procedure whereby contracting

personnel provide offerors with an

explanation of the agency’s evaluation

process and an assessment of the offerors’

proposals. But nothing in the FAR guarantees

vendors the right to view the procurement

record itself. The FAR provides only that

agencies ‘‘may exchange relevant

information’’ with agency-level protesters.

By contrast, vendors who file bid protests at

GAO may demand to see the entire record of

the procurement, and procuring agencies

must respond to such requests within

twenty-five days and produce the responsive

documents within thirty days (unless they

are withheld for a valid reason).

Finally, some vendors deem agency-level

protests to be too risky.

In many cases,

vendors who do not obtain relief through an

agency-level protest will seek relief from

GAO by pursuing their protest in that forum.

But GAO’s deadline for filing such ‘‘follow-

on protests’’ often begins to run as soon as

the vendor has actual or constructive notice

of some ‘‘adverse agency action,’’ which can

occur before a protester receives the decision

in its agency-level protest.

In this way,

delayed notification about an agency’s

decision in a bid protest can seriously

prejudice protesters’ rights at GAO.

This

causes some vendors to forego agency-level

protests altogether.

The perception that agency-level protests

lack transparency, predictability, and

accountability also makes it more likely that

protesters who do file at the agency level and

whose protests are denied will file follow-on

protests with GAO or COFC. Such follow-on

protests not only tax the limited resources of

GAO and COFC, but also can disrupt

activities at procuring agencies. For instance,

just as the filing of an agency-level protest

automatically prohibits the contract from

being awarded or performed until the agency

denies or dismisses the protest and takes

some adverse action,

a follow-on protest at

GAO may automatically prevent the contract

from being awarded or performed (if the

requisite filing deadlines are met) until GAO

denies or dismisses the protest.

Thus, when

an agency-level protest is followed by

another protest at GAO, delays in

procurements can be substantial.

Protesters, agencies, and the public would

all benefit from an improved agency-level

protest system. Protesters would benefit

because agency-level protests are typically

the least formal and least costly types of bid

protest procedures. Agencies would benefit

from an improved agency-level protest

system because greater use of agency-level

protests means more agency control over the

timing and conduct of protests and more

opportunities for agencies to superintend

their own procurement processes. And the

public would benefit from more competitive,

fairer, and more transparent agency

procurements.

Because an improved agency-level protest

system is of significant value to contractors,

agencies, and the public, this

Recommendation identifies changes to make

it more likely vendors will avail themselves

of agency-level protest procedures. The

recommended changes reflect three

overarching principles—transparency,

simplicity, and predictability—meant to

address contractors’ principal concerns about

agency-level protest systems.

Recommendation

Identification of Decisions Subject to

Agency-Level Protests

1. Agencies should clearly identify which

categories of procurement decisions may or

may not be made the subjects of agency-level

protests.

Transparency for the Process and Personnel

for Agency-Level Protests

2. Agencies should formalize and compile

in a document that is publicly available

online the procedures they apply in

adjudicating agency-level protests. In so

doing, they should be guided by the

principles set out in Recommendation 2018–

5, Public Availability of Adjudication Rules.

3. Agencies should clearly identify who

within the agency will adjudicate an agency-

level protest. They should consider

designating at least one Agency Protest

Official (APO)—a person who specializes in

handling agency-level protests—to oversee

and coordinate agency-level protests and

hear protests brought to a level above the

contracting officer. Agencies lacking the

resources to designate their own APO might

consider sharing an APO with other agencies.

Notice of the Timeline for Agency-Level

Protests

4. Agencies should consider adopting

presumptive timelines for agency-level

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00010 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6622

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

See Admin. Conf. of the U.S., Recommendation

2016–2, Aggregate Agency Adjudication, 81 FR

40,260, 40,260 (June 21, 2016).

Admin. Conf. of the U.S., Recommendation

2018–5, Public Availability of Adjudication Rules,

84 FR 2142 (Feb. 6, 2019); Admin. Conf. of the U.S.,

Recommendation 2017–1, Adjudication Materials

on Agency Websites, 82 FR 31,039 (July 5, 2017).

Cf. Admin. Conf. of the U.S., Recommendation

2018–4, Recusal Rules for Administrative

Adjudicators, 84 FR 2139 (Feb. 6, 2019).

See, e.g., Lucia v. SEC, 138 S. Ct. 2044 (2018);

Arthrex v. Smith & Nephew, 941 F.3d 1320 (Fed.

Cir. 2019), cert. granted, __S. Ct. __(Oct. 13, 2020)

(No. 19–1434).

FOIA Improvement Act of 2016, Public Law

114–185, 2, 130 Stat. 538, 538 (amending 5 U.S.C.

552(a)(2)); E-Government Act of 2002, Public Law

140–347, 206, 116 Stat. 2899, 2916 (amending 44

U.S.C. 3501).

See, e.g., Admin. Conf. of the U.S.,

Recommendation 2018–4, Recusal Rules for

Administrative Adjudicators, 84 FR 2139 (Feb. 6,

2019).

See 5 U.S.C. 554, 556–57.

The vast majority of ALJs work at the Social

Security Administration. AJs work at many

different agencies under a variety of titles,

including not only ‘‘Administrative Judge’’ but also,

by way of example, ‘‘Hearing Officer,’’

‘‘Immigration Judge,’’ ‘‘Veterans Law Judge,’’

‘‘Administrative Patent Judge,’’ and

‘‘Administrative Appeals Judge.’’

5 U.S.C. 554, 557, 3105, 4301, 5372, 7521; 5 CFR

pt. 930, subpt. B; Exec. Order No. 13,843, Executive

Order Excepting Administrative Law Judges from

the Competitive Service, 83 FR 32,755 (July 13,

2018) (issued July 10, 2018).

Kent Barnett et al., Non-ALJ Adjudicators in

Federal Agencies: Status, Selection, Oversight, and

Removal 1 (Sept. 24, 2018) (report to the Admin.

Conf. of the U.S.), https://www.acus.gov/report/non-

alj-adjudicators-federal-agencies-status-selection-

oversight-and-removal-1.

Leigh Anne Schriever, Public Availability of

Information About Adjudicators 10 (Nov. 23, 2020)

protests, similar to the ones under the

Contract Disputes Act. Agencies should also

make best efforts to notify protesters of the

timelines applicable to their agency-level

protests.

5. Agencies should clearly and

immediately provide written notice to

protesters of any adverse agency action

affecting the rights of the protester under the

challenged procurement. Agency rules

should provide that protests are deemed

denied after a specified number of days

without a decision and that agencies may

grant case-specific extensions based on

identified criteria.

Compiling the Record and Making It

Available

6. Agencies should make available to

protesters as much of the procurement record

as is feasible. To address confidential

information in the record, agencies should

consider using tools such as enhanced

debriefings.

7. Agencies should consider adopting a

thirty-day deadline, running from the date a

protest is filed, for providing protesters with

as much of the procurement record as is

feasible.

Protecting Against Adverse Consequences

8. Although the Federal Acquisition

Regulation (FAR) prohibits the award of a

contract or continued performance under an

awarded contract during an agency-level

protest, agencies should provide for a short

extension of the stay after a final decision in

an agency-level bid protest as permitted by

the FAR. The short extension should be of

sufficient duration (e.g., five days) to give the

protester time to bring a follow-on protest at

the Government Accountability Office (GAO)

or the United States Court of Federal Claims

after the agency’s decision.

9. Congress should provide that, if a

protester promptly files a GAO protest after

an adverse decision in an agency-level

protest, the agency shall not award the

contract or commence performance under the

contract during the pendency of the GAO

protest, subject to potential override in

urgent and compelling circumstances.

10. GAO should amend its bid protest

procedures to ensure that follow-on protests

at GAO are handled on an expedited basis,

to the extent feasible.

Publishing Data on Agency-Level Protests

11. Agencies should collect and annually

publish data about the bid protests they

adjudicate. To the extent feasible, the data

should at least include what the GAO

currently provides in its annual reports about

the bid protests it adjudicates (e.g., the

number of bid protests filed with the agency;

the effectiveness rate of agency-level bid

protests (the ratio of protests sustained or in

which corrective action is afforded versus

total agency-level protests filed); the number

of merits decisions by the agency; the

number of decisions sustaining the protest;

the number of decisions denying the protest;

and the time required for bid protests to be

resolved).

Administrative Conference Recommendation

2020–5

Publication of Policies Governing Agency

Adjudicators

Adopted December 17, 2020

[

NOTE

: Appendix B referenced in this

Recommendation has been omitted from this

notice because of the inaccessible images it

contains. The full appendix may be found

online at https://www.acus.gov/

recommendation/publication-policies-

governing-agency-adjudicators.]

Federal agency officials throughout the

country preside over hundreds of thousands

of adjudications each year.

As the

Administrative Conference has previously

observed, litigants, their lawyers, and other

members of the public benefit from having

ready online access to procedural rules,

decisions, and other key materials associated

with adjudications.

They also benefit from

having ready online access to the policies

and practices by which agencies appoint and

oversee administrative law judges and other

adjudicators. The availability of these

policies and practices helps inform the

public about, among other things, any actions

agencies have taken to ensure the

impartiality of administrative adjudicators

and promotes an understanding of

adjudicators’ constitutional status under the

Appointments Clause and other

constitutional provisions. The

Administrative Conference acknowledges

ongoing litigation regarding the

constitutional status of many agency

adjudicators and the continuing validity of

the means and circumstances of their

appointment and removal.

Agencies may benefit from disclosures

about agency adjudicators because it allows

them to compare their own policies with

those made publicly available by other

agencies. Agencies’ proactive disclosures,

which may sometimes already be required

under the Freedom of Information Act and

the E-Government Act, may also be more

cost-effective than agencies’ responding to

individual requests for information.

Like other recent recommendations

regarding adjudicators,

this

Recommendation pertains to officials who

preside over (1) hearings governed by the

formal hearing provisions of the

Administrative Procedure Act (APA)

and (2)

hearings that are not governed by those

provisions but are required by statute,

regulation, or executive order. It also covers

officials (agency heads excluded) who review

hearing-level adjudicators’ decisions on

appeal. For ease of reference, this

Recommendation refers to the covered

adjudicators as either ‘‘administrative law

judges’’ (ALJs) or ‘‘administrative judges’’

(AJs).

Agencies may decide to include on

their websites the disclosures identified in

this Recommendation for other adjudicators,

depending on the level of formality of the

proceedings over which they preside and

whether they serve as full-time adjudicators.

Agencies may also decide to make similar

disclosures with respect to agency heads if

their websites do not already provide

sufficient information.

This Recommendation focuses on policies

and practices relating to adjudicators that

agencies should disclose, including those

addressing appointment and qualifications;

compensation (including salaries, bonuses,

and performance incentives); duties and

responsibilities; supervision and assignment

of work; position within agencies’

organizational hierarchies; methods of

evaluating performance; limitations on ex

parte communications and other policies

ensuring separation between adjudicative

and enforcement functions; recusal and

disqualification; the process for review of

adjudications; and discipline and removal.

Many of the policies and practices

applicable to ALJs governing these matters

are already publicly available because they

are in the APA, Office of Personnel

Management rules, or other legal authorities.

Nevertheless, agencies that employ ALJs can

take steps to improve the public’s access to

this information.

ALJs, in any case, make up a small portion

of federal adjudicators. There are many more

AJs than ALJs.

AJs are regulated by a

complex mix of statutory provisions,

including civil service laws, agency rules

codified in the Code of Federal Regulations,

and agency-specific policies that take a

variety of forms. Many types of information

about AJs reside in these sources, but they

may be difficult to find.

Some relevant

VerDate Sep<11>2014 19:27 Jan 21, 2021 Jkt 253001 PO 00000 Frm 00011 Fmt 4703 Sfmt 4703 E:\FR\FM\22JAN1.SGM 22JAN1

jbell on DSKJLSW7X2PROD with NOTICES

6623

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices

(report to the Admin. Conf. of the U.S.), https://

www.acus.gov/report/final-report-public-

availability-information-about-agency-adjudicators.

Id. at 7.

5 U.S.C. 552.

Id. §552a.

Cf. Admin. Conf. of the U.S., Recommendation

2017–3, Plain Language in Regulatory Drafting, 82

FR 61,728 (Dec. 29, 2017).

sources may not be publicly available,

including internal administrative and

personnel manuals, position descriptions,

and labor agreements. This is particularly

true with respect to certain kinds of policies,

such as those relating to compensation and

performance incentives.

Of course, the

Administrative Conference recognizes that

some of these agency policies and practices

may qualify for an exemption under the

Freedom of Information Act,

Privacy Act,

or other laws and executive-branch policies.

Agency websites are the most helpful

location for agencies to make relevant

policies and practices publicly available.

Individuals most naturally seek information

about administrative policies and practices

on agencies’ websites. Agencies can situate

information about their adjudicators in a

logical and easily identifiable place on their

websites and structure their websites to

synthesize policies in plain language and

link to information from many different

sources.

This Recommendation encourages agencies

to post on their websites clear and readily

accessible descriptions of the policies

governing the appointment and oversight of

ALJs and AJs, and to include links to relevant

legal documents. How, exactly, they should

do so will of course depend on the specific

features of their adjudicative programs and

their institutional needs.

Recommendation

1. Each adjudicative agency should

prominently display on its website a short,

straightforward description of all generally

applicable policies and practices, along with

the legal authority, governing the

appointment and oversight of Administrative

Law Judges (ALJs) and Administrative Judges

(AJs), including, as applicable, those that

address:

a. Procedures for assessing, selecting, and

appointing candidates for adjudicator

positions and the legal authority under

which such appointments are made;

b. Placement of adjudicators within

agencies’ organizational hierarchies;

c. Compensation structure and

performance incentives, such as bonuses,

nonmonetary awards, and promotions;

d. Procedures for assigning cases;

e. Assignment, if any, of nonadjudicative

duties to adjudicators;

f. Limitations on ex parte communications,

including between adjudicators and other

agency officials, related to the disposition of

individual cases, as well as other policies

ensuring a separation of adjudication and

enforcement functions;

g. Standards for recusal by and

disqualification of adjudicators;

h. Administrative review of adjudicators’

decisions;

i. Supervision of adjudicators by higher-

level officials;

j. Evaluation of adjudicators, including

quantitative and qualitative methods for

appraising adjudicators’ performances, such

as case-processing goals, if any; and

k. Discipline and removal of adjudicators.

Agencies may choose not to provide access

to policies covered by a Freedom of

Information Act exemption.

2. On the same web page as the

information described in Paragraph 1

appears, each adjudicative agency should

post links to key legal documents or, when

links are not available, citations to such

documents. These documents may include

(a) federal statutes, including relevant

provisions of the Administrative Procedure

Act (APA) and other laws applicable to ALJs

and AJs; (b) agency-promulgated rules

regarding adjudicators, including Office of

Personnel Management rules applicable to

ALJs; (c) publicly available agency-

promulgated guidance documents relating to

adjudicators, including manuals, bench

books, and other explanatory materials; (d)

delegations of authority; and (e) position

descriptions. To the extent that some policies

concerning adjudicators may be a matter of

custom, such as assignment of

nonadjudicative duties, each adjudicative

agency should consider documenting those

policies to make them publicly accessible to

the extent practicable.

3. The web page containing the

information described in Paragraphs 1 and 2

should present the materials in a clear,

logical, and comprehensive fashion. One

possible method of presenting this

information appears in Appendix A. The

appendix gives one example for ALJs and

another for AJs.

4. If an agency’s mission consists

exclusively or almost exclusively of

conducting adjudications, the agency should

provide a link to the web page containing the

information described in Paragraphs 1 and 2

on the agency’s homepage. If conducting

adjudications is one of an agency’s many

functions, the agency should provide a link

to these materials from a location on the

website that is both dedicated to adjudicative

materials and logical in terms of a user’s

likelihood of finding the documents in the

selected location. One example would be an

enforcement or adjudication page or the

homepage for the component in which a

particular category of adjudicators works.

Citations to agency web pages that currently

provide this information in a way that makes

it easy for the public to locate, as well as

descriptions of how to find those pages on

agency websites, appear in Appendix B.

Appendix A

Sample Website Text for Administrative Law

Judges

About Our Administrative Law Judges

Administrative Law Judges (ALJs) at

[agency] conduct hearings and decide cases

under [insert name of authorizing act]. They

are part of the [agency component in which

ALJs are located], which is directed by [title

of office head] and has offices in [cities]. Visit

[link to agency organization chart] to see how

[office] relates to other offices at [agency].

[Agency] is committed to ensuring that all

hearings and appeals are conducted in a fair

and equitable manner. Parties are entitled to

a due process hearing presided over by an

impartial, qualified ALJ. ALJs resolve cases

involving [kinds of cases ALJs hear] in a fair,

transparent, and accessible manner. Our ALJs

are appointed by [agency official], and are

[describe qualifications]. ALJs are paid

according to the [pay scale for ALJs with link

to the scale] scale set by statute under 5

U.S.C. 5372, subject to annual pay

adjustments.

Cases are assigned to ALJs [in each

geographic office] in rotation so far as

practicable. The ALJ assigned to your case is

responsible for [job duties, like taking

evidence, hearing objections, issuing

decisions]. ALJs are required by statute to

perform their functions impartially. 5 U.S.C.

556(b). To ensure impartiality, they do not

take part in investigative or enforcement

activities, nor do they report to officials in

the [agency]’s investigative or enforcement

components. 5 U.S.C. 554(d), 3105. The ALJ

assigned to your case may not communicate

privately about the facts of your case with

other agency officials. [More details on

[agency]’s rules about communicating with

ALJs are available [location of agency-

specific ex parte prohibitions]].

By law, [agency] does not reward or

discipline ALJs for their decisions. A federal

statute provides that [agency] may remove, or

take certain other disciplinary actions,

against an ALJ it employs only for good cause

established and determined by the Merit

Systems Protection Board on the record after

opportunity for hearing before the Board. 5

U.S.C. 7521.

The agency has adopted rules of recusal

[link] that allow a participant to request that

the ALJ in charge of his or her case be

disqualified if the participant believes the

ALJ cannot fairly and impartially decide the

case.

If you are dissatisfied with an ALJ’s

decision, you can request reconsideration

from the ALJ or appeal that decision to

[agency office/official]. Visit [link] for

information on appealing an ALJ decision.

[Agency office/official] may also review your

case on [its/his or her] own initiative if there

is an issue with the ALJ’s decision.

For Further Information:

•Hiring process: [link]

•Pay rates: [link]

•How cases are assigned to ALJs: [link]

•Communicating with ALJs (ex parte

communications): [link]

•Process for addressing allegations that an

ALJ has a conflict of interest (recusal and

disqualification procedures): [link]

•How to appeal an ALJ decision: [link]

•Case-processing goals: [link]

•Process for addressing allegations of ALJ

misconduct: [link]

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see a list of all the cited cases and legislation of a document.

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see a list of all the documents that have cited the case.

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see the revised versions of legislation with amendments.

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see any amendments made to the case.

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see a visualisation of a case and its relationships to other cases. An alternative to lists of cases, the Precedent Map makes it easier to establish which ones may be of most relevance to your research and prioritise further reading. You also get a useful overview of how the case was received.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Subscribers are able to see the list of results connected to your document through the topics and citations Vincent found.

You can sign up for a trial and make the most of our service including these benefits.

Request your trial

Why Sign-up to vLex?

Over 100 Countries

Search over 120 million documents from over 100 countries including primary and secondary collections of legislation, case law, regulations, practical law, news, forms and contracts, books, journals, and more.
Thousands of Data Sources

Updated daily, vLex brings together legal information from over 750 publishing partners, providing access to over 2,500 legal and news sources from the world’s leading publishers.
Find What You Need, Quickly

Advanced A.I. technology developed exclusively by vLex editorially enriches legal information to make it accessible, with instant translation into 14 languages for enhanced discoverability and comparative research.
Over 2 million registered users

Founded over 20 years ago, vLex provides a first-class and comprehensive service for lawyers, law firms, government departments, and law schools around the world.

Adoption of Recommendations

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users

You can sign up for a trial and make the most of our service including these benefits.

Why Sign-up to vLex?

Over 100 Countries

Thousands of Data Sources

Find What You Need, Quickly

Over 2 million registered users