Analysis of GAO Protest Outcomes

Key Takeaways

• High level statistics reported annually by the GAO oversimplify wide variations in protest outcomes
• Most protests are resolved within 45 days and serial protestors are rare
• Protest sustainment rates vary from 0% to 43% depending on the procurement agency
• Small businesses file the majority of protests, but have their protests sustained less often
• Assigned GAO attorney is a major factor in protest outcome - attorney-level sustainment rates vary from 0% to 47%

Overview of GAO Report to Congress

In November 2022, the General Accountability Office (GAO) released its annual report to Congress summarizing the result of contracting protests decided during Fiscal Year 2022. General trends in the report followed those of prior years with the number of cases filed continuing to decline (down 12% from 2021) with the sustainment rate of decisions remaining in the mid-teens. Notably, the Effectiveness Rate, the rate at which the protestor receives some sort of relief from the procurement agency either from a sustainment or voluntary agency corrective action, grew to 51% in 2022.

High Level Trends in GAO Protests

With the continued decline in the number of protests and the relatively low sustainment rate (which was as high as 23% in FY 2016), we took a closer look at the data to answer key questions about the protest system that are not addressed by the GAO's report to better understand when protests are likely to receive a merit decision and ultimately be sustained.

This analysis combines raw protest docket data obtained from gao.gov with contractor registration and procurement data. See Notes on Methodology and Analysis at the end of this report for more details.

How Long Does It Take to Resolve Protests?

In most cases, protests are resolved well within the 100-day deadline. On average, in FY 2022 cases were resolved in 45 calendar days. For cases decided on merit, the average time to resolve the protest was 78 days. Approximately 45% of protests were resolved before the 30th day, the point at which agencies are required to provide a formal protest response. For agencies with at least 20 protests, the average time to resolve protests ranged from 39 days to 55 days.

Protest Resolutions By Days

Are There Sustainment Differences Between Agencies?

As shown below, total protest and sustainment rates vary greatly by agency. Unsurprisingly, the Department of Defense led the way with 779 protests, with the Department of Veterans Affairs (140) and the Department of Homeland Security (124) having the second and third most.

Agency-Level Protest Resolutions

There was significant variation in sustainment rates between agencies, with the Department of Health and Human Services having a sustainment rate of 43% versus 0% at the Department of Interior. Notably, several large agencies did not have a single sustained protest in FY 2022. Within the Department of Defense, there was a meaningful difference between the branches. Although the Army had the most protests of the branches, it had the lowest sustainment rate of only 5%. The Air Force had the highest sustainment rate of 20%.

Sustainment Rates by Agency

High sustainment rates in isolation do not necessarily signal that an agency has a procurement deficiency, as a higher sustainment rate might instead be a result of an agency being less willing to provide any sort of relief before reaching the point of a GAO decision. However, if an agency has both high merit rates and high sustainment rates, it may signal deficiencies in procurement processes, and a greater likelihood of a successful outcome for the contractor making the protest.

Merit and Sustainment Rates by Agency

How Often do Small Businesses Prevail in the Protest System?

Looking at protest rates by contractor size and economic set-aside, small businesses (which we have defined as any entity with less than $40 million in disclosed federal prime contract and subcontract obligations in the prior year) account for the majority of protests. The 12.6% sustainment rate for small businesses is modestly lower than for large businesses at 14.5%. Breaking down contractor size further shows that contractors with more than $200 million in obligations have a notably higher sustainment rate of 16.7%.

Sustainment Rates by Size and Socioeconomic Program

Service-Disabled Veteran-Owned Small Businesses (SDVOSBs) have markedly lower sustainment rates than the average of 13%. Reviewing protest decisions for SDVOSBs, there is no clear theme for the greater denials. Instead the low sustainment rate for SDVOSBs appears to be an artifact of the approximately 60% of SDVOSB contract dollars being awarded at the VA, and the VA having a very low sustainment rate generally. Women-Owned Small Businesses (WOSBs) and contractors in the 8(a) Business Development program have sustainment rates well above the average at 19.2% and 15.0%, respectively.

Does the Attorney Assigned to a Protest Matter?

Perhaps one of the most meaningful and surprising factors in whether a protest will be determined to have merit and ultimately be sustained is the GAO attorney assigned to the protest. The table below summarizes the merit and sustainment rates of all GAO attorneys with more than 40 cases during FY 2022. Merit rates varied from 13% to 47% and sustainment rates varied from 0% to 48% depending on the GAO attorney assigned.

Merit and Sustainment Rates by GAO Attorney

Unfortunately, it is difficult to tell from the available data to what extent the wide variance is due to an Attorney's general willingness to sustain protests or their use of informal means to find relief for the protestor or encouragement of Alternative Dispute Resolution (ADR) to avoid cases reaching the stage of a merit decision. While GAO attorney names assigned to protests are disclosed, we have redacted them here.

Are Serial Protestors a Problem?

In the past, both contractors and agencies have criticized serial protestors abusing the protest system by filing frivolous protests just to delay awards. This was most dramatically exemplified by a two year ban on filing protests received by the contractor Latvian Connection in 2017 after it filed more than 500 protests over several years.

However, the potential harm to market reputation from protesting too often appears to have reduced the number of serial protestors. In FY 2022, only 5 contractors filed more than 10 protests and none filed more than 16. The 69 protests in this group represented only 29 different procurements as many of the protests were supplemental protests or represented one of multiple protests against the same procurement.

Do Supplemental Protests Increase the Odds of Sustainment?

Approximately 40% of protests decided in FY 2022 had supplemental protests filed. Contractors that filed supplemental protests were approximately 16% more likely to have their protest sustained relative to those that did not. Supplemental protests were also more likely to receive a final merits decision. The higher sustainment rate associated with supplemental requests is likely driven by the contractor making positive adjustments to their case after receiving the agency response and evaluation record.

Why Are Protests Sustained?

The GAO Report notes that the three primary reasons that protests are sustained are:

1. Unreasonable Technical Evaluation
2. Flawed Selection Decision
3. Flawed Solicitation

Our manual review of protest decisions generally agrees with the above. We found that reason #1 and #2 are far more prevalent than reason #3, and sustained decisions often cite some combination of reasons #1 and #2.

Notes On Methodology and Analysis:

Definition of Protest. To align with the methodology used in GAO’s Report to Congress our analysis counts a protest as a docket entry (or “B” number). In many cases a contractor’s protest of a procurement may have several dockets, and accordingly, this analysis (and GAO's) overcounts the total number of protested procurements. We also examined the data on a procurement level, and overall trends held the same.

Completeness of Data. Docket data was obtained from gao.gov. In some cases the totals for FY 2022 did not match the figures provided in GAO's Report to Congress. In these instances, we have adjusted numbers proportionately to match the overall totals provided by GAO.

Small Business and Set-Aside Classifications. Protestors were mapped to Universal Entity Identifier (UEI) codes by matching the legal name provided in the GAO protest docket to legal names provided in contractor registration data from SAM. UEIs were then mapped to a Parent entity based on HigherGov’s parent/child mappings and total obligations measured using cleansed contract data obtained from FPDS and FSRS. SDVOSB and WOSB statuses are based on self-reported statuses in the contractor’s government registration. 8(a) status was obtained from the SBA's DSBS system.

Sources

HigherGov, GAO, FPDS, SAM, DSBS, FSRS

Author

Justin Siken
Founder of HigherGov
justin.siken@highergov.com
Linkedin
Twitter