Sparksoft Corporation

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Sparksoft Corporation

File: B-420944.2; B-420944.3

Date: December 27, 2022

DIGEST

1. Protest challenging the evaluation of the protester’s oral presentation is denied were the agency considered an evaluation criterion that, although not specifically identified in the solicitation, was reasonably related to the stated criteria. Protest challenging the evaluation of the awardee’s oral presentation is sustained where the agency relied on unsupported assumptions regarding the awardee’s proposed technical approach.

2. Protest challenging the evaluation of the awardee’s corporate experience is sustained where the evaluation relied on unsupported assumptions regarding the awardee’s ability to overcome a lack of relevant experience.

3. Protest challenging the award decision is sustained where the award decision was based in part on conclusions regarding differences between the protester’s and awardee’s proposals under the corporate experience factor that were not consistent with the solicitation’s award criteria and that relied on unsupported assumptions.

Sparksoft Corporation, a small business, of Columbia, Maryland, protests the issuance of a task order to RELI Group, Inc., a small business, of Catonsville, Maryland, by the Department of Health and Human Services, Centers for Medicare and Medicaid Services (CMS). The agency issued the task order under task order request for proposal (TORP) No. RFP-CMS-220988, for the performance of the agency’s Multidimensional Information Data Analytics System (MIDAS) requirement. The protester argues that the agency unreasonably evaluated the protester’s and awardee’s oral presentations and the awardee’s corporate experience, and that the award decision was inconsistent with the solicitation’s award criteria.

We sustain the protest.

BACKGROUND

CMS issued the solicitation on April 25, 2022, seeking proposals to “operate, maintain, and provide development services” for MIDAS, which supports the agency’s Center for Consumer Information and Insurance Oversight (CCIIO). Agency Report (AR), Tab 1, Conformed TORP at 1, 65.[1] The CCIIO “assists millions of Americans in obtaining affordable health insurance and enables more employers to offer cost-effective insurance coverage to their employees.” Id. at 65. MIDAS, in turn, “provides mission-critical functionality that CCIIO requires to implement and enforce statutory requirements for issuers.” Id. MIDAS functions supported by the contractor will include “a data repository and analytics solution to support a cloud-based data platform for capturing, sharing, aggregating, and analyzing health insurance and related information to support improvements in decision making, business practices, and services to states and federal agencies.” Id. The contractor will be required to provide services “to curate and integrate data from multiple data and operational source systems into a single information data platform.” Id. at 64. Sparksoft is the incumbent contractor for the MIDAS requirements. Contracting Officer’s Statement (COS) at 3.

The competition was limited to firms[2] who hold one of the agency’s multiple award IDIQ contracts, known as Strategic Partners Acquisition Readiness Contracts (SPARC), and was conducted under the provisions of Federal Acquisition Regulation subpart 16.5. TORP at 45. The competition was further limited to SPARC contract holders that were either Historically Underutilized Business Zone firms or women-owned small business firms. Id. The TORP anticipated issuance of a task order with fixed-price line items for a 1-year base period and four 1-year options. Id. at 1, 45.

The TORP advised that proposals would be evaluated on the basis of four factors: (1) corporate experience; (2) oral presentation; (3) section 508 compliance[3], which was to be evaluated on an acceptable/unacceptable basis; and (4) price. Id. at 57. For purposes of award, the corporate experience factor was “significantly more important” than the oral presentation factor, and the non-price factors, when combined, were “significantly more important” than price. Id. at 57-58.

CMS received proposals from Sparksoft and RELI by the closing date for receipt of proposals of May 9. AR, Tab 19, Source Selection Decision (SSD) at 2. The agency’s technical evaluation panel (TEP) evaluated the proposals as follows:[4]

	SPARKSOFT	RELI
Corporate Experience	Excellent	Highly Acceptable
Oral Presentation	Highly Acceptable	Excellent
Section 508 Compliance	Acceptable	Acceptable
Evaluated Price	$54,291,890	$51,998,479

 

Id. at 8, 11, 13.

The contracting officer, who was also the source selection authority, concluded that RELI’s proposal merited award, and issued the task order to that firm on July 27. Id. at 6; COS at 2. Sparksoft filed a protest with our Office challenging the award on August 8. On August 25, prior to filing its report responding to the protest, CMS advised our Office that it would take the following corrective action: “1. Re-evaluate the proposals; and 2. Based on re-evaluation of the proposals, issue a new award decision.” Sparksoft Corp., B-420944, Aug. 30, 2022 at 1 (unpublished decision). Based on the agency’s proposed corrective action, we concluded that the protest was rendered academic and therefore dismissed it. Id. at 1‑2.

The agency’s corrective action consisted of reconvening the TEP and reevaluating the proposals. COS at 2. The agency states that the corrective action did not result in changes to the adjectival ratings assigned to Sparksoft’s and RELI’s proposals. Id.; AR, Tab 19, SSD at 6. The contracting officer reviewed the revised evaluation and again found that RELI’s proposal merited award based on a tradeoff between the price and non-price evaluation factors. COS at 2. The contracting officer found that “[a]lthough Sparksoft represents a slightly technically superior proposal than that of RELI, RELI represents a lower price in comparison to the proposal submitted by Sparksoft and ultimately the level of technical superiority of Sparksoft’s proposal does not justify the significant price premium of approximately $2.3 million.” AR, Tab 19, SSD at 31.

The agency issued the task order to RELI on September 20, and provided a written debriefing to Sparksoft on the same day. COS at 2. This protest followed.[5]

DISCUSSION

Sparksoft raises three primary challenges to CMS’s award of the task order to RELI: (1) the agency unreasonably assigned the protester’s proposal a weakness under the oral presentation factor and evaluated RELI’s presentation on an unequal basis; (2) the agency unreasonably evaluated the awardee’s proposal under the corporate experience factor; (3) the award decision unreasonably departed from the solicitation’s award criteria.[6] For the reasons discussed below, we agree with parts of the protester’s first and second arguments, and also agree with the protester’s third argument. We therefore sustain the protest.

The task order competition here was conducted among SPARC contract holders pursuant to the IDIQ task order provisions of FAR subpart 16.5. In reviewing protests of awards in task order competitions we do not reevaluate proposals, but examine the record to determine whether the evaluations and source selection decision are reasonable and consistent with the solicitation’s evaluation criteria and applicable procurement laws and regulations, adequately documented, and treat all offerors or vendors equally. DynCorp Int’l LLC, B‑411465, B‑411465.2, Aug. 4, 2015, 2015 CPD ¶ 228 at 7; Sumaria Sys., Inc.; COLSA Corp., B‑412961, B-412961.2, July 21, 2016, 2016 CPD ¶ 188 at 10. A protester’s disagreement with the agency’s judgment regarding the evaluation of proposals, without more, is not sufficient to establish that the agency acted unreasonably. Imagine One Tech. & Mgmt., Ltd., B‑412860.4, B‑412860.5, Dec. 9, 2016, 2016 CPD ¶ 360 at 4-5.

Oral Presentation Evaluation

Sparksoft challenges CMS’s evaluation of proposals under the oral presentation factor for three reasons: (1) a weakness assigned to the protester’s presentation was unreasonable because it concerned a matter that was outside the scope of the criteria for this factor; (2) the weakness was unreasonable because the protester’s presentation in any event addressed this matter; and (3) the agency evaluated the protester’s and awardee’s oral presentations unequally. For the reasons discussed below, we find no merit to the first two arguments, but agree with the protester that the evaluation of the awardee’s oral presentation was unreasonable and sustain the protest on this basis.

The solicitation required offerors to give an oral presentation that addressed their “ability in technical aptitude, interpretation, and clearly communicating an approach.” TORP at 49. An offeror was required to “walk the technical evaluation panel and Government subject matter experts through its proposed solution and technical approach to Performance Work Statement (PWS), in particular Task C.6.2.5 Performance Stabilization and Optimization and Task C.6.2.1 [No.] 11 database and engineering activities.” Id. The presentation was to last 1.5 hours, “with time for clarifying questions and one challenge question” from the agency. Id. at 50. As relevant here, the TORP stated that offerors “shall address” the following requirements:

a. Describe the technical approach for meeting the PWS requirements in Task C.6.2.5 Performance Stabilization and Optimization, including anticipated risks and the proposed approach for mitigating each risk.

b. Describe the technical approach for database and engineering activities (Task C.6.2.1 [No.] 11) to support the various cloud data engines and databases employed in the AWS [Amazon Web Services] environment.

* * * * *

f. Describe your approach (beginning to end) to recovery from a major system failure--e.g., corruption of data lake.[7]

Id. at 50-51.

CMS assigned Sparksoft’s oral presentation four strengths and one weakness. AR, Tab 8, Oral Presentation Evaluation at 6. The weakness stated that the protester failed to address how its systems to ensure data quality would be used in the event of a system failure:

While the offeror mentions a data quality framework, they do not explain how they will use this framework to monitor data quality or work with source systems when there are data quality issues.

In describing their approach to recovering from a system failure, they did not mention using their data quality framework or discuss how they would ensure that data is not corrupted after re-running their [extract, transform, load (ETL)] code. They stated only that [DELETED] and did not indicate this [DELETED] would ensure data integrity. They stated that their [DELETED].

With the diverse sources of data MIDAS ingests, data quality is a constant problem. The offeror does not demonstrate that they understand the importance of data quality. Although they reference a technical framework to evaluate data quality, it is not clear how they would implement this framework to ensure end users have access to accurate, trustworthy data. Since the offeror did not provide an explanation of how they would handle data that fails their framework’s checks, CMS is not confident they would be able to work with source systems to resolve issues and communicate to end users about issues that may affect their analyses.

Ensuring data quality is an inherent and integral part of successfully implementing the PWS requirements for operating and maintaining a reliable database, integrating with diverse data sources, and providing data analytics support. Data quality issues in the source data and as a result of errant MIDAS code or deployments have resulted in users not trusting the data available in and reports generated from MIDAS. The offeror does not propose a solution that adequately recognizes and addresses this challenge.

There is a need for a solution that includes not just a technical tool but also an understanding of how to use that tool to effectively monitor and resolve issues. The offeror’s proposal did not demonstrate this competency.

Furthermore, their omission of data quality when describing a system failure recovery is a critical shortcoming. While [DELETED] and [DELETED] are valuable, they are not sufficient to ensure data integrity is maintained after a system failure, especially given the complexity of the ETL processes. Duplicate, missing, or otherwise corrupt data are common issues that occur in MIDAS today when jobs are run manually after a failure. Without a plan to address these problems, CMS risks end users unknowingly using invalid data to drive policy and operations.

Id. Based on the assessment of four strengths and the weakness above, the agency assigned the protester’s proposal a rating of highly acceptable for the oral presentation factor. Id.

CMS assigned RELI’s oral presentation six strengths and no weaknesses. AR, Tab 8, Oral Presentation Evaluation at 5. One of the strengths concerned the awardee’s discussion of data quality, noting that “[t]he offeror repeatedly emphasizes data quality and recognizes that MIDAS cannot be used effectively unless end users trust the data.” Id. Based on this evaluation, the agency assigned the awardee’s proposal a rating of highly acceptable for the oral presentation factor. Id.

In the award decision, the contracting officer stated that the weakness assigned to Sparksoft’s proposal represented a “critical shortcoming” in the protester’s proposed approach, and that it was a “major concern as this is a current ongoing challenge with the current MIDAS system.” AR, Tab 19, SSD at 19-21.

Unstated Criterion of Data Quality

Sparksoft argues that the weakness assigned to its proposal regarding data quality was unreasonable because the oral presentation factor did not instruct offerors to address this matter. Comments & Supp. Protest at 22-29. We conclude that while data quality was not specifically mentioned in the oral presentation factor evaluation criteria, this matter was reasonably related to the stated criterion of recovery from a major system failure.

Our Office has explained that although agencies are required to identify all major evaluation factors in a solicitation, they need not specifically identify each and every element an agency considers during an evaluation; rather, an agency may properly take into account matters that are logically encompassed by or related to the stated evaluation criteria. Trailboss Enter., Inc., B‑419209, Dec. 23, 2020, 2020 CPD ¶ 414 at 6; Northrop Grumman Sys. Corp., B‑414312 et al., May 1, 2017, 2017 CPD ¶ 128 at 12. To be reasonable, there must be a clear nexus between the stated and unstated criteria. PAE Nat. Sec. Solutions, LLC, B‑419207.2 et al., May 19, 2021, 2021 CPD ¶ 205 at 3; Raytheon Co., B‑404998, July 25, 2011, 2011 CPD ¶ 232 at 15‑16.

CMS argues that although the oral presentation factor did not specifically direct offerors to address data quality, the solicitation advised offerors that they “shall address” the following in their presentations: “f. Describe your approach (beginning to end) to recovery from a major system failure--e.g., corruption of data lake.” TORP at 50-51. The agency states that recovery from a major systems failure and resulting data corruption depends on the contractor’s ability to ensure that data are reliable. See COS at 5-6. The SSD explained that “[d]uplicate, missing, or otherwise corrupt data are common issues that occur in MIDAS today when jobs are run manually after a failure,” and that “[w]ithout a plan to address these problems, CMS risks end users unknowingly using invalid data to drive policy and operations.” AR, Tab 19, SSD at 12. The agency further states that “data quality issues in the source data and as a result of errant MIDAS code or deployments have resulted in users not trusting the data available in MIDAS and reports generated from MIDAS.” COS at 6.

Even though the oral presentation factor did not direct offerors to address data quality, we agree with the agency that data quality was reasonably encompassed within the TORP’s requirement to address an “approach (beginning to end) to recovery from a major system failure.” TORP at 51. In this regard, the agency reasonably explains that there is a clear nexus between recovery from a data corruption event and the contractor’s approach to ensuring the quality of data following that event. For this reason, we find that the agency reasonably evaluated whether the protester’s oral presentation addressed data quality in the context of recovery from a major system failure and deny this part of the protester’s challenge.

Evaluation of Sparksoft’s Oral Presentation

Sparksoft argues that, even if data quality in connection with recovery from a major system failure was a reasonable matter to consider under the oral presentation factor, the protester discussed this matter in a manner that should have addressed the agency’s concern. We conclude that the agency’s evaluation was reasonable.

As discussed above, the agency found that although the protester’s oral presentation “mentions a data quality framework, they do not explain how they will use this framework to monitor data quality or work with source systems when there are data quality issues.” AR, Tab 8, Oral Presentation Evaluation at 6. With regard to the major system failure criterion, the agency stated: “In describing their approach to recovering from a system failure, they did not mention using their data quality framework or discuss how they would ensure that data is not corrupted after re-running their ETL code.” Id. The agency further found that “[a]s a result, the TEP believes since the offeror did not provide an explanation of how they would handle data that fails their framework’s checks, CMS is not confident they would be able to work with source systems to resolve issues and communicate to end users about issues that may affect their analyses.” Id.

Sparksoft argues that its proposal addressed its data quality framework in depth and in a manner that should have led the agency to understand how the protester would address data quality during recovery from a major system failure. Protest at 23-24. In this regard, the protester contends that its description of its data quality framework explained that “data checks are conducted at ingestion and users are automatically notified of any issues,” and that “this process would work the same after the re-run of the ETL code [following a major system failure] as it does in ordinary conditions.” Id. at 24. For these reasons, the protester argues that “there was no need for Sparksoft to devote its limited time to discussing its data quality framework specifically with respect to its approach to recovery from a major system failure, because its data quality framework ensures data quality in the same way during recovery from a major system failure as it does during normal operating conditions.” Id.

CMS states that the information provided by the protester regarding its data quality framework was not presented in a manner that specifically addressed its approach to recovery from a major system failure, and that the agency therefore could not find that the protester’s general descriptions of its data quality framework addressed the agency’s concern. In this regard, the contracting officer states that the agency could not make assumptions in an offeror’s favor regarding its proposed approach: “It was not CMS’s responsibility to fill in the gap left out by the offerors; rather, it was CMS’s responsibility to evaluate the proposals based on the information that was actually included in the presentation and without making guesses or assumptions.” COS at 7. The contracting officer further explains that “there is a need for a solution that includes not just a technical tool but also an understanding of how to use that tool to effectively monitor and resolve issues,” and that the protester’s proposed approach failed to address in detail “how” it would use the tools it identified in the context of recovery from a major system failure. COS at 6.

We think the agency reasonably concluded that the protester’s oral presentation failed to directly address this matter, and that this failure constituted a weakness. The protester does not dispute that its presentation did not address data quality in the specific context of recovery from a major system failure, and the agency reasonably explains that it did not make inferences or assumptions regarding other parts of the protester’s proposal to fill in the gaps in its presentation. In this regard, it is an offeror’s responsibility to submit a well-written proposal, with adequately detailed information that clearly demonstrates compliance with the solicitation and allows a meaningful review by the procuring agency. Jacobs Tech., Inc., B‑411784, B‑411784.2, Oct. 21, 2015, 2015 CPD ¶ 342 at 8. We therefore deny this part of the protester’s challenge.

However, as discussed next, we agree with the protester that the agency’s position that offerors were required to provide information that directly addressed their proposed approaches, without the benefit of “guesses or assumptions” on the part of the agency, shows that the agency treated the offerors unequally.

Evaluation of RELI’s Oral Presentation

Sparksoft argues that CMS’s evaluation of RELI’s oral presentation reflects unequal treatment of the offerors because the agency applied a higher level of scrutiny in evaluating Sparksoft’s oral presentation regarding data quality as compared to the awardee’s oral presentation. We agree with the protester with regard to the strength assigned to RELI’s oral presentation concerning data quality, to the extent that the agency’s evaluation of the awardee’s oral presentation made unsupported assumptions regarding its proposed approach.

As discussed above, CMS found that Sparksoft’s oral presentation merited a weakness because it did not provide an adequately detailed approach to addressing data quality in the context of a recovery from a major system failure. In this regard, the contracting officer explains that offerors were expected to provide a “detailed and specific plan to address the data quality issues.” COS at 6.

The agency’s discussion of RELI’s oral presentation found that the awardee’s proposal merited a strength because it “repeatedly emphasizes data quality and recognizes that MIDAS cannot be used effectively unless end users trust the data.” AR, Tab 8, Oral Presentation Evaluation at 5. One aspect of the assigned strength concerned the awardee’s discussion of its approach to using “ETL data quality validations . . . in the event of a system failure to ensure that data is not corrupt after a recovery,” which cited [DELETED][8] as an example of “how data quality validations can be implemented.” Id.

The agency explained that “[DELETED],” and further noted the agency’s view that the [DELETED] approach “is considered a ‘gold standard’ within the industry that MIDAS should strive to model itself after.” Id. The agency concluded that the awardee’s reference to [DELETED] “means that their solution likely resembles the [DELETED] setup and would improve the usefulness of MIDAS data.” Id. The SSD found that the awardee’s proposed approach provided strengths in the area of data quality. AR, Tab 19, SSD at 11.

Sparksoft argues that CMS unreasonably assumed that RELI’s reference to [DELETED] supported an inference that the awardee’s technical approach “likely resembles the [DELETED] setup.” In this regard, the protester argues that the following portion of the awardee’s oral presentation proposal did not specifically state that its proposed solution is based on the same approach as [DELETED] in the same manner assumed by the agency:

We will ensure any corrupted data doesn’t get processed within our pipelines by [DELETED]. [DELETED]. . . . [DELETED]currently follows this approach [DELETED].

AR Tab 4.3, RELI Oral Presentation at 48:00-48:20.[9]

In response to the protest, the contracting officer states that “contrary to Sparksoft’s allegation, CMS did not ‘assume’ that RELI’s proposal included a state-of-the-art data quality framework similar to that of the [DELETED].” Supp. COS at 2. Rather, the contracting officer states that information provided in the awardee’s oral presentation specifically supported that conclusion: “Based on the information provided by RELI, the TEP and I reasonably evaluated and concluded that RELI’s description of their approach would model their solution after [DELETED].” Id.

We agree with the protester that the reference in the awardee’s oral presentation concerning [DELETED] does not specifically state that the awardee’s approach is “modeled” after [DELETED], or that it “resembles [DELETED].” We think the agency’s evaluation here assumes, without any specific support or explanation, that the reference to [DELETED] meant that RELI would model its approach on [DELETED] in a manner beyond the limited reference within the awardee’s oral presentation. We further note that the agency ascribed significant merit to [DELETED], stating that it is a “gold standard” towards which MIDAS should strive as an example, and that [DELETED] provides “[DELETED].” AR, Tab 8, Oral Presentation Evaluation at 5.

As discussed above, the contracting officer states that “[i]t was not CMS’s responsibility to fill in the gap left out by the offerors[,]” and that the agency evaluated Sparksoft’s proposal based on “information that was actually included in the presentation and without making guesses or assumptions.” COS at 7. Because the agency assumed that the awardee’s approach would be modeled on specific aspects of [DELETED] and provide specific benefits beyond what was specifically stated in the awardee’s oral presentation, we conclude that the evaluation was unreasonable. We further conclude that the agency’s evaluation treated offerors unequally by relying on its assumptions to assign a strength to the awardee’s proposal, despite stating that it could not rely on assumptions when it assigned a weakness to the protester’s proposal. We therefore sustain the protest on this basis.

Corporate Experience Evaluation

Sparksoft argues that CMS improperly found under the corporate experience factor that RELI’s proposal demonstrated experience with the solicitation’s data analytics requirements. Comments & Supp. Protest at 34-37. The protester also argues that although the agency assigned two weaknesses to the awardee’s proposal, the agency unreasonably minimized the importance of these weaknesses based on unsupported assumptions. For the reasons discussed below, we find no merit to the protester’s first argument, but agree with the protester’s second argument and sustain the protest on this basis.

The solicitation stated that the agency would evaluate offerors’ corporate experience as follows:

The Government will assess the depth and breadth in performing work similar to that envisioned by the government in pursuit of the intent described in this solicitation. Considerations may include specific technologies, architectures, stakeholders (e.g., clients, users, etc), tools, methods or anything else the Government considers relevant within this context. The Government may consider the source of the experience (e.g., prime or subcontractor) and determine if that past experience meets the requirements documented in this solicitation.

TORP at 58. The solicitation further provided that the agency would review experience within the past 3 years, and required offerors to “specifically present information on data analytics as it relates to Data Analytics Support to include designing, developing, testing, implementing, and maintaining data products, tools and associated processes, methods, and documentation.” Id. The solicitation also advised that information addressing experience could be provided regarding “a prime contractor, first tier subcontractor, or majority contributor in a teaming arrangement” that will “have meaningful involvement in contract performance.” Id. at 48; see also id. at 58.

RELI’s proposal identified three corporate experience references: (1) RELI’s National Plan and Provider Enumeration System (NPPES) contract with CMS; (2) subcontractor [DELETED]’s [DELETED] contract with CMS; and subcontractor [DELETED] contract with CMS. AR, Tab 4.1, RELI Corporate Experience Proposal at 5-12. CMS’s evaluation of RELI’s corporate experience identified 10 strengths and two weaknesses. AR, Tab 5, TEP Evaluation Report at 7-9; Tab 19, SSD at 10-11. Based on these findings, the agency assigned the awardee’s proposal an adjectival rating of highly acceptable for the corporate experience factor. Id.

Corporate Experience with Data Analytics

Sparksoft contends that CMS should have assigned a lower rating to RELI’s proposal because it did not address corporate experience with data analytics, as required by the solicitation. Comments & Supp. Protest at 35. In this regard, the protester argues that the descriptions of the three corporate experience examples performed by RELI and its proposed subcontractors did not describe the work performed and data tools used as “data analytics,” and therefore could not have been found by the agency to be relevant to the TORP requirements.

CMS and RELI each argue that the protester’s challenge relies on a narrow search of the awardee’s proposal for the term “data analytics,” which ignores numerous references to work that addresses the solicitation’s requirement for “designing, developing, testing, implementing, and maintaining data products, tools and associated processes, methods, and documentation.” See TORP at 271; COS at 8-9; Supp. COS at 4-5. For example, RELI’s proposal stated that through its work on the NPPES contract the firm “[DELETED],” and explained that “NPPES data are shared widely throughout CMS’ operational data ecosystem, such as with [the Center for Program Integrity] [application programming interface] Hub, Master Data Management, Integrated Repository, and others.” AR, Tab 4.1, RELI Corporate Experience Proposal at 6.

As another example, the description of [DELETED]’s work on the [DELETED] contract for CMS stated that the firm “identified the need for [DELETED],” “[DELETED]” thereby “[DELETED].” Id. at 8. On this record, we agree with the agency and intervenor that the protester’s arguments are based on overly-narrow searches for the term “data analytics,” and ignore the actual descriptions of the work set forth in the awardee’s proposal and cited in the agency’s evaluation.

Sparksoft also contends that the agency improperly found that RELI’s proposed subcontractor [DELETED] demonstrated corporate experience in the area of data analytics because the subcontractor was not proposed to perform this work. Comments & Supp. Protest at 36-37. The protester contends that the terms of the TORP precluded consideration of this data analytics experience because [DELETED] will not “have meaningful involvement in contract performance” for this requirement. See TORP at 261.

Here again, we agree with the agency and intervenor that although the awardee’s proposal did not specifically use the term data analytics in describing [DELETED]’s anticipated role, the proposal stated that the subcontractor will, among other things “[DELETED].” AR, Tab 4.1, RELI Corporate Experience Proposal at 12. On this record, we find that the agency reasonably concluded that [DELETED] would perform data analytics work on the contract and properly credited [DELETED] with its corporate experience in this area. Accordingly, the protester’s arguments do not provide a basis to sustain the protest.

Importance of Two Assigned Weaknesses

Next, Sparksoft argues that the award decision improperly minimized or discounted the importance of the two weaknesses assigned to RELI’s proposal under the corporate experience factor. Comments & Supp. Protest at 15-17. As relevant to these weaknesses, the PWS identified two software tools required for performance of the MIDAS task order, Databricks and SAS Viya. The PWS stated that the contractor must provide “access to business intelligence [BI] analytic tools such as Databricks to the MIDAS data user community,” and that “Databricks, an analytics and data engineering platform toolset, is the backbone of the system and gives users the ‘single view of the truth.’” TORP, PWS at 66. The PWS stated that SAS Viya will be used to “[p]erform advanced analytics on MIDAS and other system data,” and that the contractor will be required to “complete SAS Viya libraries and connection setup,” and “support the SAS Viya – Onboarding User, User support, SAS space, and Redshift workspace setup.” Id. at 74.

The first weakness assigned by the agency concerned the awardee’s lack of experience with the Databricks software tool, as follows:

While experience [No.] 2 lists Databricks in a list of tools used in the [DELETED] program, and pg. 4 of the proposal describing their proposed team describes [DELETED] having extensive experience providing full lifecycle Databricks support, including [DELETED]’s [DELETED] Spark/Cloud Engineers working on the Databricks platform support a wide range of projects, the vendor did not provide sufficient detail regarding experience using Databricks within one of the three identified experiences.

Databricks is a key tool for MIDAS. It is used for both ETL processes as well as for providing end-users an environment to perform analyses. Without demonstrated experience in administering a Databricks environment, there is a risk that the vendors will not know the nuances required for effective implementation or be able to adequately support users when they run into issues.

AR, Tab 5, Corporate Experience Evaluation at 9.

The second weakness assigned by the agency concerned the awardee’s lack of experience with the SAS Viya software tool, as follows:

While the [offeror] states that they would administer a SAS environment, the proposal does not include examples of experience using SAS, including detail to demonstrate experience administering a SAS environment or providing SAS as a tool for end users.

SAS is a unique tool that differs substantially from many other databases and BI tools. It has unique file types and performance considerations. As such, there are usually specialized developers required to work on SAS tasks which could require a learning curve.

Without the demonstrated experience, the offeror would need to learn the tool, which could . . . slow down the transition. It also places CMS at risk for insufficient support and system downtime if the [offeror] is not able to learn the new tool prior to the end of the transition out period for the current contract. Because the MIDAS system expects to move to the SAS VIYA environment in the near future without experienced administrators CMS could incur risk to the program.

Id. (emphasis added).

Despite these weaknesses, the TEP found that the awardee’s lack of corporate experience regarding the two tools was “correctable,” as follows:

The Offeror’s proposal did not provide enough detail to demonstrate experience administering a SAS environment and providing SAS as a tool for end users, or using Databricks. However, these areas should be easily correctable given their experience administering comparable tools, such as large data warehouse tools (Oracle and Snowflake) and BI tools (Power BI and Tableau), which provides them with transferable knowledge.

Id. at 7 (emphasis added).

The contracting officer similarly concluded in the award decision that the weaknesses assigned to RELI’s proposal concerning its lack of experience with Databricks and SAS Viya are “easily mitigated” based on the awardee’s experience with other tools. AR, Tab 19, SSD at 10. Based on this conclusion, the contracting officer found that, with regard to the corporate experience factor, “the advantage that Sparksoft holds over RELI is marginal given the numerous strengths of each offeror and that the weaknesses identified for RELI are minor, easily mitigated . . ., and are not critical for the overall success of the project.” Id. at 18.

Sparksoft argues that the agency does not reasonably explain why RELI will be able to “easily” overcome the lack of experience with Databricks and SAS Viya. In particular, the protester contends that while the agency found that SAS Viya is a “unique tool that differs substantially from many other databases and BI tools,” that has “unique file types and performance considerations” that usually require “specialized developers,” the agency nonetheless concluded the RELI would be able to “easily” overcome this lack of experience “given their experience administering comparable tools.” AR, Tab 5, Corporate Experience Evaluation at 7; see Tab 19, SSD at 10-11.

In response to the protest, the agency simply reiterates its view that RELI could overcome its lack of experience during performance: “It was determined that while RELI’s lack of experience with SAS or Databricks could slow down transition or incur risk, such weakness could also be easily mitigated during the transition time based on the fact that RELI had transferable skills and experience.” Supp. COS at 4.

On this record, we find that while the TEP and contracting officer recognized risk based on RELI’s lack of experience with the Databricks and SAS Viya, neither the TEP report nor the SSD reasonably explain why the agency believes the awardee will be able to “easily” overcome these risks. In particular, the agency does not reasonably explain why, in light of the “unique” nature of SAS Viya, the awardee’s experience with tools that the agency states “differ[] substantially” from SAS Viya will allow the awardee to overcome its lack of experience with this tool. AR, Tab 5, Corporate Experience Evaluation at 9. Additionally, the agency does not cite any aspects of the awardee’s proposal for the proposition that it will leverage its experience with other systems to overcome its lack of experience with the tools cited by the agency.

As discussed with regard to the evaluation of Sparksoft’s oral presentation, the contracting officer states that “[i]t was not CMS’s responsibility to fill in the gap left out by the offerors; rather, it was CMS’s responsibility to evaluate the proposals based on the information that was actually included in the presentation and without making guesses or assumptions.” COS at 7. On this record, we conclude that the agency’s findings regarding the awardee’s ability to “easily” overcome these assigned risks was not adequately supported. Here also, we conclude that the agency’s evaluation treated offerors unequally by relying on its assumptions to assign a strength to the awardee’s proposal, despite stating that it could not rely on assumptions when it assigned a weakness to the protester’s proposal. We therefore sustain the protest on this basis.

Award Decision

Sparksoft argues that the award decision was flawed for two primary reasons: (1) the decision relied on the flawed corporate experience and oral presentation evaluations; and (2) the decision departed from the solicitation’s award criteria by improperly diminishing the importance of corporate experience. For the reasons discussed above, we agree that the evaluation of the protester’s and awardee’s proposals under the corporate experience and oral presentation factors was unreasonable, and that the flawed evaluation necessarily had an effect on the award decision. We also agree with the protester with regard to its second argument that the agency improperly diminished the importance of corporate experience in the award decision, and therefore sustain the protest.

Generally, in a negotiated procurement--including when such procedures are used in task order competitions under IDIQ contracts--an agency may properly select a lower-rated, lower-priced proposal where it reasonably concludes that the price premium involved in selecting a higher-rated proposal is not justified in light of the acceptable level of technical competence available at a lower price. OGSystems, LLC, B‑417026.5, B‑417026.6, July 16, 2019, 2019 CPD ¶ 273 at 12; NOVA Corp., B‑408046, B‑408046.2, June 4, 2013, 2013 CPD ¶ 127 at 5‑6. While an agency has broad discretion in making a tradeoff between price and non-price factors, an award decision in favor of a lower-rated, lower-priced proposal must acknowledge and document any significant advantages of the higher-priced, higher-rated proposal, and explain why they are not worth the price premium. Id. An agency that fails to adequately document its source selection decision bears the risk that our Office may be unable to determine whether the decision was proper. Alpha Omega Integration, LLC, B-419812, B-419812.2, Aug. 10, 2021, 2021 CPD ¶ 302 at 7. A protester’s disagreement with the agency’s judgment, without more, does not establish that the award decision was unreasonable. Id.

Here, the contracting officer concluded that although Sparksoft’s proposal was more highly rated under the most important non-price factor, the benefits of the protester’s proposal did not justify its higher price of $2.3 million, or 4.2 percent:

Based upon this comparison of the proposals and a detailed assessment of the advantages, disadvantages, and benefits associated with each, I have determined that the proposal from RELI Group represents the best value to the Government. Although Sparksoft represents a slightly technically superior proposal than that of RELI, RELI represents a lower price in comparison to the proposal submitted by Sparksoft and ultimately the level of technical superiority of Sparksoft’s proposal does not justify the significant price premium of approximately $2.3 million.

AR, Tab 19, SSD at 31.

First, as discussed above, we conclude that CMS’s evaluation of Sparksoft’s and RELI’s proposals under the corporate experience and oral presentation factors was unreasonable. In particular, the award decision states, as noted above, that “the advantage that Sparksoft holds over RELI [under the corporate experience factor] is marginal given the numerous strengths of each offeror and that the weaknesses identified for RELI are minor, easily mitigated . . ., and are not critical for the overall success of the project.” AR, Tab 19, SSD at 18. Because we find that the agency’s evaluation under the corporate experience and oral presentation factors was unreasonable, we necessarily conclude that the award decision based on that evaluation was not reasonable. See AT&T Gov’t Solutions, Inc., B‑413012, B-413012.2, July 28, 2016, 2016 CPD ¶ 237 at 28 (protest sustained where unreasonable evaluation of proposals rendered the award decision unreasonable). We therefore sustain this aspect of the protest.

Next, Sparksoft argues that the award decision departed from the award criteria because, although corporate experience was the most important evaluation factor, the agency improperly assumed that the benefits of corporate experience were “short-lived” in a manner that diminished the value of the protester’s advantage under that factor. Comments & Supp. Protest at 19-21.

The contracting officer characterized the advantage of the protester’s proposal over the awardee’s proposal under the corporate experience factor as “slight,” and noted that the advantage arose from Sparksoft’s performance of the incumbent task order:

Sparksoft’s advantage comes from having direct corporate experience since they are the incumbent on the current work and therefore having extensive experience working with Marketplace data; supporting MIDAS users through training and coaching; having experience setting up, developing, and operating a Databricks environment for the current MIDAS effort; and experience with SAS Viya as identified by strengths in the technical evaluation for Phase I.

AR, Tab 19, SSD at 20.

The contracting officer stated that “based on the emphasis on Factor 1 [corporate experience], Sparksoft is slightly technically superior to RELI Group.” Id. at 21. The contracting officer explained that “Sparksoft has strengths that the RELI Group does not possess and Sparksoft has no identified weaknesses for Factor 1 and there is value in that.” Id. Despite this advantage, the contracting officer found that the benefits of the protester’s proposal were not worth its higher price as compared to RELI’s proposal:

The level of technical superiority of Sparksoft’s proposal does not justify the significant price premium of approximately $2.3 million price higher when compared to RELI Group’s proposal, which was assigned a Highly Acceptable rating under Factor 1-Corporate Experience and offers only slightly less technical merit when comparing the relative merit of all findings.

Id.

In support of this conclusion, the contracting office explained that “[b]enefits of experience tend to be short-lived as new contractors undergo normal learning curves, such that while corporate experience is the most important factor, small variations in quality between two vendors minimize the effect of this variation.” Id. (emphasis added).

Sparksoft argues that the contracting officer’s statement that the benefits of experience are “short-lived” is inconsistent with the solicitation’s instruction that the corporate experience factor was “significantly more important” than the oral presentation factor, and that the non-price factors were “significantly more important” than price. See TORP at 57-58.

In a similar instance, our Office found that an agency’s award decision improperly assumed that advantages in the protester’s proposal concerning its experience were limited because they stemmed from its role as the incumbent contractor. OGSystems, LLC, B-417026 et al., Jan. 22, 2019, 2019 CPD ¶ 66 at 16-17. In this regard, the SSA found that the benefits of the protester’s corporate experience would be limited to the “early stages of the contract” because the awardee would be able to provide similar levels of benefits and performance after its staff gained experience working on the contract. Id. at 16. We found that the record did not support the SSA’s conclusions that the benefits provided by the protester’s corporate experience would not endure for the duration of the contract, or that the awardee would be able achieve the same level of performance as the protester, as reflected by its experience.

Here, CMS does not explain why the “[b]enefits of experience tend to be short-lived as new contractors undergo normal learning curves,” apart from generally arguing that the award decision gave proper weight to the corporate experience factor. See Comments and Supp. Protest at 20. We note that the contracting officer’s conclusion that the benefits of experience “tend to be short-lived” is similar to the contracting officer’s conclusion that RELI will be able to “easily” overcome a lack of experience with the Databricks and SAS Viya tools, in that both conclusions assume that new contractors will experience a “learning curve” during performance that will enable them to overcome shortcomings in their proposals.

We find that neither the contemporaneous record, nor the agency’s response to the protest, adequately explain the contracting officer’s assumptions in this regard. Here again, we note that the contracting officer’s response to Sparksoft’s arguments concerning the evaluation of its oral presentation stated that “[i]t was not CMS’s responsibility to fill in the gap left out by the offerors; rather, it was CMS’s responsibility to evaluate the proposals based on the information that was actually included in the presentation and without making guesses or assumptions.” COS at 7. We therefore conclude that the source selection decision document does not reasonably support the award to RELI, and sustain the protest for this reason.

CONCLUSION AND RECOMMENDATION

We conclude that CMS’s evaluation of RELI’s proposal was unreasonable with regard to the oral presentation and corporate experience factors. We also conclude that the agency’s award decision was unreasonable because it relied on the flawed evaluations of the proposals under the oral presentation and corporate experience factors, and because the agency’s award decision was not consistent with the solicitation’s award criteria regarding corporate experience and relied on unsupported assumptions. We further find that Sparksoft was prejudiced by these errors because the protester’s proposal was more highly rated than the RELI’s proposal under the more heavily-weighted non-price factors, and because the flawed evaluations supported the agency’s decision to award the task order to RELI based on its lower-priced, lower technically rated proposal. See AT&T Gov’t Solutions, Inc., supra, at 5 (competitive prejudice is an essential element of a viable protest, as we will sustain a protest only where the protester demonstrates that, but for the agency’s improper actions, it would have had a substantial chance of receiving the award).

We recommend that the agency reevaluate RELI’s proposal consistent with the discussion above, and in a manner that ensures that both the protester’s and awardee’s proposals are evaluated on an equal basis. If the agency finds that equal treatment of the offerors requires a reevaluation of Sparksoft’s proposal as well, the agency should conduct such a reevaluation. We also recommend that the agency make a new award decision that ensures that the comparison of the offerors’ corporate experience is consistent with the solicitation’s award criteria, adequately supported, and adequately documented. To the extent that the agency believes that the benefits of corporate experience are “short-lived” in a manner that is inconsistent with the solicitation’s award criteria, the agency should amend the solicitation to ensure that offerors are advised of the agency’s views regarding the benefits of corporate experience, and solicit revised proposals.

We also recommend that the agency reimburse the protester’s reasonable costs associated with filing and pursuing the protest, including attorneys’ fees. 4 C.F.R. § 21.8(d). The protester’s certified claims for costs, detailing the time expended and costs incurred, must be submitted to the agency within 60 days after the receipt of this decision. Id. § 21.8(f).

The protest is sustained.

Edda Emmanuelli Perez
General Counsel

 

---