LQ3 InfiniBAND CLUSTER

Form / Attachment / Pricing Sheet

This document, Attachment C, is a Cost Proposal worksheet that must be completed and submitted with every bid and for every configuration. It requires a full bill of materials for a single compute server rack, listing all components and their costs.

• A bill of materials for a single compute server rack, including all components (e.g., compute server, network cables, switch, PDU).
• Cost for each listed component.
• Total cost of the LQ3 Component Servers.

Completing this cost proposal accurately is a mandatory requirement for all bidders. The information provided in this worksheet will be considered the bidder's final response regarding costs. Bidders are cautioned that Fermilab is not responsible for any incorrect or inaccurate information submitted in this checklist.

Form / Attachment / Pricing Sheet

This document, "Technical Proposal Worksheet," is a mandatory attachment (Attachment D) that must be completed and submitted with every bid and for every configuration. It serves as a detailed technical specification worksheet for the proposed compute system.

The worksheet requires bidders to provide specific technical details across several categories, including:

• Technical Contacts: Name, phone, and email.
• Motherboard: Brand, model, chipset, memory capacity, DIMM slots, PCIe slots, serial/USB ports, boot capabilities, and BIOS settings replication.
• Onboard Ethernet Adapters: Number, brand, model, speed, and MAC address availability.
• 10 GE Ethernet Adapter (if not onboard): Brand, model, speed, and MAC address availability.
• SSDs: Brand, model, capacities, operating temperature, and power consumption.
• Memory: Total installed, brand/model, speed, DIMM size, ECC support, and operating temperature.
• Chassis: Brand, model, dimensions, weight, fan count, air path, heat removal capacity, access to memory slots, tool-free opening, rail type, and drive port labeling.
• Power Supply: Manufacturer, model, PFC, power rating, power switch, operating temperature, efficiency factors, and redundancy.
• Management Ethernet Switch: Brand, model, port count, and port speed.
• Full Compute System: Max current draw (boot-up and idle), measurement methodology, and voltage.
• Service & Support: Contact information and procedures for hardware service calls and phone support.
• BIOS Changes: Description of any modifications made.
• Burn-In Procedures: Details of post-build testing.
• Proposal Summary: Specific metrics like Vendor, Processor Type, Total Nodes, Nodes/Rack, Power Draw, Processor Speed, Threads/Node, Memory details, Network Interface Card, Power/Rack, PDU details, and Switched PDUs.

Bidders must meticulously fill out all sections of this worksheet. The information provided will be considered the bidder's final response and will be used to evaluate the technical compliance and specifications of their proposed solution. Fermilab explicitly states that they are not responsible for incorrect or inaccurate information provided in this checklist.

Form / Attachment / Pricing Sheet

This document, "Exhibit A," outlines the Environment, Safety, and Health (ES&H) requirements and program expectations for subcontractors working at Fermi National Accelerator Laboratory (Fermilab).

• Points of Contact: Identifies FRA Point of Contact and Service Coordinator roles and responsibilities.
• References: Lists applicable federal regulations (DOE, OSHA, EPA), national standards (NFPA, ASME, ANSI), and state regulations.
• Enforcement: Details potential DOE civil penalties and contract fee reductions for noncompliance with 10 CFR 851 and 10 CFR 835.
• ES&H Program Requirements: Mandates subcontractor ES&H program submission, detailing required elements such as management commitment, responsibility assignment, training, hazard assessment, and waste handling.
• Hazard Analysis (HA): Requires HAs for high-risk activities, including specific hazards, mitigating actions, and PPE.
• Reporting & Training: Outlines emergency response procedures, accident reporting, man-hour reporting, and required subcontractor training (including orientation).
• Work Site Conditions: Specifies requirements for permits, utilities, transportation, traffic, temporary facilities, hazardous materials, confined spaces, utilities, PPE, burning/welding, heating, electrical work, oxygen deficiency hazards, radiation protection, and environmental protection.
• Prohibited Activities: Details restrictions on smoking, drug/alcohol use, open burning, and specific fuel storage tank requirements.
• Vehicles & Equipment: Sets standards for vehicle operation, backup alarms, roll-over protection, personnel lifts, tool maintenance, and equipment inspection.
• Fermilab Closures: Lists major holidays when construction activities are prohibited.
• ES&H Certificate (Attachment A): Requires subcontractors to acknowledge and certify compliance with 10 CFR 851 and provide medical surveillance information.
• High Risk Activities (Attachment B): Lists activities that require a Hazard Analysis.

This document is critical for bidders to understand the extensive safety, health, and environmental compliance obligations required when performing work at Fermilab. It details regulatory requirements, specific procedures, reporting protocols, and potential penalties, which are essential for accurately estimating costs and assessing the feasibility of undertaking the work.

Form / Attachment / Pricing Sheet

This document is a blank ACORD Certificate of Liability Insurance form (ACORD 25). Its purpose is to serve as a standardized template for providing proof of insurance coverage. It explicitly states that it is for informational purposes only and does not amend, extend, or alter coverage.

• Sections for detailing insurance information: Includes fields for the Producer, Insured, and Insurers.
• Coverage types: Specifies sections for Commercial General Liability, Automobile Liability, Umbrella Liability, and Workers Compensation and Employers' Liability.
• Policy details: Fields for Certificate Number, Revision Number, Policy Number, Policy Effective and Expiration Dates, and various coverage limits (e.g., Each Occurrence, Aggregate, Bodily Injury, Property Damage).
• Certificate Holder: Pre-filled with "Fermi Research Alliance LLC Fermi National Accelerator Lab."
• Cancellation clause: Notes that notice will be delivered if policies are cancelled before expiration.

This blank form indicates the type of insurance documentation that may be required for the opportunity. Potential bidders should review the types of coverage and limits detailed in this template to understand the insurance requirements they may need to meet. The pre-filled Certificate Holder section suggests that the successful contractor will likely need to name Fermi Research Alliance LLC as the certificate holder on their own certificate of insurance.

Form / Attachment

This document outlines the mandatory insurance requirements for subcontractors performing onsite non-professional services at the Fermi National Accelerator Laboratory (Fermilab). It details the types and minimum limits of coverage required by Fermi Research Alliance, LLC (FRA).

• Minimum Coverage: Specifies required limits for:
  • General Liability: $1,000,000 per occurrence, $2,000,000 general aggregate, $2,000,000 products-completed operations aggregate (maintained for 10 years).
  • Automobile Liability: $1,000,000 combined single limit for owned, non-owned, and hired automobiles.
  • Excess/Umbrella Liability: $1,000,000 per occurrence aggregate for subcontracts less than $3.5M; $5,000,000 for subcontracts greater than $3.5M.
  • Worker's Compensation: Statutory limits as required by Illinois Law.
  • Employer's Liability: $1,000,000 each accident, $1,000,000 disease each employee, $1,000,000 disease policy limit.
• Supplemental Coverages: May include Aircraft Liability ($25,000,000) and/or Builder's Risk (replacement cost) if checked, with specific conditions (e.g., earthquake, flood, debris removal, installation floater).
• Additional Insureds: All policies (except Worker's Compensation) must name Fermi Research Alliance, LLC, the University of Chicago, Universities Research Association, Inc., and the U.S. Government as additional insureds, with primary coverage and waiver of subrogation.
• Exclusions/Terms Not Allowed: Lists specific exclusions or provisions not permitted in Commercial General Liability policies.
• Evidence of Insurance: Subcontractors must provide certificates of insurance to FRA within ten (10) days of subcontract signing, with 30 days' written notice for cancellation (10 days for non-payment).
• Sub-Subcontractor Requirements: Insurance requirements, including additional insureds, apply to all sub-subcontractors. The Subcontractor is responsible for determining appropriate limits and obtaining evidence of coverage.
• Indemnification: Subcontractor indemnifies FRA for losses due to non-compliance with insurance requirements.

Potential bidders must ensure they can meet these specific insurance requirements and provide proof of coverage to be eligible for award and to perform work at Fermilab. These requirements will impact their cost and operational planning for any onsite non-professional services.

Form / Attachment / Pricing Sheet

This document, titled "Subcontractor Annual Representations & Certifications (SARC)," is a form required by Fermi Forward Discovery Group, LLC for subcontractors. It collects essential business and certification information from potential subcontractors to ensure compliance with U.S. Government contracting and subcontracting programs.

• Offeror Information: Collects details such as Offeror Name, Parent Company Name, address, contact information, EIN/TIN, DUNS #, NAICS U.S. Industry #, and SAM.gov registration status.
• Business Category: Requires representation of ownership (U.S. or Foreign), operating structure (Corporation, LLC, Partnership, etc.), and common parent information.
• Small Business Classifications: Allows the offeror to self-certify as a Large Business, Small Business, or Foreign Business, and to select applicable small business diversity categories (e.g., Small Disadvantaged Business, Woman-Owned Small Business, HUBZone).
• Executive Compensation: Inquires about the percentage of annual gross revenues from Federal financial assistance and whether executive compensation information is publicly accessible.
• Offeror Responsibility: Includes representations regarding accounting system review, responsible offeror status (per FAR 9.104), and compliance with combating trafficking in persons.
• Employment Eligibility Verification: Addresses requirements related to FAR 52.222-54 Employment Eligibility Verification (E-Verify).
• Export/Import Control Certifications: Requires agreement to comply with U.S. export control laws and regulations.
• Federal Acquisition Regulation (FAR) Certifications: Includes representations on limitations on pass-through charges.
• Certification: A final section for a duly authorized representative to certify the accuracy of all statements and acknowledge awareness of government programs.

This form is critical for any entity seeking to subcontract with Fermi Forward Discovery Group, LLC. Bidders must accurately complete and submit this SARC form as part of their proposal or in accordance with instructions provided by FermiForward. Failure to complete it correctly may impact eligibility or the ability to receive an award. It helps FermiForward assess compliance, business status, and responsibility.

Form / Attachment / Pricing Sheet

This document, "Proposal Certifications," is a form that offerors must complete and submit as part of their proposal package. It requires the offeror to make several certifications regarding compliance with various federal regulations and clauses, including those related to export/import control, the Buy American Act, independent price determination, lobbying, responsibility matters, employment reports, anti-kickback procedures, combatting trafficking in persons, and patent rights.

• Certifications #1 (Required for all proposals ≥ $10,000): Covers Export/Import Control, Buy American Act (construction materials), Rights to Proposal Data, Rights in Data-General, and Patent Rights.
• Certifications #2 (Required for all proposals ≥ $150,000): Includes Certificate of Independent Price Determination.
• Certifications #3 (Required for all proposals ≥ $550,000): Addresses Combatting Trafficking in Persons and Employment Reports on Veterans.
• Specific Clauses: Details specific FAR and DEAR clauses that require certification, such as 52.225-9 Buy American – Construction Materials, 52.227-23 Rights to Proposal Data, 52.203-11 Certification and Disclosure Regarding Payments to Influence Certain Federal Transactions, and 52.209-5 Certification Regarding Responsibility Matters.
• Revision Table: Outlines the history of changes to the document, including updates to clauses and organizational name changes.

This document is critical for bidders to understand and complete accurately. Failure to provide the required certifications can impact the evaluation of their proposal and potentially render them non-responsible. Bidders must carefully review each section to ensure they can truthfully certify compliance with the stated requirements, particularly concerning domestic sourcing (Buy American Act), data rights, pricing, and ethical conduct. The document also specifies different certification requirements based on the proposal's dollar value.

Attachment (Technical Specification)

This document, v3_SOW_FY25_LQ3_Attachment_A_Specification.docx, provides the detailed technical specifications for the Fermi Research Alliance (FRA) acquisition of an Infiniband HPC farm (LQ3). It outlines the required hardware for compute servers, Luster storage servers, Infiniband rack components, and system rack components, along with acceptance testing, warranty, and delivery requirements.

• Acquisition Scope: Purchase of an Infiniband HPC farm (LQ3) comprising 52 servers (compute, Luster, admin nodes). Vendors must utilize existing Fermilab racks and perform all integration work on-site.
• Proposal Requirements: Bidders must submit a Technical Proposal, a completed Attachment C (Technical Proposal worksheet), a single rack configuration diagram, power efficiency curves, a completed Attachment D (Cost Proposal worksheet), a completed Attachment E (Vendor Questionnaire), a delivery and server integration plan (including subcontractor info if applicable), and a Statement of Warranty.
• Hardware Specifications: Detailed configurations are provided for:
  • Compute Servers (QTY 36 base + 1-12 options): AMD EPYC Turin 9355, 1.5TB DDR5 RAM, multiple SSDs, NVIDIA ConnectX-7 400Gb/s NDR IB, and specific motherboard, BIOS, power supply, and cooling requirements.
  • Luster OSS Servers (QTY 4): AMD EPYC Turin 9135, 128GB DDR5 RAM, 24x 20TB HDDs, and NVIDIA ConnectX-7 400Gb/s NDR IB.
  • Luster Admin Servers (QTY 4): AMD EPYC Turin 9135, 128GB DDR5 RAM, and NVIDIA ConnectX-7 400Gb/s NDR IB.
  • Luster AMD MDS Servers (QTY 2): AMD EPYC Turin 9135, 192GB DDR5 RAM, multiple SSDs, and NVIDIA ConnectX-7 400Gb/s NDR IB.
  • Infiniband Rack Components: Includes a NVIDIA 64-port switch, all necessary IB cables, and optical splitter cables for connection to LQ2.
  • System Rack Components: Requirements for 47U racks, Ethernet switches, and 30A, 208V horizontal PDUs with integrated metering.
• Acceptance Testing: FRA will conduct a 15-day acceptance test, allowing no more than 5% downtime. A 'Lemon Law' defines conditions for system replacement.
• Warranty & Delivery: A five-year warranty is required for all parts, materials, and labor, commencing after the acceptance test. Specific response times (4-hour acknowledgement, 2-hour callback) and service level agreements are outlined. Vendors are responsible for physical installation, integration, and removal of packing materials, adhering to FRA safety and computing policies.

This document is crucial for bidders to formulate compliant technical and cost proposals. It provides the exact technical specifications for all required hardware, outlines the scope of integration work, details acceptance criteria, and specifies comprehensive warranty and delivery obligations that must be met.

Solicitation / RFP / RFQ

This Request for Quotation (RFQ) from Fermi National Accelerator Laboratory (Fermilab) seeks firm-fixed-price quotations for on-site support and services. The services are to be performed in accordance with the attached Scope of Work (SOW).

• On-site support and services as detailed in the attached "v3_SOW_FY25_LQ3_Attachment_A_Specification".

• The price must be firm fixed-price and include all labor and travel expenses.

• Parts needed for repair or service are not included in the quoted price.

• Exceptions to specifications and extra charges must be clearly indicated.

• Contract type: Firm Fixed Price.

• The work is for a project done under Prime Contract No. DE 89243024CSC000002 with the United States Department of Energy.

• Quotations Close: March 6, 2026, at 2:00 pm EST. Late bid criteria apply.

• Offers must include F.O.B. Fermilab delivery terms and all freight charges.

• Quotations should be submitted electronically to hairston@fnal.gov.

• Evaluation will be based on the Lowest Price Technically Acceptable (LPTA) source selection process.

• Non-cost evaluation factors for technical acceptability include: Conformance with procurement specifications, Delivery (FOB Destination), References, and Warranty of Products and Services.

1. RFQ Form RH-373289 (highlighted section completed)
2. Supplier Annual Representations & Certifications (Rev. 7)
3. PUR-466 Proposal Certs (Rev. 6)
4. Exhibit A – Attachment A (completed and signed)
5. Certificate of Insurance (Required prior to award)

• All questions, clarifications, and communications should be directed to Ryan Hairston at hairston@fnal.gov.
• Bidders should specify any government contract vehicle (e.g., GSA, ICPT) if used.
• The solicitation references FFDG Terms and Conditions for Commercial Products or Commercial Services (Rev 01/2025) or GSA/NASA SEWP Contract#.