Run IO500

You can download IO500 from the GitHub repository. Follow the instructions in the GitHub README file to install, configure, and run the benchmark!

How to Submit?

This page contains the information about the submission procedure. First, you need to run the benchmark .

The IO500 lists are released during ISC and SC each year. Submissions to the upcoming lists can be made all year. However, to be included in the next submission, we must receive the submission before the deadline listed above, unless you have previously contacted the committee and approved an extension for exceptional circumstances.

Production and Research Lists

Starting with ISC'23, we implemented a separation of the IO500 list into Production and Research lists. This better reflects the important distinction between storage systems that run in production environments and those that may use more experimental hardware and software configurations. Please see the requirements for each list and the IO500 submission rules page.

Reproducibility Questionnaire

Every submission will now receive a reproducibility score based upon the provided system details and the reproducibility questionnaire. This score will inform the community on the amount of details provided in the submission and the obtainability of the storage system. Further, this score will be used to evaluate if a submission is eligible for the Production list.

Submission Instructions

There are two options to submit to the IO500, we strongly prefer the online form:

  1. Use our online form. Users can create accounts and then update and manage all of their submissions through that account.
    1. Click on "[Register]" to create a new account
    2. You will receive an email to activate your account (also check your spam folder)
    3. Once your account is activated you can [Login]
    4. Go to the "[New Submission]" menu
    5. Complete the submission form with all required information
      • You will be able to incrementally complete the submission (i.e., your progress will be saved)
      • You are able to edit your submission until you submit it
      • Once you double-check your submission data, click on "[Submit]" to send it to the IO500 committee for review
    The form is designed to handle submissions from a wide range of storage system types, each with different numbers of hardware components. For reproducibility and analysis reasons, within each component, the easily obtainable fields are mandatory, but it is up to the submitter to select the right components. The key components are:
    • Site - High Level details of the submitting institution
    • IO500 - Summary of storage client and server information utilized in benchmark execution
    • Storage System - General storage system information and which list is targeted (Production or Research)
    • Specific Storage System Component (e.g., Lustre, DAOS, SpectrumScale) - Detailed per-file system information
      • Depending on the Storage System, there is a series of sub-components to fill out that provide the full details of the server, network, and storage of each server in the storage system.
    • Supercomputer - Detailed information regarding the client nodes utilized in benchmark execution
  2. If you experience problems with the online form, please reach out on Slack to see if this can be resolved quickly. Alternately, send an email with the following as attachments:
    • The (potentially) adapted io500.sh
    • The output directory of the benchmark (variable io500_result_dir in io500.sh).
    • If possible, please mention which system is covered of the CDCL or provide system information such that we can add the system to the CDCL!

    We will reply to you, to confirm reception and to ask any question that may arise.

Handling of the Submitted Data

Until the next release of the list, the submission committee will handle all submitted data confidentially. That means that we will not disclose any submitted data to individuals/companies, or institutions.

Privacy

We will handle all data submitted as per our data handling policy. The key point is that by submitting you and your organization give the IO500 Committee the right to publish all uploaded data (except for contact information).

Submitter Name

Submissions will be visible immediately to the members of the IO500 Steering Committee. If there is sensitivity about early visibility to your results being seen by any of the committee, please feel free to email results privately to a subset of the committee (i.e. do not use the official submission tools if you have privacy concerns).

Starting with the SC'18 list, submissions include the name of the submitter (or team) to give them the credit they deserve to execute the benchmark; this can be opted out.

Annonymity

With the online form, the submitter is able to individually opt-out the submission the name of the submitter/team (this will be then an anonymous submission).

Previous Calls for Submissions