The IO500 aims to be a robust and long-living benchmark. Nevertheless, the community recognizes the need to consider modifications occasional modifications such as including new access patterns, removing deprecated access patterns, or any other modifications deemed necessary by the community. Therefore, we have established a process to add further benchmarks, which works as follows:
- A member of the community prepares a (up to) 1-page proposal for the new access pattern to include. This should include a motivation, a rough sketch of the access pattern and justification why the pattern is important. This proposal can then be sent to the community mailing list or the steering board. Deadline: 1 month before the next community meeting — at the moment, these are the birds-of-a-feather sessions at ISC or Supercomputing.
- The steering board will give feedback to the technical quality of the proposal.
- The member is given the opportunity to present the proposal at the next following community IO500 meeting.
- Given there are no technical concerns, the IO500 benchmark will be modified for the next submission period to allow the execution of a benchmark that represents the pattern as an optional benchmarking step. Additionally, the optional field is introduced into subsequent lists and the changes to the benchmark are documented on the webpage.
- The optional pattern is kept for at least two subsequent IO500 lists and community meetings. The results and effectiveness of the new pattern are discussed during the community meetings. As a result, it may be removed, remain optional, or may become mandatory.
The committee can be reached at email@example.com.