The International Symposium on Computer Architecture (ISCA) will conduct artifact evaluation (AE) this year. AE has become a common practice in the systems community (OSDI, PLDI, PACT, MLSys), and has recently been successfully introduced to the architecture community, with ASPLOS conducting AE in the last four years, and MICRO doing so as well since 2021. We invite the authors of accepted ISCA 2023 papers to submit their artifacts to be assessed based on the ACM Artifact Review and Badging policy. Note that this submission is voluntary and will not influence the final decision regarding the papers.

Upload Artifact Submission to HotCRP

Important Dates

  • Paper Decision Notification: March 9, 2023
  • Intent to Submit: March 13, 2023
  • Artifact Submission: March 20, 2023
  • Artifact Decision: April 8, 2023

Process

The authors of accepted papers at ISCA 2023 will be invited to submit their artifacts according to the established submission guidelines. Submission will be then reviewed according to the reviewing guidelines. Papers that successfully go through AE will receive a set of ACM badges of approval printed on the papers themselves and available as meta information in the ACM Digital Library (it is now possible to search for papers with specific badges in ACM DL). Authors of such papers will have an option to include a two-page-max artifact appendix to their camera-ready paper. The optional artifact appendix pages will be free of charge.

ACM Reproducibility Badges


Artifacts Available

Artifacts Evaluated - Functional

Results Reproduced

Artifact Submission

    An artifact submission consists of two parts:
    1. The paper and a two-page appendix. Please prepare your appendi using the provided template. The appendix is expected to contain the following main sections:
      • an abstract
      • an itemized metainformation list
      • access to the artifact
      • system requirements and dependencies
      • experiment workflow
      • steps for evaluation
      • results
      Note that the paper does not need to be the final version, as the main goal of this submission is to let artifact reviewers reproduce your experiments.
    2. The artifact. Please make your artifact accessible by the reviewing committee. We do not limit the way of code delivery. However, if you would like to apply for the "Artifact Available" badge, you will need to have your artifact available at a public archival repository (for more details, see the reviewing guide).
    The artifact. Please submit your artifact on our submission site . When you submit, please provide details about the artifact's software and hardware requirements. This will be extremely helpful for the Artifact Evaluation Committee to find suitable reviewers.

Benefits

There are major benefits to introducing AE in our conferences.
  1. Dissemination of Ideas: The goal of our research is to disseminate insights and encourage people to build upon that idea. Open-sourcing the artifacts and opening up the ideas to the whole community ensures that the community can work together towards solving an important problem.
  2. Reproducibility of the Results: Artifact evaluation promotes reproducibility of experimental results and encourages code and data sharing to help the community quickly validate and compare alternative approaches.
  3. Safeguarding the Review Process: AE incentivizes people to conduct research in an ethical manner. The recent example of misconduct in our conference reviewing process has greatly hurt the reputation of this community. Introducing AE can help to restore our integrity and commitment to reproducible and ethical research.

Artifact Evaluation Organization

Artifact Evaluation Co-Chairs Affiliation
Po-An Tsai NVIDIA
Chris Fletcher UIUC

Program Committee

Committee Member Affiliation
Andres Sanchez EPFL
Boru Chen University of Illinois--Urbana Champaign
Christopher Fletcher (Chair) University of Illinois--Urbana Champaign
Guyue Huang University of California, Santa Barbara
Haojie Ye University of Michigan
Heewoo Kim University of Michigan
Hengrui Luo UCB/LBNL
Hengrui Zhang Princeton University
Jingren We Ohio State University
Kuan-Chieh Hsu University of California, Riverside
Moein Ghaniyoun The Ohio State University
Mujahid Al Rafi University of California, Merced
Nathan Gober Texas A&M University
Po-An Tsai (Chair) NVIDIA
Qixuan Yu Princeton University
Rachit Rajat University of Southern California
Ranggi Hwang KAIST
Rohan Basu Roy Northeastern University
Samuel Thomas Brown University
Sanjay Sri Vallabh Singapuram University of Michigan
Satvik Maurya University of Wisconsin-Madison
Shubham Roy IITB
Sihao Liu UCLA
Sudhanshu Gupta University of Rochester
Tianrui Wei University of California, Berkeley
Yongwon Shin POSTECH
Yoonho Choi POSTECH
Yufeng Gu University of Michigan-Ann Arbor
Yuhan Chen University of Michigan
Yunan Zhang University of California, Riverside
Yunjie Pan University of Michigan
Zhenhua Zhu Tsinghua University
Zhipeng Zhao Microsoft
Zishen Wan Georgia Tech