Add ACM CCS?

[jovanbulck]

As far as I know, ACM CCS started artifact evaluation since CCS 2023. I noticed CCS is currently prominently missing on the website.

Unfortunately, however, the CCS 2023 website seems not updated and I can't seem to find any clear overview page with artifact badges and appendices for evaluated papers. As far as I see, the official CCS'23 proceedings don't include the AE results.

At least some papers seems to have been evaluated though, e.g., this GitHub repo.

I also noticed there is a hotcrp ccs2023-ae site which mentions "32 papers accepted out of 33 submitted."

Opening this issue as it'd be great if someone could add an overview of the AE results to the website?

cc @ThorstenHolz ?

[vahldiek]

Not sure how I missed that CCS started artifact evaluation as well. I've written Eric and Thorsten an email and hope that they have a list of paper names, badges, and artifact location.

[vahldiek]

Eric plans to add 2024 when all cycles are done.

[jovanbulck]

Reviving this thread. It seems CCS AE is here to stay now already successfully organized in 2025-2024-2023. Unfortunately, the results are not in a single place afais, so it'd be great if we can add them to secartifacts somehow.

@vahldiek Any updates on this?

[vahldiek]

I pinged both chairs. Eric promised a list, but I haven't received one yet after multiple pings.

Without chair support it, is tough. ACM DL has the badges. I randomly picked one paper to find the repository. It has a footnote on the first page. There is no artifact appendix. The value in sec/sysartifacts is in the association of paper title, badges, repository/artifact url. Even if we scrape the badges of ACM DL, getting the repos automatically seems impossible.

Maybe you could ping Eric for 2024 and Thorsten for 2023, and hope for more luck?

[jovanbulck]

Thanks, very understandable! Normally, David will add the results for CCS25 in the next weeks, so I suggest we wait for those first and then I can ping the 23 and 24 chairs to hopefully fill in those as well :)

[ArquintL]

Thanks for bringing this up again! Despite CCS '23 having long passed, the artifacts seem to have been completely forgotten, which is a shame for all the work that went into them.I'm not sure what the best way forward is but I'd be more than happy to put together a PR setting up the various CCS editions and provide my own artifact (with a screenshot from HotCRP as a proof that I'm claiming the correct badges) to finally close this topic for me as the responsible AE chairs are not responding to my inquiries. Other artifacts could then be added via separate PRs

[vahldiek]

I suspect that we won't get 23/24 or even 25 results. I tried building a script to scrape ACM DL, but it doesn't seem possible. Is there any way to reliably get this information?

[J08nY]

I looked at the 2023-2025 issues of CCS in the ACM DL and:

• 2023 does not even have badges in there. (Or no artifacts actually got them.)
• 2024 has the badges in there, but no supplementary materials.
• 2025 has the badges in there and I saw like one or two artifact appendices as a supplementary material.

What a sad state of affairs for an A* conference.

[vahldiek]

@J08nY if you have any suggestions on how to approach the respective parties, please do. I've tried several times without any luck. I've proactively reached out to the CCS 2026 chairs, no luck either so far.