Cleanup for release (and LitXAlloy 0.1.1)

.github/workflows/docs.yml

@@ -27,7 +27,7 @@ jobs:
         run: uv python install 3.12
 
       - name: Install Python dependencies
-        run: uv sync --group docs
+        run: uv sync --extra docs
 
       - name: Build Sphinx documentation
         run: uv run sphinx-build docs docs/_build/html

README.md

@@ -3,10 +3,10 @@
 [![PyPI](https://img.shields.io/pypi/v/litxbench?logo=pypi&logoColor=white)](https://pypi.org/project/litxbench)
 [![LitXAlloy](https://img.shields.io/endpoint?url=https://radical-ai.github.io/litxbench/litxalloy-badge.json)](https://radical-ai.github.io/litxbench)
 
-LitXBench is a benchmark to evaluate LLMs on extracting material information synthesized in research papers. Read the preprint here.
+LitXBench is a benchmark to evaluate LLMs on extracting material information synthesized in research papers. Read the preprint [here](https://arxiv.org/pdf/2604.07649).
 
 <p align="center">
-  <img width="70%" src="https://raw.githubusercontent.com/Radical-AI/litxbench/main/core_principles.png" alt="LitXBench Principles for Accurate Extraction and Benchmarking. (1) To accurately capture a material’s properties, measurements must be linked to its processing lineage, rather than just its composition. (2) Categorical values should be mapped to canonical values to disambiguate similar values, as multiple papers may reference different properties with the same term. (3) Extracted materials are more editable and auditable when represented as code, reducing errors in the benchmark."/>
+  <img width="70%" src="https://radical-ai.github.io/litxbench/_static/core_principles.png" alt="LitXBench Principles for Accurate Extraction and Benchmarking. (1) To accurately capture a material’s properties, measurements must be linked to its processing lineage, rather than just its composition. (2) Categorical values should be mapped to canonical values to disambiguate similar values, as multiple papers may reference different properties with the same term. (3) Extracted materials are more editable and auditable when represented as code, reducing errors in the benchmark."/>
 </p>
 
 # Installation
@@ -135,4 +135,20 @@ A complete end-to-end example is available at [`examples/usage.py`](examples/usa
 
 # Paper Evaluation Scripts Warning
 
-For the evaluation scripts used in the paper, LitXBench intructs LLMs to format the extracted materials as code. This code is run by LitXBench via Python `exec`. Do NOT call untrusted LLMs as they may generate untrusted code which could be executed on your machine.
+For the evaluation scripts used in the paper, LitXBench instructs LLMs to format the extracted materials as code. This code is run by LitXBench via Python `exec`. Do NOT call untrusted LLMs as they may generate untrusted code which could be executed on your machine.
+
+# Citation
+
+If you use LitXBench in your research, please cite:
+
+```bibtex
+@article{chong2026litxbench,
+  title         = {LitXBench: A Benchmark for Extracting Experiments from Scientific Literature},
+  author        = {Curtis Chong and Jorge Colindres},
+  year          = {2026},
+  eprint        = {2604.07649},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.IR},
+  url           = {https://arxiv.org/abs/2604.07649}
+}
+```

docs/_templates/sidebar/explorer-link.html

@@ -6,5 +6,11 @@
     <li class="toctree-l1">
       <a class="reference external" href="/litxbench/explorer/">LitXAlloy Graph Viewer</a>
     </li>
+    <li class="toctree-l1">
+      <a class="reference external" href="https://github.com/Radical-AI/litxbench">GitHub</a>
+    </li>
+    <li class="toctree-l1">
+      <a class="reference external" href="https://arxiv.org/pdf/2604.07649">Paper</a>
+    </li>
   </ul>
 </div>

docs/about/contributing.rst

@@ -1,43 +1,39 @@
-Contributing
-============
-
-Contributions to LitXBench are welcome! Please open an issue or pull request on the
-`GitHub repository <https://github.com/Radical-AI/litxbench>`_.
-
-Development Setup
------------------
-
-.. code-block:: bash
+Contributing to the Leaderboard
+-------------------------------
 
-   git clone https://github.com/Radical-AI/litxbench.git
-   cd litxbench
-   uv sync --group dev
+We welcome community contributions to the :doc:`/leaderboard`. To add your method:
 
-If you want to replicate results from the paper you'll need to add `--group paper` to install the required dependencies.
+1. Run your extraction method on the LitXAlloy benchmark dataset by calling ``compare_experiments`` and ``compute_multi_level_metrics``. An example is in the `usage script <https://github.com/Radical-AI/litxbench/blob/main/examples/usage.py>`_.
+2. Open a `pull request <https://github.com/Radical-AI/litxbench/pulls>`_ that adds your results as a new row to the leaderboard table in `docs/index.rst <https://github.com/Radical-AI/litxbench/blob/main/docs/index.rst>`_. See `this example PR <https://github.com/Radical-AI/litxbench/pull/5>`_ for reference.
 
-Running Tests
--------------
+When updating `docs/index.rst <https://github.com/Radical-AI/litxbench/blob/main/docs/index.rst>`_, please include:
 
-.. code-block:: bash
+1. A link to the code that generated the results
+2. The file containing the output experiment objects from your run
+3. Any publication you'd like linked
+4. A link to the PR that submitted your result
+5. The version of LitXAlloy it was evaluated on (this version is bumped when the dataset or evaluation methods change, so scores across different versions may not be directly comparable). You can get this with:
 
-   uv run pytest
+   .. code-block:: python
 
-Contributing to the Leaderboard
--------------------------------
+      from litxbench.litxalloy import __version__
+      print(__version__)  # e.g. "0.1.0"
 
-We welcome community contributions to the :doc:`/leaderboard`. To add your method:
+Uncertainties are not required -- if your method was only run once, simply report the score without a confidence interval.
 
-1. Run your extraction method on the LitXAlloy benchmark dataset.
-2. Evaluate your results using the LitXBench evaluation pipeline (see :doc:`/user/evaluation`).
-3. Open a `pull request <https://github.com/Radical-AI/litxbench/pulls>`_ that adds your results as a new row to the leaderboard table in ``docs/leaderboard.rst``.
+Contributing to LitXBench
+-------------------------
 
-Uncertainties are not required -- if your method was only run once, simply report the score without a confidence interval.
+Contributions to LitXBench are welcome! Please open an issue or pull request on the
+`GitHub repository <https://github.com/Radical-AI/litxbench>`_.
 
-Building Docs Locally
----------------------
+Development Setup
+=================
 
 .. code-block:: bash
 
-   uv sync --group docs
-   uv run sphinx-build docs docs/_build/html
-   open docs/_build/html/index.html
+   git clone https://github.com/Radical-AI/litxbench.git
+   cd litxbench
+   uv sync --extra dev
+
+If you want to replicate results from the paper you'll need to add ``--group paper`` to install the required dependencies.

docs/conf.py

@@ -30,7 +30,7 @@
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
 html_theme = "furo"
-html_title = f"LitXBench v{version} · LitXAlloy v{litxalloy_version}"
+html_title = f"LitXBench v{version} <small style='font-size:0.65em;opacity:0.7'>· LitXAlloy v{litxalloy_version}</small>"
 html_static_path = ["_static"]
 html_css_files = ["custom.css"]
 html_js_files = ["leaderboard.js"]