Skip to content

[WIP] Slurper: SageMath documentation pages #18

Description

@katjabercic

Index SageMath reference documentation pages as Items using the Sphinx objects.inv inventory file.

WIP: Do we want SageMath functions or documentation pages?

Data access

  • GET https://doc.sagemath.org/html/en/reference/objects.inv — binary, no auth. Format: 4-line plain-text header, then zlib-compressed entries (one per line): name type priority uri dispname. Parse with zlib.decompress — no extra dependency needed.
  • Index only std:doc entries (2,952 of 5,339 total). The py:module entries have dispname == "-" (no human title, only dotted code paths) — skip them.
  • Sample entry: algebras/cubic_hecke_algebra std:doc -1 algebras/cubic_hecke_algebra.html Cubic Hecke Algebras

Item mapping

  • identifier: name field (e.g. algebras/cubic_hecke_algebra)
  • url: https://doc.sagemath.org/html/en/reference/ + uri field
  • name: dispname field (e.g. Cubic Hecke Algebras)
  • Source tag: Sg

Tasks

  • web/slurper/source_sagemath.py with a slurper class + module-level instance.
  • New Item.Source member (SAGEMATH = "Sg"); add it to Item.Source.key().
  • management/commands/import_sagemath.py and clear_sagemath.py.
  • Makefile target import-sagemath.
  • make create-migrations for the new Source choice.
  • make fix-files clean; tests pass.

Notes

  • No Wikidata property for SageMath → index-only, links via NAME_EQ only.
  • Docs are CC-BY-SA-4.0; credit SageMath with a link in the slurper docstring.
  • Consider filtering out */index pages (section indexes like "Algebras") if
    they create noise in the concept graph.

Metadata

Metadata

Assignees

No one assigned

    Labels

    draftWork in progress issue

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions