New parser

.github/workflows/ci.yml

@@ -0,0 +1,31 @@
+name: CI Pipeline
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Check out repository code
+      uses: actions/checkout@v6
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version: '3.14'
+        cache: 'pip'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+        pip install pytest 
+
+    - name: Run Test Suite
+      run: |
+        pytest

.gitignore

@@ -1,3 +1,24 @@
 *.prof
 *.html
-.env
+.env
+_md_export/
+
+# Python bytecode cache
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+
+# Pytest cache
+.pytest_cache/
+
+*/__pycache__/
+_tmp_md_export/
+
+old_bot.py
+old_refsplitter.py
+entries/
+pages/
+pretty_soup.txt
+ref_doc.txt
+dm cache sample.txt

README.md

@@ -1,18 +1,42 @@
-# About open-ref
-This is an open source project meant to allow the [DM Reference](https://www.byond.com/docs/ref/) to be edited by the community to allow us to make fixes, corrections and ammendments and expand upon it.
+# OpenRef
+OpenRef is an open-source project that makes the BYOND DM Reference easier to maintain, improve, and expand through community contributions.
 
-This repo also contains the tool used for splitting the reference provided by BYOND.com: `ref_splitter.py`.  At this time the documentation for the tool is contained in the script itself.
+Originally started as a way to edit and version-control the reference, the project evolved into a document-processing pipeline featuring HTML parsing, structured document models, automated transformations, testing, CI/CD, and community tooling.
 
-The discord bot is also available here as well, under the bot folder.
+## Features
+* HTML → structured document parsing
+* Markdown generation and transformation
+* Automated reference processing tools
+* Test suite and CI/CD pipeline
+* Discord bot integration
 
-You can join the BYONDiscord server to discuss this project [here](https://discord.gg/pTFccGgnAr)
+## Repository Contents
+### Reference Data
+A community-maintained version of the DM Reference derived from BYOND's published documentation.
 
 ## Contributing
-Anyone is welcome to submit contributions to open-ref.
-Contributions can be made either on Github.com or by cloning the repo and working locally.
-All contributions require a PR and are subject to approval.
-If you have any thoughts for changes or edits, feel free to open a discussion on the repo or discuss them on the discord server
-
-## BYOND Version
-The reference is currently built with the documentation for BYOND version 516.1644 alpha. 
-BYOND release notes can be found [here](https://www.byond.com/docs/notes/515.html).
+Contributions are welcome, including:
+* Documentation corrections
+* Additional explanations and examples
+* Tooling improvements
+* Bug fixes
+* Test coverage improvements
+
+All changes should be submitted through a Pull Request and are subject to review before being merged.
+Discussion is available through GitHub or the BYONDiscord community:
+https://discord.gg/pTFccGgnAr
+
+## Supported BYOND Version
+The current reference data is based on BYOND 516.1644 Alpha.
+
+Release notes:
+https://www.byond.com/docs/notes/516.html
+
+## Technical Focus
+This project demonstrates:
+* Parser development
+* Document transformation pipelines
+* Python application architecture
+* Automated testing
+* CI/CD workflows
+* Open-source collaboration

conftest.py

@@ -0,0 +1,25 @@
+from pathlib import Path
+import pytest
+from ref_splitter.ref_splitter import RefSplitter
+
+@pytest.fixture(scope="function")
+def mouse_down_sample_path():
+    """Returns the absolute path to the mouse_down_sample test file."""
+    return Path(__file__).parent / "tests" / "test_data" / "mouse_down_input_sample.txt"
+
+@pytest.fixture(scope="function")
+def client_sample_path():
+    """Returns the absolute path to the reference client sample file."""
+    return Path(__file__).parent / "tests" / "test_data" / "ref_client_input_sample.txt"
+
+@pytest.fixture(scope="function")
+def splitter(mouse_down_sample_path):
+    '''creates a shared splitter instance for all the tests in this module'''
+    with open(mouse_down_sample_path, 'r', encoding='utf-8') as f:
+        content = f.read()
+
+    splitter_instance = RefSplitter(content)
+    splitter_instance.prep_pages()
+    splitter_instance.build_ref_entries()
+
+    return splitter_instance