Skip to content

Refactor SentenceAwareChunker chunking logic, bump Swift tools, and pin Xcode in CI#4

Merged
joelnishanth merged 10 commits into
mainfrom
codex/investigate-ci-failure-uo15ag
Apr 26, 2026
Merged

Refactor SentenceAwareChunker chunking logic, bump Swift tools, and pin Xcode in CI#4
joelnishanth merged 10 commits into
mainfrom
codex/investigate-ci-failure-uo15ag

Conversation

@joelnishanth

Copy link
Copy Markdown
Collaborator

Motivation

  • Fix incorrect/fragile sentence-based chunking and improve handling of word counts and overlaps when building chunks.
  • Ensure the package uses the newer Swift toolchain features by updating the package tools version to 6.1.
  • Stabilize CI by pinning the Xcode version used in GitHub Actions to avoid variations from latest-stable.

Description

  • Rewrote the chunking loop in SentenceAwareChunker to use sentence indices and accumulate chunkWordCount with explicit start/end indexes, producing consistent sourceWordRange and metadata for each TextChunk.
  • Ensure chunks respect targetWords, enforce at-least-one-sentence per chunk, and implement sentence overlap rewind using overlapSentences to compute the next sentenceIndex correctly.
  • Updated // swift-tools-version in Package.swift from 5.9 to 6.1 to adopt the newer Swift toolchain.
  • Pinned the GitHub Actions Xcode selection to 16.3 in .github/workflows/ci.yml instead of latest-stable.

Testing

  • Ran swift test against the package and the test suite completed successfully.
  • CI workflow updated to run on macos-14 with Xcode 16.3 and will execute swift test as part of checks.

Codex Task

@joelnishanth
joelnishanth merged commit 237bf20 into main Apr 26, 2026
2 checks passed
@joelnishanth
joelnishanth deleted the codex/investigate-ci-failure-uo15ag branch May 6, 2026 05:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant