[chore] Switch toc generation to doctoc

[thompson-tomo]

Changes

This switches to doctoc for toc generation.

The benefit of this switch includes:

• Support for front matter and unblocks moving to pre-processed hugo content [editorial] Inline Hugo front matter #1821
• Package is mantained and accepting of enhancements/fixes

Important

Pull requests acceptance are subject to the triage process as described in Issue and PR Triage Management.
PRs that do not follow the guidance above, may be automatically rejected and closed.

Merge requirement checklist

• CONTRIBUTING.md guidelines followed.
• Change log entry added, according to the guidelines in When to add a changelog entry.
  • If your PR does not need a change log, start the PR title with [chore]
• Links to the prototypes or existing instrumentations (when adding or changing conventions)

[chalin]

Nice work!

I'd prefer to keep the current markers <!-- toc --> and <!-- tocstop --> without any extra comments, at least for this PR -- it would help us focus on the essential change in this PR (which is very minimal). Besides, IMHO folks generally know that the toc is generated by now. I like to keep things DRY and avoid repeating update instructions in every file.

[thompson-tomo]

No worries @chalin, Totally agree with you about keeping things dry, unfortunately those comments are currently hard code by doctoc. 😔

When reviewing this PR I did have a similiar concern about the noise from those comments hence I raised thlorenz/doctoc#315

In relation to the indicator start doctoc vs toc this is also fixed and when we could eventually control the header I am not sure this would be needed.

[chalin]

Btw, great job on the improvements you contributed to doctoc -- https://github.com/thlorenz/doctoc/commits?author=thompson-tomo! Thanks for doing that 🙌🏻

Totally agree with you about keeping things dry, unfortunately those comments are currently hard code by doctoc. 😔

I see 🤔

When reviewing this PR I did have a similiar concern about the noise from those comments hence I raised thlorenz/doctoc#315

Ok, I 👍🏻 that issue. Might you be planning a PR to fix that? If so, what's the usual turnaround?

[thompson-tomo]

Thanks, hopefully the updates makes thing easier/more efficient and importantly more reliable for users.

Yes I was thinking about tackling it after a refactor which included breaking changes as it became clear all the additional settings are not scaling nicely and it caused me grief when adding due to merge conflicts.

I could do a partial implementation ie the remove option if we needed to sooner but a review usually takes weeks with nudges.

Let me know your preference as who knows we might get lucky.

[chalin]

I'd like to keep this PR (and our git history) as clean as we can. We probably don't want to wait a few more weeks. Let's assume that thlorenz/doctoc#315 will get fixed ... in the meantime, to streamline this PR, we can resort to a simple Perl search/replace for prep- and post-processing before/after calling doctoc. (I'd prefer Perl over sed because it's more portable, but given that that Makefile has a SED variable, we can use that.

[thompson-tomo]

@chalin just following up on this PR.

[thompson-tomo]

@chalin could you please take another review of this.

[github-actions]

This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days.

[thompson-tomo]

@chalin i have now removed the scripts as doctoc now natively supports the compact pragma like we were achieving using the scripts.

[chalin]

@thompson-tomo - thanks for sticking with this and following up. My earlier concerns look addressed now:

• compact/doctoc markers without the extra noise
• markdown-toc kept as the mutating target
• separate markdown-toc-check for CI validation
• temporary workaround scripts removed

As final prep, pls:

• rebase, and
• regenerate a new package lock file

Note: using . as the target path shouldn't cause problems, but we might need to adjust that if it does.

[thompson-tomo]

No worries at all @chalin i have just performed a rebase & updated doctoc to the latest which includes updating the lock file.

[chalin]

LGTM 🎉

@jsuereth et all: can we get this merged soon?

[chalin]

Woot! 🎉 Thanks you @thompson-tomo! Thanks all!

[thompson-tomo]

No worries @chalin i have updated the successor pr's & split the original pr into 2 (#2971 & #3671) to make it easier to review.