Poor performance in org-mode buffers

[jams2]

Hi, thanks for the package, I've been enjoying it!

One issue I've come across is that in my main org-mode file, which is about 4000 lines long (378 headings), navigation can become very laggy when the headings are collapsed. move-beginning-of-line, next-line, etc. all become very slow. Max heading depth in my file is currently 3. The same issue doesn't appear when navigating over expanded trees.

Emacs info: GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw scroll bars) of 2024-06-19

Jinx version: 1.7 e48f333

The CPU profiler shows it's spending a lot of time in a few of jinx's functions (I'll attach the full report).

       17236  67% - timer-event-handler
       17236  67%  - apply
       17223  67%   - jinx--timer-handler
       17223  67%    - jinx--check-pending
       17223  67%       jinx--find-visible-pending
          13   0%   - #<native-comp-function F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_9>
          13   0%      jit-lock-context-fontify
        8064  31% - command-execute
        8030  31%  - byte-code
        8030  31%   - read-extended-command
        8030  31%    - read-extended-command-1
        8030  31%     - completing-read-default
        8030  31%      - apply
        8030  31%       - vertico--advice
        8030  31%        - apply
        8030  31%         - #<byte-code-function 6AE>
        8030  31%          - apply
        8030  31%           - #<native-comp-function completing-read-default>
        7467  29%            - timer-event-handler
        7467  29%             - apply
        7447  29%              - jinx--timer-handler
        7447  29%               - jinx--check-pending
        7447  29%                  jinx--find-visible-pending

profile.txt

[minad]

Hi!

I don't observe such issues in my own workflow, but such performance problems with collapsed text have been reported before. See in particular the discussion: #156 (comment)

The performance could be improved if Jinx would use a more efficient tracking of pending/unchecked regions. It should use a separate data structure instead of relying on text properties in the buffer. Patches implementing this are welcome.

[Thaodan]

How deos Jinx deal with hidden regions e.g. when narrowing? I had some lags when narowing in org-mode with Jinx enabled that went away after disabling jinx or widening.
I'm not 100%, I have to verify the issue again.
The situation was very similar as in #156 with the word suggestions popping up at the same time as the slow down occurs.

[gety9]

@jams2 @Thaodan were you guys able to solve the issue? my org mode file is unusable when i call "consult-outline" - it freezes, but if i remove jinx, and replace it with flyspell - everything is perfect.

The same issue doesn't appear when navigating over expanded trees.

Update: same situation here - if cycle visability to "SHOW ALL" then issue dissapears, "consult-outline" is blazing fast with jinx-mode enabled. If i make it "OVERVIEW" - it's unusable (5+ seconds freezes), disabling jinx-mode fixes the issues.

---

Emacs 29.4, Jinx 1.11 (so it should include 7a7f5f0 (#156))

[minad]

The interaction of consult-outline is a good insight. consult-outline fontification of each of the headings is likely the cause of your slowdown. But it may not be the same issue as the toplevel one.

EDIT: Any consult-line and consult-outline related slowdown should be gone by now, since these commands use lazy fontification since minad/consult@63a261b.

[minad]

I pushed a potential fix: bbda659

[gety9]

@minad Daniel, thx for the fix, it's much better, let me play with it couple of days, hopefully that's it.

couple more things, just in case they will be helpfull to you:
-, consult-line even on folded org-file pretty performnat, it's consult-outline which lags
-, i saw your reply about porential fix too late :) so i spent some time profiling, here is report on "consult-outline" (i "jinx-mode, profiler-start, consult-outline, profiler-stop, profiler-report"):

       14272  99% - command-execute
       14272  99%  - call-interactively
       14272  99%   - funcall-interactively
       14272  99%    - consult-outline
       12902  90%     - consult--read
       12902  90%      - apply
       12902  90%       - consult--read-1
       12902  90%        - consult--with-preview-1
       12902  90%         - #<compiled 0x1244e12e66c1d087>
       12902  90%          - completing-read
       12902  90%           - completing-read-default
       12902  90%            - apply
       12902  90%             - vertico--advice
       12902  90%              - apply
       12902  90%               - #<compiled 0x7297b5a93e6ca7b>
       12902  90%                - read-from-minibuffer
       12814  89%                 - timer-event-handler
       12814  89%                  - apply
       12814  89%                   - jinx--timer-handler
       12814  89%                    - jinx--check-pending
       12814  89%                       jinx--find-visible-pending
          10   0%                 - consult--preview-post-command-hook
          10   0%                  - #<compiled 0x159ce5286fc74a93>
          10   0%                   - #<compiled 0x191d147aac931251>
          10   0%                    - #<compiled 0x175243250cd81c51>
          10   0%                     - #<compiled 0x4bc23825c6da880>
          10   0%                        run-hooks
          10   0%                 - command-execute
          10   0%                  - call-interactively
          10   0%                   - funcall-interactively
          10   0%                      profiler-stop
        1370   9%     - funcall-with-delayed-message
        1370   9%      - #<compiled -0xe338f5ffec33b84>
        1350   9%       - consult--outline-candidates
        1092   7%        - consult--fontify-region
        1052   7%         - jit-lock-fontify-now
        1042   7%          - jit-lock--run-functions
        1032   7%           - run-hook-wrapped
        1012   7%            - #<compiled -0x570ddbc9996f03e>
        1012   7%             - font-lock-fontify-region
        1002   7%              - font-lock-default-fontify-region
         758   5%               - font-lock-fontify-keywords-region
          90   0%                  org-do-emphasis-faces
          80   0%                - org-activate-links
          80   0%                   org-activate-links--overlays
          70   0%                  org-activate-dates
          60   0%                  #<compiled 0x1caac574cedfd4fe>
          52   0%                  org-font-lock-add-priority-faces
          40   0%                - org-cite-activate
          20   0%                 - org-cite-processor-has-capability-p
          20   0%                  - org-cite-get-processor
          10   0%                     seq-find
          10   0%                 - org-cite-get-processor
          10   0%                  - seq-find
          10   0%                   - seq-do
          10   0%                    - mapc
          10   0%                       #<compiled -0x6577a408953e654>
          29   0%                  org-fontify-drawers
          20   0%                - org-activate-footnote-links
          20   0%                   org-footnote-next-reference-or-definition
          20   0%                  org-fontify-inline-src-blocks
          19   0%                  org-activate-tags
          10   0%                - org-fontify-meta-lines-and-blocks
          10   0%                   org-fontify-meta-lines-and-blocks-1
         185   1%               - font-lock-unfontify-region
         185   1%                - org-unfontify-region
          39   0%                   org-fold-region
          10   0%                   font-lock-default-unfontify-region
           9   0%                   org-remove-font-lock-display-properties
          50   0%          org-outline-level
          20   0%          consult--tofu-encode
          27   0% + ...
          11   0% + timer-event-handler

[minad]

Daniel, thx for the fix, it's much better, let me play with it couple of days, hopefully that's it.

Thanks for the feedback.

-, consult-line even on folded org-file pretty performnat, it's consult-outline which lags

Yes, makes sense. consult-line doesn't enforce fontification for large buffers, while consult-outline enforces it. consult-outline only highlights headlines though, but for Org buffers this can still be a large fraction of the whole buffer.

In any case the modified search for pending regions should do better for buffers which are folded. The problem with folded buffers is that the entire (or almost the entire) buffer is visible such that the search is naturally more expensive. In particular searching for jinx--pending properties becomes expensive. Searching for visible regions less so, since the amount of visible regions is bounded by the window size.

-, i saw your reply about porential fix too late :) so i spent some time profiling

This looks about the same what I got.

[minad]

Any consult-line and consult-outline related slowdown should be gone by now, since these commands use lazy fontification since minad/consult@63a261b.

[Thaodan]

I haven't tested the fix but today I noticed some behavior which I had behavior but within the Circe buffers where read-only text, images and the content that Jinx interacts with meet. I noticed that the poor performance or "freezes" happens whenever a popup from the text suggestions was about to draw over a TODO status of an item. The text properties I use for those are: (:inherit ((:inverse-video t :weight semibold :height 1.0 :inherit (org-todo)) org-modern-label))

[minad]

I have no explanation for that behavior. Maybe my commit also fixes this? Can you please check?