Skip to content

Refactor Code to Enable New Features #43

Description

@goossens

TL;DR;

In order to implement features like Word Wrap, Line Folding, full MiniMaps, Autocomplete and Language Server connectivity, the architecture of the TextEditor and TextDiff widgets needs to change to keep things manageable. This refactoring spills over into the API (not to crazy but it will break backwards compatibility). An alpha version of the new architecture with the added features is available in the "future" branch. Once more testing is done, this will become the master branch.

Image

API Changes

  • The old API used visual coordinates (line and column numbers) to address glyphs in a document The new API predominantly uses documents coordinates (line number and glyph index) to address glyphs (see details below).
  • Visual coordinates (rows and columns) are also available. In some cases, document coordinates and visual coordinates are the same but as soon as you use tabs, word wrapping or line folding, offsets start to appear.
  • The old API used integers for line and column numbers, The new API uses size_t for line number, glyph indices, rows and columns since they can't be negative and they are more compatible with STL (less endless typecasting).
  • Some of the API names have changed (e.g. GetWordAtScreenPos to GetWordAtMousePos).
  • API calls have been added to support new features.

History

ImGuiColorTextEdit has a long history with many people working on it over the years resulting in over 200 disjointed forks. I started using some of those forks in late 2023.

In late December 2024, I decided that it was better for me to rewrite the code from the ground up while preserving the majority of the public APIs so it would be relatively simple for me (and others) to reuse my rewrite. One of my fundamental arguments for the rewrite was that the code had become hard to read and maintain (at least for me). I decided to create a more object-oriented internal architecture (see below) that made it clear who was responsible for what by using a layered approach. While rewriting the code, a number of new features were also added.

In April 2026, another large refactoring effort was required to improve the architecture of the widgets so that new features could be implemented. The biggest change was to make a distinction between document positions and visual positions (see below) which makes the editor use more of a Model-View-Controller (MVC) design pattern. This change enables implementation of Word Wrap, Line Folding and MiniMaps but it breaks some API calls. I believe the public API is now better and more logical but it takes a little effort to port from the older versions to this new one. API calls were also added to externally implement Autocomplete and Language Server Bridges and examples are included. If you only want to use the old interface and forgo new features, the legacy release contains a snapshot that is most compatible with older versions of this editor. The legacy release will however not be maintained.

Word Wrap

When word wrap mode is activated (see SetWordWrapEnabled API call), lines are soft-wrapped within the available space based on a two part process. First line break opportunities are identified and then a greedy algorithm is used to fill the available space based on those identified opportunities. If there are no break opportunities in a line, the text is rudely truncated to fit within the space.

The first process is configurable and offers two algorithms. The first and simple algorithm works on a set of glyphs can that you can break on before or after. This is fast and seems to work fine when editing code. Theoretically, defaults could be provided based on the selected language but given that people have religious arguments about code formatting, an API is available to customize this.

The second algorithm is based on the Unicode Line Break Algorithm documented in Annex #14 of the Unicode standard. This algorithm is expressed as a set of rules and the TextEditor allows you to (de)activate individual rules to meet your needs. The example app shows how to configure the unicode algorithm to be more suitable for code editing (versus text editing).

The line break process can be configured through the SetLineBreakConfig API call.

Line Folding

When line folding mode is activated (see SetLineFoldingEnabled API call), line folding triangles appear next to relevant line numbers to collapse/expand sections of code.

For most languages, sections are based on curly brackets, square brackets and parentheses. Languages that are based on indentation (e.g. Python) are also supported.

MiniMap

The previous widgets supported optional minimaps projected on top of the vertical scrollbar to render current cursor, selections and marker locations. In the new architecture, a full optional minimap in the style of Visual Studio Code is also available and can be activated with the SetShowMiniMapEnabled API call.

Autocomplete

Full autocomplete is not part of the core TextEditor widget but it provides an optional framework to add it. Once configured, the editor takes care of activation events (triggering), state tracking, visualization and insertion of suggestions with full undo/redo. The application is responsible for providing the list of suggestions through a callback or the API. This allows simple implementations to provide suggestions in realtime and allows other implementations to do things asynchronously like reaching out to a language server.

To make life easier, two autocomplete addons are provide in the extras folder. Both of them are objects that attach themselves to the editor and do all the hard work for you. The first is a simple suggestion generator that creates a list of words based on keywords in the selected language and identifiers contained in the current document. This solution is very fast but lacks the ability to provide context specific suggestions. A second solution is available to create a bridge between the editor and an external language server which is base on a LSP Framework which in turn depends on C++20. So if you want to use this bridge in your application, you have to include the framework and ensure you are using C++20. The example application does this and you can use it as a guide.

Architecture

The block diagram below shows the new architecture. At the top is the public facing TextEditor instance and at the bottom are the private classes that store and maintain the internal state.

Image

Document

This class, as the name suggests, stores the document that's being edited. Internally, a Document is a std::vector of Lines that themselves are std::vectors of Glyphs.

The Document class translates from the external UTF-8 encoding to the internal codepoints and provides a number of member functions to manipulate content, calculate coordinates, assist searches and track overall state.

The Line and Glyph classes are used to represent the document's structure with Glyph just holding a codepoint, a color and a possible text breaking status. Both Line and Glyph also hold information used by overlays discussed below.

Overlays

Overlays are addons to a document that handle additional functionality or state. In some cases, overlays store there information in the Line/Glyph objects and the overlays are responsible for keeping the state consistent. Other overlays of the TextEditor class can use this extra information for their purposes.

Colorizer

The colorizer maintains the color state of each glyph based on the rules provided by the Language specification object. The original version used regular expressions for some languages but this was/is not very performant. The new colorizer is based on multiple state-transition engines that make it easier to express language rules and also improves performance. The new colorizer only effects lines that have changed but keep in mind, that for instance opening a multiline comment at the start of the document, causes the entire document to be re-colored. Luckily the new engine is fast enough that you don't notice this and it would only affect a single frame. I think it is also important to point out that this widget is not really intended for mega/gigabyte size text files. For those, I would still use a regular text editors (if you can find one that can handle that requirement).

Transactions

This class contains a list of Transaction records to support do/undo/redo operations. Each Transaction record contains insert and delete Actions performed on a document.

Cursors

The Cursors class maintains a list of Cursor instances and operates very much like cursors do in Visual Studio Code. The list of cursors consists of the main cursor (which is always present) and an optional set of additional cursors. The last cursor added is considered the current cursor which has meaning for some actions or API calls.

TypeSetter

This class handles the visualization of a document and maintains the link between the logical state off a document and it's visual representation. In some cases, these are the same but tabs cause horizontal offsets and word wrap as well as line folding cause vertical offsets. To support this, the editor has two coordinate systems. DocPos is used as the logical address
of a glyph in a document. It is expressed as a line number with a glyph index. This is the dominant coordinate system in the editor and it is also used in most API calls. VisPos is the other coordinate system and it addresses the visual location of a glyph on the screen expressed as a row and a column. No public APIs use this second coordinate system but it is publicly exposed as are the translation function to and from DocPos. So if required, applications can also use it.

Minimap

This class keeps the state of the optional minimap.

AutoComplete

This class keeps track of autocomplete activities when the feature is configured. Once autocomplete is triggered, this class tracks the state, interacts with the user and uses an external callback to collect suggestions in the current context.

LineFold

This class contains the algorithms for word wrap and it is used by the TypeSetter when this feature is activated..

Bracketeer

This class keeps track of where bracket pairs (parenthesis, square brackets and curly brackets) are in the document so they can be highlighted, colorized and selected when this feature is activated. It will also colorize unbalanced brackets as errors. Brackets in comments and strings are ignored.

TextEditor

As mentioned above, the TextEditor class provides the public API. As is true for Dear ImGui and the original text editor, all public member functions start with an uppercase letter. Internally, all private member functions and variables start with a lowercase letter so it's easy to see what's public and what's private.

In addition to being the public interface, the TextEditor class is also responsible for synchronizing the state of the lower levels of the architecture. When for instance the user pastes some text, TextEditor ensures that the Document gets updated, Cursors get adjusted (if required), Transaction records are created (so this paste operation can be undone and redone) and the overlays are informed of the changes so they can update their state.

The final responsibility of the TextEditor class is overall rendering and user input (keyboard and mouse) processing.

Below is a dependency graph for the major internal classes of the TextEditor.

Image

Future

In the near-term, the focus will be on bug fixes followed by optimizations.

Moving forward, this repository will include releases with a numbering scheme synchronized with Dear ImGui. This will allow people to quickly find a version of the widgets compatible with a specific Dear ImGui version.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions