Linter: Cannot enlarge memory error

[janko]

I'm currently using Herb 0.8.0, and when I tried to upgrade to 0.8.6, the linter started throwing these errors:

Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147487744 bytes, but the limit is 2147483648 bytes!
Error: Failed to initialize buffer with capacity of 128.

It's still good up until 0.8.3, then from 0.8.4 it starts throwing these errors. I don't have any .herb.yml and am running the following command:

yarn herb-lint app/views

Mind you that I'm on MacBook with only 8GB of RAM, activity monitor says I have ~1GB available, but that should be sufficient IMO.

What Claude Code thinks is the root cause

  The memory error is caused by commit a43c7e6f ("C: Deprecate hb_array_size"), which was first included in v0.8.4.

  The Change (templates/src/ast_nodes.c.erb:29-34)

  Before v0.8.4:
  node->field = field;  // NULL if no array passed

  After v0.8.4:
  if (field == NULL) {
      node->field = hb_array_init(8);  // Always allocate!
  } else {
      node->field = field;
  }

  Why This Causes Memory Issues

  1. Each hb_array_init(8) allocates:
    - sizeof(hb_array_T) (~24 bytes for struct)
    - 8 * sizeof(void*) = 64 bytes for items
    - ~88 bytes per empty array
  2. Many AST nodes have array fields (children, attributes, etc.)
  3. When parsing large template files, every AST node now creates arrays even when they're not needed (passed as NULL). This multiplies memory usage significantly.
  4. WebAssembly has a 2GB memory limit - the cumulative effect of these allocations pushes the WASM module over that limit when linting large codebases.

  Potential Fixes

  1. Revert to lazy initialization: Don't allocate arrays when NULL is passed; initialize them only when actually needed (when items are added).
  2. Use smaller initial capacity: Change hb_array_init(8) to hb_array_init(0) or hb_array_init(1) for default-created arrays.
  3. Set explicit WASM memory limits in wasm/Makefile:
  -s INITIAL_MEMORY=33554432 \  # 32MB
  -s MAXIMUM_MEMORY=2147483648  # 2GB explicitly
  3. (Though this won't fix the underlying memory growth issue)

  The most efficient fix would be option 1 - restoring lazy initialization for array fields while still supporting direct .size access by handling NULL arrays appropriately in the code that accesses them.

[marcoroth]

Very good catch, thank you @janko!

I opened #1025 to address this 🙌🏼

[marcoroth]

This is now released with https://github.com/marcoroth/herb/releases/tag/v0.8.7, curious if that resolves the issue. If not, please feel free to re-open. Thank you!

[janko]

Yes, 0.8.7 fixes the memory errors 👍🏻

However, I realized the root cause was actually #1045, where Herb was processing HTML files outside of app/views, so it probably encountered some very large ones. I still get "JavaScript heap out of memory" error on my 8GB MacBook (which is a different error), but it goes away if I force only scanning app/views/**/*.html.erb files.