Parser does not terminate within a reasonable time frame

[timkaechele]

Tried to run the herb parse command on a rather large file I use for testing. I waited for 10 minutes, and the command hasn't finished yet.

Steps to reproduce

# Download the attached html file
$ make clean
$ make -j32
$ ./herb parse $PATH_TO_DOWNLOADED_FILE

Files

example.html

LLDB Backtrace

❯ lldb ./herb parse ../example.html
(lldb) target create "./herb"
Current executable set to '/Users/timkachele/Developer/herb/herb' (arm64).
(lldb) settings set -- target.run-args  "parse" "../example.html"
(lldb) r
Process 45600 launched: '/Users/timkachele/Developer/herb/herb' (arm64)
Process 45600 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x00000001000188f0 herb`herb_visit_node(node=0x0000000a28d96340, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:15
   9   	#include "include/visitor.h"
   10  	
   11  	void herb_visit_node(const AST_NODE_T* node, bool (*visitor)(const AST_NODE_T*, void*), void* data) {
-> 12  	 if (visitor(node, data) && node != NULL) {
   13  	   herb_visit_child_nodes(node, visitor, data);
   14  	 }
   15  	}
Target 0: (herb) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x00000001000188f0 herb`herb_visit_node(node=0x0000000a28d96340, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:15
    frame #1: 0x0000000100018cb0 herb`herb_visit_child_nodes(node=0x0000000a28d9cfc0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:91:11
    frame #2: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a28d9cfc0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #3: 0x0000000100018cf8 herb`herb_visit_child_nodes(node=0x0000000a2916c300, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:101:9
    frame #4: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a2916c300, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #5: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a28dac840, errors=0x00000001006f9360) at parser.c:1318:5
    frame #6: 0x00000001000138fc herb`match_tags_visitor(node=0x0000000a28da4500, data=0x00000001006f9360) at parser_match_tags.c:27:9
    frame #7: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a28da4500, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #8: 0x0000000100018b34 herb`herb_visit_child_nodes(node=0x0000000a298abd40, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:60:9
    frame #9: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298abd40, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #10: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783daa0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #11: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298abe30, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #12: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298abe30, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #13: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298abe80, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #14: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298abe80, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #15: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298abed0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #16: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298abed0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #17: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298ac140, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #18: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298ac140, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #19: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298ac5a0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #20: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298ac5a0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #21: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783d920, errors=0x00000001006f9360) at parser.c:1318:5
    frame #22: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298ac5f0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #23: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298ac5f0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #24: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783d7a0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #25: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298ac640, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #26: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298ac640, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #27: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298ac690, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #28: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298ac690, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #29: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c4a0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #30: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298ad9a0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #31: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298ad9a0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #32: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298ad9f0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #33: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298ad9f0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #34: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298adc70, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #35: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298adc70, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #36: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c3e0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #37: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298adcc0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #38: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298adcc0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #39: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c3a0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #40: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298add10, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #41: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298add10, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #42: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298add60, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #43: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298add60, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #44: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c320, errors=0x00000001006f9360) at parser.c:1318:5
    frame #45: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298addb0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #46: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298addb0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #47: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298ade00, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #48: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298ade00, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #49: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c2a0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #50: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298b6490, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #51: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298b6490, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #52: 0x0000000100018b90 herb`herb_visit_child_nodes(node=0x0000000a298b64e0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:65:11
    frame #53: 0x0000000100018920 herb`herb_visit_node(node=0x0000000a298b64e0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:13:5
    frame #54: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c220, errors=0x00000001006f9360) at parser.c:1318:5
    frame #55: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a298b6530, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #56: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a298b6530, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #57: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c1e0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #58: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29929270, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #59: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29929270, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #60: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2783c1a0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #61: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29929310, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #62: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29929310, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #63: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a28f13ca0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #64: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29929360, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #65: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29929360, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #66: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a28f23820, errors=0x00000001006f9360) at parser.c:1318:5
    frame #67: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982b70, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #68: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982b70, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #69: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a28f23960, errors=0x00000001006f9360) at parser.c:1318:5
    frame #70: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982bc0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #71: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982bc0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #72: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2884b860, errors=0x00000001006f9360) at parser.c:1318:5
    frame #73: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982d00, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #74: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982d00, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #75: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2884b980, errors=0x00000001006f9360) at parser.c:1318:5
    frame #76: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982d50, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #77: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982d50, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #78: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2884ba00, errors=0x00000001006f9360) at parser.c:1318:5
    frame #79: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982da0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #80: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982da0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #81: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2884bce0, errors=0x00000001006f9360) at parser.c:1318:5
    frame #82: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982df0, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #83: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982df0, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #84: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a2884bf60, errors=0x00000001006f9360) at parser.c:1318:5
    frame #85: 0x000000010001395c herb`match_tags_visitor(node=0x0000000a29982e40, data=0x00000001006f9360) at parser_match_tags.c:43:9
    frame #86: 0x00000001000188fc herb`herb_visit_node(node=0x0000000a29982e40, visitor=(herb`match_tags_visitor at parser_match_tags.c:9), data=0x00000001006f9360) at visitor.c:12:7
    frame #87: 0x0000000100014068 herb`match_tags_in_node_array(nodes=0x0000000a288d8f60, errors=0x00000001006f9360) at parser.c:1318:5
    frame #88: 0x00000001000143ac herb`herb_parser_match_html_tags_post_analyze(document=0x0000000a288b7000) at parser.c:1325:3
    frame #89: 0x0000000100003da0 herb`herb_analyze_parse_tree(document=0x0000000a288b7000, source="<!doctype html>\n<html data-n-head-ssr lang=\"de\" data-n-head=\"%7B%22lang%22:%7B%22ssr%22:%22de%22%7D%7D\">\n  <head >\n    <title>Offizielle Homepage | Hertha BSC</title><meta data-n-head=\"ssr\" charset=\"utf-8\"><meta data-n-head=\"ssr\" name=\"viewport\" content=\"width=device-width, initial-scale=1\"><meta data-n-head=\"ssr\" data-hid=\"description\" name=\"description\" content=\"Auf diesen Seiten finden sich alle Informationen zum Hauptstadtclub - Profis, Nachwuchs, eSport, soziales Engagement und vieles mehr!\"><meta data-n-head=\"ssr\" data-hid=\"robots\" name=\"robots\" content=\"index, follow\"><meta data-n-head=\"ssr\" vmid=\"og:title\" property=\"og:title\" content=\"Offizielle Homepage\"><meta data-n-head=\"ssr\" vmid=\"og:description\" property=\"og:description\" content=\"Auf diesen Seiten finden sich alle Informationen zum Hauptstadtclub - Profis, Nachwuchs, eSport, soziales Engagement und vieles mehr!\"><meta data-n-head=\"ssr\" vmid=\"og:image\" property=\"og:image\" content=\"https://content.herthabsc.com/site/binaries/_bsc_1695025561589/cont"...) at analyze.c:1308:3
    frame #90: 0x0000000100012d20 herb`main(argc=3, argv=0x000000016fdff4b8) at main.c:98:5
    frame #91: 0x000000019febdd54 dyld`start + 7184
(lldb) 

[marcoroth]

Thanks for reporting @timkaechele!

It would be interesting to compare it with the commit right before this #795. I suspect that this could have quite a big difference on large files 🤔

[timkaechele]

Tried it out:

Commit 676d0c0 still works.
Commit de456b8 I waited for 5 minutes but the command didn't finish yet.

[marcoroth]

Resolved via #840, that some document attached in the issue now took ~160 ms 🎉

❯ time exe/herb parse example.html --silent
Success

________________________________________________________
Executed in  195.77 millis    fish           external
   usr time  160.26 millis    0.18 millis  160.08 millis
   sys time   31.38 millis    1.07 millis   30.31 millis