Surprise OOM errors on my computer

[TurkeyMan]

This is kinda bad:

The semantic server is using an astonishing amount of memory. Way more than my browsers with hundreds of tabs open :/
I noticed because random apps started crashing complaining with out-of-memory errors.

[TurkeyMan]

An hour or so later, it was packing 24gb and things started crashing again.

[rainers]

Are you running the latest beta? I made a few tweaks to run the GC more frequently to avoid it allowing the used memory to grow so easily due to its grow policy. That worked to some extend for me, I haven't seen memory usage above 50 GB or similar anymore.

Do you have logging still enabled? dmdserver now also writes some GC stats after a collection into the log, do you see it actually using that much memory, or does the used heap just grow indefinitely.

BTW: the beta also has a new commend "Reset Language Server" to more easily get it into a sane state again.

[TurkeyMan]

Yeah running the lastest one. I recall you saying you did direct the log to an output window, but I couldn't find that, is there something I need to do to enable that?

[TurkeyMan]

What is even all the memory? The AST can't possibly be that huge can it? If it is, maybe using DMD isn't actually a great solution for language services. C++ works really well, and doesn't use lots of memory; I guess they do something custom like your old implementation?

[rainers]

As measured here dlang/dmd#21428 (comment), building all of phobos unittests in a single invocation, dmd needs about 20 GB, which is reduced to 12 GB when using the GC. Apart from a bit of backend data, this corresponds to analyzing import std for the semantic server (if unittests are enabled).

The AST itself should only make up a small part of that (though dmdserver keeps a copy of it for faster analysis restart if other modules change). AFAICT most of the memory is allocated during semantic analysis, doing template instanciations, lowerings and other transformations. Here dmdserver is also a bit worse than dmd as it keeps references to the original AST nodes to allow tracing back analysis to the source location. CTFE should not be that bad anymore, as it uses a reusable region allocator and only extracts results from that.

AFAICT Visual Studio uses cl.exe to generate intellisense data, but places it in some databases that you can find in the .vs folders. These can grow quite large aswell, but probably consolidate the data to what's actually necessary excluding all the intermediate data.

[rainers]

. I recall you saying you did direct the log to an output window, but I couldn't find that, is there something I need to do to enable that?

There is an option "Log semantic server communication" on the D intellisense option page. This logs the client side, though, i.e. what the Visual D plugin is seeing (including a bit of GC information). The dmdserver process log is still in %TEMP%\dmdserver only.