Organize runtime initialization

[straight-shoota]

Runtime initialization is currently spread over a number of different locations in the prelude, each executing at different times.

For example, Crystal.init_runtime only initialize Crystal::Once as well as the Fiber and Thread lists.

crystal/src/crystal/main.cr

Lines 53 to 60 in 8365f55

	# :nodoc:
	def self.init_runtime : Nil
	# `__crystal_once` directly or indirectly depends on `Fiber` and `Thread`
	# so we explicitly initialize their class vars, then init crystal/once
	Thread.init
	Fiber.init
	Crystal::Once.init
	end

kernel.cr initializes execution context / scheduler, callstack and signal handlers in top-level user code:

crystal/src/kernel.cr

Lines 602 to 627 in 8365f55

	{% unless flag?(:interpreted) || flag?(:wasm32) %}
	{% if flag?(:execution_context) %}
	Fiber::ExecutionContext.init_default_context
	{% else %}
	Crystal::Scheduler.init
	{% end %}
	# load debug info on start up of the program is executed with CRYSTAL_LOAD_DEBUG_INFO=1
	# this will make debug info available on print_frame that is used by Crystal's segfault handler
	#
	# - CRYSTAL_LOAD_DEBUG_INFO=0 will never use debug info (See Exception::CallStack.load_debug_info)
	# - CRYSTAL_LOAD_DEBUG_INFO=1 will load debug info on startup
	# - Other values will load debug info on demand: when the backtrace of the first exception is generated
	Exception::CallStack.load_debug_info if ENV["CRYSTAL_LOAD_DEBUG_INFO"]? == "1"
	Exception::CallStack.setup_crash_handler
	{% if flag?(:win32) %}
	Crystal::System::Process.start_interrupt_loop
	{% else %}
	Crystal::System::Signal.setup_default_handlers
	{% end %}
	{% end %}
	{% if flag?(:interpreted) && flag?(:unix) && Crystal::Interpreter.class.has_method?(:signal_descriptor) %}
	Crystal::System::Signal.setup_default_handlers
	{% end %}

The odd thing is that this top-level user code is defined somewhere in the alphabetically-sorted require list from prelude.cr. It happens to work, but it would fail if we try to use IO in top-level code from any of the previously required files.

It would be nice to define the runtime startup sequence a bit more explicitly, and document the exact conditions for hooking into Crystal.main and Crystal.main_user_code.

There are use cases that require to run code at process startup when it is guaranteed to be single-threaded to avoid any threading issues (ENV modifications, fork). They might require different levels of runtime initialization. For the former, a working event loop would be nice, for example.

We should probably move all stdlib initialization code into methods instead of top-level user code. We can call these methods explicitly from Crystal.main/Crystal.main_user_code in order to establish a clear init sequence.

---

Add a 👍 reaction to issues you find important.

[straight-shoota]

Not sure if it's related: but a top level raise before fiber.cr and crystal/main.cr are included results in linker errors:

ld.lld: error: undefined symbol: main
>>> referenced by /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/Scrt1.o:(_start)

ld.lld: error: undefined symbol: _fiber_get_stack_top
>>> referenced by fiber.cr:143 (/mnt/wsl/src/crystal-lang/crystal/src/fiber.cr:143)
>>>               F-iber.o0.o:(*Fiber#initialize<Pointer(Void), Thread>:Nil)
collect2: error: ld returned 1 exit status

[yxhuvud]

One thing that may be related is the post-main callback that specs depend on.

Also: GC initialization.

There is also constant initialization 🤔

[straight-shoota]

Good points!

GC and Crystal::Tracing are already initialized right before Crystal.init_runtime:

crystal/src/crystal/main.cr

Lines 37 to 38 in 8365f55

	{% if flag?(:tracing) %} Crystal::Tracing.init {% end %}
	GC.init

I believe constants are initialized in the order they are declared in top-level code, or if a read happens before the declaration, it initializes on demand via Crystal::Once.

[ysbaddaden]

For the record:

• Crystal.init_runtime is a recent addition. It forces some globals to be initialized before we can initialize all the other globals (constants, class variables), namely for thread, fiber and crystal/once.

• Tracing and GC should be initialized in Crystal.init_runtime 👍

• All the above MUST happen before you can do anything that isn't strictly C level.

• Initializing globals is messy and likely needs rework, see https://gist.github.com/ysbaddaden/78a885403bdbd51048fa4b2763b91266 for details.

• We could consider moving some initialization from src/kernel.cr to Crystal.init_runtime but that will clash with any global that they depend on (and will introduce a read before init).

Maybe a __crystal_init_globals fun could be beneficial, but maybe it would be plainly artificial, since we don't control the flow (a complex initializer can spawn, yield, trigger evloop, ...) and not much different from cramming it all into __crystal_main (aka Crystal.main_user_code)... yet it would allow runtime globals to be pre-initialized, and allow to defer execution of code from src/kernel.cr and others.

Note:

• We can't have an event loop without an execution context: an event loop is attached to a context (and sometimes a scheduler). We must initialize the EC for the main thread to initialize its EV. It has always been that way (it used to be lazy, now it's explicit).

[ysbaddaden]

I.e.

1. Crystal::Tracing.init: required before anything else (LibC, no GC)
2. GC.init: required before we can allocate (LibC)
3. Crystal.init_runtime: required before we can init globals (LibC, GC)
   1. init thread
   2. init fiber
   3. init crystal/once
4. Crystal.main_user_code: misnomer, actually constant init (can call user code) -> primitive/std -> user code
   Initialization of class variables is spread out everywhere in Crystal.main_user_code (as they come).