[Core] Rocks

[EvESpirit]

This draft PR encompasses (or will encompass) the following additions/changes:

• New fast mesh-to-mesh collider system
• A "Scatterer" system (celbodies can be configured with scatter/mesh parameters, etc. See the newly edited Moon.cfg for schema)
  Meshes can be configured to use the new mesh-to-mesh collider, or they can be purely visual elements. More settings include a toggle for the scatterer system itself, a toggle for rocks casting shadows, a slider to configure rock density, scatter draw distance, and the individual sliders that define the percentage (expressed as 0.0-1.0) of the scatter draw distance which serves as a limit for drawing certain rock types (we may want 2km draw distance, but you're not going to spot tiny pebbles on even quarter that distance, so why ask the engine to draw them).

The scatter has phenomenal performance thanks to the newly implemented batched draw calls.

Besides the TODOs listed here, some more things that ought to be addressed before a potential merge is mainly this:

There exist planetary geology-grade moon rock distribution maps. Extract data from them to create a probability distribution map to mirror real life conditions as closely as possible. Other planets with rock scatter need hand-authored maps. This ought to be in the form of yet another quadtree structure. The schema for this is already designed, just not implemented:

Red Channel (Density / CFA) - The exact probability/abundance of rocks per square meter. 0 = No rocks. 255 = Maximum density.
Green Channel (Size Distribution) - The mean size of the boulders. Fresh Copernican craters have massive house-sized boulders. Older Imbrian surfaces have been pulverized into smaller cobbles by micrometeorites. 0 = Gravel/Small pebbles. 255 = Massive boulders.
Blue Channel (Geological Unit) - ID "system" to tell the scatterer what 3D model to spawn. 0-50 = Jagged, unweathered Anorthosite. 51-100 = Vesicular Basalt (Maria).* 101-150 = Breccia (Impact Ejecta).

Other than that, I'm quite happy with this. Below is the latest playtest video. Not much has changed since then, spent the whole time hunting down a silent crash (fixed as of this "initial commit", feel free to build)

[EvESpirit]

Fixed the easiest TODO from the original commit (and removed it from the original comment): Textures on the rocks now look good and are actually baked, alongside with a sample mesh, into the core.

[EvESpirit]

Fixed up the UI TODOs while I was there.

[TheGondos]

This looks good, maybe it could be used for trees on earth too?

[EvESpirit]

This looks good, maybe it could be used for trees on earth too?

Yeah absolutely. It was chiefly for rocks, but the way this is written very neatly allows for foilage too. _Small meshes can be flowers or shrubs, _Medium can be bushes, _Large can be trees, and the still yet to be implemented density map works great for pulling some earth GIS data about green coverage to have that be accourate as well. My only real concern is that even the HD textures/elev maps treat green scenery as an "elevation change" which i can't really do anything about so all the greenery will always be give or take double tall.

[n7275]

I have tested and looked at the code, and this is really cool. Would the collision model be extendable to vessel-vessel, and could we add functionality for a "collision mesh", seperate from the visual model (think docking probes). I know that's a bit out of scope, but doesn't look to far afield from what you're doing.

The only thing I might question here is exactly where some of this stuff should be implemented. Right now, a lot of it looks like it's in the D3D9 client code. How much could be shifted back to the orbiter-core. I only ask because, while the D3D9 client is our only real modern graphics client, DX9 is very obsolete, and it would be nice not to need to re-implement in any new clients, and to have the functionality in head-less orbiter installs, work toward graphics-client-over-TCP, etc.

[EvESpirit]

Thank you! Yeah this is definitely extendable for vessel-vesse and vessel-base collisions. We can have a separate collision model, but I'm gonna have to design something that's one size fits all, since what I wrote here just uses the actual mesh triangles.

And yeah I've been thinking about where to put the code too. The mesh to mesh stuff is definitely something that I wanted to shift into core, and the only real reason anything else touches D3D9 is because Vulkan still seems like a marathon sprint away, and well, it was easier for me to mess with the graphical client instead of the core for a quick prototype (something that bit me in the ass as soon as I went "huh, this should have collision").

I'll work on another commit today or tommorow to hopefully do all this stuff in one go.

[EvESpirit]

I managed to do 90% of the decoupling work, as requested. Sadly got stuck on vessel-to-vessel collisions for a couple hours before it was suddenly 1 in the morning.

What's been done as of now:

• All responsibilites of the scatterer that weren't directly related to rendering have been removed from the D3D9 OVP, with only the bare minimum code left over, namely whether or not to draw scatter shadows (render distances for Small, Medium and Large rocks are still there, I'll still move that into Visual Settings with the next commit)

With that, the Options>Visual settings tab now looks like this:

I also took the free real estate in the Options>Physics settings tab:

• Vessel-to-Base collision has been enabled, as it's stable enough (for now). Kind of janky, will resolve before marking this whole thing as ready to pull

• Vessel-to-Vessel has some code, but it's grayed out in the physics settings tab and disabled in-simulation. I managed to fix the performance issues with it, but the issues with LANDED vs INFLIGHT vehicle states proved too much for my feeble brain. I blame 25 years of tech debt, idk.

• In the simulation itself, you can navigate to the Top Menu > Functions > D3D9 Collider Visuals. For now, this shows the cubical bounding box, not necessarily the collision mesh (which is supposed to technically be the actual visual rendered mesh. TODO - fix this to represent the actual collider no matter the shape.

Some major TODOs are obviously getting the Vessel-to-Vessel mesh collider working, as well as making the collisions themselves more physically accourate.
I also really want native support for box colliders and sphere colliders at least - it will give eventual collider authoring (even if procedural) more knobs to turn to be "good". The mesh collisions the way I've implemented them are quite performant, but I see no reason why not to add genuine first class support for even faster methods of collision resolution.

Other than that, I'm still yet to implement data about the moon rock distributions from the scientific dataset I've identified originally, as well as generalizing the authoring of density maps across celbodies. After all of these are done, the PR is ready.

Not publishing a build for this since nearly all changes are under the hood. If you really want to compile this just for the vessel to base collisions, you'll have to do it yourself, sorry.

Soon I ought to have everything implemented and ready, except the Moon GIS integration workflow/support.

[EvESpirit]

All TODOs from before are completely addressed, as stated before, sans realistic Moon GIS integration workflow/support, as well as (unfortunately for now) collider visualizations.
A bit of a chaotic push series since my last comment, but I had some issues with git. And talent.

The new performant, physically accourate vessel-to-vessel collision is demonstrated here. Can still be under some circumstances glitchy, but I consider this ready. Any issues from this point on are gonna be knob-turning. The toggle for vessel-to-vessel collisions has been ungrayed.

Since my comments on the math are sparse, I would like to take a moment to document it here - since this is likely to be the first place anyone would go once this is merged to get some clarity into the code. This is a whole bunch of stuff that's in the maths aether, and has been for a very, very long time. I'm not this smart, someone else is, and it just so happens that them being smart makes my code run stupidly fast, where i had drops from 400 down to 15 during collisions before. So bear with me.

In very short, we do a point-to-triangle intersection algorithm for detection, and a standard Newtonian Rigidbody impulse solver for the reaction. The point to triangle algo is, mathematically at least, equivalent to the Möller-Trumbore triangle-intersection algorithm, except solved for a point instead of a vector.
Möller-Trumbore is like giving the code a laser pointer and asking if the ray hits a triangle, whereas what we do here is basically giving the code a point in 3D space and asking if it's currently shoved inside any particular triangle.

You still work me? Okay, good. The rest is simple.

We do vessel-to-vessel collisions in two phases to keep performance high - the broad phase, which skips checking complex geometry. The engine checks if the vessels' bounding spheres are overlapping or will overlap this frame. This is done with some dot product math of their relative position and velocity vectors to quickly discard vessels that are moving away from each other. Some of this is SIMD-able - I'm not going to bother.
Then we enter the narrow phase. If the broad-phase detects a potential hit, it samples a cached "point cloud" of vertices from the hill cache from Vessel A, and then transforms them into the local 3D space of Vessel B's meshes (and vice versa). It performs an AABB check (which stands for Axis-Aligned Bounding Box, for us semi-mortal folk), followed by barycentric coordinate math to see if any point from A has penetrated the face of a triangle in B.
It keeps track of the deepest penetration depth and the surface normal.

Once we actually have a hit, we do a bunch of really boring that aren't worth going into even here (or, are at least better commented than the math to do them). In short, velocity at contact point (includes angular momentum!), tangential friction and a slight positional correction all work together to make it feel like immovable (bases, rocks) or movable (vessels) objects. The math to calculate the normal impulse magnitude is especially gnarly:

Sorry for the jumpscare. All this does in practice for us here is that it calculates exactly how hard the vessels push away from each other along the collision normal (hence the name). We then use a standard linear and angular formula to apply the calculated impulse.

See? Even this is far too many words. For those of you that didn't fall asleep yet, all TODOs are formally done, except, again, the GIS and the collider visualizations. I'll get to it next. Hope you all enjoy this one. I'm nearly done with this.

Oh, and one more thing: Base .cfgs can now optionally configure mesh entries with the marker "COLLISION", which will make it use Vessel-to-Base collision on that particular base mesh. No define = old behavior, nothing breaks. For now I only did it on Brighton Beach (on everything except the landing pads - ocassional odd behavior). As an example:

HANGAR
POS 0 0 0
ROT 90
SCALE 12 8 244
TEX1 concrete1 0.4 0.4
TEX3 concrete1 0.5 8
COLLISION
END

[EvESpirit]

Two extra self-authored meshes and textures. Textures are clone of the original texture, I intend to make more of them. Also, to make more pointy rocks! This will do for now.

[n7275]

All TODOs from before are completely addressed, as stated before, sans realistic Moon GIS integration workflow/support, as well as (unfortunately for now) collider visualizations. A bit of a chaotic push series since my last comment, but I had some issues with git. And talent.

The new performant, physically accourate vessel-to-vessel collision is demonstrated here. Can still be under some circumstances glitchy, but I consider this ready. Any issues from this point on are gonna be knob-turning. The toggle for vessel-to-vessel collisions has been ungrayed.

Since my comments on the math are sparse, I would like to take a moment to document it here - since this is likely to be the first place anyone would go once this is merged to get some clarity into the code. This is a whole bunch of stuff that's in the maths aether, and has been for a very, very long time. I'm not this smart, someone else is, and it just so happens that them being smart makes my code run stupidly fast, where i had drops from 400 down to 15 during collisions before. So bear with me.

In very short, we do a point-to-triangle intersection algorithm for detection, and a standard Newtonian Rigidbody impulse solver for the reaction. The point to triangle algo is, mathematically at least, equivalent to the Möller-Trumbore triangle-intersection algorithm, except solved for a point instead of a vector. Möller-Trumbore is like giving the code a laser pointer and asking if the ray hits a triangle, whereas what we do here is basically giving the code a point in 3D space and asking if it's currently shoved inside any particular triangle.

You still work me? Okay, good. The rest is simple.

We do vessel-to-vessel collisions in two phases to keep performance high - the broad phase, which skips checking complex geometry. The engine checks if the vessels' bounding spheres are overlapping or will overlap this frame. This is done with some dot product math of their relative position and velocity vectors to quickly discard vessels that are moving away from each other. Some of this is SIMD-able - I'm not going to bother. Then we enter the narrow phase. If the broad-phase detects a potential hit, it samples a cached "point cloud" of vertices from the hill cache from Vessel A, and then transforms them into the local 3D space of Vessel B's meshes (and vice versa). It performs an AABB check (which stands for Axis-Aligned Bounding Box, for us semi-mortal folk), followed by barycentric coordinate math to see if any point from A has penetrated the face of a triangle in B. It keeps track of the deepest penetration depth and the surface normal.

Once we actually have a hit, we do a bunch of really boring that aren't worth going into even here (or, are at least better commented than the math to do them). In short, velocity at contact point (includes angular momentum!), tangential friction and a slight positional correction all work together to make it feel like immovable (bases, rocks) or movable (vessels) objects. The math to calculate the normal impulse magnitude is especially gnarly:

Sorry for the jumpscare. All this does in practice for us here is that it calculates exactly how hard the vessels push away from each other along the collision normal (hence the name). We then use a standard linear and angular formula to apply the calculated impulse.

See? Even this is far too many words. For those of you that didn't fall asleep yet, all TODOs are formally done, except, again, the GIS and the collider visualizations. I'll get to it next. Hope you all enjoy this one. I'm nearly done with this.

Oh, and one more thing: Base .cfgs can now optionally configure mesh entries with the marker "COLLISION", which will make it use Vessel-to-Base collision on that particular base mesh. No define = old behavior, nothing breaks. For now I only did it on Brighton Beach (on everything except the landing pads - ocassional odd behavior). As an example:

HANGAR POS 0 0 0 ROT 90 SCALE 12 8 244 TEX1 concrete1 0.4 0.4 TEX3 concrete1 0.5 8 COLLISION END

Here's the build for testing.

This is a very nice writeup. Would you be willing to add something to the technotes too? This would be a handy reference to have. (there are probably a few other documentation things that we need to add too).

[EvESpirit]

Thanks, do you mean doxygen or in-code comments or something else?

[n7275]

Thanks, do you mean doxygen or in-code comments or something else?

I ment more like what's under Doc/technotes

https://github.com/orbitersim/orbiter/tree/main/Doc%2FOrbiter%20Technical%20Reference

I think there's a section in the manual for config parameters too.

Let me know if I can help.

[EvESpirit]

All good, I was just wondering. I never really knew about the Technical Reference, just the user manual.

I looked at the relevant details more carefully and managed to cobble together a really rather nice "paper". I also added and updated some more references to the Orbiter User Manual. The whole thing is 337 lines of code and, within the PDF, lays out everything I summarized in my previous comment in dare I say excessive detail. Then again I suppose that was the intention when these were originally made, judging by the rest of it.

I'm now awaiting some input from Max-Q so I can start implementing the planetary GIS model.

[n7275]

I have done a bit of testing, more to follow. Docking ports will need some special consideration, in some cases unlocking can be a bit "chaotic"...

I'm also getting quite jittery interactions with vessels in orbit, almost like it allows too much peniteation and then the force vectors are in the wrong direction. Have not dug into too deep yet. Are you taking into account Orbiter's left handed coördinate system in all cases?

[EvESpirit]

Fixed.

[n7275]

Fixed.

Awesome. I shall test when I get home tonight. Looks like it was a coordinate system/global-local frame thing?

[EvESpirit]

There were a couple more issues than that but that was the main one yeah

[EvESpirit]

What's been done as of this comment:

• The most important channel of the real-data pipeline: the rock density maps derived from the best LRO data we have on the subject (original map res is 65x42k - this is now divided into tiles in a global quadtree and squished into only 200mb - down from a source of 4.xGB). These can be downloaded here.

• At some point a weird hard-sim crash bug was introduced within XRSound, most likely by me, that occurs when the DG tried to open the retrograde engine doors. That's now fixed too.

• Readjusted the Launchpad settings ranges for the new realistic data

We're getting very close to completion. What "needs" doing at the moment is simply to implement either one or both the other real data channels as originally discussed - however, I already forsee that our data quality for neither of those channels will be accourate enough to be worth bothering with. I might just try to hand-author them - I'll have to do that for all of the celbodies where we don't have half a billion DSLRs orbiting in a polar orbit, after all. Some good exogeology knowledge and even better NASA reference ought to provide some of the most realistic hand-authoring possible. I hope.

[n7275]

I'm still getting some odd behavior with vessel-vessel, and from observations alone, it looks like angular momentum may be applied in the wrong direction?

https://youtu.be/CzBLiwAYy0c

Vessel-Base and Vessel-Rock are very stable and seem to work as expected.

[EvESpirit]

ihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemathsihatemaths

anyway this should work I hope, I tested the same scenario many many times and tried many many things to fix the issue and I think I mostly have it? let me know

[TheGondos]

I notice your new files are dual licensed under GPL/LGPL. Orbiter is under MIT right now, not sure what this implies.

[EvESpirit]

Ah, I forgot about that. My usual go-to licenses are a bit more... crass (and permissive), so I didn't really know or care about what to license it under. I'll change it when I can. Mixing these licenses seems like a really bad idea.

[EvESpirit]

I'll leave the OVP part under the dual license since this is the precedent that Schweiger and Nikkanen established. The core part is marked as MIT.

[EvESpirit]

Okay, I know one thing well, and that's when to admit defeat. After wrestling with my novel implementation for even some time after my last post, I realized that it would always end up being inferior to other solutions even if perfected purely because of the gap problem. This is an initial commit for an entirely new, much more robust foundation that I think is as perfect as it'll realistically get while staying performant. My original implementation of the collision system was faster than this one is, but the difference is on the order of single %, and that's a loss that's perfectly acceptable given the collision quality.

Note that this still isn't finalized - as I mainly focused on vessel-to-vessel collisions in orbit and docking, both of which work flawlessly. Stuff that's known not to work is vessel-to-base and most likely also vessel-to-scatter - though the latter isn't tested. Within the next couple of days I'll fix that, since the foundation has already been verified to work well at this point. It's possible there are some other broken things, so just treat this one as a proof of concept.

The main concepts used for this new foundation is GJK/EPA (industry standard, documented collision models for complex shapes) coupled with a custom, what I like to call "convex slicer", which exists to achieve a very high collider fidelity to the actual visual mesh while maintaining very high performance. The concept itself isn't new, but the code here is. I'll rewrite the collision section of the Technical Reference with the next push, which will also contain the polish required to call this whole thing finished.

Here's some sample footage from one of the validation runs: