Unexpected Interaction Between Lines and Mesh Transparency Rendering

[qdetweiler]

Summary

When a Mesh with MeshStyle::transparencyBin = true is present in the scene and there is also a visible Line entity, the apparent opacity of the mesh is inconsistent across its faces so that the appearance of the mesh differs depending on the viewing angle, even when the line does not intersect the mesh. If the Line entity is marked invisible via a Visibility component, the same mesh is rendered with uniform transparency across its faces and a consistent appearance from all viewing angles.

---

Minimal Reproduction (rocky_demo_qt)

The reproduction is implemented in src/apps/rocky_demo_qt/rocky_demo_qt.cpp as follows:

• The demo creates a pair of overlapping square-based pyramids using MeshGeometry and Mesh. The pyramids are partially transparent (alpha = 0.4) and use a MeshStyle with transparencyBin = true.
• Both pyramids share the same latitude/longitude. One is placed at an altitude of 500 km, and the second is placed at a slightly higher altitude so that the two volumes overlap.
• The demo also creates a single horizontal Line (via LineGeometry, LineStyle, and Line) positioned north of the pyramids at latitude 15° so that it does not intersect or overlap the pyramids.
• A Visibility component is attached to the line entity, and its visible flags are set based on the ROCKY_DEMO_LINE_VISIBLE environment variable. When ROCKY_DEMO_LINE_VISIBLE is unset, the line is present in the ECS but not rendered (invisible). When the variable is set, the same line is rendered.

Here is what the mesh looks like when the line is invisible (this is the expected rendered result):

Here is what the mesh looks like when the line is visible. Notice that the left half of the pyramid from this viewing angle appears more opaque than the right half.

---

How to Reproduce

1. Replace rocky_demo_qt.cpp with the modified version
   here and build the demo application.

2. Run without the environment variable:

   ./rocky_demo_qt

   • ROCKY_DEMO_LINE_VISIBLE is unset, so:
     • The line’s Visibility.visible[] flags are all false (line is invisible).
     • The line still exists in the ECS, but it does not render.
   • Observed behavior:
     • The overlapping pyramids appear with uniform transparency.
     • The apparent opacity does not change when orbiting the camera around them.

3. Run with the environment variable set:

   ROCKY_DEMO_LINE_VISIBLE=1 ./rocky_demo_qt

   • ROCKY_DEMO_LINE_VISIBLE is set, so:
     • The line’s Visibility.visible[] flags are all true (line is rendered).
   • Observed behavior:
     • The same pyramids, with the same MeshStyle and geometry, now show inconsistent transparency:
       • One side can appear more opaque than the other.
       • The perceived opacity of the overlapping region changes as you orbit the camera.
     • This occurs even though the line is spatially separate and does not intersect the pyramids.

---

Additional Experiments

To rule out geometry or style quirks, I tried variations on how the mesh is constructed and styled, including:

• Toggling MeshStyle::writeDepth on/off.
• Toggling MeshStyle::twoPassAlpha on/off.
• Toggling MeshStyle::drawBackfaces on/off.
• Changing winding order of the faces.
• Dual-winding faces (adding triangles in both directions).
• Duplicating vs. not duplicating vertices on the faces.

In all of these cases:

• With a visible line, the view-dependent transparency behavior described above persists.
• With the line present but invisible (via Visibility), the mesh transparency is stable and appears uniform across the mesh.

[gwaldron]

Thanks for the detailed post. A couple observations:

Your vertex winding on the base triangles of the pyramid is such that the faces are pointing up (towards the interior). That might not be what you intended.

Your screenshots tell me that two-pass alpha was OFF when you took them, but the code has it ON. I know you were tweaking the settings so maybe there's a mismatch. Nevertheless, 2-pass alpha works on a per-object basis. It does not affect the interaction between 2 separate objects (at least not in the current implementation...this may change down the line).

I was unable to get the visibility of the line to affect the mesh geometry. More likely is that the transparency bin is changing the depth-sorting of the two overlapping objects based on different camera positions. If you alter your example to allow runtime toggling of the various settings that might help.

There is only so much one can do with separate depth-sorted objects that overlap like this. One option is to combine the two pyramids into a single geometry that can benefit from 2-pass alpha. But then, I do not know what your application is. Hope this helps.

[qdetweiler]

Thanks for the feedback!

You are correct that I had the vertex winding on the bottom of the pyramid incorrect. Thanks for the catch.

The screenshots I sent were taken when running with a build of the application exactly as provided in the source code attached to this issue (I also double checked that again today). This means they were taken with 2-pass alpha turned ON on my system. Given that you couldn't reproduce the result, perhaps there is some environmental difference at play?

I am running with the following:
vulkan 1.4.328.1
vsg 1.1.13
vsgqt 0.4.0
qt5 5.15.18

Unfortunately, my runtime environment is somewhat complicated at the moment. I am building and running the software in a container image based on Rocky 10, and that container is hosted on a WSL Linux distro running Rocky 9. With this setup, Vulkan is stuck using software rendering via llvmpipe. Perhaps software rendering vs. hardware rendering is causing the behavior difference?

Any other ideas?

I don't have easy access to a simpler runtime environment at the moment, but perhaps standing one up is the next troubleshooting step for this issue?