Add KHR_interactivity draft

[lexaknyazev]

Rendered version: https://github.com/KhronosGroup/glTF/blob/interactivity/extensions/2.0/Khronos/KHR_interactivity/Specification.adoc

[bhouston]

Great work! What sections are next? Also what could we release as a draft extension for SIGGRAPH 2023?

[robertdorn83]

@lexaknyazev
Currently the readonly pointer for activeCamera contains only position and rotation. Would it make also sense to add matrix to it ? I know we can build a matrix out of position and rotation because its global space, but just for consistency with the node transforms.

Also, and we already talked about partly when i remember correctly, but it would be also more consistent to add the other gltf camera properties such as activeCamera.orthographic.xmag, activeCamera.orthographic.zfar, activeCamera.perspective.aspectRatio etc. These are all known standard gltf properties.
Also maybe adding activeCamera.isOrtho, activeCamera.viewMatrix, activeCamera.projectionMatrix.
My wishlist for christmas this year ;)

[CLAassistant]

All committers have signed the CLA.

[robertdorn83]

@lexaknyazev
hey alexey, why have you added a "ref event" output on the onStart and onTick event? I have nothing in mind why we could need a reference to the event here. In Unity - and im sure in a lot of other engines too - such events don't even have a Event-Object to pass through.

[robertdorn83]

@lexaknyazev Some questions about the ref types:

• I wasn't able to find some information how to resolve a Ref (let's say a Node) to the graph node index. (can also be usefull to check if a ref belongs to the own gltf file)
  would it be with
  pointer/get (template: /node/{nodeRef}/index, type: int)
  or
  pointer/get (template: /node/{nodeRef}/, type: int) ?

• what when i want to get a mesh from a node: is /node/ []{} /mesh returning a index or ref now? Or depends it now how i setup the type in the node config? so type=Ref would give me the ref to the mesh, type=Int would give me the index? But the index would only work/is usefull, when the mesh belongs to the same gltf file as the graph.

• initial Values for Variables from type Ref or Input Sockets from type Ref: so we use here as a value just a string with /node/6/?

These pointer access cases and ref initial values seems to bemissing in the specs currently .. ?

[lexaknyazev]

how to resolve a Ref (let's say a Node) to the graph node index

That's impossible. Only the opposite direction is currently supported.

what when i want to get a mesh from a node: is /node/ []{} /mesh

All such pointers return refs, the Object Model is going to be updated.

initial Values for Variables from type Ref or Input Sockets from type Ref

Their syntax and semantics are defined in the Variables section (currently 5.3). When not set, the value is null; otherwise it must be a static (no parameters) JSON pointer string that will be resolved to a runtime reference value (if resolvable) or to null (if unresolvable) when the graph is loaded.

[robertdorn83]

I updated the Samples and Tests with the new Ref spec changes. They are currently in a separate branch: gltf Interactivity Assets
Was e bit more work than expected, so sorry for the delay :)

[robertdorn83]

@lexaknyazev

can you add some pointer samples to get Ref-Objects via json pointers? like pointer/get > /animations/0, to get the animation reference.

and: is "/animations/0" the correct format to get the reference from animation-index 0? or would it be "/animations/0/"?