Viewer: add a way to match POV of other cameras (from gltf file loaded for example)

[cournoll]

This change introduces a way to interpolate the viewer's default camera to other existing cameras in the scene. The primary goal of this feature is to enable matching the POV of cameras defined in a GLTF file, without switching the active camera to try to keep a "simple" experience for non-expert users.

• _cameraToHotSpot function compute the alpha, beta, and radius angles, as well as the target point for a camera that is not an ArcRotateCamera.
• For target selection, I used a straightforward approach, the target point is determined based on the camera's forward ray. If an intersection with the meshes from AssetContainer is found, the first hit point is used as the target. If no intersection is detected, a fallback target is calculated by projecting the distance between the camera and the AssetContainer world center along the forward ray.

[bjsplat]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[bjsplat]

Snapshot stored with reference name:
refs/pull/16076/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16076/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16076/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/16076/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/16076/merge
https://gui.babylonjs.com/?snapshot=refs/pull/16076/merge
https://nme.babylonjs.com/?snapshot=refs/pull/16076/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/16076/merge#BCU1XR#0

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16076/merge/testResults/webgl2playwright/index.html

[bjsplat]

Visualization tests for WebGPU (Experimental)

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16076/merge/testResults/webgpuplaywright/index.html

[ryantrem]

I was thinking from our initial discussions that we would simply translate these cameras directly to hotspot entries on the element (rather than adding a new cameras concept). Any reason not to go that route? If we do, then I think a few changes to this PR would be good:

1. getArcRotateCameraInfos can be a private function in ViewerElement (just an internal implementation detail for now).
2. ViewerDetails could be updated such that the model property returns an object of type Model rather than AssetContainer (breaking change on Viewer, but I still think this is ok since it is @experimental and lower level with minimal current use outside of ViewerElement). Alternatively we could just add more properties to ViewerDetails that are model specific, like the model's bounding center point.
3. If we only want hotspots for the active model, then in ViewerElements onModelChanged handler, add to the hotspot collection camera-based hotspots for that model (and remove any previous camera-based hotspots).
4. If we want hotspots for all cameras from all loaded models, then I think it is a little trickier because we need to add camera-based hotspots any time a model is loaded and remove them any time a previously loaded model is removed from the scene.

Which behavior between #3 and #4 are you guys aiming for?

[cournoll]

I was thinking from our initial discussions that we would simply translate these cameras directly to hotspot entries on the element (rather than adding a new cameras concept). Any reason not to go that route?

Not really any specific reason. Initially, during development, I wanted a distinct visual indication so the user could understand that this behavior in the viewer was tied to the cameras in the scene. But once I finished my code and realized that focusing on a hotspot or matching a camera’s POV ultimately relies on the same principle (an interpolation) I understood that we could merge the two. :)

That said, I’m still wondering how we could clarify for users where the names and values of these "undeclared hotspots" originate.

[cournoll]

1. `getArcRotateCameraInfos` can be a private function in `ViewerElement` (just an internal implementation detail for now).

I had a similar feeling about that, so it works perfectly for me.

2. `ViewerDetails` could be updated such that the `model` property returns an object of type `Model` rather than `AssetContainer` (breaking change on `Viewer`, but I still think this is ok since it is @experimental and lower level with minimal current use outside of `ViewerElement`). Alternatively we could just add more properties to `ViewerDetails` that are model specific, like the model's bounding center point.

Yes, yes, yes! Having the bounding info in ViewerDetails would be very nice for the user. Got it, I’ll add that to this PR.

3. If we only want hotspots for the active model, then in `ViewerElement`s `onModelChanged` handler, add to the hotspot collection camera-based hotspots for that model (and remove any previous camera-based hotspots).

4. If we want hotspots for all cameras from all loaded models, then I think it is a little trickier because we need to add camera-based hotspots any time a model is loaded and remove them any time a previously loaded model is removed from the scene.

We will aiming for behavior 4, so i'll try to do something that stay consistent with the Babylon viewer approach and let us fit with our needed 👍

[ryantrem]

We will aiming for behavior 4, so i'll try to do something that stay consistent with the Babylon viewer approach and let us fit with our needed 👍

Ok, I guess in this case we can just check Scene.cameras and watch Scene.onNewCameraAddedObservable and Scene.onCameraRemovedObservable. I think we should probably also have a new property/attribute like cameras-as-hotspots that is disabled by default (e.g. you have to explicitly opt into this behavior).

[ryantrem]

A few thoughts on this new type:

1. We don't expose Vector3 directly for other parts of the interface (like hotspots). Could we switch these to [x: number, y: number, z: number] like we have elsewhere?
2. Make it readonly either in the definition or in the usage (since it is a stored/persistent value, not a transient value that is just returned from a function with a single usage)? I think definition would be ok, but either works.
3. To make this more re-usable (in case in the future we want to use it for something not in world space, or for something other than models), could we rename to not include "world" or "model"?

Suggested change

	export type ModelBoundingInfo = {
	worldExtents: {
	min: Vector3;
	max: Vector3;
	};
	worldSize: Vector3;
	worldCenter: Vector3;
	};
	export type BoundingInfo = {
	extents: Readonly<{
	readonly min: readonly [x: number, y: number, z: number];
	readonly max: readonly [x: number, y: number, z: number];
	}>;
	readonly size: readonly [x: number, y: number, z: number];
	readonly center: readonly [x: number, y: number, z: number];
	};

And then in the usage below, have worldBounds?: BoundingInfo

[cournoll]

Yes good changes! For the type name, aren't you worried it might overlap with the existing BoundingInfo class in core/Culling ?

[cournoll]

Though the user can always do an import as.

[ryantrem]

That's a good point, it could be confusing. We could call it ViewerBoundingInfo or something?

[ryantrem]

Instead of having this function mutate this._modelInfo, can we:

1. Move it out to a utility function (within this file, but not inside the class) that takes an AssetContainer and returns a BoundingInfo
2. Call it in loadModel and set the bounding info for the model when it is loaded
3. Make the worldBounds property of the ModelInfo part of the readonly section (since it will always be available as it would be computed when the model loads)

[cournoll]

Indeed, we can move it out to a utility function. However, I don’t think we can move worldBounds to the readonly section because it needs to be refreshed when another animation is selected (as the camera relies on it for proper placement). Alternatively, we could compute all the worldBounds for every animation when the asset loads.

[ryantrem]

I'm a little worried about this because I can see scenarios where someone reads the hotSpots property on every frame tick for doing custom hotspot visualizations, and this would cause a lot of GC. Can we have just a single instance, and modify the instance when either a camera is added/removed or when the camerasAsHotspots property changes?

[ryantrem]

I think it could be confusing that the world bounds for a particular animation are not available until _selectAnimation is called. I think instead we should change the Model type slightly such that instead of a worldBounds property that is an array, have a getWorldBounds(animationIndex: number) function, then what we can do here is something like:

const worldBounds: BoundingInfo[] = [];

return {
  assetContainer,
  ...,
  getWorldBounds: (animationIndex: number) => {
    if (!worldBounds[animationIndex]) {
      worldBounds[animationIndex] = computeBoundingInfos(assetContainer, animationIndex);
    }
    return worldBounds[animationIndex];
  },
};

This way the computation is still cached, but anyone can access the world bounds any time (not dependent on a previous call to _selectAnimation).

[ryantrem]

From my other comment, maybe this will be removed, but if we need to keep it, it seems like it should be private.