Performance: pick from globe depth when possible

[mzschwartz5]

Feature

It's no secret that globe picking is slow. We should soon have some big improvements for globe picking, but for dense terrain data (like ArcGIS), it will still be relatively slow (at least, the first pick on each tile can be slow, but subsequent picks get faster. Still, when panning the camera around, new tiles constantly load in, and cause slight lag as the camera does slow picks to avoid collisions).

It's important to understand the difference between globe.pick and GlobeDepth. The former uses CPU-side vertex data and intersection tests to find where an arbitrary ray intersects the globe. The latter can be used to do something similar, but via a readback of the GPU globe depth buffer - and, by nature, only works for rays from the camera, not for arbitrary rays.

Now, because globe.pick works for arbitrary rays, some code (including the camera class) uses it, when they actually only need the special case of picking along a camera ray. In these cases, it may be wise to read back from the depth buffer instead.

Proposal: allow globe.pick to read back from the depth buffer instead of doing CPU-side picking. This could be via a flag, by checking if the input ray is a camera ray (same origin + direction within frustum), or just a new method entirely, to name a few options.

[mzschwartz5]

I took this performance measurement just panning around close to the surface using ArcGIS terrain. As you can see, the vast majority of the time is spent picking - but it's all ultimately coming from the camera. So, instead of doing CPU-side terrain picks, I'm proposing replacing this with depth buffer picks.

[Beilinson]

getGlobeHeight samples the globe using a ray from the cartographic position towards the ellipsoid surface, am I wrong that this ray only lines up with the GlobeDepth when the camera is looking straight down at the terrain (90 degrees)?

This might still be useful for distance from camera POV to surface, which we use quite a lot to create pseudo-zoomLevel variable, similar to the kind that exists on other map libraries (https://maplibre.org/maplibre-gl-js/docs/API/classes/Map/#getzoom)

[mzschwartz5]

Yes, I think that's right; my write up is a little inaccurate there. However, as long that ray ("from the cartographic position towards the ellipsoid surface") is in the camera's frustum at all, that means the globe depth is rendered and we can still sample the depth texture.

When the ray is out of the frustum (I.e. the camera isn't looking down at the globe even a little), we could fallback to CPU-side terrain picks. And it should be a relatively cheap computation to determine if the ray is within the camera's frustum.

Also, as you said, still useful for "distance from camera to surface" queries.

[mzschwartz5]

I should also mention that, for simple terrains like CWT, depth texture read backs might very well be slower than CPU terrain picks. However, the time to read the depth texture is independent of the complexity of the terrain, so it puts a nice maximum bound on the time to do a pick. A small sacrifice for CWT, for a potentially large speed up on ArcGIS and others.

[Beilinson]

I think its whenever the pitch is any more than around 30-45 degrees off the Y axis (haven't done the math). In the above image for example the surface below the camera cartographic isn't being rendered despite a pretty pitch.

Still think this could bring us a lot of value in our app for the zoomLevel logic, even allowing us to get a zoomLevelsDisplayed type behavior by reading a vertical strip of the depth buffer. I do want to note that adding another buffer read in the middle of rendering could slow down general rendering when not using terrain, also would this API read only the relevant pixel each time (so multiple picks would cause many reads in the same frame), or the entire bufffer (non-minor impact every frame)?

[mzschwartz5]

I think its whenever the pitch is any more than around 30-45 degrees off the Y axis

I haven't done the math either, but I'm sure it also depends on the FOV of the frustum. It's probably something like pitch > FOV

[mzschwartz5]

I do want to note that adding another buffer read in the middle of rendering could slow down general rendering when not using terrain, also would this API read only the relevant pixel each time (so multiple picks would cause many reads in the same frame), or the entire bufffer (non-minor impact every frame

Good questions. To the first point, I think the introduction of PBOs and async texture reading in #12983 would help with this concern. Of course, that's only for WebGL 2, but that's a pretty wide net these days.

To the second point, I was imagining this mostly being an optimization for camera operations, so we'd probably only want to read a given pixel or two each frame (pixel in camera's look direction, pixel straight down toward globe's surface)

[Beilinson]

I had two other ideas:

1. Similar to the current oct tree PR (and building on my quad tree comment there) build a 2d geospatial index using the bounding boxes of the projected triangles. This should result in even faster lookups with even fewer triangles (which could still be interpolated with vertical height exaggeration without rebuilding the index every time). Since terrain meshes are static, we could even use a static spatial index which is even faster (see https://github.com/mourner/flatbush)

2. Building up on the idea in this issue, maybe the best solution is to just rasterize all terrain meshes, rendering each tile to a heightmap texture when loaded, and then just sampling that texture directly for height? Vertical exaggeration could then be applied at the end and has no issue. Some custom providers like mapbox-dem for example already provide terrain in this texture format, and could be used directly. This is how maplibre gets such fast terrain height query performance.

Both of these ideas also benefit billboard clamping performance, and work for the camera regardless of the view frustum

[mzschwartz5]

Not fully sure I understand the 2d index idea, but I have some reservations about it not being rebuilt with exaggeration. Just be aware that vertical exaggeration is not an affine transformation. I know the name sounds like it's just height changing, but it's actually along geoid normals- which are mostly vertical, but not exactly. So the relative positions of projected triangles' bounding boxes may not stay the same under exaggeration.

Of course, locally, it's a pretty good approximation... so maybe that's a non-concern. (I had this same train of thought for the octree PR; just decided it was safer to rebuild than to find out 3 months down the road that there's a super niche edge case in one part of the globe)

[Beilinson]

That's a good point about the vertical exaggeration I hadn't realized that.

About my second point, take a look at https://github.com/davenquinn/cesium-martini. This does essentially the opposite of what I'm suggesting - converts a raster heightmap where each pixel encodes a height value to a terrain mesh.

However for height sampling the original height map texture is used because it is essentially an o(1) lookup time. So maybe the best idea here would be to generate on the gpu heightmap textures for each tile, and then sample that for height directly. I'm not sure how this lines up with the vertical exaggeration non-linearity you mentioned, but it could be worth exploring