Gaussian splat streaming and residency
Scalable residency​
SplatResidencyManager limits GPU bytes, logical splat rows, or independently retained source
chunks. It preserves each original prepared batch and never repacks or concatenates GPU buffers.
const residency = new SplatResidencyManager({
maxGpuBytes: 256 * 1024 * 1024,
maxResidentSplats: 2_000_000,
maxResidentChunks: 128,
onResidencyChange: batches => renderer.setProps({data: batches})
});
residency.add(preparedTile, {id: tile.id, priority: tile.priority});
residency.pin(tile.id);
residency.touch(tile.id);
await residency.load(nextTile.id, () => loadPreparedTile(nextTile), {
priority: nextTile.priority,
estimatedGpuBytes: nextTile.gpuByteLength,
estimatedSplatCount: nextTile.rowCount,
ownsData: true
});
Higher-priority chunks displace lower-priority chunks; equally prioritized chunks use
least-recently-used eviction. Pinned chunks remain resident until explicitly removed. The manager
destroys a batch only when ownsData transfers ownership explicitly. Renderer residency callbacks
run before manager-owned evicted buffers are destroyed, allowing borrowing renderers to detach
their batches safely. Supply estimatedGpuBytes and estimatedSplatCount when loading so eligible
resident chunks are evicted before a new batch allocates GPU memory; without estimates, budgets
bound retained resident allocations but cannot prevent a temporary upload spike.
Hierarchical paging and foveated level of detail​
SplatHierarchyManager selects an active frontier from source-owned page metadata. Nodes carry
world-space bounds, geometric approximation error, independent source identities, optional content
URIs, and caller-supplied asynchronous page decoders:
import {SplatHierarchyManager} from '@luma.gl/engine/splats';
const hierarchy = new SplatHierarchyManager({
roots: sourceTileRoots,
residencyBudget: {
maxGpuBytes: 256 * 1024 * 1024,
maxResidentSplats: 1_000_000,
maxResidentChunks: 32
},
maximumScreenSpaceError: 8,
maxConcurrentLoads: 4,
loadPage: async (node, {signal}) => decodeSourcePage(node.contentUri, {signal}),
onFrontierChange: batches => graphRenderer.setProps({data: batches})
});
hierarchy.update({
cameraPosition,
modelViewProjectionMatrix,
viewportSize: [width, height],
foveation: {center: [0.5, 0.5], radius: 0.2, strength: 2}
});
await hierarchy.waitForIdle();
console.log(hierarchy.frontier, hierarchy.stats, hierarchy.residencyManager.stats);
Traversal conservatively culls bounding spheres, computes projected screen-space error, and
prioritizes pages near the current gaze position. Replace-refined parents remain visible and
pinned until every visible child is resident; additive refinement retains parent detail. Requests
use bounded decoder concurrency, abort work that leaves the current view, and forward each page's
estimatedGpuBytes and estimatedSplatCount to pre-upload residency reservations. Source pages,
compressed payloads, worker scheduling, RAD parsing, and 3D Tiles transport remain application-
or loader-owned. Prepared pages remain caller-owned by default; set node.ownsData: true when
the hierarchy should destroy an asynchronously decoded page on eviction or shutdown. An externally
supplied residency manager is always borrowed and must be destroyed by its original owner.
Spark RAD row hierarchies​
Spark RAD hierarchy links belong to individual source rows, not whole source pages.
SplatRADHierarchyManager preserves this distinction using the original page-local childCounts
and source-global childStarts arrays:
import {GPUPagedSplatRenderer, SplatRADHierarchyManager} from '@luma.gl/engine/splats';
const renderer = new GPUPagedSplatRenderer(device, {viewportSize: [width, height]});
const hierarchy = new SplatRADHierarchyManager({
pageSize: 65_536,
maximumActiveRows: 1_000_000,
residencyBudget: {maxResidentSplats: 1_000_000},
lodSplatScale: 1.5,
lodRenderScale: 1.5,
lodOpacity: true,
coneFov0: 70,
onFrontierChange: frontier => renderer.setFrontier(frontier),
onPageRequest: request => scheduleSourcePage(request.rowIndex, request.priority),
onPageCancel: request => cancelSourcePage(request.rowIndex)
});
hierarchy.registerPage({
id: 'rad:0',
data: preparedRootPage,
childCounts: rootLoaderData.childCounts,
childStarts: rootLoaderData.childStarts,
ownsData: true
});
hierarchy.update({
cameraPosition,
modelViewProjectionMatrix,
viewportSize: [width, height],
foveation: {center: [0.5, 0.5], radius: 0.2, strength: 2}
});
Traversal starts at Spark's single authored root row and refines the highest-priority branches.
Authored Gaussian footprint, coarse-node opacity, camera-space angular foveation, and configurable
refinement hysteresis contribute to each node's priority. Camera changes retarget the retained
coherent tree instead of restarting from its roots, so resolved visible detail remains selected
while bounded traversal reprioritizes newly visible branches. Offscreen ancestors are still
traversed when their descendants can contribute to the view. Every unrefined or childless leaf is
retained, and an individual parent is replaced only after all of its selected child rows become
resident. Mixed parent-and-leaf source pages therefore remain correct. Each frontier entry exposes
the intact original data, batch-local activeRows, activeMask, bounds, priority, and fallback
state. Relevant page demand remains active across partial retargets; requests are canceled once
traversal proves they cannot contribute. Protected parents remain visible when the residency budget
cannot admit every required child.
hierarchy.update(view) performs synchronous traversal. Use maxTraversalRows to bound the
number of evaluated rows and continueTraversal(maximumRows) for subsequent slices of the same
retained tree. Publishing a frontier still visits its selected rows; a row budget is not a
wall-clock frame-time guarantee. Changing the configured budget with setTraversalBudget resets
traversal, so it is not a per-frame scheduling control.
For progressive worker-driven camera updates, use hierarchy.refineView(view, maximumRows).
It starts from the displayed coherent selection and publishes completed sibling replacements
after each slice, including while a changed view is still refining. It does not merge independent
cuts or draw parents over their descendants. Retargeting walks the retained cut once to update
priorities; maximumRows bounds subsequent refinement, not that retargeting pass. Keep this work
off the render thread and coalesce camera inputs to the latest view.
Retargeting may collapse a branch only after its selected descendants are all outside the current
frustum. A parent's Gaussian support is not a conservative bound on its descendants, so checking
the parent alone is not sufficient. Optional refinement exchanges replace visible sibling groups
only with a visible, small-screen-error parent. If a camera change makes the retained selection
exceed maximumActiveRows, the lowest-priority complete visible groups may be coarsened even when
their parent has high screen-space error to restore the hard cap. Page budgets remain hard limits: if retained
visible detail occupies the available pages, new refinement may wait rather than discard that
coverage. selectView(view, maximumRows) remains available for an independent, whole-cut rebalance.
With a row budget it retains the previous frontier until the new cut completes; an initially empty
scene publishes coarse coverage on its first slice. Resume with continueTraversal(maximumRows).
Omitting the budget completes selection synchronously.
Set maximumResidentPages to bound the nominal page indices reserved by selection, including
ancestors and pending sibling replacements. Leave room in the physical residency window for overlap
with the displayed view. maximumPendingPages is a soft request limit: one complete sibling
replacement may exceed it when no other requests are pending, so a partial group cannot deadlock
refinement. Both limits default to unlimited. Set frustumCulling: false to retain coarse coverage
in every direction and let the renderer clip primitives while view-cone foveation selects detail.
CPU-only selection and worker ownership​
SplatRADHierarchyManager<TData> defaults to GPUSplatData for existing GPU callers. A worker can
instead provide TData extends SplatRADHierarchyData: decoded source.positions, source.scales,
source.opacities, a revision, and the allocation/identity/lifecycle fields from
SplatResidencyData. No rendering device, color columns, spherical harmonics, or GPU buffers are
required. SplatResidencyManager<TData> and its callbacks preserve that exact data type; its byte
budget accounts for the supplied byteLength even when the allocation is CPU-only.
The hierarchy does not create workers or serialize pages. The application must acknowledge GPU
admission before allowing CPU metadata to participate in selection, preserve original page/row
identities, and retain the previous rendered pages until a complete coherent replacement is ready.
Transfer copies of activeRows, not the hierarchy's own retained arrays. Send the renderer its
latest camera independently of worker progress, and coalesce camera requests rather than queuing
every input event. Eviction of a page unused by the retained traversal does not reset visible detail;
loss of an actual traversal input still requires rebuilding its selection.
Top-level RAD metadata contains source page ranges, but not spatial page bounds. Bounds are derived conservatively after a page is decoded; missing-page requests use authored global child row links. Transport, parsing, asynchronous worker bridges, and GPU upload remain application- or loader-owned. Supply residency estimates before initiating page fetch and preparation when a hard window must prevent both unnecessary requests and transient GPU overcommit.