Skip to main content

GPU Dataframe

Overview​

@luma.gl/experimental/gpu-dataframe compiles dataframe-style expressions, filters, aggregations, ordering, and joins into a caller-owned GPUCommandGraph. It consumes Arrow-independent GPUTable and GPUVector storage while preserving source chunks and GPU-resident outputs.

When to use it​

Use GPU Dataframe when columnar data is already on the GPU or when its results will feed GPU rendering or additional analysis. It is designed for explicit, repeatable plans rather than ad hoc CPU queries or implicit downloads.

Quick start​

import {GPUCommandGraph} from '@luma.gl/gpgpu/gpu-core';
import {GPUDataFrame} from '@luma.gl/experimental/gpu-dataframe';

const graph = new GPUCommandGraph({device, id: 'dataframe-analysis'});
const frame = new GPUDataFrame({device, graph, table});
const filtered = frame.filter({column: 'duration', greaterThan: 10});

filtered.groupBy('service').mean('duration');
const compiledGraph = graph.compile();

Core concepts and data model​

  • A dataframe borrows or owns explicit GPU table storage.
  • Expressions describe work without encoding or submitting it.
  • Compilation fixes formats, chunk topology, capacities, and execution shape.
  • Filters and derived columns remain GPU-resident and can be shared with renderers or GPU Crossfilter.
  • Global ordering and bounded joins require explicit policies rather than hidden repacking.

Try the interactive example​

Operations and API index​

FamilyOperations
Data and expressionsGPUDataFrame, GPUDataFrameQuery, GPUExpression, CompiledGPUDataFrameQuery, filters, nullable derived columns
AggregationGPUDataFrameGroupByQuery, GPUDataFrameGroupedAggregationQuery, GPUDataFrameAggregationQuery, GPUDataFrameHistogramQuery, CompiledGPUDataFrameGroupedAggregation, CompiledGPUDataFrameAggregation, CompiledGPUDataFrameHistogram
OrderingGPUDataFrameSortQuery, GPUDataFrameGlobalSortQuery, CompiledGPUDataFrameSort, CompiledGPUDataFrameGlobalSort
Indexes and joinsGPUDataFrameLookupQuery, GPUDataFrameJoinQuery, CompiledGPUDataFrameLookup, CompiledGPUDataFrameJoin
Integrationretained outputs for rendering, GPU Crossfilter, telemetry, and bounded readback

See the operations reference for detailed execution and ownership contracts.

Limits and compatibility​

  • GPU Dataframe is experimental and WebGPU-only.
  • Source batch boundaries are preserved unless global behavior is requested explicitly.
  • Join uniqueness, output capacity, null behavior, and overflow are explicit contracts.
  • Submission, synchronization, readback, and fallback remain application-owned.