Performance tips

Vectro is fast with its defaults. These are the things that most often make a difference, in rough order of how often they matter.

Let Vectro do the work, then take the answer

Most time lost around Vectro is spent outside it: turning millions of rows into JavaScript objects. toRows() builds one object per row, which is fine for a page of results and slow for millions. Aggregate, group or take a top-k inside Vectro, and convert only the small answer.

• toRows(limit) still gathers every matching row before it takes the first limit; on a big filtered table, use top(k, by) or a lazy limit(n) instead.
• To hand many rows to other code, use toColumns() (typed arrays per column, text as { codes, labels }) or toArrow(), not toRows().
• Put arithmetic inside the aggregate: agg({ revenue: 'sum(price * qty)' }) computes in one fused pass, with no column built. derive makes a real column, which costs memory and a pass.

Lazy queries

A chain on a table runs step by step: sort('-price') sorts every row, then head(10) keeps ten. Start the chain with lazy() and Vectro plans it as a whole before running it: derived columns are inlined where they are used, filters move below derives, sorts and group-bys and into each side of a join, joins and sorts copy only the columns used later, and sort followed by limit becomes a top-k. explain() shows the plan.

import { Engine } from 'vectrojs';

const engine = new Engine();
const n = 1_000_000;
const orders = engine.table({
  region: Array.from({ length: n }, (_, i) => ['EU', 'US', 'APAC'][i % 3]),
  price: Float64Array.from({ length: n }, (_, i) => ((i * 7919) % 100_000) / 100),
  qty: Float64Array.from({ length: n }, (_, i) => 1 + (i % 9)),
});

const q = orders.lazy()
  .derive('revenue', 'price * qty')
  .filter("region == 'EU'")
  .sort('-revenue')
  .limit(3);
console.log(q.explain());   // the filter runs first, revenue is inlined, sort + limit is a top-k
console.log(q.toRows());

orders.dispose();
await engine.close();

Measured on 10M orders, step by step against lazy: derive, filter and total went from 19.8 to 2.9 ms; a join then a filter from 144 to 22 ms; a sort then the first 10 rows from 177 to 12 ms. SQL always compiles to a lazy plan. See lazy queries.

Smaller columns

Memory bandwidth is what most queries are limited by, so smaller columns are faster columns as well as cheaper ones.

Compress. table.compress(), or compress: true in engine.table, parseCSV and readCSV, stores each number and category column in 1, 2 or 4 bytes per row when every value comes back exactly. On 10M orders: 458 MB became 143 MB (3.2 times smaller), compressed in 0.34 s, and queries ran about as fast as on plain columns (a new column 4.0 to 3.0 ms, group by region 4.7 to 4.4 ms; top 10 went from 8.4 to 12.9 ms). Answers never change. See compressed columns.

Float32. engine.table(columns, { type: Float32Array }) (also a type option on parseCSV, readCSV and fromArrow) stores numbers in 4 bytes instead of 8. Float32 keeps about 7 significant digits, and whole numbers are exact only up to 16,777,216; a group-by on a float32 table is limited to 16,777,216 rows on its hash path. Use it for measurements and embeddings, not for money or IDs. Vector search stores embeddings as float32 for this reason.

import { Engine } from 'vectrojs';

const engine = new Engine();
const n = 1_000_000;
const columns = {
  region: Array.from({ length: n }, (_, i) => ['EU', 'US', 'APAC'][i % 3]),
  price: Float64Array.from({ length: n }, (_, i) => ((i * 7919) % 100_000) / 100),
  qty: Float64Array.from({ length: n }, (_, i) => 1 + (i % 9)),
};

for (const [name, opts] of [['float64', {}], ['float32', { type: Float32Array }], ['compressed', { compress: true }]]) {
  const t = engine.table(columns, opts);
  const { bytes } = t.memoryBytes();
  console.log(name.padEnd(10), `${(bytes / 1e6).toFixed(0)} MB`, t.filter("region == 'EU'").sum('price * qty'));
  t.dispose();
}
await engine.close();

The float32 total differs in the last digits; the compressed one is exact.

Free memory you’re done with

Tables live in the engine’s WebAssembly memory, which JavaScript’s garbage collector doesn’t manage. Call dispose() on tables and results you no longer need: the output of collect, derive, groupBy().agg, sort, top, join and loaded files. Views made by filter and select share their parent’s columns and cost nothing. Freed space is reused by the next tables; table.memoryBytes() tells you what a table takes.

await engine.close() stops the worker threads when you’re done with the engine. Datasets have await ds.close().

The 4 GB limit, and datasets

An engine holds at most 4 GB at once: that is the most a 32-bit WebAssembly address can reach. Past it, an allocation fails with vectro: out of memory (4GB limit). Intermediate results count too, so leave room for them.

For more data, use datasets: a .vectro file is read a chunk at a time, only the columns a query uses are read, and chunks its filters rule out are skipped. A billion rows that would take 37 GB as plain columns ran with at most 163 MB of engine memory. Converting CSV files to datasets once also saves parsing them on every start: a dataset opens instantly.

On data with an order (time, IDs), keep that order: write datasets sorted by the column you filter on most, so zone maps can skip whole chunks and blocks.

Threads

new Engine() in Node starts one worker thread per core and shares one block of memory with them. That is usually right. Pass threads to leave cores for other work, such as a web server’s own requests:

const engine = new Engine({ threads: 4 });   // this thread plus 3 workers

Small inputs run on the calling thread: below roughly 100,000 to 200,000 rows for a sum, waking other threads costs more than it saves. Make one engine per process and share it; the async calls on one engine queue and run one at a time, each on every core.

In a server, use the …Async calls (aggAsync, toRowsAsync, collectAsync). The calling thread then waits without blocking, so its event loop keeps serving requests while the workers compute. The plain calls block until the answer is ready, which is fine in scripts.

new Engine({ initialMemoryMB }) starts with more memory. On a single thread, memory is not shared and moves when it grows, so starting bigger avoids repeated growth when you know the size.

Threads in browsers

A page’s main thread may not block, and threads share memory only on cross-origin isolated pages, served with these headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

On such a page, Engine from vectrojs/browser runs Web Workers on every core through the …Async calls; sync calls run on the page’s thread alone. On other pages it runs on one thread. Two ways to use every core without the headers:

• SplitEngine from vectrojs/browser works on any page: each Web Worker owns a share of the rows and the page combines their small answers. It has the async API only.
• Move the work into your own worker, where blocking calls are allowed.

Browsers slow down hidden tabs, so keep the tab in front when measuring. See in the browser.

Shape the data for the engine

• Text as categories. Text columns are stored as labels with codes, and text functions, filters and group-bys on them run once per distinct label, not per row. CSV columns where most values differ (IDs, notes, emails) load as free text, which can be returned but is slower to group, filter or join on.
• Joins. Put the smaller table on the right: it is the one hashed, once, on every thread.
• Group-by keys. Labels and whole numbers in a narrow range are used as group numbers directly, the fastest path. Many groups (millions) are fine too: they are split by key range across threads.
• Dates compare as numbers. filter("ts >= '2024-06-01'") and year(ts) compile to plain arithmetic, and a date column is 8 bytes per row (4 when compressed on whole seconds).

The numbers on this page come from the README in the Vectro repository (Apple M1 Pro, Node); see the benchmarks for the full comparisons.