VectorIndex

A VectorIndex stores embeddings as float32 columns of a table, v0 to v{dim-1}, next to any other columns you give it. A search computes the score as an ordinary fused expression with the query as constants, 24 dimensions per pass, then picks the best with top(k). It is exact, not approximate: there is no index to build or tune, and the results are those of brute force. Because it is a table, filters work, and on a SplitEngine every worker searches its own rows in one round trip. Vector search is experimental: the API may change. See Vector search.

import { Engine, VectorIndex } from 'vectrojs';   // also 'vectrojs/browser', 'vectrojs/portable'

VectorIndex.create(engine, options)

Builds the index and returns a promise of it. engine is an Engine (Node, browser or portable) or a SplitEngine.

Name	Type	Default	Meaning
vectors	number[][] or a typed array	required	The embeddings: one array per row, or one flat array of n * dim values.
dim	number	required	The length of each embedding.
metric	'cosine', 'dot' or 'l2'	'cosine'	How closeness is scored (below).
columns	object	{}	Other columns, one value per vector, returned with each hit and usable in filters. Names v0, v1… and names starting with __np_ are reserved.

Metric	Score	Best
'cosine'	The dot product of the normalized vectors (vectors are normalized when stored, the query when searched).	Highest
'dot'	The dot product.	Highest
'l2'	The squared Euclidean distance.	Lowest

index.searchAsync(query, options?)

The k rows closest to query (an array of dim numbers), among the rows matching filter, best first. Returns a promise of [{ ...columns, score }].

Name	Type	Default	Meaning
k	number	10	How many hits.
filter	string	none	An expression over the other columns, as in table.filter.
params	object	{}	Values for names in filter that aren’t columns.

There is no sync form.

index.length

A getter: the number of vectors.

index.dispose()

Frees the index’s table. Returns a promise.

The index also has the properties table (the underlying table or split table), dim, metric and names (the names of the other columns).

import { Engine, VectorIndex } from 'vectrojs';

const engine = new Engine();
const docs = [
  { id: 1, year: 2019, v: [0.9, 0.1, 0.0] },
  { id: 2, year: 2021, v: [0.1, 0.9, 0.1] },
  { id: 3, year: 2022, v: [0.7, 0.6, 0.1] },
  { id: 4, year: 2023, v: [0.0, 0.2, 0.9] },
];

const index = await VectorIndex.create(engine, {
  vectors: docs.map((d) => d.v),
  dim: 3,
  columns: { id: docs.map((d) => d.id), year: docs.map((d) => d.year) },
});
console.log(index.length, index.metric);
console.log(await index.searchAsync([1, 0.2, 0], { k: 2 }));
console.log(await index.searchAsync([1, 0.2, 0], { k: 2, filter: 'year >= since', params: { since: 2021 } }));
await index.dispose();
await engine.close();

import { Engine, SplitEngine, VectorIndex } from 'vectrojs';

const dim = 16, n = 20_000;
const flat = new Float32Array(n * dim);
let seed = 1;
for (let i = 0; i < flat.length; i++) flat[i] = (seed = (seed * 16807) % 2147483647) / 2147483647;
const query = Array.from(flat.subarray(123 * dim, 124 * dim));
const columns = { row: Float64Array.from({ length: n }, (_, i) => i) };

const engine = new Engine();
const near = await VectorIndex.create(engine, { vectors: flat, dim, metric: 'l2', columns });
console.log(await near.searchAsync(query, { k: 3 }));   // row 123 first, score 0

const split = new SplitEngine({ workers: 2 });
const spread = await VectorIndex.create(split, { vectors: flat, dim, metric: 'l2', columns });
console.log(await spread.searchAsync(query, { k: 3 }));  // the same hits
await near.dispose();
await spread.dispose();
await split.close();
await engine.close();