Storage Search

The storage engine supports configurable indices that automatically index the content of storage objects written through the storageWrite API.

Only a subset of the storage object value is indexed as configured. Search these indices using the query syntax.

Nakama populates the configured indices at startup and holds the maximum number of configured entries plus a threshold. Once that threshold is surpassed, it automatically evicts the oldest entries.

These indices aren’t designed to hold indexed representations of all objects in a given collection. Instead, they enable complex queries to look up cohorts of users or entities for different use cases, such as offline matchmaking.

Creating a new index #

Create and configure indices through the runtimes.

Index names must be unique. Each index is tied to a collection, and you must specify the top-level keys of the object fields to index. If an object has none of those keys, it isn’t indexed.

Set key to index only storage objects with that specific key. Omit it (or pass an empty string) to index all object keys that match the collection and fields.

Set a maximum entry count for the index.

The optional sortableFields parameter specifies which indexed fields support custom sort ordering when listing results. These must be a subset of fields.

Multiple indices can share a collection as long as their names are unique. Nakama automatically indexes any objects that match an index’s configuration on storage write.

1
2
3
4
5
6
7
8
9

name := "IndexName"
collection := "CollectionName"
key := "" // Set to empty string to match all keys instead
fields := []string{"field1", "field2"} // Only objects containing any of these keys and respective values are indexed.
sortableFields := []string{"field1"} // Must be a subset of fields. Leave empty if no custom sort ordering is needed.
maxEntries := 1000
indexOnly := false

err := initializer.RegisterStorageIndex(name, collection, key, fields, sortableFields, maxEntries, indexOnly)

1
2
3
4
5
6
7
8
9

local name = "IndexName"
local collection = "CollectionName"
local key = "KeyName" -- Set to empty string to match all keys instead
local fields = {"field1", "field2"} -- Only objects containing any of these keys and respective values are indexed.
local sortable_fields = {"field1"} -- Must be a subset of fields. Leave empty if no custom sort ordering is needed.
local max_entries = 1000
local index_only = false

local err = nk.register_storage_index(name, collection, key, fields, sortable_fields, max_entries, index_only)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

const name = "IndexName";
const collection = "CollectionName";
const key = "KeyName"; // Set to empty string to match all keys instead
const fields = ["field1", "field2"]; // Only objects containing any of these keys and respective values are indexed.
const sortableFields = ["field1"]; // Must be a subset of fields. Leave empty if no custom sort ordering is needed.
const maxEntries = 1000;
const indexOnly = false;

try {
    initializer.registerStorageIndex(name, collection, key, fields, sortableFields, maxEntries, indexOnly);
} catch(err) {
    // Handle error
}

Index-only mode #

By default, when a query runs against an index, Nakama first resolves the matching index entries and then fetches the corresponding storage objects from the database. This ensures returned values are always complete and up-to-date.

When indexOnly is set to true at registration time, the list operation returns values directly from the index without reading from the database. This reduces database reads and can improve performance, but comes with an important caveat: the index only stores the fields that were specified at registration, so the returned objects may contain partial data — fields not included in fields will be absent from the response.

Listing from an index #

Query the index using the powerful query syntax that also powers the Nakama matchmaker.

To filter by indexed field values, prefix the field key with value..

1
2
3
4
5
6
7
8

name := "index_name"
query := "+value.field1:1 value.field2:foo"
limit := 10
order := []string{} // optional, see Sorting results below
cursor := "" // empty string for first page

objects, nextCursor, err := nk.StorageIndexList(ctx, "", name, query, limit, order, cursor)
// Pass nextCursor as cursor in the next call to retrieve the following page.

1
2
3
4
5
6
7
8

local name = "index_name"
local query = "+value.field1:1 value.field2:foo"
local limit = 10
local order = {} -- optional, see Sorting results below
local cursor = nil -- nil for first page

local objects, next_cursor = nk.storage_index_list(name, query, limit, order, nil, cursor)
-- Pass next_cursor as cursor in the next call to retrieve the following page.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

const name = "index_name";
const query = "+value.field1:1 value.field2:foo";
const limit = 10;

try {
    let result = nk.storageIndexList(name, query, limit);
    let objects = result.objects;
    let nextCursor = result.cursor; // null when there are no further results
    // Pass nextCursor as the cursor argument to retrieve the following page.
} catch(err) {
    // Handle error
}

Each index entry also includes these queryable values automatically, in addition to the configured fields:

• update_time
• key
• user_id
• version
• create_time
• read
• write
• collection

These fields don’t need the value. prefix in queries.

Sorting results #

Use the optional order parameter to sort results by storage object fields. The prefix - before a field name indicates descending order. Declare the fields you want to sort by in sortableFields when registering the index.

1
2
3

order := []string{"value.field_name", "-value.other_field"}

objects, nextCursor, err := nk.StorageIndexList(ctx, "", "<name>", "<query>", 10, order, "")

1
2
3

local order = {"value.field_name", "-value.other_field"}

local objects, next_cursor = nk.storage_index_list("<name>", "<query>", 10, order)

1
2
3
4
5
6
7
8

const order = ["value.field_name", "-value.other_field"];

try {
    let result = nk.storageIndexList("<name>", "<query>", 10, order);
    let objects = result.objects;
} catch(err) {
    // Handle error
}

To sort results by the value of any of the sortable fields, their respective keys need to be prefixed with value..

Register a storage index filter #

Register a custom filtering function per index. This function is called for each storage object eligible for the index and must return a boolean value.

Return true to index the object. Return false to exclude it from the index and delete any previously indexed entry for it.

The registered function is also applied to all eligible entries when the index is populated.

1
2
3
4
5
6
7

name := "index_name"
fn := func(ctx context.Context, logger Logger, db *sql.DB, nk NakamaModule, write *StorageWrite) bool {
    // Inspect object to decide if it should be inserted or tentatively deleted.
    return true
}

err := initializer.RegisterStorageIndexFilter(name, fn);

1
2
3
4
5
6
7

local name = "index_name"
function indexFilter(context, logger, nk, storageWrite)
    -- Inspect object to decide if it should be inserted or tentatively deleted.
    return true
end

local err = nk.register_storage_index_filter(name, indexFilter)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

const name = "index_name";

let filterFn: nkruntime.StorageIndexFilterFunction = function (ctx: nkruntime.Context, logger: nkruntime.Logger, nk: nkruntime.Nakama, storageWrite: StorageWriteRequest): boolean {
      // Inspect object to decide if it should be inserted or tentatively deleted.
      return true;
}

try {
    initializer.RegisterStorageIndexFilter(name, filterFn);
} catch(err) {
    // handle error
}

Related Pages

• Collections
• Access Controls
• Modeling for Scalability