# Storage Search

**URL:** https://heroiclabs.com/docs/nakama/concepts/storage/search/
**Summary:** The storage engine supports configurable indexing of storage objects enabling search queries over these indices.
**Keywords:** indices, search
**Categories:** nakama, search, storage

---


# Storage Search

The storage engine supports configurable indices that automatically index the content of storage objects written through the [`storageWrite`](../../../server-framework/typescript-runtime/function-reference/#storageWrite) API.

Only a subset of the storage object value is indexed as configured. Search these indices using the [query syntax](../../multiplayer/query-syntax/).

Nakama populates the configured indices at startup and holds the maximum number of configured entries plus a threshold. Once that threshold is surpassed, it automatically evicts the oldest entries.

These indices aren't designed to hold indexed representations of all objects in a given collection. Instead, they enable complex queries to look up cohorts of users or entities for different use cases, such as [offline matchmaking](../../multiplayer/matchmaker/offline-matchmaking/).

{{< note important "Note" >}}
The search system built on the Storage Engine is an eventually consistent system. Indexes are populated asynchronously, which may take a few milliseconds to reflect updates written to storage objects.
By default, the search indexes fetch their results from the Storage Engine directly, so values always reflect the most up-to-date storage object state. When using [index-only mode](#index-only-mode), results are returned directly from the index without an additional database read.
{{< / note >}}

## Creating a new index

Create and configure indices through the [runtimes](../../../server-framework/).

Index names must be unique. Each index is tied to a collection, and you must specify the top-level keys of the object fields to index. If an object has none of those keys, it isn't indexed.

Set `key` to index only storage objects with that specific key. Omit it (or pass an empty string) to index all object keys that match the collection and fields.

Set a maximum entry count for the index.

The optional `sortableFields` parameter specifies which indexed fields support custom sort ordering when listing results. These must be a subset of `fields`.

Multiple indices can share a collection as long as their names are unique. Nakama automatically indexes any objects that match an index's configuration on storage write.

{{< code type="server" >}}
```go
name := "IndexName"
collection := "CollectionName"
key := "" // Set to empty string to match all keys instead
fields := []string{"field1", "field2"} // Only objects containing any of these keys and respective values are indexed.
sortableFields := []string{"field1"} // Must be a subset of fields. Leave empty if no custom sort ordering is needed.
maxEntries := 1000
indexOnly := false

err := initializer.RegisterStorageIndex(name, collection, key, fields, sortableFields, maxEntries, indexOnly)
```
{{< /code >}}

{{< code type="server" >}}
```lua
local name = "IndexName"
local collection = "CollectionName"
local key = "KeyName" -- Set to empty string to match all keys instead
local fields = {"field1", "field2"} -- Only objects containing any of these keys and respective values are indexed.
local sortable_fields = {"field1"} -- Must be a subset of fields. Leave empty if no custom sort ordering is needed.
local max_entries = 1000
local index_only = false

local err = nk.register_storage_index(name, collection, key, fields, sortable_fields, max_entries, index_only)
```
{{< /code >}}

{{< code type="server" >}}
```typescript
const name = "IndexName";
const collection = "CollectionName";
const key = "KeyName"; // Set to empty string to match all keys instead
const fields = ["field1", "field2"]; // Only objects containing any of these keys and respective values are indexed.
const sortableFields = ["field1"]; // Must be a subset of fields. Leave empty if no custom sort ordering is needed.
const maxEntries = 1000;
const indexOnly = false;

try {
    initializer.registerStorageIndex(name, collection, key, fields, sortableFields, maxEntries, indexOnly);
} catch(err) {
    // Handle error
}
```
{{< /code >}}

## Index-only mode

By default, when a query runs against an index, Nakama first resolves the matching index entries and then fetches the corresponding storage objects from the database. This ensures returned values are always complete and up-to-date.

When `indexOnly` is set to `true` at registration time, the list operation returns values directly from the index without reading from the database. This reduces database reads and can improve performance, but comes with an important caveat: the index only stores the fields that were specified at registration, so the returned objects may contain partial data — fields not included in `fields` will be absent from the response.


## Listing from an index

Query the index using the powerful [query syntax](../../multiplayer/query-syntax/) that also powers the Nakama [matchmaker](../../multiplayer/matchmaker/).

To filter by indexed field values, prefix the field key with `value.`.

{{< code type="server" >}}
```go
name := "index_name"
query := "+value.field1:1 value.field2:foo"
limit := 10
order := []string{} // optional, see Sorting results below
cursor := "" // empty string for first page

objects, nextCursor, err := nk.StorageIndexList(ctx, "", name, query, limit, order, cursor)
// Pass nextCursor as cursor in the next call to retrieve the following page.
```
{{< /code >}}

{{< code type="server" >}}
```lua
local name = "index_name"
local query = "+value.field1:1 value.field2:foo"
local limit = 10
local order = {} -- optional, see Sorting results below
local cursor = nil -- nil for first page

local objects, next_cursor = nk.storage_index_list(name, query, limit, order, nil, cursor)
-- Pass next_cursor as cursor in the next call to retrieve the following page.
```
{{< /code >}}

{{< code type="server" >}}
```typescript
const name = "index_name";
const query = "+value.field1:1 value.field2:foo";
const limit = 10;

try {
    let result = nk.storageIndexList(name, query, limit);
    let objects = result.objects;
    let nextCursor = result.cursor; // null when there are no further results
    // Pass nextCursor as the cursor argument to retrieve the following page.
} catch(err) {
    // Handle error
}
```
{{< /code >}}


Each index entry also includes these queryable values automatically, in addition to the configured `fields`:

* `update_time`
* `key`
* `user_id`
* `version`
* `create_time`
* `read`
* `write`
* `collection`

These fields don't need the `value.` prefix in queries.

## Sorting results

Use the optional `order` parameter to sort results by storage object fields. The prefix `-` before a field name indicates descending order. Declare the fields you want to sort by in `sortableFields` when registering the index.

{{< code type="server" >}}
```go
order := []string{"value.field_name", "-value.other_field"}

objects, nextCursor, err := nk.StorageIndexList(ctx, "", "<name>", "<query>", 10, order, "")
```
{{< /code >}}

{{< code type="server" >}}
```lua
local order = {"value.field_name", "-value.other_field"}

local objects, next_cursor = nk.storage_index_list("<name>", "<query>", 10, order)
```
{{< /code >}}

{{< code type="server" >}}
```typescript
const order = ["value.field_name", "-value.other_field"];

try {
    let result = nk.storageIndexList("<name>", "<query>", 10, order);
    let objects = result.objects;
} catch(err) {
    // Handle error
}
```
{{< /code >}}

To sort results by the value of any of the sortable fields, their respective keys need to be prefixed with `value.`.

## Register a storage index filter

Register a custom filtering function per index. This function is called for each storage object eligible for the index and must return a boolean value.

Return `true` to index the object. Return `false` to exclude it from the index and delete any previously indexed entry for it.

The registered function is also applied to all eligible entries when the index is populated.

{{< code type="server" >}}
```go

name := "index_name"
fn := func(ctx context.Context, logger Logger, db *sql.DB, nk NakamaModule, write *StorageWrite) bool {
    // Inspect object to decide if it should be inserted or tentatively deleted.
    return true
}

err := initializer.RegisterStorageIndexFilter(name, fn);
```
{{< /code >}}

{{< code type="lua" >}}
```lua
local name = "index_name"
function indexFilter(context, logger, nk, storageWrite)
    -- Inspect object to decide if it should be inserted or tentatively deleted.
    return true
end

local err = nk.register_storage_index_filter(name, indexFilter)
```
{{< /code >}}

{{< code type="typescript" >}}
```typescript
const name = "index_name";

let filterFn: nkruntime.StorageIndexFilterFunction = function (ctx: nkruntime.Context, logger: nkruntime.Logger, nk: nkruntime.Nakama, storageWrite: StorageWriteRequest): boolean {
      // Inspect object to decide if it should be inserted or tentatively deleted.
      return true;
}

try {
    initializer.RegisterStorageIndexFilter(name, filterFn);
} catch(err) {
    // handle error
}

```
{{< /code >}}