Syncing — snapshot & deltas

A producer pushes data two ways: a full snapshot to establish (and periodically re-establish) the baseline, and incremental deltas for the changes in between. Both are driven by the entities you build.

Full snapshot

client.snapshot(entities) streams every entity as gzipped NDJSON and runs the three-step upload (init → upload → finalize) for you. Pass a sync or async iterable so the catalog never lands in memory.

import { client } from './voicebot';
import { product, category } from '@monoverse/voicebot-node';

async function* catalog() {
  yield category({ externalId: 'shop:category:phones', name: 'Phones', slug: 'phones' });
  for await (const row of db.streamProducts()) {
    yield product({
      externalId: `shop:product:${row.id}`,
      name: row.title,
      priceAmount: Math.round(row.price * 100),
      currency: 'UAH',
      stockStatus: row.inStock ? 'instock' : 'outofstock',
      categories: row.categorySlugs,
    });
  }
}

const result = await client.snapshot(catalog());
console.log(`snapshot ${result.syncId}: ${result.total} records`, result.counts);

The returned SnapshotResult has syncId, total, and per-kind counts.

A full snapshot is the source of truth — an empty one is refused

The backend tombstones anything not present in a full snapshot, so an accidental empty snapshot would wipe your catalog. client.snapshot() refuses to push zero entities and throws a ConfigError instead. If you only need to change a few items, use sendEvents — never an empty/partial snapshot.

The snapshot file is capped at 200 MB (snapshotMaxBytes); over that the client throws before uploading. Sync in smaller scopes if you hit it.

Incremental deltas

After the baseline, push only what changed. Wrap entities into operations and call sendEvents. An upsert replaces the whole entity — a field you leave out does not keep its old value — so build it with every field your snapshot sends, not only the one that changed:

import { client } from './voicebot';
import { product, toUpsertOperation, deleteOperation } from '@monoverse/voicebot-node';

const result = await client.sendEvents([
  toUpsertOperation(
    product({
      externalId: 'shop:product:iphone-15-pro',
      name: 'iPhone 15 Pro',
      priceAmount: 52999, // the new price
      currency: 'UAH',
      stockStatus: 'instock',
      categories: ['phones'],
    }),
  ),
  deleteOperation('product', 'shop:product:discontinued-sku'),
]);

console.log(`processed ${result.processed}, errors ${result.errorCount}`);

The client splits operations into wire batches automatically — capped at 500 ops and 5 MB per batch. Each batch carries a fresh batch_id that doubles as its Idempotency-Key, so a resign-and-resend of the same batch is not applied twice: if the first attempt was applied, the resend returns its original result. If a batch that carries upserts comes back with idempotentReplay: true, processed 0 and no errors, it was not applied yet — see Errors → Idempotency.

Reading the result

EventsResult gives you processed, errorCount, and per-batch detail. Per-op failures (a missing required field, money sent as a float) come back as typed errors — they do not throw; inspect them:

for (const batch of result.batches) {
  for (const err of batch.errors) {
    console.warn(`op ${err.index} ${err.entityKind}:${err.externalId} → ${err.error}`);
  }
}

Snapshot vs deltas — when to use which

	client.snapshot()	client.sendEvents()
What it sends	A complete baseline of every kind	Only the upserts/deletes you pass
Transport	gzip-NDJSON file upload	/events batches (≤500 ops / ≤5 MB)
Frequency	First setup, then periodically (e.g. nightly)	Often (minutes), as data changes
Deletes	Reconciles hard deletes via server tombstones	Only the deletes you send explicitly
Cost	Proportional to catalog size	Tiny — proportional to churn

Why a periodic full

Deltas carry only the changes you observe and send. A periodic full snapshot is the safety net for:

• Hard deletes you can't detect as a delta — the snapshot's server-side tombstone pass removes anything no longer present.
• Out-of-band edits that bypass your change tracking — the full re-syncs everything regardless.

Scheduling

Don't hand-code the loop — CatalogScheduler runs frequent deltas plus a periodic full, with a without-overlapping guard (a slow run never starts a second copy) and errors routed to onError so the daemon stays alive:

import { VoiceBotClient, CatalogScheduler } from '@monoverse/voicebot-node';

const client = new VoiceBotClient({ /* … */ });

const scheduler = new CatalogScheduler({
  delta: () => client.sendEvents(changedOps()),   // default cadence: every 15 min
  full: () => client.snapshot(allEntities()),     // default cadence: every 24 h
  runFullOnStart: true,
  onError: (phase, error) => console.error(`[voicebot] ${phase}`, error),
});
scheduler.start();

Run it as a long-lived process under a supervisor (systemd, Docker restart: unless-stopped, PM2) and call scheduler.stop() on shutdown. Tune deltaIntervalMs / fullIntervalMs to your catalog's churn.

Next

• Operations — status heartbeat, unpair, retry semantics, and running on a schedule.