Syncing — snapshot & deltas

A producer pushes data two ways: a full snapshot to establish (and periodically re-establish) the baseline, and incremental deltas for the changes in between. Both are driven by the entities you build.

Full snapshot

client.snapshot(entities) streams every entity as gzipped NDJSON and runs the three-step upload (init → upload → finalize) for you. Pass a sync or async iterable so the catalog never lands in memory.

import { client } from './voicebot';
import { product, category } from '@monoverse/voicebot-node';

async function* catalog() {
  yield category({ externalId: 'shop:category:phones', name: 'Phones', slug: 'phones' });
  for await (const row of db.streamProducts()) {
    yield product({
      externalId: `shop:product:${row.id}`,
      name: row.title,
      priceAmount: Math.round(row.price * 100),
      currency: 'UAH',
      stockStatus: row.inStock ? 'instock' : 'outofstock',
      categories: row.categorySlugs,
    });
  }
}

const result = await client.snapshot(catalog());
console.log(`snapshot ${result.syncId}: ${result.total} records`, result.counts);

The returned SnapshotResult has syncId, total, and per-kind counts.

A full snapshot is the source of truth — an empty one is refused

The backend tombstones anything not present in a full snapshot, so an accidental empty snapshot would wipe your catalog. client.snapshot() refuses to push zero entities and throws a ConfigError instead. If you only need to change a few items, use sendEvents — never an empty/partial snapshot.

The snapshot file is capped at 200 MB (snapshotMaxBytes); over that the client throws before uploading. Sync in smaller scopes if you hit it.

Incremental deltas

After the baseline, push only what changed. Wrap entities into operations and call sendEvents. An upsert replaces the whole entity — a field you leave out does not keep its old value — so build it with every field your snapshot sends, not only the one that changed:

import { client } from './voicebot';
import { product, toUpsertOperation, deleteOperation } from '@monoverse/voicebot-node';

const result = await client.sendEvents([
  toUpsertOperation(
    product({
      externalId: 'shop:product:iphone-15-pro',
      name: 'iPhone 15 Pro',
      priceAmount: 52999, // the new price
      currency: 'UAH',
      stockStatus: 'instock',
      categories: ['phones'],
    }),
  ),
  deleteOperation('product', 'shop:product:discontinued-sku'),
]);

console.log(`processed ${result.processed}, errors ${result.errorCount}`);

The client splits operations into wire batches automatically — capped at 500 ops and 5 MB per batch. Each batch carries a fresh batch_id that doubles as its Idempotency-Key, so a resign-and-resend of the same batch is not applied twice: if the first attempt was applied, the resend returns its original result. After a 429, a 5xx or a dropped connection the client resends a batch up to 3 times by default, re-signed each time (Operations → Retry semantics). If a batch that carries upserts comes back with idempotentReplay: true, processed 0 and no errors, an earlier attempt still holds it or never applied it (Errors → Idempotency): the client asks again after 1, 2 and 4 s, and if the answer stays the same it reports the batch as unconfirmed instead of done.

Call it from a worker

A single sendEvents call can block for minutes — per batch, up to 3 waits of 60 s on 429s plus up to 7 s of re-checks. Call it from a worker, a cron job or CatalogScheduler, never inside a web request handler.

Reading the result

EventsResult gives you processed, errorCount, unconfirmed and the per-batch detail in batches: each EventsBatchResult has batchId, processed, idempotentReplay, errors and confirmed. Per-op failures (a missing required field, money sent as a float) come back as typed errors — they do not throw; inspect them:

for (const batch of result.batches) {
  for (const err of batch.errors) {
    console.warn(`op ${err.index} ${err.entityKind}:${err.externalId} → ${err.error}`);
  }
}

Unconfirmed batches

A batch is unconfirmed (batch.confirmed is false, and it counts in result.unconfirmed) when it carries at least one upsert and the server still answers processed 0, no errors and idempotentReplay: true after the client asked again at 1, 2 and 4 s. An earlier attempt still holds that batch, or never applied it. Its processed stays 0: it is never counted as applied.

Do not advance your change watermark for that run, and on the next run send the current state of those entities again (each call mints a new batch_id):

const result = await client.sendEvents(ops);
if (result.unconfirmed === 0) {
  await saveWatermark(runStartedAt); // otherwise the next run resends these entities
}

A delete-only batch is never reported unconfirmed — resending a delete is harmless.

When a batch throws

Batches go out in order, and sendEvents stops at the first one that throws. The error carries the results of the batches sent before it as partialResult — an EventsResult declared on VoiceBotError, whose batches is empty if the first batch failed. The failing batch's outcome is unknown, and the batches after it were not sent. This covers TransientError (after the resends), ConfigError (a 4xx), ProtocolVersionError (426) and NotPairedError.

import { VoiceBotError } from '@monoverse/voicebot-node';

try {
  await client.sendEvents(ops);
} catch (err) {
  if (err instanceof VoiceBotError) {
    console.warn(`${err.partialResult?.processed ?? 0} ops landed before the failure`);
  }
  throw err;
}

Snapshot vs deltas — when to use which

	client.snapshot()	client.sendEvents()
What it sends	A complete baseline of every kind	Only the upserts/deletes you pass
Transport	gzip-NDJSON file upload	/events batches (≤500 ops / ≤5 MB)
Frequency	First setup, then periodically (e.g. nightly)	Often (minutes), as data changes
Deletes	Reconciles hard deletes via server tombstones	Only the deletes you send explicitly
Cost	Proportional to catalog size	Tiny — proportional to churn

Why a periodic full

Deltas carry only the changes you observe and send. A periodic full snapshot is the safety net for:

• Hard deletes you can't detect as a delta — the snapshot's server-side tombstone pass removes anything no longer present.
• Out-of-band edits that bypass your change tracking — the full re-syncs everything regardless.

Scheduling

Don't hand-code the loop — CatalogScheduler runs frequent deltas plus a periodic full, with a without-overlapping guard (a slow run never starts a second copy) and errors routed to onError so the daemon stays alive:

import { VoiceBotClient, CatalogScheduler } from '@monoverse/voicebot-node';

const client = new VoiceBotClient({ /* … */ });

const scheduler = new CatalogScheduler({
  delta: () => client.sendEvents(changedOps()),   // default cadence: every 15 min
  full: () => client.snapshot(allEntities()),     // default cadence: every 24 h
  runFullOnStart: true,
  onError: (phase, error) => console.error(`[voicebot] ${phase}`, error),
});
scheduler.start();

Run it as a long-lived process under a supervisor (systemd, Docker restart: unless-stopped, PM2) and call scheduler.stop() on shutdown. Tune deltaIntervalMs / fullIntervalMs to your catalog's churn.

Next

• Operations — status heartbeat, unpair, retry semantics, and running on a schedule.