Operations & troubleshooting

Running the sync day to day: the heartbeat, disconnecting, why signed requests don't blindly retry, and how to run it on a schedule.

Heartbeat — status()

status = client.status()
print(status.provider_status, status.entity_counts, status.last_session)

A signed GET that bumps the connection pulse and returns the connection status, per-kind entity counts, and the last sync session summary (StatusResult). Throttle to at most once every ~5 minutes — it's a heartbeat, not a poll loop.

Disconnect — unpair()

client.unpair()

Tells the backend to disconnect, then wipes the locally stored credentials — even if the remote call fails, so you can always re-pair. Run it before pairing a different tenant.

Retry semantics

This is the rule that trips people up: signed requests are never auto-retried on 429/5xx.

Path	Signed?	Retry policy
pair	no	retries on 429/5xx/connection (no nonce spent)
snapshot upload (PUT)	no (single-use URL)	not retried
init / finalize / send_events / status / unpair	yes	no auto-retry; opt into resign_retries

A signed request carries a single-use nonce the backend consumes the moment it arrives — before it can return a 429/5xx. A blind retry would replay a spent nonce and be rejected as a replay. So signed calls retry on connection errors only. To resend an idempotent signed call after a transient error, opt into a resign-and-resend — each retry mints a fresh nonce:

client.send_events(ops, resign_retries=2)

You can set a default resign_retries on the constructor. Because every /events batch carries a stable batch_id (its Idempotency-Key), a resend of the same batch replays rather than double-applies.

Logic errors don't raise

A 429/5xx on a signed call surfaces as a raised TransientError (it's a transport failure). But per-op logic errors (a missing required field, a bad slug, money as a float) come back inside EventsResult.errors and do not raise — read them and fix the data. See Syncing → Reading the result.

426 Upgrade Required

The backend requires a newer sync protocol than this package speaks. There is no fallback — update the package (once the PyPI release lands, pip install --upgrade monoverse-voicebot; until then, pull the repo and reinstall from source):

Terminal

pip install --upgrade monoverse-voicebot

Running on a schedule

You own the cadence — the proven pattern is frequent deltas plus one periodic full. A minimal cron-driven script:

sync_worker.py

import sys
from voicebot import client
from catalog import collect_changed_ops, full_catalog

def delta_run():
    ops = collect_changed_ops()  # since your last watermark
    if ops:
        client.send_events(ops, resign_retries=2)

def full_run():
    client.snapshot(full_catalog())

# Wire these to your scheduler:
#   */15 * * * *  python sync_worker.py delta
#   0 3 * * *     python sync_worker.py full
if __name__ == "__main__":
    (full_run if sys.argv[1:2] == ["full"] else delta_run)()

On Django/Celery you'd register delta_run and full_run as Celery-beat tasks instead of cron.

• High-churn store → delta every 5 minutes.
• Typical store → delta every 15 minutes (above).
• Schedule the full for a low-traffic hour so the larger upload doesn't compete with peak load.

Don't overlap runs

A long sync that runs past its next tick must not start a second copy. Guard with your scheduler's overlap protection (or a lock).

Long-running process? Use CatalogScheduler

If your app is a daemon (a worker, a container that stays up), the package ships a scheduler so you don't hand-code the loop — background threads, a without-overlapping guard, and on_error / on_sync callbacks that keep the daemon alive through a failed run:

worker.py

from monoverse_voicebot import CatalogScheduler

scheduler = CatalogScheduler(
    full=lambda: client.snapshot(full_catalog()),
    delta=lambda: client.send_events(collect_changed_ops(), resign_retries=2),
    run_full_on_start=True,
)
scheduler.start()  # stop() on shutdown

Defaults: delta every 15 minutes, full every 24 hours (delta_interval_s / full_interval_s to change).

The cabinet's Synchronize button doesn't reach a producer

The Synchronize button in the VoiceBot cabinet works by calling into your store over HTTP — it can only reach integrations that expose an inbound sync hook (the WordPress plugin, the Laravel package). A Python producer has no inbound endpoint, so that button does not apply to this stack: your cron/Celery/CatalogScheduler cadence is the trigger. To push immediately, run your own sync script; the cabinet's sync coverage panel reflects the result either way.

Encrypted credential storage

The shared secret is stored encrypted at rest (AES-256-GCM, cryptography) and is never logged, printed, or serialized — Credentials redacts it from repr() and str().

• Pass a full-entropy 32-byte encryption_key (openssl rand -hex 32); keep it in your secret manager / env.
• credentials_file writes the AES-GCM envelope to disk; the plaintext secret never touches disk.
• Omit credentials_file for an in-memory store — the secret lives encrypted even in process memory.
• For a KMS-backed at-rest key, implement CredentialStore and pass it as credential_store.

Rotating the encryption key

If you change encryption_key, the stored envelope can no longer be decrypted and the client raises a CredentialDecryptError. Re-pair after a key rotation.

Troubleshooting

Symptom	Cause & fix
ConfigError: …must use https://	base_url/ingest URL is plaintext http://. Switch to https:// (localhost excepted).
refusing to push an empty full snapshot	Your entity source yielded zero rows. Fix the source; never force-push an empty snapshot — it would wipe the catalog.
NotPairedError	No stored credentials. Call pair() first, or check that encryption_key/credentials_file match the run that paired.
426 protocol_upgrade_required	Update the package — there is no protocol fallback.
Per-op errors in the result	Validation/backend logic error. Common causes: missing required key, money as a float, category referenced by id instead of slug.
CredentialDecryptError on load	Wrong/rotated encryption_key. Re-pair.

Next

• Ingest Protocol Reference → Errors — the full status table and the no-retry-on-signed rationale.