# Keep your copy in sync

> Mirror listings with the changes feed without missing an edit or a removal.

Source: https://api.tailend.dev/docs/syncing-changes

This page is for anyone mirroring events, classes, camps or places locally —
a search index, a newsletter database, a cache in front of your own
API — and needs to know about an add, an edit or a removal without
re-fetching everything on a timer. `GET /v1/changes` is a single, ordered
feed across every resource type in a metro, and it's the only place a
removal shows up.

```mermaid
flowchart TD
    A["Choose and persist a since floor, then walk the list endpoints"] --> B[Store since and cursor for this metro]
    B --> C["Poll GET /v1/changes?since=&cursor="]
    C --> F["Apply every item in order; confirm archived with a detail fetch"]
    F --> G["Adopt and persist next_cursor after the page, even when data is empty"]
    G --> D{has_more?}
    D -- no --> E[Wait before polling again]
    E --> C
    D -- yes --> C
    H["Periodically:\nre-walk the list endpoints"] --> I["Reconcile: upsert anything you're missing,\nremove anything no longer served"]
    I --> B
```

## Bookkeeping

- **One resume record per metro.** Persist both the fixed `since` value and
  the latest cursor. Reusing a cursor with a newly generated `since` after a
  restart can skip changes that happened while the process was down.
- **The cursor is a decimal feed position, not an opaque token.** Store exactly
  what `meta.next_cursor` returns and pass it back as `cursor` on your next
  call; do not derive or increment it yourself.
- **Persist the cursor only after the page is fully applied, including an
  empty page.** An empty page can still advance the cursor and report
  `has_more: true` when changes were filtered from the response. If your
  process dies partway through a page, you want to replay it, not skip past
  it. Since apply is idempotent (below), replaying a page you already partly
  applied is safe.
- **Apply idempotently, keyed by `listing_id`.** Every item in the feed
  carries `listing_id` — upsert and remove against that key, not against
  `version_id` (which is only a feed position, not a stable identifier for
  the resource itself).
- **A `404` when you fetch a changed listing means it's gone.** Between the
  feed telling you about a change and you fetching the resource, it can be
  removed again. Treat a `404` on the fetch exactly like an `archived` item:
  remove it, regardless of which `kind` sent you there.
- **Poll on an interval, not in a tight loop.** Choose an interval that fits
  your freshness needs, and only wait once you've drained `has_more: false`. Back off on
  `429`/`503`: read `Retry-After` and wait that long before your next call,
  same as any other endpoint (see [Errors](https://api.tailend.dev/docs/errors.md)).
- **Always resume from the cursor you were given** rather than assuming
  "now" is caught up.
- **Reconcile periodically** by re-walking
  the list endpoints for each metro you sync, upserting everything they
  return and removing anything from your store that they no longer serve.
  This catches drift from a gap in your own polling without you having to
  trust the feed as your only source of truth forever.

## What each change `kind` means

| `kind` | What happened | What to do |
| --- | --- | --- |
| `created` | A public listing or one of its public detail records was added | Fetch by `listing_id` and upsert |
| `updated` | A public field changed that isn't covered by a more specific kind below | Fetch and upsert |
| `date_changed` | A date or schedule field changed | Fetch and upsert |
| `price_changed` | Price, tuition, or admission changed | Fetch and upsert |
| `location_changed` | The place or address changed | Fetch and upsert |
| `restored` | The listing became public; this includes the first publication of a listing initially created as a draft | Fetch and upsert |
| `archived` | The listing left public visibility — unpublished, deactivated, or deleted | Confirm with a detail fetch; remove on `404` or an archived response, otherwise upsert the live representation |

`fields` names any public fields represented by the change. It may be empty
or populated for any kind, including `created` and `archived` — see the
[API reference](https://api.tailend.dev/docs/api.md) for what each field name means.

## Example

```bash
curl -sS -H "X-API-Key: $TJ_KEY" --get https://api.tailend.dev/v1/changes \
  --data-urlencode "since=2026-09-10T00:00:00Z" \
  --data-urlencode "metro=philadelphia" \
  --data-urlencode "limit=3"
```

```json
{
  "data": [
    {
      "version_id": 18231,
      "listing_id": "2f3c9d4e-3d2b-4b7d-9a3e-1b2c3d4e5f60",
      "slug": "storytime-in-the-park-2026-09-12",
      "type": "event",
      "metro": "philadelphia",
      "changed_at": "2026-09-10T06:12:05Z",
      "kind": "date_changed",
      "fields": ["starts_at"]
    },
    {
      "version_id": 18235,
      "listing_id": "9cde4494-6c2a-4bfc-b15a-22edce13f3f7",
      "slug": "treehouse-world-fall-open-house",
      "type": "event",
      "metro": "philadelphia",
      "changed_at": "2026-09-10T08:41:19Z",
      "kind": "created",
      "fields": ["name", "status"]
    },
    {
      "version_id": 18240,
      "listing_id": "4b6b9a1e-8f21-4a11-8f4c-6a1e2b3c4d5e",
      "slug": "toddler-tinker-time",
      "type": "class",
      "metro": "philadelphia",
      "changed_at": "2026-09-10T09:03:44Z",
      "kind": "archived",
      "fields": ["status"]
    }
  ],
  "meta": {
    "since": "2026-09-10T00:00:00.000Z",
    "metro": "philadelphia",
    "limit": 3,
    "count": 3,
    "has_more": true,
    "next_cursor": "18240",
    "generated_at": "2026-09-12T16:00:00.000Z"
  }
}
```

## A sync loop (Node 18+)

Runnable as-is against a live key; swap the in-memory `store` `Map` for
whatever you're actually persisting to. It covers the whole diagram above —
initial load, poll, apply, persist the cursor, and a periodic reconciliation —
for one metro; sync several by running one of these per metro.

The list endpoints intentionally have narrower defaults than the changes
feed: list reads apply a date window, a confidence floor, and exclude
adult-only rows unless asked otherwise, while the feed is not filtered by
those rules. The example makes its mirror scope explicit with
`min_confidence=0`, `audience=all`, and a `from`/`to` window. Choose the window
your product supports, use the same parameters for every initial load and
reconciliation, and apply the same scope when processing feed items.

```javascript
const BASE = "https://api.tailend.dev";
const STATE_FILE = new URL("./tailend-sync-state.json", import.meta.url);
const KEY = process.env.TJ_KEY;
const METRO = "philadelphia";
const PLURAL = { event: "events", class: "classes", camp: "camps", place: "places" };
const POLL_MS = Number(process.env.POLL_MS);
const RECONCILE_MS = Number(process.env.RECONCILE_MS);
if (!POLL_MS || !RECONCILE_MS) throw new Error("Set POLL_MS and RECONCILE_MS to positive millisecond intervals");
const { readFile, writeFile, rename } = await import("node:fs/promises");

const store = new Map(); // listing_id -> resource; persist this for real use
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function apiGet(path, params = {}) {
  const url = new URL(path, BASE);
  for (const [key, value] of Object.entries(params)) {
    if (value !== undefined && value !== null) url.searchParams.set(key, String(value));
  }
  const res = await fetch(url, { headers: { "X-API-Key": KEY } });
  if (res.status === 429 || res.status === 503) {
    const value = res.headers.get("retry-after");
    const seconds = value && /^\d+$/.test(value) ? Number(value) : null;
    const dateDelay = value && seconds === null ? Date.parse(value) - Date.now() : null;
    await sleep(seconds !== null ? seconds * 1000 : Math.max(0, dateDelay ?? 5000));
    return apiGet(path, params);
  }
  return res;
}

async function loadState() {
  try { return JSON.parse(await readFile(STATE_FILE, "utf8")); }
  catch (error) { if (error.code === "ENOENT") return null; throw error; }
}

async function saveState(state) {
  const temporary = new URL(`${STATE_FILE.pathname}.tmp`, "file://");
  await writeFile(temporary, JSON.stringify(state));
  await rename(temporary, STATE_FILE);
}

function nextCursor(current, meta) {
  return meta.next_cursor ?? current;
}

function scopeWindow() {
  const from = new Date().toISOString().slice(0, 10);
  const end = new Date(`${from}T00:00:00Z`);
  end.setUTCDate(end.getUTCDate() + 366);
  return { from, to: end.toISOString().slice(0, 10) };
}

function listScope(type, cursor) {
  const window = scopeWindow();
  return {
    cursor,
    min_confidence: 0,
    audience: "all",
    ...(type === "places" ? {} : window),
  };
}

// Keep polling and list reconciliation inside the same mirror window.
function inScope(listing) {
  if (listing.status === "archived" || listing.is_active === false) return false;
  const { from, to } = scopeWindow();
  if (listing.type === "event") {
    // List windows use the event's metro-local calendar date, not its UTC
    // instant. Use the same field here so late-evening boundary events agree.
    const starts = listing.starts_local?.slice(0, 10);
    return starts ? starts >= from && starts <= to : false;
  }
  if (listing.type === "class") {
    if (listing.schedule?.type === "ongoing") return true;
    const starts = listing.term?.starts_on;
    const ends = listing.term?.ends_on;
    return (!starts || starts <= to) && (!ends || ends >= from);
  }
  if (listing.type === "camp") {
    return (!listing.starts_on || listing.starts_on <= to) && (!listing.ends_on || listing.ends_on >= from);
  }
  return listing.type === "place";
}

// Walks every list endpoint for the metro, upserting what it finds.
// Used both for the initial load and for periodic reconciliation.
async function walkListings(metro) {
  const seen = new Set();
  for (const type of Object.values(PLURAL)) {
    let cursor;
    do {
      const res = await apiGet(`/v1/${metro}/${type}`, listScope(type, cursor));
      if (!res.ok) throw new Error(`list failed: ${res.status}`);
      const { data, meta } = await res.json();
      for (const item of data) {
        store.set(item.id, item);
        seen.add(item.id);
      }
      cursor = meta.next_cursor ?? undefined;
    } while (cursor);
  }
  return seen;
}

async function reconcile(metro) {
  const seen = await walkListings(metro);
  for (const id of store.keys()) if (!seen.has(id)) store.delete(id);
}

async function applyChange(change) {
  const res = await apiGet(`/v1/${PLURAL[change.type]}/${change.listing_id}`);
  if (res.status === 404) {
    store.delete(change.listing_id);
    return;
  }
  if (!res.ok) throw new Error(`detail failed: ${res.status}`);
  const { data } = await res.json();
  if (data.status === "archived" || data.is_active === false) {
    store.delete(change.listing_id);
    return;
  }
  if (inScope(data)) store.set(change.listing_id, data);
  else store.delete(change.listing_id);
}

async function initializeState() {
  let state = await loadState();
  if (!state) {
    state = { since: new Date().toISOString(), cursor: null, baseline_complete: false };
    await saveState(state); // establish the floor before the initial walk
  }
  if (!state.baseline_complete) {
    await walkListings(METRO);
    state.baseline_complete = true;
    await saveState(state);
  }
  return state;
}

async function pollChanges(state, maxPages = Infinity, run = (task) => task()) {
  let pages = 0;
  while (pages < maxPages) {
    const meta = await run(async () => {
      const res = await apiGet("/v1/changes", { since: state.since, cursor: state.cursor, metro: METRO, limit: 500 });
      if (!res.ok) throw new Error(`changes failed: ${res.status}`);
      const page = await res.json();
      for (const change of page.data) await applyChange(change);
      state.cursor = nextCursor(state.cursor, page.meta); // advance even when page.data is empty
      await saveState(state);
      return page.meta;
    });
    pages += 1;
    if (!meta.has_more) await sleep(POLL_MS);
  }
}

async function main() {
  const state = await initializeState();

  // Polling and reconciliation share this queue, so a reconcile cannot delete
  // an item that a concurrent poll has just added.
  let exclusive = Promise.resolve();
  const serial = (task) => {
    const run = exclusive.then(task, task);
    exclusive = run.catch(() => {});
    return run;
  };
  setInterval(() => serial(() => reconcile(METRO)).catch((err) => console.error("reconcile failed", err)), RECONCILE_MS);

  await pollChanges(state, Infinity, serial);
}

main();
```

## Webhooks

There's no push option today — `/v1/changes` is the way to keep a copy in
sync. Change webhooks are coming, but there's no contract, payload shape, or
date to build against yet; this page will cover them once there is.

## What to do

- Key every apply by `listing_id`; don't use `version_id` for anything but
  the cursor.
- Persist `since`, the cursor, and whether the initial baseline completed
  together. If startup stops after saving the floor, finish the baseline
  before polling on restart. Adopt `next_cursor` after every
  fully applied page, even when that page contains no items.
- Confirm `archived` with a detail fetch. Remove on `404` or an archived
  representation; otherwise upsert the live representation.
- Reconcile against the list endpoints on a schedule so a gap in your own
  polling can't drift your copy forever, and serialize it with polling.
- See [Errors](https://api.tailend.dev/docs/errors.md) for `429`/`503` backoff, and the
  [API reference](https://api.tailend.dev/docs/api.md) for what each resource field means once you've
  fetched it.