My learning map Project · Global Backlog Sync

Project write-up · 2026

One backlog across many organisations

Teams and partners each plan on their own GitHub project board, in their own organisation. Leadership wants one global backlog that shows everything, always up to date. This is the small serverless system that makes that happen, explained with a 3D model you can explore, walk through, and step through one event at a time, plus the queue-based design planned next.

The problem

GitHub project boards can't natively mirror the same issue across organisations and keep fields in step. Doing it by hand doesn't scale: thousands of issues, several organisations, cards moved dozens of times a day. Earlier attempts with CI workflows were slow and couldn't react to board edits at all.

The idea

  1. Let GitHub tell us about every change. Webhooks send each new issue and each card edit to AWS as a signed event.
  2. One function decides where it goes. A routing table says which repository's issues belong on which boards; chains say which boards stay in step.
  3. Remember where everything is. A small DynamoDB "address book" maps each issue to its card on every board, so nothing ever has to search ten thousand cards twice.
  4. Write back as a bot, and ignore the bot. All writes come from one bot account, and its own echoes are ignored, so changes can't loop.

See it in 3D

🚶 Walk through it Walk through the next version Opens in a new tab: WASD or arrows to walk, mouse to look, Space to jump, N / B to step through a flow, F to follow the event, V to switch today / next.

Pick a flow and press Next to watch an event travel from a board, through AWS, and back. Click any part to read what it does. Switch to Ideal to see the next version.

How an event travels

The same steps as the 3D model, in words.

Design choices that made it work

An address book instead of searchingFinding a card on a ten-thousand-item board costs a hundred API pages. Remembering every card, and every board an issue is not on, turns that into one database lookup.
One hop, by designThe bot's own writes are ignored. That stops loops, and it means a change only travels to boards in its own chain.
Write the current value, not the remembered oneAnything retried later re-reads the source board first, so a delayed write can never put old data back.
Least privilegeThe function can write its logs, read one secret and use one table. Every event's signature is checked before anything else happens.
Respect the API budgetGitHub's GraphQL API has an hourly budget. Writes are cheap; board searches are expensive, so the design avoids them and backs off when the budget runs low.
Never invent workThe system adds and updates cards, but never deletes, recreates or bulk-edits them. Failed changes wait for a retry instead of being guessed.

What's next: queue every change

The current design does its work while GitHub waits for an answer, and a rate limit means changes have to be parked and replayed. The next version splits the work: a receiver acknowledges each event in milliseconds and puts it on a queue, and a few workers apply changes at a pace GitHub allows. A rate limit stops being a failure and becomes a short wait; anything that truly fails lands in a dead-letter queue to be fixed and replayed. Everything moves into Terraform, deployed by a pipeline.

Built with

Global Backlog Sync All maps