Replication Backend

This backend service can be used to filter the replication between a PouchDB and a CouchDB instance based on permission rules. It does this by overriding some of CouchDB`s endpoints where permissions are checked on the transmitted entities. The permission rules are defined through CASL.

Setup

This API functions as a proxy layer between a client (PouchDB) and a standard CouchDB instance. The backend can either be run as a docker container

> docker run ghcr.io/aam-digital/replication-backend:latest

or directly through npm (see below: --> "Development")

> npm install && npm start

In both cases the required and optional environment variables need to be defined. See .env.template for the full list of variables with explanations of what they do and their default values.

In case the backend is run through Docker, the args can be provided like this

> docker run -e DATABASE_URL=https://test.com/couchdb -e DATABASE_USER=replicator -e DATABASE_PASSWORD=securePassword -e JWT_SECRET=myJWTSecret -e KEYCLOAK_ADMIN_BASE_URL=https://keycloak.example.com -e KEYCLOAK_REALM=myrealm ghcr.io/aam-digital/replication-backend:latest

In case the backend is run through npm, the .env file can be adjusted.

Defining Permissions

See our Developer Documentation

Startup behavior of the permission service

RulesService loads the Config:Permissions document from the database configured via PERMISSION_DB during startup. Its behavior is fail-closed:

• Success: the loaded rules are applied to all subsequent requests.
• CouchDB returns 401 / 403: the configured DATABASE_USER / DATABASE_PASSWORD are wrong. Startup is aborted with a CRITICAL log message — the service refuses to start without a way to evaluate permissions.
• CouchDB returns 404 (permission document missing): the service enters bootstrap mode. A synthesized config grants manage all to users with the admin_app role only; every other user (including anonymous traffic) is denied. This lets an administrator sign in and seed the real Config:Permissions document, after which the live changes feed swaps in the real rules and triggers a clearLocal so all clients re-sync.
• Network error / 5xx / malformed response: the load is retried with exponential backoff (1s → 2s → 4s → 8s → 10s, capped) for up to 60s. If the live changes feed delivers a config during a backoff, the loop exits early. If the budget is exhausted, startup is aborted with a CRITICAL log.
• Defense in depth: if getRulesForUser is ever called with no config loaded (which should be unreachable given the above), it returns an empty rule set and CASL denies every action.

Startup invariant checks (CouchdbStartupInvariantsService)

This service's whole security model relies on it being the only door to its databases - every client authenticates against it, never against CouchDB directly. That assumption lives in CouchDB's own configuration, not in this codebase, so CouchdbStartupInvariantsService asserts (but never writes) it when the app starts. Only the database creation blocks startup; the read-only checks run detached and merely log CRITICAL, so a routine version upgrade against an already-misconfigured CouchDB doesn't turn into an outage.

• Both databases exist, creating them if missing (helps a fresh install only; an existing, misconfigured database is caught by the next check).
• Every database's _security restricts members to CouchDB's reserved _admin role, including databases this service never creates itself. Note that an empty _security document is not the safe state but the opposite: CouchDB treats an empty members list as "no restriction".
• CouchDB has no [jwt_keys] configured, which would otherwise let a Keycloak realm role named _admin bypass _security entirely.

The database listing and the jwt_keys check need CouchDB server admin credentials. Without them they log a "could not verify" warning, and _security is only checked on the two databases this service knows by name.

This service deliberately does not assert that CouchDB rejects anonymous requests ([chttpd] require_valid_user / require_valid_user_except_for_up); there is little to gain and we lose access to the Fauxton UI login page otherwise.

Operation

Besides the CouchDB endpoints, the backend also provides some additional endpoints that are necessary to be used at times. A swagger / OpenAPI interface can be visited at /api/ which shows all endpoints that are available.

• /admin/clear_local/{db} needs to be executed whenever a rule or a permission change might give a user more permission than the user previously had. This will restart the synchronization process for each client which makes them fetch all the documents for which they now have gained permissions.

Additionally, a separate check on the client side is necessary that cleans up the local database whenever a client looses read permissions for a document. A example for how this could look can be found here.

Audit / Changelog Recording

The backend can record a tamper-resistant change log of every entity write, so that "what changed, by whom and when" can be reconstructed for audit/legal purposes.

Because the proxy holds the authenticated Keycloak identity at write time and sees every revision a client pushes, it is the right place to record this: the "who" and the server-set timestamp are trustworthy, and conflicting branches from concurrent/multi-device edits are captured.

How it works

• Enabled via the AUDIT_ENABLED environment variable (default false). When disabled the feature is a complete no-op — no behavior change on any write path.
• On each successful write (bulk _bulk_docs replication, single PUT, and DELETE), one record is written to a separate database derived from the source db name: writes to app are recorded in app-audit. The convention is hard-wired, and each <db>-audit database is auto-created on first write.
• A record contains: the changed entityId, source database, operation (create / update / delete / baseline), the new rev and its parentRev, a server-set timestamp, the authenticated user ({ id, name, roles }), and a jsondiffpatch diff of the change. The client's local updated/created metadata ({ at, by }) is kept inside the diff; only internal CouchDB noise (_rev, _revisions, _conflicts, _attachments) is excluded.
• Seamless activation: the first change to an entity that has no prior audit record additionally emits a full-snapshot baseline record, so history is not lost when the feature is switched on for an existing system. No migration is needed.

Protection (read-only via the permission engine)

The <db>-audit database is reachable through the normal proxy and governed entirely by CASL on the AuditRecord subject:

• The system's own audit writes use the proxy's admin credentials and are written directly to CouchDB via CouchdbService, bypassing the permission-checked endpoints, so recording always succeeds.
• Any client write that targets a AuditRecord: document (to the audit DB or a forged one in the main app DB) is dropped.
• A client read of audit records is allowed only where a { subject: "AuditRecord", action: "read" } rule is granted (the proxy filters reads via ability.can('read', doc)). This is what lets the history-viewing UI read the audit DB as an ordinary read-only remote database.

Where the read rule lives: the AuditRecord read rule is part of the application's permission config (the Config:Permissions document managed in ndb-core), granted to privileged roles only — not hard-coded here. Without it, the audit DB is invisible and immutable to clients (default-deny), which is the safe default.

Documented trade-off: whole-document read rules do not strip individual fields, so a user permitted to read an audit record sees the entire diff, and read permission is coarse (the single AuditRecord subject). Mitigation: grant the read rule only to privileged roles who already see full records. Field-level redaction would require a bespoke read endpoint instead.

Known limitation

Only revisions pushed by clients are captured. PouchDB sends only leaf revisions with their ancestry, so intermediate same-device/offline edits never reach the backend and cannot be audited by any backend component. The genuine guarantee is conflict-branch capture and a trustworthy authenticated author and server timestamp.

Development

This system is Node.js application built with the NestJS framework.

To run and test this project locally:

1. npm install to download and set up all dependencies
2. npm start to run the application locally (see above for required environment variables)
3. npm test to execute unit tests

Using with the aam-services docker stack

Run this service locally while the rest of the stack runs in Docker:

1. Start the docker stack (follow the setup guide): https://github.com/Aam-Digital/aam-services/tree/main/docs/developer

2. Configure local env in this repo:

   • cp .env.template .env
   • KEYCLOAK_ADMIN_BASE_URL / KEYCLOAK_REALM already point at the dummy dev realm by default
   • Set KEYCLOAK_ADMIN_CLIENT_SECRET from the Keycloak client credentials

3. Trust the Caddy self-signed CA for HTTPS calls to keycloak.localhost:

   export NODE_EXTRA_CA_CERTS=/path/to/aam-services/docs/developer/container-data/caddy-authorities/root.crt

4. Start replication-backend locally (port 3000):

   • npm start

5. Route /db to the local service:

   • In aam-services/docs/developer/Caddyfile, enable: reverse_proxy http://host.docker.internal:3000
   • Restart Caddy: cd aam-services/docs/developer && docker compose restart reverse-proxy

Notes:

• Keep the local replication-backend running; Caddy forwards /db* to it.
• Ensure no replication-backend container is running when using the local service.

Run in a fully local environment with other services

Use the dockerized local environment to run a fully synced app including backend services on your machine: https://github.com/Aam-Digital/aam-services/tree/main/docs/developer