Skip to content

backend-plugin-system.md

Latest commit

 

History

History
383 lines (296 loc) · 21.9 KB

backend-plugin-system.md

File metadata and controls

383 lines (296 loc) · 21.9 KB
Raw

Backend Plugin System

Overview

Backend plugins run as WebAssembly (WASM) modules inside the services/api Go process. The host uses wazero — a pure-Go, zero-CGo WASM runtime — to load and execute plugin modules in a sandboxed environment.

Each plugin:

  • Is a single .wasm binary compiled from Go (via TinyGo), Rust, AssemblyScript, or any other language with a WASM target.
  • Declares the host functions it needs in its plugin.json manifest (capability-based permission model).
  • Communicates with the host exclusively through a well-defined host function bridge — it cannot read host memory, access the filesystem, or make raw network calls.

Host Function Bridge

The host function bridge exposes a set of typed functions that WASM modules can import from the env module. All host functions use a linear memory protocol: the plugin writes a request payload (Protocol Buffers) into shared linear memory and calls the host function with a pointer and length. The host reads the request, executes it, writes the response back into a plugin-provided buffer, and returns the response length.

Protobuf Message Transport

Plugin memory layout for a host call:
  [req_ptr, req_len] → plugin writes serialised proto request
  host function called → host reads request, executes, writes response
  [resp_ptr, resp_len] → plugin reads serialised proto response

The SDK (see sdk-reference.md) hides this transport entirely behind idiomatic Go or Rust function calls.

Available Host Functions

All host functions are imported from the paca module namespace.

Database Access

Function Description
paca.db_query(req_ptr, req_len, resp_ptr, max_len) → resp_len Execute a typed read query. Results are filtered to the plugin's authorised scope.
paca.db_exec(req_ptr, req_len, resp_ptr, max_len) → resp_len Execute a write (insert/update/delete) against plugin-owned tables or allowed shared tables.
paca.db_tx_begin() → tx_id Begin a database transaction.
paca.db_tx_commit(tx_id) Commit a transaction.
paca.db_tx_rollback(tx_id) Roll back a transaction.

Plugins cannot issue arbitrary SQL. They use typed query builders exposed by the SDK, which map to pre-approved query templates validated by the host bridge.

Plugin-Owned Storage

Each plugin gets a dedicated namespace in the database (plugin_data_{pluginId}) for its own tables. Schema migrations for plugin-owned tables are declared in the plugin manifest and run by the host during plugin installation/upgrade using a restricted migration runner.

Function Description
paca.storage_get(key_ptr, key_len, resp_ptr, max_len) → resp_len Get a value from the plugin's key-value store (backed by a PostgreSQL JSONB column).
paca.storage_set(key_ptr, key_len, val_ptr, val_len) → ok Set a value in the plugin's key-value store.
paca.storage_delete(key_ptr, key_len) → ok Delete a key from the plugin's key-value store.

Plugin Cache (TTL, Valkey/Redis-backed)

Backed by the host's shared Valkey/Redis instance, keyed under a per-plugin namespace so plugins can't read or overwrite each other's entries. Unlike Plugin-Owned Storage above, entries can expire and there is no durability guarantee — treat the cache as recomputable data only. If the host has no cache backend configured, cache_get always misses and cache_set/cache_delete are no-ops, so plugins should always be able to fall back to recomputing a value on a miss.

Function Description
paca.cache_get(key_ptr, key_len, value_ptr_ptr, value_len_ptr) Get a value from the plugin's cache. A zero-length response means a miss (never cached, expired, or no cache backend configured).
paca.cache_set(key_ptr, key_len, val_ptr, val_len, ttl_seconds) → ok Set a value in the plugin's cache with the given TTL in seconds. ttl_seconds = 0 stores the value with no expiry; negative values are rejected (ok = 0).
paca.cache_delete(key_ptr, key_len) → ok Delete a key from the plugin's cache.

Core Data Access (Read-Only, Scoped)

Function Description
paca.tasks_list(req_ptr, req_len, resp_ptr, max_len) → resp_len List tasks for the authorised project(s) with filter options.
paca.task_get(req_ptr, req_len, resp_ptr, max_len) → resp_len Get a single task by ID (project scope enforced).
paca.project_get(req_ptr, req_len, resp_ptr, max_len) → resp_len Get project metadata.
paca.members_list(req_ptr, req_len, resp_ptr, max_len) → resp_len List project members.

Event System

Function Description
paca.event_subscribe(event_ptr, event_len) → ok Subscribe to a named core domain event (e.g., task.created, sprint.closed). Events are delivered at plugin startup based on manifest declarations.
paca.event_emit(req_ptr, req_len) → ok Emit a plugin-namespaced event (e.g., com.paca.bdd.scenario_created) to the Valkey Stream for consumption by the realtime service or other listeners.

HTTP Response (used inside route handlers)

Function Description
paca.http_respond(req_ptr, req_len) Write an HTTP response from within a registered route handler. Includes status code, headers, and body.
paca.http_request_body(resp_ptr, max_len) → resp_len Read the incoming HTTP request body inside a route handler.
paca.http_request_headers(resp_ptr, max_len) → resp_len Read incoming request headers.
paca.http_caller_identity(resp_ptr, max_len) → resp_len Read the authenticated caller's user ID and project membership (JWT claims, validated by the host).

Logging

Function Description
paca.log(level, msg_ptr, msg_len) Write a structured log entry at the given level. Entries are tagged with the plugin ID.

Capability Permissions

Each host function group maps to a permission in plugin.json:

{
  "permissions": [
    "db:read:tasks",
    "db:read:members",
    "db:write:plugin_data",
    "http:register_routes",
    "events:subscribe:task.*",
    "events:emit"
  ]
}

The host validates the requested permissions at install time against the allowlist for the installation tier (self-hosted installations can grant all permissions; future SaaS tiers may restrict).

Plugin Manifest (plugin.json)

{
  "id": "com.paca.bdd",
  "name": "BDD Scenarios",
  "version": "1.0.0",
  "description": "Adds Given/When/Then acceptance criteria to tasks.",
  "author": "Paca Core Team",
  "license": "MIT",
  "minCoreVersion": "0.5.0",

  "frontend": {
    "remoteEntryUrl": "https://plugins.paca.app/bdd/1.0.0/remoteEntry.js",
    "extensionPoints": [
      {
        "point": "task.detail.section",
        "component": "TaskDetailSection",
        "label": "BDD Scenarios",
        "order": 10
      },
      {
        "point": "project.settings.tab",
        "component": "ProjectSettingsTab",
        "label": "BDD",
        "order": 20
      }
    ]
  },

  "backend": {
    "wasm": "bdd.wasm",
    "permissions": [
      "db:read:tasks",
      "db:write:plugin_data",
      "http:register_routes",
      "events:subscribe:task.deleted",
      "events:emit"
    ],
    "routes": [
      {
        "method": "GET",
        "path": "/tasks/:taskId/bdd-scenarios"
      },
      {
        "method": "POST",
        "path": "/tasks/:taskId/bdd-scenarios",
        "middlewares": [
          { "name": "authn" },
          { "name": "requireFreshPassword" },
          {
            "name": "requireActions",
            "scope": "project",
            "actions": ["tasks:write"]
          }
        ]
      },
      {
        "method": "POST",
        "path": "/webhook",
        "middlewares": [
          { "name": "optionalAuthn" }
        ]
      }
    ],
    "migrations": [
      "0001_create_bdd_scenarios.sql"
    ],
    "eventSubscriptions": [
      "task.deleted"
    ]
  }
}

All routes declared under backend.routes are automatically mounted at /api/v1/plugins/{pluginId}/projects/:projectId/{path}.

Minimum Host Version (minCoreVersion)

A manifest may declare minCoreVersion as a strict X.Y.Z (or vX.Y.Z) semver string — the lowest Paca (host) build the plugin is compatible with. The host enforces this on every path that persists a manifest:

  • POST /api/v1/admin/plugins (direct install)
  • POST /api/v1/admin/plugins/marketplace/install (marketplace install)
  • PATCH /api/v1/admin/plugins/:pluginId and POST /api/v1/admin/plugins/:pluginId/upgrade (manifest updates/upgrades)

If the running build's version is older than minCoreVersion, the request fails with PLUGIN_INCOMPATIBLE_HOST_VERSION before any migrations run or the new manifest is loaded into the runtime, and nothing is persisted. Omitting the field means the plugin has no minimum. A host not built from a tagged release (PACA_VERSION unset, reporting "dev") is treated as unconstrained, since there's no release version to compare against.

The error response's error field is an English-only message meant for logs/CLI use. Clients that need to render a localized message should instead use error_details, an {"error_details": {...}} object included alongside error_code/error for this code, carrying plugin_id, required_version, and host_version as plain (non-localized) strings to interpolate into their own translated copy.

Route Middleware Policy

Each backend route can declare a host-enforced middleware chain in backend.routes[].middlewares.

Supported middleware names:

  • authn
  • optionalAuthn
  • requireFreshPassword
  • requireJWTAuth
  • requireActions

requireActions options:

  • scope: global or project (default: global)
  • projectParam: route param name for project scope (default: projectId)
  • actions: IAM actions the caller must ALL be allowed, each of the form <domain>:<verb> — built-in (for example projects:read, tasks:write) or plugin-declared (for example time_logging:manage_all). The same migration converted every customPermissions[].key and requiredPermission to an action.

Each action is decided by the host's IAM engine on a resource:

  • project scope, with the project id present in the path: project/<projectId> for a built-in action, and project/<projectId>/plugin/<plugin row id> for one of the plugin's own actions, so a role can grant a plugin's permission in one project only. Grants on project/<projectId>/* cover both.
  • otherwise: the action's platform root (users:* → user/*, roles:* → role/*, plugins:* → plugin/*, settings:* → settings, settings.sso:* → sso, agents:* → agent/*, projects:* → project), or — for a plugin's own or any other non-platform action — the plugin's own resource plugin/<plugin row id> (covered by grants on plugin/* or *).

Declaring permissions

A plugin declares the permissions it adds in customPermissions. Each key is an IAM action of the form <namespace>:<verb>, where the namespace is the last segment of the plugin id with - replaced by _ (com.paca.time-logging → time_logging:manage_all). Declared actions are known to the host while the plugin is installed: role policies may name them, the role editor lists them, and requireActions/paca.permission_check can check them. scope decides which role editor lists the permission: project (default), global, or both for an action checked inside a project and across projects (plugins that used both must be reinstalled to pick up the manifest). Two plugins cannot declare the same action. A nav item's or extension point's requiredPermission is an action too, built-in or the plugin's own, and must have the form <domain>:<verb>: a malformed value (such as settings.write) would lock the page for every user, so install/update rejects it. The web app hides nav items, settings tabs, extension points, slots and view types the caller lacks requiredPermission for (inside a project, the project's effective actions decide); the backend still enforces every route.

The legacy requirePermissions middleware (dotted permission keys) is no longer supported: installing or updating a manifest that declares it fails with "requirePermissions is no longer supported; use requireActions with IAM actions", and a stored manifest still declaring it fails closed (500) at request time. Migration 000065_plugin_manifest_actions.sql rewrote every installed manifest (requirePermissions → requireActions, each key's last . replaced by :, e.g. time_logging.manage_all → time_logging:manage_all).

Legacy permissions warning. The migration rewrites the stored manifest, but a plugin package on disk may still ship a plugin.json that declares requirePermissions. GET /api/v1/plugins flags such an installed plugin with "legacy_permissions": true (only for plugins with a backend). Administrators who can write plugins (plugins:write) then see a dismissible banner on the home page listing the affected plugins, and a "legacy permissions" badge on the plugin's card in the marketplace panel. The plugin keeps working with its converted manifest, but reinstalling or updating that build is rejected until its author ships a version that uses requireActions.

The backend's paca.permission_check host function (SDK: ctx.Permissions().Check(action)) takes an IAM action too and is decided on the same resource rule, with the request's project (if any) as the project.

If middlewares is omitted, the host applies the authenticated-only default policy:

  • authn
  • requireFreshPassword

(no action check — declare requireActions explicitly; an anonymous caller is refused, any authenticated caller is admitted).

For legacy manifests, backend.routes[].public: true is still supported and means "no host auth middleware" for that route.

Plugin Runtime Lifecycle

Startup

  1. At services/api startup, the host reads the list of enabled plugins from the plugins table.
  2. For each plugin, it loads the .wasm binary from the plugin store (local disk path or object storage URL).
  3. A wazero module instance is created per plugin with its declared host function imports.
  4. The host calls the plugin's exported Init() function, passing a PluginContext proto with the plugin's ID, granted permissions, and config.
  5. The plugin registers its route handlers by calling paca.http_register_route (or the SDK equivalent) during Init().
  6. Route registrations are applied to the Gin router under the plugin's namespace.

Request Handling

When an HTTP request matches a plugin-registered route:

  1. Gin invokes the host's plugin dispatch handler.
  2. The host serialises the request context (method, path params, body) into a proto and writes it to shared memory.
  3. The host calls the plugin's exported HandleRequest(route_id) function.
  4. The plugin reads the request, executes its logic (calling host functions as needed), and calls paca.http_respond with the response.
  5. The host reads the response from shared memory and writes it to the gin.Context.

Event Delivery

  1. When a subscribed core event fires (e.g., task.deleted), the host serialises the event payload.
  2. The host calls the plugin's exported HandleEvent(event_id) function in the plugin's WASM instance.
  3. The plugin processes the event (e.g., deleting orphan BDD scenarios when a task is deleted).

Shutdown / Cleanup

The host calls the plugin's Shutdown() export before unloading. Plugins should flush any buffered state and release resources.

Plugin-Owned Database Migrations

Plugins declare SQL migration files in their bundle. Migrations are run by the host using a restricted migration runner that only allows DDL within the plugin_data_{pluginId} schema. The runner uses the same sequential migration pattern as the core (000001_name.sql, 000002_name.sql, ...).

Migrations are run:

  • On first plugin installation.
  • On plugin upgrade, running only new migration files.
  • On plugin uninstall (optional down migrations if provided).

Resource Limits

Each WASM module instance is constrained by wazero's resource controls:

Resource Default Limit
Memory 64 MB per module instance
CPU (via instruction counting) Configurable; default prevents runaway loops
Concurrent goroutines N/A — WASM is single-threaded per instance
DB connections Plugins use the host's connection pool; max 5 concurrent queries per plugin
Inbound HTTP request body 10 MB per request to a plugin route; larger requests are rejected with 413 before reaching the plugin

Limits are configurable via environment variables (see services/api/.env.example):

Variable Default Controls
PLUGINS_MAX_CALL_DURATION 5s Max time for a single plugin function call
PLUGINS_MAX_MEMORY_PAGES 1024 (64 MiB) Max WASM linear-memory pages per module instance
PLUGINS_MAX_REQUEST_BODY_BYTES 1048576 (1 MiB) Max inbound HTTP request body / event payload size

Plugin Storage Location

The host reads plugin binaries from a configured path:

  • Local (development/self-hosted): ./plugins/dist/{pluginId}/ directory.
  • Object storage (production): S3-compatible bucket, same credentials as attachment storage.

The server config entry:

plugins:
  store: local          # or "s3"
  local_path: ./plugins/dist
  s3_bucket: paca-plugins
  cdn_allowlist:
    - https://plugins.paca.app
    - https://cdn.example.com

Outbound Requests

paca.fetch checks the request URL against backend.allowedOutboundDomains. Matching is exact and case-insensitive; the literal entry "*" allows any host. Whatever the manifest says, the URL must be https and the host must not resolve to a private or internal address. The address check runs before the request, and again at dial time on the address actually connected to, so a DNS answer that changes in between (rebinding) is still refused.

Redirects are followed. Only the dial-time address check applies to them: a redirect target does not have to be in the manifest and may use http, as long as it does not resolve to a private or internal address.

Hosts on a private network

A self-hosted service on the same LAN or VPN as Paca, such as a GitLab at gitlab.internal → 10.0.0.5, is blocked by that rule. An operator can open it with PLUGIN_FETCH_PRIVATE_HOSTS on the API service, a comma-separated list of hostnames. It is empty by default.

PLUGIN_FETCH_PRIVATE_HOSTS=gitlab.internal,git.corp.example

A request to a private address then needs all of the following:

  • The hostname is in PLUGIN_FETCH_PRIVATE_HOSTS. Entries are exact ASCII names: IP literals, wildcards, URLs, host:port values, names with a leading or trailing dot and non-ASCII names are ignored, with a warning in the API log at startup. Write an internationalized name in its punycode form (xn--...).
  • The plugin's manifest names the host in allowedOutboundDomains. "*" does not count, because plugins that use it call user-supplied URLs. A plugin whose manifest does not name a listed host cannot call it at all, whatever it resolves to, and the dialer checks the manifest again on new connections. So neither a DNS answer that changes after the first check nor a pooled connection opened for another plugin opens a listed host to a "*" plugin.
  • The URL is https, and the certificate is trusted by the API container. For an internal CA, mount a bundle holding the system roots and your CA into the api service and set SSL_CERT_FILE to its path, in a Compose override or the Helm chart's pod spec.
  • The address is in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 or fc00::/7.

A redirect to a listed host must meet the same conditions; redirects to other hosts follow the rule above. Hostnames are compared in the IDNA (punycode) form that is actually dialed, so a Unicode spelling of a listed name counts as that name. The connection stays pinned to the checked address. Everything else stays blocked even for a listed host: loopback, link-local (including 169.254.169.254 and other cloud metadata endpoints), the CGNAT range 100.64.0.0/10, and IPv6 transition addresses, whatever IPv4 address they carry.

Security: listing a host lets the plugins that name it reach that internal service, and the API sends them its responses. List only hosts those plugins are meant to talk to, keep the list short, and treat a change to it like a firewall change. The list also trusts whoever controls the DNS records of those names: they can point a listed name at any other private address, so use names from DNS you control. Tailscale addresses sit in the CGNAT range and are not reachable this way.

Security Considerations

  • WASM modules have no filesystem access (wazero's WASI filesystem is not mounted).
  • WASM modules have no raw network access; all external communication must go through host functions.
  • All DB host functions enforce project-scope isolation — a plugin enabled for project A cannot query project B data.
  • Plugin WASM binaries should be signed by the publisher. The host verifies the signature against a public key stored in the plugin manifest before loading. (v1: signature check is enforced for third-party plugins; first-party plugins bypass in dev mode.)
  • WASM execution errors are caught by wazero and converted to 500 responses; the host never panics due to a plugin crash.
  • Inbound request bodies are capped (see Resource Limits) before being handed to a plugin. Plugin SDK allocators are simple bump allocators with no bounds checking of their own, so an oversized payload would otherwise advance a plugin's allocator cursor past the end of its memory; the host always resets a plugin's allocator after each call, including on failure, so a single bad request cannot leave an instance unable to serve later ones.
  • Secrets (e.g., API keys the plugin needs) are stored encrypted in the host's secrets store and passed to the plugin through paca.config_get — never baked into the WASM binary.