Running an arkush instance with Docker Compose

docker-compose.yml is the host-agnostic base stack: the API and SPA in one container, with app-owned sign-in — the service signs people in over Google itself, so no identity-proxy sidecar exists. Pair the base with an instance override that wires networking and TLS.

This guide is written for an operator with Docker and a shell, on a host with a real disk. Every act in it works from the image and this guide alone. The image carries no source tree.

It describes what the software does and grants nothing. Which versions an install may run, and for how long, is the agreement's to say.

For other deploy shapes:

• A disposable test instance on your own machine, without sign-in or GCP wiring — the harness in deploy/local/, which needs a source checkout.

The hostname

Serve the instance from a hostname on your own domain — arkush.example.org is the shape. You then hold the DNS record and the certificate, and the instance depends on no name somebody else controls.

It also decides what Google will let you do. An OAuth client's authorized domain is verified in Search Console by whoever owns the Cloud project, and that project is yours. A redirect URI on a domain you do not own therefore cannot be verified, which closes the External user type — the one an install needs when it admits accounts from outside a single Google Workspace. An Internal client skips verification, so a Workspace-only instance never meets this.

ARKUSH_PUBLIC_URL carries the address. The sign-in redirect URI, the authorized JavaScript origin, every share link and the MCP endpoint are derived from it, so moving the hostname later means re-registering the OAuth client and reissuing every link people hold.

Identity: state the mode

ARKUSH_IDENTITY says how this deployment authenticates people. It has no default, and the service refuses to boot without it: how a deployment authenticates people is the one thing that must never be guessed.

A deployment states one of two values:

Every other identity variable belongs to exactly one of them:

One set under the other mode refuses the boot by name. The deployment would run the mode it stated and ignore what you configured, so the refusal says which variable belongs to which mode. Under signin a missing half of the client pair refuses the boot too.

WARNING: a third value, dev, exists for running the software on a developer's own machine. It authenticates every tokenless caller as ARKUSH_DEV_USER, which is authentication turned off. It is not a deployment posture and no install uses it: the service refuses to boot under it unless ARKUSH_PUBLIC_URL is provably a localhost origin. It is named here so that a boot refusal quoting it reads, never as a third option to pick.

App-owned sign-in: ARKUSH_IDENTITY=signin

Google is the one sign-in provider. Configure it once in the Google Cloud console, then set the sign-in variables.

1. In the Cloud project that owns the client, open APIs & Services → OAuth consent screen. Pick the user type: Internal admits only accounts in your Google Workspace and needs no verification, External admits any Google account.
2. Set the app name and the home page URL to the instance's own name and origin. Google's brand check compares them with the landing page.
3. Under Credentials, create a Web application OAuth client. Add https://<hostname>/auth/callback/google as an authorized redirect URI.
4. Set ARKUSH_IDENTITY=signin, then set ARKUSH_SIGNIN_GOOGLE_CLIENT_ID and ARKUSH_SIGNIN_GOOGLE_CLIENT_SECRET to the client's values. Both, or the boot refuses.
5. Set ARKUSH_SIGNIN_ALLOW to who can sign in, as comma-separated entries. An entry is * (any account Google verifies), @example.org (one domain, subdomains excluded), or a full address. Unset means that nobody signs in.
6. Start the stack.
7. Open the landing page and sign in.

Sign-in asks Google for the openid and email scopes (src/api/signin.ts). Neither is sensitive, so sign-in alone needs no Google verification. The service keeps no Google token: it reads one ID token, refuses an unverified address, and mints its own session cookie. The allowlist is re-checked on every request, so an entry you remove ends that person's access at once.

BigQuery in the browser is a second, separate grant. A user_oauth datasource runs in the person's own browser under their own Google account. The browser asks for bigquery.readonly on the client that ARKUSH_BROWSER_GOOGLE_CLIENT_ID names, and that client needs https://<hostname> as an authorized JavaScript origin. bigquery.readonly is a sensitive scope. Until Google verifies the app for it, the consent screen shows a warning. The count of accounts that can grant it is then capped for the life of the project. Verification needs the privacy policy and the terms live at /privacy and /terms on the instance's own domain. An admin writes both on the admin page. Google then reviews the client. A sheet a person reads under their own account uses the non-sensitive drive.file scope through the Google Picker, which needs ARKUSH_BROWSER_GOOGLE_API_KEY and ARKUSH_BROWSER_GOOGLE_PROJECT_NUMBER. One web client can carry both the redirect URI and the JavaScript origin. An install that gives nobody browser BigQuery leaves ARKUSH_BROWSER_GOOGLE_CLIENT_ID unset and needs no verification.

Cross-site requests

The session cookie is HttpOnly and SameSite=Lax, and Secure when ARKUSH_PUBLIC_URL is https. Under https the cookie is named __Host-arkush_session, so only this host over TLS can set it, and the service ignores the bare name. A state-changing request (POST, PUT, PATCH, DELETE) whose Sec-Fetch-Site header reads cross-site answers 403 before any route reads it. A request that carries a bearer token is exempt, because the token is the credential: the MCP endpoint and /refresh/tick authenticate that way. A request with no Sec-Fetch-Site header passes. That is how an agent's POST /oauth/token and every other machine client arrive: none of them sends fetch metadata. The service sends no CORS header, so a cross-origin fetch that needs a preflight fails the preflight, and one that needs none gets an answer the page cannot read. Under header trust the front's session cookie authenticates the request, its attributes are the front's to set, and the fetch-metadata check runs behind the front unchanged.

Header trust: ARKUSH_IDENTITY=proxy

An identity-aware proxy in front signs people in and passes the authenticated email in one request header. The service reads the header that ARKUSH_AUTH_HEADER names and takes the part after the last colon, so IAP's accounts.google.com: prefix drops off (src/api/identity.ts).

1. Set ARKUSH_IDENTITY=proxy. The three ARKUSH_SIGNIN_* variables stay unset — set under this mode, one of them refuses the boot.
2. Set ARKUSH_AUTH_HEADER to the front's header, unless the front is IAP.
3. Set ARKUSH_LOGIN_URL to where an anonymous visitor signs in. The default /app is right when the front gates /app: the front itself asks the visitor to sign in. A front that lets /app load anonymously names its own sign-in start instead (oauth2-proxy: /oauth2/start). The app appends the rd= return path.
4. Set ARKUSH_LOGOUT_URL to the front's sign-out endpoint (oauth2-proxy: /oauth2/sign_out). Unset, the app shows the identity and no Sign out.
5. Configure the front's skip list from the printer below.

A Microsoft Entra ID or Okta organization puts an OIDC proxy such as oauth2-proxy in front. That proxy signs people in with the organization's provider, and the service runs header trust behind it. The service has no second sign-in provider of its own.

WARNING: In header-trust mode the container trusts the identity header without checks. A path that reaches the container around the identity layer lets any caller name any email and get that person's rights. Route every request through the front, and publish no host port on the container.

#### What a header-trust install opens at its front

The front challenges every request that its skip-auth list does not name. Three groups must answer without a session:

• the public pages — the landing page, the guides, the privacy policy, and the terms
• the stylesheets and icons those pages load
• the machine endpoints that carry their own authentication — the health probe, the MCP endpoint, the refresh tick, and the agent OAuth routes

A path missing from the list breaks a public page, and nothing reports the failure. Take the paths from the printer below, never from the groups above. The build hashes the stylesheet names, so no prose can spell them.

PUBLIC_PATHS in src/api/public-paths.ts is the list of record. The image carries the printer as a bundled script. Print the list in the shape the front reads:

docker run --rm --entrypoint node <image>:<tag> /app/scripts/proxy-paths.mjs
docker run --rm --entrypoint node <image>:<tag> /app/scripts/proxy-paths.mjs --format regex
docker run --rm --entrypoint node <image>:<tag> /app/scripts/proxy-paths.mjs --format oauth2-proxy

The first form prints each path with its reason, the second one anchored pattern per line, the third the --skip-auth-route flags.

An install that sets ARKUSH_ALLOW_ANONYMOUS=true adds --anonymous. The flag adds the paths a visitor needs to use the app without an account. An org install prints the list without the flag.

Keep /oauth/authorize out of the skip list. The consent page is authenticated like the app, because a person approves an agent's grant under their own identity.

Print the list again after every upgrade, and compare it with the config. The list grows when the app gains a public page.

CAUTION: oauth2-proxy answers /robots.txt itself, ahead of its own skip list. The printed oauth2-proxy output states what it answers, what that costs, and where the override goes.

The first admin

ARKUSH_ADMINS names who manages the instance from inside the app. An admin holds the admin page, the Users page, groups, every agent grant, and every dashboard's access. Unset means that nobody is an admin, and the admin page never appears. * refuses boot: admin is power over other people's things.

1. Put the first administrator's address in ARKUSH_ADMINS. An entry is a full address or group:<name>.
2. Recreate the container. The service reads its configuration at boot only.
3. Sign in as that address. The account menu in the toolbar gains Administration, which opens the admin page at /admin. The section links on the full pages gain Users beside it.

ARKUSH_ANALYSTS is the second right: who curates the theme library, the instance fonts and the datasource library, and who reads a private address under the service's own Google. It does not reach the warehouse — a BigQuery run answers to Google's policy on the account the datasource names. Unset means that nobody is an analyst. The single value * makes every signed-in person an analyst, which is the posture a public deployment states on purpose. * beside named entries refuses boot. Authoring needs neither right: anyone signed in creates and edits dashboards.

Offboarding a person

The identity is the email address. When a person leaves:

1. On the Users page, open the person's row and use Revoke all under Connected agents. One act ends every agent grant the account holds and signs out every browser signed in as the account. The answer names both counts.
2. Remove the address from ARKUSH_SIGNIN_ALLOW, or disable it at the identity front, so the person cannot sign in again. An allowlist that names the address also stops any agent token the account still holds: every token use and every refresh re-checks the allowlist.
3. On the Users page, transfer the person's dashboards with Transfer all under Dashboards they own, and remove their direct shares with Remove all shares under Shared with them. Or leave them shared as they stand. Shares through a group end when the group's membership changes.

NOTE: A * or @example.org allowlist entry does not name people, so step 2 removes nothing. Step 1 is what ends the access in those configurations.

What a departure stops on its own. A scheduled refresh runs as the person whose run last blessed the datasource, and the sweep asks about that person before every run. Take their Service Account Token Creator role away in Google Cloud and every schedule resting on their blessing pauses within the hour — the service keeps each policy answer for one hour. Removing a person in Google Cloud is therefore what ends their scheduled spending, and no step in this app does it. A schedule that reads a scoped address rests on the analysts allowlist instead, and pauses as soon as the allowlist stops naming the blesser: at the next container recreate for a named entry, at the membership change for a group: entry.

Two places say what stopped. POST /refresh/tick answers an unauthorized list — one entry per datasource, with the sentence that names what is missing and, for a Google role, the command that grants it (the route needs ARKUSH_REFRESH_TOKEN). Every refusal also logs one ds_refresh_unauthorized line naming the document, the datasource and the blesser, which is what the timer's own sweeps leave behind; the refresh_tick line carries the count. The dashboard itself still reads as scheduled, so nothing an author opens says so. Anyone who may still run the datasource restarts its schedule by refreshing it once.

WARNING: An address the organization later gives to a new person inherits whatever the address still holds — ownership, shares, and rights follow the email. Complete the steps above before an address can be reused.

What an override supplies

1. The Traefik labels or other routing for the host, plus TLS.
2. The env file. The base compose file passes through the variables it names, so the env file carries at least ARKUSH_IMAGE, ARKUSH_HOSTNAME, ARKUSH_IDENTITY, ARKUSH_SIGNIN_GOOGLE_CLIENT_ID, ARKUSH_SIGNIN_GOOGLE_CLIENT_SECRET, ARKUSH_SIGNIN_ALLOW, and ARKUSH_ADMINS. A variable the base file does not name goes into the override's own environment block. The table below is the complete list.
3. The compose project name, with -p. The base name arkush is deliberate: the default project name is the directory name, deploy, which collides easily.

Anonymous access is an env concern too. An instance that offers account-free use sets ARKUSH_ALLOW_ANONYMOUS=true. An org install does not. The difference stays configuration, never code.

Environment variables

The service reads its configuration from the environment at boot and never again, so a changed variable needs a recreated container. The listening log line at boot prints the effective configuration — read it to confirm what the service took. A default marked image is what the published image sets, or resolves to from its working directory.

The renamed and removed variables refuse boot by name: ARKUSH_OAUTH_CLIENT_ID, ARKUSH_GOOGLE_API_KEY, ARKUSH_GOOGLE_PROJECT_NUMBER, ARKUSH_BQ_BILLING_PROJECT, ARKUSH_INSTANCE_NAME, ARKUSH_LOGO_URL, ARKUSH_SECURITY_CONTACT, and ARKUSH_HANDBOOK_DIR. The refusal names each one and what replaced it — the four browser variables took a BROWSER prefix, the next three became the instance identity an admin sets on the admin page, and the last was read by nothing. A line left in an env file after an update would otherwise mean nothing and say nothing.

An `ARKUSH_` name this service does not read refuses the boot too. The environment is a flat namespace with no spell-check, so ARKUSH_ADMIN in place of ARKUSH_ADMINS used to start normally and grant nobody the admin right. The service now stops and names the line. Check the spelling against the table above, or unset the name.

How a value is written. Every setting takes one of the grammars below. All but the list take a closed set of spellings, and a value outside it refuses the boot rather than reading as the default:

• A true or false setting takes the word true or the word false, in any case. 1, yes and on are refused.
• A list is comma-separated, and takes any entry. Blank entries are dropped, and the entries are matched without regard to case.
• A byte size is a plain number of bytes, or a number with a KB, MB or GB suffix, as in 500MB.
• A duration is a whole number of minutes, or off.
• A signing key is 32 random bytes as 64 hex characters, the output of openssl rand -hex 32. ARKUSH_SESSION_SECRET is the one setting that takes it.

Mounting a value as a file. Every ARKUSH_ setting also reads <NAME>_FILE. Point it at a file and the service reads the value out of it, trimming the trailing newline — the shape Docker and Kubernetes secrets produce:

ARKUSH_SIGNIN_GOOGLE_CLIENT_SECRET_FILE=/run/secrets/google-client-secret

Set both and the direct value wins, so an override for one boot needs no unmounting. A path with no file at it refuses the boot.

NOTE: the settings that already end in _FILE — the registries and the instance pages — name a file the service reads for its own sake. They are unaffected by the rule above.

Get the image: a registry pull, or a tarball

A version reaches a host two ways. Pick one; both carry the same bytes, and the digest in the release set names them.

A registry pull. The deployment names the image and its tag, and docker compose pull fetches it. A licensed install is granted read on the vendor's registry, so a mirror job can fetch each tag on its own schedule. Your registry address, and the identity that reads it, belong to your own runbook.

A tarball. Where a security review scans a file before anything enters the network, take the release set for the version and load the image:

cosign verify-blob SHA256SUMS --bundle SHA256SUMS.cosign.bundle \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity https://github.com/<owner>/<repo>/.github/workflows/publish-image.yml@refs/tags/v<version>
sha256sum -c SHA256SUMS
docker load < arkush-v<version>.image.tar.gz

Verify the signature before the checksums: sha256sum -c over a list nobody verified proves that the files match the list, and nothing about who wrote it. The certificate identity is the one Verify the image below explains, at the same tag. docker load brings the image in as arkush:v<version>. Put that name in ARKUSH_IMAGE, or retag the image first.

The release set holds the image tarball, those checksums and their signature, the two SBOM files, the source maps, a file naming the registry digest, and the release note — read it before updating. The note says whether the version asks anything of an operator. The changelog inside the app says the same thing, and it is readable only after the update.

The tarball carries the linux/amd64 image. An arm64 host takes the registry pull.

Keep the source maps. They are the only way a stack trace from your install maps back to source: the image runs a minified bundle, and the vendor has no route into your host to look.

Deploy

An install needs three files on the host: the base compose file, an instance override, and an env file. The base compose file rides in the image. Take a copy of it:

mkdir -p arkush/deploy
cd arkush
docker run --rm --entrypoint cat <image>:v<version> /app/deploy/docker-compose.yml \
  > deploy/docker-compose.yml

Write the env file and the instance override in arkush/, next to deploy/. The start command below runs from arkush/ and reads both from there. The env file carries at least ARKUSH_IMAGE, ARKUSH_HOSTNAME, ARKUSH_IDENTITY, the sign-in client pair, ARKUSH_SIGNIN_ALLOW, and ARKUSH_ADMINS. What an override supplies above names what the override carries, and the table below is the complete list of variables.

ARKUSH_IMAGE is the image and the tag this install runs:

ARKUSH_IMAGE=<image>:v<version>

Compose refuses to start with it unset, and the refusal names it. Pin the version tag. latest and main move to another version at the next recreate, and the version an incident starts from is then unknown.

Start the stack:

docker compose -p <project> \
  -f deploy/docker-compose.yml \
  -f <instance-override>.yml \
  --env-file <instance>.env \
  up -d

The first start creates the three volumes. Volumes below names what each one holds.

Confirm the install:

1. Open the instance origin and sign in as an address ARKUSH_SIGNIN_ALLOW admits.
2. Make sure that the account menu carries Administration. An address in ARKUSH_ADMINS is the first admin — see The first admin above.
3. Read GET /healthz. It answers the version and ready: true.

The image carries the built app, the API service as one minified bundle with its runtime packages, the handbook, the base compose file, and the two operator scripts. It carries no source tree and no test.

The tags a version publishes. A release publishes v<version> and moves latest. Each build of the main branch publishes main and that commit's own sha-<commit>. An install pins v<version>. A deployment that tracks the main branch pins main and takes each merge.

To build the image from a source tree instead, add -f deploy/build.override.yml and --build to the command above. ARKUSH_IMAGE then names what the build is tagged as. An install needs neither: the image is the delivery.

Verify the image

Every release tag is signed and shipped with its bill of materials. Before the first pull of a version, verify the signature with cosign 3 or later, logged in to the registry as for a pull:

cosign verify ghcr.io/<owner>/arkush:v<version> \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity https://github.com/<owner>/<repo>/.github/workflows/publish-image.yml@refs/tags/v<version>

The identity names the source repository, which the image name does not carry: <repo> is the repository's own name, and the source label the docker inspect command below prints carries it in full.

A tag with no signature failed its build or its vulnerability scan. Do not deploy it. The SBOM is two CycloneDX files — the npm one lists every package the software contains, the image one lists what the image's filesystem holds. Both ride in the release set for the version, which is the durable copy, and both are also the sbom-v<version> artifact on the tag's workflow run, kept for the repository's artifact retention period (90 days by default). docker inspect --format '{{json .Config.Labels}}' <image>:<tag> answers the version, the commit, and the source repository.

The image carries the third-party notice at /app/THIRD-PARTY.md and the Apache License 2.0 text at /app/licenses/Apache-2.0.txt, beside the API bundle. To read them without a running container:

docker run --rm --entrypoint cat <image>:<tag> /app/THIRD-PARTY.md /app/licenses/Apache-2.0.txt

The image carries this guide, the security overview, the disclosure policy, and the changelog under /app/docs/: deploy.md, security-overview.md, SECURITY.md, CHANGELOG.md. A person who holds the image holds the documentation for its version. The same command reads them.

The same set is published at https://docs.arkush.app, which needs no image and no account — read it before an install exists, and read the copy in the image once one does, because that copy is the one that matches the version running.

Runtime hardening

The stack runs the container with no Linux capabilities, no path to gain privileges, a read-only root filesystem, and a ceiling of 512 tasks. The process writes only to the three volumes and to /tmp, a memory-backed filesystem capped at 1 GiB where the server DuckDB engine stages each run's inputs. An uploaded file reduced server-side is staged whole, so a 512 MiB file takes 512 MiB of memory for the length of one run. Size the host for it. A staging that exceeds the cap fails with a disk-full error and takes nothing else down. An override that mounts a hand-edited file directory or a credential keeps working. A path the process must write to is mounted, because everything outside the mounts and /tmp is read-only.

Versions and upgrades

Version numbers. A release is year.release.fix: 2026.1.0 is the first release of 2026, 2026.1.1 its first fix tag, 2026.2.0 the next feature release. The middle number restarts at 1 each January. A fix tag carries fixes only. It changes no setting, no format or schema version, no MCP tool and no public path — the release check refuses a fix tag on which one of those moved — so take a fix tag without reading further. A release tag can change any of those, and its changelog entry has an Operations item when it does. A setting or a tool that is renamed or removed keeps working until the first release that is both in a later year and at least 90 days after the release that announced the change. Until then the boot logs one line naming the successor, and a tool says so in its description. After removal, the old setting refuses the boot and names its successor. No fix is backported to an older release. To move an install that is several releases behind, take the releases in order and read each Operations item.

state.db carries a schema version in SQLite's user_version header. The version this image reads is SCHEMA_VERSION in src/api/db.ts. At boot, before the first request, the service compares the stored version with its own:

• Equal. The service starts.
• Older. The service runs every pending step in one transaction and stamps the new version. Before the first step it writes a copy of the database beside it, named for the version the plane came from: state.db.before-v3 for a plane that was at version 3. The copy replaces the previous copy, so the state directory holds one copy at most. The boot logs one schema_upgrade line with the versions, the steps, and the copy's path.
• Newer. The service refuses to start. The message names the plane's version, the highest version this image reads, and the release that introduced that version.
• Zero. A plane from before the stamp. Every boot before the stamp applied the schema version 1 names, so the service reads the plane as version 1 and continues from there as Equal or Older. An image at version 1 stamps it and writes no copy.

To upgrade an instance:

1. Read the release note in the release set, or the release's changelog entry. An item that starts Operations names what the update needs from you.
2. Set ARKUSH_IMAGE to the new tag in the env file.
3. Fetch the new version. Run the start command under Deploy above with pull in place of up -d, or docker load the new tarball.
4. Run that start command. Compose replaces the container with one on the new image. The volumes stay as they are.
5. Make sure that the boot log shows listening. If it shows a schema_upgrade line, the copy it names is your rollback target.

Rollback. In one schema version, rollback is the previous image on the same state directory. Across a schema step, rollback is the copy:

1. Stop the stack.
2. Replace state.db with state.db.before-v<N>. <N> is the version the plane had before the upgrade.
3. Delete state.db-wal and state.db-shm beside it, if present.
4. Set ARKUSH_IMAGE back to the previous version's tag.
5. Start the stack.

WARNING: The copy is the plane as it stood before the first step. Every record written after the upgrade is lost with the rollback. The bytes beside the database stay as they are.

The copy rides the state backup. Delete it when the upgrade is proven, or keep it: the next schema step replaces it.

CAUTION: An image from before the state plane is not a rollback target. Those images read no SQLite database. Pointed at a state directory, one serves an empty instance where every dashboard looks deleted. Rolling back across that line is a restore from backup.

Volumes

The stack declares these volumes:

arkush-state is the instance. state.db holds every server-owned record — dashboards and their capped version history, the shared-datasource library, the uploaded-file entries, sessions, agent grants, the two generated instance secrets, the theme library, the font index, the user registry, and the upload-quota ledger. blobs/ and fonts/ beside it hold the bytes: snapshot payloads, uploaded originals, and font faces. The session-secret file sits beside them when the deployment sets no ARKUSH_SESSION_SECRET (Rotate a secret, below).

The landing and pages volumes mount inside the state path, and they are separate volumes. A copy of the state volume alone does not hold them.

The YAML registries — the groups file or the Google group registry, the service accounts, the destinations, the ask — live at the paths their variables name, on a mount the override adds. Mount the directory that holds them, never one file. A single-file bind mount pins the inode, and an editor's atomic rename freezes the container's view The image pre-creates /data/groups, owned by the container user, as one such directory inside the state volume. A registry there rides the state backup. A registry mounted elsewhere is its own backup item.

CAUTION: arkush-state must be a real local disk. A network or object-storage mount (NFS, SMB, a Cloud Storage FUSE mount) has no working file locks, and SQLite corrupts on one.

Run exactly ONE API container per state directory. The service refuses a second one at boot: one SQLite file has one writer. A second process would also hold its own copy of the state that lives in process memory, and the two copies would diverge. That state is the live event hub's streams, the refresh sweep's in-flight flag, the directory-membership cache, the pending agent-consent codes, and the file writer's queue over the hand-edited YAML files and the instance pages.

Monitoring

GET /healthz answers 200 with the version and ready: true, and 503 with ready: false when the state directory cannot take a write or the database handle is closed. The route needs no identity and reads no document, so a container health check can poll it. Point an external uptime check at the same route. Behind Google's edge (IAP, Cloud Run) the /healthz path is reserved and answers Google's own 404 before the request reaches the service, so an external check there targets /, the landing page.

The API writes structured JSON lines to the container's stdout. Every answered request writes one line with its status and wall-clock time. Read those lines to find slow endpoints. Pin the log driver to journald in the instance override. The default json-file driver holds the logs on the container, so every recreate — an update among them — deletes them, which loses the usage events.

Backup: every path

A backup holds these items. The compose volumes are the checklist for the first three.

Two generated instance secrets — the agent OAuth key and the snapshot attestation key — are rows in state.db (src/api/db.ts). A restore of the database restores them, so agent grants and schedules survive a restore without a step of their own. The session key is not in the database (src/api/session-secret.ts): it is the session-secret file beside it, or the value in ARKUSH_SESSION_SECRET. A restore that brings the file back keeps everyone signed in. A restore of the database alone signs everyone out once, and mints no session for whoever holds the copy.

Keep the copy of the env file and the key file in a secrets store, never beside the data. A lost key is re-minted. A leaked one is not un-leaked.

Back up on a schedule. A copy on the same disk protects against an app bug or an accidental deletion. Keep a copy on another host for disk loss or host compromise.

Stop the stack to take a backup. The running service holds SQLite's exclusive lock on state.db, which is what keeps a second container off the volume — so no other process can open the database, sqlite3 and VACUUM INTO included. A stop is also what makes the backup COHERENT: the records and the bytes are two planes, and a copy taken at different moments while writes land produces a backup whose entries name snapshots the copy does not have.

Compose prefixes each volume with the project name. List them with docker volume ls. Then copy every volume through a helper container:

docker compose -p <project> stop arkush
for v in arkush-state arkush-landing arkush-pages; do
  docker run --rm -v "<project>_$v:/from:ro" -v /path/to/backup:/to alpine \
    sh -c "mkdir -p /to/$v && cp -a /from/. /to/$v/"
done
docker compose -p <project> start arkush

A bind-mounted state directory copies with cp -a from the host path instead.

A clean stop checkpoints and removes the write-ahead log, so the state copy is state.db plus the byte trees beside it.

A scripted backup checks that the log is gone rather than assuming it. A state.db-wal still beside the database means the stop did not finish, and a copy taken then drops the tail of the database — a backup that restores without complaint and is missing the last writes. Refuse the copy, start the service again, and report the refusal:

if [ -e /path/to/backup/arkush-state/state.db-wal ]; then
  echo "state.db-wal still present after stop — nothing copied" >&2
  exit 1
fi

Restore on a fresh host

1. Install Docker. Put the compose files, the override, the env file, and the YAML registries on the host, at the paths the override names.
2. When the deployment has a key file, mount it read-only at the path GOOGLE_APPLICATION_CREDENTIALS names.
3. Create the stack and do not start it, so the volumes exist: docker compose -p <project> -f ... up --no-start.
4. Copy each volume back through a helper container, and set the owner to the container user (uid 1000):

   for v in arkush-state arkush-landing arkush-pages; do
     docker run --rm -v "<project>_$v:/to" -v /path/to/backup:/from:ro alpine \
       sh -c "cp -a /from/$v/. /to/ && chown -R 1000:1000 /to"
   done

5. Set state.db to mode 0600. It holds every session, every agent grant, and the generated instance secrets:

   docker run --rm -v "<project>_arkush-state:/to" alpine chmod 600 /to/state.db

6. Delete any state.db-wal and state.db-shm left beside the database.
7. Start the stack on the image tag the backup came from.

Each document carries its own version history in the same database, so a restored document comes back with its history. There is no separate history store to reconcile.

Prove the restore before you rely on it. A file that exists is not evidence that it restores. Rehearse on a fresh host every quarter, and after every change to the backup script. The rehearsal passes when all of these hold:

1. The listening log line prints the state directory, the public URL, and the admin count you expect.
2. A known dashboard opens and its charts draw. That proves state.db and blobs/ agree.
3. A known uploaded file opens from the Data workspace.
4. A person signed in before the backup is still signed in. That proves the session key came back: the session-secret file beside the database, or the same ARKUSH_SESSION_SECRET in the env file.
5. On the Users page, a connected agent still lists its grant, and one agent call succeeds without a new consent. That proves the agent OAuth key.
6. A datasource with a schedule refreshes at the next sweep without a pause. That proves the attestation key.
7. The landing page, /privacy, and /terms show the instance's own words.
8. The admin page lists the groups, the service accounts, and the destinations the registries hold.

Decommission

The data stays readable after the software is gone. Every record is a row in state.db, an SQLite database. Every byte is a plain file beside it. The service encrypts nothing, and no key of the vendor's unlocks anything.

1. Stop the stack: docker compose -p <project> stop arkush.
2. Copy every item in the backup table above. The copy is the export.
3. Make sure that the copy opens. Each dashboard is one JSON document in the documents table, in the docs store:

   sqlite3 /path/to/backup/arkush-state/state.db \
     "SELECT id, length(text) FROM documents WHERE store = 'docs'"
   sqlite3 /path/to/backup/arkush-state/state.db \
     "SELECT text FROM documents WHERE store = 'docs' AND id = '<id>'" > <id>.json

   The `datasources` store holds the shared-datasource library the same way, and the `files` store holds the entry of each uploaded file. Snapshot payloads are Parquet files under `blobs/docs/` and `blobs/datasources/`. Uploaded originals are the files a person uploaded, under `blobs/files/`. Standard tools read all of them.

4. Remove the stack and its volumes: docker compose -p <project> down --volumes. CAUTION: this command deletes the volumes. Run it after step 3 and not before.
5. Remove the image: docker image rm <image>:<tag>.
6. Revoke what the install held outside the host:
   • the Google OAuth client of app-owned sign-in and the browser client, in the Google Cloud console,
   • the service's own Google identity: delete the key file, or remove the roles of the identity,
   • the impersonation grant of each registered service account to the service identity,
   • the Slack bot tokens in the destinations registry.

   Sessions and agent grants end with state.db. Nothing about the install is held anywhere else.

What a restart loses

The service writes every record to state.db before it answers the request. A restart loses no dashboard, no share, no grant, and no session. The state that lives in process memory is what a restart drops, and each item recovers on its own:

• Live event streams. Every open dashboard loses its stream and reconnects on its own. Presence — who is viewing — rebuilds from the reconnects.
• A sweep in flight. The shutdown cancels it. A datasource mid-run keeps its previous snapshot. The first sweep after boot runs one minute in and runs what is due.
• Pending agent consent codes. An agent mid-consent starts the consent again. A code lives five minutes in any case.
• The directory-membership cache. Under the Google groups backend, the first request that needs a group reads it from Google again.
• The OAuth rate-limit counters. They start from zero.
• The header-trust arrivals ledger. The first request an address makes after the boot logs its signin line again, once per address.
• A queued file write. A hand-edited YAML file or an instance page keeps its last complete content. An edit the writer had not yet renamed into place is lost, and the editor's next save writes it again.
• The upload-quota ledger and the Google token getters. Both rebuild at boot from the records and the credential.

On SIGTERM the service stops the timer, ends every stream, closes, and exits in at most eight seconds. docker compose stop waits ten seconds by default, so the default suffices. The first boot after an upgrade can spend a few seconds on a projection_backfill pass before the first request. It logs one line with the count.

Capacity

The service runs as exactly one process with one SQLite writer. There is no horizontal scaling: a bigger host is the scaling path. The ceilings below are constants in the tree, and none is a setting.

• Data per dashboard. 50,000 rows per snapshot (src/core/document.ts), and a 40 MiB request body. There is no per-dashboard byte ceiling: a snapshot's rows sit in the snapshot store beside the dashboard, not in the dashboard document.
• Uploaded files. 512 MiB per file (src/core/file-entry.ts). A per-person total applies only when ARKUSH_UPLOAD_QUOTA is set.
• Disk. Every save keeps a full copy of the dashboard, up to 50 versions per dashboard (src/api/sqlite-store.ts). The version archive is the bulk of state.db on a busy instance, so plan disk for it. A trashed dashboard is purged 30 days in.
• Reclaiming disk. SQLite reuses freed pages and never shrinks the file on its own, so state.db stays at the largest size it reached. To give the space back, stop the service and rebuild the file:

  sqlite3 /path/to/state.db "VACUUM INTO '/path/to/state.db.compact'"
  rm -f /path/to/state.db-wal /path/to/state.db-shm
  mv /path/to/state.db.compact /path/to/state.db
  chmod 600 /path/to/state.db

  The service holds an exclusive lock while it runs, so the stop is required. A write-ahead log belongs to the database it was written for: delete it with the file it replaces. Set the new file to mode 0600 — it holds every session, every agent grant, and the generated instance secrets. Take a backup first. This step is optional: an instance that is not short of disk never needs it.

• Upgrade copy. An upgrade across a schema step keeps one copy of state.db beside it, state.db.before-v<N> (src/api/db.ts), until the next schema step replaces it or you delete it. Plan disk for twice the database.
• Live streams. 50 per dashboard and 200 in total. Over the cap the route answers 503 (src/api/events.ts). A stream is cut and reconnected every 15 minutes.
• Container limits. The compose file caps the engine's staging space at 1 GiB of memory-backed /tmp and the container at 512 tasks. Each option carries its reason in the compose file.
• Run budgets. A server-side query, BigQuery or DuckDB, stops at 30 seconds. A request stops at 300 seconds.
• Sweep. Every five minutes by default, the first one minute after boot, never two at once.
• Identity. Sessions last 30 days and renew on use. Agent access tokens last 4 hours, refresh tokens 30 days. The agent registration, consent, and token endpoints answer 429 past 30 requests per minute per client IP.
• Group membership. The Google directory answer is cached for 10 minutes. Through an outage it is served stale for at most 24 hours, or one hour for a group the admin list names.

The project publishes no CPU or memory requirement. Every server-side run logs its wall-clock time, and a DuckDB run logs the engine's memory in use. Size the host from those lines on your own data.

Encryption at rest

The service encrypts nothing at rest, and claims nothing. state.db holds every session, every agent grant as token hashes, and the two generated secrets as plain hex; the session-secret file beside it, when one exists, is plain hex too. The blob tree holds snapshot rows in clear. Encryption at rest is the host's job: an encrypted disk or volume under the state directory, and the same under every backup copy.

Rotate a secret

Each secret rotates on its own, and none of the rotations touches the others.

The Google client secret (ARKUSH_SIGNIN_GOOGLE_CLIENT_SECRET):

1. Issue a new secret on the same OAuth client in the Google Cloud console.
2. Set the new value in the env file.
3. Recreate the container.

Nobody is signed out: the session cookie is signed with the generated session key, never with the Google secret.

The tick bearer (ARKUSH_REFRESH_TOKEN):

1. Set a new value in the env file.
2. Recreate the container.
3. Update the caller that fires POST /refresh/tick.

The in-process timer does not use it.

A Slack token. Change the token field in the destination registry, on the admin page or in the file. The service reads the file again on the next send. No restart.

The key file (GOOGLE_APPLICATION_CREDENTIALS):

1. Mint a new key for the service's own Google identity in GCP.
2. Replace the mounted file.
3. Recreate the container.
4. Delete the old key in GCP.

The session key (ARKUSH_SESSION_SECRET, or the session-secret file beside state.db). It signs every session cookie, and rotating it signs every person out — the lever for a leaked backup or a departed operator.

1. If the env file sets ARKUSH_SESSION_SECRET, set a new value from openssl rand -hex 32 and recreate the container.
2. If it does not, stop the service, delete session-secret in the state directory, and start the service. The boot mints a new file and logs session_secret_minted.

A file in the wrong shape refuses boot rather than re-mint quietly, so a damaged file is restored or deleted on purpose. To move the key out of the state directory, set the variable to the file's content, or to a fresh value when the backups that carry the file are the reason for the move.

The two generated secrets in `state.db`. Each is one row in the records table, kind = 'secret', under its own key. With the service stopped, delete the row, and the next boot mints a new one (loadOrCreateSecret, src/api/db.ts). The service never rotates one on its own, and an unreadable row refuses boot rather than re-mint quietly.

Deleting oauth revokes no agent: tokens verify by their stored hash, never by the key. The revoke act is Revoke all on the Users page (Offboarding a person, above).

docker compose -p <project> stop arkush
docker run --rm -v "<project>_arkush-state:/data" alpine \
  sh -c "apk add --no-cache sqlite >/dev/null && sqlite3 /data/state.db \
  \"DELETE FROM records WHERE kind = 'secret' AND key = 'oauth'\""
docker compose -p <project> start arkush

MCP access

Agents reach /mcp with OAuth tokens the API mints itself. The token, registration, and discovery endpoints need no browser session — the API checks its own tokens.

The consent page at /oauth/authorize is authenticated like the app: an anonymous visit redirects through /auth/sign-in and returns. On a header-trust install, the front must challenge that one route. The skip list above passes the other OAuth routes through.

Server-side BigQuery

A server-side warehouse run needs three things: ARKUSH_BQ_PROJECT, a Google credential in the container, and a registered service account named on the datasource. Leave ARKUSH_BQ_PROJECT unset and every such run answers no-bigquery, which is the posture of a deployment that holds no warehouse credential. A datasource that names no account refuses whoever asks — there is no run under the service's own identity. The DuckDB plane — preview_transform and derived-view runs — needs no credential.

That credential is the service's OWN identity, and how it proves that identity is the host's business. On a platform that attaches an identity, nothing is configured — the library reads the metadata server. On a host with none, an override mounts a service-account key file read-only and names it in GOOGLE_APPLICATION_CREDENTIALS. Keep that file outside the state directory, so a copy of the instance carries no credential.

This identity needs no BigQuery access of its own. Every server-side warehouse run names a registered service account and runs as that account by impersonation, so the service's identity only mints tokens and reads policies (below). Grant it nothing on your datasets. The accounts it impersonates still have no key anywhere — that half of the rule does not bend.

CAUTION: naming ARKUSH_BQ_PROJECT without supplying a credential is the one incoherent setting. Such a run passes the no-bigquery refusal and fails at the token mint instead, so the operator reads an error where a refusal was meant.

ARKUSH_BQ_MAX_BYTES_BILLED ("500MB", "2GB", or a plain byte count) caps the bytes one server-side run may bill. Over the cap the run fails before the scan starts, so the ceiling costs nothing to hold. Unset, no ceiling applies.

Groups

Group shares and group entries in the two allowlists resolve through one of two backends. ARKUSH_GROUPS picks which:

• file (the default) — a membership file admins edit on the admin page. Group names are short. Nothing to configure: a fresh instance has a working, empty Groups section, and the first group an admin adds creates the file.
• google — a registry of Google group ADDRESSES (groups: [team@example.org]), not members. The app reads transitive membership through the Cloud Identity API under its own identity (the cloud-identity.groups.readonly scope) — no credential of its own. Grant the runtime service account the Groups Reader admin role in the Google Workspace Admin Console (admin.google.com → Account → Admin roles) BEFORE the deploy. Admins register and remove addresses on the admin page; membership is read-only in the app, because Google owns it.

Each backend keeps its file beside the state plane — groups.yml and google-groups.yml under ARKUSH_STATE_DIR. ARKUSH_GROUPS_FILE and ARKUSH_GOOGLE_GROUPS_FILE move one somewhere else, and a path that names no file refuses the boot: you stated a location, so nothing there is a typo rather than an empty registry.

Named service accounts

The registry of accounts the service may impersonate for BigQuery runs sits at service-accounts.yml under ARKUSH_STATE_DIR, and admins register accounts on the admin page — nothing to configure. ARKUSH_SERVICE_ACCOUNTS_FILE moves the file somewhere else. Each target account needs three distinct grants in GCP:

• The impersonation grant — roles/iam.serviceAccountTokenCreator for the service's own runtime identity on the target account. The admin page's Verify attempts the real mint and prints the exact command on failure.
• The policy-read grant — roles/iam.serviceAccountViewer for the service's own runtime identity on the target account, or any role that carries iam.serviceAccounts.getIamPolicy. The service reads the account's IAM policy to answer who may use it. The impersonation grant above does not carry this permission.
• The target's own BigQuery access — roles/bigquery.jobUser on the billing project (a query runs as the target and bills ARKUSH_BQ_PROJECT) plus dataViewer on its datasets. This is the point: a per-team account reaches only its team's datasets, and it is the only reach a server-side run has — the service's own identity holds none.

Who may run under an account is decided in Google Cloud, not in this application. Grant a person roles/iam.serviceAccountTokenCreator on the target account and they may use it here. The analysts allowlist does not reach a registered account. There is no in-app list of who may run.

A policy grants to a person, to a Google group, or to a whole domain. A group resolves only where ARKUSH_GROUPS=google and an admin has registered that group address. Under the membership file the service cannot evaluate a group in a policy, and the refusal says so.

WARNING: an account reaches every dataset it was granted, for every person the policy names. Grant the account read on the datasets its team needs, and grant the role to the people who may spend it.

CAUTION: the service keeps a policy for one hour after the last successful read. If Google's IAM API is unreachable for longer than one hour, every run naming a service account refuses and every schedule that needs one pauses. Derived views, file-bound datasources and addressed reads are unaffected — none of them asks Google who may spend an account.

The file the admin page writes carries an address and a label per account, and nothing about who may use it:

serviceAccounts:
  dashboards-reader@PROJECT.iam.gserviceaccount.com:
    label: Dashboards mart, read-only

The step at the update

An install that registered accounts before this release has the impersonation grant and not the policy-read grant. Add roles/iam.serviceAccountViewer for the service's runtime identity on every registered account, or runs naming those accounts refuse with a message that prints the command. The admin page lists every registered account.

A new binding takes about a minute to reach the service — measured 2026-09-14, where the first successful read came on the fourth attempt over 60 seconds. A refusal straight after the grant is that delay, not a failed grant. Read the policy back as the service's own identity to confirm the grant landed:

gcloud iam service-accounts get-iam-policy <registered-account> \
  --impersonate-service-account=<the service's runtime identity> \
  --project=<the account's project>

Reading it as yourself proves nothing: an operator can read policies already, and the grant under test is the service's.

Scheduled deliveries to Slack

Deliveries post a dashboard into a Slack channel on a cron. To enable them:

1. Create a Slack app in the workspace. Give its bot token the files:write and chat:write scopes. Add files:read so delivered tables can thread under a message that carries chart images — without it, tables arrive as a separate channel message. Install the app.
2. Invite the bot to each channel it will post to. The bot can post only where it is a member — membership is the exposure boundary.
3. Write the destination registry on the admin page, or as the file it edits — destinations.yml under ARKUSH_STATE_DIR, which ARKUSH_DESTINATIONS_FILE moves somewhere else:

destinations:
  team-metrics:
    kind: slack
    channel: C0123456789 # the channel id, not the name
    token: xoxb-…
    label: '#team-metrics on the org Slack'

1. Leave the scheduled-refresh sweep at its default cadence, or set ARKUSH_REFRESH_INTERVAL_MINUTES. The service sweeps every five minutes on its own. The same sweep sends due deliveries and due subscriptions. To start a sweep by hand, set ARKUSH_REFRESH_TOKEN and POST /refresh/tick with that value as the bearer.
2. To let people subscribe to a dashboard in their own Slack direct messages, add the users:read.email and im:write scopes to the bot, reinstall the app in the workspace, and mark one entry with dm: true. After the reinstall, confirm that the token in the registry file is the one Slack shows. A file with one Slack entry needs no flag.

destinations:
  team-metrics:
    kind: slack
    channel: C0123456789
    token: xoxb-…
    dm: true # this bot opens the direct messages subscriptions send

The token stays server-side. Admins edit this file in the app (the admin page's Destinations section) or on disk — external edits are picked up without a restart. Unset, the feature does not exist.

The onboarding ask

The instance asks each person a few of its own questions once, at their first signed-in arrival, and stores the answers for admins to read on the Users page. An admin reads them to set the person up — grant a datasource, add them to a group. Skipping is one click and the person is never asked again.

This is the one registry with no editor in the app. Write ask.yml under ARKUSH_STATE_DIR, which ARKUSH_ASK_FILE moves somewhere else:

intro: We ask once, so we can set you up with the right data.
questions:
  team:
    label: Which team are you on?
  datasets:
    label: Which data do you expect to work with?
    hint: A warehouse dataset, a spreadsheet, a file — whatever you have.

The questions key must be present: a typo such as question: would otherwise read as a deliberately dormant file. An empty questions map is dormant — nobody is asked, and answers already stored stay readable. No file at all means the same, and the registry still records every sign-in either way.

• At most 10 questions. Past that the parse error says it is a survey.
• An id is lowercase, with no space, @ or :. Ids are the identity and labels are display text, so correcting a typo in a label leaves every stored answer where it is. Changing an id orphans the answers under the old one, where their own owner can still see and clear them.
• intro is optional and at most 300 characters. Absent, the app supplies its own opening line.
• label is required and at most 200 characters. hint is optional, with the same limit.
• A person's answer is capped at 500 characters.
• Display order is file order.

Edits apply without a restart. A corrupt file refuses the boot loudly; a corrupt edit while the service runs keeps the last good question set and logs once. Adding a question later never re-prompts anyone who has already answered or skipped — it appears unanswered in the admin's list and in that person's own About you panel.

Answers are visible to the person and to admins, and to nobody else. They never reach the usage log. A person clears their own from About you, an admin deletes a whole record from the Users page, and deleting a record means that person is asked again on their next visit.