tes
d27fd02866
Unrestricted admin (same shape as admin/volcanic-agents), password delivered via Coolify env var NATS_EXTERNAL_APPS_PASSWORD. Doc updated in both places that enumerate users. Plan file included for the record.
146 lines
8.1 KiB
Markdown
146 lines
8.1 KiB
Markdown
# NATS Expert
|
|
|
|
This project deploys a NATS server behind Coolify, reachable **over WebSockets** only. NATS' native TCP port (4222) is not exposed to the outside — the WS endpoint is the sole entry point for both applications and the managing agent.
|
|
|
|
## Deployment target
|
|
|
|
- **Coolify project:** `Volcanic MFEs` (`p4fts5a0kv3tahzjtbtfequk`)
|
|
- **Server:** `localhost` (`d0c4owww4kwo0ow08wws4ss8`)
|
|
- **Build pack:** `dockercompose`
|
|
- **Compose file:** `/docker-compose.yml`
|
|
- **Public domain:** `https://nats.tes.gd` → routed by Traefik to `nats:8080` (WebSocket listener)
|
|
- **TLS:** terminated at Traefik. NATS itself runs with `no_tls: true` on the WS listener.
|
|
|
|
Coolify's non-secret app config (project/server UUIDs, name, domain) lives in `deploy.json` at the repo root and is **gitignored** — it is regenerated locally when needed by the `deploying-to-coolify-via-api` skill flow.
|
|
|
|
## Connection details (for agents managing this NATS)
|
|
|
|
There are three users, all admin (unrestricted publish/subscribe):
|
|
|
|
- `admin` — env vars `NATS_USER` / `NATS_PASSWORD` on the Coolify application.
|
|
- `volcanic-agents` — env var `NATS_VOLCANIC_AGENTS_TOKEN` on the Coolify application. Semantically a bearer token; on the wire it goes through NATS's `password` field (NATS has no per-user `token` field — that's only valid at the top of `authorization` as a single global token).
|
|
- `external-apps` — env var `NATS_EXTERNAL_APPS_PASSWORD` on the Coolify application. Intended for third-party / external application connections. Password-style credential (random string, framed as a password rather than a bearer token). Unrestricted for now; scope down if we ever need to isolate external callers from internal traffic.
|
|
|
|
### Local credentials cache
|
|
|
|
A **gitignored** `credentials.local.json` at the repo root mirrors the live values for quick local access. Shape:
|
|
|
|
```json
|
|
{
|
|
"url": "wss://nats.tes.gd",
|
|
"users": {
|
|
"admin": { "user": "…", "password": "…", "note": "…" },
|
|
"volcanic-agents": { "user": "volcanic-agents", "password": "…", "note": "…" }
|
|
}
|
|
}
|
|
```
|
|
|
|
Read from it with `jq`, e.g. `jq -r '.users["volcanic-agents"].password' credentials.local.json`. It is **not** the source of truth — Coolify env vars are. Regenerate it from Coolify with the snippet below if it's stale or missing.
|
|
|
|
The current live values are printed by:
|
|
|
|
```bash
|
|
: "${COOLIFY_URL:?}" "${COOLIFY_KEY:?}"
|
|
APP_UUID=$(jq -r '.app.uuid' deploy.json) # from local deploy.json
|
|
curl -sS -H "Authorization: Bearer $COOLIFY_KEY" \
|
|
"$COOLIFY_URL/api/v1/applications/$APP_UUID/envs" \
|
|
| jq -r '.[] | select(.is_preview==false) | "\(.key)=\(.value)"'
|
|
```
|
|
|
|
If `deploy.json` is not present, list applications under the Volcanic MFEs project and find the one named `nats`:
|
|
|
|
```bash
|
|
curl -sS -H "Authorization: Bearer $COOLIFY_KEY" \
|
|
"$COOLIFY_URL/api/v1/projects/p4fts5a0kv3tahzjtbtfequk" \
|
|
| jq '.applications[] | {uuid, name}'
|
|
```
|
|
|
|
### WebSocket URL
|
|
|
|
```
|
|
wss://nats.tes.gd
|
|
```
|
|
|
|
The NATS server accepts standard NATS-over-WebSocket framing (RFC 6455 with the `nats` subprotocol). No custom path; connect to the root URL.
|
|
|
|
### With `nats` CLI
|
|
|
|
```bash
|
|
nats \
|
|
--server=wss://nats.tes.gd \
|
|
--user="$NATS_USER" --password="$NATS_PASSWORD" \
|
|
server info
|
|
```
|
|
|
|
### With `nats.js` / `nats.ws` (Node/browser)
|
|
|
|
```js
|
|
import { connect } from 'nats.ws'; // browser
|
|
// import { connect } from 'nats'; // node with ws
|
|
|
|
const nc = await connect({
|
|
servers: 'wss://nats.tes.gd',
|
|
user: process.env.NATS_USER,
|
|
pass: process.env.NATS_PASSWORD,
|
|
});
|
|
```
|
|
|
|
### With `nats-py`
|
|
|
|
```python
|
|
import asyncio, os
|
|
from nats.aio.client import Client as NATS
|
|
|
|
async def main():
|
|
nc = NATS()
|
|
await nc.connect(
|
|
servers=['wss://nats.tes.gd'],
|
|
user=os.environ['NATS_USER'],
|
|
password=os.environ['NATS_PASSWORD'],
|
|
)
|
|
await nc.publish('hello', b'world')
|
|
await nc.drain()
|
|
|
|
asyncio.run(main())
|
|
```
|
|
|
|
## What's inside the server
|
|
|
|
- **Listeners** (NATS binds all three inside the container; only 8080 is declared in `expose:` so Traefik picks it as the routing target)
|
|
- `4222` — native NATS protocol, reachable only from other containers on the compose network. Not routed publicly.
|
|
- `8080` — WebSocket listener, `no_tls: true`. The only port in `expose:`. Traefik terminates TLS and forwards `ws://nats:8080` from `wss://nats.tes.gd`.
|
|
- `8222` — HTTP monitoring (`/healthz`, `/varz`, `/jsz`), used by the compose healthcheck; not routed publicly.
|
|
- **JetStream**: enabled, persisted to the named volume `nats-data` mounted at `/data`. Limits: 256MB memory / 4GB file. Bump `max_file_store` in `nats-server.conf` if you need more.
|
|
- **Auth**: three users in `authorization.users` — `admin` (password from `NATS_PASSWORD`), `volcanic-agents` (password from `NATS_VOLCANIC_AGENTS_TOKEN`, semantically a bearer token), and `external-apps` (password from `NATS_EXTERNAL_APPS_PASSWORD`, for third-party callers). All three are unrestricted (no `permissions` block → admin). Substitution happens at Docker Compose parse time (the config lives inline in `docker-compose.yml` under `configs.nats-conf.content`), so NATS itself sees a static config. No accounts, no operator/JWT mode.
|
|
|
|
## Managing / redeploying
|
|
|
|
Config or compose changes → commit and push to `main`, then trigger a Coolify deploy (fire-and-forget):
|
|
|
|
```bash
|
|
: "${COOLIFY_URL:?}" "${COOLIFY_KEY:?}"
|
|
APP_UUID=$(jq -r '.app.uuid' deploy.json) # or look it up as shown above
|
|
curl -sS -X POST "$COOLIFY_URL/api/v1/deploy?uuid=$APP_UUID" \
|
|
-H "Authorization: Bearer $COOLIFY_KEY"
|
|
# → { "deployments": [ { "deployment_uuid": "…" } ] } done, don't poll
|
|
```
|
|
|
|
Rotating the password: PATCH the `NATS_PASSWORD` env on the Coolify app, then redeploy. The value is substituted into `configs.nats-conf.content` at compose parse time.
|
|
|
|
To add a second user (e.g. a scoped app account), edit the `authorization.users` array in the `configs.nats-conf.content` block in `docker-compose.yml`, and set any new credential env vars in Coolify.
|
|
|
|
## Gotchas learned the hard way
|
|
|
|
- **No per-user `token` field in NATS config.** `authorization.users = [...]` entries only accept `user+password`, `nkey`, or JWT — a `token` key inside a users entry makes `nats-server` refuse to start with `unknown field "token"`. The `token` key is only valid at the **top level** of `authorization`, and it defines a single server-wide token (mutually exclusive with `users`). To give a specific user a bearer-token-style credential, put the random token in the `password` field. On the wire it's the standard NATS user/password auth mechanism; only the semantics (opaque random string vs. human-picked password) differ.
|
|
|
|
- **Coolify's deploy status ≠ container health.** A deploy returning `status: "finished"` from `/api/v1/deployments/…` just means the build + `docker compose up -d` step ran to completion. If the container then crash-loops (e.g. bad `nats-server.conf`), the deploy is still "finished" but the app's `/api/v1/applications/{uuid}` shows `status: "restarting:unknown"` and `/api/v1/applications/{uuid}/logs` returns `Application is not running.` — no runtime logs via API. Fall back to `docker logs <container-name>` on the Coolify host to see why the process is dying. Container name pattern: `nats-<app-uuid>-<random>`.
|
|
|
|
## Files in this repo
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `docker-compose.yml` | Single `nats` service, exposes 4222/8080/8222 internally; only 8080 is routed publicly via `SERVICE_FQDN_NATS_8080` + Coolify's `docker_compose_domains`. The NATS config lives inline under `configs.nats-conf.content` (delivered to the container as `/etc/nats/nats-server.conf`) — no separate config file, no bind mount (Coolify's compose executor rewrites relative host paths into a persistent app dir and can't materialise a source file for them). |
|
|
| `.gitignore` | Keeps `deploy.json` and `credentials.local.json` out of git. |
|
|
| `deploy.json` (local only) | Coolify app config used by the `deploying-to-coolify-via-api` skill. |
|
|
| `credentials.local.json` (local only) | Cached NATS connection details for both users (WS URL, admin user/password, volcanic-agents user/password-that-is-a-token). Mirrors the Coolify env vars; regenerate from the API if stale. |
|