Deployment

Put it on a server your customers can reach

Run cob-rag-engine on a rented server with a real domain, HTTPS, backups and automatic updates. You have already run it on your own machine with the step-by-step guide; this page is the production version. Every command and setting name was checked against the repository.

Where to get it
Get cob-rag-engine from its repository (git clone https://github.com/callobuzz/cob-rag-engine.git) or download a release zip, then cd cob-rag-engine. The steps below run from that folder. Questions: callobuzz@gmail.com.

Checked against the repository on 2 October 2026. Examples use the domain yourbusiness.in; replace it with yours. Tell the page which hardware, proxy and update method you will use and it shows only what applies. Prefer to read everything? Switch on “Show all options”.

Quick startThe quick way: the installer#

On a fresh Ubuntu 22.04, Ubuntu 24.04 or Debian 12 server, one command does sections 4 to 7 below for you: it checks the server, installs Docker if it is missing, asks three questions, writes the settings with generated passwords, starts everything behind Caddy with a free HTTPS certificate, and prints the link you sign in with. Check the requirements first (server size, a web address with its DNS record pointing at the server, ports 80 and 443 free). Windows and macOS are for trying it on your own computer; use the step-by-step guide.

  1. Get the code on the server. Clone the repository there, or copy a release zip to the server (for example with scp), unzip it and enter the folder.
  2. Run the installer. It needs administrator rights.
Terminal (on the server)
git clone https://github.com/callobuzz/cob-rag-engine.git
cd cob-rag-engine
sudo bash deploy/install.sh

It asks two questions: your web address (for example chat.sharmafurniture.in), your e-mail (you sign in with it), and nothing more. Everything else, including your AI provider key and e-mail sending, is set afterwards in the dashboard’s first-run steps (and section 5.3 for e-mail). To try it without changing anything first, add --dry-run: it lists every step and writes nothing.

·What it does, and where things live#

  • It refuses to start on a server that is too small, runs another program on port 80 or 443, or runs another version of the product, and says what to do. A web address that is an IP address or a .local or .test name is refused too.
  • Everything lives in /opt/cob-rag/: the settings file (.env, readable by the administrator only), the running version (current), older versions (releases/), backups (backups/) and what was installed (state/).
  • One web address serves everything: the dashboard at /, the engine under /api/ and the chat window script at /widget.js, and the crawler under /scraper/. A nightly backup runs at 02:30 and keeps the last 7 days (--no-backup-cron turns it off). If a firewall (ufw) is already on, it opens ports 80 and 443; it never turns one on.
  • Running it again is safe. It keeps your answers and secrets and makes sure everything is in place. If a different version is already installed it stops and points you to the upgrade command below.
  • Later: sudo bash /opt/cob-rag/current/deploy/upgrade.sh <new zip or folder> (section 10), .../backup.sh and .../restore.sh (section 9).
A backup on the same server is lost with the server
Copy the folders in /opt/cob-rag/backups/ to another computer or a storage bucket now and then.

Back to top

01The plan#

In plain words: you rent a small computer in a data centre, point a web address at it, and run the same Docker Compose stack you ran at home. A program called a reverse proxy sits in front, gets a free HTTPS certificate, and passes visitors to the engine. Nothing else is reachable from the internet.

  1. Rent a server big enough for the models (section 2) from a provider near your customers (section 3).
  2. Point a domain at it, close every port except 22, 80 and 443, install Docker, clone the repository (section 4).
  3. Run the setup wizard in production mode, with real SMTP and real addresses (section 5), then start the stack (section 6).
  4. Put HTTPS in front with Caddy, Nginx or Cloudflare (section 7).
  5. Back up, then automate updates (sections 9 to 11).

Three web addresses are used throughout. Use your own names:

AddressWhat it servesInside the server
chat.yourbusiness.inThe engine API and widget.js (what customers’ websites call)127.0.0.1:8000
app.yourbusiness.inThe dashboard (what you and your customers sign in to)127.0.0.1:8090
scrape.yourbusiness.inThe web scraper, only if you use the dashboard’s Scraper page127.0.0.1:8001
Tip
Do a dry run first on a throwaway server for a day. Everything here is repeatable: the whole setup is one .env file plus the Compose stack.

Back to top

Start hereYour setup#

Three choices change which commands you see. Your answers stay in this browser; nothing is sent anywhere.

JavaScript is off, so every variant is shown and each one is labelled with the choice it applies to.

Showing this page for

CPU server · Caddy · self-hosted runner

Server hardware

CPU is right for most chatbots. A GPU only makes the local models faster.

HTTPS in front

Caddy is the simplest: it gets and renews certificates by itself.

Automatic deploys

You can start by hand and add this later.

Back to top

02Server size#

The engine itself is light. The weight is the AI models it runs for you: one that turns text into numbers (embeddings) and, optionally, one that re-orders search results (the re-ranker). Your answer-writing model runs at your AI provider, not on your server.

2.1What to rent#

SizeCPU / RAM / diskGood for
Minimum, CPU2 vCPU, 4 GB RAM, 40 GB SSDA trial, or one or two small customers. Re-ranker off. Add 2 GB of swap (below).
Recommended, CPU4 vCPU, 8 GB RAM, 80 GB SSDA real deployment with several customers, a few documents each. Leaves room for a second API worker or a busy import.
GPUNVIDIA GPU, 8 vCPU, 16 GB RAM, 100 GB SSDLarge imports, the local re-ranker, audio transcription. See VRAM below.

Two of these figures are measured and two are advice. Measured, with the earlier embedding model BAAI/bge-base-en-v1.5: the default stack used about 2.1 GB of RAM on CPU with 270,000 chunks indexed (API 0.5 GB, worker 0.4 GB, embedding container 0.6 GB, Qdrant 0.3 GB, the rest under 0.2 GB each), and the local re-ranker adds about 2.3 GB. The default is now BAAI/bge-m3, which needs about 2.3 GB for its weights in the embedding container (an estimate; it will be measured in the final test). Advice, not measured: the vCPU and disk columns, and the 2 GB headroom on top. Building the images also needs memory, which is why 4 GB is a minimum and not a comfort.

Tip
On CPU, embedding runs at about 6 passages a second on 8 vCPUs, so importing thousands of pages takes hours. For a big first import use a hosted embedding model (RAG_EMBEDDING_PROVIDER=litellm) or a GPU, then go back to the small server.

2.2What uses memory#

  • API workers. RAG_API_WORKERS (default 1). Each worker is a separate process with its own copy of the models, so memory grows with it. Raise it only with the RAM to match.
  • The re-ranker. About 2.3 GB, and the model file (about the same) is cached in the hf_cache volume. On CPU a re-ranked query took about 15 seconds in our benchmark, so keep it off on a CPU server. It is off by default (RAG_RERANKING_ENABLED=false).
  • The embedding container. tei serves BAAI/bge-m3 (RAG_TEI_MODEL_ID), about 2.3 GB in memory on CPU (an estimate). On first start it downloads the model, which takes a few minutes.
  • Worker concurrency. RAG_WORKER_CONCURRENCY (default 2) is how many ingest jobs run at once.
  • Qdrant grows with the number of chunks; the numbers above are for 270,000.
  • Optional services (Whisper for audio, Langfuse for traces) add containers we have not measured. Leave them off until you need them.

VRAM (the GPU’s own memory): about 3 GB per API worker according to .env.example, plus the re-ranker at about 2.3 GB, plus the embedding model. docker-compose.gpu.yml hands every GPU service the whole card, so they share it. Run nvidia-smi while it is busy before you decide a card is big enough.

Disk fills with Docker images, the model caches (hf_cache, tei_models), the Postgres, Qdrant and files volumes, and your customers’ uploaded files. Rent a disk you can grow.

If you must run on 4 GB, add swap before you build the images:

Terminal (on the server)
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# keep it after a reboot:
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Back to top

03Where to rent one#

Any provider that gives you a Linux server with root access works. Pick a region near your customers’ visitors: a chat bubble feels slower when every message crosses an ocean. For customers in India, an Indian city is best. Prices change often, so each row links to the provider’s own pricing page instead of quoting one.

GPU options

Think before you rent a GPU
A GPU server running all month is much dearer than a CPU one, and hourly GPU clouds can remove your machine. For most chatbots a CPU server plus a hosted embedding API is cheaper and simpler. Use a GPU for heavy imports or the local re-ranker.
Choose Ubuntu
The commands on this page assume Ubuntu 24.04 LTS (or Debian). Pick that image when you create the server and add your SSH key there.

Back to top

04Prepare the server#

4.1Point your domain at it#

In your domain’s DNS settings, add an A record for each address, pointing at the server’s public IP (and an AAAA record if you use its IPv6 address). Use the exact IP your provider shows.

DNS records
chat    A    203.0.113.10
app     A    203.0.113.10
scrape  A    203.0.113.10   # only if you use the Scraper page

203.0.113.10 is a documentation address; use yours. Check it has spread before you go on:

Terminal (your own computer)
nslookup chat.yourbusiness.in
Cloudflare
Add the records in Cloudflare, with the orange cloud (Proxied) off for now. Switch it on in section 7 after HTTPS works, or Caddy may not get its certificate.

4.2Sign in and create a working user#

Sign in over SSH as the user your provider gave you (often root), update the system, and make a normal user called deploy so you do not run everything as root.

Terminal (on the server, as root)
apt update && apt upgrade -y
adduser deploy
usermod -aG sudo deploy
# let you sign in as deploy with the same SSH key:
rsync --archive --chown=deploy:deploy ~/.ssh /home/deploy

adduser asks you to choose a password for deploy; you need it for sudo. Then sign in again as deploy (ssh deploy@chat.yourbusiness.in) and use that user for everything below.

4.3Close every port except three#

Allow SSH (22), HTTP (80) and HTTPS (443), and nothing else. Port 80 is needed to get certificates.

Terminal (on the server)
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose
Why this is not enough on its own
Docker writes its own firewall rules, and they are checked before UFW. A published port such as 5433:5432 would be open to the world even with UFW on. That is why docker-compose.yml publishes every port on ${RAG_BIND_ADDR:-127.0.0.1} and why you must leave RAG_BIND_ADDR=127.0.0.1. Only the proxy on this machine can reach the engine, the database, Valkey and Qdrant.

If you use a provider-level firewall (DigitalOcean, Hetzner and Lightsail all have one), open the same three ports there too.

4.4Install Docker#

You need Docker Engine with the Compose v2 plugin (the docker compose command, two words). The careful way is Docker’s own install guide for Ubuntu. The quick way is Docker’s convenience script:

Terminal (on the server)
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker deploy
# sign out and in again so the group applies, then:
docker compose version
Warning
Anyone in the docker group can become root on the server. Keep that group to people and tools you trust. Section 11 comes back to this.

For a GPU, also install the NVIDIA driver and the NVIDIA Container Toolkit so Docker can see the card, then check it with the same command the guide uses:

Terminal (on the server)
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi

4.5Get the code#

Clone the repository on the server (or copy a release zip there with scp and unzip it) and move into the folder.

Terminal (on the server)
cd cob-rag-engine

If you plan automatic deploys (section 11), put the code in your own private repository and use that copy, because the deploy workflow lives in a repository you control. A private repository needs a read-only deploy key so the server can fetch it.

Back to top

05Configure for production#

The setup wizard writes one file, .env, with every secret generated. For a live server run it in full mode: only full mode asks “Is this a development machine or a live deployment?” and for your web addresses.

5.1Run the wizard#

Terminal (in cob-rag-engine)
python3 scripts/setup.py --full --gpu no
Terminal (in cob-rag-engine)
python3 scripts/setup.py --full --gpu yes

Use python3 on Ubuntu: make setup runs python, which a fresh Ubuntu does not have. The wizard needs Python 3.10 or newer and nothing installed; Ubuntu 24.04 ships 3.12. Answer like this:

  • Environment: Production.
  • Super-admin e-mail: the address you will sign in with.
  • AI model key: your provider and key (they are hidden as you type).
  • E-mail: your SMTP details (section 5.3).
  • Web addresses: the public engine address https://chat.yourbusiness.in, the same for “Engine address as the dashboard’s browser sees it”, your scraper address https://scrape.yourbusiness.in (or leave the default if you will not use it), and the dashboard https://app.yourbusiness.in.
  • Interface to publish ports on: keep 127.0.0.1.

The wizard finishes by printing the exact start command for your choices, and warns you if production is missing SMTP or an https:// dashboard address. Running it again later is safe: it backs the old file up first and keeps the values you already have.

5.2Check the values that matter#

Open .env and make sure these are set. The wizard handles most; RAG_TRUST_PROXY_HEADERS, RAG_TRUSTED_PROXY_HOPS and the scraper’s origins it does not, so add them by hand.

.env (edit with nano .env)
RAG_ENV=prod
RAG_BIND_ADDR=127.0.0.1
RAG_PUBLIC_URL=https://chat.yourbusiness.in
RAG_DASHBOARD_URL=https://app.yourbusiness.in
RAG_DASHBOARD_ENGINE_URL=https://chat.yourbusiness.in
RAG_DASHBOARD_SCRAPER_URL=https://scrape.yourbusiness.in
RAG_CORS_ORIGINS=["https://app.yourbusiness.in"]
SCRAPER_CORS_ORIGINS=["https://app.yourbusiness.in"]
RAG_TRUST_PROXY_HEADERS=true
RAG_TRUSTED_PROXY_HOPS=1
RAG_ENV=prod
Hides the API docs (/docs), refuses a wildcard CORS list, and refuses the log e-mail provider. The engine will not start with RAG_EMAIL_PROVIDER=log in production.
RAG_BIND_ADDR=127.0.0.1
Every published port listens on this machine only. Never set it to 0.0.0.0: that exposes Postgres, Valkey (a Redis-compatible store) and Qdrant, and UFW does not stop it.
RAG_PUBLIC_URL
Where browsers reach the engine. It goes into the widget embed snippet you give customers.
RAG_DASHBOARD_URL
The base of every link in e-mails (sign in, reset, invitation). Must be the dashboard’s public https:// address.
RAG_DASHBOARD_ENGINE_URL, RAG_DASHBOARD_SCRAPER_URL
Where the browser finds the engine and the scraper. Never a Compose service name. The dashboard only allows connections to these two origins.
RAG_CORS_ORIGINS
The browser origins allowed to call the engine directly. The dashboard’s origin must be in it, written exactly (scheme and host, no trailing slash, as a JSON list). Production refuses ["*"]. Customers’ websites do not go here: the widget has its own per-key origin lock.
SCRAPER_CORS_ORIGINS
The same, for the scraper. Include the dashboard origin.
RAG_TRUST_PROXY_HEADERS, RAG_TRUSTED_PROXY_HOPS
Whose address the engine believes. Read section 7.3 before you leave it on. Only true behind a proxy you control, and RAG_TRUSTED_PROXY_HOPS must be the number of proxies in front of the engine (1 for Caddy or Nginx alone, 2 for Cloudflare in front of Nginx). The old name RAG_WIDGET_TRUST_PROXY_HEADERS still works.
Tip
Uploads are capped at 50 MB by default (RAG_MAX_UPLOAD_BYTES), and a plan can set a lower limit. Nginx has its own, smaller default, covered in section 7.

5.3E-mail (SMTP) is required#

Sign-in codes, sign-up verification, invitations and password resets are sent as e-mail. Production refuses the development shortcut that writes them to a log, so without SMTP e-mail is switched off and sign-up and password reset do not work. Use a sending service, not a mailbox:

.env
RAG_SMTP_HOST=email-smtp.ap-south-1.amazonaws.com
RAG_SMTP_PORT=587
RAG_SMTP_SECURITY=starttls
RAG_SMTP_USERNAME=your-smtp-username
RAG_SMTP_PASSWORD=your-smtp-password
RAG_SMTP_FROM="Your Product <no-reply@yourbusiness.in>"

That example is Amazon SES in Mumbai (the host pattern is email-smtp.<region>.amazonaws.com, with SMTP credentials from the SES console). Port 587 uses starttls, 465 uses ssl. Whatever service you use, add its SPF and DKIM records to your DNS, or customers’ mail will land in spam. Gmail works for a trial with a 16-character app password, but it is not meant for production. You test it in section 8.

Back to top

06Start the stack#

Terminal (in cob-rag-engine)
docker compose up -d --build
Terminal (in cob-rag-engine)
docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build

The overlay reserves the GPU for the engine, the worker and the embedding container. The embedding container also needs the image tag that matches your card in RAG_TEI_IMAGE (for example ghcr.io/huggingface/text-embeddings-inference:89-1.9 for an Ada card, or cuda-1.9 to be portable); the wizard sets it for you. Check the embedding log says it is starting on CUDA.

The first start builds three images and downloads the embedding model, which takes several minutes. The app container runs the database migrations itself before it starts serving. Watch it settle:

Terminal
docker compose ps
docker compose logs -f --tail=100 app
# when it is up, from the server:
curl -fsS http://127.0.0.1:8000/api/v1/health/ready

The engine has two health addresses. /api/v1/health is the detailed one: it answers {"status":"ok","services":{...}}, or "status":"degraded" with the list saying which of postgres, redis, qdrant or storage failed, and it always answers HTTP 200. /api/v1/health/ready is the one for scripts and monitors: it answers HTTP 503 when Postgres, Valkey or Qdrant is down (it does not look at object storage at all, so an uptime monitor never costs a storage request), and each check gives up after 3 seconds, so a dead dependency cannot hang the poll. Because of that, curl -f fails exactly when the engine cannot serve. Docker’s own container check keeps using the first one, so a brief Valkey restart does not mark the engine itself unhealthy.

From outside the server nothing answers yet, and that is correct: every port is on 127.0.0.1. HTTPS comes next.

Back to top

07HTTPS and the proxy#

A reverse proxy listens on ports 80 and 443, holds the HTTPS certificates, and forwards each address to the right container port on 127.0.0.1. Pick the one you chose above. All three do the same job.

7.1Install Caddy#

Caddy gets and renews free certificates from Let’s Encrypt by itself. Install it from its official repository:

Terminal (on the server)
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install -y caddy

Caddy (recommended) needs one file. Replace the contents of /etc/caddy/Caddyfile:

/etc/caddy/Caddyfile
# The engine API and widget.js
chat.yourbusiness.in {
	reverse_proxy 127.0.0.1:8000 {
		# answers stream token by token; do not buffer them
		flush_interval -1
		# X-Forwarded-For becomes exactly one entry: the address that connected to Caddy.
		# With RAG_TRUSTED_PROXY_HOPS=1 the engine reads that one entry.
		header_up X-Forwarded-For {remote_host}
	}
}

# The dashboard
app.yourbusiness.in {
	reverse_proxy 127.0.0.1:8090
}

# The scraper (delete this block if you do not use it)
scrape.yourbusiness.in {
	reverse_proxy 127.0.0.1:8001
}
Terminal (on the server)
sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddy

Caddy has no upload limit of its own, so the engine’s own cap applies.

7.1Nginx#

Install Nginx and Certbot, then add one site file. Certbot adds the HTTPS part and renews the certificates.

Terminal (on the server)
sudo apt install -y nginx certbot python3-certbot-nginx
/etc/nginx/sites-available/cob-rag
# The engine API and widget.js
server {
	listen 80;
	server_name chat.yourbusiness.in;
	# Nginx's default is 1 MB, which breaks document uploads. Match RAG_MAX_UPLOAD_BYTES (50 MB).
	client_max_body_size 50m;

	location / {
		proxy_pass http://127.0.0.1:8000;
		proxy_http_version 1.1;
		proxy_set_header Host $host;
		# one entry: the address that connected to Nginx. With RAG_TRUSTED_PROXY_HOPS=1 the
		# engine reads the last entry. ($proxy_add_x_forwarded_for is also safe with hops=1,
		# because only the entry Nginx itself appended is read.)
		proxy_set_header X-Forwarded-For $remote_addr;
		proxy_set_header X-Forwarded-Proto $scheme;
		# answers stream token by token
		proxy_buffering off;
		proxy_read_timeout 300s;
	}
}

# The dashboard
server {
	listen 80;
	server_name app.yourbusiness.in;
	location / {
		proxy_pass http://127.0.0.1:8090;
		proxy_set_header Host $host;
		proxy_set_header X-Forwarded-Proto $scheme;
	}
}

# The scraper (delete this block if you do not use it)
server {
	listen 80;
	server_name scrape.yourbusiness.in;
	location / {
		proxy_pass http://127.0.0.1:8001;
		proxy_set_header Host $host;
		proxy_set_header X-Forwarded-Proto $scheme;
	}
}
Terminal (on the server)
sudo ln -s /etc/nginx/sites-available/cob-rag /etc/nginx/sites-enabled/cob-rag
sudo nginx -t
sudo systemctl reload nginx
sudo certbot --nginx -d chat.yourbusiness.in -d app.yourbusiness.in -d scrape.yourbusiness.in

7.1Cloudflare in front of Caddy#

Cloudflare’s proxy (the orange cloud) hides your server’s address and absorbs floods. Caddy still runs on the server and holds the certificate. Do it in this order:

  1. Install Caddy and use the Caddyfile below, with the orange cloud still off. Check https://chat.yourbusiness.in/api/v1/health/ready works.
  2. In Cloudflare, set SSL/TLS to Full (strict), then switch the orange cloud on for the three records.
  3. Lock the server so only Cloudflare can connect (below). Otherwise anyone who learns the server’s address can skip Cloudflare and forge headers.

Install Caddy with the commands in 7.1 above. The Caddyfile has one special line. Behind Cloudflare, the address that connects to Caddy is a Cloudflare server, not the visitor; the visitor’s real address is in the CF-Connecting-IP header. The engine itself never reads CF-Connecting-IP, only X-Forwarded-For, so Caddy copies the visitor’s address into it.

/etc/caddy/Caddyfile
chat.yourbusiness.in {
	reverse_proxy 127.0.0.1:8000 {
		flush_interval -1
		# Cloudflare puts the real visitor here. X-Forwarded-For becomes that single entry,
		# so RAG_TRUSTED_PROXY_HOPS stays 1 (Caddy is the only proxy the engine trusts).
		header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
	}
}

app.yourbusiness.in {
	reverse_proxy 127.0.0.1:8090
}

scrape.yourbusiness.in {
	reverse_proxy 127.0.0.1:8001
}
Terminal (on the server)
sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddy

Then allow ports 80 and 443 only from Cloudflare’s published address ranges instead of from everywhere:

Terminal (on the server)
sudo ufw delete allow 80/tcp
sudo ufw delete allow 443/tcp
for ip in $(curl -s https://www.cloudflare.com/ips-v4; echo; curl -s https://www.cloudflare.com/ips-v6); do
  sudo ufw allow from "$ip" to any port 80,443 proto tcp
done
sudo ufw status numbered
Keep the list current
Cloudflare’s ranges change now and then. Re-run the loop every few months, or on a schedule. Cloudflare also cuts a request that sends nothing for 100 seconds with a 524 error; chat answers stream tokens so they are fine.

7.3The visitor’s address: RAG_TRUST_PROXY_HEADERS and RAG_TRUSTED_PROXY_HOPS#

The engine limits how fast one visitor can use a customer’s chat widget, and it records the address of every sign-in and audited action. Behind a proxy, every request seems to come from the proxy, so the engine has to be told whose address to believe. Two settings decide it:

  • RAG_TRUST_PROXY_HEADERS=false (the default): the engine ignores X-Forwarded-For completely and uses the address that connected. Behind a proxy that is the proxy, so every visitor shares one limit. (The old name RAG_WIDGET_TRUST_PROXY_HEADERS still works.)
  • RAG_TRUST_PROXY_HEADERS=true: the engine reads X-Forwarded-For, a list of addresses in which every proxy appends the address it saw connect. The visitor’s own entries come first and can be forged, so the engine reads from the right: RAG_TRUSTED_PROXY_HOPS entries from the end (1 to 10, default 1). Entries before that are never looked at, so a visitor who sends a fake X-Forwarded-For gains nothing.

RAG_TRUSTED_PROXY_HOPS is the number of proxies you run in front of the engine, counting the one that talks to it. If the header is missing, shorter than that number, or the entry is not an IP address, the engine falls back to the address that connected, so a wrong setting fails safe rather than open.

In front of the engineWhat the engine receivesRAG_TRUSTED_PROXY_HOPS
Caddy or Nginx only (the configurations on this page)One entry: the visitor’s address1
Cloudflare, then Caddy (the Cloudflare configuration above)One entry: Caddy copies CF-Connecting-IP into it1
Cloudflare, then Nginx that appends ($proxy_add_x_forwarded_for)Visitor’s address, then Cloudflare’s2
Cloudflare, then Nginx that sets X-Forwarded-For $http_cf_connecting_ipOne entry: the visitor’s address1

Check it instead of guessing. Turn the settings on, sign in from your phone on mobile data (not your own Wi-Fi), then open the dashboard’s audit log as the operator: the sign-in row must show your phone’s public address. If it shows the proxy’s or a Cloudflare address, the hop count is too high or the proxy is not setting the header; if it shows an address you did not use, the count is too low. Raise or lower RAG_TRUSTED_PROXY_HOPS by one and recreate the containers.

Cloudflare and the real visitor
The engine does not read CF-Connecting-IP. It is trustworthy only because the firewall in 7.2 lets nothing but Cloudflare reach your proxy. If you skip that lock, anyone can send their own CF-Connecting-IP straight to the server.

The same two settings also decide the address stored in sign-in sessions, the sign-in and sign-up limits and the audit log, so they matter beyond the widget. Set RAG_TRUST_PROXY_HEADERS=true only when a proxy like these is in front. If you ever expose the engine’s port directly, set it back to false.

7.4Apply the settings and test#

Changing .env needs the containers recreated. Then check from your own computer, not the server:

Terminal (on the server)
docker compose up -d --build
Terminal (your own computer)
curl -fsS https://chat.yourbusiness.in/api/v1/health/ready
curl -sI https://app.yourbusiness.in | head -1
# the docs are hidden in production; this must be 404:
curl -s -o /dev/null -w "%{http_code}\n" https://chat.yourbusiness.in/docs

Open https://app.yourbusiness.in in a browser. It should show the sign-in page with a padlock. If the page loads but sign-in fails with a CORS error, the dashboard’s origin is missing from RAG_CORS_ORIGINS (see troubleshooting).

Back to top

OptionalYour own public website#

The engine can serve a website for your brand: a home page, pricing, features, about and contact. It is off until you publish it from Public website in the dashboard and switch it on. These are the server-side parts.

·Domain and DNS#

Use the same name you point at the server in step 4.1 (for example www.yourbusiness.in). Type it, starting with https://, as the website's address under Public website. If you leave it empty the engine uses RAG_PUBLIC_URL. Links, the sitemap and share pictures always use that address and never the one a visitor happened to type.

·The website at /, the dashboard at /app#

The installer's Caddy already sends the root and the website paths to the engine and keeps the dashboard at /app, so you change nothing. Old dashboard addresses keep working. If you use Nginx or another proxy, send /, /pricing, /features, /about, /contact, /hi, /hi-latn, /robots.txt, /sitemap.xml and /_landing to the engine, and fall back to the dashboard when the engine answers 404 with the header X-Cob-Landing: off.

·Turning it off#

Switch off Show the website to visitors. The website paths answer "off" at once and your address opens the dashboard, as it did before. Nothing is deleted.

08Production checklist#

  1. Secrets stay secret. Lock the file: chmod 600 .env. Never commit it, paste it into a chat, or e-mail it. The wizard generated the admin key, session secret, database, Valkey and Qdrant passwords for you.
  2. Back up RAG_ENCRYPTION_KEY now. Copy that one line from .env into your password manager. See the warning below.
  3. Confirm production mode. RAG_ENV=prod, and the /docs check in section 7.4 returned 404.
  4. Test e-mail. Send a test message with your admin key (paste it where it says; do not save it in a file):
    Terminal (your own computer)
    curl -X POST https://chat.yourbusiness.in/api/v1/admin/smtp/test \
      -H "X-API-Key: PASTE_YOUR_RAG_ADMIN_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"to":"you@yourbusiness.in"}'
    It answers {"sent":true,...} when the message was handed to your SMTP server, or the SMTP code and error class when it was not.
  5. Create the super admin. The app does it on its first start, using RAG_SUPERADMIN_EMAIL. With SMTP set up, a set-password link arrives by e-mail; it lasts 24 hours. To print it on the console instead, or to get a fresh one:
    Terminal (on the server)
    # was a link printed at first start?
    docker compose logs app | grep -A6 "ONE-TIME SET-PASSWORD LINK"
    # issue a new one and print it here
    docker compose exec app python -m callobuzz_rag.accounts.bootstrap --resend --print-link
  6. Sign in at https://app.yourbusiness.in and set your password.
  7. Onboard a first customer. Follow section 7 of the step-by-step guide: create a customer, a knowledge base, upload a document, ask a question, add the website widget. Test the widget from a real page on that customer’s domain.
  8. Set up backups (section 9) before the customer adds anything you cannot re-upload.
Lose this key, lose the keys
RAG_ENCRYPTION_KEY encrypts each knowledge base’s stored AI-provider keys. Without it those keys cannot be read and every customer would have to enter theirs again. A backup of the database does not help: the key is not in it. Keep a copy of .env, or at least this line, somewhere that is not this server. Keep RAG_SESSION_SECRET too, or everyone is signed out.

Back to top

09Backups#

With the installer: one command
If you installed with deploy/install.sh, a backup is sudo bash /opt/cob-rag/current/deploy/backup.sh. It takes the database, the uploaded files (unless you use your own S3 provider), the crawler’s data, the search index and the settings file, writes a checksum list, and keeps the last 7 days (--keep-days N changes that). A nightly one is already set up. To restore: sudo bash /opt/cob-rag/current/deploy/restore.sh /opt/cob-rag/backups/<the backup>. It checks the checksums, refuses a backup from a newer version (--force overrides), asks you to type restore, and puts the data back; add --with-env to restore the settings file too. If you did not use the installer, or want to know how it works, read on: this is doing it by hand.

A server can vanish: a failed disk, a deleted instance, a bad update. A backup you have never restored is a hope, not a backup, so this section ends with a restore drill. Four things hold your data:

WhatWhere it livesHow it is backed up
Customers, knowledge bases, document text, chunks, accounts, keysPostgres (pgdata volume)pg_dump
Search vectorsQdrant (qdrant volume)Snapshots through its API. They can also be rebuilt from Postgres, slowly.
The uploaded original filesThe files volume (a folder on the server), or your own S3 providerA tar copy of the folder. With an external S3 provider, use its own versioning or replication.
Configuration and secrets.envA copy, kept encrypted. Includes RAG_ENCRYPTION_KEY.

9.1The backup script#

This script runs on the server as deploy. It reads the keys it needs from your .env at run time, so nothing is copied into the script.

/home/deploy/backup.sh
#!/usr/bin/env bash
# Daily backup of cob-rag-engine: Postgres, Qdrant, uploaded files and .env
set -euo pipefail

APP=/home/deploy/cob-rag-engine
BACKUPS=/home/deploy/backups
OUT="$BACKUPS/$(date +%F)"
KEEP_DAYS=14

mkdir -p "$OUT"
cd "$APP"
env_value() { grep "^$1=" .env | cut -d= -f2-; }

# 1. Postgres (a compressed dump you can restore with pg_restore)
docker compose exec -T postgres pg_dump -U rag -d rag_kernel -Fc > "$OUT/postgres.dump"

# 2. Qdrant: one snapshot per collection, downloaded, then deleted from the server
QKEY=$(env_value RAG_QDRANT_API_KEY)
QURL=http://127.0.0.1:6333
for c in $(curl -fsS -H "api-key: $QKEY" "$QURL/collections" | grep -o '"name":"[^"]*"' | cut -d'"' -f4); do
  snap=$(curl -fsS -X POST -H "api-key: $QKEY" "$QURL/collections/$c/snapshots" | grep -o '"name":"[^"]*"' | head -1 | cut -d'"' -f4)
  curl -fsS -H "api-key: $QKEY" -o "$OUT/qdrant-$c.snapshot" "$QURL/collections/$c/snapshots/$snap"
  curl -fsS -X DELETE -H "api-key: $QKEY" "$QURL/collections/$c/snapshots/$snap" > /dev/null
done

# 3. Uploaded files: the folder inside the app container, as one tar file
docker compose exec -T app tar cf - -C /var/lib/cob-rag/files . > "$OUT/files.tar"

# 4. Configuration and secrets
cp .env "$OUT/env"
chmod 600 "$OUT/env"

# 5. Keep two weeks of dated backups on this machine
find "$BACKUPS" -maxdepth 1 -type d -name '20*' -mtime +"$KEEP_DAYS" -exec rm -r {} +

# 6. Copy it OFF this machine. Uncomment one line after you set it up.
# rclone sync "$BACKUPS" yourremote:cob-rag-backups
# rsync -a --delete "$BACKUPS/" backup-user@other-server:cob-rag-backups/
echo "backup done: $OUT"
Terminal (on the server)
nano /home/deploy/backup.sh     # paste the script
chmod +x /home/deploy/backup.sh
/home/deploy/backup.sh               # run it once and read the output
ls -lh /home/deploy/backups/*/
A backup on the same server is not a backup
If the server is deleted, the backups go with it. Step 6 matters: copy to another machine or a storage bucket at another provider. The folder holds .env, so encrypt it on the way out (for example with rclone’s crypt remote). Also turn on your provider’s own server snapshots as a second layer.

If you chose an external S3 provider (RAG_STORAGE_PROVIDER=s3) instead of files on the server, delete step 3 and rely on that provider’s versioning or replication.

9.2Run it every night#

Servers usually keep UTC time. Set India time first, so “2:30 in the morning” means 2:30 in the morning for you:

Terminal (on the server)
sudo timedatectl set-timezone Asia/Kolkata
crontab -e
crontab (as deploy)
30 2 * * * /home/deploy/backup.sh >> /home/deploy/backup.log 2>&1

Check /home/deploy/backup.log the next morning, and set an alert if a backup does not run: a free service such as Healthchecks.io works by having the script call a URL when it succeeds.

9.3The restore drill#

Do this once on a spare server before you need it, and write down how long it took. On the spare: do sections 4 to 6 (so you have Docker, the code and a running stack), then put the backed-up env file in place as .env before starting. A different RAG_ENCRYPTION_KEY would make stored AI keys unreadable.

  1. Stop everything that writes, keeping the datastores running:
    Terminal (in cob-rag-engine)
    docker compose stop app worker mailer celery-beat
  2. Restore Postgres.
    Terminal (in cob-rag-engine)
    docker compose cp /home/deploy/backups/2026-10-02/postgres.dump postgres:/tmp/postgres.dump
    docker compose exec -T postgres pg_restore -U rag -d rag_kernel --clean --if-exists --no-owner /tmp/postgres.dump
    Use your own backup date in the folder name.
  3. Restore the files. Unpack the tar file into the files volume:
    Terminal (in cob-rag-engine)
    docker compose run --rm --no-deps -T --entrypoint tar app xf - -C /var/lib/cob-rag/files < /home/deploy/backups/2026-10-02/files.tar
    (With an external S3 provider this step does not apply: restore from that provider’s own versions.)
  4. Restore the vectors. Either upload each snapshot, or rebuild from Postgres once the stack is running (step 5):
    Terminal (in cob-rag-engine)
    # upload one snapshot (repeat per collection)
    curl -fsS -X POST -H "api-key: YOUR_RAG_QDRANT_API_KEY" \
      -F "snapshot=@/home/deploy/backups/2026-10-02/qdrant-COLLECTION.snapshot" \
      "http://127.0.0.1:6333/collections/COLLECTION/snapshots/upload?priority=snapshot"
    Terminal (in cob-rag-engine)
    # or rebuild a knowledge base's vectors from Postgres (run without --apply first for a dry run)
    docker compose exec app python -m callobuzz_rag.retrieval.reindex --apply --rebuild --corpus KNOWLEDGE_BASE_ID
    A rebuild drops that knowledge base’s collection and writes it again, so its searches return nothing until it finishes. It cannot restore document metadata for documents ingested by engine versions before 0.2.
  5. Start and check. Bring everything up, then ask a real question in the dashboard’s playground and open a cited source:
    Terminal (in cob-rag-engine)
    docker compose up -d
    curl -fsS http://127.0.0.1:8000/api/v1/health/ready

The Qdrant upload step is Qdrant’s own snapshot feature; its snapshot documentation covers the details and version limits. Qdrant here is v1.13.2; restore into the same version.

Back to top

10Updates and rollback#

With the installer: one command
Get the new version (a release zip, or a fresh clone of the repository in a separate folder), then run sudo bash /opt/cob-rag/current/deploy/upgrade.sh cob-rag-engine-new.zip (a zip or an unzipped folder). It takes a backup, builds the new version while the old one keeps answering, updates the settings file with any new settings, updates the database, switches, and checks the engine is healthy. If anything fails after the backup, it prints the exact command that puts the old version and its data back: sudo bash /opt/cob-rag/releases/<old version>/deploy/upgrade.sh --rollback (it asks you to type rollback). Add --dry-run to see the steps first. The text below is doing it by hand.

Updating is: back up, fetch the new code, rebuild. The app container runs alembic upgrade head before it starts serving, so database migrations apply on their own. Read CHANGELOG.md first. If your copy ran without a licence key or on an older paid-licence tier, customer billing was off even with RAG_BILLING_ENABLED=true; after this upgrade it is on, so set it to false in your settings file before you upgrade if you do not charge your businesses; an update occasionally changes what Docker creates, and the changelog says so.

Terminal (on the server)
cd /home/deploy/cob-rag-engine
/home/deploy/backup.sh
git checkout main
git pull
docker compose up -d --build
# confirm the migrations ran (a failure prints "ERROR: Alembic migration failed!")
docker compose logs app | grep -E "Migrations completed|Alembic migration failed"
curl -fsS http://127.0.0.1:8000/api/v1/health/ready

With a GPU, use the same -f docker-compose.yml -f docker-compose.gpu.yml pair on the up command as in section 6.

COB, the built-in assistant, re-reads its own documentation after every update: the app queues one sync on start, the worker ingests only the pages that changed, and the COB docs card on Engine info shows when it is done. If it ever sticks, press Resync there.

10.1Roll back#

Releases are tagged. To go back to one, check out its tag and rebuild:

Terminal (on the server)
cd /home/deploy/cob-rag-engine
git fetch --tags
git tag --list
git checkout v0.2.0
docker compose up -d --build
Code goes back; the database does not
A rollback restores old code, but migrations only move forward. If the newer release changed the database layout, the old code may not run against it. In that case restore the backup you took just before updating (section 9.3). That is the reason the update steps start with backup.sh.

To free disk after updates, remove old images: docker image prune -f.

Back to top

11Automatic deploys with GitHub Actions#

Once updating by hand is routine, you can let GitHub do it: every push to main, or every version tag, runs the tests, then updates the server and checks it is healthy. This page shows the workflow file; it is not part of the repository, and you add it to your own private copy of the repository.

Do it by hand first
Run sections 5 to 10 by hand at least once. A pipeline automates a procedure you already trust.

You chose “By hand”, so you can skip to section 12. Switch the choice above, or turn on “Show all options”, to read this section.

11.1How it fits together#

  • The tests run on a GitHub-hosted machine, so they never touch your server.
  • The deploy job runs on your server (self-hosted runner) or reaches it over SSH. It uses the clone in /home/deploy/cob-rag-engine and the .env already there, so your secrets never go into GitHub on the runner path.
  • The job takes a backup, fetches the exact commit that triggered it, runs docker compose up -d --build, checks the migrations reached the latest version, and waits for a healthy answer. If any step fails, the job fails and you are told.
  • To roll back, run the workflow again from an older tag (Actions, Run workflow, choose the tag).

11.2Install the runner on the server#

A self-hosted runner is a small program on your server that asks GitHub “is there a job for me?” over an outgoing connection. Nothing needs to be opened in the firewall. GitHub’s own instructions for adding a runner are the source of truth; the steps are:

  1. In your repository on GitHub, open Settings, Actions, Runners, New self-hosted runner, choose Linux and the architecture of your server. GitHub shows the exact download and unpack commands for the current runner version. Run those on the server as deploy, in /home/deploy/actions-runner. (The configure step refuses to run as root.)
  2. Register it with the token that page shows (it expires within the hour). The cob-prod label is what the workflow asks for:
    Terminal (on the server, in actions-runner)
    ./config.sh --url https://github.com/YOURNAME/cob-rag-engine --token TOKEN_FROM_THE_GITHUB_PAGE --labels cob-prod --name prod-vps
  3. Run it as a service so it survives reboots:
    Terminal (on the server, in actions-runner)
    sudo ./svc.sh install deploy
    sudo ./svc.sh start
    sudo ./svc.sh status
  4. In GitHub, open Settings, Environments, New environment, name it production, add yourself under Required reviewers, and limit Deployment branches and tags to main and v*. Each deploy then waits for your click.
  5. Create .github/workflows/deploy.yml in your repository with the workflow file in the next step, and push it.

11.3The workflow, with a runner on the server#

This is a complete file. The GitHub Actions in it are written with version tags, like the repository’s own ci.yml; see the notes below about pinning them.

.github/workflows/deploy.yml
# Add this to YOUR repository. It is an example, not a file in cob-rag-engine.
name: Deploy to production

on:
  push:
    branches: [main]
    tags: ["v*"]
  workflow_dispatch:           # Run workflow button; also how you roll back to a tag

# One deploy at a time, and never cancel one that is half finished.
concurrency:
  group: production-deploy
  cancel-in-progress: false

permissions:
  contents: read

jobs:
  test:
    name: Tests
    runs-on: ubuntu-latest     # GitHub's machine, not your server
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4          # pin to a full commit SHA (see the notes)
      - uses: actions/setup-python@v5      # pin to a full commit SHA
        with:
          python-version: "3.12"
          cache: pip
          cache-dependency-path: pyproject.toml
      - name: Install
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"
      - name: Unit tests
        run: pytest -q

  deploy:
    name: Deploy
    needs: test
    runs-on: [self-hosted, linux, cob-prod]
    environment: production    # required reviewers apply here
    timeout-minutes: 45
    env:
      APP_DIR: /home/deploy/cob-rag-engine
      HEALTH_URL: http://127.0.0.1:8000/api/v1/health/ready
      DEPLOY_SHA: ${{ github.sha }}
    steps:
      - name: Back up first
        run: /home/deploy/backup.sh

      - name: Fetch the exact commit
        run: |
          cd "$APP_DIR"
          git fetch --tags origin
          git checkout --detach "$DEPLOY_SHA"

      - name: Build and start
        run: |
          cd "$APP_DIR"
          docker compose up -d --build
          # GPU server: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build

      - name: Check the migrations reached the latest version
        run: |
          cd "$APP_DIR"
          docker compose exec -T app alembic current | grep "(head)"

      - name: Wait for a healthy engine
        run: |
          for i in $(seq 1 60); do
            if curl -fsS "$HEALTH_URL" > /dev/null; then
              echo "healthy"
              exit 0
            fi
            echo "waiting ($i/60)"
            sleep 5
          done
          cd "$APP_DIR"
          docker compose ps
          docker compose logs --tail=100 app
          exit 1

      - name: Remove old images
        run: docker image prune -f

The health step calls /api/v1/health/ready, which answers 503 until Postgres, Valkey and Qdrant all respond, so curl -f keeps waiting and the job fails if the engine never becomes ready.

11.4The alternative: GitHub-hosted runner and SSH#

No runner on your server: a GitHub-hosted machine signs in over SSH and runs the same steps. The cost is that SSH (port 22) must be reachable from GitHub’s changing address ranges, so allow key logins only and turn off password logins (PasswordAuthentication no in /etc/ssh/sshd_config).

  1. On your own computer, make a key used for nothing else, with no passphrase:
    Terminal (your own computer)
    ssh-keygen -t ed25519 -f cob-deploy-key -C github-deploy -N ""
    ssh-keyscan -t ed25519 chat.yourbusiness.in
  2. Append the contents of cob-deploy-key.pub to /home/deploy/.ssh/authorized_keys on the server.
  3. In GitHub, Settings, Environments, New environment, named production, with required reviewers. Add four environment secrets: DEPLOY_HOST (chat.yourbusiness.in), DEPLOY_USER (deploy), DEPLOY_SSH_KEY (the private key file’s contents) and DEPLOY_KNOWN_HOSTS (the ssh-keyscan line; compare its fingerprint with the server’s first).
  4. Create .github/workflows/deploy.yml in your repository with this complete file:
    .github/workflows/deploy.yml
    # Add this to YOUR repository. It is an example, not a file in cob-rag-engine.
    name: Deploy to production over SSH
    
    on:
      push:
        branches: [main]
        tags: ["v*"]
      workflow_dispatch:
    
    concurrency:
      group: production-deploy
      cancel-in-progress: false
    
    permissions:
      contents: read
    
    jobs:
      test:
        name: Tests
        runs-on: ubuntu-latest
        timeout-minutes: 30
        steps:
          - uses: actions/checkout@v4          # pin to a full commit SHA (see the notes)
          - uses: actions/setup-python@v5      # pin to a full commit SHA
            with:
              python-version: "3.12"
              cache: pip
              cache-dependency-path: pyproject.toml
          - name: Install
            run: |
              python -m pip install --upgrade pip
              pip install -e ".[dev]"
          - name: Unit tests
            run: pytest -q
    
      deploy:
        name: Deploy over SSH
        needs: test
        runs-on: ubuntu-latest
        environment: production
        timeout-minutes: 45
        steps:
          - name: Prepare SSH
            env:
              SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
              KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
            run: |
              install -m 700 -d ~/.ssh
              printf '%s\n' "$SSH_KEY" > ~/.ssh/deploy_key
              chmod 600 ~/.ssh/deploy_key
              printf '%s\n' "$KNOWN_HOSTS" > ~/.ssh/known_hosts
    
          - name: Deploy
            env:
              DEPLOY_SHA: ${{ github.sha }}
              DEPLOY_TARGET: ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}
            run: |
              ssh -i ~/.ssh/deploy_key "$DEPLOY_TARGET" bash -s -- "$DEPLOY_SHA" <<'REMOTE'
              set -euo pipefail
              cd /home/deploy/cob-rag-engine
              /home/deploy/backup.sh
              git fetch --tags origin
              git checkout --detach "$1"
              docker compose up -d --build
              docker compose exec -T app alembic current | grep "(head)"
              for i in $(seq 1 60); do
                if curl -fsS http://127.0.0.1:8000/api/v1/health/ready > /dev/null; then
                  echo "healthy"
                  exit 0
                fi
                sleep 5
              done
              docker compose ps
              docker compose logs --tail=100 app
              exit 1
              REMOTE

11.5Security notes#

A self-hosted runner runs whatever the workflow says, as deploy
And deploy is in the docker group, which is as good as root on the server. Use a private repository. On a public repository, anyone can open a pull request, and a workflow that runs on pull requests would run their code on your server. This workflow only runs on pushes, tags and a manual click, never on pull_request; keep it that way.
  • Require approval for outside contributors to run workflows (Settings, Actions, General) if your deploy repository is ever public. Better: keep it private and copy each new release into it.
  • Protect the triggers. Only people you trust should be able to push to main or create v* tags. Turn on branch protection and tag rules; the production environment’s reviewers are the last gate.
  • Pin actions. A version tag such as actions/checkout@v4 can be moved by whoever owns it. Replace each with the full commit SHA of the release you reviewed, with the tag in a comment, and update on purpose. See GitHub’s security hardening guide.
  • Least privilege. The workflow sets permissions: contents: read. Do not add write permissions you do not use.
  • Secrets. On the runner path, the server’s .env never leaves the server. On the SSH path, keep the four secrets in the production environment, not repository-wide, so only an approved deploy can read them. Never print them with echo.
  • Pass values through env:, as the workflow does, instead of pasting ${{ ... }} into a run: script, so a branch or tag name cannot inject commands.
  • The runner machine is production. Do not run other repositories’ jobs on it.

Back to top

12Monitoring#

You need to hear about trouble before a customer tells you. Three habits cover most of it.

12.1Look at the server#

Terminal (on the server, in cob-rag-engine)
# which containers run, and are they healthy?
docker compose ps
# recent errors from the engine, the ingest worker and the mailer
docker compose logs --since 1h app worker mailer | grep -i -E "error|traceback"
# memory and CPU per container
docker stats --no-stream
# disk: the machine, then Docker
df -h /
docker system df
free -h

Logs are rotated for you: each container keeps at most three files of 10 MB.

12.2Have something watch from outside#

Point a free uptime monitor (UptimeRobot, Better Stack, Healthchecks.io and others) at https://chat.yourbusiness.in/api/v1/health/ready every minute and alert on anything but HTTP 200: that address answers 503 when Postgres, Valkey or Qdrant is down, so a plain status check is enough. Add a second monitor for https://app.yourbusiness.in, and an alert for disk above 80 percent.

12.3Read the health answer#

The answer lists each service as true or false: api, postgres, redis, qdrant and storage (the last checks that the files folder, or the S3 bucket, is usable). The one that is false is where to look: docker compose logs <that service>. /health/ready gives the same list without storage, changes the status code, and calls the overall status ready or not_ready.

Back to top

13Connect WhatsApp (Meta app)#

The WhatsApp channel is off until you give the engine your own Meta app. This section is only needed if you offer WhatsApp. Every engine operator registers their own app, so nothing here is shared with anyone else, and Meta’s review is on you, not on the software.

13.1Register the Meta app#

In Meta for Developers create an app of type Business and add the WhatsApp product. Note the App ID and App Secret. In Meta Business Settings complete Business Verification. To offer Embedded Signup to businesses, become a Tech Provider and pass App Review for whatsapp_business_management and whatsapp_business_messaging. Meta limits a new Tech Provider to about 10 numbers a week and raises it to 200 a week once approved. Until then you can still connect numbers by hand. Then create a Facebook Login for Business configuration of the WhatsApp Embedded Signup type (version 4), add your dashboard’s address to the allowed domains and note its Configuration ID. Check Meta’s current documentation for the exact screens, which change.

13.2Point the webhook at the engine#

In the app’s WhatsApp configuration set the callback URL to https://api.example.com/api/v1/public/whatsapp/webhook (your engine’s public address, over HTTPS) and the verify token to the value you choose below. Subscribe to messages, and to smb_message_echoes if you want businesses to use their number in the WhatsApp Business app as well. The engine checks Meta’s signature (X-Hub-Signature-256) on the raw body of every delivery with your app secret and answers 401 to anything else (the one-time verify request with a wrong token is the only 403).

13.3Settings in .env#

Put these in .env (all start with RAG_WHATSAPP_), then restart the app, worker and beat:

.env
RAG_WHATSAPP_ENABLED=true
RAG_WHATSAPP_APP_ID=your-app-id
RAG_WHATSAPP_APP_SECRET=your-app-secret
RAG_WHATSAPP_VERIFY_TOKEN=a-long-random-string
RAG_WHATSAPP_CONFIG_ID=your-embedded-signup-configuration-id
RAG_WHATSAPP_GRAPH_VERSION=v23.0

The engine refuses to start with RAG_WHATSAPP_ENABLED=true and a missing app id, secret or verify token. It only talks to graph.facebook.com over HTTPS at that version. Check /api/v1/info: features.whatsapp.enabled is true. Then tick WhatsApp included on the plans that should have it.

13.4Connect a number and check it#

A workspace owner connects with Connect with Meta on the WhatsApp page. To connect for them, open the customer, Knowledge bases, Connect WhatsApp for this business and paste the phone number ID, the WhatsApp Business Account ID and a permanent token. Then send a message to the number: the chat appears on the WhatsApp page within seconds. If it does not, Check the connection on that page, and make sure the worker is running (it answers) and the callback URL is reachable from the internet. Set your fee rates under Sign-up & branding; they start at zero. Meta’s own prices are on Meta’s current price list.

Back to top

14Troubleshooting#

Each item reads symptom, why it happens, fix. Start with docker compose ps and docker compose logs <service>; they answer most questions. Problems with the local install itself are in the guide’s troubleshooting.

14.1No HTTPS certificate#

Symptom
The browser says the connection is not private, or Caddy’s log shows a failed certificate request.
Why
The domain does not point at the server yet, port 80 or 443 is closed, or (with Cloudflare) the orange cloud was switched on before the certificate was issued.
Fix
Check DNS with nslookup chat.yourbusiness.in, the firewall with sudo ufw status, and the log with sudo journalctl -u caddy --no-pager -n 50 (Nginx: sudo certbot certificates). Wait a few minutes after a DNS change and retry.

14.2The dashboard loads but sign-in fails#

Symptom
The browser console says “blocked by CORS policy”, or the sign-in button spins and then shows a network error.
Why
The dashboard’s own address is not in RAG_CORS_ORIGINS (written exactly: scheme and host, no trailing slash), or RAG_DASHBOARD_ENGINE_URL points somewhere the browser cannot reach.
Fix
Correct both in .env (section 5.2), then docker compose up -d --build. For the Scraper page, SCRAPER_CORS_ORIGINS and RAG_DASHBOARD_SCRAPER_URL work the same way.

14.3502 Bad Gateway#

Symptom
The address answers with a 502 or 503 from the proxy.
Why
The proxy is fine but the container behind it is not up: still starting (the app has a 60-second start period), restarting, or the proxy points at the wrong port.
Fix
docker compose ps, then docker compose logs --tail=100 app. From the server, curl http://127.0.0.1:8000/api/v1/health/ready must work before the proxy can. Ports: engine 8000, dashboard 8090, scraper 8001.

14.4The engine will not start in production#

Symptom
The app restarts again and again; its log says RAG_EMAIL_PROVIDER=log is refused when RAG_ENV=prod.
Why
The development e-mail provider writes verification codes into the log, so production refuses it.
Fix
Set RAG_SMTP_HOST, RAG_SMTP_FROM and the other SMTP settings (section 5.3) and remove RAG_EMAIL_PROVIDER=log. A wildcard RAG_CORS_ORIGINS is not fatal: production ignores it and allows no browser origin, which breaks the dashboard until you list its origin.

14.5E-mails do not arrive#

Symptom
No sign-in code, invitation or reset message.
Why
Wrong SMTP login or port, a sender address the service has not verified, or the message went to spam (missing SPF and DKIM).
Fix
Run the SMTP test from section 8 and read its error, then docker compose logs mailer. Transient SMTP errors are retried automatically; permanent ones are logged with the SMTP code only. Verify the sender domain at your mail service and add its DNS records.

14.6Uploads fail with 413#

Symptom
A document upload fails at once with “413 Request Entity Too Large”.
Why
Nginx allows 1 MB by default. The engine itself allows 50 MB (RAG_MAX_UPLOAD_BYTES), and a plan may set less.
Fix
Set client_max_body_size 50m; in the chat.yourbusiness.in server block, then sudo nginx -t and sudo systemctl reload nginx.

14.7The widget says “too many requests” for everyone#

Symptom
One visitor’s messages use up the limit for all visitors, or one visitor can send unlimited messages.
Why
Behind a proxy with RAG_WIDGET_TRUST_PROXY_HEADERS=false every visitor looks like the proxy. The opposite fault is the setting on with a proxy that appends to X-Forwarded-For instead of replacing it.
Fix
Set it to true and use the proxy settings in section 7, which replace the header. Recreate the containers with docker compose up -d.

14.8Containers are killed or the server freezes#

Symptom
Containers restart by themselves, builds die, or SSH stops answering during a build or a big import.
Why
Out of memory. Each API worker holds its own models, the re-ranker adds about 2.3 GB, and building the images needs memory too.
Fix
Confirm with sudo dmesg | grep -i "out of memory". Add swap (section 2), keep RAG_API_WORKERS=1, keep the re-ranker off on CPU, lower RAG_WORKER_CONCURRENCY, or rent the next size up.

14.9An update fails during migration#

Symptom
docker compose logs app ends with ERROR: Alembic migration failed! and the app keeps restarting.
Why
A database migration could not apply, for instance because of a conflict in your data.
Fix
Read the lines above the error; they name the cause. If you cannot resolve it, go back to the old tag (section 10.1) and restore the backup you took before the update (section 9.3), then open an issue with the log.

14.10The disk is full#

Symptom
Writes fail, builds stop, Postgres or Qdrant errors mention space.
Why
Old images and build cache from updates, local backups, or real growth in documents and vectors.
Fix
docker system df shows where it went. docker image prune -f removes unused images. Lower KEEP_DAYS in the backup script, copy backups off the server, and grow the disk at your provider if data is the cause.

14.11The deploy job never starts#

Symptom
The deploy job waits for a runner, or waits for approval.
Why
The runner service is stopped or offline, its labels do not match [self-hosted, linux, cob-prod], or the production environment is waiting for a reviewer.
Fix
On the server, in actions-runner, run sudo ./svc.sh status and restart it with sudo ./svc.sh start. Check Settings, Actions, Runners shows it as idle. Then approve the pending deployment on the run’s page.

Back to top

15Run a public demo#

A demo is a copy of the product that anyone can try without signing up. It is for showing the product to clients, or for a live preview on your own website. You turn it on with one setting. This section is optional: skip it if you only run the product for your own businesses.

15.1What demo mode does#

  • It prints two sign-ins on the sign-in page: one for the platform owner and one for the owner of a made-up business, Nirmal Family Clinic in Pune. Visitors click Use and sign in.
  • The clinic is ready to use. Its assistant answers in English and Hindi from five short documents and six questions with answers.
  • It switches off everything that deletes, hides a secret, sends something out or costs money: deleting, API keys, e-mail settings, WhatsApp, top-ups, plans, webhooks and password or e-mail changes. A visitor who tries one sees one friendly sentence.
  • It sends nothing out. No e-mail, no WhatsApp message and no webhook leaves the server.
  • It puts a hard daily limit on the whole demo and a smaller one on each visitor, so one person cannot use it all. When a limit is used up, the visitor gets a polite message, not an error. Everyone can see how many demo answers are left today.
  • Every night it puts everything back exactly as it was, so a visitor who changes things leaves nothing behind.

15.2What it does not do#

A demo copy has every feature, like any other copy. The demo only refuses the changes listed above.

Visitors can see each other
Everything a visitor types, uploads or changes can be seen by the next visitor until the nightly reset. The password is public on purpose. Never put a real customer’s data into a demo.

15.3Turn it on#

Use a separate server for a demo, not the one your customers use. Add these lines to .env. The platform owner signs in with the address you already set as RAG_SUPERADMIN_EMAIL, so that line must be there too.

.env
RAG_DEMO_MODE=true
RAG_DEMO_OWNER_EMAIL=owner@nirmalclinic.example.com
RAG_DEMO_PASSWORD=choose-a-long-password-for-visitors
RAG_DEMO_RESET_AT_UTC=21:30
RAG_DEMO_ANSWERS_PER_DAY=300
RAG_DEMO_ANSWERS_PER_VISITOR_PER_DAY=15
RAG_DEMO_MODEL_CALLS_PER_DAY=1200

The password is used by both sign-ins and is shown to every visitor. The owner address must be different from RAG_SUPERADMIN_EMAIL. RAG_DEMO_RESET_AT_UTC is the time of the nightly reset in UTC; 21:30 UTC is 03:00 in India. The three daily limits are explained in “Keeping the cost small” below; each takes a number from 1 to 100000.

Then recreate the containers:

Terminal (on the server)
docker compose up -d

The engine refuses to start when something is missing and names it, for example RAG_DEMO_PASSWORD. Set the model key (RAG_LLM_API_KEY) too, or the assistant cannot answer. Two minutes after the first start the demo builds itself. Allow about ten minutes for the documents to be learned, then open the sign-in page: it shows the two sign-ins.

15.4Keeping the cost small#

Every limit starts again when the nightly reset runs (RAG_DEMO_RESET_AT_UTC), so a message that says “resets at 21:30 UTC (03:00 IST)” is true for the data and for the limits. Four limits work together:

  • One visitor: RAG_DEMO_ANSWERS_PER_VISITOR_PER_DAY (default 15) is how many answers one visitor gets a day. A visitor is a keyed hash of the address the engine sees, plus the signed-in user. No address is stored. Both demo sign-ins are shared, so the address does most of the work: people behind one office or school network share one quota, and one person who uses both sign-ins gets two. Behind a reverse proxy, set RAG_TRUST_PROXY_HEADERS and RAG_TRUSTED_PROXY_HOPS correctly, or every visitor looks like the proxy.
  • The whole demo: RAG_DEMO_ANSWERS_PER_DAY (default 300) is how many answers the demo gives in a day. Only answers that call the AI model count: small talk the rules answer, cached answers and “talk to a person” requests are free and not counted. It covers the website widget, the dashboard playground and preview, conversations and COB.
  • Model calls: RAG_DEMO_MODEL_CALLS_PER_DAY (default 1200) is a hard stop under the answer limits. One answer is about four calls (label, rewrite, answer, fact check), so the default fits 300 answers with room for the playground and setup help.
  • Money: the demo plan allows 300 questions a day for the clinic and 40 questions a day to COB, and the engine stops the demo’s own AI key at about two US dollars a day.

When a visitor has used their answers they read, as a normal assistant message with no model call: “You’ve used today’s demo questions (15). The demo resets at 21:30 UTC (03:00 IST). Come back tomorrow, or install your own copy to try it without limits.” When the whole demo is used up, everyone reads “Today’s demo answers are all used up” with the same reset time. The dashboard banner and the sign-in page show “X of Y demo answers left today”, and the numbers (never addresses) are in GET /api/v1/public/branding and GET /api/v1/admin/demo. If the counters cannot be read (Redis is down), the demo stops answering with a plain “cannot answer right now” message instead of spending without a limit.

15.5The nightly reset, and doing it by hand#

The reset runs by itself at RAG_DEMO_RESET_AT_UTC. It also runs after a start when no reset has finished in the last 25 hours, which is how a new demo server fills itself. To reset now, or to see how the last reset went, run these on the server:

Terminal (on the server)
docker compose exec app python -m callobuzz_rag.demo status
docker compose exec app python -m callobuzz_rag.demo reset --wait

status shows the last good reset, the next one, today’s answers and today’s model calls. reset --wait waits until the reset is done. Both finish with exit code 0 when all is well, 1 when something failed and 2 when demo mode is off. When a reset fails, the message names the step it stopped at; run it again, and if it fails at the same step, read docker compose logs app around that time.

15.6Showing the dashboard inside another site#

By default the dashboard refuses to be shown inside any other website. That is a safety feature, and it is right for every normal install. For a live preview on another website, allow that site’s address in .env:

.env
RAG_FRAME_ANCESTORS=self https://demo.example.com

Write self and none without quote marks. Every other item must start with https:// and be a site name, with an optional port, like https://demo.yourbusiness.in. https://*.yourbusiness.in allows every sub-site. Then recreate the dashboard:

Terminal (on the server)
docker compose up -d dashboard

A wrong value stops the dashboard from starting and the message names RAG_FRAME_ANCESTORS. If the preview shows a blank frame, check that the value is exactly as above and that you recreated the dashboard. Leave the setting empty on a normal install.

Back to top

17Take online payments#

Your customers can add money to their balance, pay their plan fee or buy an add-on themselves, with UPI, a card or net banking, through Razorpay. The money reaches their balance within seconds and you do not type a top-up. This section is optional: until you switch it on, customers keep using Request top-up.

Start in Test mode
Razorpay gives you test keys (they start with rzp_test_) as soon as you sign up. No real money moves in Test mode, and you can try everything below before your business verification (KYC) is finished.

17.1In Razorpay#

  1. Create an account at razorpay.com and, for real money, finish the KYC. Make sure your website shows the pages Razorpay checks (see section 16, Before you ask a payment gateway to check your site).
  2. Under Account & Settings → API Keys generate a key. Copy the Key Id and the Key Secret now: Razorpay shows the secret once.
  3. Under Payment Capture choose Automatic.

17.2In your dashboard#

  1. Sign in as the platform owner. Open Billing → Tax and fill in your company’s name, address and state, and your GST number if you have one. The GST number’s state must be your company’s state.
  2. Open Billing → Payments, choose Razorpay, paste the Key Id and Key Secret, choose a webhook secret (any long random text) and write it in the Webhook secret box. Choose Save payment settings, then Test connection. The keys are stored encrypted and the page never shows them again.
  3. The page shows your Webhook address. In Razorpay open Account & Settings → Webhooks → Add new webhook, paste the address, type the same webhook secret, tick the events listed on the page (payment captured, payment failed, order paid, refund processed, refund failed) and save.
  4. Sign in as a customer’s owner, open Plan & billing, choose Add money and pay a small amount with Razorpay’s test card or UPI details. The balance should grow within a few seconds.

If you prefer to keep the keys out of the dashboard, set RAG_PAYMENTS_PROVIDER=razorpay, RAG_RAZORPAY_KEY_ID, RAG_RAZORPAY_KEY_SECRET and RAG_RAZORPAY_WEBHOOK_SECRET in .env and recreate the services. A value in .env wins over the dashboard, and the page says so. The webhook address must be reachable from the internet over https, so use your real domain and not localhost.

17.3GST, refunds and fees#

  • GST. A customer in your state pays CGST and SGST (9% each); a customer in another state pays IGST (18%). The balance receives the amount before GST. Without a GST number the payment carries no tax. Check the rate and your obligations with your accountant: the engine records the split, and you file the returns.
  • Refunds. Open the payment under Billing → Payments and choose Refund…. The amount comes out of the customer’s unused balance and goes back to the way they paid, within 180 days. If Razorpay refuses, the balance is put back.
  • Fees. Razorpay’s fee is copied onto each payment after settlement and shown in the P&L as Gateway fees. Customers never see it.
  • If a message from Razorpay is lost the engine finds the payment itself within five minutes, and every night it checks Razorpay’s settlement report. Deliveries that need attention on the Payments tab lists anything stuck, with a Retry button.

Back to top

18GST invoices#

Every online payment (section 17) produces a GST tax invoice at that moment, and every refund a credit note. You do not type or number anything. This section tells you what to set once, and what to keep.

18.1Set up your seller details#

  1. Open Billing → Tax as the platform owner. Fill in your company’s legal name, address and state, and your GST number if you are registered. The number’s state must be the company’s state.
  2. SAC code. The default 998315 (hosting and IT infrastructure provisioning) suits a hosted chatbot service; a different code can be set per service add-on. Ask your accountant which code fits what you sell.
  3. Invoice prefix (for example CB) and credit note prefix (CN), up to four letters or digits. A number looks like CB/26-27/00001 and the series restarts every 1 April. Changing a prefix during the year starts a fresh counter, and numbers stay unique because the prefix is part of the number.
  4. Choose whether your listed prices include GST. When they do not (the usual case), the dashboard shows the total with GST next to every price.

18.2Which document you get#

  • With a GST number: a Tax Invoice. A customer in your state pays CGST 9% + SGST 9%; a customer in another state pays IGST 18% (the place of supply is the customer’s GST state, else the state they entered).
  • Without a GST number: a Bill of Supply with no tax lines.
  • A customer outside India is an export. Fill in your LUT number (and its end date) in Billing → Tax and the invoice carries the standard wording “Supply meant for export under Letter of Undertaking without payment of integrated tax” with no tax. Without a valid LUT, 18% IGST is charged. File your LUT for the financial year with your accountant before the first export.
  • Refunds produce a Credit Note with the same tax split, naming the original invoice. After 30 November following the invoice’s financial year none can be issued; the refund dialog warns you first, and the GST already paid stays paid.
  • Not invoiced by the engine: top-ups you record by hand (bank transfers). Record those in your accounting tool.

18.3Keep them for eight years#

An issued invoice cannot be changed or deleted: a database trigger refuses it, and a correction is a credit note. The records live in two places, and your backups must cover both for at least eight years: the tax_documents table in PostgreSQL, and the PDFs in object storage under billing/tax-documents/ (the storage cleanup command never touches that folder). The records stay even after a customer’s workspace is deleted. A nightly check (R7 in Billing → Accounts → Books check) tells you if a payment ever has no invoice.

18.4For your accountant#

Open Billing → Invoices, pick the month and choose Download CSV for your accountant: one row per document with the buyer, B2B or B2C, place of supply, SAC, taxable value, CGST, SGST, IGST, total, the original invoice (for a credit note) and the payment reference. Questions to ask your accountant once:

  1. SAC 998315 or 997331 for the hosted subscription, and 998314 for setup.
  2. Is an invoice at each wallet top-up (an advance) acceptable, or do you need a receipt voucher plus an invoice?
  3. Is “computer-generated invoice” (without a signature image) enough on electronic invoices?
  4. A credit note for a refund of unused balance that was invoiced months ago.
  5. A lost chargeback: credit note or not.
  6. Reverse charge on foreign vendors such as your AI and hosting providers, and the monthly routine.
  7. Filing the LUT for this financial year, even before the first export customer.
  8. The B2C invoice-wise threshold in GSTR-1, and whether the CSV columns suit their tool.
E-invoicing is not built
Above ₹5 crore of annual turnover you must report each invoice to the government’s portal (IRN, e-invoicing). This engine does not do that. Below that turnover you do not need it.

Back to top