Quick startThe quick way: the installer#
On a fresh Ubuntu 22.04, Ubuntu 24.04 or Debian 12 server, one command does sections 4 to 7 below for you: it checks the server, installs Docker if it is missing, asks three questions, writes the settings with generated passwords, starts everything behind Caddy with a free HTTPS certificate, and prints the link you sign in with. Check the requirements first (server size, a web address with its DNS record pointing at the server, ports 80 and 443 free). Windows and macOS are for trying it on your own computer; use the step-by-step guide.
- Get the code on the server. Clone the repository there, or copy a release zip to the server (for example with
scp), unzip it and enter the folder. - Run the installer. It needs administrator rights.
git clone https://github.com/callobuzz/cob-rag-engine.git
cd cob-rag-engine
sudo bash deploy/install.shIt asks two questions: your web address (for example chat.sharmafurniture.in), your e-mail (you sign in with it), and nothing more. Everything else, including your AI provider key and e-mail sending, is set afterwards in the dashboard’s first-run steps (and section 5.3 for e-mail). To try it without changing anything first, add --dry-run: it lists every step and writes nothing.
·What it does, and where things live#
- It refuses to start on a server that is too small, runs another program on port 80 or 443, or runs another version of the product, and says what to do. A web address that is an IP address or a
.localor.testname is refused too. - Everything lives in
/opt/cob-rag/: the settings file (.env, readable by the administrator only), the running version (current), older versions (releases/), backups (backups/) and what was installed (state/). - One web address serves everything: the dashboard at
/, the engine under/api/and the chat window script at/widget.js, and the crawler under/scraper/. A nightly backup runs at 02:30 and keeps the last 7 days (--no-backup-cronturns it off). If a firewall (ufw) is already on, it opens ports 80 and 443; it never turns one on. - Running it again is safe. It keeps your answers and secrets and makes sure everything is in place. If a different version is already installed it stops and points you to the upgrade command below.
- Later:
sudo bash /opt/cob-rag/current/deploy/upgrade.sh <new zip or folder>(section 10),.../backup.shand.../restore.sh(section 9).
/opt/cob-rag/backups/ to another computer or a storage bucket now and then.01The plan#
In plain words: you rent a small computer in a data centre, point a web address at it, and run the same Docker Compose stack you ran at home. A program called a reverse proxy sits in front, gets a free HTTPS certificate, and passes visitors to the engine. Nothing else is reachable from the internet.
- Rent a server big enough for the models (section 2) from a provider near your customers (section 3).
- Point a domain at it, close every port except 22, 80 and 443, install Docker, clone the repository (section 4).
- Run the setup wizard in production mode, with real SMTP and real addresses (section 5), then start the stack (section 6).
- Put HTTPS in front with Caddy, Nginx or Cloudflare (section 7).
- Back up, then automate updates (sections 9 to 11).
Three web addresses are used throughout. Use your own names:
| Address | What it serves | Inside the server |
|---|---|---|
chat.yourbusiness.in | The engine API and widget.js (what customers’ websites call) | 127.0.0.1:8000 |
app.yourbusiness.in | The dashboard (what you and your customers sign in to) | 127.0.0.1:8090 |
scrape.yourbusiness.in | The web scraper, only if you use the dashboard’s Scraper page | 127.0.0.1:8001 |
.env file plus the Compose stack.Start hereYour setup#
Three choices change which commands you see. Your answers stay in this browser; nothing is sent anywhere.
JavaScript is off, so every variant is shown and each one is labelled with the choice it applies to.
02Server size#
The engine itself is light. The weight is the AI models it runs for you: one that turns text into numbers (embeddings) and, optionally, one that re-orders search results (the re-ranker). Your answer-writing model runs at your AI provider, not on your server.
2.1What to rent#
| Size | CPU / RAM / disk | Good for |
|---|---|---|
| Minimum, CPU | 2 vCPU, 4 GB RAM, 40 GB SSD | A trial, or one or two small customers. Re-ranker off. Add 2 GB of swap (below). |
| Recommended, CPU | 4 vCPU, 8 GB RAM, 80 GB SSD | A real deployment with several customers, a few documents each. Leaves room for a second API worker or a busy import. |
| GPU | NVIDIA GPU, 8 vCPU, 16 GB RAM, 100 GB SSD | Large imports, the local re-ranker, audio transcription. See VRAM below. |
Two of these figures are measured and two are advice. Measured, with the earlier embedding model BAAI/bge-base-en-v1.5: the default stack used about 2.1 GB of RAM on CPU with 270,000 chunks indexed (API 0.5 GB, worker 0.4 GB, embedding container 0.6 GB, Qdrant 0.3 GB, the rest under 0.2 GB each), and the local re-ranker adds about 2.3 GB. The default is now BAAI/bge-m3, which needs about 2.3 GB for its weights in the embedding container (an estimate; it will be measured in the final test). Advice, not measured: the vCPU and disk columns, and the 2 GB headroom on top. Building the images also needs memory, which is why 4 GB is a minimum and not a comfort.
RAG_EMBEDDING_PROVIDER=litellm) or a GPU, then go back to the small server.2.2What uses memory#
- API workers.
RAG_API_WORKERS(default 1). Each worker is a separate process with its own copy of the models, so memory grows with it. Raise it only with the RAM to match. - The re-ranker. About 2.3 GB, and the model file (about the same) is cached in the
hf_cachevolume. On CPU a re-ranked query took about 15 seconds in our benchmark, so keep it off on a CPU server. It is off by default (RAG_RERANKING_ENABLED=false). - The embedding container.
teiservesBAAI/bge-m3(RAG_TEI_MODEL_ID), about 2.3 GB in memory on CPU (an estimate). On first start it downloads the model, which takes a few minutes. - Worker concurrency.
RAG_WORKER_CONCURRENCY(default 2) is how many ingest jobs run at once. - Qdrant grows with the number of chunks; the numbers above are for 270,000.
- Optional services (Whisper for audio, Langfuse for traces) add containers we have not measured. Leave them off until you need them.
VRAM (the GPU’s own memory): about 3 GB per API worker according to .env.example, plus the re-ranker at about 2.3 GB, plus the embedding model. docker-compose.gpu.yml hands every GPU service the whole card, so they share it. Run nvidia-smi while it is busy before you decide a card is big enough.
Disk fills with Docker images, the model caches (hf_cache, tei_models), the Postgres, Qdrant and files volumes, and your customers’ uploaded files. Rent a disk you can grow.
If you must run on 4 GB, add swap before you build the images:
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# keep it after a reboot:
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab03Where to rent one#
Any provider that gives you a Linux server with root access works. Pick a region near your customers’ visitors: a chat bubble feels slower when every message crosses an ocean. For customers in India, an Indian city is best. Prices change often, so each row links to the provider’s own pricing page instead of quoting one.
GPU options
04Prepare the server#
4.1Point your domain at it#
In your domain’s DNS settings, add an A record for each address, pointing at the server’s public IP (and an AAAA record if you use its IPv6 address). Use the exact IP your provider shows.
chat A 203.0.113.10
app A 203.0.113.10
scrape A 203.0.113.10 # only if you use the Scraper page203.0.113.10 is a documentation address; use yours. Check it has spread before you go on:
nslookup chat.yourbusiness.in4.2Sign in and create a working user#
Sign in over SSH as the user your provider gave you (often root), update the system, and make a normal user called deploy so you do not run everything as root.
apt update && apt upgrade -y
adduser deploy
usermod -aG sudo deploy
# let you sign in as deploy with the same SSH key:
rsync --archive --chown=deploy:deploy ~/.ssh /home/deployadduser asks you to choose a password for deploy; you need it for sudo. Then sign in again as deploy (ssh deploy@chat.yourbusiness.in) and use that user for everything below.
4.3Close every port except three#
Allow SSH (22), HTTP (80) and HTTPS (443), and nothing else. Port 80 is needed to get certificates.
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose5433:5432 would be open to the world even with UFW on. That is why docker-compose.yml publishes every port on ${RAG_BIND_ADDR:-127.0.0.1} and why you must leave RAG_BIND_ADDR=127.0.0.1. Only the proxy on this machine can reach the engine, the database, Valkey and Qdrant.If you use a provider-level firewall (DigitalOcean, Hetzner and Lightsail all have one), open the same three ports there too.
4.4Install Docker#
You need Docker Engine with the Compose v2 plugin (the docker compose command, two words). The careful way is Docker’s own install guide for Ubuntu. The quick way is Docker’s convenience script:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker deploy
# sign out and in again so the group applies, then:
docker compose versiondocker group can become root on the server. Keep that group to people and tools you trust. Section 11 comes back to this.For a GPU, also install the NVIDIA driver and the NVIDIA Container Toolkit so Docker can see the card, then check it with the same command the guide uses:
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi4.5Get the code#
Clone the repository on the server (or copy a release zip there with scp and unzip it) and move into the folder.
cd cob-rag-engineIf you plan automatic deploys (section 11), put the code in your own private repository and use that copy, because the deploy workflow lives in a repository you control. A private repository needs a read-only deploy key so the server can fetch it.
05Configure for production#
The setup wizard writes one file, .env, with every secret generated. For a live server run it in full mode: only full mode asks “Is this a development machine or a live deployment?” and for your web addresses.
5.1Run the wizard#
python3 scripts/setup.py --full --gpu nopython3 scripts/setup.py --full --gpu yesUse python3 on Ubuntu: make setup runs python, which a fresh Ubuntu does not have. The wizard needs Python 3.10 or newer and nothing installed; Ubuntu 24.04 ships 3.12. Answer like this:
- Environment: Production.
- Super-admin e-mail: the address you will sign in with.
- AI model key: your provider and key (they are hidden as you type).
- E-mail: your SMTP details (section 5.3).
- Web addresses: the public engine address
https://chat.yourbusiness.in, the same for “Engine address as the dashboard’s browser sees it”, your scraper addresshttps://scrape.yourbusiness.in(or leave the default if you will not use it), and the dashboardhttps://app.yourbusiness.in. - Interface to publish ports on: keep
127.0.0.1.
The wizard finishes by printing the exact start command for your choices, and warns you if production is missing SMTP or an https:// dashboard address. Running it again later is safe: it backs the old file up first and keeps the values you already have.
5.2Check the values that matter#
Open .env and make sure these are set. The wizard handles most; RAG_TRUST_PROXY_HEADERS, RAG_TRUSTED_PROXY_HOPS and the scraper’s origins it does not, so add them by hand.
RAG_ENV=prod
RAG_BIND_ADDR=127.0.0.1
RAG_PUBLIC_URL=https://chat.yourbusiness.in
RAG_DASHBOARD_URL=https://app.yourbusiness.in
RAG_DASHBOARD_ENGINE_URL=https://chat.yourbusiness.in
RAG_DASHBOARD_SCRAPER_URL=https://scrape.yourbusiness.in
RAG_CORS_ORIGINS=["https://app.yourbusiness.in"]
SCRAPER_CORS_ORIGINS=["https://app.yourbusiness.in"]
RAG_TRUST_PROXY_HEADERS=true
RAG_TRUSTED_PROXY_HOPS=1RAG_ENV=prod- Hides the API docs (
/docs), refuses a wildcard CORS list, and refuses theloge-mail provider. The engine will not start withRAG_EMAIL_PROVIDER=login production. RAG_BIND_ADDR=127.0.0.1- Every published port listens on this machine only. Never set it to
0.0.0.0: that exposes Postgres, Valkey (a Redis-compatible store) and Qdrant, and UFW does not stop it. RAG_PUBLIC_URL- Where browsers reach the engine. It goes into the widget embed snippet you give customers.
RAG_DASHBOARD_URL- The base of every link in e-mails (sign in, reset, invitation). Must be the dashboard’s public
https://address. RAG_DASHBOARD_ENGINE_URL,RAG_DASHBOARD_SCRAPER_URL- Where the browser finds the engine and the scraper. Never a Compose service name. The dashboard only allows connections to these two origins.
RAG_CORS_ORIGINS- The browser origins allowed to call the engine directly. The dashboard’s origin must be in it, written exactly (scheme and host, no trailing slash, as a JSON list). Production refuses
["*"]. Customers’ websites do not go here: the widget has its own per-key origin lock. SCRAPER_CORS_ORIGINS- The same, for the scraper. Include the dashboard origin.
RAG_TRUST_PROXY_HEADERS,RAG_TRUSTED_PROXY_HOPS- Whose address the engine believes. Read section 7.3 before you leave it on. Only
truebehind a proxy you control, andRAG_TRUSTED_PROXY_HOPSmust be the number of proxies in front of the engine (1 for Caddy or Nginx alone, 2 for Cloudflare in front of Nginx). The old nameRAG_WIDGET_TRUST_PROXY_HEADERSstill works.
RAG_MAX_UPLOAD_BYTES), and a plan can set a lower limit. Nginx has its own, smaller default, covered in section 7.5.3E-mail (SMTP) is required#
Sign-in codes, sign-up verification, invitations and password resets are sent as e-mail. Production refuses the development shortcut that writes them to a log, so without SMTP e-mail is switched off and sign-up and password reset do not work. Use a sending service, not a mailbox:
RAG_SMTP_HOST=email-smtp.ap-south-1.amazonaws.com
RAG_SMTP_PORT=587
RAG_SMTP_SECURITY=starttls
RAG_SMTP_USERNAME=your-smtp-username
RAG_SMTP_PASSWORD=your-smtp-password
RAG_SMTP_FROM="Your Product <no-reply@yourbusiness.in>"That example is Amazon SES in Mumbai (the host pattern is email-smtp.<region>.amazonaws.com, with SMTP credentials from the SES console). Port 587 uses starttls, 465 uses ssl. Whatever service you use, add its SPF and DKIM records to your DNS, or customers’ mail will land in spam. Gmail works for a trial with a 16-character app password, but it is not meant for production. You test it in section 8.
06Start the stack#
docker compose up -d --builddocker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --buildThe overlay reserves the GPU for the engine, the worker and the embedding container. The embedding container also needs the image tag that matches your card in RAG_TEI_IMAGE (for example ghcr.io/huggingface/text-embeddings-inference:89-1.9 for an Ada card, or cuda-1.9 to be portable); the wizard sets it for you. Check the embedding log says it is starting on CUDA.
The first start builds three images and downloads the embedding model, which takes several minutes. The app container runs the database migrations itself before it starts serving. Watch it settle:
docker compose ps
docker compose logs -f --tail=100 app
# when it is up, from the server:
curl -fsS http://127.0.0.1:8000/api/v1/health/readyThe engine has two health addresses. /api/v1/health is the detailed one: it answers {"status":"ok","services":{...}}, or "status":"degraded" with the list saying which of postgres, redis, qdrant or storage failed, and it always answers HTTP 200. /api/v1/health/ready is the one for scripts and monitors: it answers HTTP 503 when Postgres, Valkey or Qdrant is down (it does not look at object storage at all, so an uptime monitor never costs a storage request), and each check gives up after 3 seconds, so a dead dependency cannot hang the poll. Because of that, curl -f fails exactly when the engine cannot serve. Docker’s own container check keeps using the first one, so a brief Valkey restart does not mark the engine itself unhealthy.
From outside the server nothing answers yet, and that is correct: every port is on 127.0.0.1. HTTPS comes next.
07HTTPS and the proxy#
A reverse proxy listens on ports 80 and 443, holds the HTTPS certificates, and forwards each address to the right container port on 127.0.0.1. Pick the one you chose above. All three do the same job.
7.1Install Caddy#
Caddy gets and renews free certificates from Let’s Encrypt by itself. Install it from its official repository:
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install -y caddyCaddy (recommended) needs one file. Replace the contents of /etc/caddy/Caddyfile:
# The engine API and widget.js
chat.yourbusiness.in {
reverse_proxy 127.0.0.1:8000 {
# answers stream token by token; do not buffer them
flush_interval -1
# X-Forwarded-For becomes exactly one entry: the address that connected to Caddy.
# With RAG_TRUSTED_PROXY_HOPS=1 the engine reads that one entry.
header_up X-Forwarded-For {remote_host}
}
}
# The dashboard
app.yourbusiness.in {
reverse_proxy 127.0.0.1:8090
}
# The scraper (delete this block if you do not use it)
scrape.yourbusiness.in {
reverse_proxy 127.0.0.1:8001
}sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddyCaddy has no upload limit of its own, so the engine’s own cap applies.
7.1Nginx#
Install Nginx and Certbot, then add one site file. Certbot adds the HTTPS part and renews the certificates.
sudo apt install -y nginx certbot python3-certbot-nginx# The engine API and widget.js
server {
listen 80;
server_name chat.yourbusiness.in;
# Nginx's default is 1 MB, which breaks document uploads. Match RAG_MAX_UPLOAD_BYTES (50 MB).
client_max_body_size 50m;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
# one entry: the address that connected to Nginx. With RAG_TRUSTED_PROXY_HOPS=1 the
# engine reads the last entry. ($proxy_add_x_forwarded_for is also safe with hops=1,
# because only the entry Nginx itself appended is read.)
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
# answers stream token by token
proxy_buffering off;
proxy_read_timeout 300s;
}
}
# The dashboard
server {
listen 80;
server_name app.yourbusiness.in;
location / {
proxy_pass http://127.0.0.1:8090;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# The scraper (delete this block if you do not use it)
server {
listen 80;
server_name scrape.yourbusiness.in;
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
}sudo ln -s /etc/nginx/sites-available/cob-rag /etc/nginx/sites-enabled/cob-rag
sudo nginx -t
sudo systemctl reload nginx
sudo certbot --nginx -d chat.yourbusiness.in -d app.yourbusiness.in -d scrape.yourbusiness.in7.1Cloudflare in front of Caddy#
Cloudflare’s proxy (the orange cloud) hides your server’s address and absorbs floods. Caddy still runs on the server and holds the certificate. Do it in this order:
- Install Caddy and use the Caddyfile below, with the orange cloud still off. Check
https://chat.yourbusiness.in/api/v1/health/readyworks. - In Cloudflare, set SSL/TLS to Full (strict), then switch the orange cloud on for the three records.
- Lock the server so only Cloudflare can connect (below). Otherwise anyone who learns the server’s address can skip Cloudflare and forge headers.
Install Caddy with the commands in 7.1 above. The Caddyfile has one special line. Behind Cloudflare, the address that connects to Caddy is a Cloudflare server, not the visitor; the visitor’s real address is in the CF-Connecting-IP header. The engine itself never reads CF-Connecting-IP, only X-Forwarded-For, so Caddy copies the visitor’s address into it.
chat.yourbusiness.in {
reverse_proxy 127.0.0.1:8000 {
flush_interval -1
# Cloudflare puts the real visitor here. X-Forwarded-For becomes that single entry,
# so RAG_TRUSTED_PROXY_HOPS stays 1 (Caddy is the only proxy the engine trusts).
header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
}
}
app.yourbusiness.in {
reverse_proxy 127.0.0.1:8090
}
scrape.yourbusiness.in {
reverse_proxy 127.0.0.1:8001
}sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddyThen allow ports 80 and 443 only from Cloudflare’s published address ranges instead of from everywhere:
sudo ufw delete allow 80/tcp
sudo ufw delete allow 443/tcp
for ip in $(curl -s https://www.cloudflare.com/ips-v4; echo; curl -s https://www.cloudflare.com/ips-v6); do
sudo ufw allow from "$ip" to any port 80,443 proto tcp
done
sudo ufw status numbered7.3The visitor’s address: RAG_TRUST_PROXY_HEADERS and RAG_TRUSTED_PROXY_HOPS#
The engine limits how fast one visitor can use a customer’s chat widget, and it records the address of every sign-in and audited action. Behind a proxy, every request seems to come from the proxy, so the engine has to be told whose address to believe. Two settings decide it:
RAG_TRUST_PROXY_HEADERS=false(the default): the engine ignoresX-Forwarded-Forcompletely and uses the address that connected. Behind a proxy that is the proxy, so every visitor shares one limit. (The old nameRAG_WIDGET_TRUST_PROXY_HEADERSstill works.)RAG_TRUST_PROXY_HEADERS=true: the engine readsX-Forwarded-For, a list of addresses in which every proxy appends the address it saw connect. The visitor’s own entries come first and can be forged, so the engine reads from the right:RAG_TRUSTED_PROXY_HOPSentries from the end (1 to 10, default 1). Entries before that are never looked at, so a visitor who sends a fakeX-Forwarded-Forgains nothing.
RAG_TRUSTED_PROXY_HOPS is the number of proxies you run in front of the engine, counting the one that talks to it. If the header is missing, shorter than that number, or the entry is not an IP address, the engine falls back to the address that connected, so a wrong setting fails safe rather than open.
| In front of the engine | What the engine receives | RAG_TRUSTED_PROXY_HOPS |
|---|---|---|
| Caddy or Nginx only (the configurations on this page) | One entry: the visitor’s address | 1 |
| Cloudflare, then Caddy (the Cloudflare configuration above) | One entry: Caddy copies CF-Connecting-IP into it | 1 |
Cloudflare, then Nginx that appends ($proxy_add_x_forwarded_for) | Visitor’s address, then Cloudflare’s | 2 |
Cloudflare, then Nginx that sets X-Forwarded-For $http_cf_connecting_ip | One entry: the visitor’s address | 1 |
Check it instead of guessing. Turn the settings on, sign in from your phone on mobile data (not your own Wi-Fi), then open the dashboard’s audit log as the operator: the sign-in row must show your phone’s public address. If it shows the proxy’s or a Cloudflare address, the hop count is too high or the proxy is not setting the header; if it shows an address you did not use, the count is too low. Raise or lower RAG_TRUSTED_PROXY_HOPS by one and recreate the containers.
CF-Connecting-IP. It is trustworthy only because the firewall in 7.2 lets nothing but Cloudflare reach your proxy. If you skip that lock, anyone can send their own CF-Connecting-IP straight to the server.The same two settings also decide the address stored in sign-in sessions, the sign-in and sign-up limits and the audit log, so they matter beyond the widget. Set RAG_TRUST_PROXY_HEADERS=true only when a proxy like these is in front. If you ever expose the engine’s port directly, set it back to false.
7.4Apply the settings and test#
Changing .env needs the containers recreated. Then check from your own computer, not the server:
docker compose up -d --buildcurl -fsS https://chat.yourbusiness.in/api/v1/health/ready
curl -sI https://app.yourbusiness.in | head -1
# the docs are hidden in production; this must be 404:
curl -s -o /dev/null -w "%{http_code}\n" https://chat.yourbusiness.in/docsOpen https://app.yourbusiness.in in a browser. It should show the sign-in page with a padlock. If the page loads but sign-in fails with a CORS error, the dashboard’s origin is missing from RAG_CORS_ORIGINS (see troubleshooting).
OptionalYour own public website#
The engine can serve a website for your brand: a home page, pricing, features, about and contact. It is off until you publish it from Public website in the dashboard and switch it on. These are the server-side parts.
·Domain and DNS#
Use the same name you point at the server in step 4.1 (for example www.yourbusiness.in). Type it, starting with https://, as the website's address under Public website. If you leave it empty the engine uses RAG_PUBLIC_URL. Links, the sitemap and share pictures always use that address and never the one a visitor happened to type.
·The website at /, the dashboard at /app#
The installer's Caddy already sends the root and the website paths to the engine and keeps the dashboard at /app, so you change nothing. Old dashboard addresses keep working. If you use Nginx or another proxy, send /, /pricing, /features, /about, /contact, /hi, /hi-latn, /robots.txt, /sitemap.xml and /_landing to the engine, and fall back to the dashboard when the engine answers 404 with the header X-Cob-Landing: off.
·Turning it off#
Switch off Show the website to visitors. The website paths answer "off" at once and your address opens the dashboard, as it did before. Nothing is deleted.
08Production checklist#
- Secrets stay secret. Lock the file:
chmod 600 .env. Never commit it, paste it into a chat, or e-mail it. The wizard generated the admin key, session secret, database, Valkey and Qdrant passwords for you. - Back up
RAG_ENCRYPTION_KEYnow. Copy that one line from.envinto your password manager. See the warning below. - Confirm production mode.
RAG_ENV=prod, and the/docscheck in section 7.4 returned 404. - Test e-mail. Send a test message with your admin key (paste it where it says; do not save it in a file):
It answers
curl -X POST https://chat.yourbusiness.in/api/v1/admin/smtp/test \ -H "X-API-Key: PASTE_YOUR_RAG_ADMIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"to":"you@yourbusiness.in"}'{"sent":true,...}when the message was handed to your SMTP server, or the SMTP code and error class when it was not. - Create the super admin. The app does it on its first start, using
RAG_SUPERADMIN_EMAIL. With SMTP set up, a set-password link arrives by e-mail; it lasts 24 hours. To print it on the console instead, or to get a fresh one:# was a link printed at first start? docker compose logs app | grep -A6 "ONE-TIME SET-PASSWORD LINK" # issue a new one and print it here docker compose exec app python -m callobuzz_rag.accounts.bootstrap --resend --print-link - Sign in at
https://app.yourbusiness.inand set your password. - Onboard a first customer. Follow section 7 of the step-by-step guide: create a customer, a knowledge base, upload a document, ask a question, add the website widget. Test the widget from a real page on that customer’s domain.
- Set up backups (section 9) before the customer adds anything you cannot re-upload.
RAG_ENCRYPTION_KEY encrypts each knowledge base’s stored AI-provider keys. Without it those keys cannot be read and every customer would have to enter theirs again. A backup of the database does not help: the key is not in it. Keep a copy of .env, or at least this line, somewhere that is not this server. Keep RAG_SESSION_SECRET too, or everyone is signed out.09Backups#
deploy/install.sh, a backup is sudo bash /opt/cob-rag/current/deploy/backup.sh. It takes the database, the uploaded files (unless you use your own S3 provider), the crawler’s data, the search index and the settings file, writes a checksum list, and keeps the last 7 days (--keep-days N changes that). A nightly one is already set up. To restore: sudo bash /opt/cob-rag/current/deploy/restore.sh /opt/cob-rag/backups/<the backup>. It checks the checksums, refuses a backup from a newer version (--force overrides), asks you to type restore, and puts the data back; add --with-env to restore the settings file too. If you did not use the installer, or want to know how it works, read on: this is doing it by hand.A server can vanish: a failed disk, a deleted instance, a bad update. A backup you have never restored is a hope, not a backup, so this section ends with a restore drill. Four things hold your data:
| What | Where it lives | How it is backed up |
|---|---|---|
| Customers, knowledge bases, document text, chunks, accounts, keys | Postgres (pgdata volume) | pg_dump |
| Search vectors | Qdrant (qdrant volume) | Snapshots through its API. They can also be rebuilt from Postgres, slowly. |
| The uploaded original files | The files volume (a folder on the server), or your own S3 provider | A tar copy of the folder. With an external S3 provider, use its own versioning or replication. |
| Configuration and secrets | .env | A copy, kept encrypted. Includes RAG_ENCRYPTION_KEY. |
9.1The backup script#
This script runs on the server as deploy. It reads the keys it needs from your .env at run time, so nothing is copied into the script.
#!/usr/bin/env bash
# Daily backup of cob-rag-engine: Postgres, Qdrant, uploaded files and .env
set -euo pipefail
APP=/home/deploy/cob-rag-engine
BACKUPS=/home/deploy/backups
OUT="$BACKUPS/$(date +%F)"
KEEP_DAYS=14
mkdir -p "$OUT"
cd "$APP"
env_value() { grep "^$1=" .env | cut -d= -f2-; }
# 1. Postgres (a compressed dump you can restore with pg_restore)
docker compose exec -T postgres pg_dump -U rag -d rag_kernel -Fc > "$OUT/postgres.dump"
# 2. Qdrant: one snapshot per collection, downloaded, then deleted from the server
QKEY=$(env_value RAG_QDRANT_API_KEY)
QURL=http://127.0.0.1:6333
for c in $(curl -fsS -H "api-key: $QKEY" "$QURL/collections" | grep -o '"name":"[^"]*"' | cut -d'"' -f4); do
snap=$(curl -fsS -X POST -H "api-key: $QKEY" "$QURL/collections/$c/snapshots" | grep -o '"name":"[^"]*"' | head -1 | cut -d'"' -f4)
curl -fsS -H "api-key: $QKEY" -o "$OUT/qdrant-$c.snapshot" "$QURL/collections/$c/snapshots/$snap"
curl -fsS -X DELETE -H "api-key: $QKEY" "$QURL/collections/$c/snapshots/$snap" > /dev/null
done
# 3. Uploaded files: the folder inside the app container, as one tar file
docker compose exec -T app tar cf - -C /var/lib/cob-rag/files . > "$OUT/files.tar"
# 4. Configuration and secrets
cp .env "$OUT/env"
chmod 600 "$OUT/env"
# 5. Keep two weeks of dated backups on this machine
find "$BACKUPS" -maxdepth 1 -type d -name '20*' -mtime +"$KEEP_DAYS" -exec rm -r {} +
# 6. Copy it OFF this machine. Uncomment one line after you set it up.
# rclone sync "$BACKUPS" yourremote:cob-rag-backups
# rsync -a --delete "$BACKUPS/" backup-user@other-server:cob-rag-backups/
echo "backup done: $OUT"nano /home/deploy/backup.sh # paste the script
chmod +x /home/deploy/backup.sh
/home/deploy/backup.sh # run it once and read the output
ls -lh /home/deploy/backups/*/.env, so encrypt it on the way out (for example with rclone’s crypt remote). Also turn on your provider’s own server snapshots as a second layer.If you chose an external S3 provider (RAG_STORAGE_PROVIDER=s3) instead of files on the server, delete step 3 and rely on that provider’s versioning or replication.
9.2Run it every night#
Servers usually keep UTC time. Set India time first, so “2:30 in the morning” means 2:30 in the morning for you:
sudo timedatectl set-timezone Asia/Kolkata
crontab -e30 2 * * * /home/deploy/backup.sh >> /home/deploy/backup.log 2>&1Check /home/deploy/backup.log the next morning, and set an alert if a backup does not run: a free service such as Healthchecks.io works by having the script call a URL when it succeeds.
9.3The restore drill#
Do this once on a spare server before you need it, and write down how long it took. On the spare: do sections 4 to 6 (so you have Docker, the code and a running stack), then put the backed-up env file in place as .env before starting. A different RAG_ENCRYPTION_KEY would make stored AI keys unreadable.
- Stop everything that writes, keeping the datastores running:
docker compose stop app worker mailer celery-beat - Restore Postgres.
Use your own backup date in the folder name.
docker compose cp /home/deploy/backups/2026-10-02/postgres.dump postgres:/tmp/postgres.dump docker compose exec -T postgres pg_restore -U rag -d rag_kernel --clean --if-exists --no-owner /tmp/postgres.dump - Restore the files. Unpack the tar file into the files volume:
(With an external S3 provider this step does not apply: restore from that provider’s own versions.)
docker compose run --rm --no-deps -T --entrypoint tar app xf - -C /var/lib/cob-rag/files < /home/deploy/backups/2026-10-02/files.tar - Restore the vectors. Either upload each snapshot, or rebuild from Postgres once the stack is running (step 5):
# upload one snapshot (repeat per collection) curl -fsS -X POST -H "api-key: YOUR_RAG_QDRANT_API_KEY" \ -F "snapshot=@/home/deploy/backups/2026-10-02/qdrant-COLLECTION.snapshot" \ "http://127.0.0.1:6333/collections/COLLECTION/snapshots/upload?priority=snapshot"A rebuild drops that knowledge base’s collection and writes it again, so its searches return nothing until it finishes. It cannot restore document metadata for documents ingested by engine versions before 0.2.# or rebuild a knowledge base's vectors from Postgres (run without --apply first for a dry run) docker compose exec app python -m callobuzz_rag.retrieval.reindex --apply --rebuild --corpus KNOWLEDGE_BASE_ID - Start and check. Bring everything up, then ask a real question in the dashboard’s playground and open a cited source:
docker compose up -d curl -fsS http://127.0.0.1:8000/api/v1/health/ready
The Qdrant upload step is Qdrant’s own snapshot feature; its snapshot documentation covers the details and version limits. Qdrant here is v1.13.2; restore into the same version.
10Updates and rollback#
sudo bash /opt/cob-rag/current/deploy/upgrade.sh cob-rag-engine-new.zip (a zip or an unzipped folder). It takes a backup, builds the new version while the old one keeps answering, updates the settings file with any new settings, updates the database, switches, and checks the engine is healthy. If anything fails after the backup, it prints the exact command that puts the old version and its data back: sudo bash /opt/cob-rag/releases/<old version>/deploy/upgrade.sh --rollback (it asks you to type rollback). Add --dry-run to see the steps first. The text below is doing it by hand.Updating is: back up, fetch the new code, rebuild. The app container runs alembic upgrade head before it starts serving, so database migrations apply on their own. Read CHANGELOG.md first. If your copy ran without a licence key or on an older paid-licence tier, customer billing was off even with RAG_BILLING_ENABLED=true; after this upgrade it is on, so set it to false in your settings file before you upgrade if you do not charge your businesses; an update occasionally changes what Docker creates, and the changelog says so.
cd /home/deploy/cob-rag-engine
/home/deploy/backup.sh
git checkout main
git pull
docker compose up -d --build
# confirm the migrations ran (a failure prints "ERROR: Alembic migration failed!")
docker compose logs app | grep -E "Migrations completed|Alembic migration failed"
curl -fsS http://127.0.0.1:8000/api/v1/health/readyWith a GPU, use the same -f docker-compose.yml -f docker-compose.gpu.yml pair on the up command as in section 6.
COB, the built-in assistant, re-reads its own documentation after every update: the app queues one sync on start, the worker ingests only the pages that changed, and the COB docs card on Engine info shows when it is done. If it ever sticks, press Resync there.
10.1Roll back#
Releases are tagged. To go back to one, check out its tag and rebuild:
cd /home/deploy/cob-rag-engine
git fetch --tags
git tag --list
git checkout v0.2.0
docker compose up -d --buildbackup.sh.To free disk after updates, remove old images: docker image prune -f.
11Automatic deploys with GitHub Actions#
Once updating by hand is routine, you can let GitHub do it: every push to main, or every version tag, runs the tests, then updates the server and checks it is healthy. This page shows the workflow file; it is not part of the repository, and you add it to your own private copy of the repository.
You chose “By hand”, so you can skip to section 12. Switch the choice above, or turn on “Show all options”, to read this section.
11.1How it fits together#
- The tests run on a GitHub-hosted machine, so they never touch your server.
- The deploy job runs on your server (self-hosted runner) or reaches it over SSH. It uses the clone in
/home/deploy/cob-rag-engineand the.envalready there, so your secrets never go into GitHub on the runner path. - The job takes a backup, fetches the exact commit that triggered it, runs
docker compose up -d --build, checks the migrations reached the latest version, and waits for a healthy answer. If any step fails, the job fails and you are told. - To roll back, run the workflow again from an older tag (Actions, Run workflow, choose the tag).
11.2Install the runner on the server#
A self-hosted runner is a small program on your server that asks GitHub “is there a job for me?” over an outgoing connection. Nothing needs to be opened in the firewall. GitHub’s own instructions for adding a runner are the source of truth; the steps are:
- In your repository on GitHub, open Settings, Actions, Runners, New self-hosted runner, choose Linux and the architecture of your server. GitHub shows the exact download and unpack commands for the current runner version. Run those on the server as
deploy, in/home/deploy/actions-runner. (The configure step refuses to run as root.) - Register it with the token that page shows (it expires within the hour). The
cob-prodlabel is what the workflow asks for:./config.sh --url https://github.com/YOURNAME/cob-rag-engine --token TOKEN_FROM_THE_GITHUB_PAGE --labels cob-prod --name prod-vps - Run it as a service so it survives reboots:
sudo ./svc.sh install deploy sudo ./svc.sh start sudo ./svc.sh status - In GitHub, open Settings, Environments, New environment, name it
production, add yourself under Required reviewers, and limit Deployment branches and tags tomainandv*. Each deploy then waits for your click. - Create
.github/workflows/deploy.ymlin your repository with the workflow file in the next step, and push it.
11.3The workflow, with a runner on the server#
This is a complete file. The GitHub Actions in it are written with version tags, like the repository’s own ci.yml; see the notes below about pinning them.
# Add this to YOUR repository. It is an example, not a file in cob-rag-engine.
name: Deploy to production
on:
push:
branches: [main]
tags: ["v*"]
workflow_dispatch: # Run workflow button; also how you roll back to a tag
# One deploy at a time, and never cancel one that is half finished.
concurrency:
group: production-deploy
cancel-in-progress: false
permissions:
contents: read
jobs:
test:
name: Tests
runs-on: ubuntu-latest # GitHub's machine, not your server
timeout-minutes: 30
steps:
- uses: actions/checkout@v4 # pin to a full commit SHA (see the notes)
- uses: actions/setup-python@v5 # pin to a full commit SHA
with:
python-version: "3.12"
cache: pip
cache-dependency-path: pyproject.toml
- name: Install
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
- name: Unit tests
run: pytest -q
deploy:
name: Deploy
needs: test
runs-on: [self-hosted, linux, cob-prod]
environment: production # required reviewers apply here
timeout-minutes: 45
env:
APP_DIR: /home/deploy/cob-rag-engine
HEALTH_URL: http://127.0.0.1:8000/api/v1/health/ready
DEPLOY_SHA: ${{ github.sha }}
steps:
- name: Back up first
run: /home/deploy/backup.sh
- name: Fetch the exact commit
run: |
cd "$APP_DIR"
git fetch --tags origin
git checkout --detach "$DEPLOY_SHA"
- name: Build and start
run: |
cd "$APP_DIR"
docker compose up -d --build
# GPU server: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build
- name: Check the migrations reached the latest version
run: |
cd "$APP_DIR"
docker compose exec -T app alembic current | grep "(head)"
- name: Wait for a healthy engine
run: |
for i in $(seq 1 60); do
if curl -fsS "$HEALTH_URL" > /dev/null; then
echo "healthy"
exit 0
fi
echo "waiting ($i/60)"
sleep 5
done
cd "$APP_DIR"
docker compose ps
docker compose logs --tail=100 app
exit 1
- name: Remove old images
run: docker image prune -fThe health step calls /api/v1/health/ready, which answers 503 until Postgres, Valkey and Qdrant all respond, so curl -f keeps waiting and the job fails if the engine never becomes ready.
11.4The alternative: GitHub-hosted runner and SSH#
No runner on your server: a GitHub-hosted machine signs in over SSH and runs the same steps. The cost is that SSH (port 22) must be reachable from GitHub’s changing address ranges, so allow key logins only and turn off password logins (PasswordAuthentication no in /etc/ssh/sshd_config).
- On your own computer, make a key used for nothing else, with no passphrase:
ssh-keygen -t ed25519 -f cob-deploy-key -C github-deploy -N "" ssh-keyscan -t ed25519 chat.yourbusiness.in - Append the contents of
cob-deploy-key.pubto/home/deploy/.ssh/authorized_keyson the server. - In GitHub, Settings, Environments, New environment, named
production, with required reviewers. Add four environment secrets:DEPLOY_HOST(chat.yourbusiness.in),DEPLOY_USER(deploy),DEPLOY_SSH_KEY(the private key file’s contents) andDEPLOY_KNOWN_HOSTS(thessh-keyscanline; compare its fingerprint with the server’s first). - Create
.github/workflows/deploy.ymlin your repository with this complete file:# Add this to YOUR repository. It is an example, not a file in cob-rag-engine. name: Deploy to production over SSH on: push: branches: [main] tags: ["v*"] workflow_dispatch: concurrency: group: production-deploy cancel-in-progress: false permissions: contents: read jobs: test: name: Tests runs-on: ubuntu-latest timeout-minutes: 30 steps: - uses: actions/checkout@v4 # pin to a full commit SHA (see the notes) - uses: actions/setup-python@v5 # pin to a full commit SHA with: python-version: "3.12" cache: pip cache-dependency-path: pyproject.toml - name: Install run: | python -m pip install --upgrade pip pip install -e ".[dev]" - name: Unit tests run: pytest -q deploy: name: Deploy over SSH needs: test runs-on: ubuntu-latest environment: production timeout-minutes: 45 steps: - name: Prepare SSH env: SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }} KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }} run: | install -m 700 -d ~/.ssh printf '%s\n' "$SSH_KEY" > ~/.ssh/deploy_key chmod 600 ~/.ssh/deploy_key printf '%s\n' "$KNOWN_HOSTS" > ~/.ssh/known_hosts - name: Deploy env: DEPLOY_SHA: ${{ github.sha }} DEPLOY_TARGET: ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }} run: | ssh -i ~/.ssh/deploy_key "$DEPLOY_TARGET" bash -s -- "$DEPLOY_SHA" <<'REMOTE' set -euo pipefail cd /home/deploy/cob-rag-engine /home/deploy/backup.sh git fetch --tags origin git checkout --detach "$1" docker compose up -d --build docker compose exec -T app alembic current | grep "(head)" for i in $(seq 1 60); do if curl -fsS http://127.0.0.1:8000/api/v1/health/ready > /dev/null; then echo "healthy" exit 0 fi sleep 5 done docker compose ps docker compose logs --tail=100 app exit 1 REMOTE
11.5Security notes#
deploydeploy is in the docker group, which is as good as root on the server. Use a private repository. On a public repository, anyone can open a pull request, and a workflow that runs on pull requests would run their code on your server. This workflow only runs on pushes, tags and a manual click, never on pull_request; keep it that way.- Require approval for outside contributors to run workflows (Settings, Actions, General) if your deploy repository is ever public. Better: keep it private and copy each new release into it.
- Protect the triggers. Only people you trust should be able to push to
mainor createv*tags. Turn on branch protection and tag rules; theproductionenvironment’s reviewers are the last gate. - Pin actions. A version tag such as
actions/checkout@v4can be moved by whoever owns it. Replace each with the full commit SHA of the release you reviewed, with the tag in a comment, and update on purpose. See GitHub’s security hardening guide. - Least privilege. The workflow sets
permissions: contents: read. Do not add write permissions you do not use. - Secrets. On the runner path, the server’s
.envnever leaves the server. On the SSH path, keep the four secrets in theproductionenvironment, not repository-wide, so only an approved deploy can read them. Never print them withecho. - Pass values through
env:, as the workflow does, instead of pasting${{ ... }}into arun:script, so a branch or tag name cannot inject commands. - The runner machine is production. Do not run other repositories’ jobs on it.
12Monitoring#
You need to hear about trouble before a customer tells you. Three habits cover most of it.
12.1Look at the server#
# which containers run, and are they healthy?
docker compose ps
# recent errors from the engine, the ingest worker and the mailer
docker compose logs --since 1h app worker mailer | grep -i -E "error|traceback"
# memory and CPU per container
docker stats --no-stream
# disk: the machine, then Docker
df -h /
docker system df
free -hLogs are rotated for you: each container keeps at most three files of 10 MB.
12.2Have something watch from outside#
Point a free uptime monitor (UptimeRobot, Better Stack, Healthchecks.io and others) at https://chat.yourbusiness.in/api/v1/health/ready every minute and alert on anything but HTTP 200: that address answers 503 when Postgres, Valkey or Qdrant is down, so a plain status check is enough. Add a second monitor for https://app.yourbusiness.in, and an alert for disk above 80 percent.
12.3Read the health answer#
The answer lists each service as true or false: api, postgres, redis, qdrant and storage (the last checks that the files folder, or the S3 bucket, is usable). The one that is false is where to look: docker compose logs <that service>. /health/ready gives the same list without storage, changes the status code, and calls the overall status ready or not_ready.
13Connect WhatsApp (Meta app)#
The WhatsApp channel is off until you give the engine your own Meta app. This section is only needed if you offer WhatsApp. Every engine operator registers their own app, so nothing here is shared with anyone else, and Meta’s review is on you, not on the software.
13.1Register the Meta app#
In Meta for Developers create an app of type Business and add the WhatsApp product. Note the App ID and App Secret. In Meta Business Settings complete Business Verification. To offer Embedded Signup to businesses, become a Tech Provider and pass App Review for whatsapp_business_management and whatsapp_business_messaging. Meta limits a new Tech Provider to about 10 numbers a week and raises it to 200 a week once approved. Until then you can still connect numbers by hand. Then create a Facebook Login for Business configuration of the WhatsApp Embedded Signup type (version 4), add your dashboard’s address to the allowed domains and note its Configuration ID. Check Meta’s current documentation for the exact screens, which change.
13.2Point the webhook at the engine#
In the app’s WhatsApp configuration set the callback URL to https://api.example.com/api/v1/public/whatsapp/webhook (your engine’s public address, over HTTPS) and the verify token to the value you choose below. Subscribe to messages, and to smb_message_echoes if you want businesses to use their number in the WhatsApp Business app as well. The engine checks Meta’s signature (X-Hub-Signature-256) on the raw body of every delivery with your app secret and answers 401 to anything else (the one-time verify request with a wrong token is the only 403).
13.3Settings in .env#
Put these in .env (all start with RAG_WHATSAPP_), then restart the app, worker and beat:
RAG_WHATSAPP_ENABLED=true
RAG_WHATSAPP_APP_ID=your-app-id
RAG_WHATSAPP_APP_SECRET=your-app-secret
RAG_WHATSAPP_VERIFY_TOKEN=a-long-random-string
RAG_WHATSAPP_CONFIG_ID=your-embedded-signup-configuration-id
RAG_WHATSAPP_GRAPH_VERSION=v23.0The engine refuses to start with RAG_WHATSAPP_ENABLED=true and a missing app id, secret or verify token. It only talks to graph.facebook.com over HTTPS at that version. Check /api/v1/info: features.whatsapp.enabled is true. Then tick WhatsApp included on the plans that should have it.
13.4Connect a number and check it#
A workspace owner connects with Connect with Meta on the WhatsApp page. To connect for them, open the customer, Knowledge bases, Connect WhatsApp for this business and paste the phone number ID, the WhatsApp Business Account ID and a permanent token. Then send a message to the number: the chat appears on the WhatsApp page within seconds. If it does not, Check the connection on that page, and make sure the worker is running (it answers) and the callback URL is reachable from the internet. Set your fee rates under Sign-up & branding; they start at zero. Meta’s own prices are on Meta’s current price list.
14Troubleshooting#
Each item reads symptom, why it happens, fix. Start with docker compose ps and docker compose logs <service>; they answer most questions. Problems with the local install itself are in the guide’s troubleshooting.
14.1No HTTPS certificate#
nslookup chat.yourbusiness.in, the firewall with sudo ufw status, and the log with sudo journalctl -u caddy --no-pager -n 50 (Nginx: sudo certbot certificates). Wait a few minutes after a DNS change and retry.14.2The dashboard loads but sign-in fails#
RAG_CORS_ORIGINS (written exactly: scheme and host, no trailing slash), or RAG_DASHBOARD_ENGINE_URL points somewhere the browser cannot reach..env (section 5.2), then docker compose up -d --build. For the Scraper page, SCRAPER_CORS_ORIGINS and RAG_DASHBOARD_SCRAPER_URL work the same way.14.3502 Bad Gateway#
docker compose ps, then docker compose logs --tail=100 app. From the server, curl http://127.0.0.1:8000/api/v1/health/ready must work before the proxy can. Ports: engine 8000, dashboard 8090, scraper 8001.14.4The engine will not start in production#
RAG_EMAIL_PROVIDER=log is refused when RAG_ENV=prod.RAG_SMTP_HOST, RAG_SMTP_FROM and the other SMTP settings (section 5.3) and remove RAG_EMAIL_PROVIDER=log. A wildcard RAG_CORS_ORIGINS is not fatal: production ignores it and allows no browser origin, which breaks the dashboard until you list its origin.14.5E-mails do not arrive#
docker compose logs mailer. Transient SMTP errors are retried automatically; permanent ones are logged with the SMTP code only. Verify the sender domain at your mail service and add its DNS records.14.6Uploads fail with 413#
RAG_MAX_UPLOAD_BYTES), and a plan may set less.client_max_body_size 50m; in the chat.yourbusiness.in server block, then sudo nginx -t and sudo systemctl reload nginx.14.7The widget says “too many requests” for everyone#
RAG_WIDGET_TRUST_PROXY_HEADERS=false every visitor looks like the proxy. The opposite fault is the setting on with a proxy that appends to X-Forwarded-For instead of replacing it.true and use the proxy settings in section 7, which replace the header. Recreate the containers with docker compose up -d.14.8Containers are killed or the server freezes#
sudo dmesg | grep -i "out of memory". Add swap (section 2), keep RAG_API_WORKERS=1, keep the re-ranker off on CPU, lower RAG_WORKER_CONCURRENCY, or rent the next size up.14.9An update fails during migration#
docker compose logs app ends with ERROR: Alembic migration failed! and the app keeps restarting.14.10The disk is full#
docker system df shows where it went. docker image prune -f removes unused images. Lower KEEP_DAYS in the backup script, copy backups off the server, and grow the disk at your provider if data is the cause.14.11The deploy job never starts#
[self-hosted, linux, cob-prod], or the production environment is waiting for a reviewer.actions-runner, run sudo ./svc.sh status and restart it with sudo ./svc.sh start. Check Settings, Actions, Runners shows it as idle. Then approve the pending deployment on the run’s page.15Run a public demo#
A demo is a copy of the product that anyone can try without signing up. It is for showing the product to clients, or for a live preview on your own website. You turn it on with one setting. This section is optional: skip it if you only run the product for your own businesses.
15.1What demo mode does#
- It prints two sign-ins on the sign-in page: one for the platform owner and one for the owner of a made-up business, Nirmal Family Clinic in Pune. Visitors click Use and sign in.
- The clinic is ready to use. Its assistant answers in English and Hindi from five short documents and six questions with answers.
- It switches off everything that deletes, hides a secret, sends something out or costs money: deleting, API keys, e-mail settings, WhatsApp, top-ups, plans, webhooks and password or e-mail changes. A visitor who tries one sees one friendly sentence.
- It sends nothing out. No e-mail, no WhatsApp message and no webhook leaves the server.
- It puts a hard daily limit on the whole demo and a smaller one on each visitor, so one person cannot use it all. When a limit is used up, the visitor gets a polite message, not an error. Everyone can see how many demo answers are left today.
- Every night it puts everything back exactly as it was, so a visitor who changes things leaves nothing behind.
15.2What it does not do#
A demo copy has every feature, like any other copy. The demo only refuses the changes listed above.
15.3Turn it on#
Use a separate server for a demo, not the one your customers use. Add these lines to .env. The platform owner signs in with the address you already set as RAG_SUPERADMIN_EMAIL, so that line must be there too.
RAG_DEMO_MODE=true
RAG_DEMO_OWNER_EMAIL=owner@nirmalclinic.example.com
RAG_DEMO_PASSWORD=choose-a-long-password-for-visitors
RAG_DEMO_RESET_AT_UTC=21:30
RAG_DEMO_ANSWERS_PER_DAY=300
RAG_DEMO_ANSWERS_PER_VISITOR_PER_DAY=15
RAG_DEMO_MODEL_CALLS_PER_DAY=1200The password is used by both sign-ins and is shown to every visitor. The owner address must be different from RAG_SUPERADMIN_EMAIL. RAG_DEMO_RESET_AT_UTC is the time of the nightly reset in UTC; 21:30 UTC is 03:00 in India. The three daily limits are explained in “Keeping the cost small” below; each takes a number from 1 to 100000.
Then recreate the containers:
docker compose up -dThe engine refuses to start when something is missing and names it, for example RAG_DEMO_PASSWORD. Set the model key (RAG_LLM_API_KEY) too, or the assistant cannot answer. Two minutes after the first start the demo builds itself. Allow about ten minutes for the documents to be learned, then open the sign-in page: it shows the two sign-ins.
15.4Keeping the cost small#
Every limit starts again when the nightly reset runs (RAG_DEMO_RESET_AT_UTC), so a message that says “resets at 21:30 UTC (03:00 IST)” is true for the data and for the limits. Four limits work together:
- One visitor:
RAG_DEMO_ANSWERS_PER_VISITOR_PER_DAY(default 15) is how many answers one visitor gets a day. A visitor is a keyed hash of the address the engine sees, plus the signed-in user. No address is stored. Both demo sign-ins are shared, so the address does most of the work: people behind one office or school network share one quota, and one person who uses both sign-ins gets two. Behind a reverse proxy, setRAG_TRUST_PROXY_HEADERSandRAG_TRUSTED_PROXY_HOPScorrectly, or every visitor looks like the proxy. - The whole demo:
RAG_DEMO_ANSWERS_PER_DAY(default 300) is how many answers the demo gives in a day. Only answers that call the AI model count: small talk the rules answer, cached answers and “talk to a person” requests are free and not counted. It covers the website widget, the dashboard playground and preview, conversations and COB. - Model calls:
RAG_DEMO_MODEL_CALLS_PER_DAY(default 1200) is a hard stop under the answer limits. One answer is about four calls (label, rewrite, answer, fact check), so the default fits 300 answers with room for the playground and setup help. - Money: the demo plan allows 300 questions a day for the clinic and 40 questions a day to COB, and the engine stops the demo’s own AI key at about two US dollars a day.
When a visitor has used their answers they read, as a normal assistant message with no model call: “You’ve used today’s demo questions (15). The demo resets at 21:30 UTC (03:00 IST). Come back tomorrow, or install your own copy to try it without limits.” When the whole demo is used up, everyone reads “Today’s demo answers are all used up” with the same reset time. The dashboard banner and the sign-in page show “X of Y demo answers left today”, and the numbers (never addresses) are in GET /api/v1/public/branding and GET /api/v1/admin/demo. If the counters cannot be read (Redis is down), the demo stops answering with a plain “cannot answer right now” message instead of spending without a limit.
15.5The nightly reset, and doing it by hand#
The reset runs by itself at RAG_DEMO_RESET_AT_UTC. It also runs after a start when no reset has finished in the last 25 hours, which is how a new demo server fills itself. To reset now, or to see how the last reset went, run these on the server:
docker compose exec app python -m callobuzz_rag.demo status
docker compose exec app python -m callobuzz_rag.demo reset --waitstatus shows the last good reset, the next one, today’s answers and today’s model calls. reset --wait waits until the reset is done. Both finish with exit code 0 when all is well, 1 when something failed and 2 when demo mode is off. When a reset fails, the message names the step it stopped at; run it again, and if it fails at the same step, read docker compose logs app around that time.
15.6Showing the dashboard inside another site#
By default the dashboard refuses to be shown inside any other website. That is a safety feature, and it is right for every normal install. For a live preview on another website, allow that site’s address in .env:
RAG_FRAME_ANCESTORS=self https://demo.example.comWrite self and none without quote marks. Every other item must start with https:// and be a site name, with an optional port, like https://demo.yourbusiness.in. https://*.yourbusiness.in allows every sub-site. Then recreate the dashboard:
docker compose up -d dashboardA wrong value stops the dashboard from starting and the message names RAG_FRAME_ANCESTORS. If the preview shows a blank frame, check that the value is exactly as above and that you recreated the dashboard. Leave the setting empty on a normal install.
16Legal pages#
Before you take money or personal data from a business you need a few public pages. The engine ships a plain-language template for each, written for India, and a screen where you fill in your details, edit the text and publish versions. Nothing is public until you publish it. This section is optional: skip it if you run the product only for your own businesses.
16.1The eight pages#
- Terms of Service: what you offer, accounts, plans and top-ups, renewal and cancelling, price changes, fair use, what happens when an account is suspended, your liability, and the courts of your city.
- Privacy Notice: written for the Digital Personal Data Protection Act, 2023. It says what data you collect and why, that you are the data fiduciary for your own customers’ accounts and a data processor for the visitors of the businesses that use the product, who else sees data, how long you keep it, a person’s rights and the 90-day answer, and breaches.
- Refund and Cancellation Policy, with real day counts: unused balance back within 7 working days, failed or duplicate payments back within 5 to 7 working days.
- Shipping and Delivery Policy: it is a digital service, delivered online, and nothing is shipped.
- Acceptable Use Policy, Data Processing Addendum (you as processor for visitors’ data, with a 48-hour breach notice), Sub-processors (a table you edit to list the companies you really use) and Grievance Officer (a name, an e-mail and an address, an answer within 48 hours and a resolution within 30 days).
16.2Fill in, edit, publish#
- Sign in as the platform owner and open Legal pages. Under Your details fill in the grievance officer, the city whose courts the Terms name and your website. Your company’s legal name and address are the Billing details.
- Open a page, read the template, change what does not fit your business, and choose Save draft. The preview shows how it will look. A word in double braces, such as your company name, is filled in when you publish; the editor lists any detail that is still missing.
- Choose Publish, add a note such as “First version”, tick that the page was reviewed and confirm. A published version never changes: publishing again makes the next version and the old one stays on record.
Each page is then a plain web page at https://your-address/legal/terms, /legal/privacy, /legal/refunds, /legal/delivery, /legal/acceptable-use, /legal/dpa, /legal/sub-processors and /legal/grievance. They open without signing in and without JavaScript. If you use the proxy from this guide, /legal/* already goes to the engine.
16.3Where the pages show up#
- Sign-up. Once both the Terms and the Privacy notice are published, the sign-up form asks people to tick that they agree, and the engine records which version each person accepted. With only one of the two published, sign-up works as before. The Who accepted tab lists everyone and downloads a CSV.
- Footers. The published pages are linked under the sign-in and sign-up pages and under every signed-in page. A switch under Your details turns that off.
- The chat widget. Its footer shows a Privacy link: the business’s own privacy policy if it set one (it is the data fiduciary for its visitors), otherwise your Privacy notice. The lead consent notice uses the same link.
16.4Before you ask a payment gateway to check your site#
Razorpay checks your website for six pages: About us, Contact us, Pricing details, Terms and conditions, Privacy policy and a Refunds policy. The Terms, Privacy and Refund pages above cover three of them; the Delivery policy and the Grievance Officer page round them out, and About, Contact and Pricing belong on your own website. The pages must be live, at an address a person can open, before you submit your site. Use the same name and address in the pages as in your account details.
Most duties of the Digital Personal Data Protection Rules apply from 13 May 2027. The templates are written for them now, so you do not have to change them later.
17Take online payments#
Your customers can add money to their balance, pay their plan fee or buy an add-on themselves, with UPI, a card or net banking, through Razorpay. The money reaches their balance within seconds and you do not type a top-up. This section is optional: until you switch it on, customers keep using Request top-up.
rzp_test_) as soon as you sign up. No real money moves in Test mode, and you can try everything below before your business verification (KYC) is finished.17.1In Razorpay#
- Create an account at razorpay.com and, for real money, finish the KYC. Make sure your website shows the pages Razorpay checks (see section 16, Before you ask a payment gateway to check your site).
- Under Account & Settings → API Keys generate a key. Copy the Key Id and the Key Secret now: Razorpay shows the secret once.
- Under Payment Capture choose Automatic.
17.2In your dashboard#
- Sign in as the platform owner. Open Billing → Tax and fill in your company’s name, address and state, and your GST number if you have one. The GST number’s state must be your company’s state.
- Open Billing → Payments, choose Razorpay, paste the Key Id and Key Secret, choose a webhook secret (any long random text) and write it in the Webhook secret box. Choose Save payment settings, then Test connection. The keys are stored encrypted and the page never shows them again.
- The page shows your Webhook address. In Razorpay open Account & Settings → Webhooks → Add new webhook, paste the address, type the same webhook secret, tick the events listed on the page (payment captured, payment failed, order paid, refund processed, refund failed) and save.
- Sign in as a customer’s owner, open Plan & billing, choose Add money and pay a small amount with Razorpay’s test card or UPI details. The balance should grow within a few seconds.
If you prefer to keep the keys out of the dashboard, set RAG_PAYMENTS_PROVIDER=razorpay, RAG_RAZORPAY_KEY_ID, RAG_RAZORPAY_KEY_SECRET and RAG_RAZORPAY_WEBHOOK_SECRET in .env and recreate the services. A value in .env wins over the dashboard, and the page says so. The webhook address must be reachable from the internet over https, so use your real domain and not localhost.
17.3GST, refunds and fees#
- GST. A customer in your state pays CGST and SGST (9% each); a customer in another state pays IGST (18%). The balance receives the amount before GST. Without a GST number the payment carries no tax. Check the rate and your obligations with your accountant: the engine records the split, and you file the returns.
- Refunds. Open the payment under Billing → Payments and choose Refund…. The amount comes out of the customer’s unused balance and goes back to the way they paid, within 180 days. If Razorpay refuses, the balance is put back.
- Fees. Razorpay’s fee is copied onto each payment after settlement and shown in the P&L as Gateway fees. Customers never see it.
- If a message from Razorpay is lost the engine finds the payment itself within five minutes, and every night it checks Razorpay’s settlement report. Deliveries that need attention on the Payments tab lists anything stuck, with a Retry button.
18GST invoices#
Every online payment (section 17) produces a GST tax invoice at that moment, and every refund a credit note. You do not type or number anything. This section tells you what to set once, and what to keep.
18.1Set up your seller details#
- Open Billing → Tax as the platform owner. Fill in your company’s legal name, address and state, and your GST number if you are registered. The number’s state must be the company’s state.
- SAC code. The default
998315(hosting and IT infrastructure provisioning) suits a hosted chatbot service; a different code can be set per service add-on. Ask your accountant which code fits what you sell. - Invoice prefix (for example
CB) and credit note prefix (CN), up to four letters or digits. A number looks likeCB/26-27/00001and the series restarts every 1 April. Changing a prefix during the year starts a fresh counter, and numbers stay unique because the prefix is part of the number. - Choose whether your listed prices include GST. When they do not (the usual case), the dashboard shows the total with GST next to every price.
18.2Which document you get#
- With a GST number: a Tax Invoice. A customer in your state pays CGST 9% + SGST 9%; a customer in another state pays IGST 18% (the place of supply is the customer’s GST state, else the state they entered).
- Without a GST number: a Bill of Supply with no tax lines.
- A customer outside India is an export. Fill in your LUT number (and its end date) in Billing → Tax and the invoice carries the standard wording “Supply meant for export under Letter of Undertaking without payment of integrated tax” with no tax. Without a valid LUT, 18% IGST is charged. File your LUT for the financial year with your accountant before the first export.
- Refunds produce a Credit Note with the same tax split, naming the original invoice. After 30 November following the invoice’s financial year none can be issued; the refund dialog warns you first, and the GST already paid stays paid.
- Not invoiced by the engine: top-ups you record by hand (bank transfers). Record those in your accounting tool.
18.3Keep them for eight years#
An issued invoice cannot be changed or deleted: a database trigger refuses it, and a correction is a credit note. The records live in two places, and your backups must cover both for at least eight years: the tax_documents table in PostgreSQL, and the PDFs in object storage under billing/tax-documents/ (the storage cleanup command never touches that folder). The records stay even after a customer’s workspace is deleted. A nightly check (R7 in Billing → Accounts → Books check) tells you if a payment ever has no invoice.
18.4For your accountant#
Open Billing → Invoices, pick the month and choose Download CSV for your accountant: one row per document with the buyer, B2B or B2C, place of supply, SAC, taxable value, CGST, SGST, IGST, total, the original invoice (for a credit note) and the payment reference. Questions to ask your accountant once:
- SAC 998315 or 997331 for the hosted subscription, and 998314 for setup.
- Is an invoice at each wallet top-up (an advance) acceptable, or do you need a receipt voucher plus an invoice?
- Is “computer-generated invoice” (without a signature image) enough on electronic invoices?
- A credit note for a refund of unused balance that was invoiced months ago.
- A lost chargeback: credit note or not.
- Reverse charge on foreign vendors such as your AI and hosting providers, and the monthly routine.
- Filing the LUT for this financial year, even before the first export customer.
- The B2C invoice-wise threshold in GSTR-1, and whether the CSV columns suit their tool.