Level 1 — Minimal Deployment

Single static binary, embedded candle inference + in-process candle sentence embeddings, zero C/C++ dependencies.

This guide covers deploying Isartor as a standalone process — no sidecars, no Docker Compose, no orchestrator. The firewall binary embeds a Gemma-2-2B-IT GGUF model via candle for Layer 2 classification and uses candle's BertModel (sentence-transformers/all-MiniLM-L6-v2) for Layer 1 semantic cache embeddings — all entirely in-process, pure Rust.

When to Use Level 1

Prerequisites

Memory budget: Gemma-2-2B Q4_K_M ≈ 1.5 GB, candle BertModel ≈ 90 MB, tokenizer ≈ 4 MB, firewall runtime ≈ 50 MB. Total: ~1.7 GB resident.

Option A: One-Click Install (Recommended)

The fastest way to get started is to leverage the pre-built, cross-platform binaries generated by the CI/CD pipeline.

Install via script:

curl -fsSL https://raw.githubusercontent.com/isartor-ai/Isartor/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/isartor-ai/Isartor/main/install.ps1 | iex

This script detects your target OS and processor architecture, downloads the correct release binary, and adds it to your path automatically.

Option B: Build from Source

1. Clone & Build

git clone https://github.com/isartor-ai/Isartor.git
cd Isartor
cargo build --release

The release binary is at ./target/release/isartor (~5 MB statically linked).

2. Configure Environment

Create a minimal .env file or export variables directly:

# Required — your cloud LLM key for Layer 3 fallback
export ISARTOR__EXTERNAL_LLM_API_KEY="sk-..."

# Optional — override defaults
export ISARTOR__GATEWAY_API_KEY="my-secret-key"
export ISARTOR__HOST_PORT="0.0.0.0:8080"
export ISARTOR__LLM_PROVIDER="openai"          # openai | azure | anthropic | xai
export ISARTOR__EXTERNAL_LLM_MODEL="gpt-4o-mini"

# Cache mode — "both" enables exact + semantic cache. Semantic embeddings
# are generated in-process via candle BertModel — no sidecar needed.
export ISARTOR__CACHE_MODE="both"

# Pluggable backends — Level 1 uses the defaults (no change needed):
#   ISARTOR__CACHE_BACKEND=memory     — in-process LRU (ahash + parking_lot)
#   ISARTOR__ROUTER_BACKEND=embedded  — in-process Candle GGUF SLM
# These are ideal for a single-process deployment with zero dependencies.

3. Start the Firewall

./target/release/isartor up

On first start, the embedded classifier will auto-download the Gemma-2-2B-IT GGUF model from Hugging Face Hub (~1.5 GB). Subsequent starts load from the local cache (~/.cache/huggingface/).

INFO  isartor > Listening on 0.0.0.0:8080
INFO  isartor::layer1::embeddings > Initialising candle TextEmbedder (all-MiniLM-L6-v2)...
INFO  isartor::layer1::embeddings > TextEmbedder ready (~90 MB BertModel loaded)
INFO  isartor::services::local_inference > Downloading model from mradermacher/gemma-2-2b-it-GGUF...
INFO  isartor::services::local_inference > Model loaded (1.5 GB), ready for inference

4. Verify

# Health check
curl http://localhost:8080/health

# Test the firewall
curl -s http://localhost:8080/api/chat \
  -H "Content-Type: application/json" \
  -H "X-API-Key: my-secret-key" \
  -d '{"prompt": "Hello, how are you?"}' | jq .

Option B: Docker (Single Container)

For environments where you prefer a container but don't need a full Compose stack.

Build the Image

cd isartor
docker build -t isartor:latest -f docker/Dockerfile .

Run

docker run -d \
  --name isartor \
  -p 8080:8080 \
  -e ISARTOR__GATEWAY_API_KEY="my-secret-key" \
  -e ISARTOR__EXTERNAL_LLM_API_KEY="sk-..." \
  -e ISARTOR__CACHE_MODE="both" \
  -e HF_HOME=/tmp/huggingface \
  -v isartor-models:/tmp/huggingface \
  isartor:latest

Note: The -v flag mounts a named volume for the Hugging Face cache so the model downloads persist across container restarts.

The official Docker image runs as non-root and uses HF_HOME=/tmp/huggingface to ensure the cache is writable.

Option C: systemd Service (Production Linux)

For long-running production deployments on bare metal or VPS.

1. Install the Binary

# Build
cargo build --release

# Install to /usr/local/bin
sudo cp target/release/isartor /usr/local/bin/isartor
sudo chmod +x /usr/local/bin/isartor

2. Create a System User

sudo useradd --system --no-create-home --shell /usr/sbin/nologin isartor

3. Create Environment File

sudo mkdir -p /etc/isartor
sudo tee /etc/isartor/env <<'EOF'
ISARTOR__HOST_PORT=0.0.0.0:8080
ISARTOR__GATEWAY_API_KEY=your-production-key
ISARTOR__EXTERNAL_LLM_API_KEY=sk-...
ISARTOR__LLM_PROVIDER=openai
ISARTOR__EXTERNAL_LLM_MODEL=gpt-4o-mini
ISARTOR__CACHE_MODE=both
ISARTOR__CACHE_BACKEND=memory
ISARTOR__ROUTER_BACKEND=embedded
RUST_LOG=isartor=info
EOF
sudo chmod 600 /etc/isartor/env

4. Create systemd Unit

sudo tee /etc/systemd/system/isartor.service <<'EOF'
[Unit]
Description=Isartor Prompt Firewall
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=isartor
Group=isartor
EnvironmentFile=/etc/isartor/env
ExecStart=/usr/local/bin/isartor
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/cache/isartor

[Install]
WantedBy=multi-user.target
EOF

5. Create Model Cache Directory

sudo mkdir -p /var/cache/isartor
sudo chown isartor:isartor /var/cache/isartor

6. Enable & Start

sudo systemctl daemon-reload
sudo systemctl enable isartor
sudo systemctl start isartor

# Check status
sudo systemctl status isartor
sudo journalctl -u isartor -f

Model Pre-Caching (Air-Gapped / Offline)

If the deployment target has no internet access, pre-download the model on a connected machine and copy it over.

On the Connected Machine

# Install huggingface-cli
pip install huggingface-hub

# Download the GGUF file
huggingface-cli download mradermacher/gemma-2-2b-it-GGUF \
  gemma-2-2b-it.Q4_K_M.gguf \
  --local-dir ./models

# Also grab the tokenizer (from the base model)
huggingface-cli download google/gemma-2-2b-it \
  tokenizer.json \
  --local-dir ./models

Transfer to Target

scp -r ./models/ user@target-host:/var/cache/isartor/

By default, hf-hub uses ~/.cache/huggingface/. In the official Docker image, Isartor sets HF_HOME=/tmp/huggingface (non-root safe). Set HF_HOME or ISARTOR_HF_CACHE_DIR to point to your pre-cached directory if needed.

Level 1 Configuration Reference

These are the most relevant ISARTOR__* variables for Level 1 deployments. For the full reference, see the Configuration Reference.

Embedded Classifier Defaults (Compiled)

Performance Expectations

Upgrading to Level 2

When your traffic outgrows Level 1, the migration path is straightforward:

1. Add the generation sidecar — ISARTOR__LAYER2__SIDECAR_URL=http://127.0.0.1:8081 (replaces embedded candle with the more powerful Phi-3-mini on GPU).
2. Optionally add an embedding sidecar — ISARTOR__EMBEDDING_SIDECAR__SIDECAR_URL=http://127.0.0.1:8082 (only needed for external embedding inference; the default L1b semantic cache already uses in-process candle BertModel).
3. Deploy via Docker Compose — See Level 2 — Sidecar Deployment.

Note: The pluggable backend defaults (cache_backend=memory, router_backend=embedded) remain appropriate for Level 2 single-host deployments. You only need to switch to cache_backend=redis and router_backend=vllm at Level 3 when scaling horizontally.

No code changes required — only environment variables and infrastructure.