AI assistant reference

AI assistant#

The AI assistant lets a signed-in member request a short plain-text summary of a discussion. The feature is off by default and requires an OpenAI-compatible chat completions server chosen by the operator.

Setup#

Configure the model with environment variables or encrypted administrator overrides on Admin → Settings → AI assistant:

• AI_BASE_URL – the base URL of the chat completions server.
• AI_MODEL – the model name or tag to use.
• AI_API_KEY – the API key, if the server requires one.
• AI_REQUEST_OPTIONS – optional JSON object merged into every request body.

After setting the values, open Admin → Settings → AI assistant and enter your password once to unlock changes for 15 minutes (Lock now ends it early). You can also type the server address, model and key on the page instead of using environment values; Remove next to a field goes back to the environment value. Then press Send test prompt and tick Show Summarize on discussions.

Changing the connection turns the assistant off until it passes a new test. The page shows the last test result, so a failed test explains itself.

vLLM#

AI_BASE_URL=http://<host>:8080/v1
AI_MODEL=<served model name>
AI_REQUEST_OPTIONS={"chat_template_kwargs":{"enable_thinking":false}}

No AI_API_KEY is needed for a local vLLM server. Set AI_REQUEST_OPTIONS for Qwen3 reasoning models so the token budget is not spent on hidden thinking.

Ollama#

AI_BASE_URL=http://<host>:11434/v1
AI_MODEL=<model tag>

No AI_API_KEY is needed for a local Ollama server.

Hosted provider#

AI_BASE_URL=https://api.openai.com/v1
AI_MODEL=<model>
AI_API_KEY=<key>

What leaves the server, and when#

Only when a signed-in member presses Summarize on a discussion they can see and no current summary exists: the discussion title, the visible (not deleted) posts' Markdown with author display names and dates, trimmed to forum.ai.prompt_characters. Nothing is sent in the background. Deleted posts, attachments, private messages and email addresses are never sent. The model's answer is stored as plain text and served until a new reply arrives.

Limits#

Setting	Value
Connect timeout	5 s
Request timeout	30 s
Prompt characters	24,000
Summary characters	2,000
Max tokens	600
Rate limit	10 per minute per member
Model calls	One per discussion version (newest post ID)

Security#

• Only administrators can set the base URL; it may be a private or LAN address on purpose.
• Outbound requests never follow redirects.
• The API key is sent only as a Bearer header and is never shown in the UI or written to logs.
• Model output is stored and rendered as escaped plain text, never as Markdown or HTML.
• Moderators can clear a stored summary.

Troubleshooting#

• The model did not answer – check the base URL is reachable from the server, the model name matches what the server serves, and the API key is correct.
• Empty answers from reasoning models – set AI_REQUEST_OPTIONS (e.g. {"chat_template_kwargs":{"enable_thinking":false}} for vLLM + Qwen3).
• Timeouts on long discussions – use a smaller model or lower forum.ai.prompt_characters.
• "A summary is already being prepared" – another member's request is still running; wait and retry.