Supported LLM providers
Vertex AI and Azure currently support only their native model families. If you need cross-provider model support (e.g., Anthropic models on Vertex AI or Azure), please contact us — we are happy to add it based on customer demand.
Environment variables
Prompt caching
GoRules AI uses prompt caching to reduce token usage and improve response times. Caching behavior depends on the provider:
No additional configuration is required — caching is handled automatically for all supported providers. Prompt caching can reduce token costs by up to 90% in some cases, though actual savings depend on the provider, model, and usage patterns.
For self-hosted deployments, ensure your load balancer has response buffering disabled or streaming enabled for optimal AI assistant experience.