Routerly 0.3.0 is on the develop channel 🧪
0.3.0 is tagged on the develop channel. This is a pre-release: the feature set is frozen and the test suite is green, but the build has not been promoted to stable yet. The goal right now is to catch anything the tests missed before it lands on latest.
To try it, pull inebrio/routerly:develop or switch your channel to develop and run routerly update run. If something breaks, open an issue.
Here is what is in 0.3.0.
Web and OAuth providers
0.3.0 adds openai-web and anthropic-web as first-class provider adapters. Both route through browser-session-authenticated endpoints instead of API keys.
The practical use case: if you already have an active ChatGPT Plus or Claude Pro subscription, you can now use it as a provider in Routerly without purchasing a separate API plan. The adapters authenticate using a browser session token and route requests through the same web interface your subscription covers.
They slot into the provider pool like any other adapter: fallback, cost tracking, and routing policies all apply. Mix them with API-key providers in the same project and Routerly treats them as ordinary pool members.
Content guardrails and PII scrubbing 🛡️
Guardrails are the most substantial new feature in this release.
Every project can define rules that inspect request and response content before it reaches the provider or the client. Each rule has two settings: a scope (request, response, or both) and an action (block or log). Rules can be combined and applied independently, so you can log a pattern in development and block the same pattern in production without duplicating configuration.
The PII scrubbing layer integrates directly with guardrails. You define named PII policies per project, specifying which entity types to redact: names, email addresses, phone numbers, financial identifiers, and more. Multiple named policies can be active on the same project simultaneously.
The harder part is ambiguous content. When a rule cannot make a binary decision from pattern matching alone, Routerly delegates to a judge model: a secondary LLM call that evaluates the full context of the exchange and returns a scored verdict. Judge tokens are tracked separately in usage logs, and the judge model is configurable per project. The full evaluation trace, including the judge’s reasoning and score, appears in the request detail view in the dashboard.
Blocked requests return the exact wire format the client expects. An OpenAI client gets an OpenAI-shaped error; an Anthropic client gets an Anthropic-shaped one.
Notifications 🔔
0.3.0 ships a complete notification system, covering channel configuration, event routing, per-project scoping, cooldowns, and an in-app inbox.
Native adapters for Slack, Microsoft Teams, PagerDuty, and Discord ship out of the box. The adapters handle their own webhook formats, authentication, and retry logic. Configure a channel once and reference it from any rule.
Event routing lets you decide which events go where. Budget thresholds, guardrail blocks, provider health changes, and error spikes are all surfaced as events. Rules map an event type to a channel, with optional project scope so a rule only fires for requests from a specific project. Channels also support configurable cooldown windows: when the same event fires repeatedly within the window, Routerly suppresses the duplicates. Per-project scoping goes further, letting you restrict a channel to events from specific projects, which keeps team channels clean when multiple teams share an instance.
The in-app inbox collects all notification events regardless of whether external channels are configured. Every admin user has their own inbox with per-notification dismiss. Clicking an inbox item opens the notification detail.
Notification channels, rules, and the inbox are all accessible from the CLI with the routerly notifications command group.
Model discovery and dynamic catalog
The model catalog is no longer baked into Routerly. In 0.3.0, it loads at runtime from one or more configurable git repositories.
The practical consequence: you can maintain your own catalog repo. Add a new model with its pricing and context-window data, push, and Routerly picks it up on the next sync interval — no Routerly update required, no waiting for a pull request to merge upstream. For teams that run internal or fine-tuned models, this also means you can add proprietary entries that will never belong in a public catalog.
The default catalog still ships with Routerly and stays up to date, so there is nothing to configure if you do not need a custom one. Pointing at an additional repo just extends what Routerly knows about.
The catalog page in the dashboard lists all available models with provider and context-length filters. The CLI exposes routerly models discover to query it from the terminal.
Prometheus metrics
A /metrics endpoint is now available in the service, exposing the standard Prometheus format. The endpoint covers request counts, latency histograms, cost totals, error rates, cache hit rates, and provider health counters. It requires a separate auth token configured in Routerly settings, distinct from the main admin credentials.
A ready-made Prometheus and Grafana configuration is available in the docker/ directory. Pull it into your existing docker-compose stack to get a working metrics pipeline without manual scrape configuration.
Metrics export integrations ship alongside the Prometheus endpoint, with adapters for forwarding events to external observability platforms.
Two-factor authentication
The admin dashboard supports TOTP-based two-factor authentication. Enrollment works through the Settings page: scan the QR code with any TOTP-compatible authenticator app, verify the first code, and 2FA is active. Subsequent logins require the current TOTP code after the password step.
Admins can require 2FA across all users from the Users settings. When enforcement is on, users without 2FA enrolled are redirected to the enrollment page and cannot access the rest of the dashboard until they complete setup. The Users admin list includes a reset 2FA button for individual accounts.
Usage enrichment
Usage tracking supports three new attribution dimensions. Pass a user identifier in requests and Routerly records it alongside cost, latency, and model data; the dashboard includes an End Users tab per project showing the breakdown. Project API tokens can also carry a session identifier, grouping all requests under the same session in usage logs, which makes per-session cost accounting straightforward. Finally, API tokens support arbitrary string tags so you can label a token with department, environment, feature flag, or any other dimension and filter usage by those tags in the dashboard.
Prompt playground
The dashboard includes a full prompt playground with a compare mode: run the same prompt through two different model configurations side by side, with per-panel parameter controls and token counts displayed independently. Presets let you save and reload configurations.
The playground is accessible at the project level and uses the project’s configured API key, so it tests against your actual routing rules and policies.
Provider health
Provider health tracking has its own tab inside the Models page. The view shows real-time and historical availability, latency percentiles (including p95 over a 1-hour window), and error rates per provider and per model. The table is sortable, filterable, and supports pagination for large provider pools.
The underlying health data is available via the API at /api/health/providers. The CLI exposes it through routerly service health.
Enterprise providers
0.3.0 adds adapters for AWS Bedrock, Azure OpenAI, and Google Vertex AI. All three slot into the standard provider pool and support the same fallback, cost tracking, and routing rules as existing providers. Bedrock uses your existing AWS credentials and handles the Converse API natively including streaming.
Model-preference routing policy
A new routing policy ships in 0.3.0: model-preference. When the client names a specific model in the request, the policy gives that model a scoring bonus, letting it win over cost or health policies without making it a hard requirement. When no model is specified, the policy abstains and the other active policies decide.
Smaller additions
Transparent pass-through proxy. Routerly now forwards any provider endpoint it does not natively handle, instead of returning an error. This matters for provider-specific capabilities such as fine-tuning or batch endpoints that Routerly does not parse.
Prompt cache token tracking. Cache-read and cache-write tokens reported by Anthropic and OpenAI are recorded separately in usage logs and factored into the per-request cost breakdown. Client-side cache_control markers pass through unchanged.
Model test button. Every model entry in the Models table and the model edit page includes a test button that fires a minimal request and reports the response, latency, and cost inline.
TTFT timeout enforcement. Routerly now enforces a configurable time-to-first-token timeout per model attempt. If a provider does not start streaming within the window, the request fails over to the next model in the pool rather than waiting indefinitely.
Gemini fix. The Gemini adapter now strips the provider prefix from the model ID before forwarding to the upstream API, fixing a routing error that surfaced when using prefixed model names.
No breaking changes
The OpenAI and Anthropic wire formats are unchanged. Existing configs, API keys, and client integrations carry over as-is.
Update
# Docker (develop channel)
docker pull inebrio/routerly:develop
docker compose up -d
# CLI install (when on the develop channel)
routerly update runThis release is on the develop channel. The latest tag still tracks the stable channel and will point to 0.3.0 when it is promoted to stable.
Source, install instructions, and the issue tracker: github.com/Inebrio/Routerly. Docs at doc.routerly.ai.
Sources
- Routerly repository: github.com/Inebrio/Routerly