[ACE_FLEET // AI_COMPUTE_EFFICIENCY_ENGINE]

Cut Your AI Inference Costs
Without Slowing Developers Down.

Route, govern, observe, and optimize every LLM API call from one control plane.

Save 30–70% on OpenAI, Azure, and Anthropic bills in under 60 seconds. No code changes.

select_your_stack →
ace_efficiency_engine · projectedlive
→ Every prompt cached, routed, and pruned before it burns a token.
prompt_tokens
−58%
cache_hit_ratio
0.71
avg_latency
18ms
cost_per_1k_calls
$0.34
Start Free Trial — 60s Setup →Calculate My Savings →
Built for Cloud APIs today. ·Scale seamlessly to Kubernetes and bare metal when you're ready.
interactive merit-order arbitrage simulatorlive
incoming api traffic 10,000 QPS
├─►[1. Primary Cloud API]Azure OpenAI PTU$0.15/1k85% loadcommitted throughput, saturated first
├─►[2. Elastic Burst API]OpenAI Direct$0.50/1k15% loadmetered spill once the PTU is full
├─►[3. Self-Hosted GPUs]vLLM Clusteroptional$0.04/1kamortisedcustomadd your own hardware and this rung fills first
live savings counter · projected
$6,533saved today
across 22.0M tokens
vs. no ladder
60%
$243.00/s vs $600.00/s
Sign up for a free compute audit on your real traffic →
Already using OpenAI / Anthropic? Change one line:
python
base_url = "https://engine.acefleet.dev/v1"
SOC2 Type IIZero Data RetentionLocal ONNX EmbeddingsSelf-Hosted VPCK8s Operator Native
the problem

Your cloud API bill is growing faster than your product.

Every layer of your AI stack — from silicon to agent frameworks — adds cost, latency, and fragility. ACE operates across all of them.

HWHardware & Accelerators·NVIDIA H100/H200 · Groq LPU · Cerebras WSE-3 · AMD Instinct
ENGInference Engines & Serving·vLLM · SGLang · TensorRT-LLM · TGI
CLDClouds & Orchestration·AWS Bedrock · Azure · GCP · CoreWeave · Kubernetes · Ray
AGTAgent Frameworks·CrewAI · LangChain · LlamaIndex · DSPy
HWSilicon & Accelerators·NVIDIA H100/H200 · Groq LPU · Cerebras WSE-3 · AMD Instinct
NETInterconnect·NVLink · InfiniBand · RoCEv2
CLDNeoclouds & Hyperscalers·CoreWeave · Nebius · Lambda · AWS · GCP · Azure
ORCOrchestration·Kubernetes · Ray · Karpenter
SRVInference Engines·vLLM · SGLang · TensorRT-LLM · TGI
OSSCustom Open-Source Models·Llama · Qwen · DeepSeek · Mistral
ENTEnterprise Platforms·Databricks · Snowflake · Vertex · Bedrock
AGTFrontier APIs & Agents·OpenAI · Anthropic · CrewAI · LangGraph · LlamaIndex · DSPy
HWHardware & Accelerators·NVIDIA H100/H200 · Groq LPU · Cerebras WSE-3 · AMD Instinct
ENGInference Engines & Serving·vLLM · SGLang · TensorRT-LLM · TGI
CLDClouds & Orchestration·AWS Bedrock · Azure · GCP · CoreWeave · Kubernetes · Ray
AGTAgent Frameworks·CrewAI · LangChain · LlamaIndex · DSPy
HWSilicon & Accelerators·NVIDIA H100/H200 · Groq LPU · Cerebras WSE-3 · AMD Instinct
NETInterconnect·NVLink · InfiniBand · RoCEv2
CLDNeoclouds & Hyperscalers·CoreWeave · Nebius · Lambda · AWS · GCP · Azure
ORCOrchestration·Kubernetes · Ray · Karpenter
SRVInference Engines·vLLM · SGLang · TensorRT-LLM · TGI
OSSCustom Open-Source Models·Llama · Qwen · DeepSeek · Mistral
ENTEnterprise Platforms·Databricks · Snowflake · Vertex · Bedrock
AGTFrontier APIs & Agents·OpenAI · Anthropic · CrewAI · LangGraph · LlamaIndex · DSPy
See how much you could save on your stack →
interactive cost calculator

How much are you overpaying for AI?

Drop in your monthly spend and see what ACE would have saved on it. Cascaded waterfall math — no overlapping percentages, no black-box multipliers.

1 · workload_profile
2 · monthly_ai_spend$100,000
$1k$500k+
3 · active_optimization_skills
projected_outputreal-time
modelled annual savings · expected case
$546,329
conservative $382,430·aggressive $655,595
Modelled on 30% cacheable · 30% routable · 30% compressible traffic
current_monthly
$100,000
optimized_monthly
$54,473
monthly_savings
$45,527
cache router prune net spend
performance_bonus
20%est. P95 latency reduction
Verify this on your own traffic →Zero code refactoring · 60-second setupACE is priced as a share of what it actually reclaims — see pricing →or get a full free compute audit →
sizing calculator

How much committed capacity
should you actually buy?

Azure PTUs, Bedrock Model Units and Vertex GSUs are cheaper per token and bill around the clock — so the question is never "how much do we use" but "how much of the day is worth pre-paying for." Answer four questions and find out. No key, no account, nothing leaves your browser.

This sizes the commitment you make to your provider. It is not a quote for ACE.

$

What you pay OpenAI, Azure, Bedrock or Vertex today, at list.

Tokens per day, in + out

Implied blended rate $0.0067/1k.

What does your day look like?

Peak-to-trough ratio. Without it there is no distribution to size against, and the honest answer to every question becomes "commit everything."

Your quoted rate against list. This moves the answer more than anything else on this page — the optimal service level is exactly 1 − committed/metered.

Provision
7,026 tok/s

Committed throughput to buy (4 machines / PTU units at $2,000/mo floor). Run it at 5,621 tok/s — the extra 20% is headroom, because past 80% utilization latency jitter breaks the SLO and the router starts steering traffic away.

A day, splitutc hour
00061218
committed 83%metered spill 17%··· provisioned  — run-at (80%)
Committed util
68%
Service level
p50
New monthly
$77,689
Saved
$22,311

estimate — estimated from the figures you entered — not measured against your traffic. The same newsvendor sizing runs against your real per-hour demand once traffic flows through the gateway, and that answer supersedes this one.

Sign Up for a Free Compute Audit →Get sizing verified against your real traffic · no credit card
select_compute_substrate →

One gateway, the full compute spectrum.

Start on a managed cloud API in sixty seconds. Add Kubernetes pods, bare-metal racks or spot fleets as your compute matures — same gateway, same keys, same dashboard.

typical user & tech stack
Azure OpenAI, AWS Bedrock, GCP Vertex, OpenAI/Anthropic APIs.
onboarding friction
< 60 seconds
Paste API key + base URL
what it unlocks
Retry budgeting only — hardware levers belong to the substrates that own hardware.
applicable gateway skills
Requestpii_nersemantic_cache
Workloadllm_router
Stackstorm_guards
▪ universal — enforced on any stack▪ stack — needs infrastructure you operate
integration

Enterprise-grade. Developer-fast.

see full onboarding doc →

15-Minute Setup. Zero Refactor.

Point your existing OpenAI or Anthropic client at the ACE proxy URL. Semantic caching, intent routing, context pruning, and agent guardrails activate automatically — your SDK calls, streaming, and tool-use stay identical.

  • · OpenAI · Anthropic · Azure OpenAI SDKs supported
  • · Streaming, tool-calling, and vision preserved
  • · Zero-config semantic caching via Redis / Qdrant
  • · Prometheus + OTLP telemetry out of the box
● app.py
# OpenAI Python SDK — swap base_url, and use your ACE key
from openai import OpenAI

client = OpenAI(
    api_key="ace_dev_...",                      # not your OpenAI key
    base_url="https://engine.acefleet.dev/v1",  # 👈 ACE intercepts
)

resp = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": heavy_prompt}],
)
Start Free — 60-Second Setup →Read Integration Docs →
ace-sidecar

Open Source.
Find out what your coding agents actually cost you.

ace-sidecar runs on your laptop, prices every turn your coding agent takes, and ranks the levers by what each is measurably worth on your own transcripts. Nothing leaves the machine — no account, no upload.

Spun out of our compute-efficiency work on ACE Fleet, and released under AGPL-3.0. The measurement is the same discipline; the scale is one developer instead of a fleet.

View the sourceread the launch post →
install
uv tool install ace-sidecar
ace up

python 3.12+ · agpl-3.0 · loopback only

what you get
  • Per-turn cost against published rates, with the rate card and the arithmetic printed beside it.
  • Levers ranked by headroom on your own transcripts — measured, not estimated.
  • Claude Code and Antigravity in one view.
  • Prometheus export at /metrics when you want the numbers elsewhere.

Two tools, not two tiers. The sidecar measures one developer's coding sessions; ACE Fleet cuts the bill in the request path for a company's production traffic.

core showcaseAI Scale Without the Compute Inflation.

Cut API costs without changing your application architecture.

Three deployment surfaces, one control plane. Pick your architecture — ACE composes the right optimizations underneath.

12 skills ·
target_persona50–70%

API & Agentic Applications

AI app builders, agent developers, and early-stage startups consuming third-party frontier APIs.

👔business_value·leadership
problem

Multi-step agents repeat prompt histories constantly, causing exponential token inflation and gross margin collapse.

ace_result

Cut monthly API spend 50%–70% without touching core application logic or degrading user experience.

🛠️technical_engine·CTO / VP ML

Multi-step agents replay prompt histories on every hop — token inflation shrinks SaaS gross margins from 80% down to 30%.

prompt_tokens
−58%
cache_hit_ratio
0.71
avg_latency
18ms
cost_per_1k_calls
$0.34
feature_01

Semantic Prompt Caching (sub-20ms)

mechanism

Vector embedding proxy (Redis / Qdrant) running cosine similarity match at ≥0.93 threshold on incoming prompt embeddings.

impact

Intercepts repetitive user queries and returns cached responses instantly at zero token cost.

feature_02

Dynamic Intent-Based LLM Routing

mechanism

Lightweight 0.5B classification router / regex intent parser intercepts outbound API calls before dispatch.

impact

Routes multi-step reasoning to GPT-4o / Claude 3.5 Sonnet; offloads JSON extraction, formatting, and classification to Llama 3.1 8B / GPT-4o-mini.

feature_03

Entropy-Based Prompt Pruning (LLMLingua-2)

mechanism

Contextual compression algorithm calculates token perplexity and strips low-information, redundant, or repetitive context prior to provider dispatch.

impact

2×–5× input token reduction while preserving generation quality.

feature_04

Agentic KV Summarizer & Loop Guardrails

mechanism

Sliding-window state compressor auto-condenses multi-turn agent trajectories (Reasoning → Action → Observation) into dense summaries.

impact

Stops exponential context-length scaling during long agent execution loops.

Try These Skills on Your Traffic →
observabilityThe receipt, not the promise.

See your exact dollar savings in real time.

ACE renders a real-time cost dashboard out of the box — cache hits, router downgrades, compaction ratios, and reclaimed spend, all fed by the same request stream that serves your traffic.

ace / dashboard / overview
tokens_saved
184.2M
+23.4%· last 7 days
reclaimed_spend
$27,481
+31.7%· vs baseline
cache_hit_ratio
62.8%
+4.1pp· rolling 24h
latency_delta
−38ms
p50 faster· vs upstream
reclaimed_spend · usd / hourlast 18h
peak
$1,842 / hr
avg
$1,140 / hr
since deploy
$184,720
request_disposition
  • semantic_cache47%
  • llm_router31%
  • prompt_compaction16%
  • passthrough6%
cost_attribution · by_provider
anthropicnet $9,240 · saved $7,180
openainet $6,120 · saved $5,240
google_vertexnet $3,480 · saved $2,310
deepseeknet $1,260 · saved $940
cost_attribution · by_model_endpoint
gpt-5.6-solnet $6,120 · saved $5,240
fable-5net $5,180 · saved $4,320
gemini-3.6-flashnet $2,940 · saved $1,980
deepseek-v4-pronet $1,040 · saved $780
router_intelligence · why_it_routed● live
  • optimized_for_cost58%
    simple prompt → cheaper model
    n = 12,481
  • optimized_for_latency27%
    primary lagging → faster peer
    n = 5,812
  • fallback / auto_retry15%
    upstream 5xx → sibling provider
    n = 3,204
traffic_switch · gpt-5.6-sol ⇢ fable-5last 24m
openai 503 detected · auto-switched · 0ms downtime
gpt-5.6-sol (primary)fable-5 (failover)
fail_open_guarantee● active

ACE is designed to be invisible. If any internal skill exceeds its latency budget, or the gateway itself encounters an error, ACE instantly degrades to a direct pass-through — your request is relayed straight to the downstream provider, pipeline bypassed, upstream body untouched.

x-ace-served-by: direct_passthroughtrigger: skill > 50ms · panic · storage unreachableproduction traffic: never drops
request_lifecycle · most_recenthover a row
requesttypemodel_routed_tocachecostsaved
Hey, could you please refactor the auth module across all of the 12 files to use the new session APIcodingfable-5 → deepseek-v4-proMISS$0.3040 $0.0101$0.2939
Given these three conflicting lab results, can you tell me which hypothesis best explains the data?reasoninggpt-5.6-sol → kimi-k3MISS$0.0700 $0.0366$0.0334
I need you to prove that the sum of the first n odd integers equals n squaredformal_mathgpt-5.6-sol → deepseek-r1MISS$0.0975 $0.0073$0.0902
Quick question — what is the difference between gross margin and contribution margin?knowledge_qasemantic_cacheHIT$0.0235 $0.0000$0.0235
Review this 340-page vendor agreement very carefully and flag every auto-renewal clauselong_docopus-4.8 → gemini-3.6-flashMISS$0.6800 $0.2040$0.4760
Navigate to the billing portal for me and download last month's invoicecomputer_usefable-5 → sonnet-5MISS$0.0840 $0.0252$0.0588
Please just classify this support ticket by urgency and product areaclassifygemini-3.6-flash → deepseek-v4-proMISS$0.0006 $0.0002$0.0004
Hi, I wanted to ask — how do I rotate an expired API key without downtime?support_faqsemantic_cacheHIT$0.0280 $0.0000$0.0280
Sorry to bother, but what is the parental leave policy for US-based employees?policy_ragsemantic_cacheHIT$0.0370 $0.0000$0.0370
hover a request to trace its lifecycle · struck text is what compaction removed, struck cost is what the caller's model would have billed
Start Free — See Your Dashboard →how it's built →snapshot · representative traffic
STAGED ROLLOUT ENGINE · SHADOW TRUST LADDER

Test risk-free in Shadow Mode before turning on live enforcement.

Every optimization skill ships with a 3-tier trust ladder: off, shadow (dry-run on real traffic, zero risk), and prod (live enforcement).

● ENGINEERING_RIGOR· ZERO_BEHAVIORAL_DRIFT

Run Verification in Shadow Mode Before Enforcing in Prod.

Mature platforms cannot risk model output mutation or unexpected latency spikes. ACE’s shadow mode evaluates skills against 100% of live production traffic, recording counterfactual accuracy metrics, latency impact, and cache candidate scores without altering a single byte of downstream output.

OFF MODE
1 SKILLS
Bypassed
SHADOW MODE
2 SKILLS
Shadow Evaluation
PROD MODE
3 SKILLS
Enforced Live
LIVE SCREENSHOT
ACE Live Skill Staged Rollout Toggles (off, shadow, prod)
✓ Verified: Granular per-skill toggle controls with zero-downtime microsecond fallback.
LIVE SKILL CONTROL MATRIX// Interactive staged rollout toggles
off shadow prod
Semantic Prompt Cachingcost saving

Vector similarity match at ≥0.93 threshold to replay responses at $0.

[MODE: PROD] Serves cached responses instantly at 0 token cost.
Dynamic Intent Routercost saving

Classifies query intent and routes to cheapest baseline model clearing quality floor.

[MODE: SHADOW] Shadow scoring live models; records counterfactual cost savings without altering routing.
Entropy Context Compactionperformance

LLMLingua-2 context pruning stripping low-information tokens before API dispatch.

[MODE: PROD] Compresses prompt context 2×–5× before provider dispatch.
Injection & Jailbreak Guardsecurity

Pre-execution scanner refusing prompt injection and jailbreak payloads.

[MODE: SHADOW] Scores attack vectors and logs security telemetry without blocking candidate prompts.
PII Redaction & NERsecurity

Dual-stage regex & BERT ONNX classifier scrubbing sensitive data before provider call.

[MODE: PROD] Redacts PII tokens live across requests and log traces.
Adaptive Circuit Breakerresilience

Automatic outlier ejection and failure shedding under upstream provider degradation.

[MODE: OFF] Skill is bypassed entirely for incoming requests.
6 core skills configured · real-time mode switchingStart Free — Configure Your Skills →
try it liveOne prompt, the full engine trace.

LIVE DEMO.

The left pane streams the engine's decision log; the right pane returns the model response with token and cost accounting.

ace / console / try
free trial · 10 of 10 sends left
Engine decision log
[req-a41f9c2d80b3] [in]: "implement fizzbuzz in typescript with tests"
[req-a41f9c2d80b3] [pii]: nothing to redact · ner_regex + ner_model scanned
[req-a41f9c2d80b3] [router]: query_category=code · complexity=medium → gpt-5-mini
[req-a41f9c2d80b3] [guard]: clean · pattern rules + classifier · classifier in shadow mode
[req-a41f9c2d80b3] [cache]: MISS · best-similarity=0.0000 (threshold 0.9200) · scope=session
[req-a41f9c2d80b3] [compact]: -0 tok · below the 40-token floor, sent unchanged
[req-a41f9c2d80b3] [serve]: gpt-5-mini @ azure-v1 · tok 10/42 (16 reasoning) · $0.000087
Responsegpt-5-mini
function fizzbuzz(n: number): string { if (n % 15 === 0) return 'FizzBuzz'; if (n % 3 === 0) return 'Fizz'; if (n % 5 === 0) return 'Buzz'; return String(n); }
tok in
10
tok out
42
cost
$0.000087
reasoning 16 tok
Router selectedgpt-5-mini
Unrealized savings$0.000870 / request
Cheaper than claude-opus-4-5$0.013914 / request
BillingBYOK_PASS_THROUGH

log lines and header values captured verbatim from the ACE production engine · model catalog 07312026

Ready to run this on your production traffic?
Start Free Trial →
watch the demo

See ACE route, cache, and save in real time.

A two-minute walkthrough of the production engine: one prompt, the full decision trace, and the cost accounting that shows up in your dashboard.

youtube · 0IOG8zqCd7w2 min walkthrough
try it first-hand

Stop Paying the Token Tax.
Reclaim Your Gross Margins Today.

Sign in to your developer console and generate your ACE_DEV_KEY on demand. No waitlist, no call, no credit card.

Generate Developer Key →Read the Docs
field notes

From the fleet. Engineering notes.

all posts →