Tensormesh Platform

Smart AI-native data.
At inference speed.

Tensormesh Platform cuts AI inference compute costs by intelligently managing your KV cache across GPU memory, CPU memory, and external storage. It adapts to your infrastructure and workload, whether that's prefix or non-prefix caching, prefill/decode disaggregation, or cross-node cache sharing, to deliver best-in-class cache hit rates and time to first token. It's just as easy to run. Helm chart–based installation gets you up fast, and a web UI plus CLI cover both initial setup and ongoing operations.

As Enterprise deploy AI workload in Self Hosted Infrastructure, the Tensormesh Platform addresses the following challenges:

AI Data Sovereignty

Data Retention in Regulated Environment

Cost Reduction

Performance Optimization

Sustainability

Capability Matrix

What's inside Tensormesh Platform

A modular platform. A production-ready release for KV cache management and prompt caching, a shared platform core for operations and multi-tenancy, and infrastructure compatibility across every stack we've seen.

Interfaces

Operator UI
Operator CLI

Data & Cache Mgmt

KV Cache Control Mgmt
Prompt Prefix Caching & External Storage Offloading
Non-Prefix Caching for Full Models (Dense)
P/D Disaggregation
Sharing KV Cache across node (P2P)

Tensormesh Shared Platform Core

K8S Operator & Helm

Security Services

Observability Stack

Multi Tenancy

Infrastructure Compatibility

Any K8S

Any Accelerator

Any Storage

Any Inference Engine

The release delivers KV cache control, prefix + non-prefix prompt caching, prefill/decode disaggregation, and P2P KV cache sharing — accessible through the operator UI and CLI. Underneath, the shared platform core provides the K8S operator, security, observability, and multi-tenancy. Everything runs on any Kubernetes cluster, any accelerator, any storage, and any inference engine.

Why Tensormesh Platform

Best-in-class KV cache.
Simple to run.

Tensormesh Platform is designed for the teams putting AI inference into production — where cache miss rate, time to first token, and operational cost decide whether a workload ships or stalls.

01 · Multi-tier KV cache

One cache, three tiers.

Manage KV state intelligently across GPU memory, CPU memory, and external storage — with the platform deciding what lives where, based on your workload and hit-rate targets.

02 · Hit-rate first

Rapidly reduce miss rate. Cut recompute.

Deliver best-in-class cache hit rates and time to first token by combining prefix caching, non-prefix caching for dense models, and P2P KV cache sharing across GPU nodes.

Advanced configurations

Prefill/decode disaggregation. Built in.

Advanced inference patterns — P/D disaggregation, KV cache sharing across nodes — are first-class capabilities, not one-off engineering projects layered on top of a stock cache.

Day 1 and Day 2

Helm to deploy. UI + CLI to operate.

Helm chart–based installation gets you live quickly. A web UI paired with a command-line interface gives operators and platform teams the tools they need for both Day 1 setup and Day 2 operations.

Get in touch

Two ways to get started

Talk to our team about deploying Tensormesh Platform on your infrastructure — or run your workload in a serverless environment first to validate the performance benefits and cost reduction.

Talk to us

Deploy on your infrastructure

Talk to our team about deploying Tensormesh Platform alongside your existing inference engines, storage, and observability stack.

Talk to an expert

Try it serverless

Run your workload in a serverless environment

Validate performance benefits and cost reduction on your own prompts — no infrastructure to set up. Launch the Tensormesh Platform Demonstrator and see the results in minutes.

Launch Demonstrator

Controls Enterprise platform controls

Fits the tools you already run.

The engine speaks Prometheus and OTLP, tiers cache across memory and pluggable backends, and isolates tenants with a per-request key so shared infrastructure does not mean shared cache state.

Ports & endpoints

MP connector

:5555

Cache-admin HTTP

:8080

Engine /metrics

:9090

OTLP → Collector

:4317

Prometheus exporter

:8889

Metrics that matter first

Hit rate

lookup_hit / requested

L1 activity

l1_read · write · evicted

L2 activity

l2_store · l2_load

Volume

num_chunks_loaded

Reuse gap

real_reuse_gap

Cache tiers & tenancy

L1

CPU memory · sizeGB

L2

filesystem · RESP · S3

Eviction

LRU · IsolatedLRU

Isolation

cache_salt

Quotas

per-tenant budgets

All metric names are the upstream lmcache_mp_* series, scraped through an OpenTelemetry Collector into your existing Prometheus and Tempo.

What Tensormesh Operator solves

Designed to operate LLM inference at scale.

On-prem inference teams need more than raw GPU capacity. Tensormesh Operator turns LMCache into reusable infrastructure across your fleet, so repeated context becomes capacity instead of cost.

Deployment

01

Tailored to your on-prem environment

Deploy LMCache inside your own Kubernetes environment, aligned to your models, GPU topology, and storage systems. Declarative configuration handles DaemonSets, services, and resource sizing automatically.

Observability

02

Visibility into where reuse helps and where it misses

Visualize how reusable context moves from GPU memory to CPU offload to external storage. The engine reports these metrics through your existing observability stack, complete with a ServiceMonitor and per-node health checks set up for you.

Isolation

03

Isolation for multi-tenant clusters

A single per-request cache_salt keeps identical prompts separate across tenants. IsolatedLRU and per-tenant quotas stop one noisy workload from evicting everyone else's entries.

Control plane

04

Advanced controls for reusable inference state

Clear, compress, move, pin, blend, and look up reusable cache entries across your fleet with controller APIs and CRD-validated policies, plus a runtime cache-admin HTTP API for live quota management on :8080

Storage

05

Pluggable storage for persistent reuse

Keep the hot set in L1 CPU memory, then spill to pluggable L2 backends such as filesystem, Redis, or object storage. Store and prefetch policies stay declarative, so warm reuse survives engine restarts.

Day-2 ops

06

Day-2 operations your platform team can own

Install through Helm, Argo CD, or Kustomize against a pinned OCI chart validated by a bundled values.schema.json. Leader-elected HA, OpenShift SCC support, and finalizer-aware cleanup fit the way production platform teams already operate.

The management layer for AI inference data

Smart AI-native data.
At inference speed.

What's inside Tensormesh Platform

Best-in-class KV cache.
Simple to run.

One cache, three tiers.

Rapidly reduce miss rate. Cut recompute.

Prefill/decode disaggregation. Built in.

Helm to deploy. UI + CLI to operate.

Runs on your stack.

Two ways to get started

Deploy on your infrastructure

Run your workload in a serverless environment

Fits the tools you already run.

Ports & endpoints

Metrics that matter first

Cache tiers & tenancy

Designed to operate LLM inference at scale.

Tailored to your on-prem environment

Visibility into where reuse helps and where it misses

Isolation for multi-tenant clusters

Advanced controls for reusable inference state

Pluggable storage for persistent reuse

Day-2 operations your platform team can own

The management layer for AI inference data

Smart AI-native data. At inference speed.

What's inside Tensormesh Platform

Best-in-class KV cache. Simple to run.

One cache, three tiers.

Rapidly reduce miss rate. Cut recompute.

Prefill/decode disaggregation. Built in.

Helm to deploy. UI + CLI to operate.

Runs on your stack.

Two ways to get started

Deploy on your infrastructure

Run your workload in a serverless environment

Fits the tools you already run.

Ports & endpoints

Metrics that matter first

Cache tiers & tenancy

Designed to operate LLM inference at scale.

Tailored to your on-prem environment

Visibility into where reuse helps and where it misses

Isolation for multi-tenant clusters

Advanced controls for reusable inference state

Pluggable storage for persistent reuse

Day-2 operations your platform team can own

Smart AI-native data.
At inference speed.

Best-in-class KV cache.
Simple to run.