01
Unused headroom exists.
The Philly study found substantial underuse even on GPUs already allocated to training jobs. The next question is whether that headroom can be shared safely and usefully. [1]
PHILLY USENIX ATC 2019Idle GPUs. Open models.Off-peak prices.
We’re building AI inference around spare GPU capacity.
The most useful work does not always need peak capacity. Give open models the spare compute they deserve.
THE RESEARCH CASE
Why build on spare GPUs?
Paying for computing power and putting it to useful work are different things. OFFPEAK starts with a practical question: can spare, authorized capacity run a suitable open model at a lower total cost?
MICROSOFT TRAINING CLUSTER · 2019
52.3%
Mean utilization of allocated GPUs
In a two-month Microsoft study of deep-learning training, allocated GPUs averaged 52.32% utilization. This shows an efficiency gap in one system. Finding capacity that can accept useful work is a separate problem. [1]
One training-cluster study. This is not a measure of all GPUs or of capacity available for resale.
02
Cheap capacity can disappear.
SpotServe studies language models on interruptible cloud machines. Its experiments show a trade-off: lower compute costs can come with more time waiting for a response. [2]
SPOTSERVE ASPLOS 202403
Getting a model ready takes work.
ServerlessLLM tackles the time spent loading model weights. Faster startup can make flexible capacity more practical, but it is only one part of delivering a useful answer. [3]
SERVERLESSLLM OSDI 2024OFFPEAK · PROPOSED PILOT
How that shapes OFFPEAK.
Useful output. Reliable requests. A sustainable cost. The pilot has to prove all three together.
Use capacity that is genuinely available. The plan requires permission to serve third-party work and an available request slot. A low GPU activity reading alone is not enough. Product plan ↗
Price the service, including the work that fails. Proposed prices respond to occupied serving slots, with a minimum that covers the measured cost of compute, operations, and failed work. Lower activity does not guarantee a discount. Product plan ↗
Measure before making a promise. Quality, successful requests, availability, and total cost must pass the pilot's checks. Showing a quote and a spending cap does not establish a price advantage.
Read the planned pilot and its limitsRead research notes 7 primary sources
These are results from the cited research systems. OFFPEAK has not reported equivalent production benchmarks.
Philly: allocated GPUs and useful work
Table 3, §3.2 reports 52.32% mean utilization of allocated GPUs in a two-month Microsoft DNN-training trace, rounded to 52.3% above. The activity metric is an upper bound on effective streaming-multiprocessor utilization. This historical study does not measure today's global GPU market, available inference slots, or authorized OFFPEAK supply.
SpotServe: cost and latency under interruptions
Figure 7 and §6.2 compare GPT-20B per-token costs and latency against on-demand-only serving. The authors report up to 54% lower cost, with latency increases of less than 18% on average and 90% at the 99th percentile. The evaluation replays extracted availability traces on AWS g4dn.12xlarge instances. These results depend on that workload, pricing, scheduling, and recovery design; they are not OFFPEAK prices or performance.
ServerlessLLM: making the model ready
Figure 6, §§7.1–7.2 report 3.6–8.2× faster checkpoint loading against tested PyTorch and Safetensors baselines. The microbenchmarks used FP16 OPT, LLaMA-2, and Falcon checkpoints, cold caches, RAID0 NVMe storage, and an eight-A5000 loading testbed. This measures loading weights, not token generation. The separate cluster workload was derived from an Azure serverless trace, not observed production LLM traffic.
Salus: sharing requires scheduling and memory control
Salus combines fast job switching and GPU memory sharing in a TensorFlow execution service. Its workload-specific experiments support the need to manage sharing deliberately. Its gains are not a forecast for modern language-model fleets, and this is not a claim that OFFPEAK has implemented Salus or training/inference co-tenancy.
Clockwork: predictable service needs admission control
Clockwork schedules DNN inference against latency objectives using predictable execution profiles. The useful lesson is to admit only the work a system can serve reliably. Traditional DNN execution differs from variable-length language-model generation, so its results do not establish OFFPEAK response times or reliability.
What a GPU-utilization reading actually means
NVIDIA defines GPU utilization as the portion of a sample interval during which at least one kernel executes. Our interpretation: that activity percentage alone cannot establish free model memory, an available serving slot, permission to serve, or profitable capacity. The proposed OFFPEAK load measure therefore uses occupied serving slots, with GPU activity as additional telemetry.
Spare cloud capacity can be reclaimed
AWS can interrupt Spot Instances when it needs that capacity back; availability varies. AWS recommends fault-tolerant applications. The implication for this research case is to budget for interruption and recovery. These are AWS product semantics, not evidence that OFFPEAK uses AWS or inherits its behavior.
What the OFFPEAK plan still needs to prove
The supplied v1.0 design proposes explicitly authorized supply, a validated model revision, fresh capacity information, and a real slot reserved at request acceptance. A quote alone reserves neither funds nor capacity. Its load measure is the 60-second smoothed fraction of occupied serving slots; GPU activity is supplemental. A measured cost floor includes supplier charges, serving and payment overhead, failed work, and margin. Failed pilot requests without a completed, retrievable result are unbilled to the buyer, but their costs still belong in the provider's economics. Expansion depends on measured quality, reliable request handling, and positive contribution on real paid work. Read the proposed pilot and its boundaries ↗
Useful compute, in every form.
Get started with a workflow.
See all
Features for useful work.
Capacity insights
Prompt to response
Wallet authorization
Quote to request
Receipt export
OFFPEAK for developers
An explicit flow from the first quote to the final receipt.
Built for work like yours.
Builders
Product classification
Classify a queue of product descriptions with an open model that meets your quality checks. Inspect capacity, cap each request, and retain the usage record.
Developers
Structured extraction
Turn unstructured text into fields your application can use. Pin the model revision and keep the prompt, quote, and output limit tied to one request.
Operations
Support routing
Sort incoming support messages by intent before they reach the right queue. Flexible text work can adapt to available capacity, with no promised discount or latency.
Researchers
Document summaries
Create short summaries for a review workflow. An explicit maximum charge makes it easier to try a bounded text task, then inspect what the model returned.
Product teams
Content tagging
Add useful labels to a content library. Start with a small example, check model quality, and keep each completed request easy to reconcile.
Finance teams
Billing reconciliation
Keep signed usage records alongside your request history. Inspect the charge, see the unused reserve released, and check receipt inclusion once a batch is published. These are planned product workflows, not customer testimonials.
Questions
OFFPEAK is being built for open-model text inference on spare GPU capacity: extraction, classification, summarization, and other repeatable work. This site is a product preview.
Proposed prices respond to available serving capacity, subject to a measured cost floor. More spare capacity can lower a quote; discounts are not guaranteed. Inspect both rates and the maximum charge before authorizing.
The website is a free preview, and any example values on it are illustrative. Paid inference and final commercial terms are still in development. The initial design uses usage-based prepaid balances, not a subscription plan.
The planned pilot uses your wallet to authorize each request, so there is no OFFPEAK-issued inference API key. Your wallet key remains a credential. Delegated budgets and direct x402 payments are later work.
Each quote fixes the prompt, model, rates, and output ceiling. You authorize a maximum charge. A successful request pays for measured usage, and the unused reserve is released. Failed pilot requests are unbilled.
Initial scope is one approved open-weight model revision, one region, and non-streaming text requests. The model, licensing, quality, and serving rates must be validated before paid access. Streaming, tools, and broader supply are future scope.
The planned pilot returns available, unreserved paid balances to the same wallet, token, and network. OFFPEAK controls the prepaid treasury and processes withdrawals. Pending transfers and reserved funds remain separate.
It checks that OFFPEAK signed a particular billing record and included it in a committed batch. It does not independently prove that a GPU performed the work, that metering was truthful, or that the model answer is correct.
The workflow gallery offers eighteen illustrative starting points for classification, extraction, summarization, and review. Read the docs to see how quotes, caps and usage records are designed to work; no wallet, funds, or live model is connected.