Home/Models
Models/Meta

Llama-3.1-70B-Instruct

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 70B instruct-tuned version is optimized for high quality dialogue usecases.

It has demonstrated strong performance compared to leading closed-source models in human evaluations.

To read more about the model release, click here. Usage of this model is subject to Meta's Acceptable Use Policy.

meta/llama-3.1-70b-instruct
Organization
Meta
Family
llama
Providers
5
Context
128,000
Output limit
4,096
Knowledge
2023-12
Release
2024-07-23
Updated
2024-07-23
Weights
Open
Input
text
Output
text
Capabilities
Tools, Temperature

Providers

Compare every cataloged offer for this model, including its provider, route, token limits, price, and capabilities.

7 offers · USD per million tokens
ProviderModel IDContextOutputInput / output · 1MReasoningToolsStructuredDetails
Amazon Bedrock
us.meta.llama3-1-70b-instruct-v1:0
128,0004,096$0.72 / $0.72NoYes—
View
Amazon Bedrock
meta.llama3-1-70b-instruct-v1:0
128,0004,096$0.72 / $0.72NoYes—
View
CoreWeave
meta-llama/Llama-3.1-70B-Instruct
131,072131,072$0.80 / $0.80NoYesYes
View
Kilo Gateway
meta-llama/llama-3.1-70b-instruct
131,07216,384$0.40 / $0.40NoYesYes
View
Merge Gateway
meta/llama-3.1-70b-instruct
128,0002,048$0.99 / $0.99NoYesNo
View
Amazon Bedrockvia OpenRouter
meta-llama/llama-3.1-70b-instruct
unknown		131,0728,192$0.72 / $0.72NoNoNo
View
DeepInfravia OpenRouter
meta-llama/llama-3.1-70b-instruct
fp8		131,07216,384$0.40 / $0.40NoYesYes
View

Prices and limits apply to each offer. A dash means the value is not provided. OpenRouter offers show the hosting provider and routing channel separately.

Pricing

Compare token rates and cache charges across available routes and hosting configurations.

USD per million tokens
Input from$0.40Per 1M input tokens
Output from$0.40Per 1M output tokens
Available offers7Across 5 providers

Lowest listed input and output prices across the offers above; they may belong to different providers. These are base rates, before conditional pricing, discounts or additional fees.

Endpoint price comparisonLatest observed snapshot · 2 lowest-priced configurations
InputOutput
DeepInfradeepinfra/turbo
$0.40$0.40
Amazon Bedrockamazon-bedrock
$0.72$0.72
meta-llama/llama-3.1-70b-instructObserved
Input
$0.40
Output
$0.40
Hosting prices through OpenRouter All endpoints and cache rates
EndpointInput / 1MOutput / 1MCache read / 1MCache write / 1M
Amazon Bedrockamazon-bedrock$0.72$0.72——
DeepInfradeepinfra/turbo$0.40$0.40——

Each row is a hosting configuration. Open an offer in Providers for its full pricing conditions and additional fees. Endpoint observations may differ from the routing catalog's base price.

Performance

Review reported latency and output-throughput percentiles from the latest observed rolling window.

Latest 30-minute observation

Time to first token · milliseconds

Latency distributionRolling 30-minute snapshot · 2 endpoints
P50P75P90P99
Amazon Bedrockamazon-bedrock
361–1,961.48 ms
DeepInfradeepinfra/turbo
338–2,124.84 ms
All latency percentiles 2 endpoints
EndpointP50P75P90P99
Amazon Bedrockamazon-bedrock361 ms431 ms750.4 ms1,961.48 ms
DeepInfradeepinfra/turbo338 ms471 ms899 ms2,124.84 ms

Output throughput · tokens per second

Throughput distributionRolling 30-minute snapshot · 2 endpoints
P50P75P90P99
Amazon Bedrockamazon-bedrock
68.5–136.34 t/s
DeepInfradeepinfra/turbo
38–47 t/s
All throughput percentiles 2 endpoints
EndpointP50P75P90P99
Amazon Bedrockamazon-bedrock68.5 t/s74 t/s89.2 t/s136.34 t/s
DeepInfradeepinfra/turbo38 t/s42 t/s44 t/s47 t/s

Percentiles describe the observed request distribution. These are snapshots of a rolling window, not a historical time series.

Uptime

Compare successful request rates across the latest 5-minute, 30-minute, and 24-hour observations.

Successful requests by window
Availability by endpointCurrent overlapping windows · 2 endpoints
Endpoint5 minutes30 minutes24 hours
Amazon Bedrockamazon-bedrock
DeepInfradeepinfra/turbo

Reported by OpenRouter at the time of collection; rate-limited requests are excluded. A dash means no measurement was reported. This is not a live availability check.

Technical details & API

Explore specifications, supported parameters, reasoning controls, and API identifiers for each OpenRouter route.

Current route specifications
meta-llama/llama-3.1-70b-instructObserved
Context
131,072
Maximum output
16,384
Tokenizer
Llama3
Input
text
Output
text
Moderation
No
Instruction format
llama3
Knowledge cutoff
2023-12-31
Added to OpenRouter
2024-07-23
Base URLhttps://openrouter.ai/api/v1
Model IDmeta-llama/llama-3.1-70b-instruct
Version IDmeta-llama/llama-3.1-70b-instruct
Weights IDmeta-llama/Meta-Llama-3.1-70B-Instruct

Supported parameters

  • frequency_penalty
  • logit_bias
  • max_tokens
  • min_p
  • presence_penalty
  • repetition_penalty
  • response_format
  • seed
  • stop
  • structured_outputs
  • temperature
  • tool_choice
  • tools
  • top_k
  • top_p

Frequently asked questions

Answers based on the model metadata and serving offers currently in the catalog.

How much does Llama-3.1-70B-Instruct cost?

Listed rates start at $0.40 per million input tokens and $0.40 per million output tokens. Prices vary by provider and configuration; see Pricing for details.

Which providers offer Llama-3.1-70B-Instruct?

There are 7 cataloged offers across 5 providers. Compare model IDs, prices and limits in Providers.

What is the context window?

The cataloged model context is 128,000 tokens. Each serving endpoint may apply a different limit.

Does it support tools and structured output?

Tool calling: Yes. Structured output: Not reported. Support can vary by endpoint.

Resources

Hugging Face ↗
Catalog history · 2 recorded revisions

Metadata observations since this model was first cataloged. These are not model release versions.

  • · 8c333a108161
  • · 68ce88e988a9