Skip to main content

Fireworks AI

https://fireworks.ai/

info

We support ALL Fireworks AI models, just set fireworks_ai/ as a prefix when sending completion requests

API Key

# env variable
os.environ['FIREWORKS_AI_API_KEY']

Sample Usage

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""
response = completion(
model="fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct",
messages=[
{"role": "user", "content": "hello from litellm"}
],
)
print(response)

Sample Usage - Streaming

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""
response = completion(
model="fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct",
messages=[
{"role": "user", "content": "hello from litellm"}
],
stream=True
)

for chunk in response:
print(chunk)

Usage with LiteLLM Proxy

1. Set Fireworks AI Models on config.yaml

model_list:
- model_name: fireworks-llama-v3-70b-instruct
litellm_params:
model: fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct
api_key: "os.environ/FIREWORKS_AI_API_KEY"

2. Start Proxy

litellm --config config.yaml

3. Test it

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "fireworks-llama-v3-70b-instruct",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'

Supported Models - ALL Fireworks AI Models Supported!

info

We support ALL Fireworks AI models, just set fireworks_ai/ as a prefix when sending completion requests

Model NameFunction Call
llama-v3p2-1b-instructcompletion(model="fireworks_ai/llama-v3p2-1b-instruct", messages)
llama-v3p2-3b-instructcompletion(model="fireworks_ai/llama-v3p2-3b-instruct", messages)
llama-v3p2-11b-vision-instructcompletion(model="fireworks_ai/llama-v3p2-11b-vision-instruct", messages)
llama-v3p2-90b-vision-instructcompletion(model="fireworks_ai/llama-v3p2-90b-vision-instruct", messages)
mixtral-8x7b-instructcompletion(model="fireworks_ai/mixtral-8x7b-instruct", messages)
firefunction-v1completion(model="fireworks_ai/firefunction-v1", messages)
llama-v2-70b-chatcompletion(model="fireworks_ai/llama-v2-70b-chat", messages)

Supported Embedding Models

info

We support ALL Fireworks AI models, just set fireworks_ai/ as a prefix when sending embedding requests

Model NameFunction Call
fireworks_ai/nomic-ai/nomic-embed-text-v1.5response = litellm.embedding(model="fireworks_ai/nomic-ai/nomic-embed-text-v1.5", input=input_text)
fireworks_ai/nomic-ai/nomic-embed-text-v1response = litellm.embedding(model="fireworks_ai/nomic-ai/nomic-embed-text-v1", input=input_text)
fireworks_ai/WhereIsAI/UAE-Large-V1response = litellm.embedding(model="fireworks_ai/WhereIsAI/UAE-Large-V1", input=input_text)
fireworks_ai/thenlper/gte-largeresponse = litellm.embedding(model="fireworks_ai/thenlper/gte-large", input=input_text)
fireworks_ai/thenlper/gte-baseresponse = litellm.embedding(model="fireworks_ai/thenlper/gte-base", input=input_text)