litellm/cookbook/LiteLLM_PromptLayer.ipynb

Using LiteLLM with PromptLayer

Promptlayer allows you to track requests, responses and prompts

LiteLLM allows you to use any litellm supported model and send data to promptlayer

Getting started docs: https://docs.litellm.ai/docs/observability/promptlayer_integration

In [ ]:

!pip install litellm

In [18]:

import litellm
from litellm import completion
import os
os.environ['OPENAI_API_KEY'] = ""
os.environ['REPLICATE_API_TOKEN'] = ""
os.environ['PROMPTLAYER_API_KEY'] = "pl_4ea2bb00a4dca1b8a70cebf2e9e11564"

# Set Promptlayer as a success callback
litellm.success_callback =['promptlayer']

Call OpenAI with LiteLLM x PromptLayer

In [ ]:

result = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "gm this is ishaan"}])
print(result)

Call Replicate-CodeLlama with LiteLLM x PromptLayer

In [ ]:

model="replicate/codellama-13b:1c914d844307b0588599b8393480a3ba917b660c7e9dfae681542b5325f228db"

result = completion(model=model, messages=[{"role": "user", "content": "gm this is ishaan"}])
print(result)

View Logs on PromptLayer