docs(langfuse.md): add doc on pass through

This commit is contained in:
Krrish Dholakia
2024-08-19 21:28:48 -07:00
parent 9e3f8949c4
commit 91aac0533b
3 changed files with 136 additions and 3 deletions
@@ -0,0 +1,132 @@
# Langfuse Endpoints (Pass-Through)
Pass-through endpoints for Langfuse - call langfuse endpoints with LiteLLM Virtual Key.
Just replace `https://us.cloud.langfuse.com` with `LITELLM_PROXY_BASE_URL/langfuse` 🚀
#### **Example Usage**
```python
from langfuse import Langfuse
langfuse = Langfuse(
host="http://localhost:4000/langfuse", # your litellm proxy endpoint
public_key="anything", # no key required since this is a pass through
secret_key="LITELLM_VIRTUAL_KEY", # no key required since this is a pass through
)
print("sending langfuse trace request")
trace = langfuse.trace(name="test-trace-litellm-proxy-passthrough")
print("flushing langfuse request")
langfuse.flush()
print("flushed langfuse request")
```
Supports **ALL** Langfuse Endpoints.
[**See All Langfuse Endpoints**](https://api.reference.langfuse.com/)
## Quick Start
Let's log a trace to Langfuse.
1. Add Langfuse Public/Private keys to environment
```bash
export LANGFUSE_PUBLIC_KEY=""
export LANGFUSE_PRIVATE_KEY=""
```
2. Start LiteLLM Proxy
```bash
litellm
# RUNNING on http://0.0.0.0:4000
```
3. Test it!
Let's log a trace to Langfuse!
```python
from langfuse import Langfuse
langfuse = Langfuse(
host="http://localhost:4000/langfuse", # your litellm proxy endpoint
public_key="anything", # no key required since this is a pass through
secret_key="anything", # no key required since this is a pass through
)
print("sending langfuse trace request")
trace = langfuse.trace(name="test-trace-litellm-proxy-passthrough")
print("flushing langfuse request")
langfuse.flush()
print("flushed langfuse request")
```
## Advanced - Use with Virtual Keys
Pre-requisites
- [Setup proxy with DB](../proxy/virtual_keys.md#setup)
Use this, to avoid giving developers the raw Google AI Studio key, but still letting them use Google AI Studio endpoints.
### Usage
1. Setup environment
```bash
export DATABASE_URL=""
export LITELLM_MASTER_KEY=""
export LANGFUSE_PUBLIC_KEY=""
export LANGFUSE_PRIVATE_KEY=""
```
```bash
litellm
# RUNNING on http://0.0.0.0:4000
```
2. Generate virtual key
```bash
curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{}'
```
Expected Response
```bash
{
...
"key": "sk-1234ewknldferwedojwojw"
}
```
3. Test it!
```python
from langfuse import Langfuse
langfuse = Langfuse(
host="http://localhost:4000/langfuse", # your litellm proxy endpoint
public_key="anything", # no key required since this is a pass through
secret_key="sk-1234ewknldferwedojwojw", # no key required since this is a pass through
)
print("sending langfuse trace request")
trace = langfuse.trace(name="test-trace-litellm-proxy-passthrough")
print("flushing langfuse request")
langfuse.flush()
print("flushed langfuse request")
```
## [Advanced - Log to separate langfuse projects (by key/team)](../proxy/team_logging.md)
+2 -2
View File
@@ -207,7 +207,7 @@ curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Content-Type: application/json' \
-d '{
"metadata": {
"logging": {
"logging": [{
"callback_name": "langfuse", # 'otel', 'langfuse', 'lunary'
"callback_type": "success" # set, if required by integration - future improvement, have logging tools work for success + failure by default
"callback_vars": {
@@ -215,7 +215,7 @@ curl -X POST 'http://0.0.0.0:4000/key/generate' \
"langfuse_secret_key": "os.environ/LANGFUSE_SECRET_KEY", # [RECOMMENDED] reference key in proxy environment
"langfuse_host": "https://cloud.langfuse.com"
}
}
}]
}
}'
+2 -1
View File
@@ -195,7 +195,8 @@ const sidebars = {
"pass_through/vertex_ai",
"pass_through/google_ai_studio",
"pass_through/cohere",
"pass_through/bedrock"
"pass_through/bedrock",
"pass_through/langfuse"
],
},
"scheduler",