tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-06-17 22:48:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
28,119 Commits 2,072 Branches 1,425 Tags
a4341ccf833c19f716e00a8fedc66fb257d5bcd6
Commit Graph

372 Commits

Author SHA1 Message Date
Cesar Garcia 45e921d533 fix: Update broken documentation links in README (#17002) 
- Update Hosted Proxy links to point to enterprise docs
- Remove "(Preview)" label from Hosted Proxy
- Fix "Supported LLM Providers" link to point to docs instead of GitHub anchor
 2025-11-23 21:54:38 -08:00
Cesar Garcia 5b0729034c docs: cleanup README and improve agent guides (#17003) 
* docs: cleanup README and improve AI agent guides

- Remove obsolete version warnings (openai>=1.0.0, pydantic>=2.0.0)
- Add note about Responses API in README
- Add GitHub templates section to CLAUDE.md, GEMINI.md, and AGENTS.md
- Remove temporary test file test_pydantic_fields.py

* update files

* update Gemini file
 2025-11-23 21:53:53 -08:00
Krrish Dholakia 6931d3013a docs(readme.md): document 8ms p95 latency 2025-11-11 19:05:50 -08:00
Cesar Garcia 656dce92d0 docs: fix streaming example in README (#16461) 
* docs: add messages variable definition in streaming example

- Add missing messages variable in streaming code example
- Makes the example complete and runnable without modifications

* docs: capitalize LiteLLM in streaming section

* docs: add gpt-4o comment
 2025-11-10 17:16:17 -08:00
Ishaan Jaffer 4d88f21393 docs fix 2025-10-30 18:25:51 -07:00
Ishaan Jaff fd073ee682 Litellm docs readme fixes (#16107) 
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config
 2025-10-30 17:05:32 -07:00
Ishaan Jaffer 40c663439a fix readme.md 2025-10-30 15:54:50 -07:00
TensorNull 074b9954a0 docs(readme): Add CometAPI to supported providers table 2025-10-16 14:18:55 +08:00
Kowyo 858f557bce docs: use docker compose instead of docker-compose 2025-09-29 11:59:53 +00:00
Franklin e437273ad5 doc: make the README document clearer (#14860) 
* Update README.md

* Update README.md
 2025-09-24 09:57:21 -07:00
Franklin b39fd688a8 doc: navigate to the correct location (#14722) 2025-09-19 15:25:18 -07:00
Tim Elfrink 9521414efa Resolve merge conflict by including both CompactifAI and OVHCloud providers 
- Keep CompactifAI provider detection logic
- Include new OVHCloud provider from main branch
- Both providers now work correctly with model prefix detection
 2025-09-14 23:03:18 +02:00
Tim Elfrink 1987556a50 Add CompactifAI provider documentation and config 
- Create comprehensive provider documentation with usage examples
- Cover basic completion, streaming, async, and function calling
- Document AWS Marketplace subscription and API key setup process
- Include proxy configuration and advanced parameter examples
- Add error handling examples and model information
- Update website sidebar to include CompactifAI in provider list
- Update README.md with CompactifAI provider reference
 2025-09-13 08:42:56 +02:00
Elias TOURNEUX 54f49b430d fix: edit readme to add OVHCloud 2025-09-12 14:29:33 +02:00
Krish Dholakia 2668112a18 Update README.md 2025-09-10 08:48:37 -07:00
Krish Dholakia c43706061f Merge branch 'main' into heroku-llms 2025-08-27 22:21:36 -07:00
Krrish Dholakia 5631d97964 docs(readme.md): cleanup 2025-08-14 17:16:05 -07:00
Krrish Dholakia 41f7901cfa docs(readme.md): add note, saying poetry is required 2025-08-14 17:15:29 -07:00
Krrish Dholakia 817b8408fa docs(readme.md): fix readme 2025-08-14 17:13:04 -07:00
Timothy Lowrimore 95d9e30448 Merge branch 'main' into heroku-llms 2025-08-11 09:45:52 -06:00
Sannan Nasir 0e53b1feab Add digitalocean provider (#12169) 
* Add digitalocean provider

* Add digitalocean provider

* Revert "Add digitalocean provider"

This reverts commit 96dda40f45b3d12ea03e861d060ec81460b7759e.

* changes

* fixes

* Update transformation

* refactoring

* rename provider to Gradient AI

* fixes

* Incorporte review comments

* revert changes

* fix typo

* revert change

* incorporated review comments

* Revert "Incorporte review comments"

This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597.

* changes

* Revert "Revert "Incorporte review comments"

This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597."

This reverts commit 68c8a198ee0d6441c3a52f6c6a49c9c95a4cb0a8.

* changes

* fixes

* Update provider_specific_fields.tsx
 2025-08-09 16:26:33 -07:00
Timothy Lowrimore dd109e7c37 adds heroku to supported providers table 2025-07-25 16:28:56 -06:00
Takashi Ishida 773717d57c improve readme: replace claude-3-sonnet because it will be retired soon (#12239) 
* improve readme: replace claude-3-sonnet-20240229 because it will be retired soon

* update the readme example to claude sonnet 4
 2025-07-03 21:50:39 -07:00
Ishaan Jaff 6b464248c7 chore - add slack support link 2025-06-30 10:23:53 -07:00
Yan 8f62b1068a Update README.md (#11586) 
It will not work if followed exactly like in readme, because .env will contain just one parameter. Fixing that
 2025-06-10 09:32:11 -07:00
Cole McIntosh f99e450d38 Update Makefile and add CONTRIBUTING.md to guide contributors on best practices and submission process (#11485) 
- Introduced a comprehensive contributing guide outlining the checklist for PR submissions, including signing the Contributor License Agreement, adding tests, and ensuring code quality.
- Updated README.md to link to the new CONTRIBUTING.md and provide a quick start for contributors.
- Enhanced Makefile with additional commands for installation and testing to streamline the development workflow.
 2025-06-06 14:19:28 -07:00
Akim Tsvigun acaa80294c Integration with Nebius AI Studio added (#11143) 
* integration with Nebius AI Studio added

* Merged with main

* Reviewer's comments resolved

* spelling error fixed

* accidental change reverted
 2025-05-27 11:05:22 -07:00
DarinVerheijke 54a0766ff9 Add provider Featherless AI (access to 4200+ models) (#10596) 
* add featherless provider

* change docs

* clean lint

* fix provider logic

* Add fixes per review

* update parameters

* update error msg

* remove featherless

* add tests

* add mock test e2e

* update to openai mock test
 2025-05-17 19:38:46 -07:00
Young Han 71d1fabc21 [README] Add Llama API in supported providers table (#10802) 
* docs: add llama-api in supported table

* docs: revert formatting
 2025-05-13 13:49:21 -07:00
Krish Dholakia d37cc63250 Add new model provider Novita AI (#7582) (#9527) 
* Add new model provider Novita AI (#7582)

* feat: add new model provider Novita AI

* feat: use deepseek r1 model for examples in Novita AI docs

* fix: fix tests

* fix: fix tests for novita

* fix: fix novita transformation

* ci: fix ci yaml

* fix: fix novita transformation and test (#10056)

---------

Co-authored-by: Jason <ggbbddjm@gmail.com>
 2025-05-12 21:49:30 -07:00
Krish Dholakia d60e485c03 Update README.md (#9616) 2025-03-28 09:28:09 -07:00
Ishaan Jaff ce35240273 Pre-Submission checklist 2025-03-10 16:21:53 -07:00
Ishaan Jaff 856ed0e1fd docs contributing 2025-03-10 15:45:22 -07:00
Ishaan Jaff c1a3cb82a9 docs on contributing 2025-03-10 14:49:27 -07:00
Ishaan Jaff ba5bdce50a Update README.md 2025-03-01 07:40:55 -08:00
Krish Dholakia 251467a525 add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support (#8684) 
* build(model_prices_and_context_window.json): mark bedrock llama as supporting vision based on docs

* Add price for Cerebras llama3.3-70b (#8676)

* docs(readme.md): fix contributing docs

point people to new mock directory testing structure s/o @vibhavbhat

* build: update contributing readme

* docs(readme.md): improve docs

* docs(readme.md): cleanup readme on tests/

* docs(README.md): cleanup doc

* feat(infinity/): support returning documents when return_documents=True

* test(test_rerank.py): add e2e testing for cohere rerank

* fix: fix linting errors

* fix(together_ai/): fix together ai transformation

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* test: mark cohere as flaky

* build: fix model supports check

* test: fix test

* test: mark flaky test

* fix: fix test

* test: fix test

---------

Co-authored-by: Yury Koleda <fut.wrk@gmail.com>
 2025-02-20 21:23:54 -08:00
Steve Cosman 37cfa0f70a Fix typo in main readme (#8574) 2025-02-17 08:19:40 -08:00
Cole McIntosh f352ed5b70 fix: update README.md API key and model example typos (#8590) 2025-02-17 07:34:15 -08:00
Sebastian Sosa 64ccf4cd6e expose port & required env vars & instructions for running in dev (#8404) 2025-02-08 16:29:44 -08:00
waterstark fbe3c58372 Added a guide for users who want to use LiteLLM with AI/ML API. (#7058) 
* Added a guide for users who want to use LiteLLM with AI/ML.

* Minor changes

* Minor changes

* Fix sidebars.js

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2025-02-05 06:20:35 -08:00
Yuki Watanabe 3f053fc99c Update MLflow calllback and documentation (#7809) 
* Update MLlfow tracer

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* image rename

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

---------

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
 2025-01-21 20:56:48 -08:00
Vini murafa bc31d8ed6b typo fix README.md (#7879) 2025-01-20 10:50:35 -08:00
Krish Dholakia c4780479a9 Litellm dev 01 10 2025 p2 (#7679) 
* test(test_basic_python_version.py): assert all optional dependencies are marked as extras on poetry

Fixes https://github.com/BerriAI/litellm/issues/7677

* docs(secret.md): clarify 'read_and_write' secret manager usage on aws

* docs(secret.md): fix doc

* build(ui/teams.tsx): add edit/delete button for updating user / team membership on ui

allows updating user role to admin on ui

* build(ui/teams.tsx): display edit member component on ui, when edit button on member clicked

* feat(team_endpoints.py): support updating team member role to admin via api endpoints

allows team member to become admin post-add

* build(ui/user_dashboard.tsx): if team admin - show all team keys

Fixes https://github.com/BerriAI/litellm/issues/7650

* test(config.yml): add tomli to ci/cd

* test: don't call python_basic_testing in local testing (covered by python 3.13 testing)
 2025-01-10 21:50:53 -08:00
Ishaan Jaff 30a90a07b2 Update README.md 2024-12-16 08:36:57 -08:00
Krish Dholakia f4b5a491b6 docs: document code quality (#7149) 
* docs: document code quality

* build(readme.md): cleanup
 2024-12-10 15:44:59 -08:00
Krish Dholakia 6bb934c0ac fix(key_management_endpoints.py): override metadata field value on up… (#7008) 
* fix(key_management_endpoints.py): override metadata field value on update

allow user to override tags

* feat(__init__.py): expose new disable_end_user_cost_tracking_prometheus_only metric

allow disabling end user cost tracking on prometheus - fixes cardinality issue

* fix(litellm_pre_call_utils.py): add key/team level enforced params

Fixes https://github.com/BerriAI/litellm/issues/6652

* fix(key_management_endpoints.py): allow user to pass in `enforced_params` as a top level param on /key/generate and /key/update

* docs(enterprise.md): add docs on enforcing required params for llm requests

* Add support of Galadriel API (#7005)

* fix(router.py): robust retry after handling

set retry after time to 0 if >0 healthy deployments. handle base case = 1 deployment

* test(test_router.py): fix test

* feat(bedrock/): add support for 'nova' models

also adds explicit 'converse/' route for simpler routing

* fix: fix 'supports_pdf_input'

return if model supports pdf input on get_model_info

* feat(converse_transformation.py): support bedrock pdf input

* docs(document_understanding.md): add document understanding to docs

* fix(litellm_pre_call_utils.py): fix linting error

* fix(init.py): fix passing of bedrock converse models

* feat(bedrock/converse): support 'response_format={"type": "json_object"}'

* fix(converse_handler.py): fix linting error

* fix(base_llm_unit_tests.py): fix test

* fix: fix test

* test: fix test

* test: fix test

* test: remove duplicate test

---------

Co-authored-by: h4n0 <4738254+h4n0@users.noreply.github.com>
 2024-12-03 23:03:50 -08:00
Krish Dholakia 3beecfb0d4 LiteLLM Minor Fixes & Improvements (11/13/2024) (#6729) 
* fix(utils.py): add logprobs support for together ai

Fixes

https://github.com/BerriAI/litellm/issues/6724

* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint

adds new `anthropic/` pass-through endpoint + refactors docs

* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id

enables seeing spend for a customer in a team

* Add integration with MLflow Tracing (#6147)

* Add MLflow logger

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* Streaming handling

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* lint

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* address comments and fix issues

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* address comments and fix issues

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* Move logger construction code

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* Add docs

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* async handlers

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* new picture

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

---------

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* fix(mlflow.py): fix ruff linting errors

* ci(config.yml): add mlflow to ci testing

* fix: fix test

* test: fix test

* Litellm key update fix (#6710)

* fix(caching): convert arg to equivalent kwargs in llm caching handler

prevent unexpected errors

* fix(caching_handler.py): don't pass args to caching

* fix(caching): remove all *args from caching.py

* fix(caching): consistent function signatures + abc method

* test(caching_unit_tests.py): add unit tests for llm caching

ensures coverage for common caching scenarios across different implementations

* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one

* fix(router.py): drop redis password requirement

* fix(proxy_server.py): fix faulty slack alerting check

* fix(langfuse.py): avoid copying functions/thread lock objects in metadata

fixes metadata copy error when parent otel span in metadata

* test: update test

* fix(key_management_endpoints.py): fix /key/update with metadata update

* fix(key_management_endpoints.py): fix key_prepare_update helper

* fix(key_management_endpoints.py): reset value to none if set in key update

* fix: update test

'

* Litellm dev 11 11 2024 (#6693)

* fix(__init__.py): add 'watsonx_text' as mapped llm api route

Fixes https://github.com/BerriAI/litellm/issues/6663

* fix(opentelemetry.py): fix passing parallel tool calls to otel

Fixes https://github.com/BerriAI/litellm/issues/6677

* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling

reduces bugs in repo

* fix(__init__.py): update provider-model mapping to include all known provider-model mappings

Fixes https://github.com/BerriAI/litellm/issues/6669

* feat(anthropic): support passing document in llm api call

* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function

* fix(factory.py): fix linting error

* add clear doc string for GCS bucket logging

* Add docs to export logs to Laminar (#6674)

* Add docs to export logs to Laminar

* minor fix: newline at end of file

* place laminar after http and grpc

* (Feat) Add langsmith key based logging (#6682)

* add langsmith_api_key to StandardCallbackDynamicParams

* create a file for langsmith types

* langsmith add key / team based logging

* add key based logging for langsmith

* fix langsmith key based logging

* fix linting langsmith

* remove NOQA violation

* add unit test coverage for all helpers in test langsmith

* test_langsmith_key_based_logging

* docs langsmith key based logging

* run langsmith tests in logging callback tests

* fix logging testing

* test_langsmith_key_based_logging

* test_add_callback_via_key_litellm_pre_call_utils_langsmith

* add debug statement langsmith key based logging

* test_langsmith_key_based_logging

* (fix) OpenAI's optional messages[].name  does not work with Mistral API  (#6701)

* use helper for _transform_messages mistral

* add test_message_with_name to base LLMChat test

* fix linting

* add xAI on Admin UI (#6680)

* (docs) add benchmarks on 1K RPS  (#6704)

* docs litellm proxy benchmarks

* docs GCS bucket

* doc fix - reduce clutter on logging doc title

* (feat) add cost tracking stable diffusion 3 on Bedrock  (#6676)

* add cost tracking for sd3

* test_image_generation_bedrock

* fix get model info for image cost

* add cost_calculator for stability 1 models

* add unit testing for bedrock image cost calc

* test_cost_calculator_with_no_optional_params

* add test_cost_calculator_basic

* correctly allow size Optional

* fix cost_calculator

* sd3 unit tests cost calc

* fix raise correct error 404 when /key/info is called on non-existent key  (#6653)

* fix raise correct error on /key/info

* add not_found_error error

* fix key not found in DB error

* use 1 helper for checking token hash

* fix error code on key info

* fix test key gen prisma

* test_generate_and_call_key_info

* test fix test_call_with_valid_model_using_all_models

* fix key info tests

* bump: version 1.52.4 → 1.52.5

* add defaults used for GCS logging

* LiteLLM Minor Fixes & Improvements (11/12/2024)  (#6705)

* fix(caching): convert arg to equivalent kwargs in llm caching handler

prevent unexpected errors

* fix(caching_handler.py): don't pass args to caching

* fix(caching): remove all *args from caching.py

* fix(caching): consistent function signatures + abc method

* test(caching_unit_tests.py): add unit tests for llm caching

ensures coverage for common caching scenarios across different implementations

* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one

* fix(router.py): drop redis password requirement

* fix(proxy_server.py): fix faulty slack alerting check

* fix(langfuse.py): avoid copying functions/thread lock objects in metadata

fixes metadata copy error when parent otel span in metadata

* test: update test

* bump: version 1.52.5 → 1.52.6

* (feat) helm hook to sync db schema  (#6715)

* v0 migration job

* fix job

* fix migrations job.yml

* handle standalone DB on helm hook

* fix argo cd annotations

* fix db migration helm hook

* fix migration job

* doc fix Using Http/2 with Hypercorn

* (fix proxy redis) Add redis sentinel support  (#6154)

* add sentinel_password support

* add doc for setting redis sentinel password

* fix redis sentinel - use sentinel password

* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714)

Fixes #6713

* (fix) using Anthropic `response_format={"type": "json_object"}`  (#6721)

* add support for response_format=json anthropic

* add test_json_response_format to baseLLM ChatTest

* fix test_litellm_anthropic_prompt_caching_tools

* fix test_anthropic_function_call_with_no_schema

* test test_create_json_tool_call_for_response_format

* (feat) Add cost tracking for Azure Dall-e-3 Image Generation  + use base class to ensure basic image generation tests pass  (#6716)

* add BaseImageGenTest

* use 1 class for unit testing

* add debugging to BaseImageGenTest

* TestAzureOpenAIDalle3

* fix response_cost_calculator

* test_basic_image_generation

* fix img gen basic test

* fix _select_model_name_for_cost_calc

* fix test_aimage_generation_bedrock_with_optional_params

* fix undo changes cost tracking

* fix response_cost_calculator

* fix test_cost_azure_gpt_35

* fix remove dup test (#6718)

* (build) update db helm hook

* (build) helm db pre sync hook

* (build) helm db sync hook

* test: run test_team_logging firdst

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>

* test: update test

* test: skip anthropic overloaded error

* test: cleanup test

* test: update tests

* test: fix test

* test: handle gemini overloaded model error

* test: handle internal server error

* test: handle anthropic overloaded error

* test: handle claude instability

---------

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
 2024-11-15 11:18:31 +05:30
Ishaan Jaff 310669e3bc (docs) add instructions on how to contribute to docker image 2024-11-13 20:52:17 -08:00
Ishaan Jaff 68fdf9de2d docs readme 2024-08-08 17:06:29 -07:00
Ishaan Jaff 5bc8b59b11 docs - use consistent name for LiteLLM proxy server 2024-08-03 12:49:35 -07:00