diff --git a/docs/my-website/docs/providers/bedrock.md b/docs/my-website/docs/providers/bedrock.md index 165ef1d12f..157a9e6a79 100644 --- a/docs/my-website/docs/providers/bedrock.md +++ b/docs/my-website/docs/providers/bedrock.md @@ -1821,40 +1821,6 @@ Here's an example of using a bedrock model with LiteLLM. For a complete list, re | Mistral 7B Instruct | `completion(model='bedrock/mistral.mistral-7b-instruct-v0:2', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` | | Mixtral 8x7B Instruct | `completion(model='bedrock/mistral.mixtral-8x7b-instruct-v0:1', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` | -## Bedrock Embedding - -### API keys -This can be set as env variables or passed as **params to litellm.embedding()** -```python -import os -os.environ["AWS_ACCESS_KEY_ID"] = "" # Access key -os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key -os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2 -``` - -### Usage -```python -from litellm import embedding -response = embedding( - model="bedrock/amazon.titan-embed-text-v1", - input=["good morning from litellm"], -) -print(response) -``` - -## Supported AWS Bedrock Embedding Models - -| Model Name | Usage | Supported Additional OpenAI params | -|----------------------|---------------------------------------------|-----| -| Titan Embeddings V2 | `embedding(model="bedrock/amazon.titan-embed-text-v2:0", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_v2_transformation.py#L59) | -| Titan Embeddings - V1 | `embedding(model="bedrock/amazon.titan-embed-text-v1", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_g1_transformation.py#L53) -| Titan Multimodal Embeddings | `embedding(model="bedrock/amazon.titan-embed-image-v1", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_multimodal_transformation.py#L28) | -| Cohere Embeddings - English | `embedding(model="bedrock/cohere.embed-english-v3", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/cohere_transformation.py#L18) -| Cohere Embeddings - Multilingual | `embedding(model="bedrock/cohere.embed-multilingual-v3", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/cohere_transformation.py#L18) - -### Advanced - [Drop Unsupported Params](https://docs.litellm.ai/docs/completion/drop_params#openai-proxy-usage) - -### Advanced - [Pass model/provider-specific Params](https://docs.litellm.ai/docs/completion/provider_specific_params#proxy-usage) ## Image Generation Use this for stable diffusion, and amazon nova canvas on bedrock diff --git a/docs/my-website/docs/providers/bedrock_embedding.md b/docs/my-website/docs/providers/bedrock_embedding.md new file mode 100644 index 0000000000..430f9a4578 --- /dev/null +++ b/docs/my-website/docs/providers/bedrock_embedding.md @@ -0,0 +1,95 @@ +## Bedrock Embedding + +## Supported Embedding Models + +| Provider | LiteLLM Route | AWS Documentation | +|----------|---------------|-------------------| +| Amazon Titan | `bedrock/amazon.*` | [Amazon Titan Embeddings](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html) | +| Cohere | `bedrock/cohere.*` | [Cohere Embeddings](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-embed.html) | +| TwelveLabs | `bedrock/us.twelvelabs.*` | [TwelveLabs](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-twelvelabs.html) | + +### API keys +This can be set as env variables or passed as **params to litellm.embedding()** +```python +import os +os.environ["AWS_ACCESS_KEY_ID"] = "" # Access key +os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key +os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2 +``` + +## Usage +### LiteLLM Python SDK +```python +from litellm import embedding +response = embedding( + model="bedrock/amazon.titan-embed-text-v1", + input=["good morning from litellm"], +) +print(response) +``` + +### LiteLLM Proxy Server + +#### 1. Setup config.yaml +```yaml +model_list: + - model_name: titan-embed-v1 + litellm_params: + model: bedrock/amazon.titan-embed-text-v1 + aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID + aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY + aws_region_name: us-east-1 + - model_name: titan-embed-v2 + litellm_params: + model: bedrock/amazon.titan-embed-text-v2:0 + aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID + aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY + aws_region_name: us-east-1 +``` + +#### 2. Start Proxy +```bash +litellm --config /path/to/config.yaml +``` + +#### 3. Use with OpenAI Python SDK +```python +import openai +client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:4000" +) + +response = client.embeddings.create( + input=["good morning from litellm"], + model="titan-embed-v1" +) +print(response) +``` + +#### 4. Use with LiteLLM Python SDK +```python +import litellm +response = litellm.embedding( + model="titan-embed-v1", # model alias from config.yaml + input=["good morning from litellm"], + api_base="http://0.0.0.0:4000", + api_key="anything" +) +print(response) +``` + +## Supported AWS Bedrock Embedding Models + +| Model Name | Usage | Supported Additional OpenAI params | +|----------------------|---------------------------------------------|-----| +| Titan Embeddings V2 | `embedding(model="bedrock/amazon.titan-embed-text-v2:0", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_v2_transformation.py#L59) | +| Titan Embeddings - V1 | `embedding(model="bedrock/amazon.titan-embed-text-v1", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_g1_transformation.py#L53) +| Titan Multimodal Embeddings | `embedding(model="bedrock/amazon.titan-embed-image-v1", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/amazon_titan_multimodal_transformation.py#L28) | +| TwelveLabs Marengo Embed 2.7 | `embedding(model="bedrock/us.twelvelabs.marengo-embed-2-7-v1:0", input=input)` | Supports multimodal input (text, video, audio, image) | +| Cohere Embeddings - English | `embedding(model="bedrock/cohere.embed-english-v3", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/cohere_transformation.py#L18) +| Cohere Embeddings - Multilingual | `embedding(model="bedrock/cohere.embed-multilingual-v3", input=input)` | [here](https://github.com/BerriAI/litellm/blob/f5905e100068e7a4d61441d7453d7cf5609c2121/litellm/llms/bedrock/embed/cohere_transformation.py#L18) + +### Advanced - [Drop Unsupported Params](https://docs.litellm.ai/docs/completion/drop_params#openai-proxy-usage) + +### Advanced - [Pass model/provider-specific Params](https://docs.litellm.ai/docs/completion/provider_specific_params#proxy-usage) \ No newline at end of file diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index f3bab0219f..ae6071b16d 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -411,6 +411,7 @@ const sidebars = { label: "Bedrock", items: [ "providers/bedrock", + "providers/bedrock_embedding", "providers/bedrock_agents", "providers/bedrock_batches", "providers/bedrock_vector_store", diff --git a/litellm/__init__.py b/litellm/__init__.py index 92319df432..3c1d6e0696 100644 --- a/litellm/__init__.py +++ b/litellm/__init__.py @@ -67,6 +67,7 @@ from litellm.constants import ( bedrock_embedding_models, known_tokenizer_config, BEDROCK_INVOKE_PROVIDERS_LITERAL, + BEDROCK_EMBEDDING_PROVIDERS_LITERAL, BEDROCK_CONVERSE_MODELS, DEFAULT_MAX_TOKENS, DEFAULT_SOFT_BUDGET, diff --git a/litellm/constants.py b/litellm/constants.py index 077059b7a3..9b44613b85 100644 --- a/litellm/constants.py +++ b/litellm/constants.py @@ -769,6 +769,12 @@ BEDROCK_INVOKE_PROVIDERS_LITERAL = Literal[ "deepseek_r1", ] +BEDROCK_EMBEDDING_PROVIDERS_LITERAL = Literal[ + "cohere", + "amazon", + "twelvelabs", +] + BEDROCK_CONVERSE_MODELS = [ "openai.gpt-oss-20b-1:0", "openai.gpt-oss-120b-1:0", diff --git a/litellm/llms/bedrock/base_aws_llm.py b/litellm/llms/bedrock/base_aws_llm.py index d9b7eb6410..8211addaf9 100644 --- a/litellm/llms/bedrock/base_aws_llm.py +++ b/litellm/llms/bedrock/base_aws_llm.py @@ -20,7 +20,11 @@ from pydantic import BaseModel from litellm._logging import verbose_logger from litellm.caching.caching import DualCache -from litellm.constants import BEDROCK_INVOKE_PROVIDERS_LITERAL, BEDROCK_MAX_POLICY_SIZE +from litellm.constants import ( + BEDROCK_EMBEDDING_PROVIDERS_LITERAL, + BEDROCK_INVOKE_PROVIDERS_LITERAL, + BEDROCK_MAX_POLICY_SIZE, +) from litellm.litellm_core_utils.dd_tracing import tracer from litellm.secret_managers.main import get_secret, get_secret_str @@ -327,6 +331,40 @@ class BaseAWSLLM: return provider return None + @staticmethod + def get_bedrock_embedding_provider( + model: str, + ) -> Optional[BEDROCK_EMBEDDING_PROVIDERS_LITERAL]: + """ + Helper function to get the bedrock embedding provider from the model + + Handles scenarios like: + 1. model=cohere.embed-english-v3:0 -> Returns `cohere` + 2. model=amazon.titan-embed-text-v1 -> Returns `amazon` + 3. model=us.twelvelabs.marengo-embed-2-7-v1:0 -> Returns `twelvelabs` + 4. model=twelvelabs.marengo-embed-2-7-v1:0 -> Returns `twelvelabs` + """ + # Handle regional models like us.twelvelabs.marengo-embed-2-7-v1:0 + if "." in model: + parts = model.split(".") + # Check if the second part (after potential region) is a known provider + if len(parts) >= 2: + potential_provider = parts[1] # e.g., "twelvelabs" from "us.twelvelabs.marengo-embed-2-7-v1:0" + if potential_provider in get_args(BEDROCK_EMBEDDING_PROVIDERS_LITERAL): + return cast(BEDROCK_EMBEDDING_PROVIDERS_LITERAL, potential_provider) + + # Check if the first part is a known provider (standard format) + potential_provider = parts[0] # e.g., "cohere" from "cohere.embed-english-v3:0" + if potential_provider in get_args(BEDROCK_EMBEDDING_PROVIDERS_LITERAL): + return cast(BEDROCK_EMBEDDING_PROVIDERS_LITERAL, potential_provider) + + # Fallback: check if any provider name appears in the model string + for provider in get_args(BEDROCK_EMBEDDING_PROVIDERS_LITERAL): + if provider in model: + return cast(BEDROCK_EMBEDDING_PROVIDERS_LITERAL, provider) + + return None + def _get_aws_region_name( self, optional_params: dict, diff --git a/litellm/llms/bedrock/embed/embedding.py b/litellm/llms/bedrock/embed/embedding.py index 67ece820b1..d4dd716a1f 100644 --- a/litellm/llms/bedrock/embed/embedding.py +++ b/litellm/llms/bedrock/embed/embedding.py @@ -5,11 +5,12 @@ Handles embedding calls to Bedrock's `/invoke` endpoint import copy import json import urllib.parse -from typing import Any, Callable, List, Optional, Tuple, Union +from typing import Any, Callable, List, Optional, Tuple, Union, get_args import httpx import litellm +from litellm.constants import BEDROCK_EMBEDDING_PROVIDERS_LITERAL from litellm.llms.cohere.embed.handler import embedding as cohere_embedding from litellm.llms.custom_httpx.http_handler import ( AsyncHTTPHandler, @@ -150,6 +151,44 @@ class BedrockEmbedding(BaseAWSLLM): raise BedrockError(status_code=408, message="Timeout error occurred.") return response.json() + + def _transform_response( + self, response_list: List[dict], model: str, provider: BEDROCK_EMBEDDING_PROVIDERS_LITERAL + ) -> Optional[EmbeddingResponse]: + """ + Transforms the response from the Bedrock embedding provider to the OpenAI format. + """ + returned_response: Optional[EmbeddingResponse] = None + if model == "amazon.titan-embed-image-v1": + returned_response = ( + AmazonTitanMultimodalEmbeddingG1Config()._transform_response( + response_list=response_list, model=model + ) + ) + elif model == "amazon.titan-embed-text-v1": + returned_response = AmazonTitanG1Config()._transform_response( + response_list=response_list, model=model + ) + elif model == "amazon.titan-embed-text-v2:0": + returned_response = AmazonTitanV2Config()._transform_response( + response_list=response_list, model=model + ) + elif provider == "twelvelabs": + returned_response = TwelveLabsMarengoEmbeddingConfig()._transform_response( + response_list=response_list, model=model + ) + + + ########################################################## + # Validate returned response + ########################################################## + if returned_response is None: + raise Exception( + "Unable to map model response to known provider format. model={}".format( + model + ) + ) + return returned_response def _single_func_embeddings( self, @@ -162,6 +201,7 @@ class BedrockEmbedding(BaseAWSLLM): aws_region_name: str, model: str, logging_obj: Any, + provider: BEDROCK_EMBEDDING_PROVIDERS_LITERAL, api_key: Optional[str] = None, ): responses: List[dict] = [] @@ -208,32 +248,9 @@ class BedrockEmbedding(BaseAWSLLM): responses.append(response) - returned_response: Optional[EmbeddingResponse] = None - - ## TRANSFORM RESPONSE ## - if model == "amazon.titan-embed-image-v1": - returned_response = ( - AmazonTitanMultimodalEmbeddingG1Config()._transform_response( - response_list=responses, model=model - ) - ) - elif model == "amazon.titan-embed-text-v1": - returned_response = AmazonTitanG1Config()._transform_response( - response_list=responses, model=model - ) - elif model == "amazon.titan-embed-text-v2:0": - returned_response = AmazonTitanV2Config()._transform_response( - response_list=responses, model=model - ) - - if returned_response is None: - raise Exception( - "Unable to map model response to known provider format. model={}".format( - model - ) - ) - - return returned_response + return self._transform_response( + response_list=responses, model=model, provider=provider + ) async def _async_single_func_embeddings( self, @@ -246,6 +263,7 @@ class BedrockEmbedding(BaseAWSLLM): aws_region_name: str, model: str, logging_obj: Any, + provider: BEDROCK_EMBEDDING_PROVIDERS_LITERAL, api_key: Optional[str] = None, ): responses: List[dict] = [] @@ -291,33 +309,10 @@ class BedrockEmbedding(BaseAWSLLM): ) responses.append(response) - - returned_response: Optional[EmbeddingResponse] = None - ## TRANSFORM RESPONSE ## - if model == "amazon.titan-embed-image-v1": - returned_response = ( - AmazonTitanMultimodalEmbeddingG1Config()._transform_response( - response_list=responses, model=model - ) - ) - elif model == "amazon.titan-embed-text-v1": - returned_response = AmazonTitanG1Config()._transform_response( - response_list=responses, model=model - ) - elif model == "amazon.titan-embed-text-v2:0": - returned_response = AmazonTitanV2Config()._transform_response( - response_list=responses, model=model - ) - - if returned_response is None: - raise Exception( - "Unable to map model response to known provider format. model={}".format( - model - ) - ) - - return returned_response + return self._transform_response( + response_list=responses, model=model, provider=provider + ) def embeddings( self, @@ -349,7 +344,12 @@ class BedrockEmbedding(BaseAWSLLM): model_id=unencoded_model_id, ) - provider = model.split(".")[0] + provider = self.get_bedrock_embedding_provider(model) + if provider is None: + raise Exception( + f"Unable to determine bedrock embedding provider for model: {model}. " + f"Supported providers: {list(get_args(BEDROCK_EMBEDDING_PROVIDERS_LITERAL))}" + ) inference_params = copy.deepcopy(optional_params) inference_params = { k: v @@ -399,9 +399,7 @@ class BedrockEmbedding(BaseAWSLLM): ) ) batch_data.append(transformed_request) - elif provider == "twelvelabs" and model in [ - "twelvelabs.marengo-embed-2-7-v1:0", - ]: + elif provider == "twelvelabs": batch_data = [] for i in input: twelvelabs_request: ( @@ -438,8 +436,9 @@ class BedrockEmbedding(BaseAWSLLM): model=model, logging_obj=logging_obj, api_key=api_key, + provider=provider, ) - return self._single_func_embeddings( + returned_response = self._single_func_embeddings( client=( client if client is not None and isinstance(client, HTTPHandler) @@ -454,7 +453,11 @@ class BedrockEmbedding(BaseAWSLLM): model=model, logging_obj=logging_obj, api_key=api_key, + provider=provider, ) + if returned_response is None: + raise Exception("Unable to map Bedrock request to provider") + return returned_response elif data is None: raise Exception("Unable to map Bedrock request to provider") diff --git a/litellm/llms/bedrock/embed/twelvelabs_marengo_transformation.py b/litellm/llms/bedrock/embed/twelvelabs_marengo_transformation.py index ffa1ed940e..fdad8a6504 100644 --- a/litellm/llms/bedrock/embed/twelvelabs_marengo_transformation.py +++ b/litellm/llms/bedrock/embed/twelvelabs_marengo_transformation.py @@ -69,15 +69,6 @@ class TwelveLabsMarengoEmbeddingConfig: if "textTruncate" not in inference_params: transformed_request["textTruncate"] = "end" - # Set default embedding options for Phase 1 (text and image) - if "embeddingOption" not in inference_params: - if is_encoded: - # For images, return both visual-text and visual-image embeddings - transformed_request["embeddingOption"] = ["visual-text", "visual-image"] - else: - # For text, return visual-text embedding - transformed_request["embeddingOption"] = ["visual-text"] - # Apply any additional inference parameters for k, v in inference_params.items(): if k not in [ @@ -94,14 +85,32 @@ class TwelveLabsMarengoEmbeddingConfig: ) -> EmbeddingResponse: """ Transform TwelveLabs response to OpenAI format. - Handles multiple embedding types in the response. + Handles the actual TwelveLabs response format: {"data": [{"embedding": [...]}]} """ embeddings: List[Embedding] = [] total_tokens = 0 for response in response_list: - if "embedding" in response: - # Single embedding response + # TwelveLabs response format has a "data" field containing the embeddings + if "data" in response and isinstance(response["data"], list): + for item in response["data"]: + if "embedding" in item: + # Single embedding response + embedding = Embedding( + embedding=item["embedding"], + index=len(embeddings), + object="embedding", + ) + embeddings.append(embedding) + + # Estimate token count (rough approximation) + if "inputTextTokenCount" in item: + total_tokens += item["inputTextTokenCount"] + else: + # Rough estimate: 1 token per 4 characters for text, or use embedding size + total_tokens += len(item["embedding"]) // 4 + elif "embedding" in response: + # Direct embedding response (fallback for other formats) embedding = Embedding( embedding=response["embedding"], index=len(embeddings), diff --git a/litellm/model_prices_and_context_window_backup.json b/litellm/model_prices_and_context_window_backup.json index da2700f7c2..87d2ddc2bc 100644 --- a/litellm/model_prices_and_context_window_backup.json +++ b/litellm/model_prices_and_context_window_backup.json @@ -308,6 +308,60 @@ "supports_image_input": true, "supports_multimodal_embedding": true }, + "us.twelvelabs.marengo-embed-2-7-v1:0": { + "input_cost_per_token": 7e-05, + "input_cost_per_second_video": 0.0007, + "input_cost_per_second_audio": 0.00014, + "input_cost_per_image": 0.0001, + "litellm_provider": "bedrock", + "max_input_tokens": 77, + "max_tokens": 77, + "mode": "embedding", + "output_cost_per_token": 0.0, + "output_vector_size": 1024, + "supports_embedding_image_input": true, + "supports_image_input": true, + "supports_multimodal_embedding": true + }, + "eu.twelvelabs.marengo-embed-2-7-v1:0": { + "input_cost_per_token": 7e-05, + "input_cost_per_second_video": 0.0007, + "input_cost_per_second_audio": 0.00014, + "input_cost_per_image": 0.0001, + "litellm_provider": "bedrock", + "max_input_tokens": 77, + "max_tokens": 77, + "mode": "embedding", + "output_cost_per_token": 0.0, + "output_vector_size": 1024, + "supports_embedding_image_input": true, + "supports_image_input": true, + "supports_multimodal_embedding": true + }, + "twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, + "us.twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, + "eu.twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, "amazon.titan-text-express-v1": { "input_cost_per_token": 1.3e-06, "litellm_provider": "bedrock", diff --git a/model_prices_and_context_window.json b/model_prices_and_context_window.json index da2700f7c2..87d2ddc2bc 100644 --- a/model_prices_and_context_window.json +++ b/model_prices_and_context_window.json @@ -308,6 +308,60 @@ "supports_image_input": true, "supports_multimodal_embedding": true }, + "us.twelvelabs.marengo-embed-2-7-v1:0": { + "input_cost_per_token": 7e-05, + "input_cost_per_second_video": 0.0007, + "input_cost_per_second_audio": 0.00014, + "input_cost_per_image": 0.0001, + "litellm_provider": "bedrock", + "max_input_tokens": 77, + "max_tokens": 77, + "mode": "embedding", + "output_cost_per_token": 0.0, + "output_vector_size": 1024, + "supports_embedding_image_input": true, + "supports_image_input": true, + "supports_multimodal_embedding": true + }, + "eu.twelvelabs.marengo-embed-2-7-v1:0": { + "input_cost_per_token": 7e-05, + "input_cost_per_second_video": 0.0007, + "input_cost_per_second_audio": 0.00014, + "input_cost_per_image": 0.0001, + "litellm_provider": "bedrock", + "max_input_tokens": 77, + "max_tokens": 77, + "mode": "embedding", + "output_cost_per_token": 0.0, + "output_vector_size": 1024, + "supports_embedding_image_input": true, + "supports_image_input": true, + "supports_multimodal_embedding": true + }, + "twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, + "us.twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, + "eu.twelvelabs.pegasus-1-2-v1:0": { + "input_cost_per_second_video": 0.00049, + "output_cost_per_token": 7.5e-06, + "litellm_provider": "bedrock", + "mode": "chat", + "supports_multimodal_input": true, + "supports_video_input": true + }, "amazon.titan-text-express-v1": { "input_cost_per_token": 1.3e-06, "litellm_provider": "bedrock", diff --git a/tests/llm_translation/test_bedrock_embedding.py b/tests/llm_translation/test_bedrock_embedding.py index f0dc9b9e78..f06132c8b5 100644 --- a/tests/llm_translation/test_bedrock_embedding.py +++ b/tests/llm_translation/test_bedrock_embedding.py @@ -76,3 +76,90 @@ def test_bedrock_embedding_models(model, input_type, embed_response): except Exception as e: pytest.fail(f"Error occurred: {e}") + + +def test_e2e_bedrock_embedding(): + """ + Test text embedding with TwelveLabs Marengo. + Validates that the transformation properly extracts embedding data from TwelveLabs response format. + """ + print("Testing text embedding...") + litellm._turn_on_debug() + response = litellm.embedding( + model="bedrock/us.twelvelabs.marengo-embed-2-7-v1:0", + input=["Hello world from LiteLLM with TwelveLabs Marengo!"], + aws_region_name="us-east-1" + ) + + # Validate response structure + assert isinstance(response, litellm.EmbeddingResponse), "Response should be EmbeddingResponse type" + assert hasattr(response, 'data'), "Response should have 'data' attribute" + assert len(response.data) > 0, "Response data should not be empty" + + # Validate first embedding + embedding_obj = response.data[0] + assert hasattr(embedding_obj, 'embedding'), "Embedding object should have 'embedding' attribute" + assert isinstance(embedding_obj.embedding, list), "Embedding should be a list of floats" + assert len(embedding_obj.embedding) > 0, "Embedding vector should not be empty" + assert all(isinstance(x, (int, float)) for x in embedding_obj.embedding), "All embedding values should be numeric" + + # Validate embedding properties + assert embedding_obj.index == 0, "First embedding should have index 0" + assert embedding_obj.object == "embedding", "Embedding object type should be 'embedding'" + + # Validate usage information + assert hasattr(response, 'usage'), "Response should have usage information" + assert response.usage is not None, "Usage should not be None" + assert response.usage.total_tokens >= 0, "Total tokens should be non-negative" + + print(f"Text embedding successful! Vector size: {len(embedding_obj.embedding)}, Response: {response}") + + + +def test_e2e_bedrock_embedding_image_twelvelabs_marengo(): + """ + Test image embedding with TwelveLabs Marengo. + Validates that the transformation properly extracts embedding data from TwelveLabs response format for images. + """ + print("Testing image embedding...") + litellm._turn_on_debug() + + # Load duck.png and convert to base64 + duck_img_path = os.path.join(os.path.dirname(__file__), "duck.png") + with open(duck_img_path, "rb") as img_file: + duck_img_data = base64.b64encode(img_file.read()).decode('utf-8') + duck_img_base64 = f"data:image/png;base64,{duck_img_data}" + + response = litellm.embedding( + model="bedrock/us.twelvelabs.marengo-embed-2-7-v1:0", + input=[duck_img_base64], + aws_region_name="us-east-1" + ) + + # Validate response structure + assert isinstance(response, litellm.EmbeddingResponse), "Response should be EmbeddingResponse type" + assert hasattr(response, 'data'), "Response should have 'data' attribute" + assert len(response.data) > 0, "Response data should not be empty" + + # Validate first embedding + embedding_obj = response.data[0] + assert hasattr(embedding_obj, 'embedding'), "Embedding object should have 'embedding' attribute" + assert isinstance(embedding_obj.embedding, list), "Embedding should be a list of floats" + assert len(embedding_obj.embedding) > 0, "Embedding vector should not be empty" + assert all(isinstance(x, (int, float)) for x in embedding_obj.embedding), "All embedding values should be numeric" + + # Validate embedding properties + assert embedding_obj.index == 0, "First embedding should have index 0" + assert embedding_obj.object == "embedding", "Embedding object type should be 'embedding'" + + # Validate usage information + assert hasattr(response, 'usage'), "Response should have usage information" + assert response.usage is not None, "Usage should not be None" + assert response.usage.total_tokens >= 0, "Total tokens should be non-negative" + + # TwelveLabs Marengo should return 1024-dimensional embeddings + expected_dimension = 1024 + assert len(embedding_obj.embedding) == expected_dimension, f"TwelveLabs Marengo should return {expected_dimension}-dimensional embeddings, got {len(embedding_obj.embedding)}" + + print(f"Image embedding successful! Vector size: {len(embedding_obj.embedding)}, Response: {response}") +