litellm/cookbook/liteLLM_Hugging_Face_Example.ipynb

Install liteLLM https://github.com/BerriAI/litellm

liteLLM provides one interface to call gpt 3.5, hugging face inference endpoints

In [1]:

!pip install litellm=="0.1.362"

Requirement already satisfied: litellm==0.1.362 in /usr/local/lib/python3.10/dist-packages (0.1.362)
Requirement already satisfied: openai<0.28.0,>=0.27.8 in /usr/local/lib/python3.10/dist-packages (from litellm==0.1.362) (0.27.8)
Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from litellm==0.1.362) (1.0.0)
Requirement already satisfied: tiktoken<0.5.0,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from litellm==0.1.362) (0.4.0)
Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai<0.28.0,>=0.27.8->litellm==0.1.362) (2.28.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai<0.28.0,>=0.27.8->litellm==0.1.362) (4.65.0)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai<0.28.0,>=0.27.8->litellm==0.1.362) (3.8.5)
Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<0.5.0,>=0.4.0->litellm==0.1.362) (2022.10.31)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<0.28.0,>=0.27.8->litellm==0.1.362) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<0.28.0,>=0.27.8->litellm==0.1.362) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<0.28.0,>=0.27.8->litellm==0.1.362) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<0.28.0,>=0.27.8->litellm==0.1.362) (2023.7.22)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (6.0.4)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (1.9.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (1.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<0.28.0,>=0.27.8->litellm==0.1.362) (1.3.1)

In [5]:

from litellm import completion
import os
user_message = "Hello, whats the weather in San Francisco??"
messages = [{ "content": user_message,"role": "user"}]

os.environ['HF_TOKEN'] = ""#@param
# get your hugging face token from here:
# https://huggingface.co/settings/tokens

# Optional if you want to run OpenAI TOO
os.environ['OPENAI_API_KEY'] = "" #@param

response = completion("stabilityai/stablecode-completion-alpha-3b-4k", messages=messages, hugging_face=True)
print("Response from stabilityai/stablecode-completion-alpha-3b-4k")
print(response['choices'][0]['message']['content'])
print("\n\n")

response = completion("bigcode/starcoder", messages=messages, hugging_face=True)
print("Response from bigcode/starcoder")
print(response['choices'][0]['message']['content'])
print("\n\n")

response = completion("google/flan-t5-xxl", messages=messages, hugging_face=True)
print("Response from google/flan-t5-xxl")
print(response['choices'][0]['message']['content'])
print("\n\n")

response = completion("google/flan-t5-large", messages=messages, hugging_face=True)
print("Response from google/flan-t5-large")
print(response['choices'][0]['message']['content'])
print("\n\n")

response = completion(model="gpt-3.5-turbo", messages=messages)
print(response['choices'][0]['message']['content'])
print(response)

Response from stabilityai/stablecode-completion-alpha-3b-4k
Hello, whats the weather in San Francisco??",
    "id": 1,
    "



Response from bigcode/starcoder
Hello, whats the weather in San Francisco??")

# print(response)

# print(response.text)

#



Response from google/flan-t5-xxl
a little cold



Response from google/flan-t5-large
cool



I'm sorry, but I am an AI language model and do not have real-time data. However, you can check the weather in San Francisco by searching for "San Francisco weather" on a search engine or checking a reliable weather website or app.