ContextQMD
Back to Llama Index

Konko

docs/examples/llm/konko.ipynb

0.14.214.6 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/Konko.ipynb" target="_parent"></a>

Konko

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

Konko API is a fully managed Web API designed to help application developers:

Konko API is a fully managed API designed to help application developers:

  1. Select the right LLM(s) for their application
  2. Prototype with various open-source and proprietary LLMs
  3. Access Fine Tuning for open-source LLMs to get industry-leading performance at a fraction of the cost
  4. Setup low-cost production APIs according to security, privacy, throughput, latency SLAs without infrastructure set-up or administration using Konko AI's SOC 2 compliant, multi-cloud infrastructure

Steps to Access Models

  1. Explore Available Models: Start by browsing through the available models on Konko. Each model caters to different use cases and capabilities.

  2. Identify Suitable Endpoints: Determine which endpoint (ChatCompletion or Completion) supports your selected model.

  3. Selecting a Model: Choose a model based on its metadata and how well it fits your use case.

  4. Prompting Guidelines: Once a model is selected, refer to the prompting guidelines to effectively communicate with it.

  5. Using the API: Finally, use the appropriate Konko API endpoint to call the model and receive responses.

To run this notebook, you'll need Konko API key. You can create one by signing up on Konko.

This example goes over how to use LlamaIndex to interact with Konko ChatCompletion models and Completion models

python
%pip install llama-index-llms-konko
python
!pip install llama-index

Call chat with ChatMessage List

You need to set env var KONKO_API_KEY

python
import os

os.environ["KONKO_API_KEY"] = "<your-api-key>"
python
from llama_index.llms.konko import Konko
from llama_index.core.llms import ChatMessage
python
llm = Konko(model="meta-llama/llama-2-13b-chat")
messages = ChatMessage(role="user", content="Explain Big Bang Theory briefly")

resp = llm.chat([messages])
print(resp)

Call chat with OpenAI Models

You need to either set env var OPENAI_API_KEY

python
import os

os.environ["OPENAI_API_KEY"] = "<your-api-key>"

llm = Konko(model="gpt-3.5-turbo")
python
message = ChatMessage(role="user", content="Explain Big Bang Theory briefly")
resp = llm.chat([message])
print(resp)

Streaming

python
message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message], max_tokens=1000)
for r in resp:
    print(r.delta, end="")

Call complete with Prompt

python
llm = Konko(model="numbersstation/nsql-llama-2-7b", max_tokens=100)
text = """CREATE TABLE stadium (
    stadium_id number,
    location text,
    name text,
    capacity number,
    highest number,
    lowest number,
    average number
)

CREATE TABLE singer (
    singer_id number,
    name text,
    country text,
    song_name text,
    song_release_year text,
    age number,
    is_male others
)

CREATE TABLE concert (
    concert_id number,
    concert_name text,
    theme text,
    stadium_id text,
    year text
)

CREATE TABLE singer_in_concert (
    concert_id number,
    singer_id text
)

-- Using valid SQLite, answer the following questions for the tables provided above.

-- What is the maximum capacity of stadiums ?

SELECT"""
response = llm.complete(text)
print(response)
python
llm = Konko(model="phind/phind-codellama-34b-v2", max_tokens=100)
text = """### System Prompt
You are an intelligent programming assistant.

### User Message
Implement a linked list in C++

### Assistant
..."""

resp = llm.stream_complete(text, max_tokens=1000)
for r in resp:
    print(r.delta, end="")

Model Configuration

python
llm = Konko(model="meta-llama/llama-2-13b-chat")
python
resp = llm.stream_complete(
    "Show me the c++ code to send requests to HTTP Server", max_tokens=1000
)
for r in resp:
    print(r.delta, end="")