結構化輸出

Ollama 現在支援結構化輸出，使其能夠將模型的輸出限制為 JSON Schema 定義的特定格式。Ollama Python 和 JavaScript 函式庫已更新以支援結構化輸出。

結構化輸出的使用案例包括

• 從文件中解析資料
• 從圖片中提取資料
• 結構化所有語言模型的回應
• 比 JSON 模式更可靠且更一致

開始使用

下載最新版本的 Ollama

升級到最新版本的 Ollama Python 或 JavaScript 函式庫

_Python

pip install -U ollama

_JavaScript

npm i ollama

若要將結構化輸出傳遞給模型，可以在 cURL 請求中使用 format 參數，或在 Python 或 JavaScript 函式庫中使用 format 參數。

cURL

curl -X POST https://127.0.0.1:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "llama3.1",
  "messages": [{"role": "user", "content": "Tell me about Canada."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "name": {
        "type": "string"
      },
      "capital": {
        "type": "string"
      },
      "languages": {
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    },
    "required": [
      "name",
      "capital", 
      "languages"
    ]
  }
}'

輸出

回應會以請求中 JSON Schema 定義的格式傳回。

{
  "capital": "Ottawa",
  "languages": [
    "English",
    "French"
  ],
  "name": "Canada"
}

Python

使用 Ollama Python 函式庫，將 Schema 作為 JSON 物件傳遞給 format 參數，可以使用 dict 或使用 Pydantic (推薦) 來使用 model_json_schema() 序列化 Schema。

from ollama import chat
from pydantic import BaseModel

class Country(BaseModel):
  name: str
  capital: str
  languages: list[str]

response = chat(
  messages=[
    {
      'role': 'user',
      'content': 'Tell me about Canada.',
    }
  ],
  model='llama3.1',
  format=Country.model_json_schema(),
)

country = Country.model_validate_json(response.message.content)
print(country)

輸出

name='Canada' capital='Ottawa' languages=['English', 'French']

JavaScript

使用 Ollama JavaScript 函式庫，將 Schema 作為 JSON 物件傳遞給 format 參數，可以使用 object 或使用 Zod (推薦) 來使用 zodToJsonSchema() 序列化 Schema。

import ollama from 'ollama';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const Country = z.object({
    name: z.string(),
    capital: z.string(), 
    languages: z.array(z.string()),
});

const response = await ollama.chat({
    model: 'llama3.1',
    messages: [{ role: 'user', content: 'Tell me about Canada.' }],
    format: zodToJsonSchema(Country),
});

const country = Country.parse(JSON.parse(response.message.content));
console.log(country);

輸出

{
  name: "Canada",
  capital: "Ottawa",
  languages: [ "English", "French" ],
}

範例

資料提取

若要從文字中提取結構化資料，請定義一個 Schema 以表示資訊。模型隨後會提取資訊，並以定義的 JSON Schema 格式傳回資料

from ollama import chat
from pydantic import BaseModel

class Pet(BaseModel):
  name: str
  animal: str
  age: int
  color: str | None
  favorite_toy: str | None

class PetList(BaseModel):
  pets: list[Pet]

response = chat(
  messages=[
    {
      'role': 'user',
      'content': '''
        I have two pets.
        A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur.
        I also have a 2 year old black cat named Loki who loves tennis balls.
      ''',
    }
  ],
  model='llama3.1',
  format=PetList.model_json_schema(),
)

pets = PetList.model_validate_json(response.message.content)
print(pets)

範例輸出

pets=[
  Pet(name='Luna', animal='cat', age=5, color='grey', favorite_toy='yarn'), 
  Pet(name='Loki', animal='cat', age=2, color='black', favorite_toy='tennis balls')
]

圖片描述

結構化輸出也可以與視覺模型一起使用。例如，以下程式碼使用 llama3.2-vision 來描述以下圖片並傳回結構化輸出

from ollama import chat
from pydantic import BaseModel

class Object(BaseModel):
  name: str
  confidence: float
  attributes: str 

class ImageDescription(BaseModel):
  summary: str
  objects: List[Object]
  scene: str
  colors: List[str]
  time_of_day: Literal['Morning', 'Afternoon', 'Evening', 'Night']
  setting: Literal['Indoor', 'Outdoor', 'Unknown']
  text_content: Optional[str] = None

path = 'path/to/image.jpg'

response = chat(
  model='llama3.2-vision',
  format=ImageDescription.model_json_schema(),  # Pass in the schema for the response
  messages=[
    {
      'role': 'user',
      'content': 'Analyze this image and describe what you see, including any objects, the scene, colors and any text you can detect.',
      'images': [path],
    },
  ],
  options={'temperature': 0},  # Set temperature to 0 for more deterministic output
)

image_description = ImageDescription.model_validate_json(response.message.content)
print(image_description)

範例輸出

summary='A palm tree on a sandy beach with blue water and sky.' 
objects=[
  Object(name='tree', confidence=0.9, attributes='palm tree'), 
  Object(name='beach', confidence=1.0, attributes='sand')
], 
scene='beach', 
colors=['blue', 'green', 'white'], 
time_of_day='Afternoon' 
setting='Outdoor' 
text_content=None

OpenAI 相容性

from openai import OpenAI
import openai
from pydantic import BaseModel

client = OpenAI(base_url="https://127.0.0.1:11434/v1", api_key="ollama")

class Pet(BaseModel):
    name: str
    animal: str
    age: int
    color: str | None
    favorite_toy: str | None

class PetList(BaseModel):
    pets: list[Pet]

try:
    completion = client.beta.chat.completions.parse(
        temperature=0,
        model="llama3.1:8b",
        messages=[
            {"role": "user", "content": '''
                I have two pets.
                A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur.
                I also have a 2 year old black cat named Loki who loves tennis balls.
            '''}
        ],
        response_format=PetList,
    )

    pet_response = completion.choices[0].message
    if pet_response.parsed:
        print(pet_response.parsed)
    elif pet_response.refusal:
        print(pet_response.refusal)
except Exception as e:
    if type(e) == openai.LengthFinishReasonError:
        print("Too many tokens: ", e)
        pass
    else:
        print(e)
        pass

提示

為了可靠地使用結構化輸出，請考慮

• 使用 Pydantic (Python) 或 Zod (JavaScript) 來定義回應的 Schema
• 在提示中加入「以 JSON 格式傳回」以幫助模型理解請求
• 將溫度設定為 0 以獲得更具決定性的輸出

接下來是什麼？

• 公開 logits 以實現受控生成
• 結構化輸出的效能和準確性改進
• 用於取樣的 GPU 加速
• JSON Schema 之外的額外格式支援