Open WebUI

https://github.com/open-webui/open-webui

首先作为结论，这个软件非常庞大，甚至有点臃肿，配置也比较复杂。

# 不知道为什么很奇怪，要安装那么多东西，所以最好放在一个新的独立的python环境里面
# 要安装几百个依赖包，其中还有一个800多M的torch包、1个700多M的cudnn（内存不够的话加swap空间）
# cuda的包几乎全装一遍，无语了，这还叫精简？这是要自己推理大模型吗？不是叫「WebUI」吗？
# 总之，首先准备venv并激活

# 安装
python -m pip install open-webui
# 刚安装完成，venv占用空间大小是：9.2G

# 我们是在服务器上安装的，需要做一些特定的步骤
# 步骤1：设定一个环境变量
# 依据：https://github.com/open-webui/open-webui/discussions/9987#discussioncomment-12313446
export ENABLE_WEBSOCKET_SUPPORT=false

# 启动
# 默认在8080端口
open-webui serve

# 做反向代理
# 步骤2：需要为Nginx开启websockets
# 依据：https://github.com/open-webui/open-webui/discussions/11053#discussioncomment-12367949
# Websockets
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

# 在反向代理、SSL证书之类的都准备好之后，应该就可以在公网上访问到了
# 第一次会设定管理员账户
# 使用管理员账号登录后，有两个设置面板，一个是「用户设置」，一个是「管理员设置」

# 记录一下Nginx的配置
server {
    listen      443 ssl http2;
    server_name ai.osakayaki.com;

    ssl_certificate     /var/www/acmesh_cert/ai.osakayaki.com/cert.pem;
    ssl_certificate_key /var/www/acmesh_cert/ai.osakayaki.com/key.pem;
    
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Range $http_range;
        proxy_set_header If-Range $http_if_range;
        proxy_redirect off;
        proxy_pass http://127.0.0.1:8080;
        
        # the max size of file to upload
        client_max_body_size 20000m;

        # Websockets
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

1. 需要使用「函数」
2. 参考资料：https://github.com/open-webui/open-webui/issues/3288
3. 所需函数：https://openwebui.com/f/justinrahb/anthropic
4. 地址：管理员设置 > 函数 > 手动添加 or 导入
5. 在导入函数之后：
6. 我们可以修改一下代码中的模型列表，不需要的模型就删掉，或者在这里（https://docs.anthropic.com/en/docs/about-claude/models/overview）找到其他新的模型，添加到代码中（get_anthropic_models函数）
7. 为这个函数设置变量（应该是代码中暴露出来的变量，也就是需要的anthropic的API Key）
8. enable这个函数（默认是关闭的）
9. 现在chat页面应该就可以看到模型了

# 记录一下这个函数的代码
# 可以打开link后选择导入（需要登录）
# 或者直接把代码手动导入（选择手动添加函数）
"""
title: Anthropic Manifold Pipe
authors: justinh-rahb and christian-taillon
author_url: https://github.com/justinh-rahb
funding_url: https://github.com/open-webui
version: 0.2.5
required_open_webui_version: 0.3.17
license: MIT
"""
 
import os
import requests
import json
import time
from typing import List, Union, Generator, Iterator
from pydantic import BaseModel, Field
from open_webui.utils.misc import pop_system_message
 
 
class Pipe:
    class Valves(BaseModel):
        ANTHROPIC_API_KEY: str = Field(default="")
 
    def __init__(self):
        self.type = "manifold"
        self.id = "anthropic"
        self.name = "anthropic/"
        self.valves = self.Valves(
            **{"ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY", "")}
        )
        self.MAX_IMAGE_SIZE = 5 * 1024 * 1024  # 5MB per image
        pass
 
    def get_anthropic_models(self):
        return [
            {"id": "claude-3-haiku-20240307", "name": "claude-3-haiku"},
            {"id": "claude-3-opus-20240229", "name": "claude-3-opus"},
            {"id": "claude-3-sonnet-20240229", "name": "claude-3-sonnet"},
            {"id": "claude-3-5-haiku-20241022", "name": "claude-3.5-haiku"},
            {"id": "claude-3-5-haiku-latest", "name": "claude-3.5-haiku"},
            {"id": "claude-3-5-sonnet-20240620", "name": "claude-3.5-sonnet"},
            {"id": "claude-3-5-sonnet-20241022", "name": "claude-3.5-sonnet"},
            {"id": "claude-3-5-sonnet-latest", "name": "claude-3.5-sonnet"},
            {"id": "claude-3-7-sonnet-20250219", "name": "claude-3.7-sonnet"},
            {"id": "claude-3-7-sonnet-latest", "name": "claude-3.7-sonnet"},
        ]
 
    def pipes(self) -> List[dict]:
        return self.get_anthropic_models()
 
    def process_image(self, image_data):
        """Process image data with size validation."""
        if image_data["image_url"]["url"].startswith("data:image"):
            mime_type, base64_data = image_data["image_url"]["url"].split(",", 1)
            media_type = mime_type.split(":")[1].split(";")[0]
 
            # Check base64 image size
            image_size = len(base64_data) * 3 / 4  # Convert base64 size to bytes
            if image_size > self.MAX_IMAGE_SIZE:
                raise ValueError(
                    f"Image size exceeds 5MB limit: {image_size / (1024 * 1024):.2f}MB"
                )
 
            return {
                "type": "image",
                "source": {
                    "type": "base64",
                    "media_type": media_type,
                    "data": base64_data,
                },
            }
        else:
            # For URL images, perform size check after fetching
            url = image_data["image_url"]["url"]
            response = requests.head(url, allow_redirects=True)
            content_length = int(response.headers.get("content-length", 0))
 
            if content_length > self.MAX_IMAGE_SIZE:
                raise ValueError(
                    f"Image at URL exceeds 5MB limit: {content_length / (1024 * 1024):.2f}MB"
                )
 
            return {
                "type": "image",
                "source": {"type": "url", "url": url},
            }
 
    def pipe(self, body: dict) -> Union[str, Generator, Iterator]:
        system_message, messages = pop_system_message(body["messages"])
 
        processed_messages = []
        total_image_size = 0
 
        for message in messages:
            processed_content = []
            if isinstance(message.get("content"), list):
                for item in message["content"]:
                    if item["type"] == "text":
                        processed_content.append({"type": "text", "text": item["text"]})
                    elif item["type"] == "image_url":
                        processed_image = self.process_image(item)
                        processed_content.append(processed_image)
 
                        # Track total size for base64 images
                        if processed_image["source"]["type"] == "base64":
                            image_size = len(processed_image["source"]["data"]) * 3 / 4
                            total_image_size += image_size
                            if (
                                total_image_size > 100 * 1024 * 1024
                            ):  # 100MB total limit
                                raise ValueError(
                                    "Total size of images exceeds 100 MB limit"
                                )
            else:
                processed_content = [
                    {"type": "text", "text": message.get("content", "")}
                ]
 
            processed_messages.append(
                {"role": message["role"], "content": processed_content}
            )
 
        payload = {
            "model": body["model"][body["model"].find(".") + 1 :],
            "messages": processed_messages,
            "max_tokens": body.get("max_tokens", 4096),
            "temperature": body.get("temperature", 0.8),
            "top_k": body.get("top_k", 40),
            "top_p": body.get("top_p", 0.9),
            "stop_sequences": body.get("stop", []),
            **({"system": str(system_message)} if system_message else {}),
            "stream": body.get("stream", False),
        }
 
        headers = {
            "x-api-key": self.valves.ANTHROPIC_API_KEY,
            "anthropic-version": "2023-06-01",
            "content-type": "application/json",
        }
 
        url = "https://api.anthropic.com/v1/messages"
 
        try:
            if body.get("stream", False):
                return self.stream_response(url, headers, payload)
            else:
                return self.non_stream_response(url, headers, payload)
        except requests.exceptions.RequestException as e:
            print(f"Request failed: {e}")
            return f"Error: Request failed: {e}"
        except Exception as e:
            print(f"Error in pipe method: {e}")
            return f"Error: {e}"
 
    def stream_response(self, url, headers, payload):
        try:
            with requests.post(
                url, headers=headers, json=payload, stream=True, timeout=(3.05, 60)
            ) as response:
                if response.status_code != 200:
                    raise Exception(
                        f"HTTP Error {response.status_code}: {response.text}"
                    )
 
                for line in response.iter_lines():
                    if line:
                        line = line.decode("utf-8")
                        if line.startswith("data: "):
                            try:
                                data = json.loads(line[6:])
                                if data["type"] == "content_block_start":
                                    yield data["content_block"]["text"]
                                elif data["type"] == "content_block_delta":
                                    yield data["delta"]["text"]
                                elif data["type"] == "message_stop":
                                    break
                                elif data["type"] == "message":
                                    for content in data.get("content", []):
                                        if content["type"] == "text":
                                            yield content["text"]
 
                                time.sleep(
                                    0.01
                                )  # Delay to avoid overwhelming the client
 
                            except json.JSONDecodeError:
                                print(f"Failed to parse JSON: {line}")
                            except KeyError as e:
                                print(f"Unexpected data structure: {e}")
                                print(f"Full data: {data}")
        except requests.exceptions.RequestException as e:
            print(f"Request failed: {e}")
            yield f"Error: Request failed: {e}"
        except Exception as e:
            print(f"General error in stream_response method: {e}")
            yield f"Error: {e}"
 
    def non_stream_response(self, url, headers, payload):
        try:
            response = requests.post(
                url, headers=headers, json=payload, timeout=(3.05, 60)
            )
            if response.status_code != 200:
                raise Exception(f"HTTP Error {response.status_code}: {response.text}")
 
            res = response.json()
            return (
                res["content"][0]["text"] if "content" in res and res["content"] else ""
            )
        except requests.exceptions.RequestException as e:
            print(f"Failed non-stream request: {e}")
            return f"Error: {e}"

管理员面板 > 设置 > 外部连接 > 直接添加OpenAI的Key即可，保存后就能看到所有OpenAI的模型了

直接和普通的OpenAI接口设置一样（管理员面板 > 设置 > 外部连接 > 直接添加OpenAI的Key）
参考：https://docs.openwebui.com/getting-started/quick-start/starting-with-openai
例子：
URL填写：https://www.gpt4novel.com/api/xiaoshuoai/ext/v1
再填写token
不需要手动添加model id，会自动从/models下获取所有可用model

用这个函数：https://openwebui.com/f/owndev/google_gemini
只需要设置一个API Key就可以用了，不用设置模型
202508测试可用

为单个对话设定：在对话界面，点击右上角Controls，即可设置，当前对话生效
为本用户设定：Settings > General > System Prompt

1. 也可以使用Docker启动
2. 开启联网搜索：Admin Panel > Web Search > Enable即可

数据都是存放在哪里的呢？比如用户名/密码等账户信息？

# AI说可以跳过可选依赖包，减小空间占用，尚未测试
pip install open-webui --no-deps
pip install fastapi uvicorn pydantic requests aiohttp