MUSAChat 在线 API 服务 API

概念介绍

大语言模型推理服务

备案信息： MUSAChat（对应备案模型：摩笔大模型），上线备案号：Beijing-MoBi-202407050022。

语言模型推理服务是摩尔线程大语言模型的 API 服务，运行于摩尔线程全自研 GPU 服务器上，对外提供 Chat Completions 接口，可用于生成文本、问答、摘要、翻译等任务。

Token

文本生成以 Token 为单位处理文本。Token 代表常见的字符序列；对于一段常规中文文本，1 个 Token 大约相当于 1.5～2 个汉字。

使用语言模型推理服务不能超过模型的最大上下文，超过后会被截断。

使用指南

接口描述

MUSAChat（对应备案模型：摩笔大模型）对话接口，用于以对话形式使用该模型的对话能力。用户提供历史对话信息和问题，通过 HTTP 接口调用 MUSAChat Serving API；模型响应后，以流式或非流式形式返回相应结果。该服务的上线备案号为 Beijing-MoBi-202407050022。

发送请求

你可以使用我们的 Chat Completions API 来发送请求，需要提供一个 API 密钥和一个模型名称。

处理响应

请求成功后会返回一个 JSON 格式的响应。

API 文档

基本信息

请求参数

Message

返回参数

Choice

Usage

Error

错误码

接入示例

非流式正确返回示例

curl --location 'http://{{endpoint}}/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "id": "5ac7deb3-75d9-4b6d-a89f-7a17895b8719",
    "model": "MusaChat-Small-4K-v0.26.2",
    "messages": [
        {
            "role": "system",
            "content": ""
        },
        {
            "role": "user",
            "content": "你好！"
        }
    ],
    "stream": false,
    "max_tokens": 512,
    "temperature": 0.0,
    "repetition_penalty": 1.0
}'
{
    "id": "5ac7deb3-75d9-4b6d-a89f-7a17895b8719",
    "object": "chat.completion",
    "created": 1677652288,
    "model": "MusaChat-Small-4K-v0.26.2",
    "choices":
    [
        {
            "index": 0,
            "message":
            {
                "role": "assistant",
                "content": "你好！有什么我可以帮助你的吗？"
            },
            "finish_reason": "stop"
        }
    ],
    "usage":
    {
        "prompt_tokens": 36,
        "completion_tokens": 13,
        "total_tokens": 49
    }
}

流式正确返回示例

curl --location 'http://{{endpoint}}/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "id": "36af8d58-2620-40ea-89dc-65253db4b889",
    "model": "MusaChat-Small-4K-v0.26.2",
    "messages": [
        {
            "role": "system",
            "content": ""
        },
        {
            "role": "user",
            "content": "你好！"
        }
    ],
    "stream": true,
    "max_tokens": 512,
    "temperature": 0.0,
    "repetition_penalty": 1.0
}'
data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"1"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"你"},"finish_reason":"2"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"好"},"finish_reason":"3"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"！"},"finish_reason":"4"}]}

...

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"提供"},"finish_reason":"44"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"帮助"},"finish_reason":"45"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{"content":"."},"finish_reason":"46"}]}

data:{"id":"36af8d58-2620-40ea-89dc-65253db4b889","object":"chat.completion.chunk","created":1694268190,"model":"MusaChat-Small-4K-v0.26.2","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data:[DONE]

非流式错误返回示例

curl --location 'http://{{endpoint}}/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "id": "5ac7deb3-75d9-4b6d-a89f-7a17895b8719",
    "model": "MusaChat-Small-4K-v0.26.2",
    "messages": [
        {
            "role": "system",
            "content": ""
        },
        {
            "role": "user",
            "content": "你好！"
        }
    ],
    "stream": false,
    "max_tokens": 512,
    "temperature": -1,
    "repetition_penalty": 1.0
}'
{
  "error": {
    "code": 40001002,
    "message": "-1 is less than the minimum of 0 - 'temperature'"
  }
}

调用方法

Curl 调用方法

curl --location 'http://{{endpoint}}/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "id": "714fea0b-f7e1-4f25-9514-cf2281161011",
    "model": "MusaChat-Small-4K-v0.26.2",
    "messages": [
        {
            "role": "system",
            "content": ""
        },
        {
            "role": "user",
            "content": "一年有多少天？"
        }
    ],
    "stream": true,
    "max_tokens": 512,
    "temperature": 0.0,
    "repetition_penalty": 1.0
}'

Python 调用方法

import requests
import json


def parse_stream_helper(line: bytes):
    if line:
        if line.strip() == b"data: [DONE]":
            return None
        if line.startswith(b"data: "):
            line = line[len(b"data: "):]
            return line.decode("utf-8")
        else:
            return None
    return None


# 是否需要流式返回
stream = False
data = {
    "messages": [{"role": "user", "content": "你好"}],
    "temperature": 0.0,
    "max_tokens": 512,
    "stream": stream,
}

api_base = "http://{{endpoint}}/v1/"
url = api_base + "/chat/completions"
response = requests.post(url, json=data)
if stream:
    answer = ""
    for line in response.iter_lines():
        item = parse_stream_helper(line)
        if item:
            item = json.loads(item)
            answer += item["choices"][0]["delta"]["content"]
    print(answer)
else:
    resp_data = response.json()
    answer = resp_data["choices"][0]["message"]["content"]
    print(answer)