ZStack 资源中心

API请求

URLs

PUT zstack/v1/ai/model-services/eval/{uuid}

Headers

Authorization: OAuth the-session-uuid

Body

{
  "deployModelEvalService": {
    "taskType": "Performance",
    "limits": 10,
    "temperature": 0.5,
    "topK": 10,
    "topP": 0.5,
    "maxNewTokens": 10,
    "repetitionPenalty": 0.5,
    "maxLength": 1024,
    "prompt": "prompt",
    "model": "model",
    "url": "url",
    "parallel": 10,
    "logEveryQuery": 10,
    "api": "openai",
    "connectTimeout": 10,
    "readTimeout": 10,
    "zoneUuid": "zoneUuidExample",
    "name": "model-eval-service",
    "type": "VirtualMachine"
  },
  "systemTags": [],
  "userTags": []
}

上述示例中systemTags、userTags字段可以省略。列出是为了表示body中可以包含这两个字段。

Curl示例

curl -H "Content-Type: application/json;charset=UTF-8" \
-H "Authorization: OAuth b86c9016b4f24953a9edefb53ca0678c" \
-X PUT -d '{"deployModelEvalService":{"taskType":"Performance","limits":10,"temperature":0.5,"topK":10,"topP":0.5,"maxNewTokens":10,"repetitionPenalty":0.5,"maxLength":1024,"prompt":"prompt","model":"model","url":"url","parallel":10,"logEveryQuery":10,"api":"openai","connectTimeout":10,"readTimeout":10,"zoneUuid":"zoneUuidExample","name":"model-eval-service","type":"VirtualMachine"}}' \
http://localhost:8080/zstack/v1/ai/model-services/eval/5634e459d1e13ff9abaf395a9517c100

参数列表


名字	类型	位置	描述	可选值	起始版本
limits	Integer	body(包含在deployModelEvalService结构中)	限制		5.1.8
temperature (可选)	Float	body(包含在deployModelEvalService结构中)	温度		5.1.8
topK (可选)	Integer	body(包含在deployModelEvalService结构中)	Top K		5.1.8
topP (可选)	Float	body(包含在deployModelEvalService结构中)	Top P		5.1.8
maxLength (可选)	Integer	body(包含在deployModelEvalService结构中)	最大长度		5.1.8
maxNewTokens (可选)	Integer	body(包含在deployModelEvalService结构中)	最大新令牌数		5.1.8
repetitionPenalty (可选)	Float	body(包含在deployModelEvalService结构中)	重复惩罚		5.1.8
uuid	String	url	资源的UUID，唯一标示该资源		5.1.8
description (可选)	String	body(包含在deployModelEvalService结构中)	资源的详细描述		5.1.8
modelUuid (可选)	String	body(包含在deployModelEvalService结构中)	模型UUID		5.1.8
zoneUuid	String	body(包含在deployModelEvalService结构中)	区域UUID		5.1.8
vmImageUuid (可选)	String	body(包含在deployModelEvalService结构中)	虚拟机镜像UUID		5.1.8
primaryStorageUuid (可选)	String	body(包含在deployModelEvalService结构中)	主存储UUID		5.1.8
datasetUuids (可选)	List	body(包含在deployModelEvalService结构中)			5.1.8
modelServiceGroupUuids (可选)	List	body(包含在deployModelEvalService结构中)			5.1.8
dockerImage (可选)	String	body(包含在deployModelEvalService结构中)	Docker镜像		5.1.8
cpuNum (可选)	Integer	body(包含在deployModelEvalService结构中)			5.1.8
name	String	body(包含在deployModelEvalService结构中)	资源名称		5.1.8
type	String	body(包含在deployModelEvalService结构中)		VirtualMachine Container	5.1.8
memorySize (可选)	Long	body(包含在deployModelEvalService结构中)	内存大小		5.1.8
l3NetworkUuids (可选)	List	body(包含在deployModelEvalService结构中)			5.1.8
systemTags (可选)	List	body	系统标签		5.1.8
userTags (可选)	List	body	用户标签		5.1.8
serviceBootUptime (可选)	Integer	body(包含在deployModelEvalService结构中)			5.1.20
taskType	String	body(包含在deployModelEvalService结构中)	任务类型	Performance Capability	5.3.6
prompt (可选)	String	body(包含在deployModelEvalService结构中)			5.3.6
model (可选)	String	body(包含在deployModelEvalService结构中)	模型		5.3.6
url (可选)	String	body(包含在deployModelEvalService结构中)	URL		5.3.6
parallel (可选)	Integer	body(包含在deployModelEvalService结构中)	并行		5.3.6
logEveryQuery (可选)	Integer	body(包含在deployModelEvalService结构中)	记录每个查询		5.3.6
api (可选)	String	body(包含在deployModelEvalService结构中)	API	openai dashscope	5.3.6
requestHeaders (可选)	Map	body(包含在deployModelEvalService结构中)	请求头		5.3.6
connectTimeout (可选)	Integer	body(包含在deployModelEvalService结构中)	连接超时		5.3.6
readTimeout (可选)	Integer	body(包含在deployModelEvalService结构中)	读取超时		5.3.6
environmentVariables (可选)	Map	body(包含在deployModelEvalService结构中)			5.3.20
startupParameters (可选)	Map	body(包含在deployModelEvalService结构中)			5.3.20
clusterUuid (可选)	String	body(包含在deployModelEvalService结构中)	集群UUID		5.3.22
rootDiskOfferingUuid (可选)	String	body(包含在deployModelEvalService结构中)			5.3.28
rootDiskSize (可选)	Long	body(包含在deployModelEvalService结构中)			5.3.28
resourceUuid (可选)	String	body(包含在deployModelEvalService结构中)	资源UUID		5.3.46
tagUuids (可选)	List	body(包含在deployModelEvalService结构中)	标签UUID列表		5.3.46
serviceLivez (可选)	String	body(包含在deployModelEvalService结构中)			5.4.6
serviceReadyz (可选)	String	body(包含在deployModelEvalService结构中)			5.4.6
requestCpuNum (可选)	Integer	body(包含在deployModelEvalService结构中)	请求CPU数量。		5.5.22
requestMemorySize (可选)	Long	body(包含在deployModelEvalService结构中)	请求内存大小。		5.5.22
projectUuid (可选)	String	body(包含在deployModelEvalService结构中)	项目UUID。		5.5.22

API返回

返回示例

{
  "inventory": {
    "uuid": "groupUuid",
    "name": "model-eval-service",
    "status": "Running"
  },
  "tasks": [
    {
      "uuid": "taskUuid",
      "name": "eval-task",
      "status": "Running"
    }
  ]
}

返回字段


名字	类型	描述	起始版本
success	Boolean	操作是否成功	5.1.8
error	ErrorCode	错误码，操作成功时该字段为null	5.1.8
inventory	ModelEvalServiceInstanceGroupInventory	模型评测服务实例组清单	5.1.8
tasks	List	模型评测任务清单列表	5.1.8

#inventory


名字	类型	描述	起始版本
uuid	String	资源的UUID，唯一标识该资源	5.1.8
modelServiceUuid	String	模型服务UUID	5.1.8
modelUuid	String	模型UUID	5.1.8
instances	List	模型服务实例列表	5.1.8
datasetRefInventories	List	数据集引用列表	5.1.8
status	String	实例组状态	5.1.8
modelServiceType	String	模型服务类型	5.1.8
type	String	实例组类型	5.1.8
name	String	实例组名称	5.1.8
description	String	实例组描述	5.1.8
yaml	String	YAML配置	5.1.8
supportMetrics	List	支持的指标列表	5.1.8
exportPath	String	导出路径	5.1.8
createDate	Timestamp	创建时间	5.1.8
lastOpDate	Timestamp	最后一次修改时间	5.1.8
limits	Integer	限制数量	5.1.8
temperature	Float	温度参数	5.1.8
topK	Integer	Top K参数	5.1.8
topP	Float	Top P参数	5.1.8
maxLength	Integer	最大长度	5.1.8
maxNewTokens	Integer	最大新生成Token数量	5.1.8
repetitionPenalty	Float	重复惩罚参数	5.1.8

#tasks


名字	类型	描述	起始版本
uuid	String	资源的UUID，唯一标识该资源	5.1.8
name	String	任务名称	5.1.8
type	String	任务类型	5.1.8
description	String	任务描述	5.1.8
percentage	Integer	任务进度百分比	5.1.8
opaque	String	扩展信息	5.1.8
status	String	任务状态	5.1.8
modelServiceGroupUuid	String	模型服务实例组UUID	5.1.8
evaluatedServiceGroupUuid	String	被评测服务实例组UUID	5.1.8
datasetUuid	String	数据集UUID	5.1.8
limits	Integer	限制数量	5.1.8
maxNewTokens	Integer	最大新生成Token数量	5.1.8
topK	Integer	Top K参数	5.1.8
temperature	Float	温度参数	5.1.8
topP	Float	Top P参数	5.1.8
prompt	String	提示词	5.1.8
repetitionPenalty	Float	重复惩罚参数	5.1.8
maxLength	Integer	最大长度	5.1.8
model	String	模型名称	5.1.8
url	String	服务URL	5.1.8
parallel	Integer	并发数	5.1.8
logEveryQuery	Integer	查询日志间隔	5.1.8
api	String	API类型	5.1.8
requestHeaders	Map	请求头	5.1.8
connectTimeout	Integer	连接超时时间	5.1.8
readTimeout	Integer	读取超时时间	5.1.8
totalScore	Double	总分	5.1.8
endTime	Timestamp	结束时间	5.1.8
createDate	Timestamp	创建时间	5.1.8
lastOpDate	Timestamp	最后一次修改时间	5.1.8

SDK示例

Java SDK

DeployModelEvalServiceAction action = new DeployModelEvalServiceAction();
action.taskType = "Performance";
action.limits = 10;
action.temperature = 0.5;
action.topK = 10;
action.topP = 0.5;
action.maxNewTokens = 10;
action.repetitionPenalty = 0.5;
action.maxLength = 1024;
action.prompt = "prompt";
action.model = "model";
action.url = "url";
action.parallel = 10;
action.logEveryQuery = 10;
action.api = "openai";
action.connectTimeout = 10;
action.readTimeout = 10;
action.zoneUuid = "zoneUuidExample";
action.name = "model-eval-service";
action.type = "VirtualMachine";
action.uuid = "5634e459d1e13ff9abaf395a9517c100";
action.sessionId = "b86c9016b4f24953a9edefb53ca0678c";
DeployModelEvalServiceAction.Result res = action.call();

Python SDK

DeployModelEvalServiceAction action = DeployModelEvalServiceAction()
action.taskType = "Performance"
action.limits = 10
action.temperature = 0.5
action.topK = 10
action.topP = 0.5
action.maxNewTokens = 10
action.repetitionPenalty = 0.5
action.maxLength = 1024
action.prompt = "prompt"
action.model = "model"
action.url = "url"
action.parallel = 10
action.logEveryQuery = 10
action.api = "openai"
action.connectTimeout = 10
action.readTimeout = 10
action.zoneUuid = "zoneUuidExample"
action.name = "model-eval-service"
action.type = "VirtualMachine"
action.uuid = "5634e459d1e13ff9abaf395a9517c100"
action.sessionId = "b86c9016b4f24953a9edefb53ca0678c"
DeployModelEvalServiceAction.Result res = action.call()

部署模型评测任务 (DeployModelEvalService)

API请求

API返回

SDK示例