Compare commits

..

16 Commits

Author SHA1 Message Date
archer
8af3d85b8a add gemini model 2025-01-25 14:25:10 +08:00
archer
fcf14af64d comment 2025-01-25 14:21:21 +08:00
heheer
991fbe254c fix interactive edge (#3659)
* fix interactive edge

* fix
2025-01-25 14:12:12 +08:00
Archer
ab0fc517dc fix: err tip (#3666)
* fix: err tip

* perf: training queue

* doc
2025-01-25 14:10:56 +08:00
Archer
92105e9a0b reload buffer (#3665)
* reload buffer

* reload buffer

* tts selector
2025-01-25 13:12:21 +08:00
Archer
d2948d7e57 feat: markdown extension (#3663)
* feat: markdown extension

* media cros

* rerank test

* default price

* perf: default model

* fix: cannot custom provider

* fix: default model select

* update bg

* perf: default model selector

* fix: usage export

* i18n

* fix: rerank

* update init extension

* perf: ip limit check

* doubao model order

* web default modle

* perf: tts selector

* perf: tts error

* qrcode package
2025-01-24 23:42:04 +08:00
heheer
02fcb6a61e export usage csv i18n (#3660)
* export usage csv i18n

* fix build
2025-01-24 19:09:08 +08:00
a.e.
4f5a12f33b fix: false triggerd org selection (#3661) 2025-01-24 19:07:36 +08:00
Archer
38efa3e050 feat: default model (#3662)
* move model config

* feat: default model
2025-01-24 18:44:43 +08:00
a.e.
5ce889942a fix: POST 500 error on dingtalk bot (#3655) 2025-01-24 14:10:40 +08:00
Archer
60c72d05d1 model perf (#3657)
* fix: model

* dataset quote

* perf: model config

* model tag

* doubao model config

* perf: config model

* feat: model test
2025-01-24 14:10:14 +08:00
Archer
99ce976b06 4.8.20 test (#3656)
* provider

* perf: model config
2025-01-23 18:32:45 +08:00
heheer
2c03abc6e1 add default model config (#3653) 2025-01-23 18:19:57 +08:00
Archer
34b510cba1 perf: usages list;perf: move components (#3654)
* perf: usages list

* team sub plan load

* perf: usage dashboard code

* perf: dashboard ui

* perf: move components
2025-01-23 17:29:39 +08:00
heheer
0c05add8b2 feat: usage filter & export & dashbord (#3538)
* feat: usage filter & export & dashbord

* adjust ui

* fix tmb scroll

* fix code & selecte all

* merge
2025-01-23 10:54:30 +08:00
Archer
e009be51e7 Aiproxy (#3649)
* model config

* feat: model config ui

* perf: rename variable

* feat: custom request url

* perf: model buffer

* perf: init model

* feat: json model config

* auto login

* fix: ts

* update packages

* package

* fix: dockerfile
2025-01-22 22:59:28 +08:00
235 changed files with 2056 additions and 5311 deletions

View File

@@ -58,7 +58,7 @@ jobs:
# Step 4 - Builds the site using Hugo
- name: Build
run: cd docSite && hugo mod get -u github.com/colinwilson/lotusdocs@6d0568e && hugo -v --minify
run: cd docSite && hugo mod get -u github.com/colinwilson/lotusdocs && hugo -v --minify
# Step 5 - Push our generated site to Vercel
- name: Deploy to Vercel

View File

@@ -58,7 +58,7 @@ jobs:
# Step 4 - Builds the site using Hugo
- name: Build
run: cd docSite && hugo mod get -u github.com/colinwilson/lotusdocs@6d0568e && hugo -v --minify
run: cd docSite && hugo mod get -u github.com/colinwilson/lotusdocs && hugo -v --minify
# Step 5 - Push our generated site to Vercel
- name: Deploy to Vercel

View File

@@ -83,7 +83,6 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [x] 统一查阅对话记录,并对数据进行标注
`6` 其他
- [x] 可视化模型配置。
- [x] 支持语音输入和输出 (可配置语音输入语音回答)
- [x] 模糊输入提示
- [x] 模板市场

View File

@@ -3,7 +3,7 @@ FROM hugomods/hugo:0.117.0 AS builder
WORKDIR /app
ADD ./docSite hugo
RUN cd /app/hugo && hugo mod get -u github.com/colinwilson/lotusdocs@6d0568e && hugo -v --minify
RUN cd /app/hugo && hugo mod get -u github.com/colinwilson/lotusdocs && hugo -v --minify
FROM fholzer/nginx-brotli:latest

Binary file not shown.

Before

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 197 KiB

View File

@@ -13,8 +13,8 @@ weight: 707
下面配置文件示例中包含了系统参数和各个模型配置:
## 4.8.20+ 版本新配置文件示例
> 从4.8.20版本开始,模型在页面中进行配置。
## 4.6.8+ 版本新配置文件示例
```json
{
"feConfigs": {
@@ -27,4 +27,4 @@ weight: 707
"pgHNSWEfSearch": 100 // 向量搜索参数。越大搜索越精确但是速度越慢。设置为100有99%+精度。
}
}
```
```

View File

@@ -7,13 +7,6 @@ toc: true
weight: 707
---
## 前置知识
1. 基础的网络知识:端口,防火墙……
2. Docker 和 Docker Compose 基础知识
3. 大模型相关接口和参数
4. RAG 相关知识:向量模型,向量数据库,向量检索
## 部署架构图
![](/imgs/sealos-fastgpt.webp)
@@ -211,8 +204,6 @@ docker restart oneapi
### 6. 配置模型
务必先配置至少一组模型,否则系统无法正常使用。
[点击查看模型配置教程](/docs/development/modelConfig/intro/)
## FAQ

View File

@@ -9,31 +9,17 @@ images: []
## 一、错误排查方式
可以先找找[Issue](https://github.com/labring/FastGPT/issues),或新提 Issue私有部署错误务必提供详细的操作步骤、日志、截图否则很难排查。
### 获取后端错误
遇到问题先按下面方式排查。
1. `docker ps -a` 查看所有容器运行状态,检查是否全部 running如有异常尝试`docker logs 容器名`查看对应日志。
2. 容器都运行正常的,`docker logs 容器名` 查看报错日志
3. 带有`requestId`的,都是 OneAPI 提示错误,大部分都是因为模型接口报错。
4. 无法解决时,可以找找[Issue](https://github.com/labring/FastGPT/issues),或新提 Issue私有部署错误务必提供详细的日志否则很难排查。
### 前端错误
前端报错时,页面会出现崩溃,并提示检查控制台日志。可以打开浏览器控制台,并查看`console`中的 log 日志。还可以点击对应 log 的超链接,会提示到具体错误文件,可以把这些详细错误信息提供,方便排查。
### OneAPI 错误
带有`requestId`的,都是 OneAPI 提示错误,大部分都是因为模型接口报错。可以参考 [OneAPI 常见错误](/docs/development/faq/#三常见的-oneapi-错误)
## 二、通用问题
### 前端页面崩溃
1. 90% 情况是模型配置不正确:确保每类模型都至少有一个启用;检查模型中一些`对象`参数是否异常(数组和对象),如果为空,可以尝试给个空数组或空对象。
2. 少部分是由于浏览器兼容问题,由于项目中包含一些高阶语法,可能低版本浏览器不兼容,可以将具体操作步骤和控制台中错误信息提供 issue。
3. 关闭浏览器翻译功能,如果浏览器开启了翻译,可能会导致页面崩溃。
### 通过sealos部署的话是否没有本地部署的一些限制
![](/imgs/faq1.png)
这是索引模型的长度限制,通过任何方式部署都一样的,但不同索引模型的配置不一样,可以在后台修改参数。
@@ -144,7 +130,7 @@ OneAPI 的 API Key 配置错误,需要修改`OPENAI_API_KEY`环境变量,并
## 四、常见模型问题
### 如何检查模型可用性问题
### 如何检查模型问题
1. 私有部署模型,先确认部署的模型是否正常。
2. 通过 CURL 请求,直接测试上游模型是否正常运行(云端模型或私有模型均进行测试)
@@ -417,7 +403,3 @@ curl --location --request POST 'https://oneapi.xxxx/v1/chat/completions' \
"tool_choice": "auto"
}'
```
### 向量检索得分大于 1
由于模型没有归一化导致的。目前仅支持归一化的模型。

View File

@@ -15,8 +15,8 @@ weight: 705
- [Git](http://git-scm.com/)
- [Docker](https://www.docker.com/)(构建镜像)
- [Node.js v20.14.0](http://nodejs.org)版本尽量一样可以使用nvm管理node版本
- [pnpm](https://pnpm.io/) 推荐版本 9.4.0 (目前官方的开发环境)
- [Node.js v18.17 / v20.x](http://nodejs.org)版本尽量一样可以使用nvm管理node版本
- [pnpm](https://pnpm.io/) 版本 8.6.0 (目前官方的开发环境)
- make命令: 根据不同平台,百度安装 (官方是GNU Make 4.3)
## 开始本地开发
@@ -77,6 +77,8 @@ Mongo 数据库需要注意,需要注意在连接地址中增加 `directConnec
可参考项目根目录下的 `dev.md`,第一次编译运行可能会有点慢,需要点耐心哦
```bash
# 给自动化脚本代码执行权限(非 linux 系统, 可以手动执行里面的 postinstall.sh 文件内容)
chmod -R +x ./scripts/
# 代码根目录下执行,会安装根 package、projects 和 packages 内所有依赖
# 如果提示 isolate-vm 安装失败可以参考https://github.com/laverdet/isolated-vm?tab=readme-ov-file#requirements
pnpm i

View File

@@ -11,9 +11,7 @@ weight: 744
从 4.8.20 版本开始,你可以直接在 FastGPT 页面中进行模型配置,并且系统内置了大量模型,无需从 0 开始配置。下面介绍模型配置的基本流程:
## 配置模型
### 1. 使用 OneAPI 对接模型提供商
## 1. 使用 OneAPI 对接模型提供商
可以使用 [OneAPI 接入教程](/docs/development/modelconfig/one-api) 来进行模型聚合,从而可以对接更多模型提供商。你需要先在各服务商申请好 API 接入 OneAPI 后,才能在 FastGPT 中使用这些模型。示例流程如下:
@@ -28,46 +26,44 @@ weight: 744
在 OneAPI 配置好模型后,你就可以打开 FastGPT 页面,启用对应模型了。
### 2. 登录 root 用户
## 2. 登录 root 用户
仅 root 用户可以进行模型配置。
### 3. 进入模型配置页面
## 3. 进入模型配置页面
登录 root 用户后,在`账号-模型提供商-模型配置`中,你可以看到所有内置的模型和自定义模型,以及哪些模型启用了。
![alt text](/image-90.png)
### 4. 配置介绍
## 4. 配置介绍
{{% alert icon="🤖 " context="success" %}}
注意:
1. 目前语音识别模型和重排模型仅会生效一个,所以配置时候,只需要配置一个即可。
2. 系统至少需要一个语言模型和一个索引模型才能正常使用。
注意:目前语音识别模型和重排模型仅会生效一个,所以配置时候,只需要配置一个即可。
{{% /alert %}}
#### 核心配置
### 核心配置
- 模型 ID接口请求时候Body 中`model`字段的值,全局唯一。
- 自定义请求地址/Key:如果需要绕过`OneAPI`,可以设置自定义请求地址和 Token。一般情况下不需要如果 OneAPI 不支持某些模型,可以使用该特性。
- 模型 ID实际发出请求的`model`值,全局唯一。
- 自定义请求地址/Token:如果需要绕过`OneAPI`,可以设置自定义请求地址和 Token。一般情况下不需要如果 OneAPI 不支持某些模型,可以使用该特性。
#### 模型类型
### 模型类型
1. 语言模型 - 进行文本对话,多模态模型支持图片识别。
2. 索引模型 - 对文本块进行索引,用于相关文本检索。
3. 重排模型 - 对检索结果进行重排,用于优化检索排名
4. 语音合成 - 将文本转换为语音
5. 语音识别 - 将语音转换为文本
3. 语音合成 - 将文本转换为语音
4. 语音识别 - 将语音转换为文本
5. 重排模型 - 对文本进行重排,用于优化文本质量
#### 启用模型
### 启用模型
系统内置了目前主流厂商的模型,如果你不熟悉配置,直接点击`启用`即可,需要注意是,`模型 ID`需要和 OneAPI 中渠道的`模型`一致。
系统内置了目前主流厂商的模型,如果你不熟悉配置,直接点击`启用`即可,需要注意是,模型 ID 需要和 OneAPI 中渠道的`模型`一致。
| | |
| --- | --- |
| ![alt text](/imgs/image-91.png) | ![alt text](/imgs/image-92.png) |
#### 修改模型配置
### 修改模型配置
点击模型右侧的齿轮即可进行模型配置,不同类型模型的配置有区别。
@@ -75,7 +71,7 @@ weight: 744
| --- | --- |
| ![alt text](/imgs/image-93.png) | ![alt text](/imgs/image-94.png) |
## 新增自定义模型
### 新增自定义模型
如果系统内置的模型无法满足你的需求,你可以添加自定义模型。自定义模型中,如果`模型 ID`与系统内置的模型 ID 一致,则会被认为是修改系统模型。
@@ -83,7 +79,7 @@ weight: 744
| --- | --- |
| ![alt text](/imgs/image-96.png) | ![alt text](/imgs/image-97.png) |
#### 通过配置文件配置
### 通过配置文件配置
如果你觉得通过页面配置模型比较麻烦,你也可以通过配置文件来配置模型。或者希望快速将一个系统的配置,复制到另一个系统,也可以通过配置文件来实现。
@@ -210,7 +206,7 @@ FastGPT 页面上提供了每类模型的简单测试,可以初步检查模型
![alt text](/imgs/image-105.png)
## 特殊接入示例
## 模型接入示例
### ReRank 模型接入
@@ -231,60 +227,6 @@ FastGPT 页面上提供了每类模型的简单测试,可以初步检查模型
[点击查看部署 ReRank 模型教程](/docs/development/custom-models/bge-rerank/)
### 接入语音识别模型
OneAPI 的语言识别接口,无法正确的识别其他模型(会始终识别成 whisper-1所以如果想接入其他模型可以通过自定义请求地址来实现。例如接入硅基流动的 `FunAudioLLM/SenseVoiceSmall` 模型,可以参考如下配置:
点击模型编辑:
![alt text](/imgs/image-106.png)
填写硅基流动的地址:`https://api.siliconflow.cn/v1/audio/transcriptions`,并填写硅基流动的 API Key。
![alt text](/imgs/image-107.png)
## 其他配置项说明
### 自定义请求地址
如果填写了该值,则可以允许你绕过 OneAPI直接向自定义请求地址发起请求。需要填写完整的请求地址例如
- LLM: {{host}}/v1/chat/completions
- Embedding: {{host}}/v1/embeddings
- STT: {{host}}/v1/audio/transcriptions
- TTS: {{host}}/v1/audio/speech
- Rerank: {{host}}/v1/rerank
自定义请求 Key则是向自定义请求地址发起请求时候携带请求头Authorization: Bearer xxx 进行请求。
所有接口均遵循 OpenAI 提供的模型格式,可参考 [OpenAI API 文档](https://platform.openai.com/docs/api-reference/introduction) 进行配置。
由于 OpenAI 没有提供 ReRank 模型,遵循的是 Cohere 的格式。[点击查看接口请求示例](/docs/development/faq/#如何检查模型问题)
### 模型价格配置
商业版用户可以通过配置模型价格,来进行账号计费。系统包含两种计费模式:按总 tokens 计费和输入输出 Tokens 分开计费。
如果需要配置`输入输出 Tokens 分开计费模式`,则填写`模型输入价格``模型输出价格`两个值。
如果需要配置`按总 tokens 计费模式`,则填写`模型综合价格`一个值。
## 如何提交内置模型
由于模型更新非常频繁,官方不一定及时更新,如果未能找到你期望的内置模型,你可以[提交 Issue](https://github.com/labring/FastGPT/issues),提供模型的名字和对应官网。或者直接[提交 PR](https://github.com/labring/FastGPT/pulls),提供模型配置。
### 添加模型提供商
如果你需要添加模型提供商,需要修改以下代码:
1. FastGPT/packages/web/components/common/Icon/icons/model - 在此目录下,添加模型提供商的 svg 头像地址。
2. 在 FastGPT 根目录下,运行`pnpm initIcon`,将图片加载到配置文件中。
3. FastGPT/packages/global/core/ai/provider.ts - 在此文件中,追加模型提供商的配置。
### 添加模型
你可以在`FastGPT/packages/service/core/ai/config/provider`目录下,找对应模型提供商的配置文件,并追加模型配置。请自行全文检查,`model`字段,必须在所有模型中唯一。具体配置字段说明,参考[模型配置字段说明](/docs/development/modelconfig/intro/#通过配置文件配置)
## 旧版模型配置说明

View File

@@ -672,7 +672,7 @@ curl --location --request POST 'http://localhost:3000/api/core/chat/getHistories
"appId": "appId",
"offset": 0,
"pageSize": 20,
"source": "api"
"source: "api"
}'
```

View File

@@ -735,7 +735,7 @@ data 为集合的 ID。
**4.8.19+**
```bash
curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listV2' \
curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listv2' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{

View File

@@ -11,7 +11,7 @@ weight: 860
在 FastGPT V4.6.4 中,我们修改了分享链接的数据读取方式,为每个用户生成一个 localId用于标识用户从云端拉取对话记录。但是这种方式仅能保障用户在同一设备同一浏览器中使用如果切换设备或者清空浏览器缓存则会丢失这些记录。这种方式存在一定的风险因此我们仅允许用户拉取近`30天``20条`记录。
分享链接身份鉴权设计的目的在于,将 FastGPT 的对话框快速、安全的接入到你现有的系统中,仅需 2 个接口即可实现。该功能目前只在商业版中提供。
分享链接身份鉴权设计的目的在于,将 FastGPT 的对话框快速、安全的接入到你现有的系统中,仅需 2 个接口即可实现。
## 使用说明

View File

@@ -60,10 +60,6 @@ FastGPT 使用了 one-api 项目来管理模型池,其可以兼容 OpenAI 、A
### 3. 配置模型
### 4. 配置模型
务必先配置至少一组模型,否则系统无法正常使用。
[点击查看模型配置教程](/docs/development/modelConfig/intro/)
## 收费

View File

@@ -1,5 +1,5 @@
---
title: 'V4.8.18(包含升级脚本)'
title: 'V4.8.18'
description: 'FastGPT V4.8.18 更新说明'
icon: 'upgrade'
draft: false

View File

@@ -1,5 +1,5 @@
---
title: 'V4.8.19(包含升级脚本)'
title: 'V4.8.19(进行中)'
description: 'FastGPT V4.8.19 更新说明'
icon: 'upgrade'
draft: false

View File

@@ -1,5 +1,5 @@
---
title: 'V4.8.20(包含升级脚本)'
title: 'V4.8.20(进行中)'
description: 'FastGPT V4.8.20 更新说明'
icon: 'upgrade'
draft: false
@@ -9,19 +9,14 @@ weight: 804
## 更新指南
### 1. 做好数据库备份
### 2. 更新环境变量
### 1. 更新环境变量
如果有很早版本用户,配置了`ONEAPI_URL`的,需要统一改成`OPENAI_BASE_URL`
### 3. 更新镜像:
### 1. 更新镜像:
- 更新 fastgpt 镜像 tag: v4.8.20-fix2
- 更新 fastgpt-pro 商业版镜像 tag: v4.8.20-fix2
- Sandbox 镜像无需更新
### 4. 运行升级脚本
### 2. 运行升级脚本
从任意终端,发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`{{host}} 替换成**FastGPT 域名**。
@@ -31,20 +26,13 @@ curl --location --request POST 'https://{{host}}/api/admin/initv4820' \
--header 'Content-Type: application/json'
```
脚本会自动把原配置文件的模型加载到新版模型配置中
自动把原配置文件的模型加载到新版模型配置中
## 完整更新内容
1. 新增 - 可视化模型参数配置,取代原配置文件配置模型。预设超过 100 个模型配置。同时支持所有类型模型的一键测试。(预计下个版本会完全支持在页面上配置渠道)。
2. 新增 - DeepSeek resoner 模型支持输出思考过程
3. 新增 - 使用记录导出和仪表盘
4. 新增 - markdown 语法扩展,支持音视频(代码块 audio 和 video
5. 新增 - 调整 max_tokens 计算逻辑。优先保证 max_tokens 为配置值,如超出最大上下文,则减少历史记录。例如:如果申请 8000 的 max_tokens则上下文长度会减少 8000
6. 优化 - 问题优化增加上下文过滤,避免超出上下文。
7. 优化 - 页面组件抽离,减少页面组件路由。
8. 优化 - 全文检索,忽略大小写。
9. 优化 - 问答生成和增强索引改成流输出,避免部分模型超时。
10. 优化 - 自动给 assistant 空 content补充 null同时合并连续的 text assistant避免部分模型抛错。
11. 优化 - 调整图片 Host 取消上传时补充 FE_DOMAIN改成发送对话前补充避免替换域名后原图片无法正常使用。
12. 修复 - 部分场景成员列表无法触底加载。
13. 修复 - 工作流递归执行,部分条件下无法正常运行。
1. 新增 - 可视化模型参数配置。预设超过 100 个模型配置。同时支持所有类型模型的一键测试。(预计下个版本会完全支持在页面上配置渠道)。
2. 新增 - 使用记录导出和仪表盘
3. 新增 - markdown 语法扩展,支持音视频(代码块 audio 和 video
4. 优化 - 页面组件抽离,减少页面组件路由
5. 优化 - 全文检索,忽略大小写
6. 优化 - 问答生成和增强索引改成流输出,避免部分模型超时。

View File

@@ -1,39 +0,0 @@
---
title: 'V4.8.21'
description: 'FastGPT V4.8.21 更新说明'
icon: 'upgrade'
draft: false
toc: true
weight: 803
---
## 更新指南
### 1. 做好数据库备份
### 2. 更新镜像:
- 更新 fastgpt 镜像 tag: v4.8.21-fix
- 更新 fastgpt-pro 商业版镜像 tag: v4.8.21-fix
- Sandbox 镜像无需更新
## 完整更新内容
1. 新增 - 弃用/已删除的插件提示。
2. 新增 - 对话日志按来源分类、标题检索、导出功能。
3. 新增 - 全局变量支持拖拽排序。
4. 新增 - LLM 模型支持 top_p, response_format, json_schema 参数。
5. 新增 - Doubao1.5 模型预设。阿里 embedding3 预设。
6. 新增 - 向量模型支持归一化配置,以便适配未归一化的向量模型,例如 Doubao 的 embedding 模型。
6. 新增 - AI 对话节点,支持输出思考过程结果,可用于其他节点引用。
7. 优化 - 网站嵌入式聊天窗口,增加窗口位置适配。
8. 优化 - 模型未配置时错误提示。
9. 优化 - 适配非 Stream 模式思考输出。
10. 优化 - 增加 TTS voice 未配置时的空指针保护。
11. 优化 - Markdown 链接解析分割规则,改成严格匹配模式,牺牲兼容多种情况,减少误解析。
12. 优化 - 减少未登录用户的数据获取范围,提高系统隐私性。
13. 修复 - 简易模式,切换到其他非视觉模型时候,会强制关闭图片识别。
14. 修复 - o1,o3 模型,在测试时候字段映射未生效导致报错。
15. 修复 - 公众号对话空指针异常。
16. 修复 - 多个音频/视频文件展示异常。
17. 修复 - 分享链接鉴权报错后无限循环。

View File

@@ -1,23 +0,0 @@
---
title: 'V4.8.22(进行中)'
description: 'FastGPT V4.8.22 更新说明'
icon: 'upgrade'
draft: false
toc: true
weight: 802
---
## 完整更新内容
1. 新增 - AI 对话节点解析 <think></think> 标签内容,便于各类模型进行思考链输出。
2. 优化 - 模型未配置时提示,减少冲突提示。
3. 优化 - 使用记录代码。
4. 修复 - 思考内容未进入到输出 Tokens.
5. 修复 - 思考链流输出时,有时与正文顺序偏差。
6. 修复 - API 调用工作流,如果传递的图片不支持 Head 检测时,图片会被过滤。已增加该类错误检测,避免被错误过滤。
7. 修复 - 模板市场部分模板错误。
8. 修复 - 免登录窗口无法正常判断语言识别是否开启。
9. 修复 - 对话日志导出,未兼容 sub path。
10. 修复 - list 接口在联查 member 时,存在空指针可能性。
11. 修复 - 工作流基础节点无法升级。

View File

@@ -7,7 +7,7 @@ toc: true
weight: 234
---
知识库搜索具体参数说明,以及内部逻辑请移步:[FastGPT知识库搜索方案](/docs/guide/knowledge_base/rag/)
知识库搜索具体参数说明,以及内部逻辑请移步:[FastGPT知识库搜索方案](/docs/course/data_search/)
## 特点
@@ -27,7 +27,7 @@ weight: 234
### 输入 - 搜索参数
[点击查看参数介绍](/docs/guide/knowledge_base/dataset_engine/#搜索参数)
[点击查看参数介绍](/docs/course/data_search/#搜索参数)
### 输出 - 引用内容

View File

@@ -20,7 +20,7 @@ weight: 502
![](/imgs/fastgpt-api1.jpg)
{{% alert icon="🍅" context="success" %}}
Tips: 安全起见,你可以设置一个额度或者过期时间,防止 key 被滥用。
Tips: 安全起见,你可以设置一个额度或者过期时间,放置 key 被滥用。
{{% /alert %}}

View File

@@ -114,15 +114,15 @@ services:
# fastgpt
sandbox:
container_name: sandbox
image: ghcr.io/labring/fastgpt-sandbox:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt-sandbox:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.17 # 阿里云
networks:
- fastgpt
restart: always
fastgpt:
container_name: fastgpt
image: ghcr.io/labring/fastgpt:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.17 # 阿里云
ports:
- 3000:3000
networks:

View File

@@ -72,15 +72,15 @@ services:
# fastgpt
sandbox:
container_name: sandbox
image: ghcr.io/labring/fastgpt-sandbox:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt-sandbox:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.17 # 阿里云
networks:
- fastgpt
restart: always
fastgpt:
container_name: fastgpt
image: ghcr.io/labring/fastgpt:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.17 # 阿里云
ports:
- 3000:3000
networks:

View File

@@ -53,15 +53,15 @@ services:
wait $$!
sandbox:
container_name: sandbox
image: ghcr.io/labring/fastgpt-sandbox:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt-sandbox:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.17 # 阿里云
networks:
- fastgpt
restart: always
fastgpt:
container_name: fastgpt
image: ghcr.io/labring/fastgpt:v4.8.21-fix # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.21-fix # 阿里云
image: ghcr.io/labring/fastgpt:v4.8.17 # git
# image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.17 # 阿里云
ports:
- 3000:3000
networks:

View File

@@ -7,7 +7,7 @@
"format-code": "prettier --config \"./.prettierrc.js\" --write \"./**/src/**/*.{ts,tsx,scss}\"",
"format-doc": "zhlint --dir ./docSite *.md --fix",
"gen:theme-typings": "chakra-cli tokens packages/web/styles/theme.ts --out node_modules/.pnpm/node_modules/@chakra-ui/styled-system/dist/theming.types.d.ts",
"postinstall": "pnpm gen:theme-typings",
"postinstall": "sh ./scripts/postinstall.sh",
"initIcon": "node ./scripts/icon/init.js",
"previewIcon": "node ./scripts/icon/index.js",
"api:gen": "tsc ./scripts/openapi/index.ts && node ./scripts/openapi/index.js && npx @redocly/cli build-docs ./scripts/openapi/openapi.json -o ./projects/app/public/openapi/index.html",

View File

@@ -16,8 +16,8 @@ export const bucketNameMap = {
}
};
export const ReadFileBaseUrl = `${process.env.FILE_DOMAIN || process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}/api/common/file/read`;
export const ReadFileBaseUrl = `${process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}/api/common/file/read`;
export const documentFileType = '.txt, .docx, .csv, .xlsx, .pdf, .md, .html, .pptx';
export const imageFileType =
'.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif, .raw, .cr2, .nef, .arw, .dng, .psd, .ai, .eps, .emf, .wmf, .jfif, .exif, .pgm, .ppm, .pbm, .jp2, .j2k, .jpf, .jpx, .jpm, .mj2, .xbm, .pcx';
'.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif';

View File

@@ -1,5 +1,5 @@
import { detect } from 'jschardet';
import { documentFileType } from './constants';
import { documentFileType, imageFileType } from './constants';
import { ChatFileTypeEnum } from '../../core/chat/constants';
import { UserChatItemValueItemType } from '../../core/chat/type';
import * as fs from 'fs';
@@ -25,7 +25,6 @@ export const detectFileEncodingByPath = async (path: string) => {
const fd = await fs.promises.open(path, 'r');
try {
// Read file head
// @ts-ignore
const { bytesRead } = await fd.read(buffer, 0, MAX_BYTES, 0);
const actualBuffer = buffer.slice(0, bytesRead);
@@ -38,49 +37,40 @@ export const detectFileEncodingByPath = async (path: string) => {
// Url => user upload file type
export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file'] | undefined => {
if (typeof url !== 'string') return;
const parseUrl = new URL(url, 'https://locaohost:3000');
// Handle base64 image
if (url.startsWith('data:')) {
const matches = url.match(/^data:([^;]+);base64,/);
if (!matches) return;
const filename = (() => {
// Check base64 image
if (url.startsWith('data:image/')) {
const mime = url.split(',')[0].split(':')[1].split(';')[0];
return `image.${mime.split('/')[1]}`;
}
// Old version file url: https://xxx.com/file/read?filename=xxx.pdf
const filenameQuery = parseUrl.searchParams.get('filename');
if (filenameQuery) return filenameQuery;
const mimeType = matches[1].toLowerCase();
if (!mimeType.startsWith('image/')) return;
// Common file https://xxx.com/xxx.pdf?xxxx=xxx
const pathname = parseUrl.pathname;
if (pathname) return pathname.split('/').pop();
})();
const extension = mimeType.split('/')[1];
if (!filename) return;
const extension = filename.split('.').pop()?.toLowerCase() || '';
if (!extension) return;
if (documentFileType.includes(extension)) {
return {
type: ChatFileTypeEnum.image,
name: `image.${extension}`,
type: ChatFileTypeEnum.file,
name: filename,
url
};
}
try {
const parseUrl = new URL(url, 'https://localhost:3000');
// Get filename from URL
const filename = parseUrl.searchParams.get('filename') || parseUrl.pathname.split('/').pop();
const extension = filename?.split('.').pop()?.toLowerCase() || '';
// If it's a document type, return as file, otherwise treat as image
if (extension && documentFileType.includes(extension)) {
return {
type: ChatFileTypeEnum.file,
name: filename || 'null',
url
};
}
// Default to image type for non-document files
if (imageFileType.includes(extension)) {
return {
type: ChatFileTypeEnum.image,
name: filename || 'null.png',
url
};
} catch (error) {
return {
type: ChatFileTypeEnum.image,
name: 'invalid.png',
name: filename,
url
};
}

View File

@@ -26,7 +26,7 @@ export const simpleText = (text = '') => {
};
export const valToStr = (val: any) => {
if (val === undefined) return '';
if (val === undefined) return 'undefined';
if (val === null) return 'null';
if (typeof val === 'object') return JSON.stringify(val);

View File

@@ -26,19 +26,13 @@ type BaseModelItemType = {
export type LLMModelItemType = PriceType &
BaseModelItemType & {
type: ModelTypeEnum.llm;
// Model params
maxContext: number;
maxResponse: number;
quoteMaxToken: number;
maxTemperature?: number;
showTopP?: boolean;
responseFormatList?: string[];
showStopSign?: boolean;
maxTemperature: number;
censor?: boolean;
vision?: boolean;
reasoning?: boolean;
// diff function model
datasetProcess?: boolean; // dataset
@@ -64,7 +58,6 @@ export type EmbeddingModelItemType = PriceType &
maxToken: number; // model max token
weight: number; // training weight
hidden?: boolean; // Disallow creation
normalization?: boolean; // normalization processing
defaultConfig?: Record<string, any>; // post request config
dbConfig?: Record<string, any>; // Custom parameters for storage
queryConfig?: Record<string, any>; // Custom parameters for query

View File

@@ -61,9 +61,6 @@ export const getModelFromList = (
model: string
) => {
const modelData = modelList.find((item) => item.model === model) ?? modelList[0];
if (!modelData) {
throw new Error('No Key model is configured');
}
const provider = getModelProvider(modelData.provider);
return {
...modelData,

View File

@@ -11,8 +11,8 @@ export type ModelProviderIdType =
| 'AliCloud'
| 'Qwen'
| 'Doubao'
| 'DeepSeek'
| 'ChatGLM'
| 'DeepSeek'
| 'Ernie'
| 'Moonshot'
| 'MiniMax'
@@ -22,7 +22,6 @@ export type ModelProviderIdType =
| 'StepFun'
| 'Yi'
| 'Siliconflow'
| 'PPIO'
| 'Ollama'
| 'BAAI'
| 'FishAudio'
@@ -168,11 +167,6 @@ export const ModelProviderList: ModelProviderType[] = [
name: i18nT('common:model_siliconflow'),
avatar: 'model/siliconflow'
},
{
id: 'PPIO',
name: i18nT('common:model_ppio'),
avatar: 'model/ppio'
},
{
id: 'Other',
name: i18nT('common:model_other'),

View File

@@ -1,12 +1,14 @@
import openai from 'openai';
import type {
ChatCompletionMessageToolCall,
ChatCompletionChunk,
ChatCompletionMessageParam as SdkChatCompletionMessageParam,
ChatCompletionToolMessageParam,
ChatCompletionContentPart as SdkChatCompletionContentPart,
ChatCompletionUserMessageParam as SdkChatCompletionUserMessageParam,
ChatCompletionToolMessageParam as SdkChatCompletionToolMessageParam,
ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam
ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam,
ChatCompletionContentPartText
} from 'openai/resources';
import { ChatMessageTypeEnum } from './constants';
import { WorkflowInteractiveResponseType } from '../workflow/template/system/interactive/type';
@@ -46,7 +48,6 @@ export type ChatCompletionMessageParam = (
| CustomChatCompletionToolMessageParam
| CustomChatCompletionAssistantMessageParam
) & {
reasoning_text?: string;
dataId?: string;
hideInUI?: boolean;
};
@@ -70,8 +71,7 @@ export type ChatCompletionMessageFunctionCall =
};
// Stream response
export type StreamChatType = Stream<openai.Chat.Completions.ChatCompletionChunk>;
export type UnStreamChatType = openai.Chat.Completions.ChatCompletion;
export type StreamChatType = Stream<ChatCompletionChunk>;
export default openai;
export * from 'openai';

View File

@@ -74,17 +74,12 @@ export type AppDetailType = AppSchema & {
export type AppSimpleEditFormType = {
// templateId: string;
aiSettings: {
[NodeInputKeyEnum.aiModel]: string;
[NodeInputKeyEnum.aiSystemPrompt]?: string | undefined;
[NodeInputKeyEnum.aiChatTemperature]?: number;
[NodeInputKeyEnum.aiChatMaxToken]?: number;
[NodeInputKeyEnum.aiChatIsResponseText]: boolean;
model: string;
systemPrompt?: string | undefined;
temperature?: number;
maxToken?: number;
isResponseAnswerText: boolean;
maxHistories: number;
[NodeInputKeyEnum.aiChatReasoning]?: boolean; // Is open reasoning mode
[NodeInputKeyEnum.aiChatTopP]?: number;
[NodeInputKeyEnum.aiChatStopSign]?: string;
[NodeInputKeyEnum.aiChatResponseFormat]?: string;
[NodeInputKeyEnum.aiChatJsonSchema]?: string;
};
dataset: {
datasets: SelectedDatasetType;
@@ -122,11 +117,6 @@ export type SettingAIDataType = {
isResponseAnswerText?: boolean;
maxHistories?: number;
[NodeInputKeyEnum.aiChatVision]?: boolean; // Is open vision mode
[NodeInputKeyEnum.aiChatReasoning]?: boolean; // Is open reasoning mode
[NodeInputKeyEnum.aiChatTopP]?: number;
[NodeInputKeyEnum.aiChatStopSign]?: string;
[NodeInputKeyEnum.aiChatResponseFormat]?: string;
[NodeInputKeyEnum.aiChatJsonSchema]?: string;
};
// variable

View File

@@ -7,8 +7,6 @@ import { StoreNodeItemType } from '../workflow/type/node';
import { DatasetSearchModeEnum } from '../dataset/constants';
import { WorkflowTemplateBasicType } from '../workflow/type';
import { AppTypeEnum } from './constants';
import { AppErrEnum } from '../../common/error/code/app';
import { PluginErrEnum } from '../../common/error/code/plugin';
export const getDefaultAppForm = (): AppSimpleEditFormType => {
return {
@@ -18,8 +16,7 @@ export const getDefaultAppForm = (): AppSimpleEditFormType => {
temperature: 0,
isResponseAnswerText: true,
maxHistories: 6,
maxToken: 4000,
aiChatReasoning: true
maxToken: 4000
},
dataset: {
datasets: [],
@@ -119,8 +116,7 @@ export const appWorkflow2Form = ({
version: node.version,
inputs: node.inputs,
outputs: node.outputs,
templateType: FlowNodeTemplateTypeEnum.other,
pluginData: node.pluginData
templateType: FlowNodeTemplateTypeEnum.other
});
} else if (node.flowNodeType === FlowNodeTypeEnum.systemConfig) {
defaultAppForm.chatConfig = getAppChatConfig({
@@ -150,18 +146,3 @@ export const getAppType = (config?: WorkflowTemplateBasicType | AppSimpleEditFor
}
return '';
};
export const checkAppUnExistError = (error?: string) => {
const unExistError: Array<string> = [
AppErrEnum.unAuthApp,
AppErrEnum.unExist,
PluginErrEnum.unAuth,
PluginErrEnum.unExist
];
if (!!error && unExistError.includes(error)) {
return error;
} else {
return undefined;
}
};

View File

@@ -46,16 +46,7 @@ export const chats2GPTMessages = ({
messages.forEach((item) => {
const dataId = reserveId ? item.dataId : undefined;
if (item.obj === ChatRoleEnum.System) {
const content = item.value?.[0]?.text?.content;
if (content) {
results.push({
dataId,
role: ChatCompletionRequestMessageRoleEnum.System,
content
});
}
} else if (item.obj === ChatRoleEnum.Human) {
if (item.obj === ChatRoleEnum.Human) {
const value = item.value
.map((item) => {
if (item.type === ChatItemValueTypeEnum.text) {
@@ -89,6 +80,15 @@ export const chats2GPTMessages = ({
role: ChatCompletionRequestMessageRoleEnum.User,
content: simpleUserContentPart(value)
});
} else if (item.obj === ChatRoleEnum.System) {
const content = item.value?.[0]?.text?.content;
if (content) {
results.push({
dataId,
role: ChatCompletionRequestMessageRoleEnum.System,
content
});
}
} else {
const aiResults: ChatCompletionMessageParam[] = [];
@@ -349,7 +349,7 @@ export const chatValue2RuntimePrompt = (value: ChatItemValueItemType[]): Runtime
};
value.forEach((item) => {
if (item.type === 'file' && item.file) {
prompt.files.push(item.file);
prompt.files?.push(item.file);
} else if (item.text) {
prompt.text += item.text.content;
}

View File

@@ -25,8 +25,7 @@ export enum ChatItemValueTypeEnum {
text = 'text',
file = 'file',
tool = 'tool',
interactive = 'interactive',
reasoning = 'reasoning'
interactive = 'interactive'
}
export enum ChatSourceEnum {

View File

@@ -70,23 +70,14 @@ export type SystemChatItemType = {
obj: ChatRoleEnum.System;
value: SystemChatItemValueItemType[];
};
export type AIChatItemValueItemType = {
type:
| ChatItemValueTypeEnum.text
| ChatItemValueTypeEnum.reasoning
| ChatItemValueTypeEnum.tool
| ChatItemValueTypeEnum.interactive;
type: ChatItemValueTypeEnum.text | ChatItemValueTypeEnum.tool | ChatItemValueTypeEnum.interactive;
text?: {
content: string;
};
reasoning?: {
content: string;
};
tools?: ToolModuleResponseItemType[];
interactive?: WorkflowInteractiveResponseType;
};
export type AIChatItemType = {
obj: ChatRoleEnum.AI;
value: AIChatItemValueItemType[];

View File

@@ -33,10 +33,8 @@ export enum WorkflowIOValueTypeEnum {
dynamic = 'dynamic',
// plugin special type
selectDataset = 'selectDataset',
// abandon
selectApp = 'selectApp'
selectApp = 'selectApp',
selectDataset = 'selectDataset'
}
export const toolValueTypeList = [
@@ -143,11 +141,6 @@ export enum NodeInputKeyEnum {
aiChatDatasetQuote = 'quoteQA',
aiChatVision = 'aiChatVision',
stringQuoteText = 'stringQuoteText',
aiChatReasoning = 'aiChatReasoning',
aiChatTopP = 'aiChatTopP',
aiChatStopSign = 'aiChatStopSign',
aiChatResponseFormat = 'aiChatResponseFormat',
aiChatJsonSchema = 'aiChatJsonSchema',
// dataset
datasetSelectList = 'datasets',
@@ -160,10 +153,6 @@ export enum NodeInputKeyEnum {
datasetSearchExtensionBg = 'datasetSearchExtensionBg',
collectionFilterMatch = 'collectionFilterMatch',
authTmbId = 'authTmbId',
datasetDeepSearch = 'datasetDeepSearch',
datasetDeepSearchModel = 'datasetDeepSearchModel',
datasetDeepSearchMaxTimes = 'datasetDeepSearchMaxTimes',
datasetDeepSearchBg = 'datasetDeepSearchBg',
// concat dataset
datasetQuoteList = 'system_datasetQuoteList',
@@ -231,8 +220,7 @@ export enum NodeOutputKeyEnum {
// common
userChatInput = 'userChatInput',
history = 'history',
answerText = 'answerText', // node answer. the value will be show and save to history
reasoningText = 'reasoningText', // node reasoning. the value will be show but not save to history
answerText = 'answerText', // module answer. the value will be show and save to history
success = 'success',
failed = 'failed',
error = 'error',

View File

@@ -140,14 +140,7 @@ export enum FlowNodeTypeEnum {
}
// node IO value type
export const FlowValueTypeMap: Record<
WorkflowIOValueTypeEnum,
{
label: string;
value: WorkflowIOValueTypeEnum;
abandon?: boolean;
}
> = {
export const FlowValueTypeMap = {
[WorkflowIOValueTypeEnum.string]: {
label: 'String',
value: WorkflowIOValueTypeEnum.string
@@ -196,6 +189,10 @@ export const FlowValueTypeMap: Record<
label: i18nT('common:core.workflow.Dataset quote'),
value: WorkflowIOValueTypeEnum.datasetQuote
},
[WorkflowIOValueTypeEnum.selectApp]: {
label: i18nT('common:plugin.App'),
value: WorkflowIOValueTypeEnum.selectApp
},
[WorkflowIOValueTypeEnum.selectDataset]: {
label: i18nT('common:core.chat.Select dataset'),
value: WorkflowIOValueTypeEnum.selectDataset
@@ -203,11 +200,6 @@ export const FlowValueTypeMap: Record<
[WorkflowIOValueTypeEnum.dynamic]: {
label: i18nT('common:core.workflow.dynamic_input'),
value: WorkflowIOValueTypeEnum.dynamic
},
[WorkflowIOValueTypeEnum.selectApp]: {
label: 'selectApp',
value: WorkflowIOValueTypeEnum.selectApp,
abandon: true
}
};
@@ -227,6 +219,3 @@ export const datasetQuoteValueDesc = `{
q: string;
a: string
}[]`;
export const datasetSelectValueDesc = `{
datasetId: string;
}[]`;

View File

@@ -123,7 +123,6 @@ export type DispatchNodeResponseType = {
temperature?: number;
maxToken?: number;
quoteList?: SearchDataResponseItemType[];
reasoningText?: string;
historyPreview?: {
obj: `${ChatRoleEnum}`;
value: string;
@@ -134,17 +133,9 @@ export type DispatchNodeResponseType = {
limit?: number;
searchMode?: `${DatasetSearchModeEnum}`;
searchUsingReRank?: boolean;
queryExtensionResult?: {
model: string;
inputTokens: number;
outputTokens: number;
query: string;
};
deepSearchResult?: {
model: string;
inputTokens: number;
outputTokens: number;
};
extensionModel?: string;
extensionResult?: string;
extensionTokens?: number;
// dataset concat
concatLength?: number;
@@ -207,11 +198,6 @@ export type DispatchNodeResponseType = {
// tool params
toolParamsResult?: Record<string, any>;
// abandon
extensionModel?: string;
extensionResult?: string;
extensionTokens?: number;
};
export type DispatchNodeResultType<T = {}> = {
@@ -234,11 +220,6 @@ export type AIChatNodeProps = {
[NodeInputKeyEnum.aiChatMaxToken]?: number;
[NodeInputKeyEnum.aiChatIsResponseText]: boolean;
[NodeInputKeyEnum.aiChatVision]?: boolean;
[NodeInputKeyEnum.aiChatReasoning]?: boolean;
[NodeInputKeyEnum.aiChatTopP]?: number;
[NodeInputKeyEnum.aiChatStopSign]?: string;
[NodeInputKeyEnum.aiChatResponseFormat]?: string;
[NodeInputKeyEnum.aiChatJsonSchema]?: string;
[NodeInputKeyEnum.aiChatQuoteRole]?: AiChatQuoteRoleType;
[NodeInputKeyEnum.aiChatQuoteTemplate]?: string;

View File

@@ -10,7 +10,6 @@ import { FlowNodeOutputItemType, ReferenceValueType } from '../type/io';
import { ChatItemType, NodeOutputItemType } from '../../../core/chat/type';
import { ChatItemValueTypeEnum, ChatRoleEnum } from '../../../core/chat/constants';
import { replaceVariable, valToStr } from '../../../common/string/tools';
import { ChatCompletionChunk } from 'openai/resources';
export const getMaxHistoryLimitFromNodes = (nodes: StoreNodeItemType[]): number => {
let limit = 10;
@@ -293,12 +292,13 @@ export const getReferenceVariableValue = ({
export const formatVariableValByType = (val: any, valueType?: WorkflowIOValueTypeEnum) => {
if (!valueType) return val;
if (val === undefined || val === null) return;
// Value type check, If valueType invalid, return undefined
if (valueType.startsWith('array') && !Array.isArray(val)) return undefined;
if (valueType === WorkflowIOValueTypeEnum.boolean) return Boolean(val);
if (valueType === WorkflowIOValueTypeEnum.number) return Number(val);
if (valueType === WorkflowIOValueTypeEnum.string) {
if (val === undefined) return 'undefined';
if (val === null) return 'null';
return typeof val === 'object' ? JSON.stringify(val) : String(val);
}
if (
@@ -364,14 +364,12 @@ export function replaceEditorVariable({
export const textAdaptGptResponse = ({
text,
reasoning_content,
model = '',
finish_reason = null,
extraData = {}
}: {
model?: string;
text?: string | null;
reasoning_content?: string | null;
text: string | null;
finish_reason?: null | 'stop';
extraData?: Object;
}) => {
@@ -383,11 +381,10 @@ export const textAdaptGptResponse = ({
model,
choices: [
{
delta: {
role: ChatCompletionRequestMessageRoleEnum.Assistant,
content: text,
...(reasoning_content && { reasoning_content })
},
delta:
text === null
? {}
: { role: ChatCompletionRequestMessageRoleEnum.Assistant, content: text },
index: 0,
finish_reason
}
@@ -420,137 +417,3 @@ export function rewriteNodeOutputByHistories(
};
});
}
// Parse <think></think> tags to think and answer - unstream response
export const parseReasoningContent = (text: string): [string, string] => {
const regex = /<think>([\s\S]*?)<\/think>/;
const match = text.match(regex);
if (!match) {
return ['', text];
}
const thinkContent = match[1].trim();
// Add answer (remaining text after think tag)
const answerContent = text.slice(match.index! + match[0].length);
return [thinkContent, answerContent];
};
// Parse <think></think> tags to think and answer - stream response
export const parseReasoningStreamContent = () => {
let isInThinkTag: boolean | undefined;
const startTag = '<think>';
let startTagBuffer = '';
const endTag = '</think>';
let endTagBuffer = '';
/*
parseReasoning - 只控制是否主动解析 <think></think>,如果接口已经解析了,仍然会返回 think 内容。
*/
const parsePart = (
part: {
choices: {
delta: {
content?: string;
reasoning_content?: string;
};
}[];
},
parseReasoning = false
): [string, string] => {
const content = part.choices?.[0]?.delta?.content || '';
// @ts-ignore
const reasoningContent = part.choices?.[0]?.delta?.reasoning_content || '';
if (reasoningContent || !parseReasoning) {
isInThinkTag = false;
return [reasoningContent, content];
}
if (!content) {
return ['', ''];
}
// 如果不在 think 标签中,或者有 reasoningContent(接口已解析),则返回 reasoningContent 和 content
if (isInThinkTag === false) {
return ['', content];
}
// 检测是否为 think 标签开头的数据
if (isInThinkTag === undefined) {
// Parse content think and answer
startTagBuffer += content;
// 太少内容时候,暂时不解析
if (startTagBuffer.length < startTag.length) {
return ['', ''];
}
if (startTagBuffer.startsWith(startTag)) {
isInThinkTag = true;
return [startTagBuffer.slice(startTag.length), ''];
}
// 如果未命中 think 标签,则认为不在 think 标签中,返回 buffer 内容作为 content
isInThinkTag = false;
return ['', startTagBuffer];
}
// 确认是 think 标签内容,开始返回 think 内容,并实时检测 </think>
/*
检测 </think> 方案。
存储所有疑似 </think> 的内容,直到检测到完整的 </think> 标签或超出 </think> 长度。
content 返回值包含以下几种情况:
abc - 完全未命中尾标签
abc<th - 命中一部分尾标签
abc</think> - 完全命中尾标签
abc</think>abc - 完全命中尾标签
</think>abc - 完全命中尾标签
k>abc - 命中一部分尾标签
*/
// endTagBuffer 专门用来记录疑似尾标签的内容
if (endTagBuffer) {
endTagBuffer += content;
if (endTagBuffer.includes(endTag)) {
isInThinkTag = false;
const answer = endTagBuffer.slice(endTag.length);
return ['', answer];
} else if (endTagBuffer.length >= endTag.length) {
// 缓存内容超出尾标签长度,且仍未命中 </think>,则认为本次猜测 </think> 失败,仍处于 think 阶段。
const tmp = endTagBuffer;
endTagBuffer = '';
return [tmp, ''];
}
return ['', ''];
} else if (content.includes(endTag)) {
// 返回内容,完整命中</think>,直接结束
isInThinkTag = false;
const [think, answer] = content.split(endTag);
return [think, answer];
} else {
// 无 buffer且未命中 </think>,开始疑似 </think> 检测。
for (let i = 1; i < endTag.length; i++) {
const partialEndTag = endTag.slice(0, i);
// 命中一部分尾标签
if (content.endsWith(partialEndTag)) {
const think = content.slice(0, -partialEndTag.length);
endTagBuffer += partialEndTag;
return [think, ''];
}
}
}
// 完全未命中尾标签,还是 think 阶段。
return [content, ''];
};
const getStartTagBuffer = () => startTagBuffer;
return {
parsePart,
getStartTagBuffer
};
};

View File

@@ -63,12 +63,14 @@ export const AiChatModule: FlowNodeTemplateType = {
key: NodeInputKeyEnum.aiChatTemperature,
renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
label: '',
value: 0,
valueType: WorkflowIOValueTypeEnum.number
},
{
key: NodeInputKeyEnum.aiChatMaxToken,
renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
label: '',
value: 2000,
valueType: WorkflowIOValueTypeEnum.number
},
@@ -89,37 +91,6 @@ export const AiChatModule: FlowNodeTemplateType = {
valueType: WorkflowIOValueTypeEnum.boolean,
value: true
},
{
key: NodeInputKeyEnum.aiChatReasoning,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.boolean,
value: true
},
{
key: NodeInputKeyEnum.aiChatTopP,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.number
},
{
key: NodeInputKeyEnum.aiChatStopSign,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
{
key: NodeInputKeyEnum.aiChatResponseFormat,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
{
key: NodeInputKeyEnum.aiChatJsonSchema,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
// settings modal ---
{
...Input_Template_System_Prompt,
@@ -130,6 +101,7 @@ export const AiChatModule: FlowNodeTemplateType = {
Input_Template_History,
Input_Template_Dataset_Quote,
Input_Template_File_Link_Prompt,
{ ...Input_Template_UserChatInput, toolDescription: i18nT('workflow:user_question') }
],
outputs: [
@@ -151,20 +123,6 @@ export const AiChatModule: FlowNodeTemplateType = {
description: i18nT('common:core.module.output.description.Ai response content'),
valueType: WorkflowIOValueTypeEnum.string,
type: FlowNodeOutputTypeEnum.static
},
{
id: NodeOutputKeyEnum.reasoningText,
key: NodeOutputKeyEnum.reasoningText,
required: false,
label: i18nT('workflow:reasoning_text'),
valueType: WorkflowIOValueTypeEnum.string,
type: FlowNodeOutputTypeEnum.static,
invalid: true,
invalidCondition: ({ inputs, llmModelList }) => {
const model = inputs.find((item) => item.key === NodeInputKeyEnum.aiModel)?.value;
const modelItem = llmModelList.find((item) => item.model === model);
return modelItem?.reasoning !== true;
}
}
]
};

View File

@@ -1,6 +1,5 @@
import {
datasetQuoteValueDesc,
datasetSelectValueDesc,
FlowNodeInputTypeEnum,
FlowNodeOutputTypeEnum,
FlowNodeTypeEnum
@@ -39,8 +38,7 @@ export const DatasetSearchModule: FlowNodeTemplateType = {
label: i18nT('common:core.module.input.label.Select dataset'),
value: [],
valueType: WorkflowIOValueTypeEnum.selectDataset,
required: true,
valueDesc: datasetSelectValueDesc
required: true
},
{
key: NodeInputKeyEnum.datasetSimilarity,

View File

@@ -43,12 +43,14 @@ export const ToolModule: FlowNodeTemplateType = {
key: NodeInputKeyEnum.aiChatTemperature,
renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
label: '',
value: 0,
valueType: WorkflowIOValueTypeEnum.number
},
{
key: NodeInputKeyEnum.aiChatMaxToken,
renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
label: '',
value: 2000,
valueType: WorkflowIOValueTypeEnum.number
},
{
@@ -58,30 +60,6 @@ export const ToolModule: FlowNodeTemplateType = {
valueType: WorkflowIOValueTypeEnum.boolean,
value: true
},
{
key: NodeInputKeyEnum.aiChatTopP,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.number
},
{
key: NodeInputKeyEnum.aiChatStopSign,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
{
key: NodeInputKeyEnum.aiChatResponseFormat,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
{
key: NodeInputKeyEnum.aiChatJsonSchema,
renderTypeList: [FlowNodeInputTypeEnum.hidden],
label: '',
valueType: WorkflowIOValueTypeEnum.string
},
{
...Input_Template_System_Prompt,

View File

@@ -1,4 +1,3 @@
import { LLMModelItemType } from '../../ai/model.d';
import { LLMModelTypeEnum } from '../../ai/constants';
import { WorkflowIOValueTypeEnum, NodeInputKeyEnum, NodeOutputKeyEnum } from '../constants';
import { FlowNodeInputTypeEnum, FlowNodeOutputTypeEnum } from '../node/constant';
@@ -78,12 +77,6 @@ export type FlowNodeOutputItemType = {
defaultValue?: any;
required?: boolean;
invalid?: boolean;
invalidCondition?: (e: {
inputs: FlowNodeInputItemType[];
llmModelList: LLMModelItemType[];
}) => boolean;
// component params
customFieldConfig?: CustomFieldConfigType;
};

View File

@@ -43,17 +43,6 @@ export type FlowNodeCommonType = {
pluginId?: string;
isFolder?: boolean;
// pluginType?: AppTypeEnum;
pluginData?: PluginDataType;
};
export type PluginDataType = {
version: string;
diagram?: string;
userGuide?: string;
courseUrl?: string;
name?: string;
avatar?: string;
error?: string;
};
type HandleType = {

View File

@@ -10,7 +10,6 @@
"echarts": "5.4.1",
"expr-eval": "^2.0.2",
"lodash": "^4.17.21",
"mssql": "^11.0.1",
"mysql2": "^3.11.3",
"json5": "^2.2.3",
"pg": "^8.10.0",

View File

@@ -1,6 +1,5 @@
import { Client as PgClient } from 'pg'; // PostgreSQL 客户端
import mysql from 'mysql2/promise'; // MySQL 客户端
import mssql from 'mssql'; // SQL Server 客户端
type Props = {
databaseType: string;
@@ -53,20 +52,6 @@ const main = async ({
const [rows] = await connection.execute(sql);
result = rows;
await connection.end();
} else if (databaseType === 'Microsoft SQL Server') {
const pool = await mssql.connect({
server: host,
port: parseInt(port, 10),
database: databaseName,
user,
password,
options: {
trustServerCertificate: true
}
});
result = await pool.query(sql);
await pool.close();
}
return {
result

View File

@@ -42,10 +42,6 @@
{
"label": "PostgreSQL",
"value": "PostgreSQL"
},
{
"label": "Microsoft SQL Server",
"value": "Microsoft SQL Server"
}
],
"required": true

View File

@@ -5,7 +5,6 @@ import { ClientSession, Types } from '../../../common/mongo';
import { guessBase64ImageType } from '../utils';
import { readFromSecondary } from '../../mongo/utils';
import { addHours } from 'date-fns';
import { imageFileType } from '@fastgpt/global/common/file/constants';
export const maxImgSize = 1024 * 1024 * 12;
const base64MimeRegex = /data:image\/([^\)]+);base64/;
@@ -26,19 +25,12 @@ export async function uploadMongoImg({
const [base64Mime, base64Data] = base64Img.split(',');
// Check if mime type is valid
if (!base64MimeRegex.test(base64Mime)) {
return Promise.reject('Invalid image base64');
return Promise.reject('Invalid image mime type');
}
const mime = `image/${base64Mime.match(base64MimeRegex)?.[1] ?? 'image/jpeg'}`;
const binary = Buffer.from(base64Data, 'base64');
let extension = mime.split('/')[1];
if (extension.startsWith('x-')) {
extension = extension.substring(2); // Remove 'x-' prefix
}
if (!extension || !imageFileType.includes(`.${extension}`)) {
return Promise.reject(`Invalid image file type: ${mime}`);
}
const extension = mime.split('/')[1];
const { _id } = await MongoImage.create({
teamId,
@@ -48,7 +40,7 @@ export async function uploadMongoImg({
expiredTime: forever ? undefined : addHours(new Date(), 1)
});
return `${process.env.NEXT_PUBLIC_BASE_URL || ''}${imageBaseUrl}${String(_id)}.${extension}`;
return `${process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}${imageBaseUrl}${String(_id)}.${extension}`;
}
const getIdFromPath = (path?: string) => {

View File

@@ -63,13 +63,6 @@ export const getMongoModel = <T>(name: string, schema: mongoose.Schema) => {
const model = connectionMongo.model<T>(name, schema);
// Sync index
syncMongoIndex(model);
return model;
};
const syncMongoIndex = async (model: Model<any>) => {
if (process.env.SYNC_INDEX !== '0' && process.env.NODE_ENV !== 'test') {
try {
model.syncIndexes({ background: true });
@@ -77,6 +70,8 @@ const syncMongoIndex = async (model: Model<any>) => {
addLog.error('Create index error', error);
}
}
return model;
};
export const ReadPreference = connectionMongo.mongo.ReadPreference;

View File

@@ -25,7 +25,7 @@ export const countGptMessagesTokens = async (
number
>({
name: WorkerNameEnum.countGptMessagesTokens,
maxReservedThreads: global.systemEnv?.tokenWorkers || 30
maxReservedThreads: global.systemEnv?.tokenWorkers || 50
});
const total = await workerController.run({ messages, tools, functionCall });

View File

@@ -24,7 +24,7 @@ export const aiTranscriptions = async ({
? { url: modelData.requestUrl }
: {
baseURL: aiAxiosConfig.baseUrl,
url: '/audio/transcriptions'
url: modelData.requestUrl || '/audio/transcriptions'
}),
headers: {
Authorization: modelData.requestAuth

View File

@@ -1,9 +1,7 @@
import OpenAI from '@fastgpt/global/core/ai';
import {
ChatCompletionCreateParamsNonStreaming,
ChatCompletionCreateParamsStreaming,
StreamChatType,
UnStreamChatType
ChatCompletionCreateParamsStreaming
} from '@fastgpt/global/core/ai/type';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { addLog } from '../../common/system/log';
@@ -40,30 +38,29 @@ export const getAxiosConfig = (props?: { userKey?: OpenaiAccountType }) => {
};
};
export const createChatCompletion = async ({
type CompletionsBodyType =
| ChatCompletionCreateParamsNonStreaming
| ChatCompletionCreateParamsStreaming;
type InferResponseType<T extends CompletionsBodyType> =
T extends ChatCompletionCreateParamsStreaming
? OpenAI.Chat.Completions.ChatCompletionChunk
: OpenAI.Chat.Completions.ChatCompletion;
export const createChatCompletion = async <T extends CompletionsBodyType>({
body,
userKey,
timeout,
options
}: {
body: ChatCompletionCreateParamsNonStreaming | ChatCompletionCreateParamsStreaming;
body: T;
userKey?: OpenaiAccountType;
timeout?: number;
options?: OpenAI.RequestOptions;
}): Promise<
{
getEmptyResponseTip: () => string;
} & (
| {
response: StreamChatType;
isStreamResponse: true;
}
| {
response: UnStreamChatType;
isStreamResponse: false;
}
)
> => {
}): Promise<{
response: InferResponseType<T>;
isStreamResponse: boolean;
getEmptyResponseTip: () => string;
}> => {
try {
const modelConstantsData = getLLMModel(body.model);
@@ -99,17 +96,9 @@ export const createChatCompletion = async ({
return i18nT('chat:LLM_model_response_empty');
};
if (isStreamResponse) {
return {
response,
isStreamResponse: true,
getEmptyResponseTip
};
}
return {
response,
isStreamResponse: false,
response: response as InferResponseType<T>,
isStreamResponse,
getEmptyResponseTip
};
} catch (error) {

View File

@@ -8,12 +8,6 @@
"maxResponse": 4000,
"quoteMaxToken": 120000,
"maxTemperature": 0.99,
"showTopP": true,
"responseFormatList": [
"text",
"json_object"
],
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -36,12 +30,6 @@
"maxResponse": 4000,
"quoteMaxToken": 120000,
"maxTemperature": 0.99,
"showTopP": true,
"responseFormatList": [
"text",
"json_object"
],
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -64,12 +52,6 @@
"maxResponse": 4000,
"quoteMaxToken": 900000,
"maxTemperature": 0.99,
"showTopP": true,
"responseFormatList": [
"text",
"json_object"
],
"showStopSign": true,
"vision": false,
"toolChoice": false,
"functionCall": false,
@@ -92,12 +74,6 @@
"maxResponse": 4000,
"quoteMaxToken": 120000,
"maxTemperature": 0.99,
"showTopP": true,
"responseFormatList": [
"text",
"json_object"
],
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -120,8 +96,6 @@
"maxResponse": 1000,
"quoteMaxToken": 6000,
"maxTemperature": 0.99,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": false,
"functionCall": false,
@@ -144,8 +118,6 @@
"maxResponse": 1000,
"quoteMaxToken": 6000,
"maxTemperature": 0.99,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": false,
"functionCall": false,

View File

@@ -8,8 +8,6 @@
"maxResponse": 8000,
"quoteMaxToken": 100000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -32,8 +30,6 @@
"maxResponse": 8000,
"quoteMaxToken": 100000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": false,
@@ -56,8 +52,6 @@
"maxResponse": 8000,
"quoteMaxToken": 100000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": false,
@@ -80,8 +74,6 @@
"maxResponse": 4096,
"quoteMaxToken": 100000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": false,

View File

@@ -5,12 +5,9 @@
"model": "deepseek-chat",
"name": "Deepseek-chat",
"maxContext": 64000,
"maxResponse": 8000,
"maxResponse": 4096,
"quoteMaxToken": 60000,
"maxTemperature": 1,
"showTopP": true,
"responseFormatList": ["text", "json_object"],
"showStopSign": true,
"maxTemperature": 1.5,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -28,11 +25,10 @@
"model": "deepseek-reasoner",
"name": "Deepseek-reasoner",
"maxContext": 64000,
"maxResponse": 8000,
"maxResponse": 4096,
"quoteMaxToken": 60000,
"maxTemperature": null,
"maxTemperature": 1.5,
"vision": false,
"reasoning": true,
"toolChoice": false,
"functionCall": false,
"defaultSystemChatPrompt": "",
@@ -43,11 +39,11 @@
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"defaultConfig": {
"temperature": null
},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}
}

View File

@@ -1,102 +1,6 @@
{
"provider": "Doubao",
"list": [
{
"model": "Doubao-1.5-lite-32k",
"name": "Doubao-1.5-lite-32k",
"maxContext": 32000,
"maxResponse": 4000,
"quoteMaxToken": 32000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm"
},
{
"model": "Doubao-1.5-pro-32k",
"name": "Doubao-1.5-pro-32k",
"maxContext": 32000,
"maxResponse": 4000,
"quoteMaxToken": 32000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm"
},
{
"model": "Doubao-1.5-pro-256k",
"name": "Doubao-1.5-pro-256k",
"maxContext": 256000,
"maxResponse": 12000,
"quoteMaxToken": 256000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm"
},
{
"model": "Doubao-1.5-vision-pro-32k",
"name": "Doubao-1.5-vision-pro-32k",
"maxContext": 32000,
"maxResponse": 4000,
"quoteMaxToken": 32000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm"
},
{
"model": "Doubao-lite-4k",
"name": "Doubao-lite-4k",
@@ -104,8 +8,6 @@
"maxResponse": 4000,
"quoteMaxToken": 4000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -128,8 +30,6 @@
"maxResponse": 4000,
"quoteMaxToken": 32000,
"maxTemperature": 1,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": false,
@@ -165,9 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-vision-lite-32k",
@@ -189,9 +87,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-pro-4k",
@@ -213,9 +109,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-pro-32k",
@@ -237,9 +131,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-pro-128k",
@@ -261,9 +153,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-vision-pro-32k",
@@ -285,25 +175,21 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Doubao-embedding-large",
"name": "Doubao-embedding-large",
"defaultToken": 512,
"maxToken": 4096,
"type": "embedding",
"normalization": true
"type": "embedding"
},
{
"model": "Doubao-embedding",
"name": "Doubao-embedding",
"defaultToken": 512,
"maxToken": 4096,
"type": "embedding",
"normalization": true
"type": "embedding"
}
]
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "ERNIE-4.0-Turbo-8K",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "ERNIE-Lite-8K",
@@ -69,9 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "ERNIE-Speed-128K",
@@ -93,9 +87,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Embedding-V1",

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gemini-1.5-pro",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gemini-2.0-flash-exp",
@@ -69,9 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gemini-2.0-flash-thinking-exp-1219",
@@ -93,9 +87,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gemini-2.0-flash-thinking-exp-01-21",
@@ -117,9 +109,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gemini-exp-1206",
@@ -141,9 +131,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "text-embedding-004",
@@ -153,4 +141,4 @@
"type": "embedding"
}
]
}
}

View File

@@ -20,9 +20,7 @@
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "llama-3.3-70b-versatile",
@@ -43,9 +41,7 @@
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-lite",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-pro",
@@ -69,9 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-standard",
@@ -93,9 +87,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-turbo-vision",
@@ -117,9 +109,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-turbo",
@@ -141,9 +131,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-vision",
@@ -165,9 +153,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "hunyuan-embedding",

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "internlm3-8b-instruct",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "abab6.5s-chat",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "speech-01-turbo",
@@ -241,4 +237,4 @@
"type": "tts"
}
]
}
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "ministral-8b-latest",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "mistral-large-latest",
@@ -69,9 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "mistral-small-latest",
@@ -93,9 +87,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}

View File

@@ -21,10 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "moonshot-v1-32k",
@@ -46,10 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "moonshot-v1-128k",
@@ -71,10 +65,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
}
]
}

View File

@@ -8,13 +8,6 @@
"maxResponse": 16000,
"quoteMaxToken": 60000,
"maxTemperature": 1.2,
"showTopP": true,
"responseFormatList": [
"text",
"json_object",
"json_schema"
],
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": true,
@@ -36,13 +29,6 @@
"maxResponse": 4000,
"quoteMaxToken": 60000,
"maxTemperature": 1.2,
"showTopP": true,
"responseFormatList": [
"text",
"json_object",
"json_schema"
],
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": true,
@@ -58,44 +44,16 @@
"fieldMap": {},
"type": "llm"
},
{
"model": "o3-mini",
"name": "o3-mini",
"maxContext": 200000,
"maxResponse": 100000,
"quoteMaxToken": 120000,
"maxTemperature": null,
"vision": false,
"toolChoice": true,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {
"stream": false
},
"fieldMap": {
"max_tokens": "max_completion_tokens"
},
"type": "llm",
"showTopP": true,
"showStopSign": true
},
{
"model": "o1-mini",
"name": "o1-mini",
"maxContext": 128000,
"maxResponse": 4000,
"quoteMaxToken": 120000,
"maxTemperature": null,
"maxTemperature": 1.2,
"vision": false,
"toolChoice": false,
"functionCall": false,
"functionCall": true,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
@@ -105,42 +63,10 @@
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {
"stream": false
"temperature": 1,
"max_tokens": null
},
"fieldMap": {
"max_tokens": "max_completion_tokens"
},
"type": "llm",
"showTopP": true,
"showStopSign": true
},
{
"model": "o1",
"name": "o1",
"maxContext": 195000,
"maxResponse": 8000,
"quoteMaxToken": 120000,
"maxTemperature": null,
"vision": true,
"toolChoice": false,
"functionCall": false,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {
"stream": false
},
"fieldMap": {
"max_tokens": "max_completion_tokens"
},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "o1-preview",
@@ -148,10 +74,10 @@
"maxContext": 128000,
"maxResponse": 4000,
"quoteMaxToken": 120000,
"maxTemperature": null,
"maxTemperature": 1.2,
"vision": false,
"toolChoice": false,
"functionCall": false,
"functionCall": true,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
@@ -161,14 +87,36 @@
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {
"temperature": 1,
"max_tokens": null,
"stream": false
},
"fieldMap": {
"max_tokens": "max_completion_tokens"
"type": "llm"
},
{
"model": "o1",
"name": "o1",
"maxContext": 195000,
"maxResponse": 8000,
"quoteMaxToken": 120000,
"maxTemperature": 1.2,
"vision": false,
"toolChoice": false,
"functionCall": true,
"defaultSystemChatPrompt": "",
"datasetProcess": true,
"usedInClassify": true,
"customCQPrompt": "",
"usedInExtractFields": true,
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"defaultConfig": {
"temperature": 1,
"max_tokens": null,
"stream": false
},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "gpt-3.5-turbo",
@@ -177,8 +125,6 @@
"maxResponse": 4000,
"quoteMaxToken": 13000,
"maxTemperature": 1.2,
"showTopP": true,
"showStopSign": true,
"vision": false,
"toolChoice": true,
"functionCall": true,
@@ -199,8 +145,6 @@
"maxResponse": 4000,
"quoteMaxToken": 60000,
"maxTemperature": 1.2,
"showTopP": true,
"showStopSign": true,
"vision": true,
"toolChoice": true,
"functionCall": true,
@@ -275,4 +219,4 @@
"type": "stt"
}
]
}
}

View File

@@ -1,4 +0,0 @@
{
"provider": "PPIO",
"list": []
}

View File

@@ -21,10 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen-plus",
@@ -46,10 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen-vl-plus",
@@ -69,9 +63,7 @@
"usedInQueryExtension": true,
"customExtractPrompt": "",
"usedInToolCall": true,
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "qwen-max",
@@ -93,10 +85,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen-vl-max",
@@ -118,9 +107,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "qwen-coder-turbo",
@@ -142,9 +129,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "qwen2.5-7b-instruct",
@@ -166,10 +151,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen2.5-14b-instruct",
@@ -191,10 +173,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen2.5-32b-instruct",
@@ -216,10 +195,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
"type": "llm"
},
{
"model": "qwen2.5-72b-instruct",
@@ -241,17 +217,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true,
"responseFormatList": ["text", "json_object"]
},
{
"model": "text-embedding-v3",
"name": "text-embedding-v3",
"defaultToken": 512,
"maxToken": 8000,
"type": "embedding"
"type": "llm"
}
]
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "Qwen/Qwen2-VL-72B-Instruct",
@@ -44,9 +42,7 @@
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"defaultConfig": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "deepseek-ai/DeepSeek-V2.5",
@@ -68,9 +64,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "BAAI/bge-m3",
@@ -207,4 +201,4 @@
"type": "rerank"
}
]
}
}

View File

@@ -19,9 +19,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "generalv3",
@@ -41,9 +39,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "pro-128k",
@@ -63,9 +59,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "generalv3.5",
@@ -85,9 +79,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "max-32k",
@@ -109,9 +101,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "4.0Ultra",
@@ -133,9 +123,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}
}

View File

@@ -19,9 +19,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1-8k",
@@ -41,9 +39,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1-32k",
@@ -63,9 +59,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1-128k",
@@ -85,9 +79,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1-256k",
@@ -107,9 +99,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1o-vision-32k",
@@ -129,9 +119,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1v-8k",
@@ -151,9 +139,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-1v-32k",
@@ -173,9 +159,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-2-mini",
@@ -195,9 +179,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-2-16k",
@@ -217,9 +199,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-2-16k-exp",
@@ -239,9 +219,7 @@
"customCQPrompt": "",
"customExtractPrompt": "",
"defaultSystemChatPrompt": "",
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "step-tts-mini",
@@ -327,4 +305,4 @@
"type": "tts"
}
]
}
}

View File

@@ -21,9 +21,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
},
{
"model": "yi-vision-v2",
@@ -45,9 +43,7 @@
"usedInToolCall": true,
"defaultConfig": {},
"fieldMap": {},
"type": "llm",
"showTopP": true,
"showStopSign": true
"type": "llm"
}
]
}

View File

@@ -11,11 +11,7 @@ import {
ReRankModelItemType
} from '@fastgpt/global/core/ai/model.d';
import { debounce } from 'lodash';
import {
getModelProvider,
ModelProviderIdType,
ModelProviderType
} from '@fastgpt/global/core/ai/provider';
import { ModelProviderType } from '@fastgpt/global/core/ai/provider';
import { findModelFromAlldata } from '../model';
import {
reloadFastGPTConfigBuffer,
@@ -31,12 +27,7 @@ import { delay } from '@fastgpt/global/common/system/utils';
export const loadSystemModels = async (init = false) => {
const getProviderList = () => {
const currentFileUrl = new URL(import.meta.url);
const filePath = decodeURIComponent(
process.platform === 'win32'
? currentFileUrl.pathname.substring(1) // Remove leading slash on Windows
: currentFileUrl.pathname
);
const modelsPath = path.join(path.dirname(filePath), 'provider');
const modelsPath = path.join(path.dirname(currentFileUrl.pathname), 'provider');
return fs.readdirSync(modelsPath) as string[];
};
@@ -100,7 +91,7 @@ export const loadSystemModels = async (init = false) => {
await Promise.all(
providerList.map(async (name) => {
const fileContent = (await import(`./provider/${name}`))?.default as {
provider: ModelProviderIdType;
provider: ModelProviderType;
list: SystemModelItemType[];
};
@@ -110,7 +101,7 @@ export const loadSystemModels = async (init = false) => {
const modelData: any = {
...fileModel,
...dbModel?.metadata,
provider: getModelProvider(dbModel?.metadata?.provider || fileContent.provider).id,
provider: dbModel?.metadata?.provider || fileContent.provider,
type: dbModel?.metadata?.type || fileModel.type,
isCustom: false
};
@@ -152,7 +143,6 @@ export const loadSystemModels = async (init = false) => {
console.error('Load models error', error);
// @ts-ignore
global.systemModelList = undefined;
return Promise.reject(error);
}
};

View File

@@ -32,14 +32,12 @@ export async function getVectorsByText({ model, input, type }: GetVectorProps) {
model: model.model,
input: [input]
},
model.requestUrl
model.requestUrl && model.requestAuth
? {
path: model.requestUrl,
headers: model.requestAuth
? {
Authorization: `Bearer ${model.requestAuth}`
}
: undefined
headers: {
Authorization: `Bearer ${model.requestAuth}`
}
}
: {}
)
@@ -56,14 +54,7 @@ export async function getVectorsByText({ model, input, type }: GetVectorProps) {
const [tokens, vectors] = await Promise.all([
countPromptTokens(input),
Promise.all(
res.data
.map((item) => unityDimensional(item.embedding))
.map((item) => {
if (model.normalization) return normalization(item);
return item;
})
)
Promise.all(res.data.map((item) => unityDimensional(item.embedding)))
]);
return {
@@ -94,15 +85,3 @@ function unityDimensional(vector: number[]) {
return resultVector.concat(zeroVector);
}
// normalization processing
function normalization(vector: number[]) {
if (vector.some((item) => item > 1)) {
// Calculate the Euclidean norm (L2 norm)
const norm = Math.sqrt(vector.reduce((sum, val) => sum + val * val, 0));
// Normalize the vector by dividing each component by the norm
return vector.map((val) => val / norm);
}
return vector;
}

View File

@@ -2,12 +2,10 @@ import { replaceVariable } from '@fastgpt/global/common/string/tools';
import { createChatCompletion } from '../config';
import { ChatItemType } from '@fastgpt/global/core/chat/type';
import { countGptMessagesTokens, countPromptTokens } from '../../../common/string/tiktoken/index';
import { chats2GPTMessages } from '@fastgpt/global/core/chat/adapt';
import { chatValue2RuntimePrompt } from '@fastgpt/global/core/chat/adapt';
import { getLLMModel } from '../model';
import { llmCompletionsBodyFormat } from '../utils';
import { addLog } from '../../../common/system/log';
import { filterGPTMessageByMaxContext } from '../../chat/utils';
import json5 from 'json5';
/*
query extension - 问题扩展
@@ -15,73 +13,72 @@ import json5 from 'json5';
*/
const title = global.feConfigs?.systemTitle || 'FastAI';
const defaultPrompt = `## 你的任务
你作为一个向量检索助手,你的任务是结合历史记录,从不同角度,为“原问题”生成个不同版本的“检索词”,从而提高向量检索的语义丰富度,提高向量检索的精度。
const defaultPrompt = `作为一个向量检索助手,你的任务是结合历史记录,从不同角度,为“原问题”生成个不同版本的“检索词”,从而提高向量检索的语义丰富度,提高向量检索的精度。
生成的问题要求指向对象清晰明确,并与“原问题语言相同”。
## 参考示例
参考 <Example></Example> 标中的示例来完成任务。
<Example>
历史记录:
"""
null
"""
原问题: 介绍下剧情。
检索词: ["介绍下故事的背景。","故事的主题是什么?","介绍下故事的主要人物。"]
----------------
历史记录:
"""
user: 对话背景。
assistant: 当前对话是关于 Nginx 的介绍和使用等。
Q: 对话背景。
A: 当前对话是关于 Nginx 的介绍和使用等。
"""
原问题: 怎么下载
检索词: ["Nginx 如何下载?","下载 Nginx 需要什么条件?","有哪些渠道可以下载 Nginx"]
----------------
历史记录:
"""
user: 对话背景。
assistant: 当前对话是关于 Nginx 的介绍和使用等。
user: 报错 "no connection"
assistant: 报错"no connection"可能是因为……
Q: 对话背景。
A: 当前对话是关于 Nginx 的介绍和使用等。
Q: 报错 "no connection"
A: 报错"no connection"可能是因为……
"""
原问题: 怎么解决
检索词: ["Nginx报错"no connection"如何解决?","造成'no connection'报错的原因。","Nginx提示'no connection',要怎么办?"]
----------------
历史记录:
"""
user: How long is the maternity leave?
assistant: The number of days of maternity leave depends on the city in which the employee is located. Please provide your city so that I can answer your questions.
Q: 护产假多少天?
A: 护产假的天数根据员工所在的城市而定。请提供您所在的城市,以便我回答您的问题。
"""
原问题: ShenYang
检索词: ["How many days is maternity leave in Shenyang?","Shenyang's maternity leave policy.","The standard of maternity leave in Shenyang."]
原问题: 沈阳
检索词: ["沈阳的护产假多少天?","沈阳的护产假政策。","沈阳的护产假标准。"]
----------------
历史记录:
"""
user: 作者是谁?
assistant: ${title} 的作者是 labring。
Q: 作者是谁?
A: ${title} 的作者是 labring。
"""
原问题: Tell me about him
检索词: ["Introduce labring, the author of ${title}." ," Background information on author labring." "," Why does labring do ${title}?"]
----------------
历史记录:
"""
user: 对话背景。
assistant: 关于 ${title} 的介绍和使用等问题。
Q: 对话背景。
A: 关于 ${title} 的介绍和使用等问题。
"""
原问题: 你好。
检索词: ["你好"]
----------------
历史记录:
"""
user: ${title} 如何收费?
assistant: ${title} 收费可以参考……
Q: ${title} 如何收费?
A: ${title} 收费可以参考……
"""
原问题: 你知道 laf 么?
检索词: ["laf 的官网地址是多少?","laf 的使用教程。","laf 有什么特点和优势。"]
----------------
历史记录:
"""
user: ${title} 的优势
assistant: 1. 开源
Q: ${title} 的优势
A: 1. 开源
2. 简便
3. 扩展性强
"""
@@ -90,20 +87,18 @@ assistant: 1. 开源
----------------
历史记录:
"""
user: 什么是 ${title}
assistant: ${title} 是一个 RAG 平台。
user: 什么是 Laf
assistant: Laf 是一个云函数开发平台。
Q: 什么是 ${title}
A: ${title} 是一个 RAG 平台。
Q: 什么是 Laf
A: Laf 是一个云函数开发平台。
"""
原问题: 它们有什么关系?
检索词: ["${title}和Laf有什么关系","介绍下${title}","介绍下Laf"]
</Example>
## 输出要求
-----
1. 输出格式为 JSON 数组,数组中每个元素为字符串。无需对输出进行任何解释。
2. 输出语言与原问题相同。原问题为中文则输出中文;原问题为英文则输出英文。
## 开始任务
下面是正式的任务:
历史记录:
"""
@@ -130,39 +125,26 @@ export const queryExtension = async ({
outputTokens: number;
}> => {
const systemFewShot = chatBg
? `user: 对话背景。
assistant: ${chatBg}
? `Q: 对话背景。
A: ${chatBg}
`
: '';
const modelData = getLLMModel(model);
const filterHistories = await filterGPTMessageByMaxContext({
messages: chats2GPTMessages({ messages: histories, reserveId: false }),
maxContext: modelData.maxContext - 1000
});
const historyFewShot = filterHistories
const historyFewShot = histories
.map((item) => {
const role = item.role;
const content = item.content;
if ((role === 'user' || role === 'assistant') && content) {
if (typeof content === 'string') {
return `${role}: ${content}`;
} else {
return `${role}: ${content.map((item) => (item.type === 'text' ? item.text : '')).join('\n')}`;
}
}
const role = item.obj === 'Human' ? 'Q' : 'A';
return `${role}: ${chatValue2RuntimePrompt(item.value).text}`;
})
.filter(Boolean)
.join('\n');
const concatFewShot = `${systemFewShot}${historyFewShot}`.trim();
const modelData = getLLMModel(model);
const messages = [
{
role: 'user',
content: replaceVariable(defaultPrompt, {
query: `${query}`,
histories: concatFewShot || 'null'
histories: concatFewShot
})
}
] as any;
@@ -172,7 +154,7 @@ assistant: ${chatBg}
{
stream: false,
model: modelData.model,
temperature: 0.1,
temperature: 0.01,
messages
},
modelData
@@ -190,41 +172,22 @@ assistant: ${chatBg}
};
}
const start = answer.indexOf('[');
const end = answer.lastIndexOf(']');
if (start === -1 || end === -1) {
addLog.warn('Query extension failed, not a valid JSON', {
answer
});
return {
rawQuery: query,
extensionQueries: [],
model,
inputTokens: 0,
outputTokens: 0
};
}
// Intercept the content of [] and retain []
const jsonStr = answer
.substring(start, end + 1)
.replace(/(\\n|\\)/g, '')
.replace(/ /g, '');
answer = answer.match(/\[.*?\]/)?.[0] || '';
answer = answer.replace(/\\"/g, '"');
try {
const queries = json5.parse(jsonStr) as string[];
const queries = JSON.parse(answer) as string[];
return {
rawQuery: query,
extensionQueries: (Array.isArray(queries) ? queries : []).slice(0, 5),
extensionQueries: Array.isArray(queries) ? queries : [],
model,
inputTokens: await countGptMessagesTokens(messages),
outputTokens: await countPromptTokens(answer)
};
} catch (error) {
addLog.warn('Query extension failed, not a valid JSON', {
answer
});
addLog.error(`Query extension error`, error);
return {
rawQuery: query,
extensionQueries: [],

View File

@@ -25,11 +25,8 @@ export function reRankRecall({
if (!model) {
return Promise.reject('no rerank model');
}
if (documents.length === 0) {
return Promise.resolve([]);
}
const { baseUrl, authorization } = getAxiosConfig();
const { baseUrl, authorization } = getAxiosConfig({});
let start = Date.now();
return POST<PostReRankResponse>(
@@ -41,7 +38,7 @@ export function reRankRecall({
},
{
headers: {
Authorization: model.requestAuth ? `Bearer ${model.requestAuth}` : authorization
Authorization: model.requestAuth ? model.requestAuth : authorization
},
timeout: 30000
}

View File

@@ -2,23 +2,33 @@ import { LLMModelItemType } from '@fastgpt/global/core/ai/model.d';
import {
ChatCompletionCreateParamsNonStreaming,
ChatCompletionCreateParamsStreaming,
ChatCompletionMessageParam,
StreamChatType
} from '@fastgpt/global/core/ai/type';
import { countGptMessagesTokens } from '../../common/string/tiktoken';
import { getLLMModel } from './model';
/*
Count response max token
*/
export const computedMaxToken = ({
export const computedMaxToken = async ({
maxToken,
model
model,
filterMessages = []
}: {
maxToken?: number;
model: LLMModelItemType;
filterMessages: ChatCompletionMessageParam[];
}) => {
if (maxToken === undefined) return;
maxToken = Math.min(maxToken, model.maxResponse);
const tokensLimit = model.maxContext;
/* count response max token */
const promptsToken = await countGptMessagesTokens(filterMessages);
maxToken = promptsToken + maxToken > tokensLimit ? tokensLimit - promptsToken : maxToken;
if (maxToken <= 0) {
maxToken = 200;
}
return maxToken;
};
@@ -30,7 +40,6 @@ export const computedTemperature = ({
model: LLMModelItemType;
temperature: number;
}) => {
if (typeof model.maxTemperature !== 'number') return undefined;
temperature = +(model.maxTemperature * (temperature / 10)).toFixed(2);
temperature = Math.max(temperature, 0.01);
@@ -42,27 +51,17 @@ type CompletionsBodyType =
| ChatCompletionCreateParamsStreaming;
type InferCompletionsBody<T> = T extends { stream: true }
? ChatCompletionCreateParamsStreaming
: T extends { stream: false }
? ChatCompletionCreateParamsNonStreaming
: ChatCompletionCreateParamsNonStreaming | ChatCompletionCreateParamsStreaming;
: ChatCompletionCreateParamsNonStreaming;
export const llmCompletionsBodyFormat = <T extends CompletionsBodyType>(
body: T & {
response_format?: any;
json_schema?: string;
stop?: string;
},
body: T,
model: string | LLMModelItemType
): InferCompletionsBody<T> => {
const modelData = typeof model === 'string' ? getLLMModel(model) : model;
if (!modelData) {
return body as unknown as InferCompletionsBody<T>;
return body as InferCompletionsBody<T>;
}
const response_format = body.response_format;
const json_schema = body.json_schema ?? undefined;
const stop = body.stop ?? undefined;
const requestBody: T = {
...body,
temperature:
@@ -72,14 +71,7 @@ export const llmCompletionsBodyFormat = <T extends CompletionsBodyType>(
temperature: body.temperature
})
: undefined,
...modelData?.defaultConfig,
response_format: response_format
? {
type: response_format,
json_schema
}
: undefined,
stop: stop?.split('|')
...modelData?.defaultConfig
};
// field map
@@ -92,7 +84,9 @@ export const llmCompletionsBodyFormat = <T extends CompletionsBodyType>(
});
}
return requestBody as unknown as InferCompletionsBody<T>;
// console.log(requestBody);
return requestBody as InferCompletionsBody<T>;
};
export const llmStreamResponseToText = async (response: StreamChatType) => {

View File

@@ -88,7 +88,7 @@ try {
ChatSchema.index({ appId: 1, chatId: 1 });
// get chat logs;
ChatSchema.index({ teamId: 1, appId: 1, updateTime: -1, sources: 1 });
ChatSchema.index({ teamId: 1, appId: 1, updateTime: -1 });
// get share chat history
ChatSchema.index({ shareId: 1, outLinkUid: 1, updateTime: -1 });

View File

@@ -1,9 +1,6 @@
import { countGptMessagesTokens } from '../../common/string/tiktoken/index';
import type {
ChatCompletionAssistantMessageParam,
ChatCompletionContentPart,
ChatCompletionContentPartRefusal,
ChatCompletionContentPartText,
ChatCompletionMessageParam,
SdkChatCompletionMessageParam
} from '@fastgpt/global/core/ai/type.d';
@@ -14,19 +11,36 @@ import { serverRequestBaseUrl } from '../../common/api/serverRequest';
import { i18nT } from '../../../web/i18n/utils';
import { addLog } from '../../common/system/log';
export const filterGPTMessageByMaxContext = async ({
export const filterGPTMessageByMaxTokens = async ({
messages = [],
maxContext
maxTokens
}: {
messages: ChatCompletionMessageParam[];
maxContext: number;
maxTokens: number;
}) => {
if (!Array.isArray(messages)) {
return [];
}
const rawTextLen = messages.reduce((sum, item) => {
if (typeof item.content === 'string') {
return sum + item.content.length;
}
if (Array.isArray(item.content)) {
return (
sum +
item.content.reduce((sum, item) => {
if (item.type === 'text') {
return sum + item.text.length;
}
return sum;
}, 0)
);
}
return sum;
}, 0);
// If the text length is less than half of the maximum token, no calculation is required
if (messages.length < 4) {
if (rawTextLen < maxTokens * 0.5) {
return messages;
}
@@ -38,7 +52,7 @@ export const filterGPTMessageByMaxContext = async ({
const chatPrompts: ChatCompletionMessageParam[] = messages.slice(chatStartIndex);
// reduce token of systemPrompt
maxContext -= await countGptMessagesTokens(systemPrompts);
maxTokens -= await countGptMessagesTokens(systemPrompts);
// Save the last chat prompt(question)
const question = chatPrompts.pop();
@@ -56,9 +70,9 @@ export const filterGPTMessageByMaxContext = async ({
}
const tokens = await countGptMessagesTokens([assistant, user]);
maxContext -= tokens;
maxTokens -= tokens;
/* 整体 tokens 超出范围,截断 */
if (maxContext < 0) {
if (maxTokens < 0) {
break;
}
@@ -88,331 +102,223 @@ export const loadRequestMessages = async ({
useVision?: boolean;
origin?: string;
}) => {
const replaceLinkUrl = (text: string) => {
const baseURL = process.env.FE_DOMAIN;
if (!baseURL) return text;
// 匹配 /api/system/img/xxx.xx 的图片链接,并追加 baseURL
return text.replace(
/(?<!https?:\/\/[^\s]*)(?:\/api\/system\/img\/[^\s.]*\.[^\s]*)/g,
(match) => `${baseURL}${match}`
);
// Load image to base64
const loadImageToBase64 = async (messages: ChatCompletionContentPart[]) => {
return Promise.all(
messages.map(async (item) => {
if (item.type === 'image_url') {
// Remove url origin
const imgUrl = (() => {
if (origin && item.image_url.url.startsWith(origin)) {
return item.image_url.url.replace(origin, '');
}
return item.image_url.url;
})();
// base64 image
if (imgUrl.startsWith('data:image/')) {
return item;
}
try {
// If imgUrl is a local path, load image from local, and set url to base64
if (imgUrl.startsWith('/') || process.env.MULTIPLE_DATA_TO_BASE64 === 'true') {
addLog.debug('Load image from local server', {
baseUrl: serverRequestBaseUrl,
requestUrl: imgUrl
});
const response = await axios.get(imgUrl, {
baseURL: serverRequestBaseUrl,
responseType: 'arraybuffer',
proxy: false
});
const base64 = Buffer.from(response.data, 'binary').toString('base64');
const imageType =
getFileContentTypeFromHeader(response.headers['content-type']) ||
guessBase64ImageType(base64);
return {
...item,
image_url: {
...item.image_url,
url: `data:${imageType};base64,${base64}`
}
};
}
// 检查下这个图片是否可以被访问,如果不行的话,则过滤掉
const response = await axios.head(imgUrl, {
timeout: 10000
});
if (response.status < 200 || response.status >= 400) {
addLog.info(`Filter invalid image: ${imgUrl}`);
return;
}
} catch (error) {
return;
}
}
return item;
})
).then((res) => res.filter(Boolean) as ChatCompletionContentPart[]);
};
const parseSystemMessage = (
content: string | ChatCompletionContentPartText[]
): string | ChatCompletionContentPartText[] | undefined => {
if (typeof content === 'string') {
if (!content) return;
return replaceLinkUrl(content);
// Split question text and image
const parseStringWithImages = (input: string): ChatCompletionContentPart[] => {
if (!useVision || input.length > 500) {
return [{ type: 'text', text: input || '' }];
}
const arrayContent = content
.filter((item) => item.text)
.map((item) => ({ ...item, text: replaceLinkUrl(item.text) }));
if (arrayContent.length === 0) return;
return arrayContent;
// 正则表达式匹配图片URL
const imageRegex =
/(https?:\/\/[^\s/$.?#].[^\s]*\.(?:png|jpe?g|gif|webp|bmp|tiff?|svg|ico|heic|avif))/gi;
const result: ChatCompletionContentPart[] = [];
// 提取所有HTTPS图片URL并添加到result开头
const httpsImages = [...new Set(Array.from(input.matchAll(imageRegex), (m) => m[0]))];
httpsImages.forEach((url) => {
result.push({
type: 'image_url',
image_url: {
url: url
}
});
});
// Too many images return text
if (httpsImages.length > 4) {
return [{ type: 'text', text: input || '' }];
}
// 添加原始input作为文本
result.push({ type: 'text', text: input });
return result;
};
// Parse user content(text and img) Store history => api messages
const parseUserContent = async (content: string | ChatCompletionContentPart[]) => {
// Split question text and image
const parseStringWithImages = (input: string): ChatCompletionContentPart[] => {
if (!useVision || input.length > 500) {
return [{ type: 'text', text: input }];
}
// 正则表达式匹配图片URL
const imageRegex =
/(https?:\/\/[^\s/$.?#].[^\s]*\.(?:png|jpe?g|gif|webp|bmp|tiff?|svg|ico|heic|avif))/gi;
const result: ChatCompletionContentPart[] = [];
// 提取所有HTTPS图片URL并添加到result开头
const httpsImages = [...new Set(Array.from(input.matchAll(imageRegex), (m) => m[0]))];
httpsImages.forEach((url) => {
result.push({
type: 'image_url',
image_url: {
url: url
}
});
});
// Too many images return text
if (httpsImages.length > 4) {
return [{ type: 'text', text: input }];
}
// 添加原始input作为文本
result.push({ type: 'text', text: input });
return result;
};
// Load image to base64
const loadUserContentImage = async (content: ChatCompletionContentPart[]) => {
return Promise.all(
content.map(async (item) => {
if (item.type === 'image_url') {
// Remove url origin
const imgUrl = (() => {
if (origin && item.image_url.url.startsWith(origin)) {
return item.image_url.url.replace(origin, '');
}
return item.image_url.url;
})();
// base64 image
if (imgUrl.startsWith('data:image/')) {
return item;
}
try {
// If imgUrl is a local path, load image from local, and set url to base64
if (imgUrl.startsWith('/') || process.env.MULTIPLE_DATA_TO_BASE64 === 'true') {
addLog.debug('Load image from local server', {
baseUrl: serverRequestBaseUrl,
requestUrl: imgUrl
});
const response = await axios.get(imgUrl, {
baseURL: serverRequestBaseUrl,
responseType: 'arraybuffer',
proxy: false
});
const base64 = Buffer.from(response.data, 'binary').toString('base64');
const imageType =
getFileContentTypeFromHeader(response.headers['content-type']) ||
guessBase64ImageType(base64);
return {
...item,
image_url: {
...item.image_url,
url: `data:${imageType};base64,${base64}`
}
};
}
// 检查下这个图片是否可以被访问,如果不行的话,则过滤掉
const response = await axios.head(imgUrl, {
timeout: 10000
});
if (response.status < 200 || response.status >= 400) {
addLog.info(`Filter invalid image: ${imgUrl}`);
return;
}
} catch (error: any) {
if (error?.response?.status === 405) {
return item;
}
addLog.warn(`Filter invalid image: ${imgUrl}`, { error });
return;
}
}
return item;
})
).then((res) => res.filter(Boolean) as ChatCompletionContentPart[]);
};
if (content === undefined) return;
if (typeof content === 'string') {
if (content === '') return;
const loadImageContent = await loadUserContentImage(parseStringWithImages(content));
if (loadImageContent.length === 0) return;
return loadImageContent;
return loadImageToBase64(parseStringWithImages(content));
}
const result = (
await Promise.all(
content.map(async (item) => {
if (item.type === 'text') {
if (item.text) return parseStringWithImages(item.text);
return;
}
if (item.type === 'file_url') return; // LLM not support file_url
if (item.type === 'image_url') {
// close vision, remove image_url
if (!useVision) return;
// remove empty image_url
if (!item.image_url.url) return;
const result = await Promise.all(
content.map(async (item) => {
if (item.type === 'text') return parseStringWithImages(item.text);
if (item.type === 'file_url') return; // LLM not support file_url
if (!item.image_url.url) return item;
return item;
})
);
return loadImageToBase64(result.flat().filter(Boolean) as ChatCompletionContentPart[]);
};
// format GPT messages, concat text messages
const clearInvalidMessages = (messages: ChatCompletionMessageParam[]) => {
return messages
.map((item) => {
if (item.role === ChatCompletionRequestMessageRoleEnum.System && !item.content) {
return;
}
if (item.role === ChatCompletionRequestMessageRoleEnum.User) {
if (item.content === undefined) return;
if (typeof item.content === 'string') {
return {
...item,
content: item.content.trim()
};
}
return item;
})
)
)
.flat()
.filter(Boolean) as ChatCompletionContentPart[];
// array
if (item.content.length === 0) return;
if (item.content.length === 1 && item.content[0].type === 'text') {
return {
...item,
content: item.content[0].text
};
}
}
if (item.role === ChatCompletionRequestMessageRoleEnum.Assistant) {
if (item.content === undefined && !item.tool_calls && !item.function_call) return;
}
const loadImageContent = await loadUserContentImage(result);
if (loadImageContent.length === 0) return;
return loadImageContent;
return item;
})
.filter(Boolean) as ChatCompletionMessageParam[];
};
const formatAssistantItem = (item: ChatCompletionAssistantMessageParam) => {
return {
role: item.role,
content: item.content,
function_call: item.function_call,
name: item.name,
refusal: item.refusal,
tool_calls: item.tool_calls
};
};
const parseAssistantContent = (
content:
| string
| (ChatCompletionContentPartText | ChatCompletionContentPartRefusal)[]
| null
| undefined
) => {
if (typeof content === 'string') {
return content || '';
}
// 交互节点
if (!content) return '';
const result = content.filter((item) => item?.type === 'text');
if (result.length === 0) return '';
return result.map((item) => item.text).join('\n');
};
if (messages.length === 0) {
return Promise.reject(i18nT('common:core.chat.error.Messages empty'));
}
// 合并相邻 role 的内容,只保留一个 role content 变成数组。 assistant 的话,工具调用不合并。
const mergeMessages = ((messages: ChatCompletionMessageParam[]): ChatCompletionMessageParam[] => {
/*
Merge data for some consecutive roles
1. Contiguous assistant and both have content, merge content
*/
const mergeConsecutiveMessages = (
messages: ChatCompletionMessageParam[]
): ChatCompletionMessageParam[] => {
return messages.reduce((mergedMessages: ChatCompletionMessageParam[], currentMessage) => {
const lastMessage = mergedMessages[mergedMessages.length - 1];
if (!lastMessage) {
return [currentMessage];
}
if (
lastMessage.role === ChatCompletionRequestMessageRoleEnum.System &&
currentMessage.role === ChatCompletionRequestMessageRoleEnum.System
) {
const lastContent: ChatCompletionContentPartText[] = Array.isArray(lastMessage.content)
? lastMessage.content
: [{ type: 'text', text: lastMessage.content || '' }];
const currentContent: ChatCompletionContentPartText[] = Array.isArray(
currentMessage.content
)
? currentMessage.content
: [{ type: 'text', text: currentMessage.content || '' }];
lastMessage.content = [...lastContent, ...currentContent];
} // Handle user messages
else if (
lastMessage.role === ChatCompletionRequestMessageRoleEnum.User &&
currentMessage.role === ChatCompletionRequestMessageRoleEnum.User
) {
const lastContent: ChatCompletionContentPart[] = Array.isArray(lastMessage.content)
? lastMessage.content
: [{ type: 'text', text: lastMessage.content }];
const currentContent: ChatCompletionContentPart[] = Array.isArray(currentMessage.content)
? currentMessage.content
: [{ type: 'text', text: currentMessage.content }];
lastMessage.content = [...lastContent, ...currentContent];
} else if (
lastMessage &&
currentMessage.role === ChatCompletionRequestMessageRoleEnum.Assistant &&
lastMessage.role === ChatCompletionRequestMessageRoleEnum.Assistant &&
currentMessage.role === ChatCompletionRequestMessageRoleEnum.Assistant
typeof lastMessage.content === 'string' &&
typeof currentMessage.content === 'string'
) {
// Content 不为空的对象,或者是交互节点
if (
(typeof lastMessage.content === 'string' ||
Array.isArray(lastMessage.content) ||
lastMessage.interactive) &&
(typeof currentMessage.content === 'string' ||
Array.isArray(currentMessage.content) ||
currentMessage.interactive)
) {
const lastContent: (ChatCompletionContentPartText | ChatCompletionContentPartRefusal)[] =
Array.isArray(lastMessage.content)
? lastMessage.content
: [{ type: 'text', text: lastMessage.content || '' }];
const currentContent: (
| ChatCompletionContentPartText
| ChatCompletionContentPartRefusal
)[] = Array.isArray(currentMessage.content)
? currentMessage.content
: [{ type: 'text', text: currentMessage.content || '' }];
lastMessage.content = [...lastContent, ...currentContent];
} else {
// 有其中一个没有 content说明不是连续的文本输出
mergedMessages.push(currentMessage);
}
lastMessage.content += currentMessage ? `\n${currentMessage.content}` : '';
} else {
mergedMessages.push(currentMessage);
}
return mergedMessages;
}, []);
})(messages);
};
const loadMessages = (
await Promise.all(
mergeMessages.map(async (item, i) => {
if (item.role === ChatCompletionRequestMessageRoleEnum.System) {
const content = parseSystemMessage(item.content);
if (!content) return;
return {
...item,
content
};
} else if (item.role === ChatCompletionRequestMessageRoleEnum.User) {
const content = await parseUserContent(item.content);
if (!content) {
return {
...item,
content: 'null'
};
}
if (messages.length === 0) {
return Promise.reject(i18nT('common:core.chat.error.Messages empty'));
}
const formatContent = (() => {
if (Array.isArray(content) && content.length === 1 && content[0].type === 'text') {
return content[0].text;
}
return content;
})();
// filter messages file
const filterMessages = messages.map((item) => {
// If useVision=false, only retain text.
if (
item.role === ChatCompletionRequestMessageRoleEnum.User &&
Array.isArray(item.content) &&
!useVision
) {
return {
...item,
content: item.content.filter((item) => item.type === 'text')
};
}
return {
...item,
content: formatContent
};
} else if (item.role === ChatCompletionRequestMessageRoleEnum.Assistant) {
if (item.tool_calls || item.function_call) {
return formatAssistantItem(item);
}
return item;
});
const parseContent = parseAssistantContent(item.content);
const loadMessages = (await Promise.all(
filterMessages.map(async (item) => {
if (item.role === ChatCompletionRequestMessageRoleEnum.User) {
return {
...item,
content: await parseUserContent(item.content)
};
} else if (item.role === ChatCompletionRequestMessageRoleEnum.Assistant) {
// remove invalid field
return {
role: item.role,
content: item.content,
function_call: item.function_call,
name: item.name,
refusal: item.refusal,
tool_calls: item.tool_calls
};
} else {
return item;
}
})
)) as ChatCompletionMessageParam[];
// 如果内容为空,且前后不再是 assistant需要补充成 null避免丢失 user-assistant 的交互
const formatContent = (() => {
const lastItem = mergeMessages[i - 1];
const nextItem = mergeMessages[i + 1];
if (
parseContent === '' &&
(lastItem?.role === ChatCompletionRequestMessageRoleEnum.Assistant ||
nextItem?.role === ChatCompletionRequestMessageRoleEnum.Assistant)
) {
return;
}
return parseContent || 'null';
})();
if (!formatContent) return;
return {
...formatAssistantItem(item),
content: formatContent
};
} else {
return item;
}
})
)
).filter(Boolean) as ChatCompletionMessageParam[];
return loadMessages as SdkChatCompletionMessageParam[];
return mergeConsecutiveMessages(
clearInvalidMessages(loadMessages)
) as SdkChatCompletionMessageParam[];
};

View File

@@ -37,7 +37,12 @@ try {
{ teamId: 1, datasetId: 1, fullTextToken: 'text' },
{
name: 'teamId_1_datasetId_1_fullTextToken_text',
default_language: 'none'
default_language: 'none',
collation: {
locale: 'simple', // 使用简单匹配规则
strength: 2, // 忽略大小写
caseLevel: false // 进一步确保大小写不敏感
}
}
);
DatasetDataTextSchema.index({ dataId: 1 }, { unique: true });

View File

@@ -5,7 +5,7 @@ import {
} from '@fastgpt/global/core/dataset/constants';
import { recallFromVectorStore } from '../../../common/vectorStore/controller';
import { getVectorsByText } from '../../ai/embedding';
import { getEmbeddingModel, getDefaultRerankModel, getLLMModel } from '../../ai/model';
import { getEmbeddingModel, getDefaultRerankModel } from '../../ai/model';
import { MongoDatasetData } from '../data/schema';
import {
DatasetDataTextSchemaType,
@@ -23,24 +23,18 @@ import json5 from 'json5';
import { MongoDatasetCollectionTags } from '../tag/schema';
import { readFromSecondary } from '../../../common/mongo/utils';
import { MongoDatasetDataText } from '../data/dataTextSchema';
import { ChatItemType } from '@fastgpt/global/core/chat/type';
import { POST } from '../../../common/api/plusRequest';
import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants';
import { datasetSearchQueryExtension } from './utils';
export type SearchDatasetDataProps = {
histories: ChatItemType[];
type SearchDatasetDataProps = {
teamId: string;
model: string;
similarity?: number; // min distance
limit: number; // max Token limit
datasetIds: string[];
searchMode?: `${DatasetSearchModeEnum}`;
usingReRank?: boolean;
reRankQuery: string;
queries: string[];
[NodeInputKeyEnum.datasetSimilarity]?: number; // min distance
[NodeInputKeyEnum.datasetMaxTokens]: number; // max Token limit
[NodeInputKeyEnum.datasetSearchMode]?: `${DatasetSearchModeEnum}`;
[NodeInputKeyEnum.datasetSearchUsingReRank]?: boolean;
/*
{
tags: {
@@ -56,96 +50,7 @@ export type SearchDatasetDataProps = {
collectionFilterMatch?: string;
};
export type SearchDatasetDataResponse = {
searchRes: SearchDataResponseItemType[];
tokens: number;
searchMode: `${DatasetSearchModeEnum}`;
limit: number;
similarity: number;
usingReRank: boolean;
usingSimilarityFilter: boolean;
queryExtensionResult?: {
model: string;
inputTokens: number;
outputTokens: number;
query: string;
};
deepSearchResult?: { model: string; inputTokens: number; outputTokens: number };
};
export const datasetDataReRank = async ({
data,
query
}: {
data: SearchDataResponseItemType[];
query: string;
}): Promise<SearchDataResponseItemType[]> => {
const results = await reRankRecall({
query,
documents: data.map((item) => ({
id: item.id,
text: `${item.q}\n${item.a}`
}))
});
if (results.length === 0) {
return Promise.reject('Rerank error');
}
// add new score to data
const mergeResult = results
.map((item, index) => {
const target = data.find((dataItem) => dataItem.id === item.id);
if (!target) return null;
const score = item.score || 0;
return {
...target,
score: [{ type: SearchScoreTypeEnum.reRank, value: score, index }]
};
})
.filter(Boolean) as SearchDataResponseItemType[];
return mergeResult;
};
export const filterDatasetDataByMaxTokens = async (
data: SearchDataResponseItemType[],
maxTokens: number
) => {
const filterMaxTokensResult = await (async () => {
// Count tokens
const tokensScoreFilter = await Promise.all(
data.map(async (item) => ({
...item,
tokens: await countPromptTokens(item.q + item.a)
}))
);
const results: SearchDataResponseItemType[] = [];
let totalTokens = 0;
for await (const item of tokensScoreFilter) {
totalTokens += item.tokens;
if (totalTokens > maxTokens + 500) {
break;
}
results.push(item);
if (totalTokens > maxTokens) {
break;
}
}
return results.length === 0 ? data.slice(0, 1) : results;
})();
return filterMaxTokensResult;
};
export async function searchDatasetData(
props: SearchDatasetDataProps
): Promise<SearchDatasetDataResponse> {
export async function searchDatasetData(props: SearchDatasetDataProps) {
let {
teamId,
reRankQuery,
@@ -550,6 +455,47 @@ export async function searchDatasetData(
tokenLen: 0
};
};
const reRankSearchResult = async ({
data,
query
}: {
data: SearchDataResponseItemType[];
query: string;
}): Promise<SearchDataResponseItemType[]> => {
try {
const results = await reRankRecall({
query,
documents: data.map((item) => ({
id: item.id,
text: `${item.q}\n${item.a}`
}))
});
if (results.length === 0) {
usingReRank = false;
return [];
}
// add new score to data
const mergeResult = results
.map((item, index) => {
const target = data.find((dataItem) => dataItem.id === item.id);
if (!target) return null;
const score = item.score || 0;
return {
...target,
score: [{ type: SearchScoreTypeEnum.reRank, value: score, index }]
};
})
.filter(Boolean) as SearchDataResponseItemType[];
return mergeResult;
} catch (error) {
usingReRank = false;
return [];
}
};
const multiQueryRecall = async ({
embeddingLimit,
fullTextLimit
@@ -634,15 +580,10 @@ export async function searchDatasetData(
set.add(str);
return true;
});
try {
return await datasetDataReRank({
query: reRankQuery,
data: filterSameDataResults
});
} catch (error) {
usingReRank = false;
return [];
}
return reRankSearchResult({
query: reRankQuery,
data: filterSameDataResults
});
})();
// embedding recall and fullText recall rrf concat
@@ -687,7 +628,31 @@ export async function searchDatasetData(
})();
// token filter
const filterMaxTokensResult = await filterDatasetDataByMaxTokens(scoreFilter, maxTokens);
const filterMaxTokensResult = await (async () => {
const tokensScoreFilter = await Promise.all(
scoreFilter.map(async (item) => ({
...item,
tokens: await countPromptTokens(item.q + item.a)
}))
);
const results: SearchDataResponseItemType[] = [];
let totalTokens = 0;
for await (const item of tokensScoreFilter) {
totalTokens += item.tokens;
if (totalTokens > maxTokens + 500) {
break;
}
results.push(item);
if (totalTokens > maxTokens) {
break;
}
}
return results.length === 0 ? scoreFilter.slice(0, 1) : results;
})();
return {
searchRes: filterMaxTokensResult,
@@ -699,53 +664,3 @@ export async function searchDatasetData(
usingSimilarityFilter
};
}
export type DefaultSearchDatasetDataProps = SearchDatasetDataProps & {
[NodeInputKeyEnum.datasetSearchUsingExtensionQuery]?: boolean;
[NodeInputKeyEnum.datasetSearchExtensionModel]?: string;
[NodeInputKeyEnum.datasetSearchExtensionBg]?: string;
};
export const defaultSearchDatasetData = async ({
datasetSearchUsingExtensionQuery,
datasetSearchExtensionModel,
datasetSearchExtensionBg,
...props
}: DefaultSearchDatasetDataProps): Promise<SearchDatasetDataResponse> => {
const query = props.queries[0];
const extensionModel = datasetSearchUsingExtensionQuery
? getLLMModel(datasetSearchExtensionModel)
: undefined;
const { concatQueries, rewriteQuery, aiExtensionResult } = await datasetSearchQueryExtension({
query,
extensionModel,
extensionBg: datasetSearchExtensionBg
});
const result = await searchDatasetData({
...props,
reRankQuery: rewriteQuery,
queries: concatQueries
});
return {
...result,
queryExtensionResult: aiExtensionResult
? {
model: aiExtensionResult.model,
inputTokens: aiExtensionResult.inputTokens,
outputTokens: aiExtensionResult.outputTokens,
query: concatQueries.join('\n')
}
: undefined
};
};
export type DeepRagSearchProps = SearchDatasetDataProps & {
[NodeInputKeyEnum.datasetDeepSearchModel]?: string;
[NodeInputKeyEnum.datasetDeepSearchMaxTimes]?: number;
[NodeInputKeyEnum.datasetDeepSearchBg]?: string;
};
export const deepRagSearch = (data: DeepRagSearchProps) =>
POST<SearchDatasetDataResponse>('/core/dataset/deepRag', data);

View File

@@ -1,5 +1,45 @@
import { DatasetTrainingSchemaType } from '@fastgpt/global/core/dataset/type';
import { addLog } from '../../../common/system/log';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { MongoDatasetTraining } from './schema';
import Papa from 'papaparse';
export const checkInvalidChunkAndLock = async ({
err,
errText,
data
}: {
err: any;
errText: string;
data: DatasetTrainingSchemaType;
}) => {
if (err?.response) {
addLog.error(`openai error: ${errText}`, {
status: err.response?.status,
statusText: err.response?.statusText,
data: err.response?.data
});
} else {
addLog.error(getErrText(err, errText), err);
}
if (
err?.message === 'invalid message format' ||
err?.type === 'invalid_request_error' ||
err?.code === 500
) {
addLog.error('Lock training data', err);
try {
await MongoDatasetTraining.findByIdAndUpdate(data._id, {
lockTime: new Date('2998/5/5')
});
} catch (error) {}
return true;
}
return false;
};
export const parseCsvTable2Chunks = (rawText: string) => {
const csvArr = Papa.parse(rawText).data as string[][];

View File

@@ -1,5 +1,5 @@
import { chats2GPTMessages } from '@fastgpt/global/core/chat/adapt';
import { filterGPTMessageByMaxContext, loadRequestMessages } from '../../../chat/utils';
import { filterGPTMessageByMaxTokens, loadRequestMessages } from '../../../chat/utils';
import type { ChatItemType } from '@fastgpt/global/core/chat/type.d';
import {
countMessagesTokens,
@@ -175,9 +175,9 @@ ${description ? `- ${description}` : ''}
}
];
const adaptMessages = chats2GPTMessages({ messages, reserveId: false });
const filterMessages = await filterGPTMessageByMaxContext({
const filterMessages = await filterGPTMessageByMaxTokens({
messages: adaptMessages,
maxContext: extractModel.maxContext
maxTokens: extractModel.maxContext
});
const requestMessages = await loadRequestMessages({
messages: filterMessages,

View File

@@ -1,5 +1,5 @@
import { createChatCompletion } from '../../../../ai/config';
import { filterGPTMessageByMaxContext, loadRequestMessages } from '../../../../chat/utils';
import { filterGPTMessageByMaxTokens, loadRequestMessages } from '../../../../chat/utils';
import {
ChatCompletion,
StreamChatType,
@@ -46,15 +46,7 @@ export const runToolWithFunctionCall = async (
externalProvider,
stream,
workflowStreamResponse,
params: {
temperature,
maxToken,
aiChatVision,
aiChatTopP,
aiChatStopSign,
aiChatResponseFormat,
aiChatJsonSchema
}
params: { temperature, maxToken, aiChatVision }
} = workflowProps;
// Interactive
@@ -180,14 +172,10 @@ export const runToolWithFunctionCall = async (
};
});
const max_tokens = computedMaxToken({
model: toolModel,
maxToken
});
const filterMessages = (
await filterGPTMessageByMaxContext({
await filterGPTMessageByMaxTokens({
messages,
maxContext: toolModel.maxContext - (max_tokens || 0) // filter token. not response maxToken
maxTokens: toolModel.maxContext - 300 // filter token. not response maxToken
})
).map((item) => {
if (item.role === ChatCompletionRequestMessageRoleEnum.Assistant && item.function_call) {
@@ -202,28 +190,27 @@ export const runToolWithFunctionCall = async (
}
return item;
});
const [requestMessages] = await Promise.all([
const [requestMessages, max_tokens] = await Promise.all([
loadRequestMessages({
messages: filterMessages,
useVision: toolModel.vision && aiChatVision,
origin: requestOrigin
}),
computedMaxToken({
model: toolModel,
maxToken,
filterMessages
})
]);
const requestBody = llmCompletionsBodyFormat(
{
model: toolModel.model,
temperature,
max_tokens,
stream,
messages: requestMessages,
functions,
function_call: 'auto',
temperature,
max_tokens,
top_p: aiChatTopP,
stop: aiChatStopSign,
response_format: aiChatResponseFormat,
json_schema: aiChatJsonSchema
function_call: 'auto'
},
toolModel
);

View File

@@ -334,7 +334,7 @@ const getMultiInput = async ({
return {
documentQuoteText: text,
userFiles: fileLinks.map((url) => parseUrlToFileType(url)).filter(Boolean)
userFiles: fileLinks.map((url) => parseUrlToFileType(url))
};
};

View File

@@ -1,5 +1,5 @@
import { createChatCompletion } from '../../../../ai/config';
import { filterGPTMessageByMaxContext, loadRequestMessages } from '../../../../chat/utils';
import { filterGPTMessageByMaxTokens, loadRequestMessages } from '../../../../chat/utils';
import {
ChatCompletion,
StreamChatType,
@@ -54,15 +54,7 @@ export const runToolWithPromptCall = async (
externalProvider,
stream,
workflowStreamResponse,
params: {
temperature,
maxToken,
aiChatVision,
aiChatTopP,
aiChatStopSign,
aiChatResponseFormat,
aiChatJsonSchema
}
params: { temperature, maxToken, aiChatVision }
} = workflowProps;
if (interactiveEntryToolParams) {
@@ -204,33 +196,30 @@ export const runToolWithPromptCall = async (
return Promise.reject('Prompt call invalid input');
}
const max_tokens = computedMaxToken({
model: toolModel,
maxToken
});
const filterMessages = await filterGPTMessageByMaxContext({
const filterMessages = await filterGPTMessageByMaxTokens({
messages,
maxContext: toolModel.maxContext - (max_tokens || 0) // filter token. not response maxToken
maxTokens: toolModel.maxContext - 500 // filter token. not response maxToken
});
const [requestMessages] = await Promise.all([
const [requestMessages, max_tokens] = await Promise.all([
loadRequestMessages({
messages: filterMessages,
useVision: toolModel.vision && aiChatVision,
origin: requestOrigin
}),
computedMaxToken({
model: toolModel,
maxToken,
filterMessages
})
]);
const requestBody = llmCompletionsBodyFormat(
{
model: toolModel.model,
stream,
messages: requestMessages,
temperature,
max_tokens,
top_p: aiChatTopP,
stop: aiChatStopSign,
response_format: aiChatResponseFormat,
json_schema: aiChatJsonSchema
stream,
messages: requestMessages
},
toolModel
);

View File

@@ -1,5 +1,5 @@
import { createChatCompletion } from '../../../../ai/config';
import { filterGPTMessageByMaxContext, loadRequestMessages } from '../../../../chat/utils';
import { filterGPTMessageByMaxTokens, loadRequestMessages } from '../../../../chat/utils';
import {
ChatCompletion,
ChatCompletionMessageToolCall,
@@ -93,15 +93,7 @@ export const runToolWithToolChoice = async (
stream,
externalProvider,
workflowStreamResponse,
params: {
temperature,
maxToken,
aiChatVision,
aiChatTopP,
aiChatStopSign,
aiChatResponseFormat,
aiChatJsonSchema
}
params: { temperature, maxToken, aiChatVision }
} = workflowProps;
if (maxRunToolTimes <= 0 && response) {
@@ -236,16 +228,11 @@ export const runToolWithToolChoice = async (
};
});
const max_tokens = computedMaxToken({
model: toolModel,
maxToken
});
// Filter histories by maxToken
const filterMessages = (
await filterGPTMessageByMaxContext({
await filterGPTMessageByMaxTokens({
messages,
maxContext: toolModel.maxContext - (max_tokens || 0) // filter token. not response maxToken
maxTokens: toolModel.maxContext - 300 // filter token. not response maxToken
})
).map((item) => {
if (item.role === 'assistant' && item.tool_calls) {
@@ -261,30 +248,31 @@ export const runToolWithToolChoice = async (
return item;
});
const [requestMessages] = await Promise.all([
const [requestMessages, max_tokens] = await Promise.all([
loadRequestMessages({
messages: filterMessages,
useVision: toolModel.vision && aiChatVision,
origin: requestOrigin
}),
computedMaxToken({
model: toolModel,
maxToken,
filterMessages
})
]);
const requestBody = llmCompletionsBodyFormat(
{
model: toolModel.model,
temperature,
max_tokens,
stream,
messages: requestMessages,
tools,
tool_choice: 'auto',
temperature,
max_tokens,
top_p: aiChatTopP,
stop: aiChatStopSign,
response_format: aiChatResponseFormat,
json_schema: aiChatJsonSchema
tool_choice: 'auto'
},
toolModel
);
// console.log(JSON.stringify(requestMessages, null, 2), '==requestBody');
// console.log(JSON.stringify(requestBody, null, 2), '==requestBody');
/* Run llm */
const {
response: aiResponse,

View File

@@ -16,16 +16,12 @@ export type DispatchToolModuleProps = ModuleDispatchProps<{
[NodeInputKeyEnum.history]?: ChatItemType[];
[NodeInputKeyEnum.userChatInput]: string;
[NodeInputKeyEnum.fileUrlList]?: string[];
[NodeInputKeyEnum.aiModel]: string;
[NodeInputKeyEnum.aiSystemPrompt]: string;
[NodeInputKeyEnum.aiChatTemperature]: number;
[NodeInputKeyEnum.aiChatMaxToken]: number;
[NodeInputKeyEnum.aiChatVision]?: boolean;
[NodeInputKeyEnum.aiChatTopP]?: number;
[NodeInputKeyEnum.aiChatStopSign]?: string;
[NodeInputKeyEnum.aiChatResponseFormat]?: string;
[NodeInputKeyEnum.aiChatJsonSchema]?: string;
[NodeInputKeyEnum.fileUrlList]?: string[];
}> & {
messages: ChatCompletionMessageParam[];
toolNodes: ToolNodeItemType[];

View File

@@ -1,15 +1,15 @@
import type { NextApiResponse } from 'next';
import { filterGPTMessageByMaxContext, loadRequestMessages } from '../../../chat/utils';
import { filterGPTMessageByMaxTokens, loadRequestMessages } from '../../../chat/utils';
import type { ChatItemType, UserChatItemValueItemType } from '@fastgpt/global/core/chat/type.d';
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
import { SseResponseEventEnum } from '@fastgpt/global/core/workflow/runtime/constants';
import {
parseReasoningContent,
parseReasoningStreamContent,
textAdaptGptResponse
} from '@fastgpt/global/core/workflow/runtime/utils';
import { textAdaptGptResponse } from '@fastgpt/global/core/workflow/runtime/utils';
import { createChatCompletion } from '../../../ai/config';
import type { ChatCompletionMessageParam, StreamChatType } from '@fastgpt/global/core/ai/type.d';
import type {
ChatCompletion,
ChatCompletionMessageParam,
StreamChatType
} from '@fastgpt/global/core/ai/type.d';
import { formatModelChars2Points } from '../../../../support/wallet/usage/utils';
import type { LLMModelItemType } from '@fastgpt/global/core/ai/model.d';
import { postTextCensor } from '../../../../common/api/requestPlusApi';
@@ -51,14 +51,13 @@ import { ModelTypeEnum } from '@fastgpt/global/core/ai/model';
export type ChatProps = ModuleDispatchProps<
AIChatNodeProps & {
[NodeInputKeyEnum.userChatInput]?: string;
[NodeInputKeyEnum.userChatInput]: string;
[NodeInputKeyEnum.history]?: ChatItemType[] | number;
[NodeInputKeyEnum.aiChatDatasetQuote]?: SearchDataResponseItemType[];
}
>;
export type ChatResponse = DispatchNodeResultType<{
[NodeOutputKeyEnum.answerText]: string;
[NodeOutputKeyEnum.reasoningText]?: string;
[NodeOutputKeyEnum.history]: ChatItemType[];
}>;
@@ -81,36 +80,29 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
maxToken,
history = 6,
quoteQA,
userChatInput = '',
userChatInput,
isResponseAnswerText = true,
systemPrompt = '',
aiChatQuoteRole = 'system',
quoteTemplate,
quotePrompt,
aiChatVision,
aiChatReasoning = true,
aiChatTopP,
aiChatStopSign,
aiChatResponseFormat,
aiChatJsonSchema,
fileUrlList: fileLinks, // node quote file links
stringQuoteText //abandon
}
} = props;
const { files: inputFiles } = chatValue2RuntimePrompt(query); // Chat box input files
stream = stream && isResponseAnswerText;
const chatHistories = getHistories(history, histories);
quoteQA = checkQuoteQAValue(quoteQA);
const modelConstantsData = getLLMModel(model);
if (!modelConstantsData) {
return Promise.reject('The chat model is undefined, you need to select a chat model.');
}
aiChatVision = modelConstantsData.vision && aiChatVision;
aiChatReasoning = !!aiChatReasoning && !!modelConstantsData.reasoning;
const chatHistories = getHistories(history, histories);
quoteQA = checkQuoteQAValue(quoteQA);
const [{ datasetQuoteText }, { documentQuoteText, userFiles }] = await Promise.all([
filterDatasetQuote({
quoteQA,
@@ -132,15 +124,9 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
return Promise.reject(i18nT('chat:AI_input_is_empty'));
}
const max_tokens = computedMaxToken({
model: modelConstantsData,
maxToken
});
const [{ filterMessages }] = await Promise.all([
getChatMessages({
model: modelConstantsData,
maxTokens: max_tokens,
histories: chatHistories,
useDatasetQuote: quoteQA !== undefined,
datasetQuoteText,
@@ -151,8 +137,8 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
userFiles,
documentQuoteText
}),
// Censor = true and system key, will check content
(() => {
// censor model and system key
if (modelConstantsData.censor && !externalProvider.openaiAccount?.key) {
return postTextCensor({
text: `${systemPrompt}
@@ -163,23 +149,26 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
})()
]);
const requestMessages = await loadRequestMessages({
messages: filterMessages,
useVision: aiChatVision,
origin: requestOrigin
});
const [requestMessages, max_tokens] = await Promise.all([
loadRequestMessages({
messages: filterMessages,
useVision: modelConstantsData.vision && aiChatVision,
origin: requestOrigin
}),
computedMaxToken({
model: modelConstantsData,
maxToken,
filterMessages
})
]);
const requestBody = llmCompletionsBodyFormat(
{
model: modelConstantsData.model,
stream,
messages: requestMessages,
temperature,
max_tokens,
top_p: aiChatTopP,
stop: aiChatStopSign,
response_format: aiChatResponseFormat as any,
json_schema: aiChatJsonSchema
stream,
messages: requestMessages
},
modelConstantsData
);
@@ -194,71 +183,34 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
}
});
const { answerText, reasoningText } = await (async () => {
if (isStreamResponse) {
if (!res) {
return {
answerText: '',
reasoningText: ''
};
}
const { answerText } = await (async () => {
if (res && isStreamResponse) {
// sse response
const { answer, reasoning } = await streamResponse({
const { answer } = await streamResponse({
res,
stream: response,
aiChatReasoning,
isResponseAnswerText,
workflowStreamResponse
});
return {
answerText: answer,
reasoningText: reasoning
answerText: answer
};
} else {
const { content, reasoningContent } = (() => {
const content = response.choices?.[0]?.message?.content || '';
// @ts-ignore
const reasoningContent: string = response.choices?.[0]?.message?.reasoning_content || '';
const unStreamResponse = response as ChatCompletion;
const answer = unStreamResponse.choices?.[0]?.message?.content || '';
// API already parse reasoning content
if (reasoningContent || !aiChatReasoning) {
return {
content,
reasoningContent
};
}
const [think, answer] = parseReasoningContent(content);
return {
content: answer,
reasoningContent: think
};
})();
// Some models do not support streaming
if (stream) {
if (aiChatReasoning && reasoningContent) {
workflowStreamResponse?.({
event: SseResponseEventEnum.fastAnswer,
data: textAdaptGptResponse({
reasoning_content: reasoningContent
})
});
}
if (isResponseAnswerText && content) {
workflowStreamResponse?.({
event: SseResponseEventEnum.fastAnswer,
data: textAdaptGptResponse({
text: content
})
});
}
// Some models do not support streaming
workflowStreamResponse?.({
event: SseResponseEventEnum.fastAnswer,
data: textAdaptGptResponse({
text: answer
})
});
}
return {
answerText: content,
reasoningText: reasoningContent
answerText: answer
};
}
})();
@@ -270,8 +222,7 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
const AIMessages: ChatCompletionMessageParam[] = [
{
role: ChatCompletionRequestMessageRoleEnum.Assistant,
content: answerText,
reasoning_text: reasoningText // reasoning_text is only recorded for response, but not for request
content: answerText
}
];
@@ -289,8 +240,7 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
});
return {
answerText: answerText.trim(),
reasoningText,
answerText,
[DispatchNodeResponseKeyEnum.nodeResponse]: {
totalPoints: externalProvider.openaiAccount?.key ? 0 : totalPoints,
model: modelName,
@@ -299,8 +249,11 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
outputTokens: outputTokens,
query: `${userChatInput}`,
maxToken: max_tokens,
reasoningText,
historyPreview: getHistoryPreview(chatCompleteMessages, 10000, aiChatVision),
historyPreview: getHistoryPreview(
chatCompleteMessages,
10000,
modelConstantsData.vision && aiChatVision
),
contextTotalLen: completeMessages.length
},
[DispatchNodeResponseKeyEnum.nodeDispatchUsages]: [
@@ -408,13 +361,12 @@ async function getMultiInput({
return {
documentQuoteText: text,
userFiles: fileLinks.map((url) => parseUrlToFileType(url)).filter(Boolean)
userFiles: fileLinks.map((url) => parseUrlToFileType(url))
};
}
async function getChatMessages({
model,
maxTokens = 0,
aiChatQuoteRole,
datasetQuotePrompt = '',
datasetQuoteText,
@@ -426,7 +378,6 @@ async function getChatMessages({
documentQuoteText
}: {
model: LLMModelItemType;
maxTokens?: number;
// dataset quote
aiChatQuoteRole: AiChatQuoteRoleType; // user: replace user prompt; system: replace system prompt
datasetQuotePrompt?: string;
@@ -493,9 +444,9 @@ async function getChatMessages({
const adaptMessages = chats2GPTMessages({ messages, reserveId: false });
const filterMessages = await filterGPTMessageByMaxContext({
const filterMessages = await filterGPTMessageByMaxTokens({
messages: adaptMessages,
maxContext: model.maxContext - maxTokens // filter token. not response maxToken
maxTokens: model.maxContext - 300 // filter token. not response maxToken
});
return {
@@ -506,59 +457,33 @@ async function getChatMessages({
async function streamResponse({
res,
stream,
workflowStreamResponse,
aiChatReasoning,
isResponseAnswerText
workflowStreamResponse
}: {
res: NextApiResponse;
stream: StreamChatType;
workflowStreamResponse?: WorkflowResponseType;
aiChatReasoning?: boolean;
isResponseAnswerText?: boolean;
}) {
const write = responseWriteController({
res,
readStream: stream
});
let answer = '';
let reasoning = '';
const { parsePart, getStartTagBuffer } = parseReasoningStreamContent();
for await (const part of stream) {
if (res.closed) {
stream.controller?.abort();
break;
}
const [reasoningContent, content] = parsePart(part, aiChatReasoning);
const content = part.choices?.[0]?.delta?.content || '';
answer += content;
reasoning += reasoningContent;
if (aiChatReasoning && reasoningContent) {
workflowStreamResponse?.({
write,
event: SseResponseEventEnum.answer,
data: textAdaptGptResponse({
reasoning_content: reasoningContent
})
});
}
if (isResponseAnswerText && content) {
workflowStreamResponse?.({
write,
event: SseResponseEventEnum.answer,
data: textAdaptGptResponse({
text: content
})
});
}
workflowStreamResponse?.({
write,
event: SseResponseEventEnum.answer,
data: textAdaptGptResponse({
text: content
})
});
}
// if answer is empty, try to get value from startTagBuffer. (Cause: The response content is too short to exceed the minimum parse length)
if (answer === '') {
answer = getStartTagBuffer();
}
return { answer, reasoning };
return { answer };
}

View File

@@ -6,11 +6,13 @@ import { formatModelChars2Points } from '../../../../support/wallet/usage/utils'
import type { SelectedDatasetType } from '@fastgpt/global/core/workflow/api.d';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
import type { ModuleDispatchProps } from '@fastgpt/global/core/workflow/runtime/type';
import { getEmbeddingModel } from '../../../ai/model';
import { deepRagSearch, defaultSearchDatasetData } from '../../../dataset/search/controller';
import { getLLMModel, getEmbeddingModel } from '../../../ai/model';
import { searchDatasetData } from '../../../dataset/search/controller';
import { NodeInputKeyEnum, NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants';
import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants';
import { DatasetSearchModeEnum } from '@fastgpt/global/core/dataset/constants';
import { getHistories } from '../utils';
import { datasetSearchQueryExtension } from '../../../dataset/search/utils';
import { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { checkTeamReRankPermission } from '../../../../support/permission/teamLimit';
import { MongoDataset } from '../../../dataset/schema';
@@ -23,19 +25,13 @@ type DatasetSearchProps = ModuleDispatchProps<{
[NodeInputKeyEnum.datasetSimilarity]: number;
[NodeInputKeyEnum.datasetMaxTokens]: number;
[NodeInputKeyEnum.datasetSearchMode]: `${DatasetSearchModeEnum}`;
[NodeInputKeyEnum.userChatInput]?: string;
[NodeInputKeyEnum.userChatInput]: string;
[NodeInputKeyEnum.datasetSearchUsingReRank]: boolean;
[NodeInputKeyEnum.collectionFilterMatch]: string;
[NodeInputKeyEnum.authTmbId]?: boolean;
[NodeInputKeyEnum.datasetSearchUsingExtensionQuery]: boolean;
[NodeInputKeyEnum.datasetSearchExtensionModel]: string;
[NodeInputKeyEnum.datasetSearchExtensionBg]: string;
[NodeInputKeyEnum.datasetDeepSearch]?: boolean;
[NodeInputKeyEnum.datasetDeepSearchModel]?: string;
[NodeInputKeyEnum.datasetDeepSearchMaxTimes]?: number;
[NodeInputKeyEnum.datasetDeepSearchBg]?: string;
[NodeInputKeyEnum.collectionFilterMatch]: string;
[NodeInputKeyEnum.authTmbId]: boolean;
}>;
export type DatasetSearchResponse = DispatchNodeResultType<{
[NodeOutputKeyEnum.datasetQuoteQA]: SearchDataResponseItemType[];
@@ -55,18 +51,13 @@ export async function dispatchDatasetSearch(
limit = 1500,
usingReRank,
searchMode,
userChatInput = '',
authTmbId = false,
collectionFilterMatch,
userChatInput,
datasetSearchUsingExtensionQuery,
datasetSearchExtensionModel,
datasetSearchExtensionBg,
datasetDeepSearch,
datasetDeepSearchModel,
datasetDeepSearchMaxTimes,
datasetDeepSearchBg
collectionFilterMatch,
authTmbId = false
}
} = props as DatasetSearchProps;
@@ -94,12 +85,25 @@ export async function dispatchDatasetSearch(
return emptyResult;
}
const datasetIds = authTmbId
? await filterDatasetsByTmbId({
datasetIds: datasets.map((item) => item.datasetId),
tmbId
})
: await Promise.resolve(datasets.map((item) => item.datasetId));
// query extension
const extensionModel = datasetSearchUsingExtensionQuery
? getLLMModel(datasetSearchExtensionModel)
: undefined;
const [{ concatQueries, rewriteQuery, aiExtensionResult }, datasetIds] = await Promise.all([
datasetSearchQueryExtension({
query: userChatInput,
extensionModel,
extensionBg: datasetSearchExtensionBg,
histories: getHistories(6, histories)
}),
authTmbId
? filterDatasetsByTmbId({
datasetIds: datasets.map((item) => item.datasetId),
tmbId
})
: Promise.resolve(datasets.map((item) => item.datasetId))
]);
if (datasetIds.length === 0) {
return emptyResult;
@@ -112,11 +116,15 @@ export async function dispatchDatasetSearch(
);
// start search
const searchData = {
histories,
const {
searchRes,
tokens,
usingSimilarityFilter,
usingReRank: searchUsingReRank
} = await searchDatasetData({
teamId,
reRankQuery: userChatInput,
queries: [userChatInput],
reRankQuery: `${rewriteQuery}`,
queries: concatQueries,
model: vectorModel.model,
similarity,
limit,
@@ -124,106 +132,59 @@ export async function dispatchDatasetSearch(
searchMode,
usingReRank: usingReRank && (await checkTeamReRankPermission(teamId)),
collectionFilterMatch
};
const {
searchRes,
tokens,
usingSimilarityFilter,
usingReRank: searchUsingReRank,
queryExtensionResult,
deepSearchResult
} = datasetDeepSearch
? await deepRagSearch({
...searchData,
datasetDeepSearchModel,
datasetDeepSearchMaxTimes,
datasetDeepSearchBg
})
: await defaultSearchDatasetData({
...searchData,
datasetSearchUsingExtensionQuery,
datasetSearchExtensionModel,
datasetSearchExtensionBg
});
});
// count bill results
const nodeDispatchUsages: ChatNodeUsageType[] = [];
// vector
const { totalPoints: embeddingTotalPoints, modelName: embeddingModelName } =
formatModelChars2Points({
model: vectorModel.model,
inputTokens: tokens,
modelType: ModelTypeEnum.embedding
});
nodeDispatchUsages.push({
totalPoints: embeddingTotalPoints,
moduleName: node.name,
model: embeddingModelName,
inputTokens: tokens
const { totalPoints, modelName } = formatModelChars2Points({
model: vectorModel.model,
inputTokens: tokens,
modelType: ModelTypeEnum.embedding
});
// Query extension
const { totalPoints: queryExtensionTotalPoints } = (() => {
if (queryExtensionResult) {
const { totalPoints, modelName } = formatModelChars2Points({
model: queryExtensionResult.model,
inputTokens: queryExtensionResult.inputTokens,
outputTokens: queryExtensionResult.outputTokens,
modelType: ModelTypeEnum.llm
});
nodeDispatchUsages.push({
totalPoints,
moduleName: i18nT('common:core.module.template.Query extension'),
model: modelName,
inputTokens: queryExtensionResult.inputTokens,
outputTokens: queryExtensionResult.outputTokens
});
return {
totalPoints
};
}
return {
totalPoints: 0
};
})();
// Deep search
const { totalPoints: deepSearchTotalPoints } = (() => {
if (deepSearchResult) {
const { totalPoints, modelName } = formatModelChars2Points({
model: deepSearchResult.model,
inputTokens: deepSearchResult.inputTokens,
outputTokens: deepSearchResult.outputTokens,
modelType: ModelTypeEnum.llm
});
nodeDispatchUsages.push({
totalPoints,
moduleName: i18nT('common:deep_rag_search'),
model: modelName,
inputTokens: deepSearchResult.inputTokens,
outputTokens: deepSearchResult.outputTokens
});
return {
totalPoints
};
}
return {
totalPoints: 0
};
})();
const totalPoints = embeddingTotalPoints + queryExtensionTotalPoints + deepSearchTotalPoints;
const responseData: DispatchNodeResponseType & { totalPoints: number } = {
totalPoints,
query: userChatInput,
model: vectorModel.model,
query: concatQueries.join('\n'),
model: modelName,
inputTokens: tokens,
similarity: usingSimilarityFilter ? similarity : undefined,
limit,
searchMode,
searchUsingReRank: searchUsingReRank,
quoteList: searchRes,
queryExtensionResult,
deepSearchResult
quoteList: searchRes
};
const nodeDispatchUsages: ChatNodeUsageType[] = [
{
totalPoints,
moduleName: node.name,
model: modelName,
inputTokens: tokens
}
];
if (aiExtensionResult) {
const { totalPoints, modelName } = formatModelChars2Points({
model: aiExtensionResult.model,
inputTokens: aiExtensionResult.inputTokens,
outputTokens: aiExtensionResult.outputTokens,
modelType: ModelTypeEnum.llm
});
responseData.totalPoints += totalPoints;
responseData.inputTokens = aiExtensionResult.inputTokens;
responseData.outputTokens = aiExtensionResult.outputTokens;
responseData.extensionModel = modelName;
responseData.extensionResult =
aiExtensionResult.extensionQueries?.join('\n') ||
JSON.stringify(aiExtensionResult.extensionQueries);
nodeDispatchUsages.push({
totalPoints,
moduleName: 'core.module.template.Query extension',
model: modelName,
inputTokens: aiExtensionResult.inputTokens,
outputTokens: aiExtensionResult.outputTokens
});
}
return {
quoteQA: searchRes,

View File

@@ -204,7 +204,6 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
{ inputs = [] }: RuntimeNodeItemType,
{
answerText = '',
reasoningText,
responseData,
nodeDispatchUsages,
toolResponses,
@@ -214,7 +213,6 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
}: Omit<
DispatchNodeResultType<{
[NodeOutputKeyEnum.answerText]?: string;
[NodeOutputKeyEnum.reasoningText]?: string;
[DispatchNodeResponseKeyEnum.nodeResponse]?: ChatHistoryItemResType;
}>,
'nodeResponse'
@@ -232,46 +230,26 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
chatNodeUsages = chatNodeUsages.concat(nodeDispatchUsages);
}
if (toolResponses !== undefined && toolResponses !== null) {
if (toolResponses !== undefined) {
if (Array.isArray(toolResponses) && toolResponses.length === 0) return;
if (
!Array.isArray(toolResponses) &&
typeof toolResponses === 'object' &&
Object.keys(toolResponses).length === 0
)
return;
if (typeof toolResponses === 'object' && Object.keys(toolResponses).length === 0) return;
toolRunResponse = toolResponses;
}
// Histories store
if (assistantResponses) {
chatAssistantResponse = chatAssistantResponse.concat(assistantResponses);
} else {
if (reasoningText) {
const isResponseReasoningText = inputs.find(
(item) => item.key === NodeInputKeyEnum.aiChatReasoning
)?.value;
if (isResponseReasoningText) {
chatAssistantResponse.push({
type: ChatItemValueTypeEnum.reasoning,
reasoning: {
content: reasoningText
}
});
}
}
if (answerText) {
// save assistant text response
const isResponseAnswerText =
inputs.find((item) => item.key === NodeInputKeyEnum.aiChatIsResponseText)?.value ?? true;
if (isResponseAnswerText) {
chatAssistantResponse.push({
type: ChatItemValueTypeEnum.text,
text: {
content: answerText
}
});
}
} else if (answerText) {
// save assistant text response
const isResponseAnswerText =
inputs.find((item) => item.key === NodeInputKeyEnum.aiChatIsResponseText)?.value ?? true;
if (isResponseAnswerText) {
chatAssistantResponse.push({
type: ChatItemValueTypeEnum.text,
text: {
content: answerText
}
});
}
}

View File

@@ -53,7 +53,7 @@ export const dispatchRunAppNode = async (props: Props): Promise<Response> => {
const userInputFiles = (() => {
if (fileUrlList) {
return fileUrlList.map((url) => parseUrlToFileType(url)).filter(Boolean);
return fileUrlList.map((url) => parseUrlToFileType(url));
}
// Adapt version 4.8.13 upgrade
return files;

View File

@@ -38,10 +38,10 @@ type HttpRequestProps = ModuleDispatchProps<{
[NodeInputKeyEnum.abandon_httpUrl]: string;
[NodeInputKeyEnum.httpMethod]: string;
[NodeInputKeyEnum.httpReqUrl]: string;
[NodeInputKeyEnum.httpHeaders]?: PropsArrType[];
[NodeInputKeyEnum.httpParams]?: PropsArrType[];
[NodeInputKeyEnum.httpJsonBody]?: string;
[NodeInputKeyEnum.httpFormBody]?: PropsArrType[];
[NodeInputKeyEnum.httpHeaders]: PropsArrType[];
[NodeInputKeyEnum.httpParams]: PropsArrType[];
[NodeInputKeyEnum.httpJsonBody]: string;
[NodeInputKeyEnum.httpFormBody]: PropsArrType[];
[NodeInputKeyEnum.httpContentType]: ContentTypes;
[NodeInputKeyEnum.addInputParam]: Record<string, any>;
[NodeInputKeyEnum.httpTimeout]?: number;
@@ -76,10 +76,10 @@ export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<H
params: {
system_httpMethod: httpMethod = 'POST',
system_httpReqUrl: httpReqUrl,
system_httpHeader: httpHeader = [],
system_httpHeader: httpHeader,
system_httpParams: httpParams = [],
system_httpJsonBody: httpJsonBody = '',
system_httpFormBody: httpFormBody = [],
system_httpJsonBody: httpJsonBody,
system_httpFormBody: httpFormBody,
system_httpContentType: httpContentType = ContentTypes.json,
system_httpTimeout: httpTimeout = 60,
[NodeInputKeyEnum.addInputParam]: dynamicInput,
@@ -244,6 +244,7 @@ export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<H
if (!httpJsonBody) return {};
if (httpContentType === ContentTypes.json) {
httpJsonBody = replaceJsonBodyString(httpJsonBody);
console.log(httpJsonBody);
return json5.parse(httpJsonBody);
}
@@ -398,6 +399,41 @@ async function fetchData({
};
}
// function replaceVariable(text: string, obj: Record<string, any>) {
// for (const [key, value] of Object.entries(obj)) {
// if (value === undefined) {
// text = text.replace(new RegExp(`{{(${key})}}`, 'g'), UNDEFINED_SIGN);
// } else {
// const replacement = JSON.stringify(value);
// const unquotedReplacement =
// replacement.startsWith('"') && replacement.endsWith('"')
// ? replacement.slice(1, -1)
// : replacement;
// text = text.replace(new RegExp(`{{(${key})}}`, 'g'), () => unquotedReplacement);
// }
// }
// return text || '';
// }
// function removeUndefinedSign(obj: Record<string, any>) {
// for (const key in obj) {
// if (obj[key] === UNDEFINED_SIGN) {
// obj[key] = undefined;
// } else if (Array.isArray(obj[key])) {
// obj[key] = obj[key].map((item: any) => {
// if (item === UNDEFINED_SIGN) {
// return undefined;
// } else if (typeof item === 'object') {
// removeUndefinedSign(item);
// }
// return item;
// });
// } else if (typeof obj[key] === 'object') {
// removeUndefinedSign(obj[key]);
// }
// }
// return obj;
// }
// Replace some special response from system plugin
async function replaceSystemPluginResponse({
response,

View File

@@ -142,7 +142,7 @@ export const checkQuoteQAValue = (quoteQA?: SearchDataResponseItemType[]) => {
if (quoteQA.length === 0) {
return [];
}
if (quoteQA.some((item) => typeof item !== 'object' || !item.q)) {
if (quoteQA.some((item) => !item.q)) {
return undefined;
}
return quoteQA;

View File

@@ -86,12 +86,9 @@ export async function addSourceMember<T extends { tmbId: string }>({
}): Promise<Array<T & { sourceMember: SourceMemberType }>> {
if (!Array.isArray(list)) return [];
const tmbIdList = list
.map((item) => (item.tmbId ? String(item.tmbId) : undefined))
.filter(Boolean);
const tmbList = await MongoTeamMember.find(
{
_id: { $in: tmbIdList }
_id: { $in: list.map((item) => String(item.tmbId)) }
},
'tmbId name avatar status',
{

View File

@@ -1,114 +1,6 @@
import { UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
import { MongoUsage } from './schema';
import { ClientSession, Types } from '../../../common/mongo';
import { addLog } from '../../../common/system/log';
import { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { ConcatUsageProps, CreateUsageProps } from '@fastgpt/global/support/wallet/usage/api';
import { i18nT } from '../../../../web/i18n/utils';
import { pushConcatBillTask, pushReduceTeamAiPointsTask } from './utils';
import { POST } from '../../../common/api/plusRequest';
import { FastGPTProUrl } from '../../../common/system/constants';
export async function createUsage(data: CreateUsageProps) {
try {
// In FastGPT server
if (FastGPTProUrl) {
await POST('/support/wallet/usage/createUsage', data);
} else if (global.reduceAiPointsQueue) {
// In FastGPT pro server
await MongoUsage.create(data);
pushReduceTeamAiPointsTask({ teamId: data.teamId, totalPoints: data.totalPoints });
if (data.totalPoints === 0) {
addLog.info('0 totalPoints', data);
}
}
} catch (error) {
addLog.error('createUsage error', error);
}
}
export async function concatUsage(data: ConcatUsageProps) {
try {
// In FastGPT server
if (FastGPTProUrl) {
await POST('/support/wallet/usage/concatUsage', data);
} else if (global.reduceAiPointsQueue) {
const {
teamId,
billId,
totalPoints = 0,
listIndex,
inputTokens = 0,
outputTokens = 0
} = data;
// billId is required and valid
if (!billId || !Types.ObjectId.isValid(billId)) return;
// In FastGPT pro server
pushConcatBillTask([
{
billId,
listIndex,
inputTokens,
outputTokens,
totalPoints
}
]);
pushReduceTeamAiPointsTask({ teamId, totalPoints });
if (data.totalPoints === 0) {
addLog.info('0 totalPoints', data);
}
}
} catch (error) {
addLog.error('concatUsage error', error);
}
}
export const createChatUsage = ({
appName,
appId,
pluginId,
teamId,
tmbId,
source,
flowUsages
}: {
appName: string;
appId?: string;
pluginId?: string;
teamId: string;
tmbId: string;
source: UsageSourceEnum;
flowUsages: ChatNodeUsageType[];
}) => {
const totalPoints = flowUsages.reduce((sum, item) => sum + (item.totalPoints || 0), 0);
createUsage({
teamId,
tmbId,
appName,
appId,
pluginId,
totalPoints,
source,
list: flowUsages.map((item) => ({
moduleName: item.moduleName,
amount: item.totalPoints || 0,
model: item.model,
inputTokens: item.inputTokens,
outputTokens: item.outputTokens
}))
});
addLog.debug(`Create chat usage`, {
source,
teamId,
totalPoints
});
return { totalPoints };
};
import { ClientSession } from '../../../common/mongo';
export const createTrainingUsage = async ({
teamId,
@@ -137,21 +29,21 @@ export const createTrainingUsage = async ({
totalPoints: 0,
list: [
{
moduleName: i18nT('common:support.wallet.moduleName.index'),
moduleName: 'support.wallet.moduleName.index',
model: vectorModel,
amount: 0,
inputTokens: 0,
outputTokens: 0
},
{
moduleName: i18nT('common:support.wallet.moduleName.qa'),
moduleName: 'support.wallet.moduleName.qa',
model: agentModel,
amount: 0,
inputTokens: 0,
outputTokens: 0
},
{
moduleName: i18nT('common:core.dataset.training.Auto mode'),
moduleName: 'core.dataset.training.Auto mode',
model: agentModel,
amount: 0,
inputTokens: 0,

View File

@@ -1,12 +0,0 @@
export type ConcatBillQueueItemType = {
billId: string;
listIndex?: number;
totalPoints: number;
inputTokens: number;
outputTokens: number;
};
declare global {
var reduceAiPointsQueue: { teamId: string; totalPoints: number }[];
var concatBillQueue: ConcatBillQueueItemType[];
}

View File

@@ -1,6 +1,5 @@
import { findAIModel } from '../../../core/ai/model';
import { ModelTypeEnum } from '@fastgpt/global/core/ai/model';
import { ConcatBillQueueItemType } from './type';
export const formatModelChars2Points = ({
model,
@@ -35,20 +34,3 @@ export const formatModelChars2Points = ({
totalPoints
};
};
export const pushReduceTeamAiPointsTask = ({
teamId,
totalPoints
}: {
teamId: string;
totalPoints: number;
}) => {
global.reduceAiPointsQueue.push({
teamId: String(teamId),
totalPoints
});
};
export const pushConcatBillTask = (data: ConcatBillQueueItemType[]) => {
global.concatBillQueue.push(...data);
};

Some files were not shown because too many files have changed in this diff Show More