perf: delete dataset (#3949 )

* fix: collection list count * fix: collection list count * ai proxy ui * perf: delete dataset * perf: add dataset text index * update doc
fix: rerank model cannot use ai proxy (#3945 )
2025-03-03 12:49:13 +08:00 · 2025-03-03 11:49:35 +08:00 · 2025-03-03 11:37:51 +08:00 · 2025-03-02 17:50:03 +08:00 · 2025-02-28 19:18:12 +08:00 · 2025-02-28 19:00:58 +08:00
481 changed files with 16576 additions and 3500 deletions
--- a/.github/workflows/docs-sync_imgs.yml
+++ b/.github/workflows/docs-sync_imgs.yml
--- a/.github/workflows/fastgpt-build-image-personal.yml
+++ b/.github/workflows/fastgpt-build-image-personal.yml
--- a/.github/workflows/fastgpt-build-image.yml
+++ b/.github/workflows/fastgpt-build-image.yml
--- a/.github/workflows/fastgpt-preview-image.yml
+++ b/.github/workflows/fastgpt-preview-image.yml
@@ -68,14 +68,3 @@ jobs:
          SEALOS_TYPE: 'pr_comment'
          SEALOS_FILENAME: 'report.md'
          SEALOS_REPLACE_TAG: 'DEFAULT_REPLACE_DEPLOY'
-
-  helm-check:
-    runs-on: ubuntu-20.04
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-      - name: Helm Check
-        run: |
-          helm dependency update files/helm/fastgpt
-          helm lint files/helm/fastgpt
-          helm package files/helm/fastgpt
--- a/.github/workflows/helm-release.yaml
+++ b/.github/workflows/helm-release.yaml
@@ -24,6 +24,6 @@ jobs:
          export APP_VERSION=${{ steps.vars.outputs.tag }}
          export HELM_VERSION=${{ steps.vars.outputs.tag }}
          export HELM_REPO=ghcr.io/${{ github.repository_owner }}
-          helm dependency update files/helm/fastgpt
-          helm package files/helm/fastgpt --version ${HELM_VERSION}-helm --app-version ${APP_VERSION} -d bin
+          helm dependency update deploy/helm/fastgpt
+          helm package deploy/helm/fastgpt --version ${HELM_VERSION}-helm --app-version ${APP_VERSION} -d bin
          helm push bin/fastgpt-${HELM_VERSION}-helm.tgz oci://${HELM_REPO}
--- a/.github/workflows/sandbox-build-image.yml
+++ b/.github/workflows/sandbox-build-image.yml
--- a/.vscode/nextapi.code-snippets
+++ b/.vscode/nextapi.code-snippets
@@ -58,7 +58,7 @@
 		"body": [
 			"import '@/pages/api/__mocks__/base';",
 			"import { root } from '@/pages/api/__mocks__/db/init';",
-			"import { getTestRequest } from '@/test/utils';",
+			"import { getTestRequest } from '@fastgpt/service/test/utils'; ;",
 			"import { AppErrEnum } from '@fastgpt/global/common/error/code/app';",
 			"import handler from './demo';",
 			"",
--- a/deploy/docker/docker-compose-milvus.yml
+++ b/deploy/docker/docker-compose-milvus.yml
@@ -114,15 +114,15 @@ services:
  # fastgpt
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -133,7 +133,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose-pgvector.yml
+++ b/deploy/docker/docker-compose-pgvector.yml
@@ -72,15 +72,15 @@ services:
  # fastgpt
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -91,7 +91,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose-zilliz.yml
+++ b/deploy/docker/docker-compose-zilliz.yml
@@ -53,15 +53,15 @@ services:
        wait $$!
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -71,7 +71,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose/docker-compose
+++ b/deploy/docker/docker-compose/docker-compose
--- a/deploy/docker/docker-compose/init.sh
+++ b/deploy/docker/docker-compose/init.sh
--- a/deploy/docker/run.sh
+++ b/deploy/docker/run.sh
--- a/deploy/helm/fastgpt/.helmignore
+++ b/deploy/helm/fastgpt/.helmignore
--- a/deploy/helm/fastgpt/Chart.lock
+++ b/deploy/helm/fastgpt/Chart.lock
--- a/deploy/helm/fastgpt/Chart.yaml
+++ b/deploy/helm/fastgpt/Chart.yaml
--- a/deploy/helm/fastgpt/README.md
+++ b/deploy/helm/fastgpt/README.md
--- a/deploy/helm/fastgpt/templates/NOTES.txt
+++ b/deploy/helm/fastgpt/templates/NOTES.txt
--- a/deploy/helm/fastgpt/templates/_helpers.tpl
+++ b/deploy/helm/fastgpt/templates/_helpers.tpl
--- a/deploy/helm/fastgpt/templates/configmap-config.yaml
+++ b/deploy/helm/fastgpt/templates/configmap-config.yaml
--- a/deploy/helm/fastgpt/templates/deployment.yaml
+++ b/deploy/helm/fastgpt/templates/deployment.yaml
--- a/deploy/helm/fastgpt/templates/hpa.yaml
+++ b/deploy/helm/fastgpt/templates/hpa.yaml
--- a/deploy/helm/fastgpt/templates/ingress.yaml
+++ b/deploy/helm/fastgpt/templates/ingress.yaml
--- a/deploy/helm/fastgpt/templates/secret-env.yaml
+++ b/deploy/helm/fastgpt/templates/secret-env.yaml
--- a/deploy/helm/fastgpt/templates/service.yaml
+++ b/deploy/helm/fastgpt/templates/service.yaml
--- a/deploy/helm/fastgpt/templates/serviceaccount.yaml
+++ b/deploy/helm/fastgpt/templates/serviceaccount.yaml
--- a/deploy/helm/fastgpt/templates/tests/test-connection.yaml
+++ b/deploy/helm/fastgpt/templates/tests/test-connection.yaml
--- a/deploy/helm/fastgpt/values.yaml
+++ b/deploy/helm/fastgpt/values.yaml
--- a/docSite/assets/imgs/appid.png
+++ b/docSite/assets/imgs/appid.png
--- a/docSite/content/zh-cn/docs/development/configuration.md
+++ b/docSite/content/zh-cn/docs/development/configuration.md
@@ -13,8 +13,8 @@ weight: 707

 下面配置文件示例中包含了系统参数和各个模型配置：

-## 4.6.8+ 版本新配置文件示例
-
+## 4.8.20+ 版本新配置文件示例
+> 从4.8.20版本开始，模型在页面中进行配置。
 ```json
 {
  "feConfigs": {
@@ -27,4 +27,4 @@ weight: 707
    "pgHNSWEfSearch": 100 // 向量搜索参数。越大，搜索越精确，但是速度越慢。设置为100，有99%+精度。
  }
 }
-```
+```
--- a/docSite/content/zh-cn/docs/development/custom-models/bge-rerank.md
+++ b/docSite/content/zh-cn/docs/development/custom-models/bge-rerank.md
@@ -31,9 +31,9 @@ weight: 920

 3 个模型代码分别为：

-1. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-base)
-2. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-large)
-3. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-v2-m3](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-v2-m3)
+1. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-base)
+2. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-large)
+3. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-v2-m3](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-v2-m3)

 ### 3. 安装依赖

--- a/docSite/content/zh-cn/docs/development/custom-models/marker.md
+++ b/docSite/content/zh-cn/docs/development/custom-models/marker.md
@@ -19,7 +19,7 @@ PDF 是一个相对复杂的文件格式，在 FastGPT 内置的 pdf 解析器

 ### 1. 按照 Marker

-参考文档 [Marker 安装教程](https://github.com/labring/FastGPT/tree/main/python/pdf-marker)，安装 Marker 模型。封装的 API 已经适配了 FastGPT 自定义解析服务。
+参考文档 [Marker 安装教程](https://github.com/labring/FastGPT/tree/main/plugins/model/pdf-marker)，安装 Marker 模型。封装的 API 已经适配了 FastGPT 自定义解析服务。

 这里介绍快速 Docker 安装的方法：

--- a/docSite/content/zh-cn/docs/development/docker.md
+++ b/docSite/content/zh-cn/docs/development/docker.md
@@ -11,7 +11,7 @@ weight: 707

 1. 基础的网络知识：端口，防火墙……  
 2. Docker 和 Docker Compose 基础知识  
-3. 大模型相关接口和参数
+3. 大模型相关接口和参数  
 4. RAG 相关知识：向量模型，向量数据库，向量检索

 ## 部署架构图
@@ -118,7 +118,7 @@ brew install orbstack
 非 Linux 环境或无法访问外网环境，可手动创建一个目录，并下载配置文件和对应版本的`docker-compose.yml`，在这个文件夹中依据下载的配置文件运行docker，若作为本地开发使用推荐`docker-compose-pgvector`版本，并且自行拉取并运行`sandbox`和`fastgpt`，并在docker配置文件中注释掉`sandbox`和`fastgpt`的部分

 - [config.json](https://raw.githubusercontent.com/labring/FastGPT/refs/heads/main/projects/app/data/config.json)
- [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/files/docker) (注意，不同向量库版本的文件不一样)
+- [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker) (注意，不同向量库版本的文件不一样)

 {{% alert icon="🤖" context="success" %}}

@@ -134,11 +134,11 @@ cd fastgpt
 curl -O https://raw.githubusercontent.com/labring/FastGPT/main/projects/app/data/config.json

 # pgvector 版本(测试推荐，简单快捷)
-curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-pgvector.yml
+curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-pgvector.yml
 # milvus 版本
-# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-milvus.yml
+# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-milvus.yml
 # zilliz 版本
-# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-zilliz.yml
+# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-zilliz.yml
 ```

 ### 2. 修改环境变量
@@ -201,6 +201,8 @@ docker restart oneapi

 在OneApi中添加合适的AI模型渠道。[点击查看相关教程](/docs/development/modelconfig/one-api/)

+只需要添加模型即可，模板已经配置好了oneapi的连接地址和令牌，无需变更。
+
 ### 5. 访问 FastGPT

 目前可以通过 `ip:3000` 直接访问(注意防火墙)。登录用户名为 `root`，密码为`docker-compose.yml`环境变量里设置的 `DEFAULT_ROOT_PSW`。
@@ -211,6 +213,8 @@ docker restart oneapi

 ### 6. 配置模型

+登录FastGPT后，进入模型配置页面，务必先配置至少一个语言模型和一个向量模型，否则系统无法正常使用。
+
 [点击查看模型配置教程](/docs/development/modelConfig/intro/)

 ## FAQ
--- a/docSite/content/zh-cn/docs/development/faq.md
+++ b/docSite/content/zh-cn/docs/development/faq.md
@@ -9,17 +9,31 @@ images: []

 ## 一、错误排查方式

-遇到问题先按下面方式排查。
+可以先找找[Issue](https://github.com/labring/FastGPT/issues)，或新提 Issue，私有部署错误，务必提供详细的操作步骤、日志、截图，否则很难排查。
+
+### 获取后端错误

 1. `docker ps -a` 查看所有容器运行状态，检查是否全部 running，如有异常，尝试`docker logs 容器名`查看对应日志。
 2. 容器都运行正常的，`docker logs 容器名` 查看报错日志
-3. 带有`requestId`的，都是 OneAPI 提示错误，大部分都是因为模型接口报错。
-4. 无法解决时，可以找找[Issue](https://github.com/labring/FastGPT/issues)，或新提 Issue，私有部署错误，务必提供详细的日志，否则很难排查。

+### 前端错误
+
+前端报错时，页面会出现崩溃，并提示检查控制台日志。可以打开浏览器控制台，并查看`console`中的 log 日志。还可以点击对应 log 的超链接，会提示到具体错误文件，可以把这些详细错误信息提供，方便排查。
+
+### OneAPI 错误
+
+带有`requestId`的，都是 OneAPI 提示错误，大部分都是因为模型接口报错。可以参考 [OneAPI 常见错误](/docs/development/faq/#三常见的-oneapi-错误)

 ## 二、通用问题

+### 前端页面崩溃
+
+1. 90% 情况是模型配置不正确：确保每类模型都至少有一个启用；检查模型中一些`对象`参数是否异常（数组和对象），如果为空，可以尝试给个空数组或空对象。
+2. 少部分是由于浏览器兼容问题，由于项目中包含一些高阶语法，可能低版本浏览器不兼容，可以将具体操作步骤和控制台中错误信息提供 issue。
+3. 关闭浏览器翻译功能，如果浏览器开启了翻译，可能会导致页面崩溃。
+
 ### 通过sealos部署的话，是否没有本地部署的一些限制？
+
 ![](/imgs/faq1.png)
 这是索引模型的长度限制，通过任何方式部署都一样的，但不同索引模型的配置不一样，可以在后台修改参数。

@@ -128,9 +142,13 @@ OneAPI 的 API Key 配置错误，需要修改`OPENAI_API_KEY`环境变量，并
 3. ....


+### Tiktoken 下载失败
+
+由于 OneAPI 会在启动时从网络下载一个 tiktoken 的依赖，如果网络异常，就会导致启动失败。可以参考[OneAPI 离线部署](https://blog.csdn.net/wanh/article/details/139039216)解决。
+
 ## 四、常见模型问题

-### 如何检查模型问题
+### 如何检查模型可用性问题

 1. 私有部署模型，先确认部署的模型是否正常。
 2. 通过 CURL 请求，直接测试上游模型是否正常运行（云端模型或私有模型均进行测试）
@@ -403,3 +421,7 @@ curl --location --request POST 'https://oneapi.xxxx/v1/chat/completions' \
  "tool_choice": "auto"
 }'
 ```
+
+### 向量检索得分大于 1
+
+由于模型没有归一化导致的。目前仅支持归一化的模型。
--- a/docSite/content/zh-cn/docs/development/intro.md
+++ b/docSite/content/zh-cn/docs/development/intro.md
@@ -15,8 +15,8 @@ weight: 705

 - [Git](http://git-scm.com/)
 - [Docker](https://www.docker.com/)（构建镜像）
- [Node.js v18.17 / v20.x](http://nodejs.org)（版本尽量一样，可以使用nvm管理node版本）
- [pnpm](https://pnpm.io/) 版本 8.6.0 (目前官方的开发环境)
+- [Node.js v20.14.0](http://nodejs.org)（版本尽量一样，可以使用nvm管理node版本）
+- [pnpm](https://pnpm.io/) 推荐版本 9.4.0 (目前官方的开发环境)
 - make命令: 根据不同平台，百度安装 (官方是GNU Make 4.3)

 ## 开始本地开发
@@ -77,8 +77,6 @@ Mongo 数据库需要注意，需要注意在连接地址中增加 `directConnec
 可参考项目根目录下的 `dev.md`，第一次编译运行可能会有点慢，需要点耐心哦

 ```bash
-# 给自动化脚本代码执行权限(非 linux 系统, 可以手动执行里面的 postinstall.sh 文件内容)
-chmod -R +x ./scripts/
 # 代码根目录下执行，会安装根 package、projects 和 packages 内所有依赖
 # 如果提示 isolate-vm 安装失败，可以参考：https://github.com/laverdet/isolated-vm?tab=readme-ov-file#requirements
 pnpm i
--- a/docSite/content/zh-cn/docs/development/modelConfig/intro.md
+++ b/docSite/content/zh-cn/docs/development/modelConfig/intro.md
@@ -23,7 +23,7 @@ weight: 744

 {{% alert icon=" " context="info" %}}
 - [SiliconCloud(硅基流动)](https://cloud.siliconflow.cn/i/TR9Ym0c4): 提供开源模型调用的平台。
- [Sealos AIProxy](https://hzh.sealos.run/?openapp=system-aiproxy): 提供国内各家模型代理，无需逐一申请 api。
+- [Sealos AIProxy](https://cloud.sealos.run/?uid=fnWRt09fZP&openapp=system-aiproxy): 提供国内各家模型代理，无需逐一申请 api。
 {{% /alert %}}

 在 OneAPI 配置好模型后，你就可以打开 FastGPT 页面，启用对应模型了。
@@ -43,7 +43,7 @@ weight: 744
 {{% alert icon="🤖 " context="success" %}}
 注意：
 1. 目前语音识别模型和重排模型仅会生效一个，所以配置时候，只需要配置一个即可。  
-2. 用于知识库文件处理的语言模型，至少需要开启一个，否则知识库会报错。
+2. 系统至少需要一个语言模型和一个索引模型才能正常使用。
 {{% /alert %}}

 #### 核心配置
@@ -467,4 +467,4 @@ OneAPI 的语言识别接口，无法正确的识别其他模型（会始终识
    "charsPointsPrice": 0
  }
 }
-```
+```
--- a/docSite/content/zh-cn/docs/development/openapi/chat.md
+++ b/docSite/content/zh-cn/docs/development/openapi/chat.md
@@ -7,6 +7,12 @@ toc: true
 weight: 852
 ---

+# 如何获取 AppId
+
+可在应用详情的路径里获取 AppId。
+
+![](/imgs/appid.png)
+
 # 发起对话

 {{% alert icon="🤖 " context="success" %}}
@@ -102,8 +108,8 @@ curl --location --request POST 'http://localhost:3000/api/v1/chat/completions' \
 {{% alert context="info" %}}
 - headers.Authorization: Bearer {{apikey}}
 - chatId: string | undefined 。
-  - 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。 不会将你的记录存储到数据库中，你也无法在记录汇总中查阅到。
-  - 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。
+  - 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。
+  - 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题，其余 message 会被忽略。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。
 - messages: 结构与 [GPT接口](https://platform.openai.com/docs/api-reference/chat/object) chat模式一致。
 - responseChatItemId: string | undefined 。如果传入，则会将该值作为本次对话的响应消息的 ID，FastGPT 会自动将该 ID 存入数据库。请确保，在当前`chatId`下，`responseChatItemId`是唯一的。
 - detail: 是否返回中间值（模块状态，响应的完整结果等），`stream模式`下会通过`event`进行区分，`非stream模式`结果保存在`responseData`中。
@@ -672,7 +678,7 @@ curl --location --request POST 'http://localhost:3000/api/core/chat/getHistories
    "appId": "appId",
    "offset": 0,
    "pageSize": 20,
-    "source: "api"
+    "source": "api"
 }'
 ```

--- a/docSite/content/zh-cn/docs/development/openapi/dataset.md
+++ b/docSite/content/zh-cn/docs/development/openapi/dataset.md
@@ -735,7 +735,7 @@ data 为集合的 ID。

 **4.8.19+**
 ```bash
-curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listv2' \
+curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listV2' \
 --header 'Authorization: Bearer {{authorization}}' \
 --header 'Content-Type: application/json' \
 --data-raw '{
--- a/docSite/content/zh-cn/docs/development/openapi/intro.md
+++ b/docSite/content/zh-cn/docs/development/openapi/intro.md
@@ -1,6 +1,6 @@
 ---
-title: 'Api Key 使用与鉴权'
-description: 'FastGPT Api Key 使用与鉴权'
+title: 'OpenAPI 介绍'
+description: 'FastGPT OpenAPI 介绍'
 icon: 'key'
 draft: false
 toc: true
@@ -27,6 +27,7 @@ FastGPT 的 API Key **有 2 类**，一类是全局通用的 key (无法直接
 | --------------------- | --------------------- |
 | ![](/imgs/fastgpt-api2.jpg) | ![](/imgs/fastgpt-api1.jpg) |

+
 ## 基本配置

 OpenAPI 中，所有的接口都通过 Header.Authorization 进行鉴权。
--- a/docSite/content/zh-cn/docs/development/openapi/share.md
+++ b/docSite/content/zh-cn/docs/development/openapi/share.md
@@ -11,7 +11,7 @@ weight: 860

 在 FastGPT V4.6.4 中，我们修改了分享链接的数据读取方式，为每个用户生成一个 localId，用于标识用户，从云端拉取对话记录。但是这种方式仅能保障用户在同一设备同一浏览器中使用，如果切换设备或者清空浏览器缓存则会丢失这些记录。这种方式存在一定的风险，因此我们仅允许用户拉取近`30天`的`20条`记录。

-分享链接身份鉴权设计的目的在于，将 FastGPT 的对话框快速、安全的接入到你现有的系统中，仅需 2 个接口即可实现。
+分享链接身份鉴权设计的目的在于，将 FastGPT 的对话框快速、安全的接入到你现有的系统中，仅需 2 个接口即可实现。该功能目前只在商业版中提供。

 ## 使用说明

--- a/docSite/content/zh-cn/docs/development/sealos.md
+++ b/docSite/content/zh-cn/docs/development/sealos.md
@@ -60,6 +60,10 @@ FastGPT 使用了 one-api 项目来管理模型池，其可以兼容 OpenAI 、A

 ### 3. 配置模型

+### 4. 配置模型
+
+务必先配置至少一组模型，否则系统无法正常使用。
+
 [点击查看模型配置教程](/docs/development/modelConfig/intro/)

 ## 收费
--- a/docSite/content/zh-cn/docs/development/upgrading/4818.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4818.md
@@ -1,5 +1,5 @@
 ---
-title: 'V4.8.18'
+title: 'V4.8.18(包含升级脚本)'
 description: 'FastGPT V4.8.18 更新说明'
 icon: 'upgrade'
 draft: false
--- a/docSite/content/zh-cn/docs/development/upgrading/482.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/482.md
@@ -20,7 +20,7 @@ SANDBOX_URL=内网地址

 ## Docker 部署

-可以拉取最新 [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/files/docker/docker-compose.yml) 文件参考
+可以拉取最新 [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker/docker-compose.yml) 文件参考

 1. 新增一个容器 `sandbox`
 2. fastgpt 和 fastgpt-pro(商业版) 容器新增环境变量: `SANDBOX_URL`
--- a/docSite/content/zh-cn/docs/development/upgrading/4820.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4820.md
@@ -17,8 +17,8 @@ weight: 804

 ### 3. 更新镜像：

- 更新 fastgpt 镜像 tag: v4.8.20-fix
- 更新 fastgpt-pro 商业版镜像 tag: v4.8.20-fix
+- 更新 fastgpt 镜像 tag: v4.8.20-fix2
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.20-fix2
 - Sandbox 镜像无需更新

 ### 4. 运行升级脚本
@@ -35,7 +35,7 @@ curl --location --request POST 'https://{{host}}/api/admin/initv4820' \

 ## 完整更新内容

-1. 新增 - 可视化模型参数配置。预设超过 100 个模型配置。同时支持所有类型模型的一键测试。（预计下个版本会完全支持在页面上配置渠道）。
+1. 新增 - 可视化模型参数配置，取代原配置文件配置模型。预设超过 100 个模型配置。同时支持所有类型模型的一键测试。（预计下个版本会完全支持在页面上配置渠道）。
 2. 新增 - DeepSeek resoner 模型支持输出思考过程。
 3. 新增 - 使用记录导出和仪表盘。
 4. 新增 - markdown 语法扩展，支持音视频（代码块 audio 和 video）。
--- a/docSite/content/zh-cn/docs/development/upgrading/4821.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4821.md
@@ -0,0 +1,39 @@
+---
+title: 'V4.8.21'
+description: 'FastGPT V4.8.21 更新说明'
+icon: 'upgrade'
+draft: false
+toc: true
+weight: 803
+---
+
+## 更新指南
+
+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.21-fix
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.21-fix
+- Sandbox 镜像无需更新
+
+## 完整更新内容
+
+1. 新增 - 弃用/已删除的插件提示。
+2. 新增 - 对话日志按来源分类、标题检索、导出功能。
+3. 新增 - 全局变量支持拖拽排序。
+4. 新增 - LLM 模型支持 top_p, response_format, json_schema 参数。
+5. 新增 - Doubao1.5 模型预设。阿里 embedding3 预设。
+6. 新增 - 向量模型支持归一化配置，以便适配未归一化的向量模型，例如 Doubao 的 embedding 模型。
+6. 新增 - AI 对话节点，支持输出思考过程结果，可用于其他节点引用。
+7. 优化 - 网站嵌入式聊天窗口，增加窗口位置适配。
+8. 优化 - 模型未配置时错误提示。
+9. 优化 - 适配非 Stream 模式思考输出。
+10. 优化 - 增加 TTS voice 未配置时的空指针保护。
+11. 优化 - Markdown 链接解析分割规则，改成严格匹配模式，牺牲兼容多种情况，减少误解析。
+12. 优化 - 减少未登录用户的数据获取范围，提高系统隐私性。
+13. 修复 - 简易模式，切换到其他非视觉模型时候，会强制关闭图片识别。
+14. 修复 - o1,o3 模型，在测试时候字段映射未生效导致报错。
+15. 修复 - 公众号对话空指针异常。
+16. 修复 - 多个音频/视频文件展示异常。
+17. 修复 - 分享链接鉴权报错后无限循环。
--- a/docSite/content/zh-cn/docs/development/upgrading/4822.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4822.md
@@ -0,0 +1,61 @@
+---
+title: 'V4.8.22(包含升级脚本)'
+description: 'FastGPT V4.8.22 更新说明'
+icon: 'upgrade'
+draft: false
+toc: true
+weight: 802
+---
+
+## 🌟更新指南
+
+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.22
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.22
+- Sandbox 镜像无需更新
+
+### 3. 运行升级脚本
+
+仅商业版，并提供 Saas 服务的用户需要运行该升级脚本。
+
+从任意终端，发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`；{{host}} 替换成**FastGPT 域名**。
+
+```bash
+curl --location --request POST 'https://{{host}}/api/admin/initv4822' \
+--header 'rootkey: {{rootkey}}' \
+--header 'Content-Type: application/json'
+```
+
+会迁移联系方式到对应用户表中。
+
+## 🚀 新增内容
+
+1. AI 对话节点解析 `<think></think>` 标签内容作为思考链，便于各类模型进行思考链输出。需主动开启模型输出思考。
+2. 对话 API 优化，无论是否传递 chatId，都会保存对话日志。未传递 chatId，则随机生成一个 chatId 来进行存储。
+3. ppio 模型提供商
+
+## ⚙️ 优化
+
+1. 模型未配置时提示，减少冲突提示。
+2. 使用记录代码。
+3. 内容提取节点，字段描述过长时换行。同时修改其输出名用 key，而不是 description。
+4. 团队管理交互。
+5. 对话接口，非流响应，增加报错字段。
+
+## 🐛 修复
+
+1. 思考内容未进入到输出 Tokens.
+2. 思考链流输出时，有时与正文顺序偏差。
+3. API 调用工作流，如果传递的图片不支持 Head 检测时，图片会被过滤。已增加该类错误检测，避免被错误过滤。
+4. 模板市场部分模板错误。
+5. 免登录窗口无法正常判断语言识别是否开启。
+6. 对话日志导出，未兼容 sub path。
+7. 切换团队时未刷新成员列表
+8. list 接口在联查 member 时，存在空指针可能性。 
+9. 工作流基础节点无法升级。
+10. 向量检索结果未去重。
+11. 用户选择节点无法正常连线。
+12. 对话记录保存时，source 未正常记录。
--- a/docSite/content/zh-cn/docs/development/upgrading/4823.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4823.md
@@ -0,0 +1,54 @@
+---
+title: 'V4.8.23'
+description: 'FastGPT V4.8.23 更新说明'
+icon: 'upgrade'
+draft: false
+toc: true
+weight: 802
+---
+
+## 更新指南
+
+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.23-fix
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.23-fix
+- Sandbox 镜像无需更新
+
+### 3. 运行升级脚本
+
+从任意终端，发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`；{{host}} 替换成**FastGPT 域名**。
+
+```bash
+curl --location --request POST 'https://{{host}}/api/admin/initv4823' \
+--header 'rootkey: {{rootkey}}' \
+--header 'Content-Type: application/json'
+```
+
+脚本会清理一些知识库脏数据，主要是多余的全文索引。
+
+## 🚀 新增内容
+
+1. 增加默认“知识库文本理解模型”配置
+2. AI proxy V1版，可替换 OneAPI使用，同时提供完整模型调用日志，便于排查问题。
+3. 增加工单入口支持。
+
+## ⚙️ 优化
+
+1. 模型配置表单，增加必填项校验。
+2. 集合列表数据统计方式，提高大数据量统计性能。
+3. 优化数学公式，转义 Latex 格式成 Markdown 格式。
+4. 解析文档图片，图片太大时，自动忽略。
+5. 时间选择器，当天开始时间自动设0，结束设置设 23:59:59，避免 UI 与实际逻辑偏差。
+6. 升级 mongoose 库版本依赖。
+
+## 🐛 修复
+
+1. 标签过滤时，子文件夹未成功过滤。
+2. 暂时移除 md 阅读优化，避免链接分割错误。
+3. 离开团队时，未刷新成员列表。
+4. PPTX 编码错误，导致解析失败。
+5. 删除知识库单条数据时，全文索引未跟随删除。
+6. 修复 Mongo Dataset text 索引在查询数据时未生效。
--- a/docSite/content/zh-cn/docs/development/upgrading/483.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/483.md
@@ -15,7 +15,7 @@ weight: 821

 ## V4.8.3 更新说明

-1. 新增 - 支持 Milvus 数据库， 可参考最新的 [docker-compose-milvus.yml](https://github.com/labring/FastGPT/blob/main/files/docker/docker-compose-milvus.yml).
+1. 新增 - 支持 Milvus 数据库， 可参考最新的 [docker-compose-milvus.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker/docker-compose-milvus.yml).
 2. 新增 - 给 chat 接口 empty answer 增加 log，便于排查模型问题。
 3. 新增 - ifelse判断器，字符串支持正则。
 4. 新增 - 代码运行支持 console.log 输出调试。
--- a/docSite/content/zh-cn/docs/guide/course/quick-start.md
+++ b/docSite/content/zh-cn/docs/guide/course/quick-start.md
@@ -7,11 +7,11 @@ toc: true
 weight: 102
 ---

-更多使用技巧，[查看视屏教程](https://www.bilibili.com/video/BV1sH4y1T7s9)
+更多使用技巧，[查看视频教程](https://www.bilibili.com/video/BV1sH4y1T7s9)

 ## 知识库

-开始前，请准备一份测试电子文档，WORD，PDF，TXT，excel，markdown 都可以，比如公司休假制度，不涉密的销售说辞，产品知识等等。
+开始前，请准备一份测试电子文档，WORD、PDF、TXT、excel、markdown 都可以，比如公司休假制度、不涉密的销售说辞、产品知识等等。

 这里使用 FastGPT 中文 README 文件为例。

@@ -31,7 +31,7 @@ weight: 102

 ![](/imgs/upload-data.png)

-点击上传后我们需要等待数据处理完成，等到我们上传的文件状态为可用。
+点击上传后我们需要等待数据处理完成，直到我们上传的文件状态为可用。

 ![](/imgs/upload-data2.png)

--- a/docSite/content/zh-cn/docs/guide/workbench/workflow/dataset_search.md
+++ b/docSite/content/zh-cn/docs/guide/workbench/workflow/dataset_search.md
@@ -7,7 +7,7 @@ toc: true
 weight: 234
 ---

-知识库搜索具体参数说明，以及内部逻辑请移步：[FastGPT知识库搜索方案](/docs/course/data_search/)
+知识库搜索具体参数说明，以及内部逻辑请移步：[FastGPT知识库搜索方案](/docs/guide/knowledge_base/rag/)

 ## 特点

@@ -27,7 +27,7 @@ weight: 234

 ### 输入 - 搜索参数

-[点击查看参数介绍](/docs/course/data_search/#搜索参数)
+[点击查看参数介绍](/docs/guide/knowledge_base/dataset_engine/#搜索参数)

 ### 输出 - 引用内容

--- a/docSite/content/zh-cn/docs/use-cases/external-integration/openapi.md
+++ b/docSite/content/zh-cn/docs/use-cases/external-integration/openapi.md
@@ -20,7 +20,7 @@ weight: 502
 ![](/imgs/fastgpt-api1.jpg)

 {{% alert icon="🍅" context="success" %}}
-Tips: 安全起见，你可以设置一个额度或者过期时间，放置 key 被滥用。
+Tips: 安全起见，你可以设置一个额度或者过期时间，防止 key 被滥用。
 {{% /alert %}}


--- a/package.json
+++ b/package.json
@@ -7,7 +7,7 @@
    "format-code": "prettier --config \"./.prettierrc.js\" --write \"./**/src/**/*.{ts,tsx,scss}\"",
    "format-doc": "zhlint --dir ./docSite *.md --fix",
    "gen:theme-typings": "chakra-cli tokens packages/web/styles/theme.ts --out node_modules/.pnpm/node_modules/@chakra-ui/styled-system/dist/theming.types.d.ts",
-    "postinstall": "sh ./scripts/postinstall.sh",
+    "postinstall": "pnpm gen:theme-typings",
    "initIcon": "node ./scripts/icon/init.js",
    "previewIcon": "node ./scripts/icon/index.js",
    "api:gen": "tsc ./scripts/openapi/index.ts && node ./scripts/openapi/index.js && npx @redocly/cli build-docs ./scripts/openapi/openapi.json -o ./projects/app/public/openapi/index.html",
--- a/packages/README.md
+++ b/packages/README.md
@@ -0,0 +1,3 @@
+# 目录说明
+
+该目录为 FastGPT 的依赖包，多端复用。
--- a/packages/global/common/error/code/dataset.ts
+++ b/packages/global/common/error/code/dataset.ts
@@ -4,6 +4,7 @@ import { ErrType } from '../errorCode';
 /* dataset: 501000 */
 export enum DatasetErrEnum {
  unExist = 'unExistDataset',
+  unExistCollection = 'unExistCollection',
  unAuthDataset = 'unAuthDataset',
  unCreateCollection = 'unCreateCollection',
  unAuthDatasetCollection = 'unAuthDatasetCollection',
@@ -28,6 +29,10 @@ const datasetErr = [
    statusText: DatasetErrEnum.unExist,
    message: 'core.dataset.error.unExistDataset'
  },
+  {
+    statusText: DatasetErrEnum.unExistCollection,
+    message: i18nT('common:error_collection_not_exist')
+  },
  {
    statusText: DatasetErrEnum.unAuthDataset,
    message: 'core.dataset.error.unAuthDataset'
--- a/packages/global/common/file/constants.ts
+++ b/packages/global/common/file/constants.ts
@@ -16,8 +16,8 @@ export const bucketNameMap = {
  }
 };

-export const ReadFileBaseUrl = `${process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}/api/common/file/read`;
+export const ReadFileBaseUrl = `${process.env.FILE_DOMAIN || process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}/api/common/file/read`;

 export const documentFileType = '.txt, .docx, .csv, .xlsx, .pdf, .md, .html, .pptx';
 export const imageFileType =
-  '.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif';
+  '.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif, .raw, .cr2, .nef, .arw, .dng, .psd, .ai, .eps, .emf, .wmf, .jfif, .exif, .pgm, .ppm, .pbm, .jp2, .j2k, .jpf, .jpx, .jpm, .mj2, .xbm, .pcx';
--- a/packages/global/common/file/tools.ts
+++ b/packages/global/common/file/tools.ts
@@ -1,5 +1,5 @@
 import { detect } from 'jschardet';
-import { documentFileType, imageFileType } from './constants';
+import { documentFileType } from './constants';
 import { ChatFileTypeEnum } from '../../core/chat/constants';
 import { UserChatItemValueItemType } from '../../core/chat/type';
 import * as fs from 'fs';
@@ -25,6 +25,7 @@ export const detectFileEncodingByPath = async (path: string) => {
  const fd = await fs.promises.open(path, 'r');
  try {
    // Read file head
+    // @ts-ignore
    const { bytesRead } = await fd.read(buffer, 0, MAX_BYTES, 0);
    const actualBuffer = buffer.slice(0, bytesRead);

@@ -37,40 +38,49 @@ export const detectFileEncodingByPath = async (path: string) => {
 // Url => user upload file type
 export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file'] | undefined => {
  if (typeof url !== 'string') return;
-  const parseUrl = new URL(url, 'https://locaohost:3000');

-  const filename = (() => {
-    // Check base64 image
-    if (url.startsWith('data:image/')) {
-      const mime = url.split(',')[0].split(':')[1].split(';')[0];
-      return `image.${mime.split('/')[1]}`;
-    }
-    // Old version file url: https://xxx.com/file/read?filename=xxx.pdf
-    const filenameQuery = parseUrl.searchParams.get('filename');
-    if (filenameQuery) return filenameQuery;
+  // Handle base64 image
+  if (url.startsWith('data:')) {
+    const matches = url.match(/^data:([^;]+);base64,/);
+    if (!matches) return;

-    // Common file： https://xxx.com/xxx.pdf?xxxx=xxx
-    const pathname = parseUrl.pathname;
-    if (pathname) return pathname.split('/').pop();
-  })();
+    const mimeType = matches[1].toLowerCase();
+    if (!mimeType.startsWith('image/')) return;

-  if (!filename) return;
-
-  const extension = filename.split('.').pop()?.toLowerCase() || '';
-
-  if (!extension) return;
-
-  if (documentFileType.includes(extension)) {
+    const extension = mimeType.split('/')[1];
    return {
-      type: ChatFileTypeEnum.file,
-      name: filename,
+      type: ChatFileTypeEnum.image,
+      name: `image.${extension}`,
      url
    };
  }
-  if (imageFileType.includes(extension)) {
+
+  try {
+    const parseUrl = new URL(url, 'https://localhost:3000');
+
+    // Get filename from URL
+    const filename = parseUrl.searchParams.get('filename') || parseUrl.pathname.split('/').pop();
+    const extension = filename?.split('.').pop()?.toLowerCase() || '';
+
+    // If it's a document type, return as file, otherwise treat as image
+    if (extension && documentFileType.includes(extension)) {
+      return {
+        type: ChatFileTypeEnum.file,
+        name: filename || 'null',
+        url
+      };
+    }
+
+    // Default to image type for non-document files
    return {
      type: ChatFileTypeEnum.image,
-      name: filename,
+      name: filename || 'null.png',
+      url
+    };
+  } catch (error) {
+    return {
+      type: ChatFileTypeEnum.image,
+      name: 'invalid.png',
      url
    };
  }
--- a/packages/global/common/string/time.ts
+++ b/packages/global/common/string/time.ts
@@ -7,12 +7,14 @@ import { i18nT } from '../../../web/i18n/utils';
 dayjs.extend(utc);
 dayjs.extend(timezone);

-export const formatTime2YMDHMW = (time?: Date) => dayjs(time).format('YYYY-MM-DD HH:mm:ss dddd');
-export const formatTime2YMDHMS = (time?: Date) =>
+export const formatTime2YMDHMW = (time?: Date | number) =>
+  dayjs(time).format('YYYY-MM-DD HH:mm:ss dddd');
+export const formatTime2YMDHMS = (time?: Date | number) =>
  time ? dayjs(time).format('YYYY-MM-DD HH:mm:ss') : '';
-export const formatTime2YMDHM = (time?: Date) =>
+export const formatTime2YMDHM = (time?: Date | number) =>
  time ? dayjs(time).format('YYYY-MM-DD HH:mm') : '';
-export const formatTime2YMD = (time?: Date) => (time ? dayjs(time).format('YYYY-MM-DD') : '');
+export const formatTime2YMD = (time?: Date | number) =>
+  time ? dayjs(time).format('YYYY-MM-DD') : '';
 export const formatTime2HM = (time: Date = new Date()) => dayjs(time).format('HH:mm');

 /**
--- a/packages/global/common/string/tools.ts
+++ b/packages/global/common/string/tools.ts
@@ -26,7 +26,7 @@ export const simpleText = (text = '') => {
 };

 export const valToStr = (val: any) => {
-  if (val === undefined) return 'undefined';
+  if (val === undefined) return '';
  if (val === null) return 'null';

  if (typeof val === 'object') return JSON.stringify(val);
--- a/packages/global/common/system/types/index.d.ts
+++ b/packages/global/common/system/types/index.d.ts
@@ -41,6 +41,7 @@ export type FastGPTConfigFileType = {
 };

 export type FastGPTFeConfigsType = {
+  show_workorder?: boolean;
  show_emptyChat?: boolean;
  register_method?: ['email' | 'phone' | 'sync'];
  login_method?: ['email' | 'phone']; // Attention: login method is diffrent with oauth
@@ -53,6 +54,7 @@ export type FastGPTFeConfigsType = {
  show_promotion?: boolean;
  show_team_chat?: boolean;
  show_compliance_copywriting?: boolean;
+  show_aiproxy?: boolean;
  concatMd?: string;

  docUrl?: string;
--- a/packages/global/core/ai/model.d.ts
+++ b/packages/global/core/ai/model.d.ts
@@ -17,6 +17,8 @@ type BaseModelItemType = {
  isActive?: boolean;
  isCustom?: boolean;
  isDefault?: boolean;
+  isDefaultDatasetTextModel?: boolean;
+  isDefaultDatasetImageModel?: boolean;

  // If has requestUrl, it will request the model directly
  requestUrl?: string;
@@ -26,11 +28,16 @@ type BaseModelItemType = {
 export type LLMModelItemType = PriceType &
  BaseModelItemType & {
    type: ModelTypeEnum.llm;
+    // Model params
    maxContext: number;
    maxResponse: number;
    quoteMaxToken: number;
    maxTemperature?: number;

+    showTopP?: boolean;
+    responseFormatList?: string[];
+    showStopSign?: boolean;
+
    censor?: boolean;
    vision?: boolean;
    reasoning?: boolean;
@@ -59,6 +66,7 @@ export type EmbeddingModelItemType = PriceType &
    maxToken: number; // model max token
    weight: number; // training weight
    hidden?: boolean; // Disallow creation
+    normalization?: boolean; // normalization processing
    defaultConfig?: Record<string, any>; // post request config
    dbConfig?: Record<string, any>; // Custom parameters for storage
    queryConfig?: Record<string, any>; // Custom parameters for query
--- a/packages/global/core/ai/provider.ts
+++ b/packages/global/core/ai/provider.ts
@@ -22,6 +22,7 @@ export type ModelProviderIdType =
  | 'StepFun'
  | 'Yi'
  | 'Siliconflow'
+  | 'PPIO'
  | 'Ollama'
  | 'BAAI'
  | 'FishAudio'
@@ -71,11 +72,6 @@ export const ModelProviderList: ModelProviderType[] = [
    name: 'Groq',
    avatar: 'model/groq'
  },
-  {
-    id: 'AliCloud',
-    name: i18nT('common:model_alicloud'),
-    avatar: 'model/alicloud'
-  },
  {
    id: 'Qwen',
    name: i18nT('common:model_qwen'),
@@ -86,6 +82,11 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_doubao'),
    avatar: 'model/doubao'
  },
+  {
+    id: 'DeepSeek',
+    name: 'DeepSeek',
+    avatar: 'model/deepseek'
+  },
  {
    id: 'ChatGLM',
    name: i18nT('common:model_chatglm'),
@@ -96,11 +97,6 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_ernie'),
    avatar: 'model/ernie'
  },
-  {
-    id: 'DeepSeek',
-    name: 'DeepSeek',
-    avatar: 'model/deepseek'
-  },
  {
    id: 'Moonshot',
    name: i18nT('common:model_moonshot'),
@@ -162,11 +158,21 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_moka'),
    avatar: 'model/moka'
  },
+  {
+    id: 'AliCloud',
+    name: i18nT('common:model_alicloud'),
+    avatar: 'model/alicloud'
+  },
  {
    id: 'Siliconflow',
    name: i18nT('common:model_siliconflow'),
    avatar: 'model/siliconflow'
  },
+  {
+    id: 'PPIO',
+    name: i18nT('common:model_ppio'),
+    avatar: 'model/ppio'
+  },
  {
    id: 'Other',
    name: i18nT('common:model_other'),
--- a/packages/global/core/ai/type.d.ts
+++ b/packages/global/core/ai/type.d.ts
@@ -1,14 +1,12 @@
 import openai from 'openai';
 import type {
  ChatCompletionMessageToolCall,
-  ChatCompletionChunk,
  ChatCompletionMessageParam as SdkChatCompletionMessageParam,
  ChatCompletionToolMessageParam,
  ChatCompletionContentPart as SdkChatCompletionContentPart,
  ChatCompletionUserMessageParam as SdkChatCompletionUserMessageParam,
  ChatCompletionToolMessageParam as SdkChatCompletionToolMessageParam,
-  ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam,
-  ChatCompletionContentPartText
+  ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam
 } from 'openai/resources';
 import { ChatMessageTypeEnum } from './constants';
 import { WorkflowInteractiveResponseType } from '../workflow/template/system/interactive/type';
@@ -48,6 +46,7 @@ export type ChatCompletionMessageParam = (
  | CustomChatCompletionToolMessageParam
  | CustomChatCompletionAssistantMessageParam
 ) & {
+  reasoning_text?: string;
  dataId?: string;
  hideInUI?: boolean;
 };
@@ -71,7 +70,8 @@ export type ChatCompletionMessageFunctionCall =
  };

 // Stream response
-export type StreamChatType = Stream<ChatCompletionChunk>;
+export type StreamChatType = Stream<openai.Chat.Completions.ChatCompletionChunk>;
+export type UnStreamChatType = openai.Chat.Completions.ChatCompletion;

 export default openai;
 export * from 'openai';
--- a/packages/global/core/app/type.d.ts
+++ b/packages/global/core/app/type.d.ts
@@ -74,13 +74,17 @@ export type AppDetailType = AppSchema & {
 export type AppSimpleEditFormType = {
  // templateId: string;
  aiSettings: {
-    model: string;
-    systemPrompt?: string | undefined;
-    temperature?: number;
-    maxToken?: number;
-    isResponseAnswerText: boolean;
+    [NodeInputKeyEnum.aiModel]: string;
+    [NodeInputKeyEnum.aiSystemPrompt]?: string | undefined;
+    [NodeInputKeyEnum.aiChatTemperature]?: number;
+    [NodeInputKeyEnum.aiChatMaxToken]?: number;
+    [NodeInputKeyEnum.aiChatIsResponseText]: boolean;
    maxHistories: number;
-    [NodeInputKeyEnum.aiChatReasoning]?: boolean;
+    [NodeInputKeyEnum.aiChatReasoning]?: boolean; // Is open reasoning mode
+    [NodeInputKeyEnum.aiChatTopP]?: number;
+    [NodeInputKeyEnum.aiChatStopSign]?: string;
+    [NodeInputKeyEnum.aiChatResponseFormat]?: string;
+    [NodeInputKeyEnum.aiChatJsonSchema]?: string;
  };
  dataset: {
    datasets: SelectedDatasetType;
@@ -119,6 +123,10 @@ export type SettingAIDataType = {
  maxHistories?: number;
  [NodeInputKeyEnum.aiChatVision]?: boolean; // Is open vision mode
  [NodeInputKeyEnum.aiChatReasoning]?: boolean; // Is open reasoning mode
+  [NodeInputKeyEnum.aiChatTopP]?: number;
+  [NodeInputKeyEnum.aiChatStopSign]?: string;
+  [NodeInputKeyEnum.aiChatResponseFormat]?: string;
+  [NodeInputKeyEnum.aiChatJsonSchema]?: string;
 };

 // variable
--- a/packages/global/core/app/utils.ts
+++ b/packages/global/core/app/utils.ts
@@ -7,6 +7,8 @@ import { StoreNodeItemType } from '../workflow/type/node';
 import { DatasetSearchModeEnum } from '../dataset/constants';
 import { WorkflowTemplateBasicType } from '../workflow/type';
 import { AppTypeEnum } from './constants';
+import { AppErrEnum } from '../../common/error/code/app';
+import { PluginErrEnum } from '../../common/error/code/plugin';

 export const getDefaultAppForm = (): AppSimpleEditFormType => {
  return {
@@ -117,7 +119,8 @@ export const appWorkflow2Form = ({
        version: node.version,
        inputs: node.inputs,
        outputs: node.outputs,
-        templateType: FlowNodeTemplateTypeEnum.other
+        templateType: FlowNodeTemplateTypeEnum.other,
+        pluginData: node.pluginData
      });
    } else if (node.flowNodeType === FlowNodeTypeEnum.systemConfig) {
      defaultAppForm.chatConfig = getAppChatConfig({
@@ -147,3 +150,18 @@ export const getAppType = (config?: WorkflowTemplateBasicType | AppSimpleEditFor
  }
  return '';
 };
+
+export const checkAppUnExistError = (error?: string) => {
+  const unExistError: Array<string> = [
+    AppErrEnum.unAuthApp,
+    AppErrEnum.unExist,
+    PluginErrEnum.unAuth,
+    PluginErrEnum.unExist
+  ];
+
+  if (!!error && unExistError.includes(error)) {
+    return error;
+  } else {
+    return undefined;
+  }
+};
--- a/packages/global/core/chat/adapt.ts
+++ b/packages/global/core/chat/adapt.ts
@@ -46,7 +46,16 @@ export const chats2GPTMessages = ({

  messages.forEach((item) => {
    const dataId = reserveId ? item.dataId : undefined;
-    if (item.obj === ChatRoleEnum.Human) {
+    if (item.obj === ChatRoleEnum.System) {
+      const content = item.value?.[0]?.text?.content;
+      if (content) {
+        results.push({
+          dataId,
+          role: ChatCompletionRequestMessageRoleEnum.System,
+          content
+        });
+      }
+    } else if (item.obj === ChatRoleEnum.Human) {
      const value = item.value
        .map((item) => {
          if (item.type === ChatItemValueTypeEnum.text) {
@@ -80,15 +89,6 @@ export const chats2GPTMessages = ({
        role: ChatCompletionRequestMessageRoleEnum.User,
        content: simpleUserContentPart(value)
      });
-    } else if (item.obj === ChatRoleEnum.System) {
-      const content = item.value?.[0]?.text?.content;
-      if (content) {
-        results.push({
-          dataId,
-          role: ChatCompletionRequestMessageRoleEnum.System,
-          content
-        });
-      }
    } else {
      const aiResults: ChatCompletionMessageParam[] = [];

@@ -349,7 +349,7 @@ export const chatValue2RuntimePrompt = (value: ChatItemValueItemType[]): Runtime
  };
  value.forEach((item) => {
    if (item.type === 'file' && item.file) {
-      prompt.files?.push(item.file);
+      prompt.files.push(item.file);
    } else if (item.text) {
      prompt.text += item.text.content;
    }
--- a/packages/global/core/dataset/type.d.ts
+++ b/packages/global/core/dataset/type.d.ts
@@ -192,6 +192,7 @@ export type DatasetCollectionItemType = CollectionWithDatasetType & {
  sourceId?: string;
  file?: DatasetFileSchema;
  permission: DatasetPermission;
+  indexAmount: number;
 };

 /* ================= data ===================== */
--- a/packages/global/core/workflow/constants.ts
+++ b/packages/global/core/workflow/constants.ts
@@ -33,8 +33,10 @@ export enum WorkflowIOValueTypeEnum {
  dynamic = 'dynamic',

  // plugin special type
-  selectApp = 'selectApp',
-  selectDataset = 'selectDataset'
+  selectDataset = 'selectDataset',
+
+  // abandon
+  selectApp = 'selectApp'
 }

 export const toolValueTypeList = [
@@ -142,6 +144,10 @@ export enum NodeInputKeyEnum {
  aiChatVision = 'aiChatVision',
  stringQuoteText = 'stringQuoteText',
  aiChatReasoning = 'aiChatReasoning',
+  aiChatTopP = 'aiChatTopP',
+  aiChatStopSign = 'aiChatStopSign',
+  aiChatResponseFormat = 'aiChatResponseFormat',
+  aiChatJsonSchema = 'aiChatJsonSchema',

  // dataset
  datasetSelectList = 'datasets',
@@ -154,6 +160,10 @@ export enum NodeInputKeyEnum {
  datasetSearchExtensionBg = 'datasetSearchExtensionBg',
  collectionFilterMatch = 'collectionFilterMatch',
  authTmbId = 'authTmbId',
+  datasetDeepSearch = 'datasetDeepSearch',
+  datasetDeepSearchModel = 'datasetDeepSearchModel',
+  datasetDeepSearchMaxTimes = 'datasetDeepSearchMaxTimes',
+  datasetDeepSearchBg = 'datasetDeepSearchBg',

  // concat dataset
  datasetQuoteList = 'system_datasetQuoteList',
--- a/packages/global/core/workflow/node/constant.ts
+++ b/packages/global/core/workflow/node/constant.ts
@@ -140,7 +140,14 @@ export enum FlowNodeTypeEnum {
 }

 // node IO value type
-export const FlowValueTypeMap = {
+export const FlowValueTypeMap: Record<
+  WorkflowIOValueTypeEnum,
+  {
+    label: string;
+    value: WorkflowIOValueTypeEnum;
+    abandon?: boolean;
+  }
+> = {
  [WorkflowIOValueTypeEnum.string]: {
    label: 'String',
    value: WorkflowIOValueTypeEnum.string
@@ -189,10 +196,6 @@ export const FlowValueTypeMap = {
    label: i18nT('common:core.workflow.Dataset quote'),
    value: WorkflowIOValueTypeEnum.datasetQuote
  },
-  [WorkflowIOValueTypeEnum.selectApp]: {
-    label: i18nT('common:plugin.App'),
-    value: WorkflowIOValueTypeEnum.selectApp
-  },
  [WorkflowIOValueTypeEnum.selectDataset]: {
    label: i18nT('common:core.chat.Select dataset'),
    value: WorkflowIOValueTypeEnum.selectDataset
@@ -200,6 +203,11 @@ export const FlowValueTypeMap = {
  [WorkflowIOValueTypeEnum.dynamic]: {
    label: i18nT('common:core.workflow.dynamic_input'),
    value: WorkflowIOValueTypeEnum.dynamic
+  },
+  [WorkflowIOValueTypeEnum.selectApp]: {
+    label: 'selectApp',
+    value: WorkflowIOValueTypeEnum.selectApp,
+    abandon: true
  }
 };

@@ -219,3 +227,6 @@ export const datasetQuoteValueDesc = `{
  q: string;
  a: string
 }[]`;
+export const datasetSelectValueDesc = `{
+  datasetId: string;
+}[]`;
--- a/packages/global/core/workflow/runtime/type.d.ts
+++ b/packages/global/core/workflow/runtime/type.d.ts
@@ -123,6 +123,7 @@ export type DispatchNodeResponseType = {
  temperature?: number;
  maxToken?: number;
  quoteList?: SearchDataResponseItemType[];
+  reasoningText?: string;
  historyPreview?: {
    obj: `${ChatRoleEnum}`;
    value: string;
@@ -133,9 +134,17 @@ export type DispatchNodeResponseType = {
  limit?: number;
  searchMode?: `${DatasetSearchModeEnum}`;
  searchUsingReRank?: boolean;
-  extensionModel?: string;
-  extensionResult?: string;
-  extensionTokens?: number;
+  queryExtensionResult?: {
+    model: string;
+    inputTokens: number;
+    outputTokens: number;
+    query: string;
+  };
+  deepSearchResult?: {
+    model: string;
+    inputTokens: number;
+    outputTokens: number;
+  };

  // dataset concat
  concatLength?: number;
@@ -198,6 +207,11 @@ export type DispatchNodeResponseType = {

  // tool params
  toolParamsResult?: Record<string, any>;
+
+  // abandon
+  extensionModel?: string;
+  extensionResult?: string;
+  extensionTokens?: number;
 };

 export type DispatchNodeResultType<T = {}> = {
@@ -221,6 +235,10 @@ export type AIChatNodeProps = {
  [NodeInputKeyEnum.aiChatIsResponseText]: boolean;
  [NodeInputKeyEnum.aiChatVision]?: boolean;
  [NodeInputKeyEnum.aiChatReasoning]?: boolean;
+  [NodeInputKeyEnum.aiChatTopP]?: number;
+  [NodeInputKeyEnum.aiChatStopSign]?: string;
+  [NodeInputKeyEnum.aiChatResponseFormat]?: string;
+  [NodeInputKeyEnum.aiChatJsonSchema]?: string;

  [NodeInputKeyEnum.aiChatQuoteRole]?: AiChatQuoteRoleType;
  [NodeInputKeyEnum.aiChatQuoteTemplate]?: string;
--- a/packages/global/core/workflow/runtime/utils.ts
+++ b/packages/global/core/workflow/runtime/utils.ts
@@ -10,6 +10,7 @@ import { FlowNodeOutputItemType, ReferenceValueType } from '../type/io';
 import { ChatItemType, NodeOutputItemType } from '../../../core/chat/type';
 import { ChatItemValueTypeEnum, ChatRoleEnum } from '../../../core/chat/constants';
 import { replaceVariable, valToStr } from '../../../common/string/tools';
+import { ChatCompletionChunk } from 'openai/resources';

 export const getMaxHistoryLimitFromNodes = (nodes: StoreNodeItemType[]): number => {
  let limit = 10;
@@ -292,13 +293,12 @@ export const getReferenceVariableValue = ({

 export const formatVariableValByType = (val: any, valueType?: WorkflowIOValueTypeEnum) => {
  if (!valueType) return val;
+  if (val === undefined || val === null) return;
  // Value type check, If valueType invalid, return undefined
  if (valueType.startsWith('array') && !Array.isArray(val)) return undefined;
  if (valueType === WorkflowIOValueTypeEnum.boolean) return Boolean(val);
  if (valueType === WorkflowIOValueTypeEnum.number) return Number(val);
  if (valueType === WorkflowIOValueTypeEnum.string) {
-    if (val === undefined) return 'undefined';
-    if (val === null) return 'null';
    return typeof val === 'object' ? JSON.stringify(val) : String(val);
  }
  if (
--- a/packages/global/core/workflow/template/system/aiChat/index.ts
+++ b/packages/global/core/workflow/template/system/aiChat/index.ts
@@ -63,14 +63,12 @@ export const AiChatModule: FlowNodeTemplateType = {
      key: NodeInputKeyEnum.aiChatTemperature,
      renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
      label: '',
-      value: undefined,
      valueType: WorkflowIOValueTypeEnum.number
    },
    {
      key: NodeInputKeyEnum.aiChatMaxToken,
      renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
      label: '',
-      value: undefined,
      valueType: WorkflowIOValueTypeEnum.number
    },

@@ -98,6 +96,30 @@ export const AiChatModule: FlowNodeTemplateType = {
      valueType: WorkflowIOValueTypeEnum.boolean,
      value: true
    },
+    {
+      key: NodeInputKeyEnum.aiChatTopP,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.number
+    },
+    {
+      key: NodeInputKeyEnum.aiChatStopSign,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },
+    {
+      key: NodeInputKeyEnum.aiChatResponseFormat,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },
+    {
+      key: NodeInputKeyEnum.aiChatJsonSchema,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },
    // settings modal ---
    {
      ...Input_Template_System_Prompt,
@@ -108,7 +130,6 @@ export const AiChatModule: FlowNodeTemplateType = {
    Input_Template_History,
    Input_Template_Dataset_Quote,
    Input_Template_File_Link_Prompt,
-
    { ...Input_Template_UserChatInput, toolDescription: i18nT('workflow:user_question') }
  ],
  outputs: [
@@ -130,6 +151,20 @@ export const AiChatModule: FlowNodeTemplateType = {
      description: i18nT('common:core.module.output.description.Ai response content'),
      valueType: WorkflowIOValueTypeEnum.string,
      type: FlowNodeOutputTypeEnum.static
+    },
+    {
+      id: NodeOutputKeyEnum.reasoningText,
+      key: NodeOutputKeyEnum.reasoningText,
+      required: false,
+      label: i18nT('workflow:reasoning_text'),
+      valueType: WorkflowIOValueTypeEnum.string,
+      type: FlowNodeOutputTypeEnum.static,
+      invalid: true,
+      invalidCondition: ({ inputs, llmModelList }) => {
+        const model = inputs.find((item) => item.key === NodeInputKeyEnum.aiModel)?.value;
+        const modelItem = llmModelList.find((item) => item.model === model);
+        return modelItem?.reasoning !== true;
+      }
    }
  ]
 };
--- a/packages/global/core/workflow/template/system/datasetSearch.ts
+++ b/packages/global/core/workflow/template/system/datasetSearch.ts
@@ -1,5 +1,6 @@
 import {
  datasetQuoteValueDesc,
+  datasetSelectValueDesc,
  FlowNodeInputTypeEnum,
  FlowNodeOutputTypeEnum,
  FlowNodeTypeEnum
@@ -38,7 +39,8 @@ export const DatasetSearchModule: FlowNodeTemplateType = {
      label: i18nT('common:core.module.input.label.Select dataset'),
      value: [],
      valueType: WorkflowIOValueTypeEnum.selectDataset,
-      required: true
+      required: true,
+      valueDesc: datasetSelectValueDesc
    },
    {
      key: NodeInputKeyEnum.datasetSimilarity,
--- a/packages/global/core/workflow/template/system/tools.ts
+++ b/packages/global/core/workflow/template/system/tools.ts
@@ -43,14 +43,12 @@ export const ToolModule: FlowNodeTemplateType = {
      key: NodeInputKeyEnum.aiChatTemperature,
      renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
      label: '',
-      value: undefined,
      valueType: WorkflowIOValueTypeEnum.number
    },
    {
      key: NodeInputKeyEnum.aiChatMaxToken,
      renderTypeList: [FlowNodeInputTypeEnum.hidden], // Set in the pop-up window
      label: '',
-      value: undefined,
      valueType: WorkflowIOValueTypeEnum.number
    },
    {
@@ -60,6 +58,30 @@ export const ToolModule: FlowNodeTemplateType = {
      valueType: WorkflowIOValueTypeEnum.boolean,
      value: true
    },
+    {
+      key: NodeInputKeyEnum.aiChatTopP,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.number
+    },
+    {
+      key: NodeInputKeyEnum.aiChatStopSign,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },
+    {
+      key: NodeInputKeyEnum.aiChatResponseFormat,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },
+    {
+      key: NodeInputKeyEnum.aiChatJsonSchema,
+      renderTypeList: [FlowNodeInputTypeEnum.hidden],
+      label: '',
+      valueType: WorkflowIOValueTypeEnum.string
+    },

    {
      ...Input_Template_System_Prompt,
--- a/packages/global/core/workflow/type/io.d.ts
+++ b/packages/global/core/workflow/type/io.d.ts
@@ -1,3 +1,4 @@
+import { LLMModelItemType } from '../../ai/model.d';
 import { LLMModelTypeEnum } from '../../ai/constants';
 import { WorkflowIOValueTypeEnum, NodeInputKeyEnum, NodeOutputKeyEnum } from '../constants';
 import { FlowNodeInputTypeEnum, FlowNodeOutputTypeEnum } from '../node/constant';
@@ -77,6 +78,12 @@ export type FlowNodeOutputItemType = {
  defaultValue?: any;
  required?: boolean;

+  invalid?: boolean;
+  invalidCondition?: (e: {
+    inputs: FlowNodeInputItemType[];
+    llmModelList: LLMModelItemType[];
+  }) => boolean;
+
  // component params
  customFieldConfig?: CustomFieldConfigType;
 };
--- a/packages/global/core/workflow/type/node.d.ts
+++ b/packages/global/core/workflow/type/node.d.ts
@@ -43,6 +43,17 @@ export type FlowNodeCommonType = {
  pluginId?: string;
  isFolder?: boolean;
  // pluginType?: AppTypeEnum;
+  pluginData?: PluginDataType;
+};
+
+export type PluginDataType = {
+  version: string;
+  diagram?: string;
+  userGuide?: string;
+  courseUrl?: string;
+  name?: string;
+  avatar?: string;
+  error?: string;
 };

 type HandleType = {
--- a/packages/global/support/user/api.d.ts
+++ b/packages/global/support/user/api.d.ts
@@ -1,5 +1,9 @@
+import { MemberGroupSchemaType, MemberGroupType } from 'support/permission/memberGroup/type';
 import { OAuthEnum } from './constant';
 import { TrackRegisterParams } from './login/api';
+import { TeamMemberStatusEnum } from './team/constant';
+import { OrgType } from './team/org/type';
+import { TeamMemberItemType } from './team/type';

 export type PostLoginProps = {
  username: string;
@@ -21,3 +25,9 @@ export type FastLoginProps = {
  token: string;
  code: string;
 };
+
+export type SearchResult = {
+  members: Omit<TeamMemberItemType, 'teamId' | 'permission'>[];
+  orgs: Omit<OrgType, 'permission' | 'members'>[];
+  groups: MemberGroupSchemaType[];
+};
--- a/packages/global/support/user/team/controller.d.ts
+++ b/packages/global/support/user/team/controller.d.ts
@@ -13,6 +13,7 @@ export type CreateTeamProps = {
  defaultTeam?: boolean;
  memberName?: string;
  memberAvatar?: string;
+  notificationAccount?: string;
 };
 export type UpdateTeamProps = Omit<ThirdPartyAccountType, 'externalWorkflowVariable'> & {
  name?: string;
@@ -39,6 +40,12 @@ export type UpdateInviteProps = {
  tmbId: string;
  status: TeamMemberSchema['status'];
 };
+
+export type UpdateStatusProps = {
+  tmbId: string;
+  status: TeamMemberSchema['status'];
+};
+
 export type InviteMemberResponse = Record<
  'invite' | 'inValid' | 'inTeam',
  { username: string; userId: string }[]
--- a/packages/global/support/user/team/type.d.ts
+++ b/packages/global/support/user/team/type.d.ts
@@ -34,6 +34,7 @@ export type TeamTagSchema = TeamTagItemType & {
  _id: string;
  teamId: string;
  createTime: Date;
+  updateTime?: Date;
 };

 export type TeamMemberSchema = {
@@ -41,6 +42,7 @@ export type TeamMemberSchema = {
  teamId: string;
  userId: string;
  createTime: Date;
+  updateTime?: Date;
  name: string;
  role: `${TeamMemberRoleEnum}`;
  status: `${TeamMemberStatusEnum}`;
@@ -79,6 +81,9 @@ export type TeamMemberItemType = {
  role: `${TeamMemberRoleEnum}`;
  status: `${TeamMemberStatusEnum}`;
  permission: TeamPermission;
+  contact?: string;
+  createTime: Date;
+  updateTime?: Date;
 };

 export type TeamTagItemType = {
--- a/packages/global/support/user/type.d.ts
+++ b/packages/global/support/user/type.d.ts
@@ -17,6 +17,7 @@ export type UserModelSchema = {
  fastgpt_sem?: {
    keyword: string;
  };
+  contact?: string;
 };

 export type UserType = {
@@ -26,9 +27,9 @@ export type UserType = {
  timezone: string;
  promotionRate: UserModelSchema['promotionRate'];
  team: TeamTmbItemType;
-  standardInfo?: standardInfoType;
  notificationAccount?: string;
  permission: TeamPermission;
+  contact?: string;
 };

 export type SourceMemberType = {
--- a/packages/service/common/file/csv.ts
+++ b/packages/service/common/file/csv.ts
@@ -0,0 +1,4 @@
+export const generateCsv = (headers: string[], data: string[][]) => {
+  const csv = [headers.join(','), ...data.map((row) => row.join(','))].join('\n');
+  return csv;
+};
--- a/packages/service/common/file/gridfs/controller.ts
+++ b/packages/service/common/file/gridfs/controller.ts
@@ -18,10 +18,10 @@ export function getGFSCollection(bucket: `${BucketNameEnum}`) {
  MongoDatasetFileSchema;
  MongoChatFileSchema;

-  return connectionMongo.connection.db.collection(`${bucket}.files`);
+  return connectionMongo.connection.db!.collection(`${bucket}.files`);
 }
 export function getGridBucket(bucket: `${BucketNameEnum}`) {
-  return new connectionMongo.mongo.GridFSBucket(connectionMongo.connection.db, {
+  return new connectionMongo.mongo.GridFSBucket(connectionMongo.connection.db!, {
    bucketName: bucket,
    // @ts-ignore
    readPreference: ReadPreference.SECONDARY_PREFERRED // Read from secondary node
--- a/packages/service/common/file/image/controller.ts
+++ b/packages/service/common/file/image/controller.ts
@@ -5,6 +5,7 @@ import { ClientSession, Types } from '../../../common/mongo';
 import { guessBase64ImageType } from '../utils';
 import { readFromSecondary } from '../../mongo/utils';
 import { addHours } from 'date-fns';
+import { imageFileType } from '@fastgpt/global/common/file/constants';

 export const maxImgSize = 1024 * 1024 * 12;
 const base64MimeRegex = /data:image\/([^\)]+);base64/;
@@ -25,12 +26,19 @@ export async function uploadMongoImg({
  const [base64Mime, base64Data] = base64Img.split(',');
  // Check if mime type is valid
  if (!base64MimeRegex.test(base64Mime)) {
-    return Promise.reject('Invalid image mime type');
+    return Promise.reject('Invalid image base64');
  }

  const mime = `image/${base64Mime.match(base64MimeRegex)?.[1] ?? 'image/jpeg'}`;
  const binary = Buffer.from(base64Data, 'base64');
-  const extension = mime.split('/')[1];
+  let extension = mime.split('/')[1];
+  if (extension.startsWith('x-')) {
+    extension = extension.substring(2); // Remove 'x-' prefix
+  }
+
+  if (!extension || !imageFileType.includes(`.${extension}`)) {
+    return Promise.reject(`Invalid image file type: ${mime}`);
+  }

  const { _id } = await MongoImage.create({
    teamId,
@@ -110,7 +118,7 @@ export async function delImgByRelatedId({
 }: {
  teamId: string;
  relateIds: string[];
-  session: ClientSession;
+  session?: ClientSession;
 }) {
  if (relateIds.length === 0) return;

--- a/packages/service/common/file/read/utils.ts
+++ b/packages/service/common/file/read/utils.ts
@@ -111,15 +111,21 @@ export const readRawContentByFileBuffer = async ({
  // markdown data format
  if (imageList) {
    await batchRun(imageList, async (item) => {
-      const src = await uploadMongoImg({
-        base64Img: `data:${item.mime};base64,${item.base64}`,
-        teamId,
-        // expiredTime: addHours(new Date(), 1),
-        metadata: {
-          ...metadata,
-          mime: item.mime
+      const src = await (async () => {
+        try {
+          return await uploadMongoImg({
+            base64Img: `data:${item.mime};base64,${item.base64}`,
+            teamId,
+            // expiredTime: addHours(new Date(), 1),
+            metadata: {
+              ...metadata,
+              mime: item.mime
+            }
+          });
+        } catch (error) {
+          return '';
        }
-      });
+      })();
      rawText = rawText.replace(item.uuid, src);
      if (formatText) {
        formatText = formatText.replace(item.uuid, src);
--- a/packages/service/common/mongo/index.ts
+++ b/packages/service/common/mongo/index.ts
@@ -38,10 +38,12 @@ const addCommonMiddleware = (schema: mongoose.Schema) => {
    schema.post(op, function (this: any, result: any, next) {
      if (this._startTime) {
        const duration = Date.now() - this._startTime;
-
        const warnLogData = {
-          query: this._query,
-          op,
+          collectionName: this.collection?.name,
+          op: this.op,
+          ...(this._query && { query: this._query }),
+          ...(this._update && { update: this._update }),
+          ...(this._delete && { delete: this._delete }),
          duration
        };

--- a/packages/service/common/mongo/init.ts
+++ b/packages/service/common/mongo/init.ts
@@ -16,16 +16,30 @@ export async function connectMongo(): Promise<Mongoose> {

  console.log('mongo start connect');
  try {
-    connectionMongo.set('strictQuery', true);
+    // Remove existing listeners to prevent duplicates
+    connectionMongo.connection.removeAllListeners('error');
+    connectionMongo.connection.removeAllListeners('disconnected');
+    connectionMongo.set('strictQuery', false);

    connectionMongo.connection.on('error', async (error) => {
      console.log('mongo error', error);
-      await connectionMongo.disconnect();
-      await delay(1000);
-      connectMongo();
+      try {
+        if (connectionMongo.connection.readyState !== 0) {
+          await connectionMongo.disconnect();
+          await delay(1000);
+          await connectMongo();
+        }
+      } catch (error) {}
    });
-    connectionMongo.connection.on('disconnected', () => {
+    connectionMongo.connection.on('disconnected', async () => {
      console.log('mongo disconnected');
+      try {
+        if (connectionMongo.connection.readyState !== 0) {
+          await connectionMongo.disconnect();
+          await delay(1000);
+          await connectMongo();
+        }
+      } catch (error) {}
    });

    await connectionMongo.connect(process.env.MONGODB_URI as string, {
--- a/packages/service/common/string/tiktoken/index.ts
+++ b/packages/service/common/string/tiktoken/index.ts
@@ -25,7 +25,7 @@ export const countGptMessagesTokens = async (
      number
    >({
      name: WorkerNameEnum.countGptMessagesTokens,
-      maxReservedThreads: global.systemEnv?.tokenWorkers || 50
+      maxReservedThreads: global.systemEnv?.tokenWorkers || 30
    });

    const total = await workerController.run({ messages, tools, functionCall });
--- a/packages/service/common/system/constants.ts
+++ b/packages/service/common/system/constants.ts
@@ -1 +1,4 @@
 export const FastGPTProUrl = process.env.PRO_URL ? `${process.env.PRO_URL}/api` : '';
+export const isFastGPTMainService = !!process.env.PRO_URL;
+// @ts-ignore
+export const isFastGPTProService = () => !!global.systemConfig;
--- a/packages/service/common/vectorStore/controller.ts
+++ b/packages/service/common/vectorStore/controller.ts
@@ -21,6 +21,7 @@ export const recallFromVectorStore = Vector.embRecall;
 export const getVectorDataByTime = Vector.getVectorDataByTime;
 export const getVectorCountByTeamId = Vector.getVectorCountByTeamId;
 export const getVectorCountByDatasetId = Vector.getVectorCountByDatasetId;
+export const getVectorCountByCollectionId = Vector.getVectorCountByCollectionId;

 export const insertDatasetDataVector = async ({
  model,
--- a/packages/service/common/vectorStore/milvus/class.ts
+++ b/packages/service/common/vectorStore/milvus/class.ts
@@ -321,6 +321,23 @@ export class MilvusCtrl {

    return total;
  };
+  getVectorCountByCollectionId = async (
+    teamId: string,
+    datasetId: string,
+    collectionId: string
+  ) => {
+    const client = await this.getClient();
+
+    const result = await client.query({
+      collection_name: DatasetVectorTableName,
+      output_fields: ['count(*)'],
+      filter: `(teamId == "${String(teamId)}") and (datasetId == "${String(datasetId)}") and (collectionId == "${String(collectionId)}")`
+    });
+
+    const total = result.data?.[0]?.['count(*)'] as number;
+
+    return total;
+  };

  getVectorDataByTime = async (start: Date, end: Date) => {
    const client = await this.getClient();
--- a/packages/service/common/vectorStore/pg/class.ts
+++ b/packages/service/common/vectorStore/pg/class.ts
@@ -240,6 +240,23 @@ export class PgVectorCtrl {
      where: [['team_id', String(teamId)], 'and', ['dataset_id', String(datasetId)]]
    });

+    return total;
+  };
+  getVectorCountByCollectionId = async (
+    teamId: string,
+    datasetId: string,
+    collectionId: string
+  ) => {
+    const total = await PgClient.count(DatasetVectorTableName, {
+      where: [
+        ['team_id', String(teamId)],
+        'and',
+        ['dataset_id', String(datasetId)],
+        'and',
+        ['collection_id', String(collectionId)]
+      ]
+    });
+
    return total;
  };
 }
--- a/packages/service/core/ai/config.ts
+++ b/packages/service/core/ai/config.ts
@@ -1,7 +1,9 @@
 import OpenAI from '@fastgpt/global/core/ai';
 import {
  ChatCompletionCreateParamsNonStreaming,
-  ChatCompletionCreateParamsStreaming
+  ChatCompletionCreateParamsStreaming,
+  StreamChatType,
+  UnStreamChatType
 } from '@fastgpt/global/core/ai/type';
 import { getErrText } from '@fastgpt/global/common/error/utils';
 import { addLog } from '../../common/system/log';
@@ -9,14 +11,17 @@ import { i18nT } from '../../../web/i18n/utils';
 import { OpenaiAccountType } from '@fastgpt/global/support/user/team/type';
 import { getLLMModel } from './model';

-export const openaiBaseUrl = process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1';
+const aiProxyBaseUrl = process.env.AIPROXY_API_ENDPOINT
+  ? `${process.env.AIPROXY_API_ENDPOINT}/v1`
+  : undefined;
+const openaiBaseUrl = aiProxyBaseUrl || process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1';
+const openaiBaseKey = process.env.AIPROXY_API_TOKEN || process.env.CHAT_API_KEY || '';

 export const getAIApi = (props?: { userKey?: OpenaiAccountType; timeout?: number }) => {
  const { userKey, timeout } = props || {};

  const baseUrl = userKey?.baseUrl || global?.systemEnv?.oneapiUrl || openaiBaseUrl;
-  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || process.env.CHAT_API_KEY || '';
-
+  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || openaiBaseKey;
  return new OpenAI({
    baseURL: baseUrl,
    apiKey,
@@ -30,7 +35,7 @@ export const getAxiosConfig = (props?: { userKey?: OpenaiAccountType }) => {
  const { userKey } = props || {};

  const baseUrl = userKey?.baseUrl || global?.systemEnv?.oneapiUrl || openaiBaseUrl;
-  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || process.env.CHAT_API_KEY || '';
+  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || openaiBaseKey;

  return {
    baseUrl,
@@ -38,29 +43,30 @@ export const getAxiosConfig = (props?: { userKey?: OpenaiAccountType }) => {
  };
 };

-type CompletionsBodyType =
-  | ChatCompletionCreateParamsNonStreaming
-  | ChatCompletionCreateParamsStreaming;
-type InferResponseType<T extends CompletionsBodyType> =
-  T extends ChatCompletionCreateParamsStreaming
-    ? OpenAI.Chat.Completions.ChatCompletionChunk
-    : OpenAI.Chat.Completions.ChatCompletion;
-
-export const createChatCompletion = async <T extends CompletionsBodyType>({
+export const createChatCompletion = async ({
  body,
  userKey,
  timeout,
  options
 }: {
-  body: T;
+  body: ChatCompletionCreateParamsNonStreaming | ChatCompletionCreateParamsStreaming;
  userKey?: OpenaiAccountType;
  timeout?: number;
  options?: OpenAI.RequestOptions;
-}): Promise<{
-  response: InferResponseType<T>;
-  isStreamResponse: boolean;
-  getEmptyResponseTip: () => string;
-}> => {
+}): Promise<
+  {
+    getEmptyResponseTip: () => string;
+  } & (
+    | {
+        response: StreamChatType;
+        isStreamResponse: true;
+      }
+    | {
+        response: UnStreamChatType;
+        isStreamResponse: false;
+      }
+  )
+> => {
  try {
    const modelConstantsData = getLLMModel(body.model);

@@ -69,6 +75,7 @@ export const createChatCompletion = async <T extends CompletionsBodyType>({
      userKey,
      timeout: formatTimeout
    });
+
    const response = await ai.chat.completions.create(body, {
      ...options,
      ...(modelConstantsData.requestUrl ? { path: modelConstantsData.requestUrl } : {}),
@@ -96,9 +103,17 @@ export const createChatCompletion = async <T extends CompletionsBodyType>({
      return i18nT('chat:LLM_model_response_empty');
    };

+    if (isStreamResponse) {
+      return {
+        response,
+        isStreamResponse: true,
+        getEmptyResponseTip
+      };
+    }
+
    return {
-      response: response as InferResponseType<T>,
-      isStreamResponse,
+      response,
+      isStreamResponse: false,
      getEmptyResponseTip
    };
  } catch (error) {
--- a/packages/service/core/ai/config/provider/AliCloud.json
+++ b/packages/service/core/ai/config/provider/AliCloud.json
@@ -1,4 +1,10 @@
 {
  "provider": "AliCloud",
-  "list": []
-}
+  "list": [
+    {
+      "model": "SenseVoiceSmall",
+      "name": "SenseVoiceSmall",
+      "type": "stt"
+    }
+  ]
+}
--- a/packages/service/core/ai/config/provider/ChatGLM.json
+++ b/packages/service/core/ai/config/provider/ChatGLM.json
@@ -8,6 +8,12 @@
      "maxResponse": 4000,
      "quoteMaxToken": 120000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "responseFormatList": [
+        "text",
+        "json_object"
+      ],
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -30,6 +36,12 @@
      "maxResponse": 4000,
      "quoteMaxToken": 120000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "responseFormatList": [
+        "text",
+        "json_object"
+      ],
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -52,6 +64,12 @@
      "maxResponse": 4000,
      "quoteMaxToken": 900000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "responseFormatList": [
+        "text",
+        "json_object"
+      ],
+      "showStopSign": true,
      "vision": false,
      "toolChoice": false,
      "functionCall": false,
@@ -74,6 +92,12 @@
      "maxResponse": 4000,
      "quoteMaxToken": 120000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "responseFormatList": [
+        "text",
+        "json_object"
+      ],
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -96,6 +120,8 @@
      "maxResponse": 1000,
      "quoteMaxToken": 6000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": true,
      "toolChoice": false,
      "functionCall": false,
@@ -118,6 +144,8 @@
      "maxResponse": 1000,
      "quoteMaxToken": 6000,
      "maxTemperature": 0.99,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": true,
      "toolChoice": false,
      "functionCall": false,
--- a/packages/service/core/ai/config/provider/Claude.json
+++ b/packages/service/core/ai/config/provider/Claude.json
@@ -1,6 +1,30 @@
 {
  "provider": "Claude",
  "list": [
+    {
+      "model": "claude-3-7-sonnet-20250219",
+      "name": "claude-3-7-sonnet-20250219",
+      "maxContext": 200000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 100000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
    {
      "model": "claude-3-5-haiku-20241022",
      "name": "claude-3-5-haiku-20241022",
@@ -8,7 +32,9 @@
      "maxResponse": 8000,
      "quoteMaxToken": 100000,
      "maxTemperature": 1,
-      "vision": false,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": true,
      "toolChoice": true,
      "functionCall": false,
      "defaultSystemChatPrompt": "",
@@ -30,6 +56,8 @@
      "maxResponse": 8000,
      "quoteMaxToken": 100000,
      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": true,
      "toolChoice": true,
      "functionCall": false,
@@ -52,6 +80,8 @@
      "maxResponse": 8000,
      "quoteMaxToken": 100000,
      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": true,
      "toolChoice": true,
      "functionCall": false,
@@ -74,6 +104,8 @@
      "maxResponse": 4096,
      "quoteMaxToken": 100000,
      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": true,
      "toolChoice": true,
      "functionCall": false,
@@ -90,4 +122,4 @@
      "type": "llm"
    }
  ]
-}
+}
--- a/packages/service/core/ai/config/provider/DeepSeek.json
+++ b/packages/service/core/ai/config/provider/DeepSeek.json
@@ -5,9 +5,12 @@
      "model": "deepseek-chat",
      "name": "Deepseek-chat",
      "maxContext": 64000,
-      "maxResponse": 4096,
+      "maxResponse": 8000,
      "quoteMaxToken": 60000,
-      "maxTemperature": 1.5,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "responseFormatList": ["text", "json_object"],
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -25,7 +28,7 @@
      "model": "deepseek-reasoner",
      "name": "Deepseek-reasoner",
      "maxContext": 64000,
-      "maxResponse": 4096,
+      "maxResponse": 8000,
      "quoteMaxToken": 60000,
      "maxTemperature": null,
      "vision": false,
@@ -42,7 +45,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    }
  ]
 }
--- a/packages/service/core/ai/config/provider/Doubao.json
+++ b/packages/service/core/ai/config/provider/Doubao.json
@@ -1,6 +1,102 @@
 {
  "provider": "Doubao",
  "list": [
+    {
+      "model": "Doubao-1.5-lite-32k",
+      "name": "Doubao-1.5-lite-32k",
+      "maxContext": 32000,
+      "maxResponse": 4000,
+      "quoteMaxToken": 32000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": false,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
+    {
+      "model": "Doubao-1.5-pro-32k",
+      "name": "Doubao-1.5-pro-32k",
+      "maxContext": 32000,
+      "maxResponse": 4000,
+      "quoteMaxToken": 32000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": false,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
+    {
+      "model": "Doubao-1.5-pro-256k",
+      "name": "Doubao-1.5-pro-256k",
+      "maxContext": 256000,
+      "maxResponse": 12000,
+      "quoteMaxToken": 256000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": false,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
+    {
+      "model": "Doubao-1.5-vision-pro-32k",
+      "name": "Doubao-1.5-vision-pro-32k",
+      "maxContext": 32000,
+      "maxResponse": 4000,
+      "quoteMaxToken": 32000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
    {
      "model": "Doubao-lite-4k",
      "name": "Doubao-lite-4k",
@@ -8,6 +104,8 @@
      "maxResponse": 4000,
      "quoteMaxToken": 4000,
      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -30,6 +128,8 @@
      "maxResponse": 4000,
      "quoteMaxToken": 32000,
      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
      "vision": false,
      "toolChoice": true,
      "functionCall": false,
@@ -65,7 +165,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-vision-lite-32k",
@@ -87,7 +189,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-pro-4k",
@@ -109,7 +213,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-pro-32k",
@@ -131,7 +237,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-pro-128k",
@@ -153,7 +261,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-vision-pro-32k",
@@ -175,21 +285,25 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Doubao-embedding-large",
      "name": "Doubao-embedding-large",
      "defaultToken": 512,
      "maxToken": 4096,
-      "type": "embedding"
+      "type": "embedding",
+      "normalization": true
    },
    {
      "model": "Doubao-embedding",
      "name": "Doubao-embedding",
      "defaultToken": 512,
      "maxToken": 4096,
-      "type": "embedding"
+      "type": "embedding",
+      "normalization": true
    }
  ]
 }
--- a/packages/service/core/ai/config/provider/Ernie.json
+++ b/packages/service/core/ai/config/provider/Ernie.json
@@ -21,7 +21,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "ERNIE-4.0-Turbo-8K",
@@ -43,7 +45,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "ERNIE-Lite-8K",
@@ -65,7 +69,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "ERNIE-Speed-128K",
@@ -87,7 +93,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "Embedding-V1",
--- a/packages/service/core/ai/config/provider/Gemini.json
+++ b/packages/service/core/ai/config/provider/Gemini.json
@@ -1,6 +1,54 @@
 {
  "provider": "Gemini",
  "list": [
+    {
+      "model": "gemini-2.0-flash",
+      "name": "gemini-2.0-flash",
+      "maxContext": 1000000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 60000,
+      "maxTemperature": 1,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
+    },
+    {
+      "model": "gemini-2.0-pro-exp",
+      "name": "gemini-2.0-pro-exp",
+      "maxContext": 2000000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 100000,
+      "maxTemperature": 1,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
+    },
    {
      "model": "gemini-1.5-flash",
      "name": "gemini-1.5-flash",
@@ -21,7 +69,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "gemini-1.5-pro",
@@ -43,7 +93,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "gemini-2.0-flash-exp",
@@ -65,7 +117,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "gemini-2.0-flash-thinking-exp-1219",
@@ -87,7 +141,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "gemini-2.0-flash-thinking-exp-01-21",
@@ -109,7 +165,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "gemini-exp-1206",
@@ -131,7 +189,9 @@
      "usedInToolCall": true,
      "defaultConfig": {},
      "fieldMap": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "text-embedding-004",
--- a/packages/service/core/ai/config/provider/Grok.json
+++ b/packages/service/core/ai/config/provider/Grok.json
@@ -1,4 +1,29 @@
 {
  "provider": "Grok",
-  "list": []
-}
+  "list": [
+    {
+      "model": "grok-3",
+      "name": "grok-3",
+      "maxContext": 128000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 128000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": false,
+      "toolChoice": false,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    }
+  ]
+}
--- a/packages/service/core/ai/config/provider/Groq.json
+++ b/packages/service/core/ai/config/provider/Groq.json
@@ -20,7 +20,9 @@
      "customExtractPrompt": "",
      "usedInToolCall": true,
      "defaultConfig": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    },
    {
      "model": "llama-3.3-70b-versatile",
@@ -41,7 +43,9 @@
      "customExtractPrompt": "",
      "usedInToolCall": true,
      "defaultConfig": {},
-      "type": "llm"
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
    }
  ]
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Archer	e860c56b77	perf: delete dataset (#3949 ) * fix: collection list count * fix: collection list count * ai proxy ui * perf: delete dataset * perf: add dataset text index * update doc	2025-03-03 12:49:13 +08:00
Archer	efac5312b4	fix: rerank model cannot use ai proxy (#3945 ) * fix: collection list count * fix: collection list count * fix: rerank model cannot use ai proxy * mongo init	2025-03-03 11:49:35 +08:00
Finley Ge	4bc7f21182	fix: add order:true to all create transactions (#3948 )	2025-03-03 11:37:51 +08:00
gggaaallleee	113e8f711f	add env proxypool (#3939 )	2025-03-02 17:50:03 +08:00
Archer	abc6dffb41	4.8.23 dev (#3932 ) * fix: collection list count * fix: collection list count * update doc * perf: init log * yml	2025-02-28 19:18:12 +08:00
gggaaallleee	f7b2a57ca3	1 (#3924 )	2025-02-28 19:00:58 +08:00
Archer	cf0aaa1091	fix: invalid dataset data clear (#3927 ) * fix: collection list count * fix: collection list count * fix: invalid dataset data clear * update ts * perf: cron clear invalid data * perf: init * perf: clear invalid code * update init * perf: clear invalid code * perf: clear invalid code * perf: init count * batch init * batch init * batch init * batch init * add comment * perf: init * fix: api proxy type	2025-02-28 17:49:20 +08:00
Archer	ac4255ea0c	4.8.23 dev (#3926 ) * fix: collection list count * fix: collection list count * fix: ts	2025-02-28 12:33:09 +08:00
Archer	df4d6f86ce	fix: delete dataset field error (#3925 ) * fix: collection list count * fix: collection list count * update doc * perf: tts selector ui * fix: delete dataset field error * doc	2025-02-28 12:29:18 +08:00
heheer	e697fda82f	fix: export chat log - chat detail order (#3923 )	2025-02-28 11:33:46 +08:00
Archer	1aa319e7aa	Update package.json (#3919 )	2025-02-27 22:25:26 +08:00
Archer	fc9e614f88	4.8.23 dev (#3917 ) * fix: icon refresh * fix: aiproxy http request * fix: collection list count * fix: collection list count * fix: tts selector name * update action	2025-02-27 22:15:48 +08:00
Archer	1121ea33bd	更新 docker.md (#3913 )	2025-02-27 17:02:06 +08:00
Finley Ge	9bbee60cde	fix: ts error (#3911 )	2025-02-27 16:31:14 +08:00
Finley Ge	9f57ad0017	fix: mongoose strictquery to false (#3906 )	2025-02-27 11:25:29 +08:00
Archer	c3d3b30d7e	update code positon (#3907 )	2025-02-27 10:30:43 +08:00
Archer	fb0eb49196	fix: pptx encoding (#3905 )	2025-02-27 10:04:59 +08:00
ZongLiang	27ebd2e8cf	Update parseOffice.ts (#3901 ) 更新本地导入pptx文件提示文件编码错误 The argument 'windows-1252' is invalid encoding. Received 'encoding'	2025-02-27 09:57:34 +08:00
Archer	81a06718d8	feat: ai proxy v1 (#3898 ) * feat: ai proxy v1 * perf: ai proxy channel crud * feat: ai proxy logs * feat: channel test * doc * update lock	2025-02-27 09:56:52 +08:00
Archer	3c382d1240	Update intro.md (#3900 )	2025-02-26 22:37:56 +08:00
Finley Ge	747bb303ec	chore: upgrade mongoose to v8.10.x for security (#3868 ) * chore: upgrade mongoose to v8.10.x for security * chore: remove duplicate code * fix: ts error	2025-02-26 18:32:19 +08:00
Finley Ge	cf9c8e9f6a	fix: leave team and refresh memberlist (#3893 )	2025-02-26 18:29:05 +08:00
Archer	5d5bee9e41	remove markdown format;refresh username;perf: latext render (#3877 ) * refresh username * remove md format * perf: latext render * ignore big image * model config	2025-02-25 16:16:30 +08:00
Archer	4f0dd96699	perf: work order tip (#3874 )	2025-02-24 20:26:15 +08:00
Finley Ge	fb6dbaf2d6	feat/workorder (#3860 ) * feat: workorder * pref: workorder button * chore: move workorder to common * chore: format code * pref: style	2025-02-24 19:59:06 +08:00
风沐白	ffc1520f4c	Update quick-start.md (#3873 )	2025-02-24 19:56:56 +08:00
Archer	255764400f	feat: model config required check;feat: dataset text model default setting (#3866 ) * feat: model config required check * feat: dataset text model default setting * perf: collection list count * fix: ts * remove index count	2025-02-24 19:55:49 +08:00
heheer	3bfe802c48	fix collection folder tags filter (#3853 ) * fix collection folder tags filter * add comment * fix	2025-02-24 17:43:31 +08:00
YeYuheng	2bf17dbb87	Marker doc update (#3869 ) * update-marker-doc * marker.md * marker.md	2025-02-24 13:13:05 +08:00
Archer	8d766372fe	update doc (#3840 ) * update doc * update doc	2025-02-20 10:40:00 +08:00
Archer	ca5717936b	update doc (#3836 )	2025-02-19 22:32:08 +08:00
Archer	6762723b10	perf: ery extension and fix filter same embedding result (#3833 ) * perf: ery extension and fix filter same embedding result * fix: extract node too long * perf: ui * perf: not chatId will auto save * fix: laf * fix: member load * feat: add completions unstream error response * feat: add completions unstream error response * updat emodel provider	2025-02-19 22:16:43 +08:00
heheer	8604cbd021	fix source name (#3834 )	2025-02-19 20:42:30 +08:00
Finley Ge	206325bc5f	chore: team, orgs, search and so on (#3807 ) * feat: clb search support username, memberName, contacts * feat: popup org names * feat: update team member table * feat: restore the member * feat: search user in team member table * feat: bind contact * feat: export members * feat: org tab could delete member * feat: org table search * feat: team notification account bind * feat: permission tab search * fix: wecom sso * chore(init): copy notificationAccount to user.contact * chore: adjust * fix: ts error * fix: useConfirm iconColor customization * pref: fe * fix: style * fix: fix team member manage * pref: enlarge team member pagesize * pref: initv4822 * fix: pageSize * pref: initscritpt	2025-02-19 17:27:19 +08:00
Archer	5fd520c794	perf: gemini config (#3828 ) * doc * doc * perf: gemini config	2025-02-19 12:00:31 +08:00
Archer	09205e4666	fix: price page init data;perf: usage code;fix: reasoning tokens;fix: workflow basic node cannot upgrade (#3816 ) * fix: img read * fix: price page init data * perf: ai model avatar * perf: refresh in change team * perf: null checker * perf: usage code * fix: reasoning tokens * fix: workflow basic node cannot upgrade * perf: model refresh * perf: icon refresh	2025-02-18 20:50:25 +08:00
Finley Ge	ccf28d83b8	fix: app version addSourcemember tmbid could be empty (#3822 )	2025-02-18 20:26:49 +08:00
LGiki	420aaad48e	chore: fix typo in docs (#3819 )	2025-02-18 20:25:51 +08:00
heheer	8ba2339890	download fetch baseurl & node select dnd (#3820 )	2025-02-18 20:25:15 +08:00
Archer	e7b8934367	Update 4818.md (#3818 )	2025-02-18 14:26:21 +08:00
Finley Ge	3e13397614	fix: refresh memberlist when switching account (#3814 )	2025-02-18 13:54:56 +08:00
Archer	b14674cc6f	fix: whisper checker;fix: img read (#3813 ) * fix: img read * fix: whisper checker * perf: dev doc * perf: dev doc * remove invalid code	2025-02-18 10:08:25 +08:00
Archer	4d20274a97	feat: think tag parse (#3805 ) (#3808 ) * feat: think tag parse * remove some model config * feat: parse think tag test	2025-02-17 20:57:36 +08:00
heheer	4447e40364	fix template market simple app (#3804 )	2025-02-17 20:56:46 +08:00
John Chen	23949230ee	fix document (#3806 ) V2版本“获取集合列表”接口的path区分了大小写，使用/api/core/dataset/collection/listv2会返回404，必须使用大写V	2025-02-17 20:55:34 +08:00
saikidev	cd7a897304	chore: add ppio provider (#3789 )	2025-02-14 17:04:43 +08:00
Archer	18aff8b8db	update yml version (#3787 )	2025-02-14 12:50:54 +08:00
Archer	d2b60ec785	fix: model check circle tip (#3786 ) * model config * feat: normalization embedding * remove log * version doc * version doc * fix: model check circle tip * uml	2025-02-14 11:42:14 +08:00
a.e.	1226fe42a1	fix: skip thirdparty sso state verification (#3721 ) (#3782 )	2025-02-14 11:39:34 +08:00
Finley Ge	abd375cdec	fix: app/dataset list api return private flag (#3784 )	2025-02-14 11:38:48 +08:00
Archer	7aacce8b0b	4.9.0 test (#3779 ) * model config * feat: normalization embedding * remove log * version doc * version doc	2025-02-13 16:27:41 +08:00
heheer	686b09afd1	chatbot not overflow (#3777 ) * chatbot not overflow * add comment	2025-02-13 15:10:22 +08:00
heheer	3cfec37e9d	fix embed chatbot default open (#3774 )	2025-02-13 13:36:56 +08:00
Archer	d3641c877c	perf: unlogin user fetch data (#3775 ) * model config * feat: normalization embedding * perf: unlogin user fetch data	2025-02-13 13:36:33 +08:00
Archer	1094c65f2b	perf: http empty params (#3773 ) * model config * feat: normalization embedding * perf: http empty params * doc	2025-02-13 10:35:11 +08:00
Archer	abe082b9ab	i18n perf (#3770 ) * model config * feat: normalization embedding * perf: mark ui * perf: i18n * fix: rerank error tip	2025-02-12 16:36:21 +08:00
heheer	132cf69372	optimize dnd drag code (#3768 )	2025-02-12 15:25:31 +08:00
heheer	06a8a5e23d	fix: simple mode variables dnd (#3767 ) * fix: simple mode variables dnd * optimize dnd drag	2025-02-12 14:36:04 +08:00
heheer	c42deab63b	global variable & interactive node dnd (#3764 )	2025-02-12 12:27:36 +08:00
Archer	58f715e878	perf: request quantity;perf: share page error circulation;perf: share chat toast (#3763 ) * model config * feat: normalization embedding * perf: share page error circulation * perf: request quantity * perf: share chat toast * perf: queue	2025-02-12 11:36:29 +08:00
Archer	116936ffa9	更新 share.md (#3757 )	2025-02-11 23:54:42 +08:00
heheer	f5d045eece	export csv format & log title debounce (#3754 )	2025-02-11 17:36:00 +08:00
sbcyk	8ac6494e60	Update chat.md (#3746 ) 示例代码的json内容少了一个引号	2025-02-11 17:31:30 +08:00
heheer	f002896a24	chat logs filter & export (#3737 ) * chat logs filter & export * export chat detail	2025-02-11 16:32:47 +08:00
Archer	8738c32fb0	4.8.21 feature (#3742 ) * model config * feat: normalization embedding * adapt unstrea reasoning response * remove select app * perf: dataset search code * fix: multiple audio video show * perf: query extension output * perf: link check * perf: faq doc * fix: ts * feat: support reasoning text output * feat: workflow support reasoning output	2025-02-11 13:53:08 +08:00
heheer	896a3f1472	add plugin unexist error tips (#3717 ) * add plugin unexist error tips * throw error when run plugin * check workflow * plugin data avoid request twice * auth owner tmbId * fix	2025-02-10 15:20:49 +08:00
John Chen	4284b78707	Update configuration.md (#3725 ) 由于4.8.20版本放弃在config.json中配置模型，在说明文档中，修正二级标题的版本号，并添加注释	2025-02-10 09:13:17 +08:00
Archer	fac5b6b50d	更新 4820.md (#3730 )	2025-02-09 10:06:08 +08:00
Archer	51e17a47fa	feat: normalization embedding;feat: model top_p param config (#3723 ) * edit form force close image select * model config * feat: normalization embedding * perf: add share page title force refresh	2025-02-08 12:16:46 +08:00
Archer	42b2046f96	4.8.21 feature (#3720 ) * agent search demo * edit form force close image select * feat: llm params and doubao1.5 * perf: model error tip * fix: template register path * package	2025-02-08 10:44:33 +08:00
heheer	bb82b515e0	feat: auto adapt outlink chatwindow position (#3707 )	2025-02-08 09:49:41 +08:00
clidxhk	fe688cdf2d	Update utils.ts (#3699 ) 本地windows平台开发，加载model列表出现两次盘符导致加载失败，修改代码确保生成的路径不会包含重复的盘符，从而避免 ENOENT 错误。	2025-02-07 09:52:08 +08:00
Archer	0d35326909	fix: yml (#3709 )	2025-02-06 16:03:45 +08:00