perf: delete dataset (#3949 )

* fix: collection list count * fix: collection list count * ai proxy ui * perf: delete dataset * perf: add dataset text index * update doc
fix: rerank model cannot use ai proxy (#3945 )
2025-03-03 12:49:13 +08:00 · 2025-03-03 11:49:35 +08:00 · 2025-03-03 11:37:51 +08:00 · 2025-03-02 17:50:03 +08:00 · 2025-02-28 19:18:12 +08:00 · 2025-02-28 19:00:58 +08:00
374 changed files with 13602 additions and 2529 deletions
--- a/.github/workflows/docs-sync_imgs.yml
+++ b/.github/workflows/docs-sync_imgs.yml
--- a/.github/workflows/fastgpt-build-image-personal.yml
+++ b/.github/workflows/fastgpt-build-image-personal.yml
--- a/.github/workflows/fastgpt-build-image.yml
+++ b/.github/workflows/fastgpt-build-image.yml
--- a/.github/workflows/fastgpt-preview-image.yml
+++ b/.github/workflows/fastgpt-preview-image.yml
@@ -68,14 +68,3 @@ jobs:
          SEALOS_TYPE: 'pr_comment'
          SEALOS_FILENAME: 'report.md'
          SEALOS_REPLACE_TAG: 'DEFAULT_REPLACE_DEPLOY'
-
-  helm-check:
-    runs-on: ubuntu-20.04
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-      - name: Helm Check
-        run: |
-          helm dependency update files/helm/fastgpt
-          helm lint files/helm/fastgpt
-          helm package files/helm/fastgpt
--- a/.github/workflows/helm-release.yaml
+++ b/.github/workflows/helm-release.yaml
@@ -24,6 +24,6 @@ jobs:
          export APP_VERSION=${{ steps.vars.outputs.tag }}
          export HELM_VERSION=${{ steps.vars.outputs.tag }}
          export HELM_REPO=ghcr.io/${{ github.repository_owner }}
-          helm dependency update files/helm/fastgpt
-          helm package files/helm/fastgpt --version ${HELM_VERSION}-helm --app-version ${APP_VERSION} -d bin
+          helm dependency update deploy/helm/fastgpt
+          helm package deploy/helm/fastgpt --version ${HELM_VERSION}-helm --app-version ${APP_VERSION} -d bin
          helm push bin/fastgpt-${HELM_VERSION}-helm.tgz oci://${HELM_REPO}
--- a/.github/workflows/sandbox-build-image.yml
+++ b/.github/workflows/sandbox-build-image.yml
--- a/.vscode/nextapi.code-snippets
+++ b/.vscode/nextapi.code-snippets
@@ -58,7 +58,7 @@
 		"body": [
 			"import '@/pages/api/__mocks__/base';",
 			"import { root } from '@/pages/api/__mocks__/db/init';",
-			"import { getTestRequest } from '@/test/utils';",
+			"import { getTestRequest } from '@fastgpt/service/test/utils'; ;",
 			"import { AppErrEnum } from '@fastgpt/global/common/error/code/app';",
 			"import handler from './demo';",
 			"",
--- a/deploy/docker/docker-compose-milvus.yml
+++ b/deploy/docker/docker-compose-milvus.yml
@@ -114,15 +114,15 @@ services:
  # fastgpt
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -133,7 +133,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose-pgvector.yml
+++ b/deploy/docker/docker-compose-pgvector.yml
@@ -72,15 +72,15 @@ services:
  # fastgpt
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -91,7 +91,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose-zilliz.yml
+++ b/deploy/docker/docker-compose-zilliz.yml
@@ -53,15 +53,15 @@ services:
        wait $$!
  sandbox:
    container_name: sandbox
-    image: ghcr.io/labring/fastgpt-sandbox:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt-sandbox:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-sandbox:v4.8.23-fix # 阿里云
    networks:
      - fastgpt
    restart: always
  fastgpt:
    container_name: fastgpt
-    image: ghcr.io/labring/fastgpt:v4.8.20-fix2 # git
-    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.20-fix2 # 阿里云
+    image: ghcr.io/labring/fastgpt:v4.8.23-fix # git
+    # image: registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.8.23-fix # 阿里云
    ports:
      - 3000:3000
    networks:
@@ -71,7 +71,7 @@ services:
      - sandbox
    restart: always
    environment:
-      # 前端访问地址: http://localhost:3000
+      # 前端外部可访问的地址，用于自动补全文件资源路径。例如 https:fastgpt.cn，不能填 localhost。这个值可以不填，不填则发给模型的图片会是一个相对路径，而不是全路径，模型可能伪造Host。
      - FE_DOMAIN=
      # root 密码，用户名为: root。如果需要修改 root 密码，直接修改这个环境变量，并重启即可。
      - DEFAULT_ROOT_PSW=1234
--- a/deploy/docker/docker-compose/docker-compose
+++ b/deploy/docker/docker-compose/docker-compose
--- a/deploy/docker/docker-compose/init.sh
+++ b/deploy/docker/docker-compose/init.sh
--- a/deploy/docker/run.sh
+++ b/deploy/docker/run.sh
--- a/deploy/helm/fastgpt/.helmignore
+++ b/deploy/helm/fastgpt/.helmignore
--- a/deploy/helm/fastgpt/Chart.lock
+++ b/deploy/helm/fastgpt/Chart.lock
--- a/deploy/helm/fastgpt/Chart.yaml
+++ b/deploy/helm/fastgpt/Chart.yaml
--- a/deploy/helm/fastgpt/README.md
+++ b/deploy/helm/fastgpt/README.md
--- a/deploy/helm/fastgpt/templates/NOTES.txt
+++ b/deploy/helm/fastgpt/templates/NOTES.txt
--- a/deploy/helm/fastgpt/templates/_helpers.tpl
+++ b/deploy/helm/fastgpt/templates/_helpers.tpl
--- a/deploy/helm/fastgpt/templates/configmap-config.yaml
+++ b/deploy/helm/fastgpt/templates/configmap-config.yaml
--- a/deploy/helm/fastgpt/templates/deployment.yaml
+++ b/deploy/helm/fastgpt/templates/deployment.yaml
--- a/deploy/helm/fastgpt/templates/hpa.yaml
+++ b/deploy/helm/fastgpt/templates/hpa.yaml
--- a/deploy/helm/fastgpt/templates/ingress.yaml
+++ b/deploy/helm/fastgpt/templates/ingress.yaml
--- a/deploy/helm/fastgpt/templates/secret-env.yaml
+++ b/deploy/helm/fastgpt/templates/secret-env.yaml
--- a/deploy/helm/fastgpt/templates/service.yaml
+++ b/deploy/helm/fastgpt/templates/service.yaml
--- a/deploy/helm/fastgpt/templates/serviceaccount.yaml
+++ b/deploy/helm/fastgpt/templates/serviceaccount.yaml
--- a/deploy/helm/fastgpt/templates/tests/test-connection.yaml
+++ b/deploy/helm/fastgpt/templates/tests/test-connection.yaml
--- a/deploy/helm/fastgpt/values.yaml
+++ b/deploy/helm/fastgpt/values.yaml
--- a/docSite/assets/imgs/appid.png
+++ b/docSite/assets/imgs/appid.png
--- a/docSite/content/zh-cn/docs/development/custom-models/bge-rerank.md
+++ b/docSite/content/zh-cn/docs/development/custom-models/bge-rerank.md
@@ -31,9 +31,9 @@ weight: 920

 3 个模型代码分别为：

-1. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-base)
-2. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-large)
-3. [https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-v2-m3](https://github.com/labring/FastGPT/tree/main/python/bge-rerank/bge-reranker-v2-m3)
+1. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-base)
+2. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-large)
+3. [https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-v2-m3](https://github.com/labring/FastGPT/tree/main/plugins/rerank-bge/bge-reranker-v2-m3)

 ### 3. 安装依赖

--- a/docSite/content/zh-cn/docs/development/custom-models/marker.md
+++ b/docSite/content/zh-cn/docs/development/custom-models/marker.md
@@ -19,7 +19,7 @@ PDF 是一个相对复杂的文件格式，在 FastGPT 内置的 pdf 解析器

 ### 1. 按照 Marker

-参考文档 [Marker 安装教程](https://github.com/labring/FastGPT/tree/main/python/pdf-marker)，安装 Marker 模型。封装的 API 已经适配了 FastGPT 自定义解析服务。
+参考文档 [Marker 安装教程](https://github.com/labring/FastGPT/tree/main/plugins/model/pdf-marker)，安装 Marker 模型。封装的 API 已经适配了 FastGPT 自定义解析服务。

 这里介绍快速 Docker 安装的方法：

--- a/docSite/content/zh-cn/docs/development/docker.md
+++ b/docSite/content/zh-cn/docs/development/docker.md
@@ -118,7 +118,7 @@ brew install orbstack
 非 Linux 环境或无法访问外网环境，可手动创建一个目录，并下载配置文件和对应版本的`docker-compose.yml`，在这个文件夹中依据下载的配置文件运行docker，若作为本地开发使用推荐`docker-compose-pgvector`版本，并且自行拉取并运行`sandbox`和`fastgpt`，并在docker配置文件中注释掉`sandbox`和`fastgpt`的部分

 - [config.json](https://raw.githubusercontent.com/labring/FastGPT/refs/heads/main/projects/app/data/config.json)
- [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/files/docker) (注意，不同向量库版本的文件不一样)
+- [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker) (注意，不同向量库版本的文件不一样)

 {{% alert icon="🤖" context="success" %}}

@@ -134,11 +134,11 @@ cd fastgpt
 curl -O https://raw.githubusercontent.com/labring/FastGPT/main/projects/app/data/config.json

 # pgvector 版本(测试推荐，简单快捷)
-curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-pgvector.yml
+curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-pgvector.yml
 # milvus 版本
-# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-milvus.yml
+# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-milvus.yml
 # zilliz 版本
-# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-zilliz.yml
+# curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/deploy/docker/docker-compose-zilliz.yml
 ```

 ### 2. 修改环境变量
@@ -201,6 +201,8 @@ docker restart oneapi

 在OneApi中添加合适的AI模型渠道。[点击查看相关教程](/docs/development/modelconfig/one-api/)

+只需要添加模型即可，模板已经配置好了oneapi的连接地址和令牌，无需变更。
+
 ### 5. 访问 FastGPT

 目前可以通过 `ip:3000` 直接访问(注意防火墙)。登录用户名为 `root`，密码为`docker-compose.yml`环境变量里设置的 `DEFAULT_ROOT_PSW`。
@@ -211,7 +213,7 @@ docker restart oneapi

 ### 6. 配置模型

-务必先配置至少一组模型，否则系统无法正常使用。
+登录FastGPT后，进入模型配置页面，务必先配置至少一个语言模型和一个向量模型，否则系统无法正常使用。

 [点击查看模型配置教程](/docs/development/modelConfig/intro/)

--- a/docSite/content/zh-cn/docs/development/faq.md
+++ b/docSite/content/zh-cn/docs/development/faq.md
@@ -142,6 +142,10 @@ OneAPI 的 API Key 配置错误，需要修改`OPENAI_API_KEY`环境变量，并
 3. ....


+### Tiktoken 下载失败
+
+由于 OneAPI 会在启动时从网络下载一个 tiktoken 的依赖，如果网络异常，就会导致启动失败。可以参考[OneAPI 离线部署](https://blog.csdn.net/wanh/article/details/139039216)解决。
+
 ## 四、常见模型问题

 ### 如何检查模型可用性问题
--- a/docSite/content/zh-cn/docs/development/intro.md
+++ b/docSite/content/zh-cn/docs/development/intro.md
@@ -15,8 +15,8 @@ weight: 705

 - [Git](http://git-scm.com/)
 - [Docker](https://www.docker.com/)（构建镜像）
- [Node.js v18.17 / v20.x](http://nodejs.org)（版本尽量一样，可以使用nvm管理node版本）
- [pnpm](https://pnpm.io/) 版本 8.6.0 (目前官方的开发环境)
+- [Node.js v20.14.0](http://nodejs.org)（版本尽量一样，可以使用nvm管理node版本）
+- [pnpm](https://pnpm.io/) 推荐版本 9.4.0 (目前官方的开发环境)
 - make命令: 根据不同平台，百度安装 (官方是GNU Make 4.3)

 ## 开始本地开发
@@ -77,8 +77,6 @@ Mongo 数据库需要注意，需要注意在连接地址中增加 `directConnec
 可参考项目根目录下的 `dev.md`，第一次编译运行可能会有点慢，需要点耐心哦

 ```bash
-# 给自动化脚本代码执行权限(非 linux 系统, 可以手动执行里面的 postinstall.sh 文件内容)
-chmod -R +x ./scripts/
 # 代码根目录下执行，会安装根 package、projects 和 packages 内所有依赖
 # 如果提示 isolate-vm 安装失败，可以参考：https://github.com/laverdet/isolated-vm?tab=readme-ov-file#requirements
 pnpm i
--- a/docSite/content/zh-cn/docs/development/modelConfig/intro.md
+++ b/docSite/content/zh-cn/docs/development/modelConfig/intro.md
@@ -23,7 +23,7 @@ weight: 744

 {{% alert icon=" " context="info" %}}
 - [SiliconCloud(硅基流动)](https://cloud.siliconflow.cn/i/TR9Ym0c4): 提供开源模型调用的平台。
- [Sealos AIProxy](https://hzh.sealos.run/?openapp=system-aiproxy): 提供国内各家模型代理，无需逐一申请 api。
+- [Sealos AIProxy](https://cloud.sealos.run/?uid=fnWRt09fZP&openapp=system-aiproxy): 提供国内各家模型代理，无需逐一申请 api。
 {{% /alert %}}

 在 OneAPI 配置好模型后，你就可以打开 FastGPT 页面，启用对应模型了。
@@ -43,8 +43,7 @@ weight: 744
 {{% alert icon="🤖 " context="success" %}}
 注意：
 1. 目前语音识别模型和重排模型仅会生效一个，所以配置时候，只需要配置一个即可。  
-2. 系统必须至少有一个语言模型和一个索引模型才能正常使用。
-3. 使用知识库功能，至少要有一个语言模型，用于知识库文件处理（可以在模型配置时候打开该开关），否则知识库会报错。
+2. 系统至少需要一个语言模型和一个索引模型才能正常使用。
 {{% /alert %}}

 #### 核心配置
@@ -468,4 +467,4 @@ OneAPI 的语言识别接口，无法正确的识别其他模型（会始终识
    "charsPointsPrice": 0
  }
 }
-```
+```
--- a/docSite/content/zh-cn/docs/development/openapi/chat.md
+++ b/docSite/content/zh-cn/docs/development/openapi/chat.md
@@ -7,6 +7,12 @@ toc: true
 weight: 852
 ---

+# 如何获取 AppId
+
+可在应用详情的路径里获取 AppId。
+
+![](/imgs/appid.png)
+
 # 发起对话

 {{% alert icon="🤖 " context="success" %}}
@@ -102,8 +108,8 @@ curl --location --request POST 'http://localhost:3000/api/v1/chat/completions' \
 {{% alert context="info" %}}
 - headers.Authorization: Bearer {{apikey}}
 - chatId: string | undefined 。
-  - 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。 不会将你的记录存储到数据库中，你也无法在记录汇总中查阅到。
-  - 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。
+  - 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。
+  - 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题，其余 message 会被忽略。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。
 - messages: 结构与 [GPT接口](https://platform.openai.com/docs/api-reference/chat/object) chat模式一致。
 - responseChatItemId: string | undefined 。如果传入，则会将该值作为本次对话的响应消息的 ID，FastGPT 会自动将该 ID 存入数据库。请确保，在当前`chatId`下，`responseChatItemId`是唯一的。
 - detail: 是否返回中间值（模块状态，响应的完整结果等），`stream模式`下会通过`event`进行区分，`非stream模式`结果保存在`responseData`中。
--- a/docSite/content/zh-cn/docs/development/openapi/dataset.md
+++ b/docSite/content/zh-cn/docs/development/openapi/dataset.md
@@ -735,7 +735,7 @@ data 为集合的 ID。

 **4.8.19+**
 ```bash
-curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listv2' \
+curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/listV2' \
 --header 'Authorization: Bearer {{authorization}}' \
 --header 'Content-Type: application/json' \
 --data-raw '{
--- a/docSite/content/zh-cn/docs/development/openapi/intro.md
+++ b/docSite/content/zh-cn/docs/development/openapi/intro.md
@@ -1,6 +1,6 @@
 ---
-title: 'Api Key 使用与鉴权'
-description: 'FastGPT Api Key 使用与鉴权'
+title: 'OpenAPI 介绍'
+description: 'FastGPT OpenAPI 介绍'
 icon: 'key'
 draft: false
 toc: true
@@ -27,6 +27,7 @@ FastGPT 的 API Key **有 2 类**，一类是全局通用的 key (无法直接
 | --------------------- | --------------------- |
 | ![](/imgs/fastgpt-api2.jpg) | ![](/imgs/fastgpt-api1.jpg) |

+
 ## 基本配置

 OpenAPI 中，所有的接口都通过 Header.Authorization 进行鉴权。
--- a/docSite/content/zh-cn/docs/development/upgrading/4818.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4818.md
@@ -1,5 +1,5 @@
 ---
-title: 'V4.8.18'
+title: 'V4.8.18(包含升级脚本)'
 description: 'FastGPT V4.8.18 更新说明'
 icon: 'upgrade'
 draft: false
--- a/docSite/content/zh-cn/docs/development/upgrading/482.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/482.md
@@ -20,7 +20,7 @@ SANDBOX_URL=内网地址

 ## Docker 部署

-可以拉取最新 [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/files/docker/docker-compose.yml) 文件参考
+可以拉取最新 [docker-compose.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker/docker-compose.yml) 文件参考

 1. 新增一个容器 `sandbox`
 2. fastgpt 和 fastgpt-pro(商业版) 容器新增环境变量: `SANDBOX_URL`
--- a/docSite/content/zh-cn/docs/development/upgrading/4821.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4821.md
@@ -1,13 +1,21 @@
 ---
-title: 'V4.8.21(进行中)'
+title: 'V4.8.21'
 description: 'FastGPT V4.8.21 更新说明'
 icon: 'upgrade'
 draft: false
 toc: true
-weight: 804
+weight: 803
 ---

+## 更新指南

+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.21-fix
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.21-fix
+- Sandbox 镜像无需更新

 ## 完整更新内容

--- a/docSite/content/zh-cn/docs/development/upgrading/4822.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4822.md
@@ -0,0 +1,61 @@
+---
+title: 'V4.8.22(包含升级脚本)'
+description: 'FastGPT V4.8.22 更新说明'
+icon: 'upgrade'
+draft: false
+toc: true
+weight: 802
+---
+
+## 🌟更新指南
+
+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.22
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.22
+- Sandbox 镜像无需更新
+
+### 3. 运行升级脚本
+
+仅商业版，并提供 Saas 服务的用户需要运行该升级脚本。
+
+从任意终端，发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`；{{host}} 替换成**FastGPT 域名**。
+
+```bash
+curl --location --request POST 'https://{{host}}/api/admin/initv4822' \
+--header 'rootkey: {{rootkey}}' \
+--header 'Content-Type: application/json'
+```
+
+会迁移联系方式到对应用户表中。
+
+## 🚀 新增内容
+
+1. AI 对话节点解析 `<think></think>` 标签内容作为思考链，便于各类模型进行思考链输出。需主动开启模型输出思考。
+2. 对话 API 优化，无论是否传递 chatId，都会保存对话日志。未传递 chatId，则随机生成一个 chatId 来进行存储。
+3. ppio 模型提供商
+
+## ⚙️ 优化
+
+1. 模型未配置时提示，减少冲突提示。
+2. 使用记录代码。
+3. 内容提取节点，字段描述过长时换行。同时修改其输出名用 key，而不是 description。
+4. 团队管理交互。
+5. 对话接口，非流响应，增加报错字段。
+
+## 🐛 修复
+
+1. 思考内容未进入到输出 Tokens.
+2. 思考链流输出时，有时与正文顺序偏差。
+3. API 调用工作流，如果传递的图片不支持 Head 检测时，图片会被过滤。已增加该类错误检测，避免被错误过滤。
+4. 模板市场部分模板错误。
+5. 免登录窗口无法正常判断语言识别是否开启。
+6. 对话日志导出，未兼容 sub path。
+7. 切换团队时未刷新成员列表
+8. list 接口在联查 member 时，存在空指针可能性。 
+9. 工作流基础节点无法升级。
+10. 向量检索结果未去重。
+11. 用户选择节点无法正常连线。
+12. 对话记录保存时，source 未正常记录。
--- a/docSite/content/zh-cn/docs/development/upgrading/4823.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/4823.md
@@ -0,0 +1,54 @@
+---
+title: 'V4.8.23'
+description: 'FastGPT V4.8.23 更新说明'
+icon: 'upgrade'
+draft: false
+toc: true
+weight: 802
+---
+
+## 更新指南
+
+### 1. 做好数据库备份
+
+### 2. 更新镜像：
+
+- 更新 fastgpt 镜像 tag: v4.8.23-fix
+- 更新 fastgpt-pro 商业版镜像 tag: v4.8.23-fix
+- Sandbox 镜像无需更新
+
+### 3. 运行升级脚本
+
+从任意终端，发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`；{{host}} 替换成**FastGPT 域名**。
+
+```bash
+curl --location --request POST 'https://{{host}}/api/admin/initv4823' \
+--header 'rootkey: {{rootkey}}' \
+--header 'Content-Type: application/json'
+```
+
+脚本会清理一些知识库脏数据，主要是多余的全文索引。
+
+## 🚀 新增内容
+
+1. 增加默认“知识库文本理解模型”配置
+2. AI proxy V1版，可替换 OneAPI使用，同时提供完整模型调用日志，便于排查问题。
+3. 增加工单入口支持。
+
+## ⚙️ 优化
+
+1. 模型配置表单，增加必填项校验。
+2. 集合列表数据统计方式，提高大数据量统计性能。
+3. 优化数学公式，转义 Latex 格式成 Markdown 格式。
+4. 解析文档图片，图片太大时，自动忽略。
+5. 时间选择器，当天开始时间自动设0，结束设置设 23:59:59，避免 UI 与实际逻辑偏差。
+6. 升级 mongoose 库版本依赖。
+
+## 🐛 修复
+
+1. 标签过滤时，子文件夹未成功过滤。
+2. 暂时移除 md 阅读优化，避免链接分割错误。
+3. 离开团队时，未刷新成员列表。
+4. PPTX 编码错误，导致解析失败。
+5. 删除知识库单条数据时，全文索引未跟随删除。
+6. 修复 Mongo Dataset text 索引在查询数据时未生效。
--- a/docSite/content/zh-cn/docs/development/upgrading/483.md
+++ b/docSite/content/zh-cn/docs/development/upgrading/483.md
@@ -15,7 +15,7 @@ weight: 821

 ## V4.8.3 更新说明

-1. 新增 - 支持 Milvus 数据库， 可参考最新的 [docker-compose-milvus.yml](https://github.com/labring/FastGPT/blob/main/files/docker/docker-compose-milvus.yml).
+1. 新增 - 支持 Milvus 数据库， 可参考最新的 [docker-compose-milvus.yml](https://github.com/labring/FastGPT/blob/main/deploy/docker/docker-compose-milvus.yml).
 2. 新增 - 给 chat 接口 empty answer 增加 log，便于排查模型问题。
 3. 新增 - ifelse判断器，字符串支持正则。
 4. 新增 - 代码运行支持 console.log 输出调试。
--- a/docSite/content/zh-cn/docs/guide/course/quick-start.md
+++ b/docSite/content/zh-cn/docs/guide/course/quick-start.md
@@ -7,11 +7,11 @@ toc: true
 weight: 102
 ---

-更多使用技巧，[查看视屏教程](https://www.bilibili.com/video/BV1sH4y1T7s9)
+更多使用技巧，[查看视频教程](https://www.bilibili.com/video/BV1sH4y1T7s9)

 ## 知识库

-开始前，请准备一份测试电子文档，WORD，PDF，TXT，excel，markdown 都可以，比如公司休假制度，不涉密的销售说辞，产品知识等等。
+开始前，请准备一份测试电子文档，WORD、PDF、TXT、excel、markdown 都可以，比如公司休假制度、不涉密的销售说辞、产品知识等等。

 这里使用 FastGPT 中文 README 文件为例。

@@ -31,7 +31,7 @@ weight: 102

 ![](/imgs/upload-data.png)

-点击上传后我们需要等待数据处理完成，等到我们上传的文件状态为可用。
+点击上传后我们需要等待数据处理完成，直到我们上传的文件状态为可用。

 ![](/imgs/upload-data2.png)

--- a/docSite/content/zh-cn/docs/use-cases/external-integration/openapi.md
+++ b/docSite/content/zh-cn/docs/use-cases/external-integration/openapi.md
@@ -20,7 +20,7 @@ weight: 502
 ![](/imgs/fastgpt-api1.jpg)

 {{% alert icon="🍅" context="success" %}}
-Tips: 安全起见，你可以设置一个额度或者过期时间，放置 key 被滥用。
+Tips: 安全起见，你可以设置一个额度或者过期时间，防止 key 被滥用。
 {{% /alert %}}


--- a/package.json
+++ b/package.json
@@ -7,7 +7,7 @@
    "format-code": "prettier --config \"./.prettierrc.js\" --write \"./**/src/**/*.{ts,tsx,scss}\"",
    "format-doc": "zhlint --dir ./docSite *.md --fix",
    "gen:theme-typings": "chakra-cli tokens packages/web/styles/theme.ts --out node_modules/.pnpm/node_modules/@chakra-ui/styled-system/dist/theming.types.d.ts",
-    "postinstall": "sh ./scripts/postinstall.sh",
+    "postinstall": "pnpm gen:theme-typings",
    "initIcon": "node ./scripts/icon/init.js",
    "previewIcon": "node ./scripts/icon/index.js",
    "api:gen": "tsc ./scripts/openapi/index.ts && node ./scripts/openapi/index.js && npx @redocly/cli build-docs ./scripts/openapi/openapi.json -o ./projects/app/public/openapi/index.html",
--- a/packages/README.md
+++ b/packages/README.md
@@ -0,0 +1,3 @@
+# 目录说明
+
+该目录为 FastGPT 的依赖包，多端复用。
--- a/packages/global/common/error/code/dataset.ts
+++ b/packages/global/common/error/code/dataset.ts
@@ -4,6 +4,7 @@ import { ErrType } from '../errorCode';
 /* dataset: 501000 */
 export enum DatasetErrEnum {
  unExist = 'unExistDataset',
+  unExistCollection = 'unExistCollection',
  unAuthDataset = 'unAuthDataset',
  unCreateCollection = 'unCreateCollection',
  unAuthDatasetCollection = 'unAuthDatasetCollection',
@@ -28,6 +29,10 @@ const datasetErr = [
    statusText: DatasetErrEnum.unExist,
    message: 'core.dataset.error.unExistDataset'
  },
+  {
+    statusText: DatasetErrEnum.unExistCollection,
+    message: i18nT('common:error_collection_not_exist')
+  },
  {
    statusText: DatasetErrEnum.unAuthDataset,
    message: 'core.dataset.error.unAuthDataset'
--- a/packages/global/common/file/constants.ts
+++ b/packages/global/common/file/constants.ts
@@ -20,4 +20,4 @@ export const ReadFileBaseUrl = `${process.env.FILE_DOMAIN || process.env.FE_DOMA

 export const documentFileType = '.txt, .docx, .csv, .xlsx, .pdf, .md, .html, .pptx';
 export const imageFileType =
-  '.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif';
+  '.jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .ico, .heic, .heif, .avif, .raw, .cr2, .nef, .arw, .dng, .psd, .ai, .eps, .emf, .wmf, .jfif, .exif, .pgm, .ppm, .pbm, .jp2, .j2k, .jpf, .jpx, .jpm, .mj2, .xbm, .pcx';
--- a/packages/global/common/file/tools.ts
+++ b/packages/global/common/file/tools.ts
@@ -1,5 +1,5 @@
 import { detect } from 'jschardet';
-import { documentFileType, imageFileType } from './constants';
+import { documentFileType } from './constants';
 import { ChatFileTypeEnum } from '../../core/chat/constants';
 import { UserChatItemValueItemType } from '../../core/chat/type';
 import * as fs from 'fs';
@@ -25,6 +25,7 @@ export const detectFileEncodingByPath = async (path: string) => {
  const fd = await fs.promises.open(path, 'r');
  try {
    // Read file head
+    // @ts-ignore
    const { bytesRead } = await fd.read(buffer, 0, MAX_BYTES, 0);
    const actualBuffer = buffer.slice(0, bytesRead);

@@ -37,40 +38,49 @@ export const detectFileEncodingByPath = async (path: string) => {
 // Url => user upload file type
 export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file'] | undefined => {
  if (typeof url !== 'string') return;
-  const parseUrl = new URL(url, 'https://locaohost:3000');

-  const filename = (() => {
-    // Check base64 image
-    if (url.startsWith('data:image/')) {
-      const mime = url.split(',')[0].split(':')[1].split(';')[0];
-      return `image.${mime.split('/')[1]}`;
-    }
-    // Old version file url: https://xxx.com/file/read?filename=xxx.pdf
-    const filenameQuery = parseUrl.searchParams.get('filename');
-    if (filenameQuery) return filenameQuery;
+  // Handle base64 image
+  if (url.startsWith('data:')) {
+    const matches = url.match(/^data:([^;]+);base64,/);
+    if (!matches) return;

-    // Common file： https://xxx.com/xxx.pdf?xxxx=xxx
-    const pathname = parseUrl.pathname;
-    if (pathname) return pathname.split('/').pop();
-  })();
+    const mimeType = matches[1].toLowerCase();
+    if (!mimeType.startsWith('image/')) return;

-  if (!filename) return;
-
-  const extension = filename.split('.').pop()?.toLowerCase() || '';
-
-  if (!extension) return;
-
-  if (documentFileType.includes(extension)) {
+    const extension = mimeType.split('/')[1];
    return {
-      type: ChatFileTypeEnum.file,
-      name: filename,
+      type: ChatFileTypeEnum.image,
+      name: `image.${extension}`,
      url
    };
  }
-  if (imageFileType.includes(extension)) {
+
+  try {
+    const parseUrl = new URL(url, 'https://localhost:3000');
+
+    // Get filename from URL
+    const filename = parseUrl.searchParams.get('filename') || parseUrl.pathname.split('/').pop();
+    const extension = filename?.split('.').pop()?.toLowerCase() || '';
+
+    // If it's a document type, return as file, otherwise treat as image
+    if (extension && documentFileType.includes(extension)) {
+      return {
+        type: ChatFileTypeEnum.file,
+        name: filename || 'null',
+        url
+      };
+    }
+
+    // Default to image type for non-document files
    return {
      type: ChatFileTypeEnum.image,
-      name: filename,
+      name: filename || 'null.png',
+      url
+    };
+  } catch (error) {
+    return {
+      type: ChatFileTypeEnum.image,
+      name: 'invalid.png',
      url
    };
  }
--- a/packages/global/common/string/time.ts
+++ b/packages/global/common/string/time.ts
@@ -7,12 +7,14 @@ import { i18nT } from '../../../web/i18n/utils';
 dayjs.extend(utc);
 dayjs.extend(timezone);

-export const formatTime2YMDHMW = (time?: Date) => dayjs(time).format('YYYY-MM-DD HH:mm:ss dddd');
-export const formatTime2YMDHMS = (time?: Date) =>
+export const formatTime2YMDHMW = (time?: Date | number) =>
+  dayjs(time).format('YYYY-MM-DD HH:mm:ss dddd');
+export const formatTime2YMDHMS = (time?: Date | number) =>
  time ? dayjs(time).format('YYYY-MM-DD HH:mm:ss') : '';
-export const formatTime2YMDHM = (time?: Date) =>
+export const formatTime2YMDHM = (time?: Date | number) =>
  time ? dayjs(time).format('YYYY-MM-DD HH:mm') : '';
-export const formatTime2YMD = (time?: Date) => (time ? dayjs(time).format('YYYY-MM-DD') : '');
+export const formatTime2YMD = (time?: Date | number) =>
+  time ? dayjs(time).format('YYYY-MM-DD') : '';
 export const formatTime2HM = (time: Date = new Date()) => dayjs(time).format('HH:mm');

 /**
--- a/packages/global/common/system/types/index.d.ts
+++ b/packages/global/common/system/types/index.d.ts
@@ -41,6 +41,7 @@ export type FastGPTConfigFileType = {
 };

 export type FastGPTFeConfigsType = {
+  show_workorder?: boolean;
  show_emptyChat?: boolean;
  register_method?: ['email' | 'phone' | 'sync'];
  login_method?: ['email' | 'phone']; // Attention: login method is diffrent with oauth
@@ -53,6 +54,7 @@ export type FastGPTFeConfigsType = {
  show_promotion?: boolean;
  show_team_chat?: boolean;
  show_compliance_copywriting?: boolean;
+  show_aiproxy?: boolean;
  concatMd?: string;

  docUrl?: string;
--- a/packages/global/core/ai/model.d.ts
+++ b/packages/global/core/ai/model.d.ts
@@ -17,6 +17,8 @@ type BaseModelItemType = {
  isActive?: boolean;
  isCustom?: boolean;
  isDefault?: boolean;
+  isDefaultDatasetTextModel?: boolean;
+  isDefaultDatasetImageModel?: boolean;

  // If has requestUrl, it will request the model directly
  requestUrl?: string;
--- a/packages/global/core/ai/provider.ts
+++ b/packages/global/core/ai/provider.ts
@@ -22,6 +22,7 @@ export type ModelProviderIdType =
  | 'StepFun'
  | 'Yi'
  | 'Siliconflow'
+  | 'PPIO'
  | 'Ollama'
  | 'BAAI'
  | 'FishAudio'
@@ -71,11 +72,6 @@ export const ModelProviderList: ModelProviderType[] = [
    name: 'Groq',
    avatar: 'model/groq'
  },
-  {
-    id: 'AliCloud',
-    name: i18nT('common:model_alicloud'),
-    avatar: 'model/alicloud'
-  },
  {
    id: 'Qwen',
    name: i18nT('common:model_qwen'),
@@ -86,6 +82,11 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_doubao'),
    avatar: 'model/doubao'
  },
+  {
+    id: 'DeepSeek',
+    name: 'DeepSeek',
+    avatar: 'model/deepseek'
+  },
  {
    id: 'ChatGLM',
    name: i18nT('common:model_chatglm'),
@@ -96,11 +97,6 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_ernie'),
    avatar: 'model/ernie'
  },
-  {
-    id: 'DeepSeek',
-    name: 'DeepSeek',
-    avatar: 'model/deepseek'
-  },
  {
    id: 'Moonshot',
    name: i18nT('common:model_moonshot'),
@@ -162,11 +158,21 @@ export const ModelProviderList: ModelProviderType[] = [
    name: i18nT('common:model_moka'),
    avatar: 'model/moka'
  },
+  {
+    id: 'AliCloud',
+    name: i18nT('common:model_alicloud'),
+    avatar: 'model/alicloud'
+  },
  {
    id: 'Siliconflow',
    name: i18nT('common:model_siliconflow'),
    avatar: 'model/siliconflow'
  },
+  {
+    id: 'PPIO',
+    name: i18nT('common:model_ppio'),
+    avatar: 'model/ppio'
+  },
  {
    id: 'Other',
    name: i18nT('common:model_other'),
--- a/packages/global/core/ai/type.d.ts
+++ b/packages/global/core/ai/type.d.ts
@@ -1,14 +1,12 @@
 import openai from 'openai';
 import type {
  ChatCompletionMessageToolCall,
-  ChatCompletionChunk,
  ChatCompletionMessageParam as SdkChatCompletionMessageParam,
  ChatCompletionToolMessageParam,
  ChatCompletionContentPart as SdkChatCompletionContentPart,
  ChatCompletionUserMessageParam as SdkChatCompletionUserMessageParam,
  ChatCompletionToolMessageParam as SdkChatCompletionToolMessageParam,
-  ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam,
-  ChatCompletionContentPartText
+  ChatCompletionAssistantMessageParam as SdkChatCompletionAssistantMessageParam
 } from 'openai/resources';
 import { ChatMessageTypeEnum } from './constants';
 import { WorkflowInteractiveResponseType } from '../workflow/template/system/interactive/type';
@@ -48,6 +46,7 @@ export type ChatCompletionMessageParam = (
  | CustomChatCompletionToolMessageParam
  | CustomChatCompletionAssistantMessageParam
 ) & {
+  reasoning_text?: string;
  dataId?: string;
  hideInUI?: boolean;
 };
@@ -71,7 +70,8 @@ export type ChatCompletionMessageFunctionCall =
  };

 // Stream response
-export type StreamChatType = Stream<ChatCompletionChunk>;
+export type StreamChatType = Stream<openai.Chat.Completions.ChatCompletionChunk>;
+export type UnStreamChatType = openai.Chat.Completions.ChatCompletion;

 export default openai;
 export * from 'openai';
--- a/packages/global/core/chat/adapt.ts
+++ b/packages/global/core/chat/adapt.ts
@@ -46,7 +46,16 @@ export const chats2GPTMessages = ({

  messages.forEach((item) => {
    const dataId = reserveId ? item.dataId : undefined;
-    if (item.obj === ChatRoleEnum.Human) {
+    if (item.obj === ChatRoleEnum.System) {
+      const content = item.value?.[0]?.text?.content;
+      if (content) {
+        results.push({
+          dataId,
+          role: ChatCompletionRequestMessageRoleEnum.System,
+          content
+        });
+      }
+    } else if (item.obj === ChatRoleEnum.Human) {
      const value = item.value
        .map((item) => {
          if (item.type === ChatItemValueTypeEnum.text) {
@@ -80,15 +89,6 @@ export const chats2GPTMessages = ({
        role: ChatCompletionRequestMessageRoleEnum.User,
        content: simpleUserContentPart(value)
      });
-    } else if (item.obj === ChatRoleEnum.System) {
-      const content = item.value?.[0]?.text?.content;
-      if (content) {
-        results.push({
-          dataId,
-          role: ChatCompletionRequestMessageRoleEnum.System,
-          content
-        });
-      }
    } else {
      const aiResults: ChatCompletionMessageParam[] = [];

@@ -349,7 +349,7 @@ export const chatValue2RuntimePrompt = (value: ChatItemValueItemType[]): Runtime
  };
  value.forEach((item) => {
    if (item.type === 'file' && item.file) {
-      prompt.files?.push(item.file);
+      prompt.files.push(item.file);
    } else if (item.text) {
      prompt.text += item.text.content;
    }
--- a/packages/global/core/dataset/type.d.ts
+++ b/packages/global/core/dataset/type.d.ts
@@ -192,6 +192,7 @@ export type DatasetCollectionItemType = CollectionWithDatasetType & {
  sourceId?: string;
  file?: DatasetFileSchema;
  permission: DatasetPermission;
+  indexAmount: number;
 };

 /* ================= data ===================== */
--- a/packages/global/core/workflow/runtime/utils.ts
+++ b/packages/global/core/workflow/runtime/utils.ts
@@ -10,6 +10,7 @@ import { FlowNodeOutputItemType, ReferenceValueType } from '../type/io';
 import { ChatItemType, NodeOutputItemType } from '../../../core/chat/type';
 import { ChatItemValueTypeEnum, ChatRoleEnum } from '../../../core/chat/constants';
 import { replaceVariable, valToStr } from '../../../common/string/tools';
+import { ChatCompletionChunk } from 'openai/resources';

 export const getMaxHistoryLimitFromNodes = (nodes: StoreNodeItemType[]): number => {
  let limit = 10;
--- a/packages/global/support/user/api.d.ts
+++ b/packages/global/support/user/api.d.ts
@@ -1,5 +1,9 @@
+import { MemberGroupSchemaType, MemberGroupType } from 'support/permission/memberGroup/type';
 import { OAuthEnum } from './constant';
 import { TrackRegisterParams } from './login/api';
+import { TeamMemberStatusEnum } from './team/constant';
+import { OrgType } from './team/org/type';
+import { TeamMemberItemType } from './team/type';

 export type PostLoginProps = {
  username: string;
@@ -21,3 +25,9 @@ export type FastLoginProps = {
  token: string;
  code: string;
 };
+
+export type SearchResult = {
+  members: Omit<TeamMemberItemType, 'teamId' | 'permission'>[];
+  orgs: Omit<OrgType, 'permission' | 'members'>[];
+  groups: MemberGroupSchemaType[];
+};
--- a/packages/global/support/user/team/controller.d.ts
+++ b/packages/global/support/user/team/controller.d.ts
@@ -13,6 +13,7 @@ export type CreateTeamProps = {
  defaultTeam?: boolean;
  memberName?: string;
  memberAvatar?: string;
+  notificationAccount?: string;
 };
 export type UpdateTeamProps = Omit<ThirdPartyAccountType, 'externalWorkflowVariable'> & {
  name?: string;
@@ -39,6 +40,12 @@ export type UpdateInviteProps = {
  tmbId: string;
  status: TeamMemberSchema['status'];
 };
+
+export type UpdateStatusProps = {
+  tmbId: string;
+  status: TeamMemberSchema['status'];
+};
+
 export type InviteMemberResponse = Record<
  'invite' | 'inValid' | 'inTeam',
  { username: string; userId: string }[]
--- a/packages/global/support/user/team/type.d.ts
+++ b/packages/global/support/user/team/type.d.ts
@@ -34,6 +34,7 @@ export type TeamTagSchema = TeamTagItemType & {
  _id: string;
  teamId: string;
  createTime: Date;
+  updateTime?: Date;
 };

 export type TeamMemberSchema = {
@@ -41,6 +42,7 @@ export type TeamMemberSchema = {
  teamId: string;
  userId: string;
  createTime: Date;
+  updateTime?: Date;
  name: string;
  role: `${TeamMemberRoleEnum}`;
  status: `${TeamMemberStatusEnum}`;
@@ -79,6 +81,9 @@ export type TeamMemberItemType = {
  role: `${TeamMemberRoleEnum}`;
  status: `${TeamMemberStatusEnum}`;
  permission: TeamPermission;
+  contact?: string;
+  createTime: Date;
+  updateTime?: Date;
 };

 export type TeamTagItemType = {
--- a/packages/global/support/user/type.d.ts
+++ b/packages/global/support/user/type.d.ts
@@ -17,6 +17,7 @@ export type UserModelSchema = {
  fastgpt_sem?: {
    keyword: string;
  };
+  contact?: string;
 };

 export type UserType = {
@@ -26,9 +27,9 @@ export type UserType = {
  timezone: string;
  promotionRate: UserModelSchema['promotionRate'];
  team: TeamTmbItemType;
-  standardInfo?: standardInfoType;
  notificationAccount?: string;
  permission: TeamPermission;
+  contact?: string;
 };

 export type SourceMemberType = {
--- a/packages/service/common/file/csv.ts
+++ b/packages/service/common/file/csv.ts
@@ -0,0 +1,4 @@
+export const generateCsv = (headers: string[], data: string[][]) => {
+  const csv = [headers.join(','), ...data.map((row) => row.join(','))].join('\n');
+  return csv;
+};
--- a/packages/service/common/file/gridfs/controller.ts
+++ b/packages/service/common/file/gridfs/controller.ts
@@ -18,10 +18,10 @@ export function getGFSCollection(bucket: `${BucketNameEnum}`) {
  MongoDatasetFileSchema;
  MongoChatFileSchema;

-  return connectionMongo.connection.db.collection(`${bucket}.files`);
+  return connectionMongo.connection.db!.collection(`${bucket}.files`);
 }
 export function getGridBucket(bucket: `${BucketNameEnum}`) {
-  return new connectionMongo.mongo.GridFSBucket(connectionMongo.connection.db, {
+  return new connectionMongo.mongo.GridFSBucket(connectionMongo.connection.db!, {
    bucketName: bucket,
    // @ts-ignore
    readPreference: ReadPreference.SECONDARY_PREFERRED // Read from secondary node
--- a/packages/service/common/file/image/controller.ts
+++ b/packages/service/common/file/image/controller.ts
@@ -26,15 +26,18 @@ export async function uploadMongoImg({
  const [base64Mime, base64Data] = base64Img.split(',');
  // Check if mime type is valid
  if (!base64MimeRegex.test(base64Mime)) {
-    return Promise.reject('Invalid image mime type');
+    return Promise.reject('Invalid image base64');
  }

  const mime = `image/${base64Mime.match(base64MimeRegex)?.[1] ?? 'image/jpeg'}`;
  const binary = Buffer.from(base64Data, 'base64');
-  const extension = mime.split('/')[1];
+  let extension = mime.split('/')[1];
+  if (extension.startsWith('x-')) {
+    extension = extension.substring(2); // Remove 'x-' prefix
+  }

-  if (!imageFileType.includes(`.${extension}`)) {
-    return Promise.reject('Invalid image file type');
+  if (!extension || !imageFileType.includes(`.${extension}`)) {
+    return Promise.reject(`Invalid image file type: ${mime}`);
  }

  const { _id } = await MongoImage.create({
@@ -115,7 +118,7 @@ export async function delImgByRelatedId({
 }: {
  teamId: string;
  relateIds: string[];
-  session: ClientSession;
+  session?: ClientSession;
 }) {
  if (relateIds.length === 0) return;

--- a/packages/service/common/file/read/utils.ts
+++ b/packages/service/common/file/read/utils.ts
@@ -111,15 +111,21 @@ export const readRawContentByFileBuffer = async ({
  // markdown data format
  if (imageList) {
    await batchRun(imageList, async (item) => {
-      const src = await uploadMongoImg({
-        base64Img: `data:${item.mime};base64,${item.base64}`,
-        teamId,
-        // expiredTime: addHours(new Date(), 1),
-        metadata: {
-          ...metadata,
-          mime: item.mime
+      const src = await (async () => {
+        try {
+          return await uploadMongoImg({
+            base64Img: `data:${item.mime};base64,${item.base64}`,
+            teamId,
+            // expiredTime: addHours(new Date(), 1),
+            metadata: {
+              ...metadata,
+              mime: item.mime
+            }
+          });
+        } catch (error) {
+          return '';
        }
-      });
+      })();
      rawText = rawText.replace(item.uuid, src);
      if (formatText) {
        formatText = formatText.replace(item.uuid, src);
--- a/packages/service/common/mongo/index.ts
+++ b/packages/service/common/mongo/index.ts
@@ -38,10 +38,12 @@ const addCommonMiddleware = (schema: mongoose.Schema) => {
    schema.post(op, function (this: any, result: any, next) {
      if (this._startTime) {
        const duration = Date.now() - this._startTime;
-
        const warnLogData = {
-          query: this._query,
-          op,
+          collectionName: this.collection?.name,
+          op: this.op,
+          ...(this._query && { query: this._query }),
+          ...(this._update && { update: this._update }),
+          ...(this._delete && { delete: this._delete }),
          duration
        };

--- a/packages/service/common/mongo/init.ts
+++ b/packages/service/common/mongo/init.ts
@@ -16,16 +16,30 @@ export async function connectMongo(): Promise<Mongoose> {

  console.log('mongo start connect');
  try {
-    connectionMongo.set('strictQuery', true);
+    // Remove existing listeners to prevent duplicates
+    connectionMongo.connection.removeAllListeners('error');
+    connectionMongo.connection.removeAllListeners('disconnected');
+    connectionMongo.set('strictQuery', false);

    connectionMongo.connection.on('error', async (error) => {
      console.log('mongo error', error);
-      await connectionMongo.disconnect();
-      await delay(1000);
-      connectMongo();
+      try {
+        if (connectionMongo.connection.readyState !== 0) {
+          await connectionMongo.disconnect();
+          await delay(1000);
+          await connectMongo();
+        }
+      } catch (error) {}
    });
-    connectionMongo.connection.on('disconnected', () => {
+    connectionMongo.connection.on('disconnected', async () => {
      console.log('mongo disconnected');
+      try {
+        if (connectionMongo.connection.readyState !== 0) {
+          await connectionMongo.disconnect();
+          await delay(1000);
+          await connectMongo();
+        }
+      } catch (error) {}
    });

    await connectionMongo.connect(process.env.MONGODB_URI as string, {
--- a/packages/service/common/string/tiktoken/index.ts
+++ b/packages/service/common/string/tiktoken/index.ts
@@ -25,7 +25,7 @@ export const countGptMessagesTokens = async (
      number
    >({
      name: WorkerNameEnum.countGptMessagesTokens,
-      maxReservedThreads: global.systemEnv?.tokenWorkers || 50
+      maxReservedThreads: global.systemEnv?.tokenWorkers || 30
    });

    const total = await workerController.run({ messages, tools, functionCall });
--- a/packages/service/common/system/constants.ts
+++ b/packages/service/common/system/constants.ts
@@ -1 +1,4 @@
 export const FastGPTProUrl = process.env.PRO_URL ? `${process.env.PRO_URL}/api` : '';
+export const isFastGPTMainService = !!process.env.PRO_URL;
+// @ts-ignore
+export const isFastGPTProService = () => !!global.systemConfig;
--- a/packages/service/common/vectorStore/controller.ts
+++ b/packages/service/common/vectorStore/controller.ts
@@ -21,6 +21,7 @@ export const recallFromVectorStore = Vector.embRecall;
 export const getVectorDataByTime = Vector.getVectorDataByTime;
 export const getVectorCountByTeamId = Vector.getVectorCountByTeamId;
 export const getVectorCountByDatasetId = Vector.getVectorCountByDatasetId;
+export const getVectorCountByCollectionId = Vector.getVectorCountByCollectionId;

 export const insertDatasetDataVector = async ({
  model,
--- a/packages/service/common/vectorStore/milvus/class.ts
+++ b/packages/service/common/vectorStore/milvus/class.ts
@@ -321,6 +321,23 @@ export class MilvusCtrl {

    return total;
  };
+  getVectorCountByCollectionId = async (
+    teamId: string,
+    datasetId: string,
+    collectionId: string
+  ) => {
+    const client = await this.getClient();
+
+    const result = await client.query({
+      collection_name: DatasetVectorTableName,
+      output_fields: ['count(*)'],
+      filter: `(teamId == "${String(teamId)}") and (datasetId == "${String(datasetId)}") and (collectionId == "${String(collectionId)}")`
+    });
+
+    const total = result.data?.[0]?.['count(*)'] as number;
+
+    return total;
+  };

  getVectorDataByTime = async (start: Date, end: Date) => {
    const client = await this.getClient();
--- a/packages/service/common/vectorStore/pg/class.ts
+++ b/packages/service/common/vectorStore/pg/class.ts
@@ -240,6 +240,23 @@ export class PgVectorCtrl {
      where: [['team_id', String(teamId)], 'and', ['dataset_id', String(datasetId)]]
    });

+    return total;
+  };
+  getVectorCountByCollectionId = async (
+    teamId: string,
+    datasetId: string,
+    collectionId: string
+  ) => {
+    const total = await PgClient.count(DatasetVectorTableName, {
+      where: [
+        ['team_id', String(teamId)],
+        'and',
+        ['dataset_id', String(datasetId)],
+        'and',
+        ['collection_id', String(collectionId)]
+      ]
+    });
+
    return total;
  };
 }
--- a/packages/service/core/ai/config.ts
+++ b/packages/service/core/ai/config.ts
@@ -1,7 +1,9 @@
 import OpenAI from '@fastgpt/global/core/ai';
 import {
  ChatCompletionCreateParamsNonStreaming,
-  ChatCompletionCreateParamsStreaming
+  ChatCompletionCreateParamsStreaming,
+  StreamChatType,
+  UnStreamChatType
 } from '@fastgpt/global/core/ai/type';
 import { getErrText } from '@fastgpt/global/common/error/utils';
 import { addLog } from '../../common/system/log';
@@ -9,14 +11,17 @@ import { i18nT } from '../../../web/i18n/utils';
 import { OpenaiAccountType } from '@fastgpt/global/support/user/team/type';
 import { getLLMModel } from './model';

-export const openaiBaseUrl = process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1';
+const aiProxyBaseUrl = process.env.AIPROXY_API_ENDPOINT
+  ? `${process.env.AIPROXY_API_ENDPOINT}/v1`
+  : undefined;
+const openaiBaseUrl = aiProxyBaseUrl || process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1';
+const openaiBaseKey = process.env.AIPROXY_API_TOKEN || process.env.CHAT_API_KEY || '';

 export const getAIApi = (props?: { userKey?: OpenaiAccountType; timeout?: number }) => {
  const { userKey, timeout } = props || {};

  const baseUrl = userKey?.baseUrl || global?.systemEnv?.oneapiUrl || openaiBaseUrl;
-  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || process.env.CHAT_API_KEY || '';
-
+  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || openaiBaseKey;
  return new OpenAI({
    baseURL: baseUrl,
    apiKey,
@@ -30,7 +35,7 @@ export const getAxiosConfig = (props?: { userKey?: OpenaiAccountType }) => {
  const { userKey } = props || {};

  const baseUrl = userKey?.baseUrl || global?.systemEnv?.oneapiUrl || openaiBaseUrl;
-  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || process.env.CHAT_API_KEY || '';
+  const apiKey = userKey?.key || global?.systemEnv?.chatApiKey || openaiBaseKey;

  return {
    baseUrl,
@@ -38,29 +43,30 @@ export const getAxiosConfig = (props?: { userKey?: OpenaiAccountType }) => {
  };
 };

-type CompletionsBodyType =
-  | ChatCompletionCreateParamsNonStreaming
-  | ChatCompletionCreateParamsStreaming;
-type InferResponseType<T extends CompletionsBodyType> =
-  T extends ChatCompletionCreateParamsStreaming
-    ? OpenAI.Chat.Completions.ChatCompletionChunk
-    : OpenAI.Chat.Completions.ChatCompletion;
-
-export const createChatCompletion = async <T extends CompletionsBodyType>({
+export const createChatCompletion = async ({
  body,
  userKey,
  timeout,
  options
 }: {
-  body: T;
+  body: ChatCompletionCreateParamsNonStreaming | ChatCompletionCreateParamsStreaming;
  userKey?: OpenaiAccountType;
  timeout?: number;
  options?: OpenAI.RequestOptions;
-}): Promise<{
-  response: InferResponseType<T>;
-  isStreamResponse: boolean;
-  getEmptyResponseTip: () => string;
-}> => {
+}): Promise<
+  {
+    getEmptyResponseTip: () => string;
+  } & (
+    | {
+        response: StreamChatType;
+        isStreamResponse: true;
+      }
+    | {
+        response: UnStreamChatType;
+        isStreamResponse: false;
+      }
+  )
+> => {
  try {
    const modelConstantsData = getLLMModel(body.model);

@@ -69,6 +75,7 @@ export const createChatCompletion = async <T extends CompletionsBodyType>({
      userKey,
      timeout: formatTimeout
    });
+
    const response = await ai.chat.completions.create(body, {
      ...options,
      ...(modelConstantsData.requestUrl ? { path: modelConstantsData.requestUrl } : {}),
@@ -96,9 +103,17 @@ export const createChatCompletion = async <T extends CompletionsBodyType>({
      return i18nT('chat:LLM_model_response_empty');
    };

+    if (isStreamResponse) {
+      return {
+        response,
+        isStreamResponse: true,
+        getEmptyResponseTip
+      };
+    }
+
    return {
-      response: response as InferResponseType<T>,
-      isStreamResponse,
+      response,
+      isStreamResponse: false,
      getEmptyResponseTip
    };
  } catch (error) {
--- a/packages/service/core/ai/config/provider/AliCloud.json
+++ b/packages/service/core/ai/config/provider/AliCloud.json
@@ -1,4 +1,10 @@
 {
  "provider": "AliCloud",
-  "list": []
-}
+  "list": [
+    {
+      "model": "SenseVoiceSmall",
+      "name": "SenseVoiceSmall",
+      "type": "stt"
+    }
+  ]
+}
--- a/packages/service/core/ai/config/provider/Claude.json
+++ b/packages/service/core/ai/config/provider/Claude.json
@@ -1,6 +1,30 @@
 {
  "provider": "Claude",
  "list": [
+    {
+      "model": "claude-3-7-sonnet-20250219",
+      "name": "claude-3-7-sonnet-20250219",
+      "maxContext": 200000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 100000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    },
    {
      "model": "claude-3-5-haiku-20241022",
      "name": "claude-3-5-haiku-20241022",
@@ -10,7 +34,7 @@
      "maxTemperature": 1,
      "showTopP": true,
      "showStopSign": true,
-      "vision": false,
+      "vision": true,
      "toolChoice": true,
      "functionCall": false,
      "defaultSystemChatPrompt": "",
@@ -98,4 +122,4 @@
      "type": "llm"
    }
  ]
-}
+}
--- a/packages/service/core/ai/config/provider/Gemini.json
+++ b/packages/service/core/ai/config/provider/Gemini.json
@@ -1,6 +1,54 @@
 {
  "provider": "Gemini",
  "list": [
+    {
+      "model": "gemini-2.0-flash",
+      "name": "gemini-2.0-flash",
+      "maxContext": 1000000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 60000,
+      "maxTemperature": 1,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
+    },
+    {
+      "model": "gemini-2.0-pro-exp",
+      "name": "gemini-2.0-pro-exp",
+      "maxContext": 2000000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 100000,
+      "maxTemperature": 1,
+      "vision": true,
+      "toolChoice": true,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm",
+      "showTopP": true,
+      "showStopSign": true
+    },
    {
      "model": "gemini-1.5-flash",
      "name": "gemini-1.5-flash",
@@ -153,4 +201,4 @@
      "type": "embedding"
    }
  ]
-}
+}
--- a/packages/service/core/ai/config/provider/Grok.json
+++ b/packages/service/core/ai/config/provider/Grok.json
@@ -1,4 +1,29 @@
 {
  "provider": "Grok",
-  "list": []
-}
+  "list": [
+    {
+      "model": "grok-3",
+      "name": "grok-3",
+      "maxContext": 128000,
+      "maxResponse": 8000,
+      "quoteMaxToken": 128000,
+      "maxTemperature": 1,
+      "showTopP": true,
+      "showStopSign": true,
+      "vision": false,
+      "toolChoice": false,
+      "functionCall": false,
+      "defaultSystemChatPrompt": "",
+      "datasetProcess": true,
+      "usedInClassify": true,
+      "customCQPrompt": "",
+      "usedInExtractFields": true,
+      "usedInQueryExtension": true,
+      "customExtractPrompt": "",
+      "usedInToolCall": true,
+      "defaultConfig": {},
+      "fieldMap": {},
+      "type": "llm"
+    }
+  ]
+}
--- a/packages/service/core/ai/config/provider/PPIO.json
+++ b/packages/service/core/ai/config/provider/PPIO.json
@@ -0,0 +1,4 @@
+{
+  "provider": "PPIO",
+  "list": []
+}
--- a/packages/service/core/ai/config/utils.ts
+++ b/packages/service/core/ai/config/utils.ts
@@ -52,6 +52,12 @@ export const loadSystemModels = async (init = false) => {
        if (model.isDefault) {
          global.systemDefaultModel.llm = model;
        }
+        if (model.isDefaultDatasetTextModel) {
+          global.systemDefaultModel.datasetTextLLM = model;
+        }
+        if (model.isDefaultDatasetImageModel) {
+          global.systemDefaultModel.datasetImageLLM = model;
+        }
      } else if (model.type === ModelTypeEnum.embedding) {
        global.embeddingModelMap.set(model.model, model);
        global.embeddingModelMap.set(model.name, model);
@@ -134,6 +140,16 @@ export const loadSystemModels = async (init = false) => {
    if (!global.systemDefaultModel.llm) {
      global.systemDefaultModel.llm = Array.from(global.llmModelMap.values())[0];
    }
+    if (!global.systemDefaultModel.datasetTextLLM) {
+      global.systemDefaultModel.datasetTextLLM = Array.from(global.llmModelMap.values()).find(
+        (item) => item.datasetProcess
+      );
+    }
+    if (!global.systemDefaultModel.datasetImageLLM) {
+      global.systemDefaultModel.datasetImageLLM = Array.from(global.llmModelMap.values()).find(
+        (item) => item.vision
+      );
+    }
    if (!global.systemDefaultModel.embedding) {
      global.systemDefaultModel.embedding = Array.from(global.embeddingModelMap.values())[0];
    }
--- a/packages/service/core/ai/type.d.ts
+++ b/packages/service/core/ai/type.d.ts
@@ -22,6 +22,9 @@ export type SystemModelItemType =

 export type SystemDefaultModelType = {
  [ModelTypeEnum.llm]?: LLMModelItemType;
+  datasetTextLLM?: LLMModelItemType;
+  datasetImageLLM?: LLMModelItemType;
+
  [ModelTypeEnum.embedding]?: EmbeddingModelItemType;
  [ModelTypeEnum.tts]?: TTSModelType;
  [ModelTypeEnum.stt]?: STTModelType;
--- a/packages/service/core/ai/utils.ts
+++ b/packages/service/core/ai/utils.ts
@@ -37,25 +37,26 @@ export const computedTemperature = ({
  return temperature;
 };

-type CompletionsBodyType = (
+type CompletionsBodyType =
  | ChatCompletionCreateParamsNonStreaming
-  | ChatCompletionCreateParamsStreaming
-) & {
-  response_format?: any;
-  json_schema?: string;
-  stop?: string;
-};
+  | ChatCompletionCreateParamsStreaming;
 type InferCompletionsBody<T> = T extends { stream: true }
  ? ChatCompletionCreateParamsStreaming
-  : ChatCompletionCreateParamsNonStreaming;
+  : T extends { stream: false }
+    ? ChatCompletionCreateParamsNonStreaming
+    : ChatCompletionCreateParamsNonStreaming | ChatCompletionCreateParamsStreaming;

 export const llmCompletionsBodyFormat = <T extends CompletionsBodyType>(
-  body: T,
+  body: T & {
+    response_format?: any;
+    json_schema?: string;
+    stop?: string;
+  },
  model: string | LLMModelItemType
 ): InferCompletionsBody<T> => {
  const modelData = typeof model === 'string' ? getLLMModel(model) : model;
  if (!modelData) {
-    return body as InferCompletionsBody<T>;
+    return body as unknown as InferCompletionsBody<T>;
  }

  const response_format = body.response_format;
@@ -91,16 +92,148 @@ export const llmCompletionsBodyFormat = <T extends CompletionsBodyType>(
    });
  }

-  // console.log(requestBody);
-
-  return requestBody as InferCompletionsBody<T>;
+  return requestBody as unknown as InferCompletionsBody<T>;
 };

-export const llmStreamResponseToText = async (response: StreamChatType) => {
+export const llmStreamResponseToAnswerText = async (response: StreamChatType) => {
  let answer = '';
  for await (const part of response) {
    const content = part.choices?.[0]?.delta?.content || '';
    answer += content;
  }
-  return answer;
+  return parseReasoningContent(answer)[1];
+};
+
+// Parse <think></think> tags to think and answer - unstream response
+export const parseReasoningContent = (text: string): [string, string] => {
+  const regex = /<think>([\s\S]*?)<\/think>/;
+  const match = text.match(regex);
+
+  if (!match) {
+    return ['', text];
+  }
+
+  const thinkContent = match[1].trim();
+
+  // Add answer (remaining text after think tag)
+  const answerContent = text.slice(match.index! + match[0].length);
+
+  return [thinkContent, answerContent];
+};
+
+// Parse <think></think> tags to think and answer - stream response
+export const parseReasoningStreamContent = () => {
+  let isInThinkTag: boolean | undefined;
+
+  const startTag = '<think>';
+  let startTagBuffer = '';
+
+  const endTag = '</think>';
+  let endTagBuffer = '';
+
+  /* 
+    parseReasoning - 只控制是否主动解析 <think></think>，如果接口已经解析了，仍然会返回 think 内容。
+  */
+  const parsePart = (
+    part: {
+      choices: {
+        delta: {
+          content?: string;
+          reasoning_content?: string;
+        };
+      }[];
+    },
+    parseReasoning = false
+  ): [string, string] => {
+    const content = part.choices?.[0]?.delta?.content || '';
+
+    // @ts-ignore
+    const reasoningContent = part.choices?.[0]?.delta?.reasoning_content || '';
+    if (reasoningContent || !parseReasoning) {
+      isInThinkTag = false;
+      return [reasoningContent, content];
+    }
+
+    if (!content) {
+      return ['', ''];
+    }
+
+    // 如果不在 think 标签中，或者有 reasoningContent(接口已解析），则返回 reasoningContent 和 content
+    if (isInThinkTag === false) {
+      return ['', content];
+    }
+
+    // 检测是否为 think 标签开头的数据
+    if (isInThinkTag === undefined) {
+      // Parse content think and answer
+      startTagBuffer += content;
+      // 太少内容时候，暂时不解析
+      if (startTagBuffer.length < startTag.length) {
+        return ['', ''];
+      }
+
+      if (startTagBuffer.startsWith(startTag)) {
+        isInThinkTag = true;
+        return [startTagBuffer.slice(startTag.length), ''];
+      }
+
+      // 如果未命中 think 标签，则认为不在 think 标签中，返回 buffer 内容作为 content
+      isInThinkTag = false;
+      return ['', startTagBuffer];
+    }
+
+    // 确认是 think 标签内容，开始返回 think 内容，并实时检测 </think>
+    /* 
+      检测 </think> 方案。
+      存储所有疑似 </think> 的内容，直到检测到完整的 </think> 标签或超出 </think> 长度。
+      content 返回值包含以下几种情况:
+        abc - 完全未命中尾标签
+        abc<th - 命中一部分尾标签
+        abc</think> - 完全命中尾标签
+        abc</think>abc - 完全命中尾标签
+        </think>abc - 完全命中尾标签
+        k>abc - 命中一部分尾标签
+    */
+    // endTagBuffer 专门用来记录疑似尾标签的内容
+    if (endTagBuffer) {
+      endTagBuffer += content;
+      if (endTagBuffer.includes(endTag)) {
+        isInThinkTag = false;
+        const answer = endTagBuffer.slice(endTag.length);
+        return ['', answer];
+      } else if (endTagBuffer.length >= endTag.length) {
+        // 缓存内容超出尾标签长度，且仍未命中 </think>，则认为本次猜测 </think> 失败，仍处于 think 阶段。
+        const tmp = endTagBuffer;
+        endTagBuffer = '';
+        return [tmp, ''];
+      }
+      return ['', ''];
+    } else if (content.includes(endTag)) {
+      // 返回内容，完整命中</think>，直接结束
+      isInThinkTag = false;
+      const [think, answer] = content.split(endTag);
+      return [think, answer];
+    } else {
+      // 无 buffer，且未命中 </think>，开始疑似 </think> 检测。
+      for (let i = 1; i < endTag.length; i++) {
+        const partialEndTag = endTag.slice(0, i);
+        // 命中一部分尾标签
+        if (content.endsWith(partialEndTag)) {
+          const think = content.slice(0, -partialEndTag.length);
+          endTagBuffer += partialEndTag;
+          return [think, ''];
+        }
+      }
+    }
+
+    // 完全未命中尾标签，还是 think 阶段。
+    return [content, ''];
+  };
+
+  const getStartTagBuffer = () => startTagBuffer;
+
+  return {
+    parsePart,
+    getStartTagBuffer
+  };
 };
--- a/packages/service/core/chat/chatSchema.ts
+++ b/packages/service/core/chat/chatSchema.ts
@@ -1,7 +1,7 @@
 import { connectionMongo, getMongoModel } from '../../common/mongo';
 const { Schema } = connectionMongo;
 import { ChatSchema as ChatType } from '@fastgpt/global/core/chat/type.d';
-import { ChatSourceMap } from '@fastgpt/global/core/chat/constants';
+import { ChatSourceEnum, ChatSourceMap } from '@fastgpt/global/core/chat/constants';
 import {
  TeamCollectionName,
  TeamMemberCollectionName
@@ -52,8 +52,10 @@ const ChatSchema = new Schema({
  },
  source: {
    type: String,
-    required: true
+    required: true,
+    enum: Object.values(ChatSourceEnum)
  },
+  sourceName: String,
  shareId: {
    type: String
  },
--- a/packages/service/core/chat/saveChat.ts
+++ b/packages/service/core/chat/saveChat.ts
@@ -1,6 +1,10 @@
 import type { AIChatItemType, UserChatItemType } from '@fastgpt/global/core/chat/type.d';
 import { MongoApp } from '../app/schema';
-import { ChatItemValueTypeEnum, ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
+import {
+  ChatItemValueTypeEnum,
+  ChatRoleEnum,
+  ChatSourceEnum
+} from '@fastgpt/global/core/chat/constants';
 import { MongoChatItem } from './chatItemSchema';
 import { MongoChat } from './chatSchema';
 import { addLog } from '../../common/system/log';
@@ -22,7 +26,8 @@ type Props = {
  variables?: Record<string, any>;
  isUpdateUseTime: boolean;
  newTitle: string;
-  source: string;
+  source: `${ChatSourceEnum}`;
+  sourceName?: string;
  shareId?: string;
  outLinkUid?: string;
  content: [UserChatItemType & { dataId?: string }, AIChatItemType & { dataId?: string }];
@@ -40,6 +45,7 @@ export async function saveChat({
  isUpdateUseTime,
  newTitle,
  source,
+  sourceName,
  shareId,
  outLinkUid,
  content,
@@ -96,6 +102,7 @@ export async function saveChat({
            pluginInputs,
            title: newTitle,
            source,
+            sourceName,
            shareId,
            outLinkUid,
            metadata: metadataUpdate,
--- a/packages/service/core/chat/utils.ts
+++ b/packages/service/core/chat/utils.ts
@@ -197,7 +197,11 @@ export const loadRequestMessages = async ({
                addLog.info(`Filter invalid image: ${imgUrl}`);
                return;
              }
-            } catch (error) {
+            } catch (error: any) {
+              if (error?.response?.status === 405) {
+                return item;
+              }
+              addLog.warn(`Filter invalid image: ${imgUrl}`, { error });
              return;
            }
          }
--- a/packages/service/core/dataset/collection/controller.ts
+++ b/packages/service/core/dataset/collection/controller.ts
@@ -25,6 +25,7 @@ import { MongoImage } from '../../../common/file/image/schema';
 import { hashStr } from '@fastgpt/global/common/string/tools';
 import { addDays } from 'date-fns';
 import { MongoDatasetDataText } from '../data/dataTextSchema';
+import { delay, retryFn } from '@fastgpt/global/common/system/utils';

 export const createCollectionAndInsertData = async ({
  dataset,
@@ -216,7 +217,7 @@ export async function createOneCollection({
        nextSyncTime
      }
    ],
-    { session }
+    { session, ordered: true }
  );

  return collection;
@@ -227,8 +228,14 @@ export const delCollectionRelatedSource = async ({
  collections,
  session
 }: {
-  collections: DatasetCollectionSchemaType[];
-  session: ClientSession;
+  collections: {
+    teamId: string;
+    fileId?: string;
+    metadata?: {
+      relatedImgId?: string;
+    };
+  }[];
+  session?: ClientSession;
 }) => {
  if (collections.length === 0) return;

@@ -259,11 +266,13 @@ export const delCollectionRelatedSource = async ({
 export async function delCollection({
  collections,
  session,
-  delRelatedSource
+  delImg = true,
+  delFile = true
 }: {
  collections: DatasetCollectionSchemaType[];
  session: ClientSession;
-  delRelatedSource: boolean;
+  delImg: boolean;
+  delFile: boolean;
 }) {
  if (collections.length === 0) return;

@@ -274,83 +283,55 @@ export async function delCollection({
  const datasetIds = Array.from(new Set(collections.map((item) => String(item.datasetId))));
  const collectionIds = collections.map((item) => String(item._id));

-  // Delete training data
-  await MongoDatasetTraining.deleteMany({
-    teamId,
-    datasetIds: { $in: datasetIds },
-    collectionId: { $in: collectionIds }
+  await retryFn(async () => {
+    await Promise.all([
+      // Delete training data
+      MongoDatasetTraining.deleteMany({
+        teamId,
+        datasetId: { $in: datasetIds },
+        collectionId: { $in: collectionIds }
+      }),
+      // Delete dataset_data_texts
+      MongoDatasetDataText.deleteMany({
+        teamId,
+        datasetId: { $in: datasetIds },
+        collectionId: { $in: collectionIds }
+      }),
+      // Delete dataset_datas
+      MongoDatasetData.deleteMany({
+        teamId,
+        datasetId: { $in: datasetIds },
+        collectionId: { $in: collectionIds }
+      }),
+      ...(delImg
+        ? [
+            delImgByRelatedId({
+              teamId,
+              relateIds: collections
+                .map((item) => item?.metadata?.relatedImgId || '')
+                .filter(Boolean)
+            })
+          ]
+        : []),
+      ...(delFile
+        ? [
+            delFileByFileIdList({
+              bucketName: BucketNameEnum.dataset,
+              fileIdList: collections.map((item) => item?.fileId || '').filter(Boolean)
+            })
+          ]
+        : []),
+      // Delete vector data
+      deleteDatasetDataVector({ teamId, datasetIds, collectionIds })
+    ]);
+
+    // delete collections
+    await MongoDatasetCollection.deleteMany(
+      {
+        teamId,
+        _id: { $in: collectionIds }
+      },
+      { session }
+    );
  });
-
-  /* file and imgs */
-  if (delRelatedSource) {
-    await delCollectionRelatedSource({ collections, session });
-  }
-
-  // Delete dataset_datas
-  await MongoDatasetData.deleteMany(
-    { teamId, datasetIds: { $in: datasetIds }, collectionId: { $in: collectionIds } },
-    { session }
-  );
-  // Delete dataset_data_texts
-  await MongoDatasetDataText.deleteMany(
-    { teamId, datasetIds: { $in: datasetIds }, collectionId: { $in: collectionIds } },
-    { session }
-  );
-
-  // delete collections
-  await MongoDatasetCollection.deleteMany(
-    {
-      teamId,
-      _id: { $in: collectionIds }
-    },
-    { session }
-  );
-
-  // no session delete: delete files, vector data
-  await deleteDatasetDataVector({ teamId, datasetIds, collectionIds });
-}
-
-/**
- * delete delOnlyCollection
- */
-export async function delOnlyCollection({
-  collections,
-  session
-}: {
-  collections: DatasetCollectionSchemaType[];
-  session: ClientSession;
-}) {
-  if (collections.length === 0) return;
-
-  const teamId = collections[0].teamId;
-
-  if (!teamId) return Promise.reject('teamId is not exist');
-
-  const datasetIds = Array.from(new Set(collections.map((item) => String(item.datasetId))));
-  const collectionIds = collections.map((item) => String(item._id));
-
-  // delete training data
-  await MongoDatasetTraining.deleteMany({
-    teamId,
-    datasetIds: { $in: datasetIds },
-    collectionId: { $in: collectionIds }
-  });
-
-  // delete dataset.datas
-  await MongoDatasetData.deleteMany(
-    { teamId, datasetIds: { $in: datasetIds }, collectionId: { $in: collectionIds } },
-    { session }
-  );
-
-  // delete collections
-  await MongoDatasetCollection.deleteMany(
-    {
-      teamId,
-      _id: { $in: collectionIds }
-    },
-    { session }
-  );
-
-  // no session delete: delete files, vector data
-  await deleteDatasetDataVector({ teamId, datasetIds, collectionIds });
 }
--- a/packages/service/core/dataset/collection/utils.ts
+++ b/packages/service/core/dataset/collection/utils.ts
@@ -97,7 +97,7 @@ export const createOrGetCollectionTags = async ({
      datasetId,
      tag: tagContent
    })),
-    { session }
+    { session, ordered: true }
  );

  return [...existingTags.map((tag) => tag._id), ...newTags.map((tag) => tag._id)];
@@ -174,6 +174,14 @@ export const syncCollection = async (collection: CollectionWithDatasetType) => {
  }

  await mongoSessionRun(async (session) => {
+    // Delete old collection
+    await delCollection({
+      collections: [collection],
+      delImg: false,
+      delFile: false,
+      session
+    });
+
    // Create new collection
    await createCollectionAndInsertData({
      session,
@@ -208,13 +216,6 @@ export const syncCollection = async (collection: CollectionWithDatasetType) => {
        updateTime: new Date()
      }
    });
-
-    // Delete old collection
-    await delCollection({
-      collections: [collection],
-      delRelatedSource: false,
-      session
-    });
  });

  return DatasetCollectionSyncResultEnum.success;
--- a/packages/service/core/dataset/controller.ts
+++ b/packages/service/core/dataset/controller.ts
@@ -7,6 +7,8 @@ import { MongoDatasetTraining } from './training/schema';
 import { MongoDatasetData } from './data/schema';
 import { deleteDatasetDataVector } from '../../common/vectorStore/controller';
 import { MongoDatasetDataText } from './data/dataTextSchema';
+import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset';
+import { retryFn } from '@fastgpt/global/common/system/utils';

 /* ============= dataset ========== */
 /* find all datasetId by top datasetId */
@@ -54,7 +56,7 @@ export async function getCollectionWithDataset(collectionId: string) {
    .populate<{ dataset: DatasetSchemaType }>('dataset')
    .lean();
  if (!data) {
-    return Promise.reject('Collection is not exist');
+    return Promise.reject(DatasetErrEnum.unExistCollection);
  }
  return data;
 }
@@ -77,40 +79,39 @@ export async function delDatasetRelevantData({

  const datasetIds = datasets.map((item) => item._id);

-  // delete training data
-  await MongoDatasetTraining.deleteMany({
-    teamId,
-    datasetId: { $in: datasetIds }
-  });
-
  // Get _id, teamId, fileId, metadata.relatedImgId for all collections
  const collections = await MongoDatasetCollection.find(
    {
      teamId,
      datasetId: { $in: datasetIds }
    },
-    '_id teamId datasetId fileId metadata',
-    { session }
+    '_id teamId datasetId fileId metadata'
  ).lean();

-  // Delete Image and file
-  await delCollectionRelatedSource({ collections, session });
+  await retryFn(async () => {
+    await Promise.all([
+      // delete training data
+      MongoDatasetTraining.deleteMany({
+        teamId,
+        datasetId: { $in: datasetIds }
+      }),
+      //Delete dataset_data_texts
+      MongoDatasetDataText.deleteMany({
+        teamId,
+        datasetId: { $in: datasetIds }
+      }),
+      //delete dataset_datas
+      MongoDatasetData.deleteMany({ teamId, datasetId: { $in: datasetIds } }),
+      // Delete Image and file
+      delCollectionRelatedSource({ collections }),
+      // Delete vector data
+      deleteDatasetDataVector({ teamId, datasetIds })
+    ]);
+  });

  // delete collections
  await MongoDatasetCollection.deleteMany({
    teamId,
    datasetId: { $in: datasetIds }
  }).session(session);
-
-  // No session delete:
-  // Delete dataset_data_texts
-  await MongoDatasetDataText.deleteMany({
-    teamId,
-    datasetId: { $in: datasetIds }
-  });
-  // delete dataset_datas
-  await MongoDatasetData.deleteMany({ teamId, datasetId: { $in: datasetIds } });
-
-  // Delete vector data
-  await deleteDatasetDataVector({ teamId, datasetIds });
 }
--- a/packages/service/core/dataset/data/dataTextSchema.ts
+++ b/packages/service/core/dataset/data/dataTextSchema.ts
@@ -1,6 +1,6 @@
 import { connectionMongo, getMongoModel } from '../../../common/mongo';
 const { Schema } = connectionMongo;
-import { DatasetDataSchemaType } from '@fastgpt/global/core/dataset/type.d';
+import { DatasetDataTextSchemaType } from '@fastgpt/global/core/dataset/type.d';
 import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
 import { DatasetCollectionName } from '../schema';
 import { DatasetColCollectionName } from '../collection/schema';
@@ -40,12 +40,13 @@ try {
      default_language: 'none'
    }
  );
+  DatasetDataTextSchema.index({ teamId: 1, datasetId: 1, collectionId: 1 });
  DatasetDataTextSchema.index({ dataId: 1 }, { unique: true });
 } catch (error) {
  console.log(error);
 }

-export const MongoDatasetDataText = getMongoModel<DatasetDataSchemaType>(
+export const MongoDatasetDataText = getMongoModel<DatasetDataTextSchemaType>(
  DatasetDataTextCollectionName,
  DatasetDataTextSchema
 );
--- a/packages/service/core/dataset/search/controller.ts
+++ b/packages/service/core/dataset/search/controller.ts
@@ -200,6 +200,7 @@ export async function searchDatasetData(
      forbidCollectionIdList: collections.map((item) => String(item._id))
    };
  };
+
  /* 
    Collection metadata filter
    标签过滤：
@@ -207,6 +208,63 @@ export async function searchDatasetData(
    2. and 标签和 null 不能共存，否则返回空数组
  */
  const filterCollectionByMetadata = async (): Promise<string[] | undefined> => {
+    const getAllCollectionIds = async ({
+      parentCollectionIds
+    }: {
+      parentCollectionIds?: string[];
+    }): Promise<string[] | undefined> => {
+      if (!parentCollectionIds) return;
+      if (parentCollectionIds.length === 0) {
+        return [];
+      }
+
+      const collections = await MongoDatasetCollection.find(
+        {
+          teamId,
+          datasetId: { $in: datasetIds },
+          _id: { $in: parentCollectionIds }
+        },
+        '_id type',
+        {
+          ...readFromSecondary
+        }
+      ).lean();
+
+      const resultIds = new Set<string>();
+      collections.forEach((item) => {
+        if (item.type !== 'folder') {
+          resultIds.add(String(item._id));
+        }
+      });
+
+      const folderIds = collections
+        .filter((item) => item.type === 'folder')
+        .map((item) => String(item._id));
+
+      // Get all child collection ids
+      if (folderIds.length) {
+        const childCollections = await MongoDatasetCollection.find(
+          {
+            teamId,
+            datasetId: { $in: datasetIds },
+            parentId: { $in: folderIds }
+          },
+          '_id type',
+          {
+            ...readFromSecondary
+          }
+        ).lean();
+
+        const childIds = await getAllCollectionIds({
+          parentCollectionIds: childCollections.map((item) => String(item._id))
+        });
+
+        childIds?.forEach((id) => resultIds.add(id));
+      }
+
+      return Array.from(resultIds);
+    };
+
    if (!collectionFilterMatch || !global.feConfigs.isPlus) return;

    let tagCollectionIdList: string[] | undefined = undefined;
@@ -326,13 +384,19 @@ export async function searchDatasetData(
      }

      // Concat tag and time
-      if (tagCollectionIdList && createTimeCollectionIdList) {
-        return tagCollectionIdList.filter((id) => createTimeCollectionIdList!.includes(id));
-      } else if (tagCollectionIdList) {
-        return tagCollectionIdList;
-      } else if (createTimeCollectionIdList) {
-        return createTimeCollectionIdList;
-      }
+      const collectionIds = (() => {
+        if (tagCollectionIdList && createTimeCollectionIdList) {
+          return tagCollectionIdList.filter((id) =>
+            (createTimeCollectionIdList as string[]).includes(id)
+          );
+        }
+
+        return tagCollectionIdList || createTimeCollectionIdList;
+      })();
+
+      return await getAllCollectionIds({
+        parentCollectionIds: collectionIds
+      });
    } catch (error) {}
  };
  const embeddingRecall = async ({
@@ -383,6 +447,7 @@ export async function searchDatasetData(
      ).lean()
    ]);

+    const set = new Map<string, number>();
    const formatResult = results
      .map((item, index) => {
        const collection = collections.find((col) => String(col._id) === String(item.collectionId));
@@ -398,8 +463,6 @@ export async function searchDatasetData(
          return;
        }

-        const score = item?.score || 0;
-
        const result: SearchDataResponseItemType = {
          id: String(data._id),
          updateTime: data.updateTime,
@@ -409,12 +472,24 @@ export async function searchDatasetData(
          datasetId: String(data.datasetId),
          collectionId: String(data.collectionId),
          ...getCollectionSourceData(collection),
-          score: [{ type: SearchScoreTypeEnum.embedding, value: score, index }]
+          score: [{ type: SearchScoreTypeEnum.embedding, value: item?.score || 0, index }]
        };

        return result;
      })
-      .filter(Boolean) as SearchDataResponseItemType[];
+      .filter((item) => {
+        if (!item) return false;
+        if (set.has(item.id)) return false;
+        set.set(item.id, 1);
+        return true;
+      })
+      .map((item, index) => {
+        if (!item) return;
+        return {
+          ...item,
+          score: item.score.map((item) => ({ ...item, index }))
+        };
+      }) as SearchDataResponseItemType[];

    return {
      embeddingRecallResults: formatResult,
@@ -717,11 +792,12 @@ export const defaultSearchDatasetData = async ({
    ? getLLMModel(datasetSearchExtensionModel)
    : undefined;

-  const { concatQueries, rewriteQuery, aiExtensionResult } = await datasetSearchQueryExtension({
-    query,
-    extensionModel,
-    extensionBg: datasetSearchExtensionBg
-  });
+  const { concatQueries, extensionQueries, rewriteQuery, aiExtensionResult } =
+    await datasetSearchQueryExtension({
+      query,
+      extensionModel,
+      extensionBg: datasetSearchExtensionBg
+    });

  const result = await searchDatasetData({
    ...props,
@@ -736,7 +812,7 @@ export const defaultSearchDatasetData = async ({
          model: aiExtensionResult.model,
          inputTokens: aiExtensionResult.inputTokens,
          outputTokens: aiExtensionResult.outputTokens,
-          query: concatQueries.join('\n')
+          query: extensionQueries.join('\n')
        }
      : undefined
  };
--- a/packages/service/core/dataset/search/utils.ts
+++ b/packages/service/core/dataset/search/utils.ts
@@ -72,12 +72,15 @@ Human: ${query}
    if (result.extensionQueries?.length === 0) return;
    return result;
  })();
+
+  const extensionQueries = filterSamQuery(aiExtensionResult?.extensionQueries || []);
  if (aiExtensionResult) {
-    queries = filterSamQuery(queries.concat(aiExtensionResult.extensionQueries));
+    queries = filterSamQuery(queries.concat(extensionQueries));
    rewriteQuery = queries.join('\n');
  }

  return {
+    extensionQueries,
    concatQueries: queries,
    rewriteQuery,
    aiExtensionResult
--- a/packages/service/core/workflow/dispatch/agent/runTool/index.ts
+++ b/packages/service/core/workflow/dispatch/agent/runTool/index.ts
@@ -334,7 +334,7 @@ const getMultiInput = async ({

  return {
    documentQuoteText: text,
-    userFiles: fileLinks.map((url) => parseUrlToFileType(url))
+    userFiles: fileLinks.map((url) => parseUrlToFileType(url)).filter(Boolean)
  };
 };

--- a/packages/service/core/workflow/dispatch/chat/oneapi.ts
+++ b/packages/service/core/workflow/dispatch/chat/oneapi.ts
@@ -4,12 +4,9 @@ import type { ChatItemType, UserChatItemValueItemType } from '@fastgpt/global/co
 import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
 import { SseResponseEventEnum } from '@fastgpt/global/core/workflow/runtime/constants';
 import { textAdaptGptResponse } from '@fastgpt/global/core/workflow/runtime/utils';
+import { parseReasoningContent, parseReasoningStreamContent } from '../../../ai/utils';
 import { createChatCompletion } from '../../../ai/config';
-import type {
-  ChatCompletion,
-  ChatCompletionMessageParam,
-  StreamChatType
-} from '@fastgpt/global/core/ai/type.d';
+import type { ChatCompletionMessageParam, StreamChatType } from '@fastgpt/global/core/ai/type.d';
 import { formatModelChars2Points } from '../../../../support/wallet/usage/utils';
 import type { LLMModelItemType } from '@fastgpt/global/core/ai/model.d';
 import { postTextCensor } from '../../../../common/api/requestPlusApi';
@@ -195,7 +192,13 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
  });

  const { answerText, reasoningText } = await (async () => {
-    if (res && isStreamResponse) {
+    if (isStreamResponse) {
+      if (!res) {
+        return {
+          answerText: '',
+          reasoningText: ''
+        };
+      }
      // sse response
      const { answer, reasoning } = await streamResponse({
        res,
@@ -210,34 +213,49 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
        reasoningText: reasoning
      };
    } else {
-      const unStreamResponse = response as ChatCompletion;
-      const answer = unStreamResponse.choices?.[0]?.message?.content || '';
-      // @ts-ignore
-      const reasoning = unStreamResponse.choices?.[0]?.message?.reasoning_content || '';
+      const { content, reasoningContent } = (() => {
+        const content = response.choices?.[0]?.message?.content || '';
+        // @ts-ignore
+        const reasoningContent: string = response.choices?.[0]?.message?.reasoning_content || '';
+
+        // API already parse reasoning content
+        if (reasoningContent || !aiChatReasoning) {
+          return {
+            content,
+            reasoningContent
+          };
+        }
+
+        const [think, answer] = parseReasoningContent(content);
+        return {
+          content: answer,
+          reasoningContent: think
+        };
+      })();

      // Some models do not support streaming
      if (stream) {
-        if (isResponseAnswerText && answer) {
+        if (aiChatReasoning && reasoningContent) {
          workflowStreamResponse?.({
            event: SseResponseEventEnum.fastAnswer,
            data: textAdaptGptResponse({
-              text: answer
+              reasoning_content: reasoningContent
            })
          });
        }
-        if (aiChatReasoning && reasoning) {
+        if (isResponseAnswerText && content) {
          workflowStreamResponse?.({
            event: SseResponseEventEnum.fastAnswer,
            data: textAdaptGptResponse({
-              reasoning_content: reasoning
+              text: content
            })
          });
        }
      }

      return {
-        answerText: answer,
-        reasoningText: reasoning
+        answerText: content,
+        reasoningText: reasoningContent
      };
    }
  })();
@@ -249,7 +267,8 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
  const AIMessages: ChatCompletionMessageParam[] = [
    {
      role: ChatCompletionRequestMessageRoleEnum.Assistant,
-      content: answerText
+      content: answerText,
+      reasoning_text: reasoningText // reasoning_text is only recorded for response, but not for request
    }
  ];

@@ -267,7 +286,7 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
  });

  return {
-    answerText,
+    answerText: answerText.trim(),
    reasoningText,
    [DispatchNodeResponseKeyEnum.nodeResponse]: {
      totalPoints: externalProvider.openaiAccount?.key ? 0 : totalPoints,
@@ -386,7 +405,7 @@ async function getMultiInput({

  return {
    documentQuoteText: text,
-    userFiles: fileLinks.map((url) => parseUrlToFileType(url))
+    userFiles: fileLinks.map((url) => parseUrlToFileType(url)).filter(Boolean)
  };
 }

@@ -500,26 +519,18 @@ async function streamResponse({
  });
  let answer = '';
  let reasoning = '';
+  const { parsePart, getStartTagBuffer } = parseReasoningStreamContent();
+
  for await (const part of stream) {
    if (res.closed) {
      stream.controller?.abort();
      break;
    }

-    const content = part.choices?.[0]?.delta?.content || '';
+    const [reasoningContent, content] = parsePart(part, aiChatReasoning);
    answer += content;
-    if (isResponseAnswerText && content) {
-      workflowStreamResponse?.({
-        write,
-        event: SseResponseEventEnum.answer,
-        data: textAdaptGptResponse({
-          text: content
-        })
-      });
-    }
-
-    const reasoningContent = part.choices?.[0]?.delta?.reasoning_content || '';
    reasoning += reasoningContent;
+
    if (aiChatReasoning && reasoningContent) {
      workflowStreamResponse?.({
        write,
@@ -529,6 +540,21 @@ async function streamResponse({
        })
      });
    }
+
+    if (isResponseAnswerText && content) {
+      workflowStreamResponse?.({
+        write,
+        event: SseResponseEventEnum.answer,
+        data: textAdaptGptResponse({
+          text: content
+        })
+      });
+    }
+  }
+
+  // if answer is empty, try to get value from startTagBuffer. (Cause: The response content is too short to exceed the minimum parse length)
+  if (answer === '') {
+    answer = getStartTagBuffer();
  }

  return { answer, reasoning };
--- a/packages/service/core/workflow/dispatch/index.ts
+++ b/packages/service/core/workflow/dispatch/index.ts
@@ -232,9 +232,14 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
      chatNodeUsages = chatNodeUsages.concat(nodeDispatchUsages);
    }

-    if (toolResponses !== undefined) {
+    if (toolResponses !== undefined && toolResponses !== null) {
      if (Array.isArray(toolResponses) && toolResponses.length === 0) return;
-      if (typeof toolResponses === 'object' && Object.keys(toolResponses).length === 0) return;
+      if (
+        !Array.isArray(toolResponses) &&
+        typeof toolResponses === 'object' &&
+        Object.keys(toolResponses).length === 0
+      )
+        return;
      toolRunResponse = toolResponses;
    }

@@ -243,12 +248,17 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
      chatAssistantResponse = chatAssistantResponse.concat(assistantResponses);
    } else {
      if (reasoningText) {
-        chatAssistantResponse.push({
-          type: ChatItemValueTypeEnum.reasoning,
-          reasoning: {
-            content: reasoningText
-          }
-        });
+        const isResponseReasoningText = inputs.find(
+          (item) => item.key === NodeInputKeyEnum.aiChatReasoning
+        )?.value;
+        if (isResponseReasoningText) {
+          chatAssistantResponse.push({
+            type: ChatItemValueTypeEnum.reasoning,
+            reasoning: {
+              content: reasoningText
+            }
+          });
+        }
      }
      if (answerText) {
        // save assistant text response
--- a/packages/service/core/workflow/dispatch/plugin/runApp.ts
+++ b/packages/service/core/workflow/dispatch/plugin/runApp.ts
@@ -53,7 +53,7 @@ export const dispatchRunAppNode = async (props: Props): Promise<Response> => {

  const userInputFiles = (() => {
    if (fileUrlList) {
-      return fileUrlList.map((url) => parseUrlToFileType(url));
+      return fileUrlList.map((url) => parseUrlToFileType(url)).filter(Boolean);
    }
    // Adapt version 4.8.13 upgrade
    return files;
--- a/packages/service/core/workflow/dispatch/tools/http468.ts
+++ b/packages/service/core/workflow/dispatch/tools/http468.ts
@@ -398,41 +398,6 @@ async function fetchData({
  };
 }

-// function replaceVariable(text: string, obj: Record<string, any>) {
-//   for (const [key, value] of Object.entries(obj)) {
-//     if (value === undefined) {
-//       text = text.replace(new RegExp(`{{(${key})}}`, 'g'), UNDEFINED_SIGN);
-//     } else {
-//       const replacement = JSON.stringify(value);
-//       const unquotedReplacement =
-//         replacement.startsWith('"') && replacement.endsWith('"')
-//           ? replacement.slice(1, -1)
-//           : replacement;
-//       text = text.replace(new RegExp(`{{(${key})}}`, 'g'), () => unquotedReplacement);
-//     }
-//   }
-//   return text || '';
-// }
-// function removeUndefinedSign(obj: Record<string, any>) {
-//   for (const key in obj) {
-//     if (obj[key] === UNDEFINED_SIGN) {
-//       obj[key] = undefined;
-//     } else if (Array.isArray(obj[key])) {
-//       obj[key] = obj[key].map((item: any) => {
-//         if (item === UNDEFINED_SIGN) {
-//           return undefined;
-//         } else if (typeof item === 'object') {
-//           removeUndefinedSign(item);
-//         }
-//         return item;
-//       });
-//     } else if (typeof obj[key] === 'object') {
-//       removeUndefinedSign(obj[key]);
-//     }
-//   }
-//   return obj;
-// }
-
 // Replace some special response from system plugin
 async function replaceSystemPluginResponse({
  response,
--- a/packages/service/package.json
+++ b/packages/service/package.json
@@ -24,7 +24,7 @@
    "jsonwebtoken": "^9.0.2",
    "lodash": "^4.17.21",
    "mammoth": "^1.6.0",
-    "mongoose": "^7.0.2",
+    "mongoose": "^8.10.1",
    "multer": "1.4.5-lts.1",
    "next": "14.2.5",
    "nextjs-cors": "^2.2.0",
--- a/packages/service/support/permission/controller.ts
+++ b/packages/service/support/permission/controller.ts
@@ -178,7 +178,7 @@ export const getClbsAndGroupsWithInfo = async ({
  ]);

 export const delResourcePermissionById = (id: string) => {
-  return MongoResourcePermission.findByIdAndRemove(id);
+  return MongoResourcePermission.findByIdAndDelete(id);
 };
 export const delResourcePermission = ({
  session,
--- a/packages/service/support/permission/inheritPermission.ts
+++ b/packages/service/support/permission/inheritPermission.ts
@@ -196,7 +196,8 @@ export async function syncCollaborators({
      permission: item.permission
    })),
    {
-      session
+      session,
+      ordered: true
    }
  );
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Archer	e860c56b77	perf: delete dataset (#3949 ) * fix: collection list count * fix: collection list count * ai proxy ui * perf: delete dataset * perf: add dataset text index * update doc	2025-03-03 12:49:13 +08:00
Archer	efac5312b4	fix: rerank model cannot use ai proxy (#3945 ) * fix: collection list count * fix: collection list count * fix: rerank model cannot use ai proxy * mongo init	2025-03-03 11:49:35 +08:00
Finley Ge	4bc7f21182	fix: add order:true to all create transactions (#3948 )	2025-03-03 11:37:51 +08:00
gggaaallleee	113e8f711f	add env proxypool (#3939 )	2025-03-02 17:50:03 +08:00
Archer	abc6dffb41	4.8.23 dev (#3932 ) * fix: collection list count * fix: collection list count * update doc * perf: init log * yml	2025-02-28 19:18:12 +08:00
gggaaallleee	f7b2a57ca3	1 (#3924 )	2025-02-28 19:00:58 +08:00
Archer	cf0aaa1091	fix: invalid dataset data clear (#3927 ) * fix: collection list count * fix: collection list count * fix: invalid dataset data clear * update ts * perf: cron clear invalid data * perf: init * perf: clear invalid code * update init * perf: clear invalid code * perf: clear invalid code * perf: init count * batch init * batch init * batch init * batch init * add comment * perf: init * fix: api proxy type	2025-02-28 17:49:20 +08:00
Archer	ac4255ea0c	4.8.23 dev (#3926 ) * fix: collection list count * fix: collection list count * fix: ts	2025-02-28 12:33:09 +08:00
Archer	df4d6f86ce	fix: delete dataset field error (#3925 ) * fix: collection list count * fix: collection list count * update doc * perf: tts selector ui * fix: delete dataset field error * doc	2025-02-28 12:29:18 +08:00
heheer	e697fda82f	fix: export chat log - chat detail order (#3923 )	2025-02-28 11:33:46 +08:00
Archer	1aa319e7aa	Update package.json (#3919 )	2025-02-27 22:25:26 +08:00
Archer	fc9e614f88	4.8.23 dev (#3917 ) * fix: icon refresh * fix: aiproxy http request * fix: collection list count * fix: collection list count * fix: tts selector name * update action	2025-02-27 22:15:48 +08:00
Archer	1121ea33bd	更新 docker.md (#3913 )	2025-02-27 17:02:06 +08:00
Finley Ge	9bbee60cde	fix: ts error (#3911 )	2025-02-27 16:31:14 +08:00
Finley Ge	9f57ad0017	fix: mongoose strictquery to false (#3906 )	2025-02-27 11:25:29 +08:00
Archer	c3d3b30d7e	update code positon (#3907 )	2025-02-27 10:30:43 +08:00
Archer	fb0eb49196	fix: pptx encoding (#3905 )	2025-02-27 10:04:59 +08:00
ZongLiang	27ebd2e8cf	Update parseOffice.ts (#3901 ) 更新本地导入pptx文件提示文件编码错误 The argument 'windows-1252' is invalid encoding. Received 'encoding'	2025-02-27 09:57:34 +08:00
Archer	81a06718d8	feat: ai proxy v1 (#3898 ) * feat: ai proxy v1 * perf: ai proxy channel crud * feat: ai proxy logs * feat: channel test * doc * update lock	2025-02-27 09:56:52 +08:00
Archer	3c382d1240	Update intro.md (#3900 )	2025-02-26 22:37:56 +08:00
Finley Ge	747bb303ec	chore: upgrade mongoose to v8.10.x for security (#3868 ) * chore: upgrade mongoose to v8.10.x for security * chore: remove duplicate code * fix: ts error	2025-02-26 18:32:19 +08:00
Finley Ge	cf9c8e9f6a	fix: leave team and refresh memberlist (#3893 )	2025-02-26 18:29:05 +08:00
Archer	5d5bee9e41	remove markdown format;refresh username;perf: latext render (#3877 ) * refresh username * remove md format * perf: latext render * ignore big image * model config	2025-02-25 16:16:30 +08:00
Archer	4f0dd96699	perf: work order tip (#3874 )	2025-02-24 20:26:15 +08:00
Finley Ge	fb6dbaf2d6	feat/workorder (#3860 ) * feat: workorder * pref: workorder button * chore: move workorder to common * chore: format code * pref: style	2025-02-24 19:59:06 +08:00
风沐白	ffc1520f4c	Update quick-start.md (#3873 )	2025-02-24 19:56:56 +08:00
Archer	255764400f	feat: model config required check;feat: dataset text model default setting (#3866 ) * feat: model config required check * feat: dataset text model default setting * perf: collection list count * fix: ts * remove index count	2025-02-24 19:55:49 +08:00
heheer	3bfe802c48	fix collection folder tags filter (#3853 ) * fix collection folder tags filter * add comment * fix	2025-02-24 17:43:31 +08:00
YeYuheng	2bf17dbb87	Marker doc update (#3869 ) * update-marker-doc * marker.md * marker.md	2025-02-24 13:13:05 +08:00
Archer	8d766372fe	update doc (#3840 ) * update doc * update doc	2025-02-20 10:40:00 +08:00
Archer	ca5717936b	update doc (#3836 )	2025-02-19 22:32:08 +08:00
Archer	6762723b10	perf: ery extension and fix filter same embedding result (#3833 ) * perf: ery extension and fix filter same embedding result * fix: extract node too long * perf: ui * perf: not chatId will auto save * fix: laf * fix: member load * feat: add completions unstream error response * feat: add completions unstream error response * updat emodel provider	2025-02-19 22:16:43 +08:00
heheer	8604cbd021	fix source name (#3834 )	2025-02-19 20:42:30 +08:00
Finley Ge	206325bc5f	chore: team, orgs, search and so on (#3807 ) * feat: clb search support username, memberName, contacts * feat: popup org names * feat: update team member table * feat: restore the member * feat: search user in team member table * feat: bind contact * feat: export members * feat: org tab could delete member * feat: org table search * feat: team notification account bind * feat: permission tab search * fix: wecom sso * chore(init): copy notificationAccount to user.contact * chore: adjust * fix: ts error * fix: useConfirm iconColor customization * pref: fe * fix: style * fix: fix team member manage * pref: enlarge team member pagesize * pref: initv4822 * fix: pageSize * pref: initscritpt	2025-02-19 17:27:19 +08:00
Archer	5fd520c794	perf: gemini config (#3828 ) * doc * doc * perf: gemini config	2025-02-19 12:00:31 +08:00
Archer	09205e4666	fix: price page init data;perf: usage code;fix: reasoning tokens;fix: workflow basic node cannot upgrade (#3816 ) * fix: img read * fix: price page init data * perf: ai model avatar * perf: refresh in change team * perf: null checker * perf: usage code * fix: reasoning tokens * fix: workflow basic node cannot upgrade * perf: model refresh * perf: icon refresh	2025-02-18 20:50:25 +08:00
Finley Ge	ccf28d83b8	fix: app version addSourcemember tmbid could be empty (#3822 )	2025-02-18 20:26:49 +08:00
LGiki	420aaad48e	chore: fix typo in docs (#3819 )	2025-02-18 20:25:51 +08:00
heheer	8ba2339890	download fetch baseurl & node select dnd (#3820 )	2025-02-18 20:25:15 +08:00
Archer	e7b8934367	Update 4818.md (#3818 )	2025-02-18 14:26:21 +08:00
Finley Ge	3e13397614	fix: refresh memberlist when switching account (#3814 )	2025-02-18 13:54:56 +08:00
Archer	b14674cc6f	fix: whisper checker;fix: img read (#3813 ) * fix: img read * fix: whisper checker * perf: dev doc * perf: dev doc * remove invalid code	2025-02-18 10:08:25 +08:00
Archer	4d20274a97	feat: think tag parse (#3805 ) (#3808 ) * feat: think tag parse * remove some model config * feat: parse think tag test	2025-02-17 20:57:36 +08:00
heheer	4447e40364	fix template market simple app (#3804 )	2025-02-17 20:56:46 +08:00
John Chen	23949230ee	fix document (#3806 ) V2版本“获取集合列表”接口的path区分了大小写，使用/api/core/dataset/collection/listv2会返回404，必须使用大写V	2025-02-17 20:55:34 +08:00
saikidev	cd7a897304	chore: add ppio provider (#3789 )	2025-02-14 17:04:43 +08:00
Archer	18aff8b8db	update yml version (#3787 )	2025-02-14 12:50:54 +08:00
Archer	d2b60ec785	fix: model check circle tip (#3786 ) * model config * feat: normalization embedding * remove log * version doc * version doc * fix: model check circle tip * uml	2025-02-14 11:42:14 +08:00
a.e.	1226fe42a1	fix: skip thirdparty sso state verification (#3721 ) (#3782 )	2025-02-14 11:39:34 +08:00
Finley Ge	abd375cdec	fix: app/dataset list api return private flag (#3784 )	2025-02-14 11:38:48 +08:00
Archer	7aacce8b0b	4.9.0 test (#3779 ) * model config * feat: normalization embedding * remove log * version doc * version doc	2025-02-13 16:27:41 +08:00