FIX: Query Extension 历史记录拼接不正确 (#1144)

* FIX: Query Extension 历史记录拼接不正确

* add .text

.github/ISSUE_TEMPLATE/bugs.md

@@ -19,7 +19,7 @@ assignees: ''
 **你的版本**
 
 - [ ] 公有云版本
-- [ ] 私有部署版本
+- [ ] 私有部署版本, 具体版本号: 
 
 **问题描述**
 

.github/ISSUE_TEMPLATE/config.yml

@@ -1,5 +1,5 @@
 blank_issues_enabled: false
 contact_links:
   - name: 微信交流群
-    url: https://doc.fastgpt.in/wechat-fastgpt.webp
+    url: https://oss.laf.run/htr4n1-images/fastgpt-qr-code.jpg
     about: FastGPT 全是问题群

.github/workflows/bot-issues-translator.yml

@@ -1,19 +0,0 @@
-name: 'Github Rebot for issues-translator'
-on:
-  issues:
-    types: [ opened ]
-  issue_comment:
-    types: [ created ]
-jobs:
-  translate:
-    permissions:
-      issues: write
-      discussions: write
-      pull-requests: write
-    runs-on: ubuntu-latest
-    steps:
-      - uses: usthe/issues-translate-action@v2.7
-        with:
-          IS_MODIFY_TITLE: true
-          BOT_GITHUB_TOKEN: ${{ secrets.GH_PAT }}
-          CUSTOM_BOT_NOTE: Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

.github/workflows/docs-image.yml

@@ -1,4 +1,4 @@
-name: Build FastGPT docs images and copy image to docker hub
+name: Build docs images and copy image to docker hub
 on:
   workflow_dispatch:
   push:
@@ -8,82 +8,67 @@ on:
       - 'main'
     tags:
       - 'v*.*.*'
+
 jobs:
   build-fastgpt-docs-images:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
+
+      - name: Get current date and time
+        id: datetime
+        run: echo "datetime=$(date +'%Y%m%d%H%M%S')" >> "$GITHUB_OUTPUT"
+
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v5
         with:
-          fetch-depth: 1
+          # list of Docker images to use as base name for tags
-      - name: Set up QEMU (optional)
+          images: |
-        uses: docker/setup-qemu-action@v2
+            ${{ secrets.DOCKER_HUB_NAME }}/fastgpt-docs
+            ghcr.io/${{ github.repository_owner }}/fastgpt-docs
+            registry.cn-hangzhou.aliyuncs.com/${{ secrets.ALI_HUB_USERNAME }}/fastgpt-docs
+          tags: |
+            ${{ steps.datetime.outputs.datetime }}
+          flavor: latest=false
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
+        uses: docker/setup-buildx-action@v3
+
+      - name: Login to DockerHub
+        uses: docker/login-action@v3
         with:
-          driver-opts: network=host
+          username: ${{ secrets.DOCKER_HUB_NAME }}
-      - name: Cache Docker layers
+          password: ${{ secrets.DOCKER_HUB_PASSWORD }}
-        uses: actions/cache@v2
+
-        with:
+      - name: Login to ghcr.io
-          path: /tmp/.buildx-cache
+        uses: docker/login-action@v3
-          key: ${{ runner.os }}-buildx-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-buildx-
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@v2
         with:
           registry: ghcr.io
           username: ${{ github.repository_owner }}
           password: ${{ secrets.GH_PAT }}
-      - name: Set DOCKER_REPO_TAGGED based on branch or tag
-        run: |
-          if [[ "${{ github.ref_name }}" == "main" ]]; then
-            echo "DOCKER_REPO_TAGGED=ghcr.io/${{ github.repository_owner }}/fastgpt-docs:latest" >> $GITHUB_ENV
-          else
-            echo "DOCKER_REPO_TAGGED=ghcr.io/${{ github.repository_owner }}/fastgpt-docs:${{ github.ref_name }}" >> $GITHUB_ENV
-          fi
-      - name: Build and publish image for main branch or tag push event
-        env:
-          DOCKER_REPO_TAGGED: ${{ env.DOCKER_REPO_TAGGED }}
-        run: |
-          docker buildx build \
-          --build-arg name=app \
-          --platform linux/amd64,linux/arm64 \
-          --label "org.opencontainers.image.source=  https://github.com/  ${{ github.repository_owner }}/FastGPT" \
-          --label "org.opencontainers.image.description=fastgpt image" \
-          --label "org.opencontainers.image.licenses=Apache" \
-          --push \
-          --cache-from=type=local,src=/tmp/.buildx-cache \
-          --cache-to=type=local,dest=/tmp/.buildx-cache \
-          -t ${DOCKER_REPO_TAGGED} \
-          -f docSite/Dockerfile \
-          .
-  push-to-docker-hub:
-    needs: build-fastgpt-docs-images
-    runs-on: ubuntu-20.04
-    if: github.repository == 'labring/FastGPT'
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v3
-      - name: Login to Docker Hub
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.DOCKER_HUB_NAME }}
-          password: ${{ secrets.DOCKER_HUB_PASSWORD }}
-      - name: Set DOCKER_REPO_TAGGED based on branch or tag
-        run: |
-          if [[ "${{ github.ref_name }}" == "main" ]]; then
-            echo "IMAGE_TAG=latest" >> $GITHUB_ENV
-          else
-            echo "IMAGE_TAG=${{ github.ref_name }}" >> $GITHUB_ENV
-          fi
-      - name: Pull image from GitHub Container Registry
-        run: docker pull ghcr.io/${{ github.repository_owner }}/fastgpt-docs:${{env.IMAGE_TAG}}
-      - name: Tag image with Docker Hub repository name and version tag
-        run: docker tag ghcr.io/${{ github.repository_owner }}/fastgpt-docs:${{env.IMAGE_TAG}} ${{ secrets.DOCKER_IMAGE_NAME }}:${{env.IMAGE_TAG}}
-      - name: Push image to Docker Hub
-        run: docker push ${{ secrets.DOCKER_IMAGE_NAME }}:${{env.IMAGE_TAG}}
 
+      - name: Login to Aliyun
+        uses: docker/login-action@v3
+        with:
+          registry: registry.cn-hangzhou.aliyuncs.com
+          username: ${{ secrets.ALI_HUB_USERNAME }}
+          password: ${{ secrets.ALI_HUB_PASSWORD }}
+
+      - name: Build and push Docker images to ghcr.io and DockerHub
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: ./docSite/Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+    outputs:
+      tags: ${{ steps.datetime.outputs.datetime }} 
   update-docs-image:
     needs: build-fastgpt-docs-images
     runs-on: ubuntu-20.04
@@ -95,4 +80,9 @@ jobs:
         env:
           KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
         with:
-          args: rollout restart deployment fastgpt-docs
+          args: set image deployment/fastgpt-docs fastgpt-docs=registry.cn-hangzhou.aliyuncs.com/${{ secrets.ALI_HUB_USERNAME }}/fastgpt-docs:${{ needs.build-fastgpt-docs-images.outputs.tags }}
+      - uses: actions-hub/kubectl@master
+        env:
+          KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
+        with:
+          args: annotate deployment/fastgpt-docs originImageName="registry.cn-hangzhou.aliyuncs.com/${{ secrets.ALI_HUB_USERNAME }}/fastgpt-docs:${{ needs.build-fastgpt-docs-images.outputs.tags }}" --overwrite

.github/workflows/docs-preview.yml

@@ -1,4 +1,4 @@
-name: deploy-docs-preview
+name: preview-docs
 
 on:
   pull_request_target:
@@ -74,7 +74,7 @@ jobs:
           alias-domains: | #Optional
             fastgpt-staging.vercel.app
   docsOutput:
-    needs: [ deploy-preview ]
+    needs: [deploy-preview]
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
@@ -92,7 +92,7 @@ jobs:
         with:
           version: v0.0.6
         env:
-          GH_TOKEN: "${{ secrets.GH_PAT }}"
+          GH_TOKEN: '${{ secrets.GH_PAT }}'
-          SEALOS_TYPE: "pr_comment"
+          SEALOS_TYPE: 'pr_comment'
-          SEALOS_FILENAME: "report.md"
+          SEALOS_FILENAME: 'report.md'
-          SEALOS_REPLACE_TAG: "DEFAULT_REPLACE_DEPLOY"
+          SEALOS_REPLACE_TAG: 'DEFAULT_REPLACE_DEPLOY'

.github/workflows/helm-release.yaml

@@ -0,0 +1,34 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - 'v*.*.*'
+  workflow_dispatch:
+
+jobs:
+  helm:
+    runs-on: ubuntu-20.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-tags: true
+          fetch-depth: 0
+      - name: Set output
+        id: vars
+        run: echo "tag=$(git describe --tags)" >> $GITHUB_OUTPUT
+      - name: Release Helm
+        run: |
+          echo ${{ secrets.GH_PAT }} | helm registry login ghcr.io -u ${{ github.repository_owner }} --password-stdin
+          export APP_VERSION=${{ steps.vars.outputs.tag }}
+          export HELM_VERSION=${{ steps.vars.outputs.tag }}
+          export HELM_REPO=ghcr.io/${{ github.repository_owner }}
+          if [[ ! "$line" =~ ^v ]]
+          then
+            unset APP_VERSION
+            unset HELM_VERSION
+          fi
+          helm dependency update files/helm/fastgpt
+          helm package files/helm/fastgpt --version ${HELM_VERSION}-helm --app-version ${APP_VERSION} -d bin
+          helm push bin/fastgpt-${HELM_VERSION}-helm.tgz oci://${HELM_REPO}

.github/workflows/preview-image.yml

@@ -54,3 +54,14 @@ jobs:
           -t ${DOCKER_REPO_TAGGED} \
           -f Dockerfile \
           .
+
+  helm-check:
+    runs-on: ubuntu-20.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Helm Check
+        run: |
+          helm dependency update files/helm/fastgpt
+          helm lint files/helm/fastgpt
+          helm package files/helm/fastgpt

.gitignore

@@ -36,4 +36,11 @@ dist/
 docSite/public/
 docSite/resources/_gen/
 docSite/.vercel
-*.local.*
+*.local.*
+
+
+.idea/
+files/helm/fastgpt/fastgpt-0.1.0.tgz
+files/helm/fastgpt/charts/*.tgz
+
+tmp/

.vscode/extensions.json

@@ -0,0 +1,5 @@
+{
+  "recommendations": [
+    "inlang.vs-code-extension"
+  ]
+}

.vscode/settings.json

@@ -2,7 +2,7 @@
     "editor.formatOnSave": true,
     "editor.mouseWheelZoom": true,
     "typescript.tsdk": "node_modules/typescript/lib",
-    "prettier.prettierPath": "./node_modules/prettier",
+    "prettier.prettierPath": "",
     "i18n-ally.localesPaths": [
         "projects/app/public/locales",
     ],
@@ -11,5 +11,5 @@
     "i18n-ally.sortKeys": true,
     "i18n-ally.keepFulfilled": true,
     "i18n-ally.sourceLanguage": "zh", // 根据此语言文件翻译其他语言文件的变量和内容
-    "i18n-ally.displayLanguage": "en", // 显示语言
+    "i18n-ally.displayLanguage": "zh", // 显示语言
 }

Dockerfile

@@ -8,7 +8,7 @@ ARG proxy
 RUN [ -z "$proxy" ] || sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
 RUN apk add --no-cache libc6-compat && npm install -g pnpm@8.6.0
 # if proxy exists, set proxy
-RUN [ -z "$proxy" ] || pnpm config set registry https://registry.npm.taobao.org
+RUN [ -z "$proxy" ] || pnpm config set registry https://registry.npmmirror.com
 
 # copy packages and one project
 COPY pnpm-lock.yaml pnpm-workspace.yaml ./
@@ -28,7 +28,7 @@ ARG proxy
 RUN [ -z "$proxy" ] || sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
 RUN apk add --no-cache libc6-compat && npm install -g pnpm@8.6.0
 # if proxy exists, set proxy
-RUN [ -z "$proxy" ] || pnpm config set registry https://registry.npm.taobao.org
+RUN [ -z "$proxy" ] || pnpm config set registry https://registry.npmmirror.com
 
 COPY ./worker /app/worker
 RUN cd /app/worker && pnpm i --production --ignore-workspace
@@ -78,7 +78,7 @@ COPY --from=builder /app/projects/$name/package.json ./package.json
 COPY --from=workerDeps /app/worker /app/worker
 # copy config
 COPY ./projects/$name/data /app/data
-
+RUN chown -R nextjs:nodejs /app/data
 
 ENV NODE_ENV production
 ENV NEXT_TELEMETRY_DISABLED 1

README.md

@@ -36,8 +36,9 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
 
 ## 🛸 在线使用
 
-- 🌐 国内临时可访问：[fastgpt.in](https://fastgpt.in/)
+- 🌍 海外版：[fastgpt.in](https://fastgpt.in/)
-- 🌍 海外版：[fastgpt.run](https://fastgpt.run/)
+
+fastgpt.run 域名会弃用。
 
 |                                    |                                    |
 | ---------------------------------- | ---------------------------------- |
@@ -57,9 +58,9 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
    - [x] 源文件引用追踪
    - [x] 模块封装，实现多级复用
    - [x] 混合检索 & 重排
-   - [ ] 自查询规划
+   - [x] Tool 模块
    - [ ] 嵌入 [Laf](https://github.com/labring/laf)，实现在线编写 HTTP 模块
-   - [ ] 插件封装功能
+   - [ ] 插件封装功能，支持低代码渲染
 
 `2` 知识库能力
    - [x] 多库复用，混用
@@ -67,10 +68,9 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
    - [x] 支持知识库单独设置向量模型
    - [x] 源文件存储
    - [x] 支持手动输入，直接分段，QA 拆分导入
-   - [x] 支持 pdf、word、txt、md 等常用文件，支持 url 读取、CSV 批量导入
+   - [x] 支持。txt， 。md， 。html， 。pdf， 。docx，pptx， 。csv， 。xlsx (有需要更多可 PR file loader)
-   - [ ] 支持 HTML、csv、PPT、Excel 导入
+   - [x] 支持 url 读取、CSV 批量导入
    - [ ] 支持文件阅读器
-   - [ ] 支持差异性文件同步
    - [ ] 更多的数据预处理方案
   
 `3` 应用调试能力
@@ -81,8 +81,8 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
    - [ ] 高级编排 DeBug 模式
   
 `4` OpenAPI 接口
-   - [x] completions 接口 (对齐 GPT 接口)
+   - [x] completions 接口 (chat 模式对齐 GPT 接口)
-   - [ ] 知识库 CRUD
+   - [x] 知识库 CRUD
    - [ ] 对话 CRUD
   
 `5` 运营能力
@@ -113,7 +113,7 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
 * [多模型配置](https://doc.fastgpt.in/docs/development/one-api/)
 * [版本更新/升级介绍](https://doc.fastgpt.in/docs/development/upgrading)
 * [OpenAPI API 文档](https://doc.fastgpt.in/docs/development/openapi/)
-* [知识库结构详解](https://doc.fastgpt.in/docs/use-cases/datasetengine/)
+* [知识库结构详解](https://doc.fastgpt.in/docs/course/dataset_engine/)
 
 <a href="#readme">
     <img src="https://img.shields.io/badge/-返回顶部-7d09f1.svg" alt="#" align="right">
@@ -121,9 +121,9 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
 
 ## 🏘️ 社区交流群
 
-添加 wx 小助手加入：
+wx 扫一下加入：
 
-![](https://otnvvf-imgs.oss.laf.run/wx300.jpg)
+![](https://oss.laf.run/cofxat-test/fastgpt-qr-code2.jpg)
 
 <a href="#readme">
     <img src="https://img.shields.io/badge/-返回顶部-7d09f1.svg" alt="#" align="right">
@@ -215,4 +215,4 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
 1. 允许作为后台服务直接商用，但不允许提供 SaaS 服务。
 2. 未经商业授权，任何形式的商用服务均需保留相关版权信息。
 3. 完整请查看 [FastGPT Open Source License](./LICENSE)
-4. 联系方式：yujinlong@sealos.io，[点击查看商业版定价策略](https://doc.fastgpt.in/docs/commercial)
+4. 联系方式：yujinlong@sealos.io，[点击查看商业版定价策略](https://doc.fastgpt.in/docs/commercial)

README_en.md

@@ -49,48 +49,52 @@ Cloud: [fastgpt.in](https://fastgpt.in/)
 
 ## 💡 Features
 
-1. Powerful visual workflows: Effortlessly craft AI applications
+`1` Application Orchestration Features
 
-   - [x] Simple mode on deck - no need for manual arrangement
+   - [x] Offers a straightforward mode, eliminating the need for complex orchestration
-   - [x] User dialogue pre-guidance
+   - [x] Provides clear next-step instructions in dialogues
-   - [x] Global variables
+   - [x] Facilitates workflow orchestration
-   - [x] Knowledge base search
+   - [x] Tracks references in source files
-   - [x] Dialogue via multiple LLM models
+   - [x] Encapsulates modules for enhanced reuse at multiple levels
-   - [x] Text magic - convert to structured data
+   - [x] Combines search and reordering functions
-   - [x] Extend with HTTP
+   - [ ] Includes a tool module
-   - [ ] Embed Laf for on-the-fly HTTP module crafting
+   - [ ] Integrates [Laf](https://github.com/labring/laf) for online HTTP module creation
-   - [x] Directions for the next dialogue steps
+   - [ ] Plugin encapsulation capabilities
-   - [x] Tracking source file references
-   - [ ] Custom file reader
-   - [ ] Modules are packaged into plug-ins to achieve reuse
 
-2. Extensive knowledge base preprocessing
+`2` Knowledge Base Features
 
-   - [x] Reuse and mix multiple knowledge bases
+   - [x] Allows for the mixed use of multiple databases
-   - [x] Track chunk modifications and deletions
+   - [x] Keeps track of modifications and deletions in data chunks
-   - [x] Supports manual entries, direct segmentation, and QA split imports
+   - [x] Enables specific vector models for each knowledge base
-   - [x] Supports URL fetching and batch CSV imports
+   - [x] Stores original source files
-   - [x] Supports Set unique vector models for knowledge bases
+   - [x] Supports direct input and segment-based QA import
-   - [x] Store original files
+   - [x] Compatible with a variety of file formats: pdf, docx, txt, html, md, csv
-   - [ ] File learning Agent
+   - [x] Facilitates URL reading and bulk CSV importing
+   - [ ] Supports PPT and Excel file import
+   - [ ] Features a file reader
+   - [ ] Offers diverse data preprocessing options
 
-3. Multiple effect testing channels
+`3` Application Debugging Features
 
-   - [x] Single-point knowledge base search test
+   - [x] Enables targeted search testing within the knowledge base
-   - [x] Feedback references and ability to modify and delete during dialogue
+   - [x] Allows feedback, editing, and deletion during conversations
-   - [x] Complete context presentation
+   - [x] Presents the full context of interactions
-   - [ ] Complete module intermediate value presentation
+   - [x] Displays all intermediate values within modules
+   - [ ] Advanced DeBug mode for orchestration
 
-4. OpenAPI
+`4` OpenAPI Interface
 
-   - [x] completions interface (aligned with GPT interface)
+   - [x] The completions interface (aligned with GPT's chat mode interface)
-   - [ ] Knowledge base CRUD
+   - [x] CRUD operations for the knowledge base
+   - [ ] CRUD operations for conversations
 
-5. Operational functions
+`5` Operational Features
+
+   - [x] Share without requiring login
+   - [x] Easy embedding with Iframe
+   - [x] Customizable chat window embedding with features like default open, drag-and-drop
+   - [x] Centralizes conversation records for review and annotation
 
-   - [x] Login-free sharing window
-   - [x] One-click embedding with Iframe
-   - [ ] Unified access to dialogue records
 
 <a href="#readme">
     <img src="https://img.shields.io/badge/-Back_to_Top-7d09f1.svg" alt="#" align="right">
@@ -112,14 +116,12 @@ Project tech stack: NextJs + TS + ChakraUI + Mongo + Postgres (Vector plugin)
 - [Configuring Multiple Models](https://doc.fastgpt.in/docs/installation/reference/models)
 - [Version Updates & Upgrades](https://doc.fastgpt.in/docs/installation/upgrading)
 
-<!-- ## :point_right: RoadMap
-- [FastGPT RoadMap](https://kjqvjse66l.feishu.cn/docx/RVUxdqE2WolDYyxEKATcM0XXnte) -->
 
-<!-- ## 🏘️ Community
+## 🏘️ Community
 
-| Community Group                                   | Assistant                                      |
+| Community Group                                   | 
-| ------------------------------------------------- | ---------------------------------------------- |
+| ------------------------------------------------- | 
-| ![](https://otnvvf-imgs.oss.laf.run/wxqun300.jpg) | ![](https://otnvvf-imgs.oss.laf.run/wx300.jpg) | -->
+| ![](https://oss.laf.run/htr4n1-images/fastgpt-qr-code.jpg) | 
 
 <a href="#readme">
     <img src="https://img.shields.io/badge/-Back_to_Top-7d09f1.svg" alt="#" align="right">

dev.md

@@ -0,0 +1,17 @@
+# 打包命令
+
+```sh
+# Build image, not proxy
+docker build -t registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.4.7 --build-arg name=app .
+
+# build image with proxy
+docker build -t registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt:v4.4.7 --build-arg name=app --build-arg proxy=taobao .
+```
+
+# Pg 常用索引
+
+```sql
+CREATE INDEX IF NOT EXISTS modelData_dataset_id_index ON modeldata (dataset_id);
+CREATE INDEX IF NOT EXISTS modelData_collection_id_index ON modeldata (collection_id);
+CREATE INDEX IF NOT EXISTS modelData_teamId_index ON modeldata (team_id);
+```