Compare commits
7 Commits
v4.8.18-al
...
v4.8.18
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
80c8897e10 | ||
|
|
e933dacb05 | ||
|
|
cdf4f0e67d | ||
|
|
4dfeb21da3 | ||
|
|
d0d1a2cae8 | ||
|
|
f1f0ae2839 | ||
|
|
10d8c56e23 |
21
.github/workflows/preview-fastgpt-image.yml
vendored
@@ -36,7 +36,7 @@ jobs:
|
||||
password: ${{ secrets.GH_PAT }}
|
||||
- name: Set DOCKER_REPO_TAGGED based on branch or tag
|
||||
run: |
|
||||
echo "DOCKER_REPO_TAGGED=ghcr.io/${{ github.repository_owner }}/fastgpt-pr:${{ github.event.pull_request.number }}" >> $GITHUB_ENV
|
||||
echo "DOCKER_REPO_TAGGED=ghcr.io/${{ github.repository_owner }}/fastgpt-pr:${{ github.event.pull_request.head.sha }}" >> $GITHUB_ENV
|
||||
- name: Build image for PR
|
||||
env:
|
||||
DOCKER_REPO_TAGGED: ${{ env.DOCKER_REPO_TAGGED }}
|
||||
@@ -44,13 +44,30 @@ jobs:
|
||||
docker buildx build \
|
||||
-f projects/app/Dockerfile \
|
||||
--label "org.opencontainers.image.source=https://github.com/${{ github.repository_owner }}/FastGPT" \
|
||||
--label "org.opencontainers.image.description=fastgpt-pr imae" \
|
||||
--label "org.opencontainers.image.description=fastgpt-pr image" \
|
||||
--label "org.opencontainers.image.licenses=Apache" \
|
||||
--push \
|
||||
--cache-from=type=local,src=/tmp/.buildx-cache \
|
||||
--cache-to=type=local,dest=/tmp/.buildx-cache \
|
||||
-t ${DOCKER_REPO_TAGGED} \
|
||||
.
|
||||
# Add write md step after build
|
||||
- name: Write md
|
||||
run: |
|
||||
echo "# 🤖 Generated by deploy action" > report.md
|
||||
echo "📦 Preview Image: \`${DOCKER_REPO_TAGGED}\`" >> report.md
|
||||
cat report.md
|
||||
|
||||
- name: Gh Rebot for Sealos
|
||||
uses: labring/gh-rebot@v0.0.6
|
||||
if: ${{ (github.event_name == 'pull_request_target') }}
|
||||
with:
|
||||
version: v0.0.6
|
||||
env:
|
||||
GH_TOKEN: '${{ secrets.GH_PAT }}'
|
||||
SEALOS_TYPE: 'pr_comment'
|
||||
SEALOS_FILENAME: 'report.md'
|
||||
SEALOS_REPLACE_TAG: 'DEFAULT_REPLACE_DEPLOY'
|
||||
|
||||
helm-check:
|
||||
runs-on: ubuntu-20.04
|
||||
|
||||
2
dev.md
@@ -1,6 +1,6 @@
|
||||
## Premise
|
||||
|
||||
Since FastGPT is managed in the same way as monorepo, it is recommended to install 'make' first during development.
|
||||
Since FastGPT is managed in the same way as monorepo, it is recommended to install ‘make’ first during development.
|
||||
|
||||
monorepo Project Name:
|
||||
|
||||
|
||||
BIN
docSite/assets/imgs/dataset1.png
Normal file
|
After Width: | Height: | Size: 81 KiB |
BIN
docSite/assets/imgs/dataset2.png
Normal file
|
After Width: | Height: | Size: 129 KiB |
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 68 KiB |
@@ -7,7 +7,7 @@ toc: true
|
||||
weight: 1210
|
||||
---
|
||||
|
||||
FastGPT 项目在 Apache License 2.0 许可下开源,同时包含以下附加条件:
|
||||
FastGPT 项目在 Apache License 2.0 许可下开源,但包含以下附加条件:
|
||||
|
||||
+ FastGPT 允许被用于商业化,例如作为其他应用的“后端即服务”使用,或者作为应用开发平台提供给企业。然而,当满足以下条件时,必须联系作者获得商业许可:
|
||||
|
||||
|
||||
@@ -9,9 +9,9 @@ weight: 707
|
||||
|
||||
由于环境变量不利于配置复杂的内容,新版 FastGPT 采用了 ConfigMap 的形式挂载配置文件,你可以在 `projects/app/data/config.json` 看到默认的配置文件。可以参考 [docker-compose 快速部署](/docs/development/docker/) 来挂载配置文件。
|
||||
|
||||
**开发环境下**,你需要将示例配置文件 `config.json` 复制成 `config.local.json` 文件才会生效。
|
||||
**开发环境下**,你需要将示例配置文件 `config.json` 复制成 `config.local.json` 文件才会生效。
|
||||
|
||||
这个配置文件中包含了系统参数和各个模型配置:
|
||||
下面配置文件示例中包含了系统参数和各个模型配置:
|
||||
|
||||
## 4.6.8+ 版本新配置文件示例
|
||||
|
||||
|
||||
@@ -19,6 +19,10 @@ images: []
|
||||
|
||||
## 二、通用问题
|
||||
|
||||
### 本地部署的限制
|
||||
|
||||
具体内容参考https://fael3z0zfze.feishu.cn/wiki/OFpAw8XzAi36Guk8dfucrCKUnjg。
|
||||
|
||||
### 能否纯本地运行
|
||||
|
||||
可以。需要准备好向量模型和LLM模型。
|
||||
@@ -41,31 +45,6 @@ images: []
|
||||
1. 问题补全需要经过一轮AI生成。
|
||||
2. 会进行3~5轮的查询,如果数据库性能不足,会有明显影响。
|
||||
|
||||
### 对话接口报错或返回为空(core.chat.Chat API is error or undefined)
|
||||
|
||||
1. 检查 AI 的 key 问题:通过 curl 请求看是否正常。务必用 stream=true 模式。并且 maxToken 等相关参数尽量一致。
|
||||
2. 如果是国内模型,可能是命中风控了。
|
||||
3. 查看模型请求日志,检查出入参数是否异常。
|
||||
|
||||
```sh
|
||||
# curl 例子。
|
||||
curl --location --request POST 'https://xxx.cn/v1/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-xxxx' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"stream": true,
|
||||
"temperature": 1,
|
||||
"max_tokens": 3000,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是谁"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### 页面中可以正常回复,API 报错
|
||||
|
||||
页面中是用 stream=true 模式,所以API也需要设置 stream=true 来进行测试。部分模型接口(国产居多)非 Stream 的兼容有点垃圾。
|
||||
@@ -111,6 +90,13 @@ FastGPT 模型配置文件中的 model 必须与 OneAPI 渠道中的模型对应
|
||||
|
||||
如果OneAPI中,没有配置对应的模型,`config.json`中也不要配置,否则容易报错。
|
||||
|
||||
### 点击模型测试失败
|
||||
|
||||
OneAPI 只会测试渠道的第一个模型,并且只会测试对话模型,向量模型无法自动测试,需要手动发起请求进行测试。[查看测试模型命令示例](/docs/development/faq/#如何检查模型问题)
|
||||
### get request url failed: Post "https://xxx dial tcp: xxxx
|
||||
|
||||
OneAPI 与模型网络不通,需要检查网络配置。
|
||||
|
||||
### Incorrect API key provided: sk-xxxx.You can find your api Key at xxx
|
||||
|
||||
OneAPI 的 API Key 配置错误,需要修改`OPENAI_API_KEY`环境变量,并重启容器(先 docker-compose down 然后再 docker-compose up -d 运行一次)。
|
||||
@@ -126,6 +112,112 @@ OneAPI 的 API Key 配置错误,需要修改`OPENAI_API_KEY`环境变量,并
|
||||
|
||||
## 四、常见模型问题
|
||||
|
||||
### 如何检查模型问题
|
||||
|
||||
1. 私有部署模型,先确认部署的模型是否正常。
|
||||
2. 通过 CURL 请求,直接测试上游模型是否正常运行(云端模型或私有模型均进行测试)
|
||||
3. 通过 CURL 请求,请求 OneAPI 去测试模型是否正常。
|
||||
4. 在 FastGPT 中使用该模型进行测试。
|
||||
|
||||
下面是几个测试 CURL 示例:
|
||||
|
||||
{{< tabs tabTotal="5" >}}
|
||||
{{< tab tabName="LLM模型" >}}
|
||||
{{< markdownify >}}
|
||||
|
||||
```bash
|
||||
curl https://api.openai.com/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-d '{
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Hello!"
|
||||
}
|
||||
]
|
||||
}'
|
||||
|
||||
```
|
||||
|
||||
{{< /markdownify >}}
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab tabName="Embedding模型" >}}
|
||||
{{< markdownify >}}
|
||||
|
||||
```bash
|
||||
curl https://api.openai.com/v1/embeddings \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"input": "The food was delicious and the waiter...",
|
||||
"model": "text-embedding-ada-002",
|
||||
"encoding_format": "float"
|
||||
}'
|
||||
```
|
||||
|
||||
{{< /markdownify >}}
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab tabName="Rerank 模型" >}}
|
||||
{{< markdownify >}}
|
||||
|
||||
```bash
|
||||
curl --location --request POST 'https://xxxx.com/api/v1/rerank' \
|
||||
--header 'Authorization: Bearer {{ACCESS_TOKEN}}' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
"model": "bge-rerank-m3",
|
||||
"query": "导演是谁",
|
||||
"documents": [
|
||||
"你是谁?\n我是电影《铃芽之旅》助手"
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
{{< /markdownify >}}
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab tabName="TTS 模型" >}}
|
||||
{{< markdownify >}}
|
||||
|
||||
```bash
|
||||
curl https://api.openai.com/v1/audio/speech \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "tts-1",
|
||||
"input": "The quick brown fox jumped over the lazy dog.",
|
||||
"voice": "alloy"
|
||||
}' \
|
||||
--output speech.mp3
|
||||
```
|
||||
|
||||
{{< /markdownify >}}
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab tabName="Whisper 模型" >}}
|
||||
{{< markdownify >}}
|
||||
|
||||
```bash
|
||||
curl https://api.openai.com/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H "Content-Type: multipart/form-data" \
|
||||
-F file="@/path/to/file/audio.mp3" \
|
||||
-F model="whisper-1"
|
||||
```
|
||||
|
||||
{{< /markdownify >}}
|
||||
{{< /tab >}}
|
||||
|
||||
{{< /tabs >}}
|
||||
|
||||
### 报错 - 模型响应为空/模型报错
|
||||
|
||||
该错误是由于 stream 模式下,oneapi 直接结束了流请求,并且未返回任何内容导致。
|
||||
@@ -165,7 +257,7 @@ curl --location --request POST 'https://api.openai.com/v1/chat/completions' \
|
||||
|
||||
需要模型提供商和 oneapi 同时支持工具调用才可使用,测试方法如下:
|
||||
|
||||
1. 通过 `curl` 向 `oneapi` 发起第一轮 stream 模式的 tool 测试。
|
||||
##### 1. 通过 `curl` 向 `oneapi` 发起第一轮 stream 模式的 tool 测试。
|
||||
|
||||
```bash
|
||||
curl --location --request POST 'https://oneapi.xxx/v1/chat/completions' \
|
||||
@@ -200,7 +292,7 @@ curl --location --request POST 'https://oneapi.xxx/v1/chat/completions' \
|
||||
}'
|
||||
```
|
||||
|
||||
2. 检查响应参数
|
||||
##### 2. 检查响应参数
|
||||
|
||||
如果能正常调用工具,会返回对应 `tool_calls` 参数。
|
||||
|
||||
@@ -238,7 +330,7 @@ curl --location --request POST 'https://oneapi.xxx/v1/chat/completions' \
|
||||
}
|
||||
```
|
||||
|
||||
3. 通过 `curl` 向 `oneapi` 发起第二轮 stream 模式的 tool 测试。
|
||||
##### 3. 通过 `curl` 向 `oneapi` 发起第二轮 stream 模式的 tool 测试。
|
||||
|
||||
第二轮请求是把工具结果发送给模型。发起后会得到模型回答的结果。
|
||||
|
||||
|
||||
41
docSite/content/zh-cn/docs/development/upgrading/4818.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: 'V4.8.18(进行中)'
|
||||
description: 'FastGPT V4.8.18 更新说明'
|
||||
icon: 'upgrade'
|
||||
draft: false
|
||||
toc: true
|
||||
weight: 806
|
||||
---
|
||||
|
||||
## 更新指南
|
||||
|
||||
### 2. 运行升级脚本
|
||||
|
||||
从任意终端,发起 1 个 HTTP 请求。其中 {{rootkey}} 替换成环境变量里的 `rootkey`;{{host}} 替换成**FastGPT 域名**。
|
||||
|
||||
```bash
|
||||
curl --location --request POST 'https://{{host}}/api/admin/initv4818' \
|
||||
--header 'rootkey: {{rootkey}}' \
|
||||
--header 'Content-Type: application/json'
|
||||
```
|
||||
|
||||
会迁移全文检索表,时间较长,迁移期间全文检索会失效,日志中会打印已经迁移的数据长度。
|
||||
|
||||
|
||||
## 完整更新内容
|
||||
|
||||
1. 新增 - 支持通过 JSON 配置直接创建应用。
|
||||
2. 新增 - 支持通过 CURL 脚本快速创建 HTTP 插件。
|
||||
3. 新增 - 商业版支持部门架构权限模式。
|
||||
4. 新增 - 支持配置自定跨域安全策略,默认全开。
|
||||
5. 新增 - 补充私有部署,模型问题排查文档。
|
||||
6. 优化 - HTTP Body 增加特殊处理,解决字符串变量带换行时无法解析问题。
|
||||
7. 优化 - 分享链接随机生成用户头像。
|
||||
8. 优化 - 图片上传安全校验。并增加头像图片唯一存储,确保不会累计存储。
|
||||
9. 优化 - Mongo 全文索引表分离。
|
||||
10. 优化 - 知识库检索查询语句合并,同时减少查库数量。
|
||||
11. 优化 - 文件编码检测,减少 CSV 文件乱码概率。
|
||||
12. 优化 - 异步读取文件内容,减少进程阻塞。
|
||||
13. 优化 - 文件阅读,HTML 直接下载,不允许在线阅读。
|
||||
14. 修复 - HTML 文件上传,base64 图片无法自动转图片链接。
|
||||
15. 修复 - 插件计费错误。
|
||||
@@ -13,4 +13,12 @@ weight: 908
|
||||
|
||||
但是,当连续问题之间的关联性较小,模型判断的准确度可能会受到限制。在这种情况下,我们可以引入全局变量的概念来记录分类结果。在后续的问题分类阶段,首先检查全局变量是否存有分类结果。如果有,那么直接沿用该结果;若没有,则让模型自行判断。
|
||||
|
||||
建议:构建批量运行脚本进行测试,评估问题分类的准确性。
|
||||
建议:构建批量运行脚本进行测试,评估问题分类的准确性。
|
||||
|
||||
## 系统编排配置中的定时执行,如果用户打开分享的连接,停留在那个页面,定时执行触发问题
|
||||
|
||||
发布后,后台生效。
|
||||
|
||||
## AI对话回答要求中的Markdown语法取消
|
||||
|
||||
在针对知识库的回答要求里有, 要给它配置提示词,不然他就是默认的,默认的里面就有该语法。
|
||||
@@ -14,4 +14,51 @@ weight: 910
|
||||
## 知识库配置里的文件处理模型是什么?与索引模型有什么区别?
|
||||
|
||||
* **文件处理模型**:用于数据处理的【增强处理】和【问答拆分】。在【增强处理】中,生成相关问题和摘要,在【问答拆分】中执行问答对生成。
|
||||
* **索引模型**:用于向量化,即通过对文本数据进行处理和组织,构建出一个能够快速查询的数据结构。
|
||||
* **索引模型**:用于向量化,即通过对文本数据进行处理和组织,构建出一个能够快速查询的数据结构。
|
||||
|
||||
## 基于知识库的查询,但是问题相关的答案过多。ai回答到一半就不继续回答。
|
||||
|
||||
FastGPT回复长度计算公式:
|
||||
|
||||
最大回复=min(配置的最大回复(内置的限制),最大上下文(输入和输出的总和)-历史记录)
|
||||
|
||||
18K模型->输入与输出的和
|
||||
|
||||
输出增多->输入减小
|
||||
|
||||
所以可以:
|
||||
|
||||
1. 检查配置的最大回复(回复上限)
|
||||
2. 减小输入来增大输出,即减小历史记录,在工作流其实也就是“聊天记录”
|
||||
|
||||
配置的最大回复:
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
1. 私有化部署的时候,后台配模型参数,可以在配置最大上文时候,预留一些空间,比如 128000 的模型,可以只配置 120000, 剩余的空间后续会被安排给输出
|
||||
|
||||
|
||||
## 受到模型上下文的限制,有时候达不到聊天记录的轮次,连续对话字数过多就会报上下文不够的错误。
|
||||
|
||||
FastGPT回复长度计算公式:
|
||||
|
||||
最大回复=min(配置的最大回复(内置的限制),最大上下文(输入和输出的总和)-历史记录)
|
||||
|
||||
18K模型->输入与输出的和
|
||||
|
||||
输出增多->输入减小
|
||||
|
||||
所以可以:
|
||||
|
||||
1. 检查配置的最大回复(回复上限)
|
||||
2. 减小输入来增大输出,即减小历史记录,在工作流其实也就是“聊天记录”
|
||||
|
||||
配置的最大回复:
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
1. 私有化部署的时候,后台配模型参数,可以在配置最大上文时候,预留一些空间,比如 128000 的模型,可以只配置 120000, 剩余的空间后续会被安排给输出
|
||||
@@ -160,6 +160,18 @@ default_doi_resolver: 'oadoi.org'
|
||||
}
|
||||
```
|
||||
|
||||
* 搜索结果为空时会返回友好提示:
|
||||
|
||||
```Bash
|
||||
{
|
||||
"result": "[]",
|
||||
"error": {
|
||||
"message": "No search results",
|
||||
"code": 500
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* 失败时通过 Promise.reject 可能返回错误信息:
|
||||
|
||||
```Bash
|
||||
|
||||
@@ -54,7 +54,7 @@ FastGPT 对外的 API 接口对齐了 OpenAI 官方接口,可以直接接入
|
||||
|
||||
1. **项目开源**
|
||||
|
||||
FastGPT 遵循附加条件 Apache License 2.0 开源协议,你可以 [Fork](https://github.com/labring/FastGPT/fork) 之后进行二次开发和发布。FastGPT 社区版将保留核心功能,商业版仅在社区版基础上使用 API 的形式进行扩展,不影响学习使用。
|
||||
FastGPT 遵循**附加条件 Apache License 2.0 开源协议**,你可以 [Fork](https://github.com/labring/FastGPT/fork) 之后进行二次开发和发布。FastGPT 社区版将保留核心功能,商业版仅在社区版基础上使用 API 的形式进行扩展,不影响学习使用。
|
||||
|
||||
2. **独特的 QA 结构**
|
||||
|
||||
|
||||
@@ -53,8 +53,9 @@ FastGPT 商业版软件根据不同的部署方式,分为 3 类收费模式。
|
||||
{{< table "table-hover table-striped-columns" >}}
|
||||
| 部署方式 | 特有服务 | 上线时长 | 标品价格 |
|
||||
| ---- | ---- | ---- | ---- |
|
||||
| Sealos全托管 | 1. 有效期内免费升级。<br>2. 免运维服务&数据库。 | 半天 | 6000元起/月(3个月起)<br>或<br>60000元起/年 |
|
||||
| 自有服务器部署 | 1. 6个版本免费升级支持。 | 14天内 | 具体价格可[联系咨询](https://fael3z0zfze.feishu.cn/share/base/form/shrcnRxj3utrzjywsom96Px4sud) |
|
||||
| Sealos全托管 | 1. 有效期内免费升级。<br>2. 免运维服务&数据库。 | 半天 | 10000元起/月(3个月起)<br>或<br>120000元起/年<br>8C32G 资源,额外资源另外收费。 |
|
||||
| Sealos全托管(多节点) | 1. 有效期内免费升级。<br>2. 免运维服务&数据库。 | 半天 | 22000元起/月(3个月起)<br>或<br>264000元起/年<br>32C128G 资源,额外资源另外收费。 |
|
||||
| 自有服务器部署 | 1. 6个版本免费升级支持。 | 14天内 | 具体价格和优惠可[联系咨询](https://fael3z0zfze.feishu.cn/share/base/form/shrcnRxj3utrzjywsom96Px4sud) |
|
||||
{{< /table >}}
|
||||
|
||||
{{% alert icon="🤖 " context="success" %}}
|
||||
|
||||
@@ -5,8 +5,9 @@ icon: 'currency_yen'
|
||||
draft: false
|
||||
toc: true
|
||||
weight: 1102
|
||||
type: redirect
|
||||
target: https://cloud.tryfastgpt.ai/price
|
||||
---
|
||||
|
||||
线上版价格请查看:[https://cloud.tryfastgpt.ai/price](https://cloud.tryfastgpt.ai/price)
|
||||
线上版价格按套餐订阅模式,具体价格和计费请查看(请正确选择版本,账号不互通):
|
||||
|
||||
- [海外版](https://cloud.tryfastgpt.ai/price)
|
||||
- [国内版](https://cloud.fastgpt.cn/price)
|
||||
@@ -32,11 +32,11 @@ weight: 602
|
||||
|
||||
1. ### 创建应用模板
|
||||
|
||||
应用模板配置以及相关资源,都会在 **projects/app/public/appMarketTemplates** 目录下。
|
||||
应用模板配置以及相关资源,都会在 **packages/templates/src** 目录下。
|
||||
|
||||

|
||||
|
||||
1. 在 **projects/app/public/appMarketTemplates** 目录下,创建一个文件夹,名称为模板对应的 id。
|
||||
1. 在**packages/templates/src** 目录下,创建一个文件夹,名称为模板对应的 id。
|
||||
2. 在刚刚创建的文件夹中,再创建一个 **template.json** 文件,复制粘贴并填写如下配置:
|
||||
|
||||
```JSON
|
||||
@@ -83,4 +83,4 @@ weight: 602
|
||||
|
||||
- 写清楚模板的介绍和功能
|
||||
- 配上模板运行的效果图
|
||||
- 模板参数填写说明,需要在 PR 中写清楚。例如,有些模板需要去某个提供商申请 key,需要附上对应的地址和教程,后续我们会加入到文档中。
|
||||
- 模板参数填写说明,需要在 PR 中写清楚。例如,有些模板需要去某个提供商申请 key,需要附上对应的地址和教程,后续我们会加入到文档中。
|
||||
|
||||
@@ -159,6 +159,14 @@ services:
|
||||
# 日志等级: debug, info, warn, error
|
||||
- LOG_LEVEL=info
|
||||
- STORE_LOG_LEVEL=warn
|
||||
# 工作流最大运行次数
|
||||
- WORKFLOW_MAX_RUN_TIMES=1000
|
||||
# 批量执行节点,最大输入长度
|
||||
- WORKFLOW_MAX_LOOP_TIMES=100
|
||||
# 自定义跨域,不配置时,默认都允许跨域(多个域名通过逗号分割)
|
||||
- ALLOWED_ORIGINS=
|
||||
# 是否开启IP限制,默认不开启
|
||||
- USE_IP_LIMIT=false
|
||||
volumes:
|
||||
- ./config.json:/app/data/config.json
|
||||
|
||||
|
||||
@@ -116,6 +116,14 @@ services:
|
||||
# 日志等级: debug, info, warn, error
|
||||
- LOG_LEVEL=info
|
||||
- STORE_LOG_LEVEL=warn
|
||||
# 工作流最大运行次数
|
||||
- WORKFLOW_MAX_RUN_TIMES=1000
|
||||
# 批量执行节点,最大输入长度
|
||||
- WORKFLOW_MAX_LOOP_TIMES=100
|
||||
# 自定义跨域,不配置时,默认都允许跨域(多个域名通过逗号分割)
|
||||
- ALLOWED_ORIGINS=
|
||||
# 是否开启IP限制,默认不开启
|
||||
- USE_IP_LIMIT=false
|
||||
volumes:
|
||||
- ./config.json:/app/data/config.json
|
||||
|
||||
|
||||
@@ -97,6 +97,14 @@ services:
|
||||
# 日志等级: debug, info, warn, error
|
||||
- LOG_LEVEL=info
|
||||
- STORE_LOG_LEVEL=warn
|
||||
# 工作流最大运行次数
|
||||
- WORKFLOW_MAX_RUN_TIMES=1000
|
||||
# 批量执行节点,最大输入长度
|
||||
- WORKFLOW_MAX_LOOP_TIMES=100
|
||||
# 自定义跨域,不配置时,默认都允许跨域(多个域名通过逗号分割)
|
||||
- ALLOWED_ORIGINS=
|
||||
# 是否开启IP限制,默认不开启
|
||||
- USE_IP_LIMIT=false
|
||||
volumes:
|
||||
- ./config.json:/app/data/config.json
|
||||
|
||||
|
||||
@@ -1,14 +1,20 @@
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
import { ErrType } from '../errorCode';
|
||||
|
||||
/* dataset: 507000 */
|
||||
const startCode = 507000;
|
||||
export enum CommonErrEnum {
|
||||
invalidParams = 'invalidParams',
|
||||
fileNotFound = 'fileNotFound',
|
||||
unAuthFile = 'unAuthFile',
|
||||
missingParams = 'missingParams',
|
||||
inheritPermissionError = 'inheritPermissionError'
|
||||
}
|
||||
const datasetErr = [
|
||||
{
|
||||
statusText: CommonErrEnum.fileNotFound,
|
||||
message: i18nT('common:error.invalid_params')
|
||||
},
|
||||
{
|
||||
statusText: CommonErrEnum.fileNotFound,
|
||||
message: 'error.fileNotFound'
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { ErrType } from '../errorCode';
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
import type { ErrType } from '../errorCode';
|
||||
/* team: 500000 */
|
||||
export enum TeamErrEnum {
|
||||
notUser = 'notUser',
|
||||
teamOverSize = 'teamOverSize',
|
||||
unAuthTeam = 'unAuthTeam',
|
||||
teamMemberOverSize = 'teamMemberOverSize',
|
||||
aiPointsNotEnough = 'aiPointsNotEnough',
|
||||
datasetSizeNotEnough = 'datasetSizeNotEnough',
|
||||
datasetAmountNotEnough = 'datasetAmountNotEnough',
|
||||
@@ -14,11 +16,22 @@ export enum TeamErrEnum {
|
||||
groupNameEmpty = 'groupNameEmpty',
|
||||
groupNameDuplicate = 'groupNameDuplicate',
|
||||
groupNotExist = 'groupNotExist',
|
||||
orgMemberNotExist = 'orgMemberNotExist',
|
||||
orgMemberDuplicated = 'orgMemberDuplicated',
|
||||
orgNotExist = 'orgNotExist',
|
||||
orgParentNotExist = 'orgParentNotExist',
|
||||
cannotMoveToSubPath = 'cannotMoveToSubPath',
|
||||
cannotModifyRootOrg = 'cannotModifyRootOrg',
|
||||
cannotDeleteNonEmptyOrg = 'cannotDeleteNonEmptyOrg',
|
||||
cannotDeleteDefaultGroup = 'cannotDeleteDefaultGroup',
|
||||
userNotActive = 'userNotActive'
|
||||
}
|
||||
|
||||
const teamErr = [
|
||||
{
|
||||
statusText: TeamErrEnum.notUser,
|
||||
message: i18nT('common:code_error.team_error.not_user')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.teamOverSize,
|
||||
message: i18nT('common:code_error.team_error.over_size')
|
||||
@@ -71,6 +84,34 @@ const teamErr = [
|
||||
{
|
||||
statusText: TeamErrEnum.userNotActive,
|
||||
message: i18nT('common:code_error.team_error.user_not_active')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.orgMemberNotExist,
|
||||
message: i18nT('common:code_error.team_error.org_member_not_exist')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.orgMemberDuplicated,
|
||||
message: i18nT('common:code_error.team_error.org_member_duplicated')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.orgNotExist,
|
||||
message: i18nT('common:code_error.team_error.org_not_exist')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.orgParentNotExist,
|
||||
message: i18nT('common:code_error.team_error.org_parent_not_exist')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.cannotMoveToSubPath,
|
||||
message: i18nT('common:code_error.team_error.cannot_move_to_sub_path')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.cannotModifyRootOrg,
|
||||
message: i18nT('common:code_error.team_error.cannot_modify_root_org')
|
||||
},
|
||||
{
|
||||
statusText: TeamErrEnum.cannotDeleteNonEmptyOrg,
|
||||
message: i18nT('common:code_error.team_error.cannot_delete_non_empty_org')
|
||||
}
|
||||
];
|
||||
|
||||
|
||||
@@ -2,25 +2,16 @@ import { ErrType } from '../errorCode';
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
/* team: 503000 */
|
||||
export enum UserErrEnum {
|
||||
unAuthUser = 'unAuthUser',
|
||||
unAuthRole = 'unAuthRole',
|
||||
binVisitor = 'binVisitor',
|
||||
account_psw_error = 'account_psw_error',
|
||||
balanceNotEnough = 'balanceNotEnough',
|
||||
unAuthSso = 'unAuthSso'
|
||||
}
|
||||
const errList = [
|
||||
{
|
||||
statusText: UserErrEnum.unAuthUser,
|
||||
message: i18nT('common:code_error.user_error.un_auth_user')
|
||||
statusText: UserErrEnum.account_psw_error,
|
||||
message: i18nT('common:code_error.account_error')
|
||||
},
|
||||
{
|
||||
statusText: UserErrEnum.binVisitor,
|
||||
message: i18nT('common:code_error.user_error.bin_visitor')
|
||||
}, // 身份校验未通过
|
||||
{
|
||||
statusText: UserErrEnum.binVisitor,
|
||||
message: i18nT('common:code_error.user_error.bin_visitor_guest')
|
||||
}, // 游客身份
|
||||
{
|
||||
statusText: UserErrEnum.balanceNotEnough,
|
||||
message: i18nT('common:code_error.user_error.balance_not_enough')
|
||||
|
||||
@@ -5,6 +5,6 @@ export const getErrText = (err: any, def = ''): any => {
|
||||
typeof err === 'string'
|
||||
? err
|
||||
: err?.response?.data?.message || err?.response?.message || err?.message || def;
|
||||
msg && console.log('error =>', msg);
|
||||
// msg && console.log('error =>', msg);
|
||||
return replaceSensitiveText(msg);
|
||||
};
|
||||
|
||||
5
packages/global/common/file/api.d.ts
vendored
@@ -1,10 +1,7 @@
|
||||
import { MongoImageTypeEnum } from './image/constants';
|
||||
import { OutLinkChatAuthProps } from '../../support/permission/chat.d';
|
||||
|
||||
export type preUploadImgProps = OutLinkChatAuthProps & {
|
||||
type: `${MongoImageTypeEnum}`;
|
||||
|
||||
expiredTime?: Date;
|
||||
// expiredTime?: Date;
|
||||
metadata?: Record<string, any>;
|
||||
};
|
||||
export type UploadImgProps = preUploadImgProps & {
|
||||
|
||||
@@ -1,61 +1,5 @@
|
||||
export const imageBaseUrl = '/api/system/img/';
|
||||
|
||||
export enum MongoImageTypeEnum {
|
||||
systemAvatar = 'systemAvatar',
|
||||
appAvatar = 'appAvatar',
|
||||
pluginAvatar = 'pluginAvatar',
|
||||
datasetAvatar = 'datasetAvatar',
|
||||
userAvatar = 'userAvatar',
|
||||
teamAvatar = 'teamAvatar',
|
||||
groupAvatar = 'groupAvatar',
|
||||
|
||||
chatImage = 'chatImage',
|
||||
collectionImage = 'collectionImage'
|
||||
}
|
||||
export const mongoImageTypeMap = {
|
||||
[MongoImageTypeEnum.systemAvatar]: {
|
||||
label: 'appAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.appAvatar]: {
|
||||
label: 'appAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.pluginAvatar]: {
|
||||
label: 'pluginAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.datasetAvatar]: {
|
||||
label: 'datasetAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.userAvatar]: {
|
||||
label: 'userAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.teamAvatar]: {
|
||||
label: 'teamAvatar',
|
||||
unique: true
|
||||
},
|
||||
[MongoImageTypeEnum.groupAvatar]: {
|
||||
label: 'groupAvatar',
|
||||
unique: true
|
||||
},
|
||||
|
||||
[MongoImageTypeEnum.chatImage]: {
|
||||
label: 'chatImage',
|
||||
unique: false
|
||||
},
|
||||
[MongoImageTypeEnum.collectionImage]: {
|
||||
label: 'collectionImage',
|
||||
unique: false
|
||||
}
|
||||
};
|
||||
|
||||
export const uniqueImageTypeList = Object.entries(mongoImageTypeMap)
|
||||
.filter(([key, value]) => value.unique)
|
||||
.map(([key]) => key as `${MongoImageTypeEnum}`);
|
||||
|
||||
export const FolderIcon = 'file/fill/folder';
|
||||
export const FolderImgUrl = '/imgs/files/folder.svg';
|
||||
export const HttpPluginImgUrl = '/imgs/app/httpPluginFill.svg';
|
||||
|
||||
4
packages/global/common/file/image/type.d.ts
vendored
@@ -1,12 +1,8 @@
|
||||
import { MongoImageTypeEnum } from './constants';
|
||||
|
||||
export type MongoImageSchemaType = {
|
||||
_id: string;
|
||||
teamId: string;
|
||||
binary: Buffer;
|
||||
createTime: Date;
|
||||
expiredTime?: Date;
|
||||
type: `${MongoImageTypeEnum}`;
|
||||
|
||||
metadata?: {
|
||||
mime?: string; // image mime type.
|
||||
|
||||
@@ -2,6 +2,7 @@ import { detect } from 'jschardet';
|
||||
import { documentFileType, imageFileType } from './constants';
|
||||
import { ChatFileTypeEnum } from '../../core/chat/constants';
|
||||
import { UserChatItemValueItemType } from '../../core/chat/type';
|
||||
import * as fs from 'fs';
|
||||
|
||||
export const formatFileSize = (bytes: number): string => {
|
||||
if (bytes === 0) return '0 B';
|
||||
@@ -16,6 +17,22 @@ export const formatFileSize = (bytes: number): string => {
|
||||
export const detectFileEncoding = (buffer: Buffer) => {
|
||||
return detect(buffer.slice(0, 200))?.encoding?.toLocaleLowerCase();
|
||||
};
|
||||
export const detectFileEncodingByPath = async (path: string) => {
|
||||
// Get 64KB file head
|
||||
const MAX_BYTES = 64 * 1024;
|
||||
const buffer = Buffer.alloc(MAX_BYTES);
|
||||
|
||||
const fd = await fs.promises.open(path, 'r');
|
||||
try {
|
||||
// Read file head
|
||||
const { bytesRead } = await fd.read(buffer, 0, MAX_BYTES, 0);
|
||||
const actualBuffer = buffer.slice(0, bytesRead);
|
||||
|
||||
return detect(actualBuffer)?.encoding?.toLocaleLowerCase();
|
||||
} finally {
|
||||
await fd.close();
|
||||
}
|
||||
};
|
||||
|
||||
// Url => user upload file type
|
||||
export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file'] | undefined => {
|
||||
|
||||
44
packages/global/common/string/http.ts
Normal file
@@ -0,0 +1,44 @@
|
||||
import parse from '@bany/curl-to-json';
|
||||
|
||||
type RequestMethod = 'get' | 'post' | 'put' | 'delete' | 'patch';
|
||||
const methodMap: { [K in RequestMethod]: string } = {
|
||||
get: 'GET',
|
||||
post: 'POST',
|
||||
put: 'PUT',
|
||||
delete: 'DELETE',
|
||||
patch: 'PATCH'
|
||||
};
|
||||
|
||||
export const parseCurl = (curlContent: string) => {
|
||||
const parsed = parse(curlContent);
|
||||
|
||||
if (!parsed.url) {
|
||||
throw new Error('url not found');
|
||||
}
|
||||
|
||||
const newParams = Object.keys(parsed.params || {}).map((key) => ({
|
||||
key,
|
||||
value: parsed.params?.[key],
|
||||
type: 'string'
|
||||
}));
|
||||
const newHeaders = Object.keys(parsed.header || {}).map((key) => ({
|
||||
key,
|
||||
value: parsed.header?.[key],
|
||||
type: 'string'
|
||||
}));
|
||||
const newBody = JSON.stringify(parsed.data, null, 2);
|
||||
const bodyArray = Object.keys(parsed.data || {}).map((key) => ({
|
||||
key,
|
||||
value: parsed.data?.[key],
|
||||
type: 'string'
|
||||
}));
|
||||
|
||||
return {
|
||||
url: parsed.url,
|
||||
method: methodMap[parsed.method?.toLowerCase() as RequestMethod] || 'GET',
|
||||
params: newParams,
|
||||
headers: newHeaders,
|
||||
body: newBody,
|
||||
bodyArray
|
||||
};
|
||||
};
|
||||
@@ -1,6 +1,9 @@
|
||||
export const HUMAN_ICON = `/icon/human.svg`;
|
||||
export const LOGO_ICON = `/icon/logo.svg`;
|
||||
export const HUGGING_FACE_ICON = `/imgs/model/huggingface.svg`;
|
||||
|
||||
export const DEFAULT_TEAM_AVATAR = `/imgs/avatar/defaultTeamAvatar.svg`;
|
||||
export const DEFAULT_ORG_AVATAR = '/imgs/avatar/defaultOrgAvatar.svg';
|
||||
export const DEFAULT_USER_AVATAR = '/imgs/avatar/BlueAvatar.svg';
|
||||
|
||||
export const isProduction = process.env.NODE_ENV === 'production';
|
||||
|
||||
@@ -73,6 +73,11 @@ export type FastGPTFeConfigsType = {
|
||||
google?: string;
|
||||
wechat?: string;
|
||||
dingtalk?: string;
|
||||
wecom?: {
|
||||
corpid?: string;
|
||||
agentid?: string;
|
||||
secret?: string;
|
||||
};
|
||||
microsoft?: {
|
||||
clientId?: string;
|
||||
tenantId?: string;
|
||||
|
||||
1
packages/global/core/ai/model.d.ts
vendored
@@ -53,6 +53,7 @@ export type VectorModelItemType = PriceType & {
|
||||
};
|
||||
|
||||
export type ReRankModelItemType = PriceType & {
|
||||
provider: ModelProviderIdType;
|
||||
model: string;
|
||||
name: string;
|
||||
requestUrl: string;
|
||||
|
||||
5
packages/global/core/app/collaborator.d.ts
vendored
@@ -1,6 +1,6 @@
|
||||
import { RequireOnlyOne } from '../../common/type/utils';
|
||||
import type { RequireOnlyOne } from '../../common/type/utils';
|
||||
import {
|
||||
UpdateClbPermissionProps,
|
||||
type UpdateClbPermissionProps,
|
||||
UpdatePermissionBody
|
||||
} from '../../support/permission/collaborator';
|
||||
import { PermissionValueType } from '../../support/permission/type';
|
||||
@@ -14,4 +14,5 @@ export type AppCollaboratorDeleteParams = {
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
|
||||
@@ -5,6 +5,8 @@ import type { FlowNodeInputItemType } from '../workflow/type/io.d';
|
||||
import { getAppChatConfig } from '../workflow/utils';
|
||||
import { StoreNodeItemType } from '../workflow/type/node';
|
||||
import { DatasetSearchModeEnum } from '../dataset/constants';
|
||||
import { WorkflowTemplateBasicType } from '../workflow/type';
|
||||
import { AppTypeEnum } from './constants';
|
||||
|
||||
export const getDefaultAppForm = (): AppSimpleEditFormType => {
|
||||
return {
|
||||
@@ -127,3 +129,20 @@ export const appWorkflow2Form = ({
|
||||
|
||||
return defaultAppForm;
|
||||
};
|
||||
|
||||
export const getAppType = (config?: WorkflowTemplateBasicType | AppSimpleEditFormType) => {
|
||||
if (!config) return '';
|
||||
|
||||
if ('aiSettings' in config) {
|
||||
return AppTypeEnum.simple;
|
||||
}
|
||||
|
||||
if (!('nodes' in config)) return '';
|
||||
if (config.nodes.some((node) => node.flowNodeType === 'workflowStart')) {
|
||||
return AppTypeEnum.workflow;
|
||||
}
|
||||
if (config.nodes.some((node) => node.flowNodeType === 'pluginInput')) {
|
||||
return AppTypeEnum.plugin;
|
||||
}
|
||||
return '';
|
||||
};
|
||||
|
||||
@@ -11,4 +11,5 @@ export type DatasetCollaboratorDeleteParams = {
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
|
||||
9
packages/global/core/dataset/type.d.ts
vendored
@@ -112,6 +112,15 @@ export type DatasetDataSchemaType = {
|
||||
rebuilding?: boolean;
|
||||
};
|
||||
|
||||
export type DatasetDataTextSchemaType = {
|
||||
_id: string;
|
||||
teamId: string;
|
||||
datasetId: string;
|
||||
collectionId: string;
|
||||
dataId: string;
|
||||
fullTextToken: string;
|
||||
};
|
||||
|
||||
export type DatasetTrainingSchemaType = {
|
||||
_id: string;
|
||||
userId: string;
|
||||
|
||||
@@ -339,6 +339,7 @@ export function replaceEditorVariable({
|
||||
const output = node.outputs.find((output) => output.id === id);
|
||||
if (output) return formatVariableValByType(output.value, output.valueType);
|
||||
|
||||
// Use the node's input as the variable value(Example: HTTP data will reference its own dynamic input)
|
||||
const input = node.inputs.find((input) => input.key === id);
|
||||
if (input) return getReferenceVariableValue({ value: input.value, nodes, variables });
|
||||
})();
|
||||
|
||||
3
packages/global/core/workflow/type/io.d.ts
vendored
@@ -49,9 +49,10 @@ export type FlowNodeInputItemType = InputComponentPropsType & {
|
||||
debugLabel?: string;
|
||||
description?: string; // field desc
|
||||
required?: boolean;
|
||||
toolDescription?: string; // If this field is not empty, it is entered as a tool
|
||||
enum?: string;
|
||||
|
||||
toolDescription?: string; // If this field is not empty, it is entered as a tool
|
||||
|
||||
// render components params
|
||||
canEdit?: boolean; // dynamic inputs
|
||||
isPro?: boolean; // Pro version field
|
||||
|
||||
@@ -14,7 +14,8 @@
|
||||
"openai": "4.61.0",
|
||||
"openapi-types": "^12.1.3",
|
||||
"json5": "^2.2.3",
|
||||
"timezones-list": "^3.0.2"
|
||||
"timezones-list": "^3.0.2",
|
||||
"@bany/curl-to-json": "^1.2.8"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/js-yaml": "^4.0.9",
|
||||
|
||||
@@ -10,22 +10,18 @@ export type CollaboratorItemType = {
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
|
||||
export type UpdateClbPermissionProps = {
|
||||
members?: string[];
|
||||
groups?: string[];
|
||||
orgs?: string[];
|
||||
permission: PermissionValueType;
|
||||
};
|
||||
|
||||
export type DeleteClbPermissionProps = RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
}>;
|
||||
|
||||
export type UpdatePermissionBody = {
|
||||
permission: PermissionValueType;
|
||||
} & RequireOnlyOne<{
|
||||
memberId: string;
|
||||
groupId: string;
|
||||
export type DeletePermissionQuery = RequireOnlyOne<{
|
||||
tmbId?: string;
|
||||
groupId?: string;
|
||||
orgId?: string;
|
||||
}>;
|
||||
|
||||
4
packages/global/support/permission/type.d.ts
vendored
@@ -1,8 +1,9 @@
|
||||
import { UserModelSchema } from '../user/type';
|
||||
import { RequireOnlyOne } from '../../common/type/utils';
|
||||
import { TeamMemberSchema } from '../user/team/type';
|
||||
import { AuthUserTypeEnum, PermissionKeyEnum, PerResourceTypeEnum } from './constant';
|
||||
import { MemberGroupSchemaType } from './memberGroup/type';
|
||||
import type { TeamMemberWithUserSchema } from '../user/team/type';
|
||||
import { AuthUserTypeEnum, type PermissionKeyEnum, type PerResourceTypeEnum } from './constant';
|
||||
|
||||
// PermissionValueType, the type of permission's value is a number, which is a bit field actually.
|
||||
// It is spired by the permission system in Linux.
|
||||
@@ -29,6 +30,7 @@ export type ResourcePermissionType = {
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
|
||||
export type ResourcePerWithTmbWithUser = Omit<ResourcePermissionType, 'tmbId'> & {
|
||||
|
||||
@@ -17,5 +17,6 @@ export enum OAuthEnum {
|
||||
wechat = 'wechat',
|
||||
microsoft = 'microsoft',
|
||||
dingtalk = 'dingtalk',
|
||||
wecom = 'wecom',
|
||||
sso = 'sso'
|
||||
}
|
||||
|
||||
3
packages/global/support/user/login/constants.ts
Normal file
@@ -0,0 +1,3 @@
|
||||
export function checkIsWecomTerminal() {
|
||||
return /wxwork/i.test(navigator.userAgent);
|
||||
}
|
||||
32
packages/global/support/user/team/org/api.d.ts
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
export type postCreateOrgData = {
|
||||
name: string;
|
||||
parentId: string;
|
||||
description?: string;
|
||||
avatar?: string;
|
||||
};
|
||||
|
||||
export type putUpdateOrgMembersData = {
|
||||
orgId: string;
|
||||
members: {
|
||||
tmbId: string;
|
||||
// role: `${OrgMemberRole}`;
|
||||
}[];
|
||||
};
|
||||
|
||||
export type putUpdateOrgData = {
|
||||
orgId: string;
|
||||
name?: string;
|
||||
avatar?: string;
|
||||
description?: string;
|
||||
};
|
||||
|
||||
export type putMoveOrgType = {
|
||||
orgId: string;
|
||||
targetOrgId: string;
|
||||
};
|
||||
|
||||
// type putChnageOrgOwnerData = {
|
||||
// orgId: string;
|
||||
// tmbId: string;
|
||||
// toAdmin?: boolean;
|
||||
// };
|
||||
12
packages/global/support/user/team/org/constant.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
import { OrgSchemaType } from './type';
|
||||
|
||||
export const OrgCollectionName = 'team_orgs';
|
||||
export const OrgMemberCollectionName = 'team_org_members';
|
||||
|
||||
export const getOrgChildrenPath = (org: OrgSchemaType) => `${org.path}/${org.pathId}`;
|
||||
|
||||
// export enum OrgMemberRole {
|
||||
// owner = 'owner',
|
||||
// admin = 'admin',
|
||||
// member = 'member'
|
||||
// }
|
||||
25
packages/global/support/user/team/org/type.d.ts
vendored
Normal file
@@ -0,0 +1,25 @@
|
||||
import type { TeamPermission } from 'support/permission/user/controller';
|
||||
import { ResourcePermissionType } from '../type';
|
||||
|
||||
type OrgSchemaType = {
|
||||
_id: string;
|
||||
teamId: string;
|
||||
pathId: string;
|
||||
path: string;
|
||||
name: string;
|
||||
avatar?: string;
|
||||
description?: string;
|
||||
updateTime: Date;
|
||||
};
|
||||
|
||||
type OrgMemberSchemaType = {
|
||||
teamId: string;
|
||||
orgId: string;
|
||||
tmbId: string;
|
||||
};
|
||||
|
||||
type OrgType = Omit<OrgSchemaType, 'avatar'> & {
|
||||
avatar: string;
|
||||
members: OrgMemberSchemaType[];
|
||||
permission: TeamPermission;
|
||||
};
|
||||
3
packages/global/support/user/team/type.d.ts
vendored
@@ -55,10 +55,11 @@ export type TeamMemberWithTeamAndUserSchema = TeamMemberSchema & {
|
||||
export type TeamTmbItemType = {
|
||||
userId: string;
|
||||
teamId: string;
|
||||
teamAvatar?: string;
|
||||
teamName: string;
|
||||
memberName: string;
|
||||
avatar: string;
|
||||
balance: number;
|
||||
balance?: number;
|
||||
tmbId: string;
|
||||
teamDomain: string;
|
||||
defaultTeam: boolean;
|
||||
|
||||
16
packages/global/support/user/utils.ts
Normal file
@@ -0,0 +1,16 @@
|
||||
export const getRandomUserAvatar = () => {
|
||||
const defaultAvatars = [
|
||||
'/imgs/avatar/RoyalBlueAvatar.svg',
|
||||
'/imgs/avatar/PurpleAvatar.svg',
|
||||
'/imgs/avatar/AdoraAvatar.svg',
|
||||
'/imgs/avatar/OrangeAvatar.svg',
|
||||
'/imgs/avatar/RedAvatar.svg',
|
||||
'/imgs/avatar/GrayModernAvatar.svg',
|
||||
'/imgs/avatar/TealAvatar.svg',
|
||||
'/imgs/avatar/GreenAvatar.svg',
|
||||
'/imgs/avatar/BrightBlueAvatar.svg',
|
||||
'/imgs/avatar/BlueAvatar.svg'
|
||||
];
|
||||
|
||||
return defaultAvatars[Math.floor(Math.random() * defaultAvatars.length)];
|
||||
};
|
||||
@@ -35,24 +35,26 @@ export const list = [...staticPluginList, ...packagePluginList];
|
||||
|
||||
/* Get plugins */
|
||||
export const getCommunityPlugins = () => {
|
||||
return list.map<SystemPluginTemplateItemType>((name) => {
|
||||
const config = require(`./src/${name}/template.json`);
|
||||
return Promise.all(
|
||||
list.map<Promise<SystemPluginTemplateItemType>>(async (name) => {
|
||||
const config = (await import(`./src/${name}/template.json`))?.default;
|
||||
|
||||
const isFolder = list.find((item) => item.startsWith(`${name}/`));
|
||||
const isFolder = list.find((item) => item.startsWith(`${name}/`));
|
||||
|
||||
const parentIdList = name.split('/').slice(0, -1);
|
||||
const parentId =
|
||||
parentIdList.length > 0 ? `${PluginSourceEnum.community}-${parentIdList.join('/')}` : null;
|
||||
const parentIdList = name.split('/').slice(0, -1);
|
||||
const parentId =
|
||||
parentIdList.length > 0 ? `${PluginSourceEnum.community}-${parentIdList.join('/')}` : null;
|
||||
|
||||
return {
|
||||
...config,
|
||||
id: `${PluginSourceEnum.community}-${name}`,
|
||||
isFolder,
|
||||
parentId,
|
||||
isActive: true,
|
||||
isOfficial: true
|
||||
};
|
||||
});
|
||||
return {
|
||||
...config,
|
||||
id: `${PluginSourceEnum.community}-${name}`,
|
||||
isFolder,
|
||||
parentId,
|
||||
isActive: true,
|
||||
isOfficial: true
|
||||
};
|
||||
})
|
||||
);
|
||||
};
|
||||
|
||||
export const getSystemPluginTemplates = () => {
|
||||
|
||||
@@ -48,6 +48,16 @@ const main = async (props: Props, retry = 3): Response => {
|
||||
});
|
||||
});
|
||||
|
||||
if (results.length === 0) {
|
||||
return {
|
||||
result: JSON.stringify([]),
|
||||
error: {
|
||||
message: 'No search results',
|
||||
code: 500
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
result: JSON.stringify(results.slice(0, 10))
|
||||
};
|
||||
|
||||
@@ -4,7 +4,7 @@ import fsp from 'fs/promises';
|
||||
import fs from 'fs';
|
||||
import { DatasetFileSchema } from '@fastgpt/global/core/dataset/type';
|
||||
import { MongoChatFileSchema, MongoDatasetFileSchema } from './schema';
|
||||
import { detectFileEncoding } from '@fastgpt/global/common/file/tools';
|
||||
import { detectFileEncoding, detectFileEncodingByPath } from '@fastgpt/global/common/file/tools';
|
||||
import { CommonErrEnum } from '@fastgpt/global/common/error/code/common';
|
||||
import { MongoRawTextBuffer } from '../../buffer/rawText/schema';
|
||||
import { readRawContentByFileBuffer } from '../read/utils';
|
||||
@@ -36,7 +36,6 @@ export async function uploadFile({
|
||||
path,
|
||||
filename,
|
||||
contentType,
|
||||
encoding,
|
||||
metadata = {}
|
||||
}: {
|
||||
bucketName: `${BucketNameEnum}`;
|
||||
@@ -45,7 +44,6 @@ export async function uploadFile({
|
||||
path: string;
|
||||
filename: string;
|
||||
contentType?: string;
|
||||
encoding: string;
|
||||
metadata?: Record<string, any>;
|
||||
}) {
|
||||
if (!path) return Promise.reject(`filePath is empty`);
|
||||
@@ -59,7 +57,7 @@ export async function uploadFile({
|
||||
// Add default metadata
|
||||
metadata.teamId = teamId;
|
||||
metadata.uid = uid;
|
||||
metadata.encoding = encoding;
|
||||
metadata.encoding = await detectFileEncodingByPath(path);
|
||||
|
||||
// create a gridfs bucket
|
||||
const bucket = getGridBucket(bucketName);
|
||||
|
||||
@@ -1,43 +1,92 @@
|
||||
import { UploadImgProps } from '@fastgpt/global/common/file/api';
|
||||
import { imageBaseUrl } from '@fastgpt/global/common/file/image/constants';
|
||||
import { MongoImage } from './schema';
|
||||
import { ClientSession } from '../../../common/mongo';
|
||||
import { ClientSession, Types } from '../../../common/mongo';
|
||||
import { guessBase64ImageType } from '../utils';
|
||||
import { readFromSecondary } from '../../mongo/utils';
|
||||
import { addHours } from 'date-fns';
|
||||
|
||||
export const maxImgSize = 1024 * 1024 * 12;
|
||||
const base64MimeRegex = /data:image\/([^\)]+);base64/;
|
||||
export async function uploadMongoImg({
|
||||
type,
|
||||
base64Img,
|
||||
teamId,
|
||||
expiredTime,
|
||||
metadata,
|
||||
shareId
|
||||
shareId,
|
||||
forever = false
|
||||
}: UploadImgProps & {
|
||||
teamId: string;
|
||||
forever?: Boolean;
|
||||
}) {
|
||||
if (base64Img.length > maxImgSize) {
|
||||
return Promise.reject('Image too large');
|
||||
}
|
||||
|
||||
const [base64Mime, base64Data] = base64Img.split(',');
|
||||
// Check if mime type is valid
|
||||
if (!base64MimeRegex.test(base64Mime)) {
|
||||
return Promise.reject('Invalid image mime type');
|
||||
}
|
||||
|
||||
const mime = `image/${base64Mime.match(base64MimeRegex)?.[1] ?? 'image/jpeg'}`;
|
||||
const binary = Buffer.from(base64Data, 'base64');
|
||||
const extension = mime.split('/')[1];
|
||||
|
||||
const { _id } = await MongoImage.create({
|
||||
type,
|
||||
teamId,
|
||||
binary,
|
||||
expiredTime,
|
||||
metadata: Object.assign({ mime }, metadata),
|
||||
shareId
|
||||
shareId,
|
||||
expiredTime: forever ? undefined : addHours(new Date(), 1)
|
||||
});
|
||||
|
||||
return `${process.env.FE_DOMAIN || ''}${process.env.NEXT_PUBLIC_BASE_URL || ''}${imageBaseUrl}${String(_id)}.${extension}`;
|
||||
}
|
||||
|
||||
const getIdFromPath = (path?: string) => {
|
||||
if (!path) return;
|
||||
|
||||
const paths = path.split('/');
|
||||
const name = paths[paths.length - 1];
|
||||
|
||||
if (!name) return;
|
||||
|
||||
const id = name.split('.')[0];
|
||||
if (!id || !Types.ObjectId.isValid(id)) return;
|
||||
|
||||
return id;
|
||||
};
|
||||
// 删除旧的头像,新的头像去除过期时间
|
||||
export const refreshSourceAvatar = async (
|
||||
path?: string,
|
||||
oldPath?: string,
|
||||
session?: ClientSession
|
||||
) => {
|
||||
const newId = getIdFromPath(path);
|
||||
const oldId = getIdFromPath(oldPath);
|
||||
|
||||
if (!newId) return;
|
||||
|
||||
await MongoImage.updateOne({ _id: newId }, { $unset: { expiredTime: 1 } }, { session });
|
||||
|
||||
if (oldId) {
|
||||
await MongoImage.deleteOne({ _id: oldId }, { session });
|
||||
}
|
||||
};
|
||||
export const removeImageByPath = (path?: string, session?: ClientSession) => {
|
||||
if (!path) return;
|
||||
|
||||
const paths = path.split('/');
|
||||
const name = paths[paths.length - 1];
|
||||
|
||||
if (!name) return;
|
||||
|
||||
const id = name.split('.')[0];
|
||||
if (!id || !Types.ObjectId.isValid(id)) return;
|
||||
|
||||
return MongoImage.deleteOne({ _id: id }, { session });
|
||||
};
|
||||
|
||||
export async function readMongoImg({ id }: { id: string }) {
|
||||
const formatId = id.replace(/\.[^/.]+$/, '');
|
||||
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
|
||||
import { connectionMongo, getMongoModel, type Model } from '../../mongo';
|
||||
import { connectionMongo, getMongoModel } from '../../mongo';
|
||||
import { MongoImageSchemaType } from '@fastgpt/global/common/file/image/type.d';
|
||||
import { mongoImageTypeMap } from '@fastgpt/global/common/file/image/constants';
|
||||
const { Schema, model, models } = connectionMongo;
|
||||
const { Schema } = connectionMongo;
|
||||
|
||||
const ImageSchema = new Schema({
|
||||
teamId: {
|
||||
@@ -14,27 +13,15 @@ const ImageSchema = new Schema({
|
||||
type: Date,
|
||||
default: () => new Date()
|
||||
},
|
||||
expiredTime: {
|
||||
type: Date
|
||||
},
|
||||
binary: {
|
||||
type: Buffer
|
||||
},
|
||||
type: {
|
||||
type: String,
|
||||
enum: Object.keys(mongoImageTypeMap),
|
||||
required: true
|
||||
},
|
||||
metadata: {
|
||||
type: Object
|
||||
}
|
||||
expiredTime: Date,
|
||||
binary: Buffer,
|
||||
metadata: Object
|
||||
});
|
||||
|
||||
try {
|
||||
// tts expired(60 Minutes)
|
||||
ImageSchema.index({ expiredTime: 1 }, { expireAfterSeconds: 60 * 60 });
|
||||
ImageSchema.index({ type: 1 });
|
||||
ImageSchema.index({ createTime: 1 });
|
||||
// delete related img
|
||||
ImageSchema.index({ teamId: 1, 'metadata.relatedId': 1 });
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { uploadMongoImg } from '../image/controller';
|
||||
import { MongoImageTypeEnum } from '@fastgpt/global/common/file/image/constants';
|
||||
import FormData from 'form-data';
|
||||
|
||||
import { WorkerNameEnum, runWorker } from '../../../worker/utils';
|
||||
@@ -8,7 +7,6 @@ import type { ReadFileResponse } from '../../../worker/readFile/type';
|
||||
import axios from 'axios';
|
||||
import { addLog } from '../../system/log';
|
||||
import { batchRun } from '@fastgpt/global/common/fn/utils';
|
||||
import { addHours } from 'date-fns';
|
||||
import { matchMdImgTextAndUpload } from '@fastgpt/global/common/string/markdown';
|
||||
|
||||
export type readRawTextByLocalFileParams = {
|
||||
@@ -22,7 +20,7 @@ export const readRawTextByLocalFile = async (params: readRawTextByLocalFileParam
|
||||
|
||||
const extension = path?.split('.')?.pop()?.toLowerCase() || '';
|
||||
|
||||
const buffer = fs.readFileSync(path);
|
||||
const buffer = await fs.promises.readFile(path);
|
||||
|
||||
const { rawText } = await readRawContentByFileBuffer({
|
||||
extension,
|
||||
@@ -114,10 +112,9 @@ export const readRawContentByFileBuffer = async ({
|
||||
if (imageList) {
|
||||
await batchRun(imageList, async (item) => {
|
||||
const src = await uploadMongoImg({
|
||||
type: MongoImageTypeEnum.collectionImage,
|
||||
base64Img: `data:${item.mime};base64,${item.base64}`,
|
||||
teamId,
|
||||
expiredTime: addHours(new Date(), 1),
|
||||
// expiredTime: addHours(new Date(), 1),
|
||||
metadata: {
|
||||
...metadata,
|
||||
mime: item.mime
|
||||
|
||||
@@ -9,10 +9,10 @@ import { jsonRes } from '../response';
|
||||
// unit: times/s
|
||||
// how to use?
|
||||
// export default NextAPI(useQPSLimit(10), handler); // limit 10 times per second for a ip
|
||||
export function useReqFrequencyLimit(seconds: number, limit: number) {
|
||||
export function useReqFrequencyLimit(seconds: number, limit: number, force = false) {
|
||||
return async (req: ApiRequestProps, res: NextApiResponse) => {
|
||||
const ip = requestIp.getClientIp(req);
|
||||
if (!ip || process.env.USE_IP_LIMIT !== 'true') {
|
||||
if (!ip || (process.env.USE_IP_LIMIT !== 'true' && !force)) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
@@ -22,10 +22,9 @@ export function useReqFrequencyLimit(seconds: number, limit: number) {
|
||||
expiredTime: addSeconds(new Date(), seconds)
|
||||
});
|
||||
} catch (_) {
|
||||
res.status(429);
|
||||
jsonRes(res, {
|
||||
code: 429,
|
||||
message: ERROR_ENUM.tooManyRequest
|
||||
error: ERROR_ENUM.tooManyRequest
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
@@ -33,7 +33,7 @@ export const jsonRes = <T = any>(
|
||||
|
||||
addLog.error(`Api response error: ${url}`, ERROR_RESPONSE[errResponseKey]);
|
||||
|
||||
return res.json(ERROR_RESPONSE[errResponseKey]);
|
||||
return res.status(code).json(ERROR_RESPONSE[errResponseKey]);
|
||||
}
|
||||
|
||||
// another error
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"provider": "OpenAI",
|
||||
"model": "text-embedding-ada-002",
|
||||
"name": "text-embedding-ada-002",
|
||||
|
||||
"defaultToken": 512, // 默认分块 token
|
||||
"maxToken": 3000, // 最大分块 token
|
||||
"weight": 0, // 权重
|
||||
|
||||
"charsPointsPrice": 0 // 积分/1k token
|
||||
}
|
||||
33
packages/service/core/ai/config/llm/gpt-4o-mini.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"provider": "OpenAI",
|
||||
"model": "gpt-4o-mini",
|
||||
"name": "GPT-4o-mini", // alias
|
||||
|
||||
"maxContext": 125000, // 最大上下文
|
||||
"maxResponse": 16000, // 最大回复
|
||||
"quoteMaxToken": 60000, // 最大引用
|
||||
"maxTemperature": 1.2, // 最大温度
|
||||
"presencePenaltyRange": [-2, 2], // 惩罚系数范围
|
||||
"frequencyPenaltyRange": [-2, 2], // 频率惩罚系数范围
|
||||
"responseFormatList": ["text", "json_object", "json_schema"], // 响应格式
|
||||
"showStopSign": true, // 是否显示停止符号
|
||||
|
||||
"vision": true, // 是否支持图片识别
|
||||
"toolChoice": true, // 是否支持工具调用
|
||||
"functionCall": false, // 是否支持函数调用(一般都可以 false 了,基本不用了)
|
||||
"defaultSystemChatPrompt": "", // 默认系统提示
|
||||
|
||||
"datasetProcess": true, // 用于知识库文本处理
|
||||
"usedInClassify": true, // 用于问题分类
|
||||
"customCQPrompt": "", // 自定义问题分类提示
|
||||
"usedInExtractFields": true, // 用于提取字段
|
||||
"customExtractPrompt": "", // 自定义提取提示
|
||||
"usedInToolCall": true, // 用于工具调用
|
||||
"usedInQueryExtension": true, // 用于问题优化
|
||||
|
||||
"defaultConfig": {}, // 额外的自定义 body
|
||||
"fieldMap": {}, // body 字段映射
|
||||
|
||||
"censor": false, // 是否开启敏感词过滤
|
||||
"charsPointsPrice": 0 // n 积分/1k token
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"provider": "BAAI",
|
||||
"model": "bge-reranker-v2-m3",
|
||||
"name": "bge-reranker-v2-m3",
|
||||
"charsPointsPrice": 0
|
||||
}
|
||||
6
packages/service/core/ai/config/stt/whisper-1.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"provider": "OpenAI",
|
||||
"model": "whisper-1",
|
||||
"name": "whisper-1",
|
||||
"charsPointsPrice": 0
|
||||
}
|
||||
32
packages/service/core/ai/config/tts/tts-1.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"provider": "OpenAI",
|
||||
"model": "tts-1",
|
||||
"name": "TTS1",
|
||||
"charsPointsPrice": 0,
|
||||
"voices": [
|
||||
{
|
||||
"label": "Alloy",
|
||||
"value": "alloy"
|
||||
},
|
||||
{
|
||||
"label": "Echo",
|
||||
"value": "echo"
|
||||
},
|
||||
{
|
||||
"label": "Fable",
|
||||
"value": "fable"
|
||||
},
|
||||
{
|
||||
"label": "Onyx",
|
||||
"value": "onyx"
|
||||
},
|
||||
{
|
||||
"label": "Nova",
|
||||
"value": "nova"
|
||||
},
|
||||
{
|
||||
"label": "Shimmer",
|
||||
"value": "shimmer"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -5,11 +5,11 @@ import { PluginSourceEnum } from '@fastgpt/global/core/plugin/constants';
|
||||
|
||||
/*
|
||||
Plugin points calculation:
|
||||
1. 商业版插件:
|
||||
1. 系统插件/商业版插件:
|
||||
- 有错误:返回 0
|
||||
- 无错误:返回 配置的点数 + 子节点点数
|
||||
2. 其他插件:
|
||||
- 返回 子节点点数
|
||||
- 无错误:返回 单次积分 + 子流程积分(可配置)
|
||||
2. 个人插件
|
||||
- 返回 子流程积分
|
||||
*/
|
||||
export const computedPluginUsage = async ({
|
||||
plugin,
|
||||
@@ -26,9 +26,9 @@ export const computedPluginUsage = async ({
|
||||
if (source !== PluginSourceEnum.personal) {
|
||||
if (error) return 0;
|
||||
|
||||
const pluginCurrentCose = plugin.currentCost ?? 0;
|
||||
const pluginCurrentCost = plugin.currentCost ?? 0;
|
||||
|
||||
return plugin.hasTokenFee ? pluginCurrentCose + childrenUsages : pluginCurrentCose;
|
||||
return plugin.hasTokenFee ? pluginCurrentCost + childrenUsages : pluginCurrentCost;
|
||||
}
|
||||
|
||||
return childrenUsages;
|
||||
|
||||
@@ -86,24 +86,21 @@ const ChatItemSchema = new Schema({
|
||||
});
|
||||
|
||||
try {
|
||||
ChatItemSchema.index({ dataId: 1 }, { background: true });
|
||||
ChatItemSchema.index({ dataId: 1 });
|
||||
/* delete by app;
|
||||
delete by chat id;
|
||||
get chat list;
|
||||
get chat logs;
|
||||
close custom feedback;
|
||||
*/
|
||||
ChatItemSchema.index({ appId: 1, chatId: 1, dataId: 1 }, { background: true });
|
||||
ChatItemSchema.index({ appId: 1, chatId: 1, dataId: 1 });
|
||||
// admin charts
|
||||
ChatItemSchema.index({ time: -1, obj: 1 }, { background: true });
|
||||
ChatItemSchema.index({ time: -1, obj: 1 });
|
||||
// timer, clear history
|
||||
ChatItemSchema.index({ teamId: 1, time: -1 }, { background: true });
|
||||
ChatItemSchema.index({ teamId: 1, time: -1 });
|
||||
|
||||
// Admin charts
|
||||
ChatItemSchema.index(
|
||||
{ obj: 1, time: -1 },
|
||||
{ background: true, partialFilterExpression: { obj: 'Human' } }
|
||||
);
|
||||
ChatItemSchema.index({ obj: 1, time: -1 }, { partialFilterExpression: { obj: 'Human' } });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
@@ -81,19 +81,19 @@ const ChatSchema = new Schema({
|
||||
});
|
||||
|
||||
try {
|
||||
ChatSchema.index({ chatId: 1 }, { background: true });
|
||||
ChatSchema.index({ chatId: 1 });
|
||||
// get user history
|
||||
ChatSchema.index({ tmbId: 1, appId: 1, top: -1, updateTime: -1 }, { background: true });
|
||||
ChatSchema.index({ tmbId: 1, appId: 1, top: -1, updateTime: -1 });
|
||||
// delete by appid; clear history; init chat; update chat; auth chat; get chat;
|
||||
ChatSchema.index({ appId: 1, chatId: 1 }, { background: true });
|
||||
ChatSchema.index({ appId: 1, chatId: 1 });
|
||||
|
||||
// get chat logs;
|
||||
ChatSchema.index({ teamId: 1, appId: 1, updateTime: -1 }, { background: true });
|
||||
ChatSchema.index({ teamId: 1, appId: 1, updateTime: -1 });
|
||||
// get share chat history
|
||||
ChatSchema.index({ shareId: 1, outLinkUid: 1, updateTime: -1 }, { background: true });
|
||||
ChatSchema.index({ shareId: 1, outLinkUid: 1, updateTime: -1 });
|
||||
|
||||
// timer, clear history
|
||||
ChatSchema.index({ teamId: 1, updateTime: -1 }, { background: true });
|
||||
ChatSchema.index({ teamId: 1, updateTime: -1 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
@@ -24,6 +24,7 @@ import { pushDataListToTrainingQueue } from '../training/controller';
|
||||
import { MongoImage } from '../../../common/file/image/schema';
|
||||
import { hashStr } from '@fastgpt/global/common/string/tools';
|
||||
import { addDays } from 'date-fns';
|
||||
import { MongoDatasetDataText } from '../data/dataTextSchema';
|
||||
|
||||
export const createCollectionAndInsertData = async ({
|
||||
dataset,
|
||||
@@ -240,12 +241,12 @@ export const delCollectionRelatedSource = async ({
|
||||
.map((item) => item?.metadata?.relatedImgId || '')
|
||||
.filter(Boolean);
|
||||
|
||||
// delete files
|
||||
// Delete files
|
||||
await delFileByFileIdList({
|
||||
bucketName: BucketNameEnum.dataset,
|
||||
fileIdList
|
||||
});
|
||||
// delete images
|
||||
// Delete images
|
||||
await delImgByRelatedId({
|
||||
teamId,
|
||||
relateIds: relatedImageIds,
|
||||
@@ -273,7 +274,7 @@ export async function delCollection({
|
||||
const datasetIds = Array.from(new Set(collections.map((item) => String(item.datasetId))));
|
||||
const collectionIds = collections.map((item) => String(item._id));
|
||||
|
||||
// delete training data
|
||||
// Delete training data
|
||||
await MongoDatasetTraining.deleteMany({
|
||||
teamId,
|
||||
datasetIds: { $in: datasetIds },
|
||||
@@ -285,11 +286,16 @@ export async function delCollection({
|
||||
await delCollectionRelatedSource({ collections, session });
|
||||
}
|
||||
|
||||
// delete dataset.datas
|
||||
// Delete dataset_datas
|
||||
await MongoDatasetData.deleteMany(
|
||||
{ teamId, datasetIds: { $in: datasetIds }, collectionId: { $in: collectionIds } },
|
||||
{ session }
|
||||
);
|
||||
// Delete dataset_data_texts
|
||||
await MongoDatasetDataText.deleteMany(
|
||||
{ teamId, datasetIds: { $in: datasetIds }, collectionId: { $in: collectionIds } },
|
||||
{ session }
|
||||
);
|
||||
|
||||
// delete collections
|
||||
await MongoDatasetCollection.deleteMany(
|
||||
|
||||
@@ -6,6 +6,7 @@ import { ClientSession } from '../../common/mongo';
|
||||
import { MongoDatasetTraining } from './training/schema';
|
||||
import { MongoDatasetData } from './data/schema';
|
||||
import { deleteDatasetDataVector } from '../../common/vectorStore/controller';
|
||||
import { MongoDatasetDataText } from './data/dataTextSchema';
|
||||
|
||||
/* ============= dataset ========== */
|
||||
/* find all datasetId by top datasetId */
|
||||
@@ -92,7 +93,7 @@ export async function delDatasetRelevantData({
|
||||
{ session }
|
||||
).lean();
|
||||
|
||||
// image and file
|
||||
// Delete Image and file
|
||||
await delCollectionRelatedSource({ collections, session });
|
||||
|
||||
// delete collections
|
||||
@@ -101,9 +102,15 @@ export async function delDatasetRelevantData({
|
||||
datasetId: { $in: datasetIds }
|
||||
}).session(session);
|
||||
|
||||
// delete dataset.datas(Not need session)
|
||||
// No session delete:
|
||||
// Delete dataset_data_texts
|
||||
await MongoDatasetDataText.deleteMany({
|
||||
teamId,
|
||||
datasetId: { $in: datasetIds }
|
||||
});
|
||||
// delete dataset_datas
|
||||
await MongoDatasetData.deleteMany({ teamId, datasetId: { $in: datasetIds } });
|
||||
|
||||
// no session delete: delete files, vector data
|
||||
// Delete vector data
|
||||
await deleteDatasetDataVector({ teamId, datasetIds });
|
||||
}
|
||||
|
||||
45
packages/service/core/dataset/data/dataTextSchema.ts
Normal file
@@ -0,0 +1,45 @@
|
||||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
const { Schema } = connectionMongo;
|
||||
import { DatasetDataSchemaType } from '@fastgpt/global/core/dataset/type.d';
|
||||
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
|
||||
import { DatasetCollectionName } from '../schema';
|
||||
import { DatasetColCollectionName } from '../collection/schema';
|
||||
import { DatasetDataCollectionName } from './schema';
|
||||
|
||||
export const DatasetDataTextCollectionName = 'dataset_data_texts';
|
||||
|
||||
const DatasetDataTextSchema = new Schema({
|
||||
teamId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamCollectionName,
|
||||
required: true
|
||||
},
|
||||
datasetId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: DatasetCollectionName,
|
||||
required: true
|
||||
},
|
||||
collectionId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: DatasetColCollectionName,
|
||||
required: true
|
||||
},
|
||||
dataId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: DatasetDataCollectionName,
|
||||
required: true
|
||||
},
|
||||
fullTextToken: String
|
||||
});
|
||||
|
||||
try {
|
||||
DatasetDataTextSchema.index({ teamId: 1, datasetId: 1, fullTextToken: 'text' });
|
||||
DatasetDataTextSchema.index({ dataId: 1 }, { unique: true });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
export const MongoDatasetDataText = getMongoModel<DatasetDataSchemaType>(
|
||||
DatasetDataTextCollectionName,
|
||||
DatasetDataTextSchema
|
||||
);
|
||||
@@ -1,4 +1,4 @@
|
||||
import { connectionMongo, getMongoModel, type Model } from '../../../common/mongo';
|
||||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
const { Schema, model, models } = connectionMongo;
|
||||
import { DatasetDataSchemaType } from '@fastgpt/global/core/dataset/type.d';
|
||||
import {
|
||||
@@ -39,10 +39,6 @@ const DatasetDataSchema = new Schema({
|
||||
type: String,
|
||||
default: ''
|
||||
},
|
||||
fullTextToken: {
|
||||
type: String,
|
||||
default: ''
|
||||
},
|
||||
indexes: {
|
||||
type: [
|
||||
{
|
||||
@@ -71,17 +67,11 @@ const DatasetDataSchema = new Schema({
|
||||
type: Number,
|
||||
default: 0
|
||||
},
|
||||
inited: {
|
||||
type: Boolean
|
||||
},
|
||||
rebuilding: Boolean
|
||||
});
|
||||
rebuilding: Boolean,
|
||||
|
||||
DatasetDataSchema.virtual('collection', {
|
||||
ref: DatasetColCollectionName,
|
||||
localField: 'collectionId',
|
||||
foreignField: '_id',
|
||||
justOne: true
|
||||
// Abandon
|
||||
fullTextToken: String,
|
||||
initFullText: Boolean
|
||||
});
|
||||
|
||||
try {
|
||||
@@ -93,13 +83,15 @@ try {
|
||||
chunkIndex: 1,
|
||||
updateTime: -1
|
||||
});
|
||||
// full text index
|
||||
DatasetDataSchema.index({ teamId: 1, datasetId: 1, fullTextToken: 'text' });
|
||||
// FullText tmp full text index
|
||||
// DatasetDataSchema.index({ teamId: 1, datasetId: 1, fullTextToken: 'text' });
|
||||
// Recall vectors after data matching
|
||||
DatasetDataSchema.index({ teamId: 1, datasetId: 1, collectionId: 1, 'indexes.dataId': 1 });
|
||||
DatasetDataSchema.index({ updateTime: 1 });
|
||||
// rebuild data
|
||||
DatasetDataSchema.index({ rebuilding: 1, teamId: 1, datasetId: 1 });
|
||||
|
||||
DatasetDataSchema.index({ initFullText: 1 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
@@ -8,8 +8,8 @@ import { getVectorsByText } from '../../ai/embedding';
|
||||
import { getVectorModel } from '../../ai/model';
|
||||
import { MongoDatasetData } from '../data/schema';
|
||||
import {
|
||||
DatasetCollectionSchemaType,
|
||||
DatasetDataSchemaType,
|
||||
DatasetDataTextSchemaType,
|
||||
SearchDataResponseItemType
|
||||
} from '@fastgpt/global/core/dataset/type';
|
||||
import { MongoDatasetCollection } from '../collection/schema';
|
||||
@@ -23,6 +23,7 @@ import { Types } from '../../../common/mongo';
|
||||
import json5 from 'json5';
|
||||
import { MongoDatasetCollectionTags } from '../tag/schema';
|
||||
import { readFromSecondary } from '../../../common/mongo/utils';
|
||||
import { MongoDatasetDataText } from '../data/dataTextSchema';
|
||||
|
||||
type SearchDatasetDataProps = {
|
||||
teamId: string;
|
||||
@@ -266,57 +267,60 @@ export async function searchDatasetData(props: SearchDatasetDataProps) {
|
||||
filterCollectionIdList
|
||||
});
|
||||
|
||||
// get q and a
|
||||
const dataList = await MongoDatasetData.find(
|
||||
{
|
||||
teamId,
|
||||
datasetId: { $in: datasetIds },
|
||||
collectionId: { $in: Array.from(new Set(results.map((item) => item.collectionId))) },
|
||||
'indexes.dataId': { $in: results.map((item) => item.id?.trim()) }
|
||||
},
|
||||
'datasetId collectionId updateTime q a chunkIndex indexes'
|
||||
)
|
||||
.populate<{ collection: DatasetCollectionSchemaType }>(
|
||||
'collection',
|
||||
'name fileId rawLink externalFileId externalFileUrl'
|
||||
)
|
||||
.lean();
|
||||
// Get data and collections
|
||||
const collectionIdList = Array.from(new Set(results.map((item) => item.collectionId)));
|
||||
const [dataList, collections] = await Promise.all([
|
||||
MongoDatasetData.find(
|
||||
{
|
||||
teamId,
|
||||
datasetId: { $in: datasetIds },
|
||||
collectionId: { $in: collectionIdList },
|
||||
'indexes.dataId': { $in: results.map((item) => item.id?.trim()) }
|
||||
},
|
||||
'_id datasetId collectionId updateTime q a chunkIndex indexes',
|
||||
{ ...readFromSecondary }
|
||||
).lean(),
|
||||
MongoDatasetCollection.find(
|
||||
{
|
||||
_id: { $in: collectionIdList }
|
||||
},
|
||||
'_id name fileId rawLink externalFileId externalFileUrl',
|
||||
{ ...readFromSecondary }
|
||||
).lean()
|
||||
]);
|
||||
|
||||
// add score to data(It's already sorted. The first one is the one with the most points)
|
||||
const concatResults = dataList.map((data) => {
|
||||
const dataIdList = data.indexes.map((item) => item.dataId);
|
||||
const formatResult = results
|
||||
.map((item, index) => {
|
||||
const collection = collections.find((col) => String(col._id) === String(item.collectionId));
|
||||
if (!collection) {
|
||||
console.log('Collection is not found', item);
|
||||
return;
|
||||
}
|
||||
const data = dataList.find((data) =>
|
||||
data.indexes.some((index) => index.dataId === item.id)
|
||||
);
|
||||
if (!data) {
|
||||
console.log('Data is not found', item);
|
||||
return;
|
||||
}
|
||||
|
||||
const maxScoreResult = results.find((item) => {
|
||||
return dataIdList.includes(item.id);
|
||||
});
|
||||
const score = item?.score || 0;
|
||||
|
||||
return {
|
||||
...data,
|
||||
score: maxScoreResult?.score || 0
|
||||
};
|
||||
});
|
||||
const result: SearchDataResponseItemType = {
|
||||
id: String(data._id),
|
||||
updateTime: data.updateTime,
|
||||
q: data.q,
|
||||
a: data.a,
|
||||
chunkIndex: data.chunkIndex,
|
||||
datasetId: String(data.datasetId),
|
||||
collectionId: String(data.collectionId),
|
||||
...getCollectionSourceData(collection),
|
||||
score: [{ type: SearchScoreTypeEnum.embedding, value: score, index }]
|
||||
};
|
||||
|
||||
concatResults.sort((a, b) => b.score - a.score);
|
||||
|
||||
const formatResult = concatResults.map((data, index) => {
|
||||
if (!data.collectionId) {
|
||||
console.log('Collection is not found', data);
|
||||
}
|
||||
|
||||
const result: SearchDataResponseItemType = {
|
||||
id: String(data._id),
|
||||
updateTime: data.updateTime,
|
||||
q: data.q,
|
||||
a: data.a,
|
||||
chunkIndex: data.chunkIndex,
|
||||
datasetId: String(data.datasetId),
|
||||
collectionId: String(data.collectionId),
|
||||
...getCollectionSourceData(data.collection),
|
||||
score: [{ type: SearchScoreTypeEnum.embedding, value: data.score, index }]
|
||||
};
|
||||
|
||||
return result;
|
||||
});
|
||||
return result;
|
||||
})
|
||||
.filter(Boolean) as SearchDataResponseItemType[];
|
||||
|
||||
return {
|
||||
embeddingRecallResults: formatResult,
|
||||
@@ -344,88 +348,224 @@ export async function searchDatasetData(props: SearchDatasetDataProps) {
|
||||
};
|
||||
}
|
||||
|
||||
let searchResults = (
|
||||
const searchResults = (
|
||||
await Promise.all(
|
||||
datasetIds.map(async (id) => {
|
||||
return MongoDatasetData.aggregate([
|
||||
{
|
||||
$match: {
|
||||
teamId: new Types.ObjectId(teamId),
|
||||
datasetId: new Types.ObjectId(id),
|
||||
$text: { $search: jiebaSplit({ text: query }) },
|
||||
...(filterCollectionIdList
|
||||
? {
|
||||
collectionId: {
|
||||
$in: filterCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
return MongoDatasetData.aggregate(
|
||||
[
|
||||
{
|
||||
$match: {
|
||||
teamId: new Types.ObjectId(teamId),
|
||||
datasetId: new Types.ObjectId(id),
|
||||
$text: { $search: jiebaSplit({ text: query }) },
|
||||
...(filterCollectionIdList
|
||||
? {
|
||||
collectionId: {
|
||||
$in: filterCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
}
|
||||
}
|
||||
}
|
||||
: {}),
|
||||
...(forbidCollectionIdList && forbidCollectionIdList.length > 0
|
||||
? {
|
||||
collectionId: {
|
||||
$nin: forbidCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
: {}),
|
||||
...(forbidCollectionIdList && forbidCollectionIdList.length > 0
|
||||
? {
|
||||
collectionId: {
|
||||
$nin: forbidCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
}
|
||||
}
|
||||
}
|
||||
: {})
|
||||
: {})
|
||||
}
|
||||
},
|
||||
{
|
||||
$sort: {
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
},
|
||||
{
|
||||
$limit: limit
|
||||
},
|
||||
{
|
||||
$project: {
|
||||
_id: 1,
|
||||
datasetId: 1,
|
||||
collectionId: 1,
|
||||
updateTime: 1,
|
||||
q: 1,
|
||||
a: 1,
|
||||
chunkIndex: 1,
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
}
|
||||
},
|
||||
],
|
||||
{
|
||||
$addFields: {
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
},
|
||||
{
|
||||
$sort: {
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
},
|
||||
{
|
||||
$limit: limit
|
||||
},
|
||||
{
|
||||
$project: {
|
||||
_id: 1,
|
||||
datasetId: 1,
|
||||
collectionId: 1,
|
||||
updateTime: 1,
|
||||
q: 1,
|
||||
a: 1,
|
||||
chunkIndex: 1,
|
||||
score: 1
|
||||
}
|
||||
...readFromSecondary
|
||||
}
|
||||
]);
|
||||
);
|
||||
})
|
||||
)
|
||||
).flat() as (DatasetDataSchemaType & { score: number })[];
|
||||
|
||||
// resort
|
||||
searchResults.sort((a, b) => b.score - a.score);
|
||||
searchResults.slice(0, limit);
|
||||
|
||||
// Get data and collections
|
||||
const collections = await MongoDatasetCollection.find(
|
||||
{
|
||||
_id: { $in: searchResults.map((item) => item.collectionId) }
|
||||
},
|
||||
'_id name fileId rawLink'
|
||||
);
|
||||
'_id name fileId rawLink externalFileId externalFileUrl',
|
||||
{ ...readFromSecondary }
|
||||
).lean();
|
||||
|
||||
return {
|
||||
fullTextRecallResults: searchResults.map((item, index) => {
|
||||
const collection = collections.find((col) => String(col._id) === String(item.collectionId));
|
||||
return {
|
||||
id: String(item._id),
|
||||
datasetId: String(item.datasetId),
|
||||
collectionId: String(item.collectionId),
|
||||
updateTime: item.updateTime,
|
||||
...getCollectionSourceData(collection),
|
||||
q: item.q,
|
||||
a: item.a,
|
||||
chunkIndex: item.chunkIndex,
|
||||
indexes: item.indexes,
|
||||
score: [{ type: SearchScoreTypeEnum.fullText, value: item.score, index }]
|
||||
};
|
||||
}),
|
||||
fullTextRecallResults: searchResults
|
||||
.map((data, index) => {
|
||||
const collection = collections.find(
|
||||
(col) => String(col._id) === String(data.collectionId)
|
||||
);
|
||||
if (!collection) {
|
||||
console.log('Collection is not found', data);
|
||||
return;
|
||||
}
|
||||
|
||||
return {
|
||||
id: String(data._id),
|
||||
datasetId: String(data.datasetId),
|
||||
collectionId: String(data.collectionId),
|
||||
updateTime: data.updateTime,
|
||||
q: data.q,
|
||||
a: data.a,
|
||||
chunkIndex: data.chunkIndex,
|
||||
indexes: data.indexes,
|
||||
...getCollectionSourceData(collection),
|
||||
score: [{ type: SearchScoreTypeEnum.fullText, value: data.score ?? 0, index }]
|
||||
};
|
||||
})
|
||||
.filter(Boolean) as SearchDataResponseItemType[],
|
||||
tokenLen: 0
|
||||
};
|
||||
};
|
||||
const fullTextRecall2 = async ({
|
||||
query,
|
||||
limit,
|
||||
filterCollectionIdList,
|
||||
forbidCollectionIdList
|
||||
}: {
|
||||
query: string;
|
||||
limit: number;
|
||||
filterCollectionIdList?: string[];
|
||||
forbidCollectionIdList: string[];
|
||||
}): Promise<{
|
||||
fullTextRecallResults: SearchDataResponseItemType[];
|
||||
tokenLen: number;
|
||||
}> => {
|
||||
if (limit === 0) {
|
||||
return {
|
||||
fullTextRecallResults: [],
|
||||
tokenLen: 0
|
||||
};
|
||||
}
|
||||
|
||||
const searchResults = (
|
||||
await Promise.all(
|
||||
datasetIds.map(async (id) => {
|
||||
return MongoDatasetDataText.aggregate(
|
||||
[
|
||||
{
|
||||
$match: {
|
||||
teamId: new Types.ObjectId(teamId),
|
||||
datasetId: new Types.ObjectId(id),
|
||||
$text: { $search: jiebaSplit({ text: query }) },
|
||||
...(filterCollectionIdList
|
||||
? {
|
||||
collectionId: {
|
||||
$in: filterCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
}
|
||||
}
|
||||
: {}),
|
||||
...(forbidCollectionIdList && forbidCollectionIdList.length > 0
|
||||
? {
|
||||
collectionId: {
|
||||
$nin: forbidCollectionIdList.map((id) => new Types.ObjectId(id))
|
||||
}
|
||||
}
|
||||
: {})
|
||||
}
|
||||
},
|
||||
{
|
||||
$sort: {
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
},
|
||||
{
|
||||
$limit: limit
|
||||
},
|
||||
{
|
||||
$project: {
|
||||
_id: 1,
|
||||
collectionId: 1,
|
||||
dataId: 1,
|
||||
score: { $meta: 'textScore' }
|
||||
}
|
||||
}
|
||||
],
|
||||
{
|
||||
...readFromSecondary
|
||||
}
|
||||
);
|
||||
})
|
||||
)
|
||||
).flat() as (DatasetDataTextSchemaType & { score: number })[];
|
||||
|
||||
// Get data and collections
|
||||
const [dataList, collections] = await Promise.all([
|
||||
MongoDatasetData.find(
|
||||
{
|
||||
_id: { $in: searchResults.map((item) => item.dataId) }
|
||||
},
|
||||
'_id datasetId collectionId updateTime q a chunkIndex indexes',
|
||||
{ ...readFromSecondary }
|
||||
).lean(),
|
||||
MongoDatasetCollection.find(
|
||||
{
|
||||
_id: { $in: searchResults.map((item) => item.collectionId) }
|
||||
},
|
||||
'_id name fileId rawLink externalFileId externalFileUrl',
|
||||
{ ...readFromSecondary }
|
||||
).lean()
|
||||
]);
|
||||
|
||||
return {
|
||||
fullTextRecallResults: searchResults
|
||||
.map((item, index) => {
|
||||
const collection = collections.find(
|
||||
(col) => String(col._id) === String(item.collectionId)
|
||||
);
|
||||
if (!collection) {
|
||||
console.log('Collection is not found', item);
|
||||
return;
|
||||
}
|
||||
const data = dataList.find((data) => String(data._id) === String(item.dataId));
|
||||
if (!data) {
|
||||
console.log('Data is not found', item);
|
||||
return;
|
||||
}
|
||||
|
||||
return {
|
||||
id: String(data._id),
|
||||
datasetId: String(data.datasetId),
|
||||
collectionId: String(data.collectionId),
|
||||
updateTime: data.updateTime,
|
||||
q: data.q,
|
||||
a: data.a,
|
||||
chunkIndex: data.chunkIndex,
|
||||
indexes: data.indexes,
|
||||
...getCollectionSourceData(collection),
|
||||
score: [
|
||||
{
|
||||
type: SearchScoreTypeEnum.fullText,
|
||||
value: item.score || 0,
|
||||
index
|
||||
}
|
||||
]
|
||||
};
|
||||
})
|
||||
.filter(Boolean) as SearchDataResponseItemType[],
|
||||
tokenLen: 0
|
||||
};
|
||||
};
|
||||
@@ -496,7 +636,8 @@ export async function searchDatasetData(props: SearchDatasetDataProps) {
|
||||
forbidCollectionIdList,
|
||||
filterCollectionIdList
|
||||
}),
|
||||
fullTextRecall({
|
||||
// FullText tmp
|
||||
fullTextRecall2({
|
||||
query,
|
||||
limit: fullTextLimit,
|
||||
filterCollectionIdList,
|
||||
|
||||
@@ -210,7 +210,7 @@ export const runToolWithToolChoice = async (
|
||||
properties[item.key] = {
|
||||
...jsonSchema,
|
||||
description: item.toolDescription || '',
|
||||
enum: item.enum?.split('\n').filter(Boolean) || []
|
||||
enum: item.enum?.split('\n').filter(Boolean) || undefined
|
||||
};
|
||||
});
|
||||
|
||||
@@ -227,6 +227,7 @@ export const runToolWithToolChoice = async (
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
// Filter histories by maxToken
|
||||
const filterMessages = (
|
||||
await filterGPTMessageByMaxTokens({
|
||||
|
||||
@@ -487,16 +487,16 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
|
||||
if (input.key === dynamicInput?.key) return;
|
||||
|
||||
// Skip some special key
|
||||
if (input.key === NodeInputKeyEnum.childrenNodeIdList) {
|
||||
if (
|
||||
[NodeInputKeyEnum.childrenNodeIdList, NodeInputKeyEnum.httpJsonBody].includes(
|
||||
input.key as any
|
||||
)
|
||||
) {
|
||||
params[input.key] = input.value;
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
// replace {{xx}} variables
|
||||
// let value = replaceVariable(input.value, variables);
|
||||
|
||||
// replace {{$xx.xx$}} variables
|
||||
// replace {{$xx.xx$}} and {{xx}} variables
|
||||
let value = replaceEditorVariable({
|
||||
text: input.value,
|
||||
nodes: runtimeNodes,
|
||||
@@ -606,6 +606,11 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
|
||||
};
|
||||
}
|
||||
|
||||
// Error
|
||||
if (dispatchRes?.responseData?.error) {
|
||||
addLog.warn('workflow error', dispatchRes.responseData.error);
|
||||
}
|
||||
|
||||
return {
|
||||
node,
|
||||
runStatus: 'run',
|
||||
|
||||
@@ -2,6 +2,7 @@ import type { ModuleDispatchProps } from '@fastgpt/global/core/workflow/runtime/
|
||||
import {
|
||||
NodeInputKeyEnum,
|
||||
NodeOutputKeyEnum,
|
||||
VARIABLE_NODE_ID,
|
||||
WorkflowIOValueTypeEnum
|
||||
} from '@fastgpt/global/core/workflow/constants';
|
||||
import {
|
||||
@@ -16,7 +17,9 @@ import { DispatchNodeResultType } from '@fastgpt/global/core/workflow/runtime/ty
|
||||
import { getErrText } from '@fastgpt/global/common/error/utils';
|
||||
import {
|
||||
textAdaptGptResponse,
|
||||
replaceEditorVariable
|
||||
replaceEditorVariable,
|
||||
formatVariableValByType,
|
||||
getReferenceVariableValue
|
||||
} from '@fastgpt/global/core/workflow/runtime/utils';
|
||||
import { ContentTypes } from '@fastgpt/global/core/workflow/constants';
|
||||
import { uploadFileFromBase64Img } from '../../../../common/file/gridfs/controller';
|
||||
@@ -104,15 +107,73 @@ export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<H
|
||||
[NodeInputKeyEnum.addInputParam]: concatVariables,
|
||||
...concatVariables
|
||||
};
|
||||
|
||||
// General data for variable substitution(Exclude: json body)
|
||||
const replaceStringVariables = (text: string) => {
|
||||
return replaceVariable(
|
||||
replaceEditorVariable({
|
||||
text,
|
||||
nodes: runtimeNodes,
|
||||
variables: allVariables
|
||||
}),
|
||||
allVariables
|
||||
);
|
||||
return replaceEditorVariable({
|
||||
text,
|
||||
nodes: runtimeNodes,
|
||||
variables: allVariables
|
||||
});
|
||||
};
|
||||
/* Replace the JSON string to reduce parsing errors
|
||||
1. Replace undefined values with null
|
||||
2. Replace newline strings
|
||||
*/
|
||||
const replaceJsonBodyString = (text: string) => {
|
||||
const valToStr = (val: any) => {
|
||||
if (val === undefined) return 'null';
|
||||
if (val === null) return 'null';
|
||||
|
||||
if (typeof val === 'object') return JSON.stringify(val);
|
||||
|
||||
if (typeof val === 'string') {
|
||||
const str = JSON.stringify(val);
|
||||
return str.startsWith('"') && str.endsWith('"') ? str.slice(1, -1) : str;
|
||||
}
|
||||
|
||||
return String(val);
|
||||
};
|
||||
|
||||
// 1. Replace {{key}} variables
|
||||
const regex1 = /{{([^}]+)}}/g;
|
||||
const matches1 = text.match(regex1) || [];
|
||||
const uniqueKeys1 = [...new Set(matches1.map((match) => match.slice(2, -2)))];
|
||||
for (const key of uniqueKeys1) {
|
||||
text = text.replace(new RegExp(`{{(${key})}}`, 'g'), () => valToStr(variables[key]));
|
||||
}
|
||||
|
||||
// 2. Replace {{key.key}} variables
|
||||
const regex2 = /\{\{\$([^.]+)\.([^$]+)\$\}\}/g;
|
||||
const matches2 = [...text.matchAll(regex2)];
|
||||
if (matches2.length === 0) return text;
|
||||
matches2.forEach((match) => {
|
||||
const nodeId = match[1];
|
||||
const id = match[2];
|
||||
|
||||
const variableVal = (() => {
|
||||
if (nodeId === VARIABLE_NODE_ID) {
|
||||
return variables[id];
|
||||
}
|
||||
// Find upstream node input/output
|
||||
const node = runtimeNodes.find((node) => node.nodeId === nodeId);
|
||||
if (!node) return;
|
||||
|
||||
const output = node.outputs.find((output) => output.id === id);
|
||||
if (output) return formatVariableValByType(output.value, output.valueType);
|
||||
|
||||
const input = node.inputs.find((input) => input.key === id);
|
||||
if (input)
|
||||
return getReferenceVariableValue({ value: input.value, nodes: runtimeNodes, variables });
|
||||
})();
|
||||
|
||||
const formatVal = valToStr(variableVal);
|
||||
|
||||
const regex = new RegExp(`\\{\\{\\$(${nodeId}\\.${id})\\$\\}\\}`, 'g');
|
||||
text = text.replace(regex, () => formatVal);
|
||||
});
|
||||
|
||||
return text.replace(/(".*?")\s*:\s*undefined\b/g, '$1: null');
|
||||
};
|
||||
|
||||
httpReqUrl = replaceStringVariables(httpReqUrl);
|
||||
@@ -176,15 +237,10 @@ export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<H
|
||||
}
|
||||
if (!httpJsonBody) return {};
|
||||
if (httpContentType === ContentTypes.json) {
|
||||
httpJsonBody = replaceStringVariables(httpJsonBody);
|
||||
|
||||
const replaceJsonBody = httpJsonBody.replace(/(".*?")\s*:\s*undefined\b/g, '$1: null');
|
||||
|
||||
// Json body, parse and return
|
||||
const jsonParse = json5.parse(replaceJsonBody);
|
||||
const removeSignJson = removeUndefinedSign(jsonParse);
|
||||
return removeSignJson;
|
||||
return json5.parse(replaceJsonBodyString(httpJsonBody));
|
||||
}
|
||||
|
||||
// Raw text, xml
|
||||
httpJsonBody = replaceStringVariables(httpJsonBody);
|
||||
return httpJsonBody.replaceAll(UNDEFINED_SIGN, 'null');
|
||||
} catch (error) {
|
||||
@@ -335,40 +391,40 @@ async function fetchData({
|
||||
};
|
||||
}
|
||||
|
||||
function replaceVariable(text: string, obj: Record<string, any>) {
|
||||
for (const [key, value] of Object.entries(obj)) {
|
||||
if (value === undefined) {
|
||||
text = text.replace(new RegExp(`{{(${key})}}`, 'g'), UNDEFINED_SIGN);
|
||||
} else {
|
||||
const replacement = JSON.stringify(value);
|
||||
const unquotedReplacement =
|
||||
replacement.startsWith('"') && replacement.endsWith('"')
|
||||
? replacement.slice(1, -1)
|
||||
: replacement;
|
||||
text = text.replace(new RegExp(`{{(${key})}}`, 'g'), () => unquotedReplacement);
|
||||
}
|
||||
}
|
||||
return text || '';
|
||||
}
|
||||
function removeUndefinedSign(obj: Record<string, any>) {
|
||||
for (const key in obj) {
|
||||
if (obj[key] === UNDEFINED_SIGN) {
|
||||
obj[key] = undefined;
|
||||
} else if (Array.isArray(obj[key])) {
|
||||
obj[key] = obj[key].map((item: any) => {
|
||||
if (item === UNDEFINED_SIGN) {
|
||||
return undefined;
|
||||
} else if (typeof item === 'object') {
|
||||
removeUndefinedSign(item);
|
||||
}
|
||||
return item;
|
||||
});
|
||||
} else if (typeof obj[key] === 'object') {
|
||||
removeUndefinedSign(obj[key]);
|
||||
}
|
||||
}
|
||||
return obj;
|
||||
}
|
||||
// function replaceVariable(text: string, obj: Record<string, any>) {
|
||||
// for (const [key, value] of Object.entries(obj)) {
|
||||
// if (value === undefined) {
|
||||
// text = text.replace(new RegExp(`{{(${key})}}`, 'g'), UNDEFINED_SIGN);
|
||||
// } else {
|
||||
// const replacement = JSON.stringify(value);
|
||||
// const unquotedReplacement =
|
||||
// replacement.startsWith('"') && replacement.endsWith('"')
|
||||
// ? replacement.slice(1, -1)
|
||||
// : replacement;
|
||||
// text = text.replace(new RegExp(`{{(${key})}}`, 'g'), () => unquotedReplacement);
|
||||
// }
|
||||
// }
|
||||
// return text || '';
|
||||
// }
|
||||
// function removeUndefinedSign(obj: Record<string, any>) {
|
||||
// for (const key in obj) {
|
||||
// if (obj[key] === UNDEFINED_SIGN) {
|
||||
// obj[key] = undefined;
|
||||
// } else if (Array.isArray(obj[key])) {
|
||||
// obj[key] = obj[key].map((item: any) => {
|
||||
// if (item === UNDEFINED_SIGN) {
|
||||
// return undefined;
|
||||
// } else if (typeof item === 'object') {
|
||||
// removeUndefinedSign(item);
|
||||
// }
|
||||
// return item;
|
||||
// });
|
||||
// } else if (typeof obj[key] === 'object') {
|
||||
// removeUndefinedSign(obj[key]);
|
||||
// }
|
||||
// }
|
||||
// return obj;
|
||||
// }
|
||||
|
||||
// Replace some special response from system plugin
|
||||
async function replaceSystemPluginResponse({
|
||||
|
||||
@@ -92,6 +92,7 @@ OutLinkSchema.virtual('associatedApp', {
|
||||
|
||||
try {
|
||||
OutLinkSchema.index({ shareId: -1 });
|
||||
OutLinkSchema.index({ teamId: 1, tmbId: 1, appId: 1 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
43
packages/service/support/permission/auth/org.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
import { TeamPermission } from '@fastgpt/global/support/permission/user/controller';
|
||||
import { AuthModeType, AuthResponseType } from '../type';
|
||||
import { TeamErrEnum } from '@fastgpt/global/common/error/code/team';
|
||||
import { authUserPer } from '../user/auth';
|
||||
import { ManagePermissionVal } from '@fastgpt/global/support/permission/constant';
|
||||
|
||||
/*
|
||||
Team manager can control org
|
||||
*/
|
||||
export const authOrgMember = async ({
|
||||
orgIds,
|
||||
...props
|
||||
}: {
|
||||
orgIds: string | string[];
|
||||
} & AuthModeType): Promise<AuthResponseType> => {
|
||||
const result = await authUserPer({
|
||||
...props,
|
||||
per: ManagePermissionVal
|
||||
});
|
||||
const { teamId, tmbId, isRoot, tmb } = result;
|
||||
|
||||
if (isRoot) {
|
||||
return {
|
||||
teamId,
|
||||
tmbId,
|
||||
userId: result.userId,
|
||||
appId: result.appId,
|
||||
apikey: result.apikey,
|
||||
isRoot,
|
||||
authType: result.authType,
|
||||
permission: new TeamPermission({ isOwner: true })
|
||||
};
|
||||
}
|
||||
|
||||
if (tmb.permission.hasManagePer) {
|
||||
return {
|
||||
...result,
|
||||
permission: tmb.permission
|
||||
};
|
||||
}
|
||||
|
||||
return Promise.reject(TeamErrEnum.unAuthTeam);
|
||||
};
|
||||
@@ -1,8 +1,8 @@
|
||||
import { MongoTeamMember } from '../../user/team/teamMemberSchema';
|
||||
import { checkTeamAIPoints } from '../teamLimit';
|
||||
import { UserErrEnum } from '@fastgpt/global/common/error/code/user';
|
||||
import { UserModelSchema } from '@fastgpt/global/support/user/type';
|
||||
import { TeamSchema } from '@fastgpt/global/support/user/team/type';
|
||||
import { TeamErrEnum } from '@fastgpt/global/common/error/code/team';
|
||||
|
||||
export async function getUserChatInfoAndAuthTeamPoints(tmbId: string) {
|
||||
const tmb = await MongoTeamMember.findById(tmbId, 'userId teamId')
|
||||
@@ -18,7 +18,7 @@ export async function getUserChatInfoAndAuthTeamPoints(tmbId: string) {
|
||||
])
|
||||
.lean();
|
||||
|
||||
if (!tmb) return Promise.reject(UserErrEnum.unAuthUser);
|
||||
if (!tmb) return Promise.reject(TeamErrEnum.notUser);
|
||||
|
||||
await checkTeamAIPoints(tmb.team._id);
|
||||
|
||||
|
||||
@@ -8,10 +8,7 @@ import { authOpenApiKey } from '../openapi/auth';
|
||||
import { FileTokenQuery } from '@fastgpt/global/common/file/type';
|
||||
import { MongoResourcePermission } from './schema';
|
||||
import { ClientSession } from 'mongoose';
|
||||
import {
|
||||
PermissionValueType,
|
||||
ResourcePermissionType
|
||||
} from '@fastgpt/global/support/permission/type';
|
||||
import { PermissionValueType } from '@fastgpt/global/support/permission/type';
|
||||
import { bucketNameMap } from '@fastgpt/global/common/file/constants';
|
||||
import { addMinutes } from 'date-fns';
|
||||
import { getGroupsByTmbId } from './memberGroup/controllers';
|
||||
@@ -21,6 +18,8 @@ import { CommonErrEnum } from '@fastgpt/global/common/error/code/common';
|
||||
import { MemberGroupSchemaType } from '@fastgpt/global/support/permission/memberGroup/type';
|
||||
import { TeamMemberSchema } from '@fastgpt/global/support/user/team/type';
|
||||
import { UserModelSchema } from '@fastgpt/global/support/user/type';
|
||||
import { OrgSchemaType } from '@fastgpt/global/support/user/team/org/type';
|
||||
import { getOrgIdSetWithParentByTmbId } from './org/controllers';
|
||||
|
||||
/** get resource permission for a team member
|
||||
* If there is no permission for the team member, it will return undefined
|
||||
@@ -67,67 +66,44 @@ export const getResourcePermission = async ({
|
||||
}
|
||||
|
||||
// If there is no personal permission, get the group permission
|
||||
const groupIdList = (await getGroupsByTmbId({ tmbId, teamId })).map((item) => item._id);
|
||||
const [groupPers, orgPers] = await Promise.all([
|
||||
getGroupsByTmbId({ tmbId, teamId })
|
||||
.then((res) => res.map((item) => item._id))
|
||||
.then((groupIdList) =>
|
||||
MongoResourcePermission.find(
|
||||
{
|
||||
teamId,
|
||||
resourceType,
|
||||
groupId: {
|
||||
$in: groupIdList
|
||||
},
|
||||
resourceId
|
||||
},
|
||||
'permission'
|
||||
).lean()
|
||||
)
|
||||
.then((perList) => perList.map((item) => item.permission)),
|
||||
getOrgIdSetWithParentByTmbId({ tmbId, teamId })
|
||||
.then((item) => Array.from(item))
|
||||
.then((orgIds) =>
|
||||
MongoResourcePermission.find(
|
||||
{
|
||||
teamId,
|
||||
resourceType,
|
||||
orgId: {
|
||||
$in: Array.from(orgIds)
|
||||
},
|
||||
resourceId
|
||||
},
|
||||
'permission'
|
||||
).lean()
|
||||
)
|
||||
.then((perList) => perList.map((item) => item.permission))
|
||||
]);
|
||||
|
||||
if (groupIdList.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
// get the maximum permission of the group
|
||||
const pers = (
|
||||
await MongoResourcePermission.find(
|
||||
{
|
||||
teamId,
|
||||
resourceType,
|
||||
groupId: {
|
||||
$in: groupIdList
|
||||
},
|
||||
resourceId
|
||||
},
|
||||
'permission'
|
||||
).lean()
|
||||
).map((item) => item.permission);
|
||||
|
||||
const groupPer = getGroupPer(pers);
|
||||
|
||||
return groupPer;
|
||||
return concatPer([...groupPers, ...orgPers]);
|
||||
};
|
||||
|
||||
/* 仅取 members 不取 groups */
|
||||
export async function getResourceAllClbs({
|
||||
resourceId,
|
||||
teamId,
|
||||
resourceType,
|
||||
session
|
||||
}: {
|
||||
teamId: string;
|
||||
session?: ClientSession;
|
||||
} & (
|
||||
| {
|
||||
resourceType: 'team';
|
||||
resourceId?: undefined;
|
||||
}
|
||||
| {
|
||||
resourceType: Omit<PerResourceTypeEnum, 'team'>;
|
||||
resourceId?: string | null;
|
||||
}
|
||||
)): Promise<ResourcePermissionType[]> {
|
||||
return MongoResourcePermission.find(
|
||||
{
|
||||
resourceType: resourceType,
|
||||
teamId: teamId,
|
||||
resourceId,
|
||||
groupId: {
|
||||
$exists: false
|
||||
}
|
||||
},
|
||||
null,
|
||||
{
|
||||
session
|
||||
}
|
||||
).lean();
|
||||
}
|
||||
|
||||
export async function getResourceClbsAndGroups({
|
||||
resourceId,
|
||||
resourceType,
|
||||
@@ -155,10 +131,17 @@ export const getClbsAndGroupsWithInfo = async ({
|
||||
resourceType,
|
||||
teamId
|
||||
}: {
|
||||
resourceId: ParentIdType;
|
||||
resourceType: Omit<`${PerResourceTypeEnum}`, 'team'>;
|
||||
teamId: string;
|
||||
}) =>
|
||||
} & (
|
||||
| {
|
||||
resourceId: ParentIdType;
|
||||
resourceType: Omit<`${PerResourceTypeEnum}`, 'team'>;
|
||||
}
|
||||
| {
|
||||
resourceType: 'team';
|
||||
resourceId?: undefined;
|
||||
}
|
||||
)) =>
|
||||
Promise.all([
|
||||
MongoResourcePermission.find({
|
||||
teamId,
|
||||
@@ -170,7 +153,7 @@ export const getClbsAndGroupsWithInfo = async ({
|
||||
})
|
||||
.populate<{ tmb: TeamMemberSchema & { user: UserModelSchema } }>({
|
||||
path: 'tmb',
|
||||
select: 'name userId',
|
||||
select: 'name userId role',
|
||||
populate: {
|
||||
path: 'user',
|
||||
select: 'avatar'
|
||||
@@ -186,6 +169,16 @@ export const getClbsAndGroupsWithInfo = async ({
|
||||
}
|
||||
})
|
||||
.populate<{ group: MemberGroupSchemaType }>('group', 'name avatar')
|
||||
.lean(),
|
||||
MongoResourcePermission.find({
|
||||
teamId,
|
||||
resourceId,
|
||||
resourceType,
|
||||
orgId: {
|
||||
$exists: true
|
||||
}
|
||||
})
|
||||
.populate<{ org: OrgSchemaType }>({ path: 'org', select: 'name avatar' })
|
||||
.lean()
|
||||
]);
|
||||
|
||||
@@ -196,6 +189,7 @@ export const delResourcePermission = ({
|
||||
session,
|
||||
tmbId,
|
||||
groupId,
|
||||
orgId,
|
||||
...props
|
||||
}: {
|
||||
resourceType: PerResourceTypeEnum;
|
||||
@@ -204,15 +198,18 @@ export const delResourcePermission = ({
|
||||
session?: ClientSession;
|
||||
tmbId?: string;
|
||||
groupId?: string;
|
||||
orgId?: string;
|
||||
}) => {
|
||||
// tmbId or groupId only one and not both
|
||||
if (!!tmbId === !!groupId) {
|
||||
// either tmbId or groupId or orgId must be provided
|
||||
if (!tmbId && !groupId && !orgId) {
|
||||
return Promise.reject(CommonErrEnum.missingParams);
|
||||
}
|
||||
|
||||
return MongoResourcePermission.deleteOne(
|
||||
{
|
||||
...(tmbId ? { tmbId } : {}),
|
||||
...(groupId ? { groupId } : {}),
|
||||
...(orgId ? { orgId } : {}),
|
||||
...props
|
||||
},
|
||||
{ session }
|
||||
@@ -250,7 +247,7 @@ export function authJWT(token: string) {
|
||||
}>((resolve, reject) => {
|
||||
const key = process.env.TOKEN_KEY as string;
|
||||
|
||||
jwt.verify(token, key, function (err, decoded: any) {
|
||||
jwt.verify(token, key, (err, decoded: any) => {
|
||||
if (err || !decoded?.userId) {
|
||||
reject(ERROR_ENUM.unAuthorization);
|
||||
return;
|
||||
@@ -436,7 +433,7 @@ export const authFileToken = (token?: string) =>
|
||||
}
|
||||
const key = (process.env.FILE_TOKEN_KEY as string) ?? 'filetoken';
|
||||
|
||||
jwt.verify(token, key, function (err, decoded: any) {
|
||||
jwt.verify(token, key, (err, decoded: any) => {
|
||||
if (err || !decoded.bucketName || !decoded?.teamId || !decoded?.fileId) {
|
||||
reject(ERROR_ENUM.unAuthFile);
|
||||
return;
|
||||
@@ -450,10 +447,10 @@ export const authFileToken = (token?: string) =>
|
||||
});
|
||||
});
|
||||
|
||||
export const getGroupPer = (groups: PermissionValueType[] = []) => {
|
||||
if (groups.length === 0) {
|
||||
export const concatPer = (perList: PermissionValueType[] = []) => {
|
||||
if (perList.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return new Permission().addPer(...groups).value;
|
||||
return new Permission().addPer(...perList).value;
|
||||
};
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import { mongoSessionRun } from '../../common/mongo/sessionRun';
|
||||
import { MongoResourcePermission } from './schema';
|
||||
import { ClientSession, Model } from 'mongoose';
|
||||
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
|
||||
import { PermissionValueType } from '@fastgpt/global/support/permission/type';
|
||||
import type { ClientSession, Model } from 'mongoose';
|
||||
import type { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
|
||||
import type { PermissionValueType } from '@fastgpt/global/support/permission/type';
|
||||
import { getResourceClbsAndGroups } from './controller';
|
||||
import { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
|
||||
import { ParentIdType } from '@fastgpt/global/common/parentFolder/type';
|
||||
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
|
||||
import type { ParentIdType } from '@fastgpt/global/common/parentFolder/type';
|
||||
|
||||
export type SyncChildrenPermissionResourceType = {
|
||||
_id: string;
|
||||
@@ -18,6 +18,7 @@ export type UpdateCollaboratorItem = {
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
|
||||
// sync the permission to all children folders.
|
||||
@@ -161,7 +162,7 @@ export async function resumeInheritPermission({
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
/*
|
||||
Delete all the collaborators and then insert the new collaborators.
|
||||
*/
|
||||
export async function syncCollaborators({
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
import { MemberGroupSchemaType } from '@fastgpt/global/support/permission/memberGroup/type';
|
||||
import { MongoGroupMemberModel } from './groupMemberSchema';
|
||||
import { TeamMemberSchema } from '@fastgpt/global/support/user/team/type';
|
||||
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
|
||||
import { MongoResourcePermission } from '../schema';
|
||||
import { getGroupPer, parseHeaderCert } from '../controller';
|
||||
import { parseHeaderCert } from '../controller';
|
||||
import { MongoMemberGroupModel } from './memberGroupSchema';
|
||||
import { DefaultGroupName } from '@fastgpt/global/support/user/team/group/constant';
|
||||
import { ClientSession } from 'mongoose';
|
||||
@@ -50,26 +47,32 @@ export const getTeamDefaultGroup = async ({
|
||||
export const getGroupsByTmbId = async ({
|
||||
tmbId,
|
||||
teamId,
|
||||
role
|
||||
role,
|
||||
session
|
||||
}: {
|
||||
tmbId: string;
|
||||
teamId: string;
|
||||
role?: `${GroupMemberRole}`[];
|
||||
session?: ClientSession;
|
||||
}) =>
|
||||
(
|
||||
await Promise.all([
|
||||
(
|
||||
await MongoGroupMemberModel.find({
|
||||
tmbId,
|
||||
groupId: {
|
||||
$exists: true
|
||||
await MongoGroupMemberModel.find(
|
||||
{
|
||||
tmbId,
|
||||
groupId: {
|
||||
$exists: true
|
||||
},
|
||||
...(role ? { role: { $in: role } } : {})
|
||||
},
|
||||
...(role ? { role: { $in: role } } : {})
|
||||
})
|
||||
undefined,
|
||||
{ session }
|
||||
)
|
||||
.populate<{ group: MemberGroupSchemaType }>('group')
|
||||
.lean()
|
||||
).map((item) => item.group),
|
||||
role ? [] : getTeamDefaultGroup({ teamId })
|
||||
role ? [] : getTeamDefaultGroup({ teamId, session })
|
||||
])
|
||||
).flat();
|
||||
|
||||
@@ -79,46 +82,6 @@ export const getGroupMembersByGroupId = async (groupId: string) => {
|
||||
}).lean();
|
||||
};
|
||||
|
||||
/**
|
||||
* Get tmb's group permission: the maximum permission of the group
|
||||
* @param tmbId
|
||||
* @param resourceId
|
||||
* @param resourceType
|
||||
* @returns the maximum permission of the group
|
||||
*/
|
||||
export const getGroupPermission = async ({
|
||||
tmbId,
|
||||
resourceId,
|
||||
teamId,
|
||||
resourceType
|
||||
}: {
|
||||
tmbId: string;
|
||||
teamId: string;
|
||||
} & (
|
||||
| {
|
||||
resourceId?: undefined;
|
||||
resourceType: 'team';
|
||||
}
|
||||
| {
|
||||
resourceId: string;
|
||||
resourceType: Omit<PerResourceTypeEnum, 'team'>;
|
||||
}
|
||||
)) => {
|
||||
const groupIds = (await getGroupsByTmbId({ tmbId, teamId })).map((item) => item._id);
|
||||
const groupPermissions = (
|
||||
await MongoResourcePermission.find({
|
||||
groupId: {
|
||||
$in: groupIds
|
||||
},
|
||||
resourceType,
|
||||
resourceId,
|
||||
teamId
|
||||
})
|
||||
).map((item) => item.permission);
|
||||
|
||||
return getGroupPer(groupPermissions);
|
||||
};
|
||||
|
||||
// auth group member role
|
||||
export const authGroupMemberRole = async ({
|
||||
groupId,
|
||||
@@ -140,8 +103,12 @@ export const authGroupMemberRole = async ({
|
||||
tmbId
|
||||
};
|
||||
}
|
||||
const groupMember = await MongoGroupMemberModel.findOne({ groupId, tmbId });
|
||||
const tmb = await getTmbInfoByTmbId({ tmbId });
|
||||
const [groupMember, tmb] = await Promise.all([
|
||||
MongoGroupMemberModel.findOne({ groupId, tmbId }),
|
||||
getTmbInfoByTmbId({ tmbId })
|
||||
]);
|
||||
|
||||
// Team admin or role check
|
||||
if (tmb.permission.hasManagePer || (groupMember && role.includes(groupMember.role))) {
|
||||
return {
|
||||
...result,
|
||||
|
||||
95
packages/service/support/permission/org/controllers.ts
Normal file
@@ -0,0 +1,95 @@
|
||||
import { TeamErrEnum } from '@fastgpt/global/common/error/code/team';
|
||||
import type { OrgSchemaType } from '@fastgpt/global/support/user/team/org/type';
|
||||
import type { ClientSession } from 'mongoose';
|
||||
import { MongoOrgModel } from './orgSchema';
|
||||
import { MongoOrgMemberModel } from './orgMemberSchema';
|
||||
import { getOrgChildrenPath } from '@fastgpt/global/support/user/team/org/constant';
|
||||
|
||||
export const getOrgsByTmbId = async ({ teamId, tmbId }: { teamId: string; tmbId: string }) =>
|
||||
MongoOrgMemberModel.find({ teamId, tmbId }, 'orgId').lean();
|
||||
|
||||
export const getOrgIdSetWithParentByTmbId = async ({
|
||||
teamId,
|
||||
tmbId
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
}) => {
|
||||
const orgMembers = await MongoOrgMemberModel.find({ teamId, tmbId }, 'orgId').lean();
|
||||
|
||||
const orgIds = Array.from(new Set(orgMembers.map((item) => String(item.orgId))));
|
||||
const orgs = await MongoOrgModel.find({ _id: { $in: orgIds } }, 'path').lean();
|
||||
|
||||
const pathIdList = new Set<string>(
|
||||
orgs
|
||||
.map((org) => {
|
||||
const pathIdList = org.path.split('/').filter(Boolean);
|
||||
return pathIdList;
|
||||
})
|
||||
.flat()
|
||||
);
|
||||
const parentOrgs = await MongoOrgModel.find(
|
||||
{
|
||||
teamId,
|
||||
pathId: { $in: Array.from(pathIdList) }
|
||||
},
|
||||
'_id'
|
||||
).lean();
|
||||
const parentOrgIds = parentOrgs.map((item) => String(item._id));
|
||||
|
||||
return new Set([...orgIds, ...parentOrgIds]);
|
||||
};
|
||||
|
||||
export const getChildrenByOrg = async ({
|
||||
org,
|
||||
teamId,
|
||||
session
|
||||
}: {
|
||||
org: OrgSchemaType;
|
||||
teamId: string;
|
||||
session?: ClientSession;
|
||||
}) => {
|
||||
return MongoOrgModel.find(
|
||||
{ teamId, path: { $regex: `^${getOrgChildrenPath(org)}` } },
|
||||
undefined,
|
||||
{
|
||||
session
|
||||
}
|
||||
).lean();
|
||||
};
|
||||
|
||||
export const getOrgAndChildren = async ({
|
||||
orgId,
|
||||
teamId,
|
||||
session
|
||||
}: {
|
||||
orgId: string;
|
||||
teamId: string;
|
||||
session?: ClientSession;
|
||||
}) => {
|
||||
const org = await MongoOrgModel.findOne({ _id: orgId, teamId }, undefined, { session }).lean();
|
||||
if (!org) {
|
||||
return Promise.reject(TeamErrEnum.orgNotExist);
|
||||
}
|
||||
const children = await getChildrenByOrg({ org, teamId, session });
|
||||
return { org, children };
|
||||
};
|
||||
|
||||
export async function createRootOrg({
|
||||
teamId,
|
||||
session
|
||||
}: {
|
||||
teamId: string;
|
||||
session?: ClientSession;
|
||||
}) {
|
||||
return MongoOrgModel.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
name: 'ROOT',
|
||||
path: ''
|
||||
}
|
||||
],
|
||||
{ session }
|
||||
);
|
||||
}
|
||||
65
packages/service/support/permission/org/orgMemberSchema.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
import { OrgCollectionName } from '@fastgpt/global/support/user/team/org/constant';
|
||||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
import {
|
||||
TeamCollectionName,
|
||||
TeamMemberCollectionName
|
||||
} from '@fastgpt/global/support/user/team/constant';
|
||||
import { OrgMemberSchemaType } from '@fastgpt/global/support/user/team/org/type';
|
||||
const { Schema } = connectionMongo;
|
||||
|
||||
export const OrgMemberCollectionName = 'team_org_members';
|
||||
|
||||
export const OrgMemberSchema = new Schema({
|
||||
teamId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamCollectionName,
|
||||
required: true
|
||||
},
|
||||
orgId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: OrgCollectionName,
|
||||
required: true
|
||||
},
|
||||
tmbId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamMemberCollectionName,
|
||||
required: true
|
||||
}
|
||||
// role: {
|
||||
// type: String,
|
||||
// enum: Object.values(OrgMemberRole),
|
||||
// required: true,
|
||||
// default: OrgMemberRole.member
|
||||
// }
|
||||
});
|
||||
|
||||
OrgMemberSchema.virtual('org', {
|
||||
ref: OrgCollectionName,
|
||||
localField: 'orgId',
|
||||
foreignField: '_id',
|
||||
justOne: true
|
||||
});
|
||||
|
||||
try {
|
||||
OrgMemberSchema.index(
|
||||
{
|
||||
teamId: 1,
|
||||
orgId: 1,
|
||||
tmbId: 1
|
||||
},
|
||||
{
|
||||
unique: true
|
||||
}
|
||||
);
|
||||
OrgMemberSchema.index({
|
||||
teamId: 1,
|
||||
tmbId: 1
|
||||
});
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
export const MongoOrgMemberModel = getMongoModel<OrgMemberSchemaType>(
|
||||
OrgMemberCollectionName,
|
||||
OrgMemberSchema
|
||||
);
|
||||
77
packages/service/support/permission/org/orgSchema.ts
Normal file
@@ -0,0 +1,77 @@
|
||||
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
|
||||
import { OrgCollectionName } from '@fastgpt/global/support/user/team/org/constant';
|
||||
import type { OrgSchemaType } from '@fastgpt/global/support/user/team/org/type';
|
||||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
import { OrgMemberCollectionName } from './orgMemberSchema';
|
||||
import { getNanoid } from '@fastgpt/global/common/string/tools';
|
||||
const { Schema } = connectionMongo;
|
||||
|
||||
export const OrgSchema = new Schema(
|
||||
{
|
||||
teamId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamCollectionName,
|
||||
required: true
|
||||
},
|
||||
pathId: {
|
||||
// path id, only used for path
|
||||
type: String,
|
||||
required: true,
|
||||
default: () => getNanoid()
|
||||
},
|
||||
path: {
|
||||
type: String,
|
||||
required: function (this: OrgSchemaType) {
|
||||
return typeof this.path !== 'string';
|
||||
} // allow empty string, but not null
|
||||
},
|
||||
name: {
|
||||
type: String,
|
||||
required: true
|
||||
},
|
||||
avatar: String,
|
||||
description: String,
|
||||
updateTime: {
|
||||
type: Date,
|
||||
default: () => new Date()
|
||||
}
|
||||
},
|
||||
{
|
||||
// Auto update updateTime
|
||||
timestamps: {
|
||||
updatedAt: 'updateTime'
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
OrgSchema.virtual('members', {
|
||||
ref: OrgMemberCollectionName,
|
||||
localField: '_id',
|
||||
foreignField: 'orgId'
|
||||
});
|
||||
// OrgSchema.virtual('permission', {
|
||||
// ref: ResourcePermissionCollectionName,
|
||||
// localField: '_id',
|
||||
// foreignField: 'orgId',
|
||||
// justOne: true
|
||||
// });
|
||||
|
||||
try {
|
||||
OrgSchema.index({
|
||||
teamId: 1,
|
||||
path: 1
|
||||
});
|
||||
OrgSchema.index(
|
||||
{
|
||||
teamId: 1,
|
||||
pathId: 1
|
||||
},
|
||||
{
|
||||
unique: true
|
||||
}
|
||||
);
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
export const MongoOrgModel = getMongoModel<OrgSchemaType>(OrgCollectionName, OrgSchema);
|
||||
@@ -6,6 +6,7 @@ import { connectionMongo, getMongoModel } from '../../common/mongo';
|
||||
import type { ResourcePermissionType } from '@fastgpt/global/support/permission/type';
|
||||
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
|
||||
import { MemberGroupCollectionName } from './memberGroup/memberGroupSchema';
|
||||
import { OrgCollectionName } from '@fastgpt/global/support/user/team/org/constant';
|
||||
const { Schema } = connectionMongo;
|
||||
|
||||
export const ResourcePermissionCollectionName = 'resource_permissions';
|
||||
@@ -23,6 +24,10 @@ export const ResourcePermissionSchema = new Schema({
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: MemberGroupCollectionName
|
||||
},
|
||||
orgId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: OrgCollectionName
|
||||
},
|
||||
resourceType: {
|
||||
type: String,
|
||||
enum: Object.values(PerResourceTypeEnum),
|
||||
@@ -51,6 +56,12 @@ ResourcePermissionSchema.virtual('group', {
|
||||
foreignField: '_id',
|
||||
justOne: true
|
||||
});
|
||||
ResourcePermissionSchema.virtual('org', {
|
||||
ref: OrgCollectionName,
|
||||
localField: 'orgId',
|
||||
foreignField: '_id',
|
||||
justOne: true
|
||||
});
|
||||
|
||||
try {
|
||||
ResourcePermissionSchema.index(
|
||||
@@ -70,6 +81,23 @@ try {
|
||||
}
|
||||
);
|
||||
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceId: 1,
|
||||
orgId: 1
|
||||
},
|
||||
{
|
||||
unique: true,
|
||||
partialFilterExpression: {
|
||||
orgId: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
@@ -87,6 +115,7 @@ try {
|
||||
}
|
||||
);
|
||||
|
||||
// Delete tmb permission
|
||||
ResourcePermissionSchema.index({
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
|
||||
@@ -19,9 +19,7 @@ export const checkDatasetLimit = async ({
|
||||
if (!standardConstants) return;
|
||||
|
||||
if (usedDatasetSize + insertLen >= datasetMaxSize) {
|
||||
return Promise.reject(
|
||||
`您的知识库容量为: ${datasetMaxSize}组,已使用: ${usedDatasetSize}组,导入当前文件需要: ${insertLen}组,请增加知识库容量后导入。`
|
||||
);
|
||||
return Promise.reject(TeamErrEnum.datasetSizeNotEnough);
|
||||
}
|
||||
|
||||
if (usedPoints >= totalPoints) {
|
||||
|
||||
@@ -3,22 +3,10 @@ const { Schema } = connectionMongo;
|
||||
import { hashStr } from '@fastgpt/global/common/string/tools';
|
||||
import type { UserModelSchema } from '@fastgpt/global/support/user/type';
|
||||
import { UserStatusEnum, userStatusMap } from '@fastgpt/global/support/user/constant';
|
||||
import { getRandomUserAvatar } from '@fastgpt/global/support/user/utils';
|
||||
|
||||
export const userCollectionName = 'users';
|
||||
|
||||
const defaultAvatars = [
|
||||
'/imgs/avatar/RoyalBlueAvatar.svg',
|
||||
'/imgs/avatar/PurpleAvatar.svg',
|
||||
'/imgs/avatar/AdoraAvatar.svg',
|
||||
'/imgs/avatar/OrangeAvatar.svg',
|
||||
'/imgs/avatar/RedAvatar.svg',
|
||||
'/imgs/avatar/GrayModernAvatar.svg',
|
||||
'/imgs/avatar/TealAvatar.svg',
|
||||
'/imgs/avatar/GreenAvatar.svg',
|
||||
'/imgs/avatar/BrightBlueAvatar.svg',
|
||||
'/imgs/avatar/BlueAvatar.svg'
|
||||
];
|
||||
|
||||
const UserSchema = new Schema({
|
||||
status: {
|
||||
type: String,
|
||||
@@ -47,7 +35,7 @@ const UserSchema = new Schema({
|
||||
},
|
||||
avatar: {
|
||||
type: String,
|
||||
default: defaultAvatars[Math.floor(Math.random() * defaultAvatars.length)]
|
||||
default: () => getRandomUserAvatar()
|
||||
},
|
||||
|
||||
promotionRate: {
|
||||
@@ -78,11 +66,8 @@ const UserSchema = new Schema({
|
||||
});
|
||||
|
||||
try {
|
||||
// login
|
||||
UserSchema.index({ username: 1, password: 1 }, { background: true });
|
||||
|
||||
// Admin charts
|
||||
UserSchema.index({ createTime: -1 }, { background: true });
|
||||
UserSchema.index({ createTime: -1 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
@@ -16,6 +16,8 @@ import { MongoMemberGroupModel } from '../../permission/memberGroup/memberGroupS
|
||||
import { mongoSessionRun } from '../../../common/mongo/sessionRun';
|
||||
import { DefaultGroupName } from '@fastgpt/global/support/user/team/group/constant';
|
||||
import { getAIApi, openaiBaseUrl } from '../../../core/ai/config';
|
||||
import { createRootOrg } from '../../permission/org/controllers';
|
||||
import { refreshSourceAvatar } from '../../../common/file/image/controller';
|
||||
|
||||
async function getTeamMember(match: Record<string, any>): Promise<TeamTmbItemType> {
|
||||
const tmb = await MongoTeamMember.findOne(match).populate<{ team: TeamSchema }>('team').lean();
|
||||
@@ -32,6 +34,7 @@ async function getTeamMember(match: Record<string, any>): Promise<TeamTmbItemTyp
|
||||
return {
|
||||
userId: String(tmb.userId),
|
||||
teamId: String(tmb.teamId),
|
||||
teamAvatar: tmb.team.avatar,
|
||||
teamName: tmb.team.name,
|
||||
memberName: tmb.name,
|
||||
avatar: tmb.team.avatar,
|
||||
@@ -77,13 +80,11 @@ export async function createDefaultTeam({
|
||||
userId,
|
||||
teamName = 'My Team',
|
||||
avatar = '/icon/logo.svg',
|
||||
balance,
|
||||
session
|
||||
}: {
|
||||
userId: string;
|
||||
teamName?: string;
|
||||
avatar?: string;
|
||||
balance?: number;
|
||||
session: ClientSession;
|
||||
}) {
|
||||
// auth default team
|
||||
@@ -100,7 +101,6 @@ export async function createDefaultTeam({
|
||||
ownerId: userId,
|
||||
name: teamName,
|
||||
avatar,
|
||||
balance,
|
||||
createTime: new Date()
|
||||
}
|
||||
],
|
||||
@@ -132,15 +132,11 @@ export async function createDefaultTeam({
|
||||
],
|
||||
{ session }
|
||||
);
|
||||
console.log('create default team and group', userId);
|
||||
await createRootOrg({ teamId: tmb.teamId, session });
|
||||
console.log('create default team, group and root org', userId);
|
||||
return tmb;
|
||||
} else {
|
||||
console.log('default team exist', userId);
|
||||
await MongoTeam.findByIdAndUpdate(tmb.teamId, {
|
||||
$set: {
|
||||
...(balance !== undefined && { balance })
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -215,7 +211,8 @@ export async function updateTeam({
|
||||
return obj;
|
||||
})();
|
||||
|
||||
await MongoTeam.findByIdAndUpdate(
|
||||
// This is where we get the old team
|
||||
const team = await MongoTeam.findByIdAndUpdate(
|
||||
teamId,
|
||||
{
|
||||
$set: {
|
||||
@@ -241,6 +238,8 @@ export async function updateTeam({
|
||||
},
|
||||
{ session }
|
||||
);
|
||||
|
||||
await refreshSourceAvatar(avatar, team?.avatar, session);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -23,10 +23,6 @@ const TeamMemberSchema = new Schema({
|
||||
type: String,
|
||||
default: 'Member'
|
||||
},
|
||||
role: {
|
||||
type: String
|
||||
// enum: Object.keys(TeamMemberRoleMap) // disable enum validation for old data
|
||||
},
|
||||
status: {
|
||||
type: String,
|
||||
enum: Object.keys(TeamMemberStatusMap)
|
||||
@@ -38,6 +34,12 @@ const TeamMemberSchema = new Schema({
|
||||
defaultTeam: {
|
||||
type: Boolean,
|
||||
default: false
|
||||
},
|
||||
|
||||
// Abandoned
|
||||
role: {
|
||||
type: String
|
||||
// enum: Object.keys(TeamMemberRoleMap) // disable enum validation for old data
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
@@ -21,10 +21,7 @@ const TeamSchema = new Schema({
|
||||
type: Date,
|
||||
default: () => Date.now()
|
||||
},
|
||||
balance: {
|
||||
type: Number,
|
||||
default: 0
|
||||
},
|
||||
balance: Number,
|
||||
teamDomain: {
|
||||
type: String
|
||||
},
|
||||
|
||||
@@ -61,11 +61,11 @@ const UsageSchema = new Schema({
|
||||
});
|
||||
|
||||
try {
|
||||
UsageSchema.index({ teamId: 1, tmbId: 1, source: 1, time: -1 }, { background: true });
|
||||
UsageSchema.index({ teamId: 1, tmbId: 1, source: 1, time: -1 });
|
||||
// timer task. clear dead team
|
||||
// UsageSchema.index({ teamId: 1, time: -1 }, { background: true });
|
||||
// UsageSchema.index({ teamId: 1, time: -1 });
|
||||
|
||||
UsageSchema.index({ time: 1 }, { background: true, expireAfterSeconds: 360 * 24 * 60 * 60 });
|
||||
UsageSchema.index({ time: 1 }, { expireAfterSeconds: 360 * 24 * 60 * 60 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
@@ -1,9 +1,27 @@
|
||||
import TurndownService from 'turndown';
|
||||
import { ImageType } from '../readFile/type';
|
||||
import { matchMdImgTextAndUpload } from '@fastgpt/global/common/string/markdown';
|
||||
import { getNanoid } from '@fastgpt/global/common/string/tools';
|
||||
// @ts-ignore
|
||||
const turndownPluginGfm = require('joplin-turndown-plugin-gfm');
|
||||
|
||||
const processBase64Images = (htmlContent: string) => {
|
||||
const base64Regex = /src="data:([^;]+);base64,([^"]+)"/g;
|
||||
const images: ImageType[] = [];
|
||||
|
||||
const processedHtml = htmlContent.replace(base64Regex, (match, mime, base64Data) => {
|
||||
const uuid = `IMAGE_${getNanoid(12)}_IMAGE`;
|
||||
images.push({
|
||||
uuid,
|
||||
base64: base64Data,
|
||||
mime
|
||||
});
|
||||
return `src="${uuid}"`;
|
||||
});
|
||||
|
||||
return { processedHtml, images };
|
||||
};
|
||||
|
||||
export const html2md = (
|
||||
html: string
|
||||
): {
|
||||
@@ -25,11 +43,14 @@ export const html2md = (
|
||||
turndownService.remove(['i', 'script', 'iframe', 'style']);
|
||||
turndownService.use(turndownPluginGfm.gfm);
|
||||
|
||||
const { text, imageList } = matchMdImgTextAndUpload(html);
|
||||
// Base64 img to id, otherwise it will occupy memory when going to md
|
||||
const { processedHtml, images } = processBase64Images(html);
|
||||
const md = turndownService.turndown(processedHtml);
|
||||
const { text, imageList } = matchMdImgTextAndUpload(md);
|
||||
|
||||
return {
|
||||
rawText: turndownService.turndown(text),
|
||||
imageList
|
||||
rawText: text,
|
||||
imageList: [...images, ...imageList]
|
||||
};
|
||||
} catch (error) {
|
||||
console.log('html 2 markdown error', error);
|
||||
|
||||
@@ -24,7 +24,11 @@ export const readFileRawText = ({ buffer, encoding }: ReadRawTextByBuffer): Read
|
||||
return buffer.toString(encoding as BufferEncoding);
|
||||
}
|
||||
|
||||
return iconv.decode(buffer, encoding);
|
||||
if (encoding) {
|
||||
return iconv.decode(buffer, encoding);
|
||||
}
|
||||
|
||||
return buffer.toString('utf-8');
|
||||
} catch (error) {
|
||||
return buffer.toString('utf-8');
|
||||
}
|
||||
|
||||
@@ -44,13 +44,15 @@ const parsePowerPoint = async ({
|
||||
}
|
||||
|
||||
// Returning an array of all the xml contents read using fs.readFileSync
|
||||
const xmlContentArray = files.map((file) => {
|
||||
try {
|
||||
return fs.readFileSync(`${decompressPath}/${file.path}`, encoding);
|
||||
} catch (err) {
|
||||
return fs.readFileSync(`${decompressPath}/${file.path}`, 'utf-8');
|
||||
}
|
||||
});
|
||||
const xmlContentArray = await Promise.all(
|
||||
files.map((file) => {
|
||||
try {
|
||||
return fs.promises.readFile(`${decompressPath}/${file.path}`, encoding);
|
||||
} catch (err) {
|
||||
return fs.promises.readFile(`${decompressPath}/${file.path}`, 'utf-8');
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
let responseArr: string[] = [];
|
||||
|
||||
|
||||
@@ -12,24 +12,24 @@ const getTemplateNameList = () => {
|
||||
return fs.readdirSync(templatesPath) as string[];
|
||||
};
|
||||
|
||||
const getFileTemplates = (): AppTemplateSchemaType[] => {
|
||||
const getFileTemplates = async (): Promise<AppTemplateSchemaType[]> => {
|
||||
const templateNames = getTemplateNameList();
|
||||
|
||||
const appMarketTemplates = templateNames.map((name) => {
|
||||
const fileContent = require(`./src/${name}/template.json`);
|
||||
return Promise.all(
|
||||
templateNames.map<Promise<AppTemplateSchemaType>>(async (name) => {
|
||||
const fileContent = (await import(`./src/${name}/template.json`))?.default;
|
||||
|
||||
return {
|
||||
...fileContent,
|
||||
templateId: `${PluginSourceEnum.community}-${name}`,
|
||||
isActive: true
|
||||
};
|
||||
});
|
||||
|
||||
return appMarketTemplates;
|
||||
return {
|
||||
...fileContent,
|
||||
templateId: `${PluginSourceEnum.community}-${name}`,
|
||||
isActive: true
|
||||
};
|
||||
})
|
||||
);
|
||||
};
|
||||
|
||||
const getAppTemplates = async () => {
|
||||
const communityTemplates = getFileTemplates();
|
||||
const communityTemplates = await getFileTemplates();
|
||||
|
||||
const dbTemplates = await MongoAppTemplate.find();
|
||||
|
||||
|
||||
@@ -18,7 +18,6 @@ const MyIconButton = ({
|
||||
}: Props) => {
|
||||
return (
|
||||
<Flex
|
||||
mr={1}
|
||||
p={1}
|
||||
color={'myGray.500'}
|
||||
rounded={'sm'}
|
||||
|
||||
@@ -31,6 +31,7 @@ export const iconPaths = {
|
||||
'common/customTitleLight': () => import('./icons/common/customTitleLight.svg'),
|
||||
'common/data': () => import('./icons/common/data.svg'),
|
||||
'common/dingtalkFill': () => import('./icons/common/dingtalkFill.svg'),
|
||||
'common/downArrowFill': () => import('./icons/common/downArrowFill.svg'),
|
||||
'common/editor/resizer': () => import('./icons/common/editor/resizer.svg'),
|
||||
'common/errorFill': () => import('./icons/common/errorFill.svg'),
|
||||
'common/file/move': () => import('./icons/common/file/move.svg'),
|
||||
@@ -95,8 +96,10 @@ export const iconPaths = {
|
||||
'common/variable': () => import('./icons/common/variable.svg'),
|
||||
'common/viewLight': () => import('./icons/common/viewLight.svg'),
|
||||
'common/voiceLight': () => import('./icons/common/voiceLight.svg'),
|
||||
'common/wallet': () => import('./icons/common/wallet.svg'),
|
||||
'common/warn': () => import('./icons/common/warn.svg'),
|
||||
'common/wechatFill': () => import('./icons/common/wechatFill.svg'),
|
||||
'common/wecom': () => import('./icons/common/wecom.svg'),
|
||||
configmap: () => import('./icons/configmap.svg'),
|
||||
copy: () => import('./icons/copy.svg'),
|
||||
'core/app/aiFill': () => import('./icons/core/app/aiFill.svg'),
|
||||
@@ -139,6 +142,7 @@ export const iconPaths = {
|
||||
'core/app/ttsFill': () => import('./icons/core/app/ttsFill.svg'),
|
||||
'core/app/type/httpPlugin': () => import('./icons/core/app/type/httpPlugin.svg'),
|
||||
'core/app/type/httpPluginFill': () => import('./icons/core/app/type/httpPluginFill.svg'),
|
||||
'core/app/type/jsonImport': () => import('./icons/core/app/type/jsonImport.svg'),
|
||||
'core/app/type/plugin': () => import('./icons/core/app/type/plugin.svg'),
|
||||
'core/app/type/pluginFill': () => import('./icons/core/app/type/pluginFill.svg'),
|
||||
'core/app/type/pluginLight': () => import('./icons/core/app/type/pluginLight.svg'),
|
||||
@@ -346,6 +350,8 @@ export const iconPaths = {
|
||||
history: () => import('./icons/history.svg'),
|
||||
infoRounded: () => import('./icons/infoRounded.svg'),
|
||||
kbTest: () => import('./icons/kbTest.svg'),
|
||||
key: () => import('./icons/key.svg'),
|
||||
keyPrimary: () => import('./icons/keyPrimary.svg'),
|
||||
menu: () => import('./icons/menu.svg'),
|
||||
minus: () => import('./icons/minus.svg'),
|
||||
'modal/AddClb': () => import('./icons/modal/AddClb.svg'),
|
||||
@@ -408,7 +414,6 @@ export const iconPaths = {
|
||||
'support/bill/shoppingCart': () => import('./icons/support/bill/shoppingCart.svg'),
|
||||
'support/bill/wallet': () => import('./icons/support/bill/wallet.svg'),
|
||||
'support/outlink/apikeyFill': () => import('./icons/support/outlink/apikeyFill.svg'),
|
||||
'support/outlink/apikeyLight': () => import('./icons/support/outlink/apikeyLight.svg'),
|
||||
'support/outlink/iframeLight': () => import('./icons/support/outlink/iframeLight.svg'),
|
||||
'support/outlink/share': () => import('./icons/support/outlink/share.svg'),
|
||||
'support/outlink/shareLight': () => import('./icons/support/outlink/shareLight.svg'),
|
||||
@@ -416,7 +421,6 @@ export const iconPaths = {
|
||||
'support/permission/privateLight': () => import('./icons/support/permission/privateLight.svg'),
|
||||
'support/permission/publicLight': () => import('./icons/support/permission/publicLight.svg'),
|
||||
'support/team/group': () => import('./icons/support/team/group.svg'),
|
||||
'support/team/key': () => import('./icons/support/team/key.svg'),
|
||||
'support/team/memberLight': () => import('./icons/support/team/memberLight.svg'),
|
||||
'support/usage/usageRecordLight': () => import('./icons/support/usage/usageRecordLight.svg'),
|
||||
'support/user/informLight': () => import('./icons/support/user/informLight.svg'),
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<g id="icon/solid/chevron-down">
|
||||
<path id="Rectangle 3101" d="M11.4695 5.33325C12.6574 5.33325 13.2523 6.76944 12.4123 7.60939L9.01223 11.0095C8.49154 11.5302 7.64732 11.5302 7.12662 11.0095L3.72653 7.60939C2.88657 6.76944 3.48146 5.33325 4.66933 5.33325H11.4695Z" fill="#667085"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 390 B |
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M9.99996 1.45117L10.0088 1.45127C11.0658 1.46249 12.1863 1.68237 13.0713 2.24383C13.9975 2.83154 14.6351 3.78214 14.6351 5.09344C14.6351 5.52986 14.5203 5.93227 14.3316 6.29147C16.4572 7.72689 18.083 10.1445 18.083 12.9349C18.083 14.9037 17.1234 16.3526 15.6002 17.2692C14.1238 18.1577 12.142 18.5394 10.0036 18.5488L9.9963 18.5488C7.85793 18.5394 5.87613 18.1577 4.39966 17.2692C2.87651 16.3526 1.91687 14.9037 1.91687 12.9349C1.91687 10.1445 3.54266 7.72689 5.66829 6.29147C5.4796 5.93227 5.36483 5.52986 5.36483 5.09344C5.36483 3.78214 6.00237 2.83154 6.92865 2.24383C7.81357 1.68237 8.93415 1.46249 9.99111 1.45127L9.99996 1.45117ZM7.82155 3.65114C7.32958 3.96329 7.0315 4.40688 7.0315 5.09344C7.0315 5.3213 7.13449 5.58907 7.40685 5.87306C7.81463 6.29825 7.73085 7.01936 7.18438 7.31745C5.1323 8.4368 3.58354 10.5741 3.58354 12.9349C3.58354 14.2641 4.18779 15.1965 5.25903 15.8411C6.37626 16.5135 8.018 16.873 9.99996 16.8821C11.9819 16.873 13.6237 16.5135 14.7409 15.8411C15.8121 15.1965 16.4164 14.2641 16.4164 12.9349C16.4164 10.5741 14.8676 8.4368 12.8155 7.31745C12.2691 7.01936 12.1853 6.29825 12.5931 5.87306C12.8654 5.58907 12.9684 5.3213 12.9684 5.09344C12.9684 4.40688 12.6703 3.96329 12.1784 3.65114C11.6468 3.31389 10.87 3.12837 9.99996 3.11794C9.12994 3.12837 8.35309 3.31389 7.82155 3.65114Z" />
|
||||
<path d="M9.68737 8.6035C9.79475 8.3133 10.2052 8.3133 10.3126 8.6035L10.9425 10.3057C10.9762 10.397 11.0482 10.4689 11.1394 10.5027L12.8417 11.1326C13.1319 11.24 13.1319 11.6504 12.8417 11.7578L11.1394 12.3877C11.0482 12.4215 10.9762 12.4934 10.9425 12.5846L10.3126 14.2869C10.2052 14.5771 9.79475 14.5771 9.68737 14.2869L9.05748 12.5846C9.02372 12.4934 8.95178 12.4215 8.86054 12.3877L7.1583 11.7578C6.86809 11.6504 6.86809 11.24 7.1583 11.1326L8.86054 10.5027C8.95178 10.4689 9.02372 10.397 9.05748 10.3057L9.68737 8.6035Z" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.9 KiB |
@@ -0,0 +1,7 @@
|
||||
<svg viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M12.9278 14.6745C12.9329 14.6814 12.9381 14.6882 12.9449 14.6934C12.9552 14.7037 12.9673 14.714 12.9793 14.7209C13.0033 14.7432 13.0256 14.7655 13.0497 14.7878C13.5031 15.2412 13.7882 15.7994 13.9049 16.385C13.9067 16.4176 13.9101 16.4503 13.9152 16.4829C13.9204 16.519 13.929 16.555 13.9376 16.5894C13.9822 16.7542 14.0698 16.9088 14.1986 17.0376C14.5936 17.4326 15.2359 17.4326 15.6326 17.0376C16.0276 16.6426 16.0276 16.0003 15.6326 15.6036C15.4935 15.4645 15.3235 15.3735 15.1432 15.3322C15.1226 15.3271 15.1037 15.3237 15.0831 15.3202C15.0624 15.3168 15.0418 15.3134 15.0212 15.3116C14.4219 15.2 13.8483 14.9115 13.3846 14.4478C13.3519 14.4152 13.3193 14.3808 13.2867 14.3465C13.1922 14.252 13.0394 14.252 12.9432 14.3465C12.8556 14.4409 12.8505 14.58 12.9278 14.6745Z" fill="#FB6500"/>
|
||||
<path d="M16.1323 15.4541C16.1391 15.449 16.146 15.4438 16.1511 15.4369C16.1615 15.4266 16.1718 15.4146 16.1786 15.4026C16.201 15.3786 16.2233 15.3562 16.2456 15.3322C16.699 14.8788 17.2571 14.5937 17.8428 14.4769C17.8754 14.4752 17.908 14.4718 17.9407 14.4666C17.9767 14.4615 18.0128 14.4529 18.0471 14.4443C18.212 14.3996 18.3666 14.3121 18.4954 14.1833C18.8904 13.7883 18.8904 13.146 18.4954 12.7493C18.1004 12.3543 17.4581 12.3543 17.0614 12.7493C16.9223 12.8884 16.8312 13.0584 16.79 13.2387C16.7849 13.2593 16.7814 13.2782 16.778 13.2988C16.7746 13.3194 16.7711 13.34 16.7694 13.3606C16.6578 13.96 16.3693 14.5336 15.9056 14.9973C15.8729 15.0299 15.8386 15.0626 15.8042 15.0952C15.7098 15.1896 15.7098 15.3425 15.8042 15.4387C15.897 15.5245 16.0361 15.5314 16.1323 15.4541Z" fill="#0082EF"/>
|
||||
<path d="M16.9103 12.2496C16.9052 12.2427 16.9 12.2358 16.8932 12.2307C16.8828 12.2204 16.8708 12.2101 16.8588 12.2032C16.8348 12.1809 16.8124 12.1585 16.7884 12.1362C16.335 11.6828 16.0499 11.1247 15.9331 10.5391C15.9314 10.5064 15.928 10.4738 15.9228 10.4412C15.9177 10.4051 15.9091 10.369 15.9005 10.3347C15.8559 10.1698 15.7683 10.0153 15.6395 9.88646C15.2445 9.49146 14.6022 9.49146 14.2055 9.88646C13.8087 10.2815 13.8105 10.9238 14.2055 11.3205C14.3446 11.4596 14.5146 11.5506 14.6949 11.5918C14.7155 11.597 14.7344 11.6004 14.755 11.6038C14.7756 11.6073 14.7962 11.6107 14.8168 11.6124C15.4162 11.724 15.9898 12.0126 16.4535 12.4763C16.4861 12.5089 16.5188 12.5432 16.5514 12.5776C16.6459 12.672 16.7987 12.672 16.8949 12.5776C16.9825 12.4848 16.9876 12.3457 16.9103 12.2496Z" fill="#2DBC00"/>
|
||||
<path d="M13.7056 11.4716C13.6988 11.4767 13.6919 11.4819 13.6867 11.4887C13.6764 11.4991 13.6661 11.5111 13.6593 11.5231C13.6369 11.5471 13.6146 11.5695 13.5923 11.5935C13.1389 12.0469 12.5807 12.332 11.9951 12.4488C11.9625 12.4505 11.9299 12.4539 11.8972 12.4591C11.8612 12.4642 11.8251 12.4728 11.7907 12.4814C11.6259 12.526 11.4713 12.6136 11.3425 12.7424C10.9475 13.1374 10.9475 13.7797 11.3425 14.1764C11.7375 14.5732 12.3798 14.5714 12.7765 14.1764C12.9156 14.0373 13.0066 13.8673 13.0479 13.687C13.053 13.6664 13.0565 13.6475 13.0599 13.6269C13.0633 13.6063 13.0668 13.5857 13.0685 13.5651C13.1801 12.9657 13.4686 12.3921 13.9323 11.9284C13.9649 11.8958 13.9993 11.8631 14.0336 11.8305C14.1281 11.7361 14.1281 11.5832 14.0336 11.487C13.9409 11.3994 13.8018 11.3943 13.7056 11.4716Z" fill="#FFCC00"/>
|
||||
<path d="M15.2136 6.51859C14.9319 5.93984 14.5524 5.4023 14.0887 4.92143C12.9123 3.70553 11.267 2.92241 9.45521 2.71804C9.13063 2.68026 8.80776 2.66309 8.49691 2.66309C8.20153 2.66309 7.8924 2.68026 7.57984 2.71461C5.75942 2.91211 4.10559 3.69179 2.92231 4.90769C2.45519 5.38856 2.07393 5.92438 1.79056 6.50142C1.40415 7.28282 1.20837 8.11575 1.20837 8.97272C1.20837 10.077 1.54498 11.1641 2.18041 12.119C2.53934 12.6599 3.15416 13.3572 3.67968 13.7848L3.38772 14.9887L3.30357 15.3287C3.28812 15.3614 3.27781 15.3974 3.26922 15.4335C3.26407 15.4558 3.26235 15.4781 3.26064 15.5022C3.25892 15.5194 3.25549 15.5365 3.25549 15.5554C3.25549 15.8645 3.50622 16.117 3.81707 16.117C3.91839 16.117 4.01285 16.0878 4.09356 16.0414C4.097 16.0397 4.09872 16.038 4.10215 16.038L4.13822 16.0174L6.21796 14.9732C6.66619 15.102 7.10927 15.1845 7.57812 15.236C7.88209 15.2703 8.19122 15.2858 8.49691 15.2858C8.80776 15.2858 9.13063 15.2669 9.45521 15.2308C10.0941 15.1587 10.7089 15.011 11.2911 14.8015C11.2275 14.7809 11.1657 14.7534 11.1056 14.7191C10.7484 14.5147 10.5646 14.1266 10.6007 13.7419C10.1834 13.8741 9.74888 13.9686 9.29893 14.0201C9.02587 14.051 8.75624 14.0665 8.4952 14.0665C8.23931 14.0665 7.9817 14.0527 7.72581 14.0235C7.67257 14.0184 7.61934 14.0098 7.5661 14.0029C7.21575 13.9565 6.87056 13.8844 6.53567 13.7865C6.46698 13.7642 6.39485 13.7539 6.321 13.7539C6.20594 13.7539 6.09431 13.7848 5.98096 13.8432C5.9655 13.8501 5.95176 13.8569 5.93631 13.8655L4.60191 14.6521H4.60019C4.57271 14.6675 4.55726 14.6744 4.5418 14.6744C4.49543 14.6744 4.45765 14.6349 4.45765 14.5868L4.50745 14.3876C4.52119 14.3344 4.54008 14.2605 4.56412 14.1712C4.62252 13.9497 4.70151 13.6457 4.76162 13.419C4.77536 13.3726 4.78738 13.3177 4.78738 13.2559C4.78738 13.0841 4.70495 12.921 4.56756 12.8214C4.49715 12.7698 4.42845 12.7166 4.35461 12.6565C4.24298 12.5655 4.13478 12.471 4.03174 12.3731C3.7415 12.0983 3.48733 11.7978 3.27438 11.4767C2.77119 10.721 2.50499 9.86232 2.50499 8.99505C2.50499 8.32184 2.65956 7.66752 2.96181 7.04926C3.18851 6.58729 3.4942 6.15794 3.87031 5.77153C4.84406 4.77202 6.21109 4.13144 7.72409 3.96657C7.98685 3.93738 8.24618 3.92364 8.49348 3.92364C8.7528 3.92364 9.02243 3.9391 9.29721 3.97001C10.8016 4.14175 12.1618 4.78576 13.1287 5.78356C13.5031 6.16996 13.807 6.60103 14.032 7.06472C14.3308 7.67954 14.482 8.3287 14.482 8.99505C14.482 9.06374 14.4768 9.13415 14.4734 9.20285C14.8632 8.96413 15.3802 9.0105 15.7168 9.34882C15.7339 9.366 15.7477 9.38489 15.7631 9.40206C15.7734 9.25952 15.7803 9.11698 15.7803 8.97444C15.7837 8.12434 15.5914 7.29828 15.2136 6.51859Z" fill="#0079DE"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.8 KiB |
@@ -0,0 +1,19 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 32 32" fill="none">
|
||||
<g clip-path="url(#clip0_16784_2993)">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M21.5271 4.37083L25.4043 8.01175C26.3626 8.91164 26.8417 9.36158 27.1828 9.89648C27.4727 10.3511 27.6882 10.849 27.8211 11.3715C27.9775 11.9863 27.9775 12.6436 27.9775 13.9582V19.4935C27.0071 19.026 25.919 18.764 24.7697 18.764C20.6828 18.764 17.3698 22.077 17.3698 26.1639C17.3698 27.502 17.7249 28.7571 18.3461 29.8401H12.1799C9.29446 29.8401 7.85176 29.8401 6.75362 29.2697C5.82822 28.789 5.07369 28.0344 4.59299 27.1091C4.02255 26.0109 4.02255 24.5682 4.02255 21.6828V10.3173C4.02255 7.43186 4.02255 5.98915 4.59299 4.89101C5.07369 3.96561 5.82822 3.21108 6.75362 2.73038C7.85176 2.15994 9.29447 2.15994 12.1799 2.15994H15.943C17.1489 2.15994 17.7519 2.15994 18.3216 2.29333C18.806 2.40673 19.2711 2.59087 19.7019 2.83981C20.2085 3.1326 20.6481 3.54535 21.5271 4.37083ZM19.1105 6.20351C18.3195 5.4451 17.924 5.0659 17.5846 5.05216C17.315 5.04124 17.0548 5.15224 16.8761 5.3544C16.6511 5.60893 16.6511 6.15685 16.6511 7.25268V8.50383C16.6511 10.1068 16.6511 10.9083 16.963 11.5205C17.2374 12.0591 17.6753 12.4969 18.2138 12.7713C18.8261 13.0833 19.6276 13.0833 21.2305 13.0833H22.6697C23.8233 13.0833 24.4001 13.0833 24.6566 12.8505C24.8602 12.6658 24.9678 12.3979 24.9487 12.1238C24.9247 11.7782 24.5084 11.379 23.6757 10.5806L19.1105 6.20351Z" fill="url(#paint0_linear_16784_2993)"/>
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M23.5057 21.6108C23.8901 20.1169 26.0117 20.1169 26.3961 21.6108C26.6044 22.4202 27.4368 22.9007 28.2419 22.6764C29.7278 22.2624 30.7886 24.0997 29.6871 25.1796C29.0903 25.7646 29.0903 26.7258 29.6871 27.3108C30.7886 28.3907 29.7278 30.228 28.2419 29.814C27.4368 29.5897 26.6044 30.0703 26.3961 30.8796C26.0117 32.3735 23.8901 32.3735 23.5057 30.8796C23.2974 30.0703 22.4651 29.5897 21.66 29.814C20.174 30.228 19.1133 28.3907 20.2148 27.3108C20.8116 26.7258 20.8116 25.7646 20.2148 25.1796C19.1133 24.0997 20.174 22.2624 21.66 22.6764C22.4651 22.9007 23.2974 22.4202 23.5057 21.6108ZM26.8127 26.2452C26.8127 27.2734 25.9791 28.107 24.9509 28.107C23.9227 28.107 23.0891 27.2734 23.0891 26.2452C23.0891 25.217 23.9227 24.3834 24.9509 24.3834C25.9791 24.3834 26.8127 25.217 26.8127 26.2452Z" fill="url(#paint1_linear_16784_2993)"/>
|
||||
</g>
|
||||
<defs>
|
||||
<linearGradient id="paint0_linear_16784_2993" x1="17.0824" y1="2.15994" x2="17.0824" y2="32" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#BAC6D8"/>
|
||||
<stop offset="1" stop-color="#9FA9C2"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint1_linear_16784_2993" x1="17.0824" y1="2.15994" x2="17.0824" y2="32" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#BAC6D8"/>
|
||||
<stop offset="1" stop-color="#9FA9C2"/>
|
||||
</linearGradient>
|
||||
<clipPath id="clip0_16784_2993">
|
||||
<rect width="32" height="32" fill="white"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.9 KiB |
|
Before Width: | Height: | Size: 1.8 KiB After Width: | Height: | Size: 1.8 KiB |
9
packages/web/components/common/Icon/icons/keyPrimary.svg
Normal file
@@ -0,0 +1,9 @@
|
||||
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<g id="icon/line/key">
|
||||
<g id="Vector">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M12.2109 12.395C14.7199 12.395 16.75 10.3644 16.75 7.86409C16.75 5.36381 14.7199 3.33317 12.2109 3.33317C9.70193 3.33317 7.67177 5.36381 7.67177 7.86409C7.67177 8.2826 7.72817 8.68531 7.83285 9.0663C8.02882 9.7796 7.86557 10.6069 7.27775 11.1964L3.33532 15.1502L3.33531 16.6665H5.02967L5.07424 16.622L5.52315 18.2974C5.43021 18.321 5.33398 18.3332 5.23648 18.3332H2.66864C2.11635 18.3332 1.66864 17.8855 1.66864 17.3332L1.66865 14.9435C1.66865 14.6347 1.79111 14.3384 2.00918 14.1197L6.09755 10.0196C6.23063 9.88611 6.27567 9.68959 6.22574 9.50784C6.08191 8.98436 6.00511 8.43318 6.00511 7.86409C6.00511 4.44126 8.78354 1.6665 12.2109 1.6665C15.6383 1.6665 18.4167 4.44126 18.4167 7.86409C18.4167 11.2869 15.6383 14.0617 12.2109 14.0617C11.651 14.0617 11.1084 13.9876 10.5923 13.8488C10.4113 13.8 10.216 13.8454 10.0833 13.9779L8.90842 15.1503L7.72991 13.9718L8.90598 12.7982C9.49218 12.2132 10.3143 12.048 11.0254 12.2393C11.4015 12.3405 11.7985 12.395 12.2109 12.395Z" fill="#3370FF"/>
|
||||
<path d="M6.13784 16.7879H6.85411C7.26832 16.7879 7.60417 16.4521 7.60417 16.0379L7.60411 16.0288V15.3216H8.43437C8.84858 15.3216 9.18437 14.9858 9.18437 14.5716C9.18437 14.1574 8.84858 13.8216 8.43437 13.8216H6.96804C6.94888 13.8216 6.92989 13.8223 6.9111 13.8237C6.89229 13.8223 6.87328 13.8216 6.85411 13.8216C6.4399 13.8216 6.10411 14.1574 6.10411 14.5716V15.2879H5.38784C4.97363 15.2879 4.63784 15.6237 4.63784 16.0379V17.5695C4.63784 17.9837 4.97363 18.3195 5.38784 18.3195C5.80206 18.3195 6.13784 17.9837 6.13784 17.5695V16.7879Z" fill="#3370FF"/>
|
||||
<path d="M14.4642 7.19029C14.4642 8.11076 13.718 8.85695 12.7975 8.85695C11.877 8.85695 11.1308 8.11076 11.1308 7.19029C11.1308 6.26981 11.877 5.52362 12.7975 5.52362C13.718 5.52362 14.4642 6.26981 14.4642 7.19029Z" fill="#3370FF"/>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.9 KiB |
@@ -1,8 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd"
|
||||
d="M12.2109 12.3945C14.7199 12.3945 16.75 10.3639 16.75 7.8636C16.75 5.36333 14.7199 3.33268 12.2109 3.33268C9.70193 3.33268 7.67177 5.36333 7.67177 7.8636C7.67177 8.28211 7.72817 8.68482 7.83285 9.06581C8.02882 9.77912 7.86557 10.6064 7.27775 11.1959L3.33532 15.1497L3.33531 16.666H5.02967L5.07424 16.6215L5.52315 18.2969C5.43021 18.3205 5.33398 18.3327 5.23648 18.3327H2.66864C2.11635 18.3327 1.66864 17.885 1.66864 17.3327L1.66865 14.943C1.66865 14.6342 1.79111 14.3379 2.00918 14.1192L6.09755 10.0191C6.23063 9.88562 6.27567 9.6891 6.22574 9.50736C6.08191 8.98387 6.00511 8.43269 6.00511 7.8636C6.00511 4.44077 8.78354 1.66602 12.2109 1.66602C15.6383 1.66602 18.4167 4.44077 18.4167 7.8636C18.4167 11.2864 15.6383 14.0612 12.2109 14.0612C11.651 14.0612 11.1084 13.9871 10.5923 13.8483C10.4113 13.7996 10.216 13.8449 10.0833 13.9774L8.90842 15.1498L7.72991 13.9713L8.90598 12.7977C9.49218 12.2127 10.3143 12.0475 11.0254 12.2389C11.4015 12.34 11.7985 12.3945 12.2109 12.3945Z" />
|
||||
<path
|
||||
d="M6.13784 16.7874H6.85411C7.26832 16.7874 7.60417 16.4516 7.60417 16.0374L7.60411 16.0283V15.3211H8.43437C8.84858 15.3211 9.18437 14.9853 9.18437 14.5711C9.18437 14.1569 8.84858 13.8211 8.43437 13.8211H6.96804C6.94888 13.8211 6.92989 13.8218 6.9111 13.8232C6.89229 13.8218 6.87328 13.8211 6.85411 13.8211C6.4399 13.8211 6.10411 14.1569 6.10411 14.5711V15.2874H5.38784C4.97363 15.2874 4.63784 15.6232 4.63784 16.0374V17.569C4.63784 17.9832 4.97363 18.319 5.38784 18.319C5.80206 18.319 6.13784 17.9832 6.13784 17.569V16.7874Z" />
|
||||
<path
|
||||
d="M14.4642 7.1898C14.4642 8.11027 13.718 8.85647 12.7975 8.85647C11.877 8.85647 11.1308 8.11027 11.1308 7.1898C11.1308 6.26932 11.877 5.52313 12.7975 5.52313C13.718 5.52313 14.4642 6.26932 14.4642 7.1898Z" />
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.8 KiB |