Feat: Images dataset collection (#4941)
* New pic (#4858) * 更新数据集相关类型,添加图像文件ID和预览URL支持;优化数据集导入功能,新增图像数据集处理组件;修复部分国际化文本;更新文件上传逻辑以支持新功能。 * 与原先代码的差别 * 新增 V4.9.10 更新说明,支持 PG 设置`systemEnv.hnswMaxScanTuples`参数,优化 LLM stream 调用超时,修复全文检索多知识库排序问题。同时更新数据集索引,移除 datasetId 字段以简化查询。 * 更换成fileId_image逻辑,并增加训练队列匹配的逻辑 * 新增图片集合判断逻辑,优化预览URL生成流程,确保仅在数据集为图片集合时生成预览URL,并添加相关日志输出以便调试。 * Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages. * Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API. * Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook. * Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets. * Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process. * 图片上传到新建的 dataset_collection_images 表,逻辑跟随更改 * 修改了除了controller的其他部分问题 * 把图片数据集的逻辑整合到controller里面 * 补充i18n * 补充i18n * resolve评论:主要是上传逻辑的更改和组件复用 * 图片名称的图标显示 * 修改编译报错的命名问题 * 删除不需要的collectionid部分 * 多余文件的处理和改动一个删除按钮 * 除了loading和统一的imageId,其他都resolve掉的 * 处理图标报错 * 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId * 去除不必要文件修改 * 报错和字段修改 * 增加上传成功后删除临时文件的逻辑以及回退一些修改 * 删除path字段,将图片保存到gridfs内,并修改增删等操作的代码 * 修正编译错误 --------- Co-authored-by: archer <545436317@qq.com> * perf: image dataset * feat: insert image * perf: image icon * fix: training state --------- Co-authored-by: Zhuangzai fa <143257420+ctrlz526@users.noreply.github.com>
This commit is contained in:
@@ -220,9 +220,11 @@ export const iconPaths = {
|
||||
import('./icons/core/dataset/feishuDatasetOutline.svg'),
|
||||
'core/dataset/fileCollection': () => import('./icons/core/dataset/fileCollection.svg'),
|
||||
'core/dataset/fullTextRecall': () => import('./icons/core/dataset/fullTextRecall.svg'),
|
||||
'core/dataset/imageFill': () => import('./icons/core/dataset/imageFill.svg'),
|
||||
'core/dataset/manualCollection': () => import('./icons/core/dataset/manualCollection.svg'),
|
||||
'core/dataset/mixedRecall': () => import('./icons/core/dataset/mixedRecall.svg'),
|
||||
'core/dataset/modeEmbedding': () => import('./icons/core/dataset/modeEmbedding.svg'),
|
||||
'core/dataset/otherDataset': () => import('./icons/core/dataset/otherDataset.svg'),
|
||||
'core/dataset/questionExtension': () => import('./icons/core/dataset/questionExtension.svg'),
|
||||
'core/dataset/rerank': () => import('./icons/core/dataset/rerank.svg'),
|
||||
'core/dataset/searchfilter': () => import('./icons/core/dataset/searchfilter.svg'),
|
||||
@@ -230,7 +232,6 @@ export const iconPaths = {
|
||||
'core/dataset/tableCollection': () => import('./icons/core/dataset/tableCollection.svg'),
|
||||
'core/dataset/tag': () => import('./icons/core/dataset/tag.svg'),
|
||||
'core/dataset/websiteDataset': () => import('./icons/core/dataset/websiteDataset.svg'),
|
||||
'core/dataset/otherDataset': () => import('./icons/core/dataset/otherDataset.svg'),
|
||||
'core/dataset/websiteDatasetColor': () => import('./icons/core/dataset/websiteDatasetColor.svg'),
|
||||
'core/dataset/websiteDatasetOutline': () =>
|
||||
import('./icons/core/dataset/websiteDatasetOutline.svg'),
|
||||
@@ -379,10 +380,12 @@ export const iconPaths = {
|
||||
fullScreen: () => import('./icons/fullScreen.svg'),
|
||||
help: () => import('./icons/help.svg'),
|
||||
history: () => import('./icons/history.svg'),
|
||||
image: () => import('./icons/image.svg'),
|
||||
infoRounded: () => import('./icons/infoRounded.svg'),
|
||||
kbTest: () => import('./icons/kbTest.svg'),
|
||||
key: () => import('./icons/key.svg'),
|
||||
keyPrimary: () => import('./icons/keyPrimary.svg'),
|
||||
loading: () => import('./icons/loading.svg'),
|
||||
menu: () => import('./icons/menu.svg'),
|
||||
minus: () => import('./icons/minus.svg'),
|
||||
'modal/AddClb': () => import('./icons/modal/AddClb.svg'),
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 21 20" >
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M2.24348 4.15292C1.9165 4.79466 1.9165 5.63474 1.9165 7.31489V12.6852C1.9165 14.3654 1.9165 15.2054 2.24348 15.8472C2.5311 16.4117 2.99005 16.8706 3.55453 17.1582C4.19627 17.4852 5.03635 17.4852 6.7165 17.4852H13.7832C15.4633 17.4852 16.3034 17.4852 16.9451 17.1582C17.5096 16.8706 17.9686 16.4117 18.2562 15.8472C18.5832 15.2054 18.5832 14.3654 18.5832 12.6852V7.31489C18.5832 5.63473 18.5832 4.79466 18.2562 4.15292C17.9686 3.58843 17.5096 3.12949 16.9451 2.84187C16.3034 2.51489 15.4633 2.51489 13.7832 2.51489H6.7165C5.03635 2.51489 4.19627 2.51489 3.55453 2.84187C2.99005 3.12949 2.5311 3.58843 2.24348 4.15292ZM7.88951 6.75656C7.88951 7.67703 7.14331 8.42322 6.22284 8.42322C5.30236 8.42322 4.55617 7.67703 4.55617 6.75656C4.55617 5.83608 5.30236 5.08989 6.22284 5.08989C7.14331 5.08989 7.88951 5.83608 7.88951 6.75656ZM12.8631 8.65525C12.5376 8.32981 12.01 8.32981 11.6845 8.65525L5.92965 14.4101C5.40468 14.9351 5.77648 15.8327 6.5189 15.8327L15.5062 15.8327C16.4267 15.8327 17.1729 15.0865 17.1729 14.1661V13.3103C17.1729 13.0892 17.0851 12.8773 16.9288 12.721L12.8631 8.65525Z" fill="#3370FF"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.2 KiB |
4
packages/web/components/common/Icon/icons/image.svg
Normal file
4
packages/web/components/common/Icon/icons/image.svg
Normal file
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 17 16" >
|
||||
<path d="M5.50794 6.8195C6.06022 6.8195 6.50794 6.37178 6.50794 5.8195C6.50794 5.26721 6.06022 4.8195 5.50794 4.8195C4.95565 4.8195 4.50794 5.26721 4.50794 5.8195C4.50794 6.37178 4.95565 6.8195 5.50794 6.8195Z" />
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M1.55029 5.85187C1.55029 4.50775 1.55029 3.83568 1.81188 3.32229C2.04197 2.87071 2.40913 2.50355 2.86072 2.27346C3.3741 2.01187 4.04617 2.01187 5.39029 2.01187H11.0436C12.3878 2.01187 13.0598 2.01187 13.5732 2.27346C14.0248 2.50355 14.3919 2.87071 14.622 3.32229C14.8836 3.83568 14.8836 4.50775 14.8836 5.85187V10.1481C14.8836 11.4922 14.8836 12.1643 14.622 12.6777C14.3919 13.1293 14.0248 13.4964 13.5732 13.7265C13.0598 13.9881 12.3878 13.9881 11.0436 13.9881H5.39029C4.04617 13.9881 3.3741 13.9881 2.86072 13.7265C2.40913 13.4964 2.04197 13.1293 1.81188 12.6777C1.55029 12.1643 1.55029 11.4922 1.55029 10.1481V5.85187ZM5.39029 3.3452H11.0436C11.7377 3.3452 12.1781 3.34624 12.5114 3.37347C12.8291 3.39944 12.9305 3.44241 12.9679 3.46146C13.1686 3.56373 13.3318 3.72691 13.434 3.92761C13.4531 3.96502 13.4961 4.06638 13.522 4.38413C13.5493 4.71745 13.5503 5.15781 13.5503 5.85187V10.1481C13.5503 10.1562 13.5503 10.1641 13.5503 10.1721L10.3165 6.93829C10.0561 6.67794 9.634 6.67794 9.37365 6.93829L3.70938 12.6026C3.5547 12.5791 3.49333 12.5524 3.46604 12.5385C3.26533 12.4363 3.10215 12.2731 2.99989 12.0724C2.98083 12.035 2.93786 11.9336 2.9119 11.6159C2.88466 11.2825 2.88363 10.8422 2.88363 10.1481V5.85187C2.88363 5.15781 2.88466 4.71745 2.9119 4.38413C2.93786 4.06638 2.98083 3.96502 2.99989 3.92761C3.10215 3.72691 3.26533 3.56373 3.46604 3.46146C3.50344 3.44241 3.6048 3.39944 3.92255 3.37347C4.25587 3.34624 4.69623 3.3452 5.39029 3.3452ZM9.84506 8.3525L5.54277 12.6548H11.0436C11.7377 12.6548 12.1781 12.6538 12.5114 12.6265C12.8291 12.6006 12.9305 12.5576 12.9679 12.5385C13.1686 12.4363 13.3318 12.2731 13.434 12.0724C13.4422 12.0563 13.4549 12.0283 13.4687 11.9762L9.84506 8.3525Z" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.0 KiB |
4
packages/web/components/common/Icon/icons/loading.svg
Normal file
4
packages/web/components/common/Icon/icons/loading.svg
Normal file
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" viewBox="0 0 48 48" >
|
||||
<path d="M47.3337 24C47.3337 36.8866 36.887 47.3333 24.0003 47.3333C11.1137 47.3333 0.666992 36.8866 0.666992 24C0.666992 11.1133 11.1137 0.666626 24.0003 0.666626C36.887 0.666626 47.3337 11.1133 47.3337 24ZM5.33366 24C5.33366 34.3093 13.691 42.6666 24.0003 42.6666C34.3096 42.6666 42.667 34.3093 42.667 24C42.667 13.6906 34.3096 5.33329 24.0003 5.33329C13.691 5.33329 5.33366 13.6906 5.33366 24Z" />
|
||||
<path d="M24.0003 2.99996C24.0003 1.71129 25.0476 0.654541 26.3298 0.783194C29.1026 1.06141 31.8097 1.83481 34.3204 3.07293C37.5303 4.6559 40.3331 6.95608 42.5119 9.79553C44.6907 12.635 46.1871 15.9376 46.8853 19.4479C47.4314 22.1934 47.4778 25.0084 47.0289 27.7588C46.8213 29.0306 45.5295 29.7687 44.2848 29.4352C43.04 29.1016 42.3169 27.8222 42.4926 26.5456C42.7752 24.4926 42.7147 22.4014 42.3083 20.3583C41.7497 17.5501 40.5526 14.908 38.8096 12.6364C37.0666 10.3649 34.8243 8.52471 32.2564 7.25833C30.3881 6.33698 28.3838 5.73731 26.3276 5.47894C25.049 5.31827 24.0003 4.28862 24.0003 2.99996Z" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.1 KiB |
Reference in New Issue
Block a user