Commit Graph

198 Commits

Author SHA1 Message Date
Byeongjin Kang
0223fc6fd5
feat: add pgvector full_text_search (#7396) 2024-08-20 11:01:13 +08:00
Chengyu Yan
1f944c6eeb
feat(api): support wenxin bge-large and tao embedding model. (#7393)
Some checks are pending
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
2024-08-19 22:25:09 +08:00
Chengyu Yan
bfd905602f
feat(api): support wenxin text embedding (#7377) 2024-08-19 09:15:19 +08:00
Weaxs
3a33062405
feat: support siliconflow rerank (#7337) 2024-08-16 20:21:41 +08:00
Xiyuan Chen
c7df6783df
Revert "feat: support pinning, including, and excluding for Model Providers and Tools" (#7324)
Some checks are pending
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
2024-08-15 23:51:00 +08:00
Xiyuan Chen
7619850855
feat: support pinning, including, and excluding for Model Providers and Tools (#7283) 2024-08-15 12:58:38 +08:00
-LAN-
32dc963556
feat(api/workflow): Add Conversation.dialogue_count (#7275)
Some checks are pending
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
2024-08-15 10:53:05 +08:00
-LAN-
b5d472fad7
test(*): Avoid import from api in tests. (#7251) 2024-08-14 14:09:26 +08:00
miendinh
f104b930cf
feat: support elasticsearch vector database (#3558)
Co-authored-by: miendinh <miendinh@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
2024-08-13 17:36:20 +08:00
KVOJJJin
935e72d449
Feat: conversation variable & variable assigner node (#7222)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-08-13 14:44:10 +08:00
shAlfred
a12ddc47e7
feat: add support of speech2text function for OpenAI-API-compatible and Siliconflow (#7197)
Some checks are pending
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
2024-08-12 21:38:59 +08:00
Yanyi Liu
5b32f2e0dd
Feat: Add model provider Text Embedding Inference for embedding and rerank (#7132) 2024-08-09 19:12:13 +08:00
shAlfred
a7162240e6
feat: add text-embedding functon and LLM models to Siliconflow (#7090) 2024-08-08 17:08:28 +08:00
小羽
34a9dbe826
Feat/add 360-zhinao provider (#7069) 2024-08-08 14:23:08 +08:00
-LAN-
aad02113c6
fix(api/core/app/segments): Allow to contains boolean value in object segment (#7047) 2024-08-07 16:20:22 +08:00
JuHyung Son
2e941bb91c
add new provider Solar (#6884) 2024-08-02 20:48:09 +08:00
Jyong
44801df8f8
fix score threshold limit be None (#6900) 2024-08-02 12:10:51 +08:00
Weaxs
cc4785f094
fix: xinference reranker return_documents (#6888) 2024-08-01 19:57:53 +08:00
SiliconFlow, Inc
3e18d32ce5
add deepseek-coder-v2 in siliconflow (#6149) 2024-07-29 18:45:19 +08:00
Giga Group
c9ff0e3961
Add model hunyuan-embedding (#6657)
Co-authored-by: sun <sun@centen.cn>
2024-07-29 18:30:52 +08:00
Bowen Liang
20268708cc
chore: improve position map conversion and tolerate empty position yaml file (#6541) 2024-07-29 10:32:11 +08:00
-LAN-
6a3bef8378
feat(api/core/app/segments): Update segment types and variables (#6734)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-07-27 14:43:51 +08:00
-LAN-
5d77dc4f58
feat(api/core/app/segments/parser.py): Remove blank segment in convert_template (#6709)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-07-26 18:19:33 +08:00
-LAN-
c6996a48a4
refactor(api/core/app/segments): Support more kinds of Segments. (#6706)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-07-26 15:03:56 +08:00
xielong
ebcc07e3e9
feat: support max_retries in jina requests (#6585) 2024-07-25 13:10:39 +08:00
-LAN-
2bc0632d0d
fix(segments): Support NoneType. (#6581) 2024-07-23 17:59:32 +08:00
-LAN-
f019bc4bd7
feat(variables): Support to_object. (#6572) 2024-07-23 16:22:06 +08:00
-LAN-
5e6fc58db3
Feat/environment variables in workflow (#6515)
Co-authored-by: JzoNg <jzongcode@gmail.com>
2024-07-22 15:29:39 +08:00
ybalbert001
4a026fa352
Enhancement: add model provider - Amazon Sagemaker (#6255)
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: crazywoola <427733928@qq.com>
2024-07-18 19:32:31 +08:00
forrestlinfeng
3b5b548af3
Add Stepfun LLM Support (#6346) 2024-07-18 07:47:18 +08:00
-LAN-
fc37887a21
refactor(api/core/workflow/nodes/http_request): Remove mask_authorization_header because its alwary true. (#6379) 2024-07-17 16:52:14 +08:00
Zhuo Qiu
63e34e5227
feat: support MyScale vector database (#6092) 2024-07-11 15:21:59 +08:00
8bitpd
7c70eb87bc
feat: support AnalyticDB vector store (#5586)
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
2024-07-09 13:32:04 +08:00
-LAN-
c436454cd4
fix(configs): Update pydantic settings in config files (#6023) 2024-07-07 12:18:15 +08:00
Yeuoly
a877d4831d
Fix/incorrect parameter extractor memory (#6038) 2024-07-07 12:17:34 +08:00
-LAN-
b217ee414f
test(test_rerank): Remove duplicate test cases. (#6024) 2024-07-06 09:44:50 +08:00
orangeclk
c490bdfbf9
fix: zhipuai pytest correction (#5934) 2024-07-03 19:19:33 +08:00
Bowen Liang
dcb72e0067
chore: apply flake8-comprehensions Ruff rules to improve collection comprehensions (#5652)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-06-27 11:21:31 +08:00
Bowen Liang
31a061ebaa
chore: cleanup test_delete_by_document_id method in opensearch vdb test (#5619) 2024-06-26 17:21:36 +08:00
Jyong
43335b5c87
delete the deprecated method (#5612) 2024-06-26 12:51:50 +08:00
Bowen Liang
2a0f03a511
refactor: extract cors configs into dify config and cleanup the config class (#5507)
Co-authored-by: takatost <takatost@gmail.com>
2024-06-25 15:48:02 +08:00
非法操作
f7900f298f
chore: refactor the http executor node (#5212) 2024-06-24 16:14:59 +08:00
-LAN-
ba67206bb9
fix(api/model_runtime/azure/llm): Switch to tool_call. (#5541) 2024-06-24 15:35:21 +08:00
Xiaoguang Sun
13fcd7a901
feat: Add program_name attribute to TiDB connection (#5499)
Signed-off-by: Xiaoguang Sun <sunxiaoguang@gmail.com>
2024-06-24 14:41:07 +08:00
LXM
e8ad0339a3
fix: tongyi json output (#5396) 2024-06-22 12:25:23 +08:00
Bowen Liang
f67b164b0d
refactor: extract db configs and celery configs into dify config (#5491) 2024-06-22 10:29:56 +08:00
Bowen Liang
8890978ad3
chore: use singular style in config class name (#5489) 2024-06-22 09:54:25 +08:00
tmuife
6a09409ec9
Add Oracle23ai as a vector datasource (#5342)
Co-authored-by: walter from vm <walter.jin@oracle.com>
2024-06-22 01:48:07 +08:00
-LAN-
142dc0afd7
refactor: Remove unused code in large_language_model.py (#5433) 2024-06-20 16:20:40 +08:00
Bowen Liang
3cc6093e4b
feat: introduce pydantic-settings for config definition and validation (#5202)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-06-19 13:41:12 +08:00
baojingyu
d160d1ed02
feat: support opensearch approximate k-NN (#5322) 2024-06-19 12:44:33 +08:00
Mitsuki Ogasahara
7305713b97
fix: allow special characters in email (#5327)
Co-authored-by: crazywoola <427733928@qq.com>
2024-06-17 21:32:59 +08:00
Jyong
ba5f8afaa8
Feat/firecrawl data source (#5232)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: takatost <takatost@gmail.com>
2024-06-15 02:46:02 +08:00
Charles Zhou
8bcc5a36bb
feat: new editor user permission profile (#4435)
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
2024-06-14 20:34:25 +08:00
quicksand
4080f7b8ad
feat: support tencent vector db (#3568) 2024-06-14 19:25:17 +08:00
Jason
b7ff765d8d
Add novita.ai as model provider (#4961) 2024-06-14 18:23:06 +08:00
Bowen Liang
f976740b57
improve: mordernizing validation by migrating pydantic from 1.x to 2.x (#4592) 2024-06-14 01:05:37 +08:00
Yeuoly
e8afc416dd
improve: CI experience (#5168) 2024-06-13 23:16:28 +08:00
Bowen Liang
cdc08a434f
feat: support Chroma vector store (#5015) 2024-06-13 18:02:18 +08:00
xielong
ea69dc2a7e
feat: support hunyuan llm models (#5013)
Co-authored-by: takatost <takatost@users.noreply.github.com>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
2024-06-12 17:24:23 +08:00
Pika
ecc7f130b4
fix(typo): misspelling (#5094) 2024-06-12 17:01:21 +08:00
Bowen Liang
7fa735a43b
chore: rename vdb tests for PGVector and PGvectoRS (#4973) 2024-06-06 07:22:49 +08:00
Weaxs
0797f9bc05
feat: support tidb vector (#4588) 2024-06-05 18:19:53 +08:00
takatost
d1dbbc1e33
feat: backend model load balancing support (#4927) 2024-06-05 00:13:04 +08:00
Bowen Liang
58db719a2c
dep: bump pandas from 1.x to 2.x (#4820) 2024-06-04 13:24:28 +08:00
Bowen Liang
3542d55e67
improve: generalize tool parameter converter (#4786) 2024-06-03 21:26:58 +08:00
zeroameli
afed3610fc
fix organize agent's history messages without recalculating tokens (#4324)
Co-authored-by: chenyongzhao <chenyz@mama.cn>
2024-05-29 15:25:20 +08:00
Yeuoly
e852a21634
Feat/workflow phase2 (#4687) 2024-05-27 22:01:11 +08:00
yalei
026175c8f7
feat: update notion extractor (#3898)
Co-authored-by: duyalei <>
2024-05-24 20:30:48 +08:00
Bowen Liang
3fda2245a4
improve: extract method for safe loading yaml file and avoid using PyYaml's FullLoader (#4031) 2024-05-24 12:08:12 +08:00
Bowen Liang
5f4df34829
improve: generalize transformations and scripts of runner and preloads into TemplateTransformer (#4487) 2024-05-20 15:56:26 +08:00
Bowen Liang
083ef2e6fc
improve: exract Code Node provider for each supported scripting language (#4164) 2024-05-17 11:58:12 +08:00
sino
6e9066ebf4
feat: support doubao llm and embeding models (#4431) 2024-05-16 11:41:24 +08:00
Yeuoly
bbef964eb5
improve: code upgrade (#4231) 2024-05-13 14:39:14 +08:00
Sebastian.W
a588df4371
Add rerank model type for LocalAI provider (#3952) 2024-05-11 11:29:28 +08:00
Bowen Liang
228de1f12a
fix: miss usage of os.path.join for URL assembly and add tests on yarl (#4224) 2024-05-10 18:14:48 +08:00
Yeuoly
8578ee0864
feat: support LLM jinja2 template prompt (#3968)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-05-10 18:08:32 +08:00
LiuVaayne
875249eb00
Feat/vector db pgvector (#3879) 2024-05-10 17:20:30 +08:00
sino
4aa21242b6
feat: add volcengine maas model provider (#4142) 2024-05-08 12:45:53 +08:00
Tomy
bb7c62777d
Add support for local ai speech to text (#3921)
Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-05-07 17:14:24 +08:00
Yeuoly
d51f52a649
fix: http authorization leakage (#4146) 2024-05-07 16:56:25 +08:00
Bowen Liang
049abd698f
improve: test CodeExecutor with code templates and extract CodeLanguage enum (#4098) 2024-05-07 12:37:18 +08:00
Bowen Liang
44857702ae
test: add integration tests on CodeExecutor with the sandbox service (#4015) 2024-05-03 08:54:40 +08:00
Jyong
3e9dbe3e0a
add pgvecto_rs support and upgrade SQLAlchemy (#3833) 2024-04-29 11:58:17 +08:00
Yeuoly
e7b4d024ee
optimize: code node has a bad error message (#3949) 2024-04-28 17:40:29 +08:00
Bowen Liang
661b30784e
chore: skip warning messages when pytest auto-collecting the vdb test class by removing Test prefix (#3906) 2024-04-27 16:36:09 +08:00
Bowen Liang
045827043d
test: improve vector store tests (#3855) 2024-04-26 19:18:42 +08:00
Bowen Liang
45dd1683fd
test: add tests covering all methods of vector store (#3849) 2024-04-25 22:27:30 +08:00
Bowen Liang
86e7330fa2
test: refactor vdb tests by visitor design pattern (#3838) 2024-04-25 18:55:49 +08:00
Bowen Liang
c54fcfb45d
extract enum type for tenant account role (#3788) 2024-04-25 18:20:08 +08:00
Bowen Liang
9cec8c1750
test: add unit tests for vector stores of Milvus, Qdrant and Weaviate (#3688) 2024-04-24 21:52:42 +08:00
takatost
3da179f77b
feat: add conversation_id and user_id in chatflow/workflow system vars (#3771)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-04-24 17:20:01 +08:00
Bowen Liang
a34e8cb0bd
test: add test for PKCS1OAEP_Cipher with gmpy2 (#3760) 2024-04-24 17:15:31 +08:00
Bowen Liang
d9b821cecc
chore: apply ruff rules on tests and app.py (#3605) 2024-04-18 20:24:05 +08:00
Yeuoly
c2acb2be60
feat: code (#3557) 2024-04-18 08:00:02 +08:00
takatost
5b447d61a6
feat: refactor tongyi models (#3496) 2024-04-15 22:28:32 +08:00
takatost
259aa97d8b
fix: test env key missing or wrong (#3430) 2024-04-12 19:08:48 +08:00
Yeuoly
a258a90291
feat: gemini pro function call (#3406) 2024-04-12 16:38:02 +08:00
Leo Q
9c01bcb3e5
feat: support setting database used in Milvus (#3003) 2024-04-09 15:39:36 +08:00
takatost
7753ba2d37
FEAT: NEW WORKFLOW ENGINE (#3160)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: jyong <718720800@qq.com>
2024-04-08 18:51:46 +08:00