Commit Graph

372 Commits

Author SHA1 Message Date
takatost
0c5892bcb6
fix: zhipuai chatglm turbo prompts must user, assistant in sequence (#1899) 2024-01-04 10:39:21 +08:00
takatost
bb7af56e69
fix: zhipuai history format wrong (#1897) 2024-01-04 01:30:23 +08:00
Chenhe Gu
77f9e8ce0f
add example api url endpoint in placeholder (#1887)
Co-authored-by: takatost <takatost@gmail.com>
2024-01-04 01:16:51 +08:00
Jyong
5ca4c4a44d
add qdrant client timeout limit (#1894)
Co-authored-by: jyong <jyong@dify.ai>
2024-01-03 22:23:04 +08:00
takatost
6333cf43a8
fix: anthropic messages empty raise errors (#1893) 2024-01-03 22:12:14 +08:00
Garfield Dai
91ee62d1ab
fix: huggingface and replicate. (#1888) 2024-01-03 18:29:44 +08:00
takatost
ede69b4659
fix: gemini block error (#1877)
Co-authored-by: chenhe <guchenhe@gmail.com>
2024-01-03 17:45:15 +08:00
waltcow
61aaeff413
Fix variable name in AgentApplicationRunner (#1884) 2024-01-03 17:44:41 +08:00
crazywoola
4d502ea44d
fix: openai embedding list out of bound (#1879) 2024-01-03 15:30:22 +08:00
Yeuoly
9134849744
fix: remove tiktoken from text splitter (#1876) 2024-01-03 13:02:56 +08:00
Garfield Dai
fcf8512956
fix: more like this. (#1875) 2024-01-03 12:51:19 +08:00
takatost
ae975b10e9
fix: openai origin credential not start with { (#1874) 2024-01-03 12:10:43 +08:00
Yeuoly
b43f1441a9
Fix/model runtime (#1873) 2024-01-03 11:36:57 +08:00
takatost
5a2aa83030
fix: ciphertext error (#1872) 2024-01-03 11:20:46 +08:00
Yeuoly
c6d59681ff
fix: xinference secret server_url (#1869) 2024-01-03 10:01:11 +08:00
takatost
3b668c0bb1
fix: IntegrityError import wrong (#1868) 2024-01-03 09:35:03 +08:00
takatost
4aed1fe8a8
fix: Azure text-davinci-003 missing (#1867) 2024-01-03 09:27:09 +08:00
takatost
2381264a3f
fix: provider create cause IntegrityError (#1866) 2024-01-03 09:12:53 +08:00
takatost
7be77c19f5
fix: default model parameter precision (#1864) 2024-01-03 08:52:22 +08:00
takatost
82247c0f14
fix: agent strategy missing in app model config (#1863) 2024-01-03 08:43:51 +08:00
takatost
d069c668f8
Model Runtime (#1858)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-01-02 23:42:00 +08:00
waltcow
06d2d8cea3
Refactor BaseVectorIndex delete method (#1853) 2023-12-30 21:49:01 +08:00
Charlie.Wei
6b57e4e0ff
Fix chitchat lost context (#1828)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2023-12-23 10:05:17 +08:00
Bannings
0b7e0cadc0
Fix Azure OpenAI Provider BASE_MODELS (#1813)
Co-authored-by: xifan <xifan@gaoding.com>
2023-12-22 14:17:15 +08:00
Charlie.Wei
64642fabc4
Parse base64 eml file (#1796)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2023-12-21 13:18:58 +08:00
Jyong
df1509983c
ppt & pptx improve (#1790)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-19 18:11:27 +08:00
Jyong
5e34f938c1
Feat/add unstructured support (#1780)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-18 23:24:06 +08:00
Jyong
2fd56cb01c
Fix/vdb index issue (#1776)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-18 21:33:54 +08:00
Jyong
ebc2cdad2e
fix annotation query exception (#1771)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-18 16:48:34 +08:00
zxhlyh
5bb841935e
feat: custom webapp logo (#1766) 2023-12-18 16:25:37 +08:00
Jyong
a71f2863ac
Annotation management (#1767)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-18 13:10:05 +08:00
Qiwen Tong
c09184fd94
update bm25 search properties (#1758)
Co-authored-by: Blade <zhangxiaobin@unixyz.cn>
2023-12-15 12:28:03 +08:00
Charlie.Wei
b0d8d196e1
azure openai add gpt-4-1106-preview、gpt-4-vision-preview models (#1751)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2023-12-14 09:55:30 +08:00
Garfield Dai
6b499b9a16
remove stripe and anthropic. (#1746) 2023-12-12 17:59:07 +08:00
Jyong
d5695b3170
check rerank document is not empty (#1740)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-11 16:12:11 +08:00
crazywoola
994fceece3
fix: qa regex (#1738) 2023-12-11 15:53:37 +08:00
Jyong
8c451eb0e6
fix only full text search in app issue (#1736)
Co-authored-by: jyong <jyong@dify.ai>
2023-12-11 15:34:29 +08:00
Pascal M
bc54cdc537
refactor: typo in dataset docstore (#1711) 2023-12-07 09:24:52 +08:00
Pascal M
5d10cf0fe6
fix: error Class 'builtins.list' is not mapped (#1710) 2023-12-07 09:24:39 +08:00
Yeuoly
d3a2c0ed34
fix wrong syntax of type definitions (#1678) 2023-12-03 20:59:13 +08:00
Wen O.Y
a1cd043fdc
fix: Incorrect order of embedded documents in CacheEmbedding (#1671) 2023-12-03 19:07:00 +08:00
WangBooth
22bc9ddc73
Hotfix/fix documents index mismatch error in rerank (#1662)
Co-authored-by: baomi.wbm <baomi.wbm@dtwave-inc.com>
2023-11-30 22:03:20 +08:00
zxhlyh
451af66be0
feat: add jina embedding (#1647)
Co-authored-by: takatost <takatost@gmail.com>
2023-11-29 14:58:11 +08:00
WangBooth
53be4d2712
fix #1637 (#1638)
Co-authored-by: baomi.wbm <baomi.wbm@dtwave-inc.com>
2023-11-28 20:05:50 +08:00
Jyong
74b2260ba6
fix score_threshold_enabled name (#1626)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-27 15:34:45 +08:00
takatost
93467cb363
fix: dataset tool missing in n-to-1 retrieve mode (#1617) 2023-11-25 16:04:22 +08:00
takatost
ea526d0822
feat: chatglm3 support (#1616) 2023-11-25 15:37:07 +08:00
takatost
0e627c920f
feat: xinference rerank model support (#1615) 2023-11-25 03:56:00 +08:00
Jyong
a5b80c9d1f
Fix/multi thread parameter (#1604) 2023-11-22 18:31:29 +08:00
Jyong
f704094a5f
fix hybrid search when document is none (#1603)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-22 17:53:42 +08:00
Jyong
b930716745
fix weaviate hybrid search issue (#1600)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-22 16:41:20 +08:00
takatost
4a55d5729d
feat: add anthropic claude-2.1 support (#1591) 2023-11-22 01:46:19 +08:00
takatost
c9368925a3
feat: add supported_model_types field and filter in provider list (#1581) 2023-11-21 15:06:47 +08:00
Jyong
0d9ce1bab0
fix multi retrieval with resource score issue (#1578)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-21 13:47:55 +08:00
Jyong
0e08526428
fix hybrid search reranking check (#1563)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-18 17:06:28 +08:00
Jyong
4588831bff
Feat/add retriever rerank (#1560)
Co-authored-by: jyong <jyong@dify.ai>
2023-11-17 22:13:37 +08:00
takatost
d654770732
feat: supports for new version of openllm (#1554) 2023-11-17 14:07:36 +08:00
Garfield Dai
8835435558
fix: change model mode. (#1520) 2023-11-13 23:13:01 +08:00
takatost
41d0a8b295
feat: [backend] vision support (#1510)
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
2023-11-13 22:05:46 +08:00
crazywoola
d0e1ea8f06
1506 remove duplicated code (#1511) 2023-11-13 19:05:32 +08:00
takatost
9de67c586f
feat: update free plan rules of spark (#1515) 2023-11-13 17:00:36 +08:00
takatost
d7ae86799c
feat: support basic feature of OpenAI new models (#1476) 2023-11-07 04:05:59 -06:00
takatost
059c089f93
fix: external data tool batch retrieve bug (#1472) 2023-11-07 01:28:22 -06:00
Garfield Dai
db43ed6f41
feat: add api-based extension & external data tool & moderation backend (#1403)
Co-authored-by: takatost <takatost@gmail.com>
2023-11-06 19:36:16 +08:00
takatost
4dfbcd0b4e
feat: support chatglm_turbo model #1443 (#1460) 2023-11-06 04:33:05 -06:00
takatost
076f3289d2
feat: add spark v3.0 llm support (#1434) 2023-10-31 03:13:11 -05:00
takatost
e122d677ad
fix: return wrong when init 0 quota in trial provider (#1394) 2023-10-21 14:02:38 -05:00
takatost
4c63cbf5b1
feat: adjust anthropic (#1387) 2023-10-20 02:27:46 -05:00
Garfield Dai
fe14130b3c
refactor advanced prompt core. (#1350) 2023-10-18 20:02:52 +08:00
wayne.wang
52ebffa857
fix: app config zhipu chatglm_std model, but it still use chatglm_lit… (#1377)
Co-authored-by: wayne.wang <wayne.wang@beibei.com>
2023-10-18 05:07:36 -05:00
takatost
7c9b585a47
feat: support weixin ernie-bot-4 and chat mode (#1375) 2023-10-18 02:35:24 -05:00
takatost
07285e5f8b
feat: optimize completion model agent (#1364) 2023-10-17 06:54:59 -05:00
Garfield Dai
695841a3cf
Feat/advanced prompt enhancement (#1340) 2023-10-13 16:47:01 +08:00
takatost
3efaa713da
feat: use xinference client instead of xinference (#1339) 2023-10-13 02:46:09 -05:00
takatost
9822f687f7
fix: max tokens of OpenAI gpt-3.5-turbo-instruct to 4097 (#1338) 2023-10-13 02:07:07 -05:00
takatost
f4be2b8bcd
fix: raise error in minimax stream generate (#1336) 2023-10-12 23:48:28 -05:00
Garfield Dai
42a5b3ec17
feat: advanced prompt backend (#1301)
Co-authored-by: takatost <takatost@gmail.com>
2023-10-12 10:13:10 -05:00
takatost
2d1cb076c6
fix: dataset segment not exist return agent response (#1329) 2023-10-12 04:40:20 -05:00
Jyong
289c93d081
Feat/improve document delete logic (#1325)
Co-authored-by: jyong <jyong@dify.ai>
2023-10-12 13:30:44 +08:00
takatost
9cba1c8bf4
fix: retriever_resource missing (#1317) 2023-10-11 14:37:11 -05:00
takatost
cbf095465c
feat: remove llm client use (#1316) 2023-10-11 14:02:53 -05:00
Jyong
7f6ad9653e
Fix/grpc gevent compatible (#1314)
Co-authored-by: jyong <jyong@dify.ai>
2023-10-11 20:48:35 +08:00
takatost
2851a9f04e
feat: optimize minimax llm call (#1312) 2023-10-11 07:17:41 -05:00
takatost
c536f85b2e
fix: compatibility issues with the tongyi model. (#1310) 2023-10-11 05:16:26 -05:00
takatost
b1352ff8b7
feat: using random sampling to check if it violates the review mechan… (#1308) 2023-10-11 04:11:20 -05:00
Jyong
5003db987d
milvus secure check fix (#1305)
Co-authored-by: jyong <jyong@dify.ai>
2023-10-11 13:11:06 +08:00
Jyong
07aab5e868
Feat/add milvus vector db (#1302)
Co-authored-by: jyong <jyong@dify.ai>
2023-10-10 21:56:24 +08:00
takatost
875dfbbf0e
fix: openllm completion start with prompt, remove it (#1303) 2023-10-10 04:44:19 -05:00
takatost
8480b0197b
fix: prompt for baichuan text generation models (#1299) 2023-10-10 13:01:18 +08:00
takatost
4ab4bcc074
feat: support openllm embedding (#1293) 2023-10-09 23:09:35 -05:00
takatost
1d4f019de4
feat: add baichuan llm support (#1294)
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
2023-10-09 23:09:26 -05:00
takatost
024250803a
feat: move login_required wrapper outside (#1281) 2023-10-08 05:21:32 -05:00
takatost
373e90ee6d
fix: detached model in completion thread (#1269) 2023-10-02 22:27:25 +08:00
takatost
41d4c5b424
fix: count down thread in completion db not commit (#1267) 2023-10-02 10:19:26 +08:00
takatost
8606d80c66
fix: request timeout when openai completion (#1265) 2023-10-01 16:00:23 +08:00
takatost
a31466d34e
fix: db session not commit before long llm call running (#1251) 2023-09-27 21:40:26 +08:00
takatost
d38eac959b
fix: wenxin model name invalid when llm call (#1248) 2023-09-27 16:29:13 +08:00
Jyong
46154c6705
Feat/dataset service api (#1245)
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
2023-09-27 16:06:32 +08:00
Garfield Dai
18c710c906
feat: support binding context var (#1227)
Co-authored-by: Joel <iamjoel007@gmail.com>
2023-09-27 14:53:22 +08:00
zxhlyh
227f9fb77d
Feat/api jwt (#1212) 2023-09-25 12:49:16 +08:00
Jyong
724e053732
Fix/qdrant data issue (#1203)
Co-authored-by: jyong <jyong@dify.ai>
2023-09-22 14:21:26 +08:00
Garfield Dai
e409895c02
Feat/huggingface embedding support (#1211)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
2023-09-22 13:59:02 +08:00
takatost
32d9b6181c
fix: transaction not commit during long LLM calls (#1213) 2023-09-22 12:43:06 +08:00
takatost
2b018fade2
fix: transaction hangs due to message commit block during long LLM calls (#1206) 2023-09-21 11:22:10 +08:00
takatost
435f804c6f
fix: gpt-3.5-turbo-instruct context size to 8192 (#1196) 2023-09-19 02:10:22 +08:00
takatost
ae3f1ac0a9
feat: support gpt-3.5-turbo-instruct model (#1195) 2023-09-19 02:05:04 +08:00
Jyong
269a465fc4
Feat/improve vector database logic (#1193)
Co-authored-by: jyong <jyong@dify.ai>
2023-09-18 18:15:41 +08:00
takatost
827c97f0d3
feat: add zhipuai (#1188) 2023-09-18 17:32:31 +08:00
takatost
c8bd76cd66
fix: inference embedding validate (#1187) 2023-09-16 03:09:36 +08:00
takatost
f9082104ed
feat: add hosted moderation (#1158) 2023-09-12 10:26:12 +08:00
Jyong
642842d61b
Feat:dataset retiever resource (#1123)
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
2023-09-10 15:17:43 +08:00
Joel
2d5ad0d208
feat: support optional query content (#1097)
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
2023-09-10 00:12:34 +08:00
takatost
c4d8bdc3db
fix: hf hosted inference check (#1128) 2023-09-09 00:29:48 +08:00
takatost
a7cdb745c1
feat: support spark v2 validate (#1086) 2023-09-01 20:53:32 +08:00
takatost
73c86ee6a0
fix: prompt of title generation (#1084) 2023-09-01 14:55:58 +08:00
takatost
33562a9d8d
feat: optimize prompt (#1080) 2023-09-01 11:46:06 +08:00
takatost
a199fa6388
feat: optimize high load sql query of document segment (#1078) 2023-09-01 10:52:39 +08:00
takatost
4c8608dc61
feat: optimize conversation title generation output must be a valid JSON (#1077) 2023-09-01 10:31:42 +08:00
takatost
df6604a734
feat: optimize generation of conversation title (#1075) 2023-09-01 02:28:37 +08:00
takatost
78e26f8b75
fix: summary no docs (#1073) 2023-08-31 20:19:26 +08:00
takatost
2191312bb9
fix: segments query missing idx hit (#1072) 2023-08-31 19:39:44 +08:00
takatost
fcc6b41ab7
feat: decrease claude model request time by set max top_k to 10 (#1071) 2023-08-31 18:23:44 +08:00
takatost
d75e8aeafa
feat: disable anthropic retry (#1067) 2023-08-31 16:44:46 +08:00
takatost
2eba98a465
feat: optimize anthropic connection pool (#1066) 2023-08-31 16:18:59 +08:00
takatost
a7a7aab7a0
fix: csv import error (#1063) 2023-08-31 15:42:28 +08:00
yezhwi
d33a269548
refactor(file extractor): file extractor (#1059) 2023-08-31 14:45:31 +08:00
yezhwi
8b8e510bfe
fix: handle AttributeError for datasets and index (#1052) 2023-08-30 11:14:16 +08:00
takatost
417c19577a
feat: add LocalAI local embedding model support (#1021)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
2023-08-29 22:22:02 +08:00
Jyong
a43e80dd9c
add qdrant migration (#1046)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-29 10:37:04 +08:00
WangBooth
ad5f27bc5f
fix openpyxl dimensions error (#1041) 2023-08-29 10:36:48 +08:00
takatost
7b3314c5db
fix: dataset desc (#1045) 2023-08-29 09:07:27 +08:00
Jyong
a55ba6e614
Fix/ignore economy dataset (#1043)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-29 03:37:45 +08:00
takatost
02452421d5
fix: pub generate message text return null (#1037) 2023-08-28 16:43:54 +08:00
takatost
0796791de5
feat: hf inference endpoint stream support (#1028) 2023-08-26 19:48:34 +08:00
takatost
6c148b223d
fix: dataset query truncated (#1026) 2023-08-26 17:35:17 +08:00
Uranus
2d9616c29c
fix: xinference last token being ignored (#1013) 2023-08-25 18:15:05 +08:00
Jyong
2d604d9330
Fix/filter empty segment (#1004)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-25 15:50:29 +08:00
Jyong
b1fd1b3ab3
Feat/vector db manage (#997)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-24 21:27:31 +08:00
takatost
9ae91a2ec3
feat: optimize xinference request max token key and stop reason (#998) 2023-08-24 18:11:15 +08:00
takatost
701e2b334f
feat: remove unnecessary prompt of baichuan (#993) 2023-08-24 15:30:59 +08:00
takatost
bd3a9b2f8d
fix: xinference-chat-stream-response (#991) 2023-08-24 14:39:34 +08:00
takatost
18d3877151
feat: optimize xinference stream (#989) 2023-08-24 13:58:34 +08:00
takatost
53e83d8697
feat: optimize baichuan prompt (#988) 2023-08-24 12:07:10 +08:00
takatost
2c30d19cbe
feat: add baichuan prompt (#985) 2023-08-24 10:22:36 +08:00
takatost
9b247fccd4
feat: adjust hf max tokens (#979) 2023-08-23 22:24:50 +08:00
takatost
a76fde3d23
feat: optimize hf inference endpoint (#975) 2023-08-23 19:47:50 +08:00
Jyong
1fc57d7358
normalize embedding (#974)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-23 19:10:11 +08:00
takatost
5cba2e7087
fix: web reader tool retrieve content empty (#957) 2023-08-22 18:01:16 +08:00
Jyong
5623839c71
update document segment (#950)
Co-authored-by: jyong <jyong@dify.ai>
2023-08-22 17:59:24 +08:00
takatost
78d3aa5fcd
fix: embedding init err (#956) 2023-08-22 17:43:59 +08:00