Commit Graph

27 Commits

Author SHA1 Message Date
Bowen Liang
39c14ec7c1
improve: unify Excel files parsing in either xls or xlsx file format by Pandas (#4965) 2024-06-20 16:14:49 +08:00
takatost
12c815c597
fix: ExtractSetting optional value missing None as default val (#5238) 2024-06-15 02:58:47 +08:00
Jyong
ba5f8afaa8
Feat/firecrawl data source (#5232)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: takatost <takatost@gmail.com>
2024-06-15 02:46:02 +08:00
Bowen Liang
f976740b57
improve: mordernizing validation by migrating pydantic from 1.x to 2.x (#4592) 2024-06-14 01:05:37 +08:00
Jyong
3b60c28b3a
deal the external image when extract docx image (#5024) 2024-06-07 20:00:39 +08:00
YC
9f8ca75a81
fixing a bug of handling header row when parsing xls file, and tune xls/xlsx parsing result to be more structured (#3600) 2024-06-05 15:28:43 +08:00
Bowen Liang
58db719a2c
dep: bump pandas from 1.x to 2.x (#4820) 2024-06-04 13:24:28 +08:00
Oliver Lee
176d91937d
fix 'NoneType' and new ContentType supported. (#4818) 2024-05-31 14:19:33 +08:00
yalei
026175c8f7
feat: update notion extractor (#3898)
Co-authored-by: duyalei <>
2024-05-24 20:30:48 +08:00
Jyong
233c4150d1
support images and tables extract from docx (#4619) 2024-05-23 18:05:23 +08:00
majian
b5204111da
Add UNSTRUCTURED_API_KEY env support (#4369) 2024-05-20 13:14:17 +08:00
Charlie.Wei
97b65f9b4b
Optimize webscraper (#4392)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2024-05-15 15:23:16 +08:00
Bowen Liang
7919596a21
fix: UP031 style rule violation (#3866) 2024-04-26 11:24:08 +08:00
Jyong
0737e930cb
chore: remove Langchain tools import (#3407) 2024-04-12 16:26:09 +08:00
chenxu9741
ad65c891e7
add xls file suport (#3321) 2024-04-12 14:53:44 +08:00
LiuVaayne
b00466f025
feat:api Add support for extracting EPUB files in ExtractProcessor (#3254)
Co-authored-by: crazywoola <427733928@qq.com>
2024-04-12 11:25:02 +08:00
Jyong
6164604462
fix dataset retrival in dataset mode (#3334) 2024-04-11 02:11:21 +08:00
Jyong
9eba6ffdd4
Optimize csv and excel extract (#3155)
Co-authored-by: jyong <jyong@dify.ai>
2024-04-08 16:34:43 +08:00
Vikey Chen
e4f686deb7
fix unstructured api,remove unused parameters (#3056) 2024-04-03 21:00:20 +08:00
Jyong
b0b0cc045f
add mutil-thread document embedding (#3016)
Co-authored-by: jyong <jyong@dify.ai>
2024-03-28 17:02:35 +08:00
Weaxs
20bd49285b
excel: get keys from every sheet (#2796) 2024-03-12 16:59:25 +08:00
Bowen Liang
b163545771
Use python-docx to extract docx files (#2654) 2024-03-07 18:24:55 +08:00
Charlie.Wei
fa7ba30ba3
Fix rebuild index&csv parsing (#2705)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2024-03-06 11:33:32 +08:00
takatost
a4d86496e1
fix: notion extractor raise 'NoneType' object has no attribute 'curre… (#2608) 2024-02-28 17:08:27 +08:00
Jyong
5b953c1ef2
Fix some RAG bugs (#2570)
Co-authored-by: jyong <jyong@dify.ai>
2024-02-27 11:39:05 +08:00
Jyong
91ea6fe4ee
Fix/langchain document schema (#2539)
Co-authored-by: jyong <jyong@dify.ai>
2024-02-23 14:16:44 +08:00
Jyong
6c4e6bf1d6
Feat/dify rag (#2528)
Co-authored-by: jyong <jyong@dify.ai>
2024-02-22 23:31:57 +08:00