Austin/firecrawl
1
0
Fork 0
mirror of https://github.com/mendableai/firecrawl.git synced 2024-11-16 03:32:22 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
969 Commits 273 Branches 1 Tag 48 MiB
fc3328f3d1
Commit Graph

969 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolas
fc3328f3d1 Update index.ts 2024-07-12 19:12:56 -04:00
Nicolas
fd18f2269b Nick: slack alerts 2024-07-12 19:07:59 -04:00
Nicolas
214a6ee608
Merge pull request #396 from mendableai/bugfix/crawl-docker 
[BUG] Fixed docker self hosting issue
 2024-07-12 16:48:16 -04:00
rafaelsideguide
f453bcf17c bugfix docker self hosting 2024-07-12 16:51:20 -03:00
Nicolas
f2a08e0ab8
Merge pull request #395 from kun432/kun432/fix-use-db-auth-check-for-selfhost 
Fix USE_DB_AUTHENTICATION checks for self-host
 2024-07-12 15:39:35 -04:00
Nicolas
0ddaac6ae0 Nick: fixed the other instances as well 2024-07-12 15:39:10 -04:00
Kuniaki Shimizu
bd986a453c fix USE_DB_AUTHENTICATION checks 2024-07-13 03:50:46 +09:00
Nicolas
b5b75086c1 Update index.ts 2024-07-12 10:44:14 -04:00
Nicolas
c07a1912e9
Merge pull request #393 from mendableai/mog/job-stuck-fix 
fix(bull): requeue jobs after restart
 2024-07-12 10:43:02 -04:00
Gergo Moricz
0d3e09e798 fix: try-catch job removal 2024-07-12 16:35:50 +02:00
Gergő Móricz
69d724714f
Merge branch 'main' into mog/job-stuck-fix 2024-07-12 16:33:34 +02:00
Nicolas
c3eecf7b9f Update index.ts 2024-07-12 10:22:06 -04:00
Gergo Moricz
10957b748b fix(bull): requeue jobs after restart 2024-07-12 13:55:53 +02:00
Nicolas
961b27811d
Merge pull request #386 from mendableai/feat/fire-engine-fallback-for-sitemap 
[Feat] Added fire-engine fallback for getting sitemaps
 2024-07-11 20:38:01 -04:00
Nicolas
84de63dbeb
Merge pull request #375 from StefanTerdell/self-host-qol 
Self-hosting quality of life fixes
 2024-07-11 20:37:39 -04:00
Nicolas
30c1118713
Merge pull request #326 from mendableai/feat/save-docs-on-supabase 
[Feat] Added implementation for saving docs on supabase
 2024-07-11 20:27:41 -04:00
Gergo Moricz
7e3a368684 fix: unpause globally 2024-07-12 00:05:35 +02:00
Gergo Moricz
ee1d41406e feat: unpause by http request 2024-07-11 23:56:36 +02:00
Gergo Moricz
f64a2d8668 fix: rename fly tomls to original 2024-07-11 23:21:02 +02:00
Gergo Moricz
bd84290b9e fix: reenable hyperdx 2024-07-11 23:20:51 +02:00
Gergo Moricz
09bca05b20 feat: fix iteration 3 (actually works) 2024-07-11 23:14:15 +02:00
Gergo Moricz
9cd7d79b64 feat: avoid double SIGINT crashing 2024-07-11 20:35:15 +02:00
Gergo Moricz
eaa8db4b19 fix(fly): raise kill timeout for graceful shutdown 2024-07-11 20:09:06 +02:00
Gergo Moricz
bffb9f8fd0 feat: stuck job restoration iteration 2 2024-07-11 20:08:21 +02:00
rafaelsideguide
86d0e88a91 removed hyperdx (they also have graceful shutdown) and tried to change the process for running on server. It didn't work. 2024-07-10 18:29:55 -03:00
rafaelsideguide
7c3cc89a80 Merge branch 'feat/save-docs-on-supabase' of https://github.com/mendableai/firecrawl into feat/save-docs-on-supabase 2024-07-09 18:48:29 -03:00
rafaelsideguide
9ad06fdf56 added fire-engine fallback for getting sitemaps 2024-07-09 16:07:53 -03:00
Gergo Moricz
1a07e9d23b feat: pick up and commit interrupted jobs from/to DB 2024-07-09 15:57:38 +02:00
Gergo Moricz
6a524e1bae feat: pick up and commit interrupted jobs from/to DB 2024-07-09 14:56:47 +02:00
Gergo Moricz
77aa46588f feat: graceful exit handler 2024-07-09 14:29:32 +02:00
Nicolas
fcc67a3c9e
Merge pull request #370 from mendableai/bug/fixing-cicd 
dependabot for security checks, fixed crawl test
 2024-07-08 19:26:18 -03:00
Eric Ciarla
51e6579d70 Update README.md 2024-07-08 17:09:17 -04:00
Eric Ciarla
f8e79dd729 Update integrations 2024-07-08 17:02:20 -04:00
Eric Ciarla
afb49e21e7 Update SDKs to MIT license 2024-07-08 13:37:53 -04:00
Nicolas
002bfdf639
Merge pull request #374 from StefanTerdell/fix-372 
Only check Supabase if configured to when cancelling job
 2024-07-07 22:53:15 -03:00
Stefan Terdell
188fe56203 Optional jobId webhook URL templating 2024-07-07 15:11:45 +02:00
Stefan Terdell
fcedcccf2d Include self hosted webhook env and allow connections to localhost 2024-07-07 15:10:35 +02:00
Stefan Terdell
a2ae5f81d9 Only check Supabase if configured to 2024-07-07 15:06:31 +02:00
Nicolas
e779dbbe26
Merge pull request #371 from mendableai/feat/issue-342 
[Feat] Added veeva to special case params
 2024-07-06 12:25:22 -03:00
rafaelsideguide
c2bba54b4f Added veeva to special case params 2024-07-05 16:58:07 -03:00
rafaelsideguide
a2cdc520e6 dependabot for security checks, fixed crawl test 2024-07-05 14:49:03 -03:00
Rafael Miller
2b36de605a
Merge pull request #349 from mendableai/dependabot/npm_and_yarn/apps/api/prod-deps-5b38a50718 
apps/api(deps): bump the prod-deps group in /apps/api with 28 updates
 2024-07-05 14:05:50 -03:00
rafaelsideguide
0ab6cef471 Merge remote-tracking branch 'origin/main' into dependabot/npm_and_yarn/apps/api/prod-deps-5b38a50718 2024-07-05 14:00:10 -03:00
Nicolas
914897c9d2 Merge branch 'main' into feat/save-docs-on-supabase 2024-07-05 12:27:22 -03:00
rafaelsideguide
538dc63035 Fixing rate-limiter-flexible package version 
Redis version <3.0.2 throws TS bug:
https://github.com/animir/node-rate-limiter-flexible/issues/228
 2024-07-05 12:12:00 -03:00
rafaelsideguide
d4e1a9724f Merge branch 'dependabot/npm_and_yarn/apps/test-suite/dev-deps-ffe2a14739' 2024-07-05 11:16:49 -03:00
Rafael Miller
c570fa92cf
Merge pull request #347 from mendableai/dependabot/npm_and_yarn/apps/test-suite/prod-deps-d16537e256 
apps/test-suite(deps): bump the prod-deps group in /apps/test-suite with 6 updates
 2024-07-05 10:18:35 -03:00
Nicolas
a7aaa7e57e
Update SELF_HOST.md 2024-07-04 17:49:09 -03:00
Nicolas
5551f704ba
Merge pull request #362 from snippet/self-host-docs-ts-playwright-service 
(Docs) Self Host added new ts playwright service instructions
 2024-07-04 17:48:41 -03:00
Nicolas
8f46b8218a
Merge pull request #361 from snippet/ts-playwright-service-docker 
setting up docker to ts playwright service
 2024-07-04 17:47:41 -03:00