Austin/firecrawl
1
0
Fork 0
mirror of https://github.com/mendableai/firecrawl.git synced 2024-11-16 11:42:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
1,738 Commits 273 Branches 1 Tag 48 MiB
030ecab6ee
Commit Graph

1071 Commits

Author SHA1 Message Date
Nicolas
030ecab6ee Update rate-limiter.ts 2024-09-13 18:09:59 -04:00
Gergo Moricz
000a316cc3 fix(fire-engine): poll more frequently
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Details
Fly Deploy / Test Suite (push) Blocked by required conditions
Details
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Rust SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Deploy app (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Rust SDK (push) Blocked by required conditions
Details
2024-09-13 16:41:27 +02:00
Gergo Moricz
f7c4cee404 fix(queue-worker): don't send LLM extract hallucination error to Sentry 2024-09-13 14:02:49 +02:00
Gergő Móricz
d497284b40 feat(api/queue): auto-remove completed jobs after 25 hours 2024-09-12 19:47:22 +02:00
Nicolas
620b02f9ca Nick: 2024-09-12 12:51:14 -04:00
Nicolas
e1171ade1b Merge branch 'main' into mog/fix-status-job-get 2024-09-12 11:35:36 -04:00
Nicolas
4cd1065ae2 Update rate-limiter.ts 2024-09-11 14:03:34 -04:00
Gergő Móricz
97ffabff3a fix(v1): converting bad docs always gives null 2024-09-10 21:21:20 +02:00
Gergő Móricz
83a165db0f fix(v0/scrape): ensure url is string 2024-09-10 21:18:53 +02:00
Gergő Móricz
ad1a6fbc74 fix(v1/map): handle invalid URLs gracefully 2024-09-10 19:41:01 +02:00
Gergő Móricz
4ebc35c9dd fix(crawl-status): add success: true 2024-09-10 18:59:09 +02:00
Nicolas
a7406031fa Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 06:53:30 -03:00
Nicolas
a17e1cac92 Rate bump 2024-09-10 06:53:24 -03:00
Gergo Moricz
b4dbf75537 fix(v1): check if url is string in blocklistMiddleware 
Fixes FIRECRAWL-SCRAPER-JS-9Z
 2024-09-10 10:25:14 +02:00
Gergo Moricz
26f2095de6 fix(v1): proper Invalid URL handling 2024-09-10 09:24:23 +02:00
Gergo Moricz
f8fbc71f91 fix(supabase-jobs): do not use RPCs 
RPCs are more failure-prone for this use case than regular queries are.
 2024-09-10 09:20:18 +02:00
Gergo Moricz
a6bcf7b438 fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supabase 2024-09-10 08:53:40 +02:00
Nicolas
17e419a7fb Nick: 2024-09-09 21:06:23 -03:00
Nicolas
ca9a781eb7 Update index.ts 2024-09-09 12:27:55 -03:00
Nicolas
22a5e85899 Update index.ts 2024-09-09 12:26:55 -03:00
Nicolas
60a15d00eb Update types.ts 2024-09-08 16:39:12 -03:00
Nicolas
fbdfa1256b Update credit_billing.ts 2024-09-08 13:07:10 -03:00
Nicolas
48c665519e Update credit_billing.ts 2024-09-07 13:42:45 -03:00
Nicolas
32097fa29d Merge branch 'main' of https://github.com/mendableai/firecrawl
Some checks failed
Fly Deploy / Pre-deploy checks (push) Has been cancelled
Details
Clean Every 30 Minutes Before 24h Completed Jobs / clean-jobs (push) Has been cancelled
Details
Fly Deploy Direct / Deploy app (push) Has been cancelled
Details
Check Queues / clean-jobs (push) Has been cancelled
Details
Fly Deploy / Test Suite (push) Has been cancelled
Details
Fly Deploy / Python SDK Tests (push) Has been cancelled
Details
Fly Deploy / JavaScript SDK Tests (push) Has been cancelled
Details
Fly Deploy / Go SDK Tests (push) Has been cancelled
Details
Fly Deploy / Rust SDK Tests (push) Has been cancelled
Details
Fly Deploy / Deploy app (push) Has been cancelled
Details
Fly Deploy / Build and publish Python SDK (push) Has been cancelled
Details
Fly Deploy / Build and publish JavaScript SDK (push) Has been cancelled
Details
Fly Deploy / Build and publish Rust SDK (push) Has been cancelled
Details
2024-09-07 13:12:52 -03:00
Nicolas
5758af3291 Update website_params.ts 2024-09-07 13:12:46 -03:00
Nicolas
79870e7305 Update excludeTags.ts 2024-09-06 20:15:26 -03:00
rafaelsideguide
82d6bf4ec8 feat(go-parser): singleton 2024-09-05 16:14:21 -03:00
Nicolas
cb630bfc34 Update fireEngine.ts 2024-09-05 14:24:10 -03:00
rafaelsideguide
8c1097e9e1 fix: pageOptions 2024-09-05 14:16:31 -03:00
rafaelsideguide
b301ffc922 added missing variables 2024-09-05 13:57:26 -03:00
Nicolas
c6f1d80992 Update crawl.ts 2024-09-05 13:03:43 -03:00
Nicolas
eb03a81152 Update crawl-status.ts 2024-09-05 12:55:04 -03:00
Nicolas
34adf43200 Merge branch 'main' into feat/go-html-to-md-parser 2024-09-05 12:13:14 -03:00
Nicolas
82cb80c817 Update map.ts
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Details
Fly Deploy / Test Suite (push) Blocked by required conditions
Details
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Rust SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Deploy app (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Rust SDK (push) Blocked by required conditions
Details
2024-09-04 23:46:18 -03:00
Nicolas
a0113dac37 Update credit_billing.ts
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Details
Fly Deploy / Test Suite (push) Blocked by required conditions
Details
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Rust SDK Tests (push) Blocked by required conditions
Details
Fly Deploy / Deploy app (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Details
Fly Deploy / Build and publish Rust SDK (push) Blocked by required conditions
Details
2024-09-04 16:54:20 -03:00
Nicolas
b7b99b53da Merge branch 'main' into nsc/check-credits-optimization 2024-09-04 16:39:18 -03:00
rafaelsideguide
85b824e122 test: what about false false? 2024-09-04 16:35:32 -03:00
rafaelsideguide
78edf13ec6 test: usedbauth envs wth 2024-09-04 16:31:42 -03:00
rafaelsideguide
cb8571abad fix: enforced dotenv config 2024-09-04 15:57:57 -03:00
Nicolas
5ecb243693 Nick: 2024-09-04 15:19:45 -03:00
rafaelsideguide
3f462eabe9 fix(cicd): 2024-09-04 12:27:46 -03:00
rafaelsideguide
364ba9f901 fix(cicd): mendable->firecrawl and waitfor 2024-09-04 12:05:02 -03:00
rafaelsideguide
ad950a6c9d fixed controller res and tests 2024-09-04 11:29:32 -03:00
rafaelsideguide
74ac8915cd details 2024-09-04 09:41:38 -03:00
rafaelsideguide
d836ba6782 added log to check response on cicd 2024-09-04 09:35:56 -03:00
Nicolas
049a11187d Nick: 2024-09-03 21:09:32 -03:00
Nicolas
3072d4a333 Nick: fixed .sort coupons and sentry to withAuth 2024-09-03 21:02:41 -03:00
Nicolas
7561fd279f Nick: debug the billing email system for free credits 2024-09-03 17:08:12 -03:00
rafaelsideguide
c5e1d77a82 added invalid html tests 2024-09-03 15:21:45 -03:00
rafaelsideguide
d60fa6e084 fixed dockerfile and function name. it's working 2024-09-03 14:08:07 -03:00