Austin/firecrawl
1
0
Fork 0
mirror of https://github.com/mendableai/firecrawl.git synced 2024-11-16 03:32:22 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Commit Graph

  • a839bc49ad
         apps/playwright-service(deps): bump the prod-deps group dependabot[bot] 2024-10-07 06:23:50 +0000
  • 96bc53ca6d
         Update README.md Nicolas 2024-10-07 00:47:57 -0300
  • 4dd1df2a58
         Update README.md Nicolas 2024-10-07 00:46:48 -0300
  • 61dcbffe66
         Update README.md Nicolas 2024-10-07 00:24:52 -0300
  • bcecb090ac
         Update README.md Nicolas 2024-10-07 00:23:25 -0300
  • f3261098fd
         Update README.md Nicolas 2024-10-07 00:20:53 -0300
  • 072a949a88
         Update README.md Nicolas 2024-10-07 00:20:20 -0300
  • 800d0306af
         Update README.md Nicolas 2024-10-07 00:17:31 -0300
  • 7efe3486a6
         Update README.md Nicolas 2024-10-07 00:17:01 -0300
  • 5292939130
         Update README.md Nicolas 2024-10-07 00:16:46 -0300
  • 36d16a1f7e
         Update README.md Nicolas 2024-10-07 00:02:41 -0300
  • 1c02187054
         Update website_qa_with_gemini_caching.ipynb Stijn Smits 2024-10-06 13:25:23 +0200
  • 33cde0588d bunch of stuff Gergő Móricz 2024-10-05 00:25:18 +0200
  • c263ceb408 Merge branch 'main' into test/k8s-load-testing test/k8s-load-testing Nicolas 2024-10-04 17:56:15 -0300
  • e473a2359a Create README.md Nicolas 2024-10-04 16:53:40 -0300
  • 33555742a7 feat(scrapeURL): more intense tests Gergő Móricz 2024-10-04 21:47:16 +0200
  • 29ca8ce845 feat(scrapeURL/transformars): uploadScreenshot Gergő Móricz 2024-10-04 21:47:11 +0200
  • fc490b9319 feat(scrapeURL/engines): scrapingbee Gergő Móricz 2024-10-04 21:47:01 +0200
  • 236a4e768c feat(scrapeURL): add forceEngine internal option Gergő Móricz 2024-10-04 21:46:48 +0200
  • 8adf50b7a8 fix(scrapeURL/engines/fire-engine;playwright): fix screenshot Gergő Móricz 2024-10-04 21:46:35 +0200
  • 1f1afeaac4 Update system-monitor.ts Nicolas 2024-10-04 15:15:04 -0300
  • a40fb3b062 Update deploy-image.yml Nicolas 2024-10-03 19:17:52 -0300
  • d0bd450f86 Merge branch 'main' of https://github.com/mendableai/firecrawl Nicolas 2024-10-03 19:07:44 -0300
  • bfbd2c83f8 Update README.md Nicolas 2024-10-03 19:07:36 -0300
  • 4a21a55925
         Merge pull request #733 from mendableai/nsc/fixed-self-host-envs Nicolas 2024-10-03 19:03:51 -0300
  • d316d52c96 fixes docker-compose and 401 error nsc/fixed-self-host-envs rafaelsideguide 2024-10-03 19:02:32 -0300
  • dba96998e3 Update fetch.ts Nicolas 2024-10-03 18:56:51 -0300
  • 668ff3c71b Update fetch.ts Nicolas 2024-10-03 18:55:39 -0300
  • 25dd16bf2a Nick: removed 401 Nicolas 2024-10-03 18:52:17 -0300
  • 93657f6a44 Update queue-worker.ts Nicolas 2024-10-03 18:44:40 -0300
  • 75658c58a2 Merge branch 'main' of https://github.com/mendableai/firecrawl Nicolas 2024-10-03 18:41:48 -0300
  • 9a056919b9 Create deploy-image.yml Nicolas 2024-10-03 18:41:41 -0300
  • 28b64fc704 Change the gracefull shutdown signal Thomas Kosmas 2024-10-04 00:40:09 +0300
  • 50c59b6de9 Update docker-compose.yaml Nicolas 2024-10-03 18:18:02 -0300
  • 497ac3328b
         Merge pull request #732 from mendableai/fix/url-validation-params Nicolas 2024-10-03 17:43:37 -0300
  • cfd776a5de fix: now urls with params are passing validation fix/url-validation-params rafaelsideguide 2024-10-03 17:37:04 -0300
  • 99ca852e5d
         Merge pull request #731 from mendableai/nsc/crawl-fixes Nicolas 2024-10-03 17:37:03 -0300
  • 85e9f7b9b9
         Merge pull request #727 from mendableai/nsc/error-js-sdk-improv Nicolas 2024-10-03 17:36:31 -0300
  • 4f7608821f Update package.json nsc/error-js-sdk-improv Nicolas 2024-10-03 17:36:20 -0300
  • f743f2b922 Update index.ts Nicolas 2024-10-03 17:34:29 -0300
  • c6a29efbed Update crawl-status.ts nsc/crawl-fixes Nicolas 2024-10-03 17:33:38 -0300
  • ddd774ed68 Nick: Nicolas 2024-10-03 17:20:57 -0300
  • 82551bb6bc Update index.test.ts Nicolas 2024-10-03 17:13:30 -0300
  • c291745385 fe load tests rafaelsideguide 2024-10-03 17:06:35 -0300
  • 49bd95327e Update types.ts Nicolas 2024-10-03 17:00:33 -0300
  • 1a1ac9fd60 Nick: Nicolas 2024-10-03 16:37:58 -0300
  • a150aa820c Nick: shouldnt fallback on a 400 + error code should be correct on page status code Nicolas 2024-10-03 15:21:42 -0300
  • a66ef18635
         Merge pull request #728 from bytrangle/patch-1 Nicolas 2024-10-03 12:50:32 -0300
  • 961a87452e
         Remove wait_until_done from python-sdk example Trang Le 2024-10-03 16:24:58 +0700
  • 489a643391 Update index.ts Nicolas 2024-10-02 20:25:52 -0300
  • 0d144244bb test: load tests for k8s rafaelsideguide 2024-10-02 18:26:37 -0300
  • 26771e2e71 debug(zod): log unsupported protocol errors Gergő Móricz 2024-10-01 22:13:28 +0200
  • 0e6c6b75ed feat(scrapeURL): better warnings Gergő Móricz 2024-10-01 22:10:30 +0200
  • d1b838322d
         Merge pull request #721 from mendableai/feat/concurrency-limit Nicolas 2024-10-01 16:15:05 -0300
  • ac5e1fc194 Update sitemap.ts feat/concurrency-limit Nicolas 2024-10-01 16:14:43 -0300
  • c6717fecaa Nick: got rid of job interval sleep and math.min Nicolas 2024-10-01 16:11:12 -0300
  • 18f9cd09e1 Nick: fixed more stuff Nicolas 2024-10-01 16:04:39 -0300
  • fe721fffbe fix(crawl-redis): normalize URL before locking Gergő Móricz 2024-10-01 19:00:33 +0200
  • c0541cc990 Update queue-worker.ts Nicolas 2024-10-01 15:38:24 -0300
  • 37299fc035 Update types.ts Nicolas 2024-10-01 15:18:11 -0300
  • 8aa07afb6d Nick: fixes Nicolas 2024-10-01 15:15:49 -0300
  • 92dbd33e57 Update queue-worker.ts Nicolas 2024-10-01 14:53:26 -0300
  • 4d5477f357 Nick: resolved conflicts Nicolas 2024-10-01 14:39:57 -0300
  • 96245e387d Update crawl.ts Nicolas 2024-10-01 14:29:53 -0300
  • 258c67ce67 Revert "feat(queue-worker): always crawl links from content even if sitemapped" Nicolas 2024-10-01 14:20:23 -0300
  • 445fc432e9 Reapply "fix(v1/crawl): always use sitemap" Nicolas 2024-10-01 14:03:07 -0300
  • 339b19ce9d Revert "fix(v1/crawl): always use sitemap" Nicolas 2024-10-01 13:59:49 -0300
  • 5dc0fcf644 fix(v1/crawl): always use sitemap Gergő Móricz 2024-10-01 18:49:44 +0200
  • 3c045c43a4 feat(queue-worker): always crawl links from content even if sitemapped Gergő Móricz 2024-10-01 18:32:53 +0200
  • 1af26fe1b4 Nick: sitemap fix Nicolas 2024-10-01 12:38:48 -0300
  • 5a115fe489 feat(scrapeURL): LLM extract Gergő Móricz 2024-09-30 23:45:43 +0200
  • 3fb1de6e8e
         Merge c171baf963 into ff4b7a835b Nicolas 2024-09-30 17:18:48 -0300
  • ff4b7a835b
         Merge pull request #685 from devflowinc/main Nicolas 2024-09-30 17:18:30 -0300
  • 986262e1d4 Update search.ts Nicolas 2024-09-30 15:23:43 -0300
  • 0dd06d33ef fix(v0/search): pass job priority Gergő Móricz 2024-09-30 19:20:24 +0200
  • 20ffdbd15c hotfix Gergő Móricz 2024-09-30 19:17:52 +0200
  • a8df85fd9b fix(acuc): remove sentry capture Gergő Móricz 2024-09-30 19:10:24 +0200
  • 7253a50fb8 feat(scrapeURL): wip readme Gergő Móricz 2024-09-30 18:45:01 +0200
  • 0efb377788
         apps/api(deps): bump the prod-deps group across 1 directory with 36 updates dependabot[bot] 2024-09-30 07:01:51 +0000
  • 4b7c72aecc
         apps/test-suite(deps-dev): bump the dev-deps group dependabot[bot] 2024-09-30 06:41:18 +0000
  • 56e8322b56
         apps/test-suite(deps): bump the prod-deps group dependabot[bot] 2024-09-30 06:40:26 +0000
  • ae85b984ac
         apps/playwright-service(deps): bump the prod-deps group dependabot[bot] 2024-09-30 06:18:17 +0000
  • 15761774e7 (add note) Gergő Móricz 2024-09-30 00:38:28 +0200
  • d7755affc1 feat(scrapeURL): feature flag priorities, engine quality sorting, PDF and DOCX support Gergő Móricz 2024-09-30 00:31:33 +0200
  • 73ea367d40 feat(transfomers/coerce): warn when a format's response is missing Gergő Móricz 2024-09-29 14:52:28 +0200
  • 4204223cb3 chore: remove unused modules Gergő Móricz 2024-09-29 00:50:04 +0200
  • 775d9946de fix(scrapeURL): report unexpected error to sentry Gergő Móricz 2024-09-29 00:46:27 +0200
  • 1c5a29c81e fix(scrapeURL): add timeout to EngineResultsTracker Gergő Móricz 2024-09-29 00:42:19 +0200
  • cda065fd39 fix(scrapeURL): error swallow Gergő Móricz 2024-09-29 00:41:10 +0200
  • 260c538846 feat(scrapeURL): first batch Gergő Móricz 2024-09-29 00:27:42 +0200
  • 76c082a8dd feat: switch logger to Winston Gergő Móricz 2024-09-28 13:26:43 +0200
  • 7732c9d062 feat: use strictNullChecking Gergő Móricz 2024-09-28 13:15:30 +0200
  • 3621e191bd feat(concurrency-limit): set limit based on plan Gergő Móricz 2024-09-28 00:19:46 +0200
  • c6a83ab92c fix(api): entrypoint Gergő Móricz 2024-09-27 22:16:27 +0200
  • e44bdf7a54 bad dockerfile Gergő Móricz 2024-09-27 21:07:11 +0200
  • f0a1a2e45b fix: increase ulimit -n in docker Gergő Móricz 2024-09-27 20:44:52 +0200
  • d5e2a80e4a fix(crawl-status): keep 10 megabyte pages if they're the only thing in the output Gergő Móricz 2024-09-27 20:41:41 +0200
  • 975f0575b4 Nick: max retries with axios-retry Nicolas 2024-09-27 12:58:57 -0400
  • 92961cf74f Merge branch 'main' of https://github.com/mendableai/firecrawl Nicolas 2024-09-27 12:23:45 -0400
  • 1fdff87b3e Update single_url.ts Nicolas 2024-09-27 12:23:44 -0400