462 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
rafaelsideguide	aa6df4305e	crawl load tests 6 and 7	2024-05-22 18:20:24 -03:00
rafaelsideguide	73f1d09d39	Update website_params.ts	2024-05-22 15:07:12 -03:00
rafaelsideguide	ff147f1f51	load testing for crawl	2024-05-22 14:26:29 -03:00
rafaelsideguide	04a0bef0fb	Merge branch 'main' into test/load-testing	2024-05-22 11:26:19 -03:00
rafaelsideguide	e4573c08ca	Update website_params.ts	2024-05-22 11:24:48 -03:00
rafaelsideguide	068a240ab4	load tests for scrape route	2024-05-22 09:30:32 -03:00
Nicolas	f915b08016	Merge pull request #174 from mendableai/nsc/fire-engine-beta ... beta: Fire-Engine fallback	2024-05-21 19:25:51 -07:00
Nicolas	cb2bd0e71f	Update index.test.ts	2024-05-21 19:03:32 -07:00
Nicolas	253abb849f	Update rate-limiter.ts	2024-05-21 18:53:58 -07:00
Nicolas	229b9908d2	Nick: only enable hyper dx in prod	2024-05-21 18:52:46 -07:00
Nicolas	a8ff295977	Update single_url.ts	2024-05-21 18:50:42 -07:00
Nicolas	a5e718b084	Nick: improvements	2024-05-21 18:34:23 -07:00
Nicolas	2e264a4c75	Update ci.yml	2024-05-21 13:24:09 -07:00
Nicolas	6285f12cd1	Merge pull request #167 from mendableai/nsc/hyper-dx-integration ... feat: HyperDX Integration	2024-05-21 13:19:38 -07:00
rafaelsideguide	75f4e34d8e	Merge branch 'main' into test/load-testing	2024-05-21 10:28:02 -03:00
rafaelsideguide	ec46065066	Update rate-limiter.ts	2024-05-21 10:07:27 -03:00
rafaelsideguide	6a3ac13fe1	Update load-test.yml	2024-05-21 10:06:02 -03:00
Nicolas	01783dc336	Update openapi.json	2024-05-20 17:10:55 -07:00
Nicolas	77a79b5a79	Nick: max num tokens for llm extract (for now) + slice the max	2024-05-20 17:07:38 -07:00
Nicolas	2644e1c029	Update .env.example	2024-05-20 13:36:51 -07:00
Nicolas	9e61d431f0	Nick: hyper dx integration init	2024-05-20 13:36:34 -07:00
Nicolas	d5d0d48848	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-05-20 10:06:52 -07:00
Nicolas	60002e79b8	Nick: python sdk bump	2024-05-20 10:06:48 -07:00
Nicolas	4bb536e5b3	Merge pull request #166 from mattjoyce/main ... Allow override of API URL	2024-05-20 09:44:47 -07:00
Matt Joyce	7e5ef4dec4	Allow override of API URL ... Allows python sdk to be used with local installs.	2024-05-20 18:46:32 +10:00
Nicolas	c74f757b53	Update rate-limiter.ts	2024-05-19 13:05:36 -07:00
Nicolas	842e197fd7	Merge pull request #151 from mendableai/feat/rate-limits ... [Feat] Added rate limits	2024-05-19 13:01:37 -07:00
Nicolas	98a39b39ab	Nick: increased rate limits	2024-05-19 12:59:29 -07:00
Nicolas	18fa15df25	Update index.test.ts	2024-05-19 12:50:06 -07:00
Nicolas	614c073af0	Nick: improvements	2024-05-19 12:45:46 -07:00
Nicolas	f473793ba3	Merge branch 'main' into feat/rate-limits	2024-05-19 12:23:34 -07:00
Nicolas	4efebf7a4b	Merge branch 'test/load-testing' of https://github.com/mendableai/firecrawl into test/load-testing	2024-05-19 12:22:51 -07:00
Nicolas	5792cd022c	Update fly.staging.toml	2024-05-19 12:22:49 -07:00
Nicolas	713f16fdc1	Update README.md	2024-05-19 00:41:12 -07:00
Nicolas	0dc108cd33	Update README.md	2024-05-18 11:32:13 -07:00
Nicolas	43d0309b09	Merge pull request #160 from elimisteve/patch-1 ... Update README.md: Typo fix	2024-05-18 11:24:23 -07:00
Steve Phillips	81563130e3	Update README.md: Typo fix ... Don't scrap, scrape!	2024-05-18 03:58:17 +02:00
Nicolas	fae8954eeb	Update SELF_HOST.md	2024-05-17 18:46:59 -07:00
rafaelsideguide	d667e1417b	added fly staging load test ... - being rate limited. Need to add the token to the rate-limit functions	2024-05-17 19:09:19 -03:00
Nicolas	7630565c26	Create fly.staging.toml	2024-05-17 14:33:59 -07:00
rafaelsideguide	7297b21dcd	Added load testing using artillery	2024-05-17 18:32:44 -03:00
rafaelsideguide	a480595aa7	Update index.test.ts	2024-05-17 15:41:27 -03:00
rafaelsideguide	54049be539	Added e2e tests	2024-05-17 15:37:47 -03:00
Nicolas	6feb21cc35	Update website_params.ts	2024-05-17 11:21:26 -07:00
Nicolas	2a1f2e396a	Merge pull request #29 from mendableai/detect-pdfs ... Fixes pdfs not found if .pdf is not present	2024-05-17 10:47:08 -07:00
Nicolas	5be208f595	Nick: fixed	2024-05-17 10:40:44 -07:00
Nicolas	eb88447e8b	Update index.test.ts	2024-05-17 10:00:05 -07:00
Nicolas	df6c3d1e7d	Merge branch 'main' into detect-pdfs	2024-05-17 09:55:51 -07:00
Nicolas	5c1e6d188c	Merge pull request #158 from mendableai/nsc/docx-support ... feat: Docx Support	2024-05-16 12:03:16 -07:00
Nicolas	9d635cb2a3	Nick: docx support	2024-05-16 11:48:02 -07:00