1019 Commits

Author	SHA1	Message	Date
rafaelsideguide	5f60a55967	workflow and npm now running v1 tests	2024-08-22 15:28:49 -03:00
rafaelsideguide	30e809966f	Merge remote-tracking branch 'origin/v1/python-sdk' into v1-webscraper	2024-08-22 15:18:05 -03:00
rafaelsideguide	a37681bdff	fix: replace jest, removed map for v0	2024-08-22 15:16:46 -03:00
rafaelsideguide	7473b74021	fix: html and rawlhtmls for pdfs	2024-08-22 15:15:45 -03:00
Nicolas	d2521612b4	Update .gitignore	2024-08-22 14:15:19 -03:00
rafaelsideguide	b1d61d8557	Merge remote-tracking branch 'origin/v1-webscraper' into v1/python-sdk	2024-08-22 13:39:09 -03:00
rafaelsideguide	ab88a75c70	fixes sdks	2024-08-22 13:38:34 -03:00
rafaelsideguide	a4686e3c8c	fixing tests	2024-08-21 15:56:48 -03:00
rafaelsideguide	fe2e8c0b7a	includehtml fix	2024-08-21 15:54:00 -03:00
rafaelsideguide	af0e47a30e	Merge remote-tracking branch 'origin/v1/node-sdk' into v1/python-sdk	2024-08-21 12:09:53 -03:00
rafaelsideguide	e9d6ca197e	tests passing now	2024-08-20 20:00:41 -03:00
Gergő Móricz	5818236659	fix: remove rawHtml properly	2024-08-20 22:51:12 +02:00
rafaelsideguide	0f48823c9c	Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk	2024-08-20 17:24:59 -03:00
rafaelsideguide	9d64c8eedc	screenshot should not be undefined also	2024-08-20 17:24:45 -03:00
Gergő Móricz	1368f9a87f	fix: treat existing screenshot as a scraper success condition	2024-08-20 22:24:18 +02:00
rafaelsideguide	70d81ca69e	Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk	2024-08-20 17:08:07 -03:00
rafaelsideguide	537fc689b6	fixing request	2024-08-20 17:08:02 -03:00
Nicolas	0c48c8a436	Nick: billing for map	2024-08-20 16:43:46 -03:00
Nicolas	39388cdc35	Update crawl.ts	2024-08-20 14:41:43 -03:00
Nicolas	674adee144	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-20 14:41:05 -03:00
Nicolas	b36faeaf54	Nick:	2024-08-20 14:39:52 -03:00
Gergő Móricz	cf32893c2e	add strict enforcement + move crawlerOptions to top-level in /crawl	2024-08-20 19:31:26 +02:00
Gergő Móricz	70d50b3640	fix(queue-worker): move dotenv config up	2024-08-20 19:25:19 +02:00
Nicolas	c5ad4dedeb	Update crawl.ts	2024-08-20 14:19:20 -03:00
Nicolas	de0dc20a02	Update credit_billing.ts	2024-08-20 14:18:14 -03:00
Nicolas	e200ec9e12	Nick:	2024-08-20 12:24:14 -03:00
Nicolas	55dad82df1	Nick: fixed map search	2024-08-20 12:17:53 -03:00
Nicolas	27903247b6	Nick: map tests and fixes	2024-08-20 12:04:08 -03:00
rafaelsideguide	fa89d2e535	v1 support for crawl/monitor status	2024-08-20 10:37:24 -03:00
rafaelsideguide	7727302ef1	Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk	2024-08-20 09:22:55 -03:00
rafaelsideguide	e160d5529a	fixed test	2024-08-20 09:22:38 -03:00
Nicolas	3dc298be54	Nick: 2x rate limits for standard and growth for /scrape	2024-08-19 13:52:54 -03:00
rafaelsideguide	32aba44167	fixing merge issues	2024-08-19 13:37:20 -03:00
rafaelsideguide	72461ce9a6	Update index.test.ts	2024-08-19 13:29:52 -03:00
rafaelsideguide	fd7fdc1d52	added blocklist middleware	2024-08-19 13:28:54 -03:00
rafaelsideguide	e516e4993b	Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk	2024-08-19 11:24:57 -03:00
Nicolas	ff84f1fe5e	Update map.ts	2024-08-16 20:42:36 -04:00
Nicolas	4314313477	Update map.ts	2024-08-16 19:56:18 -04:00
Nicolas	af9a0a6f0b	Update map.ts	2024-08-16 19:56:03 -04:00
Nicolas	ba5279eafc	Nick: all tests passing	2024-08-16 19:55:44 -04:00
Nicolas	5205c5f005	Update map.ts	2024-08-16 19:37:00 -04:00
Nicolas	0c05d096a9	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-16 19:33:58 -04:00
Nicolas	ab48353226	Nick: /map almost good	2024-08-16 19:33:57 -04:00
Gergő Móricz	eb84673b06	feat: crawl status websocket WIP	2024-08-17 01:04:14 +02:00
Gergő Móricz	e2a6ef26d3	mount v1Router under v1 path	2024-08-16 23:48:50 +02:00
Gergő Móricz	4c1b74dab3	fix(map): remove robots.txt	2024-08-16 23:46:10 +02:00
Gergő Móricz	803577eeba	feat(crawl): webhook	2024-08-16 23:42:48 +02:00
rafaelsideguide	086ba6280b	fixed markdown format	2024-08-16 18:39:13 -03:00
Gergő Móricz	aabfaf0ac5	clean up crawl-status, fix db ddos	2024-08-16 23:29:39 +02:00
rafaelsideguide	e5b807ccc4	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-16 17:57:31 -03:00