387 Commits

Author	SHA1	Message	Date
Eric Ciarla	095951aa4d	Update test	2024-06-13 17:40:00 -04:00
Eric Ciarla	5e8aa92788	Update index.ts	2024-06-13 17:33:13 -04:00
Eric Ciarla	bf10e9d392	Update index.test.ts	2024-06-13 17:28:59 -04:00
Eric Ciarla	65d63bae45	Update index.ts	2024-06-13 17:17:44 -04:00
Eric Ciarla	32e814bedc	Update index.ts	2024-06-13 17:02:30 -04:00
Nicolas	182f8d4d6c	Update index.ts	2024-06-12 18:07:05 -07:00
Nicolas	11b6d5afa5	Update fly.toml	2024-06-12 18:00:22 -07:00
Nicolas	67dc46b454	Nick: clusters	2024-06-12 17:53:04 -07:00
rafaelsideguide	d20af257ba	Added jobId to webhook data	2024-06-12 15:38:41 -03:00
rafaelsideguide	01c9f071fa	fixed	2024-06-12 11:27:06 -03:00
rafaelsideguide	dc6acbf1f0	Merge remote-tracking branch 'origin/main' into feat/allowbackwardcrawling-option	2024-06-12 11:01:05 -03:00
Nicolas	f93231499f	Merge pull request #265 from mendableai/feat/issue-264 ... [Feat] Added route to clean completed jobs and a github action cron that triggers every 24h	2024-06-11 21:33:52 -07:00
Nicolas	45dee63943	Merge pull request #262 from mendableai/nsc/webhook-self-host-fix ... Only fetch webhook from db if self host webhook not set and using db auth	2024-06-11 15:46:57 -07:00
rafaelsideguide	157fbe4a1e	added bull auth key	2024-06-11 17:52:01 -03:00
rafaelsideguide	df3a678cf4	getting back the cancel test, this should work	2024-06-11 17:46:56 -03:00
rafaelsideguide	def2ba9987	added tests	2024-06-11 17:46:25 -03:00
Nicolas	1e3e06a1d5	Update replacePaths.test.ts	2024-06-11 13:02:39 -07:00
Nicolas	2239e03269	Update replacePaths.test.ts	2024-06-11 12:54:02 -07:00
Nicolas	520739c9f4	Nick: fixed bugs associated with absolute path replacements	2024-06-11 12:43:16 -07:00
Nicolas	b87725c683	Update openapi.json	2024-06-11 12:08:49 -07:00
rafaelsideguide	ee282c3d55	Added allowBackwardCrawling option	2024-06-11 15:24:39 -03:00
rafaelsideguide	a9f93c2f1e	Added route to clean completed jobs and a github action cron that triggers every 24h	2024-06-11 14:18:05 -03:00
Nicolas	da38dad9a7	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-06-10 18:26:31 -07:00
Nicolas	9390816c1b	Update openapi.json	2024-06-10 18:26:25 -07:00
Nicolas	f6b06ac27a	Nick: ignoreSitemap, better crawling algo	2024-06-10 18:12:41 -07:00
Nicolas	1bd0327e1a	Merge branch 'main' into nsc/pageoptions-crawler	2024-06-10 17:15:10 -07:00
Nicolas	99f2ffd6d5	Update webhook.ts	2024-06-10 17:03:10 -07:00
Nicolas	7ae9778642	Update single_url.ts	2024-06-10 16:57:31 -07:00
Nicolas	913c1dd568	Nick: fetch -> axios and fix timeouts	2024-06-10 16:49:03 -07:00
Nicolas	3091f0134c	Nick:	2024-06-10 16:27:10 -07:00
Nicolas	f24ca76618	Nick: removing rate limit emails for now	2024-06-07 10:39:11 -07:00
Nicolas	98d82c4cec	Update search.ts	2024-06-06 20:02:21 -07:00
Nicolas	5e80f8af87	Nick: llm extract 50	2024-06-06 18:35:44 -07:00
rafaelsideguide	f5318ea7d7	Update index.test.ts	2024-06-06 16:50:20 -03:00
rafaelsideguide	cd7f9abcec	Update index.test.ts	2024-06-06 16:44:46 -03:00
rafaelsideguide	7b9b668b95	Update index.test.ts	2024-06-06 16:36:51 -03:00
rafaelsideguide	82e0ed4cd3	Update index.test.ts	2024-06-06 16:33:27 -03:00
rafaelsideguide	dac7612be2	Merge branch 'main' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 16:07:25 -03:00
Nicolas	c2ad358390	Nick:	2024-06-06 12:05:20 -07:00
rafaelsideguide	79ec9f04dc	Merge branch 'main' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 15:58:14 -03:00
Nicolas	de06b13deb	Update rate-limiter.ts	2024-06-06 11:56:22 -07:00
Nicolas	27a8fd0c3c	Update rate-limiter.ts	2024-06-06 11:56:00 -07:00
Nicolas	1129d33321	Update rate-limiter.ts	2024-06-06 11:53:12 -07:00
rafaelsideguide	b234b4be5a	Merge branch 'main' into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 15:44:29 -03:00
rafaelsideguide	af0bfca847	Merge branch 'main' into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 15:36:28 -03:00
rafaelsideguide	8132f22c73	nice	2024-06-06 15:36:20 -03:00
Nicolas	f1b5ec8517	Nick: fixes	2024-06-06 11:23:10 -07:00
Nicolas	deae7dcd61	Update email_notification.ts	2024-06-06 10:41:54 -07:00
Nicolas	f725fa5a97	Update email_notification.ts	2024-06-06 10:41:23 -07:00
Nicolas	0310da6729	Update rate-limiter.ts	2024-06-06 09:31:44 -07:00