Gergő Móricz
29f0d9ec94
propagate priority to fire-engine
2024-08-15 19:04:46 +02:00
Gergő Móricz
b79d3d1754
fix
2024-08-15 19:02:05 +02:00
Gergő Móricz
57730f6a35
priority changes
2024-08-15 18:58:07 +02:00
Gergő Móricz
846610681b
fix: fix posthog, add dummy crawl DB items
2024-08-15 18:55:18 +02:00
Gergő Móricz
8a5cad72f6
fix(queue-worker): variable name collision
2024-08-14 22:02:05 +02:00
Gergő Móricz
b8ec40dd72
fix(crawl): submit sitemapped jobs in bulk
2024-08-14 20:34:19 +02:00
Gergő Móricz
2ca1017fc3
fix(crawl): make request 0 of crawl jobs higher priority
2024-08-14 19:34:18 +02:00
Gergő Móricz
f4466f6bb0
fix(test-suite): add artillery
2024-08-14 19:33:09 +02:00
Gergő Móricz
cfad067a63
fix(fly): change proxy limits
2024-08-14 18:52:40 +02:00
Gergő Móricz
a6c81f9d62
fix: return all data when calling webhook
2024-08-14 17:53:47 +02:00
Gergo Moricz
2e5e480cc2
fix(crawl): call webhooks
2024-08-13 22:10:17 +02:00
Gergo Moricz
a33596de3c
fix(log_job): add crawl_id
2024-08-13 22:03:46 +02:00
Gergo Moricz
9252940b52
fix(crawl-status): sort data
2024-08-13 21:55:13 +02:00
Gergo Moricz
8dbac0268c
feat: offload crawl results to the DB
2024-08-13 21:40:59 +02:00
Gergo Moricz
4bbc9db1df
fix: prioritize scrape jobs over crawl jobs
2024-08-13 21:31:34 +02:00
Gergo Moricz
5f2af37880
fix(scrape): remove scrape job from queue after the job is done
2024-08-13 21:26:41 +02:00
Gergo Moricz
2413e33359
fix(queue-worker): remove console.log
2024-08-13 21:07:36 +02:00
Gergo Moricz
d7549d4dc5
feat: remove webScraperQueue
2024-08-13 21:03:24 +02:00
Gergő Móricz
4a2c37dcf5
Merge branch 'main' into feat/queue-scrapes
2024-08-13 20:53:49 +02:00
Gergo Moricz
86e136beca
feat: crawl to scrape conversion
2024-08-13 20:51:43 +02:00
Nicolas
c82e9f3ebc
Merge pull request #536 from mendableai/fix/e2e-tests
...
Fixed e2e tests
2024-08-13 14:44:14 -04:00
rafaelsideguide
a4be95ac27
fixed tests
2024-08-13 13:42:26 -03:00
Eric Ciarla
dd387c53c3
Merge pull request #534 from KentHsu/fix/go-sdk-module-name
...
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
fix: go-sdk module name
2024-08-13 13:53:28 +02:00
KentHsu
fd060c7ef1
fix: go-sdk module name
2024-08-13 10:10:45 +08:00
Nicolas
c6bf78cfb4
Update fly-direct.yml
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
2024-08-12 21:14:00 -04:00
Nicolas
09ca165d2e
Merge pull request #531 from kevinswiber/fix/respect-docker-env-file-comments
...
Self-host fix: Moving comments of .env.example values from end-of-line to above-line.
2024-08-12 16:54:56 -04:00
Nicolas
d06f40810c
Merge pull request #515 from wahpiangle/main
...
Update redis urls in example .env
2024-08-12 16:51:59 -04:00
Nicolas
6810338271
Update search.ts
2024-08-12 16:51:43 -04:00
Thomas Kosmas
98be29c963
Update parameters for platform.openai.com
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
2024-08-12 22:49:28 +03:00
Kevin Swiber
33aa5cf0de
Moving comments of .env.example values from end-of-line to above-line. Self-host docs suggest using .env.example as a base. However, Docker doesn't respect end-of-line comments. It sets the comment as the actual value of the variable. This fix prevents that.
2024-08-12 12:24:46 -07:00
Nicolas
9d187bf3bb
Merge pull request #529 from mendableai/nsc/redlock-cache-auth
...
Redlock cache in auth
2024-08-12 15:14:07 -04:00
Nicolas
74a5125185
Nick: removed redlock
2024-08-12 15:07:30 -04:00
Nicolas
0bd1a820ee
Update auth.ts
2024-08-12 13:42:09 -04:00
Nicolas
25a899eae3
Nick: redlock cache in auth
2024-08-12 13:37:47 -04:00
Rafael Miller
36e4b2cf49
Update .env.example
2024-08-12 10:37:00 -03:00
Quan Ming
fe179d0cb1
Update redis troubleshooting in self host guide
2024-08-10 12:39:22 +08:00
Quan Ming
a96ad4b0e2
Update redis url to use comment
2024-08-10 12:33:26 +08:00
Nicolas
e28c415cf4
Nick:
2024-08-09 14:07:46 -04:00
Gergo Moricz
5a778f2c22
fix(js-sdk): add type metadata to exports
Fly Deploy / Pre-deploy checks (push) Has been cancelled
Fly Deploy / Test Suite (push) Has been cancelled
Fly Deploy / Python SDK Tests (push) Has been cancelled
Fly Deploy / JavaScript SDK Tests (push) Has been cancelled
Fly Deploy / Go SDK Tests (push) Has been cancelled
Fly Deploy / Deploy app (push) Has been cancelled
Fly Deploy / Build and publish Python SDK (push) Has been cancelled
Fly Deploy / Build and publish JavaScript SDK (push) Has been cancelled
2024-08-09 20:05:36 +02:00
Rafael Miller
6a78f6fe78
Merge pull request #497 from KentHsu/feat/add-go-sdk
...
[Feat] Add Go SDK implementation
2024-08-09 14:58:20 -03:00
rafaelsideguide
0591000b64
bugfix includes excludes
2024-08-09 14:30:41 -03:00
Kent (Chia-Hao), Hsu
1fda882983
Merge branch 'mendableai:main' into feat/add-go-sdk
2024-08-10 00:46:15 +08:00
Quan Ming
0221872a70
Update redis urls in example .env
2024-08-10 00:40:11 +08:00
rafaelsideguide
b802ea02a1
small improvements
...
- wait for getting results on crawl: sometimes crawl takes some a second to save the data on the db and this causes response.data to be empty
- added timeout value to test script
- increased http client timeout (llm extract was failing on e2e tests)
- fixed env path on test script
2024-08-09 11:13:14 -03:00
Nicolas
bf1d325aa9
Merge pull request #512 from mendableai/nsc/digikey-tls
...
Fly Deploy / Pre-deploy checks (push) Has been cancelled
Fly Deploy / Test Suite (push) Has been cancelled
Fly Deploy / Python SDK Tests (push) Has been cancelled
Fly Deploy / JavaScript SDK Tests (push) Has been cancelled
Fly Deploy / Deploy app (push) Has been cancelled
Fly Deploy / Build and publish Python SDK (push) Has been cancelled
Fly Deploy / Build and publish JavaScript SDK (push) Has been cancelled
Digikey support
2024-08-08 12:34:18 -04:00
Nicolas
f1f5605010
Update website_params.ts
2024-08-08 12:31:58 -04:00
Nicolas
b0abad07da
Merge pull request #496 from tak-s/improve-logging-level
...
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Improve logs
2024-08-07 22:01:12 -04:00
Gergo Moricz
920b7f2f44
fix(runWebScraper): don't filter empty docs
2024-08-07 21:00:22 +02:00
Gergo Moricz
55ec96c23f
fix(queue-worker): bad job lock extension time
2024-08-07 20:24:16 +02:00
Gergo Moricz
ab7a35c581
fix(queue-worker): log lock extensions
2024-08-07 19:49:48 +02:00
