Commit Graph

1277 Commits

Author SHA1 Message Date
Gergo Moricz
b5ec47fd96 fix(runWebScraper): don't fetch next job 2024-08-07 13:53:04 +02:00
Nicolas
44c9a22731
Merge pull request #508 from mendableai/mog/js-sdk-cjs
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
fix(js-sdk): build both CJS and EJS versions
2024-08-06 19:46:20 -04:00
Gergo Moricz
020a5efdb7 Revert "Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs""
This reverts commit 5da4472842.
2024-08-07 01:27:26 +02:00
Gergő Móricz
7380d7799f
Merge branch 'main' into mog/js-sdk-cjs 2024-08-07 01:12:36 +02:00
Gergo Moricz
5f7724205f fix(js-sdk): re-add types 2024-08-07 01:06:21 +02:00
Nicolas
f294d3922c Nick: revert 2024-08-06 18:44:45 -04:00
Nicolas
5da4472842 Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs"
This reverts commit bb90e03dea, reversing
changes made to 3321ca9398.
2024-08-06 18:41:06 -04:00
Nicolas
a67a5c04c9 Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs"
This reverts commit bb90e03dea, reversing
changes made to 3321ca9398.
2024-08-06 18:02:56 -04:00
Nicolas
bb90e03dea
Merge pull request #432 from mendableai/mog/js-sdk-cjs
fix(js-sdk): build both CommonJS and ESM versions
2024-08-06 17:38:57 -04:00
Nicolas
3321ca9398
Merge pull request #504 from mendableai/feat/fullpage-screenshot
[Feat] Added fullpagescreenshot capabilities
2024-08-06 13:52:29 -04:00
Gergo Moricz
b60ee30dba fix(single_url): accept 500 2024-08-06 18:00:56 +02:00
Gergo Moricz
06751a8e21 fix(crawl-status): missing partial data after cancel 2024-08-06 17:31:20 +02:00
Gergo Moricz
810b98ec38 fix(scrape): fix timeout error code 2024-08-06 17:30:01 +02:00
Gergo Moricz
3ae95a2740 fix(scrape): consider timeout property 2024-08-06 17:25:58 +02:00
Gergo Moricz
8566ece700 fix(scrape): pass extractorOptions 2024-08-06 17:15:19 +02:00
Gergo Moricz
8e0aa69603 fix(crawl-status): partial_data 2024-08-06 17:06:21 +02:00
Gergo Moricz
1ab119c874 fix(scrape): don't double-bill for scrape 2024-08-06 16:57:23 +02:00
Gergo Moricz
7c5cda7b45 fix(queue-worker): concurrency 2024-08-06 16:57:00 +02:00
Gergo Moricz
d7d63790e5 fix(crawl-status): isCancelled should be status failed 2024-08-06 16:35:55 +02:00
Gergo Moricz
03c84a9372 cleanup and fix cancelling 2024-08-06 16:26:46 +02:00
rafaelsideguide
4d24a99d50 fix params 2024-08-06 09:34:43 -03:00
Nicolas
e195ddbef4 Merge branch 'main' into nsc/hyper-v81 2024-08-05 20:47:39 -04:00
Nicolas
72f2c3616f
Merge pull request #503 from mendableai/bugfix/empty-excludes
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
[Bug] Fixed the empty excludes.filter is undefined bug
2024-08-05 20:42:21 -04:00
rafaelsideguide
3edc3a3d15 added fullpagescreenshot capabilities, wip on fire-engine side 2024-08-05 18:17:37 -03:00
rafaelsideguide
f32e8de156 fixes the empty excludes.filter undefined bug 2024-08-05 18:13:31 -03:00
KentHsu
b2e1b2ca68 chore: add go-sdk-tests job 2024-08-04 17:33:36 +08:00
KentHsu
1378ffc138 feat: add go-sdk 2024-08-04 17:33:33 +08:00
tak-s
af9bc5c8bb Suppressed repetitive logs 2024-08-04 15:09:36 +09:00
tak-s
db926a4146 set LOGGING_LEVEL to environment 2024-08-04 15:05:53 +09:00
Nicolas
789c6cf5d7
Merge pull request #494 from mendableai/nsc/website-param-fixes
Some checks failed
Fly Deploy / Pre-deploy checks (push) Has been cancelled
Fly Deploy / Test Suite (push) Has been cancelled
Fly Deploy / Python SDK Tests (push) Has been cancelled
Fly Deploy / JavaScript SDK Tests (push) Has been cancelled
Fly Deploy / Deploy app (push) Has been cancelled
Fly Deploy / Build and publish Python SDK (push) Has been cancelled
Fly Deploy / Build and publish JavaScript SDK (push) Has been cancelled
Custom engine params fix
2024-08-02 19:29:03 -04:00
Nicolas
1742e4ceae Nick: 2024-08-02 19:25:15 -04:00
Nicolas
311f812a96 Merge branch 'nsc/amazon' 2024-08-02 17:43:53 -04:00
Nicolas
39aecd974b Update redis-health.ts 2024-08-02 17:43:45 -04:00
Nicolas
ddc0dac4e4
Merge pull request #492 from mendableai/nsc/amazon
Fix AMZN | Removal of redis alerts
2024-08-02 14:52:09 -04:00
Nicolas
2e83a8a8b4 Delete check-redis.yml 2024-08-02 14:27:19 -04:00
Nicolas
b448e3c3ad Update website_params.ts 2024-08-02 14:26:35 -04:00
Nicolas
893113a39e
Merge pull request #491 from mendableai/bugfix/issue-477
[Bug] Nested sitemaps
2024-08-02 13:03:28 -04:00
rafaelsideguide
4051630632 Update sitemap.ts 2024-08-02 11:32:48 -03:00
rafaelsideguide
8568b61015 bugfix for sitemaps 2024-08-02 11:03:01 -03:00
Nicolas
af68b7a785
Merge pull request #475 from mendableai/bugfix/issue-466
Some checks failed
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Check Redis / clean-jobs (push) Has been cancelled
[Bug] pdfs and logging pdf events, also added trycatchs for docx
2024-08-01 22:05:26 -04:00
Nicolas
72eebb0aea
Merge pull request #485 from mendableai/bugfix/issue-435
Some checks failed
Fly Deploy / Pre-deploy checks (push) Has been cancelled
Fly Deploy / Test Suite (push) Has been cancelled
Fly Deploy / Python SDK Tests (push) Has been cancelled
Fly Deploy / JavaScript SDK Tests (push) Has been cancelled
Fly Deploy / Deploy app (push) Has been cancelled
Fly Deploy / Build and publish Python SDK (push) Has been cancelled
Fly Deploy / Build and publish JavaScript SDK (push) Has been cancelled
[Bug] Issue with crawl going beyond Limit
2024-07-31 18:10:29 -04:00
rafaelsideguide
f48ff36b32 added .inc files and forced lower case comparison 2024-07-31 09:28:43 -03:00
Nicolas
4ab7fb007a
Merge pull request #484 from mendableai/nsc/fix-scaling-fix-email
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
Fix parallelization issues with caching
2024-07-30 19:21:00 -04:00
Nicolas
ad6f6eff4b Update fireEngine.ts 2024-07-30 19:15:54 -04:00
Nicolas
f9827b2151 Update credit_billing.ts 2024-07-30 19:13:17 -04:00
Nicolas
6d99dedd3c Nick: fixed tests 2024-07-30 19:11:01 -04:00
Nicolas
a28ecc1f61 Nick: caching 2024-07-30 18:59:35 -04:00
Nicolas
86c61b5648
Merge pull request #483 from mendableai/feat/improv-self-host
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
[Feat] Better self hosting guide
2024-07-30 17:08:41 -04:00
rafaelsideguide
c7a38a4ae2 Update SELF_HOST.md 2024-07-30 18:07:36 -03:00
Nicolas
52198f2991 Nick: 2024-07-30 16:15:08 -04:00