Nicolas
|
5551f704ba
|
Merge pull request #362 from snippet/self-host-docs-ts-playwright-service
(Docs) Self Host added new ts playwright service instructions
|
2024-07-04 17:48:41 -03:00 |
|
Nicolas
|
8f46b8218a
|
Merge pull request #361 from snippet/ts-playwright-service-docker
setting up docker to ts playwright service
|
2024-07-04 17:47:41 -03:00 |
|
Nicolas
|
32849b017f
|
Nick:
|
2024-07-03 20:18:11 -03:00 |
|
Nicolas
|
5ecd9cb6f5
|
Merge pull request #363 from mendableai/nsc/logging-scrapers
Logging for all scraper methods
|
2024-07-03 18:47:22 -03:00 |
|
Nicolas
|
066d92f643
|
Update single_url.ts
|
2024-07-03 18:38:17 -03:00 |
|
Nicolas
|
f5b2fbd7e8
|
Nick: revision
|
2024-07-03 18:06:53 -03:00 |
|
Nicolas
|
2d30cc6117
|
Nick: comments
|
2024-07-03 18:01:54 -03:00 |
|
Nicolas
|
90c54c32fd
|
Nick: refactor
|
2024-07-03 18:01:17 -03:00 |
|
Nicolas
|
90cf799a3c
|
Update single_url.ts
|
2024-07-03 17:56:21 -03:00 |
|
Nicolas
|
b36406e465
|
Nick: log scrpaers
|
2024-07-03 17:28:53 -03:00 |
|
Jeff Pereira
|
8d09c5f9b5
|
(Docs) Self Host added new ts playwright service instructions
|
2024-07-03 12:00:44 -07:00 |
|
Jeff Pereira
|
b4292c1ea3
|
setting up docker to ts playwright service
|
2024-07-03 11:55:39 -07:00 |
|
Nicolas
|
abb44bb112
|
Merge pull request #346 from mendableai/dependabot/pip/apps/playwright-service/prod-deps-8f04296377
apps/playwright-service(deps): bump the prod-deps group in /apps/playwright-service with 3 updates
|
2024-07-03 01:07:09 -03:00 |
|
Nicolas
|
f967daddcb
|
Merge pull request #325 from snippet/playwright-scraper-api
new playwright service
|
2024-07-03 01:04:52 -03:00 |
|
Eric Ciarla
|
2d0d5ac392
|
Update for llm-extraction-from-raw-html
|
2024-07-02 14:05:42 -04:00 |
|
rafaelsideguide
|
0175152577
|
Fixed PDF match custom scraping
Now it's working for both `https://getgc.ai/privacy` and `https://prairie.cards/products/wood-designs` usecases.
|
2024-07-02 11:25:17 -03:00 |
|
rafaelsideguide
|
96de948d6b
|
Update index.test.ts
|
2024-07-02 11:04:09 -03:00 |
|
rafaelsideguide
|
7b7154ba1e
|
bugfixed pageStatusCode
|
2024-07-02 10:51:35 -03:00 |
|
Rafael Miller
|
50eecf04a9
|
Update licence pyproject.toml
Closes #345
|
2024-07-02 10:01:49 -03:00 |
|
dependabot[bot]
|
60de6bb6e3
|
apps/playwright-service(deps): bump the prod-deps group
Bumps the prod-deps group in /apps/playwright-service with 3 updates: [hypercorn](https://github.com/pgjones/hypercorn), [fastapi](https://github.com/tiangolo/fastapi) and [playwright](https://github.com/Microsoft/playwright-python).
Updates `hypercorn` from 0.16.0 to 0.17.3
- [Changelog](https://github.com/pgjones/hypercorn/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pgjones/hypercorn/compare/0.16.0...0.17.3)
Updates `fastapi` from 0.110.0 to 0.111.0
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.110.0...0.111.0)
Updates `playwright` from 1.42.0 to 1.44.0
- [Release notes](https://github.com/Microsoft/playwright-python/releases)
- [Commits](https://github.com/Microsoft/playwright-python/compare/v1.42.0...v1.44.0)
---
updated-dependencies:
- dependency-name: hypercorn
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: prod-deps
- dependency-name: fastapi
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: prod-deps
- dependency-name: playwright
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: prod-deps
...
Signed-off-by: dependabot[bot] <support@github.com>
|
2024-07-02 12:47:09 +00:00 |
|
Rafael Miller
|
3d530b461b
|
Merge pull request #337 from Sanix-Darker/f/cleaner-docker-compose
[PROPOSAL] (docker-compose) regroup envs vars between services
|
2024-07-02 09:46:22 -03:00 |
|
Rafael Miller
|
46ddc813e0
|
Merge pull request #338 from Sanix-Darker/dependabot
[PROPOSAL] (deps): making sure all deps are always up to date
|
2024-07-02 09:46:08 -03:00 |
|
Rafael Miller
|
f0f449fe51
|
Merge pull request #336 from snippet/allow-external-content-links
[Proposal] new feature allowExternalContentLinks
|
2024-07-02 09:45:21 -03:00 |
|
rafaelsideguide
|
db4a743365
|
Added e2e test
|
2024-07-02 09:44:08 -03:00 |
|
Eric Ciarla
|
0821017f5b
|
Update README.md
|
2024-07-02 07:08:46 -04:00 |
|
Nicolas
|
42cd58a679
|
Merge pull request #332 from mendableai/feat/rawHtmlExtraction
Adds pageOptions.includeRawHtml and new extraction mode "llm-extraction-from-raw-html"
|
2024-07-01 18:23:26 -03:00 |
|
Nicolas
|
c4f423981f
|
Update pnpm-lock.yaml
|
2024-07-01 18:22:22 -03:00 |
|
rafaelsideguide
|
16aac7f8c5
|
Update single_url.ts
|
2024-07-01 18:21:15 -03:00 |
|
Nicolas
|
6d0c7a9ccd
|
Merge pull request #323 from mendableai/tests/crawl-limit-unit-tests
[Tests] Added crawl limit unit test
|
2024-07-01 17:56:04 -03:00 |
|
rafaelsideguide
|
4d6e25619b
|
minor spacing and comment stuff
|
2024-07-01 16:05:34 -03:00 |
|
Eric Ciarla
|
e1af815f8c
|
Update scrape.ts
|
2024-07-01 08:48:21 -04:00 |
|
sanix-darker
|
cf8208e3f0
|
feat(deps): making sure all deps are always up to date
|
2024-06-30 15:31:38 +02:00 |
|
sanix-darker
|
ff62b26001
|
feat: regroup what could be regroup in terms of environments variables between services (api and worker)
|
2024-06-30 15:22:21 +02:00 |
|
Eric Ciarla
|
7ae195bacc
|
Update index.test.ts
|
2024-06-29 10:13:12 -04:00 |
|
Eric Ciarla
|
837b446390
|
Update index.test.ts
|
2024-06-29 08:48:42 -04:00 |
|
Eric Ciarla
|
fe6e3aeadc
|
Update index.test.ts
|
2024-06-29 08:44:21 -04:00 |
|
Eric Ciarla
|
6c9f0dfc91
|
Add tests
|
2024-06-29 08:32:20 -04:00 |
|
Jeff Pereira
|
a5fb45988c
|
new feature allowExternalContentLinks
|
2024-06-28 17:23:40 -07:00 |
|
Eric Ciarla
|
87b54488d3
|
update to includeRawHtml
|
2024-06-28 17:07:47 -04:00 |
|
Eric Ciarla
|
70fcf2ce03
|
init
|
2024-06-28 16:39:09 -04:00 |
|
Nicolas
|
9bf74bc774
|
Update single_url.ts
|
2024-06-28 15:51:18 -03:00 |
|
Nicolas
|
7e17498bcf
|
Update single_url.ts
|
2024-06-28 15:45:16 -03:00 |
|
rafaelsideguide
|
7dffaaa3e2
|
Changed port and added "using with firecrawl" section on readme
|
2024-06-28 11:51:24 -03:00 |
|
Nicolas
|
dbfae2d9bf
|
Merge pull request #329 from george-zakharov/patch-1
Update CONTRIBUTING.md
|
2024-06-27 12:12:31 -03:00 |
|
George Zakharov
|
5a0ec070bf
|
Update CONTRIBUTING.md
Fix typo
|
2024-06-27 13:50:31 +04:00 |
|
Nicolas
|
017b0b2556
|
Merge pull request #328 from mendableai/nsc/includeOnlyTags
pageOptions.onlyIncludeTags param
|
2024-06-26 21:33:10 -03:00 |
|
Nicolas
|
9e7298945c
|
Update openapi.json
|
2024-06-26 21:25:38 -03:00 |
|
Nicolas
|
1ec0bf8adf
|
Update openapi.json
|
2024-06-26 21:22:46 -03:00 |
|
Nicolas
|
042f81ddf2
|
Update removeUnwantedElements.test.ts
|
2024-06-26 21:20:11 -03:00 |
|
Nicolas
|
388ce3cbce
|
Nick: small changes
|
2024-06-26 21:15:42 -03:00 |
|