mirror of
https://github.com/intergalacticalvariable/reader.git
synced 2024-11-16 03:32:25 +08:00
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/
dockerllmproxyragscraperself-hostedwebscraperwebscrapingwebscraping-datawebsite-screenshotwebsite-screenshot-capturer
.github/workflows | ||
.vscode | ||
backend | ||
thinapps-shared@9f0fa1dd7f | ||
.gitignore | ||
.gitmodules | ||
LICENSE | ||
package.json | ||
README.md |
reader
Development Guide
Prerequisite
- Node v18 (The build fails for Node version >18)
- Yarn
- Firebase CLI (
npm install -g firebase-tools
)
Installation
Clone the scenex repo by running the command:
git clone git@github.com:jina-ai/reader.git
git submodule init
git submodule update
After a successful clone, install the packages for backend and the webapp.
For backend, go to the backend/functions
directory and install the npm dependencies.
cd backend/functions
npm install
For the frontend (webapp), go to the webapp
directory and install the yarn dependencies.
cd webapp
yarn
Configure
Establish localhost connection:
Once the packages are installed, go to the App.vue
file inside the webapp/src/
and uncomment the below code:
connectFunctionsEmulator(functions, 'localhost', 5001);
Run The Application Now
To run the backend server, inside the backend/functions
dir run the below command:
npm run serve
To run the frontend app, inside the webapp
dir run the below command:
yarn dev
Known Errors
- If you encounter 'npm ERR! /bin/sh: pkg-config: command not found' error in Mac, run the command
brew install pkg-config cairo libpng jpeg giflib pango librsvg
Best practices
Directory structure
There are three folders:
webapp
is the frontend project ofSceneX
, knowledge requirements:
- Vue 3
- Quasar
- ...
backend
contains source code of backend logic, knowledge requirements:
- Nodejs
- Firebase
- ...
scripts
folder includes custom scripts we might need during the development or for production, currently we have the following scripts:
translate
is responsible for translating and updating our i18n language files in frontend project.
Best practices of frontend
- Quasar docs is your
best friend
. Since the frontend project highly depends on frameworkQuasar
. It is recommended to use the predefined classes and components and avoid defining your custom classes - Double check of the UI output in
Dark mode
andLight mode
. Again, use predefined classes and props. - Plugins in boot folder: create corresponding file in
boot
folder and use them inquasar.config.js
:
module.exports = configure(function() {
return {
...
boot: [
'i18n',
'axios',
'firebase',
'addressbar-color',
'quasar-lang-pack'
],
...
}
})
Best practices of backend
- Remember to deploy your functions by running:
# deploy all functions
firebase deploy --only functions
# deploy a specific function
firebase deploy --only functions:{function name}