Reduce web example bloat

This commit is contained in:
Eric Ciarla 2024-06-22 08:40:26 -04:00
parent 8e39083d8c
commit 22541362d7
65 changed files with 1 additions and 14772 deletions

View File

@ -1,11 +0,0 @@
# Required environment variables
FIRECRAWL_API_KEY=
# Optional environment variables
# LangSmith tracing from the web worker.
# WARNING: FOR DEVELOPMENT ONLY. DO NOT DEPLOY A LIVE VERSION WITH THESE
# VARIABLES SET AS YOU WILL LEAK YOUR LANGCHAIN API KEY.
NEXT_PUBLIC_LANGCHAIN_TRACING_V2=
NEXT_PUBLIC_LANGCHAIN_API_KEY=
NEXT_PUBLIC_LANGCHAIN_PROJECT=

View File

@ -1,3 +0,0 @@
{
"extends": "next/core-web-vitals"
}

View File

@ -1,38 +0,0 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# next.js
/.next/
/out/
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# local env files
.env*.local
.env
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts
.yarn

View File

@ -1,21 +0,0 @@
MIT License
Copyright (c) 2023 Jacob Lee
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -1,7 +0,0 @@
Copyright <YEAR> <COPYRIGHT HOLDER>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@ -1,72 +0,0 @@
# Local Chat With Websites
Welcome to the Local Web Chatbot! This is a direct fork of [Jacob Lee' fully local PDF chatbot](https://github.com/jacoblee93/fully-local-pdf-chatbot) replacing the chat with PDF functionality with chat with website support powered by [Firecrawl](https://www.firecrawl.dev/). It is a simple chatbot that allows you to ask questions about a website by embedding it and running queries against the vector store using a local LLM and embeddings.
## 🦙 Ollama
You can run more powerful, general models outside the browser using [Ollama's desktop app](https://ollama.ai). Users will need to download and set up then run the following commands to allow the site access to a locally running Mistral instance:
### Mac/Linux
```bash
$ OLLAMA_ORIGINS=https://webml-demo.vercel.app OLLAMA_HOST=127.0.0.1:11435 ollama serve
```
Then, in another terminal window:
```bash
$ OLLAMA_HOST=127.0.0.1:11435 ollama pull mistral
```
### Windows
```cmd
$ set OLLAMA_ORIGINS=https://webml-demo.vercel.app
set OLLAMA_HOST=127.0.0.1:11435
ollama serve
```
Then, in another terminal window:
```cmd
$ set OLLAMA_HOST=127.0.0.1:11435
ollama pull mistral
```
## 🔥 Firecrawl
Additionally, you will need a Firecrawl API key for website embedding. Signing up for [Firecrawl](https://www.firecrawl.dev/) is easy and you get 500 credits free. Enter your API key into the box below the URL in the embedding form.
## ⚡ Stack
It uses the following:
- [Voy](https://github.com/tantaraio/voy) as the vector store, fully WASM in the browser.
- [Ollama](https://ollama.ai/).
- [LangChain.js](https://js.langchain.com) to call the models, perform retrieval, and generally orchestrate all the pieces.
- [Transformers.js](https://huggingface.co/docs/transformers.js/index) to run open source [Nomic](https://www.nomic.ai/) embeddings in the browser.
- For higher-quality embeddings, switch to `"nomic-ai/nomic-embed-text-v1"` in `app/worker.ts`.
- [Firecrawl](https://www.firecrawl.dev/) to scrape the webpages and deliver them in markdown format.
## 🔱 Forking
To run/deploy this yourself, simply fork this repo and install the required dependencies with `yarn`.
There are no required environment variables, but you can optionally set up [LangSmith tracing](https://smith.langchain.com/) while developing locally to help debug the prompts and the chain. Copy the `.env.example` file into a `.env.local` file:
```ini
# No environment variables required!
# LangSmith tracing from the web worker.
# WARNING: FOR DEVELOPMENT ONLY. DO NOT DEPLOY A LIVE VERSION WITH THESE
# VARIABLES SET AS YOU WILL LEAK YOUR LANGCHAIN API KEY.
NEXT_PUBLIC_LANGCHAIN_TRACING_V2="true"
NEXT_PUBLIC_LANGCHAIN_API_KEY=
NEXT_PUBLIC_LANGCHAIN_PROJECT=
```
Just make sure you don't set this in production, as your LangChain API key will be public on the frontend!
## 🙏 Thank you!
Huge thanks to Jacob Lee and the other contributors of the repo for making this happen! Be sure to give him a follow on Twitter [@Hacubu](https://x.com/hacubu)!

View File

@ -1,74 +0,0 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
body {
color: #f8f8f8;
background: #131318;
}
body input,
body textarea {
color: black;
}
a {
color: #5ba4f8;
}
a:hover {
border-bottom: 1px solid;
}
p {
margin: 8px 0;
}
code,
pre {
color: #ffa500;
}
pre {
background-color: black;
color: #39ff14;
}
li {
padding: 4px;
}
@layer base {
label {
@apply h-6 relative inline-block;
}
[type="checkbox"] {
@apply w-11 h-0 cursor-pointer inline-block;
@apply focus:outline-0 dark:focus:outline-0;
@apply border-0 dark:border-0;
@apply focus:ring-offset-transparent dark:focus:ring-offset-transparent;
@apply focus:ring-transparent dark:focus:ring-transparent;
@apply focus-within:ring-0 dark:focus-within:ring-0;
@apply focus:shadow-none dark:focus:shadow-none;
@apply after:absolute before:absolute;
@apply after:top-0 before:top-0;
@apply after:block before:inline-block;
@apply before:rounded-full after:rounded-full;
@apply after:content-[''] after:w-5 after:h-5 after:mt-0.5 after:ml-0.5;
@apply after:shadow-md after:duration-100;
@apply before:content-[''] before:w-10 before:h-full;
@apply before:shadow-[inset_0_0_#000];
@apply after:bg-white dark:after:bg-gray-50;
@apply before:bg-gray-300 dark:before:bg-gray-600;
@apply before:checked:bg-lime-500 dark:before:checked:bg-lime-500;
@apply checked:after:duration-300 checked:after:translate-x-4;
@apply disabled:after:bg-opacity-75 disabled:cursor-not-allowed;
@apply disabled:checked:before:bg-opacity-40;
}
}

View File

@ -1,49 +0,0 @@
import "./globals.css";
import { Public_Sans } from "next/font/google";
import { Navbar } from "@/components/Navbar";
const publicSans = Public_Sans({ subsets: ["latin"] });
export default function RootLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
<html lang="en">
<head>
<title>Fully In-Browser Chat Over Documents</title>
<link rel="shortcut icon" href="/images/favicon.ico" />
<meta
name="description"
content="Upload a PDF, then ask questions about it - without a single remote request!"
/>
<meta
property="og:title"
content="Fully In-Browser Chat Over Documents"
/>
<meta
property="og:description"
content="Upload a PDF, then ask questions about it - without a single remote request!"
/>
<meta property="og:image" content="/images/og-image.png" />
<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="630" />
<meta name="twitter:card" content="summary_large_image" />
<meta
name="twitter:title"
content="Fully In-Browser Chat Over Documents"
/>
<meta
name="twitter:description"
content="Upload a PDF, then ask questions about it - without a single remote request!"
/>
<meta name="twitter:image" content="/images/og-image.png" />
</head>
<body className={publicSans.className}>
<div className="flex flex-col p-4 md:p-12 h-[100vh]">{children}</div>
</body>
</html>
);
}

View File

@ -1,7 +0,0 @@
import { ChatWindow } from "@/components/ChatWindow";
export default function Home() {
return (
<ChatWindow placeholder="Try asking something about the document you just uploaded!"></ChatWindow>
);
}

View File

@ -1,232 +0,0 @@
import { ChatWindowMessage } from "@/schema/ChatWindowMessage";
import { Voy as VoyClient } from "voy-search";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createHistoryAwareRetriever } from "langchain/chains/history_aware_retriever";
import { FireCrawlLoader } from "@langchain/community/document_loaders/web/firecrawl";
import { HuggingFaceTransformersEmbeddings } from "@langchain/community/embeddings/hf_transformers";
import { VoyVectorStore } from "@langchain/community/vectorstores/voy";
import {
ChatPromptTemplate,
MessagesPlaceholder,
PromptTemplate,
} from "@langchain/core/prompts";
import { RunnableSequence, RunnablePick } from "@langchain/core/runnables";
import {
AIMessage,
type BaseMessage,
HumanMessage,
} from "@langchain/core/messages";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import type { BaseChatModel } from "@langchain/core/language_models/chat_models";
import type { LanguageModelLike } from "@langchain/core/language_models/base";
import { LangChainTracer } from "@langchain/core/tracers/tracer_langchain";
import { Client } from "langsmith";
import { ChatOllama } from "@langchain/community/chat_models/ollama";
const embeddings = new HuggingFaceTransformersEmbeddings({
modelName: "Xenova/all-MiniLM-L6-v2",
});
const voyClient = new VoyClient();
const vectorstore = new VoyVectorStore(voyClient, embeddings);
const OLLAMA_RESPONSE_SYSTEM_TEMPLATE = `You are an experienced researcher, expert at interpreting and answering questions based on provided sources. Using the provided context, answer the user's question to the best of your ability using the resources provided.
Generate a concise answer for a given question based solely on the provided search results. You must only use information from the provided search results. Use an unbiased and journalistic tone. Combine search results together into a coherent answer. Do not repeat text.
If there is nothing in the context relevant to the question at hand, just say "Hmm, I'm not sure." Don't try to make up an answer.
Anything between the following \`context\` html blocks is retrieved from a knowledge bank, not part of the conversation with the user.
<context>
{context}
<context/>
REMEMBER: If there is no relevant information within the context, just say "Hmm, I'm not sure." Don't try to make up an answer. Anything between the preceding 'context' html blocks is retrieved from a knowledge bank, not part of the conversation with the user.`;
const _formatChatHistoryAsMessages = async (
chatHistory: ChatWindowMessage[],
) => {
return chatHistory.map((chatMessage) => {
if (chatMessage.role === "human") {
return new HumanMessage(chatMessage.content);
} else {
return new AIMessage(chatMessage.content);
}
});
};
const embedWebsite = async (url: string, firecrawlApiKey: string) => {
const webLoader = new FireCrawlLoader({
url: url,
apiKey: firecrawlApiKey,
mode: "scrape",
});
const docs = await webLoader.load();
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 500,
chunkOverlap: 50,
});
const splitDocs = await splitter.splitDocuments(docs);
self.postMessage({
type: "log",
data: splitDocs,
});
await vectorstore.addDocuments(splitDocs);
};
const queryVectorStore = async (
messages: ChatWindowMessage[],
{
chatModel,
modelProvider,
devModeTracer,
}: {
chatModel: LanguageModelLike;
modelProvider: "ollama";
devModeTracer?: LangChainTracer;
},
) => {
const text = messages[messages.length - 1].content;
const chatHistory = await _formatChatHistoryAsMessages(messages.slice(0, -1));
const responseChainPrompt = ChatPromptTemplate.fromMessages<{
context: string;
chat_history: BaseMessage[];
question: string;
}>([
["system", OLLAMA_RESPONSE_SYSTEM_TEMPLATE],
new MessagesPlaceholder("chat_history"),
["user", `{input}`],
]);
const documentChain = await createStuffDocumentsChain({
llm: chatModel,
prompt: responseChainPrompt,
documentPrompt: PromptTemplate.fromTemplate(
`<doc>\n{page_content}\n</doc>`,
),
});
const historyAwarePrompt = ChatPromptTemplate.fromMessages([
new MessagesPlaceholder("chat_history"),
["user", "{input}"],
[
"user",
"Given the above conversation, generate a natural language search query to look up in order to get information relevant to the conversation. Do not respond with anything except the query.",
],
]);
const historyAwareRetrieverChain = await createHistoryAwareRetriever({
llm: chatModel,
retriever: vectorstore.asRetriever(),
rephrasePrompt: historyAwarePrompt,
});
const retrievalChain = await createRetrievalChain({
combineDocsChain: documentChain,
retriever: historyAwareRetrieverChain,
});
const fullChain = RunnableSequence.from([
retrievalChain,
new RunnablePick("answer"),
]);
const stream = await fullChain.stream(
{
input: text,
chat_history: chatHistory,
},
{
callbacks: devModeTracer !== undefined ? [devModeTracer] : [],
},
);
for await (const chunk of stream) {
if (chunk) {
self.postMessage({
type: "chunk",
data: chunk,
});
}
}
self.postMessage({
type: "complete",
data: "OK",
});
};
// Listen for messages from the main thread
self.addEventListener("message", async (event: { data: any }) => {
self.postMessage({
type: "log",
data: `Received data!`,
});
let devModeTracer;
if (
event.data.DEV_LANGCHAIN_TRACING !== undefined &&
typeof event.data.DEV_LANGCHAIN_TRACING === "object"
) {
devModeTracer = new LangChainTracer({
projectName: event.data.DEV_LANGCHAIN_TRACING.LANGCHAIN_PROJECT,
client: new Client({
apiKey: event.data.DEV_LANGCHAIN_TRACING.LANGCHAIN_API_KEY,
}),
});
}
if (event.data.url) {
try {
self.postMessage({
type: "log",
data: `Embedding website now: ${event.data.url} with Firecrawl API Key: ${event.data.firecrawlApiKey}`,
});
await embedWebsite(event.data.url, event.data.firecrawlApiKey);
self.postMessage({
type: "log",
data: `Embedded website: ${event.data.url} complete`,
});
} catch (e: any) {
self.postMessage({
type: "error",
error: e.message,
});
throw e;
}
} else {
const modelProvider = event.data.modelProvider;
const modelConfig = event.data.modelConfig;
let chatModel: BaseChatModel | LanguageModelLike;
chatModel = new ChatOllama(modelConfig);
try {
await queryVectorStore(event.data.messages, {
devModeTracer,
modelProvider,
chatModel,
});
} catch (e: any) {
self.postMessage({
type: "error",
error: `${e.message}. Make sure you are running Ollama.`,
});
throw e;
}
}
self.postMessage({
type: "complete",
data: "OK",
});
});

View File

@ -1,125 +0,0 @@
"use client";
import { toast } from 'react-toastify';
import 'react-toastify/dist/ReactToastify.css';
import { ChatWindowMessage } from '@/schema/ChatWindowMessage';
import { useState, type FormEvent } from "react";
import { Feedback } from 'langsmith';
export function ChatMessageBubble(props: {
message: ChatWindowMessage;
aiEmoji?: string;
onRemovePressed?: () => void;
}) {
const { role, content, runId } = props.message;
const colorClassName =
role === "human" ? "bg-sky-600" : "bg-slate-50 text-black";
const alignmentClassName =
role === "human" ? "ml-auto" : "mr-auto";
const prefix = role === "human" ? "🧑" : props.aiEmoji;
const [isLoading, setIsLoading] = useState(false);
const [feedback, setFeedback] = useState<Feedback | null>(null);
const [comment, setComment] = useState("");
const [showCommentForm, setShowCommentForm] = useState(false);
async function handleScoreButtonPress(e: React.MouseEvent<HTMLButtonElement, MouseEvent>, score: number) {
e.preventDefault();
setComment("");
await sendFeedback(score);
}
async function handleCommentSubmission(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const score = typeof feedback?.score === "number" ? feedback.score : 0;
await sendFeedback(score);
}
async function sendFeedback(score: number) {
if (isLoading) {
return;
}
setIsLoading(true);
const response = await fetch("api/feedback", {
method: feedback?.id ? "PUT" : "POST",
body: JSON.stringify({
id: feedback?.id,
run_id: runId,
score,
comment,
})
});
const json = await response.json();
if (json.error) {
toast(json.error, {
theme: "dark"
});
return;
} else if (feedback?.id && comment) {
toast("Response recorded! Go to https://smith.langchain.com and check it out in under your run's \"Feedback\" pane.", {
theme: "dark",
autoClose: 3000,
});
setComment("");
setShowCommentForm(false);
} else {
setShowCommentForm(true);
}
if (json.feedback) {
setFeedback(json.feedback);
}
setIsLoading(false);
}
return (
<div
className={`${alignmentClassName} ${colorClassName} rounded px-4 py-2 max-w-[80%] mb-8 flex flex-col`}
>
<div className="flex hover:group group">
<div className="mr-2">
{prefix}
</div>
<div className="whitespace-pre-wrap">
{/* TODO: Remove. Hacky fix, stop sequences don't seem to work with WebLLM yet. */}
{content.trim().split("\nInstruct:")[0].split("\nInstruction:")[0]}
</div>
<div className="cursor-pointer opacity-0 hover:opacity-100 relative left-2 bottom-1" onMouseUp={props?.onRemovePressed}>
</div>
</div>
<div className={`${!runId ? "hidden" : ""} ml-auto mt-2`}>
<button className={`p-2 border text-3xl rounded hover:bg-green-400 ${feedback && feedback.score === 1 ? "bg-green-400" : ""}`} onMouseUp={(e) => handleScoreButtonPress(e, 1)}>
👍
</button>
<button className={`p-2 border text-3xl rounded ml-4 hover:bg-red-400 ${feedback && feedback.score === 0 ? "bg-red-400" : ""}`} onMouseUp={(e) => handleScoreButtonPress(e, 0)}>
👎
</button>
</div>
<div className={`${(feedback && showCommentForm) ? "" : "hidden"} min-w-[480px]`}>
<form onSubmit={handleCommentSubmission} className="relative">
<input
className="mr-8 p-4 rounded w-full border mt-2"
value={comment}
placeholder={feedback?.score === 1 ? "Anything else you'd like to add about this response?" : "What would the correct or preferred response have been?"}
onChange={(e) => setComment(e.target.value)}
/>
<div role="status" className={`${isLoading ? "" : "hidden"} flex justify-center absolute top-[24px] right-[16px]`}>
<svg aria-hidden="true" className="w-6 h-6 text-slate-200 animate-spin dark:text-slate-200 fill-sky-800" viewBox="0 0 100 101" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M100 50.5908C100 78.2051 77.6142 100.591 50 100.591C22.3858 100.591 0 78.2051 0 50.5908C0 22.9766 22.3858 0.59082 50 0.59082C77.6142 0.59082 100 22.9766 100 50.5908ZM9.08144 50.5908C9.08144 73.1895 27.4013 91.5094 50 91.5094C72.5987 91.5094 90.9186 73.1895 90.9186 50.5908C90.9186 27.9921 72.5987 9.67226 50 9.67226C27.4013 9.67226 9.08144 27.9921 9.08144 50.5908Z" fill="currentColor"/>
<path d="M93.9676 39.0409C96.393 38.4038 97.8624 35.9116 97.0079 33.5539C95.2932 28.8227 92.871 24.3692 89.8167 20.348C85.8452 15.1192 80.8826 10.7238 75.2124 7.41289C69.5422 4.10194 63.2754 1.94025 56.7698 1.05124C51.7666 0.367541 46.6976 0.446843 41.7345 1.27873C39.2613 1.69328 37.813 4.19778 38.4501 6.62326C39.0873 9.04874 41.5694 10.4717 44.0505 10.1071C47.8511 9.54855 51.7191 9.52689 55.5402 10.0491C60.8642 10.7766 65.9928 12.5457 70.6331 15.2552C75.2735 17.9648 79.3347 21.5619 82.5849 25.841C84.9175 28.9121 86.7997 32.2913 88.1811 35.8758C89.083 38.2158 91.5421 39.6781 93.9676 39.0409Z" fill="currentFill"/>
</svg>
<span className="sr-only">Loading...</span>
</div>
</form>
</div>
</div>
);
}

View File

@ -1,422 +0,0 @@
"use client";
import { Id, ToastContainer, toast } from "react-toastify";
import "react-toastify/dist/ReactToastify.css";
import { useRef, useState, useEffect } from "react";
import type { FormEvent } from "react";
import { ChatMessageBubble } from "@/components/ChatMessageBubble";
import { ChatWindowMessage } from "@/schema/ChatWindowMessage";
export function ChatWindow(props: { placeholder?: string }) {
const { placeholder } = props;
const [messages, setMessages] = useState<ChatWindowMessage[]>([]);
const [input, setInput] = useState("");
const [isLoading, setIsLoading] = useState(true);
const [selectedURL, setSelectedURL] = useState<string | null>(null);
const [firecrawlApiKey, setFirecrawlApiKey] = useState("");
const [readyToChat, setReadyToChat] = useState(false);
const initProgressToastId = useRef<Id | null>(null);
const titleText = "Local Chat With Websites";
const emoji = "🔥";
const worker = useRef<Worker | null>(null);
async function queryStore(messages: ChatWindowMessage[]) {
if (!worker.current) {
throw new Error("Worker is not ready.");
}
return new ReadableStream({
start(controller) {
if (!worker.current) {
controller.close();
return;
}
const ollamaConfig = {
baseUrl: "http://localhost:11435",
temperature: 0.3,
model: "mistral",
};
const payload: Record<string, any> = {
messages,
modelProvider: "ollama",
modelConfig: ollamaConfig,
};
if (
process.env.NEXT_PUBLIC_LANGCHAIN_TRACING_V2 === "true" &&
process.env.NEXT_PUBLIC_LANGCHAIN_API_KEY !== undefined
) {
console.warn(
"[WARNING]: You have set your LangChain API key publicly. This should only be done in local devlopment - remember to remove it before deploying!",
);
payload.DEV_LANGCHAIN_TRACING = {
LANGCHAIN_TRACING_V2: "true",
LANGCHAIN_API_KEY: process.env.NEXT_PUBLIC_LANGCHAIN_API_KEY,
LANGCHAIN_PROJECT: process.env.NEXT_PUBLIC_LANGCHAIN_PROJECT,
};
}
worker.current?.postMessage(payload);
const onMessageReceived = async (e: any) => {
switch (e.data.type) {
case "log":
console.log(e.data);
break;
case "init_progress":
if (initProgressToastId.current === null) {
initProgressToastId.current = toast(
"Loading model weights... This may take a while",
{
progress: e.data.data.progress || 0.01,
theme: "dark",
},
);
} else {
if (e.data.data.progress === 1) {
await new Promise((resolve) => setTimeout(resolve, 2000));
}
toast.update(initProgressToastId.current, {
progress: e.data.data.progress || 0.01,
});
}
break;
case "chunk":
controller.enqueue(e.data.data);
break;
case "error":
worker.current?.removeEventListener("message", onMessageReceived);
console.log(e.data.error);
const error = new Error(e.data.error);
controller.error(error);
break;
case "complete":
worker.current?.removeEventListener("message", onMessageReceived);
controller.close();
break;
}
};
worker.current?.addEventListener("message", onMessageReceived);
},
});
}
async function sendMessage(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
if (isLoading || !input) {
return;
}
const initialInput = input;
const initialMessages = [...messages];
const newMessages = [
...initialMessages,
{ role: "human" as const, content: input },
];
setMessages(newMessages);
setIsLoading(true);
setInput("");
try {
const stream = await queryStore(newMessages);
const reader = stream.getReader();
let chunk = await reader.read();
const aiResponseMessage: ChatWindowMessage = {
content: "",
role: "ai" as const,
};
setMessages([...newMessages, aiResponseMessage]);
while (!chunk.done) {
aiResponseMessage.content = aiResponseMessage.content + chunk.value;
setMessages([...newMessages, aiResponseMessage]);
chunk = await reader.read();
}
setIsLoading(false);
} catch (e: any) {
setMessages(initialMessages);
setIsLoading(false);
setInput(initialInput);
toast(`There was an issue with querying your website: ${e.message}`, {
theme: "dark",
});
}
}
// We use the `useEffect` hook to set up the worker as soon as the `App` component is mounted.
useEffect(() => {
if (!worker.current) {
// Create the worker if it does not yet exist.
worker.current = new Worker(
new URL("../app/worker.ts", import.meta.url),
{
type: "module",
},
);
setIsLoading(false);
}
}, []);
async function embedWebsite(e: FormEvent<HTMLFormElement>) {
console.log(e);
console.log(selectedURL);
console.log(firecrawlApiKey);
e.preventDefault();
// const reader = new FileReader();
if (selectedURL === null) {
toast(`You must enter a URL to embed.`, {
theme: "dark",
});
return;
}
setIsLoading(true);
worker.current?.postMessage({
url: selectedURL,
firecrawlApiKey: firecrawlApiKey,
});
const onMessageReceived = (e: any) => {
switch (e.data.type) {
case "log":
console.log(e.data);
break;
case "error":
worker.current?.removeEventListener("message", onMessageReceived);
setIsLoading(false);
console.log(e.data.error);
toast(`There was an issue embedding your website: ${e.data.error}`, {
theme: "dark",
});
break;
case "complete":
worker.current?.removeEventListener("message", onMessageReceived);
setIsLoading(false);
setReadyToChat(true);
toast(
`Embedding successful! Now try asking a question about your website.`,
{
theme: "dark",
},
);
break;
}
};
worker.current?.addEventListener("message", onMessageReceived);
}
const chooseDataComponent = (
<>
<div className="p-4 md:p-8 rounded bg-[#25252d] w-full max-h-[85%] overflow-hidden flex flex-col">
<h1 className="text-3xl md:text-4xl mb-2 ml-auto mr-auto">
{emoji} Local Chat With Websites {emoji}
</h1>
<ul>
<li className="text-l">
🏡
<span className="ml-2">
Welcome to the Local Web Chatbot!
<br></br>
<br></br>
This is a direct fork of{" "}
<a href="https://github.com/jacoblee93/fully-local-pdf-chatbot">
Jacob Lee&apos;s fully local PDF chatbot
</a>{" "}
replacing the chat with PDF functionality with website support. It
is a simple chatbot that allows you to ask questions about a
website by embedding it and running queries against the vector
store using a local LLM and embeddings.
</span>
</li>
<li>
<span className="ml-2">
The default LLM is Mistral-7B run locally by Ollama. You&apos;ll
need to install{" "}
<a target="_blank" href="https://ollama.ai">
the Ollama desktop app
</a>{" "}
and run the following commands to give this site access to the
locally running model:
<br />
<pre className="inline-flex px-2 py-1 my-2 rounded">
$ OLLAMA_ORIGINS=https://webml-demo.vercel.app
OLLAMA_HOST=127.0.0.1:11435 ollama serve
</pre>
<br />
Then, in another window:
<br />
<pre className="inline-flex px-2 py-1 my-2 rounded">
$ OLLAMA_HOST=127.0.0.1:11435 ollama pull mistral
</pre>
<br />
Additionally, you will need a Firecrawl API key for website
embedding. Signing up at{" "}
<a target="_blank" href="https://firecrawl.dev">
firecrawl.dev
</a>{" "}
is easy and you get 500 credits free. Enter your API key into the
box below the URL in the embedding form.
</span>
</li>
<li className="text-l">
🐙
<span className="ml-2">
Both this template and Jacob Lee&apos;s template are open source -
you can see the source code and deploy your own version{" "}
<a
href="https://github.com/ericciarla/local-web-chatbot"
target="_blank"
>
from the GitHub repo
</a>
or Jacob&apos;s{" "}
<a href="https://github.com/jacoblee93/fully-local-pdf-chatbot">
original GitHub repo
</a>
!
</span>
</li>
<li className="text-l">
👇
<span className="ml-2">
Try embedding a website below, then asking questions! You can even
turn off your WiFi after the website is scraped.
</span>
</li>
</ul>
</div>
<form
onSubmit={embedWebsite}
className="mt-4 flex flex-col justify-between items-center w-full"
>
<input
id="url_input"
type="text"
placeholder="Enter a URL to scrape"
className="text-black mb-2 w-[300px] px-4 py-2 rounded-lg"
onChange={(e) => setSelectedURL(e.target.value)}
></input>
<input
id="api_key_input"
type="text"
placeholder="Enter your Firecrawl API Key"
className="text-black mb-2 w-[300px] px-4 py-2 rounded-lg"
onChange={(e) => setFirecrawlApiKey(e.target.value)}
></input>
<button
type="submit"
className="shrink-0 px-4 py-4 bg-sky-600 rounded w-42"
>
<div
role="status"
className={`${isLoading ? "" : "hidden"} flex justify-center`}
>
<svg
aria-hidden="true"
className="w-6 h-6 text-white animate-spin dark:text-white fill-sky-800"
viewBox="0 0 100 101"
fill="none"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="M100 50.5908C100 78.2051 77.6142 100.591 50 100.591C22.3858 100.591 0 78.2051 0 50.5908C0 22.9766 22.3858 0.59082 50 0.59082C77.6142 0.59082 100 22.9766 100 50.5908ZM9.08144 50.5908C9.08144 73.1895 27.4013 91.5094 50 91.5094C72.5987 91.5094 90.9186 73.1895 90.9186 50.5908C90.9186 27.9921 72.5987 9.67226 50 9.67226C27.4013 9.67226 9.08144 27.9921 9.08144 50.5908Z"
fill="currentColor"
/>
<path
d="M93.9676 39.0409C96.393 38.4038 97.8624 35.9116 97.0079 33.5539C95.2932 28.8227 92.871 24.3692 89.8167 20.348C85.8452 15.1192 80.8826 10.7238 75.2124 7.41289C69.5422 4.10194 63.2754 1.94025 56.7698 1.05124C51.7666 0.367541 46.6976 0.446843 41.7345 1.27873C39.2613 1.69328 37.813 4.19778 38.4501 6.62326C39.0873 9.04874 41.5694 10.4717 44.0505 10.1071C47.8511 9.54855 51.7191 9.52689 55.5402 10.0491C60.8642 10.7766 65.9928 12.5457 70.6331 15.2552C75.2735 17.9648 79.3347 21.5619 82.5849 25.841C84.9175 28.9121 86.7997 32.2913 88.1811 35.8758C89.083 38.2158 91.5421 39.6781 93.9676 39.0409Z"
fill="currentFill"
/>
</svg>
<span className="sr-only">Loading...</span>
</div>
<span className={isLoading ? "hidden" : ""}>Embed Website</span>
</button>
</form>
</>
);
const chatInterfaceComponent = (
<>
<div className="flex flex-col-reverse w-full mb-4 overflow-auto grow">
{messages.length > 0
? [...messages].reverse().map((m, i) => (
<ChatMessageBubble
key={i}
message={m}
aiEmoji={emoji}
onRemovePressed={() =>
setMessages((previousMessages) => {
const displayOrderedMessages = previousMessages.reverse();
return [
...displayOrderedMessages.slice(0, i),
...displayOrderedMessages.slice(i + 1),
].reverse();
})
}
></ChatMessageBubble>
))
: ""}
</div>
<form onSubmit={sendMessage} className="flex w-full flex-col">
<div className="flex w-full mt-4">
<input
className="grow mr-8 p-4 rounded"
value={input}
placeholder={placeholder ?? "What's it like to be a pirate?"}
onChange={(e) => setInput(e.target.value)}
/>
<button
type="submit"
className="shrink-0 px-8 py-4 bg-sky-600 rounded w-28"
>
<div
role="status"
className={`${isLoading ? "" : "hidden"} flex justify-center`}
>
<svg
aria-hidden="true"
className="w-6 h-6 text-white animate-spin dark:text-white fill-sky-800"
viewBox="0 0 100 101"
fill="none"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="M100 50.5908C100 78.2051 77.6142 100.591 50 100.591C22.3858 100.591 0 78.2051 0 50.5908C0 22.9766 22.3858 0.59082 50 0.59082C77.6142 0.59082 100 22.9766 100 50.5908ZM9.08144 50.5908C9.08144 73.1895 27.4013 91.5094 50 91.5094C72.5987 91.5094 90.9186 73.1895 90.9186 50.5908C90.9186 27.9921 72.5987 9.67226 50 9.67226C27.4013 9.67226 9.08144 27.9921 9.08144 50.5908Z"
fill="currentColor"
/>
<path
d="M93.9676 39.0409C96.393 38.4038 97.8624 35.9116 97.0079 33.5539C95.2932 28.8227 92.871 24.3692 89.8167 20.348C85.8452 15.1192 80.8826 10.7238 75.2124 7.41289C69.5422 4.10194 63.2754 1.94025 56.7698 1.05124C51.7666 0.367541 46.6976 0.446843 41.7345 1.27873C39.2613 1.69328 37.813 4.19778 38.4501 6.62326C39.0873 9.04874 41.5694 10.4717 44.0505 10.1071C47.8511 9.54855 51.7191 9.52689 55.5402 10.0491C60.8642 10.7766 65.9928 12.5457 70.6331 15.2552C75.2735 17.9648 79.3347 21.5619 82.5849 25.841C84.9175 28.9121 86.7997 32.2913 88.1811 35.8758C89.083 38.2158 91.5421 39.6781 93.9676 39.0409Z"
fill="currentFill"
/>
</svg>
<span className="sr-only">Loading...</span>
</div>
<span className={isLoading ? "hidden" : ""}>Send</span>
</button>
</div>
</form>
</>
);
return (
<div
className={`flex flex-col items-center p-4 md:p-8 rounded grow overflow-hidden ${
readyToChat ? "border" : ""
}`}
>
<h2 className={`${readyToChat ? "" : "hidden"} text-2xl`}>
{emoji} {titleText}
</h2>
{readyToChat ? chatInterfaceComponent : chooseDataComponent}
<ToastContainer />
</div>
);
}

View File

@ -1,16 +0,0 @@
"use client";
import { usePathname } from 'next/navigation';
export function Navbar() {
const pathname = usePathname();
return (
<nav className="mb-4">
<a className={`mr-4 ${pathname === "/" ? "text-white border-b" : ""}`} href="/">🏴 Chat</a>
<a className={`mr-4 ${pathname === "/structured_output" ? "text-white border-b" : ""}`} href="/structured_output">🧱 Structured Output</a>
<a className={`mr-4 ${pathname === "/agents" ? "text-white border-b" : ""}`} href="/agents">🦜 Agents</a>
<a className={`mr-4 ${pathname === "/retrieval" ? "text-white border-b" : ""}`} href="/retrieval">🐶 Retrieval</a>
<a className={`mr-4 ${pathname === "/retrieval_agents" ? "text-white border-b" : ""}`} href="/retrieval_agents">🤖 Retrieval Agents</a>
</nav>
);
}

View File

@ -1,39 +0,0 @@
/** @type {import('next').NextConfig} */
const nextConfig = {
// (Optional) Export as a static site
// See https://nextjs.org/docs/pages/building-your-application/deploying/static-exports#configuration
output: 'export', // Feel free to modify/remove this option
// Override the default webpack configuration
webpack: (config, { isServer }) => {
// See https://webpack.js.org/configuration/resolve/#resolvealias
config.resolve.alias = {
...config.resolve.alias,
"sharp$": false,
"onnxruntime-node$": false,
}
config.experiments = {
...config.experiments,
topLevelAwait: true,
asyncWebAssembly: true,
};
config.module.rules.push({
test: /\.md$/i,
use: "raw-loader",
});
// Fixes npm packages that depend on `fs` module
if (!isServer) {
config.resolve.fallback = {
...config.resolve.fallback, // if you miss it, all the other options in fallback, specified
// by next.js will be dropped. Doesn't make much sense, but how it is
fs: false, // the solution
"node:fs/promises": false,
module: false,
perf_hooks: false,
};
}
return config;
},
}
module.exports = nextConfig

View File

@ -1,47 +0,0 @@
{
"name": "local-website-chatbot",
"version": "0.0.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"format": "prettier --write \"app\""
},
"engines": {
"node": ">=18"
},
"dependencies": {
"@langchain/community": "^0.2.9",
"@langchain/weaviate": "^0.0.4",
"@mendable/firecrawl-js": "^0.0.26",
"@mlc-ai/web-llm": "^0.2.42",
"@types/node": "20.4.5",
"@types/react": "18.2.17",
"@types/react-dom": "18.2.7",
"@xenova/transformers": "^2.16.0",
"autoprefixer": "10.4.14",
"encoding": "^0.1.13",
"eslint": "8.46.0",
"eslint-config-next": "13.4.12",
"jest": "^29.7.0",
"langchain": "^0.2.5",
"next": "13.4.12",
"pdf-parse": "^1.1.1",
"postcss": "8.4.27",
"react": "18.2.0",
"react-dom": "18.2.0",
"react-toastify": "^10.0.5",
"tailwindcss": "3.3.3",
"ts-node": "^10.9.2",
"typescript": "^5.4.5",
"voy-search": "^0.6.3"
},
"devDependencies": {
"prettier": "3.0.0"
},
"resolutions": {
"@langchain/core": "0.2.6"
}
}

View File

@ -1,6 +0,0 @@
module.exports = {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 308 KiB

View File

@ -1,6 +0,0 @@
export type ChatWindowMessage = {
content: string;
role: "human" | "ai";
runId?: string;
traceUrl?: string;
}

View File

@ -1,18 +0,0 @@
/** @type {import('tailwindcss').Config} */
module.exports = {
content: [
'./pages/**/*.{js,ts,jsx,tsx,mdx}',
'./components/**/*.{js,ts,jsx,tsx,mdx}',
'./app/**/*.{js,ts,jsx,tsx,mdx}',
],
theme: {
extend: {
backgroundImage: {
'gradient-radial': 'radial-gradient(var(--tw-gradient-stops))',
'gradient-conic':
'conic-gradient(from 180deg at 50% 50%, var(--tw-gradient-stops))',
},
},
},
plugins: [],
}

View File

@ -1,28 +0,0 @@
{
"compilerOptions": {
"target": "es5",
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"incremental": true,
"plugins": [
{
"name": "next"
}
],
"paths": {
"@/*": ["./*"]
}
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
}

File diff suppressed because it is too large Load Diff

View File

@ -1,3 +0,0 @@
{
"extends": "next/core-web-vitals"
}

View File

@ -1,38 +0,0 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
.yarn/install-state.gz
# testing
/coverage
# next.js
/.next/
/out/
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# local env files
.env*.local
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts
.env
node_modules

View File

@ -1,5 +0,0 @@
# Roast My Website 🔥
Welcome to Roast My Website, the ultimate tool for putting your website through the wringer! This repository harnesses the power of Firecrawl to scrape and capture screenshots of websites, and then unleashes the latest LLM vision models to mercilessly roast them.
Check it out at roastmywebsite.ai 😈

View File

@ -1,17 +0,0 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "default",
"rsc": true,
"tsx": true,
"tailwind": {
"config": "tailwind.config.ts",
"css": "src/app/globals.css",
"baseColor": "zinc",
"cssVariables": false,
"prefix": ""
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils"
}
}

View File

@ -1,11 +0,0 @@
/** @type {import('next').NextConfig} */
const nextConfig = {
env: {
G1: process.env.G1,
G2: process.env.G2,
G3: process.env.G3,
G4: process.env.G4,
},
};
export default nextConfig;

File diff suppressed because it is too large Load Diff

View File

@ -1,53 +0,0 @@
{
"name": "roastmywebsite",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@dqbd/tiktoken": "^1.0.15",
"@headlessui/react": "^2.0.4",
"@headlessui/tailwindcss": "^0.2.0",
"@mendable/firecrawl-js": "^0.0.21",
"@radix-ui/react-dialog": "^1.0.5",
"@radix-ui/react-dropdown-menu": "^2.0.6",
"@radix-ui/react-select": "^2.0.0",
"@radix-ui/react-slot": "^1.0.2",
"@radix-ui/react-switch": "^1.0.3",
"@remixicon/react": "^4.2.0",
"@tremor/react": "^3.17.2",
"@vercel/analytics": "^1.3.1",
"axios": "^1.7.2",
"class-variance-authority": "^0.7.0",
"clsx": "^2.1.1",
"cubic-spline": "^3.0.3",
"html2canvas": "^1.4.1",
"image-size": "^1.1.1",
"lucide": "^0.379.0",
"lucide-react": "^0.379.0",
"next": "14.2.3",
"next-themes": "^0.3.0",
"openai": "^4.47.3",
"react": "^18",
"react-dom": "^18",
"sonner": "^1.4.41",
"tailwind-merge": "^2.3.0",
"tailwindcss-animate": "^1.0.7",
"tiktoken": "^1.0.15"
},
"devDependencies": {
"@tailwindcss/forms": "^0.5.7",
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"eslint": "^8",
"eslint-config-next": "14.2.3",
"postcss": "^8",
"tailwindcss": "^3.4.3",
"typescript": "^5"
}
}

View File

@ -1,8 +0,0 @@
/** @type {import('postcss-load-config').Config} */
const config = {
plugins: {
tailwindcss: {},
},
};
export default config;

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 444 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 492 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 997 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

View File

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 394 80"><path fill="#000" d="M262 0h68.5v12.7h-27.2v66.6h-13.6V12.7H262V0ZM149 0v12.7H94v20.4h44.3v12.6H94v21h55v12.6H80.5V0h68.7zm34.3 0h-17.8l63.8 79.4h17.9l-32-39.7 32-39.6h-17.9l-23 28.6-23-28.6zm18.3 56.7-9-11-27.1 33.7h17.8l18.3-22.7z"/><path fill="#000" d="M81 79.3 17 0H0v79.3h13.6V17l50.2 62.3H81Zm252.6-.4c-1 0-1.8-.4-2.5-1s-1.1-1.6-1.1-2.6.3-1.8 1-2.5 1.6-1 2.6-1 1.8.3 2.5 1a3.4 3.4 0 0 1 .6 4.3 3.7 3.7 0 0 1-3 1.8zm23.2-33.5h6v23.3c0 2.1-.4 4-1.3 5.5a9.1 9.1 0 0 1-3.8 3.5c-1.6.8-3.5 1.3-5.7 1.3-2 0-3.7-.4-5.3-1s-2.8-1.8-3.7-3.2c-.9-1.3-1.4-3-1.4-5h6c.1.8.3 1.6.7 2.2s1 1.2 1.6 1.5c.7.4 1.5.5 2.4.5 1 0 1.8-.2 2.4-.6a4 4 0 0 0 1.6-1.8c.3-.8.5-1.8.5-3V45.5zm30.9 9.1a4.4 4.4 0 0 0-2-3.3 7.5 7.5 0 0 0-4.3-1.1c-1.3 0-2.4.2-3.3.5-.9.4-1.6 1-2 1.6a3.5 3.5 0 0 0-.3 4c.3.5.7.9 1.3 1.2l1.8 1 2 .5 3.2.8c1.3.3 2.5.7 3.7 1.2a13 13 0 0 1 3.2 1.8 8.1 8.1 0 0 1 3 6.5c0 2-.5 3.7-1.5 5.1a10 10 0 0 1-4.4 3.5c-1.8.8-4.1 1.2-6.8 1.2-2.6 0-4.9-.4-6.8-1.2-2-.8-3.4-2-4.5-3.5a10 10 0 0 1-1.7-5.6h6a5 5 0 0 0 3.5 4.6c1 .4 2.2.6 3.4.6 1.3 0 2.5-.2 3.5-.6 1-.4 1.8-1 2.4-1.7a4 4 0 0 0 .8-2.4c0-.9-.2-1.6-.7-2.2a11 11 0 0 0-2.1-1.4l-3.2-1-3.8-1c-2.8-.7-5-1.7-6.6-3.2a7.2 7.2 0 0 1-2.4-5.7 8 8 0 0 1 1.7-5 10 10 0 0 1 4.3-3.5c2-.8 4-1.2 6.4-1.2 2.3 0 4.4.4 6.2 1.2 1.8.8 3.2 2 4.3 3.4 1 1.4 1.5 3 1.5 5h-5.8z"/></svg>

Before

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 262 KiB

View File

@ -1 +0,0 @@
{"name":"","short_name":"","icons":[{"src":"/android-chrome-192x192.png","sizes":"192x192","type":"image/png"},{"src":"/android-chrome-512x512.png","sizes":"512x512","type":"image/png"}],"theme_color":"#ffffff","background_color":"#ffffff","display":"standalone"}

View File

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 283 64"><path fill="black" d="M141 16c-11 0-19 7-19 18s9 18 20 18c7 0 13-3 16-7l-7-5c-2 3-6 4-9 4-5 0-9-3-10-7h28v-3c0-11-8-18-19-18zm-9 15c1-4 4-7 9-7s8 3 9 7h-18zm117-15c-11 0-19 7-19 18s9 18 20 18c6 0 12-3 16-7l-8-5c-2 3-5 4-8 4-5 0-9-3-11-7h28l1-3c0-11-8-18-19-18zm-10 15c2-4 5-7 10-7s8 3 9 7h-19zm-39 3c0 6 4 10 10 10 4 0 7-2 9-5l8 5c-3 5-9 8-17 8-11 0-19-7-19-18s8-18 19-18c8 0 14 3 17 8l-8 5c-2-3-5-5-9-5-6 0-10 4-10 10zm83-29v46h-9V5h9zM37 0l37 64H0L37 0zm92 5-27 48L74 5h10l18 30 17-30h10zm59 12v10l-3-1c-6 0-10 4-10 10v15h-9V17h9v9c0-5 6-9 13-9z"/></svg>

Before

Width:  |  Height:  |  Size: 629 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

View File

@ -1,10 +0,0 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
.fill-tremor-content-emphasis {
fill: rgb(113 113 122) !important;
}

View File

@ -1,5 +0,0 @@
export async function useGithubStars() {
const res = await fetch("https://api.github.com/repos/mendableai/firecrawl");
const data = await res.json();
return data.stargazers_count;
}

View File

@ -1,68 +0,0 @@
import type { Metadata } from "next";
import { Gloria_Hallelujah } from "next/font/google";
import "./globals.css";
import { Toaster } from "sonner";
import { Analytics } from "@vercel/analytics/react";
import { useEffect, useState } from "react";
import Head from "next/head";
const inter = Gloria_Hallelujah({ weight: "400", subsets: ["latin"] });
// const inter = Inter({ subsets: ["latin"] });
const meta = {
title: "Roast My Website",
description:
"Welcome to Roast My Website, the ultimate tool for putting your website through the wringer! This repository harnesses the power of Firecrawl to scrape and capture screenshots of websites, and then unleashes the latest LLM vision models to mercilessly roast them. 😈",
cardImage: "/og.png",
robots: "follow, index",
favicon: "/favicon.ico",
url: "https://www.roastmywebsite.ai/",
};
export async function generateMetadata(): Promise<Metadata> {
return {
title: meta.title,
description: meta.description,
referrer: "origin-when-cross-origin",
keywords: ["Roast My Website", "Roast", "Website", "GitHub", "Firecrawl"],
authors: [
{ name: "Roast My Website", url: "https://www.roastmywebsite.ai/" },
],
creator: "Roast My Website",
publisher: "Roast My Website",
robots: meta.robots,
icons: { icon: meta.favicon },
metadataBase: new URL(meta.url),
openGraph: {
url: meta.url,
title: meta.title,
description: meta.description,
images: [meta.cardImage],
type: "website",
siteName: meta.title,
},
twitter: {
card: "summary_large_image",
site: "@Vercel",
creator: "@Vercel",
title: meta.title,
description: meta.description,
images: [meta.cardImage],
},
};
}
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
<html lang="en">
<body className={inter.className}>{children}</body>
<Analytics />
<Toaster />
</html>
);
}

View File

@ -1,16 +0,0 @@
// pages/index.tsx
import MainComponent from "@/components/main";
import { useGithubStars } from "./hooks/useGithubStars";
import GithubButton from "@/components/github-button";
export default async function Home() {
const githubStars = await useGithubStars();
return (
<div className="relative">
<div className="hidden md:flex z-10 absolute top-4 right-4 p-4">
<GithubButton githubStars={githubStars} />
</div>
<MainComponent />
</div>
);
}

View File

@ -1,20 +0,0 @@
"use client";
import { Github } from "lucide-react";
import { Button } from "./ui/button";
export default function GithubButton({ githubStars }: { githubStars: number }) {
return (
<Button
onClick={() => {
window.open("https://github.com/mendableai/firecrawl", "_blank");
}}
variant="outline"
size="icon"
className="px-3 w-22 gap-2"
>
<Github className="h-4 w-4" />{" "}
{/* {githubStars ? `Star us on GitHub` : "GitHub"} */}
See code on Github
</Button>
);
}

View File

@ -1,113 +0,0 @@
"use client";
import { useEffect, useRef, useState } from "react";
import { Theme, allThemes } from "@/lib/theme";
import html2canvas from "html2canvas";
import { Button } from "@/components/ui/button";
import { Input } from "./ui/input";
import { Github } from "lucide-react";
export default function MainComponent() {
const [roastUrl, setRoastUrl] = useState("");
const [loading, setLoading] = useState(false);
const [roastData, setRoastData] = useState("");
const [spiceLevel, setSpiceLevel] = useState(2);
return (
<div
className="h-screen"
style={{
background: `linear-gradient(to bottom right, rgba(255, 255, 255, 0.75) 58%, #fff, red )`,
}}
>
<main className="relative flex h-[95vh] flex-col items-center justify-center bg-transparent bg-opacity-80">
<div className="w-3/4 flex flex-col items-center gap-4">
<h1 className="font-inter-tight text-4xl lg:text-5xl xl:text-6xl font-bold bg-clip-text text-transparent bg-gradient-to-r from-zinc-500 via-zinc-900 to-zinc-900 pb-4 text-center">
<em className="relative px-1 italic animate-text transition-all text-transparent bg-clip-text bg-gradient-to-tr from-red-600 to-red-400 inline-flex justify-center items-center text-6xl lg:text-7xl xl:text-8xl">
Roast
</em>
<br />{" "}
<span className="text-4xl lg:text-5xl xl:text-6xl">My Website</span>
</h1>
<div className="flex flex-col sm:flex-row items-center gap-4 justify-center w-3/5">
<Input
type="text"
className="w-full p-2 border border-gray-300 rounded r"
placeholder="https://coconut.com/"
value={roastUrl}
onChange={(e) => setRoastUrl(e.target.value)}
/>
<Button
className="px-6 py-2 bg-red-500/25 text-red-500 0 rounded-lg hover:bg-red-300 w-1/8 whitespace-nowrap"
onClick={async () => {
if (roastUrl) {
setLoading(true);
try {
const response = await fetch(
`/api/roastWebsite?url=${encodeURIComponent(
roastUrl
)}&spiceLevel=${spiceLevel}`
);
if (!response.ok) {
throw new Error(`Error: ${response.statusText}`);
}
const data = await response.json();
setRoastData(data.roastResult);
} catch (error) {
console.error("Error:", error);
} finally {
setLoading(false);
}
}
}}
>
{loading ? "Loading..." : "Get Roasted 🌶️"}
</Button>
</div>
<div className="flex items-center justify-center mt-4">
<label
htmlFor="spice-level"
className="mr-4 font-medium text-gray-700"
>
Choose your roast level:
</label>
<select
id="spice-level"
className="cursor-pointer rounded-lg border border-gray-300 bg-white py-2 px-4 text-center shadow-sm focus:border-orange-500 focus:outline-none focus:ring-1 focus:ring-orange-500"
value={spiceLevel}
onChange={(e) => setSpiceLevel(Number(e.target.value))}
>
<option value={1}>Mild 🌶</option>
<option value={2}>Medium 🌶🌶</option>
<option value={3}>Spicy 🌶🌶🌶</option>
</select>
</div>
{loading ? (
<div className="mt-4 w-3/5 p-4 border border-gray-300 rounded shadow bg-gradient-to-r from-red-500 to-red-400 animate-pulse">
<p className="text-white text-center">Preparing your roast...</p>
</div>
) : (
roastData && (
<div className="!font-sans mt-4 w-3/5 p-4 border border-gray-300 rounded shadow">
<p>{roastData}</p>
</div>
)
)}
</div>
<div
className={`fixed bottom-0 left-0 right-0 p-4 text-white text-center font-light flex justify-center items-center gap-4`}
>
<a
href="https://firecrawl.dev"
target="_blank"
className="text-black hover:text-orange-400"
style={{ textShadow: "2px 2px 4px rgba(0, 0, 0, 0.15)" }}
>
A demo web scraping and vision extraction from Firecrawl 🔥
</a>
</div>
</main>
</div>
);
}

View File

@ -1,56 +0,0 @@
import * as React from "react"
import { Slot } from "@radix-ui/react-slot"
import { cva, type VariantProps } from "class-variance-authority"
import { cn } from "@/lib/utils"
const buttonVariants = cva(
"inline-flex items-center justify-center whitespace-nowrap rounded-md text-sm font-medium ring-offset-white transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-zinc-950 focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 dark:ring-offset-zinc-950 dark:focus-visible:ring-zinc-300",
{
variants: {
variant: {
default: "bg-zinc-900 text-zinc-50 hover:bg-zinc-900/90 dark:bg-zinc-50 dark:text-zinc-900 dark:hover:bg-zinc-50/90",
destructive:
"bg-red-500 text-zinc-50 hover:bg-red-500/90 dark:bg-red-900 dark:text-zinc-50 dark:hover:bg-red-900/90",
outline:
"border border-zinc-200 bg-white hover:bg-zinc-100 hover:text-zinc-900 dark:border-zinc-800 dark:bg-zinc-950 dark:hover:bg-zinc-800 dark:hover:text-zinc-50",
secondary:
"bg-zinc-100 text-zinc-900 hover:bg-zinc-100/80 dark:bg-zinc-800 dark:text-zinc-50 dark:hover:bg-zinc-800/80",
ghost: "hover:bg-zinc-100 hover:text-zinc-900 dark:hover:bg-zinc-800 dark:hover:text-zinc-50",
link: "text-zinc-900 underline-offset-4 hover:underline dark:text-zinc-50",
},
size: {
default: "h-10 px-4 py-2",
sm: "h-9 rounded-md px-3",
lg: "h-11 rounded-md px-8",
icon: "h-10 w-10",
},
},
defaultVariants: {
variant: "default",
size: "default",
},
}
)
export interface ButtonProps
extends React.ButtonHTMLAttributes<HTMLButtonElement>,
VariantProps<typeof buttonVariants> {
asChild?: boolean
}
const Button = React.forwardRef<HTMLButtonElement, ButtonProps>(
({ className, variant, size, asChild = false, ...props }, ref) => {
const Comp = asChild ? Slot : "button"
return (
<Comp
className={cn(buttonVariants({ variant, size, className }))}
ref={ref}
{...props}
/>
)
}
)
Button.displayName = "Button"
export { Button, buttonVariants }

View File

@ -1,122 +0,0 @@
"use client"
import * as React from "react"
import * as DialogPrimitive from "@radix-ui/react-dialog"
import { X } from "lucide-react"
import { cn } from "@/lib/utils"
const Dialog = DialogPrimitive.Root
const DialogTrigger = DialogPrimitive.Trigger
const DialogPortal = DialogPrimitive.Portal
const DialogClose = DialogPrimitive.Close
const DialogOverlay = React.forwardRef<
React.ElementRef<typeof DialogPrimitive.Overlay>,
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Overlay>
>(({ className, ...props }, ref) => (
<DialogPrimitive.Overlay
ref={ref}
className={cn(
"fixed inset-0 z-50 bg-black/80 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0",
className
)}
{...props}
/>
))
DialogOverlay.displayName = DialogPrimitive.Overlay.displayName
const DialogContent = React.forwardRef<
React.ElementRef<typeof DialogPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Content>
>(({ className, children, ...props }, ref) => (
<DialogPortal>
<DialogOverlay />
<DialogPrimitive.Content
ref={ref}
className={cn(
"fixed left-[50%] top-[50%] z-50 grid w-full max-w-lg translate-x-[-50%] translate-y-[-50%] gap-4 border border-zinc-200 bg-white p-6 shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[state=closed]:slide-out-to-left-1/2 data-[state=closed]:slide-out-to-top-[48%] data-[state=open]:slide-in-from-left-1/2 data-[state=open]:slide-in-from-top-[48%] sm:rounded-lg dark:border-zinc-800 dark:bg-zinc-950",
className
)}
{...props}
>
{children}
<DialogPrimitive.Close className="absolute right-4 top-4 rounded-sm opacity-70 ring-offset-white transition-opacity hover:opacity-100 focus:outline-none focus:ring-2 focus:ring-zinc-950 focus:ring-offset-2 disabled:pointer-events-none data-[state=open]:bg-zinc-100 data-[state=open]:text-zinc-500 dark:ring-offset-zinc-950 dark:focus:ring-zinc-300 dark:data-[state=open]:bg-zinc-800 dark:data-[state=open]:text-zinc-400">
<X className="h-4 w-4" />
<span className="sr-only">Close</span>
</DialogPrimitive.Close>
</DialogPrimitive.Content>
</DialogPortal>
))
DialogContent.displayName = DialogPrimitive.Content.displayName
const DialogHeader = ({
className,
...props
}: React.HTMLAttributes<HTMLDivElement>) => (
<div
className={cn(
"flex flex-col space-y-1.5 text-center sm:text-left",
className
)}
{...props}
/>
)
DialogHeader.displayName = "DialogHeader"
const DialogFooter = ({
className,
...props
}: React.HTMLAttributes<HTMLDivElement>) => (
<div
className={cn(
"flex flex-col-reverse sm:flex-row sm:justify-end sm:space-x-2",
className
)}
{...props}
/>
)
DialogFooter.displayName = "DialogFooter"
const DialogTitle = React.forwardRef<
React.ElementRef<typeof DialogPrimitive.Title>,
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Title>
>(({ className, ...props }, ref) => (
<DialogPrimitive.Title
ref={ref}
className={cn(
"text-lg font-semibold leading-none tracking-tight",
className
)}
{...props}
/>
))
DialogTitle.displayName = DialogPrimitive.Title.displayName
const DialogDescription = React.forwardRef<
React.ElementRef<typeof DialogPrimitive.Description>,
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Description>
>(({ className, ...props }, ref) => (
<DialogPrimitive.Description
ref={ref}
className={cn("text-sm text-zinc-500 dark:text-zinc-400", className)}
{...props}
/>
))
DialogDescription.displayName = DialogPrimitive.Description.displayName
export {
Dialog,
DialogPortal,
DialogOverlay,
DialogClose,
DialogTrigger,
DialogContent,
DialogHeader,
DialogFooter,
DialogTitle,
DialogDescription,
}

View File

@ -1,200 +0,0 @@
"use client"
import * as React from "react"
import * as DropdownMenuPrimitive from "@radix-ui/react-dropdown-menu"
import { Check, ChevronRight, Circle } from "lucide-react"
import { cn } from "@/lib/utils"
const DropdownMenu = DropdownMenuPrimitive.Root
const DropdownMenuTrigger = DropdownMenuPrimitive.Trigger
const DropdownMenuGroup = DropdownMenuPrimitive.Group
const DropdownMenuPortal = DropdownMenuPrimitive.Portal
const DropdownMenuSub = DropdownMenuPrimitive.Sub
const DropdownMenuRadioGroup = DropdownMenuPrimitive.RadioGroup
const DropdownMenuSubTrigger = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.SubTrigger>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.SubTrigger> & {
inset?: boolean
}
>(({ className, inset, children, ...props }, ref) => (
<DropdownMenuPrimitive.SubTrigger
ref={ref}
className={cn(
"flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none focus:bg-zinc-100 data-[state=open]:bg-zinc-100 dark:focus:bg-zinc-800 dark:data-[state=open]:bg-zinc-800",
inset && "pl-8",
className
)}
{...props}
>
{children}
<ChevronRight className="ml-auto h-4 w-4" />
</DropdownMenuPrimitive.SubTrigger>
))
DropdownMenuSubTrigger.displayName =
DropdownMenuPrimitive.SubTrigger.displayName
const DropdownMenuSubContent = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.SubContent>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.SubContent>
>(({ className, ...props }, ref) => (
<DropdownMenuPrimitive.SubContent
ref={ref}
className={cn(
"z-50 min-w-[8rem] overflow-hidden rounded-md border border-zinc-200 bg-white p-1 text-zinc-950 shadow-lg data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 dark:border-zinc-800 dark:bg-zinc-950 dark:text-zinc-50",
className
)}
{...props}
/>
))
DropdownMenuSubContent.displayName =
DropdownMenuPrimitive.SubContent.displayName
const DropdownMenuContent = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.Content>
>(({ className, sideOffset = 4, ...props }, ref) => (
<DropdownMenuPrimitive.Portal>
<DropdownMenuPrimitive.Content
ref={ref}
sideOffset={sideOffset}
className={cn(
"z-50 min-w-[8rem] overflow-hidden rounded-md border border-zinc-200 bg-white p-1 text-zinc-950 shadow-md data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 dark:border-zinc-800 dark:bg-zinc-950 dark:text-zinc-50",
className
)}
{...props}
/>
</DropdownMenuPrimitive.Portal>
))
DropdownMenuContent.displayName = DropdownMenuPrimitive.Content.displayName
const DropdownMenuItem = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.Item>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.Item> & {
inset?: boolean
}
>(({ className, inset, ...props }, ref) => (
<DropdownMenuPrimitive.Item
ref={ref}
className={cn(
"relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none transition-colors focus:bg-zinc-100 focus:text-zinc-900 data-[disabled]:pointer-events-none data-[disabled]:opacity-50 dark:focus:bg-zinc-800 dark:focus:text-zinc-50",
inset && "pl-8",
className
)}
{...props}
/>
))
DropdownMenuItem.displayName = DropdownMenuPrimitive.Item.displayName
const DropdownMenuCheckboxItem = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.CheckboxItem>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.CheckboxItem>
>(({ className, children, checked, ...props }, ref) => (
<DropdownMenuPrimitive.CheckboxItem
ref={ref}
className={cn(
"relative flex cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none transition-colors focus:bg-zinc-100 focus:text-zinc-900 data-[disabled]:pointer-events-none data-[disabled]:opacity-50 dark:focus:bg-zinc-800 dark:focus:text-zinc-50",
className
)}
checked={checked}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
</DropdownMenuPrimitive.CheckboxItem>
))
DropdownMenuCheckboxItem.displayName =
DropdownMenuPrimitive.CheckboxItem.displayName
const DropdownMenuRadioItem = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.RadioItem>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.RadioItem>
>(({ className, children, ...props }, ref) => (
<DropdownMenuPrimitive.RadioItem
ref={ref}
className={cn(
"relative flex cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none transition-colors focus:bg-zinc-100 focus:text-zinc-900 data-[disabled]:pointer-events-none data-[disabled]:opacity-50 dark:focus:bg-zinc-800 dark:focus:text-zinc-50",
className
)}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Circle className="h-2 w-2 fill-current" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
</DropdownMenuPrimitive.RadioItem>
))
DropdownMenuRadioItem.displayName = DropdownMenuPrimitive.RadioItem.displayName
const DropdownMenuLabel = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.Label>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.Label> & {
inset?: boolean
}
>(({ className, inset, ...props }, ref) => (
<DropdownMenuPrimitive.Label
ref={ref}
className={cn(
"px-2 py-1.5 text-sm font-semibold",
inset && "pl-8",
className
)}
{...props}
/>
))
DropdownMenuLabel.displayName = DropdownMenuPrimitive.Label.displayName
const DropdownMenuSeparator = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.Separator>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.Separator>
>(({ className, ...props }, ref) => (
<DropdownMenuPrimitive.Separator
ref={ref}
className={cn("-mx-1 my-1 h-px bg-zinc-100 dark:bg-zinc-800", className)}
{...props}
/>
))
DropdownMenuSeparator.displayName = DropdownMenuPrimitive.Separator.displayName
const DropdownMenuShortcut = ({
className,
...props
}: React.HTMLAttributes<HTMLSpanElement>) => {
return (
<span
className={cn("ml-auto text-xs tracking-widest opacity-60", className)}
{...props}
/>
)
}
DropdownMenuShortcut.displayName = "DropdownMenuShortcut"
export {
DropdownMenu,
DropdownMenuTrigger,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuCheckboxItem,
DropdownMenuRadioItem,
DropdownMenuLabel,
DropdownMenuSeparator,
DropdownMenuShortcut,
DropdownMenuGroup,
DropdownMenuPortal,
DropdownMenuSub,
DropdownMenuSubContent,
DropdownMenuSubTrigger,
DropdownMenuRadioGroup,
}

View File

@ -1,25 +0,0 @@
import * as React from "react"
import { cn } from "@/lib/utils"
export interface InputProps
extends React.InputHTMLAttributes<HTMLInputElement> {}
const Input = React.forwardRef<HTMLInputElement, InputProps>(
({ className, type, ...props }, ref) => {
return (
<input
type={type}
className={cn(
"flex h-10 w-full rounded-md border border-zinc-200 bg-white px-3 py-2 text-sm ring-offset-white file:border-0 file:bg-transparent file:text-sm file:font-medium placeholder:text-zinc-500 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-zinc-950 focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 dark:border-zinc-800 dark:bg-zinc-950 dark:ring-offset-zinc-950 dark:placeholder:text-zinc-400 dark:focus-visible:ring-zinc-300",
className
)}
ref={ref}
{...props}
/>
)
}
)
Input.displayName = "Input"
export { Input }

View File

@ -1,160 +0,0 @@
"use client"
import * as React from "react"
import * as SelectPrimitive from "@radix-ui/react-select"
import { Check, ChevronDown, ChevronUp } from "lucide-react"
import { cn } from "@/lib/utils"
const Select = SelectPrimitive.Root
const SelectGroup = SelectPrimitive.Group
const SelectValue = SelectPrimitive.Value
const SelectTrigger = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.Trigger>,
Omit<React.ComponentPropsWithoutRef<typeof SelectPrimitive.Trigger>, 'noIcon'> & { noIcon?: boolean }
>(({ className, children, noIcon, ...props }, ref) => (
<SelectPrimitive.Trigger
ref={ref}
className={cn(
"flex h-10 w-full items-center justify-between rounded-md border border-zinc-200 bg-white px-3 py-2 text-sm ring-offset-white placeholder:text-zinc-500 focus:outline-none focus:ring-2 focus:ring-zinc-950 focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1 dark:border-zinc-800 dark:bg-zinc-950 dark:ring-offset-zinc-950 dark:placeholder:text-zinc-400 dark:focus:ring-zinc-300",
className
)}
{...props}
>
{children}
<SelectPrimitive.Icon asChild>
{noIcon ? <></> : <ChevronDown className="h-4 w-4 opacity-50" />}
</SelectPrimitive.Icon>
</SelectPrimitive.Trigger>
))
SelectTrigger.displayName = SelectPrimitive.Trigger.displayName
const SelectScrollUpButton = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.ScrollUpButton>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.ScrollUpButton>
>(({ className, ...props }, ref) => (
<SelectPrimitive.ScrollUpButton
ref={ref}
className={cn(
"flex cursor-default items-center justify-center py-1",
className
)}
{...props}
>
<ChevronUp className="h-4 w-4" />
</SelectPrimitive.ScrollUpButton>
))
SelectScrollUpButton.displayName = SelectPrimitive.ScrollUpButton.displayName
const SelectScrollDownButton = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.ScrollDownButton>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.ScrollDownButton>
>(({ className, ...props }, ref) => (
<SelectPrimitive.ScrollDownButton
ref={ref}
className={cn(
"flex cursor-default items-center justify-center py-1",
className
)}
{...props}
>
<ChevronDown className="h-4 w-4" />
</SelectPrimitive.ScrollDownButton>
))
SelectScrollDownButton.displayName =
SelectPrimitive.ScrollDownButton.displayName
const SelectContent = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Content>
>(({ className, children, position = "popper", ...props }, ref) => (
<SelectPrimitive.Portal>
<SelectPrimitive.Content
ref={ref}
className={cn(
"relative z-50 max-h-96 min-w-[8rem] overflow-hidden rounded-md border border-zinc-200 bg-white text-zinc-950 shadow-md data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 dark:border-zinc-800 dark:bg-zinc-950 dark:text-zinc-50",
position === "popper" &&
"data-[side=bottom]:translate-y-1 data-[side=left]:-translate-x-1 data-[side=right]:translate-x-1 data-[side=top]:-translate-y-1",
className
)}
position={position}
{...props}
>
<SelectScrollUpButton />
<SelectPrimitive.Viewport
className={cn(
"p-1",
position === "popper" &&
"h-[var(--radix-select-trigger-height)] w-full min-w-[var(--radix-select-trigger-width)]"
)}
>
{children}
</SelectPrimitive.Viewport>
<SelectScrollDownButton />
</SelectPrimitive.Content>
</SelectPrimitive.Portal>
))
SelectContent.displayName = SelectPrimitive.Content.displayName
const SelectLabel = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.Label>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Label>
>(({ className, ...props }, ref) => (
<SelectPrimitive.Label
ref={ref}
className={cn("py-1.5 pl-8 pr-2 text-sm font-semibold", className)}
{...props}
/>
))
SelectLabel.displayName = SelectPrimitive.Label.displayName
const SelectItem = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.Item>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Item>
>(({ className, children, ...props }, ref) => (
<SelectPrimitive.Item
ref={ref}
className={cn(
"relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-zinc-100 focus:text-zinc-900 data-[disabled]:pointer-events-none data-[disabled]:opacity-50 dark:focus:bg-zinc-800 dark:focus:text-zinc-50",
className
)}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<SelectPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
</SelectPrimitive.ItemIndicator>
</span>
<SelectPrimitive.ItemText>{children}</SelectPrimitive.ItemText>
</SelectPrimitive.Item>
))
SelectItem.displayName = SelectPrimitive.Item.displayName
const SelectSeparator = React.forwardRef<
React.ElementRef<typeof SelectPrimitive.Separator>,
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Separator>
>(({ className, ...props }, ref) => (
<SelectPrimitive.Separator
ref={ref}
className={cn("-mx-1 my-1 h-px bg-zinc-100 dark:bg-zinc-800", className)}
{...props}
/>
))
SelectSeparator.displayName = SelectPrimitive.Separator.displayName
export {
Select,
SelectGroup,
SelectValue,
SelectTrigger,
SelectContent,
SelectLabel,
SelectItem,
SelectSeparator,
SelectScrollUpButton,
SelectScrollDownButton,
}

View File

@ -1,31 +0,0 @@
"use client"
import { useTheme } from "next-themes"
import { Toaster as Sonner } from "sonner"
type ToasterProps = React.ComponentProps<typeof Sonner>
const Toaster = ({ ...props }: ToasterProps) => {
const { theme = "system" } = useTheme()
return (
<Sonner
theme={theme as ToasterProps["theme"]}
className="toaster group"
toastOptions={{
classNames: {
toast:
"group toast group-[.toaster]:bg-white group-[.toaster]:text-zinc-950 group-[.toaster]:border-zinc-200 group-[.toaster]:shadow-lg dark:group-[.toaster]:bg-zinc-950 dark:group-[.toaster]:text-zinc-50 dark:group-[.toaster]:border-zinc-800",
description: "group-[.toast]:text-zinc-500 dark:group-[.toast]:text-zinc-400",
actionButton:
"group-[.toast]:bg-zinc-900 group-[.toast]:text-zinc-50 dark:group-[.toast]:bg-zinc-50 dark:group-[.toast]:text-zinc-900",
cancelButton:
"group-[.toast]:bg-zinc-100 group-[.toast]:text-zinc-500 dark:group-[.toast]:bg-zinc-800 dark:group-[.toast]:text-zinc-400",
},
}}
{...props}
/>
)
}
export { Toaster }

View File

@ -1,29 +0,0 @@
"use client"
import * as React from "react"
import * as SwitchPrimitives from "@radix-ui/react-switch"
import { cn } from "@/lib/utils"
const Switch = React.forwardRef<
React.ElementRef<typeof SwitchPrimitives.Root>,
React.ComponentPropsWithoutRef<typeof SwitchPrimitives.Root>
>(({ className, ...props }, ref) => (
<SwitchPrimitives.Root
className={cn(
"peer inline-flex h-6 w-11 shrink-0 cursor-pointer items-center rounded-full border-2 border-transparent transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-zinc-950 focus-visible:ring-offset-2 focus-visible:ring-offset-white disabled:cursor-not-allowed disabled:opacity-50 data-[state=checked]:bg-zinc-900 data-[state=unchecked]:bg-zinc-200 dark:focus-visible:ring-zinc-300 dark:focus-visible:ring-offset-zinc-950 dark:data-[state=checked]:bg-zinc-50 dark:data-[state=unchecked]:bg-zinc-800",
className
)}
{...props}
ref={ref}
>
<SwitchPrimitives.Thumb
className={cn(
"pointer-events-none block h-5 w-5 rounded-full bg-white shadow-lg ring-0 transition-transform data-[state=checked]:translate-x-5 data-[state=unchecked]:translate-x-0 dark:bg-zinc-950"
)}
/>
</SwitchPrimitives.Root>
))
Switch.displayName = SwitchPrimitives.Root.displayName
export { Switch }

View File

@ -1,24 +0,0 @@
import * as React from "react"
import { cn } from "@/lib/utils"
export interface TextareaProps
extends React.TextareaHTMLAttributes<HTMLTextAreaElement> {}
const Textarea = React.forwardRef<HTMLTextAreaElement, TextareaProps>(
({ className, ...props }, ref) => {
return (
<textarea
className={cn(
"flex min-h-[80px] w-full rounded-md border border-zinc-200 bg-white px-3 py-2 text-sm ring-offset-white placeholder:text-zinc-500 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-zinc-950 focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 dark:border-zinc-800 dark:bg-zinc-950 dark:ring-offset-zinc-950 dark:placeholder:text-zinc-400 dark:focus-visible:ring-zinc-300",
className
)}
ref={ref}
{...props}
/>
)
}
)
Textarea.displayName = "Textarea"
export { Textarea }

View File

@ -1,75 +0,0 @@
import OpenAI from "openai/index.mjs";
import { encoding_for_model } from "@dqbd/tiktoken";
/**
* Function to generate a roast for a website based on its screenshot and markdown content.
* @param roastPrompt - Initial prompt text for the roast.
* @param screenshotUrl - URL of the screenshot of the website.
* @param content - Raw markdown content of the website.
*/
export async function roastPrompt(roastPrompt: string, screenshotUrl: string, content: string) {
try {
// Initialize OpenAI with the API key from environment variables
const openai = new OpenAI({
apiKey: process.env.OPEN_AI_KEY
});
let contentTruncated = await truncateContentToFit(content, 30000);
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [
{
role: "user",
content: [
{ type: "text", text: roastPrompt },
{ type: "image_url", image_url: { url: screenshotUrl, detail: "low" } },
{ type: "text", text: contentTruncated }
],
},
],
});
// Return the first choice's message instead of logging it
return response.choices[0].message.content;
} catch (error) {
console.error("Error generating roast:", error);
// Assert error as an instance of Error to access message property
return `Error generating roast: ${(error as Error).message}`;
}
}
export function numTokensFromString(message: string, model: string): number {
const encoder = encoding_for_model(model as any);
// Encode the message into tokens
const tokens = encoder.encode(message);
// Free the encoder resources after use
encoder.free();
// Return the number of tokens
return tokens.length;
}
async function truncateContentToFit(content: string, maxTokens: number): Promise<string> {
const modifier = 4;
let contentTotruncate = content;
const numTokens = numTokensFromString(contentTotruncate, "gpt-4");
if (numTokens > maxTokens) {
// trim the document to the maximum number of tokens, tokens != characters
contentTotruncate = content.slice(0, (maxTokens * modifier));
}
return contentTotruncate
}

View File

@ -1,6 +0,0 @@
import { type ClassValue, clsx } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}

View File

@ -1,76 +0,0 @@
// pages/api/getRepoStars.ts
import { NextApiRequest, NextApiResponse } from 'next';
import FirecrawlApp from '@mendable/firecrawl-js';
import { roastPrompt } from '../../lib/LLM/llm';
const mildPrompt = "Give me a light hearted roast for this website:"
const mediumPrompt = "Give me a roast for this website:"
const spicyPrompt ="Roast this website. The person who made this website wants to be roasted really hard. I mean REALLY hard, don't hold back, and don't worry about hurting their feelings. Be mean but concisely mean. No more than 2 paragraphs."
async function getScreenShotAndHtml(url: string) {
const app = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const scrapeResult = await app.scrapeUrl(url, {
pageOptions: {
includeHtml: true,
screenshot: true
}
});
return {
screenshotUrl: scrapeResult.data.metadata.screenshot,
htmlContent: scrapeResult.data.html
};
}
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const rawUrl = typeof req.query.url === 'string' ? req.query.url : null;
const spiceLevel = typeof req.query.spiceLevel === 'string' ? parseInt(req.query.spiceLevel, 10) : null;
if (!rawUrl || spiceLevel === null || isNaN(spiceLevel)) {
res.status(400).json({ error: 'Invalid query parameters' });
return;
}
try {
const { screenshotUrl, htmlContent } = await getScreenShotAndHtml(rawUrl);
// Define a roast prompt message
let roastMessage: string;
// Determine the roast message based on the spice level
switch (spiceLevel) {
case 1:
roastMessage = mildPrompt;
break;
case 2:
roastMessage = mediumPrompt;
break;
case 3:
roastMessage = spicyPrompt;
break;
default:
// If an invalid spice level is provided, default to mild roast
roastMessage = mildPrompt;
res.status(400).json({ error: 'Invalid spice level' });
return;
}
// Convert HTML content to a markdown-like format for the roast generation
// This is a simplified conversion, assuming HTML content is already sanitized and suitable for direct usage
// Call the roastPrompt function to generate a roast
const roastResult = await roastPrompt(roastMessage, screenshotUrl, htmlContent);
// Log the roast result for debugging
// console.log("Roast Result:", roastResult);
res.status(200).json({ screenshotUrl, htmlContent, roastResult});
} catch (error: any) {
res.status(500).json({ error: error.message });
}
}

View File

@ -1,137 +0,0 @@
import type { Config } from 'tailwindcss';
import colors from 'tailwindcss/colors';
const config: Config = {
darkMode: 'class',
content: [
'./src/**/*.{js,ts,jsx,tsx}',
// Path to Tremor module
'./node_modules/@tremor/**/*.{js,ts,jsx,tsx}',
],
theme: {
transparent: 'transparent',
current: 'currentColor',
extend: {
colors: {
// light mode
tremor: {
brand: {
faint: colors.blue[50],
muted: colors.blue[200],
subtle: colors.blue[400],
DEFAULT: colors.blue[500],
emphasis: colors.blue[700],
inverted: colors.white,
},
background: {
muted: colors.gray[50],
subtle: colors.gray[100],
DEFAULT: colors.white,
emphasis: colors.gray[700],
},
border: {
DEFAULT: colors.gray[200],
},
ring: {
DEFAULT: colors.gray[200],
},
content: {
subtle: colors.gray[400],
DEFAULT: colors.gray[500],
emphasis: colors.gray[700],
strong: colors.gray[900],
inverted: colors.white,
},
},
// dark mode
'dark-tremor': {
brand: {
faint: '#0B1229',
muted: colors.blue[950],
subtle: colors.blue[800],
DEFAULT: colors.blue[500],
emphasis: colors.blue[400],
inverted: colors.blue[950],
},
background: {
muted: '#131A2B',
subtle: colors.gray[800],
DEFAULT: colors.gray[900],
emphasis: colors.gray[300],
},
border: {
DEFAULT: colors.gray[800],
},
ring: {
DEFAULT: colors.gray[800],
},
content: {
subtle: colors.gray[600],
DEFAULT: colors.gray[500],
emphasis: colors.gray[200],
strong: colors.gray[50],
inverted: colors.gray[950],
},
},
},
boxShadow: {
// light
'tremor-input': '0 1px 2px 0 rgb(0 0 0 / 0.05)',
'tremor-card':
'0 1px 3px 0 rgb(0 0 0 / 0.1), 0 1px 2px -1px rgb(0 0 0 / 0.1)',
'tremor-dropdown':
'0 4px 6px -1px rgb(0 0 0 / 0.1), 0 2px 4px -2px rgb(0 0 0 / 0.1)',
// dark
'dark-tremor-input': '0 1px 2px 0 rgb(0 0 0 / 0.05)',
'dark-tremor-card':
'0 1px 3px 0 rgb(0 0 0 / 0.1), 0 1px 2px -1px rgb(0 0 0 / 0.1)',
'dark-tremor-dropdown':
'0 4px 6px -1px rgb(0 0 0 / 0.1), 0 2px 4px -2px rgb(0 0 0 / 0.1)',
},
borderRadius: {
'tremor-small': '0.375rem',
'tremor-default': '0.5rem',
'tremor-full': '9999px',
},
fontSize: {
'tremor-label': ['0.75rem', { lineHeight: '1rem' }],
'tremor-default': ['0.875rem', { lineHeight: '1.25rem' }],
'tremor-title': ['1.125rem', { lineHeight: '1.75rem' }],
'tremor-metric': ['1.875rem', { lineHeight: '2.25rem' }],
},
},
},
safelist: [
{
pattern:
/^(bg-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
variants: ['hover', 'ui-selected'],
},
{
pattern:
/^(text-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
variants: ['hover', 'ui-selected'],
},
{
pattern:
/^(border-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
variants: ['hover', 'ui-selected'],
},
{
pattern:
/^(ring-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
},
{
pattern:
/^(stroke-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
},
{
pattern:
/^(fill-(?:slate|gray|zinc|neutral|stone|red|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet|purple|fuchsia|pink|rose)-(?:50|100|200|300|400|500|600|700|800|900|950))$/,
},
],
plugins: [require('@headlessui/tailwindcss'), require('@tailwindcss/forms')],
};
export default config;

View File

@ -1,26 +0,0 @@
{
"compilerOptions": {
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"incremental": true,
"plugins": [
{
"name": "next"
}
],
"paths": {
"@/*": ["./src/*"]
}
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
}

View File

@ -0,0 +1 @@
Full examples apps built with Firecrawl can be found at this repo: https://github.com/mendableai/firecrawl-app-examples