Migrate docs from Fern to Gitub

docs/README.md

@@ -0,0 +1,28 @@
+# R2R Documentation
+
+The most advanced AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.
+
+## Documentation Sections
+
+### [Introduction](./introduction/)
+- [System Overview](./introduction/system.md)
+- [Guides](./introduction/guides/)
+
+### [Documentation](./documentation/)
+- [Getting Started](./documentation/README.md)
+- [General Features](./documentation/general/)
+- [Retrieval](./documentation/retrieval/)
+- [Advanced Features](./documentation/advanced/)
+
+### [API & SDKs](./api/)
+- [API Reference](./api/)
+- [SDK Documentation](./api/)
+
+### [Cookbooks](./cookbooks/)
+- [Data Processing](./cookbooks/data-processing/)
+- [System Operations](./cookbooks/system-operations/)
+
+### [Self-Hosting](./self-hosting/)
+- [Installation](./self-hosting/getting-started/installation/)
+- [Configuration](./self-hosting/configuration/)
+- [Deployment](./self-hosting/deployment/)

docs/cookbooks/application.md

@@ -0,0 +1,114 @@
+R2R offers an [open-source React+Next.js application](https://github.com/SciPhi-AI/R2R-Application) designed to give developers an administrative portal for their R2R deployment, and users an application to communicate with out of the box.
+
+## Setup
+
+### Install PNPM
+
+PNPM is a fast, disk space-efficient package manager. To install PNPM, visit the [official PNPM installation page](https://pnpm.io/installation) or follow these instructions:
+
+<AccordionGroup>
+
+<Accordion icon="terminal" title="PNPM Installation">
+For Unix-based systems (Linux, macOS):
+
+```zsh
+curl -fsSL https://get.pnpm.io/install.sh | sh -
+```
+
+For Windows:
+
+```powershell
+iwr https://get.pnpm.io/install.ps1 -useb | iex
+```
+
+After installation, you may need to add PNPM to your system's PATH.
+</Accordion>
+
+</AccordionGroup>
+
+### Installing and Running the R2R Dashboard
+
+If you're running R2R with the Docker, you already have the R2R application running! Just navigate to [http://localhost:7273](http://localhost:7273).
+
+If you're running R2R outside of Docker, run the following commands to install the R2R Dashboard.
+
+1. Clone the project repository and navigate to the project directory:
+
+```zsh
+git clone https://github.com/SciPhi-AI/R2R.git
+cd R2R-Application
+```
+
+2. Install the project dependencies:
+
+```zsh
+pnpm install
+```
+
+3. Build and start the application for production:
+
+```zsh
+pnpm build
+pnpm start
+```
+
+The dashboard will be available at [http://localhost:3000](http://localhost:3000).
+
+## Features
+
+### Login
+
+To interact with R2R with the dashboard, you must first login. If it's your first time logging in, log in with the default credentials shown.
+
+By default, an R2R instance is hosted on port 7272. The login page will include this URL by default, but be sure to update the URL if your R2R instance is deployed elsewhere. For information about deploying a local R2R application server, see the [quickstart](/documentation/quickstart).
+
+![R2R Dashboard Overview](./images/application/login.png)
+
+
+### Documents
+
+The documents page provides an overview of uploaded documents and their metadata. You can upload new documents and update, download, or delete existing ones. Additionally, you can view information about each document, including the documents' chunks and previews of PDFs.
+
+![Documents Page](./images/application/oss_dashboard_documents.png)
+
+### Collections
+
+Collections allow users to create and share sets of documents. The collections page provides a place to manage your existing collections or create new collections.
+
+![Collections Page](./images/application/oss_collections_page.png)
+
+### Chat
+
+In the chat page, you can stream RAG responses with different models and configurable settings. You can interact with both the RAG Agent and RAG endpoints here.
+
+![Chat Interface](./images/application/chat.png)
+
+### Users
+
+Manage your users and gain insight into their interactions.
+
+![Users Page](./images/application/users.png)
+
+### Settings
+
+The settings page allows you to view the configuration of and edit the prompts associated with your R2R deployment.
+
+![Settings Page](./images/application/settings_config.png)
+![Settings Page](./images/application/settings_prompts.png)
+
+## Development
+
+To develop the R2R dashboard:
+
+1. Start the development server:
+
+```zsh
+pnpm dev
+```
+
+2. Run pre-commit checks (optional but recommended):
+
+```zsh
+pnpm format
+pnpm lint
+```

docs/cookbooks/custom-tools.md

@@ -0,0 +1,133 @@
+There are many cases where it is helpful to define custom tools for the RAG Agent. R2R allows for users to define custom tools, passing these definitions into the Agent at server start.
+
+### Defining New Tools
+There is a directory in the R2R repository, `/docker/user_tools`, which is mounted to the R2R docker container. It is here that we will place our custom tool files.
+
+There, we will find a README.md file, which includes a template for our new tool:
+
+
+```python
+from core.base.agent.tools.base import Tool
+
+
+class ToolNameTool(Tool):
+    """
+    A user defined tool.
+    """
+
+    def __init__(self):
+        super().__init__(
+            name="tool_name",
+            description="A natural language tool description that is shown to the agent.",
+            parameters={
+                "type": "object",
+                "properties": {
+                    "input_parameter": {
+                        "type": "string",
+                        "description": "Define any input parameters by their name and type",
+                    },
+                },
+                "required": ["input_parameter"],
+            },
+            results_function=self.execute,
+            llm_format_function=None,
+        )
+
+    async def execute(self, input_parameter: str, *args, **kwargs):
+        """
+        Implementation of the tool.
+        """
+
+        # Any custom tool logic can go here
+
+        output_response = some_method(input_parameter)
+
+        result = AggregateSearchResult(
+            generic_tool_result=[web_response],
+        )
+
+        # Add to results collector if context is provided
+        if context and hasattr(context, "search_results_collector"):
+            context.search_results_collector.add_aggregate_result(result)
+
+        return result
+```
+
+This template has two basic methods:
+
+1. `__init__` is where we define the tool. The description that we make here is shown to the agent.
+2. `execute` is where we define any custom tool logic and interact with the inputs.
+
+### Writing our new tool
+
+Below, we have an example of a toy tool, which takes an integer and string input, returning a silly message to the agent. Should your tool require additional dependencies, be sure to include them in the `user_requirements.txt` file located in the `/docker` directory.
+
+```python
+from r2r import Tool, AggregateSearchResult
+
+
+class SecretMethodTool(Tool):
+    """
+    A user defined tool.
+    """
+
+    def __init__(self):
+        super().__init__(
+            name="secret_method",
+            description="Performs a secret method.",
+            parameters={
+                "type": "object",
+                "properties": {
+                    "number": {
+                        "type": "string",
+                        "description": "An integer input for the secret method.",
+                    },
+                    "string": {
+                        "type": "string",
+                        "description": "A string input for the secret method.",
+                    },
+                },
+                "required": ["number", "string"],
+            },
+            results_function=self.execute,
+            llm_format_function=None,
+        )
+
+    async def execute(self, number: int, string: str, *args, **kwargs):
+        """
+        Implementation of the tool.
+        """
+
+        output_response = f"Your order for {number} dancing flamingos has been received. They will arrive by unicycle courier within 3-5 business dreams. Please prepare {string} for them."
+
+        result = AggregateSearchResult(
+            generic_tool_result=output_response,
+        )
+
+        context = self.context
+        # Add to results collector if context is provided
+        if context and hasattr(context, "search_results_collector"):
+            context.search_results_collector.add_aggregate_result(result)
+
+        return result
+```
+
+Finally, we can modify our configuration file's `agent` section to include our new tool:
+
+```toml
+[agent]
+rag_tools = ["secret_method"]
+```
+
+
+Finally, we can run the following and see that our agent called our new method, passed the required parameters, and understood its output:
+
+```python
+client.retrieval.agent(
+    message={"role": "user", "content": "Can you run the secret method tool? Feel free to use any parameters you want. I just want to see the output."},
+)
+```
+
+```zsh
+results=AgentResponse(messages=[Message(role='assistant', content='The secret method tool produced the following output:\n\n"Your order for 42 dancing flamingos has been received. They will arrive by unicycle courier within 3-5 business dreams. Please prepare Hello, World! for them."\n\nThis whimsical response seems to be a playful and humorous output generated by the tool.', name=None, function_call=None, tool_calls=None, tool_call_id=None, metadata={'citations': [], 'tool_calls': [{'name': 'secret_method', 'args': '{"number":"42","string":"Hello, World!"}'}], 'aggregated_search_result': '[]'}, structured_content=None, image_url=None, image_data=None)], conversation_id='12ad2d6b-1429-48ea-9077-711726d8cfde')
+```

docs/cookbooks/email.md

@@ -0,0 +1,58 @@
+Configuring your deployment to require email verification helps keep your deployment secure, prevents unauthorized account creation,
+reduces spam registrations, and ensures you have valid contact information for your users.
+
+Currently, R2R has integrations for both [Mailersend](https://www.mailersend.com/) and [Sendgrid](https://sendgrid.com/).
+
+## Setup
+Both Mailersend and Sendgrid require registration, but do offer free tiers for evaluating their services. Create an account with your desired
+provider, and generate an API key.
+
+### Mailersend
+  - [Create an account](https://www.mailersend.com/signup)
+  - [Generate an API key](https://www.mailersend.com/help/managing-api-tokens)
+
+### Sendgrid
+  - [Create an account](https://twilio.com/signup)
+  - [Generate an API key](https://www.twilio.com/docs/sendgrid/ui/account-and-settings/api-keys)
+
+## Creating a Template
+Once you have registered for an account with your email provider, you will want to create an email template. Providers will have pre-made templates, or you can build these from scratch.
+
+![A Mailersend welcome template](./images/email/mailersend.png)
+
+
+Once you save a template, you will want to make note of the template id. These will go into the configuration files.
+
+## Configuration Settings
+We can then configure our deployment with the templates, redirect URL (`frontend_url`), and from email.
+
+### Configuration File
+
+
+```toml title="mailersend.toml"
+[email]
+provider = "mailersend"
+verify_email_template_id=""
+reset_password_template_id=""
+password_changed_template_id=""
+frontend_url=""
+from_email=""
+```
+
+```toml title="sendgrid.toml"
+[email]
+provider = "sendgrid"
+verify_email_template_id=""
+reset_password_template_id=""
+password_changed_template_id=""
+frontend_url=""
+from_email=""
+```
+
+### Environment Variables
+It is required to set your provider API key in your environment:
+
+```zsh
+export MAILERSEND_API_KEY=…
+export SENDGRID_API_KEY=…
+```