feat: Use llm.txt for doc reference. rewrite this pr to include searc… (#21)

* feat: Use llm.txt for doc reference. rewrite this pr to include search doc based on frequency and relevance & fetch doc from uri

* fix: Remove unrelated files

* fix: revert code change in github workflow

* fix: readme naming fix

* fix: remove comment

* fix: refactor some code, optimize some logic

* fix:revert github workflow

Author: JackYPCOnline

.github/workflows/pypi-publish-on-release.yml

@@ -22,37 +22,37 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v4
-      with:
-        persist-credentials: false
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
 
-    - name: Set up Python
-      uses: actions/setup-python@v5
-      with:
-        python-version: '3.10'
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
 
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install hatch twine
-    - name: Validate version
-      run: |
-        version=$(hatch version)
-        if [[ $version =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
-            echo "Valid version format"
-            exit 0
-        else
-            echo "Invalid version format"
-            exit 1
-        fi
-    - name: Build
-      run: |
-        hatch build
-    - name: Store the distribution packages
-      uses: actions/upload-artifact@v4
-      with:
-        name: python-package-distributions
-        path: dist/
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install hatch twine
+      - name: Validate version
+        run: |
+          version=$(hatch version)
+          if [[ $version =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+              echo "Valid version format"
+              exit 0
+          else
+              echo "Invalid version format"
+              exit 1
+          fi
+      - name: Build
+        run: |
+          hatch build
+      - name: Store the distribution packages
+        uses: actions/upload-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
 
   deploy:
     name: Upload release to PyPI
@@ -70,10 +70,10 @@ jobs:
       id-token: write
 
     steps:
-    - name: Download all the dists
-      uses: actions/download-artifact@v4
-      with:
-        name: python-package-distributions
-        path: dist/
-    - name: Publish distribution 📦 to PyPI
-      uses: pypa/gh-action-pypi-publish@release/v1
+      - name: Download all the dists
+        uses: actions/download-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
+      - name: Publish distribution 📦 to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

.gitignore

@@ -10,4 +10,4 @@ __pycache__*
 .vscode
 dist
 venv/
-*.egg-info
+*.egg-info

README.md

@@ -32,7 +32,15 @@
   </p>
 </div>
 
-This MCP server provides documentation about Strands Agents to your GenAI tools, so you can use your favorite AI coding assistant to vibe-code Strands Agents.
+This MCP server provides curated documentation access to your GenAI tools via llms.txt files, enabling AI coding assistants to search and retrieve relevant documentation with intelligent ranking.
+
+## Features
+
+- **Smart Document Search**: TF-IDF based search with Markdown-aware scoring that prioritizes titles, headers, and code blocks
+- **Curated Content**: Indexes documentation from llms.txt files with clean, human-readable titles
+- **On-Demand Fetching**: Lazy-loads full document content only when needed for optimal performance
+- **Snippet Generation**: Provides contextual snippets with relevance scoring for quick overview
+- **Real URL Support**: Works with actual HTTPS URLs while maintaining backward compatibility
 
 ## Prerequisites
 
@@ -54,9 +62,17 @@ In `~/.aws/amazonq/mcp.json`:
 ```json
 {
   "mcpServers": {
-    "strands": {
+    "strands-agents": {
       "command": "uvx",
-      "args": ["strands-agents-mcp-server"]
+      "args": ["strands-agents-mcp-server"],
+      "env": {
+        "FASTMCP_LOG_LEVEL": "INFO"
+      },
+      "disabled": false,
+      "autoApprove": [
+        "search_docs",
+        "fetch_doc"
+      ]
     }
   }
 }
@@ -94,9 +110,17 @@ In `~/.cursor/mcp.json`:
 ```json
 {
   "mcpServers": {
-    "strands": {
+    "strands-agents": {
       "command": "uvx",
-      "args": ["strands-agents-mcp-server"]
+      "args": ["strands-agents-mcp-server"],
+      "env": {
+        "FASTMCP_LOG_LEVEL": "INFO"
+      },
+      "disabled": false,
+      "autoApprove": [
+        "search_docs",
+        "fetch_doc"
+      ]
     }
   }
 }
@@ -107,14 +131,42 @@ In `~/.cursor/mcp.json`:
 You can quickly test the MCP server using the MCP Inspector:
 
 ```bash
+# For published package
 npx @modelcontextprotocol/inspector uvx strands-agents-mcp-server
+
+# For local development
+npx @modelcontextprotocol/inspector python -m strands_mcp_server
 ```
 
 Note: This requires [npx](https://docs.npmjs.com/cli/v11/commands/npx) to be installed on your system. It comes bundled with [Node.js](https://nodejs.org/). 
 
 The Inspector is also useful for troubleshooting MCP server issues as it provides detailed connection and protocol information. For an in-depth guide, have a look at the [MCP Inspector documentation](https://modelcontextprotocol.io/docs/tools/inspector).
 
-## Server development
+## Getting Started
+
+1. **Install prerequisites**:
+   - Install [uv](https://github.com/astral-sh/uv) following the [official installation instructions](https://github.com/astral-sh/uv#installation)
+   - Make sure you have [Node.js](https://nodejs.org/) installed for npx commands
+
+2. **Configure your MCP client**:
+   - Choose your preferred MCP client from the installation examples above
+   - Add the Strands Agents MCP server configuration to your client
+
+3. **Test the connection**:
+   ```bash
+   # For published package
+   npx @modelcontextprotocol/inspector uvx strands-agents-mcp-server
+   
+   # For local development
+   npx @modelcontextprotocol/inspector python -m strands_mcp_server
+   ```
+
+4. **Start using the documentation tools**:
+   - Use `search_docs` to find relevant documentation with intelligent ranking
+   - Use `fetch_doc` to retrieve full content from specific URLs
+   - The server automatically indexes curated content from llms.txt files
+
+## Server Development
 
 ```bash
 git clone https://github.com/strands-agents/mcp-server.git

src/strands_mcp_server/__init__.py

@@ -1,10 +0,0 @@
-from . import server
-
-
-def main():
-    """Strands Agents MCP Server"""
-    server.main()
-
-
-if __name__ == "__main__":
-    main()

src/strands_mcp_server/__main__.py

@@ -1,3 +1,10 @@
-from strands_mcp_server import main
+"""Entry point for running the Strands MCP Server as a module.
 
-main()
+Usage:
+    python -m strands_mcp_server
+"""
+
+from .server import main
+
+if __name__ == "__main__":
+    main()

src/strands_mcp_server/config.py

@@ -0,0 +1,22 @@
+from dataclasses import dataclass, field
+
+
+@dataclass
+class Config:
+    """Configuration settings for the MCP server.
+
+    Attributes:
+        llm_texts_url: List of llms.txt URLs to index for documentation
+        timeout: HTTP request timeout in seconds
+        user_agent: User agent string for HTTP requests
+    """
+
+    llm_texts_url: list[str] = field(
+        default_factory=lambda: ["https://strandsagents.com/latest/llms.txt"]
+    )  # Curated list of llms.txt files to index at startup
+    timeout: float = 30.0  # HTTP request timeout in seconds
+    user_agent: str = "strands-mcp-docs/1.0"  # User agent for HTTP requests
+
+
+# Global configuration instance
+doc_config = Config()