AIML-83: Improve README documentation structure

AGENTS.md

@@ -0,0 +1,144 @@
+# AI Agent Guidelines
+
+## Issue Tracking with bd (beads)
+
+**IMPORTANT**: This project uses **bd (beads)** for ALL issue tracking. Do NOT use markdown TODOs, task lists, or other tracking methods.
+
+### Why bd?
+
+- Dependency-aware: Track blockers and relationships between issues
+- Git-friendly: Auto-syncs to JSONL for version control
+- Agent-optimized: JSON output, ready work detection, discovered-from links
+- Prevents duplicate tracking systems and confusion
+
+### Quick Start
+
+**Check for ready work:**
+```bash
+bd ready --json
+```
+
+**Create new issues:**
+```bash
+bd create "Issue title" -t bug|feature|task -p 0-4 --json
+bd create "Issue title" -p 1 --deps discovered-from:bd-123 --json
+```
+
+**Claim and update:**
+```bash
+bd update bd-42 --status in_progress --json
+bd update bd-42 --priority 1 --json
+```
+
+**Complete work:**
+```bash
+bd close bd-42 --reason "Completed" --json
+```
+
+### Issue Types
+
+- `bug` - Something broken
+- `feature` - New functionality
+- `task` - Work item (tests, docs, refactoring)
+- `epic` - Large feature with subtasks
+- `chore` - Maintenance (dependencies, tooling)
+
+### Priorities
+
+- `0` - Critical (security, data loss, broken builds)
+- `1` - High (major features, important bugs)
+- `2` - Medium (default, nice-to-have)
+- `3` - Low (polish, optimization)
+- `4` - Backlog (future ideas)
+
+### Workflow for AI Agents
+
+1. **Check ready work**: `bd ready` shows unblocked issues
+2. **Claim your task**: `bd update <id> --status in_progress`
+3. **Work on it**: Implement, test, document
+4. **Discover new work?** Create linked issue:
+   - `bd create "Found bug" -p 1 --deps discovered-from:<parent-id>`
+5. **Complete**: `bd close <id> --reason "Done"`
+6. **Commit together**: Always commit the `.beads/issues.jsonl` file together with the code changes so issue state stays in sync with code state
+
+### Jira Integration
+
+When creating beads that relate to Jira tickets:
+- **Always prepend the Jira issue ID to the bead title**
+  - Example: `"AIML-224: Consolidate RouteCoverageService tools"`
+  - NOT: `"Consolidate RouteCoverageService tools"`
+- **Set the `external_ref` parameter to the Jira issue ID**
+  - Use `--external-ref AIML-224` in CLI or `external_ref="AIML-224"` in MCP calls
+- This creates bidirectional linkage:
+  - Bead tracks local technical work (code, tests, implementation)
+  - Jira tracks project management (planning, stakeholders, timeline)
+  - Clear connection between both via issue ID in title and external_ref field
+
+### Auto-Sync
+
+bd automatically syncs with git:
+- Exports to `.beads/issues.jsonl` after changes (5s debounce)
+- Imports from JSONL when newer (e.g., after `git pull`)
+- No manual export/import needed!
+
+### MCP Server (Recommended)
+
+If using Claude or MCP-compatible clients, install the beads MCP server:
+
+```bash
+pip install beads-mcp
+```
+
+Add to MCP config (e.g., `~/.config/claude/config.json`):
+```json
+{
+  "beads": {
+    "command": "beads-mcp",
+    "args": []
+  }
+}
+```
+
+Then use `mcp__beads__*` functions instead of CLI commands.
+
+### Managing AI-Generated Planning Documents
+
+AI assistants often create planning and design documents during development:
+- PLAN.md, IMPLEMENTATION.md, ARCHITECTURE.md
+- DESIGN.md, CODEBASE_SUMMARY.md, INTEGRATION_PLAN.md
+- TESTING_GUIDE.md, TECHNICAL_DESIGN.md, and similar files
+
+**Best Practice: Use a dedicated directory for these ephemeral files**
+
+**Recommended approach:**
+- Create a `history/` directory in the project root
+- Store ALL AI-generated planning/design docs in `history/`
+- Keep the repository root clean and focused on permanent project files
+- Only access `history/` when explicitly asked to review past planning
+
+**Example .gitignore entry (optional):**
+```
+# AI planning documents (ephemeral)
+history/
+```
+
+**Benefits:**
+- ✅ Clean repository root
+- ✅ Clear separation between ephemeral and permanent documentation
+- ✅ Easy to exclude from version control if desired
+- ✅ Preserves planning history for archeological research
+- ✅ Reduces noise when browsing the project
+
+### Important Rules
+
+- ✅ Use bd for ALL task tracking
+- ✅ Always use `--json` flag for programmatic use
+- ✅ Link discovered work with `discovered-from` dependencies
+- ✅ Check `bd ready` before asking "what should I work on?"
+- ✅ Store AI planning docs in `history/` directory
+- ❌ Do NOT create markdown TODO lists
+- ❌ Do NOT use external issue trackers
+- ❌ Do NOT duplicate tracking systems
+- ❌ Do NOT clutter repo root with planning documents
+
+For more details, see README.md and QUICKSTART.md.

CLAUDE.md

@@ -2,6 +2,8 @@
 
 This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
 
+**Note**: This project uses [bd (beads)](https://github.com/steveyegge/beads) for issue tracking. Use `bd` commands instead of markdown TODOs. See AGENTS.md for workflow details.
+
 ## Project Overview
 
 This is an MCP (Model Context Protocol) server for Contrast Security that enables AI agents to access and analyze vulnerability data from Contrast's security platform. It serves as a bridge between Contrast Security's API and AI tools like Claude, enabling automated vulnerability remediation and security analysis.
@@ -10,8 +12,10 @@ This is an MCP (Model Context Protocol) server for Contrast Security that enable
 
 ### Building the Project
 - **Build**: `mvn clean install` or `./mvnw clean install`
+- **Package without tests**: `mvn clean package -DskipTests`
 - **Test**: `mvn test` or `./mvnw test`
-- **Run locally**: `java -jar target/mcp-contrast-0.0.11.jar --CONTRAST_HOST_NAME=<host> --CONTRAST_API_KEY=<key> --CONTRAST_SERVICE_KEY=<key> --CONTRAST_USERNAME=<user> --CONTRAST_ORG_ID=<org>`
+- **Run single test**: `mvn test -Dtest=HintGeneratorTest` or `mvn test -Dtest=HintGeneratorTest#specificTestMethod`
+- **Run locally**: `java -jar target/mcp-contrast-0.0.12-SNAPSHOT.jar --CONTRAST_HOST_NAME=<host> --CONTRAST_API_KEY=<key> --CONTRAST_SERVICE_KEY=<key> --CONTRAST_USERNAME=<user> --CONTRAST_ORG_ID=<org>`
 
 ### Docker Commands
 - **Build Docker image**: `docker build -t mcp-contrast .`
@@ -28,17 +32,19 @@ This is an MCP (Model Context Protocol) server for Contrast Security that enable
 
 **Main Application**: `McpContrastApplication.java` - Spring Boot application that registers MCP tools from all service classes.
 
-**Service Layer**: Each service handles a specific aspect of Contrast Security data:
+**Service Layer**: Each service handles a specific aspect of Contrast Security data and exposes `@Tool` annotated methods:
 - `AssessService` - Vulnerability analysis and trace data
 - `SastService` - Static application security testing data
 - `SCAService` - Software composition analysis (library vulnerabilities)
 - `ADRService` - Attack detection and response events
 - `RouteCoverageService` - Route coverage analysis
-- `PromptService` - AI prompt management
 
-**SDK Extensions**: Located in `sdkexstension/` package, these extend the Contrast SDK with enhanced data models and helper methods for better AI integration.
+**SDK Extensions**: Located in `sdkexstension/` package:
+- `SDKExtension.java` - Extends Contrast SDK API with additional endpoints not in standard SDK (library observations, route details, session metadata, etc.)
+- `SDKHelper.java` - Utility methods for hostname protocol handling and common operations
+- `data/` subpackages - Enhanced data models with AI-friendly representations organized by domain (application, adr, routecoverage, sca, traces, sessionmetadata)
 
-**Data Models**: Comprehensive POJOs in `data/` package representing vulnerability information, library data, applications, and attack events.
+**Data Models**: Comprehensive POJOs in `data/` package representing vulnerability information, library data, applications, and attack events used by service layer.
 
 **Hint System**: `hints/` package provides context-aware security guidance for vulnerability remediation.
 
@@ -58,19 +64,24 @@ Required environment variables/arguments:
 
 ### Technology Stack
 
-- **Framework**: Spring Boot 3.4.5 with Spring AI 1.0.0-RC1
+- **Framework**: Spring Boot 3.4.5 with Spring AI 1.0.1
 - **MCP Integration**: Spring AI MCP Server starter
 - **Contrast Integration**: Contrast SDK Java 3.4.2
-- **Testing**: JUnit 5
-- **Build Tool**: Maven with wrapper
+- **JSON Processing**: Gson (via Contrast SDK)
+- **Testing**: JUnit 5 with Spring Boot Test
+- **Build Tool**: Maven 3.6+ with wrapper
 - **Packaging**: Executable JAR and Docker container
 
 ### Development Patterns
 
-1. **MCP Tools**: Services expose methods via `@Tool` annotation for AI agent consumption
-2. **SDK Extension Pattern**: Enhanced data models extend base SDK classes with AI-friendly representations
-3. **Hint Generation**: Rule-based system provides contextual security guidance
-4. **Defensive Design**: All external API calls include error handling and logging
+1. **MCP Tools**: Services expose methods via `@Tool` annotation for AI agent consumption. Register new services in `McpContrastApplication.tools()` bean.
+2. **SDK Extension Pattern**:
+   - Use `SDKExtension` to add new Contrast API endpoints not in the standard SDK
+   - Use `SDKHelper` for common utility operations (hostname handling, etc.)
+   - Enhanced data models in `sdkexstension/data/` provide AI-friendly JSON representations
+3. **Hint Generation**: Rule-based system in `hints/` package provides contextual security guidance for vulnerability remediation
+4. **Defensive Design**: All external API calls include proper resource management (try-with-resources), error handling, and logging
+5. **Pagination Handling**: SDK extension methods handle pagination automatically (see `getLibraryObservations` for pattern)
 
 ### Security Considerations
 
@@ -81,30 +92,20 @@ This codebase handles sensitive vulnerability data. The README contains critical
 - Default log location: `/tmp/mcp-contrast.log`
 - Debug logging: Add `--logging.level.root=DEBUG` to startup arguments
 - Console logging is minimal by design for MCP protocol compatibility
+- Debug mode buffers API responses for logging (memory impact with large datasets)
 
-## Beads Workflow Requirements
-
-This project uses Beads (bd) for issue tracking. See the MCP resource `beads://quickstart` for usage details.
-
-### Managing Bead Dependencies
-
-**Command syntax:** `bd dep add <dependent-task> <prerequisite-task>`
-
-Example: If B must be done after A completes, use `bd dep add B A` (not `bd dep add A B`).
-
-Verify with `bd show <task-id>` - dependent tasks show "Depends on", prerequisites show "Blocks".
-
-### Testing Requirements Before Closing Beads
+### Troubleshooting
 
-**CRITICAL: Before closing any bead, you MUST:**
+For common issues (SSL certificates, proxy configuration, debug logging), see the "Common Issues" and "Proxy Configuration" sections in [README.md](README.md).
 
-1. **Write tests for ALL code changes** - No exceptions
-2. **Run unit tests** - `mvn test` must pass with 0 failures
-3. **Run integration tests** - `mvn verify` must pass (requires credentials in `.env.integration-test`)
-   - If credentials unavailable, verify integration tests pass in CI/CD
-4. **Verify new tests are included** - Ensure your tests ran and passed
+### Adding New MCP Tools
 
-All code changes require corresponding test coverage. Do not close beads without tests.
+To add a new tool/service:
+1. Create a new `@Service` class with methods annotated with `@Tool(description="...")`
+2. Inject dependencies (ContrastSDK, SDKExtension) via constructor
+3. Register the service in `McpContrastApplication.tools()` bean method
+4. Use `SDKExtension` to add new API endpoints if needed
+5. Create enhanced data models in appropriate `sdkexstension/data/` subpackage
 
 See INTEGRATION_TESTS.md for integration test setup and credentials.
 
@@ -122,4 +123,4 @@ This project is tracked in Jira under the **AIML** project. When creating Jira t
   - `Task` - for simple non-feature changes (refactoring, documentation, bug fixes)
   - `Epic` - for large features with many dependent tasks (typically managed by Product Management)
 
-**Access**: Use the Atlassian MCP server to read or write Jira tickets programmatically.
+**Access**: Use the Atlassian MCP server to read or write Jira tickets programmatically.