Add optional DynamoDB backend for stats

[MariusWirtz]

• Updated README and settings reference to include DynamoDB options.
• Modified settings to support both SQLite and DynamoDB for stats storage.
• Implemented DynamoDBStatsDatabase for handling stats in DynamoDB.
• Enhanced CLI and commands to accommodate the new backend.
• Added unit tests for DynamoDB settings and backend validation.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[nicolasbisurgi]

PR Review Notes

All good on my end! A couple of notes to keep them for future work

Shared Interface / Protocol

StatsDatabase and DynamoDBStatsDatabase are independent classes with no shared base class or Protocol. They happen to have overlapping method names, but nothing enforces they stay in sync. If a method is added or its signature changes in one, the other could silently drift.

Future improvement: When we move SQLite to SQLAlchemy, that would be a natural time to introduce a StatsBackend Protocol (or ABC) that all backends implement. This would catch missing/mismatched methods at type-check time rather than runtime.

Dual Dispatch in db_admin.py

Functions like list_runs, list_tasks, and get_visualization_data use isinstance(backend, str) to branch between SQLite (raw SQL) and DynamoDB (method calls). This creates parallel code paths that need to be maintained in lockstep. The backend: Any type annotations also lose type safety.

Future improvement: Once a shared interface exists, these functions should go through the backend methods for both SQLite and DynamoDB, eliminating the branching. The SQLAlchemy migration would be the right time to address this.

Minor Inconsistency

Some call sites in commands.py use get_stats_backend(settings) to normalize the backend string (lowercasing), while others pass settings.stats.backend directly. Not a bug (the factory normalizes internally), but worth making consistent.