diff --git a/.gitignore b/.gitignore index 4c8e967..e6e7472 100644 --- a/.gitignore +++ b/.gitignore @@ -15,3 +15,4 @@ .byebug_history Gemfile.lock .DS_Store +.claude/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..ea4182d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,96 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +relaton-cli is a Ruby CLI tool for managing bibliographic references to standards (ISO, IEC, IETF, NIST, etc.). It provides commands to fetch, convert, and organize standards metadata in XML, YAML, BibTeX, and HTML formats. Part of the broader Relaton/Metanorma ecosystem. + +## Common Commands + +```bash +# Install dependencies +bundle install + +# Run all tests +bundle exec rspec + +# Run a single test file +bundle exec rspec spec/relaton/cli/command_spec.rb + +# Run a specific test by line number +bundle exec rspec spec/relaton/cli/command_spec.rb:42 + +# Build the gem +bundle exec rake build + +# Lint (RuboCop, inherits from Ribose OSS guide) +bundle exec rubocop +bundle exec rubocop -a # auto-fix +``` + +## Architecture + +### Entry Point & CLI Framework + +The executable `exe/relaton` calls `Relaton::Cli.start(ARGV)` which routes to `Relaton::Cli::Command`, a Thor-based command class. Thor handles argument parsing, option definitions, and subcommand routing. + +### Command Structure + +- `lib/relaton/cli/command.rb` — Main Thor command class with top-level commands (fetch, extract, concatenate, split, yaml2xml, xml2yaml, xml2html, yaml2html, convert, fetch-data) +- `lib/relaton/cli/subcommand_collection.rb` — `relaton collection` subcommands (create, info, list, get, find, fetch, import, export) +- `lib/relaton/cli/subcommand_db.rb` — `relaton db` subcommands (create, mv, clear, fetch, fetch_all, doctype) + +### Option Forwarding Pattern + +`Command#fetch` and `SubcommandDb#fetch` use the shared `fetch_document` helper (in `Relaton::Cli` private methods at the bottom of `command.rb`). This helper transforms Thor's kebab-case option keys to snake_case symbols via `gsub("-", "_").to_sym` and splats them as `**dup_opts` to `Relaton.db.fetch` / `Relaton.db.fetch_std`. Adding a new Thor option to these commands automatically forwards it to the underlying library with no method changes needed. + +`SubcommandCollection#fetch` calls `Relaton.db.fetch` directly (not through `fetch_document`), so new options must be explicitly forwarded there. + +Current fetch options that use this pattern: `--no-cache`, `--all-parts`, `--keep-year`, `--publication-date-before`, `--publication-date-after`. + +### Core Data Classes + +- `lib/relaton/bibdata.rb` — `Relaton::Bibdata` wraps `RelatonBib::BibliographicItem`, adding URL type handling and serialization to XML/YAML/Hash. Uses `method_missing` to delegate to the underlying bibitem. +- `lib/relaton/bibcollection.rb` — `Relaton::Bibcollection` represents a collection of bibliographic items with title/author/doctype metadata. Handles XML/YAML round-tripping. +- `lib/relaton/element_finder.rb` — Mixin providing XPath utilities with namespace handling. + +### Converters (Template Method Pattern) + +- `lib/relaton/cli/base_convertor.rb` — Abstract base defining the conversion flow (convert_and_write, write_to_file_collection) +- `lib/relaton/cli/xml_convertor.rb` — XML → YAML conversion +- `lib/relaton/cli/yaml_convertor.rb` — YAML → XML conversion (includes processor detection via doctype) +- `lib/relaton/cli/xml_to_html_renderer.rb` — Renders XML/YAML to HTML using Liquid templates from `templates/` + +### File Operations + +`lib/relaton/cli/relaton_file.rb` — Static methods for extract (pull bibdata from Metanorma XML), concatenate (combine files into a collection), and split (break a collection into individual files). + +### Database (Singleton) + +`Relaton::Cli::RelatonDb` (in `lib/relaton/cli.rb`) is a Singleton managing a `Relaton::Db` instance. DB path is persisted in `~/.relaton/dbpath`. The `relaton` gem's registry auto-discovers 30+ standard-body processors. + +### Processor Detection + +`Relaton::Cli.processor(doc)` and `.parse_xml(doc)` detect the correct processor (ISO, IEC, IETF, etc.) from a document's `docidentifier` element type attribute, falling back to prefix matching. + +## Test Structure + +- `spec/acceptance/` — End-to-end CLI integration tests using `rspec-command` +- `spec/relaton/cli/` — Unit tests for command, converters, subcommands, DB +- `spec/relaton/` — Unit tests for Bibcollection and Bibdata +- `spec/support/` — Test setup: SimpleCov, WebMock, VCR, equivalent-xml matchers +- `spec/fixtures/` and `spec/vcr_cassettes/` — Test data and recorded HTTP responses + +Tests use VCR cassettes to replay HTTP interactions with standards registries. WebMock blocks real HTTP requests in tests. + +## Key Dependencies + +- `relaton ~> 1.20.0` — Core library providing DB, registry, and all standard-body processors +- `thor` / `thor-hollaback` — CLI framework +- `liquid ~> 5` — HTML template rendering +- `nokogiri` (transitive via relaton) — XML parsing + +## Ruby Version + +Requires Ruby >= 3.0.0 (set in gemspec and `.rubocop.yml`). diff --git a/docs/README.adoc b/docs/README.adoc index 344fe02..b7004e6 100644 --- a/docs/README.adoc +++ b/docs/README.adoc @@ -42,7 +42,7 @@ suports an additional `-x` or `--extension` options to use different extension. [source,console] ---- -$ relaton fetch CODE -t TYPE -f FORMAT -y YEAR -r RETRIES --all-parts --keep-year --no-cache +$ relaton fetch CODE -t TYPE -f FORMAT -y YEAR -r RETRIES --all-parts --keep-year --no-cache --publication-date-before DATE --publication-date-after DATE ---- Fetch the Relaton XML entry corresponding to the document identifier `CODE`. @@ -54,6 +54,8 @@ Fetch the Relaton XML entry corresponding to the document identifier `CODE`. * `--all-parts` fetch all parts. * `--keep-year` undated reference should return an actual reference with year. * `--no-cache` do not use cache. +* `--publication-date-before DATE` fetch only documents published before the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`. +* `--publication-date-after DATE` fetch only documents published after the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`. === relaton fetch-data @@ -358,7 +360,7 @@ Full-text search through a collection or all collections. ==== relaton collection fetch ---- -$ relaton collection fetch CODE -t TYPE -y YEAR -c COLLECTION -d DIRECTORY +$ relaton collection fetch CODE -t TYPE -y YEAR -c COLLECTION -d DIRECTORY --publication-date-before DATE --publication-date-after DATE ---- Fetch the Relaton XML entry corresponding to the document identifier `CODE` and save it into `COLLECTION`. @@ -367,6 +369,8 @@ Fetch the Relaton XML entry corresponding to the document identifier `CODE` and * `YEAR` is optional, and specifies the year of publication of the standard. * `COLLECTION` - a name of a collection. * `DIRECTORY` - optional, and specifies a path to a directory with collections. The default value is `$HOME/.relaton/collections`. +* `--publication-date-before DATE` fetch only documents published before the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`. +* `--publication-date-after DATE` fetch only documents published after the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`. ==== relaton collection export diff --git a/lib/relaton/cli/command.rb b/lib/relaton/cli/command.rb index 9381c5b..f89098d 100644 --- a/lib/relaton/cli/command.rb +++ b/lib/relaton/cli/command.rb @@ -1,3 +1,4 @@ +require "date" require "relaton/cli/relaton_file" require "relaton/cli/xml_convertor" require "relaton/cli/yaml_convertor" @@ -31,6 +32,8 @@ def version option :retries, aliases: :r, type: :numeric, desc: "Number of network retries. Default 1." option :"no-cache", type: :boolean, desc: "Ignore cache" + option :"publication-date-before", desc: "Fetch only documents published before the specified date (e.g. 2008, 2008-02, or 2008-02-02)" + option :"publication-date-after", desc: "Fetch only documents published after the specified date (e.g. 2002, 2002-01, or 2002-01-01)" def fetch(code) io = IO.new($stdout.fcntl(::Fcntl::F_DUPFD), mode: "w:UTF-8") @@ -174,6 +177,30 @@ def relaton_config private + DATE_FILTER_FORMAT = /\A\d{4}(-\d{2}(-\d{2})?)?\z/ + + def parse_date_option(value, name) + return unless value + + unless value.match?(DATE_FILTER_FORMAT) + raise ArgumentError, + "Invalid #{name}: #{value.inspect}. Expected YYYY, YYYY-MM, or YYYY-MM-DD." + end + parts = value.split("-").map(&:to_i) + Date.new(*parts.concat([1] * (3 - parts.size))) + rescue Date::Error + raise ArgumentError, + "Invalid #{name}: #{value.inspect}. Date components are out of range." + end + + def validate_date_range(date_after, date_before) + return unless date_after && date_before + return if date_after < date_before + + raise ArgumentError, + "Invalid date range: --publication-date-after (#{date_after}) must be before --publication-date-before (#{date_before})." + end + # @param code [String] # @param options [Hash] # @option options [String] :type @@ -183,6 +210,10 @@ def relaton_config def fetch_document(code, options) # rubocop:disable Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity,Metrics/AbcSize,Metrics/MethodLength year = options[:year]&.to_s dup_opts = options.dup.transform_keys { |k| k.to_s.gsub("-", "_").to_sym } + %i[publication_date_before publication_date_after].each do |key| + dup_opts[key] = parse_date_option(dup_opts[key], key.to_s.tr("_", "-").prepend("--")) if dup_opts[key] + end + validate_date_range dup_opts[:publication_date_after], dup_opts[:publication_date_before] if (processor = Relaton::Registry.instance.by_type options[:type]&.upcase) doc = Relaton.db.fetch_std code, year, processor.short, **dup_opts elsif options[:type] then return diff --git a/lib/relaton/cli/subcommand_collection.rb b/lib/relaton/cli/subcommand_collection.rb index 758a199..bae7ec3 100644 --- a/lib/relaton/cli/subcommand_collection.rb +++ b/lib/relaton/cli/subcommand_collection.rb @@ -102,13 +102,23 @@ def find(text) desc: "Type of standard to get bibliographic entry for" option :year, aliases: :y, type: :numeric, desc: "Year the standard was published" + option :"publication-date-before", desc: "Fetch only documents published before the specified date (e.g. 2008, 2008-02, or 2008-02-02)" + option :"publication-date-after", desc: "Fetch only documents published after the specified date (e.g. 2002, 2002-01, or 2002-01-01)" option :collection, aliases: :c, required: true, desc: "Collection to store a document" option :dir, aliases: :d, desc: "Directory with collections. Default is " \ "$HOME/.relaton/collections." def fetch(code) # rubocop:disable Metrics/AbcSize - doc = Relaton.db.fetch(code, options[:year]&.to_s) + opts = {} + if options[:"publication-date-before"] + opts[:publication_date_before] = parse_date_option(options[:"publication-date-before"], "--publication-date-before") + end + if options[:"publication-date-after"] + opts[:publication_date_after] = parse_date_option(options[:"publication-date-after"], "--publication-date-after") + end + validate_date_range opts[:publication_date_after], opts[:publication_date_before] + doc = Relaton.db.fetch(code, options[:year]&.to_s, **opts) if doc colfile = File.join directory, options[:collection] coll = read_collection colfile diff --git a/lib/relaton/cli/subcommand_db.rb b/lib/relaton/cli/subcommand_db.rb index 0d1722e..fd7d231 100644 --- a/lib/relaton/cli/subcommand_db.rb +++ b/lib/relaton/cli/subcommand_db.rb @@ -40,6 +40,8 @@ def clear "Default xml." option :year, aliases: :y, type: :numeric, desc: "Year the standard " \ "was published" + option :"publication-date-before", desc: "Fetch only documents published before the specified date (e.g. 2008, 2008-02, or 2008-02-02)" + option :"publication-date-after", desc: "Fetch only documents published after the specified date (e.g. 2002, 2002-01, or 2002-01-01)" def fetch(code) io = IO.new($stdout.fcntl(::Fcntl::F_DUPFD), mode: "w:UTF-8") diff --git a/spec/acceptance/relaton_fetch_spec.rb b/spec/acceptance/relaton_fetch_spec.rb index 9435b68..ca3027c 100644 --- a/spec/acceptance/relaton_fetch_spec.rb +++ b/spec/acceptance/relaton_fetch_spec.rb @@ -31,6 +31,66 @@ command = ["fetch", "--no-cache", "ISO 2146"] Relaton::Cli.start(command) end + + it "calls fetch with publication date options" do + expect(db).to receive(:fetch).with( + "ISO 2146", nil, + publication_date_before: Date.new(2008, 1, 1), + publication_date_after: Date.new(2002, 1, 1) + ) + + command = ["fetch", "--publication-date-before", "2008", + "--publication-date-after", "2002", "ISO 2146"] + Relaton::Cli.start(command) + end + + it "calls fetch with YYYY-MM publication date options" do + expect(db).to receive(:fetch).with( + "ISO 2146", nil, + publication_date_before: Date.new(2008, 6, 1), + publication_date_after: Date.new(2002, 3, 1) + ) + + command = ["fetch", "--publication-date-before", "2008-06", + "--publication-date-after", "2002-03", "ISO 2146"] + Relaton::Cli.start(command) + end + + it "calls fetch with YYYY-MM-DD publication date options" do + expect(db).to receive(:fetch).with( + "ISO 2146", nil, + publication_date_before: Date.new(2008, 6, 15), + publication_date_after: Date.new(2002, 3, 10) + ) + + command = ["fetch", "--publication-date-before", "2008-06-15", + "--publication-date-after", "2002-03-10", "ISO 2146"] + Relaton::Cli.start(command) + end + end + + context "publication date validation" do + it "rejects invalid publication date format" do + command = ["fetch", "--publication-date-before", "not-a-date", "ISO 2146"] + expect { Relaton::Cli.start(command) }.to raise_error( + ArgumentError, /Invalid --publication-date-before.*Expected YYYY/ + ) + end + + it "rejects out-of-range date components" do + command = ["fetch", "--publication-date-after", "2008-13-01", "ISO 2146"] + expect { Relaton::Cli.start(command) }.to raise_error( + ArgumentError, /Invalid --publication-date-after.*out of range/ + ) + end + + it "rejects date-after >= date-before" do + command = ["fetch", "--publication-date-after", "2010", + "--publication-date-before", "2008", "ISO 2146"] + expect { Relaton::Cli.start(command) }.to raise_error( + ArgumentError, /Invalid date range/ + ) + end end context do