ML-based row-level anomaly detection

[vb-dbrks]

Changes

This PR adds ML-based row anomaly detection to automatically find unusual rows in data (per‑record anomalies with explanations) without manually specifying thresholds so you can catch issues that rule-based checks miss. You provide recent good data. DQX trains a model and flags rows that don't fit typical patterns.

Key features:

• Auto‑discovery of columns and segmentation where appropriate
• Isolation Forest training with Spark scoring
• Explainability via SHAP-based feature contributions
• Unity Catalog / MLflow integration for model storage and versioning
• New check function: has_no_anomalies() (percentile‑based severity)
• Production defaults: severity threshold 95, contributions enabled by default, ensemble support

What’s included:

• New AnomalyEngine for training models
• Feature engineering for numeric, categorical, datetime, and boolean columns
• Model registry + drift detection metadata
• Demo notebook updates
• Documentation updates (guide + reference + install)
• Test improvements (unit + integration)

Complements Data Quality Monitoring which focuses on completeness and freshness.

Resolves #957

Tests

• manually tested (ran demos on Databricks)
• added unit tests (new tests for exact distinct + expected_anomaly_rate behavior)
• added integration tests (training, scoring, segmentation, explainability, drift)
• added end-to-end tests
• added performance tests

[github-actions]

✅ 593/593 passed, 36 skipped, 6h27m19s total

_{Running from acceptance #3996}