Add regulatory compliance evaluation pipeline cookbook

[garyatwalAI]

Summary

This PR adds a comprehensive Jupyter notebook cookbook demonstrating how to build an automated compliance evaluation pipeline using Langfuse for regulated industries (legal, financial, insurance, healthcare).

Key Changes

• New cookbook notebook (cookbook/regulatory_compliance_evaluation_pipeline.ipynb) with a complete compliance evaluation system featuring:
  • ComplianceScore and ComplianceReport dataclasses for structured compliance results
  • PIIDetector: Rule-based evaluator detecting emails, phone numbers, SSNs, credit cards, passports, and other PII patterns
  • LegalPrivilegeDetector: Hybrid rule-based + LLM-as-a-Judge evaluator identifying attorney-client privilege, work product, settlement communications, litigation strategy, and regulatory investigation privilege
  • FactualGroundingEvaluator: LLM-as-a-Judge evaluator checking if outputs are grounded in provided context (hallucination detection)
  • BiasEvaluator: LLM-as-a-Judge evaluator detecting discriminatory or biased language across protected characteristics
  • CompliancePipeline: Orchestrator class that runs all evaluators and ingests scores back into Langfuse

Notable Implementation Details

• Hybrid evaluation approach combining fast rule-based checks with nuanced LLM-based assessments
• Two-stage legal privilege detection: keyword scan followed by LLM classification for high-confidence results
• Structured compliance reporting with PASS/WARN/FAIL categories and detailed flagged items
• Seamless integration with Langfuse for score ingestion and dashboard visualization
• Extensible architecture allowing easy addition of domain-specific evaluators
• Alignment with NIST AI Risk Management Framework (GOVERN, MAP, MEASURE, MANAGE functions)
• Practical examples for litigation support, insurance claims, financial advisory, and healthcare use cases

https://claude.ai/code/session_011De5LfdoirAg729crpDFaF

[vercel]

@claude is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB