Skip to content

Commit c27d563

Browse files
btiernaybtiernay
committed
fix: make more robust
1 parent 87acc83 commit c27d563

File tree

5 files changed

+151
-174
lines changed

5 files changed

+151
-174
lines changed

‎.github/workflows/link-check.yml‎

Lines changed: 73 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,94 +1,92 @@
1+
# .github/workflows/link-check.yml
12
name: Link Checker
23

34
# Runs Lychee (link checker) on Markdown / MDX docs.
4-
# Lychee repo: https://github.com/lycheeverse/lychee
5-
# GitHub Action: https://github.com/lycheeverse/lychee-action
6-
# Config reference: https://github.com/lycheeverse/lychee#configuration
75
#
86
# What this workflow does:
9-
# - Triggers on pushes to main, pull requests, and manual workflow dispatch
10-
# - Scans all *.md and *.mdx files for broken links
11-
# - Uses .lycheeignore in the repo root to ignore known noisy URLs
12-
# - Accepts 302 redirects as valid (common for auth-required pages)
13-
# - Currently warns on failures to allow tuning before enforcing
14-
#
15-
# Manual trigger:
16-
# - Go to Actions tab → Select "Link Checker" → Click "Run workflow"
17-
# - Specify custom base URL (e.g., Mintlify preview deployment)
18-
#
19-
# To make blocking: Remove the final "Warn on failures" step
7+
# - Triggers on:
8+
# - pushes to main
9+
# - all pull requests
10+
# - manual runs via "Run workflow"
11+
# - Scans auth4genai/*.md(x) for broken HTTPS links
12+
# - Respects .lycheeignore in the repo root
13+
# - Accepts 302 redirects and 429 rate limits as valid
14+
# - Currently warns on failures instead of failing CI
2015

2116
on:
2217
push:
2318
branches:
24-
- main
19+
- main
2520
pull_request:
26-
workflow_dispatch:
27-
inputs:
28-
base_url:
29-
description: 'Base URL for resolving absolute paths (e.g., https://auth0.com or https://your-preview.mintlify.app)'
30-
required: false
31-
default: 'https://auth0.com'
32-
type: string
21+
workflow_dispatch: {}
3322

3423
jobs:
3524
link-check:
36-
name: Check external links
25+
name: Check external links in auth4genai docs
3726
runs-on: ubuntu-latest
3827

3928
steps:
40-
- name: Checkout repo
41-
uses: actions/checkout@v4
42-
43-
# If your docs are generated (Next.js / Docusaurus / etc), you can build them here
44-
# and point Lychee at the output instead. Examples in the action docs:
45-
# https://github.com/lycheeverse/lychee-action#usage
46-
# - name: Build docs
47-
# run: |
48-
# npm ci
49-
# npm run docs:build
29+
# Fetch repository contents so Lychee can scan Markdown / MDX files
30+
- name: Checkout repo
31+
uses: actions/checkout@v4
5032

51-
# TODO: Re-enable main docs checking later
52-
# - name: Check main docs links
53-
# id: lychee-main
54-
# uses: lycheeverse/lychee-action@v2
55-
# env:
56-
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
57-
# with:
58-
# args: >
59-
# --verbose
60-
# --no-progress
61-
# --base-url ${{ inputs.base_url || 'https://auth0.com' }}
62-
# --accept 200..=299,302,429
63-
# --exclude-path '../LICENSE'
64-
# main/**/*.md
65-
# main/**/*.mdx
33+
# Run Lychee on the Auth0 for AI Agents Mintlify content
34+
#
35+
# Notes on args:
36+
# - --scheme https
37+
# Only check HTTPS links. Other schemes (mailto:, file:, etc) are skipped.
38+
# - --accept 200..=299,302,429
39+
# Treat 2xx, 302, and 429 as success. 302 is common for docs redirects,
40+
# 429 is rate limiting that we do not want to treat as "broken".
41+
# - --exclude-path '.vale'
42+
# Skip the .vale directory used for prose linting.
43+
# - auth4genai/**/*.md / .mdx
44+
# Scope to the Mintlify site for Auth0 for AI Agents.
45+
#
46+
# Internal routing, API-only endpoints, and other noisy URLs are
47+
# filtered via .lycheeignore in the repo root.
48+
- name: Run Lychee on auth4genai docs
49+
id: lychee-auth4genai
50+
uses: lycheeverse/lychee-action@v2
51+
env:
52+
# Used by Lychee for authenticated GitHub requests to reduce rate limiting.
53+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
54+
with:
55+
args: >
56+
--verbose
57+
--no-progress
58+
--scheme https
59+
--accept 200..=299,302,429
60+
--exclude-path '.vale'
61+
auth4genai/**/*.md
62+
auth4genai/**/*.mdx
6663
67-
- name: Check auth4genai docs links
68-
id: lychee-auth4genai
69-
uses: lycheeverse/lychee-action@v2
70-
env:
71-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
72-
with:
73-
args: >
74-
--verbose
75-
--no-progress
76-
--base-url https://auth0.com/ai/docs
77-
--remap 'https://auth0.com/(?!docs/|ai/) https://auth0.com/ai/docs/'
78-
--accept 200..=299,302,429
79-
--exclude-path '.vale'
80-
auth4genai/**/*.md
81-
auth4genai/**/*.mdx
82-
# Auth4GenAI docs: Remap auth0.com/* paths to auth0.com/ai/docs/*
83-
# (except paths already starting with /docs/ or /ai/)
84-
# Accept 429 to handle rate limiting on auth0.com
64+
# Optional: example of how you might later add main docs checking.
65+
# Uncomment and adjust when you want to bring main/ back into scope.
66+
#
67+
# - name: Run Lychee on main docs
68+
# id: lychee-main
69+
# uses: lycheeverse/lychee-action@v2
70+
# env:
71+
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
72+
# with:
73+
# args: >
74+
# --verbose
75+
# --no-progress
76+
# --scheme https
77+
# --accept 200..=299,302,429
78+
# main/**/*.md
79+
# main/**/*.mdx
8580

86-
# For now this only warns so it does not block merges while you tune ignores.
87-
# Once the noise is under control, delete this step so Lychee failures break CI.
88-
- name: Warn on failures instead of failing CI
89-
if: steps.lychee-auth4genai.outputs.exit_code != 0
90-
run: |
91-
echo "Lychee found broken links."
92-
echo "Auth4GenAI docs exit code: ${{ steps.lychee-auth4genai.outputs.exit_code }}"
93-
echo "Check the Lychee step logs above for details."
94-
exit 0
81+
# For now, only warn on failures so you can tune .lycheeignore and fix
82+
# obvious issues without blocking merges.
83+
#
84+
# Once you are happy with the signal/noise ratio, delete this step so that
85+
# Lychee's non-zero exit code will fail the job.
86+
- name: Warn on failures instead of failing CI
87+
if: steps.lychee-auth4genai.outputs.exit_code != 0
88+
run: |
89+
echo "Lychee found broken links in auth4genai docs."
90+
echo "Auth4GenAI docs exit code: ${{ steps.lychee-auth4genai.outputs.exit_code }}"
91+
echo "Check the Lychee step logs above for details."
92+
exit 0

‎.lycheeignore‎

Lines changed: 70 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,100 @@
1+
# .lycheeignore
2+
13
# Global ignores for Lychee link checker
24
# Docs: https://github.com/lycheeverse/lychee#ignoring-links
3-
4-
# All npmjs links (these often 403 or block bots, which is noisy in CI)
5+
#
6+
# Philosophy:
7+
# - Only ignore URLs that are inherently uncheckable (require auth, localhost, API-only)
8+
# or consistently noisy (bot blocking, weird 4xx).
9+
# - Do NOT blanket-ignore normal public docs sites so we can still catch real regressions.
10+
11+
#
12+
# Mintlify internal routes and static assets (root-relative paths)
13+
# These only become real URLs after Mintlify builds the site.
14+
# Examples:
15+
# /intro/user-authentication
16+
# /how-tos/check-google-calendar-availability
17+
# /img/intro/universal_login.png
18+
#
19+
^/
20+
21+
#
22+
# Third-party sites that aggressively block or rate-limit bots
23+
#
24+
25+
# All npmjs links (often 403 / bot-blocking)
526
^https?://(www\.)?npmjs\.com/.*
627

7-
# Auth0 Dashboard and Manage (require authentication, rate limited)
28+
# LinkedIn (often 999 / 403 to automated clients)
29+
^https?://(www\.)?linkedin\.com/
30+
31+
# Cloudflare support (bot-blocking)
32+
^https://support\.cloudflare\.com/
33+
34+
#
35+
# Auth0 properties that require authentication or are internal
36+
#
37+
38+
# Auth0 Dashboard and Manage (requires interactive login)
839
^https://manage\.auth0\.com/
940

10-
# Auth0 Support site (returns 302, requires auth for some pages)
41+
# Auth0 Support (some pages gated / 302 to login)
1142
^https://support\.auth0\.com/
1243

13-
# Auth0 Community forum (rate limited, returns 404 for old posts)
44+
# Auth0 Community (older posts can 404; rate-limited)
1445
^https://community\.auth0\.com/
1546

16-
# LinkedIn (often returns 999 or 403 to automated checkers)
17-
^https?://(www\.)?linkedin\.com/
18-
19-
# Atlassian/Confluence internal wiki (requires authentication)
20-
^https://oktawiki\.atlassian\.net/
47+
# Test/example URLs in component demos
48+
^https://auth0\.com/bar
2149

22-
# Cloudflare support (blocks bots with 403)
23-
^https://support\.cloudflare\.com/
50+
#
51+
# Internal / company-only resources
52+
#
2453

25-
# GitHub (rate limits aggressively without token)
26-
# Note: If you have GITHUB_TOKEN set, you can remove this exclusion
27-
^https://github\.com/
28-
^https://www\.github\.com/
54+
# Internal Oktawiki (requires Okta auth)
55+
^https://oktawiki\.atlassian\.net/
2956

3057
# FGA Dashboard (requires authentication)
3158
^https://dashboard\.fga\.dev/
3259

33-
# OpenAI Platform (requires authentication, blocks bots)
60+
#
61+
# API-only / non-browsable endpoints
62+
#
63+
64+
# OpenAI Platform (requires login / bot protection)
3465
^https://platform\.openai\.com/
3566

36-
# Google Cloud Console (requires authentication)
67+
# Google Cloud Console & APIs dashboard (requires login)
3768
^https://console\.cloud\.google\.com/
3869
^https://console\.developers\.google\.com/
3970

40-
# Microsoft Graph API endpoints (not browsable URLs)
71+
# Microsoft Graph API endpoints (not meant for direct browser navigation)
4172
^https://graph\.microsoft\.com/
4273

43-
# Snapchat auth endpoints (not browsable URLs)
74+
# Snapchat auth endpoints (API-only)
4475
^https://auth\.snapchat\.com/
4576

46-
# Test/example URLs in component demos
47-
^https://auth0\.com/bar
77+
#
78+
# GitHub noise control
79+
#
80+
# With GITHUB_TOKEN set in the workflow, GitHub links are usually fine.
81+
# If they ever become noisy, uncomment these to silence them.
82+
# ^https://github\.com/
83+
# ^https://www\.github\.com/
4884

49-
# Invalid/malformed URLs in docs
50-
^https://sdk\.vercel\./
51-
^http://next\.js/
85+
#
86+
# Local dev and fake example hosts
87+
#
5288

53-
# Local dev and example hosts from docs (don't check when not running)
89+
# Localhost variants – never valid in CI
5490
^http://localhost(:\d+)?/
5591
^https?://localhost(:\d+)?/
92+
93+
# Example API host used in docs
5694
^http://acme-api/
95+
96+
#
97+
# Known broken URLs that should be fixed in the docs (do NOT ignore)
98+
# These are listed here as reminders only. They are intentionally
99+
# commented out so Lychee still fails on them until content is fixed.
100+
#

‎auth4genai/README.md‎

Lines changed: 6 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -80,83 +80,18 @@ We use [Vale](https://vale.sh/) to keep terminology and brand usage consistent a
8080

8181
We use [Lychee](https://lychee.cli.rs/) to check for broken links in the documentation.
8282

83-
### Automated Checks
83+
Lychee runs in CI on pull requests and on pushes to `main`. It checks external HTTPS links only and uses `.lycheeignore` to exclude URLs that require authentication, block automated traffic, or represent API endpoints. Internal Mintlify routes (for example `/intro/...`, `/mcp/...`, `/img/...`) are not checked by Lychee because the Mintlify CLI already validates routing during `mint dev`.
8484

85-
The link checker runs automatically on:
86-
- **Pull requests** - Checks all links against production (`https://auth0.com`)
87-
- **Pushes to main** - Validates links in production documentation
88-
89-
### Manual Checks
90-
91-
You can manually trigger the link checker workflow with a custom base URL:
92-
93-
1. Go to **Actions** tab in GitHub
94-
2. Select **"Link Checker"** workflow
95-
3. Click **"Run workflow"**
96-
4. Enter your base URL:
97-
- Production: `https://auth0.com` (default)
98-
- Mintlify preview: `https://your-branch-name.mintlify.app`
99-
100-
### Local Link Checking
101-
102-
To check links locally while developing:
103-
104-
#### Installation
105-
106-
```bash
107-
# macOS
108-
brew install lychee
109-
110-
# Linux (various options)
111-
# See https://lychee.cli.rs/#installation for other platforms
112-
```
113-
114-
#### Check MCP Documentation Locally
115-
116-
From the `auth4genai/` directory with `mint dev` running:
85+
Local usage of Lychee is optional. If you want to test external links manually, you can run:
11786

11887
```bash
119-
BASE_URL="http://localhost:3001"
120-
lychee -q --format detailed \
121-
--base-url "$BASE_URL" \
122-
--exclude "^${BASE_URL}/" \
123-
--exclude "^http://localhost:" \
124-
--exclude "^https://auth0\.com/bar$" \
125-
--exclude "^https://sdk\.vercel\./" \
126-
--exclude "^http://next\.js/" \
127-
--exclude "^https://graph\.microsoft\.com/" \
128-
--exclude "^https://auth\.snapchat\.com/" \
129-
--exclude "^https://www\.npmjs\.com/" \
130-
--include "^${BASE_URL}/mcp/" \
131-
--accept 200..=299,302 \
132-
--timeout 5 \
133-
**/*.md **/*.mdx
88+
lychee --scheme https --format detailed --base-url https://auth0.com auth4genai/**/*.mdx auth4genai/**/*.md
13489
```
13590

136-
**What this does:**
137-
- Checks all MCP documentation links (`/mcp/*`) against your local dev server
138-
- Excludes other localhost paths (not running locally)
139-
- Accepts 302 redirects as valid (common for auth-required pages)
140-
- Shows only errors in detailed format
141-
142-
**Adjust the port** if your `mint dev` is running on a different port (default is 3000, but may vary).
143-
144-
#### Check All Documentation Against Production
145-
146-
From the `auth4genai/` directory:
91+
To validate against a Mintlify preview deployment, substitute the base URL:
14792

14893
```bash
149-
lychee -q --format detailed --base-url https://auth0.com **/*.md **/*.mdx
94+
lychee --scheme https --format detailed --base-url https://your-branch-name.mintlify.app auth4genai/**/*.mdx auth4genai/**/*.md
15095
```
15196

152-
This checks all links against production, useful for validating external links without running a local server.
153-
154-
### Ignoring Links
155-
156-
The `.lycheeignore` file (in repository root) contains patterns for links that should be excluded from checking:
157-
- Authentication-required URLs (dashboards, management consoles)
158-
- Rate-limited services
159-
- Local development URLs
160-
- Known bot-blocking sites
161-
162-
Edit `.lycheeignore` to add new exclusion patterns as needed.
97+
All ignore rules for uncheckable URLs live in `.lycheeignore` in the repository root. If a link is consistently uncheckable, add a pattern there rather than passing flags on the command line.

‎auth4genai/how-tos/analyze-strava-activities.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to use Auth0 for AI Agents SDKs to analyze Strava activit
44
mode: "wide"
55
---
66

7-
Use the [Vercel AI SDK](https://sdk.vercel./introduction), OpenAI GPT-4, and the Auth0 Next.js SDK to analyze user fitness activity from Strava.
7+
Use the [Vercel AI SDK](https://ai-sdk.dev/docs/introduction), OpenAI GPT-4, and the Auth0 Next.js SDK to analyze user fitness activity from Strava.
88

99
<Card title="Prerequisites">
1010
Before using this example, make sure you:

0 commit comments

Comments
 (0)