-
Notifications
You must be signed in to change notification settings - Fork 4.8k
feat: Add Autodoc workflow for community connector documentation updates #66556
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 10 commits
e125bb0
5a6c2e9
5464918
688bfb3
b624218
01a621d
6b435eb
639ac1c
0c2d81e
b1037a5
4329e2f
e3d1bea
0bcd823
80717f8
37944e7
017dcfa
e19d6f3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,120 @@ | ||||||||||||||||||||
# Autodoc Workflow | ||||||||||||||||||||
# | ||||||||||||||||||||
# This workflow automatically triggers documentation updates for community-supported connectors | ||||||||||||||||||||
# when pull requests are merged to master. It uses Devin AI to review connector changes and | ||||||||||||||||||||
# update the corresponding user documentation to ensure it stays current with code changes. | ||||||||||||||||||||
# | ||||||||||||||||||||
# Workflow triggers: | ||||||||||||||||||||
# - Only on merged PRs to master branch | ||||||||||||||||||||
# - Excludes bot-authored PRs to prevent automation loops | ||||||||||||||||||||
# - Only processes community-supported connectors | ||||||||||||||||||||
|
||||||||||||||||||||
name: Autodoc | ||||||||||||||||||||
|
||||||||||||||||||||
on: | ||||||||||||||||||||
pull_request: | ||||||||||||||||||||
types: [closed] | ||||||||||||||||||||
branches: | ||||||||||||||||||||
- master | ||||||||||||||||||||
Comment on lines
14
to
17
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it reasonable to swap
with
Suggested change
|
||||||||||||||||||||
|
||||||||||||||||||||
jobs: | ||||||||||||||||||||
update-connector-documentation: | ||||||||||||||||||||
name: Update connector documentation with AI | ||||||||||||||||||||
runs-on: ubuntu-latest | ||||||||||||||||||||
# Only run for merged PRs from the same repo (not forks) and exclude bot authors | ||||||||||||||||||||
if: | | ||||||||||||||||||||
github.event.pull_request.merged == true && | ||||||||||||||||||||
|
||||||||||||||||||||
github.event.pull_request.head.repo.full_name == github.repository && | ||||||||||||||||||||
ian-at-airbyte marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||
!contains(fromJSON('["airbyteio", "octavia-bot", "octavia-bot-hoard", "github-actions[bot]", "dependabot[bot]"]'), github.event.pull_request.user.login) && | ||||||||||||||||||||
!endsWith(github.event.pull_request.user.login, '[bot]') | ||||||||||||||||||||
|
||||||||||||||||||||
steps: | ||||||||||||||||||||
# Step 1: Get the merged PR code | ||||||||||||||||||||
- name: Checkout merged PR code | ||||||||||||||||||||
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 | ||||||||||||||||||||
with: | ||||||||||||||||||||
fetch-depth: 0 # Full history needed for comprehensive analysis | ||||||||||||||||||||
|
||||||||||||||||||||
# Step 2: Analyze what files were changed in this PR | ||||||||||||||||||||
- name: Fetch list of changed files from PR | ||||||||||||||||||||
id: pr-files | ||||||||||||||||||||
env: | ||||||||||||||||||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||||||||||||||||
run: | | ||||||||||||||||||||
echo "🔍 Fetching files changed in PR #${{ github.event.pull_request.number }}..." | ||||||||||||||||||||
FILES=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \ | ||||||||||||||||||||
"https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files" | \ | ||||||||||||||||||||
jq -r '[.[].filename] | @json') | ||||||||||||||||||||
|
||||||||||||||||||||
if [ -z "$FILES" ]; then | ||||||||||||||||||||
echo "❌ Error: Failed to fetch or parse PR files." | ||||||||||||||||||||
exit 1 | ||||||||||||||||||||
|
||||||||||||||||||||
fi | ||||||||||||||||||||
|
||||||||||||||||||||
echo "✅ Successfully fetched changed files list" | ||||||||||||||||||||
echo "files=$FILES" >> $GITHUB_OUTPUT | ||||||||||||||||||||
|
||||||||||||||||||||
# Step 3: Install YAML parsing tool | ||||||||||||||||||||
- name: Install YAML parser (yq) | ||||||||||||||||||||
run: | | ||||||||||||||||||||
echo "📦 Installing yq for metadata.yaml parsing..." | ||||||||||||||||||||
sudo wget https://github.com/mikefarah/yq/releases/download/v4.47.2/yq_linux_amd64 -O /usr/local/bin/yq | ||||||||||||||||||||
sudo chmod +x /usr/local/bin/yq | ||||||||||||||||||||
echo "✅ yq v4.47.2 installed successfully" | ||||||||||||||||||||
|
||||||||||||||||||||
# Step 4: Determine if this PR affects a community-supported connector | ||||||||||||||||||||
- name: Determine if connector is community-supported | ||||||||||||||||||||
id: check-support-level | ||||||||||||||||||||
run: | | ||||||||||||||||||||
echo "🔍 Checking if PR contains community-supported connector changes..." | ||||||||||||||||||||
|
||||||||||||||||||||
# Look for connector metadata files in the standard location | ||||||||||||||||||||
METADATA_FILE=$(find . -path "*/airbyte-integrations/connectors/*/metadata.yaml" | head -n 1) | ||||||||||||||||||||
|
||||||||||||||||||||
if [ -z "$METADATA_FILE" ]; then | ||||||||||||||||||||
echo "ℹ️ No connector metadata.yaml file found - this PR doesn't affect connectors" | ||||||||||||||||||||
echo "metadata_file=false" >> $GITHUB_OUTPUT | ||||||||||||||||||||
ian-at-airbyte marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||
echo "community_support=false" >> $GITHUB_OUTPUT | ||||||||||||||||||||
exit 0 | ||||||||||||||||||||
fi | ||||||||||||||||||||
|
||||||||||||||||||||
# Parse the support level from the metadata | ||||||||||||||||||||
SUPPORT_LEVEL=$(yq '.data.supportLevel' "$METADATA_FILE") | ||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||||||||||||||
echo "📋 Found metadata file: $METADATA_FILE" | ||||||||||||||||||||
echo "📋 Support level: $SUPPORT_LEVEL" | ||||||||||||||||||||
|
||||||||||||||||||||
if [ "$SUPPORT_LEVEL" != "community" ]; then | ||||||||||||||||||||
echo "ℹ️ This connector is not community-supported (level: $SUPPORT_LEVEL)" | ||||||||||||||||||||
ian-at-airbyte marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||
echo "metadata_file=true" >> $GITHUB_OUTPUT | ||||||||||||||||||||
echo "community_support=false" >> $GITHUB_OUTPUT | ||||||||||||||||||||
exit 0 | ||||||||||||||||||||
fi | ||||||||||||||||||||
|
||||||||||||||||||||
echo "✅ Community-supported connector detected - documentation update needed" | ||||||||||||||||||||
echo "metadata_file=true" >> $GITHUB_OUTPUT | ||||||||||||||||||||
echo "community_support=true" >> $GITHUB_OUTPUT | ||||||||||||||||||||
|
||||||||||||||||||||
# Step 5: Skip documentation update for non-community connectors | ||||||||||||||||||||
- name: Skip documentation update (not community connector) | ||||||||||||||||||||
if: steps.check-support-level.outputs.metadata_file == 'false' || steps.check-support-level.outputs.community_support == 'false' | ||||||||||||||||||||
run: | | ||||||||||||||||||||
echo "⏭️ Skipping documentation update:" | ||||||||||||||||||||
echo " - Metadata file found: ${{ steps.check-support-level.outputs.metadata_file }}" | ||||||||||||||||||||
echo " - Community support: ${{ steps.check-support-level.outputs.community_support }}" | ||||||||||||||||||||
echo " - Only community-supported connectors trigger automatic documentation updates" | ||||||||||||||||||||
ian-at-airbyte marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||
exit 0 | ||||||||||||||||||||
ian-at-airbyte marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||
|
||||||||||||||||||||
# Step 6: Trigger AI documentation update for community connectors | ||||||||||||||||||||
- name: Start AI documentation update session | ||||||||||||||||||||
if: steps.check-support-level.outputs.metadata_file == 'true' && steps.check-support-level.outputs.community_support == 'true' | ||||||||||||||||||||
env: | ||||||||||||||||||||
PROMPT_TEXT: "The pull request to review is ${{ github.event.pull_request.number }}" | ||||||||||||||||||||
uses: aaronsteers/devin-action@0d74d6d9ff1b16ada5966dc31af53a9d155759f4 # Pinned to specific commit for security | ||||||||||||||||||||
with: | ||||||||||||||||||||
devin-token: ${{ secrets.DEVIN_AI_API_KEY }} | ||||||||||||||||||||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||||||||||||||||||||
playbook-macro: "!connectordocs" # Use the connector documentation playbook | ||||||||||||||||||||
prompt-text: ${{ env.PROMPT_TEXT }} | ||||||||||||||||||||
tags: | | ||||||||||||||||||||
area/documentation | ||||||||||||||||||||
team/documentation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wdyt about enabling this also for certified connectors?
Reasoning: Our certified docs aren't always much higher quality than the non-certified connector docs?