-
Notifications
You must be signed in to change notification settings - Fork 9
ci/cd pipeline #27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
NalinDalal
wants to merge
38
commits into
Mentro-Org:main
Choose a base branch
from
NalinDalal:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
ci/cd pipeline #27
Changes from all commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
14b836f
ci/cd pipeline
NalinDalal 0f4eabe
Update ci-cd.yml, optimised it
NalinDalal 439f1fa
updated the ci-cd for the correct folder
NalinDalal 4ec08a5
updated path to resolve conflicts
NalinDalal 3f2b87c
check
NalinDalal e862db7
i pray to god
NalinDalal bc883cf
improved error handling
NalinDalal 3b85d2e
updated package
NalinDalal c1ce332
updated the code as per review
NalinDalal c1e62d5
updated ci/cd as per review, removed makefile,updated error handling β¦
NalinDalal 981ba6a
update logical order to lint β test β build.
NalinDalal bfbda73
Merge branch 'Mentro-Org:main' into main
NalinDalal 600b1b4
Merge branch 'main' into main
NalinDalal dbb4e2d
Merge branch 'Mentro-Org:main' into main
NalinDalal f611653
update architecture
NalinDalal 436f5c9
check
NalinDalal 07884f9
check
NalinDalal a40c4a2
chechk
NalinDalal 8b02734
check
NalinDalal 42682eb
testing and ci-cd init
NalinDalal d14d764
redundancy removed
NalinDalal 3b1899f
exception handling
NalinDalal 15f80f2
Add GitHub Action for static analysis with golangci-lint
NalinDalal d05c98a
done
NalinDalal 487a573
removed errors
NalinDalal 513cfea
removed errors by importing correct files
NalinDalal c21f203
removed errors by importing correct files
NalinDalal 20f2c5a
build error due to dependency issues, need to clean up the imports
NalinDalal 069254e
imports updated to ensure workflow runs
NalinDalal 01e726d
mst run
NalinDalal 1ef0fa4
workflow update, should run
NalinDalal ed9df3f
updated code format which was producing error
NalinDalal 3150a15
updated code format which was producing error
NalinDalal 681da60
took care
NalinDalal 4a104d4
hope it runs
NalinDalal a7b55d2
removed problematic imports
NalinDalal 22bfd1f
done
NalinDalal d928f88
2nd workflow
NalinDalal File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| name: CI/CD Pipeline | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| tags: ['v*'] | ||
| pull_request: | ||
| branches: [main] | ||
|
|
||
| jobs: | ||
| ci: | ||
| if: github.event_name != 'push' || !startsWith(github.ref, 'refs/tags/') | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v4 | ||
| with: | ||
| go-version: '^1.20' | ||
| - name: Lint | ||
| uses: golangci/golangci-lint-action@v4 | ||
| with: | ||
| version: latest | ||
| working-directory: server | ||
|
|
||
| - name: Test | ||
| run: go test ./internal/... | ||
| working-directory: server | ||
|
|
||
| - name: Build | ||
| run: go build ./... | ||
| working-directory: server | ||
|
|
||
| cd: | ||
| if: startsWith(github.ref, 'refs/tags/') | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v4 | ||
| with: | ||
| go-version: '^1.20' | ||
| - name: Build release binary | ||
| run: go build -o build/app | ||
| working-directory: server | ||
|
|
||
| - name: Upload binaries | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: release-binaries | ||
| path: server/build/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| name: Static Analysis (golangci-lint) | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'server/**.go' | ||
| - 'server/go.mod' | ||
| - 'server/go.sum' | ||
| push: | ||
| branches: [ main ] | ||
| paths: | ||
| - 'server/**.go' | ||
| - 'server/go.mod' | ||
| - 'server/go.sum' | ||
|
|
||
| jobs: | ||
| lint: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.22' | ||
|
|
||
| - name: Install golangci-lint | ||
| uses: golangci/golangci-lint-action@v4 | ||
| with: | ||
| version: latest | ||
| working-directory: server | ||
| args: --timeout=5m | ||
|
|
||
| - name: Run golangci-lint | ||
| run: golangci-lint run ./server/... --timeout=5m |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "githubPullRequests.ignoredPullRequestBranches": [ | ||
| "main" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,42 +1,78 @@ | ||
| # Guidelines for Contributing to This Repository | ||
|
|
||
| Weβre glad you want to contribute! To get started, please **fork the repository**, create your own branch from your fork, and follow the guidelines below to ensure your changes align with the projectβs workflow and quality standards. | ||
| # Guidelines for Contributing to CodeLookout | ||
|
|
||
| ## Working on changes | ||
| Weβre glad you want to contribute! Please read these rules and setup instructions before submitting a pull request. | ||
|
|
||
| - Always **create a new branch** for your work. | ||
| - **Push regularly** to avoid data loss. | ||
| - We follow a **linear commit history** β always use `git rebase` instead of `git merge`. | ||
| ## Contribution Rules | ||
|
|
||
| ## Branch naming conventions | ||
| - **Fork the repository** and create a new branch for your work. | ||
| - **Branch naming:** | ||
| - New features: `feature/<feature-name>` | ||
| - Bug fixes: `fix/<bug-description>` | ||
| - Hotfixes: `hotfix/<critical-fix>` | ||
| - Security: `security/<change>` | ||
| - Maintenance: `maintenance/<task>` | ||
| - Other/dev: `<username>/<description>` | ||
| - **Push regularly** to your branch. | ||
| - **Linear commit history:** Use `git rebase` (not `git merge`). | ||
| - **Pull Requests:** | ||
| - Use clear, descriptive titles and detailed descriptions. | ||
| - Reference related issues if applicable. | ||
| - Use **Squash and Merge** after approval. | ||
| - **Commit messages:** | ||
| - Write clear, meaningful commit messages ([guide](https://cbea.ms/git-commit/)). | ||
| - **Tests:** | ||
| - Add or update tests for new features and bug fixes. | ||
| - **Code style:** | ||
| - Follow Go best practices and project conventions. | ||
| - **Review:** | ||
| - Address all review comments before merging. | ||
|
|
||
| Branch names should reflect the purpose of the work and follow a consistent structure. | ||
| ## Local Setup Instructions | ||
|
|
||
| ### Recommended patterns | ||
| Follow these steps to set up CodeLookout for local development: | ||
|
|
||
| | Type | Pattern | Example | Notes | | ||
| | ---------------------- | -------------------------- | ------------------------------------------------ | ---------------------------------------------------- | | ||
| | New Features | `feature/<feature-name>` | `feature/integrate-openai-apis-for-code-summary` | For new user-facing features | | ||
| | Bug Fixes | `fix/<fix-name>` | `fix/auth-error` | For bug fixes | | ||
| | Hotfixes | `hotfix/<fix-name>` | `hotfix/breaking-changes-openai-sdk` | For urgent, critical fixes | | ||
| | Security Updates/Fixes | `security/<change>` | `security/suppress-server-headers` | For security-related changes | | ||
| | Maintenance | `maintenance/<task>` | `maintenance/run-integration-tests-in-workflows` | For internal improvements or upkeep | | ||
| | Development / Other | `<username>/<description>` | `alex/job-queue-poc` | For work not covered above; often temporary branches | | ||
| 1. **Clone your fork and install dependencies:** | ||
| ```sh | ||
| git clone https://github.com/<your-username>/CodeLookout.git | ||
| cd CodeLookout | ||
| cd server | ||
| go mod download | ||
| ``` | ||
|
|
||
| > **Tip:** Avoid vague names like `fix/bug-fix`. Be clear and descriptive. | ||
| 2. **Configure environment variables:** | ||
| - Copy or create a `.env` file (see `server/development.md` for details). | ||
| - Set up a local PostgreSQL and Redis instance, or use Docker Compose. | ||
|
|
||
| ## Pull request conventions | ||
| 3. **Run database migrations:** | ||
| ```sh | ||
| ./scripts/run_migrations.sh | ||
| ``` | ||
|
|
||
| GitHub will automatically apply the default PR template located [here](https://github.com/Mentro-Org/CodeLookout/blob/main/.github/PULL_REQUEST_TEMPLATE.md). | ||
| 4. **Start the development server:** | ||
| - With Docker Compose: | ||
| ```sh | ||
| docker-compose up | ||
| ./scripts/run_smeeclient.sh # For GitHub webhook forwarding | ||
| ``` | ||
| - Or locally: | ||
| ```sh | ||
| go run cmd/main.go | ||
| ./scripts/run_smeeclient.sh | ||
| ``` | ||
|
|
||
| ### PR guidelines | ||
| 5. **Access the app:** | ||
| - API: `http://localhost:8080/api/` | ||
| - Analytics Dashboard: `http://localhost:8080/analytics` | ||
|
|
||
| - **Title**: Clearly describe the purpose of the PR. | ||
| - **Description**: Include a detailed explanation of what changes were made and why. | ||
| - **Merge strategy**: Use **`Squash and Merge`** once all reviews and checks are approved. | ||
| 6. **Run/test LLM and SonarQube CLI tools:** | ||
| ```sh | ||
| make test-llm | ||
| make test-sonarqube | ||
| ``` | ||
|
|
||
| ## Commit message conventions | ||
| 7. **See `server/development.md` for more details and troubleshooting.** | ||
|
|
||
| Good commit messages are essential for a readable project history. Please follow this guide: | ||
| --- | ||
|
|
||
| π [How to Write a Git Commit Message](https://cbea.ms/git-commit/) | ||
| Thank you for contributing to CodeLookout! |
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Makefile for testing LLM and SonarQube integrations | ||
|
|
||
| .PHONY: test-llm test-sonarqube | ||
|
|
||
|
|
||
| test-llm: | ||
| go run ./cmd/llm_cli/main.go | ||
|
|
||
| test-sonarqube: | ||
| go run ./cmd/sonarqube_cli/main.go |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,70 @@ | ||
| ## PR Review Processing Flow (Redis + Asynq + Worker) | ||
|
|
||
| +----------------------+ +------------------------+ +------------------+ | ||
| | GitHub Webhook | -----> | HTTP Server (API) | | AI Review | | ||
| | (PR Open/Update) | | - Validates Webhook | | Generator (LLM)| | ||
| +----------------------+ | - Enqueues Task (Asynq)| +------------------+ | ||
| +------------------------+ | ||
| | | ||
| v | ||
| +------------------+ | ||
| | Redis (Asynq DB)| | ||
| | - Stores Queued | | ||
| | Tasks | | ||
| +------------------+ | ||
| | | ||
| v | ||
| +-----------------------------+ | ||
| | Asynq Worker (Goroutine) | | ||
| | - Dequeues Task | | ||
| | - Fetches PR Diff | | ||
| | - Builds Prompt | | ||
| | - Calls AI API | | ||
| | - Posts Comments via GitHub | | ||
| +-----------------------------+ | ||
| ```mermaid | ||
| flowchart LR | ||
| ghwebhook[GitHub Webhook PR Open or Update] --> httpapi[HTTP Server API] | ||
| httpapi --> redis[Redis Asynq DB] | ||
| redis --> worker[Asynq Worker] | ||
| worker --> ai[AI Review Engine LLM] | ||
| ai --> ghapi[GitHub API Post Comments] | ||
|
|
||
| httpapi --> validate[Validate Webhook] | ||
| validate --> enqueue[Enqueue Task Asynq] | ||
| enqueue --> redis | ||
|
|
||
| worker --> dequeue[Dequeues Task] | ||
| dequeue --> fetch[Fetch PR Diff] | ||
| fetch --> buildprompt[Build Prompt] | ||
| buildprompt --> callai[Call AI API] | ||
| callai --> postcomment[Post Comments] | ||
| postcomment --> ghapi | ||
|
|
||
| ``` | ||
|
|
||
| +----------------------+ +------------------------+ +------------------+ | ||
| | GitHub Webhook | -----> | HTTP Server (API) | | AI Review | | ||
| | (PR Open/Update) | | - Validates Webhook | | Generator (LLM)| | ||
| +----------------------+ | - Enqueues Task (Asynq)| +------------------+ | ||
| +------------------------+ | ||
| | | ||
| v | ||
| +------------------+ | ||
| | Redis (Asynq DB)| | ||
| | - Stores Queued | | ||
| | Tasks | | ||
| +------------------+ | ||
| | | ||
| v | ||
| +-----------------------------+ | ||
| | Asynq Worker (Goroutine) | | ||
| | - Dequeues Task | | ||
| | - Fetches PR Diff | | ||
| | - Builds Prompt | | ||
| | - Calls AI API | | ||
| | - Posts Comments via GitHub | | ||
| +-----------------------------+ | ||
|
|
||
| ## Understand graceful shutdown of services | ||
| ββββββββββββββββ | ||
| β main() β | ||
| ββββββ¬ββββββββββ | ||
| β | ||
| ββ initializeDependencies() | ||
| ββ signal listener goroutine ββββ | ||
| ββ RunWorker() goroutine β (listens for ctx cancel) | ||
| ββ startServer() goroutine β (listens for ctx cancel) | ||
| ββ wg.Wait() <βββββββββββββββββββ waits for both | ||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| main[main function] --> init[Initialize Dependencies] | ||
| init --> sig[SIGINT Listener Goroutine] | ||
| init --> runworker[Run Worker Goroutine] | ||
| init --> startserver[Start Server Goroutine] | ||
| sig --> wait[Wait for Shutdown] | ||
| runworker --> wait | ||
| startserver --> wait | ||
|
|
||
|
|
||
| ``` | ||
|
|
||
| ββββββββββββββββ | ||
| β main() β | ||
| ββββββ¬ββββββββββ | ||
| β | ||
| ββ initializeDependencies() | ||
| ββ signal listener goroutine ββββ | ||
| ββ RunWorker() goroutine β (listens for ctx cancel) | ||
| ββ startServer() goroutine β (listens for ctx cancel) | ||
| ββ wg.Wait() <βββββββββββββββββββ waits for both |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "os" | ||
| "github.com/nalindalal/CodeLookout/server/internal/config" | ||
| "github.com/nalindalal/CodeLookout/server/internal/llm" | ||
| ) | ||
|
|
||
| // This CLI demonstrates LLM integration using RESTLLMClient | ||
| func main() { | ||
| cfg := config.Load() | ||
| if cfg.LLMEndpoint == "" { | ||
| fmt.Println("LLM_ENDPOINT not set in config. Exiting.") | ||
| os.Exit(1) | ||
| } | ||
| // Optionally support LLM_AUTH_TOKEN for HuggingFace | ||
| authToken := os.Getenv("LLM_AUTH_TOKEN") | ||
| client := &llm.RESTLLMClient{Endpoint: cfg.LLMEndpoint, AuthToken: authToken} | ||
| code := "func add(a int, b int) int { return a + b }" | ||
| fmt.Println("Sending code to LLM endpoint:", cfg.LLMEndpoint) | ||
| result, err := client.AnalyzeCode(code) | ||
| if err != nil { | ||
| fmt.Println("LLM error:", err) | ||
| os.Exit(1) | ||
| } | ||
| fmt.Println("LLM review result:") | ||
| fmt.Println(result) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will review later β³