RDK-60409: Component independent Tool Development for Noisy Log detection#432
RDK-60409: Component independent Tool Development for Noisy Log detection#432dshett549 wants to merge 9 commits intordkcentral:developfrom
Conversation
…ction Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
There was a problem hiding this comment.
Pull request overview
Adds a component-independent “noisy log detection” tool and CI integration to analyze generated logs for noisy levels, sensitive patterns, and severity misuse, producing HTML reports as build artifacts.
Changes:
- Introduces a YAML ruleset (
rules.yml) for sensitive pattern detection and severity/noise heuristics. - Adds a Python-based log analyzer (
noisylogdetector.py) that scans logs and generates an HTML report. - Adds a GitHub Actions workflow to run existing tests, analyze logs, and upload the resulting HTML reports as artifacts.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
rules.yml |
Defines sensitive regex patterns plus “noisy levels” and severity enforcement keywords/levels used by the analyzer. |
noisylogdetector.py |
Implements log parsing, rule-based detection, and HTML report generation. |
.github/workflows/log-quality-rules.yml |
Runs builds/tests, executes the analyzer on produced logs, and uploads HTML reports as artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: dshett549 <DEEPTHICHANDRASHEKAR_SHETTY@comcast.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| export RT_LOG_LEVEL=info | ||
| nohup ./bin/rbusTestTimeoutValuesProvider > /tmp/log_testTimeoutValuesProvider.txt & | ||
| ./bin/rbusTestTimeoutValuesConsumer > /tmp/log_testTimeoutValuesConsumer.txt |
There was a problem hiding this comment.
Some of the nohup commands redirect only stdout to the /tmp/log_*.txt files (e.g., the provider here), but stderr will still go to the Actions log and won’t be included in the files being analyzed by noisylogdetector.py. Redirect stderr as well (2>&1) so errors are captured and analyzed consistently.
|
|
||
| // ex: if the consumer pid is 12345; | ||
| // Provider1 may open a private session on 127.0.0.1:12345 | ||
| // Provider1 may open a private session on 127.0.0.1:12345 |
There was a problem hiding this comment.
In this comment example, there are multiple spaces in "open a" which looks like an accidental formatting change and makes the comment harder to read. Consider normalizing it to a single space ("open a").
| // Provider1 may open a private session on 127.0.0.1:12345 | |
| // Provider1 may open a private session on 127.0.0.1:12345 |
| if not isinstance(rules, dict): | ||
| print(f"Error: rules.yml is empty or not a valid YAML mapping.", file=sys.stderr) | ||
| sys.exit(1) |
There was a problem hiding this comment.
The error message here is hard-coded to "rules.yml" even though load_rules accepts a path parameter. If the caller passes a different rules file, this message will be misleading; use the path value in the message for consistency with the other error paths.
| with: | ||
| path: rbus | ||
| - name: Install dependencies | ||
| run: pip install pyyaml |
There was a problem hiding this comment.
This installs PyYAML via pip, but the workflow later runs the script with python3. To avoid interpreter mismatches on runners where pip may not target the same Python, install via python3 -m pip install ... (or pip3) so the yaml module is available to the exact interpreter used.
| run: pip install pyyaml | |
| run: python3 -m pip install pyyaml |
Reason for change: Component independent tool for Noisy Log detection
Test Procedure: Tested and verified
Risks: Medium
Signed-off-by: dshett549 [DEEPTHICHANDRASHEKAR_SHETTY@comcast.com]