-
Notifications
You must be signed in to change notification settings - Fork 941
feat(sdk-logs): Add FilteringLogRecordProcessor
#5991
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
JacksonWeber
wants to merge
25
commits into
open-telemetry:main
Choose a base branch
from
JacksonWeber:jacksonweber/logger-sampling
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+459
−2
Open
Changes from 3 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
ff06c8a
Add loggerConfigurator and tests.
JacksonWeber cfd6b7f
Update README.md
JacksonWeber 5ac7921
Update CHANGELOG.md
JacksonWeber c407f31
Fix lint & add @experimental tags.
JacksonWeber bf5dc25
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 8e25840
Update README.md
JacksonWeber 0dddc5d
Merge branch 'jacksonweber/logger-sampling' of https://github.com/Jac…
JacksonWeber 9bf4209
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 0975102
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 9a70d03
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber c221452
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber afc2b6b
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber f8e40d6
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 2bc4ff6
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 79ceb8e
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 75ade7f
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 19896a3
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber b7431cc
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 262a8a3
Remove loggerConfigurator and move to a logRecordProcessor.
JacksonWeber 3e2eb42
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber 4947cc3
Roll back logger configurator changes.
JacksonWeber 9648c0a
Merge branch 'jacksonweber/logger-sampling' of https://github.com/Jac…
JacksonWeber 1d85b23
Update changelog.
JacksonWeber ca3c518
Merge branch 'main' into jacksonweber/logger-sampling
JacksonWeber c55deae
Fix lint.
JacksonWeber File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
116 changes: 116 additions & 0 deletions
116
experimental/packages/sdk-logs/src/config/LoggerConfigurators.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| /* | ||
| * Copyright The OpenTelemetry Authors | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| import type { InstrumentationScope } from '@opentelemetry/core'; | ||
| import { SeverityNumber } from '@opentelemetry/api-logs'; | ||
| import type { LoggerConfig, LoggerConfigurator } from '../types'; | ||
|
|
||
| /** | ||
| * Default LoggerConfig used when no pattern matches | ||
| */ | ||
| const DEFAULT_LOGGER_CONFIG: Required<LoggerConfig> = { | ||
| disabled: false, | ||
| minimumSeverity: SeverityNumber.UNSPECIFIED, | ||
| traceBased: false, | ||
| }; | ||
|
|
||
| /** | ||
| * Configuration for a specific logger pattern | ||
| */ | ||
| export interface LoggerPattern { | ||
| /** | ||
| * The logger name or pattern to match. | ||
| * Use '*' for wildcard matching. | ||
| */ | ||
| pattern: string; | ||
|
|
||
| /** | ||
| * The configuration to apply to matching loggers. | ||
| * Partial config is allowed; unspecified properties will use defaults. | ||
| */ | ||
| config: LoggerConfig; | ||
| } | ||
|
|
||
| /** | ||
| * Creates a LoggerConfigurator from an array of logger patterns. | ||
| * Patterns are evaluated in order, and the first matching pattern's config is used. | ||
| * Supports exact matching and simple wildcard patterns with '*'. | ||
| * | ||
| * The returned configurator computes a complete LoggerConfig by merging the matched | ||
| * pattern's config with default values for any unspecified properties. | ||
| * | ||
| * @param patterns - Array of logger patterns with their configurations | ||
| * @returns A LoggerConfigurator function that computes complete LoggerConfig | ||
| * | ||
| * @example | ||
| * ```typescript | ||
| * const configurator = createLoggerConfigurator([ | ||
| * { pattern: 'debug-logger', config: { minimumSeverity: SeverityNumber.DEBUG } }, | ||
| * { pattern: 'prod-*', config: { minimumSeverity: SeverityNumber.WARN } }, | ||
| * { pattern: '*', config: { minimumSeverity: SeverityNumber.INFO } }, | ||
| * ]); | ||
| * ``` | ||
| */ | ||
| export function createLoggerConfigurator( | ||
| patterns: LoggerPattern[] | ||
| ): LoggerConfigurator { | ||
| return (loggerScope: InstrumentationScope): Required<LoggerConfig> => { | ||
| const loggerName = loggerScope.name; | ||
|
|
||
| // Find the first matching pattern | ||
| for (const { pattern, config } of patterns) { | ||
| if (matchesPattern(loggerName, pattern)) { | ||
| // Compute complete config by merging with defaults | ||
| return { | ||
| disabled: config.disabled ?? DEFAULT_LOGGER_CONFIG.disabled, | ||
| minimumSeverity: | ||
| config.minimumSeverity ?? DEFAULT_LOGGER_CONFIG.minimumSeverity, | ||
| traceBased: config.traceBased ?? DEFAULT_LOGGER_CONFIG.traceBased, | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| // No pattern matched, return default config | ||
| return { ...DEFAULT_LOGGER_CONFIG }; | ||
| }; | ||
| } | ||
|
|
||
| /** | ||
| * Matches a logger name against a pattern. | ||
| * Supports simple wildcard matching with '*'. | ||
| * | ||
| * @param name - The logger name to match | ||
| * @param pattern - The pattern to match against (supports '*' wildcard) | ||
| * @returns true if the name matches the pattern | ||
| */ | ||
| function matchesPattern(name: string, pattern: string): boolean { | ||
| // Exact match | ||
| if (pattern === name) { | ||
| return true; | ||
| } | ||
|
|
||
| // Wildcard pattern | ||
| if (pattern.includes('*')) { | ||
| const regexPattern = pattern | ||
| .split('*') | ||
| .map(part => part.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')) | ||
| .join('.*'); | ||
| const regex = new RegExp(`^${regexPattern}$`); | ||
| return regex.test(name); | ||
| } | ||
|
|
||
| return false; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that
emitis a hot-path, I think we should optimize here wherever possible.One thing I noticed is that the way it's implemented right now is:
Do we need the map lookup or should we just cache it in the logger? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The primary motivation for this feature work is just to get an implementation of trace-based log sampling and minimum severity-based log sampling implemented in a way that's easy for users to enable as part of the configuration of their
LoggerProvider.Thanks for the link to the issue in the Go repo, I'm happy to refactor this PR to avoid the creation/usage of the
LoggerConfiguratorand just implement these two filtering strategies in theLogRecordProcessor.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pichlermarc the PR has been updated to just add a new logRecordProcessor that handles both types of filtering. Please review when you have a chance.