v1.5.0 updates for pipelines, targets etc. #265
Conversation
Conditional execution
WalkthroughThis pull request significantly expands documentation coverage, introducing 15+ new cloud storage target configurations, comprehensive scheduling guides (Cron and Interval), SIEM optimization documentation, OCSF field format support, and adding scheduler and debug options to existing target configurations across the system. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/configuration/targets/console.mdx (1)
78-78: Fix typo in documentation.Line 78 contains "defails" which should be "details".
Apply this diff:
-For defails of ECS, see <Topic id="appendix-ecs">Appendix</Topic>. +For details of ECS, see <Topic id="appendix-ecs">Appendix</Topic>.
🧹 Nitpick comments (11)
docs/configuration/targets/minio.mdx (1)
3-3: Consider hyphenating compound adjective in badge text. Per standard English usage, the badge should read "Long-Term Storage" rather than "Long Term Storage" to properly form the compound adjective modifying "Storage".If other target documentation uses this badge text, consider standardizing it across all target docs for consistency.
docs/configuration/targets/ibm-cos.mdx (1)
3-3: Consider hyphenating compound adjective in badge text. Use "Long-Term Storage" instead of "Long Term Storage" to properly format the compound adjective, and ensure consistency across all badge text in target documentation.docs/configuration/targets/wasabi-cloud-storage.mdx (2)
3-3: Use hyphen for compound adjective "Long-Term Storage".Per English grammar guidelines, compound adjectives modifying a noun should be hyphenated.
- <span className="theme-doc-version-badge badge badge--secondary">Long Term Storage</span> + <span className="theme-doc-version-badge badge badge--secondary">Long-Term Storage</span>
220-232: Add disclaimer to example credentials.Code examples use placeholder credentials that could trigger false-positive credential scanning. Add a clear comment indicating these are examples, not real secrets.
### Basic Configuration The minimum configuration for a JSON Wasabi target: ```yaml + # NOTE: Replace with actual credentials. These are placeholder values. targets: - name: basic_wasabi type: wasabis3 properties: key: "ABCDEFGHIJKLMNOPQRST" secret: "abcdefghijklmnopqrstuvwxyz0123456789ABCD" region: "us-east-1" endpoint: "https://s3.us-east-1.wasabisys.com" bucket: "datastream-logs"</blockquote></details> <details> <summary>docs/about/siem-optimization.mdx (2)</summary><blockquote> `54-54`: **Consider tightening wording: "without warning" can be more concise.** Current phrasing is acceptable but can be strengthened for impact. ```diff - AI models can produce unexpected results, potentially dropping critical security events without warning. + AI models can produce unexpected results, potentially dropping critical security events silently.
244-244: Use stronger adjective than "deep" for vendor knowledge.Consider a more specific descriptor that conveys expertise and analysis rigor.
- Event filters are developed based on deep vendor knowledge and real-world security operations experience. + Event filters are developed based on comprehensive vendor analysis and real-world security operations experience.docs/configuration/targets/oracle-cloud-os.mdx (3)
3-3: Use hyphen for compound adjective "Long-Term Storage" (matches Wasabi target).Per English grammar guidelines for compound adjectives.
- <span className="theme-doc-version-badge badge badge--secondary">Long Term Storage</span> + <span className="theme-doc-version-badge badge badge--secondary">Long-Term Storage</span>
154-154: Use proper diacritics for location name: "São Paulo".Geographic names should use proper diacritical marks for accuracy and consistency.
|`sa-saopaulo-1`|Brazil East (Sao Paulo)| + |`sa-saopaulo-1`|Brazil East (São Paulo)|
222-234: Add disclaimer to example credentials.Similar to Wasabi documentation, add a comment to clarify these are placeholder values.
The minimum configuration for a JSON OCI Object Storage target: ```yaml + # NOTE: Replace with actual credentials. These are placeholder values. targets: - name: basic_oci type: oracles3 properties: key: "0a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p" secret: "AbCdEfGhIjKlMnOpQrStUvWxYz0123456789+/==" region: "us-ashburn-1" endpoint: "https://mytenancy.compat.objectstorage.us-ashburn-1.oraclecloud.com" bucket: "datastream-logs"</blockquote></details> <details> <summary>docs/configuration/scheduling/interval.mdx (1)</summary><blockquote> `332-332`: **Use hyphen for compound adjective "Cost-Optimized Route".** Per English grammar guidelines for compound adjectives modifying nouns. ```diff - ### Cost Optimized Route + ### Cost-Optimized Routedocs/configuration/targets/backblaze-b2.mdx (1)
118-118: Minor: Consider conciseness for "at a fraction of" phrase.The phrase "at a fraction of the cost" is slightly wordy. Consider "significantly lower cost" for brevity, though this is optional.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (31)
docs/about/siem-optimization.mdx(1 hunks)docs/appendix/field-formats/ocsf.mdx(1 hunks)docs/configuration/scheduling/cron.mdx(1 hunks)docs/configuration/scheduling/interval.mdx(1 hunks)docs/configuration/scheduling/overview.mdx(1 hunks)docs/configuration/targets/alibaba-oss.mdx(1 hunks)docs/configuration/targets/aws-s3.mdx(1 hunks)docs/configuration/targets/aws-security-lake.mdx(1 hunks)docs/configuration/targets/azure-blob-storage.mdx(2 hunks)docs/configuration/targets/azure-data-explorer.mdx(2 hunks)docs/configuration/targets/backblaze-b2.mdx(1 hunks)docs/configuration/targets/bigquery.mdx(2 hunks)docs/configuration/targets/clickhouse.mdx(2 hunks)docs/configuration/targets/cloudflare-r2.mdx(1 hunks)docs/configuration/targets/console.mdx(2 hunks)docs/configuration/targets/digitalocean-spaces.mdx(1 hunks)docs/configuration/targets/elasticsearch.mdx(7 hunks)docs/configuration/targets/event-hubs.mdx(2 hunks)docs/configuration/targets/file.mdx(2 hunks)docs/configuration/targets/ibm-cos.mdx(1 hunks)docs/configuration/targets/microsoft-sentinel-data-lake.mdx(3 hunks)docs/configuration/targets/microsoft-sentinel.mdx(2 hunks)docs/configuration/targets/minio.mdx(1 hunks)docs/configuration/targets/oracle-cloud-os.mdx(1 hunks)docs/configuration/targets/overview.mdx(2 hunks)docs/configuration/targets/scaleway-os.mdx(1 hunks)docs/configuration/targets/splunk-hec.mdx(11 hunks)docs/configuration/targets/syslog.mdx(2 hunks)docs/configuration/targets/wasabi-cloud-storage.mdx(1 hunks)sidebars.ts(5 hunks)topics.json(2 hunks)
🧰 Additional context used
🪛 Gitleaks (8.28.0)
docs/configuration/targets/scaleway-os.mdx
[high] 213-213: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 214-214: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 229-229: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 230-230: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 254-254: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 255-255: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 277-277: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 278-278: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 297-297: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 298-298: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 316-316: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 317-317: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 337-337: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 338-338: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
docs/configuration/targets/oracle-cloud-os.mdx
[high] 229-229: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 245-245: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 270-270: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 293-293: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 313-313: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 332-332: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
docs/configuration/targets/cloudflare-r2.mdx
[high] 211-211: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 227-227: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 252-252: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 275-275: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
[high] 295-295: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 LanguageTool
docs/configuration/targets/ibm-cos.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/backblaze-b2.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[style] ~118-~118: ‘at a fraction of’ might be wordy. Consider a shorter alternative.
Context: ...rent, simple pricing model with storage at a fraction of the cost of major cloud providers and f...
(EN_WORDINESS_PREMIUM_AT_A_FRACTION_OF)
docs/configuration/scheduling/interval.mdx
[grammar] ~332-~332: Use a hyphen to join words.
Context: ...`` ### Cost Optimized Route <Commen...
(QB_NEW_EN_HYPHEN)
docs/configuration/targets/scaleway-os.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/oracle-cloud-os.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~154-~154: Did you mean “São Paulo” (= city in Brazil)?
Context: ...lbourne)| |sa-saopaulo-1|Brazil East (Sao Paulo)| |me-jeddah-1|Saudi Arabia West (Jed...
(SAO_PAOLO)
docs/configuration/targets/wasabi-cloud-storage.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/minio.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/digitalocean-spaces.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/cloudflare-r2.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/about/siem-optimization.mdx
[style] ~54-~54: ‘without warning’ might be wordy. Consider a shorter alternative.
Context: ...ially dropping critical security events without warning. This unpredictability is unacceptable ...
(EN_WORDINESS_PREMIUM_WITHOUT_WARNING)
[style] ~244-~244: Consider a different adjective to strengthen your wording.
Context: .... Event filters are developed based on deep vendor knowledge and real-world securit...
(DEEP_PROFOUND)
[grammar] ~407-~407: Use a hyphen to join words.
Context: ... Event logs represent one of the highest volume data sources in enterprise enviro...
(QB_NEW_EN_HYPHEN)
docs/configuration/targets/aws-s3.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/configuration/targets/aws-security-lake.mdx
[grammar] ~7-~7: Use a hyphen to join words.
Context: ...SF (Open Cybersecurity Schema Framework) compliant Parquet format, enabling centr...
(QB_NEW_EN_HYPHEN)
docs/configuration/targets/alibaba-oss.mdx
[uncategorized] ~3-~3: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...c-version-badge badge badge--secondary">Long Term Storage ## Synopsis Creates a ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🔇 Additional comments (51)
docs/configuration/targets/azure-data-explorer.mdx (3)
35-36: Schema correctly updated with new scheduler fields.The
intervalandcronfields are properly added to the schema at the top-level properties, consistent with their use as target-wide execution controls.
96-102: Scheduler section well-documented; verify "realtime" default value.The new Scheduler section clearly documents both fields with references to external documentation topics. However, the default value for
intervalis listed as "realtime"—please verify this is a recognized option in your system's interval specification. If "realtime" means "no scheduling, process immediately," consider whether the description could be more explicit about this behavior.Additionally, consider adding a scheduler example to the Examples section (e.g., combining
intervalorcronwith one of the existing configurations) to demonstrate real-world usage patterns.
108-108: Debug description standardized for consistency.The description update to "Process logs but don't send to target (testing)" makes the option generic and consistent across targets, improving clarity for users testing configurations across multiple target types.
docs/configuration/targets/azure-blob-storage.mdx (1)
35-36: Scheduler and debug options addition is well-structured. The changes consistently integrate with the configuration documentation and follow the established pattern for standardizing scheduling and debug capabilities across targets. The updateddont_send_logsdescription is appropriately generalized.Also applies to: 94-100, 106-106
docs/configuration/targets/event-hubs.mdx (1)
35-40: Scheduler and debug options are properly integrated. The new sections follow the documentation pattern established across other targets, with clear descriptions and appropriate references to scheduling documentation.Also applies to: 108-121
docs/configuration/targets/clickhouse.mdx (1)
26-30: Scheduler and debug options implementation is consistent. The additions follow the established pattern and integrate cleanly with existing configuration documentation.Also applies to: 63-76
docs/configuration/targets/microsoft-sentinel-data-lake.mdx (1)
3-3: Badge update and scheduling/debug additions are well-coordinated. The terminology change from "SIEM" to "Security Lake" appropriately reflects the target's focus, and the scheduler/debug additions follow the standard pattern with generalized descriptions.Also applies to: 40-41, 88-100
docs/configuration/targets/minio.mdx (1)
1-115: Comprehensive MinIO target documentation with strong structure. The schema, configuration tables, and examples provide clear guidance for users. The integration of scheduler and debug options follows established patterns.docs/appendix/field-formats/ocsf.mdx (1)
1-130: OCSF documentation is comprehensive and well-organized. The schema class categorization, practical usage examples, and integration guidance with AWS Security Lake provide clear reference material for implementing OCSF normalization. The example configurations effectively demonstrate OCSF identifier usage.docs/configuration/targets/ibm-cos.mdx (1)
1-115: IBM COS target documentation is complete and well-structured. The configuration sections, region table, and diverse examples provide clear guidance. The integration of scheduler and debug options aligns with established patterns across targets.docs/configuration/targets/aws-security-lake.mdx (2)
1-130: AWS Security Lake documentation is exceptionally comprehensive. The extensive OCSF schema identifier table, detailed bucket configuration guidance, and diverse examples covering multiple security event types provide excellent reference material. The VirtualMetric AWS Security Lake Pack integration is well-explained, making the normalization capabilities clear to users.
49-49: Verify thataws_lakepipeline is required rather than optional. Line 49 markspipelinesas required with the comment "Must includeaws_lakepipeline for OCSF normalization." This is appropriate guidance; confirm this is an actual enforced requirement in the target implementation.docs/configuration/targets/wasabi-cloud-storage.mdx (1)
74-74: Verify normalization-mapping Topic reference.The
<Topic id="normalization-mapping">reference is used across multiple target docs. Ensure this topic exists in the navigation/topics.json file and is properly documented.docs/about/siem-optimization.mdx (1)
1-579: SIEM Optimization documentation is comprehensive and well-structured.The documentation effectively explains the Risk-Free Reduction framework, contrasts it with AI-based approaches, and provides detailed coverage of optimization techniques. The Mermaid diagrams enhance understanding, and the security/compliance considerations are well-addressed. Configuration examples are clear and practical.
docs/configuration/scheduling/interval.mdx (1)
411-431: Comparison table with Schedule (Cron) is helpful.The comparison clearly delineates when to use interval vs cron scheduling, helping users make appropriate choices for their use cases.
docs/configuration/targets/bigquery.mdx (1)
31-32: Scheduler integration and debug standardization are consistent with other targets.Scheduler fields (interval, cron) properly added to schema with corresponding documentation section and Topic references. Debug option description standardized to generic "target (testing)" language for consistency across all targets.
Also applies to: 106-111, 118-118
docs/configuration/targets/microsoft-sentinel.mdx (1)
40-41: Scheduler integration and debug standardization are properly implemented.Scheduler fields (interval, cron) and debug options follow the established pattern seen in other target updates. Documentation is consistent and references are correct.
Also applies to: 88-93, 100-100
docs/configuration/targets/overview.mdx (2)
65-241: New Debug Options section provides comprehensive guidance.The section effectively covers configuration, use cases (Development, Troubleshooting, Pipeline Validation, Staged Deployment), best practices, and security considerations. Examples are practical and warning callouts properly communicate important limitations (e.g., dont_send_logs requires debug.status).
118-120: Warning about dont_send_logs behavior is clear and important.The callout correctly documents that
dont_send_logsonly works whendebug.statusis also enabled, preventing user confusion about configuration precedence.docs/configuration/targets/file.mdx (1)
29-33: Scheduler and debug options properly integrated into file target documentation.New interval, cron, and debug configuration options are correctly added to the schema and documented with corresponding sections. Documentation follows the established pattern used across other target updates.
Also applies to: 85-97
docs/configuration/targets/syslog.mdx (2)
42-46: LGTM! Scheduler and debug configuration fields added correctly.The schema correctly adds the new scheduler fields (
interval,cron) and debug options (debug.status,debug.dont_send_logs), consistent with the PR's broader addition of these capabilities across targets.
110-123: LGTM! Documentation sections are clear and well-structured.The Scheduler and Debug Options sections properly document the new fields with appropriate cross-references to the dedicated scheduling documentation.
docs/configuration/scheduling/cron.mdx (1)
1-447: LGTM! Comprehensive and well-structured cron documentation.The cron scheduling documentation is thorough and well-organized, covering:
- Clear schema definitions for both targets and routes
- Standard cron expression format with visual diagram
- Multiple practical examples across different use cases
- Common patterns reference table
- Troubleshooting guidance
The documentation effectively supports the new scheduling capabilities being added across targets in this PR.
sidebars.ts (5)
14-14: LGTM! SIEM optimization documentation added.The new "about/siem-optimization" entry is appropriately placed in the About section.
48-49: LGTM! Organization structure reorganized.Moving "tenants" and "usage-and-limits" to the top-level Organization items improves the navigation hierarchy.
119-142: LGTM! Multiple new cloud storage targets added.The additions include Alibaba OSS, AWS Security Lake, Backblaze B2, Cloudflare R2, DigitalOcean Spaces, IBM COS, MinIO, Oracle Cloud OS, Scaleway OS, and Wasabi Cloud Storage. The targets are appropriately alphabetized within the Targets category.
316-324: LGTM! Scheduling category properly structured.The new Scheduling category with overview, cron, and interval documentation is well-organized and supports the PR's scheduler feature additions across targets.
388-388: LGTM! OCSF field format added to appendix.The new "appendix/field-formats/ocsf" entry is appropriately placed in the Field Formats subsection of the Appendix.
topics.json (2)
30-31: LGTM! Scheduling topic mappings added correctly.The new topic mappings for "cron" and "interval" correctly reference the new scheduling documentation paths and align with the Topic references used throughout the target documentation.
55-55: LGTM! OCSF field format topic mapping added.The "appendix-ocsf" mapping correctly points to the new OCSF field format documentation.
docs/configuration/targets/digitalocean-spaces.mdx (2)
1-45: LGTM! DigitalOcean Spaces target documentation is comprehensive.The new target documentation follows established patterns and includes proper schema definition with scheduler and debug options consistent with other targets in this PR.
102-115: LGTM! Scheduler and Debug Options sections are consistent.These sections match the pattern established across other targets in this PR, with proper cross-references to the dedicated scheduling documentation.
docs/configuration/targets/console.mdx (2)
19-23: LGTM! Scheduler and debug fields added correctly.The schema updates are consistent with the broader PR pattern of adding scheduler and debug capabilities to targets.
39-51: LGTM! Scheduler and Debug Options sections properly documented.These sections follow the established pattern and correctly reference the scheduling documentation.
docs/configuration/scheduling/overview.mdx (1)
1-524: LGTM! Excellent comprehensive scheduling overview.This overview documentation provides outstanding coverage of the scheduling capabilities, including:
- Clear explanations of both timing methods (Cron and Interval)
- Multiple practical use cases and scenarios
- Configuration patterns and best practices
- Performance considerations and migration strategies
The documentation will be highly valuable for users implementing scheduled data processing workflows.
docs/configuration/targets/scaleway-os.mdx (3)
1-45: LGTM! Scaleway Object Storage target documentation is well-structured.The new target documentation follows established patterns and includes proper schema definition with scheduler and debug options consistent with other targets in this PR. The emphasis on European infrastructure and GDPR compliance is appropriate for this service.
102-115: LGTM! Scheduler and Debug Options sections are consistent.These sections match the pattern established across other targets in this PR, with proper cross-references to the dedicated scheduling documentation.
190-201: LGTM! GDPR compliance and regional considerations well-documented.The documentation appropriately highlights Scaleway's European focus, GDPR compliance, and data sovereignty features, which are key differentiators for this target.
docs/configuration/targets/backblaze-b2.mdx (1)
102-115: Scheduler and Debug Options properly integrated.The Scheduler and Debug Options sections follow the established pattern across targets with clear documentation of interval, cron, and debug field behaviors.
docs/configuration/targets/alibaba-oss.mdx (2)
9-45: Schema and configuration consistently structured.The schema definition and configuration tables follow the established pattern for cloud storage targets with proper Scheduler and Debug Options integration.
203-209: Good addition: Alibaba-specific sections enhance documentation.The Storage Classes and Regional Performance sections provide valuable Alibaba Cloud context and are well-suited to the target audience. The Internal Endpoint example (lines 337-354) is particularly useful for cost optimization.
docs/configuration/targets/cloudflare-r2.mdx (1)
59-67: Cloudflare R2-specific configuration guidance is accurate.The documentation correctly specifies that
region: "auto"is typical for R2 and explains the endpoint pattern with account-id placeholders.docs/configuration/targets/splunk-hec.mdx (3)
24-25: Clarify field name change:source_type→sourcetype.The schema shows
sourcetype(line 24) andsource(line 25) fields. Ifsource_typeis being renamed tosourcetype, this is a breaking change that should be explicitly documented in migration notes or deprecation guidance. Users with existing configurations usingsource_typewill be affected.Can you confirm whether
source_typeis being deprecated or if both names should be supported for backward compatibility?
107-157: Load Balancing, Dynamic Routing, and Field Normalization sections are well-documented.The new sections clearly explain randomized endpoint selection, pipeline-based routing capabilities, and the order of operations for normalization. The practical example at lines 130-145 effectively demonstrates dynamic routing via pipelines.
183-284: Examples comprehensively demonstrate Splunk HEC target capabilities.The updated examples progress logically from basic to advanced configurations, effectively demonstrating load balancing, field normalization, secure authentication, and performance tuning. All examples properly use the updated field names (
sourcetype,source).docs/configuration/targets/elasticsearch.mdx (3)
74-87: Scheduler and Debug Options properly integrated into Elasticsearch target.The new sections follow the established pattern and documentation standards with clear field descriptions and references to scheduler topic pages.
106-187: Comprehensive new sections effectively explain Elasticsearch target capabilities.The Load Balancing, JSON Message Handling, Dynamic Index Routing, Bulk API Error Handling, and Write Actions sections provide thorough explanations with practical examples. The ECS normalization context is particularly valuable for Elastic Stack users. The inline notes about tradeoffs (filter_path optimization) demonstrate good documentation practice.
307-346: New Elasticsearch examples effectively demonstrate optimization techniques.The "Index Action" (lines 307-325) and "Minimal Response" (lines 327-345) examples are particularly valuable, demonstrating the write_action field usage and filter_path optimization. These examples complement the existing ones well and address common use cases.
docs/configuration/targets/aws-s3.mdx (3)
60-70: AWS S3 credential documentation properly explains authentication options.The credential section clearly documents the IAM role authentication alternative via the footnote, providing valuable context for AWS deployments. The session token field for temporary credentials is properly included.
120-179: AWS S3 Details section comprehensively covers file management and formats.The reorganized Details section provides clear documentation of file formats, compression options, template variables, multipart upload, and multiple buckets support. The template variables table is particularly thorough.
180-298: AWS S3 examples comprehensively demonstrate all major features.The examples progress logically from basic configuration through multiple buckets, various formats, high-reliability settings, field normalization, and debug configuration. All examples properly use placeholder credentials in standard format, making the documentation clear and safe.
|
Docs Feature Deployment https://534cef25.virtualmetric-docs.pages.dev |
This pull request adds comprehensive documentation for the Alibaba Cloud OSS target and introduces scheduler-related configuration options to the Azure Blob Storage and Azure Data Explorer targets. It also improves consistency in debug option descriptions across the documentation. The most important changes are grouped below.
New Target Documentation
alibabas3), detailing configuration schema, authentication, region support, file formats, compression, file management, templates, multipart upload, multiple bucket support, schema requirements, and example configurations.Scheduler Configuration Enhancements
intervalandcron) to the Azure Blob Storage target configuration schema and documentation, allowing users to control execution frequency and scheduled runs. [1] [2]intervalandcron) to the Azure Data Explorer target configuration schema and documentation, providing similar scheduling capabilities. [1] [2]Documentation Consistency
Field Format Documentation
Summary by CodeRabbit
Documentation
New Features
Documentation