Skip to content

Commit 206131b

Browse files
yusufozturkyusufozturk
committed
Update targets for v1.5.0
1 parent fddcff7 commit 206131b

14 files changed

+606
-340
lines changed

‎docs/configuration/targets/aws-s3.mdx‎

Lines changed: 241 additions & 302 deletions
Large diffs are not rendered by default.

‎docs/configuration/targets/azure-blob-storage.mdx‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Creates a target that writes log messages to _Azure Blob Storage_ with support f
3232
timeout: <numeric>
3333
max_size: <numeric>
3434
batch_size: <numeric>
35+
interval: <string|numeric>
36+
cron: <string>
3537
debug:
3638
status: <boolean>
3739
dont_send_logs: <boolean>
@@ -89,12 +91,19 @@ The following fields can be used for files:
8991
|`no_buffer`|N|`false`|Disable write buffering|
9092
|`field_format`|N|-|Data normalization format. See applicable <Topic id="normalization-mapping">Normalization</Topic> section|
9193

94+
### Scheduler
95+
96+
|Field|Required|Default|Description|
97+
|---|---|---|---|
98+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
99+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
100+
92101
### Debug Options
93102

94103
|Field|Required|Default|Description|
95104
|---|---|---|---|
96105
|`debug.status`|N|`false`|Enable debug logging|
97-
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to Azure (testing)|
106+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
98107

99108
## Details
100109

‎docs/configuration/targets/azure-data-explorer.mdx‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Creates an Azure Data Explorer (Kusto) target that ingests data directly into Az
3232
tables:
3333
- name: <string>
3434
schema: <string>
35+
interval: <string|numeric>
36+
cron: <string>
3537
debug:
3638
status: <boolean>
3739
dont_send_logs: <boolean>
@@ -91,12 +93,19 @@ targets:
9193
schema: "<schema definition>"
9294
```
9395

96+
### Scheduler
97+
98+
|Field|Required|Default|Description|
99+
|---|---|---|---|
100+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
101+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
102+
94103
### Debug Options
95104

96105
|Field|Required|Default|Description|
97106
|---|---|---|---|
98107
|`debug.status`|N|`false`|Enable debug logging|
99-
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to ADX (testing)|
108+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
100109

101110
## Details
102111

‎docs/configuration/targets/bigquery.mdx‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ Creates a BigQuery target that streams data directly into BigQuery tables using
2828
tables:
2929
- name: <string>
3030
schema: <string>
31+
interval: <string|numeric>
32+
cron: <string>
3133
debug:
3234
status: <boolean>
3335
dont_send_logs: <boolean>
@@ -101,12 +103,19 @@ Supported types:
101103
- `JSON` - JSON data
102104
- `RECORD` or `STRUCT` - Nested structure
103105

106+
### Scheduler
107+
108+
|Field|Required|Default|Description|
109+
|---|---|---|---|
110+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
111+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
112+
104113
### Debug Options
105114

106115
|Field|Required|Default|Description|
107116
|---|---|---|---|
108117
|`debug.status`|N|`false`|Enable debug logging|
109-
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to BigQuery (testing)|
118+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
110119

111120
## Details
112121

‎docs/configuration/targets/clickhouse.mdx‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,11 @@ Creates a ClickHouse target that sends log data to a ClickHouse database server
2323
table: <string>
2424
batch_size: <integer>
2525
field_format: <string>
26+
interval: <string|numeric>
27+
cron: <string>
28+
debug:
29+
status: <boolean>
30+
dont_send_logs: <boolean>
2631
```
2732
2833
## Configuration
@@ -55,6 +60,20 @@ The following fields are used to define the target:
5560
|`batch_size`|N|-|Number of log entries to batch before sending|
5661
|`field_format`|N|-|Data normalization format. See applicable <Topic id="normalization-mapping">Normalization</Topic> section|
5762

63+
### Scheduler
64+
65+
|Field|Required|Default|Description|
66+
|---|---|---|---|
67+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
68+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
69+
70+
### Debug Options
71+
72+
|Field|Required|Default|Description|
73+
|---|---|---|---|
74+
|`debug.status`|N|`false`|Enable debug logging|
75+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
76+
5877
## Details
5978

6079
The ClickHouse target uses the native ClickHouse protocol to efficiently send log data in batches. Logs are accumulated until the batch size is reached, then sent to the server. The default batch size is defined by the service configuration, but can be overridden.

‎docs/configuration/targets/console.mdx‎

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ Creates a console target that writes log messages to the standard output. Suppor
1616
pipelines: <pipeline[]>
1717
properties:
1818
field_format: <string>
19+
interval: <string|numeric>
20+
cron: <string>
21+
debug:
22+
status: <boolean>
23+
dont_send_logs: <boolean>
1924
```
2025
2126
## Configuration
@@ -31,26 +36,19 @@ The following are the fields used to define the target:
3136
|`status`|N|`true`|Enable/disable the target|
3237
|`field_format`|N|-|Data normalization format. See applicable <Topic id="normalization-mapping">Normalization</Topic> section|
3338

34-
Field format standards:
39+
### Scheduler
3540

36-
<TermTable>
37-
<TermCol>`ecs`</TermCol>
38-
<DefCol>Elastic Common Schema</DefCol>
39-
40-
<TermCol>`cim`</TermCol>
41-
<DefCol>Common Information Model</DefCol>
41+
|Field|Required|Default|Description|
42+
|---|---|---|---|
43+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
44+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
4245

43-
<TermCol>`asim`</TermCol>
44-
<DefCol>Advanced Security Information Model</DefCol>
45-
</TermTable>
46+
### Debug Options
4647

47-
:::note
48-
If no field_format is specified, log messages will be written to the console without any field normalization.
49-
:::
50-
51-
:::warning
52-
Using field normalization may impact the performance with high message volumes.
53-
:::
48+
|Field|Required|Default|Description|
49+
|---|---|---|---|
50+
|`debug.status`|N|`false`|Enable debug logging|
51+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
5452

5553
## Examples
5654

‎docs/configuration/targets/elasticsearch.mdx‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,10 @@ Creates an Elasticsearch target that sends data using the Bulk API. Supports mul
3030
username: <string>
3131
password: <string>
3232
interval: <string|numeric>
33-
schedule: <string>
33+
cron: <string>
34+
debug:
35+
status: <boolean>
36+
dont_send_logs: <boolean>
3437
```
3538
3639
## Configuration
@@ -59,8 +62,6 @@ The following are the fields used to define the target:
5962
|`filter_path`|N|`errors,items.*.error,items.*._index,items.*.status`|Response filter path|
6063
|`pipeline`|N|-|Ingest pipeline name|
6164
|`field_format`|N|-|Data normalization format. See applicable <Topic id="normalization-mapping">Normalization</Topic> section|
62-
|`frequency`|N|realtime|Execution frequency. See <Topic id="frequency">Frequency</Topic> for details|
63-
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
6465

6566
### Endpoint
6667

@@ -70,6 +71,20 @@ The following are the fields used to define the target:
7071
|`username`|N|-|Basic auth username|
7172
|`password`|N|-|Basic auth password|
7273

74+
### Scheduler
75+
76+
|Field|Required|Default|Description|
77+
|---|---|---|---|
78+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
79+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
80+
81+
### Debug Options
82+
83+
|Field|Required|Default|Description|
84+
|---|---|---|---|
85+
|`debug.status`|N|`false`|Enable debug logging|
86+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
87+
7388
## Details
7489

7590
The target supports multiple endpoints, authentication, compression, and ingest pipelines. Data is batched for efficient delivery and can be automatically routed to different indices.
@@ -339,8 +354,8 @@ Events are batched until either limit is reached:
339354
- **`max_payload_size_kb`**: Total size in kilobytes
340355

341356
Tune these based on your average event size:
342-
- **Small events (less than 1KB)**: Increase `batch_size`, keep default `max_payload_size_kb`
343-
- **Large events (greater than 10KB)**: Keep default `batch_size`, increase `max_payload_size_kb`
357+
- **Small events (&lt;1KB&gt;)**: Increase `batch_size`, keep default `max_payload_size_kb`
358+
- **Large events (>10KB)**: Keep default `batch_size`, increase `max_payload_size_kb`
344359
- **Mixed sizes**: Monitor both limits and adjust based on actual batch sizes
345360

346361
### Timeout

‎docs/configuration/targets/event-hubs.mdx‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,11 @@ Creates a target that sends processed messages to _Azure Event Hubs_ with suppor
3232
status: <boolean>
3333
cert_name: <string>
3434
key_name: <string>
35+
interval: <string|numeric>
36+
cron: <string>
37+
debug:
38+
status: <boolean>
39+
dont_send_logs: <boolean>
3540
```
3641
3742
## Configuration
@@ -100,6 +105,20 @@ EventHubs target supports two authentication methods:
100105
TLS certificate and key files must be placed in the service root directory.
101106
:::
102107

108+
### Scheduler
109+
110+
|Field|Required|Default|Description|
111+
|---|---|---|---|
112+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
113+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
114+
115+
### Debug Options
116+
117+
|Field|Required|Default|Description|
118+
|---|---|---|---|
119+
|`debug.status`|N|`false`|Enable debug logging|
120+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
121+
103122
## Details
104123

105124
The EventHubs target sends processed messages to Azure Event Hubs for real-time event streaming and analytics. It supports automatic batching for optimal performance, configurable retry mechanisms for reliability, and multiple authentication methods for flexible deployment scenarios.

‎docs/configuration/targets/file.mdx‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,11 @@ Creates a file target that writes log messages to files in various formats like
2626
batch_size: <integer>
2727
max_size: <integer>
2828
locations: <location[]>
29+
interval: <string|numeric>
30+
cron: <string>
31+
debug:
32+
status: <boolean>
33+
dont_send_logs: <boolean>
2934
```
3035
3136
## Configuration
@@ -77,6 +82,20 @@ targets:
7782
format: "json"
7883
```
7984

85+
### Scheduler
86+
87+
|Field|Required|Default|Description|
88+
|---|---|---|---|
89+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
90+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
91+
92+
### Debug Options
93+
94+
|Field|Required|Default|Description|
95+
|---|---|---|---|
96+
|`debug.status`|N|`false`|Enable debug logging|
97+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
98+
8099
## Details
81100

82101
The file target supports writing to multiple file locations with different formats and schemas. When using `SystemS3` field in your logs, the value will be used to route the message to the location with a matching ID.

‎docs/configuration/targets/microsoft-sentinel-data-lake.mdx‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ For more details on Microsoft Sentinel integration, refer to <Topic id="sentinel
3737
cache:
3838
timeout: <numeric>
3939
field_format: <string>
40+
interval: <string|numeric>
41+
cron: <string>
4042
debug:
4143
status: <boolean>
4244
dont_send_logs: <boolean>
@@ -83,12 +85,19 @@ The following fields are used to define the target:
8385
|`cache.timeout`|N|`300`|Stream cache timeout in seconds|
8486
|`field_format`|N|-|Data normalization format. See applicable <Topic id="normalization-mapping">Normalization</Topic> section|
8587

88+
### Scheduler
89+
90+
|Field|Required|Default|Description|
91+
|---|---|---|---|
92+
|`interval`|N|realtime|Execution frequency. See <Topic id="interval">Interval</Topic> for details|
93+
|`cron`|N|-|Cron expression for scheduled execution. See <Topic id="cron">Cron</Topic> for details|
94+
8695
### Debug Options
8796

8897
|Field|Required|Default|Description|
8998
|---|---|---|---|
9099
|`debug.status`|N|`false`|Enable debug logging|
91-
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to Sentinel (testing)|
100+
|`debug.dont_send_logs`|N|`false`|Process logs but don't send to target (testing)|
92101

93102
## Details
94103

0 commit comments

Comments
 (0)