Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions administration/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ The following descriptions apply to metrics outputted in Prometheus format by th
The following terms are key to understanding how Fluent Bit processes metrics:

- **Record**: a single message collected from a source, such as a single long line in a file.
- **Chunk**: log records ingested and stored by Fluent Bit input plugin instances. A batch of records in a chunk are tracked together as a single unit.
- **Chunk**: log records ingested and stored by Fluent Bit input plugin instances. A batch of records in a chunk is tracked together as a single unit.

The Fluent Bit engine attempts to fit records into chunks of at most `2 MB`, but the size can vary at runtime. Chunks are then sent to an output. An output plugin instance can successfully send the full chunk to the destination and mark it as successful. If an unrecoverable error is encountered, the chunk fails entirely. Otherwise, the output can request a retry.

Expand Down Expand Up @@ -165,13 +165,13 @@ The following descriptions apply to metrics outputted in JSON format by the `/ap
| `chunks.fs_chunks` | The total number of chunks saved to the filesystem. | chunks |
| `chunks.fs_chunks_up` | The count of chunks that are both in file system and in memory. | chunks |
| `chunks.fs_chunks_down` | The count of chunks that are only in the file system. | chunks |
| `input_chunks.{plugin name}.status.overlimit` | Indicates whether the input instance exceeded its configured `Mem_Buf_Limit.` | boolean |
| `input_chunks.{plugin name}.status.overlimit` | Indicates whether the input instance exceeded its configured `Mem_Buf_Limit`. | boolean |
| `input_chunks.{plugin name}.status.mem_size` | The size of memory that this input is consuming to buffer logs in chunks. | bytes |
| `input_chunks.{plugin name}.status.mem_limit` | The buffer memory limit (`Mem_Buf_Limit`) that applies to this input plugin. | bytes |
| `input_chunks.{plugin name}.chunks.total` | The current total number of chunks owned by this input instance. | chunks |
| `input_chunks.{plugin name}.chunks.up` | The current number of chunks that are in memory for this input. If file system storage is enabled, chunks that are "up" are also stored in the filesystem layer. | chunks |
| `input_chunks.{plugin name}.chunks.down` | The current number of chunks that are "down" in the filesystem for this input. | chunks |
| `input_chunks.{plugin name}.chunks.busy` | Chunks are that are being processed or sent by outputs and aren't eligible to have new data appended. | chunks |
| `input_chunks.{plugin name}.chunks.busy` | Chunks that are being processed or sent by outputs and aren't eligible to have new data appended. | chunks |
| `input_chunks.{plugin name}.chunks.busy_size` | The sum of the byte size of each chunk which is currently marked as busy. | bytes |

### V2 metrics
Expand All @@ -185,13 +185,11 @@ The following descriptions apply to metrics outputted in Prometheus format by th
The following terms are key to understanding how Fluent Bit processes metrics:

- **Record**: a single message collected from a source, such as a single long line in a file.
- **Chunk**: log records ingested and stored by Fluent Bit input plugin instances. A batch of records in a chunk are tracked together as a single unit.
- **Chunk**: log records ingested and stored by Fluent Bit input plugin instances. A batch of records in a chunk is tracked together as a single unit.

The Fluent Bit engine attempts to fit records into chunks of at most `2 MB`, but the size can vary at runtime. Chunks are then sent to an output. An output plugin instance can successfully send the full chunk to the destination and mark it as successful. If an unrecoverable error is encountered, the chunk fails entirely. Otherwise, the output can request a retry.

Some metrics are available only for specific plugins or runtime modes. For
example, the `fluentbit_input_http_server_ingress_queue_*` metrics are exposed
only for HTTP-based inputs that use `http_server.workers` greater than `1`.
Some metrics are available only for specific plugins or runtime modes. For example, the `fluentbit_input_http_server_ingress_queue_*` metrics are exposed only for HTTP-based inputs that use `http_server.workers` greater than `1`.

| Metric Name | Labels | Description | Type | Unit |
| ----------- | ------ | ----------- | ---- | ---- |
Expand Down Expand Up @@ -230,8 +228,6 @@ only for HTTP-based inputs that use `http_server.workers` greater than `1`.
| `fluentbit_output_retries_total` | name: the name or alias for the output instance | The number of times this output instance requested a retry for a chunk. | counter | chunks |
| `fluentbit_uptime` | hostname: the hostname on running Fluent Bit | The number of seconds that Fluent Bit has been running. | counter | seconds |
| `fluentbit_process_start_time_seconds` | hostname: the hostname on running Fluent Bit | The Unix Epoch time stamp for when Fluent Bit started. | gauge | seconds |
| `fluentbit_build_info` | hostname: the hostname, version: the version of Fluent Bit, os: OS type | Build version information. The returned value is originated from initializing the Unix Epoch time stamp of configuration context. | gauge | seconds |
| `fluentbit_hot_reloaded_times` | hostname: the hostname on running Fluent Bit | Collect the count of hot reloaded times. | counter | times |

#### Storage layer

Expand Down Expand Up @@ -578,6 +574,4 @@ Health status = (HC_Errors_Count > 5) OR (HC_Retry_Failure_Count > 5) IN 5 secon

## Telemetry Pipeline

[Telemetry Pipeline](https://chronosphere.io/platform/telemetry-pipeline/) is a
hosted service that lets you monitor your Fluent Bit agents including data flow,
metrics, and configurations.
[Telemetry Pipeline](https://chronosphere.io/platform/telemetry-pipeline/) is a hosted service that lets you monitor your Fluent Bit agents including data flow, metrics, and configurations.
Loading