Skip to content

Commit b164f8a

Browse files
klihubklihub
committed
docs: update logging and debugging section.
Add minimal documentation about Opentelemetry-based log exporting. Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
1 parent c2a2189 commit b164f8a

1 file changed

Lines changed: 170 additions & 14 deletions

File tree

‎docs/resource-policy/setup.md‎

Lines changed: 170 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -26,20 +26,176 @@ file is supplied using the `--config-file <config-file>` command line option.
2626

2727
## Logging and debugging
2828

29-
You can control logging with the klog options in the configuration or by
30-
setting corresponding environment variables. You can get the name of the
31-
environment variable for a klog option by prepending the `LOGGER_` prefix
32-
to the capitalized option name without any leading dashes. For instance,
33-
setting the environment variable `LOGGER_SKIP_HEADERS=true` has the same
34-
effect as setting the log.klog.Skip_headers` config option
35-
36-
Additionally, the `LOGGER_DEBUG` environment variable controls debug logs.
37-
These are globally disabled by default. You can turn on full debugging by
38-
setting `LOGGER_DEBUG='*'`.
39-
40-
When using environment variables, once configuration from a custom resource
41-
or a configuration file is taken into use, it suppresses the settings from
42-
the environment.
29+
You can use the `debug` setting in the `log` section of the configuration custom
30+
resource to control whether and which internal components produce debug log messages.
31+
32+
The `LOGGER_DEBUG` environment variable can be used to seed this for early logging
33+
before the first configuration is acquired. By default debug logging is globally
34+
turned off. You can turn on full debugging by setting `log.debug` to `[ "*" ]`. You
35+
can turn on early full debugging by setting `LOGGER_DEBUG='*'` in the environment.
36+
37+
Additionally, you can toggle temporarily forced full debugging on using the `SIGUSR1`,
38+
and toggle it again off using the same signal.
39+
40+
When using environment variables for early debug logging, once configuration from a
41+
custom resource or a configuration file is taken into use, it suppresses any setting
42+
from the environment.
43+
44+
### Exporting Logs to OpenTelemetry
45+
46+
Logs can be exported to an OpenTelemetry-compatible log collector. The following
47+
custom resource fragment enables gRPC based log exporting.
48+
49+
```yaml
50+
apiVersion: config.nri/v1alpha1
51+
kind: TopologyAwarePolicy
52+
metadata:
53+
name: default
54+
spec:
55+
...
56+
instrumentation:
57+
httpEndpoint: 8891
58+
logExportPeriod: 15s
59+
logExporter: otlp-grpc
60+
log:
61+
debug:
62+
- policy
63+
...
64+
```
65+
66+
Additionally you need to pass the collector endpoint to the resource policy plugin
67+
using the stock Opentelemetry environment variables. With the above configuration,
68+
you need to pass `OTEL\_EXPORTER\_OTLP\_LOGS\_ENDPOINT=http://otel-collector:4317`
69+
if your collector is `otel-collector` and configured to use the standard port.
70+
71+
You can set both the configuration and the necessary extra environment variable
72+
with a Helm config fragment like this:
73+
74+
```yaml
75+
config:
76+
reservedResources:
77+
cpu: 750m
78+
pinCPU: true
79+
pinMemory: true
80+
instrumentation:
81+
httpEndpoint: :8891
82+
logExporter: otlp-grpc
83+
logExportPeriod: 15s
84+
log:
85+
debug:
86+
- policy
87+
extraEnv:
88+
OTEL\_EXPORTER\_OTLP\_LOGS\_ENDPOINT: http://otel-collector:4317
89+
```
90+
91+
For testing, you can set up a corresponding collector with a file backend for log
92+
collection using, for instance, the following deployment, service, and ConfigMap.
93+
Additionally you'll need to create `/tmp/otel/data/otel-export.out` with the right
94+
permissions for your containerized collector to be able to access it.
95+
96+
```yaml
97+
apiVersion: v1
98+
kind: ConfigMap
99+
metadata:
100+
name: otel-config
101+
namespace: kube-system
102+
data:
103+
config.yaml: |
104+
receivers:
105+
otlp:
106+
protocols:
107+
grpc:
108+
endpoint: 0.0.0.0:4317
109+
http:
110+
endpoint: 0.0.0.0:4318
111+
exporters:
112+
file:
113+
path: /data/otel-export.out
114+
debug:
115+
verbosity: normal
116+
processors:
117+
batch:
118+
service:
119+
telemetry:
120+
metrics:
121+
pipelines:
122+
traces:
123+
receivers: [otlp]
124+
exporters: [file]
125+
processors: [batch]
126+
metrics:
127+
receivers: [otlp]
128+
exporters: [file]
129+
processors: [batch]
130+
logs:
131+
receivers: [otlp]
132+
exporters: [file, debug]
133+
processors: [batch]
134+
---
135+
apiVersion: apps/v1
136+
kind: Deployment
137+
metadata:
138+
name: otel-collector
139+
namespace: kube-system
140+
spec:
141+
selector:
142+
matchLabels:
143+
app: otel-collector
144+
template:
145+
metadata:
146+
labels:
147+
app: otel-collector
148+
spec:
149+
containers:
150+
- name: collector
151+
image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector
152+
resources:
153+
requests:
154+
cpu: 750m
155+
memory: 250M
156+
limits:
157+
cpu: 750m
158+
memory: 750M
159+
ports:
160+
- containerPort: 4317
161+
- containerPort: 4318
162+
volumeMounts:
163+
- name: otel-config
164+
mountPath: /etc/otelcol
165+
- name: otel-data
166+
mountPath: /data
167+
readOnly: false
168+
imagePullPolicy: IfNotPresent
169+
terminationGracePeriodSeconds: 1
170+
volumes:
171+
- name: otel-config
172+
configMap:
173+
name: otel-config
174+
- name: otel-data
175+
hostPath:
176+
path: /tmp/otel/data
177+
type: Directory
178+
---
179+
apiVersion: v1
180+
kind: Service
181+
metadata:
182+
name: otel-collector
183+
namespace: kube-system
184+
labels:
185+
app: otel-collector
186+
spec:
187+
selector:
188+
app: otel-collector
189+
ports:
190+
- name: otel-grpc
191+
port: 4317
192+
targetPort: 4317
193+
protocol: TCP
194+
- name: otel-http
195+
port: 4318
196+
targetPort: 4318
197+
protocol: TCP
198+
```
43199

44200
<!-- Links -->
45201
[configuration]: configuration.md

0 commit comments

Comments
 (0)