DaemonSet for Node-Level Log Collection

DaemonSet Log Collector

A DaemonSet ensures that one copy of a pod runs on every node in the cluster. This is the standard pattern for node-level agents like log collectors, monitoring agents, and security scanners.

Key Configuration

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: log-collector
  namespace: kube-system
  labels:
    app: "log-collector"
spec:
  selector:
    matchLabels:
      app: "log-collector"
  template:
    spec:
      serviceAccountName: log-collector
      containers:
        - name: fluentbit
          image: fluent/fluent-bit:2.2
          resources:
            requests:
              cpu: "50m"
              memory: "64Mi"
            limits:
              cpu: "200m"
              memory: "256Mi"
          volumeMounts:
            - name: varlog
              mountPath: /var/log
              readOnly: true
            - name: containers
              mountPath: /var/log/containers
              readOnly: true
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: containers
          hostPath:
            path: /var/log/containers
      tolerations:
        - key: "node-role.kubernetes.io/control-plane"
          operator: "Exists"
          effect: "NoSchedule"

Why DaemonSet?

Feature	DaemonSet	Deployment
Pods per node	Exactly 1	Variable
New node joins	Auto-schedules	No action
No replicas field	Scales with nodes	Manual replicas
Use case	Node agents	Application workloads

Tolerations for Control Plane

By default, control-plane nodes have a taint (node-role.kubernetes.io/control-plane:NoSchedule) that prevents workload pods from being scheduled there. The toleration in the DaemonSet spec allows the log collector to run on control-plane nodes too, ensuring complete log coverage.

HostPath Volumes

The hostPath volume mounts give the DaemonSet access to the node's filesystem. The log collector reads from:

• /var/log/containers/: Symlinks to container log files
• /var/log/: System logs (syslog, kubelet, etc.)

Using readOnly: true ensures the agent cannot accidentally modify or delete log files.