cAdvisor的单机监控

cAdvisor是谷哥提供的一款单机docker运行监测工具,包括CPU、内存、网络等,缺点是监控的时间段只在几分钟,没有历史查询及持久化的能力,简单查看一下没有问题,如果要用于企业级监控还是使用grafana+elk这类。

简单的测试命令如下,可以直接看到输出log,结束后,仍可以docker start mytest直接启动:
docker run --name mytest  -p 8080:8080 -v /var/run:/var/run:rw -v /:/rootfs:ro -v /sys:/sys:ro -v /var/lib/docker/:/var/lib/docker:ro  google/cadvisor:latest
-------
也可以使用docker-compose.yml配置来启动,如下:。
version: '3'
services:
  cadvisor:
    ## Image
    build: cadvisor
    ##image: google/cadvisor:v0.27.1
 
    ## Base
    privileged: false
    restart: always
    container_name: cadvisor
    hostname: ${ACADVISOR_HOSTNAME}
    command: -storage_driver=influxdb -storage_driver_db=${ACADVISOR_DB} -storage_driver_host=${INFLUXBD_SERVER_IP}:${INFLUXDB_SERVER_PORT}  
 
    ## Network
    #network_mode: host  
    #expose:
    ports:
      - "8080:8080"
    #depends_on:
    #link:
 
    ## Storage
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro 
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
 
    ## Env
    environment:
      - SERVICE_IGNORE=true

如果遇到无法启动,则可以直接用以下命令,检查日志输出结果。

docker logs mytest

————————
如果报错如下:

[abc@localhost ~]$ docker logs mytest
I0501 02:01:54.006051       1 storagedriver.go:50] Caching stats in memory for 2m0s
I0501 02:01:54.006344       1 manager.go:151] cAdvisor running in container: "/sys/fs/cgroup/cpuacct,cpu"
I0501 02:01:54.037279       1 fs.go:139] Filesystem UUIDs: map[]
I0501 02:01:54.037305       1 fs.go:140] Filesystem partitions: map[shm:{mountpoint:/dev/shm major:0 minor:79 fsType:tmpfs blockSize:0} tmpfs:{mountpoint:/dev major:0 minor:82 fsType:tmpfs blockSize:0} /dev/mapper/cl-root:{mountpoint:/var/lib/docker major:253 minor:0 fsType:xfs blockSize:0} /dev/sda1:{mountpoint:/rootfs/boot major:8 minor:1 fsType:xfs blockSize:0}]
I0501 02:01:54.039141       1 manager.go:225] Machine: {NumCores:2 CpuFrequency:2807689 MemoryCapacity:3958325248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:fef8b2542f22422eaf1adab44a0cd5f2 SystemUUID:F91D4D56-9CDF-2AB7-E229-52E12085A954 BootID:7a90ab7b-3915-4615-ab38-3df3c3824a4e Filesystems:[{Device:overlay DeviceMajor:0 DeviceMinor:78 Capacity:39700664320 Type:vfs Inodes:19394560 HasInodes:true} {Device:tmpfs DeviceMajor:0 DeviceMinor:82 Capacity:1979162624 Type:vfs Inodes:483194 HasInodes:true} {Device:/dev/mapper/cl-root DeviceMajor:253 DeviceMinor:0 Capacity:39700664320 Type:vfs Inodes:19394560 HasInodes:true} {Device:/dev/sda1 DeviceMajor:8 DeviceMinor:1 Capacity:1063256064 Type:vfs Inodes:524288 HasInodes:true} {Device:shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:483194 HasInodes:true}] DiskMap:map[253:0:{Name:dm-0 Major:253 Minor:0 Size:39720058880 Scheduler:none} 253:1:{Name:dm-1 Major:253 Minor:1 Size:2147483648 Scheduler:none} 8:0:{Name:sda Major:8 Minor:0 Size:42949672960 Scheduler:deadline}] NetworkDevices:[{Name:br-00027b44406b MacAddress:02:42:58:af:8f:6a Speed:0 Mtu:1500} {Name:ens33 MacAddress:00:0c:29:85:a9:54 Speed:1000 Mtu:1500}] Topology:[{Id:0 Memory:4294365184 Cores:[{Id:0 Threads:[0] Caches:[]} {Id:1 Threads:[1] Caches:[]}] Caches:[{Size:6291456 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I0501 02:01:54.040527       1 manager.go:231] Version: {KernelVersion:3.10.0-514.el7.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:1.13.1 DockerAPIVersion:1.26 CadvisorVersion:v0.28.3 CadvisorRevision:1e567c2}
I0501 02:01:54.057139       1 factory.go:356] Registering Docker factory
I0501 02:01:56.058709       1 factory.go:54] Registering systemd factory
I0501 02:01:56.060014       1 factory.go:86] Registering Raw factory
I0501 02:01:56.061514       1 manager.go:1178] Started watching for new ooms in manager
W0501 02:01:56.061593       1 manager.go:313] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
I0501 02:01:56.078795       1 manager.go:329] Starting recovery of all containers
I0501 02:01:56.156028       1 manager.go:334] Recovery completed
F0501 02:01:56.156054       1 cadvisor.go:156] Failed to start container manager: inotify_add_watch /sys/fs/cgroup/cpuacct,cpu: no such file or directory


这是谷歌的新版bug,需要如下解决

mount -o remount,rw '/sys/fs/cgroup'
ln -s /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuacct,cpu

————————-
附上使用截图


具体至某一容器