Hi. I have been delaying upgrading Container Station from version 2.2.14.1638 since later versions started using a lot more CPU (>40%) even while containers are idle. Today I tried upgrading to 2.4.2.83 and the issue reappeared (even after uninstalling Container Station, removing all container-station-data, and reinstalling again).
I believe I have found the culprit: I noticed that I was seeing a `runc init` process very frequently in the htop tool taking 40% of CPU. Digging deeper, it turned out that this was due to the regular Docker healthchecks happening on my containers (I have 47 containers running, and most of them have a healthcheck running every 30 seconds). This didn't pose a problem in 2.2.14.1638 (which contains an earlier version of Docker), but due to https://github.com/moby/moby/issues/39102 and https://github.com/containerd/containerd/pull/3711, it seems to cause high CPU usage with the current version of Docker being shipped with Container Station.
I tried disabling all the healthchecks in my Docker containers, and sure enough, the CPU usage came back to the usual 20%.
The upstream issue https://github.com/moby/moby/issues/39102 is still open, so I don't expect QNAP to be able to fix this, but I'm hoping to raise awareness and for QNAP to follow the development of that issue, so it can include a fixed version ASAP.
Thank you!
Container Station 2.3.4.1698 and later using too much CPU in healthchecks?
-
- Starting out
- Posts: 26
- Joined: Thu Aug 27, 2015 2:10 am
Container Station 2.3.4.1698 and later using too much CPU in healthchecks?
FYI in case you're experiencing the same, just opened this ticket on QNAP helpdesk: