UPS control of small configuation

Printers, HDDs, USB/eSATA drives, 3rd-party programs
Locked
User avatar
graemev
Know my way around
Posts: 199
Joined: Sun Feb 12, 2012 10:17 pm

UPS control of small configuation

Post by graemev »

Well I finally have a fairly simple UPS setup with my NAS. In conclusion it's not
quite right, so I will be revisiting. However it may well be suitable for other
people so I'll write it up.

What is it suitable for?

A single NAS, with a single UPS and one or more client computers.

What it does.

Following a power cut, shuts down the NAS, then shuts down the client computers.

What I want/need it to do (version2, coming later)
  1. Right after Power cut, shutdown (power hungry) Qnap
  2. When power becomes "critical" shutdown the computer (server)
  3. At the Server shutdown, turn off the UPS
  • As power comes back, Qnap and server computer both reboot (e.g. boot at power on)
  • If power is restored before computer & UPS have shutdown, bring Qnap back up (e.g. using WoL)

Step 1: Wiring it up
  • APC BX700U USB based UPS
  • Qnap TS-451+ (Draws 33w when disks spun down about 44w when spun up) on protected power out
  • Zotac Intel based box (Draws 9w) Running Buster on protected power out
  • Netgear 16 way Gigabit Ethernet switch (Draws 6w) on protected power out
  • Various network hardware (e.g switch , WiFi AP) on protected power out
My initial way to wire this is to plug the UPS into the NAS via a USB cable [A
to B type cable].
UPS-model-1.jpg
In general you need to ensure the network switch is protected (by the UPS) in
order for the notifications to be sent. However in this case it will work even
if the switch is unprotected as it is triggered by loss of access to the UPS.
(this is a bad way to do it)

How I imagine the UPS software (NUT) works on the Qnap

I suspect there are some more subtle interactions, but this is my simplified
perspective:

Three parts
  1. client
  2. server
  3. monitor
The server "talks" to the UPS, in my case via USB but also possible via Network
(SNMP, which can be confusing as this is used later, as we shall see). The
client(s) can be used to talk to the server (on any machine). The Monitor is a
specialised client which checks the status of the server an takes actions,
e.g. Mains lost, schedule a shutdown.

The software I run on the Zotac is NUT

this is available in binary in usual repositories
along with guides. The software running on the Qnap appears to be a modified copy
of this. One place where it's been modified is "access" (which machines can connect).

The monitor needs, in part at least, to actively poll the server; because if the
server vanishes (e.g. shutdown, unplugged etc) it can't simply wait to be told.

In version1 (this one) the NAS runs the NUT server. It is configured via the QNAP web interface.

In Qnap web interface, either go via "external devices" -> "UPS" or simply select the external devices
at the top of the page.
  • Select a short (1 minute) period. The NAS will begin shutdown 1 minute after power-loss (it can take quite a while , so don't delay, but it might just be a glitch)
  • Enable it as a UPS Master. The Master is physically connected to USP (e.g. via USB) and is generally shutdown last but I am abusing this in version1.
  • Add the IP address of machines you want to allow to access this "server" **
** This is used for "access control", not as you might think to list the
controlled (shutdown) machines. So I need to add Zotec which I want to
shutdown and also my client machine where I want to run clients, such as
checking the status, level of charge etc. This feature is a Qnap added
feature it's not used in the original NUT configuration files, which use
certificates to control access.

QNAP-screen1.png
The Above settings get saved (on the Qnap) in:

/etc/config/ups/upsd.conf

Code: Select all

...
ACL all 0.0.0.0/0
ACL localhost 127.0.0.1/32
ACL client_1 aa.bb.cc.dd/32
ACL client_2 ee.ff.gg.hh/32

ACCEPT localhost
ACCEPT client_1
ACCEPT client_2
REJECT all

MAXAGE		20

# =======================================================================
# MAXAGE <seconds>
# MAXAGE 15
#
# This defaults to 15 seconds.  After a UPS driver has stopped updating
# the data for this many seconds, upsd marks it stale and stops making
# that information available to clients.  After all, the only thing worse
# than no data is bad data.
#
# You should only use this if your driver has difficulties keeping
# the data fresh within the normal 15 second interval.  Watch the syslog
# for notifications from upsd about staleness.
LISTEN 0.0.0.0

But that's about the only security measure Qnap take, the file (on Qnap) /etc/config/ups/upsd.users, contains:

Code: Select all

[admin]
                password = 123456
                allowfrom = localhost
		actions = SET
		instcmds = ALL
                upsmon master           # or upsmon slave

The user is called "admin" the password is fixed a 123456. Now this is a NUT user
"admin" not the system admin user (AKA root) however, as we see later it really
is admin(root).

The file (on Qnap) /etc/config/ups/upsmon.conf, contains:

Code: Select all

# This user should not have write access to upsmon.conf.
#
# RUN_AS_USER nutmon

RUN_AS_USER admin
The nut code goes to a lot of trouble to split privilege into two parts, mostly
it runs as "RUN_AS_USER" (typically called nut or nutmon) which as per the
comment has limited access. The Qnap config uses "admin", so pretty much wide
open and only protected by a fixed password...sigh.

The file (on Qnap) /etc/config/ups/upsmon.conf, contains:

Code: Select all

# "master" means this system will shutdown last, allowing the slaves
# time to shutdown first.
#
# "slave" means this system shuts down immediately when power goes critical.
#
# Examples: 
#
# MONITOR myups@bigserver 1 monmaster blah master
# MONITOR su700@server.example.com 1 upsmon secretpass slave

MONITOR qnapups@localhost 1 admin 123456 master
This has the NUT monitor process "monitor" the daemon (server) on the Qnap (ie
on the same machine), this is the key to monitor this UPS from another
system
we will need a version of the above line, but suitably modified to use the IP address
of the Qnap.

Code: Select all

MONITOR qnapups@nas 1 admin 123456 master
(My Qnap actually has a CNAME of nas, so the above works, you need to use your
name or IP address [more later]).

Finally the file /etc/config/ups/ups.conf (on Qnap) :

Code: Select all

[qnapups]
driver = usbhid-ups
port = /dev/ttyS1
desc = "Workstation"
pollinterval=1
So it's the fairly generic driver (good) on a USB connection. The fact that it
identifies itself as a "Workstation" rather than say "NAS" is indicative of some
laziness on the part of Qnap development.

So a pretty straight forward setup:
  • daemon interacts with /dev/ttyS1 (USB port) using the driver usbhid
  • monitor process, running on Qnap, listen/talks to the above daemon and takes action
  • two client systems are allowed to talk to the daemon, e.g. query status
There are a few more worrying aspects, the file /etc/config/ups_snmptrapd.conf
does not appear to be documented anywhere, including in nut. However it
looks like SNMPTRAPD.CONF(5)

So looking @ /etc/config/ups_snmptrapd.conf (on Qnap):
#
# use fully qualified prefix... just to be safe
#
traphandle .1.3.6.1.4.1.318.0.5 /sbin/ups_snmptrap_handler POWER_LOST
traphandle .1.3.6.1.4.1.318.0.9 /sbin/ups_snmptrap_handler POWER_RESTORED
traphandle .1.3.6.1.4.1.318.0.7 /sbin/ups_snmptrap_handler BATTERY_LOW
traphandle .1.3.6.1.4.1.318.0.1 /sbin/ups_snmptrap_handler COMM_LOST
traphandle .1.3.6.1.4.1.318.0.8 /sbin/ups_snmptrap_handler COMM_RESTORED
So looking up those OID in https://github.com/humphries40/APC-UPS- ... /traps.csv:

upsOnBattery,"WARNING: The UPS has switched to battery backup power.",...,WARNING,OPERATIONAL,1.3.6.1.4.1.318.0.5
powerRestored,"INFORMATIONAL: Utility power has been restored.",...,INFORMATIONAL,OPERATIONAL,1.3.6.1.4.1.318.0.9
lowBattery,"SEVERE: The UPS batteries are low and will soon be exhausted....",...,SEVERE,DEGRADED,1.3.6.1.4.1.318.0.7
communicationLost,"SEVERE: Communication to the UPS has been lost. ...",...,SEVERE,DEGRADED,1.3.6.1.4.1.318.0.1
communicationEstablished,"INFORMATIONAL: Communication with the UPS has been established.",...,INFORMATIONAL,OPERATIONAL,1.3.6.1.4.1.318.0.8
NB, upsOnBattery also occurs during battery calibration, so don't really trust it on it's own.

So, the above are the events relating to UPS status changes.


The usage of ups_snmptrap_handler is:

Code: Select all

# /sbin/ups_snmptrap_handler -help 
Unrecognized Option: -help
Usage: /sbin/ups_snmptrap_handler BATTERY_LOW | POWER_LOST | POWER_RESTORED | COMM_LOST | COMM_RESTORED
However there are "issues"

using: /sbin/ups_snmptrap_handler COMM_RESTORED generates an error:

Code: Select all

[/etc/config/ups] # /sbin/ups_snmptrap_handler COMM_RESTORED
sh: /etc/init.d/genpowerfail.sh: Permission denied
Because it's not executable:

Code: Select all

-rw-r--r-- 1 admin administrators 9802 2021-04-05 18:41 /etc/init.d/genpowerfail.sh
So, this looks like a bug, the script is being executed, but fails due to the
execute bit not being set. However before running it blind, I replace it with a
"trace" script:


Modified /etc/init.d/genpowerfail.sh:

Code: Select all

[/etc/config/ups] # cat /etc/init.d/genpowerfail.sh
#!/bin/sh

echo "Called at $(date) with args $0 $@ "  >> /tmp/genpowerfail.log

exit 0
Then try running this with a couple of the "safe" options (not power lost)
$ /sbin/ups_snmptrap_handler COMM_RESTORED
$ /sbin/ups_snmptrap_handler POWER_RESTORED

$ tail /tmp/genpowerfail.log

# tail /tmp/genpowerfail.log
Called at Tue Apr 27 11:46:19 GMT 2021 with args /etc/init.d/genpowerfail.sh stop
Called at Tue Apr 27 11:48:52 GMT 2021 with args /etc/init.d/genpowerfail.sh stop
So, it can be seen for a "power/UPS restore type trap" it ends up passing "stop" , slightly
counter-intuitively, but would have ended up doing:

Code: Select all

            /bin/kill `/bin/pidof poweroff` 2>/dev/null
So it's intended to "cancel the shutdown". So this "bug" means, if the power recovers
the Qnap will continue to shutdown. This needs some testing but it looks like if the mains
in interupted and restored the Qnap may continue to shutdown.

Ready to rock & Roll

So, I've changed no files (permanently), so this just what happens when you use the
Qnap web interface.

To test it, I switch off the mains, the UPS beeps a lot, one minute later the Qnap beeps
and starts it's shutdown. It takes a few minutes and the UPS battery is at 93% by the time
it done. The UPS itself remains powered up and the Zotec is left running.

So, next step, bring it back up and go to the Zotec. Install some packages:
  • nut
  • nut-client
  • nut-doc
  • nut-monitor
  • nut-server
  • nut-snmp
Then as root make some edits, /etc/nut/ups.conf, add some lines:

Code: Select all

[dummy]
driver = dummy-ups
#port = dummyups.seq
port = dummyups.dev
desc = "dummy-ups in dummy mode"
Strictly you don't need this, it defines a "Dummy UPS" connected to the
Zotec (not the Qnap) this is helpful for debugging. It uses two more files
in /etc/nut:

dummyups.seq

Code: Select all

ups.status: OL
TIMER 60
ups.status: OB
TIMER 60
ups.status: OB LB
TIMER 60
The file dummyups.dev is best generated by:

Code: Select all

upsc  qnapups@nas > /etc/nut/dummyups.dev
Where nas is the name (or IP address) of your Qnap NAS

dummpups.seq cycles the dummyups between the three states. Whereas
dummpups.dev keeps the same status unless you edit the file, so you can simulate
many different changes.

Edit the file upsd.users and add the stanza:

Code: Select all

[admin]
	password = 123456
	allowfrom = localhost
	actions = SET
	instcmds = ALL
	upsmon	 master
Again, not strictly needed. This defines a (nut) user called admin , with
password 123456 on the Zotec. The idea is this "just like" the user on the
Qnap (but we won't be using root!)

Finally, the critical change, the file /etc/nut/upsmon.conf:

Add these two lines (only the first is strictly necessary):

Code: Select all

MONITOR qnapups@nas.mydomain     1 admin 123456 slave
MONITOR dummy@localhost          1 admin 123456 master
Again nas.mydomain is the name/IP address of your NAS.

Here we monitor 2 different UPS. qnapups@nas.mydomain is the real one on the
Qnap (this is the "hardwired" name+password we saw above), whereas
dummy@localhost is, as it sounds, a dummy UPS. You MAY want to remove the dummy
once you are happy. Probably commenting it out is better as you can switch it
back on for debugging without needing to keep "pulling the plug".

Now, that's enough, because; what happens now is, the power is cut, the Qnap shuts down
this makes the ups daemon (so so UPS) uncontactable, the Zotec then shuts down
because is senses the loss of the UPS.

One question is why does the Zotec shutdown just because it lost contact with
the UPS? The man page for upsmon(8):
When upsmon runs as a slave, it is relying on the distant system to tell it about the state of
the UPS. When that UPS goes critical (on battery and low battery), it immediately invokes the
local shutdown command. This needs to happen quickly. Once it disconnects from the distant
upsd(8) server, the master upsmon will start its own shutdown process. Your slaves must all shut
down before the master turns off the power or filesystem damage may result.
Since the upsmon.conf on the Zotec is marked as "slave" my guess is loss of
contact with the UPS is treated as "Critical" (since it can no longer detect low
battery). So we are rather abusing the system. The "master" just shuts down
after 1 minute (due to GUI setting on Qnap) the Zotec shuts down because it
belives itself to be a slave and is trying to shutdown before that master.

This is a little basic (and the wrong way around) version 2 of this document
will address doing it the right way round. However there are a few other things
to look at configuring because you might want a more nuanced configuration.

In /etc/nut/upssched.conf, some edits:

Code: Select all

CMDSCRIPT /bin/upssched-cmd

PIPEFN /var/run/nut/upssched.pipe

LOCKFN /var/run/nut/upssched.lock

AT COMMBAD *   START-TIMER upsgone    10
AT ONBATT  *   START-TIMER upsbattery 10
AT FSD     *   START-TIMER upsforced  10
AT NOCOMM  *   START-TIMER upsnocomm  10

AT ONLINE qnapups@nas.mydomain CANCEL-TIMER upsgone
What this does is create a series of calls to the script
/bin/upssched-cmd based on various SNMP events (e.g running on battery)
take care, the system can run on battery during calibration.

Using this allows a lot of flexibility taking different actions based on
particular events. This will be revisited more in version 2. Additionally the
script /usr/lib/systemd/system-shutdown/nutshutdown (in certain circumstances)
uses UPSDRVCTL(8) to power down the UPS; at least I think that's what it
will do :-)
You do not have the required permissions to view the files attached to this post.
User avatar
graemev
Know my way around
Posts: 199
Joined: Sun Feb 12, 2012 10:17 pm

Re: UPS control of small configuation -- might take a while for V2

Post by graemev »

Turns out NUT is not a mature as one might like. I was finding messages to the effect:

Code: Select all

  22510:Apr 30 21:29:11 zbox upsd[1234]: Connected to UPS [BX700]: usbhid-ups-BX700                                                                                                                              
  22624:Apr 30 21:32:44 zbox upsd[1234]: Connected to UPS [dummy]: dummy-ups-dummy                                                                                                                               
  22631:Apr 30 21:32:50 zbox upsd[1234]: Connected to UPS [BX700]: usbhid-ups-BX700                                                                                                                              
  23229:Apr 30 22:47:35 zbox upsd[1234]: Data for UPS [BX700] is stale - check driver                                                                                                                            
  23864:Apr 30 23:16:13 zbox upsd[1234]: Send ping to UPS [BX700] failed: Resource temporarily unavailable                                                                                                       
  23865:Apr 30 23:16:13 zbox upsd[1234]: Connected to UPS [BX700]: usbhid-ups-BX700                                                                                                                              
  23867:Apr 30 23:16:30 zbox upsd[1234]: Data for UPS [BX700] is stale - check driver                                                                                                                            
  24291:Apr 30 23:45:01 zbox upsd[1234]: Send ping to UPS [BX700] failed: Resource temporarily unavailable                                                                                                       
  24292:Apr 30 23:45:01 zbox upsd[1234]: Connected to UPS [BX700]: usbhid-ups-BX700                                                                                                                              
  24295:Apr 30 23:45:17 zbox upsd[1234]: Data for UPS [BX700] is stale - check driver
This seems to be well known. In a discussion in 2016 it was found to be fixed by moving to the libusb 1.0 (from a very old and unsupported version) this was requested and underway. It hit some snags. As of 9 days ago this was still being resolved.

Nut fails because of old libUSB --> https://alioth-lists.debian.net/piperma ... 10384.html
Nut needs to upgrade to libusb 1.0 --> https://bugs.debian.org/cgi-bin/bugrepo ... bug=810449
Upgrade of Nut to libusb 1.0 Delayed --> https://github.com/networkupstools/nut/issues/300

Looks like a plausible workaround has been found, but we have a long wait for the packages to be built and backported (assuming this happens).
Last edited by graemev on Sun May 02, 2021 2:36 am, edited 1 time in total.
elvisimprsntr

Re: UPS control of small configuation

Post by elvisimprsntr »

I connected my UPS to my https://pfsense.org firewall as a UPS master and set up QNAP as a UPS slave. Works great!

https://forum.netgate.com/topic/69828/n ... as-slave/2
User avatar
graemev
Know my way around
Posts: 199
Joined: Sun Feb 12, 2012 10:17 pm

Re: UPS control of small configuation

Post by graemev »

Yep, that's the way I'd figured it would work for me , esp. the need to use qnapups & 123456 .... fixed by Qnap and VERY VERY insecure . However the NUT package with my USB ports fails regularly, as it does for many other system. However SOME systems ONLY work with the old usblib ... you may be lucky enough to be one of those.

If you're willing, what output do you get (on pfSense) from apt depends nut-server ** :

Code: Select all

# apt depends nut-server                                                                                                                                                                       
nut-server                                                                                                                                                                                                       
  Depends: adduser                                                                                                                                                                                               
  Depends: lsb-base (>= 3.0-6)                                                                                                                                                                                   
  Depends: nut-client (= 2.7.4-5)                                                                                                                                                                                
  Depends: init-system-helpers (>= 1.18~)                                                                                                                                                                        
  Depends: libc6 (>= 2.15)                                                                                                                                                                                       
  Depends: libnspr4 (>= 2:4.9-2~)                                                                                                                                                                                
  Depends: libnss3 (>= 2:3.13.4-2~)                                                                                                                                                                              
  Depends: libupsclient4 (>= 2.7.2)                                                                                                                                                                              
  Depends: libusb-0.1-4 (>= 2:0.1.12)                                                                                                                                                                            
  Depends: libwrap0 (>= 7.6-4~)                                                                                                                                                                                  
  Depends: udev (>= 0.124-1)                                                                                                                                                                                     
  Conflicts: <nut-hal-drivers>                                                                                                                                                                                   
  Breaks: nut (<< 2.6.1-2~)                                                                                                                                                                                      
  Suggests: nut-cgi                                                                                                                                                                                              
  Suggests: nut-ipmi                                                                                                                                                                                             
  Suggests: nut-snmp                                                                                                                                                                                             
  Suggests: nut-xml                                                                                                                                                                                              
  Replaces: nut (<< 2.6.1-2~)
I don't know what package management is used on pfSense , it may be a different command but ldd(1) shows nothing, so I assume it's statically linked.

Code: Select all

# ldd /lib/nut/upsmon                                                                 
                                                                                                        |        linux-vdso.so.1 (0x00007ffe49bcd000)                                                            
                                                                                                        |        libupsclient.so.4 => /lib/x86_64-linux-gnu/libupsclient.so.4 (0x00007fd750eb5000)               
                                                                                                        |        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd750c98000)                   
                                                                                                        |        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd7508f9000)                               
                                                                                                        |        libnss3.so => /lib/x86_64-linux-gnu/libnss3.so (0x00007fd7505af000)                             
                                                                                                        |        libssl3.so => /lib/x86_64-linux-gnu/libssl3.so (0x00007fd750357000)                             
                                                                                                        |        libplds4.so => /lib/x86_64-linux-gnu/libplds4.so (0x00007fd750153000)                           
                                                                                                        |        libplc4.so => /lib/x86_64-linux-gnu/libplc4.so (0x00007fd74ff4e000)                             
                                                                                                        |        libnspr4.so => /lib/x86_64-linux-gnu/libnspr4.so (0x00007fd74fd0e000)                           
                                                                                                        |        /lib64/ld-linux-x86-64.so.2 (0x00007fd7512d0000)                                                
                                                                                                        |        libnssutil3.so => /lib/x86_64-linux-gnu/libnssutil3.so (0x00007fd74fae0000)                     
                                                                                                        |        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd74f8dc000)                             
                                                                                                        |        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd74f6c2000)                               
                                                                                                        |        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fd74f4ba000)   
elvisimprsntr

Re: UPS control of small configuation

Post by elvisimprsntr »

You don't need to use the QNAP or pfsense command line. It can be easily configured via their respective GUIs

pfsense UPS (after installing NUT via package manage GUI)
Screen Shot 2021-05-02 at 06.42.25.png
pfsense FW rule
Screen Shot 2021-05-02 at 06.45.53.png
QNAP UPS
Screen Shot 2021-05-02 at 06.43.58.png
You do not have the required permissions to view the files attached to this post.
User avatar
graemev
Know my way around
Posts: 199
Joined: Sun Feb 12, 2012 10:17 pm

Re: UPS control of small configuation

Post by graemev »

Sorry I'm at a loss to understand your reply ? I obviously don't have a pfSense . I was just interested to see if they have modified their version of nut to use the supported usblib ? I read the link you you quoted on how to setup the pfSense [which looks like what you've set in the GUI] ...if you only have access to A GUI , tools like synaptic can show the same information.
elvisimprsntr

Re: UPS control of small configuation

Post by elvisimprsntr »

graemev wrote: Sun May 02, 2021 7:31 pm Sorry I'm at a loss to understand your reply ? I obviously don't have a pfSense . I was just interested to see if they have modified their version of nut to use the supported usblib ? I read the link you you quoted on how to setup the pfSense [which looks like what you've set in the GUI] ...if you only have access to A GUI , tools like synaptic can show the same information.
Sorry, I incorrectly assumed you are running pfsense. Yes pfsense usblib work with my APCC UPS.

Running NUT 2.7.4_8
User avatar
graemev
Know my way around
Posts: 199
Joined: Sun Feb 12, 2012 10:17 pm

Re: UPS control of small configuation

Post by graemev »

In version 2. I've switched the cables around, so the USB cable from the UPS is
plugged into the Zotac. This will mean we make the Zotac the "master" from the
viewpoint of the UPS software. I've also added a client computer to control & monitor
the system. The client computer is NOT on a protected supply.
UPS-model-V2.jpg
The behaviour required is:
  1. The NAS "looks" to the Zotac as a master
  2. I can logon to the client computer and issue commands to control NUT (e.g. mute alarm, set thresholds etc)and get a status window.
Nut-Monitor-1.png
Nut-Monitor-2.png

At time of power cut
  1. The Client machine is "unprotected" so it is stopped in an uncoordinated manner.
  2. Soon after Power cut (1 minute), the (power hungry) Qnap starts shutdown, this takes several minutes
  3. When power becomes "critical" the Zotac (server) starts shutdown
  4. As the Zotac completes shutdown, it turns off the UPS
If the process runs to completion, then everything is powered off. When mains
supply is restored the UPS starts to recharge its battery, when it reaches a
certain level , it powers up the Qnap and Zotac **. Assuming these have been set to boot
on power, both restart.

** Turns out, on my UPS, this point is "zero" (ie, no delay)

If the power is restored before the Qnap is shutdown, it SHOULD cancel the
shutdown, however it's possible this is broken, as the script is not-executable***

*** In tests, however it worked fine, this script may be vestigial

If the power is restored, after the Qnap is shutdown but while the Zotac is
still running, the Zotac will cancel it's own shutdown. The remaining step is
to restart the Qnap (which shutdown but never lost power) this can be
accomplished via Wake-on-LAN (WoL) this is not yet done and will be in V3.

Step1
The Qnap needs to be reconfigured via the Web GUI.

QNAP-screen1-as-slave.png
The address xxx.yy.zzz.aaa is the address of the Zotac box, where the UPS
is plugged in.

This makes changes in the config file in the Qnap:

/etc/config/ups/ups.conf is unchanged (and presumably now, unused)
/etc/config/ups/upsmon.conf has the critical line added:

Code: Select all


MONITOR qnapups@xxx.yy.zzz.aaa 1 admin 123456 slave

Where xxx.yy.zzz.aaa is the address given on the Qnap GUI screen earlier
(the address of the Zotac box). The other stuff is more of a problem.
  • qnapups - we were never asked for a name, this MUST be the name of the UPS entry in /etc/nut/ups.conf on the Zotac (I'd used BX700, it will need to change)
  • admin - Again,never given a choice. This MUST be the userid allowed to access the ups-monitor (on Zotac)
  • 123456 - This MUST be the password used for the admin account on the Zotac
So I'll make a cry for help to @OneCD. The file /etc/config/ups/upsmon.conf has
hardwired values that really pessimises security. The RO copy will be elsewhere,
but it needs updating here in this RW copy after it's restored but before it's
used.

Until that's resolved, we have to live with what we have. We will need to make
the config/credentials match the ones supplied from the Qnap (rather back to
front).


The packages installed are listed in V1 (above) this is a Debian (Buster)
system. Most significantly it uses systemd , which has a big effect.

Step2

Setting up the Master (Zotac). These files are on the Zotac.

Before doing too much, the following edit is a good idea. It should be undone
once you are happy with the behaviour. The file /etc/nut/upsmon.conf on Zotac:

Code: Select all

# Commented out to prevent UPS powerdown
# POWERDOWNFLAG /etc/killpower
The setting of POWERDOWNFLAG is commented out. This would normally power
down the UPS and so cut power to the Qnap, even if it had not shutdown
while you're testing this out it's safer to prevent this. As I know to my cost
powering off a Qnap before it's shutdown is bad news.

File: /etc/nut/ups.conf on Zotac

Right now I have the UPS working, with the following:

Code: Select all

# Set maxretry to 3 by default, this should mitigate race with slow devices:
maxretry = 3
synchronous = yes # GPV based on comments above and the fact I'm getting lots of errors

...

#[dummy]
#driver = dummy-ups
##port = dummyups.seq
#port = dummyups.dev
#desc = "dummy-ups in dummy mode"

[BX700]
driver = usbhid-ups
port = auto
vendorid = 051d
desc = "Server"
pollinterval=10
Note the Dummy driver from V1 is commented out, this is important as I
discovered, again to my cost. Having two UPS defined like this really acts like
you have two UPS. So when you pull the plug, the BX700 eventually goes critical
but dummy is still OK and the MONITOR line says we can survive with a single
UPS. So the Zotac does not shutdown and is "surprised" when he the power is cut
and dummy totally fails to supply power :-( .

But as we can see from the above config on Qnap, the UPSname MUST be defined as
qnapups and not BX700 as I'd prefer. So we need to edit as follows:

Code: Select all

synchronous = yes # GPV based on comments above and the fact I'm getting lots of errors

#[dummy]
#driver = dummy-ups
##port = dummyups.seq
#port = dummyups.dev
#desc = "dummy-ups in dummy mode"

[qnapups]
driver = usbhid-ups
port = auto
vendorid = 051d
desc = "Server"
pollinterval=10
The second stanza needs line by line justification.
  • synchronous = yes = Comments in the file point towards this (combined with pollinterval, solved my stale messages problems)
  • [qnapups] = This name is fixed by QNAP
  • driver = usbhid-ups = This is the driver (likely to be the one you want) copied from The QNAP setting
  • port = auto = USB is dynamic, we don't/can't set a device (notwithstanding what is set on QNAP)
  • vendorid = 051d = From lsusb (see below)
  • desc = "Server" = Simple text, this is a "Server" shown by # upsc -L
  • pollinterval=10 = Only poll ever 10 seconds (default was 1) so as not to overload UPS processor
The vendorid is obtained from lsusb(1):

Code: Select all

/etc/nut# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 006: ID 051d:0002 American Power Conversion Uninterruptible Power Supply
Bus 001 Device 007: ID 0557:2213 ATEN International Co., Ltd CS682 2-Port USB 2.0 DVI KVM Switch
...
Actually you can simply miss out the line, if you only have one UPS it will find
it. You can also add productid if you have two or more from the same
company (but see above what it means to have two UPS).

The next thing we need to match up with the QNAP settings is the admin/123456
setting.

File: /etc/nut/upsd.users in Zotac

Code: Select all

...
[admin]
	password = 123456
	upsmon   master

[root]
	password = 123456
#       allowfrom = localhost No longer used/supported since 2.4
	actions = SET
	actions = FSD
	instcmds = ALL
We have defined a user called admin with a password of 123456 (to match QNAP)
but only for use by upsmon. Avoids it being used e.g. to power off the UPS by
somebody (anybody) who knew the password (123456). I've then defined a user
called root with a password 123456, don't copy this, choose your own
userid & password this definition allows a user called root (no relation) to
pretty much do anything including cutting the power.

File: /etc/nut/upsd.conf on Zotac

You only need one listen line, but we are going to allow localhost & a network interface:

Choose one of the following lines. Using 0.0.0.0 works, but means your server can be accessed via any interface
if you have multiple interfaces (e.g. Ethernet & WiFi) it would be more secure to limit access.

Code: Select all

LISTEN 127.0.0.1 3493
LISTEN xxx.yy.zzz.aaa 3493
# Listen everywhere (TBD Danger external access)
# LISTEN 0.0.0.0
File: /etc/nut/upsmon.conf on Zotac

A couple of significant lines:

Code: Select all

#MONITOR dummy@localhost          1 admin 123456 master
MONITOR qnapups@localhost        1 admin 123456 master
Again make sure to comment out the dummy UPS otherwise you never shutdown

The Schematic shows roughly how the bits are now put together (the connection
mechanism between upsd and the driver(s) is rather unclear).
UPS-schematic.jpg
Crucially the control of these is done via systemd services (this is to some
extent now Debian specific) older systems may use SystemV init scripts.
  • nut-driver.service
  • nut-monitor.service
  • nut-server.service
Now, in theory, these can be started and stopped, using:

Code: Select all

systemctl start  <service>
systemctl stop   <service>
systemctl status <service>
However don't do that!

nut-driver controls upsdrvctl and hence the device specific processes usbhid-ups & dummy-ups
nut-monitor controls upsmon
nut-server controls upsd

The nut-server has a Wants=nut-driver.service (probably ought to be requires) so
starting nut-server.service causes nut-driver.service to start and as
nut-driver.service has StopWhenUnneeded=yes it will stop when nut-server is
stopped. That is, unless you start or stop the drivers by other means. So just stick to:

Code: Select all

systemctl start/stop/status  nut-server.service
systemctl start/stop/status  nut-monitor.service
i.e. don't play with nut-driver directly.

(These services should be enabled following their install, but need restarting as you edit
the confg files)

Once restarted, the QNAP GUI shows it as connected/charged etc.
QNAP-screen1-as-slave-connected.png
So the UPS looks very like it's "plugged into" the QNAP , but this is a reference to the "remote" UPS.

test1: from client do upscmd -l -u root -p 123456 qnapups@<server>

Code: Select all

Instant commands supported on UPS [qnapups]:

beeper.disable - Disable the UPS beeper
beeper.enable - Enable the UPS beeper
beeper.mute - Temporarily mute the UPS beeper
beeper.off - Obsolete (use beeper.disable or beeper.mute)
beeper.on - Obsolete (use beeper.enable)
load.off - Turn off the load immediately
load.off.delay - Turn off the load with a delay (seconds)
shutdown.reboot - Shut down the load briefly while rebooting the UPS
shutdown.stop - Stop a shutdown in progress
test.battery.start.deep - Start a deep battery test
test.battery.start.quick - Start a quick battery test
test.battery.stop - Stop the battery test
test.panel.start - Start testing the UPS panel
test.panel.stop - Stop a UPS panel test
graeme@real:~$ upscmd -l -u root -p 123456 qnapups@zbox
Instant commands supported on UPS [qnapups]:

beeper.disable - Disable the UPS beeper
beeper.enable - Enable the UPS beeper
beeper.mute - Temporarily mute the UPS beeper
beeper.off - Obsolete (use beeper.disable or beeper.mute)
beeper.on - Obsolete (use beeper.enable)
load.off - Turn off the load immediately
load.off.delay - Turn off the load with a delay (seconds)
shutdown.reboot - Shut down the load briefly while rebooting the UPS
shutdown.stop - Stop a shutdown in progress
test.battery.start.deep - Start a deep battery test
test.battery.start.quick - Start a quick battery test
test.battery.stop - Stop the battery test
test.panel.start - Start testing the UPS panel
test.panel.stop - Stop a UPS panel test

Pulling the power cord

  1. Pull power, but replug within 1 minute (a blip in the mains) Does the QNAP cancel it's shutdown
  2. Pull power, but replug after QNAP shuts-down (e.g a real fault where power company need to switch supply) Does the Zotac cancel it's shutdown
  3. Pull power, and leave it out until everything is silent
  4. uncomment line POWERDOWNFLAG /etc/killpower in /etc/nut/upsmon.conf, restart nut-monitor.service, then pull the plug. Does the UPS shutdown after everything else
1. Less than 1 minute (30 seconds)

Code: Select all

May 10 13:24:59:qnap451plus.home daemon.warning  qulogd[11081]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: External Device, Category: UPS, Content: [External Device] Detected UPS power loss. Shutting down NAS in 1 minute(s).] 
May 10 13:25:22:qnap451plus.home daemon.warning  qulogd[11081]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: External Device, Category: UPS, Content: [External Device] UPS power restored. Canceled shutdown.] 
The QNAP started a shutdown, then cancelled it. Best outcome (and covers many of my poweroutages)

1. Greater than 1 minute (2 minute)

Code: Select all

May 10 13:33:50:qnap451plus.home daemon.warning  qulogd[11081]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: External Device, Category: UPS, Content: [External Device] Detected UPS power loss. Shutting down NAS in 1 minute(s).] 
May 10 13:35:53:qnap451plus.home daemon.info  qulogd[11081]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: Network & Virtual Switch, Category: , Content: [Network & Virtual Switch] Stopped Network & Virtual Switch.] 
May 10 13:36:03:qnap451plus.home daemon.info  qulogd[11081]:/qulogd  [ event log: Users: System, Source IP: 127.0.0.1, Computer name: localhost, Application: App Center, Category: App Status Change, Content: [App Center] Stopped Media Streaming Add-on.] 
At +one munute 30 seconds, Qnap started its shutdown. At two minute mark power restored. Qnap continued its shutdown

So, powered the QNAP back up (button on the front).

Code: Select all

May 10 13:43:00:qnap451plus.home daemon.info  qulogd[11219]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: Power, Category: NAS Power Status, Content: [Power] System started.] 
...
May 10 13:44:15:qnap451plus.home daemon.info  qulogd[11219]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: External Device, Category: UPS, Content: [External Device] USB UPS plugged in.] 
...

Now the long test...

Pull power, and leave it out until everything is silent

I intend to keep all logs in GMT, so wherever I am in the world I can compare logs. However the QNAP log here is BST (GMT+1)

Log from QNAP (on GMT+1)

Code: Select all

...
May 10 13:56:40:qnap451plus.home daemon.warning  qulogd[11219]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: External Device, Category: UPS, Content: [External Device] Detected UPS power loss. Shutting down NAS in 1 minute(s).] 
May 10 13:58:30:qnap451plus.home daemon.info  qulogd[11219]:/qulogd  [ event log: Users: admin, Source IP: 127.0.0.1, Computer name: ---, Application: Network & Virtual Switch, Category: , Content: [Network & Virtual Switch] Stopped Network & Virtual Switch.] 
May 10 13:58:44:qnap451plus.home daemon.info  qulogd[11219]:/qulogd  [ event log: Users: System, Source IP: 127.0.0.1, Computer name: localhost, Application: App Center, Category: App Status Change, Content: [App Center] Stopped Media Streaming Add-on.] 
Log from Zotac (On GMT)

Code: Select all

May 10 12:56:26 zbox upsmon[940]: UPS qnapups@localhost on battery
May 10 12:56:26 zbox upssched[3357]: Timer daemon started
May 10 12:56:27 zbox upssched[3357]: New timer: upsbattery (10 seconds)
May 10 12:56:37 zbox upssched[3357]: Event: upsbattery
May 10 12:56:37 zbox upssched-cmd: The APC-UPS is on battery [upsbattery].
May 10 12:56:52 zbox upssched[3357]: Timer queue empty, exiting

May 10 14:25:47 zbox upsmon[940]: UPS qnapups@localhost battery is low
May 10 14:25:47 zbox upsd[920]: Client admin@127.0.0.1 set FSD on UPS [qnapups]
May 10 14:25:47 zbox upsmon[940]: Executing automatic power-fail shutdown
May 10 14:25:47 zbox upsmon[940]: Auto logout and shutdown proceeding
May 10 14:25:52 zbox systemd[1]: Stopped Network UPS Tools - power device monitor and shutdown controller.
So 1 minute after the plug was pulled, the QNAP shutdown, about 90 minutes later
the Zotac shutdown. The UPS remained on because we have the line commented out
in upsmon.conf. (POWERDOWNFLAG /etc/killpower ).

So the final step is to uncomment the killpower line and see if the UPS is
powered off too.

Pull power, and leave it out until everything is silent...and see if the the
UPS light goes out (and my Ethernet switch is turned off)

Code: Select all

May 10 17:41:28 zbox upsmon[19089]: UPS qnapups@localhost battery is low
May 10 17:41:28 zbox upsd[889]: Client admin@127.0.0.1 set FSD on UPS [qnapups]
May 10 17:41:28 zbox upsmon[19089]: Executing automatic power-fail shutdown
May 10 17:41:28 zbox upsmon[19089]: Auto logout and shutdown proceeding
May 10 17:41:33 zbox systemd[1]: Stopped Network UPS Tools - power device monitor and shutdown controller.
...
May 10 17:41:33 zbox upsd[889]: mainloop: Interrupted system call
...
May 10 17:41:33 zbox upsd[889]: Signal 15: exiting
May 10 17:41:33 zbox systemd[1]: Stopping Session c1 of user lightdm.
May 10 17:41:33 zbox systemd[1]: Closed Load/Save RF Kill Switch Status /dev/rfkill Watch.
<light goes red>
Various clicks an whirrs from UPS, all small devices (Ethernet switch etc) power off, UPS light
goes off.

Wait a few minutes. Turn mains back on. Immediately Ethernet switch comes
back. This is bad news, the UPS has reactivated the "load" but there is no
battery in reserve, so if the mains fails again (a common occurrence right after
a power outage) configuration will lose power without a chance to shutdown.
There is a risky situation here. It's discussed here networkupstools.org

I think I won't set QNAP to start on power on [Control Panel -> Power -> Power
Recovery], as this leaves it exposed to starting up while power is crucially
low; then have the power cut again, because battery lacks reserves during
boot.

I have written a script shepherd_nas.sh which uses Wake On LAN to wake
the QNAP once the battery has enough charge stored. So if I set the low limit
high enough on the UPS (e.g. 25%) it should be able to cope with the situation
where the power comes back, the battery charges a tiny fraction to say 26%, the
power fails again, the Zotac shutdown begins again at 25% probably finishes by
24% . This cycle could continue a dozen or more times before the UPS would be
exhausted.
You do not have the required permissions to view the files attached to this post.
User avatar
moody_blue
Easy as a breeze
Posts: 266
Joined: Tue Jan 10, 2017 9:23 am

UPS control of small configuation

Post by moody_blue »

graemev wrote: Mon May 17, 2021 10:58 pm In version 2. I've switched the cables around, so the USB cable from the UPS is plugged into the Zotac. This will mean we make the Zotac the "master" from the viewpoint of the UPS software.
I have one APC BE700-GR. My connections are similar to your version 2, although using different devices:

1) Router is Raspi4 running OpenWRT, nut-server and nut-monitor processes. The UPS is connected to a USB2 port and managed by nut-server. nut-monitor manages Raspi4 and UPS shutdown
2) QNAP is TS-253A running QTS 4.5.4. UPS is defined as "Network UPS slave". QNAP also runs OpenHab (an IoT tool) that also gets the UPS status (mainly to shutdown zigbee lights that power on when power resumes after a power fault)
3) A laptop running Windows and this tool to monitor UPS status

UPS provides power to Raspi4, QNAP and some network devices. Batteries @ full charge last for approx 40 mins. QNAP takes 2-3 mins to shutdown and 5-7 mins to boot. This is the shutdown sequence when there is a power failure:

1) QNAP initiates shutdown 10 minutes after fault is detected or 30 secs @ 15% battery level, whatever occurs first. This assures enough power during the 2-3 minutes required for shutdown. The 10 mins are user parametrizable but the 15% battery level seems to be hardcoded in QTS
2) Raspi4 initiates shutdown after fault is detected and battery battery level drops to 10% (this is parametrizable). It also orders the UPS to shutdown 1 minute later

When power is resumed Raspi4 is powered on automatically (but not the QNAP). There is a cron process running every 20 mins in OpenWRT, when battery level is >30% and ups is online wol packets are sent to both QNAP nic's. This assures enough power for a complete power on / power off cycle if needed, and protects the QNAP against unstable power conditions (very rare, but possible).

Here are my control files in the Raspi4:

Code: Select all

root@rpi-aveiro-router /52# cat /etc/custom/qnap.sh
#!/bin/sh
if [ `upsc qnapups battery.charge` > 30 ] && [ `upsc qnapups ups.status` = "OL" ]; then
        etherwake 24:5e:be:0b:81:28
        etherwake 24:5e:be:0b:81:29

root@rpi-aveiro-router /etc/nut54# cat ups.conf
# Config file automatically generated from UCI config
[qnapups]
driver = usbhid-ups
port = auto

root@rpi-aveiro-router /etc/nut55# cat upsd.conf
# Config file automatically generated from UCI config
LISTEN ::1
LISTEN 192.168.130.1
MAXAGE 15
STATEPATH /var/run/nut
MAXCONN 10

root@rpi-aveiro-router /etc/nut54# cat upsd.users
# Config file automatically generated from UCI config
[admin]
  password = 123456
  upsmon master
And in the QNAP
ups.png
I would love to implement coordinated shutdown between Raspi4 and QNAP. NUT allows this but I did not find a find a way to have QNAP enter in such a scheme. More info in this guide
You do not have the required permissions to view the files attached to this post.
QNAP TS-253A 8G QTS 5.0.1.2145
Plex Media Server 1.29.0.6209
OpenHAB 3.4.0.M2
Unifi 7.2.92
Apache80 2454.8230
GLPI 10.0.3
Locked

Return to “Hardware & Software Compatibility”