HOWTO edit /etc/init.d/rsyslog.sh

Discussion about hard drive spin down (standby) feature of NAS.
Post Reply
User avatar
graemev
Know my way around
Posts: 100
Joined: Sun Feb 12, 2012 10:17 pm

HOWTO edit /etc/init.d/rsyslog.sh

Post by graemev » Thu Apr 22, 2021 6:15 am

Like very many people here , I've been plagued by constantly busy disks.
I've written some utilities to try to track it down , but all fall foul of the same issue:

Despite my doing all my real syslog on a different box , the rsyslog daemon on the QNAP still writes to disk.

+ It does this because the file /etc/rsyslog_only_klog.conf says:

Code: Select all

$ModLoad imklog.so
$KLogPermitNonKernelFacility on
$template KernLogFormat,"%TIMESTAMP:1:10:date-rfc3339% %TIMESTAMP:12:19:date-rfc3339% %TIMESTAMP:27:$:date-rfc3339% <%PRI%> %msg:::drop-last-lf%\n"
$outchannel kmsg_rotation, /mnt/HDA_ROOT/.logs/kmsg, 2097152, /usr/sbin/logrotate /etc/config/logrotate.d/klog
kern.* $kmsg_rotation;KernLogFormat
:syslogfacility-text, isequal, "kern" ~
:msg, contains, "kmsg" ~
/etc/init.d/rsyslog.sh
However this file is created/overwritten by /etc/init.d/rsyslog.sh:

Code: Select all

		/bin/echo '$outchannel kmsg_rotation, /mnt/HDA_ROOT/.logs/kmsg,' \
Now /mnt/HDA_ROOT./.logs/kmsg is actually in the hard disk (raid):

So I'd like to change the line, to read something like:

Code: Select all

$outchannel kmsg_rotation, /var/logs/kmsg, 2097152, /usr/sbin/logrotate /etc/config/logrotate.d/klog
but , obviously if I edit /etc/init.d/rsyslog.sh it gets lost at the next reboot , and worse:

If I edit i an kill rsyslog, it'll get restarted (managed daemon) but it won't use this script again (?right?), it'll just relaunch?

I realised that when I do:

hdparm /dev/sd?

If says sda is suspended , then spins up all the drives. I'm guessing this is because the log now contains:

2021-04-21 21:17:08 +00:00 <4> [15400.446119] program hdparm is using a deprecated SCSI ioctl, please convert it to SG_I

And writing that woke the disks up ....sigh

So any attempt at trying to debug spinups needs to start by redirecting rsyslog logs off the hard disks

Mousetick
Experience counts
Posts: 1036
Joined: Thu Aug 24, 2017 10:28 pm

Re: HOWTO edit /etc/init.d/rsyslog.sh

Post by Mousetick » Fri Apr 23, 2021 5:13 am

A couple of comments:

- The log you've been fiddling with is the kernel log (klog) which is distinct from the normal syslog. So even if you send the syslog to another system, the kernel log is still logged locally (as configured by /etc/rsyslog_only_klog.conf).
- Even if you were able to, redirecting the kernel log to /var/logs would be a bad idea. /var/log is located on the root filesystem which lives in RAM and has a very small limited capacity. If you were to use this configuration permanently on a live system, it could eventually fill up the root filesystem causing all kinds of issues.

What you can do for testing purposes, is temporarily stop the rsyslog daemon.
graemev wrote:
Thu Apr 22, 2021 6:15 am
If I edit i an kill rsyslog, it'll get restarted (managed daemon) but it won't use this script again (?right?), it'll just relaunch?
As far as I can see, rsyslogd is not managed by daemon_mgr, so if you kill it it won't automatically be restarted, unless your system is different than mine. Which firmware version are you running?

A while ago I modified the blkdevMonitor script for my own use so that it would work with recent firmware versions and it would stop rsyslogd from constantly interfering with the monitoring by writing kernel events to the on-disk log. Following are the simple before/after changes I made:

Code: Select all

$ diff -Nau blkdevMonitor.sh blkdevMonitor2.sh
--- blkdevMonitor.sh	2021-01-01 08:31:28.146460638 +0100
+++ blkdevMonitor2.sh	2021-01-28 08:34:38.456861694 +0100
@@ -27,12 +27,12 @@
 Do_Log=0
 MAXRUN=100
 FORCE_STANDBY=0
-BLKDEV_LOG=/root/blkdevMonitor_v2.log
+BLKDEV_LOG=/root/blkdevMonitor.log
 MAX_DM_NUMBER=256
 
 analyse_kmsg()
 {
-	_klog=/.klog
+	_klog=/root/blkdevMonitor-klog
 	/bin/touch $_klog
 	_standby=1
 
@@ -118,16 +118,16 @@
 	;;
 esac
 
-/bin/echo -n "Stop klogd.sh daemon... "
-/sbin/daemon_mgr klogd.sh stop "/etc/init.d/klogd.sh start"
-sleep 1
-/bin/ps | /bin/grep klogd.sh | /bin/grep -v grep 1>/dev/null 2>/dev/null
+/bin/echo -n "Stop rsyslogd daemon... "
+/bin/kill `/bin/cat /var/run/rsyslogd.pid` 1>>/dev/null 2>>/dev/null
+/bin/sleep 2
+/bin/ps | /bin/grep rsyslogd | /bin/grep -v grep 1>/dev/null 2>/dev/null
 if [ $? = 0 ]; then
 	/bin/echo "Failed"
 	exit 1
 else
 	echo "Done"
-	/bin/rm -f /var/lock/subsys/klogd.sh
+  /bin/rm -f /var/run/rsyslogd.pid
 	/usr/bin/killall dd 2>/dev/null 1>/dev/null
 fi
 
@@ -184,6 +184,6 @@
 /bin/echo "Turn off block_dump"
 /bin/echo 0 > /proc/sys/vm/block_dump
 
-/bin/echo "Start klogd.sh daemon"
-/sbin/daemon_mgr klogd.sh start "/etc/init.d/klogd.sh start &"
+/bin/echo "Start rsyslogd daemon"
+/etc/init.d/rsyslog.sh start
 
For your convenience I'm also attaching the full version of the modified script. However, be forewarned: you should not execute an unknown script obtained from a stranger online unless you have carefully reviewed and understand its contents and are confident that they're safe. Nevertheless, the script may or may not work for you. Use at your own risks.
You do not have the required permissions to view the files attached to this post.

User avatar
graemev
Know my way around
Posts: 100
Joined: Sun Feb 12, 2012 10:17 pm

Re: HOWTO edit /etc/init.d/rsyslog.sh

Post by graemev » Sat Apr 24, 2021 3:34 am

Interesting. I was pretty sure it got restarted following a kill ...in fact I ended up with several rsyslog running following several kills, but I'll test this more methodically given your lead.

BTW, it would still be nice to know where /etc/init.d/rsyslogd.sh lives BEFORE it gets copied to ramdisk.

The edit I wanted to make to rsyslog.sh is very similar to yours, for the exact same reason
Your comment "...you should not execute an unknown script..." mirrors my comment In another forum , so total agreement

FYI. The issue with hdparms generating "hdparm is using a deprecated SCSI ioctl, please convert it to SG_I" the hdparm(1) comand can use the SG interface and has done so for a long while (SG interface dates from 2001) but hdparm code has an

Code: Select all

 #ifdef SG_IO
So I'm guessing Qnap have built with this undefined for some reason. I'm assuming the kernel has support hence the message

I'm planning on writing a small command to query the sleep state (using SG) say once per hour. Then, if the disks have not spun down for say an entire day, send me an email . Because right now, I sort it, the the next QNAP revision changes something (like the container debug) and I might not spot the disk have ben spinning for months.

Mousetick
Experience counts
Posts: 1036
Joined: Thu Aug 24, 2017 10:28 pm

Re: HOWTO edit /etc/init.d/rsyslog.sh

Post by Mousetick » Sat Apr 24, 2021 4:36 am

graemev wrote:
Sat Apr 24, 2021 3:34 am
BTW, it would still be nice to know where /etc/init.d/rsyslogd.sh lives BEFORE it gets copied to ramdisk.
It lives frozen in a read-only disk image of the root filesystem that gets copied to the ram-disk during the early Linux boot phase. This disk image is stored on the DOM (flash memory) as part of what's commonly referred to as the "firmware".

More details here: viewtopic.php?f=182&t=160062&p=782651#p782651

If you want to take a peek, look for a block device (e.g. /dev/sdx) that is not mounted anywhere and has ~6 partitions. You can mount each partition (ext2 filesystem type) and examine their contents. Take care to mount read-only so as to prevent any mistake that could render your device unbootable.
FYI. The issue with hdparms generating "hdparm is using a deprecated SCSI ioctl, please convert it to SG_I" the hdparm(1) comand can use the SG interface and has done so for a long while (SG interface dates from 2001) but hdparm code has an

Code: Select all

 #ifdef SG_IO
So I'm guessing Qnap have built with this undefined for some reason. I'm assuming the kernel has support hence the message
Probably. Even though hdparm is shipped with the system, I'm not sure the system actually uses this tool, so QNAP may not care.

I hope you don't mind me asking, but is the person, maybe you, shown in your avatar's picture holding an early removable spinning hard disk (aka "disk pack") from the 60/70's?

User avatar
graemev
Know my way around
Posts: 100
Joined: Sun Feb 12, 2012 10:17 pm

Re: HOWTO edit /etc/init.d/rsyslog.sh

Post by graemev » Sat Apr 24, 2021 5:46 pm

I guess I wasn't clear

Code: Select all

hdparm -C /dev/sda
Will show if a disk is "suspended" (spun down) ...however due to the #ifdef, it triggers a kernel message, due to the rsyslog setting this updates the HDD array , which, in turn, causes it to spin up , undermining the whole exercise. I've attached the scripts I'm working on for background.

The picture is me in front of an IBM 370/145 console, holding a 3330 removable disk .
I was a systems programmer on IBM 370s (158/168) in the 70s (used an IBM 360 briefly) and managed to get my very own 'massive' 3330 for system tuning.
You do not have the required permissions to view the files attached to this post.

Mousetick
Experience counts
Posts: 1036
Joined: Thu Aug 24, 2017 10:28 pm

Re: HOWTO edit /etc/init.d/rsyslog.sh

Post by Mousetick » Sat Apr 24, 2021 8:02 pm

To go back to the original topic, that is to edit /etc/init.d/rsyslog.sh, as you probably realized by now, editing it pre-boot is not really an option. I'd like to suggest a possible approach to work around that. Apologies if you'd already considered the options or are already familiar with the procedures described below.

1. During boot, it's possible to execute a custom script (more on that further below), called 'autorun.sh'. In this custom script, you could:
2. Kill rsyslogd.
3. Overwrite /etc/rsyslog_only_klog.conf with a version that writes the kernel the log to e.g. /var/log.
4. Restart rsyslogd.

For the custom boot script, see the instructions in this wiki article to install it and configure the system to enable it:
https://wiki.qnap.com/wiki/Running_Your ... at_Startup
You can use this package (made by one of the moderators of this forum) to automate the procedure:
https://github.com/OneCDOnly/create-autorun

Even if you don't use autorun.sh to tweak rsyslogd, you may find it's a useful tool for other startup customizations.

graemev wrote:
Sat Apr 24, 2021 5:46 pm
I guess I wasn't clear

Code: Select all

hdparm -C /dev/sda
Will show if a disk is "suspended" (spun down) ...however due to the #ifdef, it triggers a kernel message, due to the rsyslog setting this updates the HDD array , which, in turn, causes it to spin up , undermining the whole exercise. I've attached the scripts I'm working on for background.
Got it. Since you were wondering why hdparm was not built by QNAP with the correct #defines, I was merely speculating that QNAP doesn't care whether it's properly built or not, because this tool is not used by the system.
The picture is me in front of an IBM 370/145 console, holding a 3330 removable disk .
I was a systems programmer on IBM 370s (158/168) in the 70s (used an IBM 360 briefly) and managed to get my very own 'massive' 3330 for system tuning.
Nice, thanks for sharing. In case you're not aware of it, you may be interested in this IBM 360/370 restoration project:
https://ibms360.co.uk/

Post Reply

Return to “HDD Spin Down (HDD Standby)”