System Volume full (no clue why)

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
catalin
New here
Posts: 4
Joined: Mon Aug 21, 2017 9:08 pm

System Volume full (no clue why)

Post by catalin »

Hello everybody,

I encounter a very nasty error - system volume is full - even if I use no more than 200Gb from 2.6Tb with my personal data.
It looks like my system files are 'growing' day by day but I was not able to identify the reason.

Had a look on this forum, a lot of similar topics but different root causes, so I'm attaching few data from my ssh session. Maybe some of you experienced similar situation or have a better understanding of the topic than me.
Please be aware that I disabled and empty the Network Recycle bin already.

Thank you in advance for your availability for my troubleshooting.

Br,
Catalin

Code: Select all

[/share/snapshot/export] # echo "NAS Model: $(getsysinfo model)"
NAS Model: TS-453A
[/share/snapshot/export] # echo "Firmware: $(getcfg system version)"
Firmware: 4.3.3
[/share/snapshot/export] # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md3 : active raid1 sdd3[0]
      2920311616 blocks super 1.0 [1/1] [U]

md2 : active raid1 sda3[0] sdb3[1]
      2920311616 blocks super 1.0 [2/2] [UU]

md1 : active raid1 sdc3[0]
      2920311616 blocks super 1.0 [1/1] [U]

md256 : active raid1 sdd2[3](S) sdc2[2](S) sdb2[1] sda2[0]
      530112 blocks super 1.0 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md13 : active raid1 sda4[0] sdd4[25] sdb4[24] sdc4[1]
      458880 blocks super 1.0 [24/4] [UUUU____________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md9 : active raid1 sda1[0] sdd1[25] sdb1[24] sdc1[1]
      530048 blocks super 1.0 [24/4] [UUUU____________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

unused devices: <none>
[/share/snapshot/export] # df -h
Filesystem                Size      Used Available Use% Mounted on
none                    250.0M    193.6M     56.4M  77% /
devtmpfs                  7.8G      8.0K      7.8G   0% /dev
tmpfs                    64.0M    960.0K     63.1M   1% /tmp
tmpfs                     7.8G     56.0K      7.8G   0% /dev/shm
tmpfs                    16.0M         0     16.0M   0% /share
tmpfs                    16.0M         0     16.0M   0% /share/snapshot/export
/dev/md9                493.5M    140.5M    352.9M  28% /mnt/HDA_ROOT
cgroup_root               7.8G         0      7.8G   0% /sys/fs/cgroup
/dev/mapper/cachedev1
                          2.6T      2.6T     23.2G  99% /share/CACHEDEV1_DATA
/dev/mapper/cachedev2
                          2.6T    979.8G      1.7T  36% /share/CACHEDEV2_DATA
/dev/mapper/cachedev3
                          2.5T      1.8T    655.6G  74% /share/CACHEDEV3_DATA
/dev/md13               355.0M    343.7M     11.3M  97% /mnt/ext
CloudDrive                2.0M      2.0M      1.0M  67% /mnt/rf/cd
tmpfs                    16.0M    144.0K     15.9M   1% /share/CACHEDEV1_DATA/.samba/lock/msg.lock
tmpfs                    16.0M         0     16.0M   0% /mnt/ext/opt/samba/private/msg.sock
tmpfs                     1.0M         0      1.0M   0% /mnt/rf/nd
tmpfs                    64.0M    960.0K     63.1M   1% /share/CACHEDEV1_DATA/.qpkg/HD_Station/tmp
/dev/md9                493.5M    140.5M    352.9M  28% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/config
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/home/httpd/RSS/images
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/init.d
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/rcS.d
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/rcK.d
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/default_config
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/platform.conf
none                    250.0M    193.6M     56.4M  77% /share/CACHEDEV1_DATA/.qpkg/HD_Station/etc/hal_util.conf
/dev/mapper/cachedev1
                          2.6T      2.6T     23.2G  99% /share/CACHEDEV1_DATA/.qpkg/HD_Station/share/CACHEDEV1_DATA
/dev/mapper/cachedev2
                          2.6T    979.8G      1.7T  36% /share/CACHEDEV1_DATA/.qpkg/HD_Station/share/CACHEDEV2_DATA
/dev/mapper/cachedev3
                          2.5T      1.8T    655.6G  74% /share/CACHEDEV1_DATA/.qpkg/HD_Station/share/CACHEDEV3_DATA
none                      7.8G         0      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/sys/fs/cgroup
udev                      7.8G      8.0K      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/dev
tmpfs                     7.8G    864.0K      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/run
none                      7.8G         0      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/run/shm
none                      7.8G         0      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/run/lock
none                      7.8G         0      7.8G   0% /share/CACHEDEV1_DATA/.qpkg/HD_Station/run/user
/dev/mapper/cachedev1
                          2.6T      2.6T     23.2G  99% /lib/modules/4.2.8/container-station
/dev/mapper/cachedev1
                          2.6T      2.6T     23.2G  99% /share/CACHEDEV1_DATA/.qpkg/container-station/system-docker/devicemapper
/dev/mapper/docker-252:0-121904489-095750efd26ee286afef59afd6673141fc57c56c9a42f51285bef58a6a6a2b28
                          9.8G    653.3M      8.7G   7% /share/CACHEDEV1_DATA/.qpkg/container-station/system-docker/devicemapper/mnt/095750efd26ee286afef59afd6673141fc57c56c9a42f51285bef58a6a6a2b28
shm                      64.0M         0     64.0M   0% /share/CACHEDEV1_DATA/.qpkg/container-station/system-docker/containers/5067368d8ea76b06fbd0fff715e6dc23e8c983447ae2ce4a686c74d3041db54e/shm


[/share/snapshot/export] # ls -alFh /share/CACHEDEV1_DATA
total 216
drwxrwxrwx   42 admin    administ    4.0K Aug 18 21:06 ./
drwxrwxrwt   34 admin    administ     920 Aug 18 21:06 ../
drwxr-xr-x    2 admin    administ    4.0K Aug 21 08:04 .@analytic/
drwxr-xr-x    2 admin    administ    4.0K Aug 16 03:04 .@backup_config/
drwxr-xr-x    2 admin    administ    4.0K Aug 16 03:04 .@backup_qbox/
drwxr-xr-x    2 admin    administ    4.0K Aug 18 18:22 .@backup_qfiling/
drwxr-xr-x    4 admin    administ    4.0K May 22  2016 .@qmariadb/
drwx------    4 admin    administ    4.0K Aug 18 18:21 .@qmonitor/
drwx------    4 admin    administ    4.0K Aug 19 07:02 .@qsync/
drwxr-xr-x    9 admin    administ    4.0K Feb 28 22:12 .@twonkymedia.db/
drwxr-xr-x    3 admin    administ    4.0K May 23  2016 .QKVM/
drwxr-xr-x    6 admin    administ    4.0K May 24  2016 .antivirus/
drwxr-xr-x    3 admin    administ    4.0K May 22  2016 .appDB/
drwxr-xr-x    2 admin    administ    4.0K May 22  2016 .idmap/
drwxr-xr-x    2 admin    administ    4.0K May 22  2016 .ldapdb/
drwxr-xr-x    2 admin    administ    4.0K Aug 18 18:11 .locks/
drwxr-xr-x    2 admin    administ    4.0K May 21  2016 .log/
drwxrwxrwx    2 admin    administ    4.0K Feb 28 23:15 .magick_tmp/
drwxr-xr-x    3 admin    administ    4.0K May 23  2016 .nvr/
drwxrwxrwx    2 admin    administ    4.0K Aug 18 18:20 .php_session/
drwxr-xr-x    4 admin    administ    4.0K Aug 18 21:08 .php_session_sys/
drwxr-xr-x    3 admin    administ    4.0K Jun  7 22:14 .qbox/
drwxrwxrwx   88 admin    administ    4.0K Aug 18 18:54 .qpkg/
drwxr-xr-x    6 admin    administ    4.0K Aug 18 18:11 .samba/
drwxrwxrwx    5 admin    administ    4.0K Aug 18 18:11 .spool/
d---------    2 admin    administ    4.0K Aug 13 17:19 .swap/
drwxr-xr-x   10 admin    administ    4.0K Aug 18 18:13 .system/
drwxrwxrwx    3 admin    administ    4.0K Feb 28 22:53 .team_folder/
drwxrwxrwx    2 admin    administ    4.0K Feb 28 23:17 .tmp/
drwxr-xr-x    3 admin    administ    4.0K Aug 18 18:13 .torrent/
drwxrwxrwx    2 admin    administ    4.0K May 21  2016 .versioning/
drwxrwxrwx    5 admin    administ    4.0K Aug 19 07:04 Back-up/
drwxrwxrwx    2 admin    administ    4.0K Aug 16 20:11 Download/
drwxrwxrwx    3 admin    administ    4.0K Aug 16 20:11 Multimedia/
drwxrwxrwx    2 admin    administ    4.0K May 21  2016 OceanKTV/
drwxrwxrwx    4 admin    administ    4.0K Aug 16 20:11 Public/
drwxrwxrwx    2 admin    administ    4.0K Nov 28  2016 QmailAgent/
drwxrwxrwx    3 admin    administ    4.0K Aug 16 20:11 Recordings/
drwxrwxrwx    4 admin    administ    4.0K Aug 16 20:11 WM_W7/
drwxrwxrwx    2 admin    administ    4.0K Aug 18 21:19 Web/
-rw-------    1 admin    administ    9.0K May 26 18:21 aquota.user
drwxrwxrwx    7 admin    administ    4.0K Aug 16 20:11 homes/
drwx------    2 admin    administ   16.0K May 22  2016 lost+found/
lrwxrwxrwx    1 admin    administ       9 May 23  2016 optware -> /dev/null
User avatar
schumaku
Guru
Posts: 43578
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: System Volume full (no clue why)

Post by schumaku »

During what operation you get this System Volume Full warning?
catalin
New here
Posts: 4
Joined: Mon Aug 21, 2017 9:08 pm

Re: System Volume full (no clue why)

Post by catalin »

In webUI and on NAS front panel.


Sent from my iPhone using Tapatalk
User avatar
Trexx
Ask me anything
Posts: 5388
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: System Volume full (no clue why)

Post by Trexx »

Run the following:

Code: Select all

du -h -d 1 /share/CACHEDEV1_DATA
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
catalin
New here
Posts: 4
Joined: Mon Aug 21, 2017 9:08 pm

Re: System Volume full (no clue why)

Post by catalin »

Hi Trexx,

Thank you for your hint.
After a long waiting time, I notice that Hybrid Deskstation could be the issue. So I decided to remove it. BINGO!!!!

So now it is showing the right size: 2.4TB.

I'll try to reinstall HD, reinstall Kodi, rebuild the library ... and hopefully the disk usage will be in the reasonable range, no more than 2.0 TB.

Thx,
Catalin
User avatar
Trexx
Ask me anything
Posts: 5388
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: System Volume full (no clue why)

Post by Trexx »

catalin wrote:Hi Trexx,

Thank you for your hint.
After a long waiting time, I notice that Hybrid Deskstation could be the issue. So I decided to remove it. BINGO!!!!

So now it is showing the right size: 2.4TB.

I'll try to reinstall HD, reinstall Kodi, rebuild the library ... and hopefully the disk usage will be in the reasonable range, no more than 2.0 TB.

Thx,
Catalin
Depends on the KODI add-ons, the icon/artwork caching/etc. It can ** up some space if you aren't careful.
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
catalin
New here
Posts: 4
Joined: Mon Aug 21, 2017 9:08 pm

Re: System Volume full (no clue why)

Post by catalin »

Trexx wrote:
catalin wrote:Hi Trexx,

Thank you for your hint.
After a long waiting time, I notice that Hybrid Deskstation could be the issue. So I decided to remove it. BINGO!!!!

So now it is showing the right size: 2.4TB.

I'll try to reinstall HD, reinstall Kodi, rebuild the library ... and hopefully the disk usage will be in the reasonable range, no more than 2.0 TB.

Thx,
Catalin
Depends on the KODI add-ons, the icon/artwork caching/etc. It can ** up some space if you aren't careful.
Thx, but do you think it can reach more than 2TB?

I think that HD upgrade mess the files and folder, du command was not able to read some files and folders.

Let's see the outcome of the HD rebuild.


Sent from my iPhone using Tapatalk
Post Reply

Return to “System & Disk Volume Management”