Want to schedule delete files and folders on attached USB drive.

Introduce yourself to us and other members here, or share your own product reviews, suggestions, and tips and tricks of using QNAP products.
Locked
User avatar
Spider99
Experience counts
Posts: 1951
Joined: Fri Oct 21, 2011 11:14 pm
Location: UK

Re: Want to schedule delete files and folders on attached USB drive.

Post by Spider99 »

OneCD wrote:Not a clue. It's possibly a case of Windows interpreting directory info as a 'created' datetime when it's not. :geek:

edit: found this: http://serverfault.com/questions/382552 ... s-vs-linux
interesting implies ext4 can store extra info for samba to present to windows that linux does not use - might explain it then
Tim

TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

progress update: still developing a script to do this (archive-old-stuff.sh)

In between other projects, I'm testing various scenarios to ensure it works as required. ;)

It should be ready shortly.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

OneCD wrote:progress update: still developing a script to do this (archive-old-stuff.sh)

In between other projects, I'm testing various scenarios to ensure it works as required. ;)

It should be ready shortly.
Thanks
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

Here it is:
archive-old-stuff.gz
Please download this into your Public share, then move and rename it as follows:

Code: Select all

mv /share/Public/archive-old-stuff.gz /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
Let's mark it as executable:

Code: Select all

chmod +x /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
You can also remove delete-old-stuff.sh as it's not needed anymore:

Code: Select all

rm /share/50014EE20BA4E444_DATA/.system/delete-old-stuff.sh
Then it's time to edit your new script:

Code: Select all

nano /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
Jump straight to the bottom and look for the line that says:

Code: Select all

ArchiveOldDirs "/share/Recordings/Camera/channel1" 10 "/share/MySmallDisk/channel1" 11
The paths will most likely need to be changed.
  1. The source path (the first one) is where your camera recordings are stored. You know where this is - I don't. I suggest you specify the exact channel as shown in the example. ;)
  2. The number 10 is the minimum number of folders (directories) to keep at this location. Every older folder in excess of this will be moved to the target location (the USB drive).
  3. The second path is where you want the videos to be placed (on your USB drive). I think this one will be correct based on your previous posts.
  4. The last number is the minimum number of folders to keep at the target location (your USB drive). Everything in excess of this will be deleted.
Adjust these as required, then save and exit the editor.

To run this script:

Code: Select all

/share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
But, as before, when you run this, it's a "dry-run". Nothing will be moved or deleted. It's a chance for you to examine what would happen. If you're happy with the lists, then edit the script, and change the dryrun value to 'false'

You may have read in an earlier post, my prediction is that as first, it will not copy and delete the correct folders until your full cycle time has elapsed (21 days). This is due to the modification times on your source folders possibly being affected by local user operations. ;)

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

Wow! I'll get to this today!

Thank you
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

Will I not need to keep 11 days CCTV on the NAS for this to work.
May of course be talking bollards!
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

You can choose whatever figures you feel comfortable with for the NAS and for your USB drive. Each can be different. I programmed in an upper limit of 100 though. ;)

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

Code: Select all

[~] # mv /share/Public/archive-old-stuff.gz /share/50014EE20BA4E444_DATA/.system          /archive-old-stuff.sh
[~] #
[~] #
[~] #
[~] #
[~] # chmod +x /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
[~] # rm /share/50014EE20BA4E444_DATA/.system/delete-old-stuff.sh
[~] # nano /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
[~] # /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh

* dryrun only! *

- excess directories that exceed the minimum of 10 in [/share/Recordings/record_          nvr/channel1]
- These would be moved to [/share/MySmallDisk/channel1]

- old directories that exceed the minimum of 11 in [/share/MySmallDisk/channel1]
- These would be deleted
[~] # [~] # chmod +x /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
-sh: [~]: command not found
[~] # [~] # rm /share/50014EE20BA4E444_DATA/.system/delete-old-stuff.sh
-sh: [~]: command not found
[~] # [~] # nano /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
-sh: [~]: command not found
[~] # [~] # /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh
-sh: [~]: command not found
[~] #
[~] # * dryrun only! *
-sh: index_default.html: command not found
[~] #
[~] # - excess directories that exceed the minimum of 10 in [/share/Recordings/record_ nvr/channel1]
-sh: -: command not found
[~] # - These would be moved to [/share/MySmallDisk/channel1]
-sh: -: command not found
[~] #
[~] # - old directories that exceed the minimum of 11 in [/share/MySmallDisk/channel1]
-sh: -: command not found
[~] # - These would be deleted
Not quite sure what happened?
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

sharper9 wrote:

Code: Select all

[~] # /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh

* dryrun only! *

- excess directories that exceed the minimum of 10 in [/share/Recordings/record_          nvr/channel1]
- These would be moved to [/share/MySmallDisk/channel1]

- old directories that exceed the minimum of 11 in [/share/MySmallDisk/channel1]
- These would be deleted
Not quite sure what happened?
Let's check your source directory:

Code: Select all

ls -la /share/Recordings/record_nvr/channel1/
And your target directory:

Code: Select all

ls -la /share/MySmallDisk/channel1/

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

Code: Select all

[~] # ls -la /share/Recordings/record_nvr/channel1/
drwxr-xr-x   10 admin    administ      4096 Mar 22 00:05 ./
drwxr-xr-x    5 admin    administ      4096 Jan 26 15:25 ../
drwxr-xr-x    2 admin    administ      4096 Jan 26 15:25 .nvr/
drwxr-xr-x   26 admin    administ      4096 Mar 22 00:05 2017-03-16/
drwxr-xr-x   27 admin    administ      4096 Mar 17 23:05 2017-03-17/
drwxr-xr-x   27 admin    administ      4096 Mar 18 23:05 2017-03-18/
drwxr-xr-x   27 admin    administ      4096 Mar 19 23:05 2017-03-19/
drwxr-xr-x   27 admin    administ      4096 Mar 20 23:05 2017-03-20/
drwxr-xr-x   27 admin    administ      4096 Mar 21 23:05 2017-03-21/
drwxr-xr-x   21 admin    administ      4096 Mar 22 17:05 2017-03-22/
[~] #
AND

Code: Select all

[~] # ls -la /share/MySmallDisk/channel1/
drwxr-xr-x   13 admin    administ      4096 Mar 22 00:05 ./
drwxrwxrwx    6 admin    administ      4096 Mar  3 16:02 ../
drwxr-xr-x    2 admin    administ      4096 Jan 26 15:25 .nvr/
drwxr-xr-x   26 admin    administ      4096 Mar 19 00:05 2017-03-13/
drwxr-xr-x   27 admin    administ      4096 Mar 20 00:06 2017-03-14/
drwxr-xr-x   27 admin    administ      4096 Mar 21 00:06 2017-03-15/
drwxr-xr-x   27 admin    administ      4096 Mar 22 00:05 2017-03-16/
drwxr-xr-x   27 admin    administ      4096 Mar 17 23:05 2017-03-17/
drwxr-xr-x   27 admin    administ      4096 Mar 18 23:05 2017-03-18/
drwxr-xr-x   27 admin    administ      4096 Mar 19 23:05 2017-03-19/
drwxr-xr-x   27 admin    administ      4096 Mar 20 23:05 2017-03-20/
drwxr-xr-x   27 admin    administ      4096 Mar 21 23:05 2017-03-21/
drwxr-xr-x    5 admin    administ      4096 Mar 22 01:05 2017-03-22/
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

Looks good! :)

To explain: The script didn't show anything to move from your NAS or to delete from your USB drive as nothing there currently qualifies.

At the moment, you only have 7 directories on your NAS. None will be moved until there are more than 10.

On your USB drive, you also have only 10 directories. None will be deleted until there are more than 11.

Wait a few days then try running the script again. ;)

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

OneCD wrote:Looks good! :)

To explain: The script didn't show anything to move from your NAS or to delete from your USB drive as nothing there currently qualifies.

At the moment, you only have 7 directories on your NAS. None will be moved until there are more than 10.

On your USB drive, you also have only 10 directories. None will be deleted until there are more than 11.

Wait a few days then try running the script again. ;)
So'
I need to put a path in for the other recordings on QNAP 'channel2'?
and then make it live?

anything else?

Thx
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
OneCD
Guru
Posts: 12037
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Want to schedule delete files and folders on attached USB drive.

Post by OneCD »

sharper9 wrote:I need to put a path in for the other recordings on QNAP 'channel2'?
Yes, add a new line to the bottom of archive-old-stuff.sh script. Please ensure the last line of the file is empty.

So, the end of that script should look like:

Code: Select all

ArchiveOldDirs "/share/Recordings/record_nvr/channel1" 10 "/share/MySmallDisk/channel1" 11
ArchiveOldDirs "/share/Recordings/record_nvr/channel2" 10 "/share/MySmallDisk/channel2" 11

sharper9 wrote:and then make it live?
I guess that means disabling the 'safety'. I think you should wait until you can verify the directories first. ;)
sharper9 wrote:anything else?
Once you know it runs correctly, then it's back to modifying your crontab. You'll be able to edit the line for delete-old-stuff.sh and change this to archive-old-stuff.sh

edit: corrected paths
Last edited by OneCD on Thu Mar 23, 2017 2:17 am, edited 1 time in total.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

I'll sort this in the morning.

Thanks again
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
User avatar
sharper9
Know my way around
Posts: 196
Joined: Fri Jun 06, 2008 6:10 pm

Re: Want to schedule delete files and folders on attached USB drive.

Post by sharper9 »

Code: Select all

[~] # /share/50014EE20BA4E444_DATA/.system/archive-old-stuff.sh

* dryrun only! *

- excess directories that exceed the minimum of 10 in [/share/Recordings/record_nvr/channel1]
- These would be moved to [/share/MySmallDisk/channel1]

- old directories that exceed the minimum of 11 in [/share/MySmallDisk/channel1]
- These would be deleted

* dryrun only! *

- excess directories that exceed the minimum of 10 in [/share/Recordings/record_nvr/channel2]
- These would be moved to [/share/MySmallDisk/channel2]

- old directories that exceed the minimum of 11 in [/share/MySmallDisk/channel2]
- These would be deleted
This look OK?
All the best

TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
Locked

Return to “Users' Corner”