Want to schedule delete files and folders on attached USB drive.
- sharper9
- Know my way around
- Posts: 196
- Joined: Fri Jun 06, 2008 6:10 pm
Re: Want to schedule delete files and folders on attached USB drive.
Things have moved on since the original USB backup plan. Underlined by your latest idea.
I have created much more space on my current main NAS harddrive since this backup drive idea was deemed at first necessary.
I'll sleep on it and give it a think.....a 1tb USB backup is a very small backup and is/was only meant to be temporary.
Till tomorrow
I have created much more space on my current main NAS harddrive since this backup drive idea was deemed at first necessary.
I'll sleep on it and give it a think.....a 1tb USB backup is a very small backup and is/was only meant to be temporary.
Till tomorrow
All the best
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
- sharper9
- Know my way around
- Posts: 196
- Joined: Fri Jun 06, 2008 6:10 pm
Re: Want to schedule delete files and folders on attached USB drive.
This was a new nas to accomodate the CCTV cameras and due to other economic calls at the time, I have only put a single 3tb drive in the two bay TS251A.
At first my existing data music/photos/cctv etc ate the 3tb drive space all but 300gb so I had a problem. I was getting warning from the QNAP at that time.
So I at first I cut down the amount of CCTV retained on the qnap and pulled as much as possible off to 'mysmalldisk' MSD.
Since then I have fine tuned the 3tb and freed up about 1TB ....while this went on I with your considerable help have been attempting to auto delete the backup on to 'MSD'.
Thing have moved on so I'm open to your ideas. I want to keep 500gb free on the nas.
ATM I keep 7 days CCTV on the NAS and my 'original aim' was to have 7 days on the NAS + say 11 days on MSD giving me a total of an 18 day archive.
I can up the CCTV from 7 to 10 day maybe on the nas (21 days in total). But I can't find a way of placing the further 11 days on 'MSD' in chronological order due the the restrains of Backup ST. (10 days NAS + 11 days on MSD = 21 days)
Hope that made sense.
Thoughts ?
At first my existing data music/photos/cctv etc ate the 3tb drive space all but 300gb so I had a problem. I was getting warning from the QNAP at that time.
So I at first I cut down the amount of CCTV retained on the qnap and pulled as much as possible off to 'mysmalldisk' MSD.
Since then I have fine tuned the 3tb and freed up about 1TB ....while this went on I with your considerable help have been attempting to auto delete the backup on to 'MSD'.
Thing have moved on so I'm open to your ideas. I want to keep 500gb free on the nas.
ATM I keep 7 days CCTV on the NAS and my 'original aim' was to have 7 days on the NAS + say 11 days on MSD giving me a total of an 18 day archive.
I can up the CCTV from 7 to 10 day maybe on the nas (21 days in total). But I can't find a way of placing the further 11 days on 'MSD' in chronological order due the the restrains of Backup ST. (10 days NAS + 11 days on MSD = 21 days)
Hope that made sense.
Thoughts ?
All the best
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
It does - I can see what you'd like to do - I'm trying to think of a simple way to do it.sharper9 wrote:I can up the CCTV from 7 to 10 day maybe on the nas (21 days in total). But I can't find a way of placing the further 11 days on 'MSD' in chronological order due the the restrains of Backup ST. (10 days NAS + 11 days on MSD = 21 days)
Hope that made sense.
Nothing has come to mind yet...
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
what you are asking for is a routine that looks at a folder and counts the contained sub directories within that folder and if its greater than x move the oldest folders to the backup drive?
moving folders older than x is the easy bit
I would guess our wizzo friend knows a simple linux command to count folders and not files?
moving folders older than x is the easy bit
I would guess our wizzo friend knows a simple linux command to count folders and not files?
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
That's crazy talk! There's no possible way to get a computer to count things...Spider99 wrote:I would guess our wizzo friend knows a simple linux command to count folders and not files?
Good idea. I'll write something up this week, just need time to consider the angles.
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
yes careful of the corners
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
Quick update. Just experimenting with ways to get a directory list as per Tim's suggestion:
This sorts selected directory (/share in my example) by modification datetime, only lists directories, ignores the first 10 directories and displays the paths of the remainder. Easy enough to move them out to an external drive. (And best I could come up with in 5 minutes.)
After they're moved, we should then run a similar check on the external drive only this time, ignore the first 11 directories, and delete the remainder, which would be the oldest ones.
If no-one can see a problem with the logic, I'll write this into a more user-friendly script with environment and validation checks.
Code: Select all
[/share] # acc=0; for dl in $(ls -dt /share/*); do [ -d "$dl" ] && { ((acc++)); [ "$acc" -gt 10 ] && echo $dl ;} ;done
/share/HDS_DATA/
/share/HDR_DATA/
/share/HDQ_DATA/
/share/HDP_DATA/
/share/HDO_DATA/
/share/HDN_DATA/
/share/HDM_DATA/
/share/HDL_DATA/
/share/HDK_DATA/
/share/HDJ_DATA/
/share/HDI_DATA/
/share/HDH_DATA/
/share/HDG_DATA/
/share/HDF_DATA/
/share/HDE_DATA/
/share/HDD_DATA/
/share/HDC_DATA/
/share/HDB_DATA/
After they're moved, we should then run a similar check on the external drive only this time, ignore the first 11 directories, and delete the remainder, which would be the oldest ones.
If no-one can see a problem with the logic, I'll write this into a more user-friendly script with environment and validation checks.
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
@OneCD
which "mod time" are you reading?
As a file could be modified within on of the directories - i guess like windows that changes the modified time of the directory - and would mess up the list and delete the wrong .........?
would it not be better to use the created time of the directory as i believe that the directory is created by the CCTV app each day?????
which "mod time" are you reading?
As a file could be modified within on of the directories - i guess like windows that changes the modified time of the directory - and would mess up the list and delete the wrong .........?
would it not be better to use the created time of the directory as i believe that the directory is created by the CCTV app each day?????
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
There's only one. 'mtime'.Spider99 wrote:which "mod time" are you reading?
Yes, in my testing, if a file is modified in a directory, this changes the modtime of that directory too.Spider99 wrote:As a file could be modified within on of the directories - i guess like windows that changes the modified time of the directory - and would mess up the list and delete the wrong .........?
Creation datetimes are not stored. Only 'atime', 'ctime' and 'mtime'. You can run a 'stat' on one of your files or directories to get an idea.Spider99 wrote:would it not be better to use the created time of the directory as i believe that the directory is created by the CCTV app each day?????
I'm trusting that once the camera recording software has written files for a single day to each directory (as OP has shown), that it won't go back and write more stuff to that directory (which would change the modtime).
So, it means that if a directory was created on 30/2/2017 ( ) at 12:00am, and was written into all day, at the end of the day, the directory will have a moddate of 30/2/2017 and with a modtime of 11:59:59pm (at the latest). This is no bad thing.
From my various 'thought-experiments', I predict that we'll start with a mix of directories being moved and deleted, but after the total cycle-time has elapsed (21 days), things will have settled down and the automated processes will keep the correct info.
Should also add that the script delete-old-stuff.sh will not be required.
Thoughts?
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
ok fine
out of interest how does windows explorer show created date on a linux share for files and directories?
out of interest how does windows explorer show created date on a linux share for files and directories?
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
You're asking the Linux fanboy how Windows does something?
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
well the data is stored somewhere and it wont be on the client pc so....
i'm asking how linux provides the info to windows?
i'm asking how linux provides the info to windows?
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
- OneCD
- Guru
- Posts: 12143
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: Want to schedule delete files and folders on attached USB drive.
Not a clue. It's possibly a case of Windows interpreting directory info as a 'created' datetime when it's not.
edit: found this: http://serverfault.com/questions/382552 ... s-vs-linux
edit: found this: http://serverfault.com/questions/382552 ... s-vs-linux
- sharper9
- Know my way around
- Posts: 196
- Joined: Fri Jun 06, 2008 6:10 pm
Re: Want to schedule delete files and folders on attached USB drive.
Morning gentlemen.
To say most of your excellent endeavours are going over my head due to my very basic IT knowledge is an understatement.
But I get the gist of the thought processes.
I can only thank you.
I'm watching.....
To say most of your excellent endeavours are going over my head due to my very basic IT knowledge is an understatement.
But I get the gist of the thought processes.
I can only thank you.
I'm watching.....
All the best
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
TS251A 8GB 2 x 3TB JBOD for LMS and loads of other stuff. 2 Hikvision DS-2CD2342WD-I
Firmware 4.4.3.1354 - Windows 11 + 2 x iPads
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: Want to schedule delete files and folders on attached USB drive.
oops been rumbled....
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)