Want to schedule delete files and folders on attached USB drive.

[sharper9]

Things have moved on since the original USB backup plan. Underlined by your latest idea.
I have created much more space on my current main NAS harddrive since this backup drive idea was deemed at first necessary.

I'll sleep on it and give it a think.....a 1tb USB backup is a very small backup and is/was only meant to be temporary.

Till tomorrow

[sharper9]

This was a new nas to accomodate the CCTV cameras and due to other economic calls at the time, I have only put a single 3tb drive in the two bay TS251A.
At first my existing data music/photos/cctv etc ate the 3tb drive space all but 300gb so I had a problem. I was getting warning from the QNAP at that time.
So I at first I cut down the amount of CCTV retained on the qnap and pulled as much as possible off to 'mysmalldisk' MSD.
Since then I have fine tuned the 3tb and freed up about 1TB ....while this went on I with your considerable help have been attempting to auto delete the backup on to 'MSD'.



Thing have moved on so I'm open to your ideas. I want to keep 500gb free on the nas.
ATM I keep 7 days CCTV on the NAS and my 'original aim' was to have 7 days on the NAS + say 11 days on MSD giving me a total of an 18 day archive.
I can up the CCTV from 7 to 10 day maybe on the nas (21 days in total). But I can't find a way of placing the further 11 days on 'MSD' in chronological order due the the restrains of Backup ST. (10 days NAS + 11 days on MSD = 21 days)

Hope that made sense.

Thoughts ?

[OneCD]

I can up the CCTV from 7 to 10 day maybe on the nas (21 days in total). But I can't find a way of placing the further 11 days on 'MSD' in chronological order due the the restrains of Backup ST. (10 days NAS + 11 days on MSD = 21 days)

Hope that made sense.

It does - I can see what you'd like to do - I'm trying to think of a simple way to do it.

Nothing has come to mind yet...

[Spider99]

what you are asking for is a routine that looks at a folder and counts the contained sub directories within that folder and if its greater than x move the oldest folders to the backup drive?

moving folders older than x is the easy bit

I would guess our wizzo friend knows a simple linux command to count folders and not files?

[OneCD]

I would guess our wizzo friend knows a simple linux command to count folders and not files?

That's crazy talk! There's no possible way to get a computer to count things...

Good idea. I'll write something up this week, just need time to consider the angles.

[Spider99]

yes careful of the corners

[OneCD]

Quick update. Just experimenting with ways to get a directory list as per Tim's suggestion:

Code: Select all

[/share] # acc=0; for dl in $(ls -dt /share/*); do [ -d "$dl" ] && { ((acc++)); [ "$acc" -gt 10 ] && echo $dl ;} ;done
/share/HDS_DATA/
/share/HDR_DATA/
/share/HDQ_DATA/
/share/HDP_DATA/
/share/HDO_DATA/
/share/HDN_DATA/
/share/HDM_DATA/
/share/HDL_DATA/
/share/HDK_DATA/
/share/HDJ_DATA/
/share/HDI_DATA/
/share/HDH_DATA/
/share/HDG_DATA/
/share/HDF_DATA/
/share/HDE_DATA/
/share/HDD_DATA/
/share/HDC_DATA/
/share/HDB_DATA/

This sorts selected directory (/share in my example) by modification datetime, only lists directories, ignores the first 10 directories and displays the paths of the remainder. Easy enough to move them out to an external drive. (And best I could come up with in 5 minutes.)

After they're moved, we should then run a similar check on the external drive only this time, ignore the first 11 directories, and delete the remainder, which would be the oldest ones.

If no-one can see a problem with the logic, I'll write this into a more user-friendly script with environment and validation checks.

[Spider99]

@OneCD

which "mod time" are you reading?

As a file could be modified within on of the directories - i guess like windows that changes the modified time of the directory - and would mess up the list and delete the wrong .........?

would it not be better to use the created time of the directory as i believe that the directory is created by the CCTV app each day?????

[OneCD]

which "mod time" are you reading?

There's only one. 'mtime'.

As a file could be modified within on of the directories - i guess like windows that changes the modified time of the directory - and would mess up the list and delete the wrong .........?

Yes, in my testing, if a file is modified in a directory, this changes the modtime of that directory too.

would it not be better to use the created time of the directory as i believe that the directory is created by the CCTV app each day?????

Creation datetimes are not stored. Only 'atime', 'ctime' and 'mtime'. You can run a 'stat' on one of your files or directories to get an idea.

I'm trusting that once the camera recording software has written files for a single day to each directory (as OP has shown), that it won't go back and write more stuff to that directory (which would change the modtime).

So, it means that if a directory was created on 30/2/2017 ( ) at 12:00am, and was written into all day, at the end of the day, the directory will have a moddate of 30/2/2017 and with a modtime of 11:59:59pm (at the latest). This is no bad thing.

From my various 'thought-experiments', I predict that we'll start with a mix of directories being moved and deleted, but after the total cycle-time has elapsed (21 days), things will have settled down and the automated processes will keep the correct info.

Should also add that the script delete-old-stuff.sh will not be required.

Thoughts?

[Spider99]

ok fine
out of interest how does windows explorer show created date on a linux share for files and directories?

[OneCD]

You're asking the Linux fanboy how Windows does something?

[Spider99]

well the data is stored somewhere and it wont be on the client pc so....

i'm asking how linux provides the info to windows?

[OneCD]

Not a clue. It's possibly a case of Windows interpreting directory info as a 'created' datetime when it's not.

edit: found this: http://serverfault.com/questions/382552 ... s-vs-linux

[sharper9]

Morning gentlemen.

To say most of your excellent endeavours are going over my head due to my very basic IT knowledge is an understatement.
But I get the gist of the thought processes.
I can only thank you.

I'm watching.....

[Spider99]

oops been rumbled....