[HELP] Why do large .pst files allmost always fail?

[The Cat]

Am I the only one that is having problems with replicating large .pst outlook files (above 1 gig)? Everynight I have a replication job that does a remote replication of the outlook data directory. The only time this replication job works is when I delete the .pst files on the destination end. Otherwise allmost always the "Error in SSH , maybe SSH of The server is disconnect." error will pop up. Is it because .pst files get bigger everyday that it is causing this problem?

[robst618]

No, you're not the only one (but sorry, I don't have a solution).

Only yesterday I finally determined that it was .PST files around the 1GB mark have been causing my backups to fail for the past month. And that is backing up locally (remote replication to a local USB disk attached to QNAP), so there are no network latency issues involved. The only checkbox enabled is the "perform incremental" one. No SSH, no compression etc.

The error I usually get is either:
- "Remote backup schedule xyz fail. (Error in rsync protocol data stream, maybe the capacity of destination is full.)" ... even though the destination disk has 100 times more capacity available than the backup needs.

- "Remote backup schedule xyz fail. (Other error with error code 24)"

After breaking out the mail folder into a separate replication job, the main replication works fine now ... but the email job still fails. I don't know why, I don't understand rsync enough ... but I hazard a guess that the QNAP CPU, upon rsync detecting the never ending changes to a PST file day to day, has trouble analysing the large PST file bit by bit (or however rsync does it, to determine the missing parts). I noticed that when syncing a large PST for the first time, and monitoring the size of the destination directory, the destination file size increases extremely slowly ... far slower than, say, replicating 20 smaller files that total the same overall size. That's what leads me to think the QNAP has problems crawling through large binary files with rsync.

I'm working on getting my users to archive off old PST content to reduce the PST file size - not sure of any other option.

btw, the replication still fails when all the PST files are no longer in use (Outlook clients shutdown on all PCs), so it's not a file locking issue.

Oh - and you're right - "almost" always. They have completed successfully on the odd occasion.

Rob

[Don]

I too have sometimes had problems with large files (20gb). The solution that works for me is to delete the file on the destination server. After that rsync works again. I'm not sure what is causing the problem on the destination side but it works.

[The Cat]

Yep, but in this case I have to delete the .pst file almost daily (I already knew the sollution, but this should not be happening). And on three different NAS. And indeed it is not a locking error. This is something the QNAP tech guys should be looking in to.

[Don]

Yep, but in this case I have to delete the .pst file almost daily (I already knew the sollution, but this should not be happening). And on three different NAS. And indeed it is not a locking error. This is something the QNAP tech guys should be looking in to.

If it something you have to do daily (almost) then as a temporary work around you could schedule a script to run on the destination NAS that will delete the .PST file right before the rsync is scheduled to kick off. Not a permanent solution but at least it will get the .PST file copied.

FWIW, I think it does have something to do with locking. When my rsync fails it is usually on a VMWare file that was open at the time the rsync kicked off. If it fails all future rsyncs will fail on this same file until I delete it on the destination NAS. Sometimes, but rarely, a reboot of the source and destination will clear the problem also. I think something becomes corrupted and deleting the file clears it.

[lentzit]

1GB does not sound much, sound like a locking problem, i have had these to...not only on on PST files but office files...to bad the device code is not more clearly. The solution that i found is to replicate the files with Robocopy to a separate location since it does not seem to bother or maby its more tollerant.

[robst618]

The daily delete of the destination file might "work", but it's hardly a solution is it ... in that case, we may as well forget rsync for the PST files & just do a daily file copy, right?

One thing I have not tried is enabling the 'Stop network file services while replicating' checkbox in the remote replication setup screen. I suspect that would prove whether it's a file locking issue or not (assuming networked client PCs are the only consumers of the PST files, not services within the NAS, which wouldn't be affected by network services being shutdown!)

Is anyone having these same issues even using the 'Stop network file services while replicating' option enabled?

[aljimenez]

Am I the only one that is having problems with replicating large .pst outlook files (above 1 gig)? Everynight I have a replication job that does a remote replication of the outlook data directory. The only time this replication job works is when I delete the .pst files on the destination end. Otherwise allmost always the "Error in SSH , maybe SSH of The server is disconnect." error will pop up. Is it because .pst files get bigger everyday that it is causing this problem?

My experience with Outlook is that even after you exit the program, some remnant .dll files can remain running holding onto the .pst file preventing other programs from copying it. You should read about the outlook.pst file issues in Microsoft knowledge base and other forums. What you are trying to do may require some other strategy to achieve, like place the outlook.pst file for each user in the QNAP to begin with, and make the backup copies in each user's pc instead... Al

[The Cat]

The reason why I know it is not a locking problem is because the local .pst file gets copied to the nas by 3th party software (sycnback se). When the desktop shuts down at the end of the day the .pst file is copied to the NAS so at night the remote replication can do it's job and I have a backup of the last working day. So the .pst file is never used directly on the NAS. It is a copy from the file on the desktop.

[aljimenez]

The reason why I know it is not a locking problem is because the local .pst file gets copied to the nas by 3th party software (sycnback se). When the desktop shuts down at the end of the day the .pst file is copied to the NAS so at night the remote replication can do it's job and I have a backup of the last working day. So the .pst file is never used directly on the NAS. It is a copy from the file on the desktop.

I guess I misunderstood where the problem is. Are you saying the problem is NOT in copying from the pc to the QNAP unit, but rather the problem is in backing up (or copying/rsyncing) from the QNAP to another device? If the answer to my question is yes, then I misunderstood the problem, and have no ideas about what is going on to help you solve the problem... Al

[The Cat]

The problem is remote replication between two QNAP devices. It looks like the rsync protocol has problems with .pst files.