What setting to use to setup qnap as veeam backup repository

QNAP NAS solution for server virtualization and clustering/HA/FT
vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Tue Feb 25, 2014 4:36 pm

can u list your installed packages with the command

Code: Select all

ipkg list_installed


and post the result, then i can compare it with my list and maybe tell u whats different.

I remember i get a couple of warning in the vsphere installer about perl-doc, but warnings are ok. I know it is installed, and the installer just continued it just doesnt know the dpkg manager.
Is the installer not installing the vsphere package, or is it only giving you warnings and installing fine?

chres
Starting out
Posts: 10
Joined: Tue Nov 26, 2013 2:08 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby chres » Wed Feb 26, 2014 3:41 am

Hi!

Thanks for the reply!

vanbeek wrote:can u list your installed packages with the command

Code: Select all

ipkg list_installed


and post the result, then i can compare it with my list and maybe tell u whats different.


Code: Select all

[/share/MD0_DATA/.qpkg/Optware/bin] # ./ipkg list_installed
bash - 3.2.49-1 - A bourne style shell
e2fslibs - 1.41.14-1 - Ext2 Filesystem Libraries
e2fsprogs - 1.41.14-1 - Ext2 Filesystem Utilities
gdbm - 1.8.3-4 - GNU dbm is a set of database routines that use extensible hashing. It works similar to the standard UNIX dbm routines.
ipkg-web - 7-7 - A web frontend for ipkg
libdb - 4.2.52-3 - Berkeley DB Libraries
module-init-tools - 3.5-1 - This package contains a set of programs for loading, inserting, and removing kernel modules for Linux (versions 2.5.48 and abov
ncurses - 5.7-3 - NCurses libraries
openssl - 0.9.8v-2 - Openssl provides the ssl implementation in libraries libcrypto and libssl, and is needed by many other applications and librari
openssl-dev - 0.9.8v-2 - openssl native development files
perl - 5.10.0-6 - Practical Extraction and Report Language.
perl-doc - 5.10.0-6 - Documentation for perl
perl-net-ssleay - 1.30-1 - Net_SSLeay - Perl extension for using OpenSSL
readline - 6.1-2 - The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are
sudo - 1.8.4.5-1 - System utility to execute commands as the superuser
Successfully terminated.


vanbeek wrote:I remember i get a couple of warning in the vsphere installer about perl-doc, but warnings are ok. I know it is installed, and the installer just continued it just doesnt know the dpkg manager.

yeah, that's what I also thought. But...

vanbeek wrote:Is the installer not installing the vsphere package, or is it only giving you warnings and installing fine?

... it doesn't seem to install anything, cause of the not met requirements that the script forces to exit here

Code: Select all

   # Exit the installation if OpenSSL or LibXML or e2fsprogs not installed on system.
   if ( ! $OpenSSL_installed || ! $LibXML2_installed || ! $e2fsprogs_installed || ! $OpenSSL_dev_installed || ! $libxml_perl_installed ) {
      uninstall_file($gInstallerMainDB);
      exit 1;


See the following messages:

Code: Select all

...
libssl-dev 0.9.7 is required for encrypted connections.
Please install libssl-dev version 0.9.7 or greater.

sh: dpkg-query: command not found
sh: dpkg-query: command not found
e2fsprogs is not installed on the system

e2fsprogs 1.38 is required for UUID.
Please install e2fsprogs 1.38 or greater.


If the dpkg-query would be there for recognizing the installed packages and its needed versions, but that obviously can't run, as it doesn't exist, how could the installer script on your qnap run through properly? Did you maybe patch the installer.pl by pre-defining the variables with the proper version numbers etc. At least that's what I tried, but seem to have missed a few or don't know exactly what (string) to put in.

chres

vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Thu Feb 27, 2014 1:03 am

ok, i see the problem, i have made some more changes in the vmwareinstall.pl file.

And i have installed some perlmodules in cpan for an other execise. I am going to do a clean install, to check witch files are the minimum to install. or if there is an ipkg package that installs the same perl-modules.

Let me get back to u on this tomorow.

chres
Starting out
Posts: 10
Joined: Tue Nov 26, 2013 2:08 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby chres » Thu Feb 27, 2014 2:23 am

vanbeek wrote:ok, i see the problem, i have made some more changes in the vmwareinstall.pl file.

:wink:

vanbeek wrote:Let me get back to u on this tomorow.

that would be really great, tyvm

vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Thu Feb 27, 2014 5:58 pm

Ok my original post was missing some stuff, because i wasn't on a clean system.
I tested the following on a clean install of 4.0.3, Hope this helps

Link the ld command for the succesfull installation of some perl modules:

Code: Select all

cd /opt/bin
ln -s ld i686-unknown-linux-gnu-ld


i added extra package compared to my original post:

Code: Select all

ipkg install module-init-tools
ipkg install perl
ipkg install openssl-dev
ipkg install perl-doc

ipkg install perl-crypt-ssleay
ipkg install perl-compress-zlib
ipkg install perl-libxml
ipkg install libxml2
ipkg install perl-html-parser
ipkg install make
ipkg install e2fslibs-dev
ipkg install gcc


Move the cpan files to a localdisk instead of the ramdisk so perl wont run out of space.

Code: Select all

cd /root/
mv ./cpan /share/MD0_DATA/.qpkg/Optware
ln -s /share/MD0_DATA/.qpkg/Optware/.cpan .cpan


add a couple of perl packages with cpan (answer yes if cpan asks to install any dependencies:

Code: Select all

cpan
install YAML
install Crypt::SSLeay
install Archive::Zip
install Class::MethodMaker
install UUID
install XML::SAX
install XML::LibXML::Common
install SOAP::Lite
install LWP::Protocol::https
install Data::Dump



I did a compare of the vmwareinstall.pl with the original and i changed the following lines:

Line 2364 Comment the line exit

Code: Select all

#   exit 1;

Line 2472 removed the -W 4 option because the ping on the qnap doesnt support it.

Code: Select all

my $internetConnect = `ping -c 10 www.vmware.com | grep -c "64 bytes"`; 

at line 2700 i replaced the '/usr/lib' directory with an existing path on a real mounted harddisk instead of a dir in memory. for example /share/Download

Code: Select all

if ((check_disk_space('.', '/usr/lib')) < 0) {




run the vmware install script

Code: Select all

./vmware-install.pl


It tells u the depmod package is not installed, answer "yes" to enter the path manual and redirect it to "/opt/sbin/depmod"
accept the license agreement,
if it asks for where to install the files, make shure u point to a path where there is enough space (if it doesnt exists the installer will create it).
for example "/share/MD0_DATA/CLI" en the rest defaults and it installs.

if everything goes wel u get

Code: Select all

Please wait while copying vSphere CLI files...

The installation of vSphere CLI 5.5.0 build-1384587 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command:
"/share/MD0_DATA/CLI/bin/vmware-uninstall-vSphere-CLI.pl".

This installer has successfully installed both vSphere CLI and the vSphere SDK
for Perl.

Enjoy,

--the VMware team




create a startup script in the folder /share/MD0_DATA/.qpkg with the name veeamstartup.sh

Code: Select all

#!/bin/sh
ln -s /opt/bin/perl /usr/bin/perl
ln -s /share/MD0_DATA/.qpkg/Optware/.cpan .cpan


edit the qpkg.conf config file in /etc/config and add the following section to create a dummy startup package

Code: Select all

[veeamautorun]
Name = veeamautorun
Version = 0.1
Author = Me
Date = 2014-01-21
Shell = /share/MD0_DATA/.qpkg/veeamautorun/veeamautorun.sh
Install_Path = /share/MD0_DATA/.qpkg/veeamautorun
QPKG_File = veeamautorun.qpkg
Enable = FALSE


Open up the webinterface and enable the newly created veeam package under apps. This will create the symbolic link to the perl command each time the device restarts. so it will survive a machine reboot.

The QNAP is done, fully prepared to be used with veeam.

chres
Starting out
Posts: 10
Joined: Tue Nov 26, 2013 2:08 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby chres » Fri Feb 28, 2014 5:34 am

Hi vanbeek!
You did it...

vanbeek wrote:Move the cpan files to a localdisk instead of the ramdisk so perl wont run out of space.

Code: Select all

mv ./cpan /share/MD0_DATA/.qpkg/Optware

This wasn't needed as cpan was already available under /share/MD0_DATA/.qpkg/Optware/bin

vanbeek wrote:add a couple of perl packages with cpan (answer yes if cpan asks to install any dependencies:

Code: Select all

cpan
install YAML
install Crypt::SSLeay
install Archive::Zip
install Class::MethodMaker
install UUID
install XML::SAX
install XML::LibXML::Common
install SOAP::Lite
install LWP::Protocol::https
install Data::Dump

I actually did an

Code: Select all

install CPAN
as during the installation of YAML there was a hint that there's a newer version of cpan (v2.0) available, followed by

Code: Select all

reload cpan


It seems that for whatever reason I had to do

Code: Select all

install Class::MethodMaker
again at the end, as during the first run of vmware-install.pl it still seemed to be missing. Or maybe I just accidentally skipped it at first. dunno.

The following has to be modified a bit obviously (just so someone else following this thread isn't thrown of the track) or else the qpkg.conf entries won't work properly
vanbeek wrote:create a startup script in the folder /share/MD0_DATA/.qpkg/veeamautorun/ with the name veeamautorun.sh

Code: Select all

#!/bin/sh
ln -s /opt/bin/perl /usr/bin/perl
ln -s /share/MD0_DATA/.qpkg/Optware/.cpan .cpan


edit the qpkg.conf config file in /etc/config and add the following section to create a dummy startup package

Code: Select all

[veeamautorun]
Name = veeamautorun
Version = 0.1
Author = Me
Date = 2014-01-21
Shell = /share/MD0_DATA/.qpkg/veeamautorun/veeamautorun.sh
Install_Path = /share/MD0_DATA/.qpkg/veeamautorun
QPKG_File = veeamautorun.qpkg
Enable = FALSE
...


thank you very much for your time and effort.

But there's one thing left that makes me wonder. Wasn't this all to direct the traffic directly from the VM to the qnap? So, I shouldn't see all the traffic running through the veeam system?
Well somehow that wasn't the case with my test run before. The lan interface of the windows machine hosting veeam was under high load during the backup process. Did I misunderstand s.th. there?

vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Mon Mar 03, 2014 11:38 pm

But there's one thing left that makes me wonder. Wasn't this all to direct the traffic directly from the VM to the qnap? So, I shouldn't see all the traffic running through the veeam system?
Well somehow that wasn't the case with my test run before. The lan interface of the windows machine hosting veeam was under high load during the backup process. Did I misunderstand s.th. there?


Yes, that was the original idea. I have to test my network traffic. Never expected that all traffic still goes trough the server and over the network twice while the agent is running.

I never tested this, my QNAP is on the internet somewhere, so for me the most important thing was to just run it over ssh, instead of having the need to setup a vpn, and then a samba share.

I will test what it does with network traffic, and i will try to see what the VEEAMaGENT proces is doing, "its only perl" so it must be readable some how.

chres
Starting out
Posts: 10
Joined: Tue Nov 26, 2013 2:08 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby chres » Tue Mar 04, 2014 5:36 am

Ahh, ok, I thought you knew. ;)
Well, if, after some reading, I understand the veeam concept correctly, the traffic always is directed through a "backup proxy server", which by default is the main veeam machine, if you didn't deploy a dedicated 2nd (windows) machine that has direct access to the datastores. This backup proxy (or better its source-side agent) then connects directly to a (target-side) agent on the linux-based repository (e.g. our qnap). Hence with this concept and our one veeam server, you'll always see the traffic run through this single machine. There's no direct esxi -> backup repository way. Putting the veeam server onto the esxi, which you'll like to be backed up, will however route the traffic only internally and therefore result in a couple thousand MBit/s. ;)

regards
chres

vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Tue Mar 04, 2014 4:14 pm

from the veeam forums:

Code: Select all

This agent helps us to do a few things better:
- perform network traffic compression and empty block removal for all Veeam Backup operations.
- in case of incremental network backup cycle: scans VM for changed blocks locally, instead of doing this over network from Veeam Backup console.
- in case of replication: agent is mandatory on target ESX because it performs replica and rollback file processing.


That makes veeam alot less effective, then i thought in general.

vanbeek
Starting out
Posts: 14
Joined: Thu May 26, 2011 10:05 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby vanbeek » Tue Mar 04, 2014 10:46 pm

chres wrote:Ahh, ok, I thought you knew. ;)


Nope i didn't, The first tests i did with veeam where of course virtual, windows only, my network monitors never monitored the internal vmware switch.
After i switch to production, and started to use the qnap. I noticed my network traffic was almost twice the backup size. so i assumed it had to do with no agent running.
i would have implemented the agent to connect to the vmware server directly, why else they need the vmware perl sdk if they are not using it....... But my assumption was clearly wrong.

I still have found a benefit of running the agent on the qnap. The full file verification of shurebackup is done by the agent, so my qnap that it somewhere on the internet is doing the check sums, and sends back if the files are OK, without the need of sending the actual data over the internet.

newtrojan
First post
Posts: 1
Joined: Fri Jul 25, 2014 3:51 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby newtrojan » Fri Jul 25, 2014 3:56 am

excellent post but I still get the same errors when I run the vmware installs script

cat: /etc/*-release: No such file or directory
sh: dpkg-query: command not found
libssl-dev 0.9.7 is required for encrypted connections.
Please install libssl-dev version 0.9.7 or greater.

sh: dpkg-query: command not found
sh: dpkg-query: command not found
e2fsprogs is not installed on the system

e2fsprogs 1.38 is required for UUID.
Please install e2fsprogs 1.38 or greater.

No Crypt::SSLeay Perl module or linker could be found on the system. Please
either install SSLeay from your distribution or install a development toolchain
and run this installer again for encrypted connections.



I have all the packages installed via CPAN and IPKG

chres
Starting out
Posts: 10
Joined: Tue Nov 26, 2013 2:08 am

Re: What setting to use to setup qnap as veeam backup reposi

Postby chres » Fri Jul 25, 2014 4:41 am

Did you also make the necessary changes to the "vmware-install.pl" script like mentioned here:
vanbeek wrote:Line 2364 Comment the line exit

Code: Select all

#   exit 1;

Line 2472 removed the -W 4 option because the ping on the qnap doesnt support it.

Code: Select all

my $internetConnect = `ping -c 10 www.vmware.com | grep -c "64 bytes"`; 

at line 2700 i replaced the '/usr/lib' directory with an existing path on a real mounted harddisk instead of a dir in memory. for example /share/Download

Code: Select all

if ((check_disk_space('.', '/usr/lib')) < 0) {


d.lansinklesscher
New here
Posts: 3
Joined: Fri Aug 29, 2014 8:38 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby d.lansinklesscher » Fri Aug 29, 2014 8:47 pm

You can get this to work easily on qts 4.1 doing the following:

In the webinterface:
-Install PERL Packages
-Create Share Config
-Creat Share Back-up

Using SSH:
Create the following file using VI
/share/Config/veeamstartup.sh
Make sure this file contains:

#!/bin/sh
ln -s /opt/bin/perl /usr/bin/perl

After saving the file give it execute rights:
chmod +x veeamstartup.sh

edit the qpkg.conf config file in /etc/config

Add this:

[veeamautorun]
Name = veeamautorun
Version = 0.1
Author = Me
Date = 2014-01-21
Shell = /share/Config/veeamstartup.sh
Install_Path = /share/Config
QPKG_File = veeamautorun.qpkg
Enable = TRUE

Reboot the Nas and you should be able to add it a Linux-server. When you add the back-up repository choose the following path: /share/back-up

Good Luck!

d.lansinklesscher
New here
Posts: 3
Joined: Fri Aug 29, 2014 8:38 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby d.lansinklesscher » Mon Sep 29, 2014 9:04 pm

Another issue that I have come across is that after a while the ramdisk of the nas runs out of diskspace. This probably is because of logs.

As a workaround I scheduled a weekly reboot.

Maybe Veeam can also address this by putting the logs on persistent storage off the nas.

fbuenemann
First post
Posts: 1
Joined: Fri Feb 03, 2012 11:52 pm

Re: What setting to use to setup qnap as veeam backup reposi

Postby fbuenemann » Wed Dec 17, 2014 2:03 pm

I'm using the following qpkg startup script on TS-659 Pro+ with firmware 4.1.2:

Code: Select all

#!/bin/sh
test -x /opt/bin/perl && ln -nfs /opt/bin/perl /usr/bin/
test -x /opt/bin/df && ln -nfs /opt/bin/df /bin/
if [ ! -L /var/log/VeeamBackup ]; then
  mkdir -p /share/MD0_DATA/VeeamBackup
  rm -rf /var/log/VeeamBackup
  ln -s /share/MD0_DATA/VeeamBackup /var/log/
fi
while cat /proc/mounts | grep -q gphotofs; do
  umount /share/external/UHCI\ Host\ Controller
done
exit 0

This also needs the the qpkg for perl and ipkg for coreutils. (ipkg version of perl would probably work too.)

It works with Veeam B&R v8 without modifications to veeam_soap.tar.

The startup script makes sure perl can be found in path, replaces busybox df with coreutils df, redirects veeam agent logs to stable storage, so the ramdisk wont fill up and unmounts the gphotofs fuse mtp filesystem driver, because it causes non-zero exit code due to errors when running df.

If you want to rotate the veeam agent logs, you can add logrotate via ipkg and a config /opt/etc/logrotate.d/veeam_backup.conf like this:

Code: Select all

/var/log/VeeamBackup/*.log
{
  daily
  missingok
  rotate 7
  compress
  notifempty
  copytruncate
  create 3104 admin administrators
}

Then add a conjob to /etc/config/crontab (this example rotates daily at 5 PM):

Code: Select all

0 17 * * * /opt/sbin/logrotate /opt/etc/logrotate.d/veeam_backup.conf


Return to “Server Virtualization & Clustering”

Who is online

Users browsing this forum: No registered users and 1 guest