[SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Introduce yourself to us and other members here, or share your own product reviews, suggestions, and tips and tricks of using QNAP products.
bonsai
First post
Posts: 1
Joined: Thu Mar 28, 2013 9:11 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by bonsai »

Hello, I'm trying to access a disk from a TS-469L with 1 sinlge volume (created as type thick). But it's not working. Mayby you can help me?

First i connect the disk to a pc with sata.
Then i perpare a usb stick with a ubuntu live and boot from it.
Next i install the LVM2 software package with the thin-provisioning-tools.

And then i do this steps:

Code: Select all

parted -l
    Model: ATA WDC WD800JD-55MU (scsi)
    Disk /dev/sda: 80,0GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number  Start   End     Size    Type     File system  Flags
    1      1049kB  368MB   367MB   primary  ntfs         boot
    2      368MB   52,4GB  52,1GB  primary  ntfs
    3      52,4GB  80,0GB  27,6GB  primary  ntfs


    Model: ATA SAMSUNG HD154UI (scsi)
    Disk /dev/sdb: 1500GB
    Sector size (logical/physical): 512B/512B
    Partition Table: gpt

    Number  Start   End     Size    File system     Name     Flags
    1      20,5kB  543MB   543MB   ext3            primary
    2      543MB   1086MB  543MB   linux-swap(v1)  primary
    3      1086MB  1491GB  1490GB                  primary
    4      1491GB  1492GB  543MB   ext3            primary
    5      1492GB  1500GB  8554MB  linux-swap(v1)  primary


    Warning: Unable to open /dev/sr0 read-write (Das Dateisystem ist nur lesbar).
    /dev/sr0 has been opened read-only.
    Error: /dev/sr0: unrecognised disk label                                 

    Error: /dev/zram0: unrecognised disk label                               

    Warning: Unable to open /dev/cloop0 read-write (Das Dateisystem ist nur lesbar).
    /dev/cloop0 has been opened read-only.
    Error: /dev/cloop0: unrecognised disk label

Code: Select all

mdadm --assemble --scan
    mdadm: /dev/md/321 has been started with 1 drive (out of 2).
    mdadm: /dev/md/13 has been started with 1 drive (out of 24).
    mdadm: /dev/md/2 has been started with 1 drive.
    mdadm: /dev/md/256 assembled from 0 drives and 1 spare - not enough to start the array.
    mdadm: /dev/md/9 has been started with 1 drive (out of 24).
    mdadm: /dev/md/256 assembled from 0 drives and 1 spare - not enough to start the array.

Code: Select all

cat /proc/mdstat
    Personalities : [raid1]
    md9 : active raid1 sdb1[25]
          530048 blocks super 1.0 [24/1] [__U_____________________]
          bitmap: 1/1 pages [4KB], 65536KB chunk

    md2 : active raid1 sdb3[0]
          1455183616 blocks super 1.0 [1/1] [U]
         
    md13 : active raid1 sdb4[25]
          458880 blocks super 1.0 [24/1] [__U_____________________]
          bitmap: 1/1 pages [4KB], 65536KB chunk

    md321 : active raid1 sdb5[0]
          8353728 blocks super 1.0 [2/1] [U_]
          bitmap: 1/1 pages [4KB], 65536KB chunk

    unused devices: <none>

Code: Select all

lvscan
      WARNING: Unrecognised segment type thin-pool
      WARNING: Unrecognised segment type thick
      inactive          '/dev/vg2/lv545' [13,88 GiB] inherit
      inactive          '/dev/vg2/tp2' [1,33 TiB] inherit
      inactive          '/dev/vg2/lv2' [1,32 TiB] inherit

Code: Select all

vgdisplay
      WARNING: Unrecognised segment type thin-pool
      WARNING: Unrecognised segment type thick
      --- Volume group ---
      VG Name               vg2
      System ID             
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  6
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                3
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               1,36 TiB
      PE Size               4,00 MiB
      Total PE              355269
      Alloc PE / Size       355269 / 1,36 TiB
      Free  PE / Size       0 / 0   
      VG UUID               2ktkBf-OyrR-eZub-NSMN-25gh-Lqyc-Ta0Vvu
So, the shown warnings confuse me.

Code: Select all

lvdisplay
      WARNING: Unrecognised segment type thin-pool
      WARNING: Unrecognised segment type thick
      --- Logical volume ---
      LV Path                /dev/vg2/lv545
      LV Name                lv545
      VG Name                vg2
      LV UUID                OAoDN1-l61r-PxPb-jSmo-vAZr-tzWY-bWdNOS
      LV Write Access        read/write
      LV Creation host, time QNAPTS469L, 2014-07-08 23:28:04 +0200
      LV Status              NOT available
      LV Size                13,88 GiB
      Current LE             3552
      Segments               1
      Allocation             inherit
      Read ahead sectors     4096
       
      --- Logical volume ---
      LV Path                /dev/vg2/tp2
      LV Name                tp2
      VG Name                vg2
      LV UUID                7yu5BA-oiTW-LPOs-auKR-1FgA-d0FA-gjdDV0
      LV Write Access        read/write
      LV Creation host, time QNAPTS469L, 2014-07-08 23:28:14 +0200
      LV Status              NOT available
      LV Size                1,33 TiB
      Current LE             347621
      Segments               1
      Allocation             inherit
      Read ahead sectors     auto
       
      --- Logical volume ---
      LV Path                /dev/vg2/lv2
      LV Name                lv2
      VG Name                vg2
      LV UUID                lZeI1R-O5M2-Gjtx-Y1Qu-cZYQ-4xNQ-WzJ0wL
      LV Write Access        read/write
      LV Creation host, time QNAPTS469L, 2014-07-08 23:28:18 +0200
      LV Status              NOT available
      LV Size                1,32 TiB
      Current LE             347108
      Segments               1
      Allocation             inherit
      Read ahead sectors     4096
With loading the needed modules for lvm ...

Code: Select all

modprobe dm_mod
lsmod| grep dm_
    dm_thin_pool           45785  0
    dm_persistent_data     54412  1 dm_thin_pool
    dm_bufio               22204  1 dm_persistent_data
    dm_bio_prison          14968  1 dm_thin_pool
    libcrc32c              12543  2 btrfs,dm_persistent_data
    dm_crypt               22622  0
    dm_multipath           22402  0
    scsi_dh                14458  1 dm_multipath
    dm_mirror              21756  0
    dm_region_hash         20121  1 dm_mirror
    dm_log                 18072  2 dm_region_hash,dm_mirror
and last

Code: Select all

lvscan
      WARNING: Unrecognised segment type thick
      ACTIVE            '/dev/vg2/lv545' [13,88 GiB] inherit
      ACTIVE            '/dev/vg2/tp2' [1,33 TiB] inherit
      inactive          '/dev/vg2/lv2' [1,32 TiB] inherit
Why a get the message "Unrecognised segment type thick"? And what can i do to activate the /dev/vg2/lv2 and mount it?

Last information

Code: Select all

    root@ubuntu:/mnt# vgscan
      Reading all physical volumes.  This may take a while...
      WARNING: Unrecognised segment type thick
      Found volume group "vg2" using metadata type lvm2

Code: Select all

    root@ubuntu:/mnt# vgdisplay
      WARNING: Unrecognised segment type thick
      --- Volume group ---
      VG Name               vg2
      System ID             
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  6
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                3
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               1,36 TiB
      PE Size               4,00 MiB
      Total PE              355269
      Alloc PE / Size       355269 / 1,36 TiB
      Free  PE / Size       0 / 0   
      VG UUID               2ktkBf-OyrR-eZub-NSMN-25gh-Lqyc-Ta0Vvu
I hope, someone can help me?

As a next test, i install a clean and small disk (200GB) alone. Install the system on it and create a volume type thick on it. Then i do the above described steps. The same result :(
markwaitangi
First post
Posts: 1
Joined: Thu Oct 02, 2014 12:37 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by markwaitangi »

Having a similar issue at the point of:

Code: Select all

 vgchange -ay 
gets

Code: Select all

  WARNING: Unrecognised segment type thick
  Refusing activation of LV lv1 containing an unrecognised segment.
  2 logical volume(s) in volume group "vg1" now active
So does anyone have any ideas of how to over come this?

Also I have

Code: Select all

lvscan

Code: Select all

  WARNING: Unrecognised segment type thick
  ACTIVE            '/dev/vg1/lv544' [20.00 GiB] inherit
  ACTIVE            '/dev/vg1/tp1' [5.40 TiB] inherit
  inactive          '/dev/vg1/lv1' [5.39 TiB] inherit
So basically how to make either the WRNING go away or the inactive to become ACTIVE
Thanks in advance
JeffryPaul
New here
Posts: 6
Joined: Tue Dec 02, 2014 3:59 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by JeffryPaul »

after installing the "thin-provisioning-tools" package, I was able to step past the thin-pool problem. Now I'm stuck with the vg1 not being activated due to the invalid "thick" segment type, which appears to be unique to QNAP, since there is no LVM documentation supporting such a segment type. Reviewed the segtype.h file in the lvm2 project found here: https://git.fedorahosted.org/cgit/lvm2.git

Thankfully I am just getting started with QNAP backup, and recovering from a NAS failure is one of my test cases. I would have expected QNAP to provide some assistance by now.
bbbh
New here
Posts: 4
Joined: Wed Oct 24, 2012 8:43 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by bbbh »

Did you try solving that issue with the segment type as thick? I am currently in that situation with running a live ubuntu usb of 14.04. I get to your part of activating the volume group but having the wrong segment type since it is thick. I have been searching all weekend on how to mount it.
Corsari
New here
Posts: 4
Joined: Sun Feb 15, 2015 10:36 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by Corsari »

Same issue here

At first sight many LV inactive

With both thin and thick warnings

After installing "thin" modules, all of them became active except one

Either if googling so much, results that this "thick" issue is always on a webpage that talks about QNAP, it is a standard "lvscan" command that reports the warning... and since "lvscan" is run on a ubuntu OS with apt-get installed LVM2 suite + thin provisioning... the message is not QNAP specific, it is built in into the "lvscan" binary.

Maybe the LVM2 mailing list could help. I've written a ticket on QNAP support ticketing system, but I'll try also the mailing list.
louiscar
Easy as a breeze
Posts: 265
Joined: Mon Aug 10, 2015 4:32 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by louiscar »

father_mande wrote:Hi,
****** Having a backup is ALWAYS the best solution

****** AS YOU KNOW ... new SMB firmware use LVM (Logical Volume Manager) under Raid to create volume for file system
Phillipe, do you know if LVM is used regardless?
I'm wondering if this is only used when thick or thin provisioning is chosen. Perhaps it uses the old system when choosing fixed?
Model : TS-453 Pro
Firmware : 5.0.0.1828
4x WD RED 3TB - Raid 5
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by schumaku »

louiscar wrote:Phillipe, do you know if LVM is used regardless?
I'm wondering if this is only used when thick or thin provisioning is chosen. Perhaps it uses the old system when choosing fixed?
All new volume types - static, thin, and thick allocated - are LVM based volumes.
louiscar
Easy as a breeze
Posts: 265
Joined: Mon Aug 10, 2015 4:32 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by louiscar »

schumaku wrote: All new volume types - static, thin, and thick allocated - are LVM based volumes.
Thanks for the clarification
Model : TS-453 Pro
Firmware : 5.0.0.1828
4x WD RED 3TB - Raid 5
alyf80
New here
Posts: 3
Joined: Fri Jan 23, 2015 1:50 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by alyf80 »

JeffryPaul wrote:after installing the "thin-provisioning-tools" package, I was able to step past the thin-pool problem. Now I'm stuck with the vg1 not being activated due to the invalid "thick" segment type, which appears to be unique to QNAP, since there is no LVM documentation supporting such a segment type. Reviewed the segtype.h file in the lvm2 project found here: https://git.fedorahosted.org/cgit/lvm2.git
Apparently QNAP extended the standard thin provisioning model of LVM2 in order to support new features (my guess is that this has to do with the ability to have thick-provisioned volumes within a thin-provisioning pool, which is a very desirable thing from a flexibility standpoint).
There are changes both in the userspace tools and in the kernel; this pretty much rules out the possibility of an easy recovery using a vanilla linux system.

Having said that, the QNAP GPL tarball at http://sourceforge.net/projects/qosgpl/ contains all the relevant sources, so I was able to build a working recovery environment by compiling and installing QNAP's kernel and LVM tools on a standard system.
louiscar
Easy as a breeze
Posts: 265
Joined: Mon Aug 10, 2015 4:32 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by louiscar »

alyf80 wrote:
JeffryPaul wrote:after installing the "thin-provisioning-tools" package, I was able to step past the thin-pool problem. Now I'm stuck with the vg1 not being activated due to the invalid "thick" segment type, which appears to be unique to QNAP, since there is no LVM documentation supporting such a segment type. Reviewed the segtype.h file in the lvm2 project found here: https://git.fedorahosted.org/cgit/lvm2.git
Apparently QNAP extended the standard thin provisioning model of LVM2 in order to support new features (my guess is that this has to do with the ability to have thick-provisioned volumes within a thin-provisioning pool, which is a very desirable thing from a flexibility standpoint).
There are changes both in the userspace tools and in the kernel; this pretty much rules out the possibility of an easy recovery using a vanilla linux system.

Having said that, the QNAP GPL tarball at http://sourceforge.net/projects/qosgpl/ contains all the relevant sources, so I was able to build a working recovery environment by compiling and installing QNAP's kernel and LVM tools on a standard system.
would love to have a bootable image of that recovery if possible for the more unix challenged amongst us or perhaps you could do a step by step tut for creating same?
Model : TS-453 Pro
Firmware : 5.0.0.1828
4x WD RED 3TB - Raid 5
alyf80
New here
Posts: 3
Joined: Fri Jan 23, 2015 1:50 am

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-50

Post by alyf80 »

louiscar wrote:
alyf80 wrote: Having said that, the QNAP GPL tarball at http://sourceforge.net/projects/qosgpl/ contains all the relevant sources, so I was able to build a working recovery environment by compiling and installing QNAP's kernel and LVM tools on a standard system.
would love to have a bootable image of that recovery if possible for the more unix challenged amongst us or perhaps you could do a step by step tut for creating same?
What I cobbled together was more of a horrible hack than a real recovery environment; building something that can be used without a fair level of linux-fu would take a considerable amount of time, which I really don't have right now.
I can outlne the process, but you will probably have to adapt things based on your environment and any problems you might encounter.

Start with a clean linux install (you need persistent storage, so a live distro is probably not the best idea). I cloned a gentoo system I happened to have at hand, but any distribution should do.
You need a working development environment, so make sure you install the relevant packages (C compiler, binutils, glibc and kernel headers,...). I don't have a complete list, you will have to sort out the missing dependencies from any errors you get.

The first step is to install thin-provisioning-tools; these are just needed for a configure check so don't bother with the sources in the QNAP tarball: just installing your distribution's package should be fine.

From the tarball extract src/LVM2.2.02.96, configure it with --with-thin=internal, compile and install. If you get errors during the configuration or complilation, check your dependencies.

Now comes the fun part, i.e. the kernel.
Extract src/linux-3.12.6; for configuration I picked a default from the kernel_cfg folder in the tarball (I used TS870/linux-3.12.6-x86_64-hal.cfg, which is for an x86 NAS with specs reasonably similar to a recent PC) and adjusted it to include drivers for the hardware in my rescue system.
Everything you need (console, input, storage, network, ...) should be compiled in rather than built as a module: this is a rather ancient kernel and the userspace tools from your distribution might not play nice with it (in my case module loading did not work at all).

Compiling the kernel is a bit of a hassle, as QNAP seems to have sprinkled the sources with calls to code in a module of their own (picd) which is disabled in the configuration and cannot be enabled as it fails to compile.
I was not interested in debugging kernel build issues, so as a quick hack I left picd disabled and added

Code: Select all

#define send_message_to_app(...) do {} while(0)
at the end of include/qnap/pic.h, just before the final #endif; this effectively converts the offending calls to no-ops.

Then you need to reboot using the kernel image you just compiled; it might take some trials to get everything right, so be sure to keep the orginal kernel around so you can always go back and make changes.
At this point you should have a working system running a patched kernel and with a patched LVM installed; if all RAID volumes are connected and running, the usual

Code: Select all

# vgchange -ay
should not give any errors, and a subsequent

Code: Select all

# lvdisplay
should output a list of all your volumes in "available" state, which you can then mount.

Now, the usual disclaimers apply: this is what I did, and it worked for me. If you try to do the same and it kills your cat (or destroys your data), well, tough luck.
I did not write a detailed HOWTO for a good reason: if what I wrote above looks like gibberish to you, you should definitely not attempt to follow it.
Also, the method above is only intended as an emergency measure to gain read-only access to your volumes so that you can copy data off them. Do not mount the volumes read-write, and do not attempt any provisioning operations on the pools!

Hope this helps,
Andrea
User avatar
marshalleq
Easy as a breeze
Posts: 303
Joined: Mon Jul 23, 2012 2:44 pm
Contact:

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by marshalleq »

There is no like button in these forums, so I'm just going to say thanks to father_mande. :)

Great post!
TS669-L (3GB RAM) fw 4.3.2.0050 Build 20170104 with 6x4TB Seagate ST4000VN000-1H41SC43 2x2TB Hitachi HDS723020BLA642 2x8TB ST8000VN0022-2EL112
QNAP UX-500P External 5 Bay USB3 Expansion Cabinet
Raspberry Pi 1,2 & 3
Mac Mini
Apple TV 3/4
Windows
Sabayon Linux
Ubuntu Linux
Android
Docker
Ryzen 1800x custom system

Web
https://www.tech-knowhow.com

Twitter
@Marshalleq
@TechKn0wH0w
buonboy
Starting out
Posts: 11
Joined: Fri Oct 05, 2018 6:30 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by buonboy »

Hello,
My NAS (TS231P2) died and does not turn on anymore, and also damaged my HDDs (RAID 1), so I can't access my data.
I got this error for both HDDs (with lvdisplay or vgdisplay):
WARNING: Unrecognized segment type tier-thin-pool
LV tp1, segment 1 invalid: does not support flag ERROR_WHEN_FULL. for tier-thin-pool segment.
Internal error: LV segments corrupted in tp1.

Any idea?
Thanks a lot.
User avatar
marshalleq
Easy as a breeze
Posts: 303
Joined: Mon Jul 23, 2012 2:44 pm
Contact:

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by marshalleq »

I'm not sure why your disks would have died, but if I were in your shoes I would contact QNAP support. Assuming you've got a new NAS. Failing that - I think there are some instructions floating around on these forums about how to get your data off the drives with a PC - would need some linux skills from memory. Probably the best option is to buy a new NAS / get your old one going (Power Supply?) and use the Helpdesk feature on it - I think it allows them to connect in and fix it up.
TS669-L (3GB RAM) fw 4.3.2.0050 Build 20170104 with 6x4TB Seagate ST4000VN000-1H41SC43 2x2TB Hitachi HDS723020BLA642 2x8TB ST8000VN0022-2EL112
QNAP UX-500P External 5 Bay USB3 Expansion Cabinet
Raspberry Pi 1,2 & 3
Mac Mini
Apple TV 3/4
Windows
Sabayon Linux
Ubuntu Linux
Android
Docker
Ryzen 1800x custom system

Web
https://www.tech-knowhow.com

Twitter
@Marshalleq
@TechKn0wH0w
buonboy
Starting out
Posts: 11
Joined: Fri Oct 05, 2018 6:30 pm

Re: [SMB] HOW-TO RECOVER data from LVM volume on a PC (UX-500P)

Post by buonboy »

Thanks for reply.
My NAS is still under warranty, it died in less than two monthes, and QNAP support already told me to send it back and they will repair or change it, but the problem is that the HDDs are corrupted after NAS failure and QNAP do not accept it, neither care about it... So how can I be sure after NAS replacment the HDDs will work again and I will have access to my data?
And in the meantime I need to access to my data... I tried with some Recovery Software in Windows, they recover my data but they don't recognize my directory structure...
I'm not a Linux expert but I tried to follow some instruction around, however I can't go further, as I get that error.
Locked

Return to “Users' Corner”