Find ZFS partition Topic is solved

How to use TestDisk to recover lost partition
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Locked
Message
Author
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Find ZFS partition

#1 Post by arm512 »

Hello, I've mistakenly destroyed GPT on HDD. (Yes, both main and backup GPT.)
Here: https://www.cgsecurity.org/wiki/TestDisk is said that Testdisk can find lost ZFS partitions.
Testdisk recovered all partition, except 2 non-raid ZFS pools.
It sees even raw partitions, I've backuped in files and raw VM files, but absolutely blind dealing with ZFS.
Can anybody help me to make Testdisk see ZFS partitions?
Top
recuperation
Posts: 2729
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Find ZFS partition

#2 Post by recuperation »

Have the ZFS partitions been encrypted?
Top
User avatar
cgrenier
Site Admin
Posts: 5432
Joined: 18 Feb 2012, 15:08
Location: Le Perreux Sur Marne, France
Contact:
Contact cgrenier

Re: Find ZFS partition

#3 Post by cgrenier »

Support for ZFS in TestDisk is very limited.
TestDisk should be able to find ZFS boot block (struct vdev_boot_header) https://git.cgsecurity.org/cgit/testdisk/tree/src/zfs.c
The beginning of a partition may looks like (hexdump output)

Code: Select all

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00003fd0  00 00 00 00 00 00 00 00  11 7a 0c b1 7a da 10 02  |.........z..z...|
                                   zio-data-bloc-tail
00003fe0  3f 2a 6e 7f 80 8f f4 97  fc ce aa 58 16 9f 90 af  |?*n........X....|
00003ff0  8b b4 6d ff 57 ea d1 cb  ab 5f 46 0d db 92 c6 6e  |..m.W...._F....n|
00004000  01 01 00 00 00 00 00 00  00 00 00 01 00 00 00 24  |...............$|
00004010  00 00 00 20 00 00 00 07  76 65 72 73 69 6f 6e 00  |... ....version.|
00004020  00 00 00 08 00 00 00 01  00 00 00 00 00 00 00 08  |................|
00004030  00 00 00 28 00 00 00 28  00 00 00 04 6e 61 6d 65  |...(...(....name|
00004040  00 00 00 09 00 00 00 01  00 00 00 09 74 61 6e 6b  |............tank|
00004050  58 31 34 30 65 00 00 00  00 00 00 24 00 00 00 20  |X140e......$... |
00004060  00 00 00 05 73 74 61 74  65 00 00 00 00 00 00 08  |....state.......|
00004070  00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 20  |............... |
00004080  00 00 00 20 00 00 00 03  74 78 67 00 00 00 00 08  |... ....txg.....|
00004090  00 00 00 01 00 00 00 00  01 5a 79 28 00 00 00 28  |.........Zy(...(|
000040a0  00 00 00 28 00 00 00 09  70 6f 6f 6c 5f 67 75 69  |...(....pool_gui|
000040b0  64 00 00 00 00 00 00 08  00 00 00 01 6a 72 a9 ee  |d...........jr..|
000040c0  37 ec a7 da 00 00 00 24  00 00 00 20 00 00 00 06  |7......$... ....|
000040d0  65 72 72 61 74 61 00 00  00 00 00 08 00 00 00 01  |errata..........|
000040e0  00 00 00 00 00 00 00 00  00 00 00 28 00 00 00 28  |...........(...(|
000040f0  00 00 00 08 68 6f 73 74  6e 61 6d 65 00 00 00 09  |....hostname....|
00004100  00 00 00 01 00 00 00 05  58 31 34 30 65 00 00 00  |........X140e...|
00004110  00 00 00 24 00 00 00 28  00 00 00 08 74 6f 70 5f  |...$...(....top_|
00004120  67 75 69 64 00 00 00 08  00 00 00 01 a1 74 3b 25  |guid.........t;%|
00004130  94 94 d9 77 00 00 00 20  00 00 00 20 00 00 00 04  |...w... ... ....|
00004140  67 75 69 64 00 00 00 08  00 00 00 01 c7 16 54 40  |guid..........T@|
00004150  90 b7 bb a2 00 00 00 2c  00 00 00 28 00 00 00 0d  |.......,...(....|
00004160  76 64 65 76 5f 63 68 69  6c 64 72 65 6e 00 00 00  |vdev_children...|
00004170  00 00 00 08 00 00 00 01  00 00 00 00 00 00 00 01  |................|
00004180  00 00 04 d8 00 00 00 38  00 00 00 09 76 64 65 76  |.......8....vdev|
00004190  5f 74 72 65 65 00 00 00  00 00 00 13 00 00 00 01  |_tree...........|
000041a0  00 00 00 00 00 00 00 01  00 00 00 24 00 00 00 20  |...........$... |
000041b0  00 00 00 04 74 79 70 65  00 00 00 09 00 00 00 01  |....type........|
000041c0  00 00 00 05 72 61 69 64  7a 00 00 00 00 00 00 20  |....raidz...... |
000041d0  00 00 00 20 00 00 00 02  69 64 00 00 00 00 00 08  |... ....id......|
000041e0  00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 20  |............... |
000041f0  00 00 00 20 00 00 00 04  67 75 69 64 00 00 00 08  |... ....guid....|
...
Top
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Re: Find ZFS partition

#4 Post by arm512 »

recuperation wrote: ↑15 Nov 2020, 09:14 Have the ZFS partitions been encrypted?
Neither LUKS nor plain. May be ZFS native encryption, but no third-party soft.
Top
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Re: Find ZFS partition

#5 Post by arm512 »

cgrenier wrote: ↑15 Nov 2020, 10:00 Support for ZFS in TestDisk is very limited.
TestDisk should be able to find ZFS boot block (struct vdev_boot_header) https://git.cgsecurity.org/cgit/testdisk/tree/src/zfs.c
The beginning of a partition may looks like (hexdump output)

Code: Select all

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00003fd0  00 00 00 00 00 00 00 00  11 7a 0c b1 7a da 10 02  |.........z..z...|
                                   zio-data-bloc-tail
00003fe0  3f 2a 6e 7f 80 8f f4 97  fc ce aa 58 16 9f 90 af  |?*n........X....|
00003ff0  8b b4 6d ff 57 ea d1 cb  ab 5f 46 0d db 92 c6 6e  |..m.W...._F....n|
00004000  01 01 00 00 00 00 00 00  00 00 00 01 00 00 00 24  |...............$|
00004010  00 00 00 20 00 00 00 07  76 65 72 73 69 6f 6e 00  |... ....version.|
00004020  00 00 00 08 00 00 00 01  00 00 00 00 00 00 00 08  |................|
00004030  00 00 00 28 00 00 00 28  00 00 00 04 6e 61 6d 65  |...(...(....name|
00004040  00 00 00 09 00 00 00 01  00 00 00 09 74 61 6e 6b  |............tank|
00004050  58 31 34 30 65 00 00 00  00 00 00 24 00 00 00 20  |X140e......$... |
00004060  00 00 00 05 73 74 61 74  65 00 00 00 00 00 00 08  |....state.......|
00004070  00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 20  |............... |
00004080  00 00 00 20 00 00 00 03  74 78 67 00 00 00 00 08  |... ....txg.....|
00004090  00 00 00 01 00 00 00 00  01 5a 79 28 00 00 00 28  |.........Zy(...(|
000040a0  00 00 00 28 00 00 00 09  70 6f 6f 6c 5f 67 75 69  |...(....pool_gui|
000040b0  64 00 00 00 00 00 00 08  00 00 00 01 6a 72 a9 ee  |d...........jr..|
000040c0  37 ec a7 da 00 00 00 24  00 00 00 20 00 00 00 06  |7......$... ....|
000040d0  65 72 72 61 74 61 00 00  00 00 00 08 00 00 00 01  |errata..........|
000040e0  00 00 00 00 00 00 00 00  00 00 00 28 00 00 00 28  |...........(...(|
000040f0  00 00 00 08 68 6f 73 74  6e 61 6d 65 00 00 00 09  |....hostname....|
00004100  00 00 00 01 00 00 00 05  58 31 34 30 65 00 00 00  |........X140e...|
00004110  00 00 00 24 00 00 00 28  00 00 00 08 74 6f 70 5f  |...$...(....top_|
00004120  67 75 69 64 00 00 00 08  00 00 00 01 a1 74 3b 25  |guid.........t;%|
00004130  94 94 d9 77 00 00 00 20  00 00 00 20 00 00 00 04  |...w... ... ....|
00004140  67 75 69 64 00 00 00 08  00 00 00 01 c7 16 54 40  |guid..........T@|
00004150  90 b7 bb a2 00 00 00 2c  00 00 00 28 00 00 00 0d  |.......,...(....|
00004160  76 64 65 76 5f 63 68 69  6c 64 72 65 6e 00 00 00  |vdev_children...|
00004170  00 00 00 08 00 00 00 01  00 00 00 00 00 00 00 01  |................|
00004180  00 00 04 d8 00 00 00 38  00 00 00 09 76 64 65 76  |.......8....vdev|
00004190  5f 74 72 65 65 00 00 00  00 00 00 13 00 00 00 01  |_tree...........|
000041a0  00 00 00 00 00 00 00 01  00 00 00 24 00 00 00 20  |...........$... |
000041b0  00 00 00 04 74 79 70 65  00 00 00 09 00 00 00 01  |....type........|
000041c0  00 00 00 05 72 61 69 64  7a 00 00 00 00 00 00 20  |....raidz...... |
000041d0  00 00 00 20 00 00 00 02  69 64 00 00 00 00 00 08  |... ....id......|
000041e0  00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 20  |............... |
000041f0  00 00 00 20 00 00 00 04  67 75 69 64 00 00 00 08  |... ....guid....|
...
Thank you, I'll search for "11 7a 0c b1 7a da 10 02" in hexeditor, but it will take a long time...
What hexeditor woul you advise?
Top
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Re: Find ZFS partition

#6 Post by arm512 »

Code: Select all

Disk /dev/sdc - 3000 GB / 2794 GiB - CHS 364801 255 63

     Partition                  Start        End    Size in sectors
  1 P Linux filesys. data           40       4095       4056 [Linux filesystem]
  2 P MS Data                     4096    2101247    2097152 [T3.D2.E1]
  3 P MS Data                  2101248 1259382783 1257281536 [Win10]
  4 P Linux filesys. data   1259382784 1263609855    4227072 [Linux filesystem]
  5 P Linux filesys. data   1263609856 1267771391    4161536
  6 P Linux filesys. data   1267771392 1968764927  700993536 <-- this is one of my ZFS, 
  which I've recovered using bash+losetup+gdisk, without testdisk. In Testdisk it was
  only freespace between 2 partitions. 
  Even now, after I recovered 1 ZFS partition, Testdisk does not detect it's FS.

And this is not-damaged disk with ZFS:

Code: Select all

Disk /dev/sdb - 2000 GB / 1863 GiB - CHS 243201 255 63

     Partition                  Start        End    Size in sectors
> 1 P Unknown                     2048       6143       4096 [tb2-d1-p1-bios-boot]
  2 P MS Data                     6144    1056767    1050624 [DISK_3_EFI] [T2.D1.E1]
  3 P MS Reserved              1056768    1093631      36864
  4 P Linux filesys. data      1093632    5287935    4194304
  5 P Solaris /usr             5287936  216043519  210755584 <-- this is ZFS, but Testdisk does not see it's FS.
  6 P Linux filesys. data    216043520 3697315839 3481272320

https://gist.github.com/krichter722/bff ... 818fb22cbd
Here krichter shows his Testdisk output:

Code: Select all

TestDisk 7.1, Data Recovery Utility, July 2019
...
Disk /dev/sdb - 2000 GB / 1863 GiB - CHS 243201 255 63
     Partition			Start        End    Size in sectors
 1 P Solaris /usr                2048 3907012607 3907010560 [zfs-2fa627ed13bb64de]
 9 P Unknown               3907012608 3907028991      16384

And in his case, ZFS is detected, but noone of my ZFS partitions, both lost and healthy, is visible forTestdisk. Why so?

All ZFS were created and used in Linux.

Code: Select all

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.10
Release:        20.10
Codename:       groovy```

uname -a
Linux n0 5.8.0-28-generic #30-Ubuntu SMP Thu Nov 5 13:24:33 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Top
recuperation
Posts: 2729
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Find ZFS partition

#7 Post by recuperation »

arm512 wrote: ↑15 Nov 2020, 12:42 And in his case, ZFS is detected, but noone of my ZFS partitions, both lost and healthy, is visible forTestdisk. Why so?
If your children are kidnapped they are kind of lost but you can never say they are healthy.
The same applies to your ZFS partitions. There is no evidence that they are healthy.

Unfortunately you did not bother providing a log file. It could be that by just looking at the locations of the other partition you could guess where the first ZFS partition should start.

Here is something you could do:

Get a new drive.
Partition that drive as GPT with just one ZFS partition. The partition does not have to fill the whole drive. Maybe that will work as well using an USB stick.
Use the machine and configuration that you used to create the now broken pool and create a new one and assign the freshly created partition above.
Run Testdisk and save the log file. That will show you if Testdisk performs under a clean, healthy configuration.
It should provide you with the first and last sector number of your ZFS partition, too.
Using a hexeditor, zero out the GPT and its backup.
Now rerun Testdisk to see what it gives.
Top
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Re: Find ZFS partition

#8 Post by arm512 »

recuperation wrote: ↑15 Nov 2020, 14:30
arm512 wrote: ↑15 Nov 2020, 12:42 And in his case, ZFS is detected, but noone of my ZFS partitions, both lost and healthy, is visible forTestdisk. Why so?
If your children are kidnapped they are kind of lost but you can never say they are healthy.
The same applies to your ZFS partitions. There is no evidence that they are healthy.

Unfortunately you did not bother providing a log file. It could be that by just looking at the locations of the other partition you could guess where the first ZFS partition should start.

Here is something you could do:

Get a new drive.
Partition that drive as GPT with just one ZFS partition. The partition does not have to fill the whole drive. Maybe that will work as well using an USB stick.
Use the machine and configuration that you used to create the now broken pool and create a new one and assign the freshly created partition above.
Run Testdisk and save the log file. That will show you if Testdisk performs under a clean, healthy configuration.
It should provide you with the first and last sector number of your ZFS partition, too.
Using a hexeditor, zero out the GPT and its backup.
Now rerun Testdisk to see what it gives.
Under healthy partitions I ment partitions on other drives with alive GPT. Testdisk doesn't see them too.

I don't think that loop device will be worse than USB-stick.
So I tryed what you've advised, but on the loop device:

Before wiping GPT:

Code: Select all

No partition found or selected for recovery
Log:

Code: Select all

Sun Nov 15 15:54:10 2020
Command line: TestDisk /log /dev/loop26

TestDisk 7.1, Data Recovery Utility, July 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org
OS: Linux, kernel 5.7.0-rc5-zbod-ym29 (#1 SMP Wed May 13 19:59:50 EEST 2020) x86_64
Compiler: GCC 9.2
ext2fs lib: 1.45.5, ntfs lib: libntfs-3g, reiserfs lib: none, ewf lib: none, curses lib: ncurses 6.1
Hard disk list
Disk /dev/loop26 - 20 GB / 18 GiB - 39062500 sectors, sector size=512

Partition table type (auto): EFI GPT
Disk /dev/loop26 - 20 GB / 18 GiB
Partition table type: EFI GPT

Analyse Disk /dev/loop26 - 20 GB / 18 GiB - 39062500 sectors
Current partition structure:
 1 P Linux filesys. data         2048   39061503   39059456

search_part()
Disk /dev/loop26 - 20 GB / 18 GiB - 39062500 sectors
Search for partition aborted

interface_write()
 
No partition found or selected for recovery
simulate write!

TestDisk exited normally.
After wiping GPT the same:

Code: Select all

No partition found or selected for recovery
Log:

Code: Select all

Sun Nov 15 16:05:34 2020
Command line: TestDisk /log /dev/loop26

TestDisk 7.1, Data Recovery Utility, July 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org
OS: Linux, kernel 5.7.0-rc5-zbod-ym29 (#1 SMP Wed May 13 19:59:50 EEST 2020) x86_64
Compiler: GCC 9.2
ext2fs lib: 1.45.5, ntfs lib: libntfs-3g, reiserfs lib: none, ewf lib: none, curses lib: ncurses 6.1
Hard disk list
Disk /dev/loop26 - 2000 MB / 1907 MiB - 3906250 sectors, sector size=512

Partition table type defaults to Intel
Disk /dev/loop26 - 2000 MB / 1907 MiB
Partition table type: EFI GPT

Analyse Disk /dev/loop26 - 2000 MB / 1907 MiB - 3906250 sectors
Bad GPT partition, invalid signature.
Trying alternate GPT
Bad GPT partition, invalid signature.
Current partition structure:
Bad GPT partition, invalid signature.
Trying alternate GPT
Bad GPT partition, invalid signature.

search_part()
Disk /dev/loop26 - 2000 MB / 1907 MiB - 3906250 sectors

interface_write()
 
No partition found or selected for recovery

search_part()
Disk /dev/loop26 - 2000 MB / 1907 MiB - 3906250 sectors

interface_write()
 
No partition found or selected for recovery
simulate write!

TestDisk exited normall
So, even under clean and healthy configuration Testdisk doesn't see ZFS at all.
Ofcource I can test ZFS created on NetBSD, Illumos/OpenSolaris deriatives, FreeBSD, and original, Oracle's Solaris ZFS. It would be interesting. I think Testdisk would see either Illumos-created ZFS or when ZFS is on Sun Partition table nested inside the GPT partition (grub-path: 'hd0,gpt1,sunpc1', similar to FreeBSD's MBR setup /dev/ad0s1a (grub's 'hd0,msdos1,bsd1')). In Solaris way there will be also partition 9, just after the pool ends... But that's not my case, I use ZoL version of ZFS.

The end of the partition doesn't matter to recover:

Code: Select all

losetup -o $OFFSET -f /dev/sd$LETTER
zpool import -o readonly=on -fN $ZPOOL -d /dev/$LOOP_DEVICE
zfs send -R .... | zfs recv ...
zpool will see it's end after beeing imported, if autoexpand=off, ofcource.
Top
recuperation
Posts: 2729
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Find ZFS partition

#9 Post by recuperation »

Unfortunately I don't have experience with ZFS.

But there are a few interesting things to note:

There are different GPT identifiers for ZFS on different operating systems. There is no reserved one for Linux.

Search ZFS in this document:
https://en.wikipedia.org/wiki/GUID_Partition_Table

I guess creating a pool does not change the identifier in the GPT. You can test that.

After creating the pool I would write at least one file into the pool to make sure that the structure gets initialized.
I still do not know where the pool specs are stored.

Your documentation did not tell if and when you created a pool.

Anyway you could save the first sector of your ZFS partition to be able to compare it with the fingerprint Testdisk is looking for.
There might be differences.
Top
arm512
Posts: 8
Joined: 14 Nov 2020, 21:36

Re: Find ZFS partition

#10 Post by arm512 »

Searching "11 7a 0c b1 7a da 10 02" was not the best idea. After find-counter turned to 8500, I decided to search "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 11 7a 0c b1 7a da 10 02" instead.
We'll see what will give.
Top
Locked

Return to “Partition Recovery”