RAID0 mdraid member disk recovery Topic is solved

How to use TestDisk to recover lost partition
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Message
Author
theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

RAID0 mdraid member disk recovery

#1 Post by theKarlisK »

While dual-booting Windows and Linux and trying to add an additional disk for the Windows environment with an identical capacity as one of the member disks in the array I made a terrible mistake and allowed Windows to "Initialize" the disk. Now my RAID0 mdraid array is broken - one of the disks contains a 16MB Microsoft Reserved partition.

The array consisted of two disks - /dev/sdb and /dev/sdd, the '/dev/sdd' is the disk that has lost it's contents:

Code: Select all

❯ sudo mdadm -D /dev/md127
/dev/md127:
        Version : 1.0
        Raid Level : raid0
    Total Devices : 1
    Persistence : Superblock is persistent

            State : inactive
Working Devices : 1

            Name : any:data
            UUID : e7ee729e:81c9fc54:9ef2cf45:8173dd35
            Events : 0

    Number   Major   Minor   RaidDevice

    -       8       16        -        /dev/sdb
Fdisk output of disks:

Code: Select all

Disk /dev/sdb: 931,51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: ST1000DX001-1CM1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1           1 1953525167 1953525167 931,5G ee GPT

Partition 1 does not start on physical sector boundary.

Code: Select all

Disk /dev/sdd: 931,51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: ST1000DM010-2EP1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: F6FFD6CE-CAB7-424A-ABB8-5A6D13E5B5D7

Device     Start   End Sectors Size Type
/dev/sdd1     34 32767   32734  16M Microsoft reserved

Partition 1 does not start on physical sector boundary.
Scanning with mdadm utilities didn't work out:

Code: Select all

sudo mdadm --assemble --scan -v
mdadm: Cannot assemble mbr metadata on /dev/sdd
Examining /dev/sdb finds the data, while examining /dev/sdd does not:

Code: Select all

❯ sudo mdadm --examine /dev/sdb
/dev/sdb:
        Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x0
    Array UUID : e7ee729e:81c9fc54:9ef2cf45:8173dd35
        Name : any:data
Creation Time : Fri Dec  4 18:03:31 2020
    Raid Level : raid0
Raid Devices : 2

Avail Dev Size : 1953525144 sectors (931.51 GiB 1000.20 GB)
Super Offset : 1953525152 sectors
        State : clean
    Device UUID : f5682d8a:9d938fc3:1d6d9493:b13a34a9

    Update Time : Fri Dec  4 18:03:31 2020
Bad Block Log : 512 entries available at offset -8 sectors
    Checksum : e73a7989 - correct
        Events : 0

    Chunk Size : 64K

Device Role : Active device 0
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)

Code: Select all

❯ sudo mdadm --examine /dev/sdd
/dev/sdd:
MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)
I've done simple data recovery and rebuilt some RAID10 mdraid arrays in the past ... but I'm not entirely sure how to proceed and recover a RAID0 array like this. With a regular disk partition I'd normally scan for the superblock, delete the existing Microsoft partition and try to recreate the disk structure / partition layout. But can I even recover/rebuild this? Or should I just give up because it's a RAID0 array and the data is most likely lost?

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#2 Post by theKarlisK »

Testdisk does seem to find "something"

Code: Select all

TestDisk 7.1, Data Recovery Utility, July 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org

Disk /dev/sdd - 1000 GB / 931 GiB - CHS 121601 255 63

     Partition                  Start        End    Size in sectors
>   P Unknown                  0   0  1 121601  80 63 1953525168
Which does look familliar to the other member disk

Code: Select all

TestDisk 7.1, Data Recovery Utility, July 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org

Disk /dev/sdb - 1000 GB / 931 GiB - CHS 121601 255 63

     Partition                  Start        End    Size in sectors
>   P Linux md 1.x RAID        0   0  1 121601  80 63 1953525168 [any:data]

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: RAID0 mdraid member disk recovery

#3 Post by recuperation »

I am not familiar with Linux MDADM raid configurations!
Even running Photorec would not yield too many files due to the sliced representation in your remaining physical drive.

I don't know if the recovery can be done by just using some smart shell commands.
I would do the following:

Get two additional drives
Empty them writing zeros
create a MDADM Raid0 on them
Search the MDADM superblock on both drives
Decode both superblock using https://raid.wiki.kernel.org/index.php/ ... ck_formats

Find the superblock on your remaining RAID0
Decode that superblock
(Assuming that you have partitioned the broken drive like the remaining one)
duplicate (maybe adjust) the partition table information from the good one to the bad one
Guess the superblock on your broken drive using the information gathered from the new working RAID0 array and the remaining drive from your broken RAID0 array.
Try out if MDADM recognizes the RAID structure with two members instead of one.

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#4 Post by theKarlisK »

That's a great suggestion - I'll look into this. I tried looking for the superblock on the trashed disk but I'm worried all traces might have gotten wiped out by windows. What I'm worried about is that, even with the disks being identical (same size, manufacturer, model, firmware) the RAID partition might not get detected properly by various recovery utilities. And without the partition I might not be able to recover the RAID metadata on the disk.

I'm also worried because of how the still intact disk layout looks - judging from this the first sector starts at 0 and last sector ends at 1953525167 (just one sector short of 1953525168 max sectors)"

Code: Select all

Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1           1 1953525167 1953525167 931,5G ee GPT
On my old system with a simillar RAID0 config consisting of 500GB drives, the RAID partition was preceded by some 100MB partition. If the partition starts and ends within the disk boundaries, I'm assuming the RAID metadata has to be present on the actual partition.
I'm currently in the process of creating an image of the trashed disk with testdisk of whatever data it was able to find - I'll try to place that image onto a 3rd disk to see if there's anything in it. If it restores the disk in it's current ruined state, then at least I have a disk to experiment on without further compromising the trashed disk.

EDIT: Oh wait, I might be in luck:

Code: Select all

❯ sudo mdadm --detail /dev/md127
/dev/md127:
           Version : 1.0
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

              Name : any:data
              UUID : e7ee729e:81c9fc54:9ef2cf45:8173dd35
            Events : 0

    Number   Major   Minor   RaidDevice

       -       8       48        -        /dev/sdd
Based on the link above, in v1.0 the superblock resides at the end of the disk.

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: RAID0 mdraid member disk recovery

#5 Post by recuperation »

That's not a surprise as this is assumably refering to the logically undammaged disk.
You need to tune the second disk in a way that MDADM recognizes it as part of your raid array.

It's all about repairing the RAID layer becore any recovery program can start its work.

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#6 Post by theKarlisK »

Well I've only managed to stress myself over this and do little to no progress... I've scanned the disk with various utilities to try to find and attempt recovering the deleted RAID partition but without success. What I keep running up against is that either no recovery tool detects the deleted/lost partition or the utility tries to scan for the RAID but can't find anything because the partition in-place is getting in the way. Inspecting the Testdisk created .dd image revealed it only created an image of the disk in it's current state, would be useful if I there were partitions with data from a failed disk but I'm trying to undo the damn Microsoft Reserved partition. But at the same time I can't bring myself to deleting the partition and trying to recreate it with fdisk - I'm worried changes like that will erase data and I'll irreversibly loose any chance of recovering anything at all.

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: RAID0 mdraid member disk recovery

#7 Post by recuperation »

There is nothing new I could tell you.

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#8 Post by theKarlisK »

Some neutral (no progress) and some positive development. So, as I mentioned before, I was able to identify partition details such as starting block, end block, superblock and other partition/RAID characteristics via various different tools/utilities but my showstopper was that Windows had converted the disk to GPT (this was a slight surprise to discover that the intact disk was a DOS disk, because I distinctly remember having it formatted as GPT disk) and that it had created the previously mentioned "Microsoft Reserved" which kept getting in the way of partition recovery. I tried cloning the trashed disk contents to a different disk to try to recreate the lost partition but that was unsuccessful (as careful as I was I think that at some point it overwrote some crucial data).

Then I switched my focus to attempt RAID0 recovery (I came across different solutions when I was trying to recover the lost partition), I found "ZAR" - Zero Assumption Recovery ( http://z-a-recovery.com/ ) it's only supported on Windows but this was not too much of a problem because I still have the Windows 10 install available on one of my SSDs. To my surprise, it was able to immediately identify that RAID0 data was present on the two RAID disks and displayed the filesystem directories that resided on the RAID (this confirmed my suspicion that mdraid stores raid information on all disks, even for RAID0). At the moment I'm in the process of restoring what data I can.

So far it looks like I've lost all my .qcow2 KVM disk images, unsure if this is because they're in binary format and software doesn't know how to recover them or because the data resided in an overwritten area, but this data was not crucial as it can be rebuilt both from the previous backups or just reconfigured again (all of these were only Test VMs anyway) - I only tried recovering just to see if it works, it was able to recover all the ISO disk images, but this data too was not crucial as it can just be re-downloaded.

I'll keep updating on my progress - someone in a similar situation might come across this day and the information might help them, I'll go back to trying to recover RAID partition after I've recovered data that I have no backups of.

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#9 Post by theKarlisK »

Okay, happy to update that this story has a happy ending! Even tho it doesn't involve testdisk, I wanted to update this with info in case anyone else ever needs to do RAID0 recovery (contrary to other Google results, I'm pleased to say that it is possible).

Through trial and error, as well as with help from Alexey from https://www.z-a-recovery.com (I was running into issues with ZAR, tho, self-inflicted pain - my peculiar partitioning was causing problems in the file recovery process) - with his assistance I was able to recover all my data from a non-functioning Linux software RAID0. A lot of luck was on my side as no data had been damaged, overwritten or any disks were physically damaged. But during the recovery process my 2TB of RAID data grew into ~6TB - this is because the best way to recover all my data, as it turned out, was to attempt RAID recovery, create disk images of the reconstructed RAID0 itself and then of each of the partitions residing on the RAID0 (so, 2TB for the RAID0 consisting of 1TB+1TB drives, and 1TB+500GB+400GB for the partitions residing on the RAID0 ... and since I was paranoid ~2TB for the recovered data copied back to my PC). It's worth noting that something, which I had underestimated, was how powerful the ZAR scan was, caused problems during recovery process for me - because ZAR would pick up so much metadata and file fragments from previous installations and partitions that it was hard to recover the needed data, but this could be a really useful aspect to someone else who's facing a severely damaged RAID data.

To describe the recovery process, as I mentioned above, I needed to use Windows because ZAR only supports Windows (the lack of RAID recovery tools on Linux was an unpleasant surprise), it's licensed software and you will need to purchase the licence, however, you don't need a licence if you only want to scan your drives to see if it even detects anything.
Screenshot_20210519_194257.png
Screenshot_20210519_194257.png (40.32 KiB) Viewed 2883 times
At the beginning (when I was still attempting to do individual file recovery) it seemed like the default settings weren't finding all the data and resulted in a lot of "fragments", so I followed one of the suggested solutions in the self-help resources on the website and disabled "Quick Scan" ... I did the same when creating the images for the RAID and partitions "just in case" but I think Quick Scan would have been enough. This can be done by going into the Advanced Settings when initially launching ZAR.
Screenshot_20210519_194330.png
Screenshot_20210519_194330.png (52.65 KiB) Viewed 2883 times
So to attempt RAID recovery I went with the "RAID Recovery" on the main menu and selected the two hard disks, which, held my RAID0 (sorry, no pictures of this step, I selected the two disks in any order and allowed software to scan them).

After it had found my RAID with a disk missing, I selected it from the list and then proceeded to create a disk image - for this, I used another externally attached disk on which to store the RAID image.
DataImage47.png
DataImage47.png (107.36 KiB) Viewed 2883 times

theKarlisK
Posts: 8
Joined: 04 May 2021, 03:15

Re: RAID0 mdraid member disk recovery

#10 Post by theKarlisK »

To create the image I clicked on "More Functions" (created a new screenshot without RAID loaded, so in my screenshot the option is greyed out, but for you it shouldn't be)
Screenshot_20210519_194455_0.png
Screenshot_20210519_194455_0.png (22.69 KiB) Viewed 2881 times
After I had the RAID image created, I'd use the "Data Recovery for Windows and Linux" and load the disk image, then, in the same fashion as the RAID image was created, select one of the lost partitions and create a new disk image.




Later, when I had all the disk images, I was able to mount the disk images of each partition on my Linux PC via:

Code: Select all

sudo mount -o loop ./VM_image.img /mnt
Additionally, during this I learned that WSL2 (Windows Subsystem for Linux) was capable to mount my XFS partitions - https://devblogs.microsoft.com/commandl ... and-wsl-2/

Which I also tried out successfully by mounting 'Games' partition (because it held no valuable data and I wasn't worried if any would get corrupted/damaged during these tests) on Windows 10 updated to 20H2, note - I used Ubuntu from Windows Store.
DataImage30.png
DataImage30.png (94.69 KiB) Viewed 2881 times
DataImage83.png
DataImage83.png (164.19 KiB) Viewed 2881 times
Continues on next page >>
Last edited by theKarlisK on 19 May 2021, 18:23, edited 1 time in total.

Locked