Evening,
I have had a disk, and it seems the controller, fail for a RAID array somebody else built years ago. It appears the array was mirror/stripe across 3x1tb drives resulting in a 1.5tb volume with single redundancy (and I have no idea why it wasn't 2tb raid5).
I have imaged the two remaining drives and pulled them up with qphotorec_win.exe in which they both show:
1000GB/931GiB flag D no partition whole disk
1498GB/1395GiB flag P type HPFS NTFS
...which I believe is correct, but when I try to extract I'm getting what's clearly a stripe missing for instance I can play FLAC files that skip every other second of the music.
I have been through everything on here that mentions raid and I can't seem to find anything that would help my situation.
Do I have to somehow add multiple images at the same time so it can find everything? Is this a GUI or windows limitation maybe? If anybody would point me in the right direction that would be appreciated!
Thanks, Alex
Odd RAID configuration
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
-
recuperation
- Posts: 3027
- Joined: 04 Jan 2019, 09:48
- Location: Hannover, Deutschland (Germany, Allemagne)
Re: Odd RAID configuration
What's up with the third device? Did it run away? You might consider describing that. Why are two remaining? This is an important part of your case description!user27 wrote: 19 Aug 2024, 21:42 Evening,
I have had a disk, and it seems the controller, fail for a RAID array somebody else built years ago. It appears the array was mirror/stripe across 3x1tb drives resulting in a 1.5tb volume with single redundancy (and I have no idea why it wasn't 2tb raid5).
I have imaged the two remaining drives and pulled them up with qphotorec_win.exe in which they both show:
Except for the rare cases listed in the description of TestDisk, TestDisk does not handle components of virtual devices.
1000GB/931GiB flag D no partition whole disk
1498GB/1395GiB flag P type HPFS NTFS
...which I believe is correct, but when I try to extract I'm getting what's clearly a stripe missing for instance I can play FLAC files that skip every other second of the music.
I have been through everything on here that mentions raid and I can't seem to find anything that would help my situation.
Do I have to somehow add multiple images at the same time so it can find everything? Is this a GUI or windows limitation maybe? If anybody would point me in the right direction that would be appreciated!
Thanks, Alex
TestDisk does not know how your storage controller operates and how the storage controller organized data on the disks under his command.
Your storage controller only shows one disk to your computer, the three ones behind are invisible for your computer.
From your description one can only guess what happened. What do you want to do exactly? What are you trying to achieve?
I hate it when I am left to guessing!
So your third disk has ran away, was stolen or broke down, anyway it is not accessible anymore, right?
Your storage controller was built for this kind of situation. It should provide you with an option to rebuild the array of disks in case you are providing the array with a new, working third disk as the storage controller seems to mirror the usable 1,5 GB on the other remaining 1,5 GB.
You would need to rebuild the array by means of the storage controller as neither your computer, TestDisk or PhotoRec has an idea about the layout of data that your controllers uses.
If the two remaining disks have no errors, you do not need TestDisk or PhotoRec anyway.
Re: Odd RAID configuration
Sorry to leave you guessing for a week; I clearly forgot to click the notify reply tick box!
The third drive was not stolen but appears to have failed catastrophically. The data within is completely inaccessible so to all intents and purposes it may as well have been stolen.
The storage controller they were connected to also seems to be dead; it was an HP controller on the motherboard and I can't get the motherboard to boot even through many hours/days of emergency recovery attempts etc the storage controller is lost.
I have obtained another identical motherboard to try to recover this but the raid menus available at boot don't have any kind of recovery option; the replacement storage controller doesn't recognise that there's an array on these two remaining disks.
What I want to do is recover the data and then rebuild the array; probably in 2TB RAID5 but I may just install larger disks; the point is really the data recovery.
I have ripped images of the two remaining disks to work with so I have in some sense reinstated the redundancy, since there are two copies of the data; I just can't access it.
The third drive was not stolen but appears to have failed catastrophically. The data within is completely inaccessible so to all intents and purposes it may as well have been stolen.
The storage controller they were connected to also seems to be dead; it was an HP controller on the motherboard and I can't get the motherboard to boot even through many hours/days of emergency recovery attempts etc the storage controller is lost.
I have obtained another identical motherboard to try to recover this but the raid menus available at boot don't have any kind of recovery option; the replacement storage controller doesn't recognise that there's an array on these two remaining disks.
What I want to do is recover the data and then rebuild the array; probably in 2TB RAID5 but I may just install larger disks; the point is really the data recovery.
I have ripped images of the two remaining disks to work with so I have in some sense reinstated the redundancy, since there are two copies of the data; I just can't access it.
-
recuperation
- Posts: 3027
- Joined: 04 Jan 2019, 09:48
- Location: Hannover, Deutschland (Germany, Allemagne)
Re: Odd RAID configuration
Here are the remaining solutions for you:
1. Use some "RAID recovery" software tool
2. Have a professional lab recover your data
3. Reverse engineering
The following is way beyond supporting TestDisk.
Attention: I never used a raid configuration - I am totally unexperienced!
Please be aware that no forum user is entitled to claim such kind of support as I may not know the holy grail or simply are not willing to spend hours on describing what one might do in your special case.
Given 3 disks with 1 TB each, a RAID5 array with single parity should have provided you with a usable space of 2 TB.
1,5 TB looks like a RAID1 (mirror) to me. A mirror where disks are mirrored would only give you 1TB of usable space. One disk would contain the data, the other disk the mirror and the third disk is useless.
Given that you had 1,5 TB of space instead, the stripes of your data have to be distributed over 3 x 1 TB = 3TB somehow.
Here is my guess how your layout could be like.
Each letter represents a block of x bytes with x being the size of stripe set:
Disk
1 2 3
A A B
B C C
D D E
E F F
G G H
H I I
Dropping one column, 1 or 2 or 3 out of the table would still provide you with all letters from A to I.
It is weird that your mainboard does not recognize your broken RAID array. This may be because some header information cannot be read out from your old array - unreadible sectors may cause this problem. If I am right you shoud have faced read errors when cloning your disks. You did not mention read errors, though!
Not knowing how similar the on-disc structures of RAID arrays are, I would check your mainboard manual for additional information anyway despite the fact that the controller does not recognize your RAID array. If the RAID function is realized by means of specific hardware, maybe the IC datasheets come with a specification how the stripe layout is set.
If that fails you have to do some work by reverse engineering:
Buy yourself three empty disks (they do not have to be new) having preferrably exact the same size as the broken ones.
Create a new RAID??? structure using the new mainboard with the working controller and your three freshly bought disks.
If your controller has its own BIOS it should show a disk with a size of 1,5 TB in the list of disk devices in your BIOS.
Do some programming:
Just write the sector number into each sector on your virtual 1,5 TB device, based on a sector size of 512 byte a four byte value ("unsigned int") is sufficiently large in your case.
Once you have filled up all sectors, remove the three disks from your controller and connect them directly to your computer.
Go through each disk and read out the sector number information and write them into an individual "map file" for each disk - you just need 4 bytes of space for each sector in your file.
You have now mapped how each sector on the virtual 1,5 TB disk maps onto the three disks.
Be aware that you are analyzing a healthy setup.
Therefore each sector number has to appear twice in your set of three "map files".
To see where sector x is located, search for the value x in all three "map files". The byte position of the found value x divided by four (length of one sector number entry) in a "map file" determines the sector number .
Example:
You find sector number 240 000 in "map file 1" at byte position 160 000. At byte position 160 000 the 40 000th entry is located as each entry has a length of 4 bytes. As a result, the sector number 240 000 is located in disk 1, sector 40 000.
You would then use the scheme to learn which of your old disks correspond to disk1, disk 2 and disk3 because your incident dimished the number of disks from three down to two.
By searching the sector numbers in the files in rising order the stripe scheme should be disclosed to you and you could directly write the content of your virtual device into a big new file on an additional "rescue" disk. If you write that information directly sector-wise into the "rescue" disk you won't have to mount the rescue file on the rescue disk as a disk device - when ready. If reading a sector x fails, you might have duplicate information on another file. Duplicate information is not always available in my scheme above.
Good luck!
1. Use some "RAID recovery" software tool
2. Have a professional lab recover your data
3. Reverse engineering
The following is way beyond supporting TestDisk.
Attention: I never used a raid configuration - I am totally unexperienced!
Please be aware that no forum user is entitled to claim such kind of support as I may not know the holy grail or simply are not willing to spend hours on describing what one might do in your special case.
Given 3 disks with 1 TB each, a RAID5 array with single parity should have provided you with a usable space of 2 TB.
1,5 TB looks like a RAID1 (mirror) to me. A mirror where disks are mirrored would only give you 1TB of usable space. One disk would contain the data, the other disk the mirror and the third disk is useless.
Given that you had 1,5 TB of space instead, the stripes of your data have to be distributed over 3 x 1 TB = 3TB somehow.
Here is my guess how your layout could be like.
Each letter represents a block of x bytes with x being the size of stripe set:
Disk
1 2 3
A A B
B C C
D D E
E F F
G G H
H I I
Dropping one column, 1 or 2 or 3 out of the table would still provide you with all letters from A to I.
It is weird that your mainboard does not recognize your broken RAID array. This may be because some header information cannot be read out from your old array - unreadible sectors may cause this problem. If I am right you shoud have faced read errors when cloning your disks. You did not mention read errors, though!
Not knowing how similar the on-disc structures of RAID arrays are, I would check your mainboard manual for additional information anyway despite the fact that the controller does not recognize your RAID array. If the RAID function is realized by means of specific hardware, maybe the IC datasheets come with a specification how the stripe layout is set.
If that fails you have to do some work by reverse engineering:
Buy yourself three empty disks (they do not have to be new) having preferrably exact the same size as the broken ones.
Create a new RAID??? structure using the new mainboard with the working controller and your three freshly bought disks.
If your controller has its own BIOS it should show a disk with a size of 1,5 TB in the list of disk devices in your BIOS.
Do some programming:
Just write the sector number into each sector on your virtual 1,5 TB device, based on a sector size of 512 byte a four byte value ("unsigned int") is sufficiently large in your case.
Once you have filled up all sectors, remove the three disks from your controller and connect them directly to your computer.
Go through each disk and read out the sector number information and write them into an individual "map file" for each disk - you just need 4 bytes of space for each sector in your file.
You have now mapped how each sector on the virtual 1,5 TB disk maps onto the three disks.
Be aware that you are analyzing a healthy setup.
Therefore each sector number has to appear twice in your set of three "map files".
To see where sector x is located, search for the value x in all three "map files". The byte position of the found value x divided by four (length of one sector number entry) in a "map file" determines the sector number .
Example:
You find sector number 240 000 in "map file 1" at byte position 160 000. At byte position 160 000 the 40 000th entry is located as each entry has a length of 4 bytes. As a result, the sector number 240 000 is located in disk 1, sector 40 000.
You would then use the scheme to learn which of your old disks correspond to disk1, disk 2 and disk3 because your incident dimished the number of disks from three down to two.
By searching the sector numbers in the files in rising order the stripe scheme should be disclosed to you and you could directly write the content of your virtual device into a big new file on an additional "rescue" disk. If you write that information directly sector-wise into the "rescue" disk you won't have to mount the rescue file on the rescue disk as a disk device - when ready. If reading a sector x fails, you might have duplicate information on another file. Duplicate information is not always available in my scheme above.
Good luck!
-
recuperation
- Posts: 3027
- Joined: 04 Jan 2019, 09:48
- Location: Hannover, Deutschland (Germany, Allemagne)
Re: Odd RAID configuration
1.8.2024 11:07 recuperation: georgejohn12 is a spammer who has been banned now.georgejohn12 wrote: 29 Aug 2024, 22:40 Your situation is challenging because you're dealing with a failed RAID 0+1 (or possibly RAID 10) array, which combines striping and mirroring.
I had to look up the definitions of those RAID types you mentioned:
https://www.starline.de/en/magazine/tec ... vs-raid-10
https://en.wikipedia.org/wiki/Nested_RA ... D_1.2B0.29
Obviously, both RAID configuration require an even number of harddisks and it looks like those configurations don't fit but
interestingly, I found a special case in the wikipedia link:
https://en.wikipedia.org/wiki/Non-stand ... MD-RAID-10
User27 most likely has a 3 disk LINUX-MD-RAID-10 configuration.
Source:
- LinuxMDRAID10.PNG (3.12 KiB) Viewed 6896 times
https://en.wikipedia.org/wiki/Non-stand ... MD-RAID-10
The stripe layout of the variant "3 drives" configuration matches my guess of how the stripe layout could be realized under the assumption 3x 1 TB, usable space 1, 5TB.
This information should be helpful when using any commercial program as listed below.
The last sentence is misleading as it only applies to a RAID0 configuration which is excluded by the parameters described by the user: 3 disks with 1 TB each and 1,5 TB of avalaible space.
When a drive in a RAID 0 configuration fails, data is effectively lost since stripes from the missing drive can't be recovered without redundancy. In your case, since you only have two out of the three drives, you’re missing a critical part of the stripe.
At this stage using the information "linux md raid 10" could be helpful, maybe one of those programs allows to select this RAID type. Maybe the mainboard was even using standard linux parameters which could be read out after recreating such an array with the new mainboard and other disks.
Here's how you might approach the recovery:
RAID Reconstruction: You will need to virtually reconstruct the RAID configuration using the two images you have. Tools like ReclaiMe Free RAID Recovery, RAID Reconstructor by Runtime Software, or R-Studio can help you do this. These tools allow you to simulate the RAID configuration and attempt to reconstruct the missing data by combining the available stripes.
Load Multiple Images: Yes, you do need to load the images of both remaining drives simultaneously to attempt a reconstruction. Most RAID recovery software will allow you to add multiple disk images, define the RAID parameters (like stripe size and order), and then try to rebuild the array.
Guesswork and Testing: Without the original RAID configuration details (e.g., stripe size, order), you may need to experiment by adjusting these parameters in the software until you find a configuration that reconstructs the files correctly.
Re: Odd RAID configuration
That's all useful info, thanks again. It is an HP Z820 which has always been on Windows 7 no 'nux involved; although it's hardware raid anyway and the OS just sees one drive.
I think your table is probably correct in that it is mirror stripe across three drives. I suspect now that it was done this way for read performance since RAID5 would be limited to a single drive speed where as this configuration should be able to do near double?
I have been able to get some of those proprietary software options to recover parts of files but nothing will give me complete files without stripe gaps.
Items of not from these include:
ReclaiMe has a specific option for HP with "delayed parity" which I assume the Z820 board is using; this suggests to me that the reason most are failing is that the HP stripes are unusual. However "confidence" never gets above 0% for the entire scan so I guess it doesn't support the three disk mirror/stripe.
DiskInternals seems to find the 1395gb volume but recovered files are incorrectly striped so I presume it doesn't understand the HP delayed parity.
RAID Reconstructor, R-Studio, and other's really didn't show any promise.
I would like to build an identical array using an identical controller and then simply replace the three drives with the two from the array to recover; however it appears to identify drives by serial number so I think I'm going to have to build an array and then clone the drives that need to be recovered onto two of those actual drives.
I think your table is probably correct in that it is mirror stripe across three drives. I suspect now that it was done this way for read performance since RAID5 would be limited to a single drive speed where as this configuration should be able to do near double?
I have been able to get some of those proprietary software options to recover parts of files but nothing will give me complete files without stripe gaps.
Items of not from these include:
ReclaiMe has a specific option for HP with "delayed parity" which I assume the Z820 board is using; this suggests to me that the reason most are failing is that the HP stripes are unusual. However "confidence" never gets above 0% for the entire scan so I guess it doesn't support the three disk mirror/stripe.
DiskInternals seems to find the 1395gb volume but recovered files are incorrectly striped so I presume it doesn't understand the HP delayed parity.
RAID Reconstructor, R-Studio, and other's really didn't show any promise.
I would like to build an identical array using an identical controller and then simply replace the three drives with the two from the array to recover; however it appears to identify drives by serial number so I think I'm going to have to build an array and then clone the drives that need to be recovered onto two of those actual drives.
-
recuperation
- Posts: 3027
- Joined: 04 Jan 2019, 09:48
- Location: Hannover, Deutschland (Germany, Allemagne)
Re: Odd RAID configuration
It is sad that you were beating around the bush by not giving this information. It takes just two searches with a search engine to find the spec sheet of your machine and a RAID guide containing a description of your RAID setup.user27 wrote: 01 Sep 2024, 21:51 That's all useful info, thanks again. It is an HP Z820 which has always been on Windows 7 no 'nux involved; although it's hardware raid anyway and the OS just sees one drive.
Please read about how RAID5 works and how striping affects speed.I think your table is probably correct in that it is mirror stripe across three drives. I suspect now that it was done this way for read performance since RAID5 would be limited to a single drive speed where as this configuration should be able to do near double?
Another hard beating around the bush! You need software that is reading out the virtual disk.I have been able to get some of those proprietary software options to recover parts of files but nothing will give me complete files without stripe gaps.
I don't understand what "Items of not from these" mean.
Items of not from these include:
Maybe the failure of the software above depends on incorrect or missing information of the user.
ReclaiMe has a specific option for HP with "delayed parity" which I assume the Z820 board is using; this suggests to me that the reason most are failing is that the HP stripes are unusual. However "confidence" never gets above 0% for the entire scan so I guess it doesn't support the three disk mirror/stripe.
DiskInternals seems to find the 1395gb volume but recovered files are incorrectly striped so I presume it doesn't understand the HP delayed parity.
RAID Reconstructor, R-Studio, and other's really didn't show any promise.
If so, I guess the RAID controller writes the serial numbers onto the disks and compares them to the serial numbers given by your disk.
I would like to build an identical array using an identical controller and then simply replace the three drives with the two from the array to recover; however it appears to identify drives by serial number so I think I'm going to have to build an array and then clone the drives that need to be recovered onto two of those actual drives.
Thank you for giving a feedback and Good luck!