Setting up Testdisk for partition repair

Using TestDisk to repair the filesystem
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Locked
Message
Author
nainsurvolte
Posts: 8
Joined: 19 Dec 2012, 23:11

Setting up Testdisk for partition repair

#1 Post by nainsurvolte »

Good day,

It`s my first real attempt at rescuing data from hard drive at such a large scale. I am wondering if someone could help me figure out what is the best approach to succeed.

I lost a Linear array formated as ext3. I tried several solution to recover data "automaticaly" using tools with more user friendly interface but with no luck beside a few files from one of the drive only. The disks are perfectly healthy, no bad sector and no error when looking at SMART status. When trying to tweak the array I believe I accidently created another one over it. Partition's are intact, drives where not formated or used in any way afterward beside trying to recover data.

I decided to give a chance to testdisk despite my limited knowledge of it. The good news, it sees 2 raid array. My guess was one is the good one, the other the one I made by mistake.

I selected the one I believe was the right one.
Next, testdisk selects NONE
Next, testdisk selects Advanced.
At that point I only have one choice, to create an Image.

My first question, is an Image a good option to recover data? My problem is the following, the Linear array is marked at 3998 GB / 3724 GiB, and I have another one that I could empty of 4000 GB / 3726 GiB and I am not sure the image will fit in.

Nonetheless, I went back, went into geometry and selected sector size of 4096. My drives are WD EARS 2TB with 4K sector.

Second question, is that something that I should select ?

I then triggered the expert mode (I saw someting on the forum on that), selected Analyse. It shows me a Linux LVM2 partition with a message "Structure Ok" at the bottom. It also says that nothing can be written because "NONE was selected.

Now my last questions

I have the Deeper search option. Considering the space I have to cover and the rate of the deeper search, I may have for a good 20hrs of scan to reach the 100%, Is there any benefit to go through the whole Deeper Search ?

Following the deeper search, what will most likely be the next steps or options ?

Thanks a lot for any answer.

Phil

nainsurvolte
Posts: 8
Joined: 19 Dec 2012, 23:11

Re: Setting up Testdisk for partition repair

#2 Post by nainsurvolte »

So I made some check here and there and now I believe my initial assessment was wrong.

The md0 partition I could see is the one I created over the good one. The other one I thought it was, is actually the current array I prepared to copy data that I would recover.

I also looked at the individual drives. Each drives as
SWAP2 Partition
Ext3 Partition
RAW partition

In the .filesystem folder of the Ext3 partition, I can see files created in Feb2012 with names like
dsk_mapping
hd_magic_num
raidtab
raidtab2web

And then there are those that seemed to have been changed about at the time everything went wrong
server.pem
Certs.info
cacert.pem
cakey.pem
server-key-nopassword.pem

Can this be of any help? Also, before doing some analysis, it marks all partition as primary. But after the analysis, they have a D for delete besides it. I am not sure what to think of that.

I decided in the meantime to create an image of 1 of the Raw partition to see what it can give me.

Thanks again,

Phil

dragonfly41
Posts: 67
Joined: 14 Sep 2012, 20:51

Re: Setting up Testdisk for partition repair

#3 Post by dragonfly41 »

The md0 partition I could see is the one I created over the good one. The other one I thought it was, is actually the current array I prepared to copy data that I would recover.
The general advice you'll find in this forum is not to use the problem drive as a target for any recovered (copied) data.
The reason is you might inadvertently overwrite data you're trying to recover.
Don't you have another external drive somewhere into which you can place your recovered files?

I have no experience myself in recovering arrays.
But can't you post your testdisk log file for others to view?

nainsurvolte
Posts: 8
Joined: 19 Dec 2012, 23:11

Re: Setting up Testdisk for partition repair

#4 Post by nainsurvolte »

Thanks for the time to look.

I don't think that posting the log would be of any use unless you specify what you need to look at. Considering the back and forth I did, it is a long log file and I can't seem to see how or if it is possible to upload it in the post.

For the time being, I'll refer to section of the log tht may help explain my situation. for technical results, I'll be ready to post new sections.

Here is the drive configuration I have today
[MythBuntu 12.04 drive]
/dev/sda - 250 GB / 232 GiB - CHS 30401 255 63, sector size=512 - ST3250820AS, S/N:5QE1JC38, FW:3.AAE
[MythTV drive for Recordings]
Disk /dev/sdb - 1000 GB / 931 GiB - CHS 121601 255 63, sector size=512 - ST31000340AS, S/N:6QJ03CQM, FW:SD15
[NEW DISKS that can be emptied to recover the files]
Disk /dev/sdc - 2000 GB / 1863 GiB - CHS 243201 255 63, sector size=512 - WDC WD20EARX-00PASB0, S/N:WD-WCAZAJ020124, FW:51.0AB51
Disk /dev/sdd - 2000 GB / 1863 GiB - CHS 243201 255 63, sector size=512 - WDC WD20EARX-00PASB0, S/N:WD-WCAZAF798502, FW:51.0AB51
[Previous JBOD disks]
Disk /dev/sde - 2000 GB / 1863 GiB - CHS 243201 255 63, sector size=512 - WDC WD20EARS-00MVWB0, S/N:WD-WMAZA3771599, FW:51.0AB51
Disk /dev/sdf - 2000 GB / 1863 GiB - CHS 243201 255 63, sector size=512 - WDC WD20EARS-00MVWB0, S/N:WD-WMAZA3770714, FW:51.0AB51
[Disk group made of sdc and sdd]
Disk /dev/mapper/vault--master-Volume1 - 4000 GB / 3726 GiB - CHS 7814037504 1 1, sector size=512
[Disk group made of sde and sdf, the old array]
Disk /dev/md0 - 3998 GB / 3724 GiB - CHS 976234880 2 4, sector size=512
[Logical Volume I guess, made of above disk group]
Disk /dev/dm-0 - 4000 GB / 3726 GiB - CHS 7814037504 1 1, sector size=512
Now I am 100% sure that /dev/mapper/vault--master-Volume1 is the same as /dev/dm-0, because I ran testdisk on /dev/dm-0 and I found the folder structure I have on my new array and some deleted folder from yesterday.

Here is the setup of the Analysis
Partition table type (auto): None
Disk /dev/md0 - 3998 GB / 3724 GiB
Partition table type: None
New geometry
Disk /dev/md0 - 3998 GB / 3724 GiB - CHS 122029360 2 4 sector_size=4096
New options :
Dump : No
Cylinder boundary : Yes
Allow partial last cylinder : No
Expert mode : Yes
FYI, note the sector size that I put to 4096 as those drives are 4K WD drive. But as stated in previous post, I don't know if this is good or not.

As stated, each individual disk has partition as such
Analyse Disk /dev/sde - 2000 GB / 1863 GiB - CHS 243202 255 63
Geometry from i386 MBR: head=255 sector=63
check_part_i386 failed for partition type 83
BAD_RS LBA=1060288 1095050
Current partition structure:
1 P Linux Swap 0 1 2 65 254 61 1060224
No ext2, JFS, Reiser, cramfs or XFS marker
2 P Linux 129 190 14 243200 254 63 3904939697
2 P Linux 129 190 14 243200 254 63 3904939697
4 P Linux 65 254 62 129 190 13 1024080

Now note, the P for Primary partition. After some analysis, it shows me the partition 1 and 4 as deleted (D). I cannot find this in the log, maybe it is a detail that only show up in the terminal, but it buged me.

The partition 4 content
dir_partition inode=2
Linux 65 254 62 129 254 63 1028162
EXT3 Sparse superblock, 526 MB / 502 MiB
Directory /
2 drwxrwxrwx 0 0 1024 14-Nov-2012 20:32 .
2 drwxrwxrwx 0 0 1024 14-Nov-2012 20:32 ..
111761 drwx------ 0 0 1024 29-Feb-2012 08:42 .systemfile
16257 drwxr-xr-x 0 0 1024 29-Feb-2012 08:41 backup
X 11 -rw-r--r-- 0 0 0 14-Nov-2012 20:32 hard_disk_wake
And .systemfile content
dir_partition inode=111761
Linux 65 254 62 129 254 63 1028162
EXT3 Sparse superblock, 526 MB / 502 MiB
Directory /.systemfile
111761 drwx------ 0 0 1024 29-Feb-2012 08:42 .
2 drwxrwxrwx 0 0 1024 14-Nov-2012 20:32 ..
111762 -rw-r--r-- 0 0 7 29-Feb-2012 08:41 dsk_mapping
111763 -rw-r--r-- 0 0 12 29-Feb-2012 08:41 hd_magic_num
111764 -rwxr--r-- 0 0 522 29-Feb-2012 08:41 raidtab
111765 -rwxr-xr-x 0 0 313 29-Feb-2012 08:41 raidtab2web
111766 -rw-r--r-- 0 0 3054 15-Dec-2012 23:01 server.pem
111767 -rwxr-xr-x 0 0 36 15-Dec-2012 23:01 Certs.info
111768 -rw-r--r-- 0 0 1379 15-Dec-2012 23:01 cacert.pem
111769 -rw-r--r-- 0 0 1743 15-Dec-2012 23:01 cakey.pem
111770 -rw-r--r-- 0 0 1675 15-Dec-2012 23:01 server-key-nopassword.pem
As for the rest, Its a bunch of the same information coming over and over as I did a lot of back and forth trying to understand what option were available to me.

And as far as using the current disk, fear not, the partition with the data are unallocated and need to be formated. I cannot write to the disks and will not write to the disk.

I'll stay tuned for any information people may need to understand my situation.

Reading a bit more around JBOD, and remembering some results and graphical information of other recovery software gave me, it made me think, could it matter what position the drives are? Currently the array if configured as
S/N:WD-WMAZA3771599 at position 0 (/dev/sde)
S/N:WD-WMAZA3770714 at position 1 (/dev/sdf)

What if the lost system format information of the old array was
S/N:WD-WMAZA3770714 at position 0 (current /dev/sdf)
S/N:WD-WMAZA3771599 at position 1 (current /dev/sde)

Apparently, from a post of someone who seem knoledgeable and JBOD issue (believe me I will never use that again), the recovery software may be looking for partition information at a place where it does not exist (begining of the drive that was once the middle of the array). In my case, maybe I screwed up to position of the drive when I recreated the array over the old one. This is just a food for thought in case it brings a lead.

Thanks,

Phil.

dragonfly41
Posts: 67
Joined: 14 Sep 2012, 20:51

Re: Setting up Testdisk for partition repair

#5 Post by dragonfly41 »

There are some other recovery tools / techniques discussed in this thread .. which might help.

http://forum.cgsecurity.org/phpBB3/reco ... t1404.html

Also you can upload a zip of testdisk.log .. or create a pastebin account and all you need to post is the pastebin url.

nainsurvolte
Posts: 8
Joined: 19 Dec 2012, 23:11

Re: Setting up Testdisk for partition repair

#6 Post by nainsurvolte »

Thanks for the reference. I almost went through the same things as the guy in this tread already.

ZAR

Tried one drive only, manage to recover some files but really not much. Mostly my music library, some TV shows and a few films, but no MOV from my the little camera. Tried the other drive alone, was unable to get anything. Tried to rebuild the array, JBOD, manyally with first drive being one then the other, failed for both. It seem the most promising tool. The interface like the old defrag tool in windows made me believe it was seeing the files, but in the end, nothing. Severe error at the end and then voila, it was closing.

Easeus

I tried once on 1 drive completely, was not there when it finished, but, software was not running anymore when I came home. I guess it did not work. I does not see the array, or I don't know how to make it see the array. I am currently running it on the other drive alone, while the other one is being imaged by Testdisk. I hate those type of sofware, they try to make the interface look like it was a push button situation.

Diskinternal Raid Recovery

I tried this one as well, but the minute I start it, it become non responsive.

The data is there and I have the feeling that to get it, I have to play smart, but unfortunately, I have limited knowledge of file system and how they work. I can see 1 or 2 link in the thread you pointed out that could be helpful are at least that I could try.

I am continuing my battle for 2 reasons,
1) to gain back some small familly videos and gain time for not recompressing the last films I added to the library (the others are backed up at a friends, me being is back up for his)
2) I took it personnally, I cannot believe that some simple command in linux makes those files unavailable.There must be something that can be done to rebuild the partition the way it was and then scan the disk in correct sequence to locate files.

It was not a working drive, simply a library with little to no read/write activities. Sometimes I wonder if making a quick format to make the array working and remove any trace of the working array, which is not the good one, would not help. Almost looks like the software are puzzled to see an array with RAW disks...

I'll post a log using a url to my BOX account once I have a fresh one that recorded coherent recovery activities and to do that, I would need answers to those questions

- Should I manually set Sector size to 4096K since drives are 4K or not ?
- Do I need to let the Deep Search mode run through completely to get something interesting to look at ?
- Are drive Image (.dd) any good to try ?

I am willing to try anything, but at 4TB, it is long, so might as well set it right before lauching the drives for a scanning spree.

Any help is appreciated,

Phil

dragonfly41
Posts: 67
Joined: 14 Sep 2012, 20:51

Re: Setting up Testdisk for partition repair

#7 Post by dragonfly41 »

I'm following these threads to learn from the experience of others.
So far I'm wary about taking the RAID / JBOD route since it seems (from reading a number of threads) that it is very difficult to recover data if one disk fails.

However .. here is a dated thread which refers to a dual disk setup.

http://www.romhack.net/index.php?post/2 ... -USB-drive

Re: your question, is an Image a good option to recover data?
The reason for "cloning" is so that you can then work on recovery of data from the copy rather than the source. It is an insurance policy. If you don't clone you run the risk of losing your data during your recovery experiments. But sometimes cloning isn't practical because it may require buying another device.

On your other questions you must hope that the testdisk development team come to your help. They know much more than I do.

If you are using ubuntu there is a rich supply of forensic tools to draw on (testdisk included).

nainsurvolte
Posts: 8
Joined: 19 Dec 2012, 23:11

Re: Setting up Testdisk for partition repair

#8 Post by nainsurvolte »

About JBOD, I am learning the same thing as I am living the experience and reading thread at the same time. At first I had 2 different drives in a NAS. I had too much stuff to use Raid 1 and too scared to used Raid 0. I though that JBOD was a no brainer, data is either on one drive or the other and if it fails, just need to read them... Wrong...

After I am done with this, I`ll go on a Linux forum and ask for te best drive configuration for what I wish to do with them and my first concern will be to make sure it is easier to salvage.

Might want to add a set of tools with wich I will play in the next few days, Carver Recovery (based on scalpel) and Scalpel. It works with disk drive Image. I'll fool around with it on a drive image of one of the Raw partition I made to see if it is worth doing a uber disk drive image of 4TB to scan and carve for data. Because if I do so, the disk image will fill the 2 new drives I bought and the only place I will have left are the 2 old disk from which the image will come from.

Thanks for the moral support. I hope to have some answer from Testdisk expert. I`ll keep people posted. Also, considering the amount of information about JBOD recovery (none) should I manage to recover something, you can be sure I'll create a guide for the others having a sword of Damocles over there head.

Phil.

dragonfly41
Posts: 67
Joined: 14 Sep 2012, 20:51

Re: Setting up Testdisk for partition repair

#9 Post by dragonfly41 »

I find the ubuntu forum to be helpful on data recovery ..

I'm sure you can find the bookmarks below by searching "data forensics" or similar

but here they are from my forensics bookmarks .. might save you some time ..

https://help.ubuntu.com/community/DataRecovery

http://www.howtogeek.com/howto/15761/re ... u-live-cd/

http://www.sleuthkit.org/autopsy/desc.php

http://www.caine-live.net/

http://linuxsleuthing.blogspot.co.uk/20 ... aging.html

http://staff.washington.edu/dittrich/forensics.html

http://ubuntu-rescue-remix.org/Version12-04

By the way make sure you are using the latest version of testdisk .. 6.14

[Later thought}

Have you tried photorec to try recovery of your MOV files?

Locked