Dramatic failure on a RAID 5 Array

Using TestDisk to repair the filesystem
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Posts: 1
Joined: 26 Jul 2016, 12:00

Dramatic failure on a RAID 5 Array

#1 Post by radix »

Hello everybody.
Few days ago one of the servers I am administering (IBM x3650 with ORACLE LINUX 7 O.S.) experienced a serious failure.
Apparently, the problem was with the RAID controller. The system got frozen during the night, exactly when it was running the daily backup task.
Therefore, the backup was not completed and a full day of work for 60 internal users could be lost.
The message displayed during the boot was:

IBM serveRAID 8k-l at PCI Bus:v04, Dev:v00, Func: 00
Fatal Error: Controller Kernel PANIC !!
Unknown firmware error
Error Code: C7

Hence, our first diagnosys was that the controller had gone.
We replaced the controller but the message was the same, hence the problem could be with the mainboard.
We replaced the server but the message was again the same. Hence, the problem was with one of the HDDs.
Removed the first disk of the array (RAID 5 with 6 HDDs) the error did not appear any more and the boot went on.
Unfortunately, the server started in Emergency mode because of an error on /dev/sda at a certain block #.

Now I am running TestDisk to try to recover -as a miminum requirement- the ORACLE database files that are on the server and that contain all the work history of 2 companies.
Of course I have the backups, but they do not cover the last day of work (including the creation of something like 3K end-user invoices) and the Oracle Transaction Log files are on the same Array (therefore, unavailable).

This is the very first time I am using TestDisk to recover data from an ORACLE LINUX 7 server.
Any advice about TestDisk usage best practices will be extremely welcome....

One more info: when starting the emergency mode, the O.S. warns that "No Linux Partitions are found on /dev/sda".

Thanks a lot in advance for your kind help.

Sponsored links

User avatar
Site Admin
Posts: 5307
Joined: 18 Feb 2012, 15:08
Location: Le Perreux Sur Marne, France

Re: Dramatic failure on a RAID 5 Array

#2 Post by cgrenier »

If the partition is on an ext2/3/4 filesystem, try TestDisk, select /dev/sda, Advanced, select the partition, List and copy the file.
Otherwise, it may be possible to mount the partition in read-only (something like mount -o ro /dev/sdaX /mnt/XXX).