I didn't wait for so long...
But I will inform with a link to this thread other users at the linux-raid mailing list that I mention in previous post, because I believe it can be of use to other mdadm users.
Here's how the story is hopefully nearing a fine ending.
In case there is no power outage (I owned a UPS, but it's... dead
now after all the years of use...), I might have recovered my data.
No, I'm not touching anything in my at8-g200 box, where the
resync is going on of
/dev/md127.
And it will take long to hopefully accomplish.
Have a look how it went.
Code: Select all
at8-g200@sysresccd % mdadm -Q /dev/sda6
/dev/sda6: is not an md array
/dev/sda6: device 2 in 5 device undetected raid6 /dev/.tmp.md127. Use mdadm --examine for more detail.
at8-g200@sysresccd %
OK. So --examine (-E) we will.
Here it is for all the members of this data raid6,
/dev/sda6 through /dev/sdf6, or /dev/sd[abcdf].
I don't understand why all but /dev/sda6 show at least one "faulty removed" member.
I can be back to read the explanation if anyone can tell us some time in the future.
Since I was given a lot by people like CG Security's author, through my Linux years,
when I can, I also explain things for the next Linux users.
For that same reason, while my data raid6 partition is resyncing (I really expect it to allow
the LVM2 to recognize it and to get my data back just fine), I'll conclude this narrative,
because I believe it might be useful to someone else in the future, just like Chad Walker's plea to which Neil Brown replied (I gave link further above) and he got his data back was useful to me.
Code: Select all
at8-g200@sysresccd % mdadm -E /dev/sda6
/dev/sda6:
Magic : a92b4efc
Version : 0.90.00
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Array Size : 558772224 (532.89 GiB 572.18 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Update Time : Thu Mar 1 10:58:46 2012
State : active
Active Devices : 5
Working Devices : 5
Failed Devices : 0
Spare Devices : 0
Checksum : 958ba4bc - correct
Events : 36125
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 2 8 6 2 active sync /dev/sda6
0 0 8 86 0 active sync /dev/sdf6
1 1 8 54 1 active sync /dev/sdd6
2 2 8 6 2 active sync /dev/sda6
3 3 8 22 3 active sync /dev/sdb6
4 4 8 38 4 active sync /dev/sdc6
at8-g200@sysresccd % mdadm -E /dev/sdb6
/dev/sdb6:
Magic : a92b4efc
Version : 0.90.00
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Array Size : 558772224 (532.89 GiB 572.18 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Update Time : Thu Mar 1 11:11:03 2012
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 3
Spare Devices : 0
Checksum : 958c3502 - correct
Events : 36129
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 3 8 22 3 active sync /dev/sdb6
0 0 8 86 0 active sync /dev/sdf6
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 22 3 active sync /dev/sdb6
4 4 0 0 4 faulty removed
at8-g200@sysresccd % mdadm -E /dev/sdc6
/dev/sdc6:
Magic : a92b4efc
Version : 0.90.00
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Array Size : 558772224 (532.89 GiB 572.18 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Update Time : Thu Mar 1 11:04:19 2012
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 1
Spare Devices : 0
Checksum : 958ba63b - correct
Events : 36126
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 4 8 38 4 active sync /dev/sdc6
0 0 8 86 0 active sync /dev/sdf6
1 1 8 54 1 active sync /dev/sdd6
2 2 0 0 2 faulty removed
3 3 8 22 3 active sync /dev/sdb6
4 4 8 38 4 active sync /dev/sdc6
at8-g200@sysresccd % mdadm -E /dev/sdd6
/dev/sdd6:
Magic : a92b4efc
Version : 0.90.00
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Array Size : 558772224 (532.89 GiB 572.18 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Update Time : Thu Mar 1 11:04:19 2012
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 1
Spare Devices : 0
Checksum : 958ba645 - correct
Events : 36126
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 1 8 54 1 active sync /dev/sdd6
0 0 8 86 0 active sync /dev/sdf6
1 1 8 54 1 active sync /dev/sdd6
2 2 0 0 2 faulty removed
3 3 8 22 3 active sync /dev/sdb6
4 4 8 38 4 active sync /dev/sdc6
at8-g200@sysresccd % mdadm -E /dev/sdf6
/dev/sdf6:
Magic : a92b4efc
Version : 0.90.00
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Array Size : 558772224 (532.89 GiB 572.18 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Update Time : Thu Mar 1 11:11:03 2012
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 3
Spare Devices : 0
Checksum : 958c353c - correct
Events : 36129
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 0 8 86 0 active sync /dev/sdf6
0 0 8 86 0 active sync /dev/sdf6
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 22 3 active sync /dev/sdb6
4 4 0 0 4 faulty removed
at8-g200@sysresccd %
There we need to start from /dev/md127 stopped, as it is below (and not showing
in /dev/mdstat), /dev/md127.
Code: Select all
at8-g200@sysresccd % mdadm -E /dev/md127
mdadm: cannot open /dev/md127: No such file or directory
at8-g200@sysresccd % cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md126 : active raid5 sda5[0] sdf5[4] sdd5[3] sdc5[2] sdb5[1]
9638400 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
md125 : active raid6 sdb2[3] sdf2[0] sdd2[1] sdc2[4] sda2[2]
19277568 blocks level 6, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices: <none>
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -A /dev/md127
mdadm: /dev/md127 not identified in config file.
at8-g200@sysresccd %
Sure enough. It's not in /proc/mdstat.
Code: Select all
at8-g200@sysresccd % mdadm -A /dev/md127 /dev/sd[abcdf]6
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd %
That was not very explanatory. So "-v" is added from now on.
Code: Select all
at8-g200@sysresccd % mdadm -v -A /dev/md127 /dev/sd[abcdf]6
mdadm: looking for devices for /dev/md127
mdadm: cannot open device /dev/sda6: Device or resource busy
mdadm: /dev/sda6 has no superblock - assembly aborted
at8-g200@sysresccd %
Yeah. Forgot to stop it. Now stopping it first.
Code: Select all
at8-g200@sysresccd % mdadm -S /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Here we'll see how the friend mdadm is more talkative.
Attempting to assemle all the members first.
Code: Select all
at8-g200@sysresccd % mdadm -v -A /dev/md127 /dev/sd[abcdf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd %
How this comes about, and why, I have no idea.
Code: Select all
at8-g200@sysresccd % mdadm -S /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Doubling the verbosity below doesn't seem to say any more. No it doesn't.
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abcdf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Let's try different combinations.
Pls. note that there is no rationale behind my different attempts.
Only mere guessing.
I think I have to leave the bash outputs below just as they were.
I like to be positive in my analyses, and not leave things for guessing.
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[bcdf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: no uptodate device for slot 2 of /dev/md127
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acdf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abdf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: no uptodate device for slot 4 of /dev/md127
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abcf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abcd]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[bcf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md127
mdadm: no uptodate device for slot 2 of /dev/md127
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acf]6
mdadm: looking for devices for /dev/md127
mdadm: cannot open device /dev/sdc6: Device or resource busy
mdadm: /dev/sdc6 has no superblock - assembly aborted
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: no uptodate device for slot 4 of /dev/md127
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 2 drives - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[adf]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdf6 is identified as a member of /dev/md127, slot 0.
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: no uptodate device for slot 4 of /dev/md127
mdadm: added /dev/sdf6 to /dev/md127 as 0
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abc]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: no uptodate device for slot 1 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[abd]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdb6 is identified as a member of /dev/md127, slot 3.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 4 of /dev/md127
mdadm: added /dev/sdb6 to /dev/md127 as 3
mdadm: /dev/md127 assembled from 1 drive - not enough to start the array.
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
And here comes the stroke of luck.
Of all the combinations above, and I don't know if there are other possible
combinations, won't spend any more time to investigate... Hey this has already
ruined my schedule as is!
This following combination is the only one that looks capable to be resuscitated.
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acd]6
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: device 2 in /dev/md127 has wrong state in superblock, but /dev/sda6 seems ok
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: /dev/md127 assembled from 3 drives - not enough to start the array while not clean - consider --force.
at8-g200@sysresccd %
That's the Neil Brown's suggestion. Only enshrined in the code, not live as for Chad Walker!
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acd]6 --forceidx
mdadm: unrecognized option '--forceidx'
Usage: mdadm --help
for help
at8-g200@sysresccd %
That's me being a little confused by the events. Of course, it's just "force", not "forceidx".
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acd]6 --force
mdadm: looking for devices for /dev/md127
mdadm: cannot open device /dev/sda6: Device or resource busy
mdadm: /dev/sda6 has no superblock - assembly aborted
at8-g200@sysresccd %
Of course, Miro the Confused forgot to stop the array.
Code: Select all
at8-g200@sysresccd % mdadm -Ss /dev/md127
mdadm: stopped /dev/md127
at8-g200@sysresccd %
Here we go. You ain't getting away from me just yet. Be good, dear friend mdadm!
Code: Select all
at8-g200@sysresccd % mdadm -v -v -A /dev/md127 /dev/sd[acd]6 --force
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda6 is identified as a member of /dev/md127, slot 2.
mdadm: /dev/sdc6 is identified as a member of /dev/md127, slot 4.
mdadm: /dev/sdd6 is identified as a member of /dev/md127, slot 1.
mdadm: clearing FAULTY flag for device 0 in /dev/md127 for /dev/sda6
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: added /dev/sda6 to /dev/md127 as 2
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: added /dev/sdc6 to /dev/md127 as 4
mdadm: added /dev/sdd6 to /dev/md127 as 1
mdadm: /dev/md127 has been started with 3 drives (out of 5).
at8-g200@sysresccd %
Right!
Code: Select all
at8-g200@sysresccd % mdadm -D /dev/md127
/dev/md127:
Version : 0.90
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Array Size : 558772224 (532.89 GiB 572.18 GB)
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Raid Devices : 5
Total Devices : 3
Preferred Minor : 127
Persistence : Superblock is persistent
Update Time : Thu Mar 1 11:04:19 2012
State : clean, degraded
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 128K
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Events : 0.36126
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 54 1 active sync /dev/sdd6
2 8 6 2 active sync /dev/sda6
3 0 0 3 removed
4 8 38 4 active sync /dev/sdc6
at8-g200@sysresccd %
Sure enough, now just add those two remaining members.
Code: Select all
at8-g200@sysresccd % mdadm /dev/md127 -a /dev/sdb6
mdadm: re-added /dev/sdb6
at8-g200@sysresccd % mdadm /dev/md127 -a /dev/sdf6
mdadm: added /dev/sdf6
at8-g200@sysresccd %
VoilĂ ! The raid6 important data partition is alive.
Code: Select all
at8-g200@sysresccd % mdadm -D /dev/md127
/dev/md127:
Version : 0.90
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Array Size : 558772224 (532.89 GiB 572.18 GB)
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Persistence : Superblock is persistent
Update Time : Thu Mar 8 09:31:59 2012
State : clean, degraded, recovering
Active Devices : 3
Working Devices : 5
Failed Devices : 0
Spare Devices : 2
Layout : left-symmetric
Chunk Size : 128K
Rebuild Status : 0% complete
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Events : 0.36131
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 54 1 active sync /dev/sdd6
2 8 6 2 active sync /dev/sda6
6 8 22 3 spare rebuilding /dev/sdb6
4 8 38 4 active sync /dev/sdc6
5 8 86 - spare /dev/sdf6
at8-g200@sysresccd %
Code: Select all
root@sysresccd /root % mdadm -D /dev/md127
/dev/md127:
Version : 0.90
Creation Time : Fri Dec 3 17:28:25 2010
Raid Level : raid6
Array Size : 558772224 (532.89 GiB 572.18 GB)
Used Dev Size : 186257408 (177.63 GiB 190.73 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 127
Persistence : Superblock is persistent
Update Time : Thu Mar 8 10:10:44 2012
State : clean, degraded, recovering
Active Devices : 3
Working Devices : 5
Failed Devices : 0
Spare Devices : 2
Layout : left-symmetric
Chunk Size : 128K
Rebuild Status : 54% complete
UUID : 03a9e99f:6af97c47:82dd0df0:537ac377
Events : 0.36149
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 54 1 active sync /dev/sdd6
2 8 6 2 active sync /dev/sda6
6 8 22 3 spare rebuilding /dev/sdb6
4 8 38 4 active sync /dev/sdc6
5 8 86 - spare /dev/sdf6
root@sysresccd /root %
In the half hour or more, since I began this current post, that's where we're at.
I'll be back just to tell you if LVM2 found its volume. Because the filesystem is on the LVM2 on top of /dev/md127.