I inadvertently overwrote the partition table on an Ubuntu system using LVM residing on, I believe a HW RAID1.
The system is/was running Ubuntu 18.04 LTS Server.
I was trying to add an external 10TB USB disk for local backups but I wrote a GPT partition table to the system disk instead of the USB disk.
I don't think I have done anything else that has modified the disks since updating the partition table.
The original machine is powered off right now.
I have used dd to make take an image copy of the disk and am trying out testdisk on that image file.
Below is the log file from an initial inspection of it.
Questions:
1. Is there a way to just extract the files from the LVM partition?
2. If I run testdisk from a liveDVD and have it update the GPT table would that make my original system bootable again?
3. Can I update the GPT table on the image file. Save it and then on a spare machine write that updated image to a raw disk to
make it bootable? If so, what tool to use for that?
Thanks for any help.
------------------------------------------------------
Using locale 'en_US.UTF-8'.
Thu Feb 18 13:54:55 2021
Command line: TestDisk /log /logname /home/john/testdiskLog.log /diske/backup_image_sda.img
TestDisk 7.0, Data Recovery Utility, April 2015
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org
OS: Linux, kernel 4.15.0-135-generic (#139-Ubuntu SMP Mon Jan 18 17:38:24 UTC 2021) x86_64
Compiler: GCC 7.2
ext2fs lib: 1.44.1, ntfs lib: libntfs-3g, reiserfs lib: none, ewf lib: none, curses lib: ncurses 6.0
User is not root!
Hard disk list
Disk /diske/backup_image_sda.img - 999 GB / 931 GiB - CHS 121535 255 63 (RO), sector size=512
Partition table type (auto): Intel
Media is opened in read-only.
Geometry from i386 MBR: head=255 sector=63
Disk /diske/backup_image_sda.img - 999 GB / 931 GiB (RO)
Partition table type: Intel
Analyse Disk /diske/backup_image_sda.img - 999 GB / 931 GiB - CHS 121535 255 63 (RO)
Geometry from i386 MBR: head=255 sector=63
BAD_RS LBA=1 0
Current partition structure:
1 P EFI GPT 0 0 2 121534 76 14 1952448511
Bad relative sector.
No partition is bootable
search_part()
Disk /diske/backup_image_sda.img - 999 GB / 931 GiB - CHS 121535 255 63 (RO)
Linux 6 127 58 25 126 37 305152
ext3 blocksize=1024 Sparse_SB, 156 MB / 149 MiB
Linux LVM 25 127 38 121533 254 63 1952034047
LVM2, 999 GB / 930 GiB
interface_write()
1 * Linux 6 127 58 25 126 37 305152
2 P Linux LVM 25 127 38 121533 254 63 1952034047
simulate write!
No extended partition
TestDisk exited normally.
Wiped out my Partition table
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
-
recuperation
- Posts: 2729
- Joined: 04 Jan 2019, 09:48
- Location: Hannover, Deutschland (Germany, Allemagne)
Re: Wiped out my Partition table
I am lacking linux experience and linux system recovery experience - so take every hint with a grain of salt.jdehart1 wrote: ↑18 Feb 2021, 21:10 Questions:
1. Is there a way to just extract the files from the LVM partition?
2. If I run testdisk from a liveDVD and have it update the GPT table would that make my original system bootable again?
3. Can I update the GPT table on the image file. Save it and then on a spare machine write that updated image to a raw disk to
make it bootable? If so, what tool to use for that?
First the report states that did not run as root. Running the report as root might give different results.
Answers:
1. You will have to try that out. Maybe it works if it's just about mirroring (Raid1). If LVM is not running in case of Raid5-array p.e. Testdisk would need to interpret LVM data to manage the member drives in able to pull out data. I doubt that.
2. The always repeating question of success guarantee cannot be answered because you can't rely on witness reports like yours. Furthermore, boot processes now can vary a lot with so many new boot techniques in place. I can't and don't want to support boot drive repairs - will typically answer recovery questions.
3. You can try that. dd or better ddrescue would be used to restore your image to a different drive. You will just have to reverse the backup process you triggered to generate your image file.
Re: Wiped out my Partition table
Thanks for the quick reply.recuperation wrote: ↑18 Feb 2021, 22:51I am lacking linux experience and linux system recovery experience - so take every hint with a grain of salt.jdehart1 wrote: ↑18 Feb 2021, 21:10 Questions:
1. Is there a way to just extract the files from the LVM partition?
2. If I run testdisk from a liveDVD and have it update the GPT table would that make my original system bootable again?
3. Can I update the GPT table on the image file. Save it and then on a spare machine write that updated image to a raw disk to
make it bootable? If so, what tool to use for that?
First the report states that did not run as root. Running the report as root might give different results.
Answers:
1. You will have to try that out. Maybe it works if it's just about mirroring (Raid1). If LVM is not running in case of Raid5-array p.e. Testdisk would need to interpret LVM data to manage the member drives in able to pull out data. I doubt that.
2. The always repeating question of success guarantee cannot be answered because you can't rely on witness reports like yours. Furthermore, boot processes now can vary a lot with so many new boot techniques in place. I can't and don't want to support boot drive repairs - will typically answer recovery questions.
3. You can try that. dd or better ddrescue would be used to restore your image to a different drive. You will just have to reverse the backup process you triggered to generate your image file.
I will try #3 first before I move to #2.
Re: Wiped out my Partition table
I have recovered my files! Whew.
I was actually able to perform #1 from above, just extract the files from the LVM partition in the disk image.
Below I will give the detailed commands used.
To recap why I am here, I had overwritten my partition table with a blank GPT table.
The disk in question had an LVM partition that contained three filesystems of interest: /, /export and /users.
Step 0: DON'T PANIC. When I realized what I had done, I stopped immediately and made a 'dd' image copy of the disk.
Step 1: Continue to not Panic. In searching the internet, I learned about 'testdisk'.
Step 2: I built a spare machine with 4 1.2 TB disks. One was the system disk and 3 were spares for experimenting
on my 1TB disk image. I also attached a 10TB USB drive. I copied the disk image to the 3 spare drives and the 10TB drive.
Lots of backup copies just in case.
Step 3: Used 'testdisk' to find the partitions, write a proper partition table and write it back out to my disk image file.
I'll see if I can find where I did that in my log file, but here is a recreation of the steps.
Step 4: Learn about LVM and mounting an LVM partition from a disk image.
Step 5: Find the starting point (in bytes) and size (in bytes) of the LVM partition:
Step 6: Use those values with losetup to point a loop device at my LVM partition.
Step 7: check the status of things with some 'blkid', 'pvs', and 'lsblk'
Step 8: Activate the volume group
Step 9: Check the device manager view of volume group
Step 10: Make mount points for the three I am interested in (don't need swap).
Step 11: Mount them
Step 12: Check that they look like I expected:
Step 13: Copy Everything to my 10TB USB disk.
Step 14: Breath a sigh of relief.
And in case you are interested in the tear down of the above, here are the commands I used for that.
I was actually able to perform #1 from above, just extract the files from the LVM partition in the disk image.
Below I will give the detailed commands used.
To recap why I am here, I had overwritten my partition table with a blank GPT table.
The disk in question had an LVM partition that contained three filesystems of interest: /, /export and /users.
Step 0: DON'T PANIC. When I realized what I had done, I stopped immediately and made a 'dd' image copy of the disk.
Step 1: Continue to not Panic. In searching the internet, I learned about 'testdisk'.
Step 2: I built a spare machine with 4 1.2 TB disks. One was the system disk and 3 were spares for experimenting
on my 1TB disk image. I also attached a 10TB USB drive. I copied the disk image to the 3 spare drives and the 10TB drive.
Lots of backup copies just in case.
Step 3: Used 'testdisk' to find the partitions, write a proper partition table and write it back out to my disk image file.
I'll see if I can find where I did that in my log file, but here is a recreation of the steps.
Code: Select all
> testdisk /diskc/backup_image_sda.img
- Select disk image [Proceed]
- Select Intel partition type [Enter]
- [Analyse]
- testdisk reports EFI GPT partition table, Bad relative sector, No partition is bootable.
- [Quick Search]
- testdisk finds two partitions: A bootable Linux partition and a Primary Linux LVM partition.
- [Enter]
- [Write], Y, [OK]
- [Quit]
- [Quit]
> mv /diskc/backup_image_sda.img /diskc/update_partition_table.img
Step 5: Find the starting point (in bytes) and size (in bytes) of the LVM partition:
Code: Select all
john@forest2:~$ fdisk -l /diskc/update_partition_table.img
Disk /diskc/update_partition_table.img: 931 GiB, 999653638144 bytes, 1952448512 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000
Device Boot Start End Sectors Size Id Type
/diskc/update_partition_table.img1 * 104448 409599 305152 149M 83 Linux
/diskc/update_partition_table.img2 409663 1952443709 1952034047 930.8G 8e Linux LVM
john@forest2:~$ sudo fdisk -l /diskc/update_partition_table.img
Disk /diskc/update_partition_table.img: 931 GiB, 999653638144 bytes, 1952448512 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000
Device Boot Start End Sectors Size Id Type
/diskc/update_partition_table.img1 * 104448 409599 305152 149M 83 Linux
/diskc/update_partition_table.img2 409663 1952443709 1952034047 930.8G 8e Linux LVM
john@forest2:~$ bc
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
409663 * 512
209747456
1952034047 * 512
999441432064
quit
Code: Select all
john@forest2:~$ sudo losetup -v -f -o 209747456 --sizelimit 999441432064 /diskc/update_partition_table.img
john@forest2:~$ sudo losetup -a
/dev/loop1: [2049]:66981149 (/var/lib/snapd/snaps/gnome-3-34-1804_66.snap)
/dev/loop8: [2049]:66981153 (/var/lib/snapd/snaps/gnome-logs_103.snap)
/dev/loop6: [2049]:66981146 (/var/lib/snapd/snaps/core18_1988.snap)
/dev/loop4: [2049]:66981144 (/var/lib/snapd/snaps/gnome-3-26-1604_100.snap)
/dev/loop2: [2049]:66981154 (/var/lib/snapd/snaps/gnome-characters_570.snap)
/dev/loop0: [2049]:66981150 (/var/lib/snapd/snaps/gnome-calculator_826.snap)
/dev/loop9: [2081]:12 (/diskc/update_partition_table.img), offset 209747456, sizelimit 999441432064
/dev/loop7: [2049]:66981148 (/var/lib/snapd/snaps/gtk-common-themes_1514.snap)
/dev/loop5: [2049]:66981158 (/var/lib/snapd/snaps/gnome-system-monitor_157.snap)
/dev/loop3: [2049]:66981130 (/var/lib/snapd/snaps/core_10823.snap)
john@forest2:~$ sudo blkid /dev/loop9
/dev/loop9: UUID="q4m1Qd-GoEj-aewp-f8OP-rt1m-Hk9D-iufZpm" TYPE="LVM2_member"
Code: Select all
john@forest2:~$ sudo blkid /dev/loop9
/dev/loop9: UUID="q4m1Qd-GoEj-aewp-f8OP-rt1m-Hk9D-iufZpm" TYPE="LVM2_member"
john@forest2:~$ sudo pvs
WARNING: Not using lvmetad because duplicate PVs were found.
WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
WARNING: After duplicates are resolved, run "pvscan --cache" to enable lvmetad.
PV VG Fmt Attr PSize PFree
/dev/loop9 vg_onlsrv3 lvm2 a-- 930.80g 0
john@forest2:~$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 2.5M 1 loop /snap/gnome-calculator/826
loop1 7:1 0 219M 1 loop /snap/gnome-3-34-1804/66
loop2 7:2 0 276K 1 loop /snap/gnome-characters/570
loop3 7:3 0 98.4M 1 loop /snap/core/10823
loop4 7:4 0 140.7M 1 loop /snap/gnome-3-26-1604/100
loop5 7:5 0 2.2M 1 loop /snap/gnome-system-monitor/157
loop6 7:6 0 55.5M 1 loop /snap/core18/1988
loop7 7:7 0 64.8M 1 loop /snap/gtk-common-themes/1514
loop8 7:8 0 548K 1 loop /snap/gnome-logs/103
loop9 7:9 0 930.8G 0 loop
├─vg_onlsrv3-lv_root 253:0 0 9.8G 0 lvm
├─vg_onlsrv3-lv_swap 253:1 0 8G 0 lvm
├─vg_onlsrv3-lv_export 253:2 0 48.8G 0 lvm
└─vg_onlsrv3-lv_users 253:3 0 864.2G 0 lvm
sda 8:0 0 1.1T 0 disk
├─sda1 8:1 0 1T 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 64G 0 part [SWAP]
sdb 8:16 0 1.1T 0 disk
└─sdb1 8:17 0 1.1T 0 part /diskb
sdc 8:32 0 1.1T 0 disk
└─sdc1 8:33 0 1.1T 0 part /diskc
sdd 8:48 0 1.1T 0 disk
└─sdd1 8:49 0 1.1T 0 part /diskd
sde 8:64 0 9.1T 0 disk /diske
sr0 11:0 1 1024M 0 rom
Code: Select all
john@forest2:~$ sudo vgchange -a y vg_onlsrv3
4 logical volume(s) in volume group "vg_onlsrv3" now active
john@forest2:~$ sudo dmsetup ls
vg_onlsrv3-lv_swap (253:1)
vg_onlsrv3-lv_root (253:0)
vg_onlsrv3-lv_export (253:2)
vg_onlsrv3-lv_users (253:3)
Code: Select all
john@forest2:~$ sudo dmsetup info /dev/dm-0
Name: vg_onlsrv3-lv_root
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: LVM-jS2YjC52D6fX4hsE4Ds6Ap6Tj7ypQ1dcNnQ75Dn3U7VVXE6cqUoqhV12pBwcNREs
john@forest2:~$ sudo dmsetup info /dev/dm-1
Name: vg_onlsrv3-lv_swap
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 253, 1
Number of targets: 1
UUID: LVM-jS2YjC52D6fX4hsE4Ds6Ap6Tj7ypQ1dcWgs4itJkTM00GK1DSeQxFJ2zOj3Cpffx
john@forest2:~$ sudo dmsetup info /dev/dm-2
Name: vg_onlsrv3-lv_export
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 253, 2
Number of targets: 1
UUID: LVM-jS2YjC52D6fX4hsE4Ds6Ap6Tj7ypQ1dcBtwgUxCw79IagR0hhg3jjr5exNSV8KVb
john@forest2:~$ sudo dmsetup info /dev/dm-3
Name: vg_onlsrv3-lv_users
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 253, 3
Number of targets: 1
UUID: LVM-jS2YjC52D6fX4hsE4Ds6Ap6Tj7ypQ1dcfqDEPGOwNBms3o8rCxG8yAH9RbUwQiXS
Code: Select all
john@forest2:~$ sudo mkdir /onlsrv_root
john@forest2:~$ sudo mkdir /onlsrv_export
john@forest2:~$ sudo mkdir /onlsrv_users
Code: Select all
john@forest2:~$ sudo mount /dev/dm-0 /onlsrv_root/
john@forest2:~$ sudo mount /dev/dm-2 /onlsrv_export
john@forest2:~$ sudo mount /dev/dm-3 /onlsrv_users
Code: Select all
john@forest2:~$ ls /onlsrv_root/
backups dev initrd.img.old mnt rec srv users
bacula-console.conf.ucftmp-jPTW5m89ug etc lib mnt2 root sys usr
bacula-fd.conf.ucftmp-nF0KvuW8vK export lib64 offload run tftpboot var
bin home lost+found packages.expandrive.gpg sbin tftpboot.export_link vmlinuz
boot initrd.img media proc scratch tmp vmlinuz.old
john@forest2:~$ ls /onlsrv_export/
backup ixp_files keeboot_files kernel_sources lost+found onl simpana UBUNTU_18_04_disklesskernel_for_tftpboot
ExpanDrive ixp_logs keeboot_kernels KVM_Images old-onlsrv.bkup patch tftpboot
john@forest2:~$ ls -1 /onlsrv_users | head 10
head: cannot open '10' for reading: No such file or directory
john@forest2:~$ ls -1 /onlsrv_users | head -10
0xsiow
aadil
aagatstein
aarellano
aarthi
aarti
aayuan
aayush
abdul
abdullah
john@forest2:~$ du -sh /onlsrv_
onlsrv_export/ onlsrv_root/ onlsrv_users/
john@forest2:~$ sudo du -sh /onlsrv_*
44G /onlsrv_export
8.3G /onlsrv_root
680G /onlsrv_users
john@forest2:~$
Step 14: Breath a sigh of relief.
And in case you are interested in the tear down of the above, here are the commands I used for that.
Code: Select all
john@forest2:~$ sudo umount /onlsrv_export
john@forest2:~$ sudo umount /onlsrv_root
john@forest2:~$ sudo umount /onlsrv_users
john@forest2:~$ sudo vgchange -a n vg_onlsrv3
0 logical volume(s) in volume group "vg_onlsrv3" now active
john@forest2:~$ sudo losetup -a
/dev/loop1: [2049]:66981149 (/var/lib/snapd/snaps/gnome-3-34-1804_66.snap)
/dev/loop8: [2049]:66981153 (/var/lib/snapd/snaps/gnome-logs_103.snap)
/dev/loop6: [2049]:66981146 (/var/lib/snapd/snaps/core18_1988.snap)
/dev/loop4: [2049]:66981144 (/var/lib/snapd/snaps/gnome-3-26-1604_100.snap)
/dev/loop2: [2049]:66981154 (/var/lib/snapd/snaps/gnome-characters_570.snap)
/dev/loop0: [2049]:66981150 (/var/lib/snapd/snaps/gnome-calculator_826.snap)
/dev/loop9: [2081]:12 (/diskc/update_partition_table.img), offset 209747456, sizelimit 999441432064
/dev/loop7: [2049]:66981148 (/var/lib/snapd/snaps/gtk-common-themes_1514.snap)
/dev/loop5: [2049]:66981158 (/var/lib/snapd/snaps/gnome-system-monitor_157.snap)
/dev/loop3: [2049]:66981130 (/var/lib/snapd/snaps/core_10823.snap)
john@forest2:~$ sudo losetup -d /dev/loop9
john@forest2:~$