deleted wrong partition (LUKS)

Message

#11 Post by **recuperation** » 11 Sep 2022, 19:42

karenmcd · #12 Post by **karenmcd** » 12 Sep 2022, 17:01

I'll make a new backup of my old LUKS drive and try again tomorrow after the copy completes. It seems that all the links and help I've been given are for fat / dos type disks. My disk must have been GPT, and I wrote the dos style one from TestDisk which seems to have badly corrupted it so that it's not even finding the LUKS partition anymore. It seems the thing that could have prevented me from doing it wrong would have been to know that over 2TB requires GPT.

karenmcd · #13 Post by **karenmcd** » 13 Sep 2022, 16:50

I'm reading gdisk documentation from https://www.rodsbooks.com/gdisk/repairing.html while I wait for the EFI/GPT quick scan to complete. It says under recovery a few things that look relevant to my situation. I'm thinking that I have a broken EFI/GPT because the disk in question is 10 TB and the partition was 6TB. Because the scan using "Intel" found what looks like a normal LUKS/Linux partition, I'm guessing that I had perhaps a hybrid MBR/GPT setup. I've also noticed that the drive I got to restore my 10TB disk appears slightly different and smaller than the original disk.

Because of the slight geometry/size difference of the new backup drive that I made a dd copy to I'm wondering if I should do this?

The e option on the experts' menu relocates the backup data structures to the end of the disk. This option can be useful in some recovery situations because, if the headers are damaged, GPT fdisk will become confused about where the backup data structures should reside, and as a result an attempt to write the data (using w) will fail. The e option causes the program to recompute this information. This option is also useful if you've resized a hardware RAID array or done a low-level copy of one disk's contents to another one of another size; in both these cases, the backup data structures will no longer reside at the end of the disk, so they should be relocated.

Once that's done it seems that perhaps doing the following in gdisk might help too. I'm thinking that the following quote might be a relevant option because running the TestDisk "Intel" scan did seem to find a normal looking Linux/LUKS partition

The f option on the recovery & transformation menu loads the MBR and generates a fresh GPT from it. This option can be useful if GPT fdisk mistakenly interpreted an MBR disk as a GPT disk, or if the GPT structures on a hybrid MBR disk were very badly damaged but the MBR definitions remain intact.

Do either of these things seem like a good idea before attempting to go any further with TestDisk?

I'm also thinking I'm in way over my head and unsure how or where to find professional disk recovery services. Searches for anything local come up with a company that told me they "only deal with businesses" and the next closest says they are 800 kilometres away but is going to ship the drive 1800 kilometres AFTER I ship it to them because "they don't actually do the work". I'm worried the disk will get damaged during shipping whereas right now it "only" has a deleted partition table. Is there anyone/place proficient in data recovery in northern BC, Canada reading these forums that I could solicit help from directly?

#14 Post by **recuperation** » 13 Sep 2022, 21:09

karenmcd wrote: ↑12 Sep 2022, 17:01 I'll make a new backup of my old LUKS drive

Instead of "I press a dd" (???) use ddrescue as described in the manual. Use the logfile feature and keep the logfile for various reasons.

and try again tomorrow after the copy completes. It seems that all the links and help I've been given are for fat / dos type disks. My disk must have been GPT,

If your disk was not enclosed in a case with sector size converting electronics GPT is the right mode for this 10 TB drive.

and I wrote the dos style one from TestDisk

Did you provide this information somewhere?!

which seems to have badly corrupted it

As this forum is about support of Testdisk and Photorec you might have considered providing a complete documentation instead of discussing various articles on different websites. You might have cross-read other postings to see how this forum operates. Five weeks after your inital posting you came to the conclusion that it might be worth telling about your actions because suddenly you fear of having something done wrong.
As you did not document your modifications I can't tell you if you successfully massacred your drive but I doubt that anyway.
Your logfile mentions LUKS, by the way.

so that it's not even finding the LUKS partition anymore. It seems the thing that could have prevented me from doing it wrong would have been to know that over 2TB requires GPT.

Yes, you need the GPT mode. There is no "hybrid GPT" as you stated. GPT contains a protective MBR to prevent legacy operating system to write on a GPT-partitioned disk.

karenmcd · #15 Post by **karenmcd** » 14 Sep 2022, 08:38

If your disk was not enclosed in a case with sector size converting electronics GPT is the right mode for this 10 TB drive.

So now I have a question. I'm copying a 10TB disk that was never used in the enclosure (internal disk located at /dev/sda) to a USB 3.0 external drive that IS in an enclosure, and attempting recovery on that new external drive (located at /dev/sdb). Does that mean that:

Code: Select all

# dd if=/dev/sda of=/dev/sdb

is not working properly because the enclosured USB external drive breadboard is interpreting the 4096 to groups of 512 like I read online @ rodsbooks.com? So it's not actually doing a proper lowlevel copy from the internal disk to the enclosed disk to make it work with mbr type systems using the 512/4096 workaround?

I mean this is super interesting to learn that companies are so resistant to changing to GPT from MBR and have come up with these weird ways to work around having to comply with GPT but f#@K!

Instead of "I press a dd" (???) use ddrescue as described in the manual. Use the logfile feature and keep the logfile for various reasons.

I super appreciate your patience and will read the documentation about and attempt ddrescue. I've never used it before, and I'll be honest, perhaps partly in my stubbornness, and partly because I'm more familiar with it, I figured dd is the better application for doing low level copies of disks. Does ddrescue get around this whole GPT / MBR problem or do I need to remove this new disk from the enclosure and work with it directly to get a proper GPT copy?

It also seems the new disk is also a tiny bit too small. I might actually be better off to go buy a slightly larger 12TB disk or something because I'm reading that it's pretty important that the new backup drive be larger than the drive I'm trying to recover anyway. IDK if it matters that I only used 6TB of the original 10TB drive, as most of the documentation says GPT backup is written on the last sector of the drive, but is overwritten when the regular table is written anyway.

Did you provide this information somewhere?!

No, I literally had just attempted that. I just wrote that to explain that it might take a few days for me to reply again, as I have to copy the original disk that has the deleted partition back to this new disk again before making another recovery attempt. So I was just giving context. As you know probably it takes a bit of time to deal with data amounts this large. Also, when I "wrote" the table it said it was unsuccessful because the partition was larger than 2TB which is what lead me to read about all this stuff anyway. I also only just today learnt that there's firmware in enclosed drives that's allowing disks larger than 2 TB to use MBR and could be interfering with this recovery perhaps.

Also looking at my post, and having re-read the instructions, I marked my Linux partition with a P instead of a D and then re-writing my start and end locations as instructed to do. I'm not sure what to do now though because my original disk is GPT and the backup is MBR probably because of the enclosure.

Should I remove the backup drive from the enclosure and use ddrescue and then re-attempt the recovery or should I get a larger drive and then ddrescue and attempt the recovery?

The GPT disk screwed up by deleting the partition table accidentally is 9314GiB and the new disk I've been working on backups of is 9313GiB. Does this matter when my LUKS partition was only using 6TB?

Again, thanks for your patience and help.

PS This is a side note and maybe I'm totally wrong here, but there is hybrid MBR/GPT https://www.rodsbooks.com/gdisk/hybrid.html . Regardless your point still stands. MBR systems will not see the GPT partitions as I understand it and I was just confused as I thought that this meant that I could read the MBR and recover partition table information from it. the only reason I brought it up or even learned about it is because from this link: https://www.rodsbooks.com/gdisk/repairing.html about repairing GPT partitions:

The f option on the recovery & transformation menu loads the MBR and generates a fresh GPT from it. This option can be useful if GPT fdisk mistakenly interpreted an MBR disk as a GPT disk, or if the GPT structures on a hybrid MBR disk were very badly damaged but the MBR definitions remain intact.

Which seemed super relevant when a person (me) accidentally dd if=GPT of=MBR(because of enclosure board)... and then use Intel TestDisk mode to try and recover a partition maybe?

karenmcd · #16 Post by **karenmcd** » 14 Sep 2022, 09:04

Code: Select all

TestDisk 7.2-WIP, Data Recovery Utility, Novembre 2020
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org

Disk /dev/sda - 10000 GB / 9313 GiB - CHS 9537535 64 32

The harddisk (10000 GB / 9313 GiB) seems too small! (< 17238299 TB / 15678142 TiB)
Check the harddisk size: HD jumper settings, BIOS detection...

The following partitions can't be recovered:
     Partition               Start        End    Size in sectors
>  BeFS                  5259034145 26342049315855935 26342044056821791 [PY~_~Q�~U�����k�^RZ�~~~Fx>�~Q^RPq>^_ja�o]
   BeFS                  8247900677 7365589529860612 7365581281959936 [}a~G^S�DA>~X~N�3(B^G�M6hu9��b~T�S]�Q�^_]
   BeFS                  8844626297 33668553830713720 33668544986087424 [��~P���~Ds�C�=�Hn~Qkں~Z~B^S~U{L~L~W^V�^B��]
I^WR^Ye)��� �:~P�E��~J-�X^[~Rd@ &�B] 28089665481544234 28089654847124480 [~^�

[ Continue ]
BeFS blocksize=512, 13487126 TB / 12266470 TiB

Also, here's the latest log file from that same run. It seems that the internal GPT disk is not copying properly to the external (in enclosure) disk properly and is just coming up with a ton of errors. Perhaps this is overcome with ddrescue. I'll read the documentation and decided what to do a bit later I guess.

Code: Select all

Tue Sep 13 07:41:08 2022
Command line: TestDisk /debug /log /dev/sda

TestDisk 7.2-WIP, Data Recovery Utility, Novembre 2020
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org
OS: Linux, kernel 5.15.60-1-MANJARO (#1 SMP PREEMPT Thu Aug 11 13:14:05 UTC 2022) x86_64
Compiler: GCC 4.4
ext2fs lib: 1.42.8, ntfs lib: libntfs-3g, reiserfs lib: 0.3.1-rc8, ewf lib: 20120504, curses lib: ncurses 5.7
Hard disk list
Disk /dev/sda - 10000 GB / 9313 GiB - CHS 9537535 64 32, sector size=512 - Seagate Expansion HDD, FW:1801

Partition table type (auto): Intel
Disk /dev/sda - 10000 GB / 9313 GiB - Seagate Expansion HDD
Partition table type: EFI GPT

Analyse Disk /dev/sda - 10000 GB / 9313 GiB - CHS 9537535 64 32
hdr_size=92
hdr_lba_self=1
hdr_lba_alt=19532873727 (expected 19532873726)
hdr_lba_start=34
hdr_lba_end=19532873694
hdr_lba_table=2
hdr_entries=128
hdr_entsz=128
Trying alternate GPT
Bad GPT partition, invalid signature.
Current partition structure:
Trying alternate GPT
Bad GPT partition, invalid signature.

search_part()
Disk /dev/sda - 10000 GB / 9313 GiB - CHS 9537535 64 32
FAT32 at 0/1/9
check_FAT: Unusual media descriptor (0xf0!=0xf8)
FAT1 : 32-3182
FAT2 : 3183-6333
start_rootdir : 6334 root cluster : 2
Data : 6334-409599
sectors : 409600
cluster_size : 1
no_of_cluster : 403266 (2 - 403267)
fat_length 3151 calculated 3151
FAT differs, FAT sectors=0-16/3151
heads/cylinder 16 (FAT) != 64 (HD)
set_FAT_info: name from BS used

FAT32 at 0/1/9
     EFI System                    40     409639     409600 [EFI System Partition] [EFI]
     FAT32, blocksize=512, 209 MB / 200 MiB
check_FAT: Bad number of sectors per cluster

SYSV4 Marker at 340678/33/28

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown                697709627 69706894472250 69706196762624 [\93
hM\F5]
     SysV4, 35689 TB / 32459 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

LVM magic value at 429677/13/22
check_FAT: Bad number of sectors per cluster

SYSV4 Marker at 473467/27/27

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown                969661306  969661305          0 ['\BD"\C9@b]
     SysV4, 0 B
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

cramfs Marker at 643344/25/1

recover_cramfs
     Linux filesys. data   1317569312 1322595384    5026072 [\E9\C7}`k\DFy|\D5[\F1\F5#;z]
     cramfs, 2573 MB / 2454 MiB
check_FAT: Bad number of sectors per cluster

SYSV4 Marker at 1023548/2/21

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               2096226388 1057051287123 1054955060736 [;@\C0\9F]
     SysV4, 540 TB / 491 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

cramfs Marker at 1132079/8/21

recover_cramfs
     Linux filesys. data   2318498068 2324375474    5877406 [<\90WѲ\81/\C1\CA9,\AD\93]
     cramfs, 3009 MB / 2869 MiB
check_FAT: Bad number of sectors per cluster
check_FAT: Bad jump in FAT partition

SYSV4 Marker at 1803262/26/13

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               3693081420 4355583639  662502220 [\A9\9E1_!]
     SysV4, 339 GB / 315 GiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
     BeFS                  5259034145 26342049315855935 26342044056821791 [PY\9F\91ʕ\CF\D8\F2\CE\F4k\A2Z\AE~~\86x>\B3\91Pq>ja\C9o]
     BeFS blocksize=512, 13487126 TB / 12266470 TiB
This partition ends after the disk limits. (start=5259034145, size=26342044056821791, end=26342049315855935, disk end=19532873727)
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               5380837929 8780917509 3400079581
     FATX, 1740 GB / 1621 GiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

SYSV4 Marker at 2672891/61/30

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               5474082749 5474082748          0 [\8C\\FA\F4(]
     SysV4, 0 B
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               5706589527 8725729984619862 8725724278030336
     WBFS, 4467570 TB / 4063232 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

SYSV4 Marker at 2986480/53/20

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               6116312755 1008282061234 1002165748480 [\EC\FD\A5ޠ\FE]
     SysV4, 513 TB / 466 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

SYSV4 Marker at 3238507/19/27

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               6632462970 602834283485817 602827651022848 [\DA\C8]\BDc]
     SysV4, 308647 TB / 280713 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

LVM magic value at 3390459/10/19

SYSV4 Marker at 3529386/40/6

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               7228183813 1419521114854660 1419513886670848 [7\ED\D5٬]
     SysV4, 726791 TB / 661012 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
check_FAT: Bad number of sectors per cluster

       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               7678740405 774164033062836 774156354322432
     WBFS, 396368 TB / 360494 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
     BeFS                  8247900677 7365589529860612 7365581281959936 [}a\87\F6DA>\98\8E\E73(B\DFM6hu9\D3\CEb\94\BFS]\DDQ\A2]
     BeFS blocksize=2097152, 3771177 TB / 3429866 TiB
This partition ends after the disk limits. (start=8247900677, size=7365581281959936, end=7365589529860612, disk end=19532873727)

SYSV4 Marker at 4037023/29/27

recover_sysv4
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               8267824058 8267824057          0 [R\9D-d\A6]
     SysV4, 0 B
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.

cramfs Marker at 4084560/40/29

recover_cramfs
     Linux filesys. data   8365180188 8368196780    3016592 [\DA\E0\89\D7\E2\E7\9CO}\A4qz_,\92\BF]
     cramfs, 1544 MB / 1472 MiB

       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               8556029188 34646046467331 34637490438144
     WBFS, 17734 TB / 16129 TiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
     BeFS                  8844626297 33668553830713720 33668544986087424 [\AD\D1\D8s\B6C\A5=\F7Hn\91kں\9A\82\95{L\8C\97\DD\AB\D5]
     BeFS blocksize=64, 17238295 TB / 15678138 TiB
This partition ends after the disk limits. (start=8844626297, size=33668544986087424, end=33668553830713720, disk end=19532873727)

       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               9203623144 30547412695 21343789552
     WBFS, 10928 GB / 10177 GiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
       0 0000 0000 00 00 00 00 00 00 00 00
     Unknown               9830950726 13705628455 3874677730
     FATX, 1983 GB / 1847 GiB
       0 0000 0000 00 00 00 00 00 00 00 00
Partition not added.
     BeFS                  10634419755 28089665481544234 28089654847124480 [\9E\C7
IRe)\F9\D7\EF \E2:\90\C2E\F2ʊ-\BEX\92d@	&\DFB]
     BeFS blocksize=524288, 14381903 TB / 13080264 TiB
This partition ends after the disk limits. (start=10634419755, size=28089654847124480, end=28089665481544234, disk end=19532873727)
Disk /dev/sda - 10000 GB / 9313 GiB - CHS 9537535 64 32
Check the harddisk size: HD jumper settings, BIOS detection...
The harddisk (10000 GB / 9313 GiB) seems too small! (< 17238299 TB / 15678142 TiB)
The following partitions can't be recovered:
     BeFS                  5259034145 26342049315855935 26342044056821791 [PY\9F\91ʕ\CF\D8\F2\CE\F4k\A2Z\AE~~\86x>\B3\91Pq>ja\C9o]
     BeFS blocksize=512, 13487126 TB / 12266470 TiB
     BeFS                  8247900677 7365589529860612 7365581281959936 [}a\87\F6DA>\98\8E\E73(B\DFM6hu9\D3\CEb\94\BFS]\DDQ\A2]
     BeFS blocksize=2097152, 3771177 TB / 3429866 TiB
     BeFS                  8844626297 33668553830713720 33668544986087424 [\AD\D1\D8s\B6C\A5=\F7Hn\91kں\9A\82\95{L\8C\97\DD\AB\D5]
     BeFS blocksize=64, 17238295 TB / 15678138 TiB
     BeFS                  10634419755 28089665481544234 28089654847124480 [\9E\C7
IRe)\F9\D7\EF \E2:\90\C2E\F2ʊ-\BEX\92d@	&\DFB]
     BeFS blocksize=524288, 14381903 TB / 13080264 TiB

Results
   P EFI System                    40     409639     409600 [EFI System Partition] [EFI]
     FAT32, blocksize=512, 209 MB / 200 MiB
   P Linux filesys. data   1317569312 1322595384    5026073 [\E9\C7}`k\DFy|\D5[\F1\F5#;z]
     cramfs, 2573 MB / 2454 MiB
   P Linux filesys. data   2318498068 2324375474    5877407 [<\90WѲ\81/\C1\CA9,\AD\93]
     cramfs, 3009 MB / 2869 MiB
   P Linux filesys. data   8365180188 8368196780    3016593 [\DA\E0\89\D7\E2\E7\9CO}\A4qz_,\92\BF]
     cramfs, 1544 MB / 1472 MiB

Hint for advanced users: dmsetup may be used if you prefer to avoid rewriting the partition table for the moment:
echo "0 409600 linear /dev/sda 40" | dmsetup create test0
echo "0 5026073 linear /dev/sda 1317569312" | dmsetup create test1
echo "0 5877407 linear /dev/sda 2318498068" | dmsetup create test2
echo "0 3016593 linear /dev/sda 8365180188" | dmsetup create test3

interface_write()
 1 P EFI System                    40     409639     409600 [EFI System Partition] [EFI]
 2 P Linux filesys. data   1317569312 1322595384    5026073 [\E9\C7}`k\DFy|\D5[\F1\F5#;z]
 3 P Linux filesys. data   2318498068 2324375474    5877407 [<\90WѲ\81/\C1\CA9,\AD\93]
 4 P Linux filesys. data   8365180188 8368196780    3016593 [\DA\E0\89\D7\E2\E7\9CO}\A4qz_,\92\BF]
simulate write!

TestDisk exited normally.

Again, much thanks! Can I send you a gratuity or to this site works fine?

#17 Post by **recuperation** » 14 Sep 2022, 22:00

If your backup drive is shorter than your source drive you will miss out the backup GPT at the end of the disk.
As ready-to-use disk that already come in housing might contain converter electronics. You are supposed to use naked disks that you either put into a docking station or into a desktop computer instead.

To avoid difficulties your backup drive should mimic the source drive and be as big as the source or even bigger.

Typically most of the consumer drives have a physical sector size of 4096 bytes with a 512 byte emulation at the interface (logical size). This is what you should buy.
Some professional drives do not come with that translation, they use 4096 bytes as well and show 4096 bytes externally at the interface.
Converting electronics typically make a drive with 4096 byte internal / 512 byte external appear as 4096/4096.

For your problem the size at the interface is relevant. Regardless of the copy tool used once such a source is copied to a disk within a converting housing the target will appear as if all partitions are lost. In reality the converting electronics makes the operating system search based upon the emulated sector size of typically 4096 byte instead of 512 byte, searching at the wrong place of the disk. As I am not 100% sure if repairing the partition table is healing everything I can only recommend the above.

I won't comment on third party articles and focus on recovery instead.
As for donations, there is a donation button hidden in plain sight at

https://www.cgsecurity.org/

in favor of the creator of the Testdisk package and this forum, Christophe Grenier.

karenmcd · #18 Post by **karenmcd** » 15 Sep 2022, 06:53

Thank you very much again for your generosity with your time and the detailed reply.

I took this new hard drive out of the case and to my surprise it was an identical model to the internal drive that I had already purchased. The external one had the bonus of being cheaper too, it's not far away in serial numbers, and has the exact same space as the original drive that I deleted the partition on when it's not in the external USB case. Interestingly the data says it's transferring at 70MB/s via SATA6G -> SATA6G instead of the 230MB/s it said it was transferring before via SATA6G -> USB 3.0 cable, but whatever. I'll have a complete and full copy this time to recover and without the 4096/512 emulation. I'll report back when it's done copying via ddrescue and with the results of testdisk.

Finally on the correct path for success this time I think.

karenmcd · #19 Post by **karenmcd** » 17 Sep 2022, 11:47

So in the end, TestDisk did not end up working for me. I do appreciate ddrescue being pointed out to me though. I will probably use it in the future when/if I ever need to recover files, but will probably stick with dd for most of my imaging needs, as I'm much more comfortable with it for now.

After multiple scans both in intel and in efi/gpt mode I was unable to correctly make a partition table that I could mount or do anything with. I got huge amounts of errors when I used efi/gpt mode (as shown in the last log a couple posts above, they did not change even after I took out the disk from the enclosure) and I could only get the first ~2TB of data to even show up in intel mode. 2.2 TB (2,199,023,255,040 bytes) to be exact even when I manually set the partition to the end of the disk. 0 32 33 -> 1215864 254 63 (these numbers are by memory they could be off, but they were the default "end" options and matched what the end of my drive looked like from the original disk minus 1 number on the first two (H and C) options im pretty sure). I suspect that there's something wrong with the default selections when I was running the efi/gpt mode. I was probably supposed to set the CHS or something, but this hard drive does not have any documentation as to what those settings are.

For more information about the lack of information for the CHS settings you can see this model hard drive: STKP10000400 which actually contained a ST10000DM005 drive inside the housing. Extensive searching through Seagate's sites only yielded this datasheet: https://www.seagate.com/www-content/dat ... -en_US.pdf which is completely useless. Other data sheets I could find for 10TB drives were for the ST10000DM004 and ...DM006, a few other drives that looked darn close. I also found https://rml527.blogspot.com/2010/10/hdd ... te-35.html this blog which also confirms that there is currently no data sheet available to the public, so I'm unable to easily find out for myself the proper CHS of the drive partly because of this maybe and mostly because I don't know what I'm doing.

I did get kinda close with these sheets, but they stopped at the 8TB and are a generation older than my drive https://www.seagate.com/files/www-conte ... -en_SG.pdf - I only am linking it incase it's useful to someone else in the future.

So I went back to a few of the links that I had posted before and started playing around with what I had learned from third party sites.

Code: Select all

   ~  sudo hexdump -C /dev/sda | grep LUKS                                                                                              ✔ 
[sudo] password for karen: 
00100000  4c 55 4b 53 ba be 00 02  00 00 00 00 00 00 40 00  |LUKS..........@.|
^C
    ~  sudo LC_ALL=C grep -a -b -P 'LUKS\xba\xbe' /dev/sda                                                               INT|INT ✘  27s  
23725:U�RRaArrAaA'U����������LUKS��@sha256mW����=ߴ%RlOBx�k�wd���@�����
                                                                      �p��u"K����#�\K�H;�v^�e5ab7af58-ffaf-408d-b97f-34039eeec2e4�iS���'*�:��
                                                                                                                                             ���HP>��?f-{"keyslots":{"0":{"type":"luks2","key_size":64,"af":{"type":"luks1","stripes":4000,"hash":"sha256"},"area":{"type":"raw","offset":"32768","size":"258048","encryption":"aes-xts-plain64","key_size":64},"kdf":{"type":"argon2id","time":6,"memory":1048576,"cpus":4,"salt":"ll4EHehBIn/g4mIaC+1BYTownXlZREGMZv9Bezh3Ph0="}}},"tokens":{},"segments":{"0":{"type":"crypt","offset":"16777216","size":"dynamic","iv_tweak":"0","encryption":"aes-xts-plain64","sector_size":4096}},"digests":{"0":{"type":"pbkdf2","keyslots":["0"],"segments":["0"],"hash":"sha256","iterations":179550,"salt":"xShUJBPIvoeiG3LBKEhS8xFVCnvEYVUqBFkhnU2RTDg=","digest":"idsBN92QTEVT38EuX2BhWiYUWZUrl4XnCbs707J5cqE="}},"config":{"json_size":"12288","keyslots_size":"16744448"}}SKUL��@sha256V��� ϙ3�سV�6Q-]����*����oa���d���P�C�I�r���vH���0y��5ab7af58-ffaf-408d-b97f-34039eeec2e4@J���1��n=S*I�{A
                                                                                                                                        ��k�����\�&�{"keyslots":{"0":{"type":"luks2","key_size":64,"af":{"type":"luks1","stripes":4000,"hash":"sha256"},"area":{"type":"raw","offset":"32768","size":"258048","encryption":"aes-xts-plain64","key_size":64},"kdf":{"type":"argon2id","time":6,"memory":1048576,"cpus":4,"salt":"ll4EHehBIn/g4mIaC+1BYTownXlZREGMZv9Bezh3Ph0="}}},"tokens":{},"segments":{"0":{"type":"crypt","offset":"16777216","size":"dynamic","iv_tweak":"0","encryption":"aes-xts-plain64","sector_size":4096}},"digests":{"0":{"type":"pbkdf2","keyslots":["0"],"segments":["0"],"hash":"sha256","iterations":179550,"salt":"xShUJBPIvoeiG3LBKEhS8xFVCnvEYVUqBFkhnU2RTDg=","digest":"idsBN92QTEVT38EuX2BhWiYUWZUrl4XnCbs707J5cqE="}},"config":{"json_size":"12288","keyslots_size":"16744448"}}d{a�G�������يp����TPOul�.�o�*訞��u!@;xw��S��bq���ÝR������7�v�+�"f���7ӺV��<*o3_�E�W�Y�o�5�n��	�j�Co���
                                                                                                                        M'���ZU��"����
^C
    ~  sudo losetup -o 0x00008000 -r -f /dev/sda                                                                                         ✔ 
    ~  sudo losetup -a                                                                                                                   ✔ 
/dev/loop1: [66306]:11403879 (/var/lib/snapd/snaps/core_13741.snap)
/dev/loop2: [0005]:162 (/dev/sda), offset 32768
/dev/loop0: [66306]:11403414 (/var/lib/snapd/snaps/core_13425.snap)
    ~  sudo cryptsetup luksOpen /dev/loop2 luksrecover                                                                                   ✔ 
Device /dev/loop2 is not a valid LUKS device.

As you can see above, this didn't work.
I used a decimal -> hexdecimal converter I found online to convert what I assumed (because that's what the default option was on the converter I found) is a 10bit decimal number "offset":"32768" which was 0x00008000. As it turns out, that was not the offset at all. I read on other people's attempts that they often found the string "LUKS" many many times in their hexdump command output. It's worth noting that for most people's cases that I read about, the first one showing up doesn't appear to be the correct one. Most people do need to use the grep 'LUKS\xba\xbe' command and be specify the extra \xba\xbe bit which helps narrow down exactly the beginning. My disk only had 1 huge LUKS parition on it because it's my main storage device, and I boot and use most of the apps on my computers from 1 and 2 TB SSDs now.
I went back to the original hexdump I attempted (quoted from much earlier in this thread here)

Code: Select all

sudo hexdump -C /dev/sda | grep LUKS
00100000  4c 55 4b 53 ba be 00 02  00 00 00 00 00 00 40 00  |LUKS..........@.|
^C

and tried to enter that in the losetup (-o for offest) command instead. Here's what that ended up looking like for me. (losetup -d /dev/loop2 deleted the one that I setup at the wrong offest earlier so I could try again at a new offset. As you can see -a lists them all)

Code: Select all

    /  sudo losetup -o 0x00100000 -f /dev/sda                                                                                                                                                              ✔ 
    /  sudo losetup -a
 /dev/loop1: [66306]:11403879 (/var/lib/snapd/snaps/core_13741.snap)
/dev/loop2: [0005]:162 (/dev/sda), offset 1048576

I found it VERY interesting that my offset ended up being this 1048576 because that number is described as "memory":1048576," in the "sudo LC_ALL=C grep -a -b -P 'LUKS\xba\xbe' /dev/sda" command I typed just earlier! I would have never guessed that. I'm super glad someone else figured out the "sudo hexdump -C /dev/sda | grep LUKS" command that ended up working for me. Here's the rest of the process that I decided to use.

Code: Select all

/dev/loop0: [66306]:11403414 (/var/lib/snapd/snaps/core_13425.snap)
    /  sudo cryptsetup luksOpen /dev/loop2 luksrecover                                                                                                                                                     ✔ 
    /  sudo mount -r /dev/mapper/luksrecover /run/media/karen/storage                                                                                                                              ✔  12s  
mount: /run/media/karen/storage: mount point does not exist.
       dmesg(1) may have more information after failed mount system call.
    /  sudo mkdir /run/media/karen/storage                                                                                                                                                              32 ✘ 
    /  sudo mount -r /dev/mapper/luksrecover /run/media/karen/storage                                                                                                                                      ✔

*******************************************************************************
edit: If you're doing data recovery it is really important to setup the loop device as read only and then use the "norecovery" flag to mount the loop device for extracting the files! The mount option is otherwise PROBABLY trying to repair/modify a log file: https://serverfault.com/questions/83989 ... -read-only

Probably, your XFS filesystem has a dirty log that needs to be replayed to give you a consistent filesystem. However, the ro mount option prevents that. Try mounting adding norecovery, for example: mount -o ro,norecovery,loop,offset=1048576 -t xfs /media/mountdevicesource/ewf1 /media/mountdest"

*******************************************************************************
I did fail at first because the first time I did "losetup -r" as recommended by one of the third party sites I had followed and linked earlier in this thread, but would get errors that I couldn't mount a read-only device from Nautilus file manager for some reason.... (see edit above) anyway.... I'm not sure if from a forensics perspective if this was safe or the correct thing to do? I did end up deciding to mount it from the CLI instead of in Gnome Nautilus. Using "mount" and the -r flag just because I saw it was an option and thought maybe it's a better/safer because I didn't do that when I setup the loop device, and I remember reading here in the official documentation that when a drive mounts that it can cause further damage to a drive you're attempting recovery on. Would appreciate feedback on this if possible?

I gotta say when I got asked for my LUKS password for the first time here, my heart skipped a beat with excitement that finally something looked right!

Im typing this message now to the forum while I'm copying all the data to a newly partitioned and formatted LUKS drive (which was the original drive I accidentally deleted the partition on because I've been working from the ddrescue copy). I've got an UPS rated for my system and will be sure to get all the data and now will have an extra drive to backup the backup when I'm done. I am also going to learn how to backup my partition table and LUKS header now so that if anything like this should happen again, I'll have better options, and I'm going to automate doing nightly backups. So grateful to get my data back!

I'd be more than happy to have it pointed out to me how I could have recovered this with TestDisk if I did something wrong, or made an error someplace. Even though the software didn't work for my case I'm very grateful for this forum and the information that I got from the users, moderator and documentation that in the end did contribute to me getting my data back. Hopefully I documented this well enough for any other newbies who find themselves in this position, and maybe there will be a followup from someone telling newbies what not to do... or in other words how I messed up using TestDisk

I will be donating to this project regardless. Thanks again!

#20 Post by **recuperation** » 17 Sep 2022, 13:24

Congratulations! You managed to suceed manually.

karenmcd wrote: ↑17 Sep 2022, 11:47 So in the end, TestDisk did not end up working for me. I do appreciate ddrescue being pointed out to me though. I will probably use it in the future when/if I ever need to recover files, but will probably stick with dd for most of my imaging needs, as I'm much more comfortable with it for now.

After multiple scans both in intel and in efi/gpt mode
Could you still post a logfile of Testdisk when being in GPT mode? I would like to see if this makes any difference. Your old MBR-style log file showed the LUKS partition but with an obviously unusable location information. Did you already have your 10 TB drive removed from the enclosure?
I was unable to correctly make a partition table that I could mount or do anything with. I got huge amounts of errors when I used efi/gpt mode (as shown in the last log a couple posts above, they did not change even after I took out the disk from the enclosure) and I could only get the first ~2TB of data to even show up in intel mode. 2.2 TB (2,199,023,255,040 bytes) to be exact even when I manually set the partition to the end of the disk. 0 32 33 -> 1215864 254 63 (these numbers are by memory they could be off, but they were the default "end" options and matched what the end of my drive looked like from the original disk minus 1 number on the first two (H and C) options im pretty sure). I suspect that there's something wrong with the default selections when I was running the efi/gpt mode. I was probably supposed to set the CHS or something, but this hard drive does not have any documentation as to what those settings are.

For more information about the lack of information for the CHS settings

CHS only plays a role in legacy operating systems. You did not name the operating systems you used.

you can see this model hard drive: STKP10000400 which actually contained a ST10000DM005 drive inside the housing. Extensive searching through Seagate's sites only yielded this datasheet: https://www.seagate.com/www-content/dat ... -en_US.pdf which is completely useless. Other data sheets I could find for 10TB drives were for the ST10000DM004 and ...DM006, a few other drives that looked darn close. I also found https://rml527.blogspot.com/2010/10/hdd ... te-35.html this blog which also confirms that there is currently no data sheet available to the public, so I'm unable to easily find out for myself the proper CHS of the drive partly because of this maybe and mostly because I don't know what I'm doing. I did get kinda close with these sheets, but they stopped at the 8TB and are a generation older than my drive https://www.seagate.com/files/www-conte ... -en_SG.pdf - I only am linking it incase it's useful to someone else in the future.

So I went back to a few of the links that I had posted before and started playing around with what I had learned from third party sites.
Code: Select all
   ~  sudo hexdump -C /dev/sda | grep LUKS                                                                                              ✔ 
[sudo] password for karen: 
00100000  4c 55 4b 53 ba be 00 02  00 00 00 00 00 00 40 00  |LUKS..........@.|
^C
    ~  sudo LC_ALL=C grep -a -b -P 'LUKS\xba\xbe' /dev/sda                                                               INT|INT ✘  27s  
23725:U�RRaArrAaA'U����������LUKS��@sha256mW����=ߴ%RlOBx�k�wd���@�����
                                                                      �p��u"K����#�\K�H;�v^�e5ab7af58-ffaf-408d-b97f-34039eeec2e4�iS���'*�:��
                                                                                                                                             ���HP>��?f-{"keyslots":{"0":{"type":"luks2","key_size":64,"af":{"type":"luks1","stripes":4000,"hash":"sha256"},"area":{"type":"raw","offset":"32768","size":"258048","encryption":"aes-xts-plain64","key_size":64},"kdf":{"type":"argon2id","time":6,"memory":1048576,"cpus":4,"salt":"ll4EHehBIn/g4mIaC+1BYTownXlZREGMZv9Bezh3Ph0="}}},"tokens":{},"segments":{"0":{"type":"crypt","offset":"16777216","size":"dynamic","iv_tweak":"0","encryption":"aes-xts-plain64","sector_size":4096}},"digests":{"0":{"type":"pbkdf2","keyslots":["0"],"segments":["0"],"hash":"sha256","iterations":179550,"salt":"xShUJBPIvoeiG3LBKEhS8xFVCnvEYVUqBFkhnU2RTDg=","digest":"idsBN92QTEVT38EuX2BhWiYUWZUrl4XnCbs707J5cqE="}},"config":{"json_size":"12288","keyslots_size":"16744448"}}SKUL��@sha256V��� ϙ3�سV�6Q-]����*����oa���d���P�C�I�r���vH���0y��5ab7af58-ffaf-408d-b97f-34039eeec2e4@J���1��n=S*I�{A
                                                                                                                                        ��k�����\�&�{"keyslots":{"0":{"type":"luks2","key_size":64,"af":{"type":"luks1","stripes":4000,"hash":"sha256"},"area":{"type":"raw","offset":"32768","size":"258048","encryption":"aes-xts-plain64","key_size":64},"kdf":{"type":"argon2id","time":6,"memory":1048576,"cpus":4,"salt":"ll4EHehBIn/g4mIaC+1BYTownXlZREGMZv9Bezh3Ph0="}}},"tokens":{},"segments":{"0":{"type":"crypt","offset":"16777216","size":"dynamic","iv_tweak":"0","encryption":"aes-xts-plain64","sector_size":4096}},"digests":{"0":{"type":"pbkdf2","keyslots":["0"],"segments":["0"],"hash":"sha256","iterations":179550,"salt":"xShUJBPIvoeiG3LBKEhS8xFVCnvEYVUqBFkhnU2RTDg=","digest":"idsBN92QTEVT38EuX2BhWiYUWZUrl4XnCbs707J5cqE="}},"config":{"json_size":"12288","keyslots_size":"16744448"}}d{a�G�������يp����TPOul�.�o�*訞��u!@;xw��S��bq���ÝR������7�v�+�"f���7ӺV��<*o3_�E�W�Y�o�5�n��	�j�Co���
                                                                                                                        M'���ZU��"����
^C
    ~  sudo losetup -o 0x00008000 -r -f /dev/sda                                                                                         ✔ 
    ~  sudo losetup -a                                                                                                                   ✔ 
/dev/loop1: [66306]:11403879 (/var/lib/snapd/snaps/core_13741.snap)
/dev/loop2: [0005]:162 (/dev/sda), offset 32768
/dev/loop0: [66306]:11403414 (/var/lib/snapd/snaps/core_13425.snap)
    ~  sudo cryptsetup luksOpen /dev/loop2 luksrecover                                                                                   ✔ 
Device /dev/loop2 is not a valid LUKS device.
As you can see above, this didn't work.
I used a decimal -> hexdecimal converter I found online to convert what I assumed (because that's what the default option was on the converter I found) is a 10bit decimal number "offset":"32768" which was 0x00008000. As it turns out, that was not the offset at all. I read on other people's attempts that they often found the string "LUKS" many many times in their hexdump command output. It's worth noting that for most people's cases that I read about, the first one showing up doesn't appear to be the correct one. Most people do need to use the grep 'LUKS\xba\xbe' command and be specify the extra \xba\xbe bit which helps narrow down exactly the beginning. My disk only had 1 huge LUKS parition on it because it's my main storage device, and I boot and use most of the apps on my computers from 1 and 2 TB SSDs now.
I went back to the original hexdump I attempted (quoted from much earlier in this thread here)
Code: Select all
sudo hexdump -C /dev/sda | grep LUKS
00100000  4c 55 4b 53 ba be 00 02  00 00 00 00 00 00 40 00  |LUKS..........@.|
^C
and tried to enter that in the losetup (-o for offest) command instead. Here's what that ended up looking like for me. (losetup -d /dev/loop2 deleted the one that I setup at the wrong offest earlier so I could try again at a new offset. As you can see -a lists them all)
Code: Select all
    /  sudo losetup -o 0x00100000 -f /dev/sda                                                                                                                                                              ✔ 
    /  sudo losetup -a
 /dev/loop1: [66306]:11403879 (/var/lib/snapd/snaps/core_13741.snap)
/dev/loop2: [0005]:162 (/dev/sda), offset 1048576
I found it VERY interesting that my offset ended up being this 1048576 because that number is described as "memory":1048576," in the "sudo LC_ALL=C grep -a -b -P 'LUKS\xba\xbe' /dev/sda" command I typed just earlier! I would have never guessed that. I'm super glad someone else figured out the "sudo hexdump -C /dev/sda | grep LUKS" command that ended up working for me. Here's the rest of the process that I decided to use.
Code: Select all
/dev/loop0: [66306]:11403414 (/var/lib/snapd/snaps/core_13425.snap)
    /  sudo cryptsetup luksOpen /dev/loop2 luksrecover                                                                                                                                                     ✔ 
    /  sudo mount -r /dev/mapper/luksrecover /run/media/karen/storage                                                                                                                              ✔  12s  
mount: /run/media/karen/storage: mount point does not exist.
       dmesg(1) may have more information after failed mount system call.
    /  sudo mkdir /run/media/karen/storage                                                                                                                                                              32 ✘ 
    /  sudo mount -r /dev/mapper/luksrecover /run/media/karen/storage                                                                                                                                      ✔ 
I did fail at first because the first time I did "losetup -r" as recommended by one of the third party sites I had followed and linked earlier in this thread, but would get errors that I couldn't mount a read-only device from Nautilus file manager for some reason.... anyway.... I'm not sure if from a forensics perspective if this was safe or the correct thing to do? I did end up deciding to mount it from the CLI instead of in Gnome Nautilus. Using "mount" and the -r flag just because I saw it was an option and thought maybe it's a better/safer because I didn't do that when I setup the loop device, and I remember reading here in the official documentation that when a drive mounts that it can cause further damage to a drive you're attempting recovery on. Would appreciate feedback on this if possible?

I did not find it. Any disk activity can worsen the situation. Mounting involves at least various read operations.

I gotta say when I got asked for my LUKS password for the first time here, my heart skipped a beat with excitement that finally something looked right!

Im typing this message now to the forum while I'm copying all the data to a newly partitioned and formatted LUKS drive (which was the original drive I accidentally deleted the partition on because I've been working from the ddrescue copy). I've got an UPS rated for my system and will be sure to get all the data and now will have an extra drive to backup the backup when I'm done. I am also going to learn how to backup my partition table

You can do this using Testdisk.

and LUKS header

That should be a LUKS command. This is a very good idea because the most recent LUKS version spreads out the password hashes over lots of sectors.
Once an affected sector becomes unreadable you loose access to your LUKS partition.

now so that if anything like this should happen again, I'll have better options, and I'm going to automate doing nightly backups. So grateful to get my data back!

I'd be more than happy to have it pointed out to me how I could have recovered this with TestDisk

Please provide the logfile requested above with the drive being in the enclosure it came with. I am interested in finding out why the log line referring to LUKS was incomplete.

if I did something wrong, or made an error someplace.

I would like to find that out.

cgsecurity.org

deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)

Re: deleted wrong partition (LUKS)