deleted wrong partition (LUKS)

How to use TestDisk to recover lost partition
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Message
Author
User avatar
karenmcd
Posts: 19
Joined: 03 Aug 2022, 01:15

Re: deleted wrong partition (LUKS)

#21 Post by karenmcd »

CHS only plays a role in legacy operating systems. You did not name the operating systems you used.
You're right? But it was in the log file? Edit: I normally include an inxi -F with support requests. I don't know why I forgot this time. My bad. I did say that I was trying to recover from the Manjaro machine in the first post, but didn't make it clear that was the same machine that I had accidentally deleted the partition on.

I am using Manjaro UEFI/GTP x3 (1 desktop, 1 workstation and 1 laptop) another system is Debian 11/Windows 10 efi/gtp dual boot, and my laptop is QubesOS/Debian 11 dual boot on 2 separate SSDs also efi/gtp. I was attempting to repair a separate Windows 11/Dell machine for a family member. Was down to either the motherboard being faulty or the SSD, so I installed the SSD into my Manjaro desktop, and attempted to install Windows on it. I had tried to use the DELL recovery tool, then install Manarjo, Debian and Ubuntu onto the Dell machine all without success earlier. All of the gnu-linux based installations had selected the NVME drive first to partition and install an OS on by default. They also didn't want to make changes to the partitions by default and all had confirmation screens for actually writing the partition table explaining with BOLD CAPS that I was making destructive changes to the disk. Windows does not have this "feature". I quickly skipped through setup skimming all the EULA BOLD CAPS Vogon Poetry lines, and it chose by default to install on my SLOW 'storage' SATA drive which has many system backups on it, raw video footage for a project I'm scrubbing/editing and colour grading on instead of the very fast PCIE 4.0 4x NVME 1TB drive. It did not ask for confirmation after choosing to delete the partition. Windows also as a "feature" wrote the changes without confirmation as well and showed me a blank 10TB SATA disk next to a badly corrupted (but working as it turns out) 1TB NVME. That's when I facepalmed and started this journey. (*as for the DELL story, we got a replacement motherboard through warranty and the NVME (and rest of the components for that matter) seems to have survived the mobo death.

I did the repairs mostly on the Manjaro Desktop system that I deleted the 10TB partition on. I also moved the SATA drive and did some of the recovery on the Manjaro workstation, so I could work on some of the film with some missing footage until I got the newer data back like I did last night finally. I had a slightly older and smaller backup that was about 2 months behind (I referred to it earlier in this thread being too small for the recovery attempt), but was missing some irreplaceable footage shot this summer, which only emphasizes the importance of REGULAR backups.
Please provide the logfile requested above with the drive being in the enclosure it came with. I am interested in finding out why the log line referring to LUKS was incomplete.
I will re-scan the copy I have now working after the copy is complete to give you a log file with the disk outside of the enclosure.

The most recent logfile contained in this thread is already one from inside the enclosure using efi/gtp mode viewtopic.php?p=36505#p36505

The first one was in Intel mode not in this thread and deleted. I didn't bother with it anymore after you asked me to re-scan using the beta branch instead of the stable one. I can rescan it with the breadboard (the plastic case is broken) USB 3.0 interface in efi/gtp mode in the beta version after I have my data back if you like. It will probably take 2 days or so given the transfer rates and then scanning again.

recuperation
Posts: 2720
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: deleted wrong partition (LUKS)

#22 Post by recuperation »

As you are not aware of the intent of my question here is the background.

Harddrives have been produced for a long time with a sector size of 512 bytes.
Then came the so-called invention of advanced format disks.

https://www.seagate.com/tech-insights/a ... master-ti/
https://documents.westerndigital.com/co ... format.pdf

Just a quick annotation, they increased the number of ECC bytes from 50 per sector with 512 bytes to 100 for a sector with 4096 bytes in size.
What Seagate and Western Digital try to sell us is the increase of efficiency. They do not mention that the new sector size was eightfold whereas the the number of ECC bytes only doubled. As everybody can see, the share of ECC bytes to sector size was only 25% as before, brave new world, what an innovation!


The first thing what happened is that those new drives failed to boot because at the time the change took place the sector size was hardcoded as 512 in every operating system.

To ensure compatibility, the manufacturers emulated a sector size of 512 bytes while maintaining the physical sector size of 4096 bytes. At that time the term "sector size" was not sufficient anymore to describe reality, "sector size" splitted up in "physical" (internal) and "logical" at the SATA-interface.
While compatibility was restored perfomance degraded when partitions were located at a sector number that was not a clean multiple of eight (4096 bytes/512bytes).

By repairing one problem a new problem was created when dealing with big drives. The MBR style partition table was not able to address more than 2(,2) Gigabytes.
The problem with a sector size of 512 bytes is due to the restriction of a 32-bit-wide sector number: 2^32*512 gives you the 2 Gigabyte figure.

Now the hard drive manufacturers reversed the 4096 byte emulation by putting some converting electronics in their enclosure, so that the operating system was confronted with a sector size of 4096 bytes (or other figures, I have no personal experience with that):

4096 internal -> 512 bytes external at the SATA-interface - 4096 byte at the USB plug.

When you remove a disk that had been set in such an enclosure your operating system does not see a sector size of 4096 bytes but a size of 512 byte. As the operating system is using the sector size of 512 byte it will look at the wrong location on the disk when trying to access the partitions. Furthermore I can't say for sure if that creates additional problems.

The converter electronics - if it's in there - allows you to address the whole 10 TB with a MBR-style partition table because of the sector size of 4096 bytes.
But you would also be able to address 10 TB with a sector size of only 512 bytes.

Removing the disk from the enclosure creates problems because you might remove a translation layer that would be missing when installing the drive directly in a desktop computer.

I guess that was the problem because your logfile contained a LUKS line without a proper start address:
viewtopic.php?p=36505#p36505

When running "hdparm -i /dev/sdx" (replace the "x" as needed) it shows you the sector size. If it yields different results with and without enclosure you know that some converting process is going on in the interface.

If you don't remember how you partitioned your drive and the partition table is broken you have that additional problem.

Typically people are either lazy or are lacking the knowledge of what to report here. That's why I created the following questionnaire where some questions target specifically at indicators for their MBR/GPT setup.

Code: Select all

0. Please provide your Testdisk logfile.

1.Which operating systems can be booted from your computer where the incident happened?
List them all!

2. Which version of Testdisk do you use?

3.Do you prevent/reduce write access to the failed drive/file system?
[Yes/no]

4. If yes, how is that done?

[ ] I removed the failed drive and connected it to another computer (not linux) as an external drive => risky
[ ] I am using a live linux from a USB stick on the machine with the broken drive => good
[ ] I am booting a linux system on a different system and connect the drive externally once the linux finished booting => good

5. Is the broken drive a drive where an operating system resides on or is it a data drive?

6. What technology is your disk (HDD, SDD, USB stick, Compact Flash card, SD card,...)?

7. What is the size of your disk?

8. Who is the maker of your failed drive?

9. What is the model?

10. Is the drive something you bought "naked" one or does it come with a housing and a connector for a computer (p.e. like "WD My Passort")?

11. If possible, provide a logfile from smartmontools!
Instructions:
https://forum.cgsecurity.org/phpBB3/viewtopic.php?f=5&t=10910

12. What has been the partitioning scheme used on the failed drive (MBR (old partition table style), GPT, Superfloppy)

13. How many partitions have been on the broken drive, what was their size, what was their file system?

14. Is your drive visible in your operating system (Windows: Disk management, Linux use lsblk command, get information using hdparm command)

15. Is the partition scheme containing your partitions still visible?

14. Describe the supposed event when your system went from "OK" to "broken"!

15. Is your disk showing signs of failures such as 
	-clicking noises
	-permanent reboot (spindel speed up followed by a stop)
	-no spindel speed up

?

16. Do you use encryption, if yes, which one?

17. If you use encryption, what is the scope?

[ ] full drive
[ ] partition
[ ] file container
[ ] single files

User avatar
karenmcd
Posts: 19
Joined: 03 Aug 2022, 01:15

Re: deleted wrong partition (LUKS)

#23 Post by karenmcd »

When running "hdparm -i /dev/sdx" (replace the "x" as needed) it shows you the sector size. If it yields different results with and without enclosure you know that some converting process is going on in the interface.
I noticed in the man pages that the -i flag only provides information from the kernel module during boot. Running the command on the disk with the USB interface resulted in an error only - I forgot to record it but it was something to the effect of "there's no device". I had a quick glance at the man pages and found out that -I (upper case i) gives more information directly from the drive instead of the kerneal, so I tried that and it spilled some information. After rebooting there was no difference if I booted the computer with the USB interface plugged in or not. The command showed the same results with both flags with the USB interface.

With the USB interface, plugged in after boot.

Code: Select all

    /  sudo hdparm -I /dev/sdb                                                                                                                                                                  ✔  1m 10s  

/dev/sdb:

ATA device, with non-removable media
Standards:
	Likely used: 1
Configuration:
	Logical		max	current
	cylinders	0	0
	heads		0	0
	sectors/track	0	0
	--
	Logical/Physical Sector size:           512 bytes
	device size with M = 1024*1024:           0 MBytes
	device size with M = 1000*1000:           0 MBytes 
	cache/buffer size  = unknown
Capabilities:
	IORDY not likely
	Cannot perform double-word IO
	R/W multiple sector transfer: not supported
	DMA: not supported
	PIO: pio0 
Here it is plugged in before boot. Looks basically the same to me.

Code: Select all

    ~  sudo hdparm -I /dev/sdb                                                                                                                                                                             ✔ 
[sudo] password for karen: 

/dev/sdb:

ATA device, with non-removable media
Standards:
	Likely used: 1
Configuration:
	Logical		max	current
	cylinders	0	0
	heads		0	0
	sectors/track	0	0
	--
	Logical/Physical Sector size:           512 bytes
	device size with M = 1024*1024:           0 MBytes
	device size with M = 1000*1000:           0 MBytes 
	cache/buffer size  = unknown
Capabilities:
	IORDY not likely
	Cannot perform double-word IO
	R/W multiple sector transfer: not supported
	DMA: not supported
	PIO: pio0
Here is what it looks like without the USB interface (SATA III port), I am unsure if these ports support hot swapping, and I'm not about to try it out! :D (the command was run after a cold boot) There's considerably more information.

Code: Select all

    ~  sudo hdparm -I /dev/sdb                                                                                                                                 ✔ 
[sudo] password for karen: 

/dev/sdb:

ATA device, with non-removable media
	Model Number:       ST10000DM005-3AW101                     
	Serial Number:      WP005865
	Firmware Revision:  DN04    
	Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
	Used: unknown (minor revision code 0xffff) 
	Supported: 11 10 9 8 7 6 5 
	Likely used: 11
Configuration:
	Logical		max	current
	cylinders	16383	16383
	heads		16	16
	sectors/track	63	63
	--
	CHS current addressable sectors:    16514064
	LBA    user addressable sectors:   268435455
	LBA48  user addressable sectors: 19532873728
	Logical  Sector size:                   512 bytes [ Supported: 512 4096 ]
	Physical Sector size:                  4096 bytes
	Logical Sector-0 offset:                  0 bytes
	device size with M = 1024*1024:     9537536 MBytes
	device size with M = 1000*1000:    10000831 MBytes (10000 GB)
	cache/buffer size  = unknown
	Form Factor: 3.5 inch
	Nominal Media Rotation Rate: 7200
Capabilities:
	LBA, IORDY(can be disabled)
	Queue depth: 32
	Standby timer values: spec'd by Standard, no device specific minimum
	R/W multiple sector transfer: Max = 16	Current = 16
	Recommended acoustic management value: 254, current value: 0
	DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
	     Cycle time: min=120ns recommended=120ns
	PIO: pio0 pio1 pio2 pio3 pio4 
	     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
	Enabled	Supported:
	   *	SMART feature set
	    	Security Mode feature set
	   *	Power Management feature set
	   *	Write cache
	   *	Look-ahead
	   *	WRITE_BUFFER command
	   *	READ_BUFFER command
	   *	DOWNLOAD_MICROCODE
	    	Power-Up In Standby feature set
	   *	SET_FEATURES required to spinup after power up
	    	SET_MAX security extension
	   *	48-bit Address feature set
	   *	Mandatory FLUSH_CACHE
	   *	FLUSH_CACHE_EXT
	   *	SMART error logging
	   *	SMART self-test
	   *	Media Card Pass-Through
	   *	General Purpose Logging feature set
	   *	WRITE_{DMA|MULTIPLE}_FUA_EXT
	   *	64-bit World wide name
	   *	IDLE_IMMEDIATE with UNLOAD
	    	Write-Read-Verify feature set
	   *	WRITE_UNCORRECTABLE_EXT command
	   *	{READ,WRITE}_DMA_EXT_GPL commands
	   *	Segmented DOWNLOAD_MICROCODE
	   *	unknown 119[6]
	   *	unknown 119[7]
	    	unknown 119[8]
	    	unknown 119[9]
	   *	Gen1 signaling speed (1.5Gb/s)
	   *	Gen2 signaling speed (3.0Gb/s)
	   *	Gen3 signaling speed (6.0Gb/s)
	   *	Native Command Queueing (NCQ)
	   *	Phy event counters
	   *	Idle-Unload when NCQ is active
	   *	READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
	   *	DMA Setup Auto-Activate optimization
	   *	Device-initiated interface power management
	   *	Software settings preservation
	    	unknown 78[7]
	   *	SMART Command Transport (SCT) feature set
	   *	SCT Write Same (AC2)
	   *	SCT Error Recovery Control (AC3)
	   *	SCT Features Control (AC4)
	   *	SCT Data Tables (AC5)
	    	unknown 206[7]
	    	unknown 206[12] (vendor specific)
	    	unknown 206[14] (vendor specific)
	   *	SANITIZE_ANTIFREEZE_LOCK_EXT command
	   *	SANITIZE feature set
	   *	OVERWRITE_EXT command
	   *	Extended number of user addressable sectors 
Security: 
	Master password revision code = 65534
		supported
	not	enabled
	not	locked
		frozen
	not	expired: security count
		supported: enhanced erase
	850min for SECURITY ERASE UNIT. 850min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c500dc97e018
	NAA		: 5
	IEEE OUI	: 000c50
	Unique ID	: 0dc97e018
Checksum: correct
    ~                                                                                                                                                                                                       ✔     ~  sudo hdparm -I /dev/sda                                                                                                                                                                      ✔  6s  

/dev/sda:

ATA device, with non-removable media
	Model Number:       ST10000DM005-3AW101                     
	Serial Number:      WP007FRD
	Firmware Revision:  DN04    
	Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
	Used: unknown (minor revision code 0xffff) 
	Supported: 11 10 9 8 7 6 5 
	Likely used: 11
Configuration:
	Logical		max	current
	cylinders	16383	16383
	heads		16	16
	sectors/track	63	63
	--
	CHS current addressable sectors:    16514064
	LBA    user addressable sectors:   268435455
	LBA48  user addressable sectors: 19532873728
	Logical  Sector size:                   512 bytes [ Supported: 512 4096 ]
	Physical Sector size:                  4096 bytes
	Logical Sector-0 offset:                  0 bytes
	device size with M = 1024*1024:     9537536 MBytes
	device size with M = 1000*1000:    10000831 MBytes (10000 GB)
	cache/buffer size  = unknown
	Form Factor: 3.5 inch
	Nominal Media Rotation Rate: 7200
Capabilities:
	LBA, IORDY(can be disabled)
	Queue depth: 32
	Standby timer values: spec'd by Standard, no device specific minimum
	R/W multiple sector transfer: Max = 16	Current = 16
	Recommended acoustic management value: 254, current value: 0
	DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
	     Cycle time: min=120ns recommended=120ns
	PIO: pio0 pio1 pio2 pio3 pio4 
	     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
	Enabled	Supported:
	   *	SMART feature set
	    	Security Mode feature set
	   *	Power Management feature set
	   *	Write cache
	   *	Look-ahead
	   *	WRITE_BUFFER command
	   *	READ_BUFFER command
	   *	DOWNLOAD_MICROCODE
	    	Power-Up In Standby feature set
	   *	SET_FEATURES required to spinup after power up
	    	SET_MAX security extension
	   *	48-bit Address feature set
	   *	Mandatory FLUSH_CACHE
	   *	FLUSH_CACHE_EXT
	   *	SMART error logging
	   *	SMART self-test
	   *	Media Card Pass-Through
	   *	General Purpose Logging feature set
	   *	WRITE_{DMA|MULTIPLE}_FUA_EXT
	   *	64-bit World wide name
	   *	IDLE_IMMEDIATE with UNLOAD
	    	Write-Read-Verify feature set
	   *	WRITE_UNCORRECTABLE_EXT command
	   *	{READ,WRITE}_DMA_EXT_GPL commands
	   *	Segmented DOWNLOAD_MICROCODE
	   *	unknown 119[6]
	   *	unknown 119[7]
	    	unknown 119[8]
	    	unknown 119[9]
	   *	Gen1 signaling speed (1.5Gb/s)
	   *	Gen2 signaling speed (3.0Gb/s)
	   *	Gen3 signaling speed (6.0Gb/s)
	   *	Native Command Queueing (NCQ)
	   *	Phy event counters
	   *	Idle-Unload when NCQ is active
	   *	READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
	   *	DMA Setup Auto-Activate optimization
	   *	Device-initiated interface power management
	   *	Software settings preservation
	    	unknown 78[7]
	   *	SMART Command Transport (SCT) feature set
	   *	SCT Write Same (AC2)
	   *	SCT Error Recovery Control (AC3)
	   *	SCT Features Control (AC4)
	   *	SCT Data Tables (AC5)
	    	unknown 206[7]
	    	unknown 206[12] (vendor specific)
	    	unknown 206[14] (vendor specific)
	   *	SANITIZE_ANTIFREEZE_LOCK_EXT command
	   *	SANITIZE feature set
	   *	OVERWRITE_EXT command
	   *	Extended number of user addressable sectors 
Security: 
	Master password revision code = 65534
		supported
	not	enabled
	not	locked
		frozen
	not	expired: security count
		supported: enhanced erase
	852min for SECURITY ERASE UNIT. 852min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c500e314f853
	NAA		: 5
	IEEE OUI	: 000c50
	Unique ID	: 0e314f853
Checksum: correct

User avatar
karenmcd
Posts: 19
Joined: 03 Aug 2022, 01:15

Re: deleted wrong partition (LUKS)

#24 Post by karenmcd »

Just a quick annotation, they increased the number of ECC bytes from 50 per sector with 512 bytes to 100 for a sector with 4096 bytes in size.
What Seagate and Western Digital try to sell us is the increase of efficiency. They do not mention that the new sector size was eightfold whereas the the number of ECC bytes only doubled. As everybody can see, the share of ECC bytes to sector size was only 25% as before, brave new world, what an innovation!
Brave new world indeed! while we get only 25% increase in ECC bytes and 8fold increase in sector size, it also seems we also get a whopping 10% increase in space efficiency! Totally worth it right? :roll: NEW! IMPROVED! FEATURES!

It appears it'll be days before I can get those TestDisk logs from these drives for you (USB vs non-USB). Apparently there's a well documented issue with sata -> sata data transfers on the X570 chipset that is slowing these transfers down to the speeds I was complaining about earlier, and everyone at AMD/ASUS/ASROCK etc... is just trying to pretend it's not a problem, hah!

recuperation
Posts: 2720
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: deleted wrong partition (LUKS)

#25 Post by recuperation »

No hurry. 8-)

User avatar
karenmcd
Posts: 19
Joined: 03 Aug 2022, 01:15

Re: deleted wrong partition (LUKS)

#26 Post by karenmcd »

editing...

Locked