Why does PhotoRec write so much data, but recovers so little?

Using PhotoRec to recover lost data
Forum rules
When asking for technical support:
- Search for posts on the same topic before posting a new question.
- Give clear, specific information in the title of your post.
- Include as many details as you can, MOST POSTS WILL GET ONLY ONE OR TWO ANSWERS.
- Post a follow up with a "Thank you" or "This worked!"
- When you learn something, use that knowledge to HELP ANOTHER USER LATER.
Before posting, please read https://www.cgsecurity.org/testdisk.pdf
Locked
Message
Author
ciupicri
Posts: 4
Joined: 25 Sep 2022, 13:07

Why does PhotoRec write so much data, but recovers so little?

#1 Post by ciupicri »

I'm using PhotoRec to recover the files from three partitions having 120, 300 and 500 GB, so a total of 920 GB.
The directory where there files are stored has only 60 GB, but imagine my reaction when I discovered that PhotoRec wrote 20000 GB on the SSD.

To sum it up:
  • original: 920 GB
  • recovered: 60 GB
  • written: 20000 GB (20 TB)

If PhotoRec needs to write some temporary files, it should do it under /tmp or under another temporary directory specified by the user, so it doesn't wear SSDs unnecessarily.

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Why does PhotoRec write so much data, but recovers so little?

#2 Post by recuperation »

Which version of Photorec do you use?
What are the settings?

ciupicri
Posts: 4
Joined: 25 Sep 2022, 13:07

Re: Why does PhotoRec write so much data, but recovers so little?

#3 Post by ciupicri »

I'm using testdisk-7.1-8.fc36.x86_64

If I remember correctly last time I selected only Zip, Office, PDF and MS Access Database because I'm interested only in these kind of files and also to be faster.

P.S. 46880 GB written so far.

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Why does PhotoRec write so much data, but recovers so little?

#4 Post by recuperation »

You should use the 7.2-WIP version instead.
What about the other settings?

Selecting only a few file/family types can lead to huge files because Photorec will only terminate one file if it finds another file header.

What is the product name of your SSD?

ciupicri
Posts: 4
Joined: 25 Sep 2022, 13:07

Re: Why does PhotoRec write so much data, but recovers so little?

#5 Post by ciupicri »

It's a Crucial CT2000MX500SSD1 SSD. smartctl says that Total_LBAs_Written is 100163814094 (from which about 1300 GB have been used to install Fedora and clone the disks to recover).

If you're thinking of hidden write amplification, the /sys/block/sda/stat file confirms that 40214 GB have been written after booting. Also iotop showed photorec writing with 100 MB/s from time to time.

In the mean photorec has finished and recovered only 109 files (using 5102 MB of disk space).

Are you talking about these settings? I've used the default.

Code: Select all

Paranoid : Yes (Brute force disabled)
Keep corrupted files : No
Expert mode : No
Low memory: No
The RPM from the grenier/testdisk copr is good?

recuperation
Posts: 2718
Joined: 04 Jan 2019, 09:48
Location: Hannover, Deutschland (Germany, Allemagne)

Re: Why does PhotoRec write so much data, but recovers so little?

#6 Post by recuperation »

ciupicri wrote: 27 Sep 2022, 16:45 It's a Crucial CT2000MX500SSD1 SSD. smartctl says that Total_LBAs_Written is 100163814094 (from which about 1300 GB have been used to install Fedora and clone the disks to recover).

If you're thinking of hidden write amplification, the /sys/block/sda/stat file confirms that 40214 GB have been written after booting. Also iotop showed photorec writing with 100 MB/s from time to time.

In the mean photorec has finished and recovered only 109 files (using 5102 MB of disk space).
I don't understand what "only" should mean.
What drive are you recovering from?

Are you talking about these settings? I've used the default.

Code: Select all

Paranoid : Yes (Brute force disabled)
Keep corrupted files : No
Expert mode : No
Low memory: No
How much memory does your machine have? What else was running on that machine during the time Photorec was running?

The RPM from the grenier/testdisk copr is good?
From where, please?

Assigning any difference in Total_LBAs_Written to Testdisk/Photorec activity is not a good idea. Neither is /sys/block/sda/stat. As I am mostly a windows user I can't tell you which linux tool shows you the aggregated write activity for the Photorec executable, unfortunately.

The best thing to do is to run the latest version. When I checked a couple of different linux distributions in terms of their data recovery software content, those few I looked at only had 7.1 either installed or in their repository. I compiled 7.2-WIP as explained in the Testdisk manual on Linuxmint 21 where I had to install the compiler tools first.

ciupicri
Posts: 4
Joined: 25 Sep 2022, 13:07

Re: Why does PhotoRec write so much data, but recovers so little?

#7 Post by ciupicri »

only was to emphasize the huge signal / noise ratio: 5102 MB recovered versus ~40214 GB written to the disk.

grenier/testdisk Copr. If I remember correctly it's mentioned somewhere here.

User avatar
cgrenier
Site Admin
Posts: 5432
Joined: 18 Feb 2012, 15:08
Location: Le Perreux Sur Marne, France
Contact:

Re: Why does PhotoRec write so much data, but recovers so little?

#8 Post by cgrenier »

ciupicri wrote: 03 Oct 2022, 12:05 only was to emphasize the huge signal / noise ratio: 5102 MB recovered versus ~40214 GB written to the disk.

grenier/testdisk Copr. If I remember correctly it's mentioned somewhere here.
For most file format, PhotoRec stops to recover a file when a new file is found. It tests the recovered file and
- drop it if it's invalid,
- keep the recovered files and if the correct filesize can be found, truncate it.
Selecting only the zip file family (same problem when recovering from a disk where only zip archives are written) is one of the worst situation as a zip file begins by ZIP_FILE_ENTRY (0x04034B50), this signature can also be
found for each compressed file, so PhotoRec will not stop when a new zip file is found. In this case, PhotoRec will recover a very huge zip file before testing it and truncate to its correct size.

grenier/testdisk copr repository is mentioned in https://www.cgsecurity.org/testdisk.pdf
I am using copr as a CI (continuous integration). The build targets are CentOS and Fedora, the rpm spec file build the git version of the project. (Note, the build doesn't work anymore for CentOS 6 and CentOS 8 stream.)

Locked