photorec on large blockdev >10TB slowing down after a couple min, stuck at 100% CPU, not finding existing signatures

[TheRealM]

Hello dear Christophe, dear community,

actually for years, every now and then, I had to use photorec and testdisk to recover lost partitions and files, mostly on small storage. For the first time now, I am dealing with something >4TB, a 10,7TB reassembled mdadm RAID 5 that was partially overwritten to the extent that an ext4 recovery is not feasible nor possible.

- the target storage for the recovered data is a relatively fast SSD
- the source storage is technically fine, the disks are in a good shape and show no defects neither in SMART nor on reading the media
- the source ofc is in read-only mode
- the source is being read in 512 byte sectors to restore as many even small fragments as possible

The system processing is an

- old AMD FX-6300
- 16GB memory
- Ubuntu 25.04 with the latest testdisk-package installed from the canonical repo

Settings used are:

- expert mode yes
- keep broken files yes
- bruteforce no

What I had noticed is, that a couple minutes in, photorec started to heavily slow down and consume 100% of one CPU core. It started to refresh the statistics much slower and processed very slow, sometimes not reading any data off the disks at all.

When then I paused it "STOP" and resumed "N" it started right away finding new files and iterating over the sectors much quicker. Until a couple seconds later it would slow down again. There is enough IO bandwidth left and there is plenty of unused memory.

I activated the log "/log" and saw that even writing the log lines stopped in the middle of a line only to progress once I paused for a brief moment and then resumed. I ran strace on the photorec process to find that the reads its processing are tickeling in very slowly once in that state. You could literally read line by line at that pace.

Since I used your software already a couple of times, I wanted to try and see how it works when I shrank the blockdev size down to something more usual, lets say about 5TB. So I inserted a loopdevice with losetup -o <half the sectors*512> /dev/loop20 /dev/md1 and ran photorec on that. No problems, running much faster and finding all files well. Even after half an hour no noticable slowdown. For my use-case this is okay enough because I could just split the blockdevice, since it likely holds no large files that pass over the lets say 4TB-boundaries. At least its highly unlikely and I could re-check across the boundary later on.

Is this expected behaviour? I am willing to fully debug and cooperate on this, to help you with your great software, which I really like. Let me know how to proceed or what I can still do to narrow it down further.

[recuperation]

Please post your TestDisk log file and a smartmontools log file for each of your RAID member disks:

viewtopic.php?f=5&t=10910