Would like assistance in adding a custom signature and using TestDisk or PhotoRec to retrieve the file with the signatur

[leni1]

Hello

I have a disk image I made using `ddrescue` that is on an external hard drive.
I would like assistance in adding a custom signature to PhotoRec and then use PhotoRec to retrieve the file.

The file type in question is JEX (.jex extension) which stands for "Joplin Export File" https://joplinapp.org/help/apps/import_export#exporting

Here is some sample output from hexdump:

Code: Select all

$ hexdump -C test_note.jex | head
00000000  34 63 39 31 34 63 35 37  31 66 35 30 34 63 64 30  |4c914c571f504cd0|
00000010  62 63 38 64 64 38 39 34  36 39 66 35 63 35 62 35  |bc8dd89469f5c5b5|
00000020  2e 6d 64 00 00 00 00 00  00 00 00 00 00 00 00 00  |.md.............|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000060  00 00 00 00 30 30 30 36  34 34 20 00 00 00 00 00  |....000644 .....|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 30 30 30 30  |............0000|
00000080  30 30 30 35 35 30 20 00  31 34 36 33 36 35 33 31  |000550 .14636531|
00000090  37 32 31 00 30 31 32 36  33 33 20 00 30 00 00 00  |721.012633 .0...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Link to file used: https://pixeldrain.com/u/ewdTb8G7

Code: Select all

$ hexdump -C test_note_2.jex | head
00000000  33 33 33 61 38 61 36 39  33 66 34 31 34 63 66 66  |333a8a693f414cff|
00000010  61 66 34 62 39 37 37 35  39 66 33 36 61 35 31 38  |af4b97759f36a518|
00000020  2e 6d 64 00 00 00 00 00  00 00 00 00 00 00 00 00  |.md.............|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000060  00 00 00 00 30 30 30 36  34 34 20 00 00 00 00 00  |....000644 .....|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 30 30 30 30  |............0000|
00000080  30 30 31 33 32 37 20 00  31 34 36 33 36 35 33 32  |001327 .14636532|
00000090  31 35 34 00 30 31 32 35  36 34 20 00 30 00 00 00  |154.012564 .0...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Link to file used https://pixeldrain.com/u/ewdTb8G7

Code: Select all

$ hexdump -C QubesIncoming/personal/25_06_2024.jex | head
00000000  31 31 65 64 31 39 64 33  64 61 65 35 34 34 33 62  |11ed19d3dae5443b|
00000010  61 34 62 38 31 39 33 39  39 35 66 62 64 34 33 35  |a4b8193995fbd435|
00000020  2e 6d 64 00 00 00 00 00  00 00 00 00 00 00 00 00  |.md.............|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000060  00 00 00 00 30 30 30 36  34 34 20 00 00 00 00 00  |....000644 .....|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 30 30 30 30  |............0000|
00000080  30 30 36 31 34 34 20 00  31 34 36 33 36 35 31 34  |006144 .14636514|
00000090  31 30 30 00 30 31 32 36  31 35 20 00 30 00 00 00  |100.012615 .0...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Link to file used https://pixeldrain.com/u/ewdTb8G7

fidentify returns unknown on these files.
I am not sure how to turn this into a signature since the example here https://www.cgsecurity.org/wiki/Add_you ... o_PhotoRec doesn't match my use case.
Let me know if there's any other information I should provide.

Edit: corrected the link to JEX file description

[recuperation]

Thank you for your good case description!

Here is the recipe:

Creating a custom signature is about finding the common part that is contained in all files of the desired file type (here: "jex").
You delivered three files which is a good basis to start guessing the signature.

Can you see something that is common for all three files? I do!
Where is that something, at which position from the beginning of the file do you find it?
What is your proposal?
We need three parts of information for that custom signature, as described in your link:

Code: Select all

   extension name
    offset of the signature
    signature or magic value

What are these?
How would you write this into a text file according to the rules outlined in your link?

[leni1]

Can you see something that is common for all three files? I do!
Where is that something, at which position from the beginning of the file do you find it?

The second line seems be held in common for all three files

Code: Select all

00000020  2e 6d 64 00 00 00 00 00  00 00 00 00 00 00 00 00  |.md.............|

What is your proposal?
We need three parts of information for that custom signature, as described in your link:

Code: Select all

   extension name
    offset of the signature
    signature or magic value

What are these?
How would you write this into a text file according to the rules outlined in your link?

Not sure regarding what the offset of the signature should be
Not sure also regarding signature or magic value
(I regret to say that I am new to this )

My proposal would be to take the second line and turn it into hexadecimal format as shown in the documentation so that I am able to make a photorec.sig file with it using vim.

[recuperation]

Where is that something, at which position from the beginning of the file do you find it?

One byte (of your file) can be represented by two symbols of the hexadecimal system. You see 16 bytes per line in your hexdump.
The first column of your hexdump show you the relative position of the first element of the series of 16 bytes in a hexadecimal coding:
00 (hex) = 0 (dec)
10 (hex) = 16 (dec)
20 (hex) = 32 (dec)
30 (hex) = 48 (dec)

The hexadecimal system (based of 16 numbers instead of the usual 10 in our decimal system) count is:
0 1 2 3 4 5 6 7 8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 .... 90 91 .... a0 a1 a2... b0 b1 b2 ... f0 f1 ... ff

ff (hex) represent 255 (dec).

The whole thing is easier than you think - you don't have to be a nerd to create your custom signature.

[leni1]

So I've understood the formula correctly, the signature file would end up looking something like

Code: Select all

jex 32 0x2e6d6400000000000000000000000000

where 32 is the offset and the rest of the line stays as is since it is already in hexadecimal format.

Correct me if I got it wrong

[recuperation]

Perfect.
Write the
ASCII-file
with your preferred text editor and save the file.

Now dig out the TestDisk manual, read chapter 12, two traps are described there. Make sure to exclude them.
Tell me what you checked!

Now my proposal:
Put one jex file combined with other files on a small USB pen drive and run PhotoRec against that USB pen drive to make sure, that your jex file is being recognized. Depending on fragmentation on the pen drive the jex file may not be recognized correctly but at least the beginning containing the signature should be found.

If you succeed, you are ready to go.

If Photorec does not find anything using your signature file, shorten it to the the 3 bytes that are not zero (0x2e6d64).

The issue with the length of our assumed fingerprint (we are only guessing as we don't have a file format description by its authors) is comparable to a Covid-Test:
If you are very specific and using a long signature you will not have false positives (other file types that look the same by chance). On the other hand, you might miss some jex files, because all the trailing zeros may not be part of the fingerprint but only looking at three files lead us to believe so.
If you are less specific you will not miss any real jex files, but you will also find many false positives (files with the signature that are nevertheless not jex files).

[leni1]

The traps that are likely:

Warning: If you are using a .photorec.sig in your HOME directory, be warned that when running photorec as
root (ie. via the sudo command), photorec will search in /root/.photorec.sig, not in your user home directory.
So you may have to copy the .photorec.sig file first.

Check your custom signature with fidentify
fidentify now perfectly identify the file

Code: Select all

[kmaster@adsl ~]$ fidentify /home/kmaster/src/testfiles/sample.pfi
/home/kmaster/src/testfiles/sample.pfi: pfi

If fidentify doesn’t recognize the signature, check your signature, it may be incorrect
• verify that the signature file is a true ASCII text file. It must not begin by EF BB BF (UTF-8 Byte Order
Mark) or FF FE (UTF-16 LE BOM) by example.
• verify the filename of your signature file

The other about not including non-printable chars should not affect me since I will be using the hexadecimal format.

Will try out your proposal and see how it goes

[leni1]

So this worked...sort of

What worked: I was able to get the jex files and none of them were not Joplin Export files

What was strange: The image of my data is on a 5TB drive and in the PhotoRec options, I chose to search the whole partition. The search ends at ~1 hour or so mark, alerting me that I do not have enough space on the destination. By ends, I mean the search pauses, asks me to confirm that the destination directory is correct and then returns me to the screen showing how many passes it has made, with estimated time having tripled from ~11hrs to ~120hrs.

The image is of a failing HDD that was NTFS partitioned and of size 250GB (the data itself is ~230GB in size). The destination is a folder on a 1TB drive which should have enough space.

Is there a way to correct/work around this behaviour?

[recuperation]

As opposed to your initial postings I had to read the last one three times.
Could you specify exactly what you mean by "this behaviour"?

[leni1]

As opposed to your initial postings I had to read the last one three times.
Could you specify exactly what you mean by "this behaviour"?

What was strange: The image of my data is on a 5TB drive and in the PhotoRec options, I chose to search the whole partition. The search ends at ~1 hour or so mark, alerting me that I do not have enough space on the destination. By ends, I mean the search pauses, asks me to confirm that the destination directory is correct and then returns me to the screen showing how many passes it has made, with estimated time having tripled from ~11hrs to ~120hrs.

The above is the behaviour I meant.

The image is of a failing HDD that was NTFS partitioned and of size 250GB (the data itself is ~230GB in size). The destination is a folder on a 1TB drive which should have enough space.

with this line describing the environment in which I am doing the data recovery. I trust that makes it clear.