UFS - 0 bytes file broke filesystem header?

Message

Markweiser · #1 Post by **Markweiser** » 12 Apr 2016, 18:25

A human error broke an UFS filesystem (Solaris).

When trying to do a backup of the filesytem (c0t0d0s3) the ufsdump(1M) hasn't been correctly used.
I will explain the background that led to this ...

The admin used:

Code: Select all

# ufsdump 0f /dev/dsk/c0t0d0s3 > output_1
root@ats-br000432 # ufsdump 0f /dev/dsk/c0t0d0s3 > output_1
Usage: ufsdump [0123456789fustdWwnNDCcbavloS [argument]] filesystem

This is a bad usage, so it created a file called output_1 with 0 bytes:

Code: Select all

# ls -la output_1
-rw-r--r--   1 root   root         0 abr 12 14:12 output_1

Then, the syntax used was:

Code: Select all

# ufsdump 0f /dev/rdsk/c0t0d0s3 output_1

Which wrote that 0 bytes file output_1 to /dev/rdsk/c0t0d0s3 - which was the partition slice

Now, interestingly, due to being a 0 bytes file, we thought that this would cause no harm to the filesystem, but it did.
When trying to ls in the mountpoint, the partition claimed there was an I/O error.

I assume, at some point, the filesystem 'header' was affected, right? Or was it the slice information?

The vital files that are inside the partition which we do require are *.dbf files (Oracle database), at this moment we've been running testdisk and photorec to try recovering it, we could get more than 58694 at the moment, but no dbf files, this is current status from photorec:

Code: Select all

PhotoRec 7.0, Data Recovery Utility, April 2015
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org

Disk dsk_corrupt_c0t0d0s3 - 135 GB / 126 GiB (RO)
     Partition                  Start        End    Size in sectors
   P UFS 2 - Little Endian     0   0  1 16501 254 63  265104630 [/database]


Pass 1 - Reading sector   51779678/265104630, 60845 files found
Elapsed time 0h29m07s - Estimated time to completion 1h59m57
txt: 36762 recovered
gif: 13052 recovered
class: 4161 recovered
png: 3112 recovered  
tx?: 1964 recovered
tar: 1111 recovered
zip: 316 recovered
gz: 127 recovered 
elf: 93 recovered 
jpg: 32 recovered
others: 115 recovered
  Stop

Does any of you have suggestions? Why and how this 0-byte file affected UFS? What can we try to do?

By the way, normally, when superblock problems happen on extX, we can use mkfs.extX -S to recreate them, is there an equivalent with testdisk?

Appreciate a lot your time

newfs -N output

Code: Select all

# newfs -N /dev/rdsk/c0t0d0s3
Warning: 2826 sector(s) in last cylinder unallocated
/dev/rdsk/c0t0d0s3:     265104630 sectors in 43149 cylinders of 48 tracks, 128 sectors
        129445,6MB in 2697 cyl groups (16 c/g, 48,00MB/g, 5824 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
 98464, 196896, 295328, 393760, 492192, 590624, 689056, 787488, 885920,
Initializing cylinder groups:
.....................................................
super-block backups for last 10 cylinder groups at:
 264150944, 264241184, 264339616, 264438048, 264536480, 264634912, 264733344,
 264831776, 264930208, 265028640

Edit: first fsck impressions about it:

Code: Select all

** /dev/rdsk/c0t0d0s3
** Last Mounted on /database
** Phase 1 - Check Blocks and Sizes
INCORRECT DISK BLOCK COUNT I=11 (400 should be 208)
CORRECT?

The rest is attached (fsck -N) -

c0t0d0s3_fsck-N_20160412.zip: fsck output (fsck -N); (43.49 KiB) Downloaded 259 times

Also, testdisk logfile with /debug and /log set

testdisk.zip: testdisk logfile; (43.04 KiB) Downloaded 250 times

and a hexdump of first 5000 lines of the filesystem (od -x /dev/rdsk/c0t0d0s3 |head -5000)

hex_5000lines-.zip: hexdump first 5000 lines of the filesystem; (23.33 KiB) Downloaded 313 times