Sep 232016

For the first time I have started using LVM on my home server and things have been alright for an year. I am able to resize, replace disks , expand size of my mount points or move mount to a new disk without any trouble at all.

While its not relevant I am using CentOS 7.

I have had to force reboot the server a couple of times over the year and woefully XFS complained of corruption 24 hours ago and it was time to figure this out. I kept dropping into emergency mode upon reboot. There is also another reason for this problem. I have bad memory, I bought some refurbished FB-DIMM DDR2 with ECC and ironically not all of it is good.

Anyway TL;DR

Filesystem : XFS

Boot with rescue CD/USB whatever and dont mount anything

Using LVM? yes: Try with lvscan to confirm

I  run xfs_repair /dev/sdx1 and it complains of bad superblock and it also fails to find a good secondary superblock. Something like `Sorry, could not find valid secondary superblock
Exiting now.`

Wait! I need to repair from the Virtual Volume

start with

vgscan -v --mknodes

find and create /dev/xxx/ nodes for the lvm.

Now activate the volumes (it does not mount them)

vgchange -a y

Now see what volumes are active with another lvscan

I have /dev/centos/root which is what I need to fix. It is an XFS volume so this time I do

xfs_repair /dev/centos/root

bunch of output follows…. repair is done.

Lets mount and test

mkdir /mnt/test && mount /dev/centos/root /mnt/test

Looks good?

Unmount and reboot without rescue CD.