Search This Blog

2014-03-19

Read-Only Filesystems And Corrupted Journals

Sometimes after rebooting during init you will run into scenarios where the root filesystem or other filesystems are mounted read-only. This is generally due to problems with journaling being corrupted and Linux mounting read-only to prevent further damage to the filesystem because it feels that it is currently in an unrecoverable state.

Generally you're looking for these messages in dmesg:
dmesg | grep -i 'journal|read-only'
Write protecting the kernel read-only data: 530k
kjournald starting.  Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
Aborting journal on device dm-0.
EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only
journal commit I/O error

Steps to recover from read-only corrupted journal.

Reboot and enter single-user mode
init 1

remount the filesystem read-only
remount -n -o remount,ro /dev/sda1

Check the disk for errors
fsck -y /dev/sda1

Remove the Journal
tune2fs -O ^has_journal /dev/sda1

Re-mount as RW
mount -n -o remount,rw /dev/sda1

Re-create the journal
tune2fs -j /dev/sda1

re-mount the filesystem read-only again
remount -n -o remount,ro /dev/sda1

Check the file system (and have the journal inode block info backed up

fsck -f /dev/sda1

Reboot and it should mount the root filesystem fine again.

http://benohead.com/rebuild-journal-on-a-ext3-file-system/
http://www.cyberciti.biz/tips/linux-filesytem-goes-read-only.html
http://www.linuxquestions.org/questions/linux-newbie-8/aborted-journal-and-volume-remounted-read-only-812216/
https://access.redhat.com/site/solutions/26049

No comments:

Post a Comment