Code:
fsck -fCV /dev/md0p1
and paste the output
EDIT: and the output from
Code:
dmesg | grep md0
Topic: - page 2. (Read 639 times)
You sure there is an ext4 filesystem to be expected? (Just asking, no offence meant)
Otherwise I'd be keen to hear of any ideas from makrospex, I am getting a bit lost, the loop part still doesn't let go on me.
Just as some therapeutic exercise, could you post the output of
(I am really just fishing for new ideas here)
At this point I need to ask: Did this contain a wallet that you didn't have backuped otherwise or any other unique and valuable data or just a synched btc node that you want to get back online without synching from genesis?
Otherwise I'd be keen to hear of any ideas from makrospex, I am getting a bit lost, the loop part still doesn't let go on me.
Just as some therapeutic exercise, could you post the output of
Code:
dumpe2fs -h /dev/md0p1
(I am really just fishing for new ideas here)
At this point I need to ask: Did this contain a wallet that you didn't have backuped otherwise or any other unique and valuable data or just a synched btc node that you want to get back online without synching from genesis?
At least there are valid superblock backups, that's a start I guess.
Please try the mounts with the device I suggested above and not /dev/md0:
i.e.
Please try the mounts with the device I suggested above and not /dev/md0:
i.e.
Code:
mount -t ext4 -o ro,offset=32768 /dev/md0p1 /mnt/md0
Code:
root@bitcorn:/mnt/md0# mount -t ext4 -o ro,offset=32768 /dev/md0p1 /mnt/md0
mount: /mnt/md0: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
from mke2fs: /dev/md0p1 alignment is offset by 1024 bytes.
This may result in very poor performance, (re)-partitioning suggested.
Might that have something to do with it ? Add 1024 to whatever the offsets say ?
Hmm, not sure about the offset, you could try with 1024 and by adding 1024, you could also try to fsck it with the backup superblock added:
Code:
fsck.ext4 -b 32768 /dev/md0p1
Otherwise I am bit confused about the loop part still, did you by chance have for example an encrypted filesystem on top of that raid device?
^The software just simply tells us where these backup superblocks would be stored on a partition of given size.
Please try the mounts with the device I suggested above and not /dev/md0:
i.e.
Code:
mount -t ext4 -o ro,offset=32768 /dev/md0p1 /mnt/md0
doh
then it's making out the backup superblock adresses via
Code:
mke2fs -n
My fucking spacebar is making me mad, something sticksunder it, so i have to punch it after every word to get a space in the text
Agree with makrospex, next try would be actually finding and restoring a good backup of the superblock of m0p1, I think he meant to say:
Code:
mke2fs -n /dev/md0p1
to find the superblocks.
Triplecheck it has the -n before you answer 'y' to the following prompt!
Regarding the mounts to try from makrospex, please try these again with /dev/md0p1 instead of /dev/md0, i.e.:
Code:
mount -t ext4 -o=ro,offset xxx /dev/md0p1 /mnt/md0
Maybe also check that /mnt/md0 is empty, otherwise the mount might fail with a slightly different error message that you could overlook easily.
First a question: Did one of the succeed?
So is showing the contents of the raid?
Code:
mount -t ext4 -o=ro,offset xxx /dev/md0 /mnt/md0
So
Code:
ls -la /mnt/md0
Was not able to get it to mount at all using the offset calculations.
doh
then it's making out the backup superblock adresses via
Code:
mke2fs -n
My fucking spacebar is making me mad, something sticksunder it, so i have to punch it after every word to get a space in the text
@Bob: Can you show us /etc/mdadm/mdadm.conf and when it was last modified (so we know it was not recreated recently):
It might gives us a new idea..
Code:
cat /etc/mdadm/mdadm.conf; stat /etc/mdadm/mdadm.conf
* psycodad needs much more coffee..
I commented out the mdadm.conf ARRAY stuff. I figured it looked fucked up enough that it was effectively useless. I have no idea what I'm doing.
Code:
#ARRAY /dev/md0 metadata=1.2 spares=1 name=xserver:0 UUID=e0104263:8aef7ef8:2d66045f:26a69f31
#ARRAY /dev/md0 metadata=1.2 spares=1 name=xserver:0 UUID=6314fb8d:1466fbbd:5aa88188:793c3019
#ARRAY /dev/md0 metadata=1.2 spares=1 name=bitcorn:0 UUID=4b7f5741:476a4550:ba1184b2:76efd22e
#ARRAY /dev/md/0 metadata=1.2 UUID=4b7f5741:476a4550:ba1184b2:76efd22e name=bitcorn:0
#ARRAY /dev/md/bitcorn:0 metadata=1.2 name=bitcorn:0 UUID=4196079a:155fab25:eb7fa044:37adb2ce
I hope you made a backup of the file, which you can move back in place?
Or roll back any changes?
Let's continue with the "script":
First a question: Did one of the
Code:
mount -t ext4 -o=ro,offset xxx /dev/md0 /mnt/md0
So
Code:
ls -la /mnt/md0
If yes, copy away what you need and then:
Code:
umount /dev/md0
mke2fs -t ext4 -n /dev/md0p1
and paste the output into the reply.
Don't give it up yet, you seem frustrated, it's too early, man
EDIT: First please follow Psycodad's guidance, i crossposted but was definitely later than him.
@Bob: Can you show us /etc/mdadm/mdadm.conf and when it was last modified (so we know it was not recreated recently):
It might gives us a new idea..
Code:
cat /etc/mdadm/mdadm.conf; stat /etc/mdadm/mdadm.conf
* psycodad needs much more coffee..
I commented out the mdadm.conf ARRAY stuff. I figured it looked fucked up enough that it was effectively useless. I have no idea what I'm doing.
Code:
#ARRAY /dev/md0 metadata=1.2 spares=1 name=xserver:0 UUID=e0104263:8aef7ef8:2d66045f:26a69f31
#ARRAY /dev/md0 metadata=1.2 spares=1 name=xserver:0 UUID=6314fb8d:1466fbbd:5aa88188:793c3019
#ARRAY /dev/md0 metadata=1.2 spares=1 name=bitcorn:0 UUID=4b7f5741:476a4550:ba1184b2:76efd22e
#ARRAY /dev/md/0 metadata=1.2 UUID=4b7f5741:476a4550:ba1184b2:76efd22e name=bitcorn:0
#ARRAY /dev/md/bitcorn:0 metadata=1.2 name=bitcorn:0 UUID=4196079a:155fab25:eb7fa044:37adb2ce
Okay, that looks pretty weird, I guess that system has seen quite a few changes already.
I am still circling around this partition type 0xfd on m0p1. Just for fun and giggles could you try to see if there is another raid device on top of m0p1 when you scan now?
Code:
mdadm --assemble --scan
Not trying to be a smartass at all (ok, maybe a tiny lil bit
), that was what I meant by:If dumpe2fs finds no superblock backups, there is the risky way of using mke2fs in simulation mode, by basically 'dry'-reformatting your partition the computer will come up with the same superblock addresses as the first time it formatted it.
RL calls, will check back tomorrow.
It seems dangerous, it will ask you to answer y to reformat your partition (even though with -n it doesn't). Though I tried it on USB stick and it does indeed not reformat (even the wording and confirmation is confusion as the author of mke2fs once admitted).
But yes, that would be one of the next steps I'd also propose.
After that I only have testdisk in my mind, but that's rather dangerous too easy to do something wrong.
Right, testdisk would also not be suitable for guided use, i would want my own hands on that.
So the next steps are clear now. I'd like to know if the offset mount of /dev/md0 was successful, but i'll have to wait for the TC to show up again.
Also mdadm.conf contents...
Since the source of the errors was a power outage and the filesystem wasn't in sync, i'm positive that the problem will finally be solved.
bad blocks would be worse.
Also, i seem to be in luck this week. Two days ago i shadow-copied an old laptop's hdd, which was making occasional noises of horror, into a 18gig .vhd file for use in Virtualbox for almost six hours. Roughly fifteen minutes after completion, the laptop rebooted and got stuck at the bios splash screen. Then i removed the drive, connected it via usb-ide cable and copied over the .vhd file to the virtualbox host. The file was vital, but the drive died just before i wanted to disconnected it later (i was lazy).
With so much luck, i'm confident
Quote
-n
Causes mke2fs to not actually create a filesystem, but display what it would do if it were to create a filesystem. This can be used to determine the location of the backup superblocks for a particular filesystem, so long as the mke2fs parameters that were passed when the filesystem was originally created are used again. (With the -n option added, of course!)
Causes mke2fs to not actually create a filesystem, but display what it would do if it were to create a filesystem. This can be used to determine the location of the backup superblocks for a particular filesystem, so long as the mke2fs parameters that were passed when the filesystem was originally created are used again. (With the -n option added, of course!)
I suggest the default values ("auto" at setup) were used on filesystem creation, so
Code:
mke2fs -n /dev/md0p1
should output the same backup block numbers for use with
Code:
fsck -B ...
Waddaya think?
Not trying to be a smartass at all (ok, maybe a tiny lil bit
), that was what I meant by:If dumpe2fs finds no superblock backups, there is the risky way of using mke2fs in simulation mode, by basically 'dry'-reformatting your partition the computer will come up with the same superblock addresses as the first time it formatted it.
RL calls, will check back tomorrow.
It seems dangerous, it will ask you to answer y to reformat your partition (even though with -n it doesn't). Though I tried it on USB stick and it does indeed not reformat (even the wording and confirmation is confusion as the author of mke2fs once admitted).
But yes, that would be one of the next steps I'd also propose.
After that I only have testdisk in my mind, but that's rather dangerous too easy to do something wrong.
Moinmoin,
Thanks makrospex for picking up where I left :-)
Thanks makrospex for picking up where I left :-)
Gern geschehen
sgdisk -p /dev/md0
Code:
root@bitcorn:/dev# sgdisk -p /dev/md0
Disk /dev/md0: 15627544576 sectors, 7.3 TiB
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 00000000-0000-0000-0000-000000000000
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 15627544542
Partitions will be aligned on 8-sector boundaries
Total free space is 8141 sectors (4.0 MiB)
Number Start (sector) End (sector) Size Code Name
1 5118 15611104253 7.3 TiB FD00
2 15611104256 15627541487 7.8 GiB 8200
While p2 is correctly showing as type "linux swap" (0x82), p1 shows as "Linux raid autodetect" (0xfd).
IMHO it should show as 0x83 for an ext4 partition. Looking for some confirmation or shown to be wrong from makrospex here.
@Bob: Can you show us /etc/mdadm/mdadm.conf and when it was last modified (so we know it was not recreated recently):
Code:
cat /etc/mdadm/mdadm.conf; stat /etc/mdadm/mdadm.conf
It might gives us a new idea..
* psycodad needs much more coffee..
I realize that i need one too
I was also wondering, looks like *cough* linuxnoob partitioning to me.
Now i'll paste what i was about to post before reading your reply:
I read into the mke2fs man page and found the -n switch:
Quote
-n
Causes mke2fs to not actually create a filesystem, but display what it would do if it were to create a filesystem. This can be used to determine the location of the backup superblocks for a particular filesystem, so long as the mke2fs parameters that were passed when the filesystem was originally created are used again. (With the -n option added, of course!)
Causes mke2fs to not actually create a filesystem, but display what it would do if it were to create a filesystem. This can be used to determine the location of the backup superblocks for a particular filesystem, so long as the mke2fs parameters that were passed when the filesystem was originally created are used again. (With the -n option added, of course!)
I suggest the default values ("auto" at setup) were used on filesystem creation, so
Code:
mke2fs -n /dev/md0p1
should output the same backup block numbers for use with
Code:
fsck -B ...
Waddaya think?
EDIT: knowing mdadm.conf contents would surely help, but BobLawBlaw has to wake up first. Damned timeshifts.
Moinmoin,
Thanks makrospex for picking up where I left :-)
Anyway, there is one thing that strikes me as very odd, but it may be too early or I may be simply too stupid (or both), but...:
While p2 is correctly showing as type "linux swap" (0x82), p1 shows as "Linux raid autodetect" (0xfd).
IMHO it should show as 0x83 for an ext4 partition. Looking for some confirmation or shown to be wrong from makrospex here.
@Bob: Can you show us /etc/mdadm/mdadm.conf and when it was last modified (so we know it was not recreated recently):
It might gives us a new idea..
Thanks makrospex for picking up where I left :-)
Anyway, there is one thing that strikes me as very odd, but it may be too early or I may be simply too stupid (or both), but...:
sgdisk -p /dev/md0
Code:
root@bitcorn:/dev# sgdisk -p /dev/md0
Disk /dev/md0: 15627544576 sectors, 7.3 TiB
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 00000000-0000-0000-0000-000000000000
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 15627544542
Partitions will be aligned on 8-sector boundaries
Total free space is 8141 sectors (4.0 MiB)
Number Start (sector) End (sector) Size Code Name
1 5118 15611104253 7.3 TiB FD00
2 15611104256 15627541487 7.8 GiB 8200
While p2 is correctly showing as type "linux swap" (0x82), p1 shows as "Linux raid autodetect" (0xfd).
IMHO it should show as 0x83 for an ext4 partition. Looking for some confirmation or shown to be wrong from makrospex here.
@Bob: Can you show us /etc/mdadm/mdadm.conf and when it was last modified (so we know it was not recreated recently):
Code:
cat /etc/mdadm/mdadm.conf; stat /etc/mdadm/mdadm.conf
It might gives us a new idea..
* psycodad needs much more coffee..
Please calculate the value manually and omit the quotes.
Yeah, figured I'd just enter the calculated values. No freaking idea what /dev/loop0 has anything to do with anything.
Linux really makes me sad sometimes
EDIT: Sleep time. Back in a few hours. Thanks again !
you're welcome.
The "loop block device" lets you use a file as if it were a block device (a virtual hard drive, for example).
https://en.wikipedia.org/wiki/Loop_device
Windows made me even more sad, tbh.
But that's a longer story, and way off topic
Code:
mount -t ext4 -o ro,offset=((5118*4096)) /dev/md0 /path/to/mountpoint
Code:
mount -t ext4 -o ro,offset=((5118*512)) /dev/md0 /path/to/mountpoint
Code:
root@bitcorn:/dev# mount -t ext4 -o ro,offset=((5118*4096)) /dev/md0 /mnt/md0
bash: syntax error near unexpected token `('
root@bitcorn:/dev# mount -t ext4 -o ro,offset=((5118*512)) /dev/md0 /mnt/md0
bash: syntax error near unexpected token `('
Poop.
Arrr :/
Please calculate the value manually and omit the quotes.
mount -t ext4 -o ro,offset=20963328 /dev/md0 /mnt/md0
or
mount -t ext4 -o ro,offset=2620416 /dev/md0 /mnt/md0
Good morning
So you have a drive sector size of 4096 bytes = 512 bytes * 8 and we know the starting sector number.
Psycodad's opinion on recreating the filesystem with mke2fs seems a valid option to me there.
But first, i want you to try to mount partition1 directly, but without trying to read the bad superblock,which (iirc) should work this way:
or
I don't know exactly which size is the right one to calc the offset, but imho it's the physical sector size (=command 1 should work, imo).
If either mount command succeeds, check if your files are there in /path/to/mountpoint.
Partition 1 (which is /dev/md0p1 in linux) will be mounted in read-only mode.
BUT: The filesystem could alsohave errors, that's why it is mounted in read-only mode. At least you are able to access the data for backup, but the final goal should be recreation of a healthy superblock, so fsck can do it's work before you remount in read-write mode.
If none of the above commands work, i'd try mke2fs to create a new superblock and handle the drive conventionally.
Wait, peeking for the first superblock backup location by multiples of the sector size for would be a valid option before that, but i want to read Psycodad's opinion on this first.
For future filesystem creation(s):
Optimally, logical sector size would match physical sector size and the start of any partition should be a multiple of that, unless you have to store many very small files (below physical sector size).
So you have a drive sector size of 4096 bytes = 512 bytes * 8 and we know the starting sector number.
Quote
Number Start (sector) End (sector) Size Code Name
1 5118 15611104253 7.3 TiB FD00
1 5118 15611104253 7.3 TiB FD00
Psycodad's opinion on recreating the filesystem with mke2fs seems a valid option to me there.
But first, i want you to try to mount partition1 directly, but without trying to read the bad superblock,which (iirc) should work this way:
Code:
mount -t ext4 -o ro,offset=((5118*4096)) /dev/md0 /path/to/mountpoint
or
Code:
mount -t ext4 -o ro,offset=((5118*512)) /dev/md0 /path/to/mountpoint
I don't know exactly which size is the right one to calc the offset, but imho it's the physical sector size (=command 1 should work, imo).
If either mount command succeeds, check if your files are there in /path/to/mountpoint.
Partition 1 (which is /dev/md0p1 in linux) will be mounted in read-only mode.
BUT: The filesystem could alsohave errors, that's why it is mounted in read-only mode. At least you are able to access the data for backup, but the final goal should be recreation of a healthy superblock, so fsck can do it's work before you remount in read-write mode.
If none of the above commands work, i'd try mke2fs to create a new superblock and handle the drive conventionally.
Wait, peeking for the first superblock backup location by multiples of the sector size for
Code:
fsck -B ...
For future filesystem creation(s):
Optimally, logical sector size would match physical sector size and the start of any partition should be a multiple of that, unless you have to store many very small files (below physical sector size).
No sleep yet 
I think we are on a similar pathway.
Seems to appear in some recovery situations automatically, according to some outbound forum threads.
(possibly also created during reorg/rebuilds?)
Maybe try
first?
With a forced repair afterwards, following a backup, if not already created.
Hurr... fsck on mountpoint... Durr... I'm special.
you and me both, guess who had to learn that you can't mount multiple file systems on a single mount point...
That's the way it goes.
Like Zen, only linux-wise
Good points/ideas! Will be interesting to see the output of sgdisk and if it confirms my current theory.
I think we are on a similar pathway.
Never seen that md127 before, would check /etc/mdadm/mdadm.conf, but that's only a wild guess.
Seems to appear in some recovery situations automatically, according to some outbound forum threads.
(possibly also created during reorg/rebuilds?)
After revisiting the below, I noticed that md0p1 has no recognized filesystem. So trying to rebuild the journal with tune2fs is probably of no help. But it indeed hints to a broken super block me thinks.
...
It definitely should read TYPE="ext4" at the md0p1 line, but there is no partition type detected at all.
Based on this the best recommendation I can come up with at this point is trying to restore a backup superblock on md0p1.
If dumpe2fs finds no superblock backups, there is the risky way of using mke2fs in simulation mode, by basically 'dry'-reformatting your partition the computer will come up with the same superblock addresses as the first time it formatted it.
RL calls, will check back tomorrow.
...
It definitely should read TYPE="ext4" at the md0p1 line, but there is no partition type detected at all.
Based on this the best recommendation I can come up with at this point is trying to restore a backup superblock on md0p1.
If dumpe2fs finds no superblock backups, there is the risky way of using mke2fs in simulation mode, by basically 'dry'-reformatting your partition the computer will come up with the same superblock addresses as the first time it formatted it.
RL calls, will check back tomorrow.
Maybe try
Code:
fsck -t ext4 /dev/md0p1
first?
With a forced repair afterwards, following a backup, if not already created.
Hurr... fsck on mountpoint... Durr... I'm special.
you and me both, guess who had to learn that you can't mount multiple file systems on a single mount point...
That's the way it goes.
Like Zen, only linux-wise
Hurr... fsck on mountpoint... Durr... I'm special.
you and me both, guess who had to learn that you can't mount multiple file systems on a single mount point...
Good points/ideas! Will be interesting to see the output of sgdisk and if it confirms my current theory.
I also didn't find much about /dev/md127 on the net. any idea psycodad?
Never seen that md127 before, would check /etc/mdadm/mdadm.conf, but that's only a wild guess.
After revisiting the below, I noticed that md0p1 has no recognized filesystem. So trying to rebuild the journal with tune2fs is probably of no help. But it indeed hints to a broken super block me thinks.
re: lsblk -f, etc. re: It crashed due to power failure, for the sake of argument. Cant be sure of LVM. I'm a bit of a tard when it comes to Linux.
Code:
root@bitcorn:/# lsblk -f
root@bitcorn:/# blkid
/dev/sda1: UUID="BA8D-865F" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="080f6638-f09f-4518-a785-6212e1b305b8"
/dev/sda2: UUID="42a6267e-28ba-4c93-8554-3dea69f5eb2e" TYPE="ext4" PARTUUID="f25d3ecd-9158-42b4-b502-4853b96bba78"
/dev/sda3: UUID="9bdd83b3-0bb0-474e-a83e-8394a1fb2e8b" TYPE="swap" PARTUUID="75ae0f41-c850-4153-af77-b2085f465a4d"
/dev/sdb: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="8757b0af-bb26-b0e6-f0e1-eb5672344df3" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/sdc: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="600cedf1-06f6-f8a1-5c27-528f7588eada" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/sdd: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="9c08f657-41d9-1a24-33b9-0f993e7f0d6f" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/md0: PTTYPE="gpt"
/dev/md0p1: PARTUUID="11d10ea4-e100-3949-b9d2-c30072ef7648"
/dev/md0p2: UUID="3df11d9c-51f2-48d8-9ab0-49b6dcd47753" TYPE="swap" PARTUUID="53d67f15-7962-ed43-840a-881c288fc86a"
root@bitcorn:/#
root@bitcorn:/# blkid
/dev/sda1: UUID="BA8D-865F" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="080f6638-f09f-4518-a785-6212e1b305b8"
/dev/sda2: UUID="42a6267e-28ba-4c93-8554-3dea69f5eb2e" TYPE="ext4" PARTUUID="f25d3ecd-9158-42b4-b502-4853b96bba78"
/dev/sda3: UUID="9bdd83b3-0bb0-474e-a83e-8394a1fb2e8b" TYPE="swap" PARTUUID="75ae0f41-c850-4153-af77-b2085f465a4d"
/dev/sdb: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="8757b0af-bb26-b0e6-f0e1-eb5672344df3" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/sdc: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="600cedf1-06f6-f8a1-5c27-528f7588eada" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/sdd: UUID="b7b2edc8-cd74-ad69-e5e4-81961dcc8509" UUID_SUB="9c08f657-41d9-1a24-33b9-0f993e7f0d6f" LABEL="bitcorn:0" TYPE="linux_raid_member"
/dev/md0: PTTYPE="gpt"
/dev/md0p1: PARTUUID="11d10ea4-e100-3949-b9d2-c30072ef7648"
/dev/md0p2: UUID="3df11d9c-51f2-48d8-9ab0-49b6dcd47753" TYPE="swap" PARTUUID="53d67f15-7962-ed43-840a-881c288fc86a"
root@bitcorn:/#
It definitely should read TYPE="ext4" at the md0p1 line, but there is no partition type detected at all.
Based on this the best recommendation I can come up with at this point is trying to restore a backup superblock on md0p1.
If dumpe2fs finds no superblock backups, there is the risky way of using mke2fs in simulation mode, by basically 'dry'-reformatting your partition the computer will come up with the same superblock addresses as the first time it formatted it.
RL calls, will check back tomorrow.
Jump to: