Last night, my mail server died...
Jan. 28th, 2004 02:02 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
fleetfootmikeRather messily. Throwing spurious disk errors.
So. Last night I tried to do what I could to fix the disk errors, including turning UDMA off on the offending drive. That fixed it for a bit, and most things were working, except useful things like the virus scanner, and anything to do with Debian package installs, all of which died with 'Segmentation Fault'.
At half past midnight, I gave up, went outside to the server room (discovering it had snowed in the process) to turn it off, and went to bed.
This morning dawned crisp and even brightly...
9.15am: download a new Debian net install CD from the comfort of my armchair on to the machine in the office with the CD-R.
9.40am: Finally run out of displacement activity to put off going out in the cold again, girded my loins, went out, started the Debian CD burning, and spent 10 mins wrestling to get the server out of the rack.
9.50am: bring machine it inside, along with spare hard drive, IDE ribbon cable.
9.55am: outside to collect Debian CD.
10.00am: plug machine up to power, keyboard, monitor and network port in the semi-warm indoors. Move iffy disk and CD drive to IDE secondary, hang brand new 20G drive off primary. Hit power.... Machine finds new drive and CD, but not old drive.
10:05am: jiggle cables, reroute cables, check jumpers on HD's. Curse idiot who put massive ATI TV tuner card in mini tower, thus making cable routing a complete sod. Remember idiot is self. Curse self. Boot again. Same problem.
10:10am: reboot. Set BIOS to boot from CD first. hit eject button on CD to insert Debian CD. Zip.
10:15am: remove suspect HD. more cable jiggling, apply paperclip to eject CD drawer, insert CD, close draw. Boot. Nada.
10:20am: retrieve box from car containing nice Yamaha CD-RW from![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-userinfo.gif)
bedlamhouse.
10:22am: realise nice CD-RW is SCSI, and about as much immediate use as a chocolate fireguard. Gird loins and head outside again
10.30am: finish reviewing pile of mostly dead PCs in office - reject DVD-ROM in one as being known flaky, reject pile of strange CD drives in cupboard (including two external ones and an original Creative SB 8x CD, whose interface I do not wish to contemplate). Find Acer 50x drive in fourth PC down pile, remove.
10:35am: swap mounting rails over from old CD to newer Acer drive.
10:40am: make tea, start writing this LJ entry.
10:50am: fit Acer drive in server. Pause. Find paperclip. Remove Debian CD from old drive using paperclip.
10:55am: apply power. CD drawer opens, disk spins.
11:00am: remove power. On a whim, reattach suspect hard drive. Reboot with Debian CD in drive. BINGO! Both HD's and CD found, machine starts to boot installer.
11:05am: start fresh Debian install on new HD. It might not need it now, but I'm in the mood to be paranoid: that old HD was giving some very very odd errors.
11:09am: admire shininess of new Debian installer
11;10am; CD fails to read network drivers. Swear, reboot.
11:15am: fine second time round, partition new disk, leave machine badblock scanning, have early lunch while admiring shininess of new Debian installer some more
11:30am: return to machine to find various errors, reboot, try again. Reboot, try yet again with old HD disconnected. Still craps out. Peer, as instructed, in /target/var/log/debbootstrap,log, to find the very helpful 'Segmentation Fault'.
12:00noon: spent half an hour considering alternatives. Begin to suspect old HD is in fact fine, and can be plonked in different machine pretty safely. Temporarily cannibalise old Linux workstation.,...
12:15pm: Old HD fsck's fine, after a tantrum or two, in old workstation. Take deep breath, let it continue booting.
12:20pm: Well, it makes it to a root prompt, and MailScanner seems to be clearing the mail queue.
12:21pm: break for delivering James to doctors, via Anne, and a walk in the park withbardling
2:00pm: It's still up, and still behaving. Make tea.
So. Last night I tried to do what I could to fix the disk errors, including turning UDMA off on the offending drive. That fixed it for a bit, and most things were working, except useful things like the virus scanner, and anything to do with Debian package installs, all of which died with 'Segmentation Fault'.
At half past midnight, I gave up, went outside to the server room (discovering it had snowed in the process) to turn it off, and went to bed.
This morning dawned crisp and even brightly...
9.15am: download a new Debian net install CD from the comfort of my armchair on to the machine in the office with the CD-R.
9.40am: Finally run out of displacement activity to put off going out in the cold again, girded my loins, went out, started the Debian CD burning, and spent 10 mins wrestling to get the server out of the rack.
9.50am: bring machine it inside, along with spare hard drive, IDE ribbon cable.
9.55am: outside to collect Debian CD.
10.00am: plug machine up to power, keyboard, monitor and network port in the semi-warm indoors. Move iffy disk and CD drive to IDE secondary, hang brand new 20G drive off primary. Hit power.... Machine finds new drive and CD, but not old drive.
10:05am: jiggle cables, reroute cables, check jumpers on HD's. Curse idiot who put massive ATI TV tuner card in mini tower, thus making cable routing a complete sod. Remember idiot is self. Curse self. Boot again. Same problem.
10:10am: reboot. Set BIOS to boot from CD first. hit eject button on CD to insert Debian CD. Zip.
10:15am: remove suspect HD. more cable jiggling, apply paperclip to eject CD drawer, insert CD, close draw. Boot. Nada.
10:20am: retrieve box from car containing nice Yamaha CD-RW from
bedlamhouse.10:22am: realise nice CD-RW is SCSI, and about as much immediate use as a chocolate fireguard. Gird loins and head outside again
10.30am: finish reviewing pile of mostly dead PCs in office - reject DVD-ROM in one as being known flaky, reject pile of strange CD drives in cupboard (including two external ones and an original Creative SB 8x CD, whose interface I do not wish to contemplate). Find Acer 50x drive in fourth PC down pile, remove.
10:35am: swap mounting rails over from old CD to newer Acer drive.
10:40am: make tea, start writing this LJ entry.
10:50am: fit Acer drive in server. Pause. Find paperclip. Remove Debian CD from old drive using paperclip.
10:55am: apply power. CD drawer opens, disk spins.
11:00am: remove power. On a whim, reattach suspect hard drive. Reboot with Debian CD in drive. BINGO! Both HD's and CD found, machine starts to boot installer.
11:05am: start fresh Debian install on new HD. It might not need it now, but I'm in the mood to be paranoid: that old HD was giving some very very odd errors.
11:09am: admire shininess of new Debian installer
11;10am; CD fails to read network drivers. Swear, reboot.
11:15am: fine second time round, partition new disk, leave machine badblock scanning, have early lunch while admiring shininess of new Debian installer some more
11:30am: return to machine to find various errors, reboot, try again. Reboot, try yet again with old HD disconnected. Still craps out. Peer, as instructed, in /target/var/log/debbootstrap,log, to find the very helpful 'Segmentation Fault'.
12:00noon: spent half an hour considering alternatives. Begin to suspect old HD is in fact fine, and can be plonked in different machine pretty safely. Temporarily cannibalise old Linux workstation.,...
12:15pm: Old HD fsck's fine, after a tantrum or two, in old workstation. Take deep breath, let it continue booting.
12:20pm: Well, it makes it to a root prompt, and MailScanner seems to be clearing the mail queue.
12:21pm: break for delivering James to doctors, via Anne, and a walk in the park with
bardling2:00pm: It's still up, and still behaving. Make tea.
