Various recovery ideas just aren’t working – there’s no way I can even see data on the /usr partition, let alone figure out where these legendary superblocks are. /usr, by the way, includes all the user accounts, websites, mail and so on. So bad news. And no backups.
Next task: Get a new drive in the server, set it up for what I can remember, and start sending and receiving email again. I’ll try and do that today.
After that: The data recovery company I’ve had recommended is closed until the 5th, so we won’t know till after then what’s been lost. Conversation indicates this might be a hardware failure - some problem with the electronics on the drive itself, or something - so hopefully all the data is intact. And I’m a bit scared to take the drive apart and move the platters into a new case myself.
After that: Get some backup system working (shutting the stable door after etc, but I’d be even stupider than I’ve already been to leave open the chance of this happening again); Move across the old data (if there is any).
13.48
So I’m on the datafloor. I’ve been on TCR, bought a new hard drive (80Gb) and a screwdriver. I’ve made the install floppies for FreeBSD – floppies which, it seems, are getting less and less reliable over the years. So they’ve been tested too, but only once, and not all the way through.
The moment of truth. Hook up the monitor and keyboard, put the kernel floppy in, and turn it on.
“Run” by Air is playing on my iPod.
13.53
kernel is loading… successful. Now it wants disk 2, mfsroot. This is one I haven’t checked all the way through.
This is my favourite aspect of FreeBSD. That you can give it two floppies, a few configuration parameters, then it pulls 2Gb of data off the net and installs itself. Wonderful. The tension added by potentially corrupt shit floppies only adds to the excitement. If it works.
14.07
Okay, start again. What ethernet card do I have? Does it matter? What do all these user options mean? Where’s my manual?
14.16
Subnet wrong, start again.
It’s probably worth saying how I got into this server nonsense. I wrote a rather database-intensive web toy that was popular for a while. After a year or so it got spidered rather heavily, battered my host’s box, and I got kicked off.
Well fortunately I had a test server in my room I’d been learning on because I’d talked my way into doing a website for college, knowing nothing about any of this. The server was a second-hand hard-used machine, and it got pulled into service hosting my site under a desk at a friend’s company.
That machine lasted a year before it started dying on a daily basis, but in that time I learned a little about updating a few things, running virtual hosts and email for friends (using linuxconf). I didn’t really know what I was doing.
But then it was time for a new server. I asked some friends, bought some kit, put it together and installed FreeBSD (because that was what we used at work).
That was two years ago. In that time I’ve used having the server to teach myself more, locked myself out of it once (I locked myself out of the previous one many more times than that) and had to come down to deal with it.
That was the time I got a key fob to the datacentre, historicalfact having graduated from a cupboard that stank of mouse shit to a rack in an office to a rack in the Docklands.
And that’s how I managed to get down here today, know how to come in and attempt to diagnose the problem, bring the drive home, buy one, fit it (back on the datafloor) and start installing the OS. Without really knowing what I’m doing.
Bollocks. And because I’m not paying attention I’ve managed to start installing the box without any software.
Oh, or maybe not. It seems to be working. I wish I had more luck with drives however.
14.58
Right, various packages installing now. The plan is to get it into a state where I can install other stuff by ssh from somewhere warming than this, get mail/www/dns working, test reboot the box, then go home. But first these essentials have to install. What was I saying?
Oh yes, that I had a server in my room and that was how I started learning.
The webapp got written, btw, and I got to learn more about running Perl on Windows talking to an Access db (and the oddities of such a db) than I ever wanted to know. And it works, and as far as I know, still works.
That was when I decided to look for work in the internet when I finished college - the dot com days still looked rosy - or rather, that was when I started a weblog and got talking to people who assured me such a thing was possible, and due in no small part to the fact I knew a little about lots of things, I got a job and there we go. It was a small company too, so I got to do a lot of little things and ended up learning even more about servers, databases, programming and so on.
It’s friendly sysadmins, experimenting and losing data all the way along.
But programming. Before I went to college I was interested in computers, sure, but only as far as they could help me make magazines and I could write stories on them (I love print design and typography; DTP was fantastic). I used BBSs and played on the internet once or twice enough to know I wanted in-room access to my Mac when I started at college, but nothing more than that.
Firstly I learned HTML at the end of my first year. Even that wouldn’t have happened except that I got a reputation as a bit of a geek (printing out unusual stories from websites and pinning them to my door every day can’t have helped) and my neighbour, a good friend, needed a website.
I doubt I would’ve done it except my exams finished a week earlier than everyone else I knew and - after beaming sunshine for months - it rained for that whole week, solid. So I made a website.
The next year we had to take a programming class. And I swear I’d tried programming between being 14 (when I’d last used BBC Basic) and 1997/8 (that year) and not got on with it. But the class used Pascal, and I don’t know, something about it was just what I was looking for. I didn’t really understand what I was doing to begin with, but the concept of dataflow, loops, abstraction and so on started dropping into place.
Backtrack a little to the project before that class.
Ah, and I’ll have to pause a second because I need to set up the users.
(“Dirty Dream Number Two” by Belle & Sebastian is playing on my iPod.)
15.21
And… I’m ssh’d in to historicalfact from the rest area on over the wireless. Work to do, I’ll finish the story in a bit.
15:30
Seeing as it’s downloading software, it might also be worth saying why I had that server in my room at college, having before that being a Mac user and steering well clear of anything command-line shaped.
We used to have dinner every night in Hall. Cheap food, not too bad. One year the college authorities decided to make us sign up before Hall so they knew how much food to make, otherwise the wastage would push the prices us.
Cue: Uproar.
Cue: Outrage.
The place to sign up, see, was in the Lodge, the reception to the college. Whereas Hall was 20 or 30 yards from the student rooms, the Lodge was, oh, about 100 yards. Being lazy students, this was too much. (I’m sure there were other reasons too, although I can’t remember them. Something like: how can we possibly be expected to know where we’re going to have dinner 20 minutes before dinnertime? Because, of course, being students, our lives were busy with urgent appointments, last-minute uninterruptable engagements and cetera.)
Being an aforementioned lazy student I also thought it was a silly idea. Being also a student with an internet connection in my room (we all did; the college computer rooms were filled with abused old PCs that would crash regularly. I appropriated a couple of these later for the servers I’ve mentioned) I said that we should have a webapp to sign up, because that’d be much easier. For “I said” read “I mouthed off loudly about how rubbish paper was and how great and easy webapps would be in the computer room, in public”.
Pause for the vacation.
The first I knew about what had happened was when a friend of mine, also a student but much better connected, congratulated me for offering to help out with the Hall problem, and thanks very much.
So I went to see the sysadmin, who was also a friend, and it turned out that I’d been heard talking about webapps and somebody had told someone else who’d told someone else who’d understood it as an offer, and accepted, without talking to me, and told everyone I was doing it.
I couldn’t rightly say no, at this point. Ah, and I remember now. I’d already got the server with Linux on it in my room because I needed to run Perl for my 4th-year project. But that was it. I’d already had the 30 minute “please close your open mail relay that Red Hat installed otherwise we’ll shut you down” security talk (our college sysadmin was really helpful and sorted me out with all of this. Great bloke).
The server became a proper server, with Apache and MySQL and so on, and I learned about RPMs and compiling and so on (although I’d been practicing on my mum’s computer over the previous summer).
That took a lot longer than I thought.
qmail with auth and tls is now back up for historicalfact, and some of the users are accepting mail (I’ve only put back the users who have accounts on the historicalfact domain). courier imap (with ssl) is also up, but untested: when people sync their accounts over imap they’re going to lose all their current email. This is bad. I’ll have to warn them all in advance of giving them their passwords back.
Right, back home now since it looks like the box will come back up if I reboot it. Tonight: more domains and a little www action I think. And food.
Various recovery ideas just aren’t working – there’s no way I can even see data on the /usr partition, let alone figure out where these legendary superblocks are. /usr, by the way, includes all the user accounts, websites, mail and so on. So bad news. And no backups.
Next task: Get a new drive in the server, set it up for what I can remember, and start sending and receiving email again. I’ll try and do that today.
After that: The data recovery company I’ve had recommended is closed until the 5th, so we won’t know till after then what’s been lost. Conversation indicates this might be a hardware failure - some problem with the electronics on the drive itself, or something - so hopefully all the data is intact. And I’m a bit scared to take the drive apart and move the platters into a new case myself.
After that: Get some backup system working (shutting the stable door after etc, but I’d be even stupider than I’ve already been to leave open the chance of this happening again); Move across the old data (if there is any).
13.48
So I’m on the datafloor. I’ve been on TCR, bought a new hard drive (80Gb) and a screwdriver. I’ve made the install floppies for FreeBSD – floppies which, it seems, are getting less and less reliable over the years. So they’ve been tested too, but only once, and not all the way through.
The moment of truth. Hook up the monitor and keyboard, put the kernel floppy in, and turn it on.
“Run” by Air is playing on my iPod.
13.53
kernel is loading… successful. Now it wants disk 2, mfsroot. This is one I haven’t checked all the way through.
This is my favourite aspect of FreeBSD. That you can give it two floppies, a few configuration parameters, then it pulls 2Gb of data off the net and installs itself. Wonderful. The tension added by potentially corrupt shit floppies only adds to the excitement. If it works.
14.07
Okay, start again. What ethernet card do I have? Does it matter? What do all these user options mean? Where’s my manual?
14.16
Subnet wrong, start again.
It’s probably worth saying how I got into this server nonsense. I wrote a rather database-intensive web toy that was popular for a while. After a year or so it got spidered rather heavily, battered my host’s box, and I got kicked off.
Well fortunately I had a test server in my room I’d been learning on because I’d talked my way into doing a website for college, knowing nothing about any of this. The server was a second-hand hard-used machine, and it got pulled into service hosting my site under a desk at a friend’s company.
That machine lasted a year before it started dying on a daily basis, but in that time I learned a little about updating a few things, running virtual hosts and email for friends (using linuxconf). I didn’t really know what I was doing.
But then it was time for a new server. I asked some friends, bought some kit, put it together and installed FreeBSD (because that was what we used at work).
That was two years ago. In that time I’ve used having the server to teach myself more, locked myself out of it once (I locked myself out of the previous one many more times than that) and had to come down to deal with it.
That was the time I got a key fob to the datacentre, historicalfact having graduated from a cupboard that stank of mouse shit to a rack in an office to a rack in the Docklands.
And that’s how I managed to get down here today, know how to come in and attempt to diagnose the problem, bring the drive home, buy one, fit it (back on the datafloor) and start installing the OS. Without really knowing what I’m doing.
Bollocks. And because I’m not paying attention I’ve managed to start installing the box without any software.
Oh, or maybe not. It seems to be working. I wish I had more luck with drives however.
14.58
Right, various packages installing now. The plan is to get it into a state where I can install other stuff by ssh from somewhere warming than this, get mail/www/dns working, test reboot the box, then go home. But first these essentials have to install. What was I saying?
Oh yes, that I had a server in my room and that was how I started learning.
The webapp got written, btw, and I got to learn more about running Perl on Windows talking to an Access db (and the oddities of such a db) than I ever wanted to know. And it works, and as far as I know, still works.
That was when I decided to look for work in the internet when I finished college - the dot com days still looked rosy - or rather, that was when I started a weblog and got talking to people who assured me such a thing was possible, and due in no small part to the fact I knew a little about lots of things, I got a job and there we go. It was a small company too, so I got to do a lot of little things and ended up learning even more about servers, databases, programming and so on.
It’s friendly sysadmins, experimenting and losing data all the way along.
But programming. Before I went to college I was interested in computers, sure, but only as far as they could help me make magazines and I could write stories on them (I love print design and typography; DTP was fantastic). I used BBSs and played on the internet once or twice enough to know I wanted in-room access to my Mac when I started at college, but nothing more than that.
Firstly I learned HTML at the end of my first year. Even that wouldn’t have happened except that I got a reputation as a bit of a geek (printing out unusual stories from websites and pinning them to my door every day can’t have helped) and my neighbour, a good friend, needed a website.
I doubt I would’ve done it except my exams finished a week earlier than everyone else I knew and - after beaming sunshine for months - it rained for that whole week, solid. So I made a website.
The next year we had to take a programming class. And I swear I’d tried programming between being 14 (when I’d last used BBC Basic) and 1997/8 (that year) and not got on with it. But the class used Pascal, and I don’t know, something about it was just what I was looking for. I didn’t really understand what I was doing to begin with, but the concept of dataflow, loops, abstraction and so on started dropping into place.
Backtrack a little to the project before that class.
Ah, and I’ll have to pause a second because I need to set up the users.
(“Dirty Dream Number Two” by Belle & Sebastian is playing on my iPod.)
15.21
And… I’m ssh’d in to historicalfact from the rest area on over the wireless. Work to do, I’ll finish the story in a bit.
15:30
Seeing as it’s downloading software, it might also be worth saying why I had that server in my room at college, having before that being a Mac user and steering well clear of anything command-line shaped.
We used to have dinner every night in Hall. Cheap food, not too bad. One year the college authorities decided to make us sign up before Hall so they knew how much food to make, otherwise the wastage would push the prices us.
Cue: Uproar.
Cue: Outrage.
The place to sign up, see, was in the Lodge, the reception to the college. Whereas Hall was 20 or 30 yards from the student rooms, the Lodge was, oh, about 100 yards. Being lazy students, this was too much. (I’m sure there were other reasons too, although I can’t remember them. Something like: how can we possibly be expected to know where we’re going to have dinner 20 minutes before dinnertime? Because, of course, being students, our lives were busy with urgent appointments, last-minute uninterruptable engagements and cetera.)
Being an aforementioned lazy student I also thought it was a silly idea. Being also a student with an internet connection in my room (we all did; the college computer rooms were filled with abused old PCs that would crash regularly. I appropriated a couple of these later for the servers I’ve mentioned) I said that we should have a webapp to sign up, because that’d be much easier. For “I said” read “I mouthed off loudly about how rubbish paper was and how great and easy webapps would be in the computer room, in public”.
Pause for the vacation.
The first I knew about what had happened was when a friend of mine, also a student but much better connected, congratulated me for offering to help out with the Hall problem, and thanks very much.
So I went to see the sysadmin, who was also a friend, and it turned out that I’d been heard talking about webapps and somebody had told someone else who’d told someone else who’d understood it as an offer, and accepted, without talking to me, and told everyone I was doing it.
I couldn’t rightly say no, at this point. Ah, and I remember now. I’d already got the server with Linux on it in my room because I needed to run Perl for my 4th-year project. But that was it. I’d already had the 30 minute “please close your open mail relay that Red Hat installed otherwise we’ll shut you down” security talk (our college sysadmin was really helpful and sorted me out with all of this. Great bloke).
The server became a proper server, with Apache and MySQL and so on, and I learned about RPMs and compiling and so on (although I’d been practicing on my mum’s computer over the previous summer).
That took a lot longer than I thought.
qmail with auth and tls is now back up for historicalfact, and some of the users are accepting mail (I’ve only put back the users who have accounts on the historicalfact domain). courier imap (with ssl) is also up, but untested: when people sync their accounts over imap they’re going to lose all their current email. This is bad. I’ll have to warn them all in advance of giving them their passwords back.
Right, back home now since it looks like the box will come back up if I reboot it. Tonight: more domains and a little www action I think. And food.