November 16th, 2012 ravi
So… yeah. About the service the last year. Let’s just say my professional life got really, really, REALLY busy which started to compete with my personal life. The professional life involved a substantial amount of travel which unfortunately never really overlapped with my (now) fiancée’s. Of my personal life I have the reduced time with my fiancée and my personal geeky projects. One of those projects is a shell server which I’ve been running since 1996. Thankfully it’s not running on 1996 hardware or OS, but in the grand scheme of things it may as well have been.
The server is some Compaq running FreeBSD 6.3. The kernel is old. Very old. So much that I can’t build many new and shiny things like a modern enough Perl to do O-Auth with a module among other things. Now the logical thing would be to upgrade, but the server is 2000 miles away and doing the upgrade to an 8 kernel meant going to 7 first. Not the most trivial of tasks nor timely. So I decided to call a mulligan and start from scratch.
The new server is a HP DL380 with a bunch of fast disk and an external JBOD array with some SATA disk. I got FreeBSD 9 going and went to town migrating services to new and shiny homes. Then I ran into a challenge with how to move mail. The issue is, in part, that shell users got local mail in a mbox format. Virtual users are in maildir format. In the new world mail won’t live where the shell users are. Anyhow, long story short the migration stalled. I got everything but the mail moved over which means this service became split. The web bits live in the happy new home while the actual functional bits that take in email and make the posts is on the old dilapidated server.
Then I lost a drive on the internal array (RAID5).
Then I lost a drive on the external array (zfs).
So here I was mid migration, up to my eyeballs with work, and the amount of time for my personal life shrinking. I had to make an unfortunate compromise of letting the service coast as-is and hope I’d be able to get my life back in order.
So here I am in a flat in Austin, TX after day 1 of the Circuit Of The Americas F1 US Grand Prix. My flight out here I had the foresight to dump the mail spools for the Caltrain account and then wrote up some perl to parse through the message headers and generate a CSV file to import into the DB. About an hour ago I completed that import, and about an hour before that I finally figured out what was wrong with the web site. Stupid AllowOverrides.
In case these details make you more frustrated because they’re admittedly not difficult sysadmin problems to fix consider this: I’m not a sysadmin. I’m a network engineer who plays a sysadmin on TV and I don’t get paid to do it. It’s all out of pocket expenses to maintain things.
Bottom line, however, is I failed the community, the service, and myself by letting things go as long as I did. While the system has been functional in this maintenance-mode period, many people have been (im)patiently waiting for their key to contribute to the system. All requests that I was able to identify have been generated and are in the DB. All that is left is to send the mass email with the keys. That is something I will do in the next day or so. My goal is for the 92 people to have their key for Monday’s commute.
Again, my sincerest apologies.
March 10th, 2012 ravi
I’m writing up a more in-depth post about this on my personal blog, but I wanted to give a quick summary of why keys have not been generated and mailed to you.
I run a very, very old version of FreeBSD that is not capable of running a modern compiler. This, in addition to other things, means I’m not capable of compiling the tools that are needed to support the users on my host or this service as a whole.
I currently have a bunch of applications running in memory right now that I cannot start again because I can’t compile a modern version of it.
I’ve overcome the dependencies for this service specifically and with the help of a friend we’ll be scraping through all the key requests and shipping them out to you ASAP.
March 10th, 2012 ravi
I wanted to mention that I will be on a SXSW 2012 panel, Sexy Data Solutions for Public Transit Systems, which will be at the AT&T Conference Hotel.
If you’re at SXSW it would be great if you could make it. Hop on the shuttle of take a stroll. The weather should be well cleared up by then!
July 19th, 2011 ravi
I haven’t updated the blog in about 6 months mostly because the service is fairly self sufficient. Unfortunately I’m very behind on generating update keys and I know a lot of you have mailed requesting them more than once. I haven’t forgotten about the requests so much as I got in the weeds with respect to free time.
I’ll try my best to get these kicked out soon so more can help make the service better.
January 29th, 2011 ravi
John Murphy (@murphstahoe) asked for some stats of this service. I’ve been meaning to write something that sifts through the data and I figured others may be interested.
I would have dropped this behind the new WordPress <more> tag, but this theme doesn’t support it. Oh well.
Check out the compiled data.
September 1st, 2010 ravi
Community! Lindsay Schauer of Twitter write a great blog post about this service for the new Twitter Tales site.
Also as a followup to an editorial update Mark Otto succinctly rephrased my response with “We don’t follow you for personal opinions; we follow you as a community wanting to know how we’re getting to/from work each day.” Well said, Mark!
The service is also approaching the 3 year anniversary and there are 4,490 @caltrain follows, 465 @bikecar, and 534 email keys have bee issued. Quite an accomplishment, I think, for a small grassroots project.
Thank you to everyone who has continued to help make this a valuable service to the community.
May 11th, 2010 ravi
The new year brought with it a new job, new apartment, and new car. As a result of the former 2 I was completely in the weeds with any projects like this service. Thankfully it runs on its own however being in the weeds meant nearly 100 key requests sat unanswered. Part of the problem was how I input the data to the DB. It was anything but graceful.
A friend of mine has been coaching me to learn Ruby for some very specific reasons (so please don’t tell me I should learn some other language). I took the opportunity to write a script that will speed up the data entry and parsing. After actually successfully doing that I was able to plow through the pending requests.
The new car means I haven’t been taking Caltrain for the last several months. This is in part because I’m no longer within a reasonable biking distance to 4th/King and I would have to either bike from Mountain View or take VTA to my new office. I have a rule that excluding walking I will not take more than 3 modes of transportation to work. That said I need to really need to figure out timing to drive to Millbrae (with or without my bike) and take Caltrain to Mountain View where I either bike or take VTA. but of course Caltrain has been about as reliable as MUNI so I’ve scrapped the whole idea of taking Caltrain all together for the time being. I’d really like to use it again as I’d be gaining at least 1h30m of work time.
Anyhow things are back on track and its a good thing too because Twitter is discontinuing support for basic authentication. That means unless I get moving it will outright break. I better get cracking!
Thanks again to everyone who continues to make the service the success that it is.
March 2nd, 2010 ravi
It is March already and I am 2 months behind in generating update keys for people. I am very sorry for not getting to it, but I started a new job in early January which has been consuming 97.45% of my time and then in February I thought it would be a great idea to move. Now that I’ve moved in I will spend a little time going through all the emails and generating those keys.
Thank you for your patients.
December 8th, 2009 ravi
…public transit providers in the U.S. had a similar campaign.
Also I know I’be been slacking off quite a bit with issuing new post keys and I am sorry. I have no excuse other than being very lazy. Perhaps I will do them tomorrow.
October 22nd, 2009 ravi
Jennifer Van Grove wrote a piece about BART partnering up with a location-based social network site Foursquare. One person tweeted that Caltrain should also embrace the service and a few more shared the feeling. I can’t help but to wonder how this partnership will help Caltrain or Caltrain riders any more than the BART partnership does. Am I missing a bigger picture here? Can someone clue me in, please?