Today two amazing things happened and I would like to share them with you in a two-part series (woah, look at me going all high-tech on my reader). This is the non-nerdy story, the one everyone can read.

Here’s the second part of this series:

At the BP

My driving experience almost entirely consisted today of my car beeping at me when I went up or down a hill. I was running out of petrol and I wasn’t surprised since I did drive all the way to Byron Bay and back on the weekend. 560 on the clock, I knew the petrol tank’s life wasn’t to be much longer. After carraday (where we go to Carraway Pier and enjoy delicious burgers + chips), I went to the BP on Milton Road, around the corner from where I live. Today’s the last day for Queenslanders to fill up before the government takes off the subsidy and petrol stations start gouging (read: imitating South Australia’s) prices. So of course the line’s long. I fill up, and go inside, get money out and wait in the line of 7 or so people in order to get served by the lone clerk.

Then she walked in front of me.

This heavy-set woman who looked like she did weight lifting with semi-trailers walked past, about the same height as me but really, really built. She was wearing a leopard-print shirt, skirt and fish-net stockings. Then she walks over to the door and enters the security code, up to the counter and starts serving people.

Then I realise the chiselled chin, the bulge in the throat. She was once (or still is) a man. “Next, thanks!”, she says in a deep baritone voice, unmistakably male. I was shocked at first, it’s not something that you see every day.

Now step back 10, 20 or 30 years. This person would’ve been beaten or worse for doing what they’re doing now. If they were alive, they probably wouldn’t be employed.

You’re thinking right now that I’m old-fashioned. That I think that society has slipped. That I believe this person should be beaten and unemployed. That’s where you’re wrong. Very, very, wrong.

I got served at the counter by this person, who commented on my shirt, “When are you going to finish that?” because I was wearing my “Top 10 reasons I procrastinate [line break] 1.” shirt and asked if I would like both my tax receipt and change. This got a laugh out of me and my initial shock disappeared, I realised this person is too a human and why should it matter to me what they believe in or how they want to dress? These people are confident in what they think about themselves and they go out without caring what the outside world thinks of them. Good on them for doing what they think is right, and damning the rest of the world’s opinion. I applaud them.

It was also the most friendly service I ever received from a petrol station attendant.

Today two amazing things happened and I would like to share them with you in a two-part series (woah, look at me going all high-tech on my reader). Here’s the first:

Hampton Catlin

Today Hampton Catlin was talking with Dr Nic about Ruby 1.9 issues he was having with his wikimedia-mobile project, specifically he was getting incompatible character encodings: ASCII-8BIT and UTF-8. This is a guy I admire and look up to and think he’s “the shit”. He came to my company looking for help and it was my (and Bo’s) task to help him figure out what’s going on. Honoured.

The search box on his site was a bit wrong for fanciful languages like that German:

and some pages threw some more interesting errors:

I’d seen this error before in my Ruby 1.9 testing, but that was so long ago that I had forgotten what context or even if I fixed it. Probably not.

I remembered someone linking to this post by Dave Thomas a while ago but forgot the link, but thanks to Google I was able to enter “Ruby 1.9 encodings” and it knew exactly what I was after. I followed the “instructions” and put # encoding: utf-8 at the top of the merb executable and the buffer.rb file in HAML (which, it turns out had no bearing on the final result). No luck. Then Hampton mentioned he put -KU on the end of the ruby interpreter which randomly fixed/broke random things. So I tried that, and got a couple of degrees of success.

I opened up irb1.9 -KU (yeah, I’m so cool I have two versions of Ruby installed, at the same time) and I knew of the encoding method you could call on a string in order to get the encoding of that string. So I tried something simple: “Ryan”.encoding which gave UTF-8 so I tried the German text and I wasn’t surprised when that also returned UTF-8. So what’s going on?

Well, turns out that even though we specified # encoding: utf-8 in the merb executable and even in a meta tag in the HTML, the HAML that was getting sent to the parser was being sent in ASCII-8BIT! Around this point Bo came in and we discovered the lovely force_encoding method for strings in order to… well, I’m sure you can figure it out.

This is the misbehaving line in haml 2.0.9 and to fix it we just do result.force_encoding(“UTF-8″) and that forces whatever’s being appended to the buffer to always be UTF-8!

Hampton was happy, we were happy, and karma rewarded me with a delicious steak sandwich + icecream with banana slices with maple syrup on top.

When I moved to Queensland I quickly realised that there’s no CommunityCPS branches up here in the sunshine state so it was suggested that I change to The Commonwealth Bank which I did and was painless. I changed every single point of debiting from the account I could find except for iPower (my hosting company, my mistake) and Allianz.

Fucking Allianz.

So I call them up about a month or so ago and say “Hey guys, about my insurance with you… I want to change the bank account it’s debited out of it.” The guy on the other end of the phone sounds like a McLackey, but my faith in large-corporation call-centers is about as existent as God, so I’m unphased by this. What I was phased by however he said:

We do not have an account with that name, or that license plate in our systems..

So I give him every single shred of information I can think of that would link to me, my old addresses, phone numbers, everything. According to him I didn’t exist. So who were the $109.31 monthly debits going to?

Fucking Allianz. That’s who. Said so on the bank statement.

So I get a little pissy with the guy and say “Righto, I’ll just close the account and you can have your people call my people and we’ll change the details then?” The guy’s shocked, says “Alri..” and I hang up.

Today

Today I get a call from my old bank and they say there’s a negative amount on my debit account which should never happen. I’m told that if I deposit the $109.31 into the account then they can close that for me, which is what I should’ve done a while ago. They’ll send me out paperwork too (yet another company who fails to realise the year is 2009), but what can you do? I guess I could plant a tree or something.

Then I get a call back from them. They ask if I want to cancel my car insurance along with my account. I go “lolwut” or something to that effect, and the lady explains that I have insurance being debited out of my old account and that they “honor” Allianz which is why I had a negative amount. We then go through the process of changing my payment details to my Commonwealth account and my address to my new place, then all is well.

Just annoyed that Allianz has no record of CommunityCPS’s account with me, and also at myself for not remembering I had insurance with my bank, not with Allianz itself.

Today I spent an hour and a half writing a Sinatra app (and rocking out to Foo Fighters) for a problem that’s been bugging me personally and I’m sure many of you for a very long time. In the past, there hasn’t been a comprehensive list of TextMate 2 features. The remaining features to be added to this release are hidden from the general public which causes a lot of speculation on exactly WHAT will be in TextMate 2.

But fear not dear TextMate lover/fanatic/fetishist! There is a website for you to view the coming features in TextMate 2 and even suggest your own! This site is http://fixedbytm2.com and, as you can now probably see, shows you, in bite-sized pieces so you’re not too overwhelmed exactly what features and bugs are going to be fixed in TextMate 2!

The code’s available on github, naturally.

I went in to work today to pick up some things and on the way there the fuel light came on so I thought I would fill up on the way back at one of the two servos on the way. I go in to work, grab the stuff and go to the Foodworks down the road to get money out of the ATM there. It tells me to stick my card in and then take it out immediately after.

I do this. Again. And Again. And Again.

After about the 10th try I get frustrated, grab some snack food and head to the counter with the purpose of getting out my money. I ask the cashier for $200 and she says “Um… I could probably only give you about $20″, which annoys me a little but I know the servo right across the road has an ATM in it that I could try. I walk back to my car so I can drive it across the road to the servo, fill up on fuel and cash. Great idea! So I cross at the traffic lights but the servo is on the other side of that road, and in between me an it is a very long line of cars looking to go the other way.

Scratch that servo.

So I know there’s a Shell just before I get on the Western Freeway that I can go to if I take the scenic route. To me, it’s always seemed like a large well-”endowed” servo with everything you want, including an ATM. I get there and fill up on Petrol and I look inside the shop and see they don’t have the mystical ATM that I so dearly crave. No worries, when I pay for petrol I’ll just ask for the money out then. So I did. Cashier could only give me $50, and I take it.

But I still need a bit more to get me through the week.

Not a problem! There’s an ATM near my local takeaway joints that I know will give me the money I need! I go there and park in the final spot remaining in the parking lot and basically get out of my car and in front of the ATM in one smooth movement. I insert my card.

It feels like something’s stopping the card from going in.

I take it out and try it again.

Only then do I look at the screen to realise that the ATM is switched off.

I drive the remaining 500m home, dejected.

I think this is a sign the world wants me to get out and exercise and it’s showing me this by just about forcing me (indirectly) to walk to the ATM at the Commonwealth Bank on Park Road, just up the road from where I live. So off I go then.

Today at work our iPhone developer asked:

“How many requests per second do you think the server could handle?”

This was for a given action, getting the scores of some games for a given sport in a given league in the current week of the current season. We’ll call it “scores”. This action sends back JSON data containing information about the games. The data is populated from an external source via an endlessly looping rake task that sleeps for a couple of seconds before re-querying the data source.

So I set about finding how many requests it did at the time.

Enter, the Bench

To do this I used something that comes with Apache called Apache Bench (ab on the command-line). This lets you bench how well a webserver responds to a number of requests and you can set, among other things, the concurrency of these requests.

So I did ab -n 100 http://127.0.0.1:3000/sports/sport_name/leagues/league_name/games/scores and got in the results this like:

Requests per second:    5.52 [#/sec] (mean)

Only 5 and a half requests a second! That’s nothing! So what’d I do?

Queries

First stop was queries. We retrieved records from one model, and then based on that find we retrieved records from another model and based on that find more records, and even on that find more records!

We used something like this example:

@sport = Sport.find(params[:sport_id])
@league = @sport.leagues.find(params[:league_id])
@season = @league.current_season
@week = @season.current_week
@competitions = @week.competitions 

In our app, sports are always going to have leagues and leagues are going to have seasons which are going to have weeks and weeks are going to have games, so we can use a joins option in our initial sport find in order to get it to cut down on the number of queries.

@sport = Sport.find(params[:sport_id], :joins => { :leagues => { :seasons => { :weeks => :competitions } } })
@league = @sport.leagues.find(params[:league_id])
@season = @league.current_season
@week = @season.current_week
@competitions = @week.competitions 

At this stage we see a triple-fold increase plus a little bit more with our app now doing 17 requests per second! Woah!

Requests per second:    17.43 [#/sec] (mean)

Caching

Since our action is most likely going to be hammered by a lot of users frantically refreshing to see the latest scores, caching is definitely the way to go here. Caching will store the page on disk and serve that file rather than going through the Rails stack which is, as we’ve seen, only serving 17 requests a second. How do we cache? Well, for development mode we’re going to have to turn on caching in development.rb:

config.action_controller.perform_caching = true

And we’re going to have to put in our controller a method call to tell Rails to cache the page:

caches_page :scores

After this we’ll restart the Rails server (Mongrel) and access our scores action. For the first request we’ll see in our log/development.log that it says: Cached page: /sports/sport_name/leagues/league_name/games/scores.json (1.6ms) indicating that this page has been cached. Future requests to this page will not be logged in development.log because Rails is serving the file sports/sportname/leagues/leaguename/games/score.json directly from the public folder.

Running Apache Bench again and we get:

Requests per second: 921.93 [#/sec] (mean)

That’s a 54 times increase! Now we’re talking! So if we have a single mongrel serving 921, almost 922, requests a second, surely we can only go so much faster, right?

Enter, the Passenger

So I hooked up my app to run on Passenger using the ever-awesome Passenger Preference Pane which makes it a cinch to get your app running on passenger on your dev machine.

So now we’ll change our ab call to point to our passenger app: ab -n 1000 http://games.local/sports/sport_name/leagues/league_name/games/scores. I snuck in the number 1000 just to really emphasise what you’re going to see next:

Requests per second: 3184.45 [#/sec] (mean)

This 578 times faster than our initial single-mongrel server!

What’s also interesting in this output is right at the bottom:

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      3
  80%      4
  90%      5
  95%      5
  98%      7
  99%      8
 100%     24 (longest request)

99% of our requests were served in less than 10 milliseconds and the longest only took just over double that. And this is on a Macbook Pro! Imagine what kind of speeds you can get on a super-server!

Nginx

So I installed nginx using passenger-install-nginx-module which’ll install nginx too if you don’t already have which is handy! It inserted some default config and I set it up to listen on port 81 so it wouldn’t conflict with Apache and let her rip. Running the same apache bench (with a different port for nginx) I get:

Requests per second: 6461.48 [#/sec] (mean)

This is a massive twice as fast improvement over Apache. Word on the street is that it uses less RAM too! That’s 1,235 times faster than what our original app was serving at.

• Note: YMMV. Also note that these tests are done locally. Latency will play a large part in making your remote connections slower, but overall your app will be faster by using proper queries and caching.

Sometimes you want to see a query (or queries) that were executed by a model call that you just did. To do this you have to either set up some hax in your irb configuration or you could now use SQL Display.

SQL Display’s been an idea that has been thrown around in the Mocra offices for a while and after Bo Jeanes’ and Ben Hoskings’ conversation on twitter last night about the syntax I decided to try my hand at it. The code itself is… not pretty. I warn you of this now before you go trumping through the source code and discover something analogous to a large heap of elephant dung; but it’s functional!

One of the fun things about implementing this plugin was how ActiveRecord implements its logger. When it initialises a new connection it passes the logger object at the time to the connection adapter object so it’s set there For All Time.

SQL Display works by storing what the old logger was, setting the new logger to a file called tmp/sql_display.log, re-establishing the connection, running your query, removing all the colour & extraneous crap from the logs and stores it, removes the log, sets the logger back and again re-establishes the connection.

Patches welcome.

I submitted a ticket for Cucumber, and I hope that it gets accepted:

I encountered a problem when I was using cucumber today. I changed a small detail in one of my Rails controllers and when I ran the features many of them turned red! Disaster! So I scrolled back up, found one that was failing and copy and pasted the file:line syntax into the cucumber command and made them all work! What I didn’t like about this process was that I had to scroll back up through a mixed mountain of red and green features. I only wanted to know which of my features were failing. Trudging through this mess, I thought “If only cucumber showed me the failing features in the summary, along with all the other useful information it displays there” and then I remembered “Cucumber is an open-source project, I could fork it and add it during my spare time!”. So I did. My commit that adds this functionality is added here: http://github.com/radar/cucumber… for all to gaze upon, criticise and ideally improve upon. At the moment it does everything but exceptions that are raised during Scenarios, something I intend to look on later this evening or perhaps one of you lovely people could do that. I’d really love to see this merged into the core since I believe (along with my coworkers and friends) this is “extremely fucking useful”. Thanks for reading!

And now the ticket’s been accepted and applied! Thanks to Aslak for this!

When you go to sign up to Basecamp it displays a form where you can enter your details, but only if you have Javascript turned on. If you have Javascript turned off, then you’re presented with a message telling you to turn on Javascript.

I made my own little gist showing what can be done with the noscript tag here and it’s just a minimalist version of what the Basecamp guys have done.

Unfortunately Gist is still a little feature-bare so I had to make 3 gists for the files, even though you can put multiple files in the same gist.

index.html

noscript.css

style.css

Ironically, you’re not able to view the files above with Javascript turned off.

As a twenty-something man, I enjoy the brief interludes between the days that I work. Many of you will refer to them as “weekends”, I refer to them as “me time”. During my “me time” I enjoy sleeping in. Occasionally however this is cut tragically short by the people that I live with bickering about certain things like beds not being made or, more often, internet usage.

A certain teenager who shall not be named was kicked off the computer earlier this evening and went into his room to do whatever it is that teenagers do without the internet. After his darling mother went to bed, he went out of his room and as I discovered when I went to get a drink, went straight on the computer. Now if darling mother comes out and finds darling child on the computer, Hell shall break loose and God help us all.

This is where I profess my love for Billion. Specifically their BiPAC 7401VGPR3 model which allows me to, among other things, implement a firewall that I can tell to filter everything between certain times. These certain times just so happen to be midnight (27 minutes away) until 7am. And I don’t even need to RTFM!

So, ideally tomorrow morning I’ll be getting up at the lovely beautiful time of 10:30 and darling mother and darling child will not have their early morning bickering fight. Everyone’s happy.