skip main content

Posts Tagged ‘Ruby’

Always adding http_proxy support when using Ruby’s Net:HTTP

posted by Duncan at 8:57 am on April 12th, 2011

Recently there have been a number of times I’ve needed to use Ruby’s Net:HTTP library. Because I develop inside a company firewall, all HTTP traffic has to go through a proxy. This means our libraries always need the Proxy support that Net:HTTP has. This means we now have a common idiom, and here it is:

require 'ostruct'
url = URI.parse("")
proxy = ENV['http_proxy'] ? URI.parse(ENV['http_proxy']) :
conn = Net::HTTP::Proxy(, proxy.port).new(, url.port)

Simple! You can use this every time, and your lib will have proxy support based on the http_proxy environment variable being set. If it’s not set, then it will just carry on as expected.

rQRCode 0.3.3 released. Bug Fixes

posted by Duncan at 6:37 pm on February 5th, 2011

Quick post to say I pushed a 0.3.3 release of rQRCode which fixes a few bugs that have been causing people pain. If you’ve never seen the library, you can check it out over at the site. Otherwise it’s just:

gem install rqrcode

BBC Gardeners’ Planner in your calendar part deux

posted by Duncan at 1:01 pm on October 20th, 2009

This is just a small extension to the original post: Subscribe to the BBC Gardeners’ Planner with the help of Sinatra.

I mentioned there that I needed to add some caching to the application, and also that it didn’t live anywhere. Well it now has caching (a feeds cached for a day) thanks to rack/cache and it now has a url:

Which you can now use in your iCal enabled calendar, one of which is Google Calendar, which is used in the photo below where I have subscriber to:

Google Calendar

I was having some strange problems the other day, where Google Calendar was showing the calendar titles as long email addresses. I think this may be a Google Calendar index thing (anyone know?), as it appears to fix itself over time so ignore it.

Subscribe to the BBC Gardeners’ Planner with the help of Sinatra

posted by Duncan at 9:45 am on October 15th, 2009

[UPDATE] I have posted an update with a url you can use

I really like being in my garden, so I was looking at the BBC Gardening site the other day for some gardening tips. The hardest part for me, is the lack of experience I have in knowing what needs to be done and when.

The BBC Gardening website solves this by having a gardeners’ planner. Great I thought, there’s loads of really useful stuff in there. The problem is I have to keep going back to the site, to check what I’m suppose to be doing. As it’s a planner, it would be great of I could have this information to put in my own calendar, so I could see it along with other jobs I have to do. Even better if I could subscribe to this information, so if it ever changes, I’m kept up-to-date. Maybe they will provide this one day (I really think they should), but in the mean time I decided to roll my own:

Subscribing to the BBC Garden planner

I’ve created a web app that reads this planner data, and converts it into the iCal format that you can now subscribe to. It’s written using Sinatra, and I choose to run it under the awesome Passenger via mod_passenger. It’s currently not hosted anywhere (other than on my laptop), so you’ll have to run it on your own server. Maybe that will change if I can find somewhere to stick it, I’ll keep you posted. I think some caching may be needed before that happens though.

Once running on your computer of choice, you only really have one uri to stick in your calendar of choice. You can extend this uri by passing in section names to filter the information displayed in the calendar. Here’s a few examples:

# to subscribe to a calendar of everything ( quite big! )
# to subscribe to a calendar of tips on just trees, ponds and wildlife
# to view all the available sections

And here’s what it looks like in Apple’s iCal once you have subscibed:

BBC Garden planner in iCal

Get your own copy of the app from github. You get up and running in the same way you would with any Sinatra/Passenger app. The structure and config file is already written, so you really just need to do the vhost and symlinking :

Happy Gardening!

Controlling the spotlights at work with Sinatra and OSC

posted by Duncan at 8:47 pm on April 24th, 2009

Nic bought some spotlights the other week for work, so we could hook them up to our continuous integration server, and they would let us know using the medium of colour, when things had failed!

Before we did that, I thought I’d have a little play. Tristan and Chris having been building a fun new game (Read more via the Radio Labs blog soon) which uses nowplaying data from BBC 6 Music to power it. Knowing when a track is starting and ending is also useful, so as a bit of fun, I wrote a little Ruby script that made the spotlights behave like traffic lights:

  1. Green – when a track started playing
  2. Flashing Amber – when a track was about to finish
  3. Red – when the track finally finished

spotlightPhoto by Tristan

To control the lights we’re using ROSC open sound control for Ruby. It’s very good, but required building to install, and the interface is nice, but not simple enough if all you’re doing is turning a light on or off and changing it’s colour. So in order to make the script even more simple, I thought I’d write a little http proxy so that your could control the lights by just going to a url. It meant even the non-programmers could have fun replicating disco lighting. The urls look like this:


The proxy is written using Sinatra. I love Sinatra. If you thought prototyping was simple with Rails, well with Sinatra + Passenger it takes simplicity to a whole new level.

You can download the lightcontroller source from Github as normal, you’ll have to tweak a few settings for your needs. You may even find it a useful template for controlling something else.

expires_in Rails FileStore fragment caching in about 6 lines

posted by Duncan at 2:32 pm on April 3rd, 2009

I have been building a Rails app that creates RSS feeds based on a lot of screen scraping behind the scenes. I really needed a way to cache the data object that feeds those specific pages. In rails you can do page, action and fragment caching, using many different methods, and storing that cache data in many different locations.

After deciding I needed Fragment caching, it turns out that if you need your cache data to magically expire over time, you need to be using memcached (I couldn’t), other wise you have to roll you own solution, based on cache sweeping, or in-code testing of whether your content has expired.

I’m using FileStore for caching (cached files live on the file system) and I have no database, and all the feed data comes from external feeds that change over time, so unless I fetch it all again, I don’t know.

What I needed was the ability to have a expires_in parameter whilst using FileStore. So this is what the code below does in surprisingly few lines. The code is an amalgamation of many posts I found doing almost the same thing, but this is what works for me.

First we extend FileStore:

# stick this in a file somewhere in your path e.g lib/fs_extend.rb
class ActiveSupport::Cache::FileStore
  def read(name, options = nil) 
    ttl = 0
    ttl = options[:expires_in] if options.is_a?(Hash) && 
    fn = real_file_path(name)
    return if ttl > 0 && File.exists?(fn) && (File.mtime(fn) < ( - ttl)), 'rb') { |f| Marshal.load(f) } rescue nil

And then in your controller I have:

class FooController < ApplicationController 
  def do_stuff
    key = 'someuniquekey'
    expire_fragment(key) unless 
        @data = read_fragment(key, :expires_in => 1.hour) || 
                                  write_fragment(key, some_complex_data)
    respond_to do |format|

This has reduced many 5 second requests to 32ms requests, so hurrah!

Installing Nokogiri on CentOS5

posted by Duncan at 10:45 am on November 13th, 2008

Nokogiri is an HTML, XML, SAX, and Reader parser with XPath and CSS selector support. It claims to be a drop in replacement for Hpricot, and it claims to be faster. I think the latter has been addressed but the drop-in bit, well. You see, the Hpricot’s gem had always in the past, just installed, but with Nokogiri I had dependency problems. Turns out I needed a few extra libs that were not installed on my machine. Here’s how I got things going again:

# It seems you need the latest version of rubygems
sudo gem update --system
# and some RPM's
sudo yum install libxml libxml-devel libxslt libxslt-devel
# now the gem should install ok
sudo gem install nokogiri

Twammer, sending stuff from Twitter to Yammer

posted by Duncan at 8:12 pm on September 28th, 2008

I use Twitter. I don’t really post much, but I do like to dip in and hear what my friends are doing. Yammer is like a private Twitter for companies. We have just had a network set up for the BBC, which is interesting. Again, I like the idea of posting what I am doing and dipping into what other people are doing around the organisation.

So you can see my problem, I’m doubling up here a bit. When I post something to Twitter, and I think it would be relevant for work too, I don’t want to have to open a different interface and post it again. Now there are services like which will sync every site you belong to, but I what to decide what gets posted at work and what doesn’t in this instance. The simple answer is to build something that looks at my Twitter feed and if it sees anything new, then posts it off to Yammer. My friend Mr Humfrey does that very thing. I wanted more than this though, I wanted to filter, plus I wanted to make it easy to use, so I have written this very simple app that does the job:

  1. Read my Twitter feed every n minutes for changes
  2. Keep messages with my chosen keyword
  3. Remove keyword, and send off to Yammer

Twammer usage:

# Display help
./twammer -h
# check Twitter messages with last 5 mins (default)
# filter by #bbc but don't send anything to Yammer
./twammer -t 47983 -u user -p pass -f bbc -q
# check Twitter messages within last 10 mins
# send anything to Yammer. Show output
./twammer -t 47983 -u user -p pass -d 10 -v

This is designed to be run as a cron job. You must also make sure you run it at the same interval as the –delay in the app. Here’s what my crontab looks like:

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/duncan/bin/twammer -t 47983 -u user -p pass -f bbc > /dev/null

[UPDATE] Changed to using the correct API instead of scraping the RSS

rQRCode, a Ruby library for encoding QR Codes

posted by Duncan at 3:37 pm on February 24th, 2008

[Update]: I have also posted a bit more about the upcoming QR Code work for BBC /programmes.

We implemented QR Codes on BBC programmes (A project I’m one of the Software Engineer’s on, at the BBC) a few weeks back. They’ve been talked about since so I won’t repeat things, only to say it was a simple implementation using a JavaScript library by Kazuhiko Arase and took all of 5 minutes of my lunch hour to add. It started as an email by colleague Michael Smethhurst asking if we could/should implement them, and was also the first I had really heard of them.

Since then, I have done lots more research and think there is great mileage there for promotional material and advertising. At the moment the codes are built on the client side, but this will be moved to the server soon so we can start caching the pages, and also so we can provide the code in a more useful format (maybe images instead of the current HTML table).

So to the point of this post. During my research to understand QR Codes more and because of the lack of free readable spec (Anyone know if there is one?) I decided to reverse engineer the JavaScript library into a Ruby Gem as there didn’t seem to be one out there. This helped me understand the how QR Codes are encoded, while at the same time giving something back for other people to use.

You can find out more over at it’s project page on Github:

So that should get you going. Enjoy!

Show Us Your Gems

posted by Duncan at 4:12 pm on February 12th, 2007

What a good idea. Here’s what I have installed currently:

$ gem list|grep '^[a-zA-Z]'
actionmailer (1.3.3, 1.3.2)
actionpack (1.13.3, 1.13.2)
actionwebservice (1.2.3, 1.2.2)
activerecord (1.15.3, 1.15.2)
activesupport (1.4.2, 1.4.1)
acts_as_versioned (0.2.3)
amazon-ec2 (0.1.0)
builder (2.0.0)
camping (1.5.180)
camping-omnibus (1.5.180)
capistrano (1.99.1, 1.4.1, 1.4.0)
cgi_multipart_eof_fix (2.1)
daemons (1.0.4)
deprec (1.3.1)
fastercsv (1.2.0)
fastthread (0.6.3)
ferret (0.11.2)
flexmock (0.6.1, 0.6.0, 0.5.0)
gem_plugin (0.2.2)
highline (1.2.7)
hoe (1.2.0)
hpricot (0.5)
map_by_method (0.6.0)
markaby (0.5)
metaid (1.0)
mocha (0.4.0)
mongrel (1.0.1)
mysql (2.7)
needle (1.3.0)
net-sftp (1.1.0)
net-ssh (1.1.1, 1.1.0, 1.0.10)
railroad (0.3.3)
rails (1.2.3, 1.2.2)
railsbench (0.9.2)
rake (0.7.2, 0.7.1)
rb-appscript (0.3.0)
RedCloth (3.0.3)
rfacebook (0.6.2)
rmagick (1.15.6, 1.15.2)
rspec (0.8.2,
ruby-openid (1.1.4)
ruby-yadis (0.3.4)
rubyforge (0.4.1)
sources (0.0.1)
sqlite3-ruby (
stemmer (1.0.1)
termios (0.9.4)
test-spec (0.3.0)
what_methods (1.0.1)
xml-simple (1.0.10)

back to the top