Rack::Zetetic::Campaign

A Rack-based URL Shortener for Link Tracking

If you’re like us and you like to send out a newsletter once in a while about one of your new products – in particular if you’re running promotional ad campaigns and your own marketing efforts – you’ve probably tried tacking on Google’s various utm_.* variables to the URLs you are sending people. These are really handy in that Google’s analytics code looks for these when users visit your website, allowing you to start tracking which of your ads, promos, and newsletters is performing well, and which are bombing.

The only problem with this is that you end up trying to put links like this into emails, and they’re long enough to be problematic on display, in particular in plain-text emails:

Write a blog post about Strip telling us what you like and 
what you don't like, and we'll hook you up with a candy-
bar! Click here to learn more: (not an actual link)

http://getstrip.com/promos/blogging?utm_medium=email&utm_term=&utm_campaign=new+promotional+offer&utm_content=textlink&utm_source=newsletter&

We’ve all seen emails where that URL gets garbled and cut into two lines.

What we used to do for this was to employ a URL shortener like bit.ly or tinyurl.com, resulting in links like http://bit.ly/abc123. This solves the problem of munging, but brings on two new problems: your links are dependent on bit.ly always being available and working correctly, and they’re not always easy to “fix” once they’re out there. In particular if you decide to change up the various UTM variables you’re sending to Google Analytics. PITA.

Suffice it to say, many sites are choosing to implement their own URL shorteners to ensure their carefully crafted links don’t rot, and we decided that we’d do ours as a tiny Rack application with a YAML file:

# Rack-up config
require 'rubygems'
require 'rack/zetetic/rack-campaign'
run Rack::Zetetic::Campaign.new('/path/to/your/campaigns.yml')

To install:

gem install rack-campaign

Here we list our campaigns out in YAML:

blog-rack-campaign:
  url: http://dev.zetetic.net/blog/2010/06/11/introducing-rack-campaign
  tokens:
    campaign: rack-campaign
    source: blog
    medium: internets
    content: example link

blog-getstrip:
  url: http://getstrip.com
  tokens:
    campaign: rack-campaign
    source: blog
    medium: internets
    term: strip rocks!
    content: example link

At the top level of our Nginx config, we define an upstream directive for Unicorn, which will host our Rack application over a unix socket (mwahahaha):

# define upstream for campaign routing
upstream campaigns {
  server unix:/www/campaigns/tmp/sockets/unicorn.sock;
}

Then, in each of our Nginx vhosts, we add a location directive for /c, telling the server to forward all such requests to the unicorn process:

location /c {
  proxy_redirect     off;
  proxy_set_header   Host             $host;
  proxy_set_header   X-Real-IP        $remote_addr;
  proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
  proxy_max_temp_file_size 0;

  # forward everything else to the mongrel cluster
  if (!-f $request_filename) {
   proxy_pass http://campaigns;
   break;
  }
}

Now, we can pass out the link you may have seen on Twitter:

/c/blog-rack-campaign

When you visit it, you get redirected to:

http://www.zetetic.net/blog/2010/06/11/introducing-rack-campaign?utm_campaign=rack-campaign&utm_source=blog&utm_medium=internets&utm_content=example%20link

Isn’t that neat!?

This needs to perform very well, so we did a little testing with Apache’s ab testing tool and found it runs as fast as a normal nginx rewrite. Here’s what a basic rewrite looks like (/software rewrites to /code):

$ ab -n 500 -c 10 /software
...snip...
Non-2xx responses:      500
Total transferred:      191000 bytes
HTML transferred:       92500 bytes
Requests per second:    4626.33 [#/sec] (mean)
Time per request:       2.162 [ms] (mean)
Time per request:       0.216 [ms] (mean, across all concurrent requests)
Transfer rate:          1721.00 [Kbytes/sec] received
...snip...
Percentage of the requests served within a certain time (ms)
  50%      2
  66%      2
  75%      2
  80%      2
  90%      2
  95%      2
  98%      2
  99%      2
 100%      2 (longest request)

And here’s how rack-campaign is performing:

$ ab -n 500 -c 10 /c/blog-rack-campaign
...snip...
Non-2xx responses:      500
Total transferred:      167500 bytes
HTML transferred:       7000 bytes
Requests per second:    2749.78 [#/sec] (mean)
Time per request:       3.637 [ms] (mean)
Time per request:       0.364 [ms] (mean, across all concurrent requests)
Transfer rate:          896.43 [Kbytes/sec] received
...snip...
Percentage of the requests served within a certain time (ms)
  50%      2
  66%      2
  75%      3
  80%      3
  90%      4
  95%      4
  98%     13
  99%     14
 100%     39 (longest request)

Obviously, there’s some differences but so far, it looks like rack-campaign is doing well enough here. I’m sure it could be faster, and I’m sure that over time a YAML file could become a cumbersome manner of storage, but for now it’s good ’nuff. Feel free to fork it and use a SQLite database (which in this case would likely be very fast).

Installation

gem install rack-campaign

A thing of minor note: we’ve updated the main zetetic.net website. Aside from giving a few of our pages a long-needed refresh, we’ve decided that we really wanted to get all this content out of the database and onto the filesystem.

For a long time, we’ve been using the Radiant CMS to run both the website and the blog. It’s had it’s ups and downs. Radiant is pretty cool, a Rails-based application that has a lot of brilliant tricks for building out a website with a lot of shared content and cross-marketing. However, for us it’s always been a tad volatile and clunky. For instance, it’s easy to accidentally double-submit on a new post to the blog which then creates a page with no parent_id and that makes your whole site impossible to access.

Major version upgrades have been a real nightmare, especially when it comes to getting plugins updated. Plugins are what make Radiant so dynamic, but they tend to be very brittle, and are complex to upgrade with a major update of Radiant. Simple things like tagging or comments really ought to be part of the core system.

Then there’s the memory usage, or rather Ruby’s memory allocator and garbage collector (which could be considered a form of entropy). We don’t see the need to incur and manage this overhead just to provide static HTML content and images.

Finally, we wanted to be able to build out our pages in HAML, manage them over time with Git, and we’ve found that the Ruby gem staticmatic is just the thing for this. It provides a Rails-like project structure, partials, layout capabilities, etc. I even managed to hack in an automatically generated sitemap.xml.

At this point our website is almost entirely static content, with a little jQuery here and there where needed, and the blog itself is still on Radiant, soon to be moved over to another engine. When we have some spare cycles I’ll try to post some of the tips and tricks we employed to make it happen, but it’s nothing that any competent Ruby programmer can’t handle.

As an aside, the staticmatic gem is maintained by Stephen Bartholomew, and he’s doing a great job of actively maintaining the project. The mailing list is pretty active and helpful, and there seems to be a good community of people using it. That tends to give a tool a longer shelf-life and makes us even more confident in our decision to make the switch.

The argument about the iPhone OS being too “closed”, or that Apple is controlling what you do with it too tightly, is still being pretty hotly debated, and I’m loathe to wade into it because most of what could be said has been said, usually with a lot of invective that’s unhelpful. For the most part, while I find it annoying, I try to keep in mind that nobody is forcing anyone to buy iPhones. Take it or leave it.

Yesterday, John Gruber brought up this interesting proposition by Jason Snell in which the latter proposes that allowing the “side-loading” of native apps would end the debate, and end the claims of critics that iPhone OS is too “closed.” I don’t agree with him that this will be the direct consequence, but I think it is worth considering. Happily, Gruber has pulled the money quotes for us:

I don’t think the company needs to stop controlling what apps get in the App Store. All Apple needs to do is add a new feature, buried several menu items down in the Settings app, that mirrors the one found on Android devices: an option that lets you install Apps from “unknown sources.” If a user tried to turn this option on, they’d get a scary warning about how these sources couldn’t be trusted, and that they may lead to instability, crashes, loss of data, you name it. Scary stuff.

Most users will never find that setting. Many who do will be loath to turn it on. But by putting it there, Apple immediately shuts up every single claim that the iPhone isn’t open.

Obviously, I like this idea, but I don’t think it “immediately shuts up every single claim that the iPhone isn’t open.” Gruber responds:

Personally, I’d welcome such a move, but I don’t think it would have the effect Snell envisions. Snell’s argument is that Apple should do this to nip the argument that the iPhone is too closed. But if Apple did exactly what Snell argues, critics would still harp on the closed App Store. iPhone critics have seldom let facts get in their way.

Maybe John’s right about critics of the App Store, at least some of them. But I think there’s another angle of influence to consider here, a better reason to do this, that will help achieve the goal of calming critics who might scare away potential customers, telling them that the device is “too closed.” Consider Gruber’s further argument later on in his post:

If there are people who think the iPad can’t read PDFs or play music and videos that aren’t purchased from the iTunes Store, then surely there would be people who’d think you can only install apps from the App Store even if sideloading were a supported option, as per Snell’s suggestion.

This is more likely than not the result of the tech-head discussion spilling over to blogs that normal people read and daily conversation in a kind of painful telephone game. It’s almost like watching political campaign spin and slogans propagating.

I’d like to offer a bit of dissent here. Perhaps I’m not the kind of critic John is writing about, since I’m already an iPhone developer, but this would certainly make me very happy and alleviate most of my concerns (in particular the very high risk of investing months of “opportunity cost” on an application that could be rather capriciously rejected from the App Store). Also, and more importantly I think, I’m one of those people who gets asked by friends, “should I buy an iPhone? I heard those things are really closed, that you can’t do what you want with them.” My answer to those folks would suddenly be radically different*.

Then there’s all my sysadmin friends buying Droids instead of iPhones. I think they’d have a radically different view of all this, too. Being able to load what they want is specifically the concern they have. In my circle of tech heads, we don’t care that the App Store is closed, we care that we are prevented from loading what we want onto the device. We’d still happily sell software on the App Store (I think few would choose to go it alone) and buy software on the App Store. And we’d be more inclined to develop software for the device itself. In fact, you’d probably have an explosion of freeware, further entrenching iPhone OS.

I think a lot of tech folks – who are not normal folks, but who end up being a product’s pro-bono (or de facto?) evangelists to the normal folks – do see the prohibition against side-loading native apps as being really heinous. They’re like howling monkeys on the issue! And they are very much the people coloring the discussion, starting this telephone game of “the iphone is locked down,” resulting in normal people thinking that they can’t read PDFs or load their own music. Maybe I’m wrong, but I think these are dots appropriately connected.

I think all this other stuff about the App Store itself being too closed is just window dressing for that issue. I don’t think Apple needs to do this to convince people to buy the iPhone or the iPad – so far it would seem they clearly don’t – so I don’t expect to see it happen.

An aside: I don’t think it can reasonably be said that the prevention of side-loading native apps is solely to protect the user experience, as it’s also been exercised by Apple in ways that have nothing to do with protecting the user experience (rejecting Google Voice, punting that 4Chan image scraper or whatever it was, etc). Furthermore, it obviously gives them a monopoly on revenue for the whole market place. The App Store is very convenient, but it’s very much the only game in town by force.

Allowing the side-loading of native apps really would change the discussion quite a bit, and does cut the rug out from under the whole “what iPhone doesn’t, Droid does (unless Verizon doesn’t want you to)” ridiculousness.

* Usually my answer is, “yeah, they’re great, I love it. No, you can’t load whatever you want onto it, but it’s still really awesome, despite regularly dropping all my calls.” And that right there is probably why Apple won’t change the policy. Still, “Yes, buy it, yes, you can do whatever you want with it, it’s really awesome, despite regularly dropping all my calls,” is a much better endorsement.

We’re a hair’s breath away from releasing the latest and greatest version of Strip, our password manager and data vault for iPhone OS. Just a few tweaks to make, then we’re on to App Store submission. Strip Sync is also available for general beta, just pop on over to the contact form and drop us a note if you’d like to participate.

We recently merged the development branch into master and tagged the release. I love looking at these stats:

113 files changed, 6956 insertions(+), 4447 deletions(-)

More prep work to do, we’ll keep you posted on our progress. Beta testers – the feedback has been great, please keep it coming!

One last thing to note: the Ad Hoc development and app signing quagmire that Apple has developed to control how we handle beta testing has been a huge hindrance to many of our beta testers. We’re sorry about the complexity, there’s not much we can do about it, but we seem to have most folks getting along alright.

Because I’m totally comfortable with allowing the phone company with the most contempt for its customers to access my bank account information:

I can’t wait to stop paying these jokers their “subsidy” once the contract ends and my monthly bill doesn’t go down. Need to be able to make phone calls, plain and simple. Why are they even bothering to spend R&D money on this junk? Perhaps they recognized that if they couldn’t force this stuff on their subscribers, they can at least make it available through the App Store. Still doesn’t make any sense.

It’s been suggested by Marco Arment and others that one of the reasons the iPhone isn’t available on Verizon yet is because Apple doesn’t go for allowing the carriers to dictate anything about the device, and Verizon is pretty much #1 at doing just that. I’m curious of any of my VZW-Droid-owning friends can elaborate on whether this is currently the case for them? Are there apps on there you don’t want and can’t delete?