Last night the folks at Twitter released a significant update to their API. Among the changes was a massive change to the way the Twitter API responds to time-sensitive queries. In short, the since parameter and If-Modified-Since
header are no longer supported.

For those unfamiliar with the API, these features allowed searches for Public Tweets or Direct Messages that occurred after a specific point in time. It was intended to both reduce load on Twitter’s service and allow API clients to selectively retrieve tweets that they haven’t seen yet based on a time stamp.

The fact that the API changed is not shocking. The Twitter team had a solid reason behind it, as it seems date based queries weren’t handled efficiently enough for widespread use. There is even an alternate API search, based on message id, that can provide a similar function. Therefore these changes could really stand to benefit the developer community as a whole by providing increased capacity and performance for the API.

The API change was dangerous

First, and most importantly, the API was modified in a way that it would simply begin to ignore since parameters as if they weren’t even there. This means that clients querying the API with the expectation of retrieving a limited subset of tweets they hadn’t seen yet would, all of a sudden, start seeing a completely unfiltered feed of tweets, including ones from the past.

Consider the hundreds of applications like our own apps Tempo and PingMe, which process Twitter direct messages they haven’t yet seen to perform operations on a user’s data. Applications polling the Twitter API using the since parameter would be flooded by old, duplicate messages that they had already processed, possibly causing rampant duplication and perhaps data corruption.

edit: Let me draw an analogy: imagine that one day before release MySQL (or pick your favorite DB) decided to deprecate and ignore the the greater than > operator on queries. No errors – it would just pretend that query clause didn’t exist and return all rows for queries that used a >. How would that affect your application?

It is just poor form for a public API to change in such a way that the results of API operations become unpredictable like this. The API release notes claim this approach was used to ensure that existing applications didn’t break, but in fact would almost guarantee catastrophic failure of applications dependent on the function.

Errors are OK

Don’t get me wrong, it’s absolutely Twitter’s prerogative to change the API, and they can do so whenever they want. But with tens of millions of users and thousands of application and sites using the Twitter API, they should be more responsible about these kinds of changes.

If the API drops support for certain operations, just return an error message. Sure, it will be annoying to get error until the application is fixed, but at least you don’t risk corrupting thousands of down-stream integrations with improper data.

24 is only good enough for Jack

To make the situation worse, Twitter gave their developer community 1 day notice about the breaking change. Imagine if you didn’t get that email, or consider a case where an API release occurred without notification.

edit: When you expose a public API it becomes a contract with your developer community. If you make a breaking change to the API you have a fundamental responsibility to effectively communicate the change and ensure that it doesn’t create massive and unpredictable results for clients.

As Twitter grows and becomes more popular in the mainstream it needs to establish a process for notification on API changes and stick with it. Give the developer community enough time to properly code and test changes to their applications before a release. If it’s not convenient to do on the main API, perhaps they could consider some sort of business service that would provide premium access and a stabler environment for groups and teams like ours that really depend on it.

I just made a couple of updates to our NSDate (Helper) project, thanks to a suggestion from Stephan Burlot in our comments. We’re not the only ones who think working with NSDate is a big pain! The code I had in there to determine relative days ago was based off of NSCalendar and while accurate in its own special way, wasn’t quite what you’d want for a relative time display.

In addition to the tweaks made in the code, I finally got around to creating a more illustrative set of examples in the README, and put up a corresponding document in our software archive. Check out how it works over at Github.

I miss my old Treo’s bluetooth sync capability. And for a device that does syncing (although in a much more limited way) and has bluetooth, it’s somewhat obnoxious that I have to sync over the wire with my iPhone, that wireless sync isn’t available.

It’s amazing that all smart phones still commonly cripple the bluetooth capabilities of the devices to keep you from using them and for all sorts of reasons – Apple doesn’t want you looking at the file system, Verizon wants you to buy ring tones over their EVDO network, etc. I was hoping Apple would be a pioneer here, tell the carriers to shove it, and go further than Palm did, making other bluetooth capabilities available.

On a mostly unrelated note, there has got to be a better way to package up and bundle Cocoa code you frequently use in different applications.

We all make mistakes. Or we make decisions that we have to later revisit, to put it kindly! I’ve got a Rails app where I wasn’t using UTC back in the beginning, and this was pre Rails 2.1, so ActiveRecord has been storing all the time stamps into postgres as Eastern time. I need to move all those dates over to UTC for this particular app to go through its next growth spurt, so I threw together a migration that does just that. I’d like to think it couldn’t get more DRY, hope it’s useful to anyone else out there. It’s dependent on Postgres, but you could adapt the increment statement to suit your own DB or to be more generic:

The last two weeks have seen a ton of development work here at Zetetic. We’ve been working to cut over one of our bigger client’s single-sign-on systems to a new location, we’ve built new features into the Strip beta (update coming soon, promise!), and we’ve completed a pile of performance, stability, and maintenance improvements for Tempo, our time tracker.

I should say that a lot of this work was made possible by an extra pair of hands; we’ve recently brought Bret Morgan onto our team as a developer. He’s been doing a lot of great Rails development work on Tempo and helping out with testing and Q&A for Strip.

Without going into all the changes we’ve made in Tempo, there are two I’d like to highlight:

Message Handler Stability

Tempo provides our customers with various ways to log time remotely and start timers, and our message handler is responsible for scooping up email and Twitter messages. We did some tweaking that dramatically improves its ability to handle the unexpected (you wouldn’t believe the weird stuff that comes in from MS Exchanges and Outlook systems), touches up its ability to respond to erroneous messages, and ensures that Message Handler keeps it up. We had some older daemonizing code in there, and ripped it out in favor of the Looper module we posted to Github recently. We’ve been using Looper in PingMe’s handlers for quite some time now, it was time to gemify it and use it in Tempo.

Tag Operations Improved ++bajillion

Sometimes you need to re-name your tags in Tempo, sometimes you need to add a tag like ‘oracle’ to everything tagged ‘apex’, and sometimes you just need to delete a tag from all of the entries in the current report. To provide this capability we added the Batch Tag interface, and it got the job done, but it was horribly slow. We got down and dirty with the SQL and updated our tag caching strategy to get it right. Operations on hundreds and hundreds of entries that used to take really long times (often leading to time-outs, I’m sad to say) are now complete within a second or two.

The batch tags interface still needs work, I will concede. We’re not entirely happy with the UI itself and expect to overhaul it soon.

We are planning to roll out these updates and more on Monday night EDT, and we’ll publish a full list of improvements made once it’s complete.

So let’s talk Strip. A number of people are already finding that the simple Categories → Entries navigation can be a little restrictive. We’re working on a number of features to allow for easy re-categorization of Entries, but there are two new features that will make accessing your data a lot easier:

Recents

Simple idea, extremely convenient, a tab displaying the most recent entries you’ve opened up:

I use this more than the Categories → Entries navigation already.

Search

Sometimes I just don’t remember where I put something, and I don’t really want to figure it out, I just need the information, and I need it now. I also tend to think that perfect taxonomies are a thing of fantasy, and I get annoyed at software that expects me to be a perfect person. Accordingly, the new Search interface allows me to start typing the name of an Entry, or any data I’ve stored in Strip and it will pull up any matching Entries:

I think you can expect to see these new features among others in the next version of the Strip beta early next week. Thanks for all the great feedback, and keep it coming. If you’ve recently e-mailed us about joining the beta, or you’d like to join the beta, be sure to send us your device’s UDID to support@zetetic.net. The Ad Hoc Helper app (free) will actually take care of the details for you.