Last night the folks at Twitter released a significant update to their API. Among the changes was a massive change to the way the Twitter API responds to time-sensitive queries. In short, the since parameter and If-Modified-Since
header are no longer supported.
For those unfamiliar with the API, these features allowed searches for Public Tweets or Direct Messages that occurred after a specific point in time. It was intended to both reduce load on Twitter’s service and allow API clients to selectively retrieve tweets that they haven’t seen yet based on a time stamp.
The fact that the API changed is not shocking. The Twitter team had a solid reason behind it, as it seems date based queries weren’t handled efficiently enough for widespread use. There is even an alternate API search, based on message id, that can provide a similar function. Therefore these changes could really stand to benefit the developer community as a whole by providing increased capacity and performance for the API.
The API change was dangerous
First, and most importantly, the API was modified in a way that it would simply begin to ignore since parameters as if they weren’t even there. This means that clients querying the API with the expectation of retrieving a limited subset of tweets they hadn’t seen yet would, all of a sudden, start seeing a completely unfiltered feed of tweets, including ones from the past.
Consider the hundreds of applications like our own apps Tempo and PingMe, which process Twitter direct messages they haven’t yet seen to perform operations on a user’s data. Applications polling the Twitter API using the since parameter would be flooded by old, duplicate messages that they had already processed, possibly causing rampant duplication and perhaps data corruption.
edit: Let me draw an analogy: imagine that one day before release MySQL (or pick your favorite DB) decided to deprecate and ignore the the greater than > operator on queries. No errors – it would just pretend that query clause didn’t exist and return all rows for queries that used a >. How would that affect your application?
It is just poor form for a public API to change in such a way that the results of API operations become unpredictable like this. The API release notes claim this approach was used to ensure that existing applications didn’t break, but in fact would almost guarantee catastrophic failure of applications dependent on the function.
Errors are OK
Don’t get me wrong, it’s absolutely Twitter’s prerogative to change the API, and they can do so whenever they want. But with tens of millions of users and thousands of application and sites using the Twitter API, they should be more responsible about these kinds of changes.
If the API drops support for certain operations, just return an error message. Sure, it will be annoying to get error until the application is fixed, but at least you don’t risk corrupting thousands of down-stream integrations with improper data.
24 is only good enough for Jack
To make the situation worse, Twitter gave their developer community 1 day notice about the breaking change. Imagine if you didn’t get that email, or consider a case where an API release occurred without notification.
edit: When you expose a public API it becomes a contract with your developer community. If you make a breaking change to the API you have a fundamental responsibility to effectively communicate the change and ensure that it doesn’t create massive and unpredictable results for clients.
As Twitter grows and becomes more popular in the mainstream it needs to establish a process for notification on API changes and stick with it. Give the developer community enough time to properly code and test changes to their applications before a release. If it’s not convenient to do on the main API, perhaps they could consider some sort of business service that would provide premium access and a stabler environment for groups and teams like ours that really depend on it.
I just made a couple of updates to our NSDate (Helper) project, thanks to a suggestion from Stephan Burlot in our comments. We’re not the only ones who think working with NSDate is a big pain! The code I had in there to determine relative days ago was based off of NSCalendar and while accurate in its own special way, wasn’t quite what you’d want for a relative time display.
In addition to the tweaks made in the code, I finally got around to creating a more illustrative set of examples in the README, and put up a corresponding document in our software archive. Check out how it works over at Github.
I miss my old Treo’s bluetooth sync capability. And for a device that does syncing (although in a much more limited way) and has bluetooth, it’s somewhat obnoxious that I have to sync over the wire with my iPhone, that wireless sync isn’t available.
It’s amazing that all smart phones still commonly cripple the bluetooth capabilities of the devices to keep you from using them and for all sorts of reasons – Apple doesn’t want you looking at the file system, Verizon wants you to buy ring tones over their EVDO network, etc. I was hoping Apple would be a pioneer here, tell the carriers to shove it, and go further than Palm did, making other bluetooth capabilities available.
On a mostly unrelated note, there has got to be a better way to package up and bundle Cocoa code you frequently use in different applications.
Last night we deployed a pretty substantial update to Tempo, our time tracker. Many of the changes are unseen, nuanced, and/or performance-related, so I want to give our customers an update on what we’ve done. It looks like a short list, but it was quite a bit of work. Details follow for some of the more significant changes.
Improved Time Zone Support
This was a huge update. In order to support future planned features we needed to overhaul our handling of time localization. There were a couple of hiccups last night during the deploy related to this, but we’ve got it worked out now. Tempo does browser-based detection of your time zone for new users, setting your time zone preference. On the Account screen you can now change this preference, should you so desire.
Remote Command Stability Upgrade
As mentioned last week, we’ve done a bit of work to make sure that our remote command message handlers are rock solid in the face of weird input via email, and outages on Twitter. The remote command handlers are Looper daemons that check to see if you’ve sent in time entries or timers by email or Twitter.
Batch Tagging Performance
Also mentioned here previously, the batch tagging interface received a serious speed bump. This was a bit tricky, because we’ve been caching tags as a concatenated string on the entries for performance. Nothing a little PL/PgSQL can’t fix.
Rails Upgraded
It’s no secret that Tempo is a RubyOnRails application. It started it out on 1.2 and we’ve kept upgrading with each update to the framework; we’re now on the 2.3 series. This brings significant performance and flexibility improvements. It may not seem like a big deal, but the amount of work that goes into making these upgrades a success is well worth the sweat considering the capabilities it opens up for Tempo in the future.
Potential XSS Vulnerabilities Closed
Being a security company, we try to never take the appearance of security for granted. We decided to give our code a thorough review to plug any potential cross-site-scripting vulnerabilities. Generally speaking, such holes allow an attacker to execute malicious Javascript in your browser. Such code can execute as you inside the application in question, which can wreak all sorts of havoc. Fortunately, we’ve been really on top of this from the beginning, so we only found a couple of instances in Tempo where this was possible, and plugged up the few holes we found.
Coming Soon!
That’s that for this update, but there’s a lot more coming around the corner. We have a new feature that’s almost ready, and we’re working with nGen Works to develop a new graphic design and improve the user experience dramatically.
We’ll be pushing a number of maintenance and performance improvements out to Tempo tonight, at 10pm EDT. This should only briefly interrupt service for our users. More information on the various changes will be posted here once the update is complete.