Zetetic

With the new smart timers, I no longer need PingMe to remind me to sum up my time for the day. Thanks, old buddy!

(Although the street cleaning reminders are still pretty clutch…)

A long time ago I read Neal Stephenson’s essay In the Beginning…was the Command Line, an absurdly fun romp through the evolution of the modern operating system (at least up until the time it was written, it’s a bit dated now), in which he compares Windows 95 to a crummy station wagon, classic Mac OS to a cheesy Euro-sedan, and Linux to free tanks that no one wants. It’s a kind of ode to a hacker’s love for good tools, and discusses how interfaces have evolved from the command line as their ancestor.

We here at Zetetic are still big fans of the command line, which makes sense given our line of work. We prefer many good tools that do simple things that we can link together to do very powerful things. That’s the legacy of many of the oldest command line tools that are still in heavy use today, and the driving aesthetic behind some emerging tools like Git. (If I’m speaking somewhat generally here, it’s because I wish to avoid alienating readers with grotesque demonstrations of piped UNIX commands ;-)

On the web this aesthetic seems to be driving a lot of innovation as many online services, including ours, are striving to be good at some very specific things and to offer easy integration points with related services. More and more, services are turning to Twitter to provide users with mobile features and access, which makes a lot of sense – not everyone can afford an iPhone, but almost everyone has text messaging capabilities in their pocket and a service like Twitter makes a useful transport (when it’s working).

And thus, we come full circle back to the command line, in the year 2008. How long will the honeymoon last? Hard to say, but I’m going to put my money on it being around until the next drastic change in how we physically interact with computers.

Those of us who are real big nerds and power-users tend to have keyboard shortcuts for everything and we like to avoid using a mouse to get around an interface quickly. We don’t want to go to the web to fill out a form and click ten things, we don’t want to navigate an interface with our fingers on our phone’s web browser.

What we want is to tell some service or system, in very short, simple terms, what to do for us. And if you’re like Stephen and I, you don’t really want to spell everything out, you want to provide just the barest amount of information needed so you can move on to the next thing.

We embraced this notion at Zetetic immediately when we first built PingMe, an interactive reminder service that provides a text based command line interface. In normal human-speak, you can send simple commands in a text message to create reminders while you’re on the go.

When building Tempo we saw that this capability was easily adapted to time entry. There are numerous ways to enter time with Tempo, but the most powerful, quickest, and probably the least understood so far is the command line. It’s a format of input that allows for mobile time entry by e-mail or twitter (giving you SMS and IM by extension), and it’s actually available in the web interface, too, shown below:

Instead of waiting for a web form to load and clicking through the interface, looking for what is desired, the user gets the immediate satisfaction of telling the system what’s up, but with only the text “1 hr design meeting in NYC @project @tag”. It’s faster, and there’s a sense of empowerment there. It’s like the difference between working within a bureaucracy and a small company. Less waiting around for everyone else, more personal responsibility. You don’t get a list of what tags or clients you have, but you already know because you use them all the time.

While folks aren’t flocking to the command line input form in our web interface, a lot of Tempo users are really taken with submitting their time entries over Twitter and E-mail, and are using this command line syntax. While I think we’re the first time tracking service to provide this kind of interface, we’re certainly not the only ones who think the idea is a good one, and that’s a good sign that we’re on the right track.

Over the past few months we've been getting a lot of great feed back from users of Tempo, we've been squashing bugs, and we've been working really hard on a number of new features and service integrations. We've also been continuously striving to streamline the interface and make it even easier to use.

This Sunday, May 11th, from 9pm to 12am Eastern, Tempo will be offline while we perform the latest batch of updates -- this is a big one! Stay tuned...

P.S. During the same time period, PingMe will be unavailable for system maintenance -- no pings will be sent and the web interface will be unavailable.

Over a year ago, when we were first putting together PingMe, a system not entirely unlike Twitter, we had a pretty good idea what the real potential was for a revenue model and how valuable it could be.

A few weeks back we posited these thoughts in public because the discussion of Twitter's likely potential for revenue keeps coming up in the blogosphere and it seems like everyone is missing the obvious: sticking small, meme-sized ads in the tweets themselves, based on context and relationships, and exploiting the nearly unbounded impression space. Nice to see somebody agrees with us about the context part of things, even though I think ReadWriteWeb is getting the means wrong.

I'm pretty sure that if Twitter started sending direct advertisements to their users, as opposed to embedded ones, they'd chase their users away to the clone services that are starting to emerge and which will mature. I don't think people will pay to subscribe to Twitter to escape the ads; when a service has been free for almost two years (a long time on the Internet), that kind of conversion is probably not in the cards (although not impossible). The potential revenue in just embedding ads is so high that I doubt they'd risk angering their user base with direct ads.

On top of that, it was a few months ago that some Tweets were arriving on my phone with an embedded ad for Twitter itself at the end of it, signaling, IMHO, that that would be the ad space in the future.

I should mention that since we started using our direct-embedding method in PingMe messages to briefly mention our other products and brands, we've yet to have any complaints from our users. You can't ask for better than that (well, aside from goal conversions).

GoPingMe.com experienced a brief service interruption today where the site was temporarily replaced with a copy of Tempo's homepage. We sincerely apologize to any users that weren't able to create or modify Pings on the site eariler. Thankfully the delivery of schedule PingMe reminders was not affected at all.

We've also been contacted speculating whether we were shutting down or canceling the free PingMe service. Don't worry PingMe isn't going anywhere. This problem was the result of a mis-keyed IP address in our web server configuration that crossed traffic between application servers.

Again, we're very sorry for any inconvenience.

Suppose you have something really important to do every week, like checking that your car is still legal for street cleaning (raise your hand if you've been towed). You'd probably be tempted to set up a ping that both repeats and pesters. This way you'd get pestered about it each day to make sure that you don't forget, and PingMe would reschedule it for next week once you completed the task.

In the past, this feature has been missing from PingMe. When you received a pestering ping that also has a repeat schedule, and you replied with 'off' or 'done' or 'stop' (or 'ok' or 'okay') to stop the pester, say from your phone, the ping would be marked as done, and it would turn off. For good. But most people really set pester & repeat pings up so that after they turn off the pester for today, they'll still get the ping tomorrow (or whenever the next scheduled repeat is, if you follow me).

Well now you have options! We've changed the behavior of two of the stop words so that they only stop the pester of a ping and not the repeat. As of this morning, replying to a ping with 'ok', 'okay' or 'done' will stop only the pester of a pestering & repeating ping. Replying with 'stop' or 'off' will turn the ping off as before.

To sum it up, once you go and move your car to the other side of the street, you can reply to the ping 'ok' and it will stop bothering you until tomorrow, when it's time to move your car again.

==> /var/log/pingme/PingMeReceiver.log <==
[INFO] change: 17827, Preparing
[INFO] change: 17827, stripping message part 0...
[INFO] change: 17827, Processing.
[INFO] change: 17827, found a stop message on this line:  Ok
[INFO] change: 17827, this is a stop message for ping 16427, 
[INFO] change: 17827, user requests to stop pester
[INFO] change: 17827, nagging and recurring ping, clearing events for reschedule

It's alive!

Over at Ryan's Scraps, in a post about the new TimeWithZone functionality in edge Rails, there are a pair of comments that I want to highlight. A fella named Ben asks "Couldn’t this be pushed deeper so that current_user.registered_at is a TimeWithZone?"

Then there's a response from the main guy who developed the TimeWithZone functionality, Geoff Buesig, in regards to how they intend it to be used (and with a bunch of other neat and helpful notes that you should check out):

1.TimeWithZone is similar to the Duration class, in that, you should never need to create an instance directly—in the TWZ case, you’ve got the #in_time_zone, #in_current_time_zone, #change_time_zone and #change_time_zone_to_current methods on Time and DateTime instances that will handle that for you.

So, for example, you can do this:

current_user.registered_at.in_current_time_zone

... and the result will automatically be wrapped in a TimeWithZone

What Ben is asking for, and what Geoff seems to be distancing himself from, is exactly what we here at Zetetic would find incredibly useful: the ability to harness our database backend's time zone support, PostgreSQL's 'timestamp with time zone'.

Here's the deal. PingMe was designed for users around the globe so it supports time zones. We set it up so that all timestamps (:datetime) were stored in UTC in the database, and converted to the user's local time on display. We also convert from the user's local time on datetime input. Nothing fancy or unexpected there, really. And hey, the tzinfo gem supports DST, so we're good, right?

Well, PingMe is a scheduling system. It has a scheduler daemon that's constantly checking to see which pings need to be sent out, then it creates outbound events for the dispatcher daemons to deliver. Never mind the terminology, the important thing here is that it's working in UTC. And that Rails is storing the timestamps in Postgres' default TIMESTAMP WITHOUT TIME ZONE data type. Here's an illustrative query:

  def lock_a_block(type_name)
    before = (Time.now.utc).to_s(:db)
    
    ActiveRecord::Base.connection.execute(
    <<-END_OF_SQL
      UPDATE events SET dispatcher = '#{@name}'
      WHERE id IN (
        SELECT e.id FROM 
          (( events e INNER JOIN targets t ON e.target_id = t.id )
          INNER JOIN pings p ON e.ping_id = p.id)
          INNER JOIN target_types tt ON t.target_type_id = tt.id
        WHERE 
          tt.const = '#{type_name}'
          AND 
          (
            (e.dt_when < '#{before}' AND e.status = '#{Event::STATUS_PENDING}')
            OR
            (e.retry_at < '#{before}' AND e.status = '#{Event::STATUS_RETRY}')
          )
          AND e.dispatcher IS NULL
          AND t.activated_at IS NOT NULL
          AND (p.is_done = 'f' OR p.is_done IS NULL)
          AND (p.deleted_at IS NULL)
        ORDER BY
          e.dt_when ASC
        LIMIT #{@block_size}
        );
    END_OF_SQL
    )
  end

So the app is providing a UTC timestamp for the before variable, and the timestamps are in UTC in the database. What happens when DST begins or ends? Nothing changes. Everything is sent at the set time, for UTC. So a ping set for 5pm EST was stored at 12:00 UTC, and when 5pm shifts an hour for EDT, that ping is still stored at 12:00 UTC and will be sent either an hour early or an hour late, depending on the circumstance.

The only way we could break this up to work off the time zone setting on the user model is to execute separate queries for all of our users all the time joining against their timezone. Ridiculous! And following Geoff's notion of things above, it's just not a clean solution -- storing the ping's time without the time zone is decidedly *inaccurate*. I hate to say it.

I think the best solution is not to store in UTC here, but to store as a timestamp with time zone. I realize that sounds like an impure solution, but it's not: PostgreSQL actually stores the data in UTC and can do all sorts of magical conversions for us. We could still use the code above and work in proper UTC, but any DST on the timezone would be respected:

WHERE ... e.dt_when AT TIME ZONE 'UTC' < '#{before}'

And that is why I hope Geoff changes his mind, because we do need TimeWithZone as a data type in Rails, or perhaps a col definition that will provide a TimeWithZone instead of Time objects:

col.datetime :col_name, :with_time_zone => true

As an aside, we don't leave PingMe users to hang when DST rolls around, we update the relevant time stamps via SQL. But I would like to get us to a better solution. Being able to store TimeWithZone would do just the thing.

We're big Twitter fans, and for quite some time we've wanted to allow PingMe users to interact with our service through Twitter. A number of folks have asked for it and, selfishly, we also wanted this capability for ourselves. Now, if you're a twittaholic, you can access all mobile PingMe functionality straight through the service you know and love.

On a side note, this feature also introduces an alternate way to use SMS messages with PingMe. Previously, in order to send and receive SMS messages with our users (in a cost-efficient way) PingMe would send reminders through a provider's sms-to-email gateways. These gateway's are provided by most (but unfortunately not all) cell carriers, and some people pay an extra fee for the capability. Now that we've added support Twitter, you can use their service as a universal transport for SMS or even Instant Messaging.

In this post I'll step you through the process of using PingMe with Twitter. Various details about how messages to create and update pings in this way are covered in older articles and our help section, so I'm going to stick to just the bits pertinent to Twitter.

To get started I'll assume you already have a Twitter account, and are logged in to their web site. To be able to send get messages from PingMe on twitter, you have to "follow" the PingMe twitter account, 'gpm', like so:

Now that you've got your twitter account set up, log in to PingMe and click that "Add target" link under the Targets listing on the right side of the page. There's not much to do but select 'Twitter' from the type drop-down and then enter your username on twitter:

Note that if you skipped the first step, where you follow gpm on Twitter, you'll get an error in that last step.

Now that you've got a Twitter target for your account, you can have your pings sent there just like any other target:

So let's try creating a ping from Twitter. We'll use the web interface for our example, but keep in mind that you can do this in all the ways you interact with Twitter - including from your phone or IM. What we are doing is sending a direct message to gpm ('d gpm ...') that contains a create-ping instruction.

The syntax is very similar to the format we use for creating remote pings from e-mail and SMS. The '5h' tells ping me "five hours", the "p:10" tells us to pester you every ten minutes until you respond with 'done', and the 't:t' tells us that you want this ping sent to your Twitter targets (click here for info on setting default targets). The only new trick here is the addition of the '+' sign. Since Twitter is conversational, we have to have a way of distinguishing your create messages from your updates, so after the direct message bit, you begin your ping creation with a plus sign.

Updates, as you might have expected, are simpler. The only caveat is that we don't necessarily know which ping you are trying to update, so we assume it is the most recent one sent to you. You can send an update like this:

That message will update the most recent ping sent to you to be sent again in 30 minutes.

Here at Zetetic we do a lot of logging, and a lot of looking at logs. In particular, we have a couple of daemon processes implemented in Rails for PingMe that handle our message queueing and parsing of incoming messages (when you reply to your pings or create new ones by remote). If you have any experience with message queueing systems, you'll recall that these are not easy things to maintain, and require access to really good diagnostics. And if you are familiar with Rails you'll recall that there are no time-stamps prepended to the log messages, making it very difficult sometimes to track down what happened *when*.

I did a quick bit of poking around and came across this fantastic article with a number of tips in terms of logging. Their solution for the issue of formatting the messages (so that you can have timestamps) is to subclass Logger, and instantiate that.

However, we have our own Loggers all over the place, in our daemons, they use the Logger class which has been patched by Rails to have that timestamp-less format. What we do from there is replace the Rails logger instance with our own (there are a few reasons for this, having to do with forking processes, resources, and the nature of daemons that I don't want to get into), which works beautifully:

logger = Logger.new("#{config[:log_dir]}/#{config[:name]}.log", 'daily')
unless config[:log_level].blank?
  begin
    logger.level = Logger.const_get(config[:log_level])
  rescue StandardError => e
    logger.level = Logger::INFO
    logger.error "An exception occurred while setting log level to #{config[:log_level]}, setting to INFO.  Exception: #{e.message}"
  end
else
  logger.level = Logger::INFO
end

logger.info "Initialized log @ #{Time.now.utc} with log_level #{logger.level.to_s}"
logger.info "Starting up Dispatcher #{config[:name]}..."

# over-ride the active record logger (which would be closed now)
ActiveRecord::Base.logger = logger
ActionMailer::Base.logger = logger

I really don't feel like subclassing Logger, I just want to adjust the default behavior, since we're using the same loggers everywhere. So I opened up config/environment.rb, and at the bottom of it, added this:

# re-patch logger to restore format patched out by Rails
class Logger
  def format_message(severity, timestamp, program, message)
    "#{timestamp.to_formatted_s(:db)} #{program}: [#{severity}] #{message}\n"
  end
end

Works fantastic! Thanks to Maintainable Software for their post.

I almost forgot to mention in today's earlier post that there are two more new features that many people have requested.

If you open up a ping, and click on the drop-down menu for repeat settings you'll see two new ones, 'Mon-Fri' and 'Bi-weekly':

Mon-Fri pings don't execute over the weekend. Now I won't get my 7am "Wake up!" ping when I'm trying to sleep in, and that makes me very happy.

Bi-weekly pings are just that, they repeat every two weeks instead of every week, on the same day of the week as the initial date you set.

Login to PingMe now to start taking advantage of the work-week feature ;-)

Your feedback is driving PingMe's development, so please keep these feature requests coming, and send them to support@gopingme.com. Some of them spur us on to implementing features we want for ourselves, while others are just awesome ideas that we can't help but adopt.

Update:

You can also specify these new intervals when creating pings by remote. Use the value 'f' with the repeating flag to set a Mon-Fri ping, like this:

h 4pm hit the road before traffic gets bad r:f

Similarly, use the value 'b' for bi-weekly pings:

h9 m45 project status meeting r: b

Last night we pushed a new version of PingMe out so now is a good time to mention some feature requests we've very happily implemented based on your feedback over the last few weeks.

In an update a couple of weeks ago we added two new flags you can provide while creating Pings by remote. This went out on our mailing list, but not here and it isn't in the Help page yet, so I'd like to recap:

Specify the 'c' tag to turn Require Confirmation on or off on a Ping, like so:

1d h6p Call your mother c:y
-or-
1d h6p Call your mother c:n

You could also type out yes or no if you are so inclined. Going hand in hand with that addition a couple of weeks ago is the new Always Confirm Pings setting on the user profile that we rolled out last night. Many of you wanted to Require Confirmation to always be enabled on your pings (we did, too), so we added this flag to your profile to do just that:

That's really handy for us, it means we don't even have to use the new c: flag to enable Require Confirmation when we create pings by remote.

Last night's update also included one more oft-requested feature that we also wanted to have, which makes creating pings on the web and remotely a bit quicker: default targets.

New users will find that the initial E-mail target we create for them is set to be a default. You can make any target one of your defaults by editing it in the sidebar and checking the Default? flag:

One last change we made was a stability update. A couple of PingMe users have made the mistake of setting their profile e-mail account in Jott to their PingMe address, rather than setting up a contact called PingMe. This can cause a pretty nasty mail loop, so we implemented some fixes and checks to prevent that from ever happening again.

If you are having any trouble receiving your pings, or sending pings from Jott, send an e-mail to support@zetetic.net and we'll get you back on track.

Many PingMe users are really excited to be using Jott with PingMe. But it seems a few users that have recently joined the service are getting stuck on how to correctly specify when the Ping should be sent.

Human beings tend to be a lot better at learning how to write than computers are at learning to read. Therefore, when we were first designing a short-hand for creating Pings remotely we had two priorities for the "time" language:

1. Make it very quick and terse so that users would have to do as little typing as possible from their mobile phone
2. Make it easy for PingMe's computers to reliably parse

PingMe's time codes do exactly that - they allow you to specify the minimum amount of information necessary to convey a date and time. 'h12 m30 call mom t:m' is a lot less typing, than say, 'remind me to call mom at twelve thirty on my cell phone', and it's 100 times easier for our computers to process.

When we introduced the ability to create Pings by speaking via the Jott service (with which are totally unaffiliated), we found that the time codes fit well there too. Numbers and and interval codes (days, weeks, hours, minutes) were reliably transcribed by the system in our testing with a very low failure rate. Our primary concern is always reliability, so we didn't focus as much on a natural language translation: you basically speak the exact same time codes that you would type into an email or SMS interface to PingMe.

Sure, it would be really great to be able to say "remind me to call mom at twelve thirty on my cell phone," or something similar. That will take more work on our part, and we need to evaluate how critical a feature that is for our users. To put it in perspective, we've processed thousands of time-coded messages, and many users are already comfortable with this interface.

In the meantime, lets review how to specify a Ping remotely and go over what you need to say to get the job done when you create a Ping by voice using Jott (or via SMS for that matter).

Messages to create a Ping must begin with some information telling PingMe when to send the reminder. The time codes we use for this information are documented in our help here, but here's a basic example:

h12 m30 Call mom

The full set of intervals and their synonyms follows:

  'min'    => [ 'm', 'min', 'mins', 'minute', 'minutes' ],
  'hour'   => [ 'h', 'hr', 'hrs', 'hour', 'hours' ],
  'day'    => [ 'd', 'ds', 'day', 'days' ],
  'week'   => [ 'w', 'ws', 'wk', 'wks', 'week', 'weeks' ],
  'month'  => [ 'mo', 'mos', 'mon', 'mons', 'month', 'months' ],
  'year'   => [ 'y', 'yr', 'yrs', 'year', 'years' ]

When PingMe receives a message, it evaluates it left to right. It sees the h12pm and sets the ping to go off at 12pm today. Note that if 12pm today has already passed, this Ping will not be sent! Then PingMe sees the 'm30' and updates the Ping to be sent at 12:30pm. Another form of time code is to specify a number and then the interval name, like this:

1mon 1d h6p Call mom

In this case PingMe sees the '1mon' and recognizes it to mean '1 month from now.' Then '1d' pushes the time ahead one more day. Notice that we've switched back to the other format to set the time to 6pm! PingMe long-hand for this message would be:

1 month 1 day hour 6pm Call mom

So this statement means 'call mom in one month and one day at six o'clock PM' The example is perhaps a little contrived but gives you an idea of how flexible the time codes are.

Also notice that you can abbreviate the interval names, but you don't have to. When speaking into Jott you should use their full names, which seems to help ensure accuracy, like this:

one month day fourteen call mom

If you have feedback on the time codes or Jott interfaces please let us know.

Last night we saw a really big surge in Japanese users on PingMe. Irashaimasen! We wondered what was up and saw that this article on a Japanese website might have been the cause.

We were really curious to find out what the article says so we popped it into Google Translate, but their Japanese support is still a little wonky. Can anyone help us out? Is it a decent review?

We've been hard at work on "PingMe":http://www.gopingme.com over the past few weeks. Here are a few improvements to the way you can create Pings remotely from your mobile devices. *Create Pings with Email Subject* Hardcore Blackberry and Email wizards everywhere often type quick notes directly in the subject line of a message. It saves keystrokes and lets you easily find messages by subject in your message list view. Now you can use this messaging style to create Pings by placing the entire create message in the subject and leaving the body part empty. !http://www.zetetic.net/files/ping-in-subject.png! *New Flags for Remote Pings* We've added some new flags that let you specify confirmation mode and set tags when creating a Ping by e-mail or from your Phone: # Specifying *c:y* or *c:yes* after your message will require confirmation on a Ping # Tag your mobile Pings using the *z:* flag, followed by a list of tags, separated by spaces, commas, or both Here's an example that would create a tagged ping that requires confirmation: bq. 1d h5p Call Dunder-Miflin, close paper deal c:y z: followup calls sales

Timezones are a difficult thing, whether you're traveling or communicating across them. While the jet-lag will always catch up with you, PingMe won't let you down when you're crossing timezones and borders.

In the Profile screen you can select your Timezone from a drop-down list. It provides many different timezones to account for the small adjustments to time that are made by various local governments around the world.

Let's suppose that you are traveling from the East coast of the United States to the West coast. Changing your Timezone to 'US - Pacific' will change the display time of your current pings by three hours. They will still be sent at the same time, which is to say that a Ping set for 5pm Eastern time will be sent at 2pm Pacific time. This is great if you don't want to miss that conference call with your boss at 1pm Eastern time.

But what if you're not just making a short stay, what if you're moving? Or maybe you have daily Pings (like "wake up," which seems to be very popular) that you want to go off at the same time of day in the new time zone? We've got you covered.

When you select a new Timezone on the profile screen, a check box appears labeled "Move all pings to this timezone?" If you select this option and save your profile, you'll find a Ping that was scheduled to be sent at 5pm Eastern time will now be sent at 5pm Pacific time.