We’re big Twitter fans, and for quite some time we’ve wanted to allow PingMe users to interact with our service through Twitter. A number of folks have asked for it and, selfishly, we also wanted this capability for ourselves. Now, if you’re a twittaholic, you can access all mobile PingMe functionality straight through the service you know and love.

On a side note, this feature also introduces an alternate way to use SMS messages with PingMe. Previously, in order to send and receive SMS messages with our users (in a cost-efficient way) PingMe would send reminders through a provider’s sms-to-email gateways. These gateway’s are provided by most (but unfortunately not all) cell carriers, and some people pay an extra fee for the capability. Now that we’ve added support Twitter, you can use their service as a universal transport for SMS or even Instant Messaging.

In this post I’ll step you through the process of using PingMe with Twitter. Various details about how messages to create and update pings in this way are covered in older articles and our help section, so I’m going to stick to just the bits pertinent to Twitter.

To get started I’ll assume you already have a Twitter account, and are logged in to their web site. To be able to send get messages from PingMe on twitter, you have to “follow” the PingMe twitter account, ‘gpm’, like so:

Now that you’ve got your twitter account set up, log in to PingMe and click that “Add target” link under the Targets listing on the right side of the page. There’s not much to do but select ‘Twitter’ from the type drop-down and then enter your username on twitter:

Note that if you skipped the first step, where you follow gpm on Twitter, you’ll get an error in that last step.

Now that you’ve got a Twitter target for your account, you can have your pings sent there just like any other target:

So let’s try creating a ping from Twitter. We’ll use the web interface for our example, but keep in mind that you can do this in all the ways you interact with Twitter – including from your phone or IM. What we are doing is sending a direct message to gpm (‘d gpm …’) that contains a create-ping instruction.

The syntax is very similar to the format we use for creating remote pings from e-mail and SMS. The ‘5h’ tells ping me “five hours”, the “p:10” tells us to pester you every ten minutes until you respond with ‘done’, and the ‘t:t’ tells us that you want this ping sent to your Twitter targets (click here for info on setting default targets). The only new trick here is the addition of the ‘+’ sign. Since Twitter is conversational, we have to have a way of distinguishing your create messages from your updates, so after the direct message bit, you begin your ping creation with a plus sign.

Updates, as you might have expected, are simpler. The only caveat is that we don’t necessarily know which ping you are trying to update, so we assume it is the most recent one sent to you. You can send an update like this:

That message will update the most recent ping sent to you to be sent again in 30 minutes.

Over the last few months we’ve put a lot of work into PingMe‘s scheduling system because it was necessary if we wanted to expand the service and make it more reliable. Before I get into what we are doing differently I’ll take a moment to describe the previous situation and our setup.

PingMe has a number of daemons – independent processes that are always running, scheduling pings, sending them out, and processing messages that you send to the service. These daemons are implemented in Ruby using Ruby On Rails. This allows them to be tightly coupled with the PingMe web application – the daemon processes and the webapp operate with the same model, which helps us keep the code pretty clean.

The ones of most concern are the dispatchers, the daemons who’s job it was to check for new pings to deliver, and then reschedule them for the next delivery (if necessary). Getting the concurrency right was rather tricky and involved some real nerding out in Postgres (our database engine of choice). Basically, the dispatchers had to do what’s called mutex locking in order to guarantee that different dispatchers would try to send out the same message. The locking code is a neat trick, btw, and it’s still in use, it’s served us well:

  def acquire_mutex
    ActiveRecord::Base.connection.execute(
    <<-END_OF_SQL
      LOCK mutex IN ACCESS EXCLUSIVE MODE;
    END_OF_SQL
    )
  end

Different database engines have different facilities for this sort of thing, but basically doing this within a transaction caused the other dispatchers to wait until the lock was released. What were they waiting for? A chance to grab a block of pings to dispatch.

Now the rescheduling of pings, and the scheduling of pings was honestly a not very clean thing to begin with. We had callbacks on the Ping model that would create the actual instances of an outbound message for delivery (we called these Events), and then the dispatchers would need to block those callbacks in certain situations to cause a reschedule. It worked, I don’t want to get into the details of it, but it had one particular problem:

Events are an instance of a Ping associated with a Target for delivery. The one dispatcher we were running would do it’s selection of events to deliver based on target-types. Once we created the new Twitter target type and added a new dispatcher that only handled that one target type (this was all in our dev environment), the daemon would conflict with the other dispatcher. Which ever daemon picked up a ping first for it’s target type and then marked the ping as done was basically preventing the other daemon for processing the ping for its other targets.

The solution was to implement a new daemon, that we called Scheduler, and to move all the rescheduling code into this one serial process. Once we stripped all that out of the Ping model and the dispatcher code, we had a much leaner and faster system. We can now run as many dispatchers as our memory allows and configure them to handle various target types.

Here at Zetetic we do a lot of logging, and a lot of looking at logs. In particular, we have a couple of daemon processes implemented in Rails for PingMe that handle our message queueing and parsing of incoming messages (when you reply to your pings or create new ones by remote). If you have any experience with message queueing systems, you’ll recall that these are not easy things to maintain, and require access to really good diagnostics. And if you are familiar with Rails you’ll recall that there are no time-stamps prepended to the log messages, making it very difficult sometimes to track down what happened when.

I did a quick bit of poking around and came across this fantastic article with a number of tips in terms of logging. Their solution for the issue of formatting the messages (so that you can have timestamps) is to subclass Logger, and instantiate that.

However, we have our own Loggers all over the place, in our daemons, they use the Logger class which has been patched by Rails to have that timestamp-less format. What we do from there is replace the Rails logger instance with our own (there are a few reasons for this, having to do with forking processes, resources, and the nature of daemons that I don’t want to get into), which works beautifully:

logger = Logger.new("#{config[:log_dir]}/#{config[:name]}.log", 'daily')
unless config[:log_level].blank?
  begin
    logger.level = Logger.const_get(config[:log_level])
  rescue StandardError => e
    logger.level = Logger::INFO
    logger.error "An exception occurred while setting log level to #{config[:log_level]}, setting to INFO.  Exception: #{e.message}"
  end
else
  logger.level = Logger::INFO
end

logger.info "Initialized log @ #{Time.now.utc} with log_level #{logger.level.to_s}"
logger.info "Starting up Dispatcher #{config[:name]}..."

# over-ride the active record logger (which would be closed now)
ActiveRecord::Base.logger = logger
ActionMailer::Base.logger = logger

I really don’t feel like subclassing Logger, I just want to adjust the default behavior, since we’re using the same loggers everywhere. So I opened up config/environment.rb, and at the bottom of it, added this:

# re-patch logger to restore format patched out by Rails
class Logger
  def format_message(severity, timestamp, program, message)
    "#{timestamp.to_formatted_s(:db)} #{program}: [#{severity}] #{message}\n"
  end
end

Works fantastic! Thanks to Maintainable Software for their post.

Update 25-APR-2008: This plugin has been updated for Rails 2.0. Update 06-JUN-2008: This plugin now includes acts_as_union, and we moved the repository to GitHub.

When we started integrating simple social networking features into PingMe we wanted to easily represent a bi-directional relationship between users in the system. When a user signs up for PingMe they can invite another user to join them. Once an invite is accepted, the users become mutual friends, or contacts in PingMe parlance, and can send Pings to each other.

Most importantly, we wanted the relationship to be bidirectional -when Jack is a friend of Jane then Jane should alsobe a friend of Jack.

Unfortunately we quickly realized that thismodel was not going to be so easy. The usual way of representing this type of network relationship using ActiveRecord is with an intermediate HABTM join, or with a self-referential has_many :through association. For example one might define a simple person model and then a join table tostore the friendship relation:

create_table :people, :force => true do |t|
  t.column :name, :string
end
  create_table :friends, {:id => false} do |t|
  t.column :person_id, :integer, :null => false
  t.column :person_id_friend, :integer, :null => false      # target of the relationship
end

The problem is that this model requires two rows in the intermediate table to make a relationship bi-directional.

jane = Person.create(:name => 'Jane')
jack = Person.create(:name => 'Jack')
jane.friends << jack
jane.friends.include?(jack)    =>  true   # Jack is Janes friend
jack.friends.include?(jane)    =>  false  # Jane is NOT Jack's friend

In short, you must explicitly define the reverse relation in order for this to work.

jack.friends << jane
jack.friends.include?(jane)    =>  true  # now they're buds

This can be implemented in a fairly DRY way using association callbacks as documented in Rails Recipes, but things start to get ugly when you want to express the relationship through a "proper" join model (like for an Invite) using has_many :through.

create_table :invites do |t|
  t.column :person_id, :integer, :null => false           # source of the relationship
  t.column :person_id_friend, :integer, :null => false    #  target of the relationship
  t.column :code, :string                                 # random invitation code
  t.column :message, :text                                # invitation message
  t.column :is_accepted, :boolean
  t.column :accepted_at, :timestamp                       # when did they accept?
end

In this case creating a reverse relationship is much more complex and could require the duplication of multiple values, making the data model decidedly non-DRY.

Enter acts_as_network

acts_as_network is a plugin that we developed for PingMe to resolve some of these issues. It drives the social networking features of the site. It'sintended to simplify the definition and storage of reciprocal relationships between entities using ActiveRecord by exposing a "network" of 2-wayconnections.

What makes it special is that it does this in a DRY way using only a single record in an intermediate has_and_belongs_to_many join table or has_many :through join model. There is no redundancy, and you need only one instance of an association or join model to represent both directions of the relationship. Consider this more desirable implementation:

class Invite <ActiveRecord>::Base
  belongs_to :person                    # the source of the invite
  belongs_to :person_target,            # the target of the invite
  :class_name => 'Person',
  :foreign_key => 'person_id_target'
end

class Person <ActiveRecord>::Base
  acts_as_network :friends, :through => :invites, :conditions => ["is_accepted = ?", true]
end

In this case acts_as_network implicitly defines five new properties on the Person model

person.invites_out        # has_many invites originating from me to others
person.invites_in         # has_many invites originating from others to me
person.friends_out        # has_many friends :through outbound accepted invites from me to others
person.friends_in         # has_many friends :through inbound accepted invites from others to me
person.friends            # the union of the two friend sets
- all people who I have invited all the people who have invited me and
Now...
# Jane invites Jack to be friends
invite = Invite.create(:person => jane, :person_target => jack, :message => "let's be friends!")
jane.friends.include?(jack)    =>  false   # Jack is not yet Jane's friend
jack.friends.include?(jane)    =>  false   # Jane is not yet Jack's friend either

invite.is_accepted = true  # Now Jack accepts the invite
invite.save and jane.reload and jack.reload

jane.friends.include?(jack)    =>  true   # Jack is Janes friend now
jack.friends.include?(jane)    =>  true   # Jane is also Jacks friend

So much cleaner!

Most of this magic is actually accomplished with a UnionCollection class that provides useful application-space functionalityfor emulating set unions across ActiveRecord collections. Once initialized, the UnionCollection itself will act as an array containing all of the records from each of its member sets, but its more interesting feature is that it will intelligently forward ActiveRecord method calls likefind, find_all_by_*, etc. to its member sets.

Check it out

Further documentation is available online, and you can easily install acts_as_network as a plugin to try it out:

% script/plugin install git://github.com/sjlombardo/acts_as_network.git
% rake doc:plugins

Please check it out and let us know what you think.

Last night we pushed a new version of PingMe out so now is a good time to mention some feature requests we’ve very happily implemented based on your feedback over the last few weeks.

In an update a couple of weeks ago we added two new flags you can provide while creating Pings by remote. This went out on our mailing list, but not here and it isn’t in the Help page yet, so I’d like to recap:

Specify the ‘c’ tag to turn Require Confirmation on or off on a Ping, like so:

1d h6p Call your mother c:y
-or-
1d h6p Call your mother c:n

You could also type out yes or no if you are so inclined. Going hand in hand with that addition a couple of weeks ago is the new Always Confirm Pings setting on the user profile that we rolled out last night. Many of you wanted to Require Confirmation to always be enabled on your pings (we did, too), so we added this flag to your profile to do just that:

That’s really handy for us, it means we don’t even have to use the new c: flag to enable Require Confirmation when we create pings by remote.

Last night’s update also included one more oft-requested feature that we also wanted to have, which makes creating pings on the web and remotely a bit quicker: default targets.

New users will find that the initial E-mail target we create for them is set to be a default. You can make any target one of your defaults by editing it in the sidebar and checking the Default? flag:

One last change we made was a stability update. A couple of PingMe users have made the mistake of setting their profile e-mail account in Jott to their PingMe address, rather than setting up a contact called PingMe. This can cause a pretty nasty mail loop, so we implemented some fixes and checks to prevent that from ever happening again.

If you are having any trouble receiving your pings, or sending pings from Jott, send an e-mail to support@zetetic.net and we’ll get you back on track.