There seems to be a bit of interest in the Ruby community lately in putting together handy libraries for creating Ruby daemons, so I thought I’d throw our own particular solution into the mix. Our solution intentionally avoids a lot of the common daemonizing tasks (like detaching from the terminal) so it might not be quite appropriate to call it daemonizing, but it fills our requirements nicely. (As an aside, check out Kenneth Kalmer’s new library daemon-kit for a nice actual-daemon implementation in Ruby.)

We originally tried coding things by hand using Daemons, and then modularizing our oft-repeated code to stay DRY, but we kept running into stability issues. Also, when you fork a Ruby daemon, particularly one that’s loaded up a large stack in memory, you end up at least briefly using twice as much memory. So, memory hogging and crashing needed to be avoided.

None of our daemons required advanced process handling, either. They were simple, single instance workers that had very particular tasks (like schedulers and message handlers for PingMe). We found ourselves asking, “do we really need to be forking child processes for this?”

The answer is no. Instead we are using a module we just pushed to Github that we call Looper. It allows us to take any bit of code or a class and run it as a kind of daemon. Or, more specifically, the class uses Looper to take advantage of its simple signal trapping, loop handling (including sleeping), and the class can rely on Looper to catch any unhandled exception, report it, and keep on keepin’ on.

The loopme method takes a block that it runs for you, handles sleeping between runs, and will catch any unhandled exceptions that bubble up. Thus, if you want to exit on a particular exception, you’ve got to rescue it in your code and set @run to false. It also sets up signal trapping so that signals TERM, INT and HUP will all cause the loop to end (I’d be open to changing this behavior or adding additional signal responses if anybody has an interest).

Here’s an example “daemon” that uses looper to kick it:

require 'looper'
require 'twitter'

class DoSomething
  include Looper

  def initialize(config)
    @run = true
    # do config stuff, etc...
    @sleep = config[:sleep].nil? ? 60 : config[:sleep]
  end # initialize

  def run
    loopme(@sleep) do
      begin
        # this is where the meat of your code goes...
        messages = twitter.direct_messages({:since => Time.now.strftime("%a, %d %b %Y %H:%M:%S %Z")})
      rescue Twitter::EpicFailure => e
        puts "bailing out, dude!" 
        # set run to false to put the kabosh on the next run
        @run = false
      end
    end
  end
end

# and here's how we kick it off:
DoSomething.new( { :sleep => 10 } ).run

Pretty simple, right? No coding up the signals, the sleeping, the global exception handler, etc. You can take a look at looper.rb to see exactly what it does for you. From here, starting and stopping our script is really easy, and can be re-used for each of our daemons. It boils down to starting it up with nohup (and script/runner because we like having our rails stack), and then killing it later with the PID:

$ nohup script/runner -e RAILS_ENV /path/to/DoSomething.rb

There’s no need to use Rails’ script/runner, you could just call ruby itself and save yourself a lot of memory ;-)

The following is an init script that we use to start and stop any of our Looper daemons:

#!/bin/bash
#
# chkconfig: 345 70 30 
# description: control script for daemons 
#

usage(){
  echo $"Usage: $0 {start|stop|restart} {daemon1|godzilla|voltron}"
}

# daemon name is required as second param
if [ "$2" = "" ]
  then 
    usage
    exit 5
fi

# RAILS_ENV?
if [ "$RAILS_ENV" = "" ]; then
  RAILS_ENV='development'
fi

# RAILS_DIR?
if [ "$RAILS_ENV" = "production" ]; then
  RAILS_DIR='/www/app/current'
else # assume development
  RAILS_DIR='/www/dev.app/current'
fi

PROGRAM="${RAILS_DIR}/daemons/${2}/looper.rb"
RUNNER="${RAILS_DIR}/script/runner"

LOG="/var/log/app/daemon_${RAILS_ENV}.log"

# if the daemon is missing, well, the daemon is missing!
test -f $PROGRAM || exit 5

# See how we were called
case "$1" in
  start)
  nohup $RUNNER -e $RAILS_ENV $PROGRAM >> $LOG 2>&1 &
    ;;
  stop)
    PID=`ps auxw | grep ${PROGRAM} | grep -v grep | awk '{ print $2 }'`
    echo "Stopping PID: ${PID}"
    kill $PID
    sleep 2
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
    usage
    exit 1
  ;;
esac

exit 0

Today’s dose of win:

“Alright!! It’s working against the production servers – monitoring 6 web apps across three servers. My boss is very happy. That means I am even happier. I have been working with Nagios now for roughly 4 months. The user login test was the final piece of this project. Thanks again for the help.”

How cool is that? We have a plugin for Nagios called ckFormLogin that monitors login processes for popular asset management systems (Oracle CoreID Access Manager, CA eTrust Siteminder, Sun Access Manager, &c.), and when people have a little trouble with it, we’re pretty happy to help them out. But rarely do you get cool feedback like that.

Nagios itself is a really useful and highly configurable piece of monitoring software. The interface is a bit Soviet, sure, but it works great, and it’s open source. So is the ckFormLogin plugin.

This post is probably not what you’d expect. But our time-tracker service Tempo has some great charting capabilities. Behold, the effects of a recent virus (of which I will spare you the horrific details) on my productivity:

Above is the Heads-Up-Display, at the top of the main screen. It reads pretty easily – my productivity was sent to bed!

Check out the utilization chart, which is a bit more instructive as to what happened, and when. That’s accessible by clicking the productivity percentage number box in the HUD, or just pulling it up from the Charts tab:

Brutal.

Utilization is a measurement concept in Tempo that is supposed to give you a ballpark idea how much of your time is accounted for, compared to how much time is generally available in a work-day. So, for every day in a report, the par for the day is considered to be 7 to 9 hours (which is a pretty common length for a workday), and that’s the horizontal, light-blue bar going across the graph between 7 and 9 on the Y-axis. The bars then show us a visual comparison of our utilization for the time periods. If I were to expand the date range enough, these would be grouped by week, then by month. Here’s what my utilization looks like since Nov 1st, 2008, until now:

So 2008 was a pretty productive year for us here at Zetetic. I can’t really go through it all right here and right now, but we’re quite excited about the new year, improvements we’ll be making to our existing products, and the new products and services we’re currently building.

And with that I’m going back to bed. Offers to send me comfort whisky, cheese burgers and nurses will not be turned away outright.

I guess it kinda makes sense that Tetris could interrupt the brain from developing PTSD, it definitely interrupted my childhood for a few solid months at a time: (h/t Tim F.)

These results support Holmes’s theory that Tetris can help to prevent PTSD flashbacks by occupying the brain’s energies during the narrow time window when traumatic memories are consolidated. …snip…

But EMDR is only used to treat PTSD and there are several ways of doing that. Holmes’s goal is more proactive – she wants to find ways of preventing the symptoms from appearing in the first place. There are a few potential options for doing that, from drugs to psychotherapy, but few can be delivered so quickly or cheaply as a quick game of Tetris on a handheld machine. The game has another big advantage in that it affects a person’s reactions to an event but not their actual memories of it – Holmes notes that they would feel relief, but their ability to, say, testify in court wouldn’t be diminished.

Yes, I just downloaded the version that’s available in the App Store. What an easy preventative to make available for people — pretty much everyone has a cell phone, especially troops in combat. Doesn’t take much processing power or graphics to run Tetris!

In the environment of one of our clients, we use Oracle’s Application Express product quite heavily for their identity and access reporting — we’ve got five full blown apps running in that framework at this point. Once in a while we hit a snag in development of an application and post the solution here.

Imagine you’ve got this asset management application for a very large organization, called Conglomo, which has thousands of franchises all over the world. The application holds files that users can see in a portal app. Basically, you upload files, you assign them to organizations (franchises, districts, regions, &c.) and users, and when a user logs in, she can see the files assigned to her or to any orgs in her hierarchy. From there you have a management screen for files, that is basically a report listing them out, and giving you some filters to narrow it down. Say you want to see all files linked to a particular org. The piece of data you need is really the org’s unique ID number, but users need a real way to search for a franchise named “Jack’s Conglomo Outlets Inc,” perhaps and in particular, the one in Bakersfield, California. They don’t want to have to look for org # 5003886, they don’t even know the number.

Well, the ready made Popoup Key LOV form element is just the thing. You drop in some SQL to generate the LOV, and voila you get a nice searchable list. Using a query in this way you can easily concatenate a bunch of fields about the organizations so that when the use is searching for a particular dealer, they have other factors to look at besides the name (and many of them are named quite similarly):

select 
 id || '-> ' || arnumber ||' '|| name ||' ('|| division ||' '|| type || ')' as d, 
 id as r
from   organization
order by 1

This gives us a Popup Key LOV that looks like this:

When you select an org, the form element drops the display name into that disabled text area, and in a hidden element it drops the key’s value:

<input type="hidden" name="p_t05" value="105" id="P5_ORG_HIDDENVALUE">

So in APEX-land, when we make a report, it’s nothing like what you see in other frameworks, really. We just have one big query that draws up our table. So when we add a new element above, we need to amend the WHERE clause on our query to include that criteria if it was specified. Here’s an example of what I mean:

SELECT
  f.id,
  f.title,
  f.description,
  f.file_name,
  to_char(f.created_at, 'dd-MON-yyyy HH:MM AM') as created_at
FROM 
  naap_files f 
WHERE

...SNIP...

AND ( :P5_ORG IS NULL OR f.id IN (
    SELECT nof.file_id FROM organization_files_join_table nof
    WHERE nof.organization_id = :P5_ORG
  )
)

So, in this simple case, we’ll get all files if :P5_ORG IS NULL (that’s our Popup Key LOV element in our form). If it’s not null, we limit the list of files to those that are listed in the join table as belonging to the specified Org ID number.

And this really works out great until you need to clear the field because you’re no longer interested in files belonging to Org 105. In the image above you can see a “Clear this field” link in the popup. That’s actually the null option in the elements configuration, you can see it here:

When you click the null option, “Clear this field,” you’ll see the disabled text element get cleared. However, it isn’t really NULL that is sent back up to the host when you submit the form next. The hidden field we saw before is actually set to the string 'undefined'.

<input type="hidden" name="p_t05" value="undefined" id="P5_ORG_HIDDENVALUE">

So, when you submit the form, you’re not sending an empty string, which APEX just turns into a NULL value, you’re literally getting the string undefined, which your query then binds as the org id number! And then no rows come back! FAIL! This may be fixed in later versions of APEX (we’re working in 3.0x), but I did some digging and it doesn’t look like it.

Problems sending an actual NULL value for a select list or some other multi-select element in APEX are nothing new, basically the value that is actually sent is '%null%', even though you specify nothing at all. There are a few workaround, but I usually drop an application computation into each of my apps that zaps these guys and terms them into actual NULL values. I found this solution somewhere inside the APEX forum, I didn’t come up with it myself, but it’s pretty much SOP for anyone doing this kinda work, a PL/SQL block that is run On Submit After Computations and Validation:

BEGIN
    FOR rItem IN
      ( SELECT ITEM_NAME
         FROM APEX_APPLICATION_PAGE_ITEMS
         WHERE APPLICATION_ID   = TO_NUMBER(:APP_ID)
           AND PAGE_ID          = TO_NUMBER(:APP_PAGE_ID)
           AND LOV_DISPLAY_NULL = 'Yes'
           AND LOV_DEFINITION   IS NOT NULL
           AND LOV_NULL_VALUE   IS NULL
      )
    LOOP
        IF V(rItem.ITEM_NAME) = '%null' || '%'
        THEN
            Apex_Util.set_session_state(rItem.ITEM_NAME, NULL);
        END IF;
    END LOOP;
END;

Since we have this in place, we can just update this bad boy to zap ‘undefined’ values as easily:

BEGIN
    FOR rItem IN
      ( SELECT ITEM_NAME
         FROM APEX_APPLICATION_PAGE_ITEMS
         WHERE APPLICATION_ID   = TO_NUMBER(:APP_ID)
           AND PAGE_ID          = TO_NUMBER(:APP_PAGE_ID)
           AND LOV_DISPLAY_NULL = 'Yes'
           AND LOV_DEFINITION   IS NOT NULL
           AND LOV_NULL_VALUE   IS NULL
      )
    LOOP
        IF (V(rItem.ITEM_NAME) = '%null' || '%') OR (V(rItem.ITEM_NAME) = 'undefined')
        THEN
            Apex_Util.set_session_state(rItem.ITEM_NAME, NULL);
        END IF;
    END LOOP;
END;

It’s a little odd that even in this fairly recent version of APEX you still have to jump a hoop or two to handle null value submissions, and even weirder that you get '%null%' in some cases and 'undefined' in others. Any web framework has to deal with this kinda thing, but a little consistency would be nice. Hope this is a good primer and introduction to the problem if you’re new to APEX. Although, I have to say that aside from a few weird quirks like this, APEX is absolutely awesome for a situation in which you need to throw together a webapp in a client’s environment and they aren’t interested in any “risky” new fangeled technology like Rails!

It’s hard to convince large companies to work with emerging platforms. There is always a perception of risk that makes tried-and-true platforms like Oracle APEX attractive. I still sometimes yearn to have used Rails for one of these projects, but honestly I can throw together these applications MUCH faster and with way less overhead. I may not have quite the flexibility I get with Rails, but this is all really PL/SQL running on a giant modified mod_plsql, and you can do all kinds of crazy stuff in PL/SQL.