Who's Resetting Passwords?

2012-03-15 11:55:35 -0400

Monitoring a large Active Directory and Windows environment with Zetetic Combine can turn up some interesting things... in this case we took a look at a Combine beta program participant's administrative password reset activity over the last few weeks.

It's a colorful chart, and a long one! Over 50 distinct support and administrative folks have been helping locked-out users around the globe. Of course, it's always good policy to make sure the people exercising these rights are doing so properly... all the more reason to implement good security log collection and analysis policies.

(The names have been obscured to protect the innocent, as well as the mostly-innocent.)

Forefront Identity Manager + Lotus Notes

2012-03-14 13:13:20 -0400

At Zetetic we do a lot of work for our clients with Microsoft's identity management stack: Active Directory, AD Federation Services, Forefront Identity Manager 2010, and its predecessors Identity Lifecycle Manager 2007 and Microsoft Identity Integration Server 2003.  FIM is a great tool, but--perhaps unsurprisingly--its connector to Lotus Notes has always come up a bit short because:

  • It can only talk to the Domino Names and Address Book with predefined fields and behaviors
  • It's limited to exactly one running Notes management agent (MA) at a time
  • It has a tendency to crash the FIM Synchronization Service, usually requiring a reboot, when the Notes API gets out of whack
  • Microsoft only officially supports a few legacy versions of the Notes client and server

(It's also impossible to prevent the standard Notes MA from complaining about replication-save conflicts or incomplete records, which then foul up the delta import process, but I digress...)

Having worked for years in the Notes and Domino development arena before switching full-time to C#, it seemed only natural to build a MA that bridges the gap between the many, many real-world business uses of Notes databases and FIM / ILM, while at the same time keeping the large, complex Notes client API out of the FIM server process.  

Our solution to this problem is a two-parter:

1. A backend Windows service that manages all native access to the Notes API, and exposes a facade of Notes sessions, databases, documents, and the Notes Certification Authority, via .NET 2 TCP Remoting, and,

2. An Extensible MA for FIM / ILM that communicates with the .NET Remoting service to implement FIM's import and export operations.

The result is a FIM agent that uses only .NET managed code, moving all the native Notes stuff into a separate process--or even better, a separate host.  Upgrading or moving FIM then no longer means dealing with the Notes client, or whether the 64-bit FIM process will have trouble with 32-bit Notes, or even trying to find the older editions of Notes that the FIM product supports.

We think this solution is an ideal fit for any FIM / ILM customer with Notes business databases or identity infrastructure in their environment because it greatly extends FIM's reach to read and update Notes databases, while also providing a serious boost in reliability.  It's tremendously useful for application and messaging migration, easy to set up, and we provide full support to get your FIM to Notes and Domino integration configured and running.

For more information: Zetetic Management Agent for IBM Lotus Notes and Domino

SQLCipher for Android 1.1 Released

2012-03-13 17:11:53 -0400

A new build of SQLCipher for Android is now available, binaries can be found here. With this latest release we've added support on the Android platform spanning versions 2.1 through 4.0.3. Due to our expanded support, a new ICU zip file entitled icudt46l.zip is being included which replaces the older icudt44l.zip. A tutorial covering application integration of SQLCipher for Android and building the source can be found here.

Point In Time Recovery From Backup using PostgreSQL Continuous Archving

2012-03-09 13:55:44 -0500

Last year, around this time we, published details of how we setup PostgresSQL backups using Continuous Archiving. This technique involves continuous WAL archives combined with a daily "baseline" backup script:

Backups are only half the picture, though; the ability to easily restore data is just as critical. Unfortunately, the recovery process, as documented, can be a bit complex. Here is an distilled, exact step-by-step procedure we use to recover from backup.


This assumes that hot-backups are enabled in postgres. Ensure the following are setup in postgresql.conf on the box where the backups are located

wal_level = hot_standby                 # minimal, archive, or hot_standby
archive_mode = on               # allows archiving to be done
archive_command = 'cp -n %p /backup/postgresql/%f'

Understanding the Backup Folder Structures

The PostgreSQL WAL-based backup approach is structured on a base backup, and then a set of transaction logs that can be applied on it.

Via the backup script, daily baselines are stored in the PostgreSQL backup folder, /backup/pg_backups, in folders like this:

drwxr-xr-x    2 106      113           4096 Aug 24 05:15 2011-08-23.09:00:01
drwxr-xr-x    2 106      113           4096 Aug 25 05:16 2011-08-24.09:00:01
drwxr-xr-x    2 106      113           4096 Aug 26 05:16 2011-08-25.09:00:01
drwxr-xr-x    2 106      113           4096 Aug 27 05:16 2011-08-26.09:00:01
drwxr-xr-x    2 106      113           4096 Aug 27 05:19 2011-08-27.09:00:01

In this case, the folder is named after the current date and the base backup. For example, the folder named 2011-08-25.09:00:01 would contain the base backup from 9 AM UTC, along with the WAL log files archived between 2011-08-25.09:00:01 and 2011-08-26.09:00:01. Thus, to restore to any point in that time you should use the files from that backup folder:

  1. base.tar.bz2 - full backup of the postgresql data folder (i.e. /var/lib/postgresql/9.x/main)
  2. pit-wal.tar.bz2 - point in time logs captured for transactions active while the base snapshot was being taken
  3. full-wal.tar.bz2 - the archive of all transaction logs present between the base backup and the following day's base backup

In the case of the current date, the folder corresponding to today's date will contain the base backup (base.tar.bz2) and the point in time logs pit-wal.tar.bz2. However, the full-wal.tar.bz2 file will not be present because the day's logs have not been archived yet. Instead, the current day's archive logs are stored in the archive logs directory, /backup/postgresql.

There are two options for data restoration:

  1. restore up to the last backup point in the current date
  2. restore to a previous day's backups

The process between these two is basically the same, the only difference is whether the WAL logs are extracted from the full-wal.tar.bz2 or taken from /backup/postgresql

Step 1. Copy data backup

Copy all the files from the appropriate backup folder to your home directory. The following assumes you have the archive files in the home directory of ~postgres (substitute as necessary).

In order to perform the restore, we'll create a temporary directory in which to stage the files.

cd ~postgres
mkdir restore
cd restore

Step 2. Extract base data and PIT logs

tar xjf ~postgres/base.tar.bz2
tar xjf ~postgres/pit-wal.tar.bz2
cd backup/postgresql

Step 3. Extract WAL data

  • If you are restoring from a previous day
    tar xjf ~postgres/full-wal.tar.bz2
  • If you are restoring from the current day
    cp /path/to/today/backup/postgresql/* .

Step 4. Shutdown Postgres

# shutdown postgres
sudo su -
service postgresql stop

Step 4. Restore main data directory

Start by moving into the base version directory. Move the existing main folder out of the way, then move the restored base back into place

cd /var/lib/postgresql/9.x/
mv main main.bak
mv ~postgres/restore/var/lib/postgresql/9.x/main main
cd main

Step 5. Create a recovery.conf

Now that the base backup is restored, you must to tell PostgreSQL how to apply the recovery procedure. First, create file called recovery.conf in the data directory, /var/lib/postgresql/9.x/main/recovery.conf. The contents should minimally include the following line, adjusted to the location of the WAL files.

restore_command = 'cp /home/postgres/restore/backup/postgresql/%f %p'

If you only want to apply transactions up to a specific point in time, for instance to recover to the middle of a specific day, add the recovery_target_time (note the time stamp will be relative to UTC):

recovery_target_time = '2011-08-27 04:00:00.0'

Step 6. Start PostgreSQL

service postgresql start

When PostgreSQL has finished restoring the files the recovery.conf file will be renamed to recovery.done.

Note: Information and additional details from:


SQLCipher in The Busy Coder's Guide to Advanced Android Development

2012-03-06 16:10:18 -0500

The latest version of the Busy Coder's Guide to Advanced Android Development, published by CommonsWare, has recently been released. Chapter 21, Advanced Databases, is dedicated to database techniques like encryption. SQLCipher for Android is featured prominently along with a tutorial and discussion of proper integration techniques.

If you've been looking for a good book on advanced Android development topics that includes SQLCipher database encryption you should definitely check it out!