Codebook Sync Replication

Codebook's Sync feature is designed to allow you to copy your passwords and other data to all your devices, and then exchange any changes made to those records over time. That includes additions, deletions, and changes to the same records.

As you edit your data, Codebook records these changes into what we call changesets, and stores them in the local encrypted database. These changesets can be used to replay the changes you've made on another device. When it's time to sync, Codebook writes groups of these changes out to small encrypted SQLCipher database files we call shards that are then exchanged with the sync service in use. Each is encrypted using your Sync Key. Codebook retrieves all changesets it hasn't "seen" yet from the sync service and applies those changes locally.

Changing How Sync Operates

There are times where it may be useful to change how Codebook Sync operates, for instance when you want to wipe out all data and restore from another device or cloud service, or overwrite all data on another device or cloud service with known good data. These options are available under the hidden Operations menu. It's best to contact our support team for guidance before using these advanced features.

When you have a record in Codebook on two of your devices, let's say an entry for your Yahoo! email address and password, and you modify this entry in Codebook on both your iPhone and your Mac, and then you sync, what happens? This is called a conflict. To resolve such conflicts on sync, Codebook considers the most recent change to a record to be authoritative, and that's the one that "wins" in order to bring both devices up to date.

Thus, if you were to update the password field in Codebook on your iPhone, and then update the same password to a different value on your Mac, and then sync over WiFi, both copies of Codebook will recognize that the change from your Mac is the most recent (and therefore authoritative), and that's the value that will be in place on both copies after sync completes.

What happens if you add a new field to the record on both devices, for instance a Website URL? Both additional fields will be kept: the new fields are considered unique, thus they don't actually conflict, so after sync they will both be present on both devices.

This change tracking and conflict resolution strategy allows you to store your Codebook data on multiple devices (not just two, as in the examples above), and bring them back to an updated state via sync with the most recent changes, even if one device is allowed to go "stale" for a long period of time.

Due to the last-update-wins strategy for conflict resolution, it is possible to run into an edge case involving deletes. If you add a new entry to a particular category on one device, and then delete that whole category from your other device, and run sync, the category will be deleted on both devices. However, the new entry you added has not been lost! It's still in the database, but there is no category to associate it with and display it in the application. Thus, it's an "orphaned" record.

The same potential for orphaned records exists with entries and fields: if you add fields to an entry on one device, and then on the other device you delete the same entry containing those new fields, and sync them together, the entry in question will be deleted on both sides. However, the new fields still exist in the database, orphaned from the entry record they belong to.

To recover orphaned records and display them in the application, run the Integrity Check feature. Integrity Check will first check for any orphaned entries, and if they exist, it will create a new Category called Unfiled and store them there. Next it checks for orphaned fields, and if it finds any it will store them on an entry named Orphaned Fields, in the Unfiled category. Integrity Check will report if there were any orphaned records and how many there were when it completes.

Codebook has been around since 2008 and has provided a Sync feature for a long time. In Codebook 4 we updated Sync to separate user changes into discrete chunks of data that can be passed around on sync (rather than entire databases being exchanged and compared). The new system relies on every record having accurate "audit" timestamps that indicate when a record was modified. An old bug (since fixed) in Codebook 3 caused some databases to have some records that lacked a proper audit timestamp in the updated_at property. As one can imagine, this causes problems for the last-update-wins strategy Sync uses to exchange changes and resolve conflicts.

If you are seeing unreliable sync behavior this may be the cause. We have recently introduced some advanced Sync Operations features to help us deal with these scenarios, handling "bad" data in a reliable way to create a new "good" database. Please get in touch with our support team so we can advise you on their use.