Microsoft ADAM provides a nice LDIF export tool, roughly equivalent to ldapsearch, called ldifde. However, the ADAM directory itself tracks a number of internal attributes that will cause a subsequent import of a generated LDIF to fail. In order to get a “clean” export, you need to selectively omit, via the -o command line flag, those operational attributes that you’re not interested in exporting (line breaks inserted for readability):

ldifde -f c:\people.ldif 
  -d "ou=people,dc=xyz,dc=com" 
  -s localhost 
  -t 389 
  -r "(objectclass=*)" 
  -o   "whenCreated,whenChanged,uSNCreated,
          uSNChanged,name,objectGUID,badPwdCount,
          badPasswordTime,pwdLastSet,objectSid,objectCategory,
          dSCorePropagationData,lastLogonTimestamp,
          distinguishedName,instanceType,lockoutTime"

The output generated by the command can now be cleanly imported into another ldap directory, or into a separate ADAM instance using a simple import:

ldifde -i -f c:\people.ldif -s localhost -t 389

Matt Flynn, in a post about AD password sync, asks:

What if a Virtual Directory could pass authentication requests to Active Directory the way that ADAM does? … Would this be useful functionality in a virtual directory? Is it technically feasible?

+1 for virtual directory pass-through authentication.

It’s definitely technically feasible and works very well to drive consolidation of authentication services. From past experience it’s one of the most powerful benefits of virtual directory technology. In fact, this feature was key to the value proposition and purchasing decision for several of the prominent deployments I’ve worked on. It was also one of the key topics we discussed at the DIDW 2005 Virtual directory panel.

Yesterday we discovered 100+ notifications for spam messages contributed through an online form by a malicious bot. We usually get a few of these per day at identicentric, because of this blog and various other unauthenticated forms, but the volume has never been enough to warrant decisive action. Nevertheless, Friday night’s activity “stepped over a line” and, much to our chagrin, spam continued to pour in over the course of Saturday morning at a rate of 15-20 per hour.

There are several established approaches to battling form spam. Some techniques requiring a user to enter in random characters displayed in a embedded image on the page. Others rely on logging IP addresses on form load, so that the processing script can reject bulk form submissions. Some attempt to use mod-rewrite to block form spam based on missing or specific Referrers or known blacklisted IP segments, with mixed results.

We wanted a dead-simple, general purpose solution that could be used to block spam on any form submissions, without dependencies on the back-end processor. Conceptually, mod-rewrite seemed like a nice fit because it could be implemented on Unix or windows (using ISAPIRewrite), and it was completely externalized from the form-backing application. Yet, the referrer and IP filtering techniques were unsuitable as they could result in long rewrite configurations, frequent ongoing maintenance, or incompatibility with many personal-firewall packages.

Our solution wound up being very simple, and involved setting a cookie using JavaScript that could be detected using mod-rewrite. It relies on the fact that spam-bots are dumb, not cookie-aware, and certainly aren’t JavaScript aware.
Here’s how it works.

Start off by creating a small .js file and including it in the page with the form. Expose a single function called setFormAllowCookie(), or something similar. This function, when called, will set a browser cookie named “formallowed” to a value of “true”.

function setFormAllowCookie() {
  var cookieName = “formallowed”;
  var cookieValue = “true”;
  document.cookie= cookieName + “=” + escape(cookieValue) + “; path=/”;
  return true;
}

Include the .js file into the page with the form. This is easily embedded in practically any html page. Next, add an onload oronsubmit to the bodytag or formtag respectively that calls setFormAllowCookie().

The final step is to configure a rewrite rule that redirects form submissions to an error page if the cookie is not present in the request, like this (show protecting WordPress comments):

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_URI} /wp-comments-post.php
RewriteCond %{HTTP_COOKIE} !formallowed=true
RewriteRule (.*) http://blog.xyz.com/error.html
</IfModule>

Pros:

• This solution should continue to work until form-scrapers become cookie and JavaScript aware.
• This approach does not introduce dependencies on the form processing application.

Cons:

• Requires JavaScript and Cookies, potentially interfering with a subset of form submitters.
• Might not work against bots that are manually configured to attack a site, as a human could easily figure out the appropriate cookie to set.

It’s a judgement call as to whether the pros outweigh the cons with this approach, with the answer depending largely on the form’s target user base. In our minds the results speak for themselves: it took about 15 minutes to implement this approach to stop the initial barrage of spam and, according to logs, has operated with a 100% success rate against subsequent attempts.

Sean O’Neill from Sun points out some very valid reasons against using email addresses as unique identifiers within identity systems. I agree with him on all points, except for one:

So the recommendation still remains to utilize a numeric value or alpha/numeric value for UID and put up with user’s complaints they are not easy to remember.

Even within highly secure environments user perceptions can be very important. Customer facing applications, high-volume ordering systems, business partner extranets, and even large scale identity deployments within the enterprise are all faced with the challenges of balancing good data practices with user experience. There is no doubt that changing unique identifiers is a Bad Thing™, largely because they are used to map between different systems. However, playing the devils advocate, exposing poorly chosen UIDs to end-users can cause a wide range of problems including increased help-desk traffic, reduced usage of shared credential management services, and even the creation of duplicate user registrations.

Luckily, there is a middle ground made possible by separating the concept of the Username from that of the unique identifier (although they will remain interconnected entities at some level). First, at provisioning time, each Identity must be assigned a globally unique, persistent unique identifier. This is by no means a new concept, and it is often referred to as a GUID, so we’ll use that term here as well. In a properly implemented system this GUID will never change over the life of an identity. Next, each identity should be assigned, by some means, a friendly, easy to remember Username for the purpose of authentication.

The key to success with this relatively simple approach lies not within the separation of identifiers, but in how they are used. Basically, applications, databases, services or resources that reference the identity should always use the GUID. Period. The only entity in the entire universe that should ever reference the Username is the human authenticating to the system. After credential validation, the authentication system can simply map the GUID to provide a unique identifier to other resources.

Consider how this works in practice. Lets say you have an central authentication system using a popular web access management platform. Each user has an identity record in the central service. Each user also has access to one or more applications that have been integrated with the central service. Each of these applications has its own database back-end, auditing functions, and other services that require a UID. As Sean points out, when the Username is the UID, the entire identity system is fragile – a name change, typo, or marriage can break the mapping between the authentication service and the applications.

Now reconsider this situation when the Username and UID are separate. Jane Doe logs into her applications with the Username jdoe. Then the authentication service maps that back to her GUID, 09103510 (or whatever…), and passes that value back to the application she’s using. Now all of databases, services, transactions, historical audit logs, etc. are all tied to GUID. If Jane marries
John Tailor, none of the backend systems change. She can log in tomorrow with jtailor and her applications won’t even realize a difference. This same model extends nicely into more flexible systems too, as people could just as easily select their own usernames.

By decoupling the Username from the UID, an identity system can enjoy the benefits of strict unique identifier assignment along side complete flexibility in username assignment. Best of all it can be implemented with most (although not all) common authentication technologies like JAAS, Web plug-in style access management systems, PAM, SAML, LDAP, etc., with assignment driven by your choice of provisioning tools. While its not appropriate for every scenario, its definitely worth examining as an option while establishing standards for identifier assignment.

Thanks to Johannes Ernst for pointing out this gem by Peter Gutmann about the broken state of XML Security. Johannes is right – the article brings up a set of excellent points.

I often find myself asking the same question the original author Mr. Gutmann poses in his article – why insist on doing it the hard way? Within the context of web services, I’ve seen developers with little security training go down the road of WS-Security solutions that invariably break or are rendered useless by poor integration. Most often the project in question has three straightforward requirements:

1. authenticate the service provider
2. ensure that the message is not modified in transit by third parties without detection, and
3. ensure that the message is not readable in transit by third parties.

The requirements are dead simple – but its so easy and tempting to get lost in the details of the XML-way-to-do-things that the obvious solution is never used. If you’re using SOAP for web services (which most people are), and you’re adhering to the WS-I Basic Profile 1.0 (which most people should be), then you’re using HTTP as a transport. The fact is that SSL/TLS provides a practical method of security communications between service providers and consumers over HTTP, including XML web services. It has practically universal platform support, good performance characteristics, encrypts all data sent between parties, prevents data tampering, supports authentication of both servers and clients, and there is widespread availability of acceleration hardware. Plus, the security is externalized, so it drastically reduces required integration time, and places no dependencies on developers for “proper” implementation. It’s a very pragmatic and elegant solution to several web service system security problems.

Don’t get me wrong. WSS is great and has a wide array of very legitimate uses, and I’m not downplaying its importance in the web service arena. Nor am I’m trying to imply that SSL/TLS is a security panacea – especially if your XML transport is not HTTP. But good system security is about choices – and often the simplest choice that meets a particular requirements is the best choice. So if your web service implementation has straightforward security requirements, my advice is to take a close hard look at SSL/TLS.