Socializing Eventual Consistency

I've followed Werner's blog for a while now and his latest on eventual consistency is one of my favorite posts. I recently advocated for an eventual consistency approach for a service offering we're putting together and lost. Part of that I think was that the technology to build an eventually-consistent client could really use some improvement and I'll talk to that in a later post. More at play though was the general issue of pushing people's comfort zones with new concepts.

I was somewhat conflicted after the discussion because the Agile part of my brain wanted to believe that the simplest thing we could do was flow with out skill set and just write to an SQL database. The systems experience in me was screaming the whole way at building a tightly-coupled, highly intertwined system when staring at a green field that didn't really have highly-structured relational data.

As always, I could have done a better job in making my case, focusing more on the technical and less on the emotional, but I sure wish I would have had read Werner's blog before the discussion. Two things really stood out:

"Eventually consistency is not some esoteric property of extreme distributed systems. Many modern RDBMS systems that provide primary-backup reliability implement their replication techniques in both synchronous and asynchronous modes."

When I talk to people about things like eventual consistency, I generally get a response along the lines of "that's cool, but we're not Amazon". No, most companies aren't, but I'm not sure that we shouldn't leverage all the brain power and learning from Amazon to our benefit. You can only make buggy whips for so long while thought leaders speed ahead.

The other piece that stood out was:

"Inconsistency can be tolerated for two reasons: for improving read and write performance under highly concurrent conditions and for handling partition cases where a majority model would render part of the system unavailable even though the nodes are up and running."

Customers are always going to want the C, A and P from commercial software, but experience tells us that they are not all possible. If you buy Werner's arguments, the P is impossible for distributed systems. Most business-focused users I've encountered, when really pressed, almost always choose the A over the C.

When trying to socialize concepts like these, there is truly an art to helping people out of their comfort zones. I'm not as good at it as many people, posts like Werner's go a long way towards helping with that.

Now, if they can just fix that SimpleDB API...