The big news in the data and cloud worlds last week was Amazon's introduction of DynamoDB, a NoSQL database service for internet scale applications:
"Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. With a few clicks in the AWS Management Console, customers can launch a new Amazon DynamoDB database table, scale up or down their request capacity for the table without downtime or performance degradation, and gain visibility into resource utilization and performance metrics."
The DynamoDB announcement, along with implications for enterprises and the NoSQL ecosystem, has been well covered. Instead of focusing on the final product, I wanted to take a bit of a tangent, inspired by Werner Vogels great post on the genesis of DynamoDB.
In his post, Vogels recounts the history of NoSQL at Amazon, covering Dynamo (Dynamo PDF), SimpleDB, use cases, successes, shortcomings, lessons and the current evolution to DynamoDB.
Besides the obvious, I took away three lessons from the DynamoDB evolution that every software architect should embrace:
1. Technology adopters, in this case developers, will forgo superior processing fit for management simplicity:
"As we spoke to many senior engineers and service owners, we saw a clear pattern start to emerge in their explanations of why they didn't adopt Dynamo more broadly: while Dynamo gave them a system that met their reliability, performance, and scalability needs, it did nothing to reduce the operational complexity of running large database systems. Since they were responsible for running their own Dynamo installations, they had to become experts on the various components running in multiple data centers. Also, they needed to make complex tradeoff decisions between consistency, performance, and reliability. This operational complexity was a barrier that kept them from adopting Dynamo.
During this period, several other systems appeared in the Amazon ecosystem that did meet their requirements for simplified operational complexity, notably Amazon S3 and Amazon SimpleDB. These were built as managed web services that eliminated the operational complexity of managing systems while still providing extremely high durability. Amazon engineers preferred to use these services instead of managing their own databases like Dynamo, even though Dynamo's functionality was better aligned with their applications’ needs."
2. Architects of service-oriented business applications need to stratify business capabilities to a variety of data persistence mechanisms. The question is not SQL vs. NoSQL. The questions are transaction volume, velocity and value:
"This non-relational, or NoSQL, database was targeted at use cases that were core to the Amazon ecommerce operation, such as the shopping cart and session service. Any downtime or performance degradation in these services has an immediate financial impact and their fault-tolerance and performance requirements for their data systems are very strict. These services also require the ability to scale infrastructure incrementally to accommodate growth in request rates or dataset sizes. Another important requirement for Dynamo was predictability. This is not just predictability of median performance and latency, but also at the end of the distribution (the 99.9th percentile), so we could provide acceptable performance for virtually every customer."
3. For the enterprise software architect audience, we need to adopt characteristics from the product architect's mindset:
Read Vogel's post, for DynamoDB and more.
