> For many idempotent operations, it seems likely the most fault-tolerant and latency-optimizing pattern is to ask multiple endpoints concurrently - take the first to respond, and discard or terminate the duplicates (and if one fails, so what?). Obviously this isn't cheap; sufficient bus/network and endpoint capacity is necessary - but isn't it the natural progression of a trend?
That's called latency leveling. It's actually done (at least on writes) by the different open source Dynamo implementations.
In addition to idempotence, unless your requests are being serialized through a single gateway (i.e., you're taking advantage of the TCP ordering guarantee), this also requires commutativity.
Don't get me wrong, it's a great approach and has many advantages (and I'll heartily endorse it for many applications) but there are a few drawbacks:
1. When you do want read read-your-writes consistency even under failures, you have to use quorum protocols. Again, quorum protocols are great, but they have drawbacks: you have to wait for the slowest node to finish, you need to aggressively increase replication to achieve read-your-writes and maintain fault tolerance (when |Writes| + |Reads| > replication factor, you can only survive the failure of 1 node with replication factor is 3).
2. Getting de-duplicated commit log with even a partial ordering to ship to another application (or to another instance of the storage system, e.g., in a different datacenter) is more difficult. Although I suppose you could, e.g., notify the other application that a change has taken place and have the other application _itself_ make (batched?) requests for the full after image (as opposed to just the individual mutation) of the data.
Either way, there's no write and wrong, just different approaches with separate trade offs.
[Edit: just read your bio and noticed you're from Amazon/AWS: A far more heterogenous environment than Google. I wonder how the new kinds of applications Google is building are influencing the infrastructure trade offs underneath.]
That's called latency leveling. It's actually done (at least on writes) by the different open source Dynamo implementations.
In addition to idempotence, unless your requests are being serialized through a single gateway (i.e., you're taking advantage of the TCP ordering guarantee), this also requires commutativity.
Don't get me wrong, it's a great approach and has many advantages (and I'll heartily endorse it for many applications) but there are a few drawbacks:
1. When you do want read read-your-writes consistency even under failures, you have to use quorum protocols. Again, quorum protocols are great, but they have drawbacks: you have to wait for the slowest node to finish, you need to aggressively increase replication to achieve read-your-writes and maintain fault tolerance (when |Writes| + |Reads| > replication factor, you can only survive the failure of 1 node with replication factor is 3).
2. Getting de-duplicated commit log with even a partial ordering to ship to another application (or to another instance of the storage system, e.g., in a different datacenter) is more difficult. Although I suppose you could, e.g., notify the other application that a change has taken place and have the other application _itself_ make (batched?) requests for the full after image (as opposed to just the individual mutation) of the data.
Either way, there's no write and wrong, just different approaches with separate trade offs.
[Edit: just read your bio and noticed you're from Amazon/AWS: A far more heterogenous environment than Google. I wonder how the new kinds of applications Google is building are influencing the infrastructure trade offs underneath.]