Showing posts with label distributed database. Show all posts
Showing posts with label distributed database. Show all posts
Thursday, December 12, 2019
Wednesday, August 9, 2017
Replication Factor in Couchbase
One of the core requirement for Distributed DBs is to be as High Availability as possible. What this literally means is that, even if node/nodes go down the DB should function (on its own or with minimum intervention). This is possible only if there are backup copies of the data.
Replication factor controls number of replicas or backup of an item/data/document stored in a DB. The general rule is to have replica for each node which can fail in the cluster.
Let's check how one of the famous NoSQL distributed Db handles Replication Factor.
Couchbase
Default replication factor is 1 in Couchbase (if it's enabled). Drop down field (as shown below) has only 3 values i.e. 1, 2 and 3. Practically, it doesn't make sense to have replication factor more than 3 no matter how large your cluster is.
So even if you have only one node and enable replicas then in the same node there will be two copies of the same data (one original and one backup). Once you add more nodes to the cluster original and replicas will get re-distributed automatically.
Recommendation:
Number of Nodes <= 5 - RF = 1
5 <= Number Of Nodes <= 10 - RF =2
Number of Nodes > 10 - RF = 3
Number of nodes mentioned above is only for data nodes if you are using Multi Dimensional Scaling. If you are not using MDS then also above rule should hold good.
In the event of failure we can fail over (manually or automatically) to replicas.
- In a 5 node cluster with 1 replica. If one node goes down cluster can fail it over. Now before the the failed node is up, what if another node goes down ? You are out of luck. You will have to add another node to the cluster.
- After a node goes down and it's failed over try to replace that node ASAP and perform rebalance. Rebalance creates the replica copies if there are enough nodes available.
References
Saturday, March 25, 2017
Understanding Consistency in Distributed Databases
In Distributed DBs, Consistency is evaluated based on CAP theorem. This post doesn't go into CP vs AP debate of CAP, but just address what consistency means in Distributed DBs. It starts with ideal scenario and then concludes with practically possible consistency level.
In perfect world, a distributed database system will be considered Consistent if an update is applied to all nodes at same logical time, i.e. it's instantaneous and global just like updating a DB with single node.
As shown above, client makes call to insert data and then DB initiates insertion to all data nodes (primary and replica) at same time. And because all nodes are similar so they take exactly same time to insert record. Is it possible ?
You might argue that, all nodes should be able to perform the operation in same time; so it's possible.
If you still feel this is possible - think hard. We are talking about distributed DB, which means they are at different places. All, sorts of things are possible while communicating - hardware might fail, network connectivity might fail or slow down.
How you synchronize, if update on one node fails ?
Perfect world is impossible for distributed DB !
In Practical world, a distributed database system is considered consistent if it applies update to all nodes in a synchronized way and then confirms to the client that update is successful. So, if there is one primary nodes and 2 replicas. Then DB should make sure that replicas get updated along with primary nodes. This can be done, but it comes at a cost.
Practical world is strongly consistent, but latency goes for a toss.
Practical world is not going to help us either!
We have only one option, store/update data on only one (primary/master) node and then update other nodes asynchronously.
Let's see how it's going to work when next request comes to read the data which is still undergoing replication to other nodes in asynchronous manner. In this case, system will behave depending on how the request gets handled. Below are possibilities:
In perfect world, a distributed database system will be considered Consistent if an update is applied to all nodes at same logical time, i.e. it's instantaneous and global just like updating a DB with single node.
You might argue that, all nodes should be able to perform the operation in same time; so it's possible.
If you still feel this is possible - think hard. We are talking about distributed DB, which means they are at different places. All, sorts of things are possible while communicating - hardware might fail, network connectivity might fail or slow down.
How you synchronize, if update on one node fails ?
Perfect world is impossible for distributed DB !
In Practical world, a distributed database system is considered consistent if it applies update to all nodes in a synchronized way and then confirms to the client that update is successful. So, if there is one primary nodes and 2 replicas. Then DB should make sure that replicas get updated along with primary nodes. This can be done, but it comes at a cost.
In practical world, DB will ensure that all nodes are updated before confirming to the client that operation is successful. In worst case, each node can take it's own sweet time to ensure that write/update is successful.
This can clearly mean that time, t1 >> t2 + t3 + t4
Host, might be unreachable for some time or it can go down. And unless, all nodes are updated; request will not be successful.
Your update is going to be slow and worst case really slow which will increase latency numbers and reduce throughput. Such system will not scale.
This can clearly mean that time, t1 >> t2 + t3 + t4
Host, might be unreachable for some time or it can go down. And unless, all nodes are updated; request will not be successful.
Your update is going to be slow and worst case really slow which will increase latency numbers and reduce throughput. Such system will not scale.
Practical world is strongly consistent, but latency goes for a toss.
Practical world is not going to help us either!
Consistency in Distributed DBs
Quite obviously above approaches (which will bring strong consistency to system) is not practical, and it wont be exaggeration to say Impossible!. So, what option we have ?We have only one option, store/update data on only one (primary/master) node and then update other nodes asynchronously.
Let's see how it's going to work when next request comes to read the data which is still undergoing replication to other nodes in asynchronous manner. In this case, system will behave depending on how the request gets handled. Below are possibilities:
- Any node can return value. Now, if the primary node returns the value then it will be most recent but if a node which is yet to get the update will return stale or old value. This is Week Consistency. This approach is used to achieve high availability.
- Only the primary node handles the request so you get the latest value. If DB can ensure that the data is returned from the primary node then client will get most recent value. This is Eventual Consistency.
Subscribe to:
Posts (Atom)