RAM sizing Data Node of Couchbase Cluster

This post talks about finding out how much RAM does your Couchbase cluster needs for holding your Data (in RAM)! 

RAM Calculator 

RAM is one of the most crucial areas to size correctly. Cached documents allow the reads to be served at low latency and high throughput.  Please note that, this doesn't not incorporate RAM requirement from the host/VM OS and other applications running along with Couchbase.

Enter below fields to estimate RAM -

Sample Document        (key)    (Value) 

This is required as document content length as well as ID length impacts RAM. Be mindful of the size aspect when deciding your key generation strategy. 

# Replicas                                        

Couchbase only supports upto 3 replicas. So enter either - 1, 2 or 3.

% Of Data you want to be in RAM  %

For best throughput you need to have all your documents in RAM i.e. 100% . This way any request will be served from RAM and there will be no IO.  In the field please enter only the value like 80, 100 etc. 

# Documents                                   

Number of documents in the cluster. When your application is starting from scratch then you can start with a number depending on the load of the application and then you need to evaluate it regularly and adjust your RAM quota if required. So, you can start with say 10000 or 1000000 documents. 

Type of Storage                                SSD        HDD

If storage is SSD then overhead % is 25 else it's 30%. SSD will bring better performance in disk throughput and latency. SSD storage will help improved performance if all data is not in the RAM. 

Couchbase Version                        < 2.1       2.1 or higher  

Size of meta data for 2.1 and higher versions is 56 bytes but for lower versions it's 64. 

High Water Mark                             %

If you want to use default value enter 85. 

If the amount of RAM used by documents reaches high water mark (upper threshold), both primary and replica documents are ejected until the memory usage reaches low Water Mark (lower threshold). 

                                                          

Based on the RAM requirement for the cluster, you can plan how many nodes are required. Another important aspect in deciding number of data nodes is how you expect your system to behave if 1, 2 or more nodes go down at the same time. This link, I have discussed about Replication factor and how it affects your system performance. So, take your call wisely!

The value got calculated as explained in the Couchbase link, here.
Reference for calculating document size is, here. 

--- happy sizing :)

Create Couchbase Cluster using Docker

This post walks you through steps to setup Couchbase cluster using Docker. Post expects that you already have Docker installed in your machine (Either of Linux, Mac or Windows)

Before starting the real work, make sure that Docker daemon is up and running.

$ sudo docker info

If daemon is NOT up; you will have something like below on your console:
Cannot connect to the Docker daemon. Is the docker daemon running on this host ?

If daemon is up, then you will have the details like Containers, OS, CPU, OS Name, docker root directory etc printed on the console.

To start Docker daemon (ref)

$ sudo service docker start

Run Couchbase as Dockerized Container

$ sudo docker run -d --name cb1 couchbase

On some of the platforms (like Mac) you need to specify the port mapping as well.

$ sudo docker run -d --name cb1 -p 8091-8094:8091-8094 -p 11210:11210 couchbase:enterprise-4.6.3

Above command runs Couchbase in detached mode (-d) and name is cb1.

$ sudo docker inspect cb1

Above command list all metadata of the cluster in JSON format. One of the important details to look for is IP address (with key IPAddress) of the couchbase server.

To launch couchbase in your browser use URL: http://<<ip_address>>:8091

Lanuch URL and then customize/configure couchbase server. To make it usable, you need to provide details like bucket, RAM, services which you want to run (if it supports multi dimensional scaling). Also, check the radio button 'Start a New Cluster' on CONFIGURE SERVER page.

This way, you will be able to successfully configure your first node.

Create Couchbase Cluster

Above step successfully launched one instance of Couchbase. To create a cluster you need to have minimum two instances of couchbase. 

Create another instance with a different name cb2. And then launch url as discussed in above step. But, be careful this time to Join a cluster (instead of start a new cluster). In the table, give your first instance IP address and then username and password.

Once, the new node is joined to the previous node your cluster is created. As, a final step go ahead and Rebalance your Couchbase to distribute data among nodes. 

References
https://github.com/wsargent/docker-cheat-sheet#networks

---

happy learning !