Scalability - Getting hang of Seconds

This post list some of the most important numbers which are important for engineers to do back of envelope calculations. This post, I have focussed on seconds. If you hear someone telling that his service gets 1 million hits a day; don't get bogged down with the numbers, it just means he gets 10 requests per second.

Seconds

# Seconds in a day = 86400  (=24*60*60)

                                = 0.85*10^5

                                = 10^5

                                = 0.1 million

# Seconds in a month = 2592000 (=30*86400)

                                     =  2.5*10^6

                                     = 2.5 millions

If an online site gets 10 million hits per day then it means on an average it gets 100 requests/sec.

If an online site gets 10 million hits per month then it means on an average it gets 4 requests/sec.

# Seconds in a year = 31104000 (=12*259200)

                                  = 31 millions

                                  = Pie * 10^7

   If we treat a year as 365.25 days then also, # seconds in a year would be 3,155,7600 which would approximate to 31 million. 

# Seconds in a century = 3,155,760,000 seconds (considering 1 year = 365.25 days)

                                       = 3.15 billions 

                                     

Nanocentury is 1 billionth of a century. So, a nanocentry = 3.15 seconds.

i.e. Pie seconds are there in a nano century. This is also known as Duff's Rule. 

                    

Scaling your Data Storage

The traditional approach of storing all data is - store it in a single data source and handle all read-write operations in a more centralized fashion. In this approach, you keep adding more memory and/or processing power (or buy a bigger and more powerful machine) as the load on your system grows. Well, this works for most cases, but a time will come when this doesn't scale. Upgrading hardware doesn't always help to achieve scalability.

So obvious solution to this problem is - start partitioning your data. 
This post covers possible partitioning approaches.

Let's take an example of an online system where you are storing user profile details and photographs (of course, along with other details). 

Dividing Schema / Scale Up

One way to partition data is - store profile detail on one server and photographs on other.  This way only specific read and write queries are sent and hence scaling is possible. This approach of partitioning is known as vertical partitioning. This approach basically divides your schema. So if you want to fetch complete data of a given user, you need to fetch data from two different data sources. 

Replicating Schema / Sharding / Scale Out

In this approach, you replicate the schema and then decide what data goes where. So all instances are exactly the same.  In this approach, profile details, as well as the photograph of a user, will be in a single instance.

Advantages:

• If an instance goes down, only some of the users will get affected (High Availability), so your system overall doesn't get impacted. 
• There is no master-slave thing here so we can perform write in parallel. Writing is a major bottleneck for many systems. 
• We can load balance our web server and access shard instances on different paths, this leads to faster queries. 
• Data which are used together are stored together. This leads to data denormalization, but need to keep in mind that, it doesn't mean that data is not separated logically (so we are still storing profile and photographs logically separate). So no complex join, data is read and written in one operation. 
• Segregating data in smaller shards helps in performance, as it can remain in cache. It also makes managing data easier, fast backup and restore. 

http://highscalability.com/unorthodox-approach-database-design-coming-shard
https://www.citusdata.com/blog/2017/08/09/principles-of-sharding-for-relational-databases/

Containerize your application to achieve Scale

This post talks in general about Containers; their evolution and contribution in scaling systems.

Once upon a time, applications used to run on servers configured on bare mettle sitting in companies own data centers. Provisioning used to take anywhere from few days to few weeks. Then came Virtual Machines which use hardware visualization to provide isolation. They take time in minutes to create as they require significance resource.  Then finally; here, comes a brand new guy in the race, which takes 300 ms to couple of seconds to bootstrap a new instance; yes I am talking about containers. They don't use hardware virtualization. They interface directly with the host's linux kernel.

Managing VMs at scale is not easy.  In-fact, I find difficult to manage even couple of VMs :D So just imagine how difficult it would be for companies like Google and Amazon which operate at internet scale.

Two features which has been part of Linux Kernel since 2007 are cgroups and namespaces. Engineers at Google started exploring process isolation using these kernel features (to manage and scale their millions of computing units). This eventually resulted in what we know today as containers. Containers inherently are light weight and that makes them super flexible and fast. If containers even think of misbehaving, they can easily be replaced by another brand new container because the cost of doing so is not high at all. This means, they need to be run in a managed and well guarded environment. Their small footprint help in using them for specific purpose and they can easily be scheduled and re-arranged/load balanced. 

So one thing is quite clear, Containers are not brand new product or technology. They use existing features of OS. 

With containers the actual problem of making every component of a system resilient and bullet proof doesn’t hold good. This seems contradictory - we want to make systems more resilient but containers themselves are very fragile. This means any component deployed in them automatically becomes non-reliable. 
We can design our system with assumption that containers are fragile. If any instance failed - just mark it bad, replace it with a new instance. With containers the real hard problems are not isolation but orchestration and scheduling.

Read more in details on Containers vs VMs

Containers are also described as jail which guards the inmates to make sure that they behave themselves. Currently, one of the most popular container is Docker. And at the same time there are tools available to manage or orchestrate them (one of the most popular one is Kubernetes from Google).

Happy learning!!!