The Cost of Concurrency

Concurrency is not free!

Modern libraries provide a wonderful abstraction for programmers, so doing certain task concurrently or asynchronously is quite trivial. It is as simple as instantiating an object and calling few methods on it, and you are done! These libraries are abstracted in such a way that they don't even remind to programmers that you are going to deal with threads. And this is where the lazy programmer can take things for granted.

You need to process 100 task, create 50 threads.
     Collection<Task> task = fetchTasks();   //from somewhere
     int numberOfThreads = 50;
    obj.executeConcurrently(tasks, numberOfThreads);
In object oriented world, all it takes is a method call. 

To understand the cost of concurrency, let's take a step back and ask yourself how is it implemented? It is implemented through locks. Locks provide mutual exclusion and ensure that the visibility of change occurs in an ordered manner. 

Locks are expensive because they require arbitration when contended. This arbitration is achieved by a context switch at the OS level which will suspend threads waiting for lock until it is released. Context switch might cause performance penalty as OS might decide to do some other housekeeping job and so will lose the cached instruction and data. This is even more evident in multicore CPUs where each core has its own cache.  In the worst case, this might cause latency equivalent to that of an I/O operation. 

Another aspect of concurrency is managing the lifecycle of threads. OS does the dirty job of creating threads and managing them on behalf of your platform (or runtime environment). There are certain limits on the number of threads which can be created at the system level. So definitely, proper thoughts should be given on how many threads are required to accomplish a job.

Don't blindly decide to execute task concurrently!

Concurrency or Thread Model of Java

Thread Model in Java is built around shared memory and Locking!

Concurrency basically means two or more tasks happen in parallel and they compete to access a resource.  In the object-oriented world, the resource would be an object which could abstract a database, file, socket, network connection etc. In the concurrent environment, multiple threads try to get hold of the same resource. Locks are used to ensuring consistency in case there is a possibility of concurrent execution of a piece of code. Let's cover these aspects briefly:

Concurrent Execution is about

1. Mutual Exclusion or Mutex
2. Memory Consistency or Visibility of change

Mutual Exclusion

Mutual exclusion ensures that at a given point of time only one thread can modify a resource.  If you write an algorithm which guarantees that a given resource can be modified by (only) one thread then mutual exclusion is not required.  

Visibility of Change

This is about propagating changes to all threads. If a thread modifies the value of a resource and then (right after that) another thread wants to read the same then the thread model should ensure that read thread/task gets the updated value. 

The most costly operation in a concurrent environment contends write access. Write access to a resource, by multiple threads, requires expensive and complex coordination. Both read as well as write requires that all changes are made visible to other threads. 

Locks

Locks provide mutual exclusion and ensure that visibility of change is guaranteed (Java implements locks using the synchronized keyword which can be applied on a code block or method).

Read about the cost of locks, here. 

Related Post: http://geekrai.blogspot.in/2014/02/atomic-operations-in-java.html

Case Study: http://martinfowler.com/articles/lmax.html
References: Java Memory Model

Atomic operations in Java

Before jumping to the topic;  let's first understand the keyword, Atom. Wikipedia defines Atom as something that can not be divided further (i.e. uncuttable or indivisible).

Java and other modern programming languages use the term atomicity for operations/methods, which gets executed as one unit. An atomic operation is either fully done or not done, there is no intermediate state.  

       
       public class SharedStorage{
             private int a = 5;

             int increment(){

                    a++

               }
        }

Method, increment() in above class is one of the most trivial operations possible in Java (or any language). Is it atomic ? 

No, it is NOT.
The method is NOT indivisible as it consists of sub-operations (read the number and then increment it by one). So at any given time, it could be in either state (i.e. reading value or incrementing it by one). 

Importance of Atomic Operations

As we saw, even one of the simplest method is not atomic. The legitimate question is, is it really important?

If your application has a single thread of execution, it doesn't make any difference. Single threaded applications are sequential and they execute in the given order. So practically, the question of atomicity doesn't arise. 

Atomicity starts playing a role only if there is a possibility that an operation which modifies some shared data (or state) can be called by multiple threads at the same time. Assume that, two threads call increment() method of above class at the same time. What would be the value of a after completion of both threads? 

Final value of a could be 6 or 7. Let's understand this -
t1.start();
t2.start();

Even if thread, t2 starts after thread, t1 in your source code; JVM doesn't guarantee the same behavior at the time of execution. Sequential executions don't hold good anymore here. 

Java uses shared memory for communication between two threads. At the time of execution, both threads could read value of a at same time, hence, the incremented value will be 6. Or at the different time or on a different platform it a could have value as 7. This is a matter of concern!

In this post, I will be discussing how Java helps you to make operations atomic and also about some of the inbuilt atomic classes.

Important learning is; atomicity arises only when two or more threads can access a shared resource. 

Atomic Operations

We have seen earlier, even trivial operations like increment are not atomic in Java. Does it mean that no operation is atomic? Certainly NOT. Java ensures that some basic operations are always atomic:

• Read and write for primitive variables (except long and double)
• Read and write for reference variables 
• Read and write for any variable (including long and double) and references declared as volatile

Java 7 specification clearly states that read or write to a non-volatile long or double is treated as two separate actions one for each 32 bit half.  Thus read and write on long/double are not atomic. Also, restriction of 32 bit is not applicable for references. So read and write on all references are atomic, irrespective of the size. 

Java also provides a second mechanism to make any shared attribute atomic by declaring it volatile [Java 7 Spec]. If a field is declared as volatile Java Memory Model ensures that all threads see a consistent value for the variable. 

Let's consider non-trivial operations which are commonplace in any normal application. We have discussed already, the increment method in SharedStorage class is NOT atomic.
How do you handle such operations in a concurrent environment?
How can we ensure that even if multiple threads invoke the methods, it remains in a consistent state?

Java provides another way to make an operation atomic.

 public class SharedStorage {  
      private int counter = 5;  //need to make it volatile
      public int getCounter(){  
           return counter;  
      }  
      public synchronized void increment(){  
           counter++;  
      }  
 }  

If you declare any method as synchronized, it means only one thread can invoke it at a time. This makes the operation atomic implicitly ( and makes thread-safe as well).

There is still a problem with above class ?
Assume that thread, t1 calls increment method, and in mean time another thread, t2 calls get method to read the value of counter. Which value will be read by thread t2 ; 5 or 6 ? Answer to this question is still uncertain; you can get any value.

Synchronizing increment method has solved one part of the problem. But it has not made the overall state of program consistent in a concurrent environment.

There are two alternatives to fix this :

1. Either declare counter attribute as volatile. OR
2. Synchronize the getCounter() method. 

You might question, why do we need to declare counter as volatile? We discussed earlier that read on primitives (except long and double) is atomic.
getCounter() method would have been atomic in the absence of synchronized increment method. We are forced to declare counter as volatile only due to synchronized increment method. To make a class fully thread safe all operations which are reading or modifying the state needs to be synchronized.

Java Provided Atomic APIs

Java has rich set of API for making life easier for programmers. So if you just need a atomic counter variable, you can use inbuilt class, instead of creating a new class altogether.
The java.util.concurrent.atomic package provides such classes. You can use AtomicInteger, AtomicLong etc as per your requirement instead of doing it from scratch.

Thread-local variable in Java

Instances of a class maintain their own state through non-static data attributes. So, if you create 100 instances of Student class, they all have their own copy of name, age etc. Like this, life continues for objects and it's all cool!

Along with objects, Java has another level of abstraction (more lower level than objects); threads. The obvious question is, how are you going to manage state among multiple threads?  

In below code snippet, local variable, state can't keep value specific to each thread.

 sampleMethod(){  
  String state = "default";  
  Thread t1 = new Thread(){  };  //thread 1  
  Thread t2 = new Thread(){  };  //thread 2
 }  

Literally,  thread-local means, something which is local to a thread. One way to keep a variable local to a thread is;  inherit Thread class and add state.  And create unique objects for each thread. This solution can only work in trivial situations; and in some cases it might not be even feasible. Definitely there is a need for a cleaner approach.

 public class MyThread extends Thread{  
     String state;  
     public void run() { ... }   
 }   

java.lang.ThreadLocal<T> class

Java provides a specific class (since version 1.2) named as ThreadLocal, which allows you to associate a per-thread value with a value-holding object. This object provides get and set accessor methods that maintain a separate copy of the value for each thread that uses it. Below is a simplest example to illustrate the point. 

1:  public class ThreadLocalDemo {  
2:       private static ThreadLocal<String> tl = new ThreadLocal<String>();  
3:       public static void main(String[] args) {  
4:            tl.set("Main Thread");  
5:            Runnable r = new Runnable(){  
6:                 public void run(){  
7:                      tl.set("Sub Thread");  
8:                      System.out.println("Thread Local in Sub Thread :"+ tl.get());  
9:                      tl.remove();  
10:                 }                 
11:            };  
12:            System.out.println("Thread Local in Main :"+ tl.get());  
13:            Thread t = new Thread(r);  
14:            t.start();  
15:            tl.remove();  
16:       }  
17:  }  

Output:

Thread Local in Main : Main Thread

Thread Local in Sub Thread :Sub Thread

Note that, in the above class there are two threads (JVM runs this program in a separate thread and calls the main method and then there is a second thread (as t) created by the main method). The output clearly confirms that values stored in ThreadLocal object by two threads are preserved without any external synchronization. So, irrespective of the number of threads, ThreadLocal will create and maintain one unique storage for each thread. Method, get is used to return most recent value passed to set method from the currently executing thread. Conceptually, ThreadLocal can be visualized as a Map which stores value for each thread. (i.e. ThreadLocal<T>  equivalent to  Map<Thread, T>).

Let's go through a non-trivial example. Below is a ThreadLocalHolder class, which creates five threads for displaying thread-local behavior. Inner class, MyThreadLocal manages all methods of ThreadLocal class. Method, initialValue() is used to set an initial value.

1:  import java.util.Random;  
2:  public class ThreadLocalHolder implements Runnable {  
3:       private static Random rand = new Random(21);  
4:       class MyThreadLocal {  
5:            private ThreadLocal<Integer> tl = new ThreadLocal<Integer>() {  
6:                 protected Integer initialValue() {  
7:                      return rand.nextInt(100);  
8:                 }  
9:            };  
10:            public Integer getValue() {  
11:                 return tl.get();  
12:            }  
13:            public void setValue(Integer v) {  
14:                 tl.set(v);  
15:            }  
16:            public void remove() {  
17:                 tl.remove();  
18:            }  
19:       }  
20:       @Override  
21:       public void run() {  
22:            MyThreadLocal tlc = new MyThreadLocal();  
23:            tlc.setValue(rand.nextInt(1000));  
24:            System.out.println(" Val for thread : " + tlc.getValue());  
25:            // task logic  
26:            tlc.remove();  
27:       }  
28:       public static void main(String[] args) throws InterruptedException {  
29:            ThreadLocalHolder tlh = new ThreadLocalHolder();  
30:            for (int i = 0; i < 5; i++) {  
31:                 Thread t = new Thread(tlh);  
32:                 t.start();  
33:            }  
34:       }  
35:  }  

Output:
 Val for thread :  144
 Val for thread :  724
 Val for thread :  320
 Val for thread :  627
 Val for thread :  478

Final Note

You should be very careful in using ThreadLocal class. The value stored in this object stays as long as the thread is active.
So it could create an issue if your application is running in an Application server thread? In such case, the object stored in ThreadLocal will never get reclaimed as long as your application is up and running. 

--

Time to call it a post..do drop your feedback/comments.