Performance Engineering – JVM – Thread Pooling, Multi-threading concepts

JVM – Thread Pooling

• Threads allows a program to work parallelly on different tasks at the same time. In a single threaded application, only one thread can execute, meaning only one task can run at any point in time.
• If the application simply creates a new thread for every new HTTP request, and the system receives more requests than it can handle — immediately, the application will suddenly stop responding to all requests when the overhead of all those threads exceeds the capacity of the system. With a limit on the number of the threads that can be created, the application will not be servicing HTTP requests as quickly as they come in, but it will be servicing them as quickly as the system can sustain. That is the reason we have a thread pool so that we can minimize the management of threads and focus on our business logic.
• Thread Pool is a collection of threads which can directly work on tasks assigned by us. Threads are also reused to work on tasks waiting in the queue.
• Types of Thread Pools
  • Fixed Thread Pool
    • Number of Threads will be Fixed. Eg: Using ExecutorService in Java
    • Whenever you are sure on the number of threads that might be used, go for Fixed Thread Pool
  • Cached Thread Pool
    • We will create the thread whenever there is a need, and keep on creating these threads until an extent.
    • Cached Thread Pool is not to be used for long running tasks.
  • Single-Threaded Thread Pool
    • A single thread is being created to perform all the activities
  • Scheduled Thread Pool
    • Tasks to be taken based on the time delay via Scheduled Thread Pool

• Core Pool Size

  • Defines as “How many threads to be created per pool?”
  • When the incoming tasks goes beyond the Core Pool Size, the tasks will be assigned to the queue
    • • Queuing – Bounded Queue, Un-Bounded Queue
      • Tasks can be queued until a certain limit in the Bounded Queue, which is frequently leveraged
      • Tasks can be queued without having any limit in the Un-Bounded Queue
      • Once the bounded queue has been set with value as pool maximum size, once the inbound task count reach this pool maximum size, that task gets rejected

• Multi-threading concepts

  • Thread Life Cycle

• • • Refer below to the Thread Life Cycle

• • • New
      • This the state where the thread is created.
    • Ready/Runnable

• • • • When the start() method is invoked to a particular thread, it will switch state from New to Ready or Runnable state
    • Running
      • • The Thread will switch to Running state when run() method is invoked. which means when the process is executing. But it may go back to Ready/Runnable state and come back to Running state and this can happen again and again also.
    • Blocked
      •  This is the state where one thread blocked on the lock because other thread has already acquired this lock.
      • For performance engineers, this state is very important to monitor
    • Waiting
      • In this state, the thread will wait forever until if there is any interruption. Usually the invocation of  join() or wait() method will put the thread in waiting state.
      • For performance engineers, this state is very important to monitor
    • Timed Waiting
      • When sleep() method or join() or wait() methods with timeout are invoked, that state is known as Timed Waiting. Name itself explains that the thread will wait for a certain given time.
    • Dead
      • This state represents the completion of process.

• JVM Thread Dump Analysis

• • Whenever your application have issues(crashing/taking more time to respond/etc), we will either take a Heap Dump or Thread Dump that gives us the current snapshot to analyze if the issue is with memory or threads
  • Thread dumps are vital artifacts to diagnose CPU spikes, deadlocks, memory problems, unresponsive applications, poor response times, and other system problems.
  • Ways to take the Thread Dumps
    • jstack

• • • • ‘jstack’ is an effective command line tool to capture thread dumps. The jstack tool is shipped in JDK_HOME\bin folder. Here is the command that you need to issue to capture thread dump: jstack -l <pid> > <file-path>
      • Jstack tool is included in JDK since Java 5. If you are running in older version of java, consider using other options
    • Kill -3
      • In major enterprises for security reasons only JREs are installed in production machines. Since jstack and other tools are only part of JDK, you wouldn’t be able to use jstack. In such circumstances, ‘kill -3’ option can be used
      • kill -3 <pid>

• • • JVisualVM
      • Java VisualVM is a graphical user interface tool that provides detailed information about the applications while they are running on a specified Java Virtual Machine (JVM). It’s located in JDK_HOME\bin\jvisualvm.exe. It’s part of Sun’s JDK distribution since JDK 6 update 7.
      • Launch the jvisualvm. On the left panel, you will notice all the java applications that are running on your machine. You need to select your application from the list (see the red color highlight in the below diagram). This tool also has the capability to capture thread dumps from the java processes that are running in remote host as well.

• • • JMC
      • Java Mission Control (JMC) is a tool that collects and analyze data from Java applications running locally or deployed in production environments. This tool has been packaged into JDK since Oracle JDK 7Update 40. This tool also provides an option to take thread dumps from the JVM. JMC tool is present in JDK_HOME\bin\jmc.exe
      • Once you launch the tool, you will see all the Java processes that are running on your local host. Note: JMC also has the ability to connect with java processes running on a remote host. Now on the left panel click on the “Flight Recorder” option that is listed below the Java process for which you want to take thread dumps. Now you will see the “Start Flight Recording” wizard, as shown in the below figure.

• • • • Here in the “Thread Dump” field, you can select the interval in which you want to capture thread dump. As per the above example, every 60 seconds thread dump will be captured. After the selection is complete start the Flight recorder. Once recording is complete, you will see the thread dumps in the “Threads” panel, as shown in the figure below.

• • • JCMD
      • The jcmd tool was introduced with Oracle’s Java 7. It’s useful in troubleshooting issues with JVM applications. It has various capabilities such as identifying java process Ids, acquiring heap dumps, acquiring thread dumps, acquiring garbage collection statistics
      • Using the below JCMD command you can generate thread dump:
        • jcmd <pid> Thread.print > <file-path>
    • APM Tool – App Dynamics
      •  Few Application Performance Monitoring tools provide options to generate thread dumps. If you are monitoring your application through App Dynamics (APM tool), below are the instructions to capture thread dump:
        • Create an action, selecting Diagnostics->Take a thread dump in the Create Action window.
        • Enter a name for the action, the number of samples to take, and the interval between the thread dumps in milliseconds.
        • If you want to require approval before the thread dump action can be started, check the Require approval before this Action checkbox and enter the email address of the individual or group that is authorized to approve the action. See Actions Requiring Approval for more information
        • Click OK.

• Analyze the generated Thread Dump

  • Let’s have a look at the generated thread dump. The first line displays the timestamp while the second line informs about the JVM:

• • Next section shows the Safe Memory Reclamation (SMR) and non-JVM internal threads:

• • Then, the dump displays the list of threads. Each thread contains the following information:
    • Name: it can provide useful information if developers include a meaningful thread name
    • Whether the thread is a daemon or a non-daemon thread
    • Priority (prior): the priority of the thread
    • Java ID (tid): the unique ID given by the JVM
    • Native ID (nid): the unique ID given by the OS, useful to extract correlation with CPU or memory processing
    • State: the actual state of the thread
    • Stack trace: the most important source of information to decipher what is happening with our application
    • We can see from top to bottom what the different threads are doing at the time of the snapshot. Let’s focus only on the interesting bits of the stack waiting to consume the message:

• • • At a first glance, we see that the main stack trace is executing java.io.BufferedReader.readLine
    • which is the expected behavior. If we look further down we’ll see all the JVM methods executed by our application behind the scenes. Therefore, we are able to identify the root of the problem by looking at the source code or other internal JVM processing.
    • At the end of the dump, we’ll notice there are several additional threads performing background operations such as Garbage Collection (GC) or object termination:

• • • Finally, the dump displays the Java Native Interface (JNI) references. We should pay special attention to this when memory leak occurs because they aren’t automatically garbage collected:

• • • Thread dumps are fairly similar in their structure, but we’ll want to get rid of the non-important data generated for our use case. On the other hand, we’ll need to keep and group the important information from the tons of logs produced by the stack trace.
    • Better to have multiple thread dumps, and search the blocked thread id from these multiple thread dumps, and if we happen to find the thread id in multiple thread dumps, meaning that this thread is still in blocked state. Similarly if we find any thread that is in waiting state/running statein multiple thread dumps, then these too can be considered as the error scenarios and have to be avoided.
      • Report all these issues and problematic line of code details to dev team and ensure resolution and retest for the issue resolution

 

-By  Saravanan & Sai Krishna