232ByteWatcher from JCrete

Author: Daniel Shaya, Chris Newland and Dr. Heinz M. KabutzDate: 2015-09-29Java Version: 8Category: Performance

Abstract: Even though standard Java is not real-time, we often use it in time-sensitive applications. Since GC events are often the reason for latency outliers, such code seeks to avoid allocation. In this newsletter we look at some tools to add to your unit tests to check your allocation limits.

Welcome to the 232nd edition of The Java(tm) Specialists' Newsletter, sent to you from the beautiful Island of Crete. Thank you to all those who were willing to take my place on Martin Thompson's course. In the end, we had such a lot of interest that we had to raffle the place. Thompson certainly is a popular guy! In a funny twist of fate, the winner works for LMAX, the company that Martin used to work for :-)

I spent the last few days as a "synodos" to my dear wife at the Heraklion Venizelou Hospital. You are probably wondering what a "synodos" entails? Let me explain and you will learn a little bit about Greek culture. In South Africa, hospitals have two categories of assistants to the doctors. Right below the doctor is the "sister". These have medical degrees and can do a surprising number of tasks, such as take the blood pressure, dress wounds, check your heart, save your life, etc. As a child, I preferred injections from a sister rather than from a doctor as they had more practice and were generally more skilled at those tasks. Below the sisters are the "nurses". These would also have some medical training I think, but not nearly as much as the sisters. They do the jucky stuff, like washing the patients, changing bedpans, etc. Don't ever ever ever call a sister a "nurse" in South Africa or your next injection will be painful ;-)

In Greece, the hospitals do not employ nurses, but only sisters. You will get the essential medical help, but that's it. So without nurses, how do the old frail people get to the bathrooms? This is where the "synodos" comes in. Every patient has at least one "synodos" sitting with them 24/7. This would be a relative or a friend. The word "synodos" is a companion or an escort. "Odos" means street, so the literal meaning is probably someone who walks the road with you. Beautiful. In most hospitals around the world, there are specific visiting hours. We had un-visiting hours! Whenever the doctors or sisters needed to do something to one of the patients, they shooed all of us out onto the balcony.

At first, this was a huge culture shock to us. Imagine a room with 8 patients, plus another 12 "synodos", all crammed together like sardines! But having experienced it all, I must say that there are some upsides to this system. Time just flies by, because you are constantly chatting to people. You don't even need a television. Plus it was heartening seeing these old ladies surrounded by people that loved them, 24/7. The lady next to us had her son-in-law sitting next to her for several days. One lady spent 3 days and nights without sleep sitting next to a friend. Here you don't get a call in the morning "Oh, your mother passed away last night." For someone getting old, what better way, than to have excellent medical care (Venizelou is fantastic) and the companionship of those that you love? Please tell me if you have something like the "synodos" system in your country.

javaspecialists.teachable.com: Please visit our new self-study course catalog to see how you can upskill your Java knowledge.

ByteWatcher from JCrete

After my newsletter on String's new substring mechanism, Daniel Shaya wrote a series of articles on the topic of allocation. This eventually resulted in a tool we named "ByteWatcher". Daniel and I got together in a JCrete session to brainstorm and start this article together. The idea was to write and publish it in one hour. We did not manage :-) However, I hope that the result is better than what we would've given you in such a hurry.

Regression testing is important. Most projects these days are filled to the brim with unit tests. Often the amount of code in the test branch will be far more than the code in the main branch. We usually test:

1. Correctness: We make sure that no fence-post, null pointer and similar bugs have crept into our code base. We also test that the code fulfills the contracts of what it is supposed to do.
2. Performance: Most software is expected to finish within a certain amount of time. This might be explicitly stated or assumed. For example, if you open IntelliJ IDEA, you expect that to happen within a few seconds. You would also expect a large drive to take an hour or more to fully back up. A trade might need to happen in under 10 microseconds. It is generally a good idea to state what these expectations are and to have tests that check that we do not exceed them.
3. Size Matters (sometimes): With ubiquitous storage becoming the norm, size of a distribution is not nearly as important as it used to be. Some might include some tests to make sure this does not get out of hand though.

But there is another attribute that has been difficult to test in the past: How many bytes are allocated on the heap by this function?

Java is not real-time. And despite the common misconception, real-time also doesn't mean fast. It means that we are predictable in our slowness :-) However, we do use Java in applications where we want to be fast and have as predictable latencies as possible, right up to the high percentiles. These latencies get blown out of the water when a GC event occurs. For this and many other reasons, such applications seek to avoid allocating objects on the heap. With ByteWatcher we can add a couple of lines into our regression tests that will check whether the expected allocation has been exceeded or not. Let's take a method that only works with stack variables:

public int methodThatDoesNotAllocateAnything() {
  int x = 42;
  for (int i = 0; i < 10000; i++) {
    x += i % 10;
    x /= i % 2 + 1;
  }
  return x;
}

To test this really does not allocate anything, we can use the ByteWatcherRegressionTestHelper:

private final ByteWatcherRegressionTestHelper helper =
    new ByteWatcherRegressionTestHelper();

@Test
public void testNoAllocationMethod() {
  helper.testAllocationNotExceeded(
      this::methodThatDoesNotAllocateAnything,
      0 // no allocation!
  );
}

On the other hand, if we want to test that a byte[] allocation does not exceed our expected values, we could test it like this:

@Test
public void testByteArrayAllocation() {
  System.out.println(methodThatDoesNotAllocateAnything());
  helper.testAllocationNotExceeded(
      () -> {
        byte[] data = new byte[100];
      },
      120 // 100 bytes, plus the object header, plus int length
  );
}

The ByteWatcherRegressionTestHelper is fairly trivial and relies on the ByteWatcherSingleThread. It can be created for either the current thread or any other arbitrary thread. In most cases, the unit test would probably be monitoring the current thread in order to make the test more predictable.

import static org.junit.Assert.*;

public class ByteWatcherRegressionTestHelper {
  private final ByteWatcherSingleThread bw;

  public ByteWatcherRegressionTestHelper(Thread thread) {
    bw = new ByteWatcherSingleThread(thread);
  }

  public ByteWatcherRegressionTestHelper() {
    this(Thread.currentThread());
  }

  public void testAllocationNotExceeded(
      Runnable job, long limit) {
    bw.reset();
    job.run();
    long size = bw.calculateAllocations();
    assertTrue(String.format("exceeded limit: %d using: %d%n",
        limit, size), size <= limit);
  }
}

The code for ByteWatcherSingleThread is a combined effort by Daniel Shaya, Chris Newland (Mr JITWatch) and myself. One of the features is that we try to figure out how many bytes are wasted when we measure how many bytes are wasted :-) Usually this number is 336, but if the JIT compiler runs whilst we are measuring, then it could be far more. We thus calibrate on start-up, by calling threadAllocatedBytes() 10000 times, sleeping for a bit, then doing that all 10 times. Hopefully by the time that the constructor returns, the code would've been all compiled and would thus not turn on the JIT Compiler on that thread during the test. The rest of the code is fairly self-explanatory. We simply grab the property getThreadAllocatedBytes from the ThreadMXBean, like I showed in my newsletter on String.substring().

import javax.management.*;
import java.lang.management.*;
import java.util.concurrent.atomic.*;

/**
 * A class to measure how much allocation there has been on an
 * individual thread.  The class would be useful to embed into
 * regression tests to make sure that there has been no
 * unintended allocation.
 */
public class ByteWatcherSingleThread {
  private static final String ALLOCATED = " allocated ";
  private static final String GET_THREAD_ALLOCATED_BYTES =
      "getThreadAllocatedBytes";
  private static final String[] SIGNATURE =
      new String[]{long.class.getName()};
  private static final MBeanServer mBeanServer;
  private static final ObjectName name;

  private final String threadName;
  private final Thread thread;

  private final Object[] PARAMS;
  private final AtomicLong allocated = new AtomicLong();
  private final long MEASURING_COST_IN_BYTES; // usually 336
  private final long tid;
  private final boolean checkThreadSafety;

  static {
    try {
      name = new ObjectName(
          ManagementFactory.THREAD_MXBEAN_NAME);
      mBeanServer = ManagementFactory.getPlatformMBeanServer();
    } catch (MalformedObjectNameException e) {
      throw new ExceptionInInitializerError(e);
    }
  }

  public ByteWatcherSingleThread() {
    this(Thread.currentThread(), true);
  }

  public ByteWatcherSingleThread(Thread thread) {
    this(thread, false);
  }

  private ByteWatcherSingleThread(
          Thread thread, boolean checkThreadSafety) {
    this.checkThreadSafety = checkThreadSafety;
    this.tid = thread.getId();
    this.thread = thread;
    threadName = thread.getName();
    PARAMS = new Object[]{tid};

    long calibrate = threadAllocatedBytes();
    // calibrate
    for (int repeats = 0; repeats < 10; repeats++) {
      for (int i = 0; i < 10_000; i++) {
        // run a few loops to allow for startup anomalies
        calibrate = threadAllocatedBytes();
      }
      try {
        Thread.sleep(50);
      } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        break;
      }
    }
    MEASURING_COST_IN_BYTES = threadAllocatedBytes() - calibrate;
    reset();
  }

  public long getMeasuringCostInBytes() {
    return MEASURING_COST_IN_BYTES;
  }

  public void reset() {
    checkThreadSafety();

    allocated.set(threadAllocatedBytes());
  }

  long threadAllocatedBytes() {
    try {
      return (long) mBeanServer.invoke(
          name,
          GET_THREAD_ALLOCATED_BYTES,
          PARAMS,
          SIGNATURE
      );
    } catch (Exception e) {
      throw new IllegalArgumentException(e);
    }
  }

  /**
   * Calculates the number of bytes allocated since the last
   * reset().
   */
  public long calculateAllocations() {
    checkThreadSafety();
    long mark1 = ((threadAllocatedBytes() -
        MEASURING_COST_IN_BYTES) - allocated.get());
    return mark1;
  }

  private void checkThreadSafety() {
    if (checkThreadSafety &&
        tid != Thread.currentThread().getId())
      throw new IllegalStateException(
          "AllocationMeasure must not be " +
              "used over more than 1 thread.");
  }

  public Thread getThread() {
    return thread;
  }

  public String toString() {
    return thread.getName() + ALLOCATED + calculateAllocations();
  }
}

We also wrote a ByteWatcher, which you can use to monitor specific threads in your applications. The ByteWatcher would not work well for unit testing, since the call-back would come from a thread managed within the ByteWatcher. An assertion failure would thus only serve to stop the watching task. We need to be careful to never throw exceptions inside our callback code!

import java.util.*;
import java.util.concurrent.*;
import java.util.function.*;
import java.util.stream.*;

/**
 * Created by daniel on 22/07/2015.
 * This class allows a user to receive callbacks if
 * threads are destroyed or created.
 * Its primary function is to alert the user if any
 * thread has exceeded a specified amount of allocation.
 */
public class ByteWatcher {
  public static final int SAMPLING_INTERVAL =
      Integer.getInteger("samplingIntervalMillis", 500);
  public static final Consumer<Thread> EMPTY = a -> { };
  public static final BiConsumer<Thread, Long> BI_EMPTY =
      (a, b) -> { };
  private final Map<Thread, ByteWatcherSingleThread> ams;
  private volatile Consumer<Thread> threadCreated = EMPTY;
  private volatile Consumer<Thread> threadDied =
      EMPTY;
  private volatile ByteWatch byteWatch = new ByteWatch(
      BI_EMPTY, Long.MAX_VALUE
  );

  private static class ByteWatch
      implements BiConsumer<Thread, Long>, Predicate<Long>{
    private final long threshold;
    private final BiConsumer<Thread, Long> byteWatch;

    public ByteWatch(BiConsumer<Thread, Long> byteWatch,
                     long threshold) {
      this.byteWatch = byteWatch;
      this.threshold = threshold;
    }

    public void accept(Thread thread, Long currentBytes) {
      byteWatch.accept(thread, currentBytes);
    }

    public boolean test(Long currentBytes) {
      return threshold < currentBytes;
    }
  }

  private final ScheduledExecutorService monitorService =
      Executors.newSingleThreadScheduledExecutor();

  public ByteWatcher() {
    // do this first so that the worker thread is not considered
    // a "newly created" thread
    monitorService.scheduleAtFixedRate(
        this::checkThreads,
        SAMPLING_INTERVAL, SAMPLING_INTERVAL,
        TimeUnit.MILLISECONDS);

    ams = Thread.getAllStackTraces()
        .keySet()
        .stream()
        .map(ByteWatcherSingleThread::new)
        .collect(Collectors.toConcurrentMap(
            ByteWatcherSingleThread::getThread,
            (ByteWatcherSingleThread am) -> am));
    // Heinz: Streams make sense, right? ;-)
  }

  public void onThreadCreated(Consumer<Thread> action) {
    threadCreated = action;
  }

  public void onThreadDied(Consumer<Thread> action) {
    threadDied = action;
  }

  public void onByteWatch(
      BiConsumer<Thread, Long> action, long threshold) {
    this.byteWatch = new ByteWatch(action, threshold);
  }

  public void shutdown() {
    monitorService.shutdown();
  }

  public void forEach(Consumer<ByteWatcherSingleThread> c) {
    ams.values().forEach(c);
  }

  public void printAllAllocations() {
    forEach(System.out::println);
  }

  public void reset() {
    forEach(ByteWatcherSingleThread::reset);
  }

  private void checkThreads() {
    Set<Thread> oldThreads = ams.keySet();
    Set<Thread> newThreads = Thread.getAllStackTraces().keySet();

    Set<Thread> diedThreads = new HashSet<>(oldThreads);
    diedThreads.removeAll(newThreads);

    Set<Thread> createdThreads = new HashSet<>(newThreads);
    createdThreads.removeAll(oldThreads);

    diedThreads.forEach(this::threadDied);
    createdThreads.forEach(this::threadCreated);
    ams.values().forEach(this::bytesWatch);
  }

  private void threadCreated(Thread t) {
    ams.put(t, new ByteWatcherSingleThread(t));
    threadCreated.accept(t);
  }

  private void threadDied(Thread t) {
    threadDied.accept(t);
  }

  private void bytesWatch(ByteWatcherSingleThread am) {
    ByteWatch bw = byteWatch;
    long bytesAllocated = am.calculateAllocations();
    if (bw.test(bytesAllocated)) {
      bw.accept(am.getThread(), bytesAllocated);
    }
  }
}

The GIT project that Daniel and I used to build this code can be found here.

Related Projects

After I sent out this newsletter, I was pointed to two similar projects: allocheck, which uses try-with-resource as a wrapper for checking allocation. Brian's mechanism also counts number of objects created, not just bytes:

public void doSomeWork() {
    try (Allocheck _ = new Allocheck(1, 1)) {
        for (int i = 0; i < 5; i++) {
            new Object().hashCode();
            Integer.parseInt(String.valueOf(i));
        }
    }
}

Matthew Painter wrote a Java Agent that tracks allocation directly from the byte code. See his project sio2box.

Thank you so much for reading this and I hope this will be a useful addition to your testing toolbox.

Kind regards from a sunny Crete

Heinz

Comments

We are always happy to receive comments from our readers. Feel free to send me a comment via email or discuss the newsletter in our JavaSpecialists Slack Channel (Get an invite here)

Related Articles

Browse the Newsletter Archive