078MemoryCounter for Java 1.4

Author: Dr. Heinz M. KabutzDate: 2003-09-29Java Version: 1.4Category: Performance

Abstract: Our old MemoryCounter needed to be updated for Java 1.4. In previous versions of Java, each primitive field used at least 4 bytes, even a boolean. This changed in Java 1.4, where boolean and byte use a 1 byte, short and char use 2 bytes, int and float use 4 and long and double use 8.

Welcome to the 78th edition of The Java(tm) Specialists' Newsletter. Approximately two years ago, I published a newsletter (#29) that showed how we could count the bytes being used by any Java object. I have now turned that newsletter into a little program, that you can use to automatically determine memory usage.

This past week I taught UML and Java at the Tsinghua University in Beijing, China. My audience were professors, lecturers and researchers from all around China. They were very attentive and participated well during the discussion times. It was a great pleasure spending six days teaching them. After climbing The Great Wall yesterday and meeting the Computer Science Department this morning, I now have some time to churn out yet another newsletter :-)

China is, well, memorable. You simply cannot put it in words. In the week that I have been here, I have already taken about 350 pictures. It is definitely a place that one should visit one day. When I asked one professor how many software developers there were in China, he said "One Million or more". I thought I had misunderstood, so I asked him to repeat. That is a HUGE workforce.

javaspecialists.teachable.com: Please visit our new self-study course catalog to see how you can upskill your Java knowledge.

MemoryCounter for Java 1.4

Update: Please also read later newsletter 142 - Instrumentation Memory Counter..

Before I start, I must warn you that I am using classes that will only work for Java 1.4 onwards. In addition, the memory sizes have changed in JDK 1.4, so there are some differences between 1.4 and 1.3. This class is just supposed to give you *some* idea of how big an object is.

After my newsletter #29, I received a few emails telling me that it would be much easier to serialize the object and then count the bytes. If I need to know how much bandwidth my objects will use, then such an approach would make sense. My approach is used for measuring RAM memory. Another advantage of my approach is that the objects do not have to be serializable.

The first class we have is called MemorySizes and will tell my memory counter class how much memory each element takes. I have determined these through experiments.

import java.util.*;

public class MemorySizes {
  private final Map primitiveSizes = new IdentityHashMap() {
    {
      put(boolean.class, new Integer(1));
      put(byte.class, new Integer(1));
      put(char.class, new Integer(2));
      put(short.class, new Integer(2));
      put(int.class, new Integer(4));
      put(float.class, new Integer(4));
      put(double.class, new Integer(8));
      put(long.class, new Integer(8));
    }
  };
  public int getPrimitiveFieldSize(Class clazz) {
    return ((Integer) primitiveSizes.get(clazz)).intValue();
  }
  public int getPrimitiveArrayElementSize(Class clazz) {
    return getPrimitiveFieldSize(clazz);
  }
  public int getPointerSize() {
    return 4;
  }
  public int getClassSize() {
    return 8;
  }
}

Next we have the class that counts the memory sizes. For an explanation of this code, I would suggest you look at the original newsletter.

import java.lang.reflect.*;
import java.util.*;

/**
 * This class can estimate how much memory an Object uses.  It is
 * fairly accurate for JDK 1.4.2.  It is based on the newsletter #29.
 */
public final class MemoryCounter {
  private static final MemorySizes sizes = new MemorySizes();
  private final Map visited = new IdentityHashMap();
  private final Stack stack = new Stack();

  public synchronized long estimate(Object obj) {
    assert visited.isEmpty();
    assert stack.isEmpty();
    long result = _estimate(obj);
    while (!stack.isEmpty()) {
      result += _estimate(stack.pop());
    }
    visited.clear();
    return result;
  }

  private boolean skipObject(Object obj) {
    if (obj instanceof String) {
      // this will not cause a memory leak since
      // unused interned Strings will be thrown away
      if (obj == ((String) obj).intern()) {
        return true;
      }
    }
    return (obj == null)
        || visited.containsKey(obj);
  }

  private long _estimate(Object obj) {
    if (skipObject(obj)) return 0;
    visited.put(obj, null);
    long result = 0;
    Class clazz = obj.getClass();
    if (clazz.isArray()) {
      return _estimateArray(obj);
    }
    while (clazz != null) {
      Field[] fields = clazz.getDeclaredFields();
      for (int i = 0; i < fields.length; i++) {
        if (!Modifier.isStatic(fields[i].getModifiers())) {
          if (fields[i].getType().isPrimitive()) {
            result += sizes.getPrimitiveFieldSize(
                fields[i].getType());
          } else {
            result += sizes.getPointerSize();
            fields[i].setAccessible(true);
            try {
              Object toBeDone = fields[i].get(obj);
              if (toBeDone != null) {
                stack.add(toBeDone);
              }
            } catch (IllegalAccessException ex) { assert false; }
          }
        }
      }
      clazz = clazz.getSuperclass();
    }
    result += sizes.getClassSize();
    return roundUpToNearestEightBytes(result);
  }

  private long roundUpToNearestEightBytes(long result) {
    if ((result % 8) != 0) {
      result += 8 - (result % 8);
    }
    return result;
  }

  protected long _estimateArray(Object obj) {
    long result = 16;
    int length = Array.getLength(obj);
    if (length != 0) {
      Class arrayElementClazz = obj.getClass().getComponentType();
      if (arrayElementClazz.isPrimitive()) {
        result += length *
            sizes.getPrimitiveArrayElementSize(arrayElementClazz);
      } else {
        for (int i = 0; i < length; i++) {
          result += sizes.getPointerSize() +
              _estimate(Array.get(obj, i));
        }
      }
    }
    return result;
  }
}

I recently changed this code significantly to include the new IdentityHashMap and to add support for interned Strings. This would not have been possible without a good suite of unit tests. I normally do not include my unit tests in the newsletters, since they do use a lot of space. However, I would like to hear from you if you have a JDK 1.4.x where the results are different. The unit tests pass on my machine, using JDK 1.4.2, build 1.4.2-b28.

import java.util.*;
import junit.framework.TestCase;
import junit.swingui.TestRunner;

public class MemoryCounterTest extends TestCase {
  public static void main(String[] args) {
    TestRunner.run(MemoryCounterTest.class);
  }

  private static final MemoryCounter mc = new MemoryCounter();

  public MemoryCounterTest(String name) {
    super(name);
  }

  public void testString() {
    assertEquals(64, mc.estimate(new String("Hello World!")));
  }

  public void testIntegerToString() {
    for (int i=0; i<1; i++) {
      assertEquals(72, mc.estimate("" + i));
    }
  }

  static class Entry implements Map.Entry {
    final Object key;
    Object value;
    final int hash;
    Entry next;

    Entry(int h, Object k, Object v, Entry n) {
      value = v;
      next = n;
      key = k;
      hash = h;
    }

    public Object getKey() {
      return key;
    }

    public Object getValue() {
      return value;
    }

    public Object setValue(Object value) {
      return value;
    }
  }

  public void testHashMap() {
    assertEquals(120, mc.estimate(new HashMap()));

    Byte[] all = new Byte[256];
    for (int i = -128; i < 128; i++) {
      all[i + 128] = new Byte((byte) i);
    }
    assertEquals(5136, mc.estimate(all));

    HashMap hm = new HashMap();
    for (int i = -128; i < 128; i++) {
      hm.put("" + i, new Byte((byte) i));
    }
    assertEquals(30776, mc.estimate(hm));
  }

  public void testVector() {
    assertEquals(80, mc.estimate(new Vector(10)));
  }

  public void testObject() {
    assertEquals(8, mc.estimate(new Object()));
  }

  public void testInteger() {
    assertEquals(16, mc.estimate(new Integer(1)));
  }

  public void testCharArray() {
    assertEquals(40, mc.estimate("Hello World!".toCharArray()));
  }

  public void testByte() {
    assertEquals(16, mc.estimate(new Byte((byte) 10)));
  }

  public void testThreeBytes() {
    assertEquals(16, mc.estimate(new ThreeBytes()));
  }

  public void testSixtyFourBooleans() {
    assertEquals(72, mc.estimate(new SixtyFourBooleans()));
  }

  public void testThousandBooleansObjects() {
    Boolean[] booleans = new Boolean[1000];

    for (int i = 0; i < booleans.length; i++)
      booleans[i] = new Boolean(true);

    assertEquals(20016, mc.estimate(booleans));
  }

  public void testThousandBytes() {
    assertEquals(1016, mc.estimate(new byte[1000]));
  }

  public void testEmptyArrayList() {
    assertEquals(80, mc.estimate(new ArrayList()));
  }

  public void testFullArrayList() {
    ArrayList arrayList = new ArrayList(10000);

    for (int i = 0; i < 10000; i++) {
      arrayList.add(new Object());
    }

    assertEquals(120040, mc.estimate(arrayList));
  }

  public void testFullLinkedList() {
    LinkedList linkedList = new LinkedList();

    for (int i = 0; i < 10000; i++) {
      linkedList.add(new Object());
    }

    assertEquals(320048, mc.estimate(linkedList));
  }

  public void testComplexClass() {
    assertEquals(48, mc.estimate(new ComplexClass()));
  }

  public void testBooleanArray() {
    assertEquals(27, mc.estimate(new boolean[11]));
  }

  public void testShortArray() {
    assertEquals(38, mc.estimate(new short[11]));
  }

  public void testIntArray() {
    assertEquals(60, mc.estimate(new int[11]));
  }

  public void testFloatArray() {
    assertEquals(60, mc.estimate(new float[11]));
  }

  public void testLongArray() {
    assertEquals(104, mc.estimate(new long[11]));
  }

  public void testDoubleArray() {
    assertEquals(104, mc.estimate(new double[11]));
  }

  static class ThreeBytes {
    byte b0;
    byte b1;
    byte b2;
  }

  private static class ComplexClass {
    ComplexClass cc = this;
    boolean z;
    byte b;
    char c;
    double d;
    float f;
    int i;
    long l;
    short s;
  }

  private static class SixtyFourBooleans {
    boolean a0;
    boolean a1;
    boolean a2;
    boolean a3;
    boolean a4;
    boolean a5;
    boolean a6;
    boolean a7;
    boolean b0;
    boolean b1;
    boolean b2;
    boolean b3;
    boolean b4;
    boolean b5;
    boolean b6;
    boolean b7;
    boolean c0;
    boolean c1;
    boolean c2;
    boolean c3;
    boolean c4;
    boolean c5;
    boolean c6;
    boolean c7;
    boolean d0;
    boolean d1;
    boolean d2;
    boolean d3;
    boolean d4;
    boolean d5;
    boolean d6;
    boolean d7;
    boolean e0;
    boolean e1;
    boolean e2;
    boolean e3;
    boolean e4;
    boolean e5;
    boolean e6;
    boolean e7;
    boolean f0;
    boolean f1;
    boolean f2;
    boolean f3;
    boolean f4;
    boolean f5;
    boolean f6;
    boolean f7;
    boolean g0;
    boolean g1;
    boolean g2;
    boolean g3;
    boolean g4;
    boolean g5;
    boolean g6;
    boolean g7;
    boolean h0;
    boolean h1;
    boolean h2;
    boolean h3;
    boolean h4;
    boolean h5;
    boolean h6;
    boolean h7;
  }
}

I would be happy to hear from you if this newsletter is useful to you. Just pop me a quick email telling me how this can help you.

Kind regards

Heinz

P.S. My original code has loads of comments. I stripped them out for the newsletter to save space.

Comments

We are always happy to receive comments from our readers. Feel free to send me a comment via email or discuss the newsletter in our JavaSpecialists Slack Channel (Get an invite here)

Related Articles

Browse the Newsletter Archive