059When Arguments Get out of Hand...

Author: Dr. Heinz M. KabutzDate: 2002-11-11Java Version: 1.4.1Category: Performance

Abstract: We turn the dial to full blast and find out how many parameters a method may contain. Wholly impractical, but fun.

Welcome to the 59th edition of The Java(tm) Specialists' Newsletter sent to 5008 Java Specialists in 89 countries. We have broken through the 5000 mark, I am very grateful to all of you for "spreading the word" about this newsletter.

The original topic of this newsletter was "Verrrrrry looooong Strings and other things". One of my subscribers from Poland suggested that the topic did not really represent the content, so I renamed it.

A few monts ago, my daugter pulled off one of my keys from my Asus notebook. Te irritating part is tat te company from wom I bougt te notebook went bankrupt, and it is now really ard to get spare parts in Sout Africa. I would tell you wic key it is if I could ... ;-)

I started typing this newsletter whilst sitting in a coach, speeding across the English country-side. I spent three days in October at a large company in London, presenting my Design Patterns Course and followed that up with a quick visit to mom-in-law in Sidmouth. Whilst in London, I met up with a number of my newsletter subscribers from the company, and they gave me a memorable introduction to English beverages. Thanks especially to Joe for organising it. That Design Patterns Course was the best I've had so far, gauged by the discussions we had around the various subtleties of the patterns.

javaspecialists.teachable.com: Please visit our new self-study course catalog to see how you can upskill your Java knowledge.

Verrrrrry Looooong Strings and Other Things

One of the things I enjoy doing, is seeing how far I can take things. For example, the rev counter of my Alfa Romeo is limited to 7000 revolutions per minute (revs). The question is, for each of the gears, what is the km/h equivalent of 7000 revs? I found out the hard way on the first day when I had just gotten the car. I was sitting next to a Mercedes at a traffic light, waiting for it to change to green. As it did, I threw in the first gear, charging it up to 60km/h, smugly watching the Mercedes in my rearview mirror, then threw in the second gear, and promptly got the accelerator stuck underneath the floormat! I discovered that the second gear goes up to 100 km/h at 7000 revs. The merc driver must have thought: "Typical show-off Alfa Romeo driver!" After some desperate moments, I managed to pull the floormat back, much to my relief, as I was heading for two speedbumps. Fortunately the breaks are extremely responsive. I have since discovered that gear #3 goes up to 140km/h, and gear #4 is not actually limited to 7000 revs (I don't know why), but it hits 7000 revs at about 180km/h. I do not know at what point gear #5 hits 7000 revs...

Back to Java. At what point do the wheels start falling off? Here are some limitations of the Java Virtual Machine, taken from the VM Spec. They are implicit due to the data structure for the class file format. Consider these restrictions as the rev counter in your JVM:

1. The per-class or per-interface constant pool is limited to 65535 entries by the 16-bit constant_pool_count field of the ClassFile structure. This acts as an internal limit on the total complexity of a single class or interface.
2. The amount of code per non-native, non-abstract method is limited to 65536 bytes by the sizes of the indices in the exception_table of the Code attribute, in the LineNumberTable attribute, and in the LocalVariableTable attribute.
3. The greatest number of local variables in the local variables array of a frame created upon invocation of a method is limited to 65535 by the size of the max_locals item of the Code attribute giving the code of the method. Note that values of type long and double are each considered to reserve two local variables and contribute two units toward the max_locals value, so use of local variables of those types further reduces this limit.
4. The number of fields that may be declared by a class or interface is limited to 65535 by the size of the fields_count item of the ClassFile structure. Note that the value of the fields_count item of the ClassFile structure does not include fields that are inherited from superclasses or superinterfaces.
5. The number of methods that may be declared by a class or interface is limited to 65535 by the size of the methods_count item of the ClassFile structure. Note that the value of the methods_count item of the ClassFile structure does not include methods that are inherited from superclasses or superinterfaces.
6. The number of direct superinterfaces of a class or interface is limited to 65535 by the size of the interfaces_count item of the ClassFile structure.
7. The size of an operand stack in a frame is limited to 65535 values by the max_stack field of the Code_attribute structure. Note that values of type long and double are each considered to contribute two units toward the max_stack value, so use of values of these types on the operand stack further reduces this limit.
8. The number of local variables in a frame is limited to 65535 by the max_locals field of the Code_attribute structure and the 16-bit local variable indexing of the Java virtual machine instruction set.
9. The number of dimensions in an array is limited to 255 by the size of the dimensions opcode of the multianewarray instruction and by the constraints imposed on the multianewarray, anewarray, and newarray instructions.
10. The number of method parameters is limited to 255 by the definition of a method descriptor, where the limit includes one unit for this in the case of instance or interface method invocations. Note that a method descriptor is defined in terms of a notion of method parameter length in which a parameter of type long or double contributes two units to the length, so parameters of these types further reduce the limit.
11. The length of field and method names, field and method descriptors, and other constant string values is limited to 65535 characters by the 16-bit unsigned length item of the CONSTANT_Utf8_info structure. Note that the limit is on the number of bytes in the encoding and not on the number of encoded characters. UTF-8 encodes some characters using two or three bytes. Thus, strings incorporating multibyte characters are further constrained.

Ok, that list sounds quite exhausting. Imagine typing in more than 255 parameters! What's the point of worrying about these restrictions?

When I think about going beyond these restrictions, I am thinking about auto-generated code, one of the many perls mentioned in The Pragmatic Programmer: From Journeyman to Master [ISBN 020161622X] . No-one is going to type in a method that contains more than 65535 instructions. At least, if they did, I would be happy for their code to fail. However, you might want to write some code that generates code automatically, and who knows, perhaps for some reason, you want to let a method contain that many instructions.

More Than 255 Parameters (Rule 10 Above)

I do not want to cover all the restrictions, just a few. The rest will be left as an exercise to the reader.

The first one I want to look at is the restriction to only allow 255 parameters. This includes the this parameter that is automatically passed into non-static methods. Therefore, non-static methods are actually limited to 254 parameters. What happens when you auto-generate a method with 5000 parameters?

import java.io.*;

public class ManyParametersGenerator {
  public static void main(String[] args) throws IOException {
    int LENGTH = Integer.parseInt(args[0]);
    System.out.println("Creating java file with " + LENGTH + 
      " parameters");
    // First, we generate a class with many parameters in a method
    PrintStream out = new PrintStream(
      new BufferedOutputStream(
        new FileOutputStream("ManyParameters.java")));
    out.println("public class ManyParameters {");
    out.println("  public int f(");
    for(int i=0; i<LENGTH; i++) {
      out.print("      int i" + i);
      if (i == LENGTH-1) out.println(") {");
      else out.println(",");
    }
    out.println("    int j = 0 ");
    for(int i=0; i<LENGTH; i++) {
      out.println("      + i" + i);
    }
    out.println("      ;");
    out.println("    System.out.println(j);");
    out.println("    return j;");
    out.println("  }"); 
    out.println("}"); 
    out.close();
    // Second, we generate a class that tests our strange class
    out = new PrintStream(
      new BufferedOutputStream(
        new FileOutputStream("ManyParametersTest.java")));
    out.println("public class ManyParametersTest {");
    out.println("  public static void main(String[] args) {");
    out.println("    ManyParameters mp = new ManyParameters();");
    out.println("    int j = mp.f(");
    for(int i=0; i<LENGTH; i++) {
      out.print("      " + (i + 10));
      if (i == LENGTH-1) out.println(");");
      else out.println(",");
    }
    // we also calculate what "j" should actually be
    int j = 0;
    for(int i=0; i<LENGTH; i++) {
      j += i+10;
    }
    out.println("    System.out.println(j);");
    out.println("    System.out.println(" + j + ");");
    out.println("  }");
    out.println("}"); 
    out.close();
  }
}

All code in this newsletter was compiled and run with Sun Microsystems' JDK 1.4.1_01 on Windows 2000 Professional, unless stated otherwise.

Let us run this with 10, 100, 254, 255 and 5000 parameters, followed by a compile and execution of our test classes:

Run with 10 parameters:
> java ManyParametersGenerator 10
Creating java file with 10 parameters
> javac ManyParameters.java ManyParametersTest.java 
> java ManyParametersTest 
145
145
145

Run with 100 parameters:
> java ManyParametersGenerator 100
Creating java file with 100 parameters
> javac ManyParameters.java ManyParametersTest.java 
> java ManyParametersTest 
5950
5950
5950

Run with 254 parameters:
> java ManyParametersGenerator 254
Creating java file with 254 parameters
> javac ManyParameters.java ManyParametersTest.java 
> java ManyParametersTest 
34671
34671
34671

Run with 255 parameters:
> java ManyParametersGenerator 255
Creating java file with 255 parameters
> javac ManyParameters.java ManyParametersTest.java 
> java ManyParametersTest 
Exception in thread "main" java.lang.VerifyError: (class: Many
ParametersTest, method: main signature: ([Ljava/lang/String;)V
) Signature (IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII`L#

Let's take a quick sanity check break here. It compiles fine, but the class verifier blows up, because there are 256 parameters (if we include the this pointer that implicitely gets sent to the method). Surely, the number of parameters could also be checked by the compiler? So, why do it in the verifier? The reason is that if it were only done in the compiler, I could write my own compiler that generated methods with more than 255 parameters, and that could cause your JVM to break. Think applets. However, it would make sense to me to also restrict the standard compiler to only allow 255 parameters. Perhaps they thought that no-one would be crazy enough to use more than 255 parameters anyway...

Run with 5000 parameters:
> java ManyParametersGenerator 5000
Creating java file with 5000 parameters
> javac ManyParameters.java ManyParametersTest.java 
> java ManyParametersTest 
The system is out of resources.
Consult the following stack trace for details.
java.lang.StackOverflowError
  at com.sun.tools.javac.v8.comp.Attr.attribExpr(Attr.java:279)
  at com.sun.tools.javac.v8.comp.Attr.visitBinary(Attr.java:965)
  at com.sun.tools.javac.v8.tree.Tree$Binary.accept(Tree.java:1014)
  at com.sun.tools.javac.v8.comp.Attr.attribTree(Attr.java:256)
  at com.sun.tools.javac.v8.comp.Attr.attribExpr(Attr.java:279)
etc.

What do we learn from this experience? Not very much. We hear the Java engine revving a bit, but besides that, all we notice is that the engine is limited to 7000 revs. However, we do now know how far we can push Java. Not very.

Long Constant Strings (Rule 11 Above)

Constant Strings must be shorter than 65536 characters. This also means that variable names, method names and field names must be less than 65536 characters. Oh no! Again, usually the only conceivable time when you could end up with a String that long is with automatically generated code, although I know of one company that embedded their SQL queries in constant Strings and then hit this limit!

import java.io.*;

public class BigStringGenerator {
  public static void main(String[] args) throws IOException {
    int LENGTH = Integer.parseInt(args[0]);
    System.out.println("Creating java file with string of length " + LENGTH);
    PrintStream out = new PrintStream(
      new BufferedOutputStream(
        new FileOutputStream("BigString.java")));
    out.println("public class BigString {");
    out.print("  public String big = ");
    out.print("\"");
    for(int i=0; i<LENGTH; i++) {
      out.print((char)((i%26)+'A'));
    }
    out.print("\"");
    out.println(";");
    out.println("}"); 
    out.close();
    out = new PrintStream(
      new BufferedOutputStream(
        new FileOutputStream("BigStringTest.java")));
    out.println("public class BigStringTest {");
    out.println("  public static void main(String[] args) {");
    out.println("    try {");
    out.println("      BigString bs = new BigString();");
    out.println("      System.out.println(bs.big.length());");
    out.println("    } catch(Throwable t) { System.err.println(t); }");
    out.println("  }");
    out.println("}"); 
    out.close();
  }
}

Let us run this with Strings of size 65535, 65536, 600000 and 6000000, followed by a compile and execution of our test classes:

Run with String of length 65535:
> java BigStringGenerator 65535
Creating java file with string of length 65535
> javac BigString.java BigStringTest.java
> java BigStringTest 
65535

Run with String of length 65536:
> java BigStringGenerator 65536
Creating java file with string of length 65536
> javac BigString.java BigStringTest.java
> java BigStringTest 
Exception in thread "main" java.lang.ClassFormatError: BigString (Illegal constant pool type)
      at java.lang.ClassLoader.defineClass0(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:502)
      at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
      at java.net.URLClassLoader.defineClass(URLClassLoader.java:250)
      at java.net.URLClassLoader.access$100(URLClassLoader.java:54)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:193)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:186)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:299)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:265)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:255)
      at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315)

Run with String of length 600000:
> java BigStringGenerator 600000
Creating java file with string of length 600000
> javac BigString.java BigStringTest.java
> java BigStringTest 
Exception in thread "main" java.lang.ClassFormatError: BigString (Illegal constant pool type)
      at java.lang.ClassLoader.defineClass0(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:502)
      at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
      at java.net.URLClassLoader.defineClass(URLClassLoader.java:250)
      at java.net.URLClassLoader.access$100(URLClassLoader.java:54)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:193)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:186)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:299)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:265)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:255)
      at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315)

Run with String of length 6000000:
> java BigStringGenerator 6000000
Creating java file with string of length 6000000
> javac BigString.java BigStringTest.java
The system is out of resources.
Consult the following stack trace for details.
java.lang.OutOfMemoryError
> java BigStringTest 
Exception in thread "main" java.lang.ClassFormatError: BigString (Truncated class file)

Ok, that was pushing it a bit. However, I am a bit worried that you cannot change the maximum heap memory when you run the javac tool.

What Are the Missing Fields? (Rule 4 Above)

I'm confused (a bit). In rule 4, we see that the number of fields is limited to 65535. However, when I pushed the gas pedal, I found that I was only allowed 65521 data members (in the JDK 1.4.1_01). Are there hidden fields, or did the compiler writers not check the boundary conditions properly? The JDK 1.3.1_03 contains a bug that only allows you to have 12996 data members, so it is a 404% improvement. Have a look at the following code:

import java.io.*;

public class BigClassGenerator {
  public static void main(String[] args) throws IOException {
    int LENGTH = Integer.parseInt(args[0]);
    System.out.println("Creating java file with " + LENGTH + 
      " data members");
    PrintStream out = new PrintStream(
      new BufferedOutputStream(
        new FileOutputStream("BigClass.java")));
    out.println("public class BigClass {");
    for(int i=0; i<LENGTH; i++) {
      out.println("private int a" + i + ";");
    }
    out.println("}"); 
    out.close();
  }
}

Try this on JDK 1.4.1_01 and JDK 1.3.1_03 and see if you get the same results as me. Assuming that JDK 1.4.1_01 is the most correct version of Java we have available at the moment, what are the missing 14 fields? Are there hidden fields? Perhaps hidden static fields?

Classes in a Dependency Chain

I don't know where this last idea would be knocking against the rules. In case you have made up your mind that the latest and greatest JDK 1.4.1_01 is the answer for all your compilation problems, have a look at this example:

import java.io.*;

public class ClassGenerator {
  public static void main(String[] args) throws IOException {
    int NUMBER_OF_CLASSES = Integer.parseInt(args[0]);
    for (int i=0; i<NUMBER_OF_CLASSES; i++) {
      String filename = "A" + i + ".java";
      System.out.println(filename);
      PrintStream out = new PrintStream(
        new FileOutputStream(filename));
      out.println("public class A" + i + " { private A" +
        ((i+1)%NUMBER_OF_CLASSES) + " other; }");
      out.close();
    }
  }
}

In JDK 1.4.1_01 if you run this with a value of 176 or higher, you get the following result:

The system is out of resources.
Consult the following stack trace for details.
java.lang.StackOverflowError
     at java.util.Hashtable.get(Hashtable.java:329)
     at java.util.Properties.getProperty(Properties.java:480)
     at java.lang.System.getProperty(System.java:574)
     at sun.security.action.GetPropertyAction.run(GetPropertyAction.java:66)
     at java.security.AccessController.doPrivileged(Native Method)
     at sun.io.Converters.getDefaultEncodingName(Converters.java:66)
     at sun.nio.cs.StreamDecoder.forInputStreamReader(StreamDecoder.java:69)
     at java.io.InputStreamReader.<init>(InputStreamReader.java:57)
     at com.sun.tools.javac.v8.parser.Scanner.<init>(Scanner.java:139)
     at com.sun.tools.javac.v8.JavaCompiler.parse(JavaCompiler.java:231)
     at com.sun.tools.javac.v8.JavaCompiler.complete(JavaCompiler.java:305)
     ... etc.

In JDK 1.3.1_03, I ran the test for 8000 classes, it compiled fine, but took a while to complete. So, don't be lulled into a false sense of security with the new compilers. Your long SQL Queries might now compile, but don't write programs that are too complex ;-)

Until our next newsletter...

Heinz

P.S. Please let me know if you found this newsletter particularly interesting. I have been thinking about this topic since January 2000 when I was flying back from Germany after a serious consulting expedition with my colleague Paul van Spronsen and we started bashing the JVM around while sitting 30'000 feet above the ground. You have just read thoughts that span 3 years :-)

Comments

We are always happy to receive comments from our readers. Feel free to send me a comment via email or discuss the newsletter in our JavaSpecialists Slack Channel (Get an invite here)

Related Articles

Browse the Newsletter Archive