Java Performance Considerations: Here is some collection of notes for making java applications perform better & scale better.Usually we need consider the java application performance at 3 levels:

1. Refactoring/writing java code.
2. JVM fine tuning depending on the context.
3. Application server parameter tuning. (server configuration files)

Above all are equally important but I guess 2 & 3 are important for application administrator. I believe every java developer should be administrator as well, so all are important to be considered. General strategies for faster programs, like purging obsolete code, and the well-known 80-20 rule (optimizing the 20 percent of your code that consumes 80 percent of processing time) can be applied while optimizing the existing code or reviewing code . Anyway it's painful to refactor existing code written with old techniques & also by some one else.Whether you are refactoring or writing new code performance improvement techniques should be helpful.By end of the day the speed/efficiency of program depends the # of cpu cycles & memory space that were utilized for accomplishing a task. Balance between the memory & cpu cycles are contextual & I guess that's where technical knack/experience comes into picture.
One of the main technical reason for using the java platform is the availability of inbuilt profiling inspite of productivy power provided by the functional/scripting languages, Profiling provides details about the # of threads, memory utilization & the CPU usage.
We have both commerial & free tools for profiling a application. JConsole/VisualVM & Netbeans inbuilt profiler have fairly good options to generate required reports.
I found Visual VM as the best option for profiling the application. It's lightweight & extremely non-intrusive.
Saving cpu cycles, network calls & keeping source code compact should be the main driving factor for improving performance. Most of the performance improving techniques are obsolete with modren JVMs. For example the usage of StringBuilder (Or using + operator) against the synchronized StringBuilder are becoming non issue with compiler, VM becoming more mature in handling these automatically & even there is no need inline (using final variables).
So micro benchmarking & refactoring of java code has very less relevance in improving performance of java application. Upgrading JDK version should automatically push the performance by 20-30% :-)

Performance related decisions while coding/designing happes based on CPU Vs Memory, I tried out printing sizeOf various java objects.

new SimpleDateFormat() : 4248 bytes

StrigBuffer : 72 bytes
StringBuilder : 72 bytes

new String() : 40 bytes
new String("1234") : 48 bytes

Boolean.TRUE : 16 bytes
new Object() : 8 bytes

new Integer(2) : 16 bytes

new Vector() : 80 bytes
new ArrayList(0) : 40 bytes
new ArrayList() : 80 bytes
Collections.EMPTY_LIST : 16 bytes
Collections.emptyList() : 16 bytes

class T { } 24 bytes
class T1 { int a=10; } 24 bytes
class T2 { int a=10; SimpleDateFormat sdf=null; } 32 bytes
class T3 { int a=10; Object obj=null;} 32 bytes
class T4 { int a=10; int getA() { return a; } void setA(int _a) { a=_a;} } 24 bytes

JVM Options:
Josep D. Mocker has done a wonderful job of collecting JVM options in a single place - this compilation will be more useful if any real time problems were solved with these options.There is well written document about profiling about the profiler by NetBeans, but applicalbe to java in general
OutOfMemoryException->Most of the time this problem can be sloved by setting maximum memory, but even after setting to hight value (>2GB) there is possibility of outofmemory error because of permanent generation memory that is allocated for classloader.
The permanent generation is allocated outside of the normal heap and holds objects of the VM itself such as class objects and method objects. If we have programs that load many classes (like deployment of many BPEL processes in a batch), we may need a larger permanent generation. It's a separate heap inside the heap, the standard value for this is 48MB.
This bug can be solved by setting these 2 JVM parameters:-XX:PermSize=256m -XX:MaxPermSize=256m
Since the sizing of this is done independently from the other generations, this means that even if you setup a heap of 2Gb, you might still encounter problems in the permanent generation cause if you do not specify this it will fallback on the defaults .The same value has been assigned for 2 values because we want to minimize large garbage collection here.
In the past I have tried to check with various parameters & could not see any improvements whatsoever on performance.
Sun Hot-Spot Engineering team says JRE6 works best with default options rather than fine tuning; You can look into these benchmarks for details. Long back I wrote simple code to do profiling, you can re-use this poor man's profiler if you don;t have patience to set up a profiler or if you want monitor some portion of the the code,feel free to use this code. I also likes this simple idea for load testing simple web applications.
Server Fine tuning:
Weblogic (BEA,Oracle), Websphere (IBM)have wealth of information about the configuration in their sites. I found these 20 Tips for Using Tomcat in Production useful .

JVM Tuning options - Good collection of tuning options.