Sunday, October 12, 2008

Dependency Injection with Guice, JUnit 4 & Get-Set problem:
I was evaluating Guice,JUnit-4,while trying out with these same I wrote a sample application using the same. & also tried out a possible solution for the problem with data conversion.
In a typical web application we also use heavily with get/set to convert from UI representation of object to back end implementation object. Many a times (In my experience most of the time) they are all heavy parallel structures required by framework (Like classic struts forces UI object to extend & ActionFormBean, limited data type support) or data type representation in the different UI models. With the advent of domain driven design we have more rich domain objects which usually contains many other data/behaviour which should not be or need not be exposed to the UI layer.
The problem with this conversion is that they are not only look dumb consuming lot of source code lines but they are error-prone & since there is no contract with back end we are forced to unit test the code as we cannot rely on compile time checks. I also tried a possible solution for this, 3 years back I tried out implementing this in a classic struts based web application & was successful in reducing the number of bugs.

The sample application is user management system. (Please note that the code has been written in such a way to show the usage Guice, JUnit & type safe data conversion & shouldn't be mistaken as real time design,there are no exceptions, validation etc...). It follows a typical MVC pattern followed in web applications.

Model Layer
User.java Represents a User domain object. As we notice it also contains the information that is populated through context (Logged in user, context, date, etc... usually through HttpSession or EJBContext) in the form of AuditInfo object. Here the fullName field (which doesn't make sense to UI) representing the firstName & fullName as 'firstName,lastName' in a single field. This is done to show how easily back end object can be refactored to accomadate client requirements. This can be even applied to data types. Since extracting of interface from any class is supported by all the IDEs, there is no coding effort required here.


IUserService - Exposing the functionality of user management service

UserServiceMockImpl.java - Implements the service by saving the content in a file using object serialization.



UI Layer
IUser.java - Now this is the trick we have IUser that's required by UI layer (which is always a subset of back end object) that's being implemented by the UI model object.

UserForm.java - UI bean honouring java bean spec & can extend the classes like ActionFormBean representing the HTML form in the screen.



controller
UserController.java - Controls the logic flow b/w UI & the back end. Here is the place we will introduce the dependency injection. In a typical web application audit information is captured in HttpSession object, since in test envioronment we will not be having references to HttpRequest, HttpResponse etc... we will inject those data with our Guice & also we will inject service implementation which could be changed without disturbing the other layers.



Unit test layer
MockBinder.java - As I am obsessed with fluent interface, generics & type safety, it's really enjoyable to wire up dependencies using Guice APIs rather than through XML.

UserTest.java - & now finally we have unit testing code with JUnit4 test case show-casing the usage of the applications.

------- Final Summary-------

  • Guice simplifies the testing a lot doing everything in java, If you just want dependency Injection, Guice beats all other frameworks (Spring...) hands down in ease of use

  • JUnit4 is much easy to use with annotations & now we don;t have to extend any class & don't have to write methds with test...(). I really liked the way we can test the Exceptions

  • Adding interface to both backend & UI objects we can get rid of get/set noise.
    Although it makes(or forces) back end to be aware of UI & other form of clients, in some cases (like we we use ORMs instead of JDBCTemplate) one time conversion logic shifts from UI to backend & it might put more development load on persistent layer team. My guess is these objections can be ignored because of the benefit of to cleaner & less error prone code. In case of distrbuted enviornment all the extra fields can be set to null to reduce the payload, but anyway number of rows are much more important than the number of columns in deciding the data size.

    Hope that this sample application help new comers to understand/appreciate the value of Guice & JUnit4 libraries.

    References;
    Guice Dependency Injection Framework from Google
  • JUnit 4 - with Java5 features & is quite different from older versions.


    Thursday, August 14, 2008

    What value could custom client SDK can provide over SOAP toolkit(s)?

    1. Single library to deal.
    Developers need not have to spend time in knowing or generating stubs & which framework to choose for for the same (XML marshaling/un-marshaling).This single library client can hide this implementation methodology by exposing easy to use APIs & hiding these techniques behind the scene using some kind of dependency injection APIs. (Jax-ws, Axis, XFire, Xstream so on...). The single library can be collection of core + optional service libraries in the same fashion provided by the Spring Like spring-jdbc.jar, spring-jndi.jar so on... with spring-core.jar as kernal providing infrastructure for all other services.

    2. Handling backward compatibility issues
    Carefully designed client SDK can solve backward/upward compatibility issues without compromising on the design elegance, If we don't want to use new SOAP version, it should be OK to use existing version most of the time. Client SDK can be designed in such a way user is shielded from SOAP schema changes that does not require recompilation or rebuild what so ever.For using new APIs user just has to drop in new library(jar) that's it. Well designed Mock objects can even save the need to goto sandbox during development.

    3. Easy to use fluent APIs.
    Domain specific language (DSL) design can bring lot of changes & provide better user experience with fail safe APIs. Fluent APIs can bring lot of +ve changes. For me this is the single most important feature.
    For example:

    MailBuilder.mail().
    .from(praveen.manvi@yahoo.com)
    .to(pmanvi@aol.com)
    .withSubject("Fluent Mail API")
    .withBody("Fluent API & DSL can make developer life simpler")
    .attach(new File("c:/test.txt")).
    .send();

    is definitely much easier to use. This is ok for 90% of users to start, if we have hooks to get into Transport & other Java Mail APIs,it can simplify the way we code & importantly understand without compromising the power for advanced users.

    4. Meaningful Exception hierarchy stack for better recoverable actions
    Users don't have to deal with SoapFault Error code. Just like Spring DAOException hierarchy translating the SQLException & SQL error codes giving meaningful RuntimeException (even exteding it to hibernate, ibatis,jpa) API providers can make developer's life easier. Like SQLException & SQL error code APIFaultException & fault code is difficult use & introduces lot of boiler plate code (because of checked exception) without providing the meaningful recovery options. Hibernate, Spring & C# have proved that checked exception is like communism, works best in theory only.

    http://static.springframework.org/spring/docs/2.5.x/reference/dao.html


    5. Automatic logging soap XML handshake (Making this switch on & off on need basis) between client & server. This comes as very handy & we can even make them asynchronous & log this information to to different data sources like file & database by providing proper hooks. This automation improves the performance as well as avoids the need to develop auditing APIs.

    6. Most of error handling/validation can done by the SDK framework, there is very less chance for error requests coming to server improving server performance. 99% of the time server will deal with valid requests

    7. Meaningful defaults for all the APIs & actions, can reduce initial learning curve & improve the perception. It can help user to get into the the details step by step instead of throwing him everything upfront.

    8. Can enforce the best practices. & develop new interfaces that are very difficult to achieve with simple plane SOAP.

    For example a utility can be developed to export the data in chunks from large files like Excel sheet, a csv file or a PDF., as the sending large files & batch processing with SOAP is not mature (mtom...)

    9. Encourage more developer community participation. Easy to use APIs improves the possibility of
    developer to develop new tools & application using the SDK especially desktop tools.

    10. Get rid technical limitation of programming technique. SOAP inherintly not object orieneted & it's procedural style of programming is not what a modren developer would like to work.

    I just evaluated different SDK provided by various well known API providers & most of them don't really provide above facilities & there is very good opportunity of 3'rd party developers to fill the gap over here.

    Same concepts can be carried out for developing C# APIs. As for as dynamic langauges(Groovy,Ruby, PHP) concerned I guess they don't need any SDK, they are smart enough to deal with XML directly without any code generation.

    Many a times developers need to design wrappers over SOAP apis -- that involves substantial effort. Client SDK can save the time & money here. Like Spring removed the necessity of having in-house framework for standard java enterprise project by not tying the implementation any specific technology but to the interface & also DSL developed with business in mind can make writing code like writing an essay.

    References:
    http://developer.ebay.com/DevZone/XML/docs/HowTo/BestPractices/JavaToJsdkToSoap.html - eBay SOAP SDK sample
    https://www.paypal.com/IntegrationCenter/ic_sdk-resource.html - Paypal API
    http://code.google.com/p/fluentmailapi/ - Fluent Mail APIs
    http://www.javaworld.com/javaworld/jw-08-2008/jw-08-dsls-in-java-3.html - DSL in java
    http://www.toolsforteams.com/code/facebook-api/ - Facebook APis
    http://code.google.com/p/java-twitter/ - Twitter APIs
    http://www.oracle.com/technology/tech/soa/mastering-soa-series/part4.html - Rich UI for web services
    http://www.javabeat.net/articles/29-introduction-to-google-guice-5.html - Guice Samples

    Thursday, July 24, 2008

    Some wise words on design - I recently was listening to a presentation in InfoQ on design principles of successful large scale enterprise financial application based on Spring & Hibernate. I noted some points while listening to that. Although these are well known established principles I thought it's easier to refer & document when ever we need to come up new design document.
    • Minimize data movement
    • Task Parallel execution where ever possible
    • Physically partition of data
    • Optimized reads & writes of volatile data
    • Minimize contention -- Different connection pooling
    • Asynchronous Decoupling
    • Complete Business Logic performed by data
    • Caching frequently accessed data
    Like all popular scalable architecture (eBay, Yahoo) the transaction (distrubuted) & state management were simple (stateless & transactionless for 90% of operations)
    JDBCTemplate - fantastic example of Dependency injection
    Spring promotes good design - modularity, loose coupling separation of concerns
    Testability provides better confidence for refactoring
    Hibernate promotes domain driven design - 2'nd level caching is gr8

    Contents of design document can be divided into following categories:
    • Architectural review
    • Jumpstart programs
    • Deployment strategy
    • Platform change
    • Proof of concept
    • Performance tuning
    • Development
    Capacity Planning questions:
    • How much memory footprint the application requires
    • How many threads the application components need
    • How many connections each application has to handle
    • How many system resources like file handles are required
    • How much parallelism is attainable in a single process
    • How many CPUs can be leveraged at the same time from within the same hardware
    • What are the synchronization primitives available in case of multiple processes and nodes
    • How we coordinate shared accesses and leases
    • The allowable number of nodes that a unit of application job can cross, taking into consideration the performance criterion
    I also listened to Joshua Bloch's presentation on design guide lines, Joshua Bloch is currently Java Architect at Google & has great influence on Java's future. He also influenced (JDK5) & wrote many JDK classes as distinguished Engineer at Sun. His assertions carries lot of weight.
    Importance of API design for internal and external use.
    • Public APIs are forever - one chance to get it right
    • Good code is modular–each module has an API
    • Thinking in terms of APIs improves code quality
    • Easy to use, even without documentation
    • Don’t let implementation details “leak” into API
    • Make classes and members as private as possible
    I guess the most important one was "when in doubt, don't provide the API, leave it".

    "There's a natural law in programming language and API design : as backwards compatibility increases, elegance decreases." - Bill Venners
    So be minimalist,
    because of the compatibility requirement, it's much easier to put things in than to take them out. So don't add anything to the API that you're not sure you need

    Sunday, July 13, 2008

    Bio-Rhythmic Cycles - I got curious about bio-rhythmic cycles & wanted to try out the the results on my life.

    For those who don't know what bio rhythmic cycles are, here is explanation:

    The theory of biorhythms claims that one's life is affected by rhythmic biological cycles, and seeks to make predictions regarding these cycles and the personal ease of carrying out tasks related to the cycles. These inherent rhythms are said to control or initiate various biological processes and are classically composed of three cyclic rhythms that are said to govern human behavior and demonstrate innate periodicity in natural physiological change: the physical, the emotional, and the intellectual (or mental) cycles. Others claim there are additional rhythms, some of which may be combinations of the three primary cycles. Some proponents think that biorhythms may be potentially related to bioelectricity and its interactions in the body


    I wrote a sample application in swing, which basically takes the date of birth as input & generate the charts.


    Here is the sample screenshot, I used JFreecharts & some Date manipulating APIs to create above application. BTW is there any way I can show applet in this blog? Please let me know.

    The results were interesting, Most of the events in my life had some kind of correlation with the above graph & explanation:

    Physical cycle - 23 days; coordination strength well-being

    Emotional cycle - 28 days; creativity sensitivity mood perception awareness

    Intellectual cycle - 33 days; alertness analytical functioning logical analysis memory or recall communication

    I also tried correlate our god Sachin's best innings with these cycles & surprisingly, cycle behaviour used to match with his best time :-)

    Anyway I will try out with some other important figures before coming to conclusion these cycles have any impact or it's just a humbug. BTW, I am little skeptical about all astrology stuff.

    So I am planning to develop a RESTful sample as well sample mobile application accepting date of birth & generating the charts possibly using Google charts instead of Jfreecharts

    Wednesday, July 09, 2008

    GeoNames.org - provides geocoding services. GeoNames provide a huge databases of place names (with coordinates) organized in a hierarchy (eg. country/state/region/province/etc...). Also there is a query that allows to retrieve the geo tags within a range from a specific point. Geonames is integrating geographical data such as names of places in various languages,elevation, population, alternate names, Administrator division type,continent, Address,timezone & weather observation.

    Geonames provides RESTful based webservices with both XML & JSON rendering. These seems to defined by seasoned developer & have Style enum (Short,Medium,Long, Full) for defining result verbosity & many such developer friendly criteria APIs.

    For off-line applications, geographical database is available for download free of charge under a creative commons distrubution license & can leverage the service by downloading all the data. Geonames.org claims that it has already serving up to over 3 million web service requests per day & has the ability scale.

    I guess applications (especially free & open source ones) have excellent opportunity to integrate with this service. Web applications can be relieved from maintaining the huge database of geo services. They can just store the geo-ids & can rely geonames to fetch the details. Hierarchical data is always to difficult store & fetch in standard RDBMS & now there is no need to maintain this data. GeoNames includes the reference implementation in all popular languages. I just checked the java implementation & it was not that good. It could be have been made more easier (or fluent).

    The important point to note is that high-profile users of GeoNames include Slide.com & LinkedIn.

    Reference
    http://www.geonames.org/export/ws-overview.html
    http://www.geonames.org/export/place-hierarchy.html
    http://en.wikipedia.org/wiki/GeoNames

    Sunday, June 22, 2008

    Some notes on web services, REST & Fluent Interface. Theoretically we can make use any language to write web service client, & of course that's the exact reason web service was invented offering loose coupling for implementing distributed computing environment for businesses. But there were many requirements (schema definition/validaion, security,performance, service discovery, SLAs,BLAs, language etc..) to achieve high cohesion, hence SOAP, XML, UDDI & WSDL were necessary. I guess for most of integrations these technologies make developer's life un-necessarily tougher & are forced to learn/unlearn these vocabulary.They are all XML-RPC in disguise & may not be much better than the existing binary RPC solutions (RMI,CORBA, COM...). I am leaning towards REST proponents who are arguing that this complexity is not really worth of benefits it offers & is really waste of time, bandwidth & finally money. These are looking similar to EJBs as I am going through this book (considered as first printed book on REST)
    If we see the existing web service landscape (I am referring WSDL, SOAP only), most of them are giving SDK for different languages (eBay SDK, Google Data API...) proving that SDK is more important than the WSDL. SDK always beats the SOAP toolkit hands down.

    SOAP prevent independent evolution of client & server. For exampleSOAP toolkits provide (Jax-ws,Axis) code generation with strongly typed APIs, even adding new optional parameter, system will break until we generate thte stub again,Which basically diefies the logic of loose coupling.

    With REST,HTML forms provide default & hidden variables,Service end point change is effortless, we can have re-directs based on situation & also partition, scale well depending on the requirements.Means don't have redeploy & test specifically for this.

    separating reads from writes. SOAP has no inherent support for this, although we can design.
    caching GET requests including hardware solutions, Caching @ HTTP always yields better results.
    compression: Since REST uses HTTP, you can use compression such as gzip.
    No need for additional jars or library
    explorability -clicking around a RESTful API in their browser & use right away.
    REST architecture serves as the basis for the most biggest and most successful information system the world has ever seen

    Cons of REST
    security: Though, security considered as weakest area of REST not sure why (HTTPS, authentic)
    Tools will make working with SOAP just simple, IDE & tools cannot provide same kind features as SOAP to REST.
    (Anyway we had same arguments for using EJB in the past as well)


    Here is a sample Java APIs for web service APIs to help out developer community to directly working java instead of WSDL. I noted following points from going through the code.

    • Contains 28 methods & 768 lines


    • Expects user to update the Source code or use the properties file to inject startup values (Namely username, password, AccountID so on & so forth) & is not easy to pass/change these values @ runtime.


    • Uses Exception, RemoteException providing little recovery options from service faults & forcing client code to become ugly, clearing suggesting RuntimeExceptions are better suited here to use.


    • One single class for many APIs, making shpping client API size bigger if we want to deal with single API only


    • No way to see the SOAP XML files that get handshaked b/w client & server.


    • Uses Axis for marshalling/unmarshiling, We have Sun benchmarks suggesting latest jax-ws API/implementaion beat Axis/xfire hands down.


    • The API is not fluent (or user friendly) & does not make use latest JDK1.5 APIs


    • Uses lot of Array values as input/output params making client code uglier
    Before providing my solution for this I do understand that I don't have much knowledge of the context (of client applications) using the APIs. So please be adviced I might have over-looked many issues.

    I generated stub code using jax-ws APIs & netbeans. JAX-WS is the successor to JAX-RPC. It requires Java 5.0, and is not backwards-compatible to JAX-RPC.Netbeans has amazing plugins for about everything! I also wrote some wrappers around the campaign service API & here is how client code looke in the JUnit test case. (BTW JUnit4 has amazing features with POJO annotations & I guess this unit testing library is the best testimonial of effective usage of the java annotations.)

    Now JDK6 users can turn their normal POJO in to web services and deploy it with ease without getting to deal with different web services stack. Before JDK6 one had to download some webservices toolkit and learn how to use

    @Test
    public void testGetCampaigns() {

    List<Campaign>campaigns = new CampaignServiceTemplate() {
    @Override
    public YAccount getYAccount() {
    return YAccount.builder.userName("uyiui")
    .password("test123")
    .license("uyiuu7687687gg")
    .masterAccountID("988533")
    .accountId("2086880350")
    .build();
    }
    }.getCampaigns(13897897l,797897l, 89678l);
    Assert.assertNotNull(campaigns);
    }

    Above code definately looks much cleaner easy to use. It makes use of generics, varargs, SOAPHandler java-ws interceptor,template pattern & fluent interface to make code look simpler & robust by validating all the inputs that takes care of all the issues I listed above sample code.

    One task which I wanted to avoid was generating the source using tools Axis/jax-ws, if I want to write some adhoc testing I don't think doing in java make really sense. Ruby, Groovy where dynamic class reference options are amazing solutions here.

    For stock quote client Groovy code looks something like this.

    import groovyx.net.ws.WSClient
    proxy = new WSClient("http://www.webservicex.net/stockquote.asmx?WSDL", this.class.classLoader)
    quote = proxy.GetQuote('YHOO')
    print quote

    We, Java programmers are unable to make use of Groovy APIs like this, we will be the losers. Coding can't be simpler/easier than this :-)

    But don't get me wrong, Java has a place & will continue to occupy the major portions of software development as Joshua Bloch says "One thing that makes Java such a pleasure to use is that it is a safe language."

    So when we stick to Java,fluent interface or easy to use APIs have lot of importance.A fluent interface is a DSL embedded in a language,it consists of a set of classes that have been organized to allow you to write code that nearly reads as English essay. Beautiful Code is often associated with essay-writing.

    Reference:

    Immutable objects improves the code reliability.

    Joshua bloch discusses about the design consideration

    Jax-ws faster than Axis - Benchmarks

    Java options for writing web service clients

    SoapUI has wonderful set of applications & plugins to make web service testinf very easy task. I tried out their NetBeans plug in , It's simply amazing.SoapUI is a great tool for any developer who wants to test web services to verify that they are working correctly.

    Good DZone link explaining functional testing using these APIs. I will update my experience after trying these out.

    GoAPI -> Good for searching APIs.
    Restas Architecture - "
    Simpler is better, and REST is generally simpler than SOAP"

    Sunday, June 08, 2008

    Java Logging Frameworks - why to use sl4j?
    One of my friend was asking why should I with sl4j over log4j & java logging. My answer was "because it was written by the same author who wrote log4j & is now adopted by finest java framework authors (Tapestry & Wicket)" sl4j is the best available way to solve logging problems. It's latest & hence best suited for any new projects. This way some decisions are easy & safe to make. Logging is one of the most boring & non-debatable topic. At least for issues like this it's best to follow the best minds. During my initial days of coding I have seen mindless java wrappers over existing logging frameworks without any value addition. Log4j is definitely superior option to standard java logging both in terms of speed & availability of appenders. But when we write client apps or want to make our jar size compact & cannot afford to include log4j.jar java logging might be still better option which is rare case anyway. With log4j , in order to bypass the expensive string concatenation if statement was necessary. It would have been nice if APIs to provide some features that would alleviate things like this, and maybe make it easier to toggle the display of log statements for code readability.
    if (log.isDebugEnabled()){
    log.debug("Logging " + String.valueOf(a) + "some information :"+someBigEntity.toString());
    }

    I never liked double if conditions. Less if conditions directly proportional to good code quality.
    sl4j utilizes parameterized messaging (like logger.debug("The new value {} is replacing {}.", newValue, oldValue);) & does'nt have class loader & memory leak problem. As JVM is becoming smarter & smarter with every new version, micro bench marking knowledge are becoming totally irrelevant. The experience is becoming baggage & anti pattern most of the time as for micro bench marking is concerned.

    Here is one experiment I did with latest JVM with decompilation .

    public class JavaLogTest {
    // Get status from Logger.isDebugEnabled();
    private static final boolean DEBUG=false;
    public static void main(String[] args) {
    if(DEBUG) {
    System.out.println("Performance issue");
    debug("Some Big"+"Concatenation here"+"& also creation of objects.... "+new java.util.Date()+" which is not good for performance");
    }
    debug("testing"+" ! testing again");

    debug("testing"+new Integer(10).toString() +new java.util.Date());
    }
    public static void debug(String str){
    System.out.println(str);
    }
    }

    I decompiled the class with decompiler & here is the result what I got;

    import java.io.PrintStream;
    java.util.Date;
    public class JavaLogTest{
    private static final boolean DEBUG = false;
    public JavaLogTest() { }
    public static void main(String args[]) {

    debug("testing ! testing again");
    debug((new StringBuilder()).append("testing").append((new Integer(10)).toString()).append(new Date()).toString());
    }
    public static void debug(String s) {

    System.out.println(s);
    }
    }

    & the results were interesting;
    1. It doesn't make sense to use StringBuffer for appending, JVM is intelligent to make use of StringBuilder which is not synchronized also
    2. When we append strings only there is no need to use if statement even with log4j
    3. We can take out the code from the byte code (pre-processor directive DEBUG) which can reduce the total jar size
    I found scala solution for this was interesting. Dynamic languages indeed bring lot of paradigm shift in thinking!

    Thursday, May 15, 2008

    Java Language Vs Java Platform - The paradox :
    Now it looks like Java is at crossroad both as language & as platform. There are 2 formidable arguments for both Java platform & as Java language. The companies like Thoughtworks are arguing that Java as programming language has reached it's dead end & dynamic languages JRuby & Groovy are the future. Bruce eckel echoed the same saying Scala is "the current best exit strategy for Java" & adding new features (Closures etc...) will make language complex, big & finally unusable.
    Google has provided interesting twist to the above thinking with GWT, GData APIs & Android. Google is saying 'java as language is great, it's static typing, IDEs, large user base helps to create robust application even by mediocre programmers' but not the platform. So GWT is path-breaking technology in that sense as it compiles the java code to javascript code instead of bytecode & so the android.
    So what's the truth & future?
    Let's analyze how Java performed on it's promise:
    Java as a language - My guess is it has done fairly good job here. Market is always right, now we have java programmers outnumbering any other language.
    Java as library - If you look into sourceforge.net or any other common place producing software code Java is the #1 language inspite of great productivity provided by the new dynamic languages like Ruby, now we have superior set of libraries in java compared to any other language.
    Java as security model - Done fairly good job.
    Java as platform - Now we have profilers monitoring everything, Java is becoming full fledged operating system now.
    Java has done fairly well on all these account & looks is going to influence in the future as well, we cannot stop Java from evolving & it has to add new features in order to remain competetive (generics, closures & so on...) & has to compete with C# which is adding new features at amazing speed.
    Steve Yegge has written a brilliant article defending dynamic languages. Polygot programming (using multiple languages) is also becoming very popular among architects.

    Monday, April 14, 2008

    Java Performance Considerations: Here is some collection of notes for making java applications perform better & scale better.Usually we need consider the java application performance at 3 levels:
    1. Refactoring/writing java code.
    2. JVM fine tuning depending on the context.
    3. Application server parameter tuning. (server configuration files)
    Above all are equally important but I guess 2 & 3 are important for application administrator. I believe every java developer should be administrator as well, so all are important to be considered. General strategies for faster programs, like purging obsolete code, and the well-known 80-20 rule (optimizing the 20 percent of your code that consumes 80 percent of processing time) can be applied while optimizing the existing code or reviewing code . Anyway it's painful to refactor existing code written with old techniques & also by some one else.Whether you are refactoring or writing new code performance improvement techniques should be helpful.By end of the day the speed/efficiency of program depends the # of cpu cycles & memory space that were utilized for accomplishing a task. Balance between the memory & cpu cycles are contextual & I guess that's where technical knack/experience comes into picture.
    One of the main technical reason for using the java platform is the availability of inbuilt profiling inspite of productivy power provided by the functional/scripting languages, Profiling provides details about the # of threads, memory utilization & the CPU usage.
    We have both commerial & free tools for profiling a application. JConsole/VisualVM & Netbeans inbuilt profiler have fairly good options to generate required reports.
    I found Visual VM as the best option for profiling the application. It's lightweight & extremely non-intrusive.
    Saving cpu cycles, network calls & keeping source code compact should be the main driving factor for improving performance. Most of the performance improving techniques are obsolete with modren JVMs. For example the usage of StringBuilder (Or using + operator) against the synchronized StringBuilder are becoming non issue with compiler, VM becoming more mature in handling these automatically & even there is no need inline (using final variables).
    So micro benchmarking & refactoring of java code has very less relevance in improving performance of java application. Upgrading JDK version should automatically push the performance by 20-30% :-)

    Performance related decisions while coding/designing happes based on CPU Vs Memory, I tried out printing sizeOf various java objects.

    new SimpleDateFormat() : 4248 bytes

    StrigBuffer : 72 bytes
    StringBuilder : 72 bytes

    new String() : 40 bytes
    new String("1234") : 48 bytes

    Boolean.TRUE : 16 bytes
    new Object() : 8 bytes
    new Integer(2) : 16 bytes

    new Vector() : 80 bytes
    new ArrayList(0) : 40 bytes
    new ArrayList() : 80 bytes
    Collections.EMPTY_LIST : 16 bytes
    Collections.emptyList() : 16 bytes

    class T { } 24 bytes
    class T1 { int a=10; } 24 bytes
    class T2 { int a=10; SimpleDateFormat sdf=null; } 32 bytes
    class T3 { int a=10; Object obj=null;} 32 bytes
    class T4 { int a=10; int getA() { return a; } void setA(int _a) { a=_a;} } 24 bytes


    JVM Options:
    Josep D. Mocker has done a wonderful job of collecting JVM options in a single place - this compilation will be more useful if any real time problems were solved with these options.There is well written document about profiling about the profiler by NetBeans, but applicalbe to java in general
    OutOfMemoryException->Most of the time this problem can be sloved by setting maximum memory, but even after setting to hight value (>2GB) there is possibility of outofmemory error because of permanent generation memory that is allocated for classloader.
    The permanent generation is allocated outside of the normal heap and holds objects of the VM itself such as class objects and method objects. If we have programs that load many classes (like deployment of many BPEL processes in a batch), we may need a larger permanent generation. It's a separate heap inside the heap, the standard value for this is 48MB.
    This bug can be solved by setting these 2 JVM parameters:-XX:PermSize=256m -XX:MaxPermSize=256m
    Since the sizing of this is done independently from the other generations, this means that even if you setup a heap of 2Gb, you might still encounter problems in the permanent generation cause if you do not specify this it will fallback on the defaults .The same value has been assigned for 2 values because we want to minimize large garbage collection here.
    In the past I have tried to check with various parameters & could not see any improvements whatsoever on performance.
    Sun Hot-Spot Engineering team says JRE6 works best with default options rather than fine tuning; You can look into these benchmarks for details. Long back I wrote simple code to do profiling, you can re-use this poor man's profiler if you don;t have patience to set up a profiler or if you want monitor some portion of the the code,feel free to use this code. I also likes this simple idea for load testing simple web applications.
    Server Fine tuning:
    Weblogic (BEA,Oracle), Websphere (IBM)have wealth of information about the configuration in their sites. I found these 20 Tips for Using Tomcat in Production useful .
    JVM Tuning options - Good collection of tuning options.

    Monday, February 11, 2008

    2008 - Programming Technologies to look for:
    Comet Programming:
    a web server sends data to a web browser (client) asynchronously without any need for the client to explicitly request it, So it's a server side AJAX (supported in Tomcat6). This is great feature & opens up lot of possibilities with server side events. It's like RMI's Callbackinterface where client can listen to events from UnicastRemoteObject.It has great potential where it's not advisable for client to poll . Servlet2.5/Tomcat6 will support Comet programming & I am sure all the frameworks will start laveraging the same. But I am little skeptical about the popularity of this; although the statements from several authors are exciting.

    Web Technology Frameworks:
    I tried out some samples with component frameworks. Since I have been working swing programming model, it was so easy to write Listener anonymous classes. I wrote samples with GWT, wings & Echo.I did more work with wingS as they were using YUI library as backbone.
    I also followed Wicket & Tapestry very closely. I am yet to work with real project to really appreciate the necessity of having to define HTML & Java artifacts for each page. zk framework,JSF & many other frameworks compete for this space. Wicket is my favorite component oriented framework although I am not sure it's scalability capabilities. As tapestry claims to work well with highly scalable sites, there is neither such explicit asserstions or sample sites avaiable for Wicket to prove that it's possible to create highly scalable sites, anyway I am excited to work with new Wicket release that will get released in coming months leveraging JDK1.5 features. Although personally I love swing programming model,my guess is page-centric MVC is going to be mainstream web programming model in 2008 as well. RoR, Grails are offering great productivity gains in this space. Grails1.0 got released in Feb 1'st week & I believe that 2008 will be Grails year as for as page centric web sites development is concerned & it's going to be Wicket year for developing complex web applications.

    Closures & DSLs with dynamic languages:
    Last year there was lot of discussion on aspect oriented programming & dependency injection. Dynamic lanaguages have brought lot changes to the programming parctices with Groovy & Ruby leading the way.It was difficult to commit to any of these two languages (Groovy & Ruby (JRuby) ) , Although Groovy looks to be easiest to select for Java programmer I was little skeptical as Sun choose to invest over Ruby/JRuby over Groovy & Thoughtworks , a technically intelligent company invested in Ruby. But after trying out few samples I am convinced that Groovy is the best way for dynamic programming for Java programmers. Closures, DSLs & weak typing languages are simply unstoppable on JVM. I bought Groovy In Action book.

    Closures:-> As a swing programmer, I have been using anonymous classes a lot. Spring's JDBCTemplate... & other template wrappers were huge hits for me. Not only closures will help solving resource handling( File, Database, User Interfaces, transactions (open<>close)), they also help better thinking (recurse & iterative). Joel explained them well here. The main point I guess with closures is that we can assign a variable to a method & use just like any object & you can pass on these parameter to a other methods as explained superbly here.
    I used to wonder about the reason for final variable only accessibility in anonymous classes. I found explanation here, where Guy Stele says java should not allocate any memory in heap without "new" call & allowing non-final variables violates this. He also says dynamic class loading ,reflection already violates this principle. Anyway it's clear that anonymous classes are not equal to closures. Paul Graham argued brilliantly for closures saying succinctness is power long back. Paul's concern over java for not providing power & Joel concern of java being simple, I guess are now solved with dynamic languages.With James Gosling asserting that it was resource/time constraint that prevented Closure inclusion in original JDK, closures are likely to reach java mainstream this year.

    Some useful links on this topic:
    Review of Groovy/Grails books from Matt Raible
    Higher Order Functions with Groovy - GINA book doesn't cover this topic
    Closures samples
    Some closure theory
    Closure proposals from Bob lee (Guice creator) & Josh Bloch where they argue removal of new keyword to support closures.
    comparison of different closure proposals:
    Closure Debate from IBM
    Language comparison:
    A closure sample in Groovy

    Some how Fluent interfaces were not familiar or popular in Java world. Standard Java bean definition is not really good way to define & populate objects. Martin Fowler defined them as back as Dec 05. Although fluent interface & builder pattern cannot be implemented in java easily my guess it's happening mainly because of Java Bean spec (which are well used by frameworks(ex.hibernate) & tools (IDEs)), we can also write java code like,
    new Person("test","test").email("praveenm").firstName("Praveen").lastName("Manvi"); instead of calling set method on each variable. Many JDK standard classes like StringBuffer, ProcessBuilder already using this. I guess more & more APIs will start using fluent interfaces like JAXB2 commons.I am also planning to write a plug-in in netbeans/eclipse to extract fluent interfaces from existing classes. Venkat provides a very neat explanation about the necessity & usage of DSLs & XML limitations to be used as full fledged DSL.
    Cloud Computing, Grid computing, Virtualization might reach masses this year.
    OSGI - OSGi is going to change the deployment and run time model for enterprise apps.(Although I am not able fully appreciate/understand the value even after going through literature, The immedeate advanatage I see as of now as a Swing programmer that I can run may java apps with single JVM which is painful now), Although Oracle, Bea (Oracle?), IBM & more importantly Spring showing lot of interest & excitement, Sun doesn't seem to be that excited
    So exciting days ahead...
    Bookmark and Share