Client-side server monitoring with Jolokia and JMX

The choice of Java monitoring tools is tremendous (random selection and order powered by Google):

javamelody
psi-probe
JVisualVM
jconsole
JAMon
Java JMX Nagios Plugin	N/A

Besides, there are various dedicated tools e.g. for ActiveMQ, JBoss, Quartz scheduler, Tomcat/tcServer... So which one should you use as an ultimate monitoring dashboard? Well, none of them provide out-of-the-box features you might require. In some applications you have to constantly monitor the contents and size of a given JMS queue. Others are known to have memory or CPU problems. I have also seen software where system administrators had to constantly run some SQL query and check the results or even parse the logs to make sure some essential background process is running. Possibilities are endless, because it really depends on the software and its use-cases. To make matters worse, your customer doesn't care about GC activity, number of open JDBC connections and whether this nasty batch process is not hanging. It should just work.

In this post we will try to develop easy, cheap, but yet powerful management console. It will be built around the idea of a single binary result – it works or not. If this single health indicator is green, no need to go deeper. But! If it turned red, we can easily drill-down. It is possible because instead of showing hundreds of unrelated metrics we will group them in a tree-like structure. The health status of each node in a tree is as bad as the worst child. This way if anything bad happens with our application, it will bubble-up.

Automatically generating WADL in Spring MVC REST application

Last time we have learnt the basics of WADL. The language itself is not as interesting to write a separate article about it, but the title of this article reveals why we needed that knowledge.

Many implementations of JSR 311: JAX-RS: The Java API for RESTful Web Services provide runtime WADL generation out-of-the-box: Apache CXF, Jersey and Restlet. RESTeasy still waiting. Basically these frameworks examine Java code with JSR-311 annotations and generate WADL document available under some URL. Unfortunately Spring MVC not only does not implement the JSR-311 standard (see: Does Spring MVC support JSR 311 annotations?), but it also does not generate WADL for us (see: SPR-8705), even though it is perfectly suited for exposing REST services.

For various reasons I started developing server-side REST services with Spring MVC and after a while (say, thirdy resources later) I started to get a bit lost. I really needed a way to catalogue and document all available resources and operations. WADL seemed like a great choice.

Fortunately Spring framework is open for extension and it is easy to add new features based on existing infrastructure if you are willing to dig through the code for a while. In order to generate WADL I needed a list of URIs that an application handles, what HTTP methods are implemented and – ideally – which Java method handles each one of them. Obviously Spring does that job already somewhere during boot-strapping MVC DispatcherServlet - scanning for @Controller, @RequestMapping, @PathVariable, etc. - so it seems smart to reuse that information rather then performing the job again.

Guess what, it looks like all the information we need is kept in an oddly named RequestMappingHandlerMapping class. Here is a debugger screenshot just to give you an overview how rich information is available:

But it gets even better: RequestMappingHandlerMapping is actually a Spring bean which you can easily inject and use:

Gentle introduction to WADL (in Java)

WADL (Web Application Description Language) is to REST what WSDL is to SOAP. The mere existence of this language causes a lot of controversy (see: Do we need WADL? and To WADL or not to WADL). I can think of few legitimate use cases for using WADL, but if you are here already, you are probably not seeking for yet another discussion. So let us move forward to the WADL itself.

In principle WADL is similar to WSDL, but the structure of the language is much different. Whilst WSDL defines a flat list of messages and operations either consuming or producing some of them, WADL emphasizes the hierarchical nature of RESTful web services. In REST, the primary artifact is the resource. Each resource (noun) is represented as an URI. Every resource can define both CRUD operations (verbs, implemented as HTTP methods) and nested resources. The nested resource has a strong relationship with a parent resource, typically representing an ownership.

A simple example would be http://example.com/api/books resource representing a list of books. You can (HTTP) GET this resource, meaning to retrieve the whole list. You can also GET the http://example.com/api/books/7 resource, fetching the details of 7th book inside books resource. Or you can even PUT new version or DELETE the resource altogether using the same URI. You are not limited to a single level of nesting: GETting http://example.com/api/books/7/reviews?page=2&size=10 will retrieve the second page (up to 10 items) of reviews of 7th book. Obviously you can also place other resources next to books, like http://example.com/api/readers

The requirement arose to formally and precisely describe every available resource, method, request and response, just like WSDL guys were able to do. WADL is one of the options to describe “available URIs", although some believe that well-written REST service should be self-descriptive (see HATEOAS). Nevertheless here is a simple, empty WADL document:

<application xmlns="http://wadl.dev.java.net/2009/02">
    <resources base="http://example.com/api"/>
</application>

Nothing fancy here. Note that the <resources> tag defines base API address. All named resources, which we are just about to add, are relative to this address. Also you can define several <resources> tags to describe more than one APIs. So, let's add a simple resource:

Enabling JMX in Hibernate, EhCache, Quartz, DBCP and Spring

Continuing our journey with JMX (see: ...JMX for human beings) we will learn how to enable JMX support (typically statistics and monitoring capabilities) in some popular frameworks. Most of this information can be found on project's home pages, but I decided to collect it with few the addition of some useful tips.

Hibernate (with Spring support)

Exposing Hibernate statistics with JMX is pretty simple, however some nasty workarounds are requires when JPA API is used to obtain underlying SessionFactory

class JmxLocalContainerEntityManagerFactoryBean() extends LocalContainerEntityManagerFactoryBean {
 override def createNativeEntityManagerFactory() = {
  val managerFactory = super.createNativeEntityManagerFactory()
  registerStatisticsMBean(managerFactory)
  managerFactory
 }

 def registerStatisticsMBean(managerFactory: EntityManagerFactory) {
  managerFactory match {
   case impl: EntityManagerFactoryImpl =>
    val mBean = new StatisticsService();
    mBean.setStatisticsEnabled(true)
    mBean.setSessionFactory(impl.getSessionFactory);
    val name = new ObjectName("org.hibernate:type=Statistics,application=spring-pitfalls")
    ManagementFactory.getPlatformMBeanServer.registerMBean(mBean, name);
   case _ =>
  }
 }

}

Note that I have created a subclass of Springs built-in LocalContainerEntityManagerFactoryBean. By overriding createNativeEntityManagerFactory() method I can access EntityManagerFactory and by trying to downcast it to org.hibernate.ejb.EntityManagerFactoryImpl we were able to register Hibernate Mbean.
One more thing has left. Obviously we have to use our custom subclass instead of org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean. Also, in order to collect the actual statistics instead of just seeing zeroes all the way down we must set the hibernate.generate_statistics flag.

@Bean
def entityManagerFactoryBean() = {
 val entityManagerFactoryBean = new JmxLocalContainerEntityManagerFactoryBean()
 entityManagerFactoryBean.setDataSource(dataSource())
 entityManagerFactoryBean.setJpaVendorAdapter(jpaVendorAdapter())
 entityManagerFactoryBean.setPackagesToScan("com.blogspot.nurkiewicz")
 entityManagerFactoryBean.setJpaPropertyMap(
  Map(
   "hibernate.hbm2ddl.auto" -> "create",
   "hibernate.format_sql" -> "true",
   "hibernate.ejb.naming_strategy" -> classOf[ImprovedNamingStrategy].getName,
   "hibernate.generate_statistics" -> true.toString
  ).asJava
 )
 entityManagerFactoryBean
}

Here is a sample of what can we expect to see in JvisualVM (don't forget to install all plugins!):

Spring pitfalls: transactional tests considered harmful

One of the Spring killer-features is an in-container integration testing. While EJB lacked this functionality for many years (Java EE 6 finally addresses this, however I haven't, ekhem, tested it), Spring from the very beginning allowed you to test the full stack, starting from web tier, through services all the way down to the database.

Database is the problematic part. First you need to use in-memory self-contained database like H2 to decouple your tests from an external database. Spring helps with this to a great degree, especially now with profiles and embedded database support. The second problem is more subtle. While typical Spring application is almost completely stateless (for better or worse), database is inherently stateful. This complicates integration testing since the very first principle of writing tests is that they should be independent on each other and repeatable. If one test writes something to the database, another test may fail; also the same test may fail on subsequent call due to database changes.

Obviously Spring handles this problem as well with a very neat trick: prior to running every test Spring starts a new transaction. The whole test (including its setup and tear down) runs within the same transaction which is... rolled back at the end. This means all the changes made during the test are visible in the database just like if they were persisted. However rollback after every test wipes out all the changes and the next test works on a clean and fresh database. Brilliant!

Unfortunately this is not yet another article about Spring integration testing advantages. I think I have written hundreds if not thousands of such tests and I truly appreciate the transparent support Spring framework gives. But I also came across numerous quirks and inconsistencies introduces by this comfortable feature. To make matters worse, very often so-called transactional tests are actually hiding errors convincing the developer that the software works, while it fails after deployment! Here is a non-exhaustive but eye-opening collection of issues:

@Test
public void shouldThrowLazyInitializationExceptionWhenFetchingLazyOneToManyRelationship() throws Exception {
  //given
  final Book someBook = findAnyExistingBook();

  //when
  try {
    someBook.reviews().size();
    fail();
  } catch(LazyInitializationException e) {
    //then
  }
}

Spring pitfalls: proxying

Being a Spring framework user and enthusiast for many years I came across several misunderstandings and problems with this stack. Also there are places where abstractions leak terribly and to effectively and safely take advantage of all the features developers need to be aware of them. That is why I am starting a Spring pitfalls series. In the first part we will take a closer look at how proxying works.

Bean proxying is an essential and one of the most important infrastructure features provided by Spring. It is so important and low-level that for most of the time we don't even realize that it exists. However transactions, aspect-oriented programming, advanced scoping, @Async support and various other domestic use-cases wouldn't be possible without it. So what is proxying?

Here is an example: when you inject DAO into service, Spring takes DAO instances and injects it directly. That's it. However sometimes Spring needs to be aware of each and every call made by service (and any other bean) to DAO. For instance if DAO is marked transactional it needs to start a transaction before call and commit or rolls back afterwards. Of course you can do this manually, but this is tedious, error-prone and mixes concerns. That's why we use declarative transactions on the first place.

So how does Spring implement this interception mechanism? There are three methods from simplest to most advanced ones. I won't discuss their advantages and disadvantages yet, we will see them soon on a concrete examples.

Java dynamic proxies

Simplest solution. If DAO implements any interface, Spring will create a Java dynamic proxy implementing that interface(s) and inject it instead of the real class. The real one still exists and the proxy has reference to it, but to the outside world – the proxy is the bean. Now every time you call methods on your DAO, Spring can intercept them, add some AOP magic and call the original method.

CGLIB generated classes

The downside of Java dynamic proxies is a requirement on the bean to implement at least one interface. CGLIB works around this limitation by dynamically subclassing the original bean and adding interception logic directly by overriding every possible method. Think of it as subclassing the original class and calling super version amongst other things:

Logging exceptions root cause first

The 0.9.30 release of Logback logging library brings new awesome feature: logging stack traces starting from root (innermost) exception rather than from the outermost one. Of course my excitement has nothing to do with the fact that I contributed this feature...



To paraphrase Cecil B. de Mille: “The way to make a blog post is to begin with a stack trace and work up to a climax” - here it goes:

The details aren't important yet, but from 100ft view you can see long stack trace with several exceptions wrapping each other (causing). We'll go back to this stack trace, but first some basics. If you throw an exception it will be logged in a way showing how the stack was looking in the moment when the exception was from. At the very bottom you will either see static main() or Thread.run() proceeded by methods invoked up to the first stack trace line indicating the place where the actual exception was thrown. This is very convenient since you can see the whole control flow that resulted in the exception:

The evolution of Spring dependency injection techniques

Looking back at the history of Spring framework you will find out that the number of ways you can implement dependency injection is growing in every release. If you've been working with this framework for more than a month you'll probably find nothing interesting in this retrospective article. Nothing hopefully except the last example in Scala, language that accidentally works great with Spring.

First there was XML [full source]:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd ">

    <bean id="foo" class="com.blogspot.nurkiewicz.Foo">
        <property name="bar" ref="bar"/>
        <property name="jdbcOperations" ref="jdbcTemplate"/>
    </bean>

    <bean id="bar" class="com.blogspot.nurkiewicz.Bar" init-method="init"/>

    <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
        <property name="driverClassName" value="org.h2.Driver"/>
        <property name="url" value="jdbc:h2:mem:"/>
        <property name="username" value="sa"/>
    </bean>

    <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
        <constructor-arg ref="dataSource"/>
    </bean>
</beans>

This simple application only fetches H2 database server time and prints it with full formatting:

What features of Java have been dropped in Scala?

Despite more complex and less intuitive syntax compared to Java, Scala actually drops several features of Java, sometimes for good, other times providing replacements on the standard library level. As you will see soon, Scala isn't a superset of Java (like Groovy) and actually removes a lot of noise. Below is a catalogue of the missing features.

break and continue in loops

Every time I see code like this:

while(cond1) {
    work();
    if(cond2)
        continue;
    rest();
}

I feel as if it has been written by a guy who truly misses the times when goto wasn't yet considered harmful. Hands up who finds this version more readable:

EJB 3.1 Cookbook by Richard Reese review

Recently I received a copy of EJB 3.1 Cookbook (courtesy to Packt Publishing) with a kind request to prepare a review. And since I have just got my new Kindle reader, this was a great opportunity to test them together.

EJB 3.1 Cookbook by Richard Reese aims to provide “a collection of simple but incredibly effective recipes” – quoting the subtitle. The book covers expected part of the EJB stack: session and message-driven beans, JPA, security, interceptors, timer service and web services. Contents are

organized around so called recipes – short (no more than three pages long) micro-chapters focused on one specific issue. This is the most advantageous feature of this publication: all recipes all self-contained most of the time, so one can jump between them and apply the most suitable in given scenario. This makes the book suitable for both the beginners, which should read it from cover to cover and slightly more experienced developers. However the latter ones will probably prefer more comprehensive sources and skip significant parts of the book.

Although I found the book structure very convenient, contents (both granularity and volume) are highly subjective and controversial. On the one hand the author devotes five recipes to describe separately each JMS message type (string, byte, stream, map and object) – each one is almost identical. Whilst he could only list different types, he fills half of the JMS chapter this way (sic!) On the other hand, there is only one recipe explaining new JPA 2.0 Criteria API – to make matters worse, only using weakly typed queries. Probably one of the most important new features in EJB 3.1 stack has been covered on two pages. And this API is not particularly easy to grasp. To depict you the scale – the art of adding Bean Validation annotations (@NotNull etc.) on fields required ten pages and eight recipes... Where a half-page bullet-list would suffice.

The last chapter – EJB Techniques – is very intriguing. Lots of valuable and accurate tips have been given, like the difference between Date and Calendar, logging and dealing with exceptions, effective String manipulations. Despite appearances, this is not the basic Java knowledge and many experienced programmers still don't know that SimpleDateFormat is not thread-safe and that Date class does not store time zone. I am really happy that this kind of knowledge found its way in the book. Unfortunately – in the wrong one. That being said, there are more appropriate books, not focused on EJB or even Java EE. Here it just looks like putting anything useful to reach dozen chapters in total.

Poor man's CRUD: jqGrid, REST, AJAX, and Spring MVC in one house

More than two years back I wrote an article on how two implement elegant CRUD in Struts2. Actually I had to devote two articles on that subject because the topic was so broad. Today I have taken much more lightweight and modern approach with a set of popular and well established frameworks and libraries. Namely, we will use Spring MVC on the back-end to provide REST interface to our resources, fabulous jqGrid plugin for jQuery to render tabular grids (and much more!) and we will wire up everything with a pinch of JavaScript and AJAX.

Back-end is actually the least interesting part of this showcase, it could have been implemented using any server-side technology capable of handling RESTful requests, probably JAX-RS should now be considered standard in this field. I have chosen Spring MVC without any good reason, but it's also not a bad choice for this task. We will expose CRUD operations over REST interface; the list of best selling books in history will be our domain model (can you guess who is on the podium?)

@Controller
@RequestMapping(value = "/book")
public class BookController {

  private final Map<Integer, Book> books = new ConcurrentSkipListMap<Integer, Book>();

  @RequestMapping(value = "/{id}", method = GET)
  public @ResponseBody Book read(@PathVariable("id") int id) {
    return books.get(id);
  }

  @RequestMapping(method = GET)
  public @ResponseBody Page<Book> listBooks(
      @RequestParam(value = "page", required = false, defaultValue = "1") int page,
      @RequestParam(value = "max", required = false, defaultValue = "20") int max) {
    final ArrayList<Book> booksList = new ArrayList<Book>(books.values());
    final int startIdx = (page - 1) * max;
    final int endIdx = Math.min(startIdx + max, books.size());
    return new Page<Book>(booksList.subList(startIdx, endIdx), page, max, books.size());
  }
}

Few things need explanation. First of all for the purposes of this simple showcase I haven't used any database, all the books are stored in an in-memory map inside a controller. Forgive me. Second issue is more subtle. Since there seems to be no agreement on how to handle paging with RESTful web services, I used simple query parameters. You may find it ugly, but I find abusing Accept-Ranges and Range headers together with 206 HTTP response code even uglier.

Last notable detail is the Page<Book> wrapper class:

Enabling load balancing and failover in Apache CXF

A while ago we've faced the requirement of load-balancing web services clients based on Apache CXF. Also the clients should automatically fail-over when some of the servers are down. To make it even worse, the list of servers target addresses was to be obtained from external service and updated at runtime. Eventually we ended up with home-grown load-balancing micro-library (ESB/UDDI/WS-Addressing seemed like an interesting alternatives, but they were an overkill in our situation). If we only knew Apache CXF already supports all these features (almost) out of the box?

Don't blame us though, only reference to this feature points to a very poor documentation page (if you call 404 “poor”). If it's not in official documentation, I would expect to find it in Apache CXF Web Service Development book – unfortunately, bad luck there as well. But hey, isn't exploring such features yourself even greater fun? This is the client configuration we are starting with:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:jaxws="http://cxf.apache.org/jaxws"
       xmlns:clustering="http://cxf.apache.org/clustering"
       xmlns:util="http://www.springframework.org/schema/util">

    <jaxws:client id="testServiceClient"
                  serviceClass="com.blogspot.nurkiewicz.cxfcluster.SimpleService"
                  address="http://serverA/simple">
    </jaxws:client>

</beans>

MongoDB and recording appenders for Logback

Today I am giving you two new appenders for Logback: one for MongoDB and one which I called recording appender. Just as a reminder, appenders (both in Log4J and Logback) are an abstraction of your application logs destination. The most common are file and console appenders, followed by several others built-in. MongoDB appender is pretty straightforward, so I will start by describing the recording appender.

Recording appender

As you already know, one of the biggest benefits of using logging frameworks are logging levels. By carefully choosing levels for each logging statement we can easily filter which logs should be present in our log files and which shouldn't. Also we can apply different logging strategies for different environments. This is in theory. In practice we often face the choice between: log everything just in case and handle gigabytes of meaningless log files or log only warnings and errors but when they actually occur, they are meaningless as well, lacking important debugging context. The idea isn't new (see [1], [2] and [3] for example), but somehow decent implementation is missing in both Log4J and Logback. And the idea is simple – as long as there is nothing wrong happening with the system: do not log anything or log very little – but silently memorize all debug logs in some cyclic buffer. And whenever disaster occurs (any log with ERROR level, probably an exception), dump the buffer first to provide meaningful context.

Writing custom logging appenders is pretty straightforward. Following is the essence of my recording appender:

Jolokia + Highcharts = JMX for human beings

Java Management Extensions (JMX) is a well established, but not widespread technology allowing to monitor and manage every JVM. It provides tons of useful information, like CPU, thread and memory monitoring. Also every application can register its own metrics and operations in so called MBeanServer. Several libraries take advantage of JMX: Hibernate, EhCache and Logback and servers like Tomcat or Mule ESB, to name a few. This way one can monitor ORM performance, HTTP worker threads utilization, but also change logging levels, flush caches, etc. If you are creating your own library or container, JMX is a standard for monitoring, so please don't reinvent a wheel. Also Spring has a wonderful support for JMX.

If this standard is so wonderful, why aren't we using it all day long? Well, history of JMX reaches the dark ages of J2EE. Although the specification isn't that complicated, there are at least two disadvantages of JMX effectively discouraging people from using it. First one lies on the server side. At the foundation of Java Management Extensions is an MBeanServer where you register MBeans. Each MBean exposes its properties (attributes) and operations for external access. This is fine (especially when Spring is used: 1 line of XML + two annotations), but there's a catch. By default the MBeanServer exposes itself via RMI, which is certainly not the top XXI century protocol...

The second drawback of JMX lies on the client side. JConsole, although not terrible, has very limited functionality. If we want to present our JMX-enabled application to the customer, showing JConsole as a client is a bit embarrassing. It is capable of showing graphs, but you cannot display more than one attribute at the same composite graph and you also can't observe attributes from different MBeans at the same time. Last but not least, again, we're living in the XXI century, Swing client? Weird RMI port? What about Web 2.0 rave? Knowing how much I love charts (and how data visualization is important for diagnosing and correlating facts) I felt really disappointed by JConsole capabilites. And the only rival of JConsole seems dead.

Jolokia – bridge JMX over HTTP

I knew exactly what I wanted: HTTP transport for JMX server, so that I can easily access MBeanServer from outside without the RMI mess. Jolokia meets my expectations perfectly. This small library (about 170 kiB) connects to a given MBeanServer and exposes it via REST-like interface. Just deploy the jolokia.war file on your servlet container and use whatever HTTP client you want to monitor your JVM!

$ curl localhost:8080/jolokia
{
  "request" : { "type" : "version" },
  "status" : 200,
  "timestamp" : 1300561261,
  "value" : {
      "agent" : "0.83",
      "info" : {
          "product" : "tomcat",
          "vendor" : "Apache",
          "version" : "7.0.10"
        },
      "protocol" : "4.1"
    }
}


$ curl localhost:8080/jolokia/read/java.lang:type=Memory/HeapMemoryUsage
{
  "request" : {
      "attribute" : "HeapMemoryUsage",
      "mbean" : "java.lang:type=Memory",
      "type" : "read"
    },
  "status" : 200,
  "timestamp" : 1300561367,
  "value" : {
      "committed" : "169607168",
      "init" : "49404160",
      "max" : "702808064",
      "used" : "16635472"
    }
}

Tenfold increase in server throughput with Servlet 3.0 asynchronous processing

It is not a secret that Java servlet containers aren't particularly suited for handling large amount of concurrent users. Commonly established thread-per-request model effectively limits the number of concurrent connections to the number of concurrently running threads JVM can handle. And because every new thread introduces significant increase of memory footprint and CPU utilization (context switches), handling more than 100-200 concurrent connections seems like a ridiculous idea in Java. At least it was in pre-Servlet 3.0 era.

In this article we will write scalable and robust file download server with throttled speed limit. Second version, leveraging Servlet 3.0 asynchronous processing feature, will be able to handle ten times bigger load using even less threads. No additional hardware required, just few wise design decisions.

Token bucket algorithm

Building a file download servers we have to consciously manage are our resources, especially network bandwidth. We don't want a single client to consume the whole traffic, we might even want to throttle the download limit dynamically at runtime, based on user, time of the day, etc. - and of course everything happens during heavy load. Developers love reinventing the wheel, unfortunately all our requirements are already addressed by absurdly simple token bucket algorithm.

The explanation in Wikipedia is pretty good, but since we'll adjust the algorithm a bit for our needs, here's even simpler description. First there was a bucket. In this bucket there were uniform tokens. Each token is worth 20 kiB (I will be using real values from our application) of raw data. Every time a client ask for a file, the server tries to take one token from the bucket. If it succeeds, he sends 20 kiB to the client. Repeat last two sentences. What if the server fails to obtain the token because the bucket is already empty? He waits.

So where are the tokens coming from? Background process fill the bucket from time to time. Now it becomes clear. If this background process adds 100 new tokens every 100 ms (10 times per second), each worth 20 kiB, the server is capable of sending 20 MiB/s (100 times 20 kiB times 10) max, shared amongst all the clients. Of course if the bucket is full (1000 tokens), new tokens are ignored. This works amazingly well – if bucket is empty, clients are waiting for next bucket filling cycle; and by controlling the bucket capacity we can limit total bandwidth.

Enough of talking, our simplistic implementation of token bucket starts with an interface (whole source code is available on GitHub in global-bucket branch):

Spring framework without XML... At all!

First, there was EJB 2.1 with countless XML files all over. It won't be such a big exaggeration to say that for every line of business code you had to create at least 10 lines of framework code and two pages of XML. Local and remote interfaces, manual JNDI lookup, triply nested try-catches, checked RemoteExceptions everywhere... this was enterprise. There were even tools to generate some of this boilerplate automatically.

Then couple of guys came and created Spring framework. After being forced to cast by obscure PortableRemoteObject.narrow() it was like taking a deep breath of fresh air, like writing poetry after working in coal mine. Time went by (BTW do you remember how many years ago was the last major JRE release?) and Sun learnt their lesson. EJB 3.0 was even simpler compared to Spring, XML-free, annotations, dependency injection. 3.1 was another great step toward simplicity, being more and more often compared to Spring. Logically current state of the art EJB specification might be considered as a subset of what Spring offers, I am actually surprised why there is no EJB spec. implementation in plain Spring (oh, wait...), considering its out-of-the-box support for JPA 1.0/2.0, JSR-250, JSR-330, JAX-WS/RS compatible solutions and others. But even though, Spring framework is nowadays perceived as a slow, heavyweight and hard to maintain, mainly due to reliance on XML descriptors. Once simple, now Spring is a whipping boy in the JEE framework battle.

I don't like politics, I won't defend my beloved framework writing lengthy essays. Instead I will take simple, but not trivial Spring application and quickly rewrite it so that it won't use XML. Not reduce the amount of XML, not leave only few untouchable lines. No XML - at all.

For the purposes of this article I created very simple Spring web application (base version under xml branch, final on master on my GitHub account) using JDBC, JMS and JMX, just not to make things trivial. Every change I made to the source code will be reflected in a separate commit to this repository. Step by step I will be removing XML configuration until there will be no Spring XML left. This is where we start:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:amq="http://activemq.apache.org/schema/core"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
            http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
             http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core-5.4.2.xsd
             http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

    <context:mbean-export />

    <bean id="fooService" class="com.blogspot.nurkiewicz.FooService">
        <property name="jmsOperations" ref="jmsTemplate" />
    </bean>

    <bean id="fooRequestProcessor" class="com.blogspot.nurkiewicz.FooRequestProcessor">
        <property name="fooRepository" ref="fooRepository" />
    </bean>

    <bean id="fooRepository" class="com.blogspot.nurkiewicz.FooRepository" init-method="init">
        <property name="jdbcOperations" ref="jdbcTemplate" />
    </bean>


    <!-- JDBC -->
    <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="org.h2.Driver" />
        <property name="url" value="jdbc:h2:~/workspace/h2/spring-noxmal;DB_CLOSE_ON_EXIT=FALSE;TRACE_LEVEL_FILE=4;AUTO_SERVER=TRUE" />
        <property name="username" value="sa" />
        <property name="password" value="" />
    </bean>

    <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
        <constructor-arg ref="dataSource" />
    </bean>

    <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <constructor-arg ref="dataSource" />
    </bean>

    <tx:annotation-driven />


    <!-- JMS -->
    <bean id="jmsConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
        <constructor-arg>
            <bean class="org.apache.activemq.ActiveMQConnectionFactory">
                <property name="brokerURL" value="tcp://localhost:61616" />
            </bean>
        </constructor-arg>
    </bean>

    <amq:queue id="requestsQueue" physicalName="requests" />

    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <constructor-arg ref="jmsConnectionFactory" />
        <property name="defaultDestination" ref="requestsQueue" />
    </bean>

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="jmsConnectionFactory" />
        <property name="destination" ref="requestsQueue" />
        <property name="sessionTransacted" value="true"/>
        <property name="concurrentConsumers" value="5"/>
        <property name="messageListener">
            <bean class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
                <constructor-arg ref="fooRequestProcessor" />
                <property name="defaultListenerMethod" value="process"/>
            </bean>
        </property>
    </bean>

</beans>

Activiti processes and executions explained

I was interested in Activiti process engine long before it reached its first stable 5.0 version. Now, when 5.1 was released, I decided to play a bit with this framework, especially paying attention to Spring and JUnit support. But one of the first impediments encountered was the difference between process instance and execution, as well as between sub process and call activity. I am hoping after reading this article you won't encounter the same problems when starting with Activiti.

As you know, even a monkey can learn a new Java framework after reading documentation, but the real fun comes when you meet the technology by studying its source code (often tracking bugs and looking for solutions). And I must admit that Activiti code-base was a pleasure to read, nicely structured and designed. For some reason it is not deployed to Alfresco's repository, so to take full advantage of your BPMN journey, start by:

svn co http://svn.codehaus.org/activiti/activiti/tags/activiti-5.1
cd activiti-5.1
mvn install source:jar -DskipTests -Pdistro

OK, for starters take a look at this process:

Speeding up Spring integration tests

The biggest problem with unit testing using Spring testing support* is the time it takes to initialize the Spring framework context. Every new test case adds precious seconds to overall build time. After a while it will take minutes or even hours to fully build the application, while most of this time is consumed by Spring itself. But we'll start from the basics.

In order to make JUnit aware of Spring framework test support, simply add these annotations on test case class:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
@Transactional
public class MainControllerTest {
    //...
}

While @Transactional is not necessary, it will greatly simplify testing when database is involved (details here). In IntelliJ IDEA 10 (I just took this brand new version for a test drive) these annotations will raise the following error to occur:

And suggested solution:

Functional Java Developers’ Day 2010

Ted Newards’ talk about functional programming in Java and his workshop on Scala (hence the article title) were the most memorable events during the third JDD conference that I attended last week. Sadly most memorable, and almost the only ones. But first things first.

After spending endless hours with great Enterprise JavaBeans 3.0 book by Bill Burke I’ve expected something fabulous, but Bills’ lecture about JAX-RS was just average, with no coding, only plain API introduction. Scott Davis' presenting REST and ROA last year was way much better. Thankfully Angelika Langer talk about Java concurrency pitfalls was much more interesting, although one might argue that she just gave a summary of marvelous Java Concurrency in Practice book, that I have recommended long time ago. One new thing I’ve learnt is that updating volatile variable guarantees all other non-volatile variable updated earlier by the same thread to be visible by other threads, as if they were volatile as well (are you following?) After Angelika there was a talk about Java performance testing, but I had a copy of The Art of Application Performance Testing book in my knapsack, so the presentation didn’t caught a lot of my attention.

After delicious lunch Ted Neward gave a talk on functional programming concepts in Java. Charismatic, entertaining, surprising – I knew he will make a great show. In the middle of third or fourth slide he asked innocently who prefers slides to live coding and after finding no such person on the audience, he instantly closed the presentation and opened notepad, writing some Java code from scratch. It looked so natural that I almost believed that he actually had any slides further – but of course this was all set up. Ted mentioned Functional Java library as a way to enable functional style of programming in plain old Java. Another libraries I can point out if you have to stick with this language are: Fun4J, Google's Guava and LambdaJ. Also take a look at Lombok to write more concise POJOs.

I’ve seen Piotr Walczyszyn several times evangelizing Flex and AIR so it came a bit of a surprise that I really enjoyed his talk, even though I don’t like front-end programming. But a real surprise was the Linda Rising presentation dealing with the problem of introducing change and convincing our coworkers to it. From time to time we need to look at our job from 10,000 ft perspective, see how ridiculously we sometimes behave and how irrational our choices are. See Pragmatic Thinking and Learning... for a more in-depth analysis on this. Also I recently read marvelous Dreaming in Code by Scott Rosenberg. Linda’s talk complemented these books excellently.

On the next day I sacrificed three presentations to participate in Ted Neward’s workshop on Scala – continuing the subject of functional programming. He introduced Scala language step by step, emphasizing the differences and commons misconceptions. I will repeat after him: Scala is compiled, statically typed, consistent language, don’t compare it with Groovy, these are completely different tools. I was doing my best to follow Ted’s examples on my computer and finally I wrote few simple (compiling!) lines of Scala code. Too bad my first presentation on Scala (during GeeCON 2009) by Luc Duponcheel wasn’t that good and simply beyond my comprehension. Here are few examples of Scala concepts that significantly caught my attention. POJO in Scala:

import org.apache.commons.lang.builder._

class Book(author: String, title: String, year: Int) {

  override def toString =
    new ToStringBuilder()
      .append("author", author)
      .append("title", title)
      .append("year", year)
      .toString()
}

Testing for exceptions in JUnit revised

In his recent post the author of fantastic mocking framework Mockito collected few rules about testing exceptions. What caught my attention is the advice to use JUnit rules (nomen est omen!) for testing exceptions. ExpectedException rule gathers advantages of both expected @Test attribute clarity and try-catch strictness. Here is the example:

public class DefaultFooServiceTest {

 private FooService fooService = new DefaultFooService();

 @Rule
 public ExpectedException exception = new ExpectedException();

 @Test
 public void shouldThrowNpeWhenNullName() throws Exception {
  //given
  String name = null;

  //when
  exception.expect(NullPointerException.class);
  fooService.echo(name);

  //then
 }

}

Szczepan claims that ExpectedException fits into given/when/then test template nicely. I disagree! Look at the code snippet above – what is the most natural place to put assertions on exception being thrown? From the obvious reasons it must be the last line before the line that actually throws the exceptions. So you have a choice to put assertion as the last statement in given block or as first in when block. You are right, this is how this test should look like in an ideal world:

@Test
public void shouldThrowNpeWhenNullName() throws Exception {
 //given
 String name = null;

 //when
 fooService.echo(name);

 //then
 exception.expect(NullPointerException.class);
}