JPA Enhancement done right

JPA Enhancement done right

When it comes to handling databases in a highlevel language there are basically 2 different approaches:

a.) Systems which give you access to all the gory details and therefor have a pretty high entry barrier.
b.) Systems which are perfectly easy to use for samples and simple apps – but you need all the (well hidden) gory details if you use that stuff in real world projects.

Without a doubt JPA is more the category b.) type of project.

Using JPA

If you look at a classical JPA entity, you will find something similar to our following example:

import javax.persistence.*;

@Entity
public class Customer {
  @Id
  @GeneratedValue
  private Long id;

  @Column(length = 50)
  private String firstName;

  public Long getId() {
    return id;
  }

  public String getFirstName() {
    return firstName;
  }

  public void setFirstName(String firstName) {
    this.firstName = firstName; 
  }
}

Usually the entities get created with the ‘new’ keyword and then stored by calling EntityManager#persist()

Customer c = new Customer();
c.setFirstName("John");
entityManager.persist(c);

Looks easy, isn’t?

Enhancement

A lot of the ‘easyness’ of JPA actually comes by the feature of ‘enhancing’ classes which are annotated with @Entity. And this enhancement step is usually well hidden deep inside the code.

So let’s first look what this ‘enhancement’ does for you!

Whenever you touch a JPA entity, it will track the following information for each and every non-transient attribute of the entity:

* _dirty: if you change firstName from ‘John’ to ‘Barry’, this field becomes ‘dirty’. JPA uses this information to decide which fields/entities needs to get written to the database.

* _loaded: JPA entities might contain fields which have FetchType.LAZY. They will only get loaded from the database once they are accessed the first time. This is pretty neat if you have a fat @ElementCollection which takes some time to load. Some older JPA impls did rely on if(field == null) but what about c.setFirstName(null); to clear out this info in the database? How to distinguish between a field which didn’t yet get laoded (and is null) from a field which got loaded and afterwards set to null?
_loaded will be used to track this info.

But where is this _dirty and _loaded info in my entities? Well, this is exactly what the ‘enhancement’ step is doing. It adds this information to the entity class file for you!

Basically the setFirstName in the ‘enhanced’ class will finally look somehow like the following (completely transparent for the user!):

public void setFirstName(String firstName) {
  if (safeEquals(this.firstName, firstName) {
    return;
  }

  //EntityManager must not get accessed in different threads
  assertEntityManagerLocking();

  setDirty(FIRSTNAME_FIELD);
  this.firstName = firstName;
}

Technically there are 3 different ways to achieve this

1.) Using Proxies/Subclassing at runtime

A Proxy is basically a dynamically created Subclass. This is the default mode for Hibernate and has quite a few drawbacks.

1.a.) you can only proxy methods. It’s not possible to intercept the field access inside the class level. This makes it much harder to correctly write EntityListeners and other methods which juggle around with fields.

1.b.) Only property access for classes loaded via a query or EntityManager#find() can get tracked. It’s not possible to do the same for Entities just created via ‘new’ because there is no proxy in this case.

1.c.) Lazy loading is not possible in this case. All fields (including expensive @ElementCollection reads) will be performed eagerly.

1.d.) Hibernate seems to have a mode where it dynamically replaces Lists stored in an entity with their own ProxyLists for managed entities. That way they can at least do some LazyLoading. But doing the dirty-check requires to compare each entry in those Lists with the database – that’s damn slow!

2.) Using Compilation Enhancement

You can use either ant or maven to ‘enhance’ the class files of your entities to contain all the additional code which is needed to work properly. This is the prefered way as it provides the full set of features and also the best performance at runtime. Actually this option comes in 2 different flavours:

2.a.) Build Time Enhancement: The class file get’s modified even before it will get packaged into the JAR. I will come back to this again a bit later and explain it in detail.

2.b.) Deploy Time Enhancement: The class file get’s enhanced by the ‘deployer’ in a JavaEE container. If you upload your WAR/EAR to your server, it will get unpackaged, all classes with an @Entity will automatically get enhanced immediately afterwards by the container.

3.) Runtime Enhancement (via ClassTransformer)

At the first glance the JavaAgent [1] looked like a smart way to perform the enhancement at runtime, but it turned out that there are fundamental problems with this approach. Again there are 2 different flavours for this way of enhancement:
3.a.) using the Java5 -javaagent option.
3.b.) using the Java6 Instrumentation#retransformClasses. This allows to drop out loaded classes and replace them with their ‘transformed’ version.

The major problem with both of the aforementioned mechanism is that they use the SystemClassLoader to register the ClassTransformer and subsequently also to load the classes needed to perform the transformation!

This has the bad side effect that the SystemClassloader gets polluted with some impl classes of your persistence provider. But not all, and if the transformation is finished, the jar containing the Transformer will get detached from the JVM again. This will leave you with 30% of Hibernate/OpenJPA/EclipseLink lying around in your SystemClassPath and the rest is not available.

Not that worse, one might say, because all those classes are still available in your web application. Too bad that a nice little security mechanism kicks in and destroys all our dreams. The Servlet specification defines that any container must prevent redefining ‘system classes’ – that is all classes which get loaded via the SystemClassLoader. Thus in any spec conform servlet container (e.g. tomcat [2]) you are now bound to the few classes of your jpa provider which are available in your SystemClassLoader. This will result in tons of NoClassDefFound errors and many tears…

The Winner: Build Time Enhancement

Technically only Built Time Enhancement and Deploy Time Enhancement do work sufficiently at all. The reason I don’t use Deploy Time Enhancement is that you are not able to test and debug locally what you have running in your server. All the unit tests are not worth much if you don’t run the code which gets executed in production. Thus Build Time Enhancement is the clear winner to me.

The openjpa-maven-plugin

Quite some time ago we wrote the openjpa-maven-plugin [3] which allows to do all the enhancement steps while building your project with maven. This plugin got moved over to the OpenJPA project itself and will first be available with the upcoming OpenJPA-2.2.0 version (expect this to be out this month). A similar tool is available as ant-task. There is also a maven plugin for doing the same with hibernate [4].

IDE pitfalls

Most IDEs nowadays compile the classes on the fly. If you don’t have a special IDE plugin installed which ‘fixes’ the entity classes afterwards, you might get errors reporting that you ‘try to run unenhanced classes’. Usually this happens only the first time or if you change an Entity class.

In this case simply enhance them again on the commandline.
$> mvn clean process-classes

Afterwards you can work in your IDE without any further problems.

Glossar:

[1] javaagent: http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/package-summary.html

[2] tomcat WebAppClassLoader: http://svn.apache.org/repos/asf/tomcat/trunk/java/org/apache/catalina/loader/WebappClassLoader.java
see public synchronized Class loadClass(String name, boolean resolve)

// (0.2) Try loading the class with the system class loader, to prevent
// the webapp from overriding J2SE classes

[3] http://mojo.codehaus.org/openjpa-maven-plugin/usage.html

[4] http://mojo.codehaus.org/maven-hibernate3/hibernate3-maven-plugin/

OpenJPA: http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_pc_enhance

Hibernate: http://docs.jboss.org/hibernate/core/3.3/reference/en/htm/performance.html#performance-fetching-lazyproperties