Hibernate optimizations beyond JPA

Optimizations on the persistence layer not supported by JPA

With the JPA 2 standard a great support came for storing and referencing data in collections. 

Entities can now reference other objects that are of the type basic, used for primitive type.

The annotation for supporting this is @ElementCollection.

With this a separate table is created used to store the primitive types. 

It can be specified further with @CollectionTable and @Column.

For small amount of types you would define the class of the entity containing the primitive type as @Embeddable. 

If the size of the primitive types is big/very big, you would leave the default behavior and let the persistence provider to lazy load the primitives types as they are needed.

More thoughts have to put into a real relationship between entities.

If you don't pay attention you could very easily run into the famous N+1-SELECT-problem.

If an entity has a one-to-many relationship to another entity and no specific loading strategy is defined. The standard lazy-loading will lead to the fact, that on every access to an attribute of the other entity, this entity will be loaded separately. 

Common use case:

1) N Entities are loaded from a persistence service

2) Looping over this result(N) set and applying certain business logic on it

3) With this the entity graph is used and the code navigates to the one-to-many relationship 

4) With the lazy-loading configuration of this relationship for every item in the loop another fetch of the one-to-many relationship is done. The persistence provider will generate an additional SELECT.

What to do in order to optimize this situation?

• usage of @BatchSize  (non-JPA compliant)
• usage of SUBSELECT-FETCH  (non-JPA compliant)
• own criteria/JPQL for joining both entities
• ...

But with the first 2 points we are already outside of the JPA standard.

@BatchSize could ease the problem that the fetching is reduced from n+1 to a n+batchSize+1 problem.

@Entity 

public class A

@ManyToOne

@BatchSize(size=5)

public B getBs() {

  ..

}

The second point leads to completely load the collection after entity A is loaded.

@Entity 

public class A

@ManyToOne

@Fetch(FetchMode.SUBSELECT)

public B getBs() {

  ..

}

But this is again a Hibernate dependency and Hibernate supports this on collections, and all XToMany-relationships. 

Leaving it to 3. point and the danger of retrieving too much data.

Optimistic / pessimistic locking LockModeType

What is the difference between optimistic locking and pessimistic locking?
First of all locking is intended for managing transactions.
If transactions can run serial, that means one after another,
nothing can go wrong:
The transaction fulfills the ACID properties:
- atomic
- consistent
- isolated
- durable

From database view a transaction should always have these properties:
It is executed as a whole or not - atomic.
It drives the state of the database from one consistent state to another state - consistent.
Transactions do not influence each other - isolated.
Transactions and there changes are saved durably in the database  - durable.
From JPA or Hibernate view the easiest way is using the highest isolation level:
Serializable.
Serial transactions in high frequent systems have a bad performance and scalability.
Therefore the level of serialization of an application / system is reduced to increase performance and throughput.
You pay this by problems which occur on account of certain isolation levels:

• Dirty Reads: Changes are read by other transactions before they are committed. If a rollback takes place you have read something wrong.
• Non repeatable Reads: Rows are read, another transaction makes changes, rows are read again, other data exists as to begin of the transaction
• Phantom Reads: a query delivers different results during a transaction

Normally you have some places in your application or system which have to handle transaction in a highly secure way. The business logic needs a high transaction security and this is payed by less performance.

How is transaction management done in JPA?
In JPA you normally have 2 scenarios:

1. JEE - environment and existence of a JTA manager
2. no JEE- environment and management of transactions by the application itself

Assuming point 2) the entity manager is used for the management of transactions:
the EntityTransaction- Interface, which is delivered by EntityManager.getTransaction().
Here you have the usually methods for transaction management defined:

• begin()
• commit()
• rollback()

In the Spring-environment the transactions are managed by the annotation declared on the method declaration:
@Transactional(isolation=Isolation.READ_COMMITTED, propagation=Propagation.REQUIRED)

If the method is stated with @Transactional you have the behaviour like above:
the isolation level is set by the database default(usually READ_COMMITTED) and a transaction is needed.

Locking -strategies
JPA behaves like you know it from Hibernate already:
If an entity is annotated with @Version optimistic locking takes place for this entity.
Optimistic locking is the mechanism where objects are not explicitly locked at the beginning of a transaction.
It assumes that optimistically no conflict will take place. Therefore at the moment of persisting/writing on commit a version check takes place:
Every update on an entity increments the version of an entity.
The version - property of an entity can only be written by the JPA-provider.
On commit the entity is checked if it has a different version value.
If so, an OptimisticLockException is thrown.
If not the entity with the new version number is persisted in the database.
The OptimisticLockException should be handled by the application.
This behaviour is delivered by using @Version.
With Hibernate as the JPA-provider and setting the isolation level of the transaction on Repeatable Read or Serializable, the version checking is done explicitly with a select for the Entity for retrieving the actual version used in the database. This is the Hibernate specific LockMode.READ.
If the cache is used for version checking this corresponds to LockMode.NONE.
LockMode.UPGRADE corresponds to the JPA mode LockModeType.READ.

A more restrictive OptimisticLocking - mechanism can be configured by selecting the isolation level.
For that the objects which should be locked are locked by:
EntityManager.lock(object, LockModeType t);
If LockModeType.READ is set, normally during commit the corresponding object is locked by select .... from table for update:
on row level an exclusive lock is set, which is set for a very short time span.
There is LockModeType.WRITE which increments the version - also if nothing has changed on the entity.

Pessimistic Locking, that means locking of objects on transaction begin and keeping the lock during transaction is done by these 2 PessimisticLockModes:
- LockModeType.PESSIMISTIC_READ -->
entity can be read by other transactions but no changes can be made
- LockModeType.PESSIMISTIC_WRITE -->
entity can not be read or written by other transactions

Hibernate 4.1 new features loading per @NaturalId

What features come along with the new Hibernate 4 Release?
In Release 4, to be correct 4.1-Release, one can load entities with naturalIds.

What are naturalIds?
NaturalIds are keys with professional uniqueness, which can be used in certain queries or loading situations. It is recommendate that technical keys are used, but if those entities have attributes with professional uniqueness, those attributs can be assigned hibernate annotations
@NaturalId. Those properties must be unique, because
an unique constraint will be created on that column in the database.


Why should I bother about @NaturalId - annotations anyway?
Assumpted that there is an entity taxpayer. It certainly has a property named taxnumber.
The taxnumber should be unique.
So this property is a candidate for a definition of an naturalId.

public class Taxpayer {
  @Id
  @GeneratedValue
   private Long id;
   private String firstname;
   ....
   @NaturalId
   private String taxnumber;
   ...
}

Now you have an advantage, which could be used already in older hibernate versions:
You can make use of those naturalIds in queries:
Session s = ...
Criteria crit = s.createCritera(Taxpayer.class);
Taxpayer t = (Taxpayer) crit.add(Restrictions.naturalId("taxnumber").set("89089083AST8908").uniqueResult();

New 4.1 feature:
With 4.1 you can define @NaturalId in the entity and use it on the session object, that means you
can load an entity with help of byNaturalId()!

Session s = ...
TayPayer t = (TaxPayer) s.bySessionId(Taxpayer.class).using("taxnumber", "89089083AST8908");


Is there an disadvantage?
Unfortunately, the @NaturalId is a hibernate specific feature.
So you can't express / use @NaturalIds in JPA.
It seems, that @NaturalId will not be included in the upcoming JPA 2.1 standard.
See: Early Draft JPA 2.1

JPA2 Feature Saving the order of elements of an association

In JPA 2 there is a new feature for assuring that an association has always the same order of elements.
That means the order of the elements of an association, which is realized by a list, is stored in the database as well.
There is a 1:n - association.
The 1-side has the according collection of entities of the n-side.
This collection can be defined as list in the specific entity class.
If you want, that the list behaves deterministic, that the order must be stored:
therefore you can use the new annotation in JPA 2:

@OrderColumn

@Entity
public class A implements Serializable {
   ...
   @OneToMany
   @OrderColumn
   private List<B> bs = new ArrayList<B>();
   ...
}    

@Entity
public class B implements Serializable {
 ...
 @Id
 @Generated
  private Long id;
  private String orderNumber;
 ...
}

In this case in the entity B a new field with the name BS_ORDER is created, which can not be seen by the entity itself. It is used for saving the order of the associated entities of B to A.
So the order of elements of the collection is always the same, when the entities of A are load for e.g. with mode FetchType.EAGER.
When would you use it?
If the sort order @OrderBy has an bad impact on access execution or the order of subject-specific attributes / columns still produces an undefined order.


Saving of elements per key of an association, which is realized by Map, was possible in the previous JPA Version:
In the example above bs is not created as List, but as Map.
Then the annotation @MapKey can define the key with whom one can access the collection / association.
The Default @MapKey always defines the primary key of the entity for accessing the map.
If you want to use another column as the key, you can use the parameter name:

@MapKey(name="columnName").
The result of the above example is:
   @OneToMany
   @MapKey(name="orderNumber")
   private Map<String, B> bs = new HashMap<String,B>();

For what do we need an key based access of an association?
It makes sense, if you know, that the access of this annotation regarding the business logic
usually is done per using the orderNumber and the primary key is a more technical key, which is used in other subordinated use cases like archiving the data.
So one can express very clean:

String aOrderNumber;
....
B myBEntity = a.getBs().get(aOrderNumber);

Treatment of Value Types in JPA Hibernate

How is value types approached in JPA / Hibernate?

First of all the definition of value types.
Value types are no real entities. They have no life cycles, that means they will not be created, later on edited or deleted. They exist since beginning of system life.
True entities, which are marked with @Entity, do have a life cycle and they a corresponding representation in the database.
Now there a certain cases in which one wants to use value types:

• as constants, which are stored as attribute to a entity in the database
• values, which are stored next to a certain entity

Constants can be made easily in Java with enums:
public enum EnumXY{
   constantValue1, constantValue2, constantValue3;
}

They are used with every entity:
@Entity
public class MyEntity implements Serializable {
    ...
    @Enumerated
    private EnumXY flag;
    ...
}

In the table per default the position of the value(ordinal number) of the corresponding entity is stored as integer. If the MyEntity flag gets the constantValue3 assigned, than value 2 is stored in the table of MyEntity.
This can be configured with @Enumerated(EnumType.ORDINAL) (default) or @Enumerated(EnumType.STRING). The latter the name of the constant will be stored in the corresponding column of the database table. Here it is constantValue3.

What happens if someone wants to store multiple values of a value type to an entity?
This is also possible in JPA 2:
With the help of @ElementCollection you can express, that the collection is made of values of a value type and not a collection of normal entities.
 @ElementCollection
private List<String> basetypeValues = new ArrayList<String>();
First of all  CollectionsOfElements is the hibernate specific annotation.
In JPA you should use @ElementCollection.
Example:
@Entity
public class X implements Serializable {
  @Id
  @Generated
  private Long id;
  @ElementCollection
   private List<String> remarks = new ArrayList<String>();
    ...
}

@Test
public void testElementCollection() {
  ...
  X x = new X();
  private List<String> remarks = new ArrayList<String>();
  remarks.add("remark1");
  remarks.add("remark2");
  remarks.add("remark3");
  x.setRemarks(remarks);
  em.persist(x);
  ...
}

What happens in the database?
An "ElementCollection" will always be stored in a separate table.
This can be defined more precisely with @CollectionTable. JoinColum(s) is used for the definition of the foreignKey(s):
@CollectionTable(
   name = "nameOfCollectionTable",
   joinColumns = @JoinColum(name="fk")
)

Result:

Table X:
id column1 column2
3 .....         ....

Table Remarks:
id remarks 
3  remark1
3  remark2
3  remark3

JPA inheritance SINGLE_TABLE JOINED TABLE_PER_CLASS

The advantage of using a O/R mapper like Hibernate by use of the JPA specification is for sure that one can develop independently of a certain database system and for a programmer you don't have to change the object-oriented world for doing persistence.
Therefore you want definitely use inheritance with JPA.
There are different strategies for realizing inheritance:
Default is SINGLE_TABLE, if only the annotation @Inheritance has been assigned.
There are 2 other strategies possible:

@Inheritance(strategy=InheritanceType.JOINED) und
@Inheritance(strategy=InheritanceType.TABLE_PER_CLASS).

So what do these strategies mean?
The default(SINGLE_TABLE) maps all sub classes of the inheritance chain in one database table.
With JOINED all abstract classes and concrete classes of the inheritance chain are stored in separate tables.
With TABLE_PER_CLASS every concrete class of the inheritance chain gets its own dedicated table.

What strategy should be used in what situation?

The answer of this question derives of mainly 2 aspects:

• Polymorphy
• Performance

Everybody wants to have high performance - but there are situations in which flexibility and expandability are more important than performance: e.g. new features, configurations or subordinate use cases.
There are situations in which the inheritance chain refers to important transaction data and affects core use cases - in that case performance is more important that flexibility or expandability.

SINGLE_TABLE has the advantage of a high performance data access, because everything is stored in one table. If the concrete subclasses highly differ and the amount of very different sub classes is high, the result is a wide table. That may result in an unfavorable tablespace and less rows of the table will be cached by the database. In this case a SINGLB_TABLE strategy is a bad decision.

Disadvantage data integrity:
With SINGLE_TABLE all not primary keys must be nullable, because all sub classes are stored in this table 
and therefore some types will have no values for certain columns.
If you do know that the most and the most important data access situations want to fetch concrete types from persistence context, TABLE_PER_CLASS is a good choice.
With this strategy polymorphic queries have very poor performance because of the resulting UNION queries generated by the JPA provider like Hibernate and the use of polymorphic associations is not possible, because the abstract types are not stored in the database table.
With JOINED polymorphic queries are possible because of the @DescriminiatorValue which results in a column for distinguishing the several subclasses. Those queries are realized with the help of outer joins - much better in performance than UNION selects. Concrete types can be queried with help of inner joins.
There the name of the strategy comes from.


Realization
There is only one abstract class. The subclasses inherit from this class and they all are marked with @Entity.
With SINGLE_TABLE and JOINED according to the JPA standard a @DiscriminatorValue must be provided. It is of type string and it's name is per default dtype and is used for distinguishing the concrete classes from each other: in the database of this table the corresponding string value is stored in the column dtype.

@Inheritance(strategy=InheritanceType.SINGLE_TABLE)
public abstract class BaseClass implements Serializable {
@Id
@GeneratedValue
private Long id;
private String common;
 ...
}

@Entity
@DiscriminatorValue(value="A")
public class A extends BaseClass {
 private String specificA;
...
}
@Entity
@DiscriminatorValue(value="B")

public class B extends BaseClass {
 private String specificB;
 ...
}
Following table is the result:
dtype|id|common|specificA|specificB

Having an instance A the column specificB will always be null.
On the other side polymorphic queries(select * from BaseClass) or querying a concrete subclass (select * from A ...) are lightning fast.

If it would be TABLE_PER_CLASS polymorphic queries are not possible.
With JOINED all queries after concrete subclasses are inner joins,
polymorphic queries outer joins.
So you could say JOINED is a trade off between performance and expandability/flexibility.

JPA Many-To-Many relationship mappedBy

A many-to-many-relationship is realized by the JPA annotation @ManyToMany.
On both sides  there is a collection marked with that annotation. One of this side must be the owner / manager of the relationship - mappedBy:
@Entity
public class A {

@Id
  private Long id;
  ...
  @ManyToMany
  private List<B> colBs;
}

@Entity

public class B {
  @Id
  private Long id;

  ...
  @ManyToMany(mappedBy="colBs")

  List<A> colAs;
}



So entity b is owner of the n-m-relationship.

A n-m-relationship is always stored in a distinct table:
The name is made of: 
entity name of the owner of the relationship + _ + entity name of the other side.
In this case: A_B.
The foreign key column for generating the relation is per default made by this pattern:
Attribute name of the referred side + _ + primary key name.
In this example: colbs_id respectively colas_id

JPA Hibernate one-to-many orphanRemoval

On a 1-n-relationship there is again the possibility for expression the relationship uni- or bidirectionally:
Unidirectional:
On the 1-side there is a collection, which is annotated with @OneToMany.
The collection consists of entities of the n-side.
Different parameters can be set:
- mappedBy
The owner of the relationship. Defined on the 1-side, than the n-side is owner of the relationship or the manager of this relationship. So here the instance variable of the n-site is stated.
Thus on the n-side a foreignKey field on default is maintained, in order for handling the relationship in the database.
- cascade
- fetch

If the relationship should be bidirectional, than also on the n-side an annotation by the instance variable must be set:
@ManyToOne
Here there are more interesting annotation / parameter which can be set.
But first of all the default
If there is no name stated with @JoinColumn, the foreignKey column has the following name:
name of the instance variable + _ + name of the primary key field

- @JoinColumn with the parameters:

• referencedColumnName - Name of the foreignKey field
• nullable - Must the relationship be set?
• orphanRemoval - with true during deleting of the entity on the 1-side all referenced entities on the n-side will be deleted. This makes sense if a composition should be used. In some cases  @Embeddable respectively @Embedded could be an alternative. For the realisation of the composition, that means the parts of the composition will be managed by the head of the composition and the parts should never exists without the head or the composition, one should set on the mappedBy the 1-side as owner. Therefore during deletion of the entity on the 1-side the JPA-provider respectively Hibernate will also delete the n-side.
• cascade - with cascade = CascadeType.PERSIST one can define that calling persist on an entity all other related entities on the n-side will also be made persistent.
• fetch - in relation to performance / load behaviour one should think at least twice

binary assoziation JPA

one-to-one-relationship with JPA

For this there is the JPA annotation @OneToOne.
Per default this is done by foreignKey in the database.

Unidirectional

In a unidirectional @OneToOne-relationship the entity from which one can navigate to the other entity gets the @OneToOne-annotation.

mappedBy

if the one-to-one relationship of the two entities should be bidirectionally,
on every entity the @OneToOne-annotation must be set and with help of the mappedBy-attribute it is defined which side manages the relationship.
The relationship with the entity, which has the mappedBy, will be be managed by the other side and the parameter will be set to the referred property.
The result is that is defined on which side the foreignKey-column will be applied  to.
The entity without the mappedBy-attribute gets in the according table the foreignKey-column, so that the relationship can be expressed. The name is derived from:
Table name of the referred table + _ + column name of the primary key.


@Entity
public class A {
 @OneToOne
 B b;
 ...
}

@Entity
public class B {

@OneToOne(mappedBy="b")
A a;

Persist @OneToOne per JPA

During persist of an @OneToOne-relationship one must set the relation pragmatically and both entities / every side must be made persistent by means of calling the persist-method.

cascade=CascadeType.PERSIST

To avoid, that both entities of an @OneToOne-relationship must be saved separately,
one can take advantage of the cascade-attribute of the @OneToOne-relationship:
@OneToOne(cascade=CascadeType.PERSIST).
As a result the relationship of an entity will also be made persistent.
An OneToOne-relationship owns the entity without the mappedBy-attribute 
and has the foreignKey-column , so that the entity and the connected other entity will be made persistent.
So no need for calling persist() twice, calling persist on entity A is enough, B will also be made persistent.

optional

There is a parameter optional in context of an @OneToOne-relationship, which decides if the relationship must be set or not.

@OneToOne(optional=true)

Caching via Hibernate JPA

JPA Caching

For optimization of an existing application, it is very useful caching catalog data and master data.
Unlike transaction data, which refer to certain processes and are needed in certain usecases, master data are involved in nearly every process.
So it is clever to thing about caching these data.
So called key data or catalog data have in common that they are read all the time, but nearly never changed.
How can caching strategies with JPA / Hibernate can be done?

    Activation of the Hibernate Second Level Caches
    Configuration of the cache provider
    Filtering of the cache candidates and tagging the entities / associations.

1. Activation of the Hibernate Second Level Caches

In the persistence.xml or hibernate.cfg.xml the property hibernate.cache.use_second_level_cache must be set to true.
With that besides the first level cache, which is in charge of all managed entities of the according hibernate session, a second cache in hibernate is activated, which works beyond transactions.

The first level cache is bound to the actual transaction and is cleared as soon as the transaction is terminated(committed).

The second level cache is not bound to the hibernate session, it is bound to the entity manager / HibernateSessionFactory.
Thus entities can be handed out without a callback to the database has to be done, if the entities are not loaded in the actual transaction, but a transaction before has already loaded/written those entities.


2. Configuration of the cache provider

Hibernate has an own implementation for the second level cache.
But this is only a test implementation and should never be used for productive use.
Therefore one must select the according cache provider and this should be set in the configuration:
hibernate.cache.region.factory_class

There are some cache provider: JBoss Cache, Infinispan, ...
Infinispan will be the official cache provider in JBoss and is highly in use.

The introduction of regions(Interface region.factory_class) is new:
There is a possibility for caching query results.
Rules and approach for such a cache differs from an application wide Second-Level-Cache, who caches entities beyond transactions.
Thus the second level cache can be divided in different regions in order for configuring certain cache mechanism, eviction policies etc.

3. Filtering of the cache candidates and tagging the entities / associations.

In order to cache an entity by hibernate, it must be annotated with
@org.hibernate.annotations.Cache respectively JPA @javax.persistence.Cacheable

An association must be annotated with @org.hibernate.annotations.Cache in the according getter method of the involved entity.
Following book copes with Caching and Query Cache(see chapter 9):

Treatment of IDs / performance and optimization

If a Object/Relation Mapper like e.g. Hibernate is used, then every managed entity, which will be made persistent, must have an ID.
There must be something, which clearly identifies an entity.
In JPA the annotation @Id is used.


The attributes for identification can be chosen by the following aspects:

1. Usage of a more professional key
2. Usage/generation of a technical key

Alternative 1 can not be found for every entity.
Alternative 2 means that for new objects new IDs have to be generated.

JPA Generators

With JPA generators are been used:

@GeneratedValue

There are 3 generators, one of them is selected by the annotation parameter strategy:

1. IDENTITY
2. SEQUENCE
3. TABLE
4. AUTO

IDENTITY

With IDENTITY an autoincrement-column is used, if the underlying database can support this strategy.

SEQUENCE

With SEQUENCE an decidedly databased generator is used, who is in charge for incrementing the value.

TABLE

With TABLE a table hibernate_sequences is used, which holds the values of the generator.

AUTO

With AUTO hibernate decides on account of the configured database dialect, which generator should be used in particular.

JPA PERFORMANCE

Performance

Basically a technical key means poorer performance, because during persist the EntityManager/HibernateSessionFactory has to read / collect the assigned value of the database.

Those databases which do not provide these important operations per API suffer from bad insert performance.

Is there no special operation or opportunity for efficient read of this data, poorer insert operations will be recognized as in other database systems.

The AUTO-strategy can destroy performance on certain database systems.
 On a oracle database (and also e.g. in a DB2 database) a sequence will be used, BUT with an allocationSize of 1.

This can not be configured with the AUTO-strategy - what means that
the insert performance is bad, because after every insert the next value of the sequence has to be collected using the database driver.
By default generating manually or by use of a database tool on DB2/Oracle, usually  the next 20 IDs will be cached, which can than be used by the EntityManager in order to avoid asking the sequence object after every insert operation.

 CREATE SEQUENCE XY
      START WITH 1
      INCREMENT BY 1
      NO MAXVALUE
      NO CYCLE
      CACHE 20;
     INSERT INTO ORDERS (ORDERNO, CUSTNO)
       VALUES (NEXT VALUE FOR XY, 123456);

Performance @GeneratedValue

If performance is crucial the annotation @GeneratedVaule can be removed, so that the JPA provider is not in charge for generating the ID and setting the ID during the persist operation.
In that case one must generate the ID on your own - which can be regarding database systems and driver possibilities considerably faster.

UUID class

In the java.util-package there is since Java 1.5 a UUID class which can be used for generating UUIDs.
Eventually one should combine the generated value with a secure distinct information according to the context, so that the UUID will also be by chance unique.

The benefit is that after calling the persist method a look up by JPA / driver for the assigned value is not needed anymore.

A very good literature for this topic is found in the following book

(chapter 4 section mapping the Primary Key):