ESB concept / bus

1) Explanation
The ESB (Enterprise Service Bus) is the holy grail of applications integration.
It is actually an integration platform used to allow applications to integrate more easily into an enterprise landscape.
The ESB defines the infrastructure, the foundation, in which the different applications and services of an enterprise landscape will integrate. It stands for a specific architecture style enforcing a communication bus to be used for inter-application or inter-services communication. Depending on the structuring of the enterprise landscape, that means service oriented or application based, the bus is used for service integration in comparison to "only" connecting different applications over a common communication bus.

2) Initial situation
Having an enterprise landscape with many applications that need to communicate with each other,
lead to many point-to-point communications with all its disadvantages.
That are implementing retry mechanism in each application that interacts with another application.
Dealing with the complexity of timeouts(connection and read timeout) - finding the right timeout window, distinguishing between operations that can and that shouldn't be repeated.

3) Benefits
Business events that happened are published by the source application to  the bus.
Typically there is an application interested in this event, so it will consume it as this kind of processing would have been  done by a point-to-point communication.
But now comes the benefits of an enterprise bus into play. If business requirements changes and enterprise landscape grow more and more applications might potentially be interested in that kind of events. So the producing application does not have to be changed at all.
Additional systems that are interested in that event just subscribe to the topic or if there is a dispatcher application in place, that consumes special kind of messages laying on the bus, will then dispatch requests to the corresponding systems.
Integration scenarios are typically configured / implemented with the help of a DSL leading to more maintainability and faster realization.

ESB allows:
- Heterogeneous environments get interoperable

- Service virutalization aka as location transparency
The service consumer and the service provider is decoupled by the ESB.
With service virtualization the service consumer has not be reconfigured if the service provider is changed regarding
   endpoint information.

- Service versioning
With message transformation the message of an interface that is not support anymore is translated into a message of
  new interface version.
Benefitting from again the fact that the service consumer is decoupled from the service provider.
The additional "layer" can be used to allow a technical mapping between an old interface and the new one.
Allowing business adaption independent of interface availabilities.

- Transaction-based message processing
The message is taken from queue and during the processing of the defined workflow involving maybe different services
   and service calls, the whole transaction is only committed if the process arrived the last service that is a database
   adapter. The whole process is committed as the flow reaches the db adapter.
So the ESB can be used to coordinate distributed transactions with different services involved. The client has only to  
   mark the begin and end of a transaction.

The coordination work is done by the bus.

4) Background
Basic groundwork was done by Hophe and Woolf with their work on EIP - Enterprise Integration Platform.
They have described common integration scenarios, abstracted them into different patterns, and categorized them into the following 6 categories:

- Message endpoints
All kind of patterns for connecting applications that should integrate with each other.
E.g.:
- Polling consumer - An adapter that polls periodically a data source and consume the data that needs to be
            processed.
         - Service activator - A component that locates a service of an application that then will be called.

- Message construction
All kind of patterns that deal with messages itself.
     E.g.:
- Correlation Identifier  -  The correlation id is used to map messages to a transaction Messages that took part in a
           transaction should have the same correlation id.
  - Message expiration - Message have a period of time in which they need to be processed. If this time passed
           without processing they are dropped by the message broker or moved do a dead letter queue.

- Message channels
All kind of patterns that describe how message are delivered to message endpoints.
E.g.:
 - Guaranteed delivery - Producer can rely to 100% on the fact that if a topic or queue has confirmed the receive of a
           message, that the message will be later on processed. After the confirmation the resources of the JMS client are
           freed and he can continue with further actions/processing.
-  Point-to-point channels - A message channel pattern to realize a synchronous communication based on JMS.
           Sender blocks until receiver of a message has processed and delivered the result to the sender.

- Message routing
All kind of pattern that do not change the semantics of a message - no content change. But based on certain rules
     messages are forwarded to different endpoint.
E.g.:
- Aggregator - Split message are composed to a new resulting message.
- Splitter - A message is split in several parts that result in new messages.

- Message transformation
All kind of patterns that change the content of a message.
E.g.:
- Enricher - A message is enriched with further information from other services, data sources etc.
- Message translator - Transforming a message of a data model A into a message of a data model B without
          changing the semantic of the message.

- System management
All kind of patterns that can't be categorized in the above categories or have general supporting characteristics.

GAE and JPA

How does JPA work in a GAE environment?

The Google App Engine(GAE) supports JPA, but the persistence is not done in a relational database.

It uses a NoSQL-database, using BigTable technology.

So there have to be some restrictions:

1) Polymorph queries

2) Aggregation functions
3) Transactional behavior: in a transaction only objects of the entity group may be changed
4) ...

1) Especially in an object-oriented abstraction where the data model knows about inheritance relations between the entities and they get persisted accordingly e.g. each class will be saved into a separate table. 

On querying such a structure with JPQL GAE does not allow you the use of polymorph queries:

A extends B extends C

If you are not interested in retrieving a special entity type, it's very handy to retrieve all entities based on C, the top super class, and apply abstracted treatment in cases where common treatment can be done.

So a "from C where …." - JPQL query will be possible on a relational database, but unfortunately will fail in a GAE environment.

2) Aggregation functions like SUM, AVG, … are not usable on GAE.
The like -operator is limited in use - it can only be used on the end of a search - token e.g. ... like 'Adam R%' ....

3) The transactional behavior JPA offers is limited on the GAE platform.
In one single transaction A only objects of the same entity group may be changed.
Their changes will then be applied accordingly in the database.
An entity group is a collection or grouping of objects that creates a data structure. This data structure consists of root objects and dependent objects. Instances of an entity group are so called
- root objects (starting entity)
- and from that dependent objects.
On creation of an instance objects can point to a parent entity.
The entity without a parent entity are the so called root entity.
Datasets of this entity groups reside on different nodes of the cluster in the distributed data storage. But one single dataset is normally physically available on one node. So that during data processing communication overhead is reduced.  

One may question how the relationship inside an entity group is realized as GAE does not run on a relational database?
The relationship is not handled like on relational database by using attribute/s or in JPA language properties. The primary key of the entity is used to route through the hierarchy:
In such an entity group we have a parent having a primary key and all the children have a pk that contains the parent pk. The pk normally has the type and a certain id. Hence a child pk consists of the type + id of parent + own id.
For example: Invoice(5)/InvoiceItem(1)

Because of this @ManyToMany and joining is not usable as well.