Java Quick References…

Here is some stuff that every Java developer must be aware of:

• XML schema namespaces (default namespace, target namespace etc): http://www.oracle.com/technology/pub/articles/srivastava_namespaces.html
• Java 5, JDBC RowSets: http://java.sun.com/developer/Books/JDBCTutorial/chapter5.html , http://download.oracle.com/docs/cd/B28359_01/java.111/b31224/jcrowset.htm#i630230, http://edocs.bea.com/wls/docs90/jdbc/rowsets.html#1041182
• JVM tuning: http://raymondtay.blogspot.com/2008/09/j2eeweblogic-performance-tuning-with.html , http://blogs.sun.com/jonthecollector/

Hibernate Quick References…

Here are some common tips that every hibernate developer must know. Will post detailed examples later, just posting a few quick links right now for ready reference:

• Hibernate tips and tricks: https://www.hibernate.org/118.html
• Hibernate performance Tips: http://docs.jboss.org/hibernate/stable/core/reference/en/html/performance.html
• Hibernate II level caching: http://www.javalobby.org/java/forums/t48846.html , http://blog.dynatrace.com/2009/03/24/understanding-caching-in-hibernate-part-three-the-second-level-cache/
• Hibernate fetch strategies and lazy fetching: http://www.javalobby.org/java/forums/m91885142.html
• Hibernate n+1 selects problem: http://www.realsolve.co.uk/site/tech/hib-tip-pitfall.php?name=n1selects
• Hibernate inheritance mapping: http://docs.jboss.org/hibernate/stable/core/reference/en/html/inheritance.html

Template Pattern and Spring

As you might be aware, Template is a GoF behavioral pattern. Here I first explain what the Template pattern is and then show an example of the pattern as implemented in the Spring framework:

Template works on the philosophy of Abstract classes, wherein the Abstract class implements some boiler plate functionality and let the subclasses implement the specifics.

For example, when you want to execute a JDBC query against a database, you have to implement the following steps:

1. Obtain a DataSource
2. Use the DataSource to get a Connection object
3. Use the Connection object to create a Statement (or a PreparedStatement)
4. Execute a query to get the ResultSet
5. Iterate over the results and create a list of VOs (as an example of a business logic)
6. Close the ResultSet
7. Close the Statement
8. Close the Connection

Out of the above steps, most of these are boiler plate code while steps #4 and #5 reflects the actual business logic. If you create a Template class (essentially an Abstract class) that models these steps and allow subclasses to customize the steps #4 and #5 then it will help subclasses reuse the boiler plate code.

Spring does this by providing classes like JdbcTemplate, HibernateTemplate, JmsTemplate etc.  Here is how things work in the Spring world:

Example code:
List results = getHibernateTemplate().executeFind(new HibernateCallback() {
                   public Object doInHibernate(Session session) {
                       Query query = session.createQuery("from Event");
                       query.setMaxResults(2);
                       return query.list();
                   }
});

Unix Find and Grep

A lot of times we have to recursively find files and grep some text in each of the files, here is a handly command to do so:

find . -name “*.log” -exec grep “mytext” {} \;

Structural Patterns

• Proxy could be used when you want to lazy-instantiate an object, or hide the fact that you’re calling a remote service, or control access to the object.

• Decorator is also called “Smart Proxy.” This is used when you want to add functionality to an object, but not by extending that object’s type. This allows you to do so at runtime. Java I/O reader, writers are a fine example of this.

  

• Adapter the adapter design pattern (often referred to as the wrapper pattern or simply a wrapper) translates one interface for a class into a compatible interface. An adapter allows classes to work together that normally could not because of incompatible interfaces. Adapter is used when you’re trying to unify the interfaces of some incompatible classes that already exist. The Adapter functions as a kind of translator to implementations that could be considered legacy.

  

• Facade is a higher-level (read: simpler) interface to a subsystem of one or more classes. Think of Facade as a sort of container for other objects, as opposed to simply a wrapper.

• Bridge is very similar to Adapter, but it is used for code that is more likely to be greenfield. We call it Bridge when you define both the abstract interface and the underlying implementation i.e. you’re not adapting to some legacy or third-party code, you’re the designer of all the code but you need to be able to swap out different implementations.

Perhaps the most confusing of these is Bridge, here is how a wise guy has tried to define it:

The Bridge pattern is an application of the old advice, “prefer composition over inheritance”. It becomes handy when you must subclass different times in ways that are orthogonal with one another. Say you must implement a hierarchy of colored shapes. You wouldn’t subclass Shape with Rectangle and Circle and then subclass Rectangle with RedRectangle, BlueRectangle and GreenRectangle and the same for Circle, would you? You would prefer to say that each Shape has a Color and to implement a hierarchy of colors, and that is the Bridge Pattern.

Here is an example of a Bridge:

Usage:

PersistenceImplementor implementor = null;
if (true){
	implementor = new DabatasePersistenceImplementor();
}else{
	implementor = new FileSystemPersistenceImplementor();
}
Persistence persistenceAPI = new PersistenceImp(implementor);
Object o = persistenceAPI.findById("12343755");
// do changes to the object then persist
persistenceAPI.persist(o);
persistenceAPI.deleteById("2323");

Application Performance Optimization Tips

In my experience, mostly database is the bottleneck, so the prime objective should be to optimize queries and to reduce the number of database calls and bring as much data together as possible.

1. Try to bring multiple result sets together from the database, use Stored procs if needed.

2. Use JDBC batch updates to club together multiple database updates in a single database call.

3. Use Java caching of results wherever possible (using caching libraries, static variables etc).

4. Another possibility is to use static db views / summary tables, which can be nightly refreshed etc.

5. Appropriate use of indexes in db tables can make a lot of difference, especially in case of complex joins etc.

6. Reduce the amount of Java reflection being used in code or cache the reflection stuff.

7. Do selective logging, check if log enabled before issuing a log statement.

8. Tune app server params like increase connection pool size, -Xms, -Xmx, -XX:PermSize, -XX:MaxPermSize etc.

9. Consider appropriate clustering of app servers, JMS queues etc and asynchronous execution of complex jobs (using JMS) etc

10. Use optimized XML parsing, serialization libraries etc.

Using good profiling tools to find out pain areas is extremely important.

In my experience, mostly database is the bottleneck, so the prime objective should be to optimize queries and to reduce the number of database calls and bring as much data together as possible. Using good profiling tools to find out the pain areas is extremely important.

1. Try to bring multiple result sets together from the database, use Stored procs if needed.
2. Use JDBC batch updates to club together multiple database updates in a single database call.
3. Use Java caching of results wherever possible (using caching libraries, static variables etc).
4. Another possibility is to use static db views / summary tables, which can be nightly refreshed etc.
5. Appropriate use of indexes in db tables can make a lot of difference, especially in case of complex joins etc.
6. Reduce the amount of Java reflection being used in code or cache the reflection stuff.
7. Do selective logging, check if log enabled before issuing a log statement.
8. Tune app server params like increase connection pool size, -Xms, -Xmx, -XX:PermSize, -XX:MaxPermSize etc.
9. Consider appropriate clustering of app servers, JMS queues etc and asynchronous execution of complex jobs (using JMS) etc.
10. Use optimized XML parsing, serialization libraries etc.

Abstract Factory Pattern

I had implemented an Abstract Factory in one of my projects. Posting the details of the implementation here.

The requirement is that for CDR database, there are 2 datasources, one for HongKong and one for US. And for ProdProfile database, there is only datasource for both locations. The location is not known at the compile time so we can’t inject datasource in the factory at compile time. Also passing the location in the getDao() method looks ugly since location is not required for all type of DAOs. And even ProductProfile datasource may become location specific in future.

I have tried to make one factory for each Location. The factory will instantiate different DAOs, whether the DAO is injected in the factory is location specific or not is controlled by Spring config.

AbstractDaoFactory:

CdrDao getCdrDao()

ProdProfileDao getProdProfileDao()

USDaoFactory extends AbstractDaoFactory

CdrDao getCdrDao()

ProdProfileDao getProdProfileDao()

HKDaoFactory extends AbstractDaoFactory

CdrDao getCdrDao()

ProdProfileDao getProdProfileDao()

CdrDao

setDS()

loadProductList()

…

CdrDaoImpl: 2 spring beans for this Impl, but different DS injected in these: USCdrDaoImpl, HKCdrDaoImpl

ChecklistDao

setDS()

loadChecklist()

ChecklistDaoImpl: just 1 spring bean for this Impl, so both factories will return the same instance

Class AbstractDaoFactory:

+ CdrDao getCdrDao()

+ ProdProfuleDao getProdProfileDao()

Class USDaoFactory extends AbstractDaoFactory

+ CdrDao getCdrDao()

+ ProdProfileDao getProdProfileDao()

Class HKDaoFactory extends AbstractDaoFactory

+ CdrDao getCdrDao()

+ ProdProfileDao getProdProfileDao()

Interface CdrDao

+ setDs(DataSource ds)

+ loadLegCodes()

Interface ProdProfileDao

+ setDs(DataSource ds)

+ loadAllSeries()

Class CdrDaoImpl: created 2 spring beans for this Impl, but different DS injected in these: USCdrDaoImpl, HKCdrDaoImpl

Class ProdProfileDaoImpl: created just 1 spring bean for this Impl, so both factories will return the same instance

Notes on QoS and Capacity Planning

The architecture you create must address the following service-level requirements: performance, scalability, reliability, availability, extensibility, maintainability, manageability, and security.

You will have to make trade-offs between these requirements. For example, if the most important service-level requirement is the performance of the system, you might sacrifice the maintainability and extensibility of the system to ensure that you meet the performance quality of service.

Performance

The performance requirement is usually measured in terms of response time for a given screen transaction per user. In addition to response time, performance can also be measured in transaction throughput, which is the number of transactions in a given time period, usually one second.

Some of the performance considerations to be taken care while designing systems are: Network overheads (making multiple fine grained network calls vis-à-vis a single course grained network call), Memory issues (Heap size for objects created on heap, Perm size for reflection and dynamically created objects), Logging issues (amount of logging, synchronous/asynchronous i/o etc), Concurrency issues (is thread safety needed).

Scalability

Scalability is the ability to support the required quality of service as the system load increases without changing the system. A system can be considered scalable if, as the load increases, the system still responds within the acceptable limits.

The capacity of a system is defined as the maximum number of processes or users a system can handle and still maintain the quality of service. If a system is running at capacity and can no longer respond within an acceptable time frame, then it has reached its maximum scalability. Scalability can be added vertically or horizontally:

Vertical scaling in a J2EE application involves adding additional servers running on the same machine. With vertical scaling, the machine’s processing power, CPU usage, and JVM heap memory configurations are the main factors in deciding how many server instances should be run on one machine (also known as the server-to-CPU ratio).

Horizontal scaling involves adding more machines to the cluster, thus increasing the overall system capacity.

Vertical scaling typically does not have an impact on the architecture, but the architecture must be created with special considerations for a horizontal scaling.

Reliability

Reliability ensures the integrity and consistency of the application and all its transactions.

As the load increases on your system, your system must continue to process requests and handle transactions as accurately as it did before the load increased.

Availability

Availability ensures that a service/resource is always accessible. By setting up an environment of redundant components and failover, an individual component can fail and have a negative impact on reliability, but the service is still available due to the redundancy.

Load balancing (and Failover) is a mechanism where the server load is distributed to different nodes within the server cluster, based on a load balancing policy. There are many different algorithms to define the load distribution policy, ranging from a simple round robin algorithm to more sophisticated algorithms like minimum load, weight-based, last access time etc.

Two popular methods of load balancing in a cluster are DNS round robin and hardware load balancing. DNS round robin provides a single logical name, returning any IP address of the nodes in the cluster. This option is inexpensive, simple, and easy to set up, but it doesn’t provide any server affinity or high availability. In contrast, hardware load balancing offers virtual IP addressing. Here, the load balancer shows a single IP address for the cluster, which maps the addresses of each machine in the cluster. The load balancer receives each request and rewrites headers to point to other machines in the cluster. If we remove any machine in the cluster, the requests are still served seamlessly. Hardware load balancing has obvious advantages in terms of server affinity and high availability.

Extensibility

Extensibility is the ability to add additional functionality or modify existing functionality without impacting existing system functionality. You should consider the following when you create the architecture and design to help ensure extensibility: low coupling, interfaces, and encapsulation.

Maintainability

Maintainability is the ability to correct flaws in the existing functionality without impacting other components of the system. When creating an architecture and design, you should consider the following to enhance the maintainability of a system: low coupling, modularity, and documentation.

Manageability

Manageability deals with system monitoring of the QoS requirements and the ability to change the system configuration to improve the QoS dynamically without changing the system.

Security

Security is the ability to ensure that the system cannot be compromised. Security includes not only issues of confidentiality and integrity (SQL injection, URL injection etc), but also relates to Denial-of-Service (DoS) attacks that impact availability. Creating an architecture that is separated into functional components makes it easier to secure the system because you can build security zones around the components. If a component is compromised, then it is easier to contain the security violation to that component.

Hibernate Inheritance

Hibernate supports different types of inheritance, here we explain it with the help of some examples. Please consider the following structures for the purpose of the examples:

Tables:

payment_t: pmt_id (PK), amount, paid_by, date_time

cash_payment_t: pmt_id (PK), currency

credit_payment_t: pmt_id (PK), cc_no, cc_type

cheque_payment_t: pmt_id (PK), cheque_no

Value objects:

Interfaces:

PaymentVo

Classes:

CashPaymentVo

CreditCardPaymentVo

ChequePaymentVo

Remove ^M characters from Unix file

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.

To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).