Tomcat clustering and session persistency in AWS

Tomcat supports clustering out-of-the-box (see http://tomcat.apache.org/tomcat-7.0-doc/cluster-howto.html). However, since Amazon EC2 environment has some pecualiar features such as not supporting IP multicasting (http://aws.amazon.com/vpc/faqs/#R4), the traditional(?) way of Tomcat clustering cannot be used. Instead of using direct communication between different cluster nodes, you can use HTTP session sharing via database. Although this is not as fast or flexible as direct communication, it's still better than no clustering.

How session sharing works

Tomcat copies session data to a database when it's shut down. It also makes a periodical copying of idle sessions every now and then (the configuration given below doesn't define the exact maximum session idle time before Tomcat does replication to database. The configured maximum time is about maxIdleBackup + processExpiresFrequency * engine.backgroundProcessorDelay. Google for more details...).

Because all the Tomcats are sharing a commong session repository i.e. database, they all have access to each other's sessions that have been persisted at least once. Whenever a new Tomcat instance starts, it reads the existing sessions from common session repository and is then able to serve any request to any Tomcat in the cluster up to start-up time of the instance.

Because of delays in sharing the sessions, you should enable sticky sessions in you load balancer. This means that each request from a user will be directed to the same Tomcat as long as user HTTP session exists.

Use application generated cookie in ELB

In order to use database session sharing, you must use sticky sessions in your load balancer. Amazon ELB has the options of using either ELB generated session cookie or application generated session cookie. I think that in this case one should use application generated session cookie (JSESSIONID), because ELB generated session cookie may have different expiration time than your application's session. If ELB cookie expires before you application's session, user may get served by different Tomcat, although application's session has not changed. This may lead into nasty problems.

Setup RDS based database for Tomcat session storing

The easiest way of using a common session replication database in Amazon is RDS service. Of course you may also set up your own database on one EC2 instance, but this would be a single point of failure. So, I'm using RDS and MySQL in this example. Create a session database and a session table in the database.

CREATE DATABASE tomcat_data DEFAULT CHARACTER SET UTF8 COLLATE utf8_swedish_ci;
GRANT ALL ON tomcat_data.* TO "tomcat"@"%" IDENTIFIED BY "tomcat";


create table tomcat_data.tomcat_sessions (
  session_id varchar(100) not null primary key,
  valid_session char(1) not null,
  max_inactive int not null,
  last_access bigint not null.
  app_name varchar(255),
  session_data mediumblob,
  KEY kapp_name(app_name)
);

MySQL connection from Tomcat

You need to copy the mysql-connector-java-x.x.xx.jar under $CATALINA/lib (on Ubuntu, it's /usr/share/tomcat6/lib/ depending on your Ubuntu/Tomcat version). This JAR is needed to create connection from Tomcat to MySQL.

Here's context.xml you need. Remember to change localhost to something more meaningful in the XML file.

<Context>

...

    <!-- The idle time in seconds before persisting session is
         maxIdleBackup + processExpiresFrequency * engine.backgroundProcessorDelay.
         The backgroundProcessorDelay is around 10 seconds. This configuration should make Tomcat
         to persist sessions after 20 seconds of idling and hold them in memory as long as each
         instance wants -->
    <Manager className="org.apache.catalina.session.PersistentManager"
      saveOnRestart="true"
      minIdleSwap="-1"
      maxIdleSwap="-1"
      maxIdleBackup="20"
      processExpiresFrequency="1">
      <Store className="org.apache.catalina.session.JDBCStore"
        connectionURL="jdbc:mysql://localhost/tomcat_data"
        driverName="com.mysql.jdbc.Driver"
        connectionName="tomcat"
        connectionPassword="tomcat"
        sessionIdCol="session_id"
        sessionValidCol="valid_session"
        sessionMaxInactiveCol="max_inactive"
        sessionLastAccessedCol="last_access"
        sessionTable="tomcat_sessions"
        sessionAppCol="app_name"
        sessionDataCol="session_data"
      />
    </Manager>
...

</Context>

Problems in this solution

If Tomcat dies unexpectedly, the new sessions served by dead instance are lost. Most likely with the configuration above, sessions newer than 30 seconds will disappear. Also new changes that have been done to session data will be lost.

Session related database traffic could grow quite high in a busy web site. I have no cure for this, but of course you can at least use a different database for sessions than what's being used for other data.