Troubleshooting Tomcat

Tomcat 5.5 or later is required!  Tomcat6 is recommended.  Are you  using an older version of Tomcat?

General troubleshooting

The first thing to do when troubleshooting tomcat is to check the tomcat logs. They are found at C:\Program Files\Apache Software Foundation\Tomcat 6.0\logs - look for the one named stdout_yyyymmdd.txt where yyyymmdd is today's date. If this log is too long and you can't find the relevant part, you can stop the tomcat service (in Windows using control panel, administrative tools, services), rename the file to stdout_yyyymmdd_old.txt, restart tomcat and it will start writing a new log at stdout_yyyymmdd.txt.

Sluggish performance

Confirm that you have installed tomcat from the tomcat.apache.org website. The apt-get install of Ubuntu tomcat6 has terrible performance consequences.

Running Tomcat as a Service on Ubuntu

If you're trying to run Tomcat as a serving on Ubuntu, you will likely run into permission issues. The easiest way to solve this issue is to disable the Java security manager in /etc/init.d/tomcat6.

Error:

java.security.AccessControlException: access denied (java.io.FilePermission /usr/share/tomcat6/webapps/openmrs/WEB-INF/dwr-modules.xml delete)

Solution:

# Use the Java security manager? (yes/no)
TOMCAT6_SECURITY=no

Error: prunsrv.c Failed creating ... jvm.dll

If you cannot start the Tomcat service, try checking the tomcat logs at C:/Program Files/Apache Software Foundation/Tomcat/logs. If these logs give errors like "Failed creating java C:\Program Files\Java\jre1.6.0\bin\client\jvm.dll" then do the following:

  • Search for msvcr71.dll on your hard drive
  • Copy this file to C:\Windows\System32

Tomcat hangs and stops responding when uploading the war file

When uploading the war file, Tomcat hangs and stops responding. This typically occurs if you have not defined a MySQL user account that OpenMRS can use to access the database or if you have not granted this user full access to the OpenMRS database.

The default username is test with password test. The default username/password can be overridden in the openmrs_runtime_properties.properties configuration file.

To fix this: use Navicat or the MySQL administration tool (available from MySQL) to verify that you have a user with username test and password test and this user has full access to the OpenMRS database.

Tomcat error when uploading war file - FileUploadBase$SizeLimitExceededException

When uploading the war file, you can get the following error message with Tomcat's default settings.

description The server encountered an internal error () that prevented it from fulfilling this request.
exception
java.lang.IllegalStateException: org.apache.tomcat.util.http.fileupload.FileUploadBase$SizeLimitExceededException: the request was rejected because its size (54624465) exceeds the configured maximum (52428800)

To fix this: Go to the web.xml of the manager application (for instance it could be under /tomcat7/webapps/manager/WEB-INF/web.xml).
Increase the max-file-size and max-request-size:

<multipart-config>
<!– 50MB max –>
<max-file-size>52428800</max-file-size>
<max-request-size>52428800</max-request-size>
<file-size-threshold>0</file-size-threshold>
</multipart-config>

Cannot connect to Tomcat on port 8080

This port is sometimes used by other programs, such as Popfile and TivoServer. You can use the Windows "netstat -ao" command to discover if another process is using port 8080.

In one case, software for a HP Laserjet 2840 printer was trying to use port 8005, which conflicts with Tomcat startup.

java.lang.OutOfMemoryError PermGen space

This error will appear after Tomcat has exhausted all of its memory. This is typically caused by using the "Update" or "Reload" function on a webapp too many times. Currently, Tomcat and/or the JVM isn't releasing all of the memory when a webapp is destroyed/recreated. After several reloads Tomcat is out of its allotted memory and will hang or freeze. Read more at ?Troubleshooting Memory Errors.

Can't deploy applications

This solution assumes you are using a Linux machine. If you are not, you may have to "translate" the general method into a Windows environment. First, undeploy the application from the Tomcat web interface. Next, log into the server's command line interface as root and find the process id using

ps ax | grep tomcat

It will likely be the first item returned and you will know because it's about three or four lines long. Find the process id (the first number on that line), and kill it with

kill -9 processid

Now restart the service with

service tomcat6 start

Log back into the Tomcat web interface and deploy your application.

Reporting OpenMRS Errors by Email

OpenMRS logs all errors using the log4j framework and while this is useful, serious errors are often written to file and forgotten. Fortunately, log4j supports email alerts through the SMTPAppender.

Find your log4j.xml file, add the following.

<appender name="EMAIL"  class="org.apache.log4j.net.SMTPAppender">
	<param name="BufferSize" value="512" />
	<param name="SMTPHost" value="localhost" />
	<param name="From" value="log4j@openmrs.org" />
	<param name="To" value="errors@openmrs.org" />
	<param name="Subject" value="OpenMRS Error" />
	<layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern"
			value="%p - %C{1}.%M(%L) |%d{ISO8601}| %m%n" />
	</layout>
	<filter class="org.apache.log4j.varia.LevelRangeFilter">
		<param name="LevelMin" value="ERROR"/>
		<param name="LevelMax" value="FATAL"/>
	</filter>
</appender>

You will also have to modify the root.

<root>
	<level value="WARN" />
	<appender-ref ref="CONSOLE" />
	<appender-ref ref="EMAIL" />
</root>

Be sure to change SMTPHost, From, To, and Subject to reflect your environment. The layout of the email can be controlled using ConversionPattern (although there seems to be a bug with now stack traces are printed). LevelMin and LevelMax changes which log messages are emailed. In the above example, WARN messages are passed to CONSOLE and EMAIL. Only log messages that are classified between ERROR and FATAL (inclusive) are emailed.

For more information on log4j and email, check out
Reporting Application Errors by Email and AuthSMTPAppender.

Enabling secure (SSL) data transfer in Tomcat

SSL, or Secure Socket Layer, is a technology which allows web browsers and web servers to communicate over a secured connection. This means that the data being sent is encrypted by one side, transmitted, then decrypted by the other side before processing. This is a two-way process, meaning that both the server AND the browser encrypt all traffic before sending out data. On the web, unsecure data arise often passed over HTTP (port 80) while secure data is passed over HTTPS (port 443).

Because the security of medical information is essential, it is important for all OpenMRS data to be passed over HTTPS. Of course, in addition to securing internet communication, implementers must also ensure the local network is secure, security patches are applied, strong passwords are used, and the OpenMRS machine physically secured.

To enable SSL in Tomcat, follow the instructions at Apache's Tomcat SSL Configuration page. InstallCert.java can help you install your self-created certificate on your client machines. GoDaddy/Verisign certificates work out of the box.

Once SSL is running, implementers may wish to redirect all unsecure requests to secure channels. To do this, a few configuration files in Tomcat must be changed.

First, make sure all Tomcat applications require HTTPS to operate. Find the $TOMCAT_HOME/conf/web.xml file. At the bottom of the file, immediately before make sure the following is uncommented (or added).

<!-- == Force all applications to use SSL == -->
	<security-constraint>
	    <web-resource-collection>
	        <web-resource-name>All applications</web-resource-name>
	        <url-pattern>/*</url-pattern>
	    </web-resource-collection>
	    <user-data-constraint>
	        <description>Redirect all requests to HTTPS</description>
	        <transport-guarantee>CONFIDENTIAL</transport-guarantee>
	    </user-data-constraint>
	</security-constraint>

</web-app>

Then, ensure all requests to the HTTP port are forwarded to the HTTPS port. Find the $TOMCAT_HOME/conf/server.xml file. In the HTTP connector section, ensure the redirectPort is pointing towards your HTTPS connector. Finally, ensure the HTTPS connector has SSL enabled. An example minimum configuration is below.

<Connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443" />

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
           maxThreads="150" scheme="https" secure="true"
           clientAuth="false" sslProtocol="TLS" />

Lost tomcat password

You can retrieve the tomcat admin password from C:\Program Files\Apache Software Foundation\Tomcat 6.0\conf\tomcat-users.xml

"SEVERE: Error listenerStart" in tomcat logfile

You deploy the openmrs.war, but it fails to start, with the only clue two simple messages in the tomcat logfile...

SEVERE: Error listenerStart
SEVERE: Context [/openmrs] startup failed due to previous errors

Well, it is most likely a problem with the application_data_directory. Investigate these questions:

  • have you created the runtime properties file?
  • have you specified application_data_directory in the runtime properties?
  • does the directory exist?
  • does tomcat have read/write permissions to the directory?

If that all checks out, then you probably have security violations with your tomcat configuration.

Error Loading Persisted Sessions

Tomcat tries to restore the exact memory state after each restart. OpenMRS does not depend on this, so you can ignore the annoying warnings printed to the logs that look like this:

SEVERE: IOException while loading persisted sessions: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException:

To turn off session persistence, Go to tomcathome/conf/server.xml and uncomment the "<Manager pathname="" />" line

ElException in parsing some JSP pages (Updated tomcat 7 and may be other servlet containers that conform to EL Spec 2.2)

In the latest version of EL Spec v2.2, Java identifiers and reserved keywords cannot be used in EL expression. In OpenMRS we have some parts in JSP that do not conform to this new spec. To workaround this problem, you can add "-Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true" to CATALINA_OPTS environment variable - See discussion: Identifier creation doesn't work in trunk

Troubleshooting Installation

For more troubleshooting tips see Troubleshooting Installation.