Eclipse Jetty Operations Guide

The Eclipse Jetty Operations Guide targets sysops, devops, and developers who want to install Eclipse Jetty as a standalone server to deploy web applications.

Introduction

If you are new to Eclipse Jetty, read here to download, install, start and deploy web applications to Jetty.

Eclipse Jetty Features

If you know Eclipse Jetty already, jump to a feature:

TODO

  • Jetty Overview

  • Jetty Modules

  • Rewrite Modules

Introduction to Eclipse Jetty

This section will get you started with Eclipse Jetty.

Quick Jetty Setup

Jetty is distributed in an artifact that expands in a directory called $JETTY_HOME, which should not be modified.

Configuration for Jetty is typically done in one (or more) other directories called $JETTY_BASE.

The following commands can be used to setup a $JETTY_BASE directory that supports deployment of *.war files and a clear-text HTTP connector:

$ export JETTY_HOME=/path/to/jetty-home
$ mkdir /path/to/jetty-base
$ cd /path/to/jetty-base
$ java -jar $JETTY_HOME/start.jar --add-module=server,http,deploy

This will create a $JETTY_BASE/start.d/ directory and other directories that contain the configuration of the server, including the $JETTY_BASE/webapps/ directory, in which standard *.war files can be deployed.

To deploy Jetty’s demo web applications, run this command:

$ java -jar $JETTY_HOME/start.jar --add-module=demo

Now you can start the Jetty server with:

$ java -jar $JETTY_HOME/start.jar

Point your browser at http://localhost:8080 to see the web applications deployed in Jetty.

The Jetty server can be stopped with ctrl-c in the terminal window.

The following sections will guide you in details about downloading, installing and starting Jetty, as well as deploying your web applications to Jetty.

Downloading Eclipse Jetty

The Eclipse Jetty distribution is available for download from https://www.eclipse.org/jetty/download.html

The Eclipse Jetty distribution is available in both zip and gzip formats; download the one most appropriate for your system, typically zip for Windows and gzip for other operating systems.

Installing Eclipse Jetty

After the download, unpacking Eclipse Jetty will extract the files into a directory called jetty-home-VERSION, where VERSION is the version that you downloaded, for example 10.0.0, so that the directory is called jetty-home-10.0.0.

Unpack Eclipse Jetty compressed file in a convenient location, for example under /opt.

For Windows users, you should unpack Jetty to a path that does not contain spaces.

The rest of the instructions in this documentation will refer to this location as $JETTY_HOME, or ${jetty.home}.

It is important that only stable release versions are used in production environments. Versions that have been deprecated or are released as Milestones (M), Alpha, Beta or Release Candidates (RC) are not suitable for production as they may contain security flaws or incomplete/non-functioning feature sets.

If you are new to Jetty, you should read the Jetty architecture section below to become familiar with the terms used in this documentation. Otherwise, you can jump to the section on starting Jetty.

Eclipse Jetty Architecture Overview

There are two main concepts on which the Eclipse Jetty standalone server is based:

  • The Jetty module system, that provides the Jetty features

  • The $JETTY_BASE directory, that provides a place where you configure the modules, and therefore the features you need for your web applications

After installing Jetty, you will want to set up a $JETTY_BASE directory where you configure Jetty modules.

Eclipse Jetty Architecture: Modules

The Jetty standalone server is made of components that are assembled together, configured and started to provide different features.

A Jetty module is made of one or more components that work together to provide typically one feature, although they may provide more than one feature.

A Jetty module is nothing more than Jetty components assembled together like you would do using Java APIs, just done in a declarative way using configuration files. What you can do in Java code to assemble Jetty components can be done using Jetty modules.

A Jetty module may be dependent on other Jetty modules: for example, the http Jetty module depends on the server Jetty module which in turn depends on the threadpool and logging Jetty modules.

Every feature in a Jetty server is enabled by enabling the corresponding Jetty module(s).

For example, if you enable only the http Jetty module, then your Jetty standalone server will only be able to listen to a network port for clear-text HTTP requests. It will not be able to process secure HTTP (i.e. https) requests, it will not be able to process WebSocket, or HTTP/2 or any other protocol because the correspondent modules have not been enabled.

You can even start a Jetty server without listening on a network port — for example because you have enabled a custom module you wrote that provides the features you need.

This allows the Jetty standalone server to be as small as necessary: modules that are not enabled are not loaded, don’t waste memory, and you don’t risk a client using a module that you did not know was even there.

For more detailed information about the Jetty module system, see this section.

Eclipse Jetty Architecture: $JETTY_BASE

Instead of managing multiple Jetty implementations out of several different distribution locations, it is possible to maintain a separation between the binary installation of the standalone Jetty (known as ${jetty.home}), and the customizations for your specific environment(s) (known as ${jetty.base}). In addition to easy management of multiple server instances, is allows for quick, drop-in upgrades of Jetty. There should always only be one Jetty Home (per version of Jetty), but there can be multiple Jetty Base directories that reference it.

This separation between $JETTY_HOME and $JETTY_BASE allows upgrades without affecting your web applications. $JETTY_HOME contains the Jetty runtime and libraries and the default configuration, while a $JETTY_BASE contains your web applications and any override of the default configuration.

For example, with the $JETTY_HOME installation the default value for the network port for clear-text HTTP is 8080. However, you want that port to be 6060, because you are behind a load balancer that is configured to forward to the backend on port 6060.

Instead, you want to configure the clear-text HTTP port in your $JETTY_BASE. When you upgrade Jetty, you will upgrade only files in $JETTY_HOME, and all the configuration in $JETTY_BASE will remain unchanged.

Installing the Jetty runtime and libraries in $JETTY_HOME also allows you to leverage file system permissions: $JETTY_HOME may be owned by an administrator user (so that only administrators can upgrade it), while $JETTY_BASE directories may be owned by a less privileged user.

If you had changed the default configuration in $JETTY_HOME, when you upgrade Jetty, say from version 10.0.0 to version 10.0.1, your change would be lost. Maintaining all the changes in $JETTY_HOME, and having to reconfigure these with each upgrade results in a massive commitment of time and effort.

To recap:

$JETTY_BASE
  • This is the location for your configurations and customizations to the Jetty distribution.

$JETTY_HOME
  • This is the location for the Jetty distribution binaries, default XML IoC configurations, and default module definitions.

Jetty Home should always be treated as a standard of truth. All configuration modifications, changes and additions should always be made in the appropriate Jetty Base directory.
Eclipse Jetty Architecture: $JETTY_HOME and $JETTY_BASE Configuration Resolution

Potential configuration is resolved from these 2 directory locations. When Jetty starts up in processes configuration from them as follows:

Check Jetty Base First

If the referenced configuration exists, relative to the defined Jetty base, it is used.

Check Jetty Home Second

If the referenced configuration exists, relative to the defined Jetty home, it is used.

Use java.io.File(String pathname) Logic

Lastly, use the reference as a java.io.File(String pathname) reference, following the default resolution rules outlined by that constructor.In brief, the reference will be used as-is, be it relative (to current working directory, aka $\{user.dir}) or absolute path, or even network reference (such as on Windows and use of UNC paths).

Starting Eclipse Jetty

Eclipse Jetty as a standalone server has no graphical user interface, so configuring and running the server is done from the command line.

Recall from the Eclipse Jetty standalone server architecture section that Jetty is based on modules, that provides features, and on $JETTY_BASE, the place where you configure which module (and therefore which feature) you want to enable, and where you configure module parameters.

Jetty is started by executing $JETTY_HOME/start.jar, but first we need to create a $JETTY_BASE:

$ JETTY_BASE=/path/to/jetty.base
$ cd $JETTY_BASE
$ java -jar $JETTY_HOME/start.jar
ERROR : Nothing to start, exiting ...

Usage: java -jar $JETTY_HOME/start.jar [options] [properties] [configs]
       java -jar $JETTY_HOME/start.jar --help  # for more information

The error is normal, since the $JETTY_BASE you just created is empty and therefore there is no configuration to use to assemble the Jetty server.

However, it shows that start.jar takes parameters, whose details can be found in this section.

You can explore what modules are available out of the box via:

$ java -jar $JETTY_HOME/start.jar --list-modules=*

Try to enable the http module (see also this section for additional information):

$ java -jar $JETTY_HOME/start.jar --add-module=http
INFO  : mkdir ${jetty.base}/start.d
INFO  : server          transitively enabled, ini template available with --add-module=server
INFO  : logging-jetty   transitively enabled
INFO  : http            initialized in ${jetty.base}/start.d/http.ini
INFO  : resources       transitively enabled
INFO  : threadpool      transitively enabled, ini template available with --add-module=threadpool
INFO  : logging/slf4j   dynamic dependency of logging-jetty
INFO  : bytebufferpool  transitively enabled, ini template available with --add-module=bytebufferpool
INFO  : mkdir ${jetty.base}/resources
INFO  : copy ${jetty.home}/modules/logging/jetty/resources/jetty-logging.properties to ${jetty.base}/resources/jetty-logging.properties
INFO  : Base directory was modified

Now you can start Jetty:

$ java -jar $JETTY_HOME/start.jar
2020-09-11 15:35:17.451:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-10T11:01:33.608Z; git: b10a14ebf9b200da388f4f9a2036bd8117ee0b11; jvm 11.0.8+10
2020-09-11 15:35:17.485:INFO :oejs.AbstractConnector:main: Started ServerConnector@2d52216b{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2020-09-11 15:35:17.496:INFO :oejs.Server:main: Started Server@44821a96{STARTING}[10.0.0-SNAPSHOT,sto=5000] @553ms

Note how Jetty is listening on port 8080 for clear-text HTTP/1.1 connections.

After having enabled the http module, the $JETTY_BASE directory looks like this:

JETTY_BASE
├── resources
│   └── jetty-logging.properties (1)
└── start.d (2)
    └── http.ini (3)
1 The resources/jetty-logging.properties file has been created because the http modules depends on the server module, which in turn depends on the logging module; the logging module created this file that can be configured to control the server logging level.
2 The start.d/ directory contains the configuration files for the modules.
3 The start.d/http.ini file is the http module configuration file, where you can specify values for the http module properties.

In the http.ini file you can find the following content (among other content):

http.ini
--module=http (1)
# jetty.http.port=8080 (2)
...
1 This line enables the http module and should not be modified.
2 This line is commented out and specifies the default value for the module property jetty.http.port, which is the network port that listens for clear-text HTTP connections.

You can change the module property jetty.http.port value directly from the command line:

$ java -jar $JETTY_HOME/start.jar jetty.http.port=9999

To make this change persistent, you can edit the http.ini file, uncomment the module property jetty.http.port and change its value to 9999:

http.ini
--module=http
jetty.http.port=9999
...

If you restart Jetty, the new value will be used:

$ java -jar $JETTY_HOME/start.jar
2020-09-11 15:35:17.451:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-10T11:01:33.608Z; git: b10a14ebf9b200da388f4f9a2036bd8117ee0b11; jvm 11.0.8+10
2020-09-11 15:35:17.485:INFO :oejs.AbstractConnector:main: Started ServerConnector@2d52216b{HTTP/1.1, (http/1.1)}{0.0.0.0:9999}
2020-09-11 15:35:17.496:INFO :oejs.Server:main: Started Server@44821a96{STARTING}[10.0.0-SNAPSHOT,sto=5000] @553ms

Note how Jetty is now listening on port 9999 for clear-text HTTP/1.1 connections.

If you want to enable support for different protocols such as secure HTTP/1.1 or HTTP/2, or configured Jetty behind a load balancer, read this section.

The Jetty server is now up and running, but it has no web applications deployed, so it just replies with 404 Not Found to every request. It is time to deploy your web applications to Jetty.

For more detailed information about the Jetty start system, you can read the Jetty start system section.

Deploying Web Applications to Eclipse Jetty

For the purpose of deploying web applications to Jetty, there are two types of resources that can be deployed:

  • Standard Web Application Archives, in the form of *.war files or *.war directories, defined by the Servlet specification. Their deployment is described in this section.

  • Jetty context XML files, that allow you to customize the deployment of standard web applications, and also allow you use Jetty components, and possibly custom components written by you, to assemble your web applications. Their deployment is described in this section.

Deploying Standard *.war Web Applications

A standard Servlet web application is packaged in either a *.war file or in a directory with the structure of a *.war file.

Recall that the structure of a *.war file is as follows:

mywebapp.war
├── index.html (1)
└── WEB-INF (2)
    ├── classes/ (3)
    ├── lib/ (4)
    └── web.xml (5)
1 Publicly accessible resources such as *.html, *.jsp, *.css, *.js files, etc. are placed in *.war or in sub-directories of the *.war.
2 WEB-INF is a special directory used to store anything related to the web application that must not be publicly accessible, but may be accessed by other resources.
3 WEB-INF/classes stores the web application compiled *.class files
4 WEB-INF/classes stores the web application *.jar files
5 WEB-INF/web.xml is the web application deployment descriptor defines the components and the configuration of your web application.

To deploy a standard web application, you need to enable the deploy module (see the deploy module complete definition here).

$ java -jar $JETTY_HOME/start.jar --add-module=deploy
INFO  : webapp          transitively enabled, ini template available with --add-module=webapp
INFO  : security        transitively enabled
INFO  : servlet         transitively enabled
INFO  : deploy          initialized in ${jetty.base}/start.d/deploy.ini
INFO  : mkdir ${jetty.base}/webapps
INFO  : Base directory was modified

The deploy module creates the $JETTY_BASE/webapps directory, the directory where *.war files or *.war directories should be copied so that Jetty can deploy them.

The deploy module only provides the feature of deploying web applications.

Whether these web applications are served via clear-text HTTP/1.1, or secure HTTP/1.1, or secure HTTP/2 (or even all of these protocols) depends on whether the correspondent Jetty modules have been enabled. Refer to the section about protocols for further information.

Now you need to copy a web application to the $JETTY_BASE/webapps directory:

curl https://repo1.maven.org/maven2/org/eclipse/jetty/test-jetty-webapp/10.0.0/test-jetty-webapp-10.0.0.war --output $JETTY_BASE/webapps/test.war

The $JETTY_BASE directory is now:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
├── start.d
│   ├── deploy.ini
│   └── http.ini
└── webapps
    └── test.war

Now start Jetty:

$ java -jar $JETTY_HOME/start.jar
2020-09-16 09:53:38.182:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-16T07:47:47.334Z; git: d45455b32d96f516d39e03b53e91502a34b04f37; jvm 15+36-1562
2020-09-16 09:53:38.205:INFO :oejdp.ScanningAppProvider:main: Deployment monitor [file:///tmp/jetty.base/webapps/] at interval 1
2020-09-16 09:53:38.293:WARN :oejshC.test:main: The async-rest webapp is deployed. DO NOT USE IN PRODUCTION!
2020-09-16 09:53:38.298:INFO :oejw.StandardDescriptorProcessor:main: NO JSP Support for /test, did not find org.eclipse.jetty.jsp.JettyJspServlet
2020-09-16 09:53:38.306:INFO :oejss.DefaultSessionIdManager:main: DefaultSessionIdManager workerName=node0
2020-09-16 09:53:38.306:INFO :oejss.DefaultSessionIdManager:main: No SessionScavenger set, using defaults
2020-09-16 09:53:38.307:INFO :oejss.HouseKeeper:main: node0 Scavenging every 660000ms
2020-09-16 09:53:38.331:INFO :oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@45b4c3a9{Async REST Webservice Example,/test,[file:///tmp/jetty-0_0_0_0-8080-test_war-_test-any-15202033063643714058.dir/webapp/, jar:file:///tmp/jetty-0_0_0_0-8080-test_war-_test-any-15202033063643714058.dir/webapp/WEB-INF/lib/example-async-rest-jar-11.0.0-SNAPSHOT.jar!/META-INF/resources],AVAILABLE}{/tmp/jetty.base/webapps/test.war}
2020-09-16 09:53:38.338:INFO :oejs.AbstractConnector:main: Started ServerConnector@543295b0{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2020-09-16 09:53:38.347:INFO :oejs.Server:main: Started Server@5ffead27{STARTING}[10.0.0-SNAPSHOT,sto=5000] @593ms

Now you can access the web application by pointing your browser to http://localhost:8080/test.

If you want to customize the deployment of your web application, for example by specifying a contextPath different from the file/directory name, or by specifying JNDI entries, or by specifying virtual hosts, etc. read this section.

Using start.jar

TODO: review in light of Jetty 10

The most basic way of starting the Jetty standalone server is to execute the start.jar from a base directory containing the configuration of jetty:

[] mkdir /var/jetty-base
[] cd /var/jetty-base
[] java -jar $JETTY_HOME/start.jar
ERROR : No enabled jetty modules found!
...

Jetty is a highly modularized web server container. Very little is mandatory and required, and most components are optional; you enable or disable them according to the needs of your environment. You use start.jar to configure the modules:

[] java -jar $JETTY_HOME/start.jar --add-module=http
...
INFO  : http            initialized in ${jetty.base}/start.d/http.ini
...

[] java -jar $JETTY_HOME/start.jar
2020-12-02 09:31:45.563:INFO :oejs.Server:main: jetty-10.0.0; built: 2020-11-30T14:38:10.953Z; git: cac070fdb134df192252e99653d99d7c3fb91b87; jvm 15.0.1+9
2020-12-02 09:31:45.601:INFO :oejs.AbstractConnector:main: Started ServerConnector@3eb7fc54{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2020-12-02 09:31:45.615:INFO :oejs.Server:main: Started Server@55a561cf{STARTING}[10.0.0,sto=5000] @760ms

At its most basic, you configure Jetty from two elements:

  1. A set of libraries and directories that make up the server classpath.

  2. A set of Jetty XML configuration files (IoC style) that establish how to build the Jetty server and its components.

Instead of editing these directly, Jetty 9.1 introduced more options on how to configure Jetty (these are merely syntactic sugar that eventually resolve into the two basic configuration components).

Jetty Startup Features include:

  • A separation of the Jetty distribution binaries in ${jetty.home} and the environment specific configurations (and binaries) found in ${jetty.base} (detailed in Managing Jetty Base and Jetty Home.)

  • You can enable a set of libraries and XML configuration files via the newly introduced module system.

  • All of the pre-built XML configuration files shipped in Jetty are now parameterized with properties that you can specify in your ${jetty.base}/start.ini (demonstrated in Quick Start Configuration).

These are powerful new features, made to support a variety of styles of configuring Jetty, from a simple property based configuration, to handling multiple installations on a server, to customized stacks of technology on top of Jetty, and even the classic, custom XML configurations of old.

For example, if you use the ${jetty.base} concepts properly, you can upgrade the Jetty distribution without having to remake your entire tree of modifications to Jetty. Simply separate out your specific modifications to the ${jetty.base}, and in the future, just upgrade your ${jetty.home} directory with a new Jetty distribution.

Executing start.jar

When executed start.jar performs the following actions:

  • Loads and parses all INIs found in ${jetty.base}/start.d/*.ini as command line arguments.

  • Loads and parses ${jetty.base}/start.ini as command line arguments.

  • Parses actual command line arguments used to execute start.jar itself.

  • Resolves any XML configuration files, modules, and libraries using base vs. home resolution steps:

    1. Checks whether file exists as relative reference to ${jetty.base}.

    2. Checks whether file exists as relative reference to ${jetty.home}.

    3. Uses default behavior of java.io.File (Relative to System.getProperty ("user.dir") and then as absolute file system path).

  • Loads any dependent modules (merges XXNK, library, and properties results with active command line).

  • Builds out server classpath.

  • Determines run mode as one of:

    • Shows informational command line options and exit.

    • Executes Jetty normally, waits for Jetty to stop.

    • Executes a forked JVM to run Jetty in, waits for forked JVM to exit.

start.jar Command Line Options

Command Line Options
--help

Obtains the current list of command line options and some basic usage help.

--version

Shows the list of server classpath entries, and prints version information found for each entry.

--list-classpath

Similar to --version, shows the server classpath.

--list-config

Lists the resolved configuration that will start Jetty.

  • Java environment

  • Jetty environment

  • JVM arguments

  • Properties

  • Server classpath

  • Server XML configuration files

--dry-run

Print the command line that the start.jar generates, then exit. This may be used to generate command lines when the start.ini includes -X or -D arguments:

$ java -jar start.jar --dry-run > jetty.sh
$ . jetty.sh
--dry-run=<parts>

Print specific parts of the command line. The parts are a comma separated list of:

  • "java" - the JVM to run

  • "opts" - the JVM options (eg -D and -X flags)

  • "path" - the JVM class path or JPMS modules options

  • "main" - the main class to run

  • "args" - the arguments passed to the main class

It is possible to decompose the start command:

$ OPTS=$(java -jar start.jar --dry-run=opts,path)
$ MAIN=$(java -jar start.jar --dry-run=main)
$ ARGS=$(java -jar start.jar --dry-run=args)
$ java $OPTS -Dextra=opt $MAIN $ARGS extra=arg

Alternatively to create an args file for java:

$ java -jar start.jar --dry-run=opts,path,main,args > /tmp/args
$ java @/tmp/args
--exec

Forces the start to use a forked instance of java to run Jetty. Some modules include --exec in order to set java command line options. Some start options, such as --jpms also imply --exec

--exec-properties=<filename>

Assign a fixed name to the file used to transfer properties to the sub process. This allows the generated properties file to be saved and reused. Without this option, a temporary file is used.

--commands=<filename>

Instructs start.jar to use each line of the specified file as arguments on the command line.

Debug and Start Logging
--debug

Enables debugging output of the startup procedure.

Note: This does not set up debug logging for Jetty itself. For information on logging, please see the section on Configuring Jetty Logging.]

--start-log-file=<filename>

Sends all startup output to the filename specified. Filename is relative to ${jetty.base}. This is useful for capturing startup issues where the Jetty-specific logger has not yet kicked in due to a possible startup configuration error.

Module Management
--list-modules

Lists all the modules defined by the system. Looks for module files using the normal ${jetty.base} and ${jetty.home} resolution logic. Also lists enabled state based on information present on the command line, and all active startup INI files.

--list-modules=<tag>(,<tag>)*

List modules by tag. Use '*' for all tags. Prefix a tag with '-' to exclude the tag. The special tag "internal" is always excluded unless it is explicitly included.

--list-all-modules

List all modules.

--module=<name>,(<name>)*

Enables one or more modules by name (use --list-modules to see the list of available modules). This enables all transitive (dependent) modules from the module system as well. If you use this from the shell command line, it is considered a temporary effect, useful for testing out a scenario. If you want this module to always be enabled, add this command to your ${jetty.base}/start.ini.

--add-to-start=<name>,(<name>)*

Enables a module by appending lines to the ${jetty.base}/start.ini file. The lines that are added are provided by the module-defined INI templates. Note: Transitive modules are also appended. If a module contains an .ini template with properties, you can also edit these properties when activating the module. To do this, simply list the property and its value after the -add-to-start command, such as in the following example:

$ java -jar start.jar --add-to-start=http jetty.http.port=8379 jetty.http.host=1.2.3.4

Doing this will uncomment the property in the associated .ini file and set it to the value specified.

--update-ini

Used to update a specified property or properties that exist in an existing .ini file. Jetty scans the command line, ${jetty.base} and ${jetty.home} for .ini files that have the specified property and update it accordingly.

[my-base]$ java -jar /path/to/jetty-home/start.jar --update-ini jetty.http.port=8417
ConfigSource <command-line>
ConfigSource ${jetty.base}
INFO  : http            property updated jetty.http.port=8417
INFO  : http            updated ${jetty.base}/start.d/http.ini
ConfigSource ${jetty.home}
--create-startd

Creates a ${jetty.base}/start.d/ directory. If a ${jetty.base}/start.ini file already exists, it is copied to the ${jetty.base}/start.d directory.

With respect to start.ini and start.d/.ini files, only *one of these methods should be implemented. Mixing a start.ini with module specific ini files in the {$jetty.base}/start.d directory can lead to server issues unless great care is taken. Please see Start.ini vs. Start.d for more information.

--write-module-graph=<filename>

Advanced feature: Creates a graphviz dot file of the module graph as it exists for the active ${jetty.base}.

# generate module.dot
$ java -jar start.jar --module=websocket --write-module-graph=modules.dot

# post process to a PNG file
$ dot -Tpng -o modules.png modules.dot

See graphviz.org for details on how to post-process this dotty file into the output best suited for your needs.

--create-files

Create any missing files that are required by initialized modules. This may download a file from the network if the module provides a URL.

--skip-file-validation=<modulename>(,<modulename)*

Disable the [files] section validation of content in the ${jetty.base} directory for a specific module. Useful for modules that have downloadable content that is being overridden with alternatives in the ${jetty.base}` directory.

This advanced option is for administrators that fully understand the configuration of their ${jetty.base} and are willing to forego some of the safety checks built into the jetty-start mechanism.
--approve-all-licenses

Approve all license questions. Useful for enabling modules from a script that does not require user interaction.

Startup / Shutdown Command Line
--stop

Sends a stop signal to the running Jetty instance.

Note: The server must have been started with various stop properties for this to work.

STOP.PORT=<number>

The port to use to stop the running Jetty server. This is an internal port, opened on localhost, used solely for stopping the running Jetty server. Choose a port that you do not use to serve web traffic.

Required for --stop to function.

STOP.KEY=<alphanumeric>

The passphrase defined to stop the server.

Required for --stop to function.

STOP.WAIT=<number>

The time (in seconds) to wait for confirmation that the running Jetty server has stopped. If not specified, the stopper waits indefinitely for the server to stop.

If the time specified elapses, without a confirmation of server stop, then the --stop command exits with a non-zero return code.

You can configure a port number for Jetty to listen on for a stop command, so you are able to stop it from a different terminal. This requires the use of a "secret" key, to prevent malicious or accidental termination. Use the STOP.PORT and STOP.KEY (or -DSTOP.PORT= and -DSTOP.KEY=, respectively, which will set these as system parameters) parameters as arguments to the start.jar:

> java -jar ${JETTY_HOME}/start.jar STOP.PORT=1234 STOP.KEY=secretpassword

Then, to stop Jetty from a different terminal, you need to supply this port and key information. You can either use a copy of the Jetty distribution, the jetty-maven-plugin, the jetty-ant plugin, or a custom class to accomplish this. Here’s how to use the Jetty distribution, leveraging start.jar, to perform a stop:

> java -jar start.jar STOP.PORT=8181 STOP.KEY=abc123 --stop
To perform a graceful shutdown of Jetty, the stats module must be enabled.
Advanced Commands
--lib=<classpath>

Add arbitrary classpath entries to the the server classpath.

--include-jetty-dir=<path>

Include an extra Jetty directory to use as a source for configuration details. This directory behaves similarly to ${jetty.base} but sits at a layer between ${jetty.base} and ${jetty.home}. This allows for some complex hierarchies of configuration details.

--download=<http-uri>|<location>

If the file does not exist at the given location, download it from the given http URI. Note: location is always relative to ${jetty.base}. You might need to escape the slash "\|" to use this on some environments.

maven.repo.uri=[url]

The url to use to download Maven dependencies. Default is https://repo1.maven.org/maven2/.

Shaded Start.jar

If you have a need for a shaded version of start.jar (such as for Gradle), you can achieve this via a Maven dependency.

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-start</artifactId>
    <version>{VERSION}</version>
    <classifier>shaded</classifier>
</dependency>

Start.jar without exec or forking.

Some Jetty modules include the --exec option so that java command line options can be set. Also some start.jar options (eg. --jpms) include an implicit --exec. To start jetty without forking a new JVM instance from the start JVM, the --dry-run option can be used to generate a command line:

$ CMD=$(java -jar start.jar --dry-run)
$ $CMD

It is possible to decompose the start command so that it can be modified:

$ OPTS=$(java -jar start.jar --dry-run=opts,path)
$ MAIN=$(java -jar start.jar --dry-run=main)
$ ARGS=$(java -jar start.jar --dry-run=args)
$ java $OPTS -Dextra=opt $MAIN $ARGS extra=arg

Alternatively to create an args file for java:

$ java -jar start.jar --dry-run=opts,path,main,args > /tmp/args
$ java @/tmp/args

Customizing Web Application Deployment

Most of the times you want to be able to customize the deployment of your web applications, for example by changing the contextPath, or by adding JNDI entries, or by configuring virtual hosts, etc.

The customization is performed by the deploy module by processing Jetty context XML files.

The deploy module contains the DeploymentManager component that scans the $JETTY_BASE/webapps directory for changes, following the deployment rules described in this section.

Hot vs Static Deployment

The DeploymentManager scans the $JETTY_BASE/webapps directory for changes every N seconds, where N is configured via the jetty.deploy.scanInterval property.

By default, the scan interval is 1 second, which means that hot deployment is enabled: if a file is added/changed/removed from the $JETTY_BASE/webapps directory, the DeploymentManager will notice the change and respectively deploy/redeploy/undeploy the web application.

Setting the scan interval to 0 means that static deployment is enabled, and the DeploymentManager will not scan the $JETTY_BASE/webapps directory for changes. This means that to deploy/redeploy/undeploy a web application you will need to stop and restart Jetty.

The following command line disables hot deployment by specifying the jetty.deploy.scanInterval property on the command line, and therefore only for this particular run:

$ java -jar $JETTY_HOME/start.jar jetty.deploy.scanInterval=0

To make static deployment persistent, you need to edit the deploy module configuration file, $JETTY_BASE/start.d/deploy.ini, uncomment the module property jetty.deploy.scanInterval and change its value to 0:

deploy.ini
--module=deploy
jetty.deploy.scanInterval=0
...

Deployment Rules

Adding a *.war file, a *.war directory, a Jetty context XML file or a normal directory to $JETTY_BASE/webapps causes the DeploymentManager to deploy the new web application.

Updating a *.war file or a Jetty context XML file causes the DeploymentManager to redeploy the web application, which means that the Jetty context component representing the web application is stopped, then reconfigured, and then restarted.

Removing a *.war file, a *.war directory, a Jetty context XML file or a normal directory from $JETTY_BASE/webapps causes the DeploymentManager to undeploy the web application, which means that the Jetty context component representing the web application is stopped and removed from the Jetty server.

When a file or directory is added to $JETTY_BASE/webapps, the DeploymentManager derives the web application contextPath from the file or directory name, with the following rules:

  • If the directory name is, for example, mywebapp/, it is deployed as a standard web application if it contains a WEB-INF/ subdirectory, otherwise it is deployed as a web application of static content. The contextPath would be /mywebapp (that is, the web application is reachable at http://localhost:8080/mywebapp/).

  • If the directory name is ROOT, case insensitive, the contextPath is / (that is, the web application is reachable at http://localhost:8080/).

  • If the directory name ends with .d, for example config.d/, it is ignored, although it may be referenced to configure other web applications (for example to store common files).

  • If the *.war file name is, for example, mywebapp.war, it is deployed as a standard web application with the context path /mywebapp (that is, the web application is reachable at http://localhost:8080/mywebapp/).

  • If the file name is ROOT.war, case insensitive, the contextPath is / (that is, the web application is reachable at http://localhost:8080/).

  • If both the mywebapp.war file and the mywebapp/ directory exist, only the file is deployed. This allows the directory with the same name to be the *.war file unpack location and avoid that the web application is deployed twice.

  • A Jetty context XML file named mywebapp.xml is deployed as a web application by processing the directives contained in the XML file itself, which must set the contextPath.

  • If both mywebapp.xml and mywebapp.war exist, only the XML file is deployed. This allows the XML file to reference the *.war file and avoid that the web application is deployed twice.

Deploying Jetty Context XML Files

A Jetty context XML file is a Jetty XML file that allows you to customize the deployment of web applications.

Recall that the DeploymentManager component of the Jetty deploy module gives priority to Jetty context XML files over *.war files or directories.

To deploy a web application using a Jetty context XML file, simply place the file in the $JETTY_BASE/webapps directory.

A simple Jetty context XML file, for example named wiki.xml is the following:

wiki.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
  <Set name="contextPath">/wiki</Set> (2)
  <Set name="war">/opt/myapps/myapp.war</Set> (3)
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies the web application contextPath, which may be different from the *.war file name.
3 Specifies the file system path of the *.war file.

The $JETTY_BASE directory would look like this:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
├── start.d
│   ├── deploy.ini
│   └── http.ini
└── webapps
    └── wiki.xml
The *.war file may be placed anywhere in the file system and does not need to be placed in the $JETTY_BASE/webapps directory.
If you place both the Jetty context XML file and the *.war file in the $JETTY_BASE/webapps directory, remember that they must have the same file name, for example wiki.xml and wiki.war, so that the DeploymentManager deploys the web application only once using the Jetty context XML file (and not the *.war file).

You can use the features of Jetty XML files to avoid to hard-code file system paths or other configurations in your Jetty context XML files, for example by using system properties:

wiki.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/wiki</Set>
  <Set name="war"><SystemProperty name="myapps.dir"/>/myapp.war</Set>
</Configure>

Note how the *.war file path is now obtained by resolving the system property myapps.dir that you can specify on the command line when you start Jetty:

$ java -jar $JETTY_HOME/start.jar -Dmyapps.dir=/opt/myapps

Configuring JNDI Entries

A web application may reference a JNDI entry, such as a JDBC DataSource from the web application web.xml file. The JNDI entry must be defined in a Jetty XML file, for example a context XML like so:

mywebapp.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="wac" class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/mywebapp</Set>
  <Set name="war">/opt/webapps/mywebapp.war</Set>
  <New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg><Ref refid="wac"/></Arg>
    <Arg>jdbc/myds</Arg>
     <Arg>
        <New class="com.mysql.cj.jdbc.MysqlConnectionPoolDataSource">
           <Set name="url">jdbc:mysql://localhost:3306/databasename</Set>
           <Set name="user">user</Set>
           <Set name="password">password</Set>
        </New>
     </Arg>
  </New>
</Configure>

For more information and examples on how to use JNDI in Jetty, refer to the JNDI feature section.

Class com.mysql.cj.jdbc.MysqlConnectionPoolDataSource is present in the MySQL JDBC driver file, mysql-connector-java-<version>.jar, which must be available on the server’s classpath .

If the class is instead present within the web application, then the JNDI entry must be declared in a WEB-INF/jetty-env.xml file - see the JNDI feature section for more information and examples.

Configuring Virtual Hosts

A virtual host is an internet domain name, registered in the Domain Name Server (DNS), for an IP address such that multiple virtual hosts will resolve to the same IP address of a single server instance.

If you have multiple web applications deployed on the same Jetty server, by using virtual hosts you will be able to target a specific web application.

For example, you may have a web application for your business and a web application for your hobbies , both deployed in the same Jetty server. By using virtual hosts, you will be able to have the first web application available at http://domain.biz/, and the second web application available at http://hobby.net/.

Another typical case is when you want to use different subdomains for different web application, for example a project website is at http://project.org/ and the project documentation is at http://docs.project.org.

Virtual hosts can be used with any context that is a subclass of ContextHandler.

Virtual Host Names

Jetty supports the following variants to be specified as virtual host names:

www.hostname.com

A fully qualified domain name. It is important to list all variants as a site may receive traffic for both www.hostname.com and hostname.com.

*.hostname.com

A wildcard domain name which will match only one level of arbitrary subdomains. *.foo.com will match www.foo.com and m.foo.com, but not www.other.foo.com.

10.0.0.2

An IP address may be set as a virtual host to indicate that a web application should handle requests received on the network interface with that IP address for protocols that do not indicate a host name such as HTTP/0.9 or HTTP/1.0.

@ConnectorName

A Jetty ServerConnector name to indicate that a web application should handle requests received on the ServerConnector with that name, and therefore received on a specific IP port. A ServerConnector name can be set via http://www.eclipse.org/jetty/javadoc/11.0.0/org/eclipse/jetty/server/AbstractConnector.html#setName(java.lang.String).

www.√integral.com

Non-ASCII and IDN domain names can be set as virtual hosts using Puny Code equivalents that may be obtained from a Punycode/IDN converters. For example if the non-ASCII domain name www.√integral.com is given to a browser, then the browser will make a request that uses the domain name www.xn—​integral-7g7d.com, which is the name that should be added as the virtual host name.

Virtual Hosts Configuration

If you have a web application mywebapp.war you can configure its virtual hosts in this way:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/mywebapp</Set>
  <Set name="war">/opt/webapps/mywebapp.war</Set>
  <Set name="virtualHosts">
    <Array type="java.lang.String">
      <Item>mywebapp.com</Item>
      <Item>www.mywebapp.com</Item>
      <Item>mywebapp.net</Item>
      <Item>www.mywebapp.net</Item>
    </Array>
  </Set>
</Configure>

Your web application will be available at:

  • http://mywebapp.com/mywebapp

  • http://www.mywebapp.com/mywebapp

  • http://mywebapp.net/mywebapp

  • http://www.mywebapp.net/mywebapp

You configured the contextPath of your web application to /mywebapp.

As such, a request to http://mywebapp.com/other will not match your web application because the contextPath does not match.

Likewise, a request to http://other.com/mywebapp will not match your web application because the virtual host does not match.

Same Context Path, Different Virtual Hosts

If you want to deploy different web applications to the same context path, typically the root context path /, you must use virtual hosts to differentiate among web applications.

You have domain.war that you want to deploy at http://domain.biz/ and hobby.war that you want to deploy at http://hobby.net.

To achieve this, you simply use the same context path of / for each of your webapps, while specifying different virtual hosts for each of your webapps:

domain.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/</Set>
  <Set name="war">/opt/webapps/domain.war</Set>
  <Set name="virtualHosts">
    <Array type="java.lang.String">
      <Item>domain.biz</Item>
    </Array>
  </Set>
</Configure>
hobby.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/</Set>
  <Set name="war">/opt/webapps/hobby.war</Set>
  <Set name="virtualHosts">
    <Array type="java.lang.String">
      <Item>hobby.net</Item>
    </Array>
  </Set>
</Configure>
Different Port, Different Web Application

Sometimes it is required to serve different web applications from different IP ports, and therefore from different ServerConnectors.

For example, you want requests to http://localhost:8080/ to be served by one web application, but requests to http://localhost:9090/ to be served by another web application.

This configuration may be useful when Jetty sits behind a load balancer.

In this case, you want to configure multiple connectors, each with a different name, and then reference the connector name in the web application virtual host configuration:

domain.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/</Set>
  <Set name="war">/opt/webapps/domain.war</Set>
  <Set name="virtualHosts">
    <Array type="java.lang.String">
      <Item>@port8080</Item>
    </Array>
  </Set>
</Configure>
hobby.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/</Set>
  <Set name="war">/opt/webapps/hobby.war</Set>
  <Set name="virtualHosts">
    <Array type="java.lang.String">
      <Item>@port9090</Item>
    </Array>
  </Set>
</Configure>

Web application domain.war has a virtual host of @port8080, where port8080 is the name of a Jetty connector.

Likewise, web application hobby.war has a virtual host of @port9090, where port9090 is the name of another Jetty connector.

See this section for further information about how to configure connectors.

Configuring *.war File Extraction

By default, *.war files are uncompressed and its content extracted in a temporary directory. The web application resources are served by Jetty from the files extracted in the temporary directory, not from the files within the *.war file, for performance reasons.

If you do not want Jetty to extract the *.war files, you can disable this feature, for example:

mywebapp.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/mywebapp</Set>
  <Set name="war">/opt/webapps/mywebapp.war</Set>
  <Set name="extractWAR">false</Set>
</Configure>

Overriding web.xml

You can configure an additional web.xml that complements the web.xml file that is present in the web application *.war file. This additional web.xml is processed after the *.war file web.xml. This allows you to add host specific configuration or server specific configuration without having to extract the web application web.xml, modify it, and repackage it in the *.war file.

mywebapp.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/mywebapp</Set>
  <Set name="war">/opt/webapps/mywebapp.war</Set>
  <Set name="overrideDescriptor">/opt/webapps/mywebapp-web.xml</Set>
</Configure>

The format of the additional web.xml is exactly the same as a standard web.xml file, for example:

mywebapp-web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd"
         version="4.0">
  <servlet>
    <servlet-name>my-servlet</servlet-name>
    <init-param>
      <param-name>host</param-name>
      <param-value>192.168.0.13</param-value>
    </init-param>
  </servlet>
</web-app>

In the example above, you configured the my-servlet Servlet (defined in the web application web.xml), adding a host specific init-param with the IP address of the host.

Configuring Eclipse Jetty Connectors and Protocols

Connectors are the network components through which Jetty accepts incoming network connections.

Each connector listens on a network port and can be configured with ConnectionFactory components that understand one or more network protocols.

Understanding a protocol means that the connector is able to interpret incoming network bytes (for example, the bytes that represent an HTTP/1.1 request) and convert them into more abstract objects (for example an HttpServletRequest object) that are then processed by applications. Conversely, an abstract object (for example an HttpServletResponse) is converted into the correspondent outgoing network bytes (the bytes that represent an HTTP/1.1 response).

Like other Jetty components, connectors are enabled and configured by enabling and configuring the correspondent Jetty module.

Recall that you must always issue the commands to enable Jetty modules from within the $JETTY_BASE directory, and that the Jetty module configuration files are in the $JETTY_BASE/start.d/ directory.

You can obtain the list of connector-related modules in this way:

$ java -jar $JETTY_HOME/start.jar --list-modules=connector

Configuring Clear-Text HTTP/1.1

Clear text HTTP/1.1 is enabled with the http Jetty module with the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-module=http
INFO  : mkdir ${jetty.base}/start.d
INFO  : server          transitively enabled, ini template available with --add-module=server
INFO  : logging-jetty   transitively enabled
INFO  : http            initialized in ${jetty.base}/start.d/http.ini
INFO  : resources       transitively enabled
INFO  : threadpool      transitively enabled, ini template available with --add-module=threadpool
INFO  : logging/slf4j   dynamic dependency of logging-jetty
INFO  : bytebufferpool  transitively enabled, ini template available with --add-module=bytebufferpool
INFO  : mkdir ${jetty.base}/resources
INFO  : copy ${jetty.home}/modules/logging/jetty/resources/jetty-logging.properties to ${jetty.base}/resources/jetty-logging.properties
INFO  : Base directory was modified

After having enabled the http module, the $JETTY_BASE directory looks like this:

JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    └── http.ini

The http.ini file is the file that you want to edit to configure network and protocol parameters — for more details see this section.

Note that the http Jetty module depends on the server Jetty module.

Some parameters that you may want to configure are in fact common HTTP parameters that are applied not only for clear-text HTTP/1.1, but also for secure HTTP/1.1 or for clear-text HTTP/2 or for encrypted HTTP/2, and these configuration parameters may be present in the server module configuration file.

You can force the creation of the server.ini file via:

$ java -jar $JETTY_HOME/start.jar --add-module=server

Now the $JETTY_BASE directory looks like this:

JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    └── server.ini

Now you can edit the server.ini file — for more details see this section.

Configuring Secure HTTP/1.1

Secure HTTP/1.1 is enabled with both the ssl and https Jetty modules with the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-modules=ssl,https
INFO  : mkdir ${jetty.base}/start.d
INFO  : server          transitively enabled, ini template available with --add-module=server
INFO  : logging-jetty   transitively enabled
INFO  : resources       transitively enabled
INFO  : https           initialized in ${jetty.base}/start.d/https.ini
INFO  : ssl             initialized in ${jetty.base}/start.d/ssl.ini
INFO  : threadpool      transitively enabled, ini template available with --add-module=threadpool
INFO  : logging/slf4j   transitive provider of logging/slf4j for logging-jetty
INFO  : logging/slf4j   dynamic dependency of logging-jetty
INFO  : bytebufferpool  transitively enabled, ini template available with --add-module=bytebufferpool
INFO  : mkdir ${jetty.base}/resources
INFO  : copy ${jetty.home}/modules/logging/jetty/resources/jetty-logging.properties to ${jetty.base}/resources/jetty-logging.properties
INFO  : Base directory was modified

The command above enables the ssl module, that provides the secure network connector, the KeyStore configuration and TLS configuration — for more details see this section. Then, the https module adds HTTP/1.1 as the protocol secured by TLS.

The $JETTY_BASE directory looks like this:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── https.ini
    └── ssl.ini

Note that the KeyStore file is missing, because you have to provide one with the cryptographic material you want (read this section to create your own KeyStore). You need to configure these two properties by editing ssl.ini:

  • jetty.sslContext.keyStorePath

  • jetty.sslContext.keyStorePassword

As a quick example, you can enable the test-keystore module, that provides a KeyStore containing a self-signed certificate:

$ java -jar $JETTY_HOME/start.jar --add-modules=test-keystore
INFO  : test-keystore   initialized in ${jetty.base}/start.d/test-keystore.ini
INFO  : mkdir ${jetty.base}/etc
INFO  : copy ${jetty.home}/modules/test-keystore/test-keystore.p12 to ${jetty.base}/etc/test-keystore.p12
INFO  : Base directory was modified

The $JETTY_BASE directory is now:

├── etc
│   └── test-keystore.p12
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── https.ini
    ├── ssl.ini
    └── test-keystore.ini

Starting Jetty yields:

$ java -jar $JETTY_HOME/start.jar
2020-09-22 08:40:49.482:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-21T14:44:05.094Z; git: 5c33f526e5b7426dd9644ece61b10184841bb8fa; jvm 15+36-1562
2020-09-22 08:40:49.709:INFO :oejus.SslContextFactory:main: x509=X509@14cd1699(mykey,h=[localhost],w=[]) for Server@73a1e9a9[provider=null,keyStore=file:///tmp/jetty.base/etc/test-keystore.p12,trustStore=file:///tmp/jetty.base/etc/test-keystore.p12]
2020-09-22 08:40:49.816:INFO :oejs.AbstractConnector:main: Started ServerConnector@2e1d27ba{SSL, (ssl, http/1.1)}{0.0.0.0:8443}
2020-09-22 08:40:49.828:INFO :oejs.Server:main: Started Server@2f177a4b{STARTING}[10.0.0-SNAPSHOT,sto=5000] @814ms

Note how Jetty is listening on port 8443 for the secure HTTP/1.1 protocol.

If you point your browser at https://localhost:8443/ you will get a warning from the browser about a "potential security risk ahead", or that "your connection is not private", or similar message depending on the browser.

This is normal because the certificate contained in test-keystore.p12 is self-signed — and as such not signed by a recognized certificate authority — and therefore browsers do not trust it.

Configuring HTTP/2

HTTP/2 is the successor of the HTTP/1.1 protocol, but it is quite different from HTTP/1.1: where HTTP/1.1 is a duplex, text-based protocol, HTTP/2 is a multiplex, binary protocol.

Because of these fundamental differences, a client and a server need to negotiate what version of the HTTP protocol they speak, based on what versions each side supports.

To ensure maximum compatibility, and reduce the possibility that an intermediary that only understands HTTP/1.1 will close the connection when receiving unrecognized HTTP/2 bytes, HTTP/2 is typically deployed over secure connections, using the TLS protocol to wrap HTTP/2.

Browsers only support secure HTTP/2.

The protocol negotiation is performed by the ALPN TLS extension: the client advertises the list of protocols it can speak, and the server communicates to the client the protocol chosen by the server.

For example, you can have a client that only supports HTTP/1.1 and a server that supports both HTTP/1.1 and HTTP/2:

Diagram

Nowadays, it’s common that both clients and servers support HTTP/2, so servers prefer HTTP/2 as the protocol to speak:

Diagram

When you configure a connector with the HTTP/2 protocol, you typically want to also configure the HTTP/1.1 protocol. The reason to configure both protocols is that you typically do not control the clients: for example an old browser that does not support HTTP/2, or a monitoring console that performs requests using HTTP/1.1, or a heartbeat service that performs a single HTTP/1.0 request to verify that the server is alive.

Secure vs Clear-Text HTTP/2

Deciding whether you want to configure Jetty with secure HTTP/2 or clear-text HTTP/2 depends on your use case.

You want to configure secure HTTP/2 when Jetty is exposed directly to browsers, because browsers only support secure HTTP/2.

Diagram

You may configure clear-text HTTP/2 (mostly for performance reasons) if you offload TLS at a load balancer (for example, HAProxy) or at a reverse proxy (for example, nginx).

Diagram

You may configure clear-text HTTP/2 (mostly for performance reasons) to call microservices deployed to different Jetty servers (although you may want to use secure HTTP/2 for confidentiality reasons).

Diagram

Configuring Secure HTTP/2

When you enable secure HTTP/2 you typically want to enable also secure HTTP/1.1, for backwards compatibility reasons: in this way, old browsers or other clients that do not support HTTP/2 will be able to connect to your server.

You need to enable:

  • the ssl Jetty module, which provides the secure connector and the KeyStore and TLS configuration

  • the http2 Jetty module, which adds ALPN handling and adds the HTTP/2 protocol to the secured connector

  • optionally, the https Jetty module, which adds the HTTP/1.1 protocol to the secured connector

Use the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-modules=ssl,http2,https

As when enabling the https Jetty module, you need a valid KeyStore (read this section to create your own KeyStore).

As a quick example, you can enable the test-keystore module, that provides a KeyStore containing a self-signed certificate:

$ java -jar $JETTY_HOME/start.jar --add-modules=test-keystore

Starting Jetty yields:

$ java -jar $JETTY_HOME/start.jar
2020-09-29 19:00:47.316:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-29T13:28:40.441Z; git: 9c0082610528a846b366ae26f4c74894579a8e48; jvm 15+36-1562
2020-09-29 19:00:47.528:INFO :oejus.SslContextFactory:main: x509=X509@7770f470(mykey,h=[localhost],w=[]) for Server@24313fcc[provider=null,keyStore=file:///tmp/jetty.base/etc/test-keystore.p12,trustStore=file:///tmp/jetty.base/etc/test-keystore.p12]
2020-09-29 19:00:47.621:INFO :oejs.AbstractConnector:main: Started ServerConnector@73700b80{SSL, (ssl, alpn, h2, http/1.1)}{0.0.0.0:8443}
2020-09-29 19:00:47.630:INFO :oejs.Server:main: Started Server@30ee2816{STARTING}[10.0.0-SNAPSHOT,sto=5000] @746ms

Note how Jetty is listening on port 8443 and the protocols supported are the sequence (ssl, alpn, h2, http/1.1).

The (ordered) list of protocols after alpn are the application protocols, in the example above (h2, http/1.1).

When a new connection is accepted by the connector, Jetty first interprets the TLS bytes, then it handles the ALPN negotiation knowing that the application protocols are (in order) h2 and then http/1.1.

You can customize the list of application protocols and the default protocol to use in case the ALPN negotiation fails by editing the alpn module properties.

The HTTP/2 protocol parameters can be configured by editing the http2 module properties.

Configuring Clear-Text HTTP/2

When you enable clear-text HTTP/2 you typically want to enable also clear-text HTTP/1.1, for backwards compatibility reasons and to allow clients to upgrade from HTTP/1.1 to HTTP/2.

You need to enable:

  • the http Jetty module, which provides the clear-text connector and adds the HTTP/1.1 protocol to the clear-text connector

  • the http2c Jetty module, which adds the HTTP/2 protocol to the clear-text connector

$ java -jar $JETTY_HOME/start.jar --add-modules=http,http2c

Starting Jetty yields:

$ java -jar $JETTY_HOME/start.jar
2020-09-30 09:18:36.322:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-09-29T22:40:09.015Z; git: ba5f91fe00a68804a586b7dd4e2520c8c948dcc8; jvm 15+36-1562
2020-09-30 09:18:36.349:INFO :oejs.AbstractConnector:main: Started ServerConnector@636be97c{HTTP/1.1, (http/1.1, h2c)}{0.0.0.0:8080}
2020-09-30 09:18:36.361:INFO :oejs.Server:main: Started Server@3c72f59f{STARTING}[10.0.0-SNAPSHOT,sto=5000] @526ms

Note how Jetty is listening on port 8080 and the protocols supported are HTTP/1.1 and h2c (i.e. clear-text HTTP/2).

With this configuration, browsers and client applications will be able to connect to port 8080 using:

  • HTTP/1.1 directly (e.g. curl --http1.1 http://localhost:8080):

GET / HTTP/1.1
Host: localhost:8080
  • HTTP/1.1 with upgrade to HTTP/2 (e.g. curl --http2 http://localhost:8080):

GET / HTTP/1.1
Host: localhost:8080
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings:
  • HTTP/2 directly (e.g. curl --http2-prior-knowledge http://localhost:8080):

50 52 49 20 2a 20 48 54 54 50 2f 32 2e 30 0d 0a
0d 0a 53 4d 0d 0a 0d 0a 00 00 12 04 00 00 00 00
00 00 03 00 00 00 64 00 04 40 00 00 00 00 02 00
00 00 00 00 00 1e 01 05 00 00 00 01 82 84 86 41
8a a0 e4 1d 13 9d 09 b8 f0 1e 07 7a 88 25 b6 50
c3 ab b8 f2 e0 53 03 2a 2f 2a

The HTTP/2 protocol parameters can be configured by editing the http2c module properties.

Configuring Secure Protocols

Secure protocols are normal protocols such as HTTP/1.1, HTTP/2 or WebSocket that are wrapped by the TLS protocol. Any network protocol can be wrapped with TLS.

The https scheme used in URIs really means tls+http/1.1 (or tls+http/2) and similarly the wss scheme used in URIs really means tls+websocket, etc. Senders wrap the underlying protocol bytes (e.g. HTTP bytes or WebSocket bytes) with the TLS protocol, while receivers first interpret the TLS protocol to obtain the underlying protocol bytes, and then interpret the wrapped bytes.

The ssl Jetty module allows you to configure a secure network connector; if other modules require encryption, they declare a dependency on the ssl module.

It is the job of other Jetty modules to configure the wrapped protocol. For example, it is the https module that configures the wrapped protocol to be HTTP/1.1. Similarly, it is the http2 module that configures the wrapped protocol to be HTTP/2. If you enable both the https and the http2 module, you will have a single secure connector that will be able to interpret both HTTP/1.1 and HTTP/2.

Recall from the section about modules, that only modules that are explicitly enabled get their module configuration file (*.ini) saved in $JETTY_BASE/start.d/, and you want $JETTY_BASE/start.d/ssl.ini to be present so that you can configure the connector properties, the KeyStore properties and the TLS properties.
Customizing KeyStore and TLS Configuration

Secure protocols have a slightly more complicated configuration since they require to configure a KeyStore. Refer to the KeyStore section for more information about how to create and manage a KeyStore.

For simple cases, you only need to configure the KeyStore path and KeyStore password as explained in this section.

For more advanced configuration you may want to configure the TLS protocol versions, or the ciphers to include/exclude, etc. The correct way of doing this is to create a custom Jetty XML file and reference it in $JETTY_BASE/start.d/ssl.ini:

ssl.ini
jetty.sslContext.keyStorePassword=my_passwd! (1)
etc/tls-config.xml (2)
1 Configures the jetty.sslContext.keyStorePassword property with the KeyStore password.
2 References your newly created $JETTY_BASE/etc/tls-config.xml.

The ssl.ini file above only shows the lines that are not commented out (you can leave the lines that are commented unmodified for future reference).

You want to create the $JETTY_BASE/etc/tls-config.xml with the following template content:

tls-config.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Ref refid="sslContextFactory">
    ... (1)
  </Ref>
</Configure>
1 Here goes your advanced configuration.

The tls-config.xml file references the sslContextFactory component (created by the ssl Jetty module) that configures the KeyStore and TLS parameters, so that you can now call its APIs via XML, and you will have full flexibility for any advanced configuration you want (see below for few examples).

Refer to the SslContextFactory javadocs for the list of methods that you can call through the Jetty XML file.

Use module properties whenever possible, and only resort to use a Jetty XML file for advanced configuration that you cannot do using module properties.
Customizing TLS Protocol Versions

By default, the SSL protocols (SSL, SSLv2, SSLv3, etc.) are already excluded because they are vulnerable. To explicitly add the exclusion of TLSv1.0 and TLSv1.1 (that are also vulnerable — which leaves only TLSv1.2 and TLSv1.3 available), you want to use this XML:

tls-config.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Ref refid="sslContextFactory">
    <Call name="addExcludeProtocols">
      <Arg>
        <Array type="String">
          <Item>TLSv1.0</Item>
          <Item>TLSv1.1</Item>
        </Array>
      </Arg>
    </Call>
  </Ref>
</Configure>
Customizing TLS Ciphers

You can precisely set the list of excluded ciphers, completely overriding Jetty’s default, with this XML:

tls-config.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Ref refid="sslContextFactory">
    <Set name="ExcludeCipherSuites">
      <Array type="String">
        <Item>^TLS_RSA_.*$</Item>
        <Item>^.*_RSA_.*_(MD5|SHA|SHA1)$</Item>
        <Item>^.*_DHE_RSA_.*$</Item>
        <Item>SSL_RSA_WITH_DES_CBC_SHA</Item>
        <Item>SSL_DHE_RSA_WITH_DES_CBC_SHA</Item>
        <Item>SSL_DHE_DSS_WITH_DES_CBC_SHA</Item>
        <Item>SSL_RSA_EXPORT_WITH_RC4_40_MD5</Item>
        <Item>SSL_RSA_EXPORT_WITH_DES40_CBC_SHA</Item>
        <Item>SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA</Item>
        <Item>SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA</Item>
      </Array>
    </Set>
  </Ref>
</Configure>

Note how each array item specifies a regular expression that matches multiple ciphers, or specifies a precise cipher to exclude.

You can choose to create multiple XML files, and reference them all from $JETTY_BASE/start.d/ssl.ini, or put all your custom configurations in a single XML file.

Renewing the Certificates

When you create a certificate, you must specify for how many days it is valid.

The typical validity is 90 days, and while this period may seem short, it has two advantages:

  • Reduces the risk in case of compromised/stolen keys.

  • Encourages automation, i.e. certificate renewal performed by automated tools (rather than manually) at scheduled times.

To renew a certificate, you must go through the same steps you followed to create the certificate the first time, and then you can reload the KeyStore without the need to stop Jetty.

Watching and Reloading the KeyStore

Jetty can be configured to monitor the directory of the KeyStore file, and reload the SslContextFactory component if the KeyStore file changed.

This feature can be enabled by activating the ssl-reload Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=ssl-reload

For more information about the configuration of the ssl-reload Jetty module, see this section.

Using Conscrypt as TLS Provider

By default, the standard TLS provider that comes with the JDK is used.

The standard TLS provider from OpenJDK is implemented in Java (no native code), and its performance is not optimal, both in CPU usage and memory usage.

A faster alternative, implemented natively, is Google’s Conscrypt, which is built on BoringSSL, which is Google’s fork of OpenSSL.

As Conscrypt eventually binds to a native library, there is a higher risk that a bug in Conscrypt or in the native library causes a JVM crash, while the Java implementation will not cause a JVM crash.

To use Conscrypt as the TLS provider just enable the conscrypt Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=conscrypt
Configuring SNI

Server Name Indication (SNI) is a TLS extension that clients send to indicate what domain they want to connect to during the initial TLS handshake.

Modern TLS clients (e.g. browsers) always send the SNI extension; however, older TLS clients may not send the SNI extension.

Being able to handle the SNI is important when you have virtual hosts and a KeyStore with multiple certificates, one for each domain.

For example, you have deployed over a secure connector two web applications, both at context path /, one at virtual host one.com and one at virtual host two.net. The KeyStore contains two certificates, one for one.com and one for two.net.

There are three ssl module properties that control the SNI behavior on the server: one that works at the TLS level, and two that works at the HTTP level.

The property that works at the TLS level is:

jetty.sslContext.sniRequired

Whether SNI is required at the TLS level, defaults to false.

Its behavior is explained by the following table:

Table 1. Behavior of the jetty.sslContext.sniRequired property

sniRequired=false

sniRequired=true

SNI = null

client receives default certificate

client receives TLS failure

SNI = wrong.org

client receives default certificate

client receives TLS failure

SNI = one.com

client receives one.com certificate

client receives one.com certificate

The default certificate is the certificate returned by the TLS implementation in case there is no SNI match, and you should not rely on this certificate to be the same across Java vendors and versions, or Jetty versions, or TLS provider vendors and versions.

In the example above it could be either the one.com certificate or the two.net certificate.

When jetty.sslContext.sniRequired=true, clients that don’t send a valid SNI receive a TLS failure, and their attempt to connect to the server fails. The details of this failure may not be reported and could be difficult to figure out that the failure is related to an invalid SNI.

For this reason, other two properties are defined at the HTTP level, so that clients can received an HTTP 400 response with more details about what went wrong while trying to connect to the server:

jetty.ssl.sniRequired

Whether SNI is required at the HTTP level, defaults to false.

Its behavior is similar to the jetty.sslContext.sniRequired property above, and is explained by the following table:

Table 2. Behavior of the jetty.ssl.sniRequired property

sniRequired=false

sniRequired=true

SNI = null

Accept

Reject: 400 Bad Request

SNI = wrong.org

Accept

Reject: 400 Bad Request

SNI = one.com

Accept

Accept

When jetty.ssl.sniRequired=true, the SNI is matched against the certificate sent to the client, and only if there is a match the request is accepted.

When the request is accepted, there could be an additional check controlled by the following property:

jetty.ssl.sniHostCheck

Whether the certificate sent to the client matches the Host header, defaults to true.

Its behavior is explained by the following table:

Table 3. Behavior of the jetty.ssl.sniHostCheck property

sniHostCheck=false

sniHostCheck=true

certificate = one.com
Host: wrong.org

Accept

Reject: 400 Bad Request

certificate = one.com
Host: one.com

Accept

Accept

In the normal case with the default server configuration, for a TLS clients that sends SNI, and then sends an HTTP request with the correct Host header, Jetty will pick the correct certificate from the KeyStore based on the SNI received from the client, and accept the request.

Accepting the request does not mean that the request is responded with an HTTP 200 OK, but just that the request passed successfully the SNI checks and will be processed by the server. If the request URI is for a resource that does not exist, the response will likely be a 404 Not Found.

You may modify the default values of the SNI properties if you want stricter control over old/broken TLS clients or bad HTTP requests.

Configuring Eclipse Jetty Behind a Load Balancer or Reverse Proxy

You may need to configure one or more Jetty instances behind an intermediary, typically a load balancer such as HAProxy, or a reverse proxy such as Apache HTTP Server or Nginx.

Diagram

HAProxy can communicate either HTTP/1.1 or HTTP/2 to backend servers such as Jetty.

Apache HTTP Server and Nginx can only speak HTTP/1.1 to backend servers such as Jetty, and have no plans to support HTTP/2 towards backend servers.

In these setups, typically the proxy performs TLS offloading, and the communication with backend servers happens in clear-text. It is possible, however, to configure the proxy so that all the bytes arriving from the client are tunnelled opaquely to the backend Jetty server (that therefore needs to perform the TLS offloading) and viceversa the bytes arriving from the Jetty server are tunnelled opaquely to the client.

Also in these setups, the TCP/IP connection terminating on the Jetty servers does not originate from the client, but from the proxy, so that the remote IP address and port number may be reported incorrectly in backend server logs, or worse applications may not work because they need to be able to differentiate different clients based on the client IP address.

For this reason, intermediaries typically implement at least one of several de facto standards to communicate information about the original client connection to the backend Jetty server.

Jetty supports two methods to process client information sent by intermediaries:

In both methods, web applications that call HttpServletRequest.getRemoteAddr() will receive the remote client IP address as specified by the client information sent by the intermediary, not the physical IP address of TCP connection with the intermediary. Likewise, HttpServletRequest.getRemotePort() will return the remote client IP port as specified by the client information sent by the intermediary, and HttpServletRequest.isSecure() will return whether the client made a secure request using the https scheme as specified by the client information sent by the intermediary.

Configuring Jetty for the Forwarded Header

The Forwarded HTTP header is added by the intermediary with information about the client and the client request, for example:

GET / HTTP/1.1
Host: domain.com
Forwarded: for=2.36.72.144:21216;proto=https

In the example above, the intermediary added the Forwarded header specifying that the client remote address is 2.36.72.144:21216 and that the request was made with the https scheme.

Support for the Forwarded HTTP header (and its predecessor X-Forwarded-* headers) is enabled with the http-forwarded Jetty module with the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-module=http-forwarded
INFO  : http-forwarded  initialized in ${jetty.base}/start.d/http-forwarded.ini
INFO  : Base directory was modified

With the http-forwarded Jetty module enabled, Jetty interprets the Forwarded header and makes its information available to web applications via the standard Servlet APIs.

For further information about configuring the http-forwarded Jetty module, see this section.

Configuring Jetty for the Proxy Protocol

The Proxy Protocol is the de facto standard, introduced by HAProxy, to communicate client information to backend servers via the TCP connection, rather than via HTTP headers.

The information about the client connection is sent as a small data frame on each newly established connection. This mechanism is therefore independent of any protocol, so it can be used for TLS, HTTP/1.1, HTTP/2, etc.

There are 2 versions of the proxy protocol: v1 and v2, both supported by Jetty.

Proxy protocol v1 is human readable, but it only carries information about the client TCP connection (IP address and IP port).

Proxy protocol v2 has a binary format, carries the information about the client TCP connection, and can carry additional arbitrary information encoded in pairs (type, value) where type is a single byte that indicates the value’s meaning, and value is a variable length byte array that can encode user-defined data.

Support for the proxy protocol can be enabled for the clear-text connector or for the secure connector (or both).

To enable proxy protocol support for the clear-text connector, enable the proxy-protocol Jetty module with the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-module=proxy-protocol
INFO  : proxy-protocol  initialized in ${jetty.base}/start.d/proxy-protocol.ini
INFO  : Base directory was modified

Starting Jetty yields:

$ java -jar $JETTY_HOME/start.jar
2020-10-12 18:44:25.246:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-10-12T13:49:35.796Z; git: 1cd15e8d85feb308527c3df560734fc2ca1bc13c; jvm 15+36-1562
2020-10-12 18:44:25.267:INFO :oejdp.ScanningAppProvider:main: Deployment monitor [file:///tmp/jetty.base/webapps/]
2020-10-12 18:44:25.276:INFO :oejs.AbstractConnector:main: Started ServerConnector@7a8c8dcf{[proxy], ([proxy], http/1.1)}{0.0.0.0:8080}
2020-10-12 18:44:25.285:INFO :oejs.Server:main: Started Server@5c5eefef{STARTING}[10.0.0-SNAPSHOT,sto=5000] @486ms

Note how in the example above the list of protocols for the clear-text connector is first proxy and then http/1.1. For every new TCP connection, Jetty first interprets the proxy protocol bytes with the client information; after this initial proxy protocol processing, Jetty interprets the incoming bytes as HTTP/1.1 bytes.

Similarly, to enable proxy protocol support for the secure connector, enable the proxy-protocol-ssl Jetty module with the following command (issued from within the $JETTY_BASE directory):

$ java -jar $JETTY_HOME/start.jar --add-module=proxy-protocol-ssl
INFO  : proxy-protocol-ssl initialized in ${jetty.base}/start.d/proxy-protocol-ssl.ini
INFO  : Base directory was modified

Starting Jetty yields:

$ java -jar $JETTY_HOME/start.jar
2020-10-12 19:09:38.397:INFO :oejs.Server:main: jetty-10.0.0-SNAPSHOT; built: 2020-10-12T13:49:35.796Z; git: 1cd15e8d85feb308527c3df560734fc2ca1bc13c; jvm 15+36-1562
2020-10-12 19:09:38.417:INFO :oejdp.ScanningAppProvider:main: Deployment monitor [file:///tmp/jetty.base/webapps/]
2020-10-12 19:09:38.605:INFO :oejus.SslContextFactory:main: x509=X509@4a7f959b(mykey,h=[localhost],w=[]) for Server@5403f35f[provider=null,keyStore=file:///tmp/jetty.base/etc/test-keystore.p12,trustStore=file:///tmp/jetty.base/etc/test-keystore.p12]
2020-10-12 19:09:38.697:INFO :oejs.AbstractConnector:main: Started ServerConnector@5afa3c9{[proxy], ([proxy], ssl, http/1.1)}{0.0.0.0:8443}
2020-10-12 19:09:38.705:INFO :oejs.Server:main: Started Server@54d9d12d{STARTING}[10.0.0-SNAPSHOT,sto=5000] @785ms

Note how in the example above the list of protocols for the secure connector is first proxy, then ssl and then http/1.1.

Configuring HAProxy and Jetty for HTTP/1.1 and HTTP/2

HAProxy is an open source solution that offers load balancing and proxying for TCP and HTTP based application, and can be used as a replacement for Apache or Nginx when these are used as reverse proxies.

The deployment proposed here has HAProxy playing the role that Apache and Nginx usually do: to perform the TLS offloading (that is, decrypt incoming bytes and encrypt outgoing bytes) and then forwarding the now clear-text traffic to a backend Jetty server, speaking either HTTP/1.1 or HTTP/2. Since HAProxy’s TLS offloading is based on OpenSSL, it is much more efficient than the Java implementation shipped with OpenJDK.

After you have installed HAProxy on your system, you want to configure it so that it can perform TLS offloading.

HAProxy will need a single file containing the X509 certificates and the private key, all in PEM format, with the following order:

  1. The site certificate; this certificate’s Common Name refers to the site domain (for example: CN=*.webtide.com) and is signed by Certificate Authority #1.

  2. The Certificate Authority #1 certificate; this certificate may be signed by Certificate Authority #2.

  3. The Certificate Authority #2 certificate; this certificate may be signed by Certificate Authority #3; and so on until the Root Certificate Authority.

  4. The Root Certificate Authority certificate.

  5. The private key corresponding to the site certificate.

Refer to the section about KeyStores for more information about generating the required certificates and private key.

Now you can create the HAProxy configuration file (in Linux it’s typically `/etc/haproxy/haproxy.cfg). This is a minimal configuration:

haproxy.cfg
global
tune.ssl.default-dh-param 1024

defaults
timeout connect 10000ms
timeout client 60000ms
timeout server 60000ms

frontend fe_http (1)
mode http
bind *:80
# Redirect to https
redirect scheme https code 301

frontend fe_https (2)
mode tcp
bind *:443 ssl no-sslv3 crt /path/to/domain.pem ciphers TLSv1.2 alpn h2,http/1.1
default_backend be_http

backend be_http (3)
mode tcp
server domain 127.0.0.1:8282 send-proxy-v2
1 The fe_http front-end accepts connections on port 80 and redirects them to use the https scheme.
2 The fe_https front-end accepts connections on port 443, and it is where the TLS decryption/encryption happens. You must specify the path to the PEM file containing the TLS key material (the crt /path/to/domain.pem part), the ciphers that are suitable for HTTP/2 (ciphers TLSv1.2), and the ALPN protocols supported (alpn h2,http/1.1). This front-end then forwards the now decrypted bytes to the backend in mode tcp. The mode tcp says that HAProxy will not try to interpret the bytes but instead opaquely forwards them to the backend.
3 The be_http backend will forward (again in mode tcp) the clear-text bytes to a Jetty connector that talks clear-text HTTP/2 and HTTP/1.1 on port 8282. The send-proxy-v2 directive sends the proxy protocol v2 bytes to the backend server.

On the Jetty side, you need to enable the following modules:

$ java -jar $JETTY_HOME/start.jar --add-modules=proxy-protocol,http2c,http,deploy

You need to specify the host (127.0.0.1) and port (8282) you have configured in HAProxy when you start Jetty:

$ java -jar $JETTY_HOME/start.jar jetty.http.host=127.0.0.1 jetty.http.port=8282

You want the Jetty connector that listens on port 8282 to be available only to HAProxy, and not to remote clients.

For this reason, you want to specify the jetty.http.host property on the command line (or in $JETTY_BASE/start.d/http.ini to make this setting persistent) to bind the Jetty connector only on the loopback interface (127.0.0.1), making it available to HAProxy but not to remote clients.

If your Jetty instance runs on a different machine and/or on a different (sub)network, you may want to adjust both the back-end section of the HAProxy configuration file and the jetty.http.host property to match accordingly.

With this configuration for HAProxy and Jetty, browsers supporting HTTP/2 will connect to HAProxy, which will decrypt the traffic and send it to Jetty. Likewise, HTTP/1.1 clients will connect to HAProxy, which will decrypt the traffic and send it to Jetty.

The Jetty connector, configured with the http2c and the http modules is able to distinguish whether the incoming bytes are HTTP/2 or HTTP/1.1 and will handle the request accordingly.

The response is relayed back to HAProxy, which will encrypt it and send it back to the remote client.

This configuration offers you efficient TLS offloading, HTTP/2 support and transparent fallback to HTTP/1.1 for clients that don’t support HTTP/1.1.

Configuring KeyStores

A KeyStore is a file on the file system that contains a private key and a public certificate, along with the certificate chain of the certificate authorities that issued the certificate. The private key, the public certificate and the certificate chain, but more generally the items present in a KeyStore, are typically referred to as "cryptographic material".

Keystores may encode the cryptographic material with different encodings, the most common being PKCS12, and are typically protected by a password.

Refer to the secure protocols section for more information about how to configure a secure connector using a KeyStore.

Creating a KeyStore

KeyStores are created with the JDK tool $JAVA_HOME/bin/keytool.

The following command creates a KeyStore file containing a private key and a self-signed certificate:

keytool
    -genkeypair (1)
    -validity 90 (2)
    -keyalg RSA (3)
    -keysize 2048 (4)
    -keystore /path/to/keystore.p12 (5)
    -storetype pkcs12 (6)
    -dname "CN=domain.com, OU=Unit, O=Company, L=City, S=State, C=Country" (7)
    -ext san=dns:www.domain.com,dns:domain.org (8)
    -v (9)
1 the command to generate a key and certificate pair
2 specifies the number of days after which the certificate expires
3 the algorithm must be RSA (the DSA algorithm does not work for web sites)
4 indicates the strength of the key
5 the keyStore file
6 the keyStore type, stick with the standard PKCS12
7 the distinguished name (more below) — customize it with your values for CN, OU, O, L, S and C
8 the extension with the subject alternative names (more below)
9 verbose output

The command prompts for the KeyStore password that you must choose to protect the access to the KeyStore.

The important parts of the command above are the Common Name (CN) part of the distinguished name, and the subject alternative names (SAN).

The CN value must be the main domain you own and that you want to use for your web applications. For example, if you have bought domains domain.com and domain.org, you want to specify CN=domain.com as your main domain.

Furthermore, to specify additional domains or subdomains within the same certificate, you must specify the SAN extension. In the example above, san=dns:www.domain.com,dns:domain.org specifies www.domain.com and domain.org as alternative names for your web applications (that you can configure using virtual hosts).

In rare cases, you may want to specify IP addresses, rather than domains, in the SAN extension. The syntax in such case is san=ip:127.0.0.1,ip:[::1], which specifies as subject alternative names IPv4 127.0.0.1 and IPv6 [::1].

Creating a Certificate Signing Request

Self-signed certificates are not trusted by browsers and generic clients: you need to establish a trust chain by having your self-signed certificate signed by a certificate authority (CA).

Browsers and generic clients (e.g. Java clients) have an internal list of trusted certificate authorities root certificates; they use these trusted certificates to verify the certificate they received from the server when they connect to your web applications.

To have your self-signed certificate signed by a certificate authority you first need to produce a certificate signing request (CSR):

keytool
    -certreq (1)
    -file domain.com.csr (2)
    -keystore keystore.p12 (3)
1 the command to generate a certificate signing request
2 the file to save the CSR
3 the keystore that contains the self-signed certificate

Then, you have to send the CSR file to the certificate authority of your choice, and wait for their reply (they will probably require a proof that you really own the domains indicated in your certificate).

Eventually, the certificate authority will reply to you with one or more files containing the CA certificate chain, and your certificate signed by their certificate chain.

Importing the Signed Certificate

The file you receive from the CA is typically in PEM format, and you must import it back into the same KeyStore file you used to generate the CSR. You must import both the certificate chain and your signed certificate.

First, import the certificate chain:

keytool
    -importcert (1)
    -alias ca (2)
    -file chain_from_ca.pem (3)
    -keystore keystore.p12 (4)
    -trustcacerts (5)
    -v (6)
1 the command to import certificates
2 use the ca alias to differentiate from the alias of the server certificate
3 the file containing the certificate chain received from the CA
4 your KeyStore file
5 specify that you trust CA certificates
6 verbose output

Then, import the signed certificate:

keytool
    -importcert
    -file signed_certificate.pem
    -keystore keystore.p12
    -trustcacerts
    -v

Now you have a trusted certificate in your KeyStore that you can use for the domains of your web applications.

Refer to the section about configuring secure protocols to configure the secure connector with your newly created KeyStore.

Creating a KeyStore for Client Certificate Authentication

For the majority of secure web sites, it is the client (typically the browser) that validates the certificate sent by the server (by verifying the certificate chain). This is the server domain certificate.

However, the TLS protocol supports a mutual authentication mode where also the client must send a certificate to the server, that the server validates.

You typically want to sign the client certificate(s) with a server certificate that you control, and you must distribute the client certificate(s) to all the clients that need it, and redistribute the client certificates when they expire. The server authentication certificate may be different from the server domain certificate, but it’s typically stored in the same KeyStore for simplicity (although under a different alias).

First, you want to create the private key and server authentication certificate that you will use to sign client certificates:

keytool
    -genkeypair
    -alias server_authn (1)
    -validity 90
    -keyalg RSA
    -keysize 2048
    -keystore keystore.p12 (2)
    -storetype pkcs12
    -dname "CN=server_authn, OU=Unit, O=Company, L=City, S=State, C=Country" (3)
    -ext bc=ca:true (4)
    -v
1 use the server_authn alias to differentiate from the alias of the server certificate
2 the KeyStore file
3 the CN is not that important, since this certificate will not be validated by clients
4 the extension with the basic constraints (more below)
The important difference with the creation of a server certificate is the basic constraints extension (bc) that indicates that this certificates acts as a certificate authority (ca:true).

Now you want to export both the private key and server authentication certificate. Unfortunately, the keytool program cannot export private keys, so you need to use a different command line program like openssl, or a graphical program like KeyStore Explorer.

Let’s use openssl to export the server authentication private key:

openssl
    pkcs12
    -in keystore.p12
    -nodes
    -nocerts
    -out server_authn.key

Now let’s export the server authentication certificate:

keytool
    -exportcert
    -keystore keystore.p12
    -rfc
    -file server_authn.crt
    -v

At this point, you want to create a client KeyStore, so that you can sign the client certificate with the server authentication cryptographic material:

keytool
    -genkeypair
    -validity 90
    -keyalg RSA
    -keysize 2048
    -keystore client_keystore.p12 (1)
    -storetype pkcs12
    -dname "CN=client, OU=Unit, O=Company, L=City, S=State, C=Country" (2)
    -v
1 the client KeyStore file
2 the CN is not that important, since it will not be validated by the server

Now produce a certificate signing request (CSR):

keytool
    -certreq
    -file client.csr
    -keystore client_keystore.p12

Now you need to sign the CSR, but again the keytool program does not have this functionality, and you must resort again to use openssl:

openssl
    x509
    -req
    -days 90
    -in client.csr
    -CA server_authn.crt
    -CAkey server_authn.key
    -CAcreateserial
    -sha256
    -out signed.crt

Now you need to import the server authentication certificate and the signed client certificate into the client KeyStore.

First, the server authentication certificate:

keytool
    -importcert
    -alias ca
    -file server_authn.crt
    -keystore client_keystore.p12
    -v

Then, the signed client certificate:

keytool
    -importcert
    -file signed.crt
    -keystore client_keystore.p12
    -v

Now you can distribute client_keystore.p12 to your client(s).

Refer to the section about configuring secure protocols to configure the secure connector to require client authentication.

Module bytebufferpool

The bytebufferpool module allows you to configure the server-wide ByteBuffer pool.

The module file is $JETTY_HOME/modules/bytebufferpool.mod:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Configures the ByteBufferPool used by ServerConnectors.

[depends]
logging

[xml]
etc/jetty-bytebufferpool.xml

[ini-template]
### Server ByteBufferPool Configuration
## Minimum capacity to pool ByteBuffers
#jetty.byteBufferPool.minCapacity=0

## Maximum capacity to pool ByteBuffers
#jetty.byteBufferPool.maxCapacity=65536

## Capacity factor
#jetty.byteBufferPool.factor=1024

## Maximum queue length for each bucket (-1 for unbounded)
#jetty.byteBufferPool.maxQueueLength=-1

## Maximum heap memory retainable by the pool (-1 for unlimited)
#jetty.byteBufferPool.maxHeapMemory=-1

## Maximum direct memory retainable by the pool (-1 for unlimited)
#jetty.byteBufferPool.maxDirectMemory=-1

Among the configurable properties, the most relevant are:

TODO

Module deploy

The deploy module provides the deployment feature through a DeploymentManager component that watches a directory for changes (see how to deploy web applications for more information).

Files or directories added in this monitored directory cause the DeploymentManager to deploy them as web applications; updating files already existing in this monitored directory cause the DeploymentManager to re-deploy the correspondent web application; removing files in this monitored directory cause the DeploymentManager to undeploy the correspondent web application (see also here for more information).

The module file is $JETTY_HOME/modules/deploy.mod:

[description]
Enables web application deployment from the $JETTY_BASE/webapps/ directory.

[depend]
webapp

[lib]
lib/jetty-deploy-${jetty.version}.jar

[files]
webapps/

[xml]
etc/jetty-deploy.xml

[ini-template]
# Monitored directory name (relative to $jetty.base)
# jetty.deploy.monitoredDir=webapps
# - OR -
# Monitored directory path (fully qualified)
# jetty.deploy.monitoredPath=/var/www/webapps

# Defaults Descriptor for all deployed webapps
# jetty.deploy.defaultsDescriptorPath=${jetty.base}/etc/webdefault.xml

# Monitored directory scan period (seconds)
# jetty.deploy.scanInterval=1

# Whether to extract *.war files
# jetty.deploy.extractWars=true

Among the configurable properties, the most relevant are:

  • jetty.deploy.monitoredDir, to change the name of the monitored directory.

  • jetty.deploy.scanInterval, to change the scan period, that is how frequently the DeploymentManager wakes up to scan the monitored directory for changes. Setting jetty.deploy.scanInterval=0 disabled hot deployment so that only static deployment will be possible (see also here for more information).

Module http

The http module provides the clear-text connector and support for the clear-text HTTP/1.1 protocol, and depends on the server module.

The module properties to configure the clear-text connector are:

### Clear-Text HTTP Connector Configuration

## The host/address to bind the connector to.
# jetty.http.host=0.0.0.0

## The port the connector listens on.
# jetty.http.port=8080

## The connector idle timeout, in milliseconds.
# jetty.http.idleTimeout=30000

## The number of acceptors (-1 picks a default value based on number of cores).
# jetty.http.acceptors=-1

## The number of selectors (-1 picks a default value based on number of cores).
# jetty.http.selectors=-1

## The ServerSocketChannel accept queue backlog (0 picks the platform default).
# jetty.http.acceptQueueSize=0

## The thread priority delta to give to acceptor threads.
# jetty.http.acceptorPriorityDelta=0

## Whether to enable the SO_REUSEADDR socket option.
# jetty.http.reuseAddress=true

## Whether to enable the TCP_NODELAY socket option on accepted sockets.
# jetty.http.acceptedTcpNoDelay=true

## The SO_RCVBUF socket option to set on accepted sockets.
## A value of -1 indicates that the platform default is used.
# jetty.http.acceptedReceiveBufferSize=-1

## The SO_SNDBUF socket option to set on accepted sockets.
## A value of -1 indicates that the platform default is used.
# jetty.http.acceptedSendBufferSize=-1

Among the configurable properties, the most relevant are:

  • jetty.http.port, default 8080, is the network port that Jetty listens to for clear-text HTTP/1.1 connections.

  • jetty.http.idleTimeout, default 30 seconds, is the amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources.

  • jetty.http.acceptors, default -1 (i.e. an accept heuristic decides the value based on the number of cores), is the number of threads that compete to accept connections.

  • jetty.http.selectors, default -1 (i.e. a select heuristic decides the value based on the number of cores), is the number of NIO selectors (with an associated thread) that manage connections.

Configuration of Acceptors

Accepting connections is a blocking operation, so a thread is blocked in the accept() call until a connection is accepted, and other threads are blocked on the lock acquired just before the accept() call.

When the accepting thread accepts a connection, it performs a little processing of the just accepted connection, before forwarding it to other components.

During this little processing other connections may be established; if there is only one accepting thread, the newly established connections are waiting for the accepting thread to finish the processing of the previously accepted connection and call again accept().

Servers that manage a very high number of connections that may (naturally) come and go, or that handle inefficient protocols that open and close connections very frequently (such as HTTP/1.0) may benefit of an increased number of acceptor threads.

Configuration of Selectors

Performing a NIO select() call is a blocking operation, where the selecting thread is blocked in the select() call until at least one connection is ready to be processed for an I/O operation. There are 4 I/O operations: ready to be accepted, ready to be connected, ready to be read and ready to be written.

A single NIO selector can manage thousands of connections, with the assumption that not many of them will be ready at the same time.

For a single NIO selector, the ratio between the average number of selected connections over the total number of connections for every select() call depends heavily on the protocol but also on the application.

Multiplexed protocols such as HTTP/2 tend to be busier than duplex protocols such as HTTP/1.1, leading to a higher ratio.

REST applications that exchange many little JSON messages tend to be busier than file server applications, leading to a higher ratio.

The higher the ratio, the higher the number of selectors you want to have, compatibly with the number of cores — there is no point in having 64 selector threads on a single core hardware.

Module http2

The http2 module enables support for the secure HTTP/2 protocol.

The module properties are:

## Specifies the maximum number of concurrent requests per session.
# jetty.http2.maxConcurrentStreams=128

## Specifies the initial stream receive window (client to server) in bytes.
# jetty.http2.initialStreamRecvWindow=524288

## Specifies the initial session receive window (client to server) in bytes.
# jetty.http2.initialSessionRecvWindow=1048576

## Specifies the maximum number of keys in all SETTINGS frames received by a session.
# jetty.http2.maxSettingsKeys=64

## Specifies the maximum number of bad frames and pings per second,
## after which a session is closed to avoid denial of service attacks.
# jetty.http2.rateControl.maxEventsPerSecond=20

The jetty.http2.rateControl.maxEventsPerSecond property controls the number of "bad" or "unnecessary" frames that a client may send before the server closes the connection (with code ENHANCE_YOUR_CALM) to avoid a denial of service.

For example, an attacker may send empty SETTINGS frames to a server in a tight loop. While the SETTINGS frames don’t change the server configuration and each of them is somehow harmless, the server will be very busy processing them because they are sent by the attacker one after the other, causing a CPU spike and eventually a denial of service (as all CPUs will be busy processing empty SETTINGS frames).

The same attack may be performed with PRIORITY frames, empty DATA frames, PING frames, etc.

Module http2c

The http2c module enables support for the clear-text HTTP/2 protocol.

The module properties are:

## Specifies the maximum number of concurrent requests per session.
# jetty.http2c.maxConcurrentStreams=1024

  ## Specifies the initial stream receive window (client to server) in bytes.
# jetty.http2c.initialStreamRecvWindow=65535

## Specifies the initial session receive window (client to server) in bytes.
# jetty.http2.initialSessionRecvWindow=1048576

## Specifies the maximum number of keys in all SETTINGS frames received by a session.
# jetty.http2.maxSettingsKeys=64

## Specifies the maximum number of bad frames and pings per second,
## after which a session is closed to avoid denial of service attacks.
# jetty.http2.rateControl.maxEventsPerSecond=20

The jetty.http2.rateControl.maxEventsPerSecond property controls the number of "bad" or "unnecessary" frames that a client may send before the server closes the connection (with code ENHANCE_YOUR_CALM) to avoid a denial of service.

For example, an attacker may send empty SETTINGS frames to a server in a tight loop. While the SETTINGS frames don’t change the server configuration and each of them is somehow harmless, the server will be very busy processing them because they are sent by the attacker one after the other, causing a CPU spike and eventually a denial of service (as all CPUs will be busy processing empty SETTINGS frames).

The same attack may be performed with PRIORITY frames, empty DATA frames, PING frames, etc.

Module http-forwarded

The http-forwarded module provides support for processing the Forwarded HTTP header (defined in RFC 7239) and the now obsoleted X-Forwarded-* HTTP headers.

The module properties are:

### ForwardedRequestCustomizer Configuration

## Whether to process only the RFC7239 "Forwarded" header.
## "X-Forwarded-*" headers are not processed.
# jetty.httpConfig.forwardedOnly=false

## Whether the address obtained from "Forwarded: by=" or
## "X-Forwarded-Server" is used in the request authority.
# jetty.httpConfig.forwardedProxyAsAuthority=false

## Whether the "X-Forwarded-Port" header is used in the request authority,
## or else it is the remote client port.
# jetty.httpConfig.forwardedPortAsAuthority=true

## The name of the RFC 7239 HTTP header.
# jetty.httpConfig.forwardedHeader=Forwarded

## The name of the obsolete forwarded host HTTP header.
# jetty.httpConfig.forwardedHostHeader=X-Forwarded-Host

## The name of the obsolete forwarded server HTTP header.
# jetty.httpConfig.forwardedServerHeader=X-Forwarded-Server

## The name of the obsolete forwarded scheme HTTP header.
# jetty.httpConfig.forwardedProtoHeader=X-Forwarded-Proto

## The name of the obsolete forwarded for HTTP header.
# jetty.httpConfig.forwardedForHeader=X-Forwarded-For

## The name of the obsolete forwarded port HTTP header.
# jetty.httpConfig.forwardedPortHeader=X-Forwarded-Port

## The name of the obsolete forwarded https HTTP header.
# jetty.httpConfig.forwardedHttpsHeader=X-Proxied-Https

## The name of the obsolete forwarded SSL session ID HTTP header.
# jetty.httpConfig.forwardedSslSessionIdHeader=Proxy-ssl-id

## The name of the obsolete forwarded SSL cipher HTTP header.
# jetty.httpConfig.forwardedCipherSuiteHeader=Proxy-auth-cert

Module https

The https module provides the HTTP/1.1 protocol to the ssl module.

The module file is $JETTY_HOME/modules/https.mod:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Adds HTTPS protocol support to the TLS(SSL) Connector.

[tags]
connector
https
http
ssl

[depend]
ssl

[optional]
alpn
http2
http-forwarded

[xml]
etc/jetty-https.xml

Module server

The server module provides generic server support, and configures generic HTTP properties that apply to all HTTP protocols, the scheduler properties and the server specific properties.

The server module depends on the threadpool module, the bytebufferpool module and the logging module.

The module file is $JETTY_HOME/modules/server.mod:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Enables the core Jetty server on the classpath.

[optional]
jvm
ext
resources

[depend]
threadpool
bytebufferpool
logging

[lib]
lib/jetty-jakarta-servlet-api-5.0.*.jar
lib/jetty-http-${jetty.version}.jar
lib/jetty-server-${jetty.version}.jar
lib/jetty-xml-${jetty.version}.jar
lib/jetty-util-${jetty.version}.jar
lib/jetty-io-${jetty.version}.jar

[xml]
etc/jetty.xml

[ini-template]
### Common HTTP configuration
## Scheme to use to build URIs for secure redirects
# jetty.httpConfig.secureScheme=https

## Port to use to build URIs for secure redirects
# jetty.httpConfig.securePort=8443

## Response content buffer size (in bytes)
# jetty.httpConfig.outputBufferSize=32768

## Max response content write length that is buffered (in bytes)
# jetty.httpConfig.outputAggregationSize=8192

## Max request headers size (in bytes)
# jetty.httpConfig.requestHeaderSize=8192

## Max response headers size (in bytes)
# jetty.httpConfig.responseHeaderSize=8192

## Whether to send the Server: header
# jetty.httpConfig.sendServerVersion=true

## Whether to send the Date: header
# jetty.httpConfig.sendDateHeader=false

## Max per-connection header cache size (in nodes)
# jetty.httpConfig.headerCacheSize=1024

## Whether, for requests with content, delay dispatch until some content has arrived
# jetty.httpConfig.delayDispatchUntilContent=true

## Maximum number of error dispatches to prevent looping
# jetty.httpConfig.maxErrorDispatches=10

## HTTP Compliance: RFC7230, RFC7230_LEGACY, RFC2616, RFC2616_LEGACY, LEGACY
# jetty.httpConfig.compliance=RFC7230

## Cookie compliance mode for parsing request Cookie headers: RFC2965, RFC6265
# jetty.httpConfig.requestCookieCompliance=RFC6265

## Cookie compliance mode for generating response Set-Cookie: RFC2965, RFC6265
# jetty.httpConfig.responseCookieCompliance=RFC6265

## Relative Redirect Locations allowed
# jetty.httpConfig.relativeRedirectAllowed=false

### Server configuration
## Whether ctrl+c on the console gracefully stops the Jetty server
# jetty.server.stopAtShutdown=true

## Timeout in ms to apply when stopping the server gracefully
# jetty.server.stopTimeout=5000

## Dump the state of the Jetty server, components, and webapps after startup
# jetty.server.dumpAfterStart=false

## Dump the state of the Jetty server, components, and webapps before shutdown
# jetty.server.dumpBeforeStop=false

## Scheduler Configuration
# jetty.scheduler.name=
# jetty.scheduler.deamon=false
# jetty.scheduler.threads=-1

Among the configurable properties, the most relevant are:

TODO

Module ssl

The ssl module provides the secure connector, and allows you to configure the KeyStore properties and the TLS parameters, and depends on the server module.

Secure Connector Properties

The module properties to configure the secure connector are:

### TLS (SSL) Connector Configuration

## The host/address to bind the connector to.
# jetty.ssl.host=0.0.0.0

## The port the connector listens on.
# jetty.ssl.port=8443

## The connector idle timeout, in milliseconds.
# jetty.ssl.idleTimeout=30000

## The number of acceptors (-1 picks a default value based on number of cores).
# jetty.ssl.acceptors=-1

## The number of selectors (-1 picks a default value based on number of cores).
# jetty.ssl.selectors=-1

## The ServerSocketChannel accept queue backlog (0 picks the platform default).
# jetty.ssl.acceptQueueSize=0

## The thread priority delta to give to acceptor threads.
# jetty.ssl.acceptorPriorityDelta=0

## Whether to enable the SO_REUSEADDR socket option.
# jetty.ssl.reuseAddress=true

## Whether to enable the TCP_NODELAY socket option on accepted sockets.
# jetty.ssl.acceptedTcpNoDelay=true

## The SO_RCVBUF socket option to set on accepted sockets.
## A value of -1 indicates that the platform default is used.
# jetty.ssl.acceptedReceiveBufferSize=-1

## The SO_SNDBUF socket option to set on accepted sockets.
## A value of -1 indicates that the platform default is used.
# jetty.ssl.acceptedSendBufferSize=-1

## Whether client SNI data is required for all secure connections.
## When SNI is required, clients that do not send SNI data are rejected with an HTTP 400 response.
# jetty.ssl.sniRequired=false

## Whether client SNI data is checked to match CN and SAN in server certificates.
## When SNI is checked, if the match fails the connection is rejected with an HTTP 400 response.
# jetty.ssl.sniHostCheck=true

## The max age, in seconds, for the Strict-Transport-Security response header.
# jetty.ssl.stsMaxAgeSeconds=31536000

## Whether to include the subdomain property in any Strict-Transport-Security header.
# jetty.ssl.stsIncludeSubdomains=true

Among the configurable properties, the most relevant are:

jetty.ssl.port

the network port that Jetty listens to for secure connections — default 8443.

jetty.http.idleTimeout

the amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources — default 30000 milliseconds.

jetty.http.acceptors

the number of threads that compete to accept connections — default -1 (i.e. an accept heuristic decides the value based on the number of cores). Refer to this section for more information about acceptor threads.

jetty.http.selectors

the number of NIO selectors (with an associated thread) that manage connections — default -1 (i.e. a select heuristic decides the value based on the number of cores). Refer to this section for more information about selector threads.

The module properties to configure the KeyStore and TLS parameters are:

### SslContextFactory Configuration
## Note that OBF passwords are not secure, just protected from casual observation.

## Whether client SNI data is required for all secure connections.
## When SNI is required, clients that do not send SNI data are rejected with a TLS handshake error.
# jetty.sslContext.sniRequired=false

## The Endpoint Identification Algorithm.
## Same as javax.net.ssl.SSLParameters#setEndpointIdentificationAlgorithm(String).
# jetty.sslContext.endpointIdentificationAlgorithm=

## The JSSE Provider.
# jetty.sslContext.provider=

## The KeyStore file path (relative to $JETTY_BASE).
# jetty.sslContext.keyStorePath=etc/keystore.p12

## The TrustStore file path (relative to $JETTY_BASE).
# jetty.sslContext.trustStorePath=etc/keystore.p12

## The KeyStore password.
# jetty.sslContext.keyStorePassword=

## The Keystore type.
# jetty.sslContext.keyStoreType=PKCS12

## The KeyStore provider.
# jetty.sslContext.keyStoreProvider=

## The KeyManager password.
# jetty.sslContext.keyManagerPassword=

## The TrustStore password.
# jetty.sslContext.trustStorePassword=

## The TrustStore type.
# jetty.sslContext.trustStoreType=PKCS12

## The TrustStore provider.
# jetty.sslContext.trustStoreProvider=

## Whether client certificate authentication is required.
# jetty.sslContext.needClientAuth=false

## Whether client certificate authentication is desired, but not required.
# jetty.sslContext.wantClientAuth=false

## Whether cipher order is significant.
# jetty.sslContext.useCipherSuitesOrder=true

## The SSLSession cache size.
# jetty.sslContext.sslSessionCacheSize=-1

## The SSLSession cache timeout (in seconds).
# jetty.sslContext.sslSessionTimeout=-1

## Whether TLS renegotiation is allowed.
# jetty.sslContext.renegotiationAllowed=true

## The max number of TLS renegotiations per connection.
# jetty.sslContext.renegotiationLimit=5
KeyStore Properties and TLS Properties

Among the configurable properties, the most relevant are:

jetty.sslContext.keyStorePath

the KeyStore path on the file system. If it is a relative path, it is relative to $JETTY_BASE. Defaults to $JETTY_BASE/etc/keystore.p12.

jetty.sslContext.keyStorePassword

the KeyStore password, which you want to explicitly configure. The password may be obfuscated with the Jetty Password Tool.

If you need to configure client certificate authentication, you want to configure one of these properties (they are mutually exclusive):

jetty.sslContext.needClientAuth

whether client certificate authentication should be required.

jetty.sslContext.wantClientAuth

whether client certificate authentication should be requested.

If you configure client certificate authentication, you need to configure and distribute a client KeyStore as explained in this section.

Module ssl-reload

The ssl-reload module provides a periodic scanning of the directory where the KeyStore file resides. When the scanning detects a change to the KeyStore file, the correspondent SslContextFactory.Server component is reloaded with the new KeyStore configuration.

The module properties are:

# Monitored directory scan period, in seconds.
# jetty.sslContext.reload.scanInterval=1

Module test-keystore

The test-keystore module provides a KeyStore containing a self-signed certificate for domain localhost.

The module file is $JETTY_HOME/modules/test-keystore.mod:

[description]
Test keystore with test SSL Certificate.
DO NOT USE IN PRODUCTION!!!

[tags]
demo
ssl

[depend]
ssl

[files]
basehome:modules/test-keystore/test-keystore.p12|etc/test-keystore.p12

[ini]
jetty.sslContext.keyStorePath?=etc/test-keystore.p12
jetty.sslContext.trustStorePath?=etc/test-keystore.p12
jetty.sslContext.keyStoreType?=PKCS12
jetty.sslContext.keyStorePassword?=OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4

Note how properties jetty.sslContext.keyStorePath and jetty.sslContext.keyStorePassword are configured, only if not already set (via the ?= operator), directly in the module file, rather than in a *.ini file. This is done to avoid that these properties accidentally overwrite a real KeyStore configuration.

Module threadpool

The threadpool module allows you to configure the server-wide thread pool.

The module file is $JETTY_HOME/modules/threadpool.mod:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Enables and configures the Server thread pool.

[depends]
logging

[xml]
etc/jetty-threadpool.xml

[ini-template]

### Server Thread Pool Configuration
## Minimum Number of Threads
#jetty.threadPool.minThreads=10

## Maximum Number of Threads
#jetty.threadPool.maxThreads=200

## Number of reserved threads (-1 for heuristic)
#jetty.threadPool.reservedThreads=-1

## Thread Idle Timeout (in milliseconds)
#jetty.threadPool.idleTimeout=60000

## Whether to Output a Detailed Dump
#jetty.threadPool.detailedDump=false

Among the configurable properties, the most relevant are:

TODO

Jetty XML

The Jetty XML format is a straightforward mapping of XML elements to Java APIs so that any object can be instantiated and getters, setters, and methods can be called.

The Jetty XML format is very similar to that of frameworks like Spring or Plexus, although it predates all of them and it’s typically more powerful as it can invoke any Java API.

The Jetty XML format is used in Jetty modules to create the Jetty server components, as well as in Jetty XML context files to configure web applications, but it can be used to call any Java API.

Jetty XML Syntax

The Jetty XML syntax defines XML element that allow you to call any Java API and that allow you to interact in a simpler way with the Jetty module system and the Jetty deploy system.

The Jetty XML elements define attributes such as id, name, class, etc. that may be replaced by correspondent elements, so that these XML documents are equivalent:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Get id="stderr" class="java.lang.System" name="err">
    <Call name="println" arg="HELLO" />
  </Get>
</Configure>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Get>
    <Id>stderr</Id>
    <Name>err</Name>
    <Class>java.lang.System</Class>
    <Call>
      <Name>println</Name>
      <Arg>HELLO</Arg>
    </Call>
  </Get>
</Configure>

The version using attributes is typically shorter and nicer to read, but sometimes the attribute value cannot be a literal string (for example, it could be the value of a system property) and that’s where elements gives you the required flexibility.

<Configure>

Element Configure must be the root element of the XML document.

The following Jetty XML creates an empty String and assigns it the id mystring:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="mystring" class="java.lang.String" />

This is equivalent to the following Java code:

var mystring = new String();

If an object with the id mystring already exists, then it is not created again but rather just referenced.

Typically the <Configure> element is used to configure a Server instance or ContextHandler subclasses such as WebAppContext that represent web applications.

<Arg>

Element Arg is used to pass arguments to constructors and method calls.

The following example creates a minimal Jetty Server:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.server.Server">
  <Arg type="int">8080</Arg>
</Configure>

Arguments may have a type attribute that explicitly performs type coercion.

Arguments may also have a name attribute, which is matched with the corresponding Java annotation in the source class, that helps to identify arguments:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.server.Server">
  <Arg name="port" type="int">8080</Arg>
</Configure>
<New>

Element <New> creates a new object of the type specified by the mandatory class attribute. A sequence of Arg elements, that must be contiguous and before other elements, may be present to specify the constructor arguments.

Within element <New> the newly created object is in scope and may be the implicit target of other, nested, elements.

The following example creates an ArrayList:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <New id="mylist" class="java.util.ArrayList">
    <Arg type="int">16</Arg>
  </New>
</Configure>

This is equivalent to the following Java code:

var mylist = new ArrayList(16);
<Call>

Element <Call> invokes a method specified by the mandatory name attribute. A sequence of Arg elements, that must be contiguous and before other elements, may be present to specify the method arguments.

Within element <Call> the return value, if the return type is not void, is in scope and may be the implicit target of other, nested, elements.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <New class="java.util.ArrayList">
    <Call name="listIterator">
      <Arg type="int">0</Arg>
    </Call>
    <Call name="next" />
  </New>
</Configure>

This is equivalent to the following Java code:

new ArrayList().listIterator(0).next();

It is possible to call static methods by specifying the class attribute:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Call id="myhost" name="getByName" class="java.net.InetAddress">
    <Arg>jdk.java.net</Arg>
  </Call>
</Configure>

This is equivalent to the following Java code:

var myhost = InetAddress.getByName("jdk.java.net");
<Get>

Element <Get> retrieves the value of a JavaBean property specified by the mandatory name attribute.

If the JavaBean property is foo (or Foo), <Get> first attempts to invoke method getFoo(); failing that, attempts to retrieve the value from field foo (or Foo).

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="server" class="org.eclipse.jetty.server.Server">
  <!-- Invokes getter method server.getVersion() -->
  <Get id="version" name="version" />

  <!-- Gets the System.err field -->
  <Get class="java.lang.System" name="err">
    <Call name="println">
      <Arg>Jetty</Arg>
    </Call>
  </Get>
</Configure>
<Set>

Element <Set> stores the value of a JavaBean property specified by the mandatory name attribute.

If the JavaBean property is foo (or Foo), <Set> first attempts to invoke method setFoo(…​) with the value in the scope as argument; failing that, attempts to store the value in the scope to field foo (or Foo).

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="server" class="org.eclipse.jetty.server.Server">
  <!-- The value in the <Set> scope is the string "true" -->
  <Set name="dryRun">true</Set>

  <!-- The value in the <Set> scope is the instance created by <New> -->
  <Set name="requestLog">
    <New class="org.eclipse.jetty.server.CustomRequestLog" />
  </Set>
</Configure>
<Map> and <Entry>

Element <Map> allows the creation of a new java.util.Map implementation, specified by the class attribute — by default a HashMap.

The map entries are specified with a sequence of <Entry> elements, each with exactly 2 <Item> elements, for example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Map class="java.util.concurrent.ConcurrentHashMap">
    <Entry>
      <Item>host</Item>
      <Item>
        <Call class="java.net.InetAddress" name="getByName">
          <Arg>localhost</Arg>
        </Call>
      </Item>
    </Entry>
  </Map>
</Configure>
<Put>

Element <Put> is a convenience element that puts a key/value pair into objects that implement java.util.Map. You can only specify the key value via the name attribute, so the key can only be a literal string (for keys that are not literal strings, use the <Call> element).

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <New class="java.util.Properties">
    <Put name="host">
      <Call class="java.net.InetAddress" name="getByName">
        <Arg>localhost</Arg>
      </Call>
    </Put>
  </New>
</Configure>
<Array> and <Item>

Element <Array> creates a new array, whose component type may be specified by the type attribute.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
  <Array type="java.lang.Object">
    <Item /> <!-- null -->
    <Item>literalString</Item>
    <Item type="String"></Item> <!-- empty string -->
    <Item type="Double">1.0D</Item>
    <Item>
      <New class="java.lang.Exception" />
    </Item>
  </Array>
</Configure>
<Ref>

Element <Ref> allows you to reference an object via the refid attribute`, putting it into scope so that nested elements can operate on it. You must give a unique id attribute to the objects you want to reference.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<!-- The Jetty Server has id="server" -->
<Configure id="server" class="org.eclipse.jetty.server.Server">
  <Get class="java.lang.System" name="err">
    <!-- Here the System.err field is in scope, but you
         want to operate on the server to get its version -->
    <Ref refid="server">
      <!-- Store the server version under id="myversion" -->
      <Get id="myversion" name="version" />
    </Ref>

    <Call name="println">
      <!-- Reference the server version stored above -->
      <Arg>Server version is: <Ref refid="myversion" /></Arg>
    </Call>
  </Get>
</Configure>
<Property>

Element <Property> retrieves the value of the Jetty module property specified by the name attribute, and it is mostly used when creating custom Jetty modules or when using Jetty context XML files.

The deprecated attribute allows you to specify a comma separated list of old, deprecated, property names for backward compatibility.

The default attribute allows you to specify a default value for the property, if it has not been explicitly defined.

For example, you may want to configure the context path of your web application in this way:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">
    <Property name="com.myapps.mywiki.context.path" default="/wiki" />
  </Set>
  <Set name="war">/opt/myapps/mywiki.war</Set>
</Configure>

The contextPath value is resolved by looking for the Jetty module property com.myapps.mywiki.context.path; if this property is not set, then the default value of /wiki is used.

<SystemProperty>

Element <SystemProperty> retrieves the value of the JVM system property specified by the name attribute, via System.getProperty(…​).

The deprecated attribute allows you to specify a comma separated list of old, deprecated, system property names for backward compatibility.

The default attribute allows you to specify a default value for the system property value, if it has not been explicitly defined.

The following example creates a minimal Jetty Server that listens on a port specified by the com.acme.http.port system property:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="server" class="org.eclipse.jetty.server.Server">
  <Arg type="int">
    <SystemProperty name="com.acme.http.port" default="8080" />
  </Arg>
</Configure>
<Env>

Element <Env> retrieves the value of the environment variable specified by the name attribute, via System.getenv(…​).

The deprecated attribute allows you to specify a comma separated list of old, deprecated, environment variable names for backward compatibility.

The default attribute allows you to specify a default value for the environment variable value, if it has not been explicitly defined.

The following example creates a minimal Jetty Server that listens on a port specified by the COM_ACME_HTTP_PORT environment variable:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="server" class="org.eclipse.jetty.server.Server">
  <Arg type="int">
    <Env name="COM_ACME_HTTP_PORT" default="8080" />
  </Arg>
</Configure>
Type Coercion

Elements that have the type attribute explicitly perform the type coercion of the string value present in the XML document to the Java type specified by the type attribute.

Supported types are the following:

  • all primitive types and their boxed equivalents, for example type="int" but also type="Integer" (short form) and type="java.lang.Integer" (fully qualified form)

  • java.lang.String, in both short form and fully qualified form

  • java.net.URL, in both short form and fully qualified form

  • java.net.InetAddress, in both short form and fully qualified form

Scopes

Elements that create new objects or that return a value create a scope. Within these elements there may be nested elements that will operate on that scope, i.e. on the new object or returned value.

The following example illustrates how scopes work:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="server" class="org.eclipse.jetty.server.Server">
  <Arg type="int">8080</Arg>
  <!-- Here the Server object has been created and is in scope -->

  <!-- Calls the setter on the Server object that is in scope -->
  <Set name="stopTimeout">5000</Set>

  <!-- Creates a new object -->
  <New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
    <!-- Here the HttpConfiguration just created is in a nested scope -->

    <!-- Calls the setter on the HttpConfiguration object that is in scope -->
    <Set name="secureScheme">https</Set>
  </New>

  <!-- Calls the getter on the Server object that is in scope -->
  <Get name="ThreadPool">
    <!-- Here the ThreadPool object returned by the getter is in a nested scope -->

    <!-- Calls the setter on the ThreadPool object that is in scope -->
    <Set name="maxThreads" type="int">256</Set>
  </Get>

  <!-- Gets the System.err field -->
  <Get class="java.lang.System" name="err">
    <!-- Here the System.err object is in scope -->

    <!-- Equivalent to: var myversion = server.getVersion() -->
    <Ref refid="server">
      <!-- Here the "server" object is in scope -->
      <Get id="myversion" name="version" />
    </Ref>

    <!-- Calls println() on the System.err object -->
    <Call name="println">
      <Arg>Server version is: <Ref refid="myversion" /></Arg>
    </Call>
  </Get>
</Configure>

HTTP Session Management

HTTP sessions are a concept within the Servlet API which allow requests to store and retrieve information across the time a user spends in an application. Jetty offers a number of pluggable alternatives for managing and distributing/persisting sessions. Choosing the best alternative is an important consideration for every application as is the correct configuration to achieve optimum performance.

HTTP Session Overview

Terminology

Before diving into the specifics of how to plug-in and configure various alternative HTTP session management modules, let’s review some useful terminology:

Session

is a means of retaining information across requests for a particular user. The Servlet Specification defines the semantics of sessions. Some of the most important characteristics of sessions is that they have a unique id and that their contents cannot be shared between different contexts (although the id can be): if a session is invalidated in one context, then all other sessions that share the same id in other contexts will also be invalidated. Sessions can expire or they can be explicitly invalidated.

SessionIdManager

is responsible for allocating session ids. A Jetty server can have at most 1 SessionIdManager.

HouseKeeper

is responsible for periodically orchestrating the removal of expired sessions. This process is referred to as "scavenging".

SessionHandler

is responsible for managing the lifecycle of sessions. A context can have at most 1 SessionHandler.

SessionCache

is a L1 cache of in-use session objects. The SessionCache is used by the SessionHandler.

SessionDataStore

is responsible for all clustering/persistence operations on sessions. A SessionCache uses a SessionDataStore as a backing store.

CachingSessionDataStore

is an L2 cache of session data. A SessionCache can use a CachingSessionDataStore as its backing store.

More details on these concepts can be found in the Programming Guide.

SessionDataStores implementations interact with other, usually third party, systems responsible for storing and/or distributing session information. Sessions can be distributed without being persisted. They can also be persisted without being distributed. Because persisting session information to a shared store is a very common way of distributing (also known as "clustering") sessions, in the documentation we will often refer to just "persisting".

Session Modules

There are a number of modules that offer pluggable alternatives for http session management. You can design how you want to cache and store http sessions by selecting alternative combinations of session modules.

For example, Jetty ships with two alternative implementations of the SessionCache:

There are at least 6 alternative implementations of the SessionDataStore that you can use to persist/distribute your http sessions:

It is worth noting that if you do not configure any session modules, Jetty will still provide HTTP sessions that are cached in memory but are never persisted.

The Base Session Module

The sessions module is the base module that all other session modules depend upon. As such it will be transitively enabled if you enable any of the other session modules: you need to explicitly enable it if you wish to change any settings from their defaults.

Enabling the sessions module puts the $JETTY_HOME/etc/sessions/id-manager.xml file onto the execution path and generates a $JETTY_BASE/start.d/sessions.ini file.

The id-manager.xml file instantiates a DefaultSessionIdManager and HouseKeeper. The former is used to generate and manage session ids whilst the latter is responsible for periodic scavenging of expired sessions.

Configuration

The $JETTY_BASE/start.d/sessions.ini file contains these configuration properties:

jetty.sessionIdManager.workerName

This uniquely identifies the jetty server instance and is applied to the SessionIdManager. You can either provide a value for this property, or you can allow Jetty to try and synthesize a workerName - the latter option is only advisable in the case of a single, non-clustered deployment. There are two ways a default workerName can be synthesized:

  • if running on Google AppEngine, the workerName will be formed by concatenating the values of the environment variables JETTY_WORKER_INSTANCE and GAE_MODULE_INSTANCE

  • otherwise, the workerName will be formed by concatenating the environment variable JETTY_WORKER_INSTANCE and the literal 0.

So, if you’re not running on Google AppEngine, and you haven’t configured one, the workerName will always be: node0.

If you have more than one Jetty instance, it is crucial that you configure the workerName differently for each instance.
jetty.sessionScavengeInterval.seconds

This is the period in seconds between runs of the HouseKeeper, responsible for orchestrating the removal of expired sessions. By default it will run approximately every 600 secs (ie 10 mins). As a rule of thumb, you should ensure that the scavenge interval is shorter than the <session-timeout> of your sessions to ensure that they are promptly scavenged. On the other hand, if you have a backend store configured for your sessions, scavenging too frequently can increase the load on it.

Don’t forget that the <session-timeout> is specified in web.xml in minutes and the value of the jetty.sessionScavengeInterval.seconds is in seconds.
Session Scavenging

The HouseKeeper is responsible for the periodic initiation of session scavenge cycles. The jetty.sessionScavengeInterval.seconds property in $JETTY_BASE/start.d/sessions.ini controls the periodicity of the cycle.

The HouseKeeper semi-randomly adds an additional 10% to the configured sessionScavengeInterval. This is to prevent multiple nodes in a cluster that are all started at once from syncing up scavenge cycles and placing extra load on the configured persistence mechanism.

A session whose expiry time has been exceeded is considered eligible for scavenging. The session might be present in a SessionCache and/or present in the session persistence/clustering mechanism.

Scavenging occurs for all contexts on a server at every cycle. The HouseKeeper sequentially asks the SessionHandler in each context to find and remove expired sessions. The SessionHandler works with the SessionDataStore to evaluate candidates for expiry held in the SessionCache, and also to sweep the persistence mechanism to find expired sessions.

The sweep takes two forms: once per cycle the SessionDataStore searches for sessions for its own context that have expired; infrequently, the SessionDataStore will widen the search to expired sessions in all contexts. The former finds sessions that are no longer in this context’s SessionCache, and using some heuristics, are unlikely to be in the SessionCache of the same context on another node either. These sessions will be loaded and fully expired, meaning that HttpSessionListener.destroy() will be called for them. The latter finds sessions that have not been disposed of by scavenge cycles on any other context/node. As these will be sessions that expired a long time ago, and may not be appropriate to load by the context doing the scavenging, these are summarily deleted without HttpSessionListener.destroy() being called.

A combination of these sweeps should ensure that the persistence mechanism does not fill over time with expired sessions.

As aforementioned, the sweep period needs to be short enough to find expired sessions in a timely fashion, but not so often that it overloads the persistence mechanism.

Modules for HTTP Session Caching

In this section we will look at the alternatives for the SessionCache, i.e. the L1 cache of in-use session objects. Jetty ships with 2 alternatives: an in-memory cache, and a null cache. The latter does not actually do any caching of sessions, and can be useful if you either want to minimize your support for sessions, or you are in a clustered deployment without a sticky loadbalancer.

The scenarios go into more detail on this.

Caching in Memory

If you wish to change any of the default configuration values you should enable the session-cache-hash module. The name "hash" harks back to historical Jetty session implementations, whereby sessions were kept in memory using a HashMap.

Configuration

The $JETTY_BASE/start.d/session-cache-hash.ini contains the following configurable properties:

jetty.session.evictionPolicy

Integer, default -1. This controls whether session objects that are held in memory are subject to eviction from the cache. Eviction means that the session is removed from the cache. This can reduce the memory footprint of the cache and can be useful if you have a lot of sessions. Eviction is usually used in conjunction with a SessionDataStore that persists sessions. The eviction strategies and their corresponding values are:

-1 (NO EVICTION)

sessions are never evicted from the cache. The only way they leave are via expiration or invalidation.

0 (EVICT AFTER USE)

sessions are evicted from the cache as soon as the last active request for it finishes. The session will be passed to the SessionDataStore to be written out before eviction.

>= 1 (EVICT ON INACTIVITY)

any positive number is the time in seconds after which a session that is in the cache but has not experienced any activity will be evicted. Use the jetty.session.saveOnInactiveEvict property to force a session write before eviction.

If you are not using one of the session store modules, ie one of the session-store-xxxxs, then sessions will be lost when the context is stopped, or the session is evicted.
jetty.session.saveOnInactiveEvict

Boolean, default false. This controls whether a session will be persisted to the SessionDataStore if it is being evicted due to the EVICT ON INACTIVITY policy. Usually sessions will be written to the SessionDataStore whenever the last simultaneous request exits the session. However, as SessionDataStores can be configured to skip some writes (see the documentation for the session-store-xxx module that you are using), this option is provided to ensure that the session will be written out.

Be careful with this option, as in clustered scenarios it would be possible to "re-animate" a session that has actually been deleted by another node.
jetty.session.saveOnCreate

Boolean, default false. Controls whether a session that is newly created will be immediately saved to the SessionDataStore or lazily saved as the last request for the session exits. This can be useful if the request dispatches to another context and needs to re-use the same session id.

jetty.session.removeUnloadableSessions

Boolean, default false. Controls whether the session cache should ask a SessionDataStore to delete a session that cannot be restored - for example because it is corrupted.

jetty.session.flushOnResponseCommit

Boolean, default false. If true, if a session is "dirty" - ie its attributes have changed - it will be written to the SessionDataStore as the response is about to commit. This ensures that all subsequent requests whether to the same or different node will see the updated session data. If false, a dirty session will only be written to the backing store when the last simultaneous request for it leaves the session.

jetty.session.invalidateOnShutdown

Boolean, default false. If true, when a context is shutdown, all sessions in the cache are invalidated and deleted both from the cache and from the SessionDataStore.

No Caching

You may need to use the session-cache-null module if your clustering setup does not have a sticky load balancer, or if you want absolutely minimal support for sessions. If you enable this module, but you don’t enable a module that provides session persistence (ie one of the session-store-xxx modules), then sessions will neither be retained in memory nor persisted.

Configuration

The $JETTY_BASE/start.d/session-cache-null.ini contains the following configurable properties:

jetty.session.saveOnCreate

Boolean, default false. Controls whether a session that is newly created will be immediately saved to the SessionDataStore or lazily saved as the last request for the session exits. This can be useful if the request dispatches to another context and needs to re-use the same session id.

jetty.session.removeUnloadableSessions

Boolean, default false. Controls whether the session cache should ask a SessionDataStore to delete a session that cannot be restored - for example because it is corrupted.

jetty.session.flushOnResponseCommit

Boolean, default false. If true, if a session is "dirty" - ie its attributes have changed - it will be written to the backing store as the response is about to commit. This ensures that all subsequent requests whether to the same or different node will see the updated session data. If false, a dirty session will only be written to the backing store when the last simultaneous request for it leaves the session.

Modules for Persistent HTTP Sessions: File System

The session-store-file Jetty module supports persistent storage of session data in a filesystem.

Persisting sessions to the local file system should never be used in a clustered environment.

Enabling this module creates the $JETTY_BASE/sessions directory. By default session data will be saved to this directory, one file representing each session.

File names follow this pattern:

[expiry]_[contextpath]_[virtualhost]_[id]

expiry

This is the expiry time in milliseconds since the epoch.

contextpath

This is the context path with any special characters, including /, replaced by the underscore character. For example, a context path of /catalog would become _catalog. A context path of simply / becomes just _.

virtualhost

This is the first virtual host associated with the context and has the form of 4 digits separated by . characters: [digit].[digit].[digit].[digit]. If there are no virtual hosts associated with a context, then 0.0.0.0 is used.

id

This is the unique id of the session.

Putting all of the above together as an example, a session with an id of node0ek3vx7x2y1e7pmi3z00uqj1k0 for the context with path /test with no virtual hosts and an expiry of 1599558193150 would have a file name of:

1599558193150__test_0.0.0.0_node0ek3vx7x2y1e7pmi3z00uqj1k0

Configuration

The $JETTY_BASE/start.d/sessions.ini file contains the following properties which may be modified to customise filesystem session storage:

jetty.session.storeDir

The default is $JETTY_BASE/sessions. This is a path that defines the location for storage of session files.

jetty.session.file.deleteUnrestorableFiles

Boolean, default false. If set to true, unreadable files will be deleted. This is useful to prevent repeated logging of the same error when the scavenger periodically (re-)attempts to load the corrupted information for a session in order to expire it.

jetty.session.gracePeriod.seconds

Integer, default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

Modules for Persistent HTTP Sessions: JDBC

Enabling the session-store-jdbc module configures Jetty to persist session data in a relational database.

Configuration

After enabling the module, the $JETTY_BASE/start.d/session-store-jdbc.ini file contains the following customizable properties:

jetty.session.gracePeriod.seconds

Integer, default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

db-connection-type

Default datasource. Set to either datasource or driver depending on the type of connection being used. Depending which you select, there are additional properties available:

datasource
jetty.session.jdbc.datasourceName

Name of the remote datasource.

driver
jetty.session.jdbc.driverClass

Name of the JDBC driver that controls access to the remote database, such as com.mysql.jdbc.Driver

jetty.session.jdbc.driverUrl

URL of the database which includes the driver type, host name and port, service name and any specific attributes unique to the database, such as a username. As an example, here is a mysql connection with the username appended: jdbc:mysql://127.0.0.1:3306/sessions?user=sessionsadmin.

jetty.session.jdbc.blobType

Optional. Default blob or bytea for Postgres. This is the keyword used by the particular database to identify the blob data type. If netiher default is suitable you can set this value explicitly.

jetty.session.jdbc.longType

Optional. Default bigint or number(20) for Oracle. This is the keyword used by the particular database to identify the long integer data type. Set this explicitly if neither of the default values is appropriate.

jetty.session.jdbc.stringType

Optional. Default varchar. This is the keyword used by the particular database to identify character type. If the default is not suitable, you can set this value explicitly.

jetty.session.jdbc.schema.schemaName
jetty.session.jdbc.schema.catalogName

Optional. The exact meaning of these two properties is dependent on your database vendor, but can broadly be described as further scoping for the session table name. See https://en.wikipedia.org/wiki/Database_schema and https://en.wikipedia.org/wiki/Database_catalog. These extra scoping names can come into play at startup time when Jetty determines if the session table already exists, or otherwise creates it on-the-fly. If you have employed either of these concepts when you pre-created the session table, or you want to ensure that Jetty uses them when it auto-creates the session table, then you have two options: either set them explicitly, or let Jetty infer them from a database connection (obtained using either a Datasource or Driver according to the db-connection-type you have configured). To set them explicitly, uncomment and supply appropriate values for the jetty.session.jdbc.schema.schemaName and/or jetty.session.jdbc.schema.catalogName properties. Alternatively, to allow Jetty to infer them from a database connection, use the special string INFERRED instead. If you leave them blank or commented out, then the sessions table will not be scoped by schema or catalog name.

jetty.session.jdbc.schema.table

Default JettySessions. This is the name of the table in which session data is stored.

jetty.session.jdbc.schema.accessTimeColumn

Default accessTime. This is the name of the column that stores the time - in ms since the epoch - at which a session was last accessed

jetty.session.jdbc.schema.contextPathColumn

Default contextPath. This is the name of the column that stores the contextPath of a session.

jetty.session.jdbc.schema.cookieTimeColumn

Default cookieTime. This is the name of the column that stores the time - in ms since the epoch - that the cookie was last set for a session.

jetty.session.jdbc.schema.createTimeColumn

Default createTime. This is the name of the column that stores the time - in ms since the epoch - at which a session was created.

jetty.session.jdbc.schema.expiryTimeColumn

Default expiryTime. This is name of the column that stores - in ms since the epoch - the time at which a session will expire.

jetty.session.jdbc.schema.lastAccessTimeColumn

Default lastAccessTime. This is the name of the column that stores the time - in ms since the epoch - that a session was previously accessed.

jetty.session.jdbc.schema.lastSavedTimeColumn

Default lastSavedTime. This is the name of the column that stores the time - in ms since the epoch - at which a session was last written.

jetty.session.jdbc.schema.idColumn

Default sessionId. This is the name of the column that stores the id of a session.

jetty.session.jdbc.schema.lastNodeColumn

Default lastNode. This is the name of the column that stores the workerName of the last node to write a session.

jetty.session.jdbc.schema.virtualHostColumn

Default virtualHost. This is the name of the column that stores the first virtual host of the context of a session.

jetty.session.jdbc.schema.maxIntervalColumn

Default maxInterval. This is the name of the column that stores the interval - in ms - during which a session can be idle before being considered expired.

jetty.session.jdbc.schema.mapColumn

Default map. This is the name of the column that stores the serialized attributes of a session.

Modules for Persistent HTTP Sessions: MongoDB

Enabling the session-store-mongo module configures Jetty to store session data in MongoDB.

Because MongoDB is not a technology provided by the Eclipse Foundation, you will be prompted to assent to the licenses of the external vendor (Apache in this case) during the install. Jars needed by MongoDB are downloaded and stored into a directory named $JETTY_BASE/lib/nosql/.

If you want to use updated versions of the jar files automatically downloaded by Jetty, you can place them in the associated $JETTY_BASE/lib/ directory and use the --skip-file-validation=<module name> command line option to prevent errors when starting your server.
Configuration

The $JETTY_BASE/start.d/session-store-mongo.ini file contains these configurable properties:

jetty.session.mongo.dbName

Default is "HttpSessions". This is the name of the database in MongoDB used to store the session collection.

jetty.session.mongo.collectionName

Default is "jettySessions". This is the name of the collection in MongoDB used to store all of the sessions.

The connection type-

You can connect to MongoDB either using a host/port combination, or a URI. By default, the host/port method is selected, but you can change this by commenting out the unwanted method, and uncommenting the other one.

connection-type=address

Used when utilizing a direct connection to the MongoDB server.

jetty.session.mongo.host

Host name or address for the remote MongoDB instance.

jetty.session.mongo.port

Port number for the remote MongoDB instance.

connection-type=uri

Used when utilizing MongoURI for secured connections.

jetty.session.mongo.connectionString

The string defining the MongoURI value, such as mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]. More information on how to format the MongoURI string can be found in the official documentation for mongo.

You will only use one connection-type at a time, either address or uri. If both are utilized in your session-store-mongo.ini, only the last connection-type configured in the file will be used.

jetty.session.gracePeriod.seconds

Integer, in seconds. Default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

Modules for Persistent HTTP Sessions: Infinispan

In order to persist/cluster sessions using Infinispan, Jetty needs to know how to contact Infinispan. There are two options: a remote Infinispan instance, or an in-process Infinispan instance. The former is referred to as "remote" Infinispan and the latter as "embedded" Infinispan. If you wish Jetty to be able to scavenge expired sessions, you will also need to enable the appropriate infinispan-[remote|embedded]-query module.

Remote Infinispan Session Module

The session-store-infinispan-remote module configures Jetty to talk to an external Infinispan instance to store session data.

Because Infinispan is not a technology provided by the Eclipse Foundation, you will be prompted to assent to the licenses of the external vendor (Apache in this case).

Infinispan-specific jar files are download to the directory named $JETTY_BASE/lib/infinispan/.

In addition to adding these modules to the classpath of the server it also added several ini configuration files to the $JETTY_BASE/start.d directory.

If you have updated versions of the jar files automatically downloaded by Jetty, you can place them in the associated $JETTY_BASE/lib/ directory and use the --skip-file-validation=<module name> command line option to prevent errors when starting your server.
Configuration

The $JETTY_BASE/start.d/session-store-infinispan-remote.ini contains the following configurable properties:

jetty.session.infinispan.remoteCacheName

Default "sessions". This is the name of the cache in Infinispan where sessions will be stored.

jetty.session.infinispan.idleTimeout.seconds

Integer, in seconds, default 0. This is the amount of time, in seconds, that a session entry in Infinispan can be idle (ie neither read nor written) before Infinispan will delete its entry. Usually, you do not want to set a value for this, as you want Jetty to manage all session expiration (and call any HttpSessionListeners). You should enable the infinispan-remote-query to allow jetty to scavenge for expired sessions. If you do not, then there is the possibility that sessions can be left in Infinispan but no longer referenced by any Jetty node (so called "zombie" or "orphan" sessions), in which case you can use this feature to ensure their removal.

You should make sure that the number of seconds you specify is larger than the configured maxIdleTime for sessions.
jetty.session.gracePeriod.seconds

Integer, default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

Remote Infinispan Query Module

The infinispan-remote-query module allows Jetty to scavenge expired sessions. Note that this is an additional module, to be used in conjunction with the session-store-infinispan-remote module.

There are no configuration properties associated with this module.

Embedded Infinispan Session Module

Enabling the session-store-infinispan-embedded module runs an in-process instance of Infinispan.

Because Infinispan is not a technology provided by the Eclipse Foundation, you will be prompted to assent to the licenses of the external vendor (Apache in this case). Infinispan-specific jar files will be downloaded and saved to a directory named $JETTY_BASE/lib/infinispan/.

If you have updated versions of the jar files automatically downloaded by Jetty, you can place them in the associated $JETTY_BASE/lib/ directory and use the --skip-file-validation=<module name> command line option to prevent errors when starting your server.
Configuration

The $JETTY_BASE/start.d/session-store-infinispan-embedded.ini contains the following configurable properties:

jetty.session.infinispan.idleTimeout.seconds

Integer, in seconds, default 0. This is the amount of time, in seconds, that a session entry in Infinispan can be idle (ie neither read nor written) before Infinispan will delete its entry. Usually, you do not want to set a value for this, as you want Jetty to manage all session expiration (and call any HttpSessionListeners). You should enable the infinispan-embedded-query to allow Jetty to scavenge for expired sessions. If you do not, then there is the possibility that expired sessions can be left in Infinispan.

You should make sure that the number of seconds you specify is larger than the configured maxIdleTime for sessions.
jetty.session.gracePeriod.seconds

Integer, default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

Embedded Infinispan Query Module

The infinispan-embedded-query module allows Jetty to scavenge expired sessions.

There are no configuration properties associated with this module.

Converting Session Format for Jetty-9.4.13

From Jetty-9.4.13 onwards, we have changed the format of the serialized session when using a remote cache (ie using hotrod). Prior to release 9.4.13 we used the default Infinispan serialization, however this was not able to store sufficient information to allow Jetty to properly deserialize session attributes in all circumstances. See issue https://github.com/eclipse/jetty.project/issues/2919 for more background.

We have provided a conversion program which will convert any sessions stored in Infinispan to the new format.

We recommend that you backup your stored sessions before running the conversion program.

How to use the converter:

java -cp jetty-jakarta-servlet-api-4.0.2.jar:jetty-util-{VERSION}.jar:jetty-server-{VERSION}.jar:infinispan-remote-9.1.0.Final.jar:jetty-infinispan-{VERSION}.jar:[other classpath]  org.eclipse.jetty.session.infinispan.InfinispanSessionLegacyConverter

Usage:  InfinispanSessionLegacyConverter [-Dhost=127.0.0.1] [-Dverbose=true|false] <cache-name> [check]
The classpath

Must contain the servlet-api, jetty-util, jetty-server, jetty-infinispan and infinispan-remote jars. If your sessions contain attributes that use application classes, you will also need to also put those classes onto the classpath. If your session has been authenticated, you may also need to include the jetty-security and jetty-http jars on the classpath.

Parameters

When used with no arguments the usage message is printed. When used with the cache-name parameter the conversion is performed. When used with both cache-name and check parameters, sessions are checked for whether or not they are converted.

-Dhost

you can optionally provide a system property with the address of your remote Infinispan server. Defaults to the localhost.

-Dverbose

defaults to false. If true, prints more comprehensive stacktrace information about failures. Useful to diagnose why a session is not converted.

cache-name

the name of the remote cache containing your sessions. This is mandatory.

check

the optional check command will verify sessions have been converted. Use it after doing the conversion.

To perform the conversion, run the InfinispanSessionLegacyConverter with just the cache-name, and optionally the host system property. The following command will attempt to convert all sessions in the cached named my-remote-cache on the machine myhost, ensuring that application classes in the /my/custom/classes directory are on the classpath:

java -cp jetty-jakarta-servlet-api-4.0.2.jar:jetty-util-{VERSION}.jar:jetty-server-{VERSION}.jar:infinispan-remote-9.1.0.Final.jar:jetty-infinispan-{VERSION}.jar:/my/custom/classes  org.eclipse.jetty.session.infinispan.InfinispanSessionLegacyConverter -Dhost=myhost my-remote-cache

If the converter fails to convert a session, an error message and stacktrace will be printed and the conversion will abort. The failed session should be untouched, however it is prudent to take a backup of your cache before attempting the conversion.

Modules for Persistent HTTP Sessions: Hazelcast

Hazelcast can be used to cluster session information in one of two modes: either remote or embedded. Remote mode means that Hazelcast will create a client to talk to other instances, possibly on other nodes. Embedded mode means that Hazelcast will start a local instance and communicate with that.

Remote Hazelcast Clustering

Enabling the session-store-hazelcast-remote module allows jetty to communicate with a remote Hazelcast instance to cluster session data.

Because Hazelcast is not a technology provided by the Eclipse Foundation, you will be prompted to assent to the licenses of the external vendor (Apache in this case).

Hazelcast-specific jar files will be downloaded and saved to a directory named $JETTY_BASE/lib/hazelcast/.

If you have updated versions of the jar files automatically downloaded by Jetty, you can place them in the associated $JETTY_BASE/lib/ directory and use the --skip-file-validation=<module name> command line option to prevent errors when starting your server.
Configuration

The start.d/session-store-hazelcast-remote.ini contains a list of all the configurable options for the Hazelcast module:

jetty.session.hazelcast.mapName

The default is "jetty-distributed-session-map". This is the name of the Map in Hazelcast where sessions will be stored.

jetty.session.hazelcast.onlyClient

Boolean, default true. The Hazelcast instance will be configured in client mode.

jetty.session.hazelcast.configurationLocation

Optional. This is the path to an external Hazelcast xml configuration file.

jetty.session.hazelcast.useQueries

Boolean, default false. If true, Jetty will use Hazelcast queries to find sessions to scavenge. If false sessions that are not currently in a session cache cannot be scavenged, and will need to be removed by some external process.

jetty.session.hazelcast.addresses

Optional. These are the addresses of remote Hazelcast instances with which to communicate.

jetty.session.gracePeriod.seconds

Integer, in seconds. Default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

Be aware that if your session attributes contain classes from inside your webapp (or Jetty classes) then you will need to put these classes onto the classpath of all of your Hazelcast instances.
Embedded Hazelcast Clustering

This will run an in-process instance of Hazelcast. This can be useful for example during testing. To enable this you enable the session-store-hazelcast-embedded module.

Because Hazelcast is not a technology provided by the Eclipse Foundation, you will be prompted to assent to the licenses of the external vendor (Apache in this case).

Hazelcast-specific jar files will be downloaded to a directory named $JETTY_BASE/lib/hazelcast/.

Configuration

The $JETTY_BASE/start.d/start.d/session-store-hazelcast-embedded.ini contains a list of all the configurable options for the Hazelcast module:

jetty.session.hazelcast.mapName

The default is "jetty-distributed-session-map". This is the name of the Map in Hazelcast where sessions will be stored. jetty.session.hazelcast.hazelcastInstanceName Default is "JETTY_DISTRIBUTED_SESSION_INSTANCE". This is the unique name of the Hazelcast instance that will be created.

jetty.session.hazelcast.configurationLocation

Optional. This is the path to an external Hazelcast xml configuration file.

jetty.session.hazelcast.useQueries

Boolean, default false'. If `true, Jetty will use Hazelcast queries to find expired sessions to scavenge. If false sessions that are not currently in a session cache cannot be scavenged, and will need to be removed by some external process.

jetty.session.gracePeriod.seconds

Integer, in seconds. Default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

If your session attributes contain classes from inside your webapp (or jetty classes) then you will need to put these classes onto the classpath of all of your hazelcast instances. In the case of embedded hazelcast, as it is started before your webapp, it will NOT have access to your webapp’s classes - you will need to extract these classes and put them onto the jetty server’s classpath.

Modules for Persistent HTTP Sessions: Google Cloud DataStore

Jetty can store http session information into GCloud by enabling the session-store-gcloud module.

Preparation

You will first need to create a project and enable the Google Cloud api: https://cloud.google.com/docs/authentication#preparation. Take note of the project id that you create in this step as you need to supply it in later steps.

Communicating with GCloudDataStore
When Running Jetty Outside of Google Infrastructure

Before running Jetty, you will need to choose one of the following methods to set up the local environment to enable remote GCloud DataStore communications.

  1. Using the GCloud SDK:

    • Ensure you have the GCloud SDK installed: https://cloud.google.com/sdk/?hl=en

    • Use the GCloud tool to set up the project you created in the preparation step: gcloud config set project PROJECT_ID

    • Use the GCloud tool to authenticate a google account associated with the project created in the preparation step: gcloud auth login ACCOUNT

  2. Using environment variables

    • Define the environment variable GCLOUD_PROJECT with the project id you created in the preparation step.

    • Generate a JSON service account key and then define the environment variable GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key.json

When Running Jetty Inside of Google Infrastructure

The Google deployment tools will automatically configure the project and authentication information for you.

Configuring Indexes for Session Data

Using some special, composite indexes can speed up session search operations, although it may make write operations slower. By default, indexes will not be used. In order to use them, you will need to manually upload a file that defines the indexes. This file is named index.yaml and you can find it in your distribution in $JETTY_BASE/etc/sessions/gcloud/index.yaml.

Follow the instructions here to upload the pre-generated index.yaml file.

Communicating with the GCloudDataStore Emulator

To enable communication using the GCloud Emulator:

  • Ensure you have the GCloud SDK installed: https://cloud.google.com/sdk/?hl=en

  • Follow the instructions here on how to start the GCloud datastore emulator, and how to propagate the environment variables that it creates to the terminal in which you run Jetty.

Enabling the Google Cloud DataStore Module

The session-store-gcloud module provides GCloud support for storing session data.

Because the Google Cloud DataStore is not a technology provided by the Eclipse Foundation, when enabling the module you will be prompted to assent to the licenses of the external vendor.

As GCloud requires certain Java Commons Logging features to work correctly, Jetty routes these through SLF4J by transitively enabling the jcl-slf4j module during installation. Therefore, you will also need to enable one of the SLF4J implementation modules. You can either choose one ahead of time and enable it at the same time as the session-store-gcloud module, or you can just enable session-store-gcloud module and it will print out a list of available SLF4J implementations. You can then choose one and enable it.

If you want to use updated versions of the jar files automatically downloaded during the module enablement, you can place them in the associated $JETTY_BASE/lib/ directory and use the --skip-file-validation=<module name> command line option to prevent errors when starting your server.

Configuration

The $JETTY_BASE/start.d/session-store-gcloud.ini file contains all of the configurable properties for the session-store-gcloud module:

jetty.session.gcloud.maxRetries

Integer. Default 5. Maximum number of retries to connect to GCloud DataStore to write a session.

jetty.session.gcloud.backoffMs

Integer in milliseconds. Default 1000. Number of milliseconds between successive attempts to connect to the GCloud DataStore to write a session.

jetty.session.gracePeriod.seconds

Integer, in seconds. Default 3600. Used during session scavenging. Multiples of this period are used to define how long ago a stored session must have expired before it should be scavenged.

jetty.session.savePeriod.seconds

Integer, in seconds, default is 0. Whenever a session is accessed by a request, its lastAccessTime and expiry are updated. Even if your sessions are read-mostly, the lastAccessTime and expiry will always change. For heavily-used, read-mostly sessions you can save some time by skipping some writes for sessions for which only these fields have changed (ie no session attributes changed). The value of this property is used to skip writes for these kinds of sessions: the session will only be written out if the time since the last write exceeds the value of this property.

You should be careful in the use of this property in clustered environments: if you set too large a value for this property, the session may not be written out sufficiently often to update its expiry time thus making it appear to other nodes that it has expired. Thorough consideration of the maxIdleTime of the session when setting the savePeriod is imperative - it would be undesirable to set a savePeriod that is larger than the maxIdleTime.

jetty.session.gcloud.namespace

Optional. Sets the namespace for GCloud Datastore to use. If set, partitions the visibility of session data between webapps, which is helpful for multi-tenant deployments. More information can be found here.

Configuration of the stored session object and its fields names-

You should very rarely, if ever, need to change these defaults.

jetty.session.gcloud.model.kind

The default is "GCloudSession". This is the type of the object that is stored in GCloud.

jetty.session.gcloud.model.id

The default is "id". This is the session id.

jetty.session.gcloud.model.contextPath

The default is "contextPath". This is the canonicalized context path of the context to which the session belongs.

jetty.session.gcloud.model.vhost

The default is "vhost". This is the canonicalized virtual host of the context to which the session belongs.

jetty.session.gcloud.model.accessed

The default is "accessed". This is the current access time of the session.

jetty.session.gcloud.model.lastAccessed

The default is "lastAccessed". This is the last access time of the session.

jetty.session.gcloud.model.createTime

The default is "createTime". This is the time, in ms since the epoch, at which the session was created.

jetty.session.gcloud.model.cookieSetTime

The default is "cookieSetTime". This is the time at which the session cookie was last set.

jetty.session.gcloud.model.lastNode

The default is "lastNode". This is the workerName of the last node to manage the session.

jetty.session.gcloud.model.expiry

The default is "expiry". This is the time, in ms since the epoch, at which the session will expire.

jetty.session.gcloud.model.maxInactive

The default is "maxInactive". This is the session timeout in ms.

jetty.session.gcloud.model.attributes

The default is "attributes". This is a map of all the session attributes.

Modules for Persistent HTTP Sessions: The L2 Session Data Cache

If your chosen persistence technology is slow, it can be helpful to locally cache the session data. The CachingSessionDataStore is a special type of SessionDataStore that locally caches session data, which makes reads faster. It writes-through to your chosen type of SessionDataStore when session data changes.

MemcachedSessionDataMap

The MemcachedSessionDataMap uses memcached to perform caching of SessionData.

To enable it with the Jetty distribution, enable the session-store-cache module, along with your chosen session-store-xxxx module.

Configuration

The $JETTY_BASE/start.d/session-store-cache.ini contains the following configurable properties:

jetty.session.memcached.host

Default value is localhost. This is the host on which the memcached server resides.

jetty.session.memcached.port

Default value is 11211. This is the port on which the memcached server is listening.

jetty.session.memcached.expirySec

Default value 0. This is the length of time in seconds that an item can remain in the memcached cache, where 0 indicates indefinitely.

jetty.session.memcached.heartbeats

Default value true. Whether the memcached system should generate heartbeats.

Session Scenarios

Minimizing Support for Sessions

The standard support for webapps in Jetty will use sessions cached in memory, but not persisted/clustered, with a scavenge for expired sessions that occurs every 10 minutes. If you wish to pare back support for sessions because you know your app doesn’t use them (or use JSPs that use them), then you can do the following:

If you wish to do any further minimization, you should consult the Programming Guide.

Clustering with a Sticky Load Balancer

Preferably, your cluster will utilize a sticky load balancer. This will route requests for the same session to the same Jetty instance. In this case, the DefaultSessionCache can be used to keep in-use session objects in memory. You can fine-tune the cache by controlling how long session objects remain in memory with the eviction policy settings.

If you have a large number of sessions or very large session objects, then you may want to manage your memory allocation by controlling the amount of time session objects spend in the cache. The EVICT_ON_SESSION_EXIT eviction policy will remove a session object from the cache as soon as the last simultaneous request referencing it exits. Alternatively, the EVICT_ON_INACTIVITY policy will remove a session object from the cache after a configurable amount of time has passed without a request referencing it.

If your sessions are very long lived and infrequently referenced, you might use the EVICT_ON_INACTIVITY_POLICY to control the size of the cache.

If your sessions are small, or relatively few or stable in number or they are read-mostly, then you might select the NEVER_EVICT policy. With this policy, session objects will remain in the cache until they either expire or are explicitly invalidated.

If you have a high likelihood of simultaneous requests for the same session object, then the EVICT_ON_SESSION_EXIT policy will ensure the session object stays in the cache as long as it is needed.

Clustering Without a Sticky Load Balancer

Without a sticky load balancer requests for the same session may arrive on any node in the cluster. This means it is likely that the copy of the session object in any SessionCache is likely to be out-of-date, as the session was probably last accessed on a different node. In this case, your choices are to use either the NullSessionCache or to de-tune the DefaultSessionCache. If you use the NullSessionCache all session object caching is avoided. This means that every time a request references a session it must be read in from persistent storage. It also means that there can be no sharing of session objects for multiple requests for the same session: each will have their own independent session object. Furthermore, the outcome of session writes are indeterminate because the Servlet Specification does not mandate ACID transactions for sessions.

If you use the DefaultSessionCache, there is a risk that the caches on some nodes will contain out-of-date session information as simultaneous requests for the same session are scattered over the cluster. To mitigate this somewhat you can use the EVICT_ON_SESSION_EXIT eviction policy: this will ensure that the session is removed from the cache as soon as the last simultaneous request for it exits. Again, due to the lack of session transactionality, the ordering outcome of write operations cannot be guaranteed. As the session is cached while at least one request is accessing it, it is possible for multiple simultaneous requests to share the same session object.

Handling Corrupted or Unreadable Session Data

For various reasons it might not be possible for the SessionDataStore to re-read a stored session. One scenario is that the session stores a serialized object in its attributes, and after a re-deployment there in an incompatible class change. Setting the $JETTY_BASE/start.d/session-cache-hash.ini or $JETTY_BASE/start.d/session-cache-null.ini property jetty.session.removeUnloadableSessions to true will allow the unreadable session to be removed from persistent storage. This can be useful for preventing the scavenger from continually generating errors on the same expired, but un-readable session.

Faster Web Application Deployment

The auto discovery features of the Servlet Specification can make deployments slow and uncertain. Auto discovery of web application configuration can be useful during the development as it allows new features and frameworks to be enabled simply by dropping in a jar file. However for production deployment, the need to scan the contents of many jars can have a significant impact at startup time.

The quickstart module allows a webapp to be pre-scanned, making startup predictable and faster. During scanning all declarative configuration (ie from web.xml, web-fragment.xml and annotations) are encoded into an effective web.xml, called WEB-INF/quickstart-web.xml, which can be inspected to understand what will be deployed. NOTE:: Programmatic configuration is not encoded into the generated quickstart-web.xml file.

With quickstart, webapps that took many seconds to scan and deploy can now be deployed in a few hundred milliseconds.

Enabling

Enable the quickstart module for your jetty base:

$ cd $JETTY-BASE
$ java -jar $JETTY_HOME/start.jar --add-module=quickstart

The $JETTY-BASE/start.d/quickstart.ini file contains these configurable parameters:

jetty.quickstart.mode

The values are:

AUTO

Allows jetty to run either with or without a quickstart-web.xml file. If jetty detects the file, then it will be used, otherwise the app is started normally.

GENERATE

In this mode, jetty will generate a quickstart-web.xml file and then terminate. Use this mode first before changing to either AUTO or QUICKSTART.

QUICKSTART

In this mode, if jetty does not detect a quickstart-web.xml file then jetty will not start.

jetty.quickstart.origin

Use this parameter to set the name of the attribute in the quickstart-web.xml file that contains the origin of each element. Knowing the descriptor or annotation from which each element derives can be useful for debugging. Note that the origin attribute does not conform to the web xml schema, so if you deploy with xml validation, you’ll see errors. It is probably best to do a few trial runs with the attribute set, then turn it off for final generation.

jetty.quickstart.xml

Use this parameter to change the name of the generated file. By default this is quickstart-web.xml in the webapp’s WEB-INF directory. The file named by this parameter will always be interpreted relative to WEB-INF.

If your webapp is a war file, you will need to either first unpack it yourself, or use a context xml file (or code equivalent) that calls WebAppContext.setExtractWAR(true). If you allow Jetty to do the unpacking, it will use the usual mechanisms to find the location to which to unpack. Note that by default Jetty unpacks to a temporary location which is not reused between executions. So either specify the directory to which to unpack, or make a work directory in your base to ensure the unpacked war is preserved and reused across restarts.

Annotations

Enable the annotations module if your webapp - or any of its third party libraries - uses any of the following:

  • Annotations:

    • @Resource

    • @Resources

    • @PostConstruct

    • @PreDestroy

    • @DeclaredRoles

    • @RunAs

    • @MultipartConfig

    • @WebServlet

    • @WebFilter

    • @WebListener

    • @WebInitParam

    • @ServletSecurity, @HttpConstraint, @HttpMethodConstraint

    • @HandlesTypes

  • javax.servlet.ServletContainerInitializers

  • JSP

Annotation Scanning

According to more recent versions of the Servlet Specification, the web.xml file can contain the attribute metadata-complete. If this is set to true, then no annotation scanning takes place, and your descriptor must contain the equivalent xml statements of any annotations.

If it is metadata-complete=false, or your web.xml predates the inclusion of this attribute, annotation scanning is required to take place.

To prevent annotation scanning you can use the WebAppContext.setConfigurationDiscovered(false) method. Here’s an example context xml file that calls this method:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
  <Set name="configurationDiscovered">false</Set> (2)
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies that scanning should not take place.

However, despite metadata-complete=true, scanning of classes may still occur because of javax.servlet.ServletContainerInitializers. Classes implementing this interface are found by Jetty using the javax.util.ServiceLoader mechanism, and if one is present and it includes the @HandlesTypes annotation, then Jetty must scan the class hierarchy of the web application. This may be very time-consuming if you have many jars.

We will now look at ways to limit the jars that are scanned.

The container classpath

By default, Jetty will not scan any classes that are on the container’s classpath.

Sometimes, you may have third party libraries on the container’s classpath that you need to be scanned. In this case, use the org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern context attribute to define which container jars and class directories to scan. The value of this attribute is a regular expression.

Here’s an example from a context xml file that includes any jar whose name starts with "foo-" or "bar-", or a directory named "classes":

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
    <Call name="setAttribute"> (2)
      <Arg>org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern</Arg> (3)
      <Arg>.*/foo-[^/]*\.jar$|.*/bar-[^/]*\.jar$|.*/classes/.*</Arg> (4)
    </Call>
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies a context attribute.
3 Specifies the name of the context attribute.
4 Specifies the value of the context attribute.

Note that the order of the patterns defines the ordering of the scanning of the jars or class directories.

The webapp classpath

By default Jetty will scan all classes from WEB-INF/classes and all jars from WEB-INF/lib according to the order, if any, established by absolute or relative ordering clauses in web.xml.

If your webapp contains many jar files that you know do not contain any annotations, you can significantly speed up deployment by omitting them from scanning. However, be careful if your webapp uses a ServletContainerInitializer with an @HandlesTypes annotation that you don’t exclude jars that contain classes matching the annotation.

Use the org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern context attribute to define a regular expression for jars and class directories to select for scanning.

Here’s an example of a context xml file that sets a pattern that matches any jar on the webapp’s classpath that starts with spring-:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
    <Call name="setAttribute">  (2)
      <Arg>org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern</Arg>  (3)
      <Arg>.*/spring-[^/]*\.jar$</Arg>  (4)
    </Call>
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies a context attribute.
3 Specifies the name of the context attribute.
4 Specifies the value of the context attribute.
Multi-threading

By default Jetty performs annotation scanning in a multi-threaded manner in order to complete it in the minimum amount of time.

If you don’t want multi-threaded scanning, you can configure Jetty to revert to single-threaded scanning. There are several options to configure this:

  1. Set the context attribute org.eclipse.jetty.annotations.multiThreaded to false

  2. Set the Server attribute org.eclipse.jetty.annotations.multiThreaded to false

  3. Set the System property org.eclipse.jetty.annotations.multiThreaded to false

Method 1 will only affect the current webapp. Method 2 will affect all webapps deployed to the same Server instance. Method 3 will affect all webapps deployed in the same JVM.

By default, Jetty will wait a maximum of 60 seconds for all of the scanning threads to complete. You can set this to a higher or lower number of seconds by doing one of the following:

  1. Set the context attribute org.eclipse.jetty.annotations.maxWait

  2. Set the Server attribute org.eclipse.jetty.annotations.maxWait

  3. Set the System property org.eclipse.jetty.annotations.maxWait

Method 1 will only affect the current webapp. Method 2 will affect all webapps deployed to the same Server instance. Method 3 will affect all webapps deployed in the same JVM.

ServletContainerInitializers

The javax.servlet.ServletContainerInitializer class can exist in: the container’s classpath, the webapp’s WEB-INF/classes directory, the webapp’s WEB-INF/lib jars, or any external extraClasspath that you have configured on the webapp.

The Servlet Specification does not define any order in which a ServletContainerInitializer must be called when the webapp starts. By default Jetty will call them in the following order:

  1. ServletContainerInitializers from the container’s classpath

  2. ServletContainerInitializers from WEB-INF/classes

  3. ServletContainerInitializers from WEB-INF/lib jars in the order established in web.xml, or in the order that the SCI is returned by the javax.util.ServiceLoader if there is no ordering.

Exclusions

By default, as according to the Servlet Specification, all ServletContainerInitializer that are discovered are invoked.

Sometimes, depending on your requirements, you may need to prevent some being called at all.

In this case, you can define the org.eclipse.jetty.containerInitializerExclusionPattern context attribute.

This is a regular expression that defines patterns of classnames that you want to exclude. Here’s an example of setting the context attribute in a context xml file:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
    <Call name="setAttribute">  (2)
      <Arg>org.eclipse.jetty.containerInitializerExclusionPattern</Arg>  (3)
      <Arg>com.acme.*|com.corp.SlowContainerInitializer</Arg>  (4)
    </Call>
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies a context attribute.
3 Specifies the name of the context attribute.
4 Specifies the value of the context attribute.

In this example we exclude all ServletContainerInitializer instances in the com.acme package, and the specific class com.corp.SlowContainerInitializer.

It is possible to use exclusion and ordering together to control ServletContainerInitializer invocation - the exclusions will be applied before the ordering.

Ordering

If you need ServletContainerInitializer classes called in a specific order, you can use the context attribute org.eclipse.jetty.containerInitializerOrder. Set it to a list of comma separated class names of ServletContainerInitializers in the order that you want them applied.

You may optionally use the wildcard character * once in the list. It will match all ServletContainerInitializer classes not explicitly named in the list.

Here is an example context xml file that ensures the com.example.PrioritySCI will be called first, followed by the com.acme.FooSCI, then all other SCIs:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
    <Call name="setAttribute">  (2)
      <Arg>org.eclipse.jetty.containerInitializerOrder</Arg>  (3)
      <Arg>org.eclipse.jetty.websocket.javax.server.JavaxWebSocketServletContainerInitializer, com.acme.FooSCI, *</Arg>  (4)
    </Call>
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies a context attribute.
3 Specifies the name of the context attribute.
4 Specifies the value of the context attribute.

Java Server Pages

Jetty supports JSP via the jsp module, which is based on Apache Jasper:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Enables JSP for all web applications deployed on the server.

[depend]
servlet
annotations
apache-jsp

Logging has been bridged to Jetty logging, so you can enable logging for the org.apache.jasper package, subpackages and classes as usual.

Configuration of the JSP servlet

The org.eclipse.jetty.jsp.JettyJspServlet is the servlet responsible for serving jsps.

It is configured as the default jsp servlet in the webdefault.xml file. Notice that Jetty identifies the jsp servlet by the presence of the id=jsp attribute in the <servlet> declaration.

That file maps the org.eclipse.jetty.jsp.JettyJspServlet to the following partial urls:

  • *.jsp

  • *.jspf

  • *.jspx

  • *.xsp

  • *.JSP

  • *.JSPF

  • *.JSPX

  • *.XSP

You can change to a different servlet, change or add <init-param>s or add extra <servlet-mapping>s in your web.xml file.

Here’s an example of adding an <init-param> to augment the definitions from the standard webdefault.xml file:

 <servlet id="jsp">  (1)
     <servlet-name>jsp</servlet-name> (2)
     <init-param>
         <param-name>keepgenerated</param-name> (3)
         <param-value>true</param-value>  (4)
     </init-param>
   </servlet>
1 This identifies this servlet as the jsp servlet to Jetty.
2 This identifies this declaration as augmenting the already-defined servlet called jsp.
3 This init param controls whether the jsp servlet retains the *.java files generated during jsp compilation.
4 This sets the value of the init param

Another element you might consider adding to the default setup is async-supported:

<servlet id="jsp">  (1)
  <servlet-name>jsp</servlet-name>  (2)
  <async-supported>true</async-supported>  (3)
</servlet>
1 This identifies this servlet as the jsp servlet to Jetty.
2 This identifies this declaration as augmenting the already-defined servlet called jsp.
3 By default, the jsp servlet does not support async.

There are many configuration parameters for the Apache Jasper jsp servlet, here are some of them:

Table 4. Jsp Servlet Parameters
init param Description Default webdefault.xml

checkInterval

If non-zero and development is false, background jsp recompilation is enabled. This value is the interval in seconds between background recompile checks.

0

classpath

The classpath is dynamically generated if the context has a URL classloader. The org.apache.catalina.jsp_classpath context attribute is used to add to the classpath, but if this is not set, this classpath configuration item is added to the classpath instead.`

-

classdebuginfo

Include debugging info in class file.

TRUE

compilerClassName

If not set, defaults to the Eclipse jdt compiler.

-

compiler

Used if the Eclipse jdt compiler cannot be found on the classpath. It is the classname of a compiler that Ant should invoke.

compilerTargetVM

Target vm to compile for.

1.8

1.8

compilerSourceVM

Sets source compliance level for the jdt compiler.

1.8

1.8

development

If true recompilation checks occur at the frequency governed by modificationTestInterval.

TRUE

displaySourceFragment

Should a source fragment be included in exception messages

TRUE

dumpSmap

Dump SMAP JSR45 info to a file.

FALSE

enablePooling

Determines whether tag handler pooling is enabled.

TRUE

engineOptionsClass

Allows specifying the Options class used to configure Jasper. If not present, the default EmbeddedServletOptions will be used.

-

errorOnUseBeanInvalidClassAttribute

Should Jasper issue an error when the value of the class attribute in an useBean action is not a valid bean class

TRUE

fork

Only relevant if you use Ant to compile jsps: by default Jetty will use the Eclipse jdt compiler.

TRUE

-

genStrAsCharArray

Option for generating Strings as char arrays.

FALSE

ieClassId

The class-id value to be sent to Internet Explorer when using <jsp:plugin> tags.

clsid:8AD9C840-044E-11D1-B3E9-00805F499D93

javaEncoding

Pass through the encoding to use for the compilation.

UTF8

jspIdleTimeout

The amount of time in seconds a JSP can be idle before it is unloaded. A value of zero or less indicates never unload.

-1

keepgenerated

Do you want to keep the generated Java files around?

TRUE

mappedFile

Support for mapped Files. Generates a servlet that has a print statement per line of the JSP file 

TRUE

maxLoadedJsps

The maximum number of JSPs that will be loaded for a web application. If more than this number of JSPs are loaded, the least recently used JSPs will be unloaded so that the number of JSPs loaded at any one time does not exceed this limit. A value of zero or less indicates no limit.

-1

modificationTestInterval

If development=true, interval between recompilation checks, triggered by a request.

4

quoteAttributeEL

When EL is used in an attribute value on a JSP page, should the rules for quoting of attributes described in JSP.1.6 be applied to the expression

TRUE

-

recompileOnFail

If a JSP compilation fails should the modificationTestInterval be ignored and the next access trigger a re-compilation attempt? Used in development mode only and is disabled by default as compilation may be expensive and could lead to excessive resource usage.

FALSE

scratchDir

Directory where servlets are generated. The default is the value of the context attribute javax.servlet.context.tempdir, or the system property java.io.tmpdir if the context attribute is not set.

strictQuoteEscaping

Should the quote escaping required by section JSP.1.6 of the JSP specification be applied to scriplet expression.

TRUE

-

suppressSmap

Generation of SMAP info for JSR45 debugging.

FALSE

trimSpaces

Should template text that consists entirely of whitespace be removed from the output (true), replaced with a single space (single) or left unchanged (false)? Note that if a JSP page or tag file specifies a trimDirectiveWhitespaces value of true, that will take precedence over this configuration setting for that page/tag. trimmed?

FALSE

xpoweredBy

Generate an X-Powered-By response header.

FALSE

FALSE

NOTE

If the value you set doesn’t take effect, try using all lower case instead of camel case, or capitalizing only some of the words in the name, as Jasper is inconsistent in its parameter naming strategy.

JavaServer Pages Standard Tag Libraries

The JavaServer Pages Standlard Tag Library (JSTL) is part of the Jetty distribution, and is available via the jstl module:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Enables JSTL for all web applications deployed on the server.

[depend]
jsp
apache-jstl

When enabled, Jetty will make the JSTL tags available for your webapps.

JavaServer Faces Taglibs

If you want to use JSF with your webapp, you should copy the relevant jars from your implementation of choice into your $jetty.base directory, ideally into $jetty.base/lib/ext. If that directory does not exist, enable the ext module, which will create the directory and ensure all jars within it are put onto the container classpath.

Then you will need to tell Jetty which of those jars contains the *.tld files. To accomplish that, you need to specify either the name of the file or a pattern that matches the name/s of the file/s as the org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern context attribute. You will need to preserve the existing value of the attribute, and add in your extra pattern.

Here’s an example of using a context xml file to add in a pattern to match files starting with jsf-, which contain the *.tld files:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure class="org.eclipse.jetty.webapp.WebAppContext"> (1)
    <Call name="setAttribute"> (2)
      <Arg>org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern</Arg> (3)
      <Arg.*/jetty-servlet-api-[^/]*\.jar$|.*/javax.servlet.jsp.jstl-.*\.jar$|.*/org.apache.taglibs.taglibs-standard-impl-.*\.jar$|.*/jsf-[^/]*\.jar$></Arg> (4)
    </Call>
</Configure>
1 Configures a WebAppContext, which is the Jetty component that represents a standard Servlet web application.
2 Specifies a context attribute.
3 Specifies the name of the context attribute.
4 Adds the additional pattern .*/jsf-[^/]*\.jar$ to those already existing.

JNDI

Enable the plus module in order to be able to use JNDI resources in your webapp. If you already have the annotations module enabled, then it will already be enabled.

If you have extra jars associated with your JNDI resources, eg database drivers etc, that are not located inside your webapp then you should place those jars into your $jetty.base/lib/ext directory. If your base doesn’t already contain this directory, then enable the ext module, and Jetty will create the directory for you and ensure its contents are on the server’s classpath.

You can now declare JNDI resources and reference them within your webapps.

Declaring resources

You must declare the objects you want bound into the environment so that you can then hook them into your webapp via env-entry, resource-ref and resource-env-refs in web.xml, web-fragment.xml or override-web.xml.

You make these declarations in Jetty XML files that are either external or internal to your webapp. A server or context XML file is external to your webapp. The special WEB-INF/jetty-env.xml file is internal to your webapp. See the section on Jetty XML files for more information on how to choose in which XML file to place your declarations.

For now, let’s look at what you declare in the XML file, regardless of its location.

Declaring a JDNI resource to be referenced later in your webapp is accomplished by declaring new instances of the following types:

org.eclipse.jetty.plus.jndi.EnvEntry

Used for env-entry type of entries

org.eclipse.jetty.plus.jndi.Resource

Used for most other type of resources

org.eclipse.jetty.plus.jndi.Transaction

For a JTA manager

org.eclipse.jetty.plus.jndi.Link

For the link between a web.xml resource name and a naming entry

Declarations of each of these types follow a similar pattern:

<New class="org.eclipse.jetty.plus.jndi.xxxx"> (1)
  <Arg><!-- scope --></Arg> (2)
  <Arg><!-- name --></Arg>  (3)
  <Arg><!-- value --></Arg> (4)
</New>
1 Defines a resource to Jetty.
2 Specifies the scope of the resource.
3 Specifies the name of the resource which will be looked up by the webapp relative to the java:comp/env namespace.
4 Specifies the value of the resource.
org.eclipse.jetty.plus.jndi.EnvEntry

Sometimes it is useful to pass configuration information to a webapp at runtime that you either cannot or cannot conveniently code into a web.xml <env-entry>. In such cases, you can use the org.eclipse.jetty.plus.jndi.EnvEntry class, and optionally even override an entry of the same name in web.xml.

Here’s an example that defines the equivalent of an env-entry called mySpecialValue with value 4000 that overrides an <env-entry> declaration of the same name in web.xml:

<New class="org.eclipse.jetty.plus.jndi.EnvEntry"> (1)
  <Arg></Arg> (2)
  <Arg>mySpecialValue</Arg> (3)
  <Arg type="java.lang.Integer">4000</Arg> (4)
  <Arg type="boolean">true</Arg> (5)
</New>
1 Define an EnvEntry that corresponds to an <env-entry>.
2 Scoped at the JVM level.
3 The name of the entry, corresponding to a lookup by the webapp of java:comp/env/mySpecialValue.
4 The value of the entry, in this case the integer value 4000.
5 true means to override the value of an <env-entry> of the same name in web.xml.

Note that if you don’t want to override the web.xml value, simply omit the last argument, or set it to false.

The Servlet Specification allows binding only the following object types to an env-entry:

  • java.lang.String

  • java.lang.Integer

  • java.lang.Float

  • java.lang.Double

  • java.lang.Long

  • java.lang.Short

  • java.lang.Character

  • java.lang.Byte

  • java.lang.Boolean

Jetty is a little more flexible and allows you to also bind:

Be aware that if you take advantage of this feature, your web application is not portable.

org.eclipse.jetty.plus.jndi.Resource

You can configure any type of resource that you want to refer to in web.xml via a resource-ref or resource-env-ref by using the org.eclipse.jetty.plus.jndi.Resource type of naming entry.

You provide the scope, the name of the object (relative to java:comp/env) and a POJO, javax.naming.Reference or javax.naming.Referenceable instance.

Let’s examine how to configure some of the most common types of resources.

DataSources

In this example, we’ll configure a Derby DataSource named jdbc/myds:

<Configure id='wac' class="org.eclipse.jetty.webapp.WebAppContext">
  <New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg><Ref refid="wac"/></Arg>
    <Arg>jdbc/myds</Arg>
    <Arg>
      <New class="org.apache.derby.jdbc.EmbeddedDataSource">
        <Set name="DatabaseName">test</Set>
        <Set name="createDatabase">create</Set>
      </New>
    </Arg>
  </New>
</Configure>

This would linked into the webapps JNDI namespace via an entry in a web.xml like so:

<resource-ref>
  <res-ref-name>jdbc/myds</res-ref-name>
  <res-type>javax.sql.DataSource</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

When configuring Resources, ensure that the type of object you configure matches the type of object you expect to look up in java:comp/env. For database connection factories, this means that the object you register as a Resource must implement the javax.sql.DataSource interface.

Also note that the J2EE Specification recommends storing DataSources relative to jdbc/ and thus looked up by the application as java:comp/env/jdbc/xxx. Eg The Datasource bound in Jetty as jdbc/users would be looked up by the application as java:comp/env/jdbc/users

JMS Queues, Topics and ConnectionFactories

Jetty can bind any implementation of the JMS destinations and connection factories.

Here is an example of binding an ActiveMQ in-JVM connection factory:

<Configure id='wac' class="org.eclipse.jetty.webapp.WebAppContext">
  <New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg><Ref refid='wac'/></Arg>
    <Arg>jms/connectionFactory</Arg>
    <Arg>
      <New class="org.apache.activemq.ActiveMQConnectionFactory">
        <Arg>vm://localhost?broker.persistent=false</Arg>
      </New>
    </Arg>
  </New>
</Configure>

The corresponding entry in web.xml to bind the ConnectionFactory into the webapp’s JNDI namespace would be:

<resource-ref>
  <res-ref-name>jms/connectionFactory</res-ref-name>
  <res-type>javax.jms.ConnectionFactory</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

The J2EE Specification recommends storing JMS connection factories under jms/. Eg The ConnectionFactory bound in Jetty as jms/inqueue would be looked up by the application as java:comp/env/jms/inqueue.

Mail

To configure access to javax.mail.Session from within a webapp, declare an org.eclipse.jetty.plus.jndi.Resource with an org.eclipse.jetty.jndi.factories.MailSessionReference that will hold the mail configuration and create the instance of the Session when it is referenced:

<Configure id='wac' class="org.eclipse.jetty.webapp.WebAppContext">
  <New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg><Ref refid="wac"/></Arg>
    <Arg>mail/Session</Arg>
    <Arg>
      <New class="org.eclipse.jetty.jndi.factories.MailSessionReference"> (1)
        <Set name="user">fred</Set> (2)
        <Set name="password">OBF:1xmk1w261z0f1w1c1xmq</Set> (3)
        <Set name="properties"> (4)
          <New class="java.util.Properties">
            <Put name="mail.smtp.host">XXX</Put>
            <Put name="mail.from">me@me</Put>
            <Put name="mail.debug">true</Put>
          </New>
        </Set>
      </New>
    </Arg>
  </New>
</Configure>
1 Use the org.eclipse.jetty.jndi.factories.MailSessionReference class to hold the configuration.
2 Set the username for the mail instance.
3 Set the password for the mail instance - use Jetty’s secure password obfuscation to obscure the password.
4 Set all other applicable properties.

The webapp performs a lookup for java:comp/env/mail/Session at runtime and obtains a javax.mail.Session that has the correct configuration to permit it to send email via SMTP.

Jetty does not provide the javax.mail and javax.activation jars.

Note also that the J2EE Specification recommends storing JavaMail connection factories under mail/. Eg The MailSessionReference bound to jetty as mail/smtp would be looked up by the application as java:comp/env/mail/smtp.

org.eclipse.jetty.plus.jndi.Transaction

To perform distributed transactions with your resources, a transaction manager that supports the JTA interfaces is required. The transaction manager is looked up by the application as java:comp/UserTransaction.

Jetty does not ship with a JTA manager, but does provide the infrastructure to plug in the JTA manager of your choice.

Use the org.eclipse.jetty.plus.jndi.Transaction object in a Jetty XML file to configure the JTA manager.

The following example configures the Atomikos transaction manager:

<New id="tx" class="org.eclipse.jetty.plus.jndi.Transaction">
  <Arg>
    <New class="com.atomikos.icatch.jta.J2eeUserTransaction"/>
  </Arg>
</New>

Jetty will automatically bind this JTA manager to the webapp’s JNDI namespace at java:comp/UserTransaction.

Usually, the name you provide for the org.eclipse.jetty.plus.jndi.Resource is the same name you reference in web.xml. This ensures that the two are linked together and thus accessible to your webapp.

However, if the names cannot be the same, then it is possible to effectively alias one to another using an org.eclipse.jetty.plus.jndi.Link.

Let’s look at an example.

Supposing you have a declaration for a Datasource named jdbc/workforce in a Jetty context XML file, but your web.xml wants to link to a <resource-ref> named jdbc/employees, and you cannot edit the web.xml. You can create a WEB-INF/jetty-env.xml file with an org.eclipse.jetty.plus.jndi.Link that ties together the names jdbc/workforce and jdbc/employees:

The context XML file declares jdbc/workforce:

<Configure id='wac' class="org.eclipse.jetty.webapp.WebAppContext">
  <New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg><Ref refid="wac"/></Arg>
    <Arg>jdbc/workforce</Arg>
    <Arg>
      <New class="org.apache.derby.jdbc.EmbeddedDataSource">
        <Set name="DatabaseName">test</Set>
        <Set name="createDatabase">create</Set>
      </New>
    </Arg>
  </New>
</Configure>

The web.xml refers to it as jdbc/employees:

<resource-ref>
  <res-ref-name>jdbc/employees</res-ref-name>
  <res-type>javax.sql.DataSource</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

Create a WEB-INF/jetty-env.xml file with a org.eclipse.jetty.plus.jndi.Link to link these names together:

<New class="org.eclipse.jetty.plus.jndi.Link">
  <Arg><Ref refid='wac'/></Arg>
  <Arg>jdbc/employees</Arg> (1)
  <Arg>jdbc/workforce</Arg>  (2)
</New>
1 The name as referenced in the web.xml file.
2 The name as referenced in the context XML file.
Jetty XML files

You can define naming resources in three places:

Server XML file

Naming resources defined in a server XML file are scoped at either the JVM level or the org.eclipse.jetty.server.Server level. The classes for the resource must be visible at the Jetty container level. If instead the classes for the resource only exist inside your webapp, you must declare it in a WEB-INF/jetty-env.xml file.

Context XML file

Entries in a context XML file should be scoped at the level of the webapp to which they apply (although it is possible to use a less strict scoping level of Server or JVM, but not recommended). As with resources declared in a server XML file, classes associated with the resource must be visible on the container’s classpath.

WEB-INF/jetty-env.xml

Naming resources in a WEB-INF/jetty-env.xml file are scoped to the webapp in which the file resides. While you can enter JVM or Server scopes if you choose, we do not recommend doing so. The resources defined here may use classes from inside your webapp.

Resource scoping

Naming resources within Jetty belong to one of three different scopes, in increasing order of restrictiveness:

JVM scope: The name is unique across the JVM instance, and is visible to all application code. This scope is represented by a null first parameter to the resource declaration. For example:

<New id="cf" class="org.eclipse.jetty.plus.jndi.Resource">
  <Arg></Arg>  (1)
  <Arg>jms/connectionFactory</Arg>
  <Arg>
    <New class="org.apache.activemq.ActiveMQConnectionFactory">
       <Arg>vm://localhost?broker.persistent=false</Arg>
    </New>
  </Arg>
</New>
1 Empty first arg equates to JVM scope for the object bound to name jms/connectionFactory.

Server scope: The name is unique to a Server instance, and is only visible to applications associated with that instance. This scope is represented by referencing the Server instance as the first parameter to the resource declaration. For example:

<New id="cf" class="org.eclipse.jetty.plus.jndi.Resource">
  <Arg><Ref refid="Server"/></Arg>  (1)
  <Arg>jms/connectionFactory</Arg>
  <Arg>
    <New class="org.apache.activemq.ActiveMQConnectionFactory">
      <Arg>vm://localhost?broker.persistent=false</Arg>
    </New>
  </Arg>
</New>
1 We refer to the id Server which identifies the default org.eclipse.jetty.server.Server instance.

Webapp scope: The name is unique to the org.eclipse.jetty.webapp.WebAppContext instance, and is only visible to that application. This scope is represented by referencing the instance as the first parameter to the resource declaration. For example:

<New class="org.eclipse.jetty.plus.jndi.Resource">
  <Arg><Ref refid='wac'/></Arg> (1)
  <Arg>jms/connectionFactory</Arg>
  <Arg>
    <New class="org.apache.activemq.ActiveMQConnectionFactory">
      <Arg>vm://localhost?broker.persistent=false</Arg>
    </New>
  </Arg>
</New>
1 We refer to an instance of an org.eclipse.jetty.webapp.WebAppContext which has been previously defined.

JAAS

JAAS implements a Java version of the standard Pluggable Authentication Module (PAM) framework.

JAAS can be used for two purposes:

  • for authentication of users, to reliably and securely determine who is currently executing Java code, regardless of whether the code is running as an application, an applet, a bean, or a servlet

  • for authorization of users to ensure they have the access control rights (permissions) required to do the actions performed

JAAS authentication is performed in a pluggable fashion. This permits applications to remain independent from underlying authentication technologies. New or updated authentication technologies can be plugged under an application without requiring modifications to the application itself.

See Java Authentication and Authorization Service (JAAS) Reference Guide for more information about JAAS.

The Jetty JAAS support aims to dictate as little as possible whilst providing a sufficiently flexible infrastructure to allow users to drop either one of the JAAS Login Modules that ships with Jetty, or their own custom LoginModules.

Configuration

The jaas module

Enable the jaas module:

# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html

[description]
Enables JAAS for deployed web applications.

[depend]
server

[lib]
lib/jetty-jaas-${jetty.version}.jar

[xml]
etc/jetty-jaas.xml

[ini-template]
## The file location (relative to $jetty.base) for the
## JAAS "java.security.auth.login.config" system property
# jetty.jaas.login.conf=etc/login.conf

The configurable items in the resulting $jetty.base/start.d/jaas.ini file are:

jetty.jaas.login.conf

This is the location of the file that will be referenced by the System property java.security.auth.login.config: Jetty sets this System property for you based on the value of this property. The value of this property is assumed to be relative to the $jetty.base. The default value is etc/login.conf, which resolves to $jetty.base/etc/login.conf. If you don’t want to put your login module configuration file here, you can change this property to point to where it is.

See more about the contents of this file in the Configuring JAAS section.

Configure the webapp for JAAS

The <realm-name> in web.xml will be used to identify the org.eclipse.jetty.jaas.JAASLoginService declaration that integrates JAAS with Jetty.

For example, this web.xml contains a realm called Test JAAS Realm:

<login-config>
  <auth-method>FORM</auth-method>
  <realm-name>Test JAAS Realm</realm-name> (1)
  <form-login-config>
    <form-login-page>/login/login</form-login-page>
    <form-error-page>/login/error</form-error-page>
  </form-login-config>
</login-config>
1 The name of the realm, which must be identical to the name of an org.eclipse.jetty.jaas.JAASLoginService declaration.

We now need to declare an org.eclipse.jetty.jaas.JAASLoginService that references the realm name of Test JAAS Realm. Here’s an example of a suitable XML snippet:

<New class="org.eclipse.jetty.jaas.JAASLoginService">
  <Set name="Name">Test JAAS Realm</Set> (1)
  <Set name="LoginModuleName">xyz</Set> (2)
</New>
1 The name is the same as that declared in the <realm-name> in web.xml.
2 The name that identifies a set of javax.security.auth.spi.LoginModule configurations that comprise the JAAS config file identified in the jetty.jaas.login.conf property of the jaas module.

The org.eclipse.jetty.jaas.JAASLoginService can be declared in a couple of different places, pick whichever suits your purposes best:

  • If you have more than one webapp that you would like to use the same security infrastructure, then you can declare your org.eclipse.jetty.jaas.JAASLoginService as a bean that is added to the org.eclipse.jetty.server.Server. The file in which you declare this needs to be on Jetty’s execution path. The recommended procedure is to create a file in your $jetty.base/etc directory and then ensure it is on the classpath either by adding it to the Jetty start command line, or more conveniently to a custom module.

    Here’s an example of this type of XML file:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure id="Server" class="org.eclipse.jetty.server.Server">

  <Call name="addBean">
    <Arg>
      <New class="org.eclipse.jetty.jaas.JAASLoginService">
        <Set name="name">Test JAAS Realm</Set>
        <Set name="LoginModuleName">xyz</Set>
      </New>
    </Arg>
  </Call>

</Configure>
  • Alternatively, if you want to use JAAS with a specific webapp only, you declare your org.eclipse.jetty.jaas.JAASLoginService in a context XLM file specific to that webapp:

    <?xml version="1.0"?>
    <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">
    
    <Configure class="org.eclipse.jetty.webapp.WebAppContext">
    
      <Set name="securityHandler">
        <New class="org.eclipse.jetty.security.ConstraintSecurityHandler">
         <Set name="loginService">
           <New class="org.eclipse.jetty.jaas.JAASLoginService">
             <Set name="name">Test JAAS Realm</Set>
             <Set name="loginModuleName">xyz</Set>
           </New>
         </Set>
        </New>
      </Set>
    
    </Configure>
Configure JAAS

We now need to setup the contents of the file we specified as the jetty.jaas.login.conf property when we configured the jaas module. Refer to the syntax rules of this file for a full description.

Remembering the example we set up previously, the contents of the $jetty.base/etc/login.conf file could look as follows:

xyz  { (1)
       com.acme.SomeLoginModule required debug=true; (2)
       com.other.OtherLoginModule optional; (3)
     };
1 The name of the configuration exactly as specified in your org.eclipse.jetty.jaas.JAASLoginService declaration.
2 The first LoginModule declaration, containing the classname of the LoginModule and its configuration properties.
3 A second LoginModule declaration. You can provide as many LoginModule alternatives as you like, with a minimum of one. Refer to the JAAS documentation for more information on the standard configuration properties, and how JAAS interprets this file.

Provided LoginModules

Passwords can be stored in clear text, obfuscated or checksummed. The class org.eclipse.jetty.util.security.Password should be used to generate all varieties of passwords,the output from which can be put in to property files or entered into database tables.
JDBCLoginModule

The org.eclipse.jetty.jaas.spi.JDBCLoginModule stores user passwords and roles in a database accessed via JDBC calls. You can configure the JDBC connection information, as well as the names of the table and columns storing the username and credential, and the names of the table and columns storing the roles.

Here is an example login module configuration file entry for it using an HSQLDB driver:

jdbc { (1)
      org.eclipse.jetty.jaas.spi.JDBCLoginModule required (2)(3)
      dbUrl="jdbc:hsqldb:." (4)
      dbUserName="sa" (5)
      dbDriver="org.hsqldb.jdbcDriver" (6)
      userTable="myusers" (7)
      userField="myuser" (8)
      credentialField="mypassword" (9)
      userRoleTable="myuserroles" (10)
      userRoleUserField="myuser" (11)
      userRoleRoleField="myrole"; (12)
      };
1 The name of the configuration.
2 The name of the LoginModule class.
3 A standard JAAS flag making successful authentication via this LoginModule mandatory.
4 The JDBC url used to connect to the database.
5 The name of the JDBC user to use for the connection.
6 The name of the JDBC Driver class.
7 The name of the table holding the user authenication information.
8 The name of the column holding the user name.
9 The name of the column holding the user credential.
10 The name of the table holding the user authorization information.
11 The name of the column holding the user name.
12 The name of the column holding the user role.

The properties 7-12 are used to format the following queries:

  select <credentialField> from <userTable>
          where <userField> =?
  select <userRoleRoleField> from <userRoleTable>
          where <userRoleUserField> =?

Credential and role information is lazily read from the database when a previously unauthenticated user requests authentication. Note that this information is only cached for the length of the authenticated session. When the user logs out or the session expires, the information is flushed from memory.

Note that passwords can be stored in the database in plain text or encoded formats - see the note on "Passwords/Credentials" above.

DataSourceLoginModule

Similar to the org.eclipse.jetty.jaas.spi.JDBCLoginModule, but using a javax.sql.DataSource to connect to the database instead of a JDBC driver. The javax.sql.DataSource is obtained at runtime by performing a JNDI lookup on java:comp/env/${dnJNDIName}.

A sample login module configuration for this LoginModule:

ds { (1)
     org.eclipse.jetty.jaas.spi.DataSourceLoginModule required (2)(3)
     dbJNDIName="ds" (4)
     userTable="myusers" (5)
     userField="myuser" (6)
     credentialField="mypassword" (7)
     userRoleTable="myuserroles" (8)
     userRoleUserField="myuser" (9)
     userRoleRoleField="myrole"; (10)
    };
1 The name of the configuration.
2 The name of the LoginModule class.
3 A standard JAAS flag making successful authentication via this LoginModule mandatory.
4 The JNDI name, relative to java:comp/env/ to lookup to obtain the javax.sql.DataSource.
5 The name of the table holding the user authenication information.
6 The name of the column holding the user name.
7 The name of the column holding the user credential.
8 The name of the table holding the user authorization information.
9 The name of the column holding the user name.
10 The name of the column holding the user role.
PropertyFileLoginModule

With this login module implementation, the authentication and role information is read from a property file.

props { (1)
        org.eclipse.jetty.jaas.spi.PropertyFileLoginModule required  (2)(3)
        file="/somewhere/somefile.props"; (4)
      };
1 The name of the configuration.
2 The name of the LoginModule class.
3 A standard JAAS flag making successful authentication via this LoginModule mandatory.
4 The location of a properties file containing the authentication and authorization information.

The property file must be of the format:

<username>: <password> [,<rolename> ...]

Here’s an example:

fred: OBF:1xmk1w261u9r1w1c1xmq,user,admin
harry: changeme,user,developer
tom: MD5:164c88b302622e17050af52c89945d44,user
dick: CRYPT:adpexzg3FUZAk,admin

The contents of the file are fully read in and cached in memory the first time a user requests authentication.

LdapLoginModule

The org.eclipse.jetty.jaas.spi.LdapLoginModule uses LDAP to access authentication and authorization information stored in a directory. The LDAP connection information and structure of the authentication/authorization data can be configured.

Here’s an example:

example  { (1)
   org.eclipse.jetty.jaas.spi.LdapLoginModule required  (2)(3)
   contextFactory="com.sun.jndi.ldap.LdapCtxFactory" (4)
   hostname="ldap.example.com" (5)
   port="389" (6)
   bindDn="cn=Directory Manager" (7)
   bindPassword="directory" (8)
   authenticationMethod="simple" (9)
   useLdaps="true" (10)
   userBaseDn="ou=people,dc=alcatel" (11)
   userRdnAttribute="uid" (12)
   userIdAttribute="cn" (13)
   userPasswordAttribute="userPassword" (14)
   userObjectClass="inetOrgPerson" (15)
   roleBaseDn="ou=groups,dc=example,dc=com" (16)
   roleNameAttribute="cn" (17)
   roleMemberAttribute="uniqueMember" (18)
   roleObjectClass="groupOfUniqueNames"; (19)
   forceBindingLogin="false" (20)
   debug="false" (21)
   };
1 The name of the configuration.
2 The name of the LoginModule class.
3 A standard JAAS flag making successful authentication via this LoginModule mandatory.
4 The name of the context factory to use for the LDAP connection.
5 The hostname for the LDAP connection. Optional.
6 The port for the LDAP connection. Optional.
7 The caller security Principal. Optional.
8 The caller security credential. Optional.
9 The security level for the LDAP connection environment. Optional.
10 If true, use ldaps instead of ldap for the connection url.
11 The distinguished name of the directory to search for user information.
12 The name of the attribute for the user roles.
13 The name of the attribute for the user id.
14 The name of the attribute for the user password.
15 The ObjectClass for users.
16 The distinguished name of the directory to search for role information.
17 The name of the attribute for roles.
18 The name of the attribute storing the user for the roles ObjectClass.
19 The name of the ObjectClass for roles.
20 If true, the authentication proceeds on the basis of a successful LDAP binding using the username and credential provided by the user. If false, then authentication proceeds based on username and password information retrieved from LDAP.
21 If true, failed login attempts are logged on the server.

Eclipse Jetty Monitoring & Management

Monitoring and management of a Jetty server is important because it allows you to monitor the status of the server ("Is the server processing requests?") and to manage — i.e. read and possibly change — its configuration.

The ability to read and change the Jetty configuration is very important for troubleshooting Jetty — please refer to the troubleshooting section for more information.

Jetty relies on the Java Management Extensions (JMX) APIs included in OpenJDK to provide monitoring and management.

The JMX APIs support a JVM-local MBeanServer, accessible only from within the JVM itself (or by tools that can attach to a running JVM), and a way to expose the MBeanServer to remote clients via Java’s RMI (Remote Method Invocation).

Enabling Local JMX Support

As with many other Jetty features, local JMX support is enabled with the jmx Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=jmx

With the jmx Jetty module enabled, Jetty components will be exported as JMX MBeans to the JVM platform MBeanServer, so that they can be accessed by JMX compliant tools.

Each Jetty component will export to its correspondent MBean relevant configuration parameters, so that a JMX tool can read and possibly change the component configuration through the MBean.

Note that the Jetty MBeans are registered into the platform MBeanServer, but are not available to remote clients: they are local to the JVM.

This configuration is useful when you develop and test your Jetty server locally.

JMX compliant tools such as Java Mission Control (JMC) can be started locally on your machine and can attach to other JVMs running on your machine, showing you the registered MBeans among which you will find the Jetty MBeans.

Enabling only the local JMX support is the most secure option for monitoring and management, but only users that have local access to the JVM will be able to browse the MBeans. If you need to access the MBeans from a remote machine, read this section.

Enabling Remote JMX Support

There are two ways to configure a Jetty server so that it is possible to access the JVM platform MBeans from remote clients:

  • Use the com.sun.management.jmxremote and related system properties when starting Jetty. Unfortunately, this solution does not work well with firewalls, and will not be discussed further.

  • Use the jmx-remote Jetty module.

Both ways use Java’s Remote Method Invocation (RMI) to communicate between the client and the server.

Refresher: How RMI Works

A server application that wants to make an object available to remote clients must export the object.

Exporting an object creates an RMI stub that contains the host/port of the RMI server that accepts incoming invocations from clients and forwards them to the object. During the creation of the RMI stub, the host stored in the RMI stub is retrieved from the local name resolution system (for example, in Linux, from /etc/hosts).

The RMI stub is then sent, along with a name that uniquely identifies the object, to the RMI registry. The RMI registry is a service that maps names to RMI stubs; it may be external to both clients and server, although often it is part of the server JVM.

When a client application wants to connect to the server object using RMI, it first connects to the RMI registry to download the RMI stub for the RMI server; recall that the RMI stub contains the host/port to connect to the RMI server. Then, the client uses the RMI stub to connect to the RMI server, typically to a host/port that may be different from the RMI registry host/port (in particular, by default the RMI server port will be different from the RMI registry port).

Remote access to the platform MBeans, and therefore the Jetty MBeans, is enabled by the jmx-remote Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=jmx-remote

This command creates the jmx-remote.ini file:

JETTY_BASE
└── start.d
    └── jmx-remote.ini

Enabling the jmx-remote module transitively enables the jmx module as well.

The configuration for the RMI registry and the RMI server is specified by a JMXServiceURL. The string format of an RMI JMXServiceURL is the following:

service:jmx:rmi://<rmi_server_host>:<rmi_server_port>/jndi/rmi://<rmi_registry_host>:<rmi_registry_port>/jmxrmi

Below you can find examples of JMXServiceURLs:

service:jmx:rmi:///jndi/rmi:///jmxrmi
where:
  rmi_server_host = local host address
  rmi_server_port = randomly chosen
  rmi_registry_host = local host address
  rmi_registry_port = 1099

service:jmx:rmi://0.0.0.0:1099/jndi/rmi://0.0.0.0:1099/jmxrmi
where:
  rmi_server_host = any address
  rmi_server_port = 1099
  rmi_registry_host = any address
  rmi_registry_port = 1099

service:jmx:rmi://localhost:1100/jndi/rmi://localhost:1099/jmxrmi
where:
  rmi_server_host = loopback address
  rmi_server_port = 1100
  rmi_registry_host = loopback address
  rmi_registry_port = 1099

The default JMXServiceURL configured by the jmx-remote module is the following:

service:jmx:rmi://localhost:1099/jndi/rmi://localhost:1099/jmxrmi

With the default configuration, only clients that are local to the server machine can connect to the RMI registry and RMI server - this is done for security reasons. However, even with this local-only configuration, it would still be possible to access the MBeans from remote using an SSH tunnel, as explained in this section.

By specifying an appropriate JMXServiceURL, you can fine tune the network address the RMI registry and the RMI server bind to, and the ports that the RMI registry and the RMI server listen to. The RMI server and RMI registry hosts and ports can be the same (as in the default configuration) because RMI is able to multiplex traffic arriving to one port to multiple RMI objects.

If you need to allow JMX remote access through a firewall, you must open both the RMI registry and the RMI server ports. The default configuration simplifies the firewall configuration because you only need to open port 1099.

When Jetty is started with the jmx-remote module enabled, the RMI stub of the Jetty component that provides access to the MBeans is exported to the RMI registry.

The RMI stub contains the host/port to connect to the RMI server, but the host is typically the machine host name, not the host specified in the JMXServiceURL (the latter is only used to specify the network address the RMI server binds to).

To control the host stored in the RMI stub you need to set the system property java.rmi.server.hostname with the desired value in the module configuration file, jmx-remote.ini.

If your client cannot connect to the server, the most common cause is a mismatch between the RMI server host of the JMXServiceURL and the RMI server host of the RMI stub.

You can customize the RMI server host/port, the RMI registry host/port and the system property java.rmi.server.hostname by editing the jmx-remote.ini configuration file. Further information about the jmx-remote module configuration can be found here.

Remote JMX Access with Port Forwarding via SSH Tunnel

You can access JMX MBeans on a remote machine when the RMI ports are not open, for example because of firewall policies, but you have SSH access to the machine, using local port forwarding via an SSH tunnel.

In this case you want to configure the JMXServiceURL that binds the RMI server and the RMI registry to the loopback interface only and to the same port:

service:jmx:rmi://localhost:1099/jndi/rmi://localhost:1099/jmxrmi

You must set the system property -Djava.rmi.server.hostname=localhost so that the RMI stub contains localhost as the host name to connect to. This is, incidentally, the default configuration of the jmx-remote module.

Then you setup the local port forwarding with the SSH tunnel:

$ ssh -L 1099:localhost:1099 <user>@<machine_host>

Thanks to the local port forwarding of the SSH tunnel, when the client connects to localhost:1099 on your local computer, the traffic will be forwarded to machine_host and when there, the SSH daemon will forward the traffic to localhost:1099 on machine_host, which is exactly where the RMI server and the RMI registry listens to.

The client first contacts the RMI registry, so it connects to localhost:1099 on your local computer; the traffic is forwarded to machine_host through the SSH tunnel, connects to the RMI registry and the RMI stub is downloaded to the client.

Then the client uses the RMI stub to connect to the RMI server. The RMI stub contains localhost as the RMI server host because that is what you have configured with the system property java.rmi.server.hostname.

The client will connect again to localhost:1099 on your local computer, this time to contact the RMI server; the traffic is forwarded to machine_host through the SSH tunnel, arrives to machine_host and connects to the RMI server.

Remote JMX Access Authentication & Authorization

The standard javax.management.remote.JMXConnectorServer class, used by the jmx-remote module to provide remote JMX access to Jetty MBeans, provides several options to authenticate and authorize users. For a complete guide to controlling authentication and authorization in JMX, see the official JMX documentation.

The simplest way to control JMX authentication and authorization is to specify two files: one contains username and password pairs, and the other contains username and permission pairs.

This is achieved by enbling the jmx-remote-auth Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=jmx-remote-auth

Enabling the jmx-remote-auth Jetty module creates the following files:

$JETTY_BASE
├── etc
│   ├── jmxremote.access
│   ├── jmxremote.password
│   └── jmx-remote-auth.xml
└── start.d
    ├── jmx-remote-auth.ini
    └── jmx-remote.ini

Then you edit the $JETTY_BASE/etc/jmxremote.password file, adding the username/password pairs that you need:

$JETTY_BASE/etc/jmxremote.password
# The file format is: <username> <password>
alice wonderland
bob marley

You must also edit the $JETTY_BASE/etc/jmxremote.access file to give permissions to your users:

$JETTY_BASE/etc/jmxremote.access
# The file format is: <username> <readonly|readwrite>
alice readwrite
bob readonly

The above files define user alice with password wonderland to have readwrite access, and user bob with password marley to have readonly access.

Securing Remote JMX Access with TLS

The JMX communication via RMI happens by default in clear-text, but it is possible to secure the JMX communication via RMI with TLS.

If you want to reuse the configuration that you are using for the https module, you can just enable the jmx-remote-ssl.xml Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-module=jmx-remote-ssl

The jmx-remote-ssl Jetty module depends on the ssl Jetty module that in turn requires a KeyStore (read this section for more information).

The KeyStore must contain a valid certificate signed by a Certification Authority. Having certificates signed by a Certification Authority simplifies by a lot the configuration needed to get the RMI communication over TLS working properly.

The RMI mechanic is the usual one: the RMI client (typically a monitoring console) will connect first to the RMI registry (using TLS), download the RMI stub that contains the address and port of the RMI server to connect to, then connect to the RMI server (using TLS).

This also mean that if the RMI registry and the RMI server are on different hosts, the RMI client must have available the cryptographic material to validate the certificates from both hosts. This is where having certificates signed by a Certification Authority simplifies the configuration: if they are signed by a well known Certification Authority, the client does not need any extra configuration — everything will be handled by the Java runtime.

If the certificates are not signed by a Certification Authority (for example the certificate is self-signed), then you need to specify the TLS system properties that allow RMI (especially when acting as an RMI client) to retrieve the cryptographic material necessary to establish the TLS connection.

When the RMI server exports the JMXConnectorServer it acts as an RMI client towards the RMI registry, and as such you must specify the TLS system properties as detailed below.

You must edit the $JETTY_BASE/start.d/jmx-remote-ssl.ini file and add the TrustStore path and password:

$JETTY_BASE/start.d/jmx-remote-ssl.ini
--module=jmx-remote-ssl

# System properties necessary for non-trusted certificates.
-Djavax.net.ssl.trustStore=/path/to/trustStore.p12
-Djavax.net.ssl.trustStorePassword=password

The TrustStore must contain the certificate you want to trust.

If you are using self-signed certificates, the KeyStore already contains the self-signed certificate and therefore the KeyStore can be used as a TrustStore, and the system properties above can refer to the KeyStore path and password.

JMX compliant tools that offer a graphical user interface also must be started specifying the TrustStore path and password.

For example, to launch Java Mission Control (JMC):

$ jmc -vmargs -Djavax.net.ssl.trustStore=/path/to/trustStore.p12 -Djavax.net.ssl.trustStorePassword=password