Server I/O Architecture

The Jetty server libraries provide the basic components and APIs to implement a network server.

They build on the common Jetty I/O Architecture and provide server specific concepts.

The Jetty server libraries provide I/O support for TCP/IP sockets (for both IPv4 and IPv6) and, when using Java 16 or later, for Unix-Domain sockets.

Support for Unix-Domain sockets is interesting when Jetty is deployed behind a proxy or a load-balancer: it is possible to configure the proxy or load balancer to communicate with Jetty via Unix-Domain sockets, rather than via the loopback network interface.

The central I/O server-side component are org.eclipse.jetty.server.ServerConnector, that handles the TCP/IP socket traffic, and org.eclipse.jetty.unixdomain.server.UnixDomainServerConnector, that handles the Unix-Domain socket traffic.

ServerConnector and UnixDomainServerConnector are very similar, and while in the following sections ServerConnector is used, the same concepts apply to UnixDomainServerConnector, unless otherwise noted.

A ServerConnector manages a list of ConnectionFactorys, that indicate what protocols the connector is able to speak.

Creating Connections with ConnectionFactory

Recall from the Connection section of the Jetty I/O architecture that Connection instances are responsible for parsing bytes read from a socket and generating bytes to write to that socket.

On the server-side, a ConnectionFactory creates Connection instances that know how to parse and generate bytes for the specific protocol they support — it can be either HTTP/1.1, or TLS, or FastCGI, or the PROXY protocol.

For example, this is how clear-text HTTP/1.1 is configured for TCP/IP sockets:

// Create the HTTP/1.1 ConnectionFactory.
HttpConnectionFactory http = new HttpConnectionFactory();

Server server = new Server();

// Create the connector with the ConnectionFactory.
ServerConnector connector = new ServerConnector(server, http);
connector.setPort(8080);

server.addConnector(connector);
server.start();

With this configuration, the ServerConnector will listen on port 8080.

Similarly, this is how clear-text HTTP/1.1 is configured for Unix-Domain sockets:

// Create the HTTP/1.1 ConnectionFactory.
HttpConnectionFactory http = new HttpConnectionFactory();

Server server = new Server();

// Create the connector with the ConnectionFactory.
UnixDomainServerConnector connector = new UnixDomainServerConnector(server, http);
connector.setUnixDomainPath(Path.of("/tmp/jetty.sock"));

server.addConnector(connector);
server.start();

With this configuration, the UnixDomainServerConnector will listen on file /tmp/jetty.sock.

ServerConnector and UnixDomainServerConnector only differ by how they are configured — for ServerConnector you specify the IP port it listens to, for UnixDomainServerConnector you specify the Unix-Domain path it listens to.

Both configure ConnectionFactorys in exactly the same way.

When a new socket connection is established, ServerConnector delegates to the ConnectionFactory the creation of the Connection instance for that socket connection, that is linked to the corresponding EndPoint:

Diagram

For every socket connection there will be an EndPoint + Connection pair.

Wrapping a ConnectionFactory

A ConnectionFactory may wrap another ConnectionFactory; for example, the TLS protocol provides encryption for any other protocol. Therefore, to support encrypted HTTP/1.1 (also known as https), you need to configure the ServerConnector with two ConnectionFactorys — one for the TLS protocol and one for the HTTP/1.1 protocol, like in the example below:

// Create the HTTP/1.1 ConnectionFactory.
HttpConnectionFactory http = new HttpConnectionFactory();

// Create and configure the TLS context factory.
SslContextFactory.Server sslContextFactory = new SslContextFactory.Server();
sslContextFactory.setKeyStorePath("/path/to/keystore.p12");
sslContextFactory.setKeyStorePassword("secret");

// Create the TLS ConnectionFactory,
// setting HTTP/1.1 as the wrapped protocol.
SslConnectionFactory tls = new SslConnectionFactory(sslContextFactory, http.getProtocol());

Server server = new Server();

// Create the connector with both ConnectionFactories.
ServerConnector connector = new ServerConnector(server, tls, http);
connector.setPort(8443);

server.addConnector(connector);
server.start();

With this configuration, the ServerConnector will listen on port 8443. When a new socket connection is established, the first ConnectionFactory configured in ServerConnector is invoked to create a Connection. In the example above, SslConnectionFactory creates a SslConnection and then asks to its wrapped ConnectionFactory (in the example, HttpConnectionFactory) to create the wrapped Connection (an HttpConnection) and will then link the two Connections together, in this way:

Diagram

Bytes read by the SocketChannelEndPoint will be interpreted as TLS bytes by the SslConnection, then decrypted and made available to the DecryptedEndPoint (a component part of SslConnection), which will then provide them to HttpConnection.

The application writes bytes through the HttpConnection to the DecryptedEndPoint, which will encrypt them through the SslConnection and write the encrypted bytes to the SocketChannelEndPoint.

Choosing ConnectionFactory via Bytes Detection

Typically, a network port is associated with a specific protocol. For example, port 80 is associated with clear-text HTTP, while port 443 is associated with encrypted HTTP (that is, the TLS protocol wrapping the HTTP protocol, also known as https).

In certain cases, applications need to listen to the same port for two or more protocols, or for different but incompatible versions of the same protocol, which can only be distinguished by reading the initial bytes and figuring out to what protocol they belong to.

The Jetty server libraries support this case by placing a DetectorConnectionFactory in front of other ConnectionFactorys. DetectorConnectionFactory accepts a list of ConnectionFactorys that implement ConnectionFactory.Detecting, which will be called to see if one of them recognizes the initial bytes.

In the example below you can see how to support both clear-text and encrypted HTTP/1.1 (i.e. both http and https) on the same network port:

// Create the HTTP/1.1 ConnectionFactory.
HttpConnectionFactory http = new HttpConnectionFactory();

// Create and configure the TLS context factory.
SslContextFactory.Server sslContextFactory = new SslContextFactory.Server();
sslContextFactory.setKeyStorePath("/path/to/keystore.p12");
sslContextFactory.setKeyStorePassword("secret");

// Create the TLS ConnectionFactory,
// setting HTTP/1.1 as the wrapped protocol.
SslConnectionFactory tls = new SslConnectionFactory(sslContextFactory, http.getProtocol());

Server server = new Server();

// Create the detector ConnectionFactory to
// detect whether the initial bytes are TLS.
DetectorConnectionFactory tlsDetector = new DetectorConnectionFactory(tls); (1)

// Create the connector with both ConnectionFactories.
ServerConnector connector = new ServerConnector(server, tlsDetector, http); (2)
connector.setPort(8181);

server.addConnector(connector);
server.start();
1 Creates the DetectorConnectionFactory with the SslConnectionFactory as the only detecting ConnectionFactory. With this configuration, the detector will delegate to SslConnectionFactory to recognize the initial bytes, which will detect whether the bytes are TLS protocol bytes.
2 Creates the ServerConnector with DetectorConnectionFactory as the first ConnectionFactory, and HttpConnectionFactory as the next ConnectionFactory to invoke if the detection fails.

In the example above ServerConnector will listen on port 8181. When a new socket connection is established, DetectorConnectionFactory is invoked to create a Connection, because it is the first ConnectionFactory specified in the ServerConnector list. DetectorConnectionFactory reads the initial bytes and asks to its detecting ConnectionFactorys if they recognize the bytes. In the example above, the detecting ConnectionFactory is SslConnectionFactory which will therefore detect whether the initial bytes are TLS bytes. If one of the detecting ConnectionFactorys recognizes the bytes, it creates a Connection; otherwise DetectorConnectionFactory will try the next ConnectionFactory after itself in the ServerConnector list. In the example above, the next ConnectionFactory after DetectorConnectionFactory is HttpConnectionFactory.

The final result is that when new socket connection is established, the initial bytes are examined: if they are TLS bytes, a SslConnectionFactory will create a SslConnection that wraps an HttpConnection as explained here, therefore supporting https; otherwise they are not TLS bytes and an HttpConnection is created, therefore supporting http.

Writing a Custom ConnectionFactory

This section explains how to use the Jetty server-side libraries to write a generic network server able to parse and generate any protocol..

Let’s suppose that we want to write a custom protocol that is based on JSON but has the same semantic as HTTP; let’s call this custom protocol JSONHTTP, so that a request would look like this:

{
  "type": "request",
  "method": "GET",
  "version": "HTTP/1.1",
  "uri": "http://localhost/path",
  "fields": {
    "content-type": "text/plain;charset=ASCII"
  },
  "content": "HELLO"
}

In order to implement this custom protocol, we need to:

  • implement a JSONHTTPConnectionFactory

  • implement a JSONHTTPConnection

  • parse bytes and generate bytes in the JSONHTTP format

  • design an easy to use API that applications use to process requests and respond

First, the JSONHTTPConnectionFactory:

public class JSONHTTPConnectionFactory extends AbstractConnectionFactory
{
    public JSONHTTPConnectionFactory()
    {
        super("JSONHTTP");
    }

    @Override
    public Connection newConnection(Connector connector, EndPoint endPoint)
    {
        JSONHTTPConnection connection = new JSONHTTPConnection(endPoint, connector.getExecutor());
        // Call configure() to apply configurations common to all connections.
        return configure(connection, connector, endPoint);
    }
}

Note how JSONHTTPConnectionFactory extends AbstractConnectionFactory to inherit facilities common to all ConnectionFactory implementations.

Second, the JSONHTTPConnection. Recall from the echo Connection example that you need to override onOpen() to call fillInterested() so that the Jetty I/O system will notify your Connection implementation when there are bytes to read by calling onFillable(). Furthermore, because the Jetty libraries are non-blocking and asynchronous, you need to use IteratingCallback to implement onFillable():

public class JSONHTTPConnection extends AbstractConnection
{
    // The asynchronous JSON parser.
    private final AsyncJSON parser = new AsyncJSON.Factory().newAsyncJSON();
    private final IteratingCallback callback = new JSONHTTPIteratingCallback();

    public JSONHTTPConnection(EndPoint endPoint, Executor executor)
    {
        super(endPoint, executor);
    }

    @Override
    public void onOpen()
    {
        super.onOpen();

        // Declare interest in being called back when
        // there are bytes to read from the network.
        fillInterested();
    }

    @Override
    public void onFillable()
    {
        callback.iterate();
    }

    private class JSONHTTPIteratingCallback extends IteratingCallback
    {
        private ByteBuffer buffer;

        @Override
        protected Action process() throws Throwable
        {
            if (buffer == null)
                buffer = BufferUtil.allocate(getInputBufferSize(), true);

            while (true)
            {
                int filled = getEndPoint().fill(buffer);
                if (filled > 0)
                {
                    boolean parsed = parser.parse(buffer);
                    if (parsed)
                    {
                        Map<String, Object> request = parser.complete();

                        // Allow applications to process the request.
                        invokeApplication(request, this);

                        // Signal that the iteration should resume when
                        // the application completed the request processing.
                        return Action.SCHEDULED;
                    }
                    else
                    {
                        // Did not receive enough JSON bytes,
                        // loop around to try to read more.
                    }
                }
                else if (filled == 0)
                {
                    // We don't need the buffer anymore, so
                    // don't keep it around while we are idle.
                    buffer = null;

                    // No more bytes to read, declare
                    // again interest for fill events.
                    fillInterested();

                    // Signal that the iteration is now IDLE.
                    return Action.IDLE;
                }
                else
                {
                    // The other peer closed the connection,
                    // the iteration completed successfully.
                    return Action.SUCCEEDED;
                }
            }
        }

        @Override
        protected void onCompleteSuccess()
        {
            getEndPoint().close();
        }

        @Override
        protected void onCompleteFailure(Throwable cause)
        {
            getEndPoint().close(cause);
        }
    }
}

Again, note how JSONHTTPConnection extends AbstractConnection to inherit facilities that you would otherwise need to re-implement from scratch.

When JSONHTTPConnection receives a full JSON object it calls invokeApplication(…​) to allow the application to process the incoming request and produce a response.

At this point you need to design a non-blocking asynchronous API that takes a Callback parameter so that applications can signal to the implementation when the request processing is complete (either successfully or with a failure).

A simple example of this API design could be the following:

  • Wrap the JSON Map into a JSONHTTPRequest parameter so that applications may use more specific HTTP APIs such as JSONHTTPRequest.getMethod() rather than a generic Map.get("method")

  • Provide an equivalent JSONHTTPResponse parameter so that applications may use more specific APIs such as JSONHTTPResponse.setStatus(int) rather than a generic Map.put("status", 200)

  • Provide a Callback (or a CompletableFuture) parameter so that applications may indicate when the request processing is complete

This results in the following API:

class JSONHTTPRequest
{
    // Request APIs
}

class JSONHTTPResponse
{
    // Response APIs
}

interface JSONHTTPService
{
    void service(JSONHTTPRequest request, JSONHTTPResponse response, Callback callback);
}

The important part of this simple API example is the Callback parameter that makes the API non-blocking and asynchronous.