Dynamic Content with HandlerMapping

Recently I was challenged to enable a Spring Boot API micro service with serving a frontend, but without the frontend being packaged with the service, and in such a way that the frontend could change at any time without redeploying the backend. Not only that, but multiple frontends could be served from the same hostname depending on the route the browser requests. Here's my solution.

First, I'll describe what's running in production and then share a minimal reproduction of the concept. This reproduction was done on my own hardware and does not share any code with the production solution. While creating this reproduction, I figured out how to use HandlerMapping and HandlerExecutionChain which unfortunately have very little documentation online. I hope someone some day finds this demonstration of HandlerMapping useful.

Production

Let me introduce the micro service on the side: this is what manages our deployments at this time. It instruments AWS to change elastic beanstalk, push updated cloudformation stacks, and so on.

This service was initially made so that our new quality assurance team could deploy our applications without depending on someone to run command line scripts with a specific git commit hash.

Do everything you can to support your QA team. When they can do a good job and do a good job, as an on call engineer you'll only have to worry about black swan events like Let's Encrypt DST Root CA X3 Expiration (September 2021).

I imagine that being told to run some magical command line text (a script) with some other magical text (a git commit hash) which if done wrong could take down production is like being told to restart an airplane mid flight using an actively wiggling slimy squid arm with just the right amount of salt. You don't know how this alchemy works but you're expected to perform it without any feedback on if it were done correctly.

This microservice was extended to support a new concept: Dynamic Frontend. Previously it only supported whole applications, where the app update would either be updating beanstalk, running a CDK script, or syncing an S3 bucket with some frontend content at the root and invalidating index.html. Now it tracks which frontends have been built, deployed, which hostnames have which frontends deployed. With this information and the metadata described next it generates a frontend manifest that describes which content is to be shown on which routes.

Additional metadata such as the routes for this frontend is collected on every commit and saved alongside a zip of the frontend content and the html to inject into the head, body, and footer. This all goes into the deployables bucket during Continuous Integration (CI) and are called artifacts. Each of these is tracked and submitted to the microservice during CI with information from the commit.

The old static sites each have a dedicated bucket, CloudFront distribution, and hosted zone record per app and stage (e.g. dev, prod), these will still be supported but dynamic frontend is quite different. The assets are unpacked into the bucket in an immutable fashion prefixed by app-name/commit-hash/. This bucket will serve multiple application contents for all stages and no objects are replaced. Some time later we will figure out garbage collection.

It may seem obvious to say it, but make sure your resources are uploaded where they can be reached before you give the links to them. Not doing so is a classic race condition.

When working in an immutable fashion, order your world-affecting actions in reverse order from the point of external reference. In Haskell and erlang for example, a linked list is a pair of a value and a pointer to either the next pair or the end of the list. This pair cannot be modified after it is created, it is immutable.

So how is an immutable list made? Again, in reverse order. So a list like [1, 2, 3] is actually something like this: {1, {2, {3, {}}}}. The values are instantiated in the following order

• three = {3, {}}
• two = {2, three}
• one = {1, two}

How does this relate to deploying? Well to have a server with your latest app serve traffic to your end users..

• First you need your application compiled.
• Second, you need to make a docker container. (if you use that)
• Third, you need to set up a server with the docker container.
• Finally you need to switch away from the old application somehow.

This last part is usually mutable, but all the steps before it are immutable. You usually don't need to build a docker container twice for the same commit.

Always keep the dependency graph in mind as you make changes across immutable and mutable things.

This means the assets need to be deployed before the html points to it. That’s the second to last step of the deployment process: unpacking the zip from the deployables bucket onto the public distribution bucket. Once done, then the manifest can be generated. This is stored atomically into the database, rather than being computed on demand. Within some Time To Live (TTL), the backends will query the current manifest and cache it in memory.

This manifest contains all the html to inject for every active frontend app and version, every host name that is configured, a default host name to fall back to, and within each host name a configuration of route patterns and app-version to use. It is under 10kb, can be fetched in 30ms, and is only hit once per minute per production server. Sandbox servers have a TTL of 5 seconds for convenience.

A micro library of sorts reviews the request and provides dynamic content to use, or if no content will fall back on the clunky ancient frontend bundled with the monolith. This is done within the index.jsp that serves all unmapped paths (convenient for SPAs), so no API routes are affected.

JSP is a pretty old technology standard. You may have seen PHP, well this is quite different. It's like a bunch of compiler macros got together to compile java files on the fly to make a Handler. Everything is string built over time and you can't help but see like 15 blank lines at the top of the html response body. Treat it like wood and rope, I really cannot recommend raising its technological complexity with other things.

Don't rely on JSTL if it isn't already in your project, it will likely fail and give confusing errors. Sometimes stack overflow reaches for these like oh you can use conditional tags and embed your conditional results in those. Don't make it too complicated, just go with <% if (...) {} %>.

Also don't rely on JSPF to reuse code across pages. I've tried to refactor things into this technology, it felt like I was stepping into the mess that web components have with passing parameters everywhere, figuring out how to loop properly and so on. When there's a divergence that has to be accounted in a shared JSPF, it's... easier to just un-refactor it back into the site it is being used. These JSPs aren't being edited every week, only once every 6 months. Moving completely out the JSP into something like Thymeleaf or Pebble is advised.

Just in case, the crunchy old monolith keeps a cached copy in redis if the deployment microservice is down and a new instance arrives without a copy in memory. Production has to keep on running even if other things are down.. including the thing that deploys production.

So in short the following happens in this order:

1. When an API path with a certain prefix is used, spring will handle the request.
2. When an old old tomcat servlet controller is mapped to a uri path it will handle the request.
3. If a file exists that should be served (and isn’t a jsp) and the path matches, send that
4. When the fallback jsp is used, it will see if the uri path is managed by dynamic frontend, if so it will inject the new html into the page and omit the html for the old single page app
5. Otherwise the fallback jsp is used with the old single page app bundled with the monolith.

The solution is by no means clean, but is low risk backwards compatibility ever clean?

A spring boot version

After this launched successfully and with praise from all teams affected, I worked on a solution for our spring boot projects. This isn’t needed yet, but I expect it to be useful for internal Microservices and new public backends going forward. Our current split frontend / backend projects struggle with CORS, often requiring my intervention or… someone just outright turning CORS bypass everything on so another can do their job.

I think I’m the only one that actually understands CORS at my workplace and at times that kinda scares me. Back in 2013 or so when it became a thing in Chrome it was the most frustrating thing to figure out. I resorted to the same insecure brute force tactics those in my engineering team are doing to make progress on their projects. I’ll do what I can to make development fun, productive, and secure.

While developing the spring solution, I searched for a different design. While trying out Vaadin, a Java client and server solution, I saw it was able to inject its own routes into spring and I dug for hours to figure out what was going on.

I pieced together some hints from stackoverflow, documentation going back to 2003, debug breakpoints in the heart of Spring to figure out a solution which does not rely on servlet filters.

Servlet filters

While the initial reimplementation did in fact use a filter, doing so comes with several consequences. First I lose control of prioritizing API requests, instead I would have to watch for signals like NotMappedException to implement this. And somehow inject the filter between the application and the top level filter. No thank you.

Unfortunately, I couldn't even hope to fall back on NotMappedException as you will see soon.

Servlet filters are great for stuff like logging, attaching info to requests, syncing and flushing sessions or the like. You know, like express middleware.

I’ve seen how filters are used in that old monolith, conditionally handling requests at different levels makes first and second class citizens in a way almost no one can debug. It’s not a good long term decision even though it seems to be the easiest one.

HandlerMapping

What is a HandlerMapping? It is an abstraction that provides a found handler or null. What is a Handler? I did not know exactly what going into this. But it seems to be a lot like a Servlet Filter that can choose the right Controller to feed a request to and send the response back to the client (or up the filter chain). I suspect that Spring returns a handler that renders views, encodes json, etc. the HandlerMapping just tracks all the routes found during startup and returns that handler on a successful match.

At application start, the application will instantiate several HandlerMapping classes, one is set up with all of the controller RequestMapping / GetMapping / etc. annotated functions, another is set up for resources. By default there are five of them. I won’t go into detail on each one. To make this work I need to insert a custom HandlerMapping in the middle. API and local files should be served without being interrupted while dynamic frontend routes can take the rest.

So with reflection, spring collects the HandlerMapping instances and orders them.

Unfortunately, the Order annotation did not work for me. Instead it is another ancient and optional interface called Ordered to be implemented on the HandlerMapping implementing class. Figuring this out took me about 2 hours. When not available or not set it is Integer.MAX_VALUE, which essentially means last and unordered.

Next is the HandlerMapping interface, the parameters aren’t even httpServlets, which makes getting the URI out an annoying exercise of unchecked casting. I’d recommend using this class called AbstractHandlerMapping. It’s getHandlerInternal function provides more useful inputs but the output is just… an Object.

I get that Java 1.4 from 2002 wasn’t capable and all sorts of type losing hacks had to be used. But really? An Object? And no useful documentation as to what it should be?

With more browsing around, I found the interface HttpRequestHandler in another spring dependency. It seems obvious enough and so I try it. Thankfully it works! It looks a lot like a Servlet Filter though.

Unfortunately static content is handled at priority Integer.MAX_VALUE - 1 and a missing resource is not bubbled up despite spring.mvc.throw-exception-if-no-handler-found=true being set.

Everyone's solution for this is to disable resource mappings with spring.resources.add-mappings=false...

Is it just not possible to have a programmatic 404 page with resources? I spent 10 hours trying to figure out how handle a 404 for a missing resource. It seems that Spring does not consider this to be an interesting and useful feature...

I tried various things, from filters, controller advice, to even reinventing the resource handler mapping by enumerating all resources under a path and adding mappings one at a time. None of these solutions were satisfactory. I lacked confidence in traversing the resources securely.

However, while exploring all of the possibilities I could muster in my personal time, I found a neat feature: AntPathMatcher is another useful 2003 relic that allows one to pattern match on paths. It feels a little like xpath. This is used in the workaround for the resource priority problem.

Simple HandlerMapping

What's the shortest useful HandlerMapping possible? I think it's this:

package dev.cendyne.handlerdemo;

import org.springframework.beans.factory.InitializingBean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.handler.AbstractHandlerMapping;

import javax.servlet.http.HttpServletRequest;

@Configuration
public class ProofOfConceptHandler extends AbstractHandlerMapping implements InitializingBean {
    @Override
    public void afterPropertiesSet() throws Exception {
        setOrder(11);
    }

    @Override
    protected Object getHandlerInternal(HttpServletRequest request) throws Exception {
        String uri = request.getRequestURI();
        if (uri.equals("/proof-of-concept")) {
            return new SimpleStaticHandler("text/plain", "Proof!");
        }
        return null;
    }
}

Here's what to make note of:

1. Extend AbstractHandlerMapping which implements HandlerMapping
2. Match the request in the getHandlerInternal and return a handler when it is appropriate
3. Explicitly set the order with setOrder within afterPropertiesSet which comes from InitializingBean

The Handler

Earlier I referenced SimpleStaticHandler, it really isn't too complicated:

package dev.cendyne.handlerdemo;

import lombok.Value;
import org.springframework.web.HttpRequestHandler;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

@Value
public class SimpleStaticHandler implements HttpRequestHandler {
    String contentType;
    String responseContent;

    @Override
    public void handleRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.setStatus(200);
        response.setContentType(contentType);
        try (var writer = response.getWriter()) {
            writer.write(responseContent);
        }
    }
}

So it turns out javax has been renamed to jakarta. Future versions of spring will use that instead, though javax will still be maintained in name for the next version. More can be read at Transition from Java EE to Jakarta EE. I guess this is good since Oracle has a history of being mean.

Anyway, here you can see what makes this look a lot like a Servlet Filter. The handler receives a request and response and is expected to respond to the client or throw an exception. A filter however would pass on the request if it does not handle it. The HandlerMapping handles that concern.

Dynamic HandlerMapping

So what can we do to make this dynamic? First, I'll make a simple interface to represent the domain here. Either we get a result to send to the client, or we do not.

Also, it's dynamic so it can change, either by API or something in the environment.

package dev.cendyne.handlerdemo;

import java.util.Optional;

public interface DynamicFrontendService {
    Optional<String> getHtmlForRoute(String uri);
    void addHtml(String uri, String html);
}

The actual implementation isn't all that interesting, it wraps around a ConcurrentHashMap

To add some data on startup, the following runs each time.

@Bean
public CommandLineRunner loadData(DynamicFrontendService dynamicFrontend) {
    return (args) -> {
        dynamicFrontend.addHtml("/hello", "<h1>Hello World</h1>");
        dynamicFrontend.addHtml("/aloha", "<h1>Aloha kakahiaka</h1>");
    };
}

To actually do something dynamically, here's the HandlerMapping before we add exclusions for resources.

@Configuration
@RequiredArgsConstructor
public class DynamicFrontendHandler extends AbstractHandlerMapping implements InitializingBean {
    private final DynamicFrontendService dynamicFrontend;
    @Override
    public void afterPropertiesSet() throws Exception {
        setOrder(10);
    }

    @Override
    protected Object getHandlerInternal(HttpServletRequest request) throws Exception {
        return dynamicFrontend.getHtmlForRoute(request.getRequestURI())
                .map(content -> new SimpleStaticHandler("text/html", content))
                .orElse(null);
    }
}

See? It wasn't all that complicated.

But if I add content to /robots.txt and there is a resource for robots.txt, then this handler will obscure the original resource which is not desirable.

Accounting for static resources

To allow the dynamic handler mapping to take on nearly any url for a single page application, but pass through resources such as.. favicon.ico, robots.txt, images/cendyne-does-java.jpg they need to be excluded.

This can be done with the AntPathMatcher mentioned before and a configurable property value.

@Configuration
@RequiredArgsConstructor
public class DynamicFrontendHandler extends AbstractHandlerMapping implements InitializingBean {
    private final DynamicFrontendService dynamicFrontend;
    private final AntPathMatcher antPathMatcher = new AntPathMatcher();

    @Autowired
    @Value("${frontend.static-paths:}")
    String staticPathsStr;

    List<String> staticPaths = Collections.emptyList();

    @Override
    public void afterPropertiesSet() throws Exception {
        setOrder(10);

        if (staticPathsStr != null && !staticPathsStr.isBlank()) {
            staticPaths = Arrays.asList(staticPathsStr.split(","));
        }
    }

    @Override
    protected Object getHandlerInternal(HttpServletRequest request) throws Exception {
        String uri = request.getRequestURI();
        // Do not consider static paths
        for (String staticPath : staticPaths) {
            if (antPathMatcher.match(staticPath, uri)) {
                return null;
            }
        }

        return dynamicFrontend.getHtmlForRoute(uri)
                .map(content -> new SimpleStaticHandler("text/html", content))
                .orElse(null);
    }
}

Notably, this means now application.properties can be set with

frontend.static-paths=/robots.txt,/favicon.ico,/images/**

See the **? That's part of the Ant path matching expression. It means all files within that directory and any sub directories. So, /images/cendyne-does-java.jpg and /images/anything/really.png will be avoided with a succinct expression.

Next, that @Value annotation..

@Autowired
@Value("${frontend.static-paths:}")
String staticPathsStr;

@Autowired tells Spring to inject in this object, it happens to be a String. That type alone won't be useful. While @Qualifier can be used to specify which named bean should be injected, we want something easily configurable. That can come through either a configuration object or directly with a Spring Expression Language (SpEL) expression. Here, the value will be pulled from frontend.static-paths, the colon at the end is the fallback expression which is blank. So the empty string may be the supplied value if not provided by the application developer.

aplication.properties is an accessible mechanism for libraries and projects to coordinate together without costly initialization rituals and boilerplate. Best to allow library tuning through that by default.

Conclusion

HandlerMapping from 2003 is an incredibly flexible invention. It justs lacks constructive by-example documentation. If you're interested in seeing the full sources for this minimal reproduction, see the github repository cendyne/spring-boot-handler-demo.

I would have expected Spring to have a way to handle 404s with resources involved. All of StackOverflow saying just disable resources is not a decent answer. The alternative may be to use a configurer and list out resources one by one and then any resource-only paths like /images/**. But this is tedious work that I expect to be lost and forgotten.