Persisting state between browser and PWA instance on Safari for iOS

Introduction #

This post came out of some experiments to add WiFi functionality to a doorbell in order to send push notifications to my phone (and other devices). As I was looking for a way to actually trigger the push notification on the phone, I came across Web Push, which is a standard that allows websites to send push notifications. Surprisingly, the standard is actually supported on most platforms, even on iOS (as of version 16.4). So, I decided to implement the "frontend" as a Progressive Web App. I'm not really a web person myself, but the API is easy to understand, and I found a great guide which I used as the base for my front end.

As I want make my frontend accessible from the internet, I needed some form of authentication to prevent unauthorized access. I specifically wanted to avoid having user accounts with passwords, since password-based authentication is a bad user experience: No one wants to remember a long password, particularly not for something rarely used like a doorbell notification app. This leads to users choosing simple, easy-to-remember passwords, which can be cracked with brute-force. Passkeys offer a solution to this problem, by generating a secure key that the user doesn't have to remember directly - instead, the device presents the key automatically. To avoid unauthorized access, the device usually requires the user to authenticate themselves to the device, mostly with the same method used to unlock the screen (e.g. Face ID on an iPhone). I decided against passkeys however, since they are quite new and I was not sure whether they are well supported on all possible browser / OS / device combinations. Instead, I am using a somewhat similar system, which works as follows:

1. The server generates a signup token in advance. This token allows a user to register a device themselves, e.g. by scanning a QR code with a link to the registration endpoint with the token embedded
2. The client sends the token to the server. The server will generate a device ID, which is essentially used like a passkey. This device ID is sent to the client as a cookie
3. In all following requests, the client automatically presents the cookie, and the server will look for the device ID in the database

With this, the user signup flow will look like this:

1. The user scans a QR code or clicks on a link with the signup token
2. A page is displayed, which instructs the user to install the PWA. At this piont, the device is already registered.
3. The PWA starts on a home page, which finally prompts the user to enable push notifications.

Think different #

As is often the case, things are not as straightforward when it comes to Apples Safari, particularly on iOS. First off, Web Push on iOS only works for PWAs installed to the home screen. This is actually the only reason why adding the app is a mandatory step in the signup flow above - on other platforms, all current features would work just fine from a browser tab. But that's not all: See, I first tested the token-based signup on my laptop running chrome. I would click the token link, install the PWA and be greeted with my home screen. Delighted, I tried the same with iOS: Scan the QR code with the link, add the app to home screen, and open it - just to be greeted with an Authentication failure.

So, I attached my phone to my laptop to inspect the running PWA. Normally, websites on Safari for iOS requires a Mac, but thanks to ios-safari-remote-debug-kit, we can use Safaris Web Inspector in a regular chrome browser. I navigated to the storage tab in the inspector, only to see that there were no cookies stored for the PWA. I switched to the regular Safari tab, which was still open, and the cookie was just fine there.

As it turns out, a PWA in Safari uses a new context that is isolated from the regular browser ^1. This means that any state we could store locally, be it in cookies, local storage, session storage, IndexedDB, would be wiped out in the homescreen-installed version of the PWA. However, this behavior breaks my desired signup flow since a link, be it from a message or QR code, will always be opened in a Safari tab. So, I had to find a trick to transfer at least a bit of information from the tab to the installed version of the app.

I found a trick using cache storage, which was apparently shared between safari tabs and installed instances of the same page in earlier iOS versions. However, I tried a demo of it on my phone (which is running iOS 17 beta), and it didn't work. So, we have to come up with something else.

Manifest tricks #

If we assume the different site instances as perfectly isolated from the browser storage side, the only thing that connects them is the web app manifest: It is downloaded using a regular HTTP request initiated in the browser tab session, and is then used to set up the PWA installation. The manifest contains a start_url parameter, which specifies the site that should be displayed on launch of the app. This gives us an opportunity to embed a small token that we can use to track the user temporarily. This idea is not completely mine, it is based on this answer to the same problem on StackOverflow

To make this process as secure as possible, we will use a temporary token instead of the permanent device ID. This token is implemented as follows:

1. As before, the client makes a request to the registration endpoint in a Safari tab. The response still contains just the device ID cookie
2. In the HTML response, a link to the manifest is included. The client makes a request to fetch the manifest, with the cookie included in the request
3. Upon getting a manifest request, the server creates a new temporary token (just a random UUID) and saves the pair of device ID and temporary token (also called tempId going further). The server generates the manifest dynamically, appending a query string of ?tempId=<generated tempId> to the start_url
4. The client installs the PWA. On first launch, a request is made to the start page, with the tempID included in the query string
5. The server sees the tempId and sets a cookie with the associated deviceId. This way, we essentially restore the lost deviceId for the PWA. The server authenticates the client just as it would if it saw a deviceId cookie, and returns with the start web page. In this process, the server also deletes the tempId association.
6. On future requests, the deviceId cookie is sent as usual. The now unknown tempId is simply ignored by the server

With my backend implemented in ExpressJS, here are the relevant code snippets:

// authentication middleware. has to run before everything else
// This also handles tempIDs, which are matched against the association list
// and exchanged for the corresponding deviceId cookie
// in this case, the request is also authenticated, so we don't have to care
// about the deviceId cookie or tempId in any other code
app.use(errorHandlingWrapper(async (req, res, next) => {
    console.debug('raw URL: ', req.url);
	// URLs exempt from authentication
    // strip out query string (?...) from URL
    const realUrl = req.url.split('?')[0];
    console.debug('cleaned URL: ', realUrl);
    if (['/register', '/register.html', '/register.js', '/mint', '/worker.js', '/localforage.js', '/manifest.json', '/icon.png'].includes(realUrl)) {
        console.debug('URL ' + realUrl + ': is exempt from auth, passing on to next middleware');
        next();
	    return;
    }

    // in normal authentication flow, the device should send a cookie with its deviceId
    let deviceId = req.cookies.deviceId;

    // iOS separates pwa and browser environment, so there is no cookie in the PWA, even if signup was completed in the browser
    // therefore, we need a hack: if a signed-in device requests the manifest, the start_url will contain a token
    // that is associated to the client id server side. we check for this token, and if it is present and matches a client id,
    // we give the client the deviceId cookie again
    if (deviceId === undefined) {
        const queryStringStart = req.url.indexOf('?');
        if (queryStringStart > -1) {
            const queryParams = new URLSearchParams(req.url.substring(queryStringStart));
            const tempId = queryParams.get('tempId');
            if (tempId !== null) {
                const assocIndex = tempIdAssocs.findIndex((assoc) => assoc.tempId === tempId)
                deviceId = tempIdAssocs[assocIndex].deviceId;
                tempIdAssocs = tempIdAssocs.splice(assocIndex, 1);
                // cookie should expire in 400 days, this is the max
                res.cookie('deviceId', deviceId, { maxAge: 1000 * 60 * 60 * 24 * 400});
            }
        }
    }
    if (deviceId === undefined) {
        const err = new Error("No deviceID sent");
        err.statusCode = 400;
        throw err;
    }
    
    // either way, we should now have a device id
    let device = await Device.findOne({_id: deviceId}).exec();
    if (device === null) {
        const err = new Error("invalid deviceId");
        err.statusCode = 401;
        throw err;
    }

    // ensureDevice throws if there is no valid device id in the request
    req.device = device;
    console.debug('URL ' + realUrl + ': auth succeeded');
    next();
}));

// Dynamic manifest generation
app.get('/manifest.json', errorHandlingWrapper(async (req, res) => {
    // if we have a client ID, modify the manifest to include the corresponding temp id in the start URL
    let deviceManifest;
    if (req.cookies.deviceId !== undefined) {
        let assoc = tempIdAssocs.find((assoc) => assoc.deviceId === req.cookies.deviceId);
        if (assoc === undefined) {
            assoc = {deviceId: req.cookies.deviceId, tempId: crypto.randomUUID()};
            tempIdAssocs.push(assoc);
        }
            
        deviceManifest = structuredClone(manifest);
        deviceManifest.start_url += '?tempId=' + assoc.tempId;
    } else {
        deviceManifest = manifest;
    }

    res.status(200).json(deviceManifest);
}));

// both of the middleware functions above must be registered before any other middleware that actually serves responses, in particular the static file middleware

Per default, the browser does not include any credentials (in particular no cookies) in the manifest request. To change this behavior, the manifest <link> must contain the crossorigin="use-credentials" attribute:

<link rel="manifest" href="/manifest.json" crossorigin="use-credentials"/>

With these modifications, we can finally see the PWA working as intended on iOS.

Conclusion #

We have found a way to track a user from the safari tab to the PWA installation of the same app. It is not perfect, and the method we use is discouraged by the web app manifest specifications. Ultimately, the problem lies with Apple and their decision to isolate the PWA instance from the browser, which has no real benefit in my eyes (please leave a comment if you can think of anything). It is not clear whether there is hope in getting this fixed, since Apple has a reputation for supporting web technologies (in particular around PWAs) as little as they can get away with in order to push developers to develop native applications.

Appendix: Debugging Safari on iOS #

As we have seen above, ios-safari-remote-debug-kit can be used to debug Safari on iOS from Chrome on a regular PC. It is built on:

• ios-webkit-debug-proxy developed by Google, which exposes the Safari debug interface to clients speaking the Chrome DevTools Protocol
• Safari Web Inspector, which is open-sourced by Apple

ios-safari-remote-debug-kit provides an easy wrapper around them and makes some necessary patches to the Safari Web inspector. The instructions in the repository describe everything that is needed to get the inspector to run, and once you're at that point, any guide about the Safari web inspector should answer further questions.

Comments

If you have any questions or comments, please feel free to reach out to me by sending an email to blog(at)krisnet.de.

• Previous: Investigation: Why does an Arduino clone not work as a USB-Serial