Specify how updates work

[adrianhopebailie]

Is there a reason that the spec only defines a declarative way to request that a user agent obtain and process a manifest?

Likewise, it would be useful to have an explicit way to request that the user-agent update its version of the manifest.

[marcoscaceres]

@adrianhopebailie yeah, we might need to add an explicit means to request an update to mitigate security issues: that is, we want to be able to show what's changed and give the end-user a chance to review or dismiss the update.

Excuse the crappy mock-up I slapped together below, but want to illustrate the point:

The UA would also need to show more info, like if the display mode or start URL changed, etc. Basically, anything that could be a concern to a user.

[marcoscaceres]

@adrianhopebailie, about "installing explicitly", most browser vendors would like to pursue the installabiltiy signals route. If that doesn't work out, we might need to add an .requestInstall() API. However, we have resisted that option because we don't want install buttons all over the web.

[adrianhopebailie]

What is the logic a user agent should apply to decide if the installation of one manifest is actually a new install or an update/replacement of an existing one? It would seem that this should be related to scope but I'm not sure if that's explicitly defined anywhere?

Put differently, why would you say this?

we want to be able to show what's changed and give the end-user a chance to review or dismiss the update.

If I visit the same URL a week apart I get no warnings that the content has changed since the last time I visited?

If the "effect" of a manifest is limited to functions that will not impact security or privacy then the normal controls for managing those risks should suffice surely?

[marcoscaceres]

If I visit the same URL a week apart I get no warnings that the content has changed since the last time I visited?

But you totally do. For instance, Safari shows you the following when you go to bbc.com/news:

And Chrome:

There are a lot of indicators built into browsers to make users aware of what is going on. And, if for instance, you retuned to the bbc.com and their cert had expired, the browser would warn you.

The same with native apps, like providing either, or both, an accessible history showing what's being updated:

The reason it feels different than traditional software updates is that the Web's same origin policy and browser sandboxing protects web pages from meddling with the underlying OS. Web manifest changes that:

1. it can add an icon to the user's homescreen.
2. it can set the name of the application.
3. it can take away the browser's chrome.
4. it can redirect the browser somewhere unexpected (via start_url)
5. it can hide information about the user in the start URL (or other URLs - if those URLs are rewritten and captured via a service worker, for instance)

If the "effect" of a manifest is limited to functions that will not impact security or privacy then the normal controls for managing those risks should suffice surely?

But they would impact security and privacy (see all the recently added security and privacy notes in the spec). So doing a "sneaky" switcharoo of the icons and name of the icons that the user is not aware of would be really bad. At least, at Mozilla I know we would be absolutely against just allowing applications to switch any part of the manifest without the user knowing about it (and modifying them).

[adrianhopebailie]

I'm sorry but I think of myself as a pretty "experienced" web user and I definitely don't consider, the fact that a browser indicates to me a page is being fetched and loaded, as a hint that that the underlying content has changed. I think that is over-reaching a little.

Are you suggesting that an average user of the web understands the difference between a page loaded from the cache and one fetched from the Web and ALSO understands that the latter MAY mean the code of the Web page is different to the last time they loaded that page?

The fact that start_url can change is the only reason all this complex change control is required at all. Wouldn't it be better to bind a manifest to a start_url?

Perhaps if you could answer my initial question it would help me understand this better:

What is the logic a user agent should apply to decide if the installation of one manifest is actually a new install or an update/replacement of an existing one? It would seem that this should be related to scope but I'm not sure if that's explicitly defined anywhere?

It seems to me that it's possible to have a URL that falls under overlapping scopes of two different manifests. In which case, if I land on that URL, which application-context, if any, is loaded?

[marcoscaceres]

It seems to me that it's possible to have a URL that falls under overlapping scopes of two different manifests. In which case, if I land on that URL, which application-context, if any, is loaded?

We haven't worked out what to do about overlapping scopes. Hope to get to that soon.

However, the problem of requesting an explicit update remains. Relates to #384

[boyofgreen]

Binding to a start URL doesn't quite fit the grander idea of scope either. Since I may "deep link" into an app, the scope needs to cover all that would be part of the app. The idea we are trying to solve in when you have an application like gmail, where mail.google.com is the scope, but contacts.google.com should also be in scope. In contrast, docs.google.com and www.google.com should not be in scope.

[benfrancis]

So now that the update text has been removed, as far as the spec is currently concerned a manifest gets set at install time and can never be updated? Although I'm aware of the risk of app identity theft, I'm not sure this is better than what was there before. Seems like a high priority to address updates.

[marcoscaceres]

On 27 Apr 2016, at 9:13 PM, Ben Francis [email protected] wrote:

So now that the update text has been removed, as far as the spec is currently concerned a manifest gets set at install time and can never be updated?

Correct.
Although I'm aware of the risk of app identity theft, I'm not sure this is better than what was there before. Seems like a high priority to address updates.

Yes. I agree.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[cheshrkat]

I just ran into this while spiking the use of app manifest. It's great to get onto the device, but the spec definitely needs to talk about the way(s) you can update the content.

[RobDolinMS]

I'm thinking there are two possible scenarios here:
A) Every time the UA boots the app (and is connected to the Internet), the browser checks for an updated Web App Manifest
B) The Web App Manifest includes something like a time-to-live (TTL) value and UAs should not run the app if the TTL has expired

Is one (or both) of these what you're interested in @adrianhopebailie ?

[kenchris]

AFAIU, the Android team is looking into updating the app data given manifest changes as part of the work on WebAPK. It would be great to have their feedback. @PaulKinlan ?

[marcoscaceres]

I think we should limit the declarative features of the manifest as much as possible, particularly ones where we can do better (for developers) by making them programmatic. I believe there is, nevertheless, and as @RobDolinMS suggests, the opportunity for a good mix of collaboration between the browser and the web application: where the browser can automatically check for an update (as service workers do every 24 hours) - or the developer can request an update, through an API.

As I argued in #446 (comment), the end user needs to be in control of the update process (similarly to how they go through the installation process), because we are asking the user agent to make a pretty scary change at the OS level (the application icon, name, start_url, and possibly other things can change).

[marcoscaceres]

@PaulKinlan, feedback would be tremendously useful, particularly if there is going to be some kind of impact on the API surface. It may be we reuse the onbeforeinstall infrastructure here to perform the update - which would make most sense.

[nunoperalta]

It is really unfortunate we can't update our PWA. This can be considered a stopper in many projects.

manifest.json should have a "version" that can be updated, and if browsers detect that that version number has changed, they can ask the user whether they would like to receive the update or not.

I was asking about this in a StackOverflow question... https://stackoverflow.com/questions/47762022/apply-changes-to-web-app-manifest-on-an-installed-pwa

[marcoscaceres]

Version is orthogonal to updating. Nonetheless, update is important and something I hope we solve in 2018.

[dominickng]

Note that Chrome now will poll the web manifest of an installed PWA on Android from time to time and automatically update if changes are detected. A more explicit updating mechanism may be better though since Chrome's process is entirely user-transparent.

[nunoperalta]

I had tried yesterday to make an update to the app name in the manifest and then change the link to "manifest.json?v=2", which ensures the browser will download the new version. However, the app name did not change.

[dominickng]

Changing the manifest URL shouldn't be required (and it may even interfere with the process).

Assuming you have the app installed via the improved add to home screen (i.e. it appears in the Android app list), when you launch the app Chrome will fetch the manifest, parse it, and compare its data to the data we have baked into the app. If things differ, an update is requested and will go through when possible. Hence this being entirely user-transparent, it should just happen in the background.

[ewsizeit]

What URL is Chrome using to fetch the manifest? Is it the original url the manifest was fetched from when it was installed or the start_url member?

I see a danger in updating using the manifest presented by the page in start_url. What if you make a mistake in an update, for example a misspelled domain name. All installed apps would update and then have no way of fetching the right manifest again. Basically locking yourself out of your installed app base.

Irregardless of the implementation details, that's great news. We can start producing apps without worrying that they will be impossible to update. The ideal would be to specify this in the standard.

[madmoizo]

when you launch the app Chrome will fetch the manifest, parse it, and compare its data to the data we have baked into the app. If things differ, an update is requested

@dominickng I'm afraid it does not work or please provide more details about the update process. I tried with a static name manifest.json and with a dynamic one manifest.[hash].json and Chrome does not update the manifest.
I opened an issue with a screencast example (even if it's more a spec issue...) https://bugs.chromium.org/p/chromium/issues/detail?id=806518#c4

[Edit] comment from blink team:

We check for updates when a webapk is launched, but throttle to once every 3 days (IIRC)

[mgiuca]

@alancutter has been working on the Chrome manifest update logic, and should be able to answer questions on this.

[alancutter]

Chrome compares the latest manifest against what is currently installed throttled to once per day. If changes are found it will wait until all app windows are closed then update its web app installation to reflect those changes (app name excluded).

I can imagine sites that want control over this behaviour would like a checkForManifestUpdates() function and a beforemanifestupdate event that can be event.preventDefault()ed.

[g-ortuno]

I suspect we do something different on Android though.

[alancutter]

Android does the same except the check is done on a central server and manifest updates are pushed out globally. This would require major changes to support any kind of update API.

[dominickng]

@alancutter @g-ortuno the manifest fetching and parsing for updates is still done on-device on Android, with parsed values and icons being sent to the server to compute whether or not an update is needed based on those. Supporting explicit updating would probably just require some extra flag to "force an update", which may already exist.

The implementation side should be orthogona though - here we should be answering the question of whether this is a sensible and desirable functionality that should be supported.

[marcoscaceres]

Some documentation for Chrome for Android: https://web.dev/manifest-updates/

[evanstade]

Version is orthogonal to updating.

Can you explain what you mean by this @marcoscaceres ? It seems straightforward enough to define a rule such as, "if the user agent finds that the version in the manifest is higher than the version at last install/update time, it MAY apply changes found in the manifest to the installed app"

I imagine a version field would also be useful for orthogonal things, like helping developers identify the code that's being run (e.g. on a user's system when a bug is reported).

Chrome compares the latest manifest against what is currently installed throttled to once per day. If changes are found it will wait until all app windows are closed then update its web app installation to reflect those changes

(Desktop) Chrome is forced to not just compare the manifest to the last seen manifest, but also re-download and compare all images every time, which is a fairly intensive operation. Hence the throttling. But if Chrome were able to skip the update by avoiding this step and only updating after an explicit update signal (via version comparison), the throttling might not be necessary. The throttling is a known/common source of confusion, to the extent that we have documentation that we frequently point confused developers to.

[dmurph]

Quick thoughts on version:

• Would be a nice hint
• Relying on it would obviously break existing use cases, but we could throttle non-hint cases to reduce resources, and guarantee resources
• Possibly a privacy risk - version could be a user id. Would have to be not accessible, and even then, it reveals some info about the value when the manifest updates?
  • but I guess we could reset version on browsing data cleared?

I see value as an improvement of update predictability.

[aarongustafson]

Related: https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/VersionHistory/explainer.md

[marcoscaceres]

@evanstade wrote:

Can you explain what you mean by this @marcoscaceres ? It seems straightforward enough to define a rule such as, "if the user agent finds that the version in the manifest is higher than the version at last install/update time, it MAY apply changes found in the manifest to the installed app"

Sure, the problem is: how do you compute "higher"? Do you do a simple ascii comparison? Or would you require implementing something like sem-ver, which would lead to a lot of complexity (e.g., "01.01.10-beta-1" GT "1.1.10-beta-1""). The best we can do is "is it different"?. What if people don't want to use sem-ver? and so on...

I imagine a version field would also be useful for orthogonal things, like helping developers identify the code that's being run (e.g. on a user's system when a bug is reported).

Sure, but that's equally handled by "version" being an opaque string (as in, "oh, looks like that's good ol' version 'vva.234fad' at it again").

[aspiers]

I'm strongly in favour of a versioning scheme to enable predictable updates. As we all know, versioning is a solution to this problem which is used and well-understood across the whole industry, and it should be easy to retain backwards compatibility.

I would suggest using semver unless there are compelling alternatives or reasons not to (which seem unlikely to me at this point). Semver is a very well-trodden path and avoids reinventing wheels. I don't think it's compelling to say that semver leads to a lot of complexity, because it's essentially a very simple concept which is already well understood by many developers and already has multiple implementations.

In other words, update mechanisms via versioning is already a solved problem, so I think it makes sense to treat it as one rather than looking for a new solution. Reinventing wheels would mean a whole new updates and/or versioning scheme for developers to learn, even though it would only exist in this one place.

[evanstade]

The best we can do is "is it different"?

I think we'd be OK with this and still using it to help manage updates. A lower version number might indicate that a user ought to be down-graded and I can't think of a reason not to support that.

We are concerned about maintaining functionality (at least for now) for apps that don't specify a version at all. Thus, Chrome's ideal behavior would be something like:

• store the version specified during most recent update/install
• when processing a manifest for update
  • if version is not found, use current update procedure (i.e. compare all manifest fields and image contents to determine need for updating, but throttle this to ~once/day)
  • if version is found
    • and matches saved version, do not update
    • and does not match saved version, do update, without throttling (probably -- might need to consider some amount of throttling still)
• (possibly) display version in user-visible app info surface(s), to assist with debugging in the field

If I'm not mistaken, the only part of the above behavior which turns into actual spec is

if version is found, and matches saved version, do not update

App developers should be motivated to use version because it helps reduce unnecessary bandwidth usage (i.e. the user agent does not need to download all images again to make sure they haven't changed) and makes updating more predictable (reduces need for user agent to throttle).