feat: add extension gallery

[CompuIves]

I think this is pretty ready for an early review now. This PR enables the extensions gallery service, and all the services it requires to function. It allows you to install extensions from the extension marketplace (https://open-vsx.org/) either in web only, or on the remote if a remote is connected.

To enable this, I made some changes:

• Downgrade semver to v5.5.0, which is used within VSCode. This way we can stub their module.
• Allow for passing a custom product configuration to the client & the server, this allows us to define a different extension marketplace (that is not from Microsoft)
• Imported all extra required services, including their stubs

[CGNonofr]

Impressive work, well done!

Is there a way for extensions to be kept after a reload?

There are networks errors, and not working images:


do you know why?

When trying to install a remote extension in the server, I get a FileOperationError: Unable to resolve nonexistent file '<project>/dist/extensions'

Also, could you please simplify the git history by removing commits that are reverted later?

[CompuIves]

do you know why?

Yes, removing res.setHeader('Cross-Origin-Resource-Policy', 'cross-origin') makes it work, because currently we don't allow loading resources (like images) from other origins. I'm not sure if we want to remove it, though.

When trying to install a remote extension in the server, I get a FileOperationError: Unable to resolve nonexistent file '/dist/extensions'

Argh, I thought I had fixed it with this line, but maybe I got the directory wrong or we need more. The explanation is this: VSCode tries to look for system extensions (which are extensions preinstalled on the system, can only be disabled and not uninstalled) in a specific directory. If that directory does not exist, it will just error out and things will be broken. The directory it looks for is in the directory where the server.js got executed from (or so I thought). It could perhaps be that this directory resolution changed after the asset registration change.

Is there a way for extensions to be kept after a reload?

I think our storage service work might've fixed this! I'll rebase soon.

Also, could you please simplify the git history by removing commits that are reverted later?

Yep!

[CompuIves]

Also, could you please simplify the git history by removing commits that are reverted later?

Will do! I'll squash everything together and separate the demo changes in another commit.

[CGNonofr]

Yes, removing res.setHeader('Cross-Origin-Resource-Policy', 'cross-origin') makes it work, because currently we don't allow loading resources (like images) from other origins. I'm not sure if we want to remove it, though.

I'm not 100% familiar with those http headers, can you elaborate on what it does exactly and why removing it resolves the error ?

[CompuIves]

I'm not 100% familiar with those http headers, can you elaborate on what it does exactly and why removing it resolves the error ?

Actually... I think I know why it works in VSCode. The ExtensionResourceLoaderService has this code when resolving a resource:

		const requestInit: RequestInit = {};
		if (this.isExtensionGalleryResource(uri)) {
			requestInit.headers = await this.getExtensionGalleryRequestHeaders();
			requestInit.mode = 'cors'; /* set mode to cors so that above headers are always passed */
		}

And our SimpleExtensionResourceLoaderService doesn't have the cors adding yet. I'm adding that now.

[CGNonofr]

I'm not 100% familiar with those http headers, can you elaborate on what it does exactly and why removing it resolves the error ?

Actually... I think I know why it works in VSCode. The ExtensionResourceLoaderService has this code when resolving a resource:

		const requestInit: RequestInit = {};
		if (this.isExtensionGalleryResource(uri)) {
			requestInit.headers = await this.getExtensionGalleryRequestHeaders();
			requestInit.mode = 'cors'; /* set mode to cors so that above headers are always passed */
		}

And our SimpleExtensionResourceLoaderService doesn't have the cors adding yet. I'm adding that now.

Well spotted!

I think now ExtensionResourceLoaderService without customization should be enough

[CompuIves]

I think now ExtensionResourceLoaderService without customization should be enough

I was initially thinking that, but the browser version is not exported. So I can do two things:

1. Copy the code over (which I've done in the last commit)
2. Add to the VSCode diff to export the service

Disadvantage of copying is that we won't keep up with VSCode changes. I can make it an export, what do you think?

[CGNonofr]

I think now ExtensionResourceLoaderService without customization should be enough

I was initially thinking that, but the browser version is not exported. So I can do two things:

1. Copy the code over (which I've done in the last commit)
2. Add to the VSCode diff to export the service

Disadvantage of copying is that we won't keep up with VSCode changes. I can make it an export, what do you think?

That's what the patch is for, we already did it plenty of times already

[CompuIves]

Okay! This is a pretty important commit, I could use your review on it.

I noticed that extensions would start disabled, when loading VSCode. I took a look at the extensionService, and they have some enable logic when the extension service boots. I copied that code, instead of using the service directly, because I believe our service has some custom logic for inline extensions. I tested this code both with the extension gallery override and without, and it still works! Now the extensions also get enabled properly (and they say how long it took to enable them!)

[CompuIves]

Okay, applied that change

[CompuIves]

Cleaned up the commits now as well

[CGNonofr]

@CompuIves I've addressed myself all my concern, I'd like to know what you think about the last few commits

[CompuIves]

Wow, thanks a lot! I'll do a review now.

[CompuIves]

Some small questions/notes. For the rest it looks good!

[CompuIves]

Awesome!! Looking forward to this change 😄

[CGNonofr]

wrongly rebased, no conflict anymore!

[CompuIves]

Awesome to see this!!! Congrats on the release! 🎉

[CGNonofr]

Awesome to see this!!! Congrats on the release! 🎉

There are still a few issues, and the demo failed to deploy, fix incoming!

[CGNonofr]

Can you check #256 ? :)