Summary of what's changed in WSL since the last stable update

[SRGOM]

I'm hoping you guys have something on this already. I remember @benhillis @aseering and a few others whose id's I don't recall being the know-it-all's six months back. Do you guys have any links I can check out which tell me what's changed with WSL between anniversary and creator's updates?

[carpet92]

@SRGOM You can see here https://msdn.microsoft.com/en-us/commandline/wsl/release_notes

[benhillis]

We're also planning a blog post or two for when we get closer to Creator's Update release.

[MikeGitb]

Are you - on the long run - targeting complete (modulo bugs) feature parity with the real linux kernel or are there syscalls or functionalities thereof that you deliberately don't / can't port?

Edit: By long run I mean primarily the time horizon for the next win 10 Update after the creator's update.

[fpqc]

@MikeGitb Right now, they're working on completeness vis-a-vis 64-bit syscalls and performance (and the console team is working on a new console API), but I've been able to harass Ben into giving some ideas that they're considering for some kind of graphics support.

A very very large part of the Linux Kernel is device and filesystem drivers. In general, since Windows already drives these interfaces natively with its own drivers, MS's approach will probably be some sort of generic WSL drivers for the already-abstracted hardware. If filesystem support happens, it will be built directly into the NT kernel, not just the Linux subsystem.

[iz0eyj]

I believe that to have a good success at least Firefox and Chrome must be executable without problems by any user.
If the browser will not be used simply almost no one will use the WSL.
The order of priorities should be revised in this direction.
Of course that's just my opinion

[MikeGitb]

@iz0eyj: Why would you run one of those browsers from within the WSL instead of just using the Windows version of it?

[iz0eyj]

@MikeGitb Because that's what everyone will want to do justly. The WSL expresses the maximum of the mixing potential on the power of a single desktop Win10 with the flexibility and the large amount of them freely installable applications of Linux.
It 'will be obvious that the most interest for graphical applications, and between them the part of the eon will the graphics packages like Gimp, for some LibreOffice, for still other development environments ... but above all there will be the browser, because it is with them that the user builds the relationship with the Internet.
In this cluster now it works pretty well almost everything EXCEPT browsers, which are the most important thing.
What you can see in the picture is my desktop, in which Windows 10 and WSL are mixed to obtain a power hitherto never experienced the true union between the owner and the free software.
At times like now use Edge, other (usually when I charging machine) prefer to use Firefox from WSL ... still others are both open. not that I pay much attention to the face which I am using, for me the end is the same thing because WSL is making free software an integral part of Windows, and I think this is absolutely brilliant!
And that's what everyone will expect, and will demand, by WSL.

I ask forgiveness for my bad English.

[iz0eyj]

@MikeGitb: Another observation: since last summer WSL officially became a Win10 subsystem; This means that now we can no longer ask why launch a Linux instead of a Windows program, because now any ELF-64 program is fully in a Windows program.
This also means that from next April, the ELF-64 will land on half a billion Desktop compatibility making Microsoft the majority shareholder in the market of Free Software Desktop.
I do not know if this is a desired outcome or a simple side-effect to which he had not thought of, but that's what will happen in a month.

[aseering]

@iz0eyj -- how can a Windows be a majority stakeholder in desktop Free Software applications when it explicitly does not support desktop Free Software applications (because it does not support X Windows natively)?

[iz0eyj]

@aseering:
I do not think the majority of people very interests that is officially supported or unsupported, partly because I doubt anyone is going in and read it. It will affect only what you can do.
If a single click from the package manager will install Gimp or LibreOffice will be happy if with a click can install Firefox and Chrome (or Chromium) will even be excited.
The problem is that now Gimp and LibreOffice installing them work well, while Firefox only works kicked him and Chrome does not work.

[iz0eyj]

@aseering:
I think not support X WSL side is a great idea because it would make unnecessarily heavier the same WSL. What sense would include a second graphical environment when the original works fine and installing X on it you will get good performance?
Do not go 3D games? ok ... buy an Xbox :)

[aseering]

@iz0eyj -- LibreOffice and Gimp do not work with stock WSL either. They also require X Windows.

[iz0eyj]

@aseering:
Adam is true, three mouse clicks to download and install VcXsrv the reach of the most naive of the end user, a simple procedure that will be shown in all the magazines and blogs.
In fact it is not even correct to say that WSL not natively run the software ... it runs it but does not do graphics rendering.
Users interested in having this can be tens of millions, impossible to avoid saying that the graphics "is not officially supported".

[aseering]

@iz0eyj -- fair point that non-graphical features of the software do run natively. It's also true that VcXsrv is easy to download and install. But it's also true, in my experience, that VcXsrv is buggy :-) Sometimes it works well for me; sometimes it does not.

I'm not on the WSL team; I'm just another user. So I don't get to decide their priorities :-) But if you are correct that graphical applications were very important and lots of people were using them, then there should be should be lots of people asking for graphical applications to be fixed. So, why don't you try this experiment?: Open up the Bash on Windows UserVoice. Then read the titles of the most-popular issues. You say that there's lots of interest in graphical applications. How much interest is there in it really? How many people have asked for it?

If you know people who want it, tell them to go post on that page. Go write articles and blog posts; get people (not just you -- lots of people) complaining that they need this. If lots of people do want Linux graphical apps to work better, and if requests involving apps like Chrome and Firefox get voted much higher on that list, then I'm confident that the WSL team will make them work better.

If you're right that there are tens of millions of interested people, then it should be very easy to get a few hundred votes on an issue to add support. Is there enough interest to get that many votes?

[iz0eyj]

@aseering:
Adam, I'm a developer and an Insider, you are a developer and an Insider (much better than me on Linux) ... we are not regular users.
We come here on Github and we write something, or go on Uservoice to propose or vote for something ... but do not think for a minute that a common user to be here or go on Uservoice because never will, as never use WSL from the command line (and this you should know that better than me because you gave a lot to do in the desktop environment debugging from the first moment).
The difference is only in the degree of satisfaction that otterrano ordinary users when WSL (which is a highly important subsystem) will be proposed.
All of Win are accustomed to "click to install, click to run", and that they will want to WSL, and one of the few things not to meet this habit unfortunately there are browsers.
And browsers have an extraordinary strategic importance, because you can bet it will be the first thing everyone will expect to see work (well).
Do not think like a programmer and insider, fell in the average user's shoes, because compared to them we are a small handful.

On VcXsrv ... to me it works great, in almost a year since I use it I have never experienced a crash. The only thing that sometimes gives me problems is the "Copy & Paste".

I am sorry for my bad English.

[aseering]

@iz0eyj -- regarding browsers, why would people who are not developers care whether their browser is an ELF (Linux) binary or a Windows binary? It's one-click either way; it can be invoked from a bash shell either way. It can integrate with, for example, xdg-open, so that a Windows binary is the default Linux system browser.

You're right that we're developers. You think that lots of people want to use graphical Linux browsers. I think that most people (most non-developers) don't. So why don't we stop talking and do what developers do best -- write some code?

You keep saying that getting graphical Linux applicatuons working is a one-click (or three-click, etc) process. But, frankly, that's simply not true right now. You have to install bash itself at the command line; you have to set the DISPLAY environment variable after installing VcXsrv; you have to install a bunch of packages using apt inorder to get a graphical package manager; and you still have to launch bash apps from the command line because Linux menu shortcuts aren't added to the Windows start menu. You should make a one-click installer that solves those problems. If you do, and if it becomes as popular as you say, I will help you get Firefox and Chrome working.

[iz0eyj]

@aseering:

Yes, I thought several times to make an automatic configurator, but not the usual script. What I had in mind is a Win32 program, fully graphical and then average household users, able to take care of both the WSL as a graphic configuration environment that its subsequent maintenance.
I believe a lot in the WSL project, to have opened the doors to the support of ELF-64 on the Windows kernel opens the door to incredible scenery, and I do not see the negative.
Unfortunately I left the *nix environments for two decades, and so I do not control them more effectively, however, I sense the strength that can come from the union of two worlds when used simultaneously.

[MikeGitb]

MS has stated very clearly from the beginning that - for now - their target audience are developers that want to use Linux command line tools and/or create Linux command line applications themselves. I'm pretty sure (or rather hope), they will stay focused on that for quite some more time before committing serious resources to the support of graphical apps. Otherwise we get just stuck with half baked support for command line and half baked support for Graphical user applications.
Although I certainly wouldn't be disgruntled if WSL would receive official support for graphical applications in the long run.
Back to the specific topic of web browsers: I still don't see what advantage it would have to run e.g. FF under WSL when the exact same Software with the same functionality runs natively on windows. If this is just about having a software repository on windows chocolatey might be good enough for you.

[fpqc]

ELF64 is just the image format for Linux and other 64-bit variants of Unix executables. There's actually a way to have cygwin compile to ELF64 if you want. ELF64 offers very few benefits over PE32+ (and at a significant increase in parser complexity). Executable format isn't the problem. The reason you can't run a FreeBSD ELF64 on Linux is that the syscall interface and libraries are different.

[fpqc]

@MikeGitb Ben Hillis mentioned in a thread of mine here that they were at least brainstorming options for implementing graphics natively in Windows, so it is on their radar.

#882 (comment)

[iz0eyj]

@fpqc:
Exactly, and in addition the team is present in all open thread on the topic Desktop Manager, Browser and graphics in general.
In any case, a bug is a bug, whether it blocks an editor or browser, and as such should be corrected if possible.
With regard to the direct support of WSL graphics, personally I think it preferable to continue to keep the X server on Win32 because the only real benefit would be achieved by moving it would be to have more bandwidth, after all unnecessary accounts because hardly anyone would play in ELF-64, but the benefit would pay in terms of heaviness and can lower security.

[fpqc]

@iz0eyj uhh look at @therealkenc 's udev-stub, which makes chromium work.

[therealkenc]

@aseering

So, why don't you try this experiment?: Open up the Bash on Windows UserVoice. Then read the titles of the most-popular issues. You say that there's lots of interest in graphical applications. How much interest is there in it really? How many people have asked for it?

Answer is 426 people before it got locked.

That said, running Chrome (with libudev-stub), Firefox, LibreOffice, or Roller Coaster Tycoon on WSL is pretty pointless. Fun, maybe. But pointless.

Running VSCode with vscode-cpptools on WSL, however, is an entirely legitimate development use case. But to that end, there is still much WSL work to be done before broaching anything to do with "graphics". The cpptools guys did some nice additions recently to help with interop by the way.

[iz0eyj]

If this is so and then WSL is intended for a few thousand developers who usreanno (why ever then a developer should develop on WSL?) What is the point that I waste my time?
If the targets are only developer they can debug it alone, I certainly will not let the development of .NET / Win32 code to dedicate myself to ELF-64, so it does not concern me, and I do not care

[iz0eyj]

However it does not at all clear what "intended for developers."
A developer should convince his customers to come back to the non-GUI software because WSL does not officially support the graphics?
Or maybe the aim is to develop of WSL then to run the software on a Linux operating system that supports the graphics?

I believe that those who think one of two things is wrong, because no professional developer or software company will do one of two things if not for pure hobby level.

[aseering]

@iz0eyj -- why would I care about the market (Linux desktop) that only reaches a few percent of users, when I as a developer can develop for the market (Linux server) that can be a platform for 99+% of users?

[iz0eyj]

@aseering:
First I could also answer that "officially WSL does not support server", second thing because if properly oriented WSL make dispnibile the Desktop environment of hundreds of millions of desktop, take a market nonexistent today.
In fact WSL is a "Linux" Linux QUALITIES, both for servr that for the desktop, because it offers the direct execution of ELF-64 on a better kernel of the Linux.
Using an integrated graphical environment (with mixed desktops, not alternative to each other) you get a flexibility never seen to date, as well as using it in a server environment you will get more robust system, quick and easy to administer.

[therealkenc]

@iz0eyj

Or maybe the aim is to develop of WSL then to run the software on a Linux operating system

In a word, yes. In two words, cross platform.

• Phones/Tablets (iPhone and Android, basically)
• Cloud (Azure, AWS, Google)
• Games (which are inherently cross platform)
• Using tools that matter (gcc/clang, node, java, docker, and git)

All of which kind of sucked on Windows (read: Visual Studio) in say 2012, and developers fled to MacBooks. Microsoft has since improved this astronomically, and WSL is part of that.

What's your "professional development use case"? ELF binaries running on Ubuntu? I don't think so, unless you are in some specialized vertical scientific and engineering areas. In which case you have a real Linux box.

On the other hand, if you are developing Pokémon Go, where the money is, you are probably doing it on a Mac. For now.

[iz0eyj]

@therealkenc:
What you say is absolutely correct, but there is no single reason in the world why a developer should employ WSL instead of a Linux machine.
In this sense WSL is absolutely useless.

[fpqc]

@iz0eyj eh? I'm pretty sure at least 90% of the work that went into Project Astoria is part of WSL now.

[iz0eyj]

@fpqc: yes it is, but Astoria has never reached the public

[SRGOM]

Here's the blog post I asked for: https://blogs.msdn.microsoft.com/commandline/2017/04/11/windows-10-creators-update-whats-new-in-bashwsl-windows-console/

Thank you guys for the wonderful work. Closing

[therealkenc]

It's a nice post. I can't help by being struck with some irony though:

..most mainstream developer tools now work as expected, including:

• Dev tools: vim, emacs, nano, git, gdb, etc.
• Languages & platforms: vim, emacs, nano, git, gdb, etc.Node.js & npm, Ruby & Gems, Java & Maven, Python & Pip, C/C++, C# & .NET Core & Nuget, Go, Rust, Haskell, Elixir/Erlang, etc.

Gdb, git, node (ie libuv), .NET Core, Go and Haskell all have known unimplemented surface and/or performance issues in Creator's Update that are pretty serious blockers. Meanwhile:

...you may also have been following along with some intrepid explorations into running X/GUI apps ... we don’t do anything to block/prevent them from running... but know that we are still focusing all our efforts on delivering a really solid command-line experience

Xlib (and by extension libgtk) is a client TCP socket protocol library that has no known issues since #313 and #493 were squashed. OpenGL (mesa llvmpipe) is a number cruncher that doesn't enter kernel space to begin with.

Just sayin'.

[aseering]

@therealkenc -- well, there's graphical applications, and then there are graphical applications :-) If you're just pushing pixels, then, yeah, that infrastructure is in good shape. But I think it's fair to say that a lot of the infrastructure behind at least some popular desktop applications (fuse filesystems -- #412 , GPU acceleration, etc) has not been a focus.

Also -- are there any specific tickets that tracks node issues related to libuv? I can't find any offhand. I haven't had trouble using node for performance-insensitive tasks personally. And the deluge of commentary from node users about network-interface issues seemed to taper off after that fix landed, and nothing new has obviously taken its place :-)

[aseering]

Also, yeah, WSL team -- I do appreciate this post, and I'm impressed by all the amazing stuff that you've shipped. I agree that the post is overreaching a bit, though. For gdb in particular: The practical gdb user experience has not improved significantly in my experience since prior to Anniversary. (See #204 , for example.) I have periodically been testing it, but I haven't been reporting bugs because y'all have indicated that gdb was a post-Creators priority; no point in filing a bunch of detail tickets when the basics don't work yet, the second-order tickets will just get lost :-) gdb bugs are a major reason why, when I summarized its status to our engineering group at work, I was not able to recommend that our backend team (which develops in C++) offer WSL as a primary dev environment. I appreciate that the broader C++ dev workflow is a technical challenge to support and there aren't that many of us using it, so it makes sense that you wouldn't prioritize it. But that still means that it doesn't work yet :-)

[SRGOM]

Adding @bitcrazed who authored that post and is generally receptive to feedback. Rich, would you be open to trying to push #2041? Let us know if and how we can help. I don't want WSL to go the way of some other products where people don't try an MS product again because of the initial disappointment (no offense to you guys, you've built an amazing thing). MS has already lost the a lot of PR battles with devs, but this is one feature which will need a good community around it for everyone to benefit.

[therealkenc]

Regarding nodejs, the libuv test suite fails on the following. It also hangs on one test, and skips others. Clean on Real Linux, natch. Full results here. Any fail in libuv represents a fail in node; one would just have to write some javascript that exercises the same functionality as the libuv test.

# Assertion failed in test/test-tcp-bind-error.c on line 56: r == UV_EADDRINUSE
# Assertion failed in test/test-tcp-create-socket-early.c on line 168: r == UV_EINVAL
# Assertion failed in test/test-tcp-oob.c on line 89: 5 == r
# Assertion failed in test/test-tcp-bind6-error.c on line 60: r == UV_EADDRINUSE
# Assertion failed in test/test-udp-create-socket-early.c on line 108: r == UV_EINVAL
# Assertion failed in test/test-poll.c on line 608: 0 != uv_poll_init(uv_default_loop(), &poll_handle, fd)
# Assertion failed in test/test-fs.c on line 613: r == UV_ENAMETOOLONG

[fpqc]

@SRGOM I think it's just a nonstarter. The lxss drivers have frequently moved in tandem with improvements to Windows kernel subsystems. For example, read the post about the winsock kernel and WSL

[therealkenc]

@fpqc - You are working under the architectural assumption that the WSL winsock functionality has to live in lxss.sys/lxcore.sys, and not in a win32 driver that could be released at will. [Sorry, inside voice. Forget I said it.]

[aseering]

@therealkenc , libuv -- interesting. Apparently whatever those tests test are not things that happen frequently when using node to host simple Web services? Otherwise I would have expected to either run into it myself, or have seen reports about it. The node userbase has historically not been shy about reporting cases where they think WSL is broken :-)

[therealkenc]

VSCode is a node app that hosts a bunch of web services. You've probably seen reports about it not working. The hang in the libuv test might be related to the hang I'm seeing in VSCode. Of course, VSCode also draws some pixels along the way. But that part works fine, because nothing about drawing pixels has been broken for a long time.

[iz0eyj]

[Sorry, inside voice. Forget I said it.] :)

[SRGOM]

@benhillis is there a blog post for the latest release as well?

[sunilmut]

@SRGOM - We have one for 17677. Nothing interesting has changed in WSL in 17682. I am putting the release notes out now.

[SRGOM]

@sunilmut I meant the latest stable windows release with changes over the last 6 months

[benhillis]

This is the closest thing we have:
https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/

[SRGOM]

This is what I wanted, thanks. I'll go read the individual notes for smaller stuff. @benhillis

[SRGOM]

This is what I wanted, thanks. I'll go read the individual notes for smaller stuff. @benhillis

[bitcrazed]

Please note:

1. We've released a "What's New in WSL" blog post with each major release since its initial launch, and will continue to do so in the future
2. For an overview of what's changed in each Insiders release, please review WSL's Release Notes

[Biswa96]

There is a new folder name wslfs (just like rootfs). But release notes don't mention that. And wslconfig.exe has new option /upgrade.

I found what this upgrade option does. It change the mounted file system type of root i.e /. The file system type changes from rootfs to wslfs. Then it changes the Version from ONE to TWO in Lxss registry path.

[benhillis]

[Biswa96]

There is no explanation about this feature in 17686 "Ensure newly created reparse points are listed as such in the parent directory". Will there be any docs about it?

It's seem to be interesting. Also add the deatils of this named pipe:

 CreateNamedPipeW(L"\\\\.\\pipe\\wslconfig_upgrade_{distro_guid}, ...);

[bitcrazed]

Great suggestion for @tara-raj 😉

[onomatopellan]

Seeing it in action in build 17704 I finally understand one of the things that wslconfig /upgrade <distro> does. Upgrading to wslfs unifies the way the unix file permissions are stored on LxFs and DrvFs. Before the upgrade LxFs files used LXATTRB extended attribute to store the permissions but now both use LXGID LXMOD and LXUID.

I wonder if this finally would help the possibility of mounting LxFs folders on external partitions.

[benhillis]

Edited comment to avoid others being confused.

[mailinglists35]

@bitcrazed
Please note:

We've released a "What's New in WSL" blog post with each major release since its initial launch, and will continue to do so in the future
For an overview of what's changed in each Insiders release, please review WSL's Release Notes

it would be awesome to also publish what's on the roadmap.