Magic Command

[geyang]

Hi @n-riesco,

Thanks for this fantastic library! I am trying to run some command line command in the notebook, and realized that the iPython magic commands are not available.

For example, I was thinking about doing:

!sudo npm install -g   superagent

but then it turned out that this is recognized as invalid javascript.

Is this by design?

Best,
Ge

[n-riesco]

It is by design: Node.js (and hence IJavascript) can already run shell commands; for example:

In[1]: require("child_process").execSync("npm install superagent")

The issue here is not the lack of magics, but the fact that currently IJavascript doesn't listen to stdin and hence a sudo request wouldn't be able to read your password.

I will implement a handler for stdin in the future (if you really need this functionality, I could do it sooner), but I wouldn't advise you to use sudo from a notebook, because the Jupyter protocol would transmit your password in plain text.

[geyang]

Thanks @n-riesco ! Your comment is very helpful.

btw, have you looked at the jupyter-nodejs project? I have always used yours but I'm wondering what are the pros and cons of each. As the person who built ijs, I think you know more about the details and feature set more than anyone else! ^_^

[beilharz]

Hi @n-riesco! Thanks for this project.
interactivecomputing/ijs is another project where you opinion on differences would help me...

[n-riesco]

@episodeyang and @beilharz Sorry about the silence (other commitments are keeping me busy). I will comment on this further in the future, but for the time being I will refer you to what I wrote at jupyter-nodejs issue #4.

[awb99]

I think the magic command would really make sense; then one can add the "npm install xxx" in the top cell of a worksheet. Otherwise one has to run this installs in the terminal, and there they can get easily lost.

[n-riesco]

@awb99 The main problem with magics is that they aren't valid Javascript statements. Imagine the following case. Let's say we are using Hydrogen and the IPython kernel to write our code. Then, we end up with a file, that looks like a python script, but it isn't. And it can only be run within Hydrogen.

I also gave above a simple example of how you easy is to run npm install xxx in IJavascript:

child_process.execSync("npm install superagent");

[awb99]

@n-riesko I understand your point. Let me give my point: I am using Jupyter/python to do data mining, experiments and prototyping in python. I now have integrated your Javascript kernel into my Jupyter docker image and want to start being productive in Javascript. Now one mayor issue I use the ! Function is to install dependencies via pip (python package manager). I think the same is true for Javascript with Npm. Also on python! Is not correct syntax. The developers of Jupyter have spent 15 years in designing it so it works very seamlessly in python. I think the features that they have added will make sense in other languages too. However is not the most important feature as it can be done in the terminal also. More important is it to be able to use all. Kind of controls (sliders /combo box /...) that python has to bind to models. This should be much easier in Javascript than in python I guess. This controls cannot be done in a terminal; therefore are more important.

[n-riesco]

@awb99 I think your last comment brings up a different subject: notebook extensions and interactive output (sliders, combo box). I'm very interested in getting IJavascript and Plotly.js working seamlessly.

IJavascript already provides some tools to customise its HTML output. And although I was planning to advertise this in the next major release of IJavascript, there is new functionality already available in the latest version. See a brief tutorial here and here.

[awb99]

@n-riesco Very cool! BTW this is a working docker image that I created that uses ijavascript (it also has python 2 and python 3) : https://github.com/awb99/myjupyter/blob/master/node/Dockerfile

[crutchcorn]

I really really would like to see the magic command %paste work if at all possible. I've been looking for any repl that would allow this to work, but I haven't found one yet

[n-riesco]

@crutchcorn This isn't really a question about magics. The functionality for %paste is provided by the frontend (not the kernel).

Unfortunately, jupyter console doesn't provide %paste (or %cpaste like ipython). See the discussion here.

If you want to use ipython and %cpaste with IJavascript, (assuming the kernelspec for IJavascript is already installled) you only need to run:

$ ipython --kernel=javascript

---

BTW, %paste is a good example of why magics don't belong in kernels. They are a frontend concept and should be handled by frontends.

[crutchcorn]

@n-riesco I'm really dumb, sorry for bothering 😅

[n-riesco]

@crutchcorn Not bothering at all! And it wasn't a dumb question!

[n-riesco]

JavaScript's logical NOT operator

Linking to #157 (comment) to help people find the issue opened to discuss this matter.

[apowers313]

#229 proposes an API to add commands. A proof of concept is here and a working package can be found here.

Here's a demo of the proposal in action:

There are a few TODO's sprinkled throughout the code and it may make sense to hang the API off of the $$ global for use in npm modules. As @n-riesco comments in #229 he prefers a more strict interpreter rather than all the syntactic sugar that gets thrown into jupyter. I'm not sure if it's worth maintaining a second fork for this with all the overhead of merging in features and bug fixes, so this might be DOA.

[n-riesco]

@apowers313 The reason jp-kernel exists is so that a project like yours doesn't have to deal with forking (and also to encourage people to name their own kernels; IJavascript is a kernel with very stable requirements and API; other kernels may choose otherwise).

In principle, the main burden for a project like yours would be fixing bugs related to magic commands. I'd deal with anything related to jp-kernel.

[apowers313]

I've figured out that if I monkey patch vm.runInThisContext I don't have to build a different kernel and create a transpiler (effectively the transpiler gets run every time runInThisContext is called). I can now boil down the entire change to include magics to just requiring a module.

Technically this could be a startup script (--startup-script=path) instead of a new kernel. If I break out my code from IJavascriptEX and publish it as a separate module, it could just be loaded into IJavascript if people want magics.

The only downside I see (other than the ugliness of monkey patching) is that people who want magics would have to go through some additional configuration steps and / or pass some command line options every time they fire up Jupyter. Any thoughts about adding a --startup-module config option to ijsinstall to automatically load a specified module every time Jupyter fires up? Or maybe I can add a npm postinstall script to my new module to add some config option to the installed IJavascript kernel (e.g. npm install magicpatch configures IJavascript to load magicpatch every time Jupyter is started)?

Or some other way of making the user experience better for people that want magics?

[n-riesco]

I've figured out that if I monkey patch vm.runInThisContext I don't have to build a different kernel and create a transpiler (effectively the transpiler gets run every time runInThisContext is called). I can now boil down the entire change to include magics to just requiring a module.

That's a neat idea!

Any thoughts about adding a --startup-module config option to ijsinstall to automatically load a specified module every time Jupyter fires up?

Why wouldn't --startup-script=path do the job?
ijsinstall installs a kernelspec that launches the kernel with all the given arguments.

Another option is setting NODE_OPTIONS="-r xxxxx", but this would affect all the node.js instances (bad idea, e.g. if running Atom).

[apowers313]

Why wouldn't --startup-script=path do the job?

I think that might work, I just didn't like the idea of making the user type that on the command line every time. I can use a postinstall script with some combination of jupyter kernelspec list and modifying kernel.json to add it automatically when the module is installed. If the user has another startup-script, this trick would overwrite it... might be nice to allow an array of startup scripts.

[n-riesco]

As a user, I wouldn't want a kernel modify kernelspecs I have already installed.

And as maintainer of IJavascript, I don't want the users of your kernel to think they are using the IJavascript kernel.

[apowers313]

Here's a solid working beta version:
https://www.npmjs.com/package/magicpatch

I'm going to use it for a few weeks and then I'll bump it to 1.0. If anyone finds bugs, feel free to submit issues.

Any chance you'd be willing to throw a mention / link in the IJavascript README?