any way to use this live?

[konsumer]

When I first saw this, I thought it might be a way to open a live canvas context and draw on the screen. It seems like from the examples, it's all offline (create canvas, draw on it, then save image.)

Is there interest in a live canvas? I am pretty new to deno & rust, but have dabbled with rust, and have a lot of experience with node. I'd love to help. It'd be awesome to be able to make a fast native game with roughly the same API as browser. What would be involved? Maybe sdl2 or something with webgpu?

[DjDeveloperr]

You're right: this module quite only serves for server side image manipulation (at least for now). You can probably copy the image data to some window frame buffer or something, in fact I've done this exact before by putting raw image data on a window opened using winit, every frame. It just worked, but main problem is the copying going on when rendering it.

I do not have any experience with SDL, but I'll try looking into that.

At the moment, WebGPU in Deno does not have any way to render to a window. There were little progress to support SwapChain for plugins which would allow rendering directly to a window, but as plugins died, so is the progress on that right now (which is pretty sad, I really wanted that...)

Btw as for winit, there used to be a plugin that wrapped winit and exposed TypeScript bindings. The plugin was pane, but no longer works because plugins were removed in favor of FFI, it might be ported to use FFI though.

[konsumer]

It seems like sdl2 does not really have support for in-memory images (so I would need to make a new image file every frame, which just sounds horrible for performance and disk-thrashing.) More than that it uses a funny TCP-based interop system (rather than ffi or native plugin) so I dunno, it may be kind of a bad path. I have worked with SDL in the past, in C, and could probly make a simple little lib in rust that has a really simple api:

• make a window (with height/width/title/etc) and make image texture that fills screen
• update image texture & refresh screen (for running on change)

I like your idea of winit, too, but I can't quite tell what is needed to draw an image on the window.

[konsumer]

Ok! Here is what I'm gonna do:

I will make a deno FFI-wrapper lib for minifb that works with this lib. It has a very simple API, and is cross-platform. That way users can do offline rendering with canvas, as it is, or use that to make a window. Seems like a pretty good best-of-all-worlds solution, and it might be useful for other deno non-canvas things.

Still not quite sure how to distribute my rust lib with the deno ts file. Will need to do some research.

[konsumer]

Hmm, I fought with it for a long time, and limitations in deno ffi (no buffers, strings or arrays!) and my lack of familiarity with rust made this super-hard for me. My start to it is here. I might come back to it, but this just seems impossibly stupid to get a really simple thing done.

[DjDeveloperr]

minifb looks really nice and perfect for this case! But yeah, then there's Deno FFI limitations.. strings and buffer support will yet be added (there is open PR). If Deno FFI supported getting pointer to JS allocated ArrayBuffer, it would be honestly so good. We could perhaps just get direct Array Buffer View for the WASM memory and pass it's pointer directly to minifb, zero copying. EDIT: buffers support already does this! We'll be able to pass JS Uint8Arrays without any copying as arguments. At this point due to the limitations it does seem impossible without doing some hacky workarounds which will do only the worse. We can only wait for buffers support to be added yet. I'd be willing to contribute to the bindings too!

[DjDeveloperr]

For this purpose, I've just added a getRawBuffer method to Canvas which returns Uint8Array (pointing to WASM memory) containing underlying RGBA data. Let's wait for buffer support in FFI next 🙌

[DjDeveloperr]

Playing around the buffers PR and your deno_minifb 👀

[konsumer]

That's awesome! How did you get it talking between deno & rust lib? That is where I gave up. I had a dumb little rust main that could popup a window like that, but couldn't figure out how to send a simple string for window-title, or a buffer that could be modified in deno-space.

I guess I missed UInt8Arrays already work in ffi with rust. That is huge! What does it look like on the rust side? After passing the initial canvas buffer, how do we make changes to it?

[konsumer]

I think maybe my remaining issue is rust-side (and not knowing how to expose the right types), but it seems to me like this might be doable right now with a UInt8Array for title & the pixel-buffer, but I can't quite figure it out. buffer and usize are also not mentioned in the types, should I be reading somewhere else, or am I misunderstanding?

[konsumer]

I updated deno_minifb to follow this, and it feels like I got a bit further, but I still get this error on deno-side:

error: TS2322 [ERROR]: Type '"buffer"' is not assignable to type 'NativeType'.
    parameters: ["buffer", "usize", "usize"],

This is with deno 1.13.2 (release, x86_64-unknown-linux-gnu). Do I need another version?

It's having issue with this:

const lib = Deno.dlopen(path, {
  window_new: {
    parameters: ["buffer", "usize", "usize"],
    result: "usize"
  },

  window_update: {
    paramaters: ["usize", "buffer", "usize", "usize"],
    result: "void"
  }
})

for this:

extern crate minifb;

use minifb::Window;
use minifb::WindowOptions;

#[no_mangle]
pub extern "C" fn window_new(title: &String, width: &usize, height: &usize) -> usize {
    let win =
        &mut Window::new(title, *width, *height, WindowOptions::default()).unwrap_or_else(|e| {
            panic!("{}", e);
        });
    return win as *const _ as usize;
}

#[no_mangle]
pub extern "C" fn window_update(
    window: &mut Window,
    buffer: &mut Vec<u32>,
    width: &usize,
    height: &usize,
) {
    window.update_with_buffer(&buffer, *width, *height).unwrap();
}

[konsumer]

Ah, sorry, just looked closer at your screenshot. You are using deno-ffi-buffers-pr. Where can I find this?

[DjDeveloperr]

I've built Deno from this PR: denoland/deno#11648 also haven't got much far with rendering yet, there seems to be something up with window hanging up / not responding. I can PR my progress after cleaning it up

[DjDeveloperr]

So after investigating more, seems like update_with_buffer calls are very slow and end up hanging the thread.

[DjDeveloperr]

It was limit_update_rate causing this, lol. Now I'm banging my head against another big issue... either canvas buffer is not RGBA or update with buffer doesn't take RGBA buffer :/ I'm getting B color swapped with R

[DjDeveloperr]

So canvas buffer is for sure RGBA, but minifb seems to look for BGRA. I'm not sure how we can work around this..

[konsumer]

I think it's RGB (they call it 0RGB in source comments), but u32 vs u8, with ignored alpha-data. It seems to be a RGB buffer made withu32[width*height] rather than u8[height*width*4].

I found this in the comments of the source:

fn from_u8_rgb(r: u8, g: u8, b: u8) -> u32 {
    let (r, g, b) = (r as u32, g as u32, b as u32);
    (r << 16) | (g << 8) | b
}

Maybe the best thing would be to convert u8[height*width*4] to u32[height*width] in rust-space for performance (since buffer has to be in u8 for ffi.)

[DjDeveloperr]

Ah, I think this probably has something to do with endianness since the *const u8 I'm passing to Rust is probably being transmuted to &[u32] and causing the whole format to be reversed i.e. 0RGB/ARGB -> BGRA/BGR0.
So we'll have to do manual conversion of &[u8] to &[u32]... it will make allocation but no other way around.

[DjDeveloperr]

So it's fixed by converting it to that 👀

[konsumer]

Awesome! love to play with the code.

[konsumer]

You wanna PR to the deno_minifb library?

[DjDeveloperr]

Yeah, I'll do PR in a bit

[konsumer]

I think I would like to eventually add the other minifb stuff (mouse and key handling, mostly, would be useful.) Without the deno buffer PR being merged, and also no callbacks, it will probly take a bit for my lib to be useful, but it seems like a nice general but minimal cross-platform way to do this kind of thing in deno.

[DjDeveloperr]

I have some ideas for event loop for mouse and key handling stuff, I would love to help with that

[konsumer]

Like I said, rust is not my strong-suit. I will add you as a collab, and you can do whatever makes sense, no PR needed.

[DjDeveloperr]

Since this feature is gonna be implemented by external libs, I'll close this issue for now

[konsumer]

Lately, I have been working on node-raylib (it needs to be updated to latest raylib), as an alternative, so I am happy to transfer deno-minifb to you, since you are doing all the work on it, anyway!

node-raylib isn't a nice familiar canvas API, but raylib runs very well on the pi, and is pretty simple to use, and node-raylib is still in javascript, so it should meet my needs of "easy to use library for making minigames on pi device", even if it does require learning a new js API to draw things on screen.

[andykais]

@konsumer deno sadly does not run on a raspberry pi denoland/deno#2295. It helps to give that issue a 👍 if you want to see it worked on though

[konsumer]

bummer