Is it okay to implement keystroke record / playback?

[DeeNewcum]

It would be nice to have complex macros like the QMK / ZMK keyboard firmware, but I'm not ready to implement that. But I'm definitely interested in keystroke recording/playback, as a primitive macro system. Would that be welcomed as part of this project?

[Gadgetoid]

That might constitute scope creep, but it's an interesting idea.

Of course the ultimate evolution of this would be to display icons on an attached touchscreen (totally not biased about which one we might choose) and enable a kind-of Stream Deck style macro system.

Heck even Keybow could be attached to the Pi 400s GPIO and supply some additional dedicated macro keys.

I think it's better implemented not directly in pi400kb but as a way of getting input data in/out of pi400kb to process elsewhere. Like a pipe for key events?

The biggest single issue with any overloaded functionality is how we pick a shortcut key combo that takes you out of regular keyboard mode into macro mode. Otherwise all your input goes both ways and weird things will happen.

There's an open issue - #27 - requesting a change of grab/release shortcut, so perhaps the first step on this road is to figure out:

1. How to support arbitrary keyboard shortcuts to do higher level things like grab/release, macro/normal
2. How to make it easy for the user to change those

Then we can say, perhaps, that in "macro" mode key events get sent to another service (written in C, Python, whatever) for processing and recording. Keep this codebase clean but extensible.

I'm mostly rambling here but the TLDR is:

1. This sounds interesting and has some good benefits for cool projects
2. We should try to implement it in an extensible and unobtrusive way because it's nonetheless a bit out of the remit of pi400kb

Sheesh even my TLDR is long.

[DeeNewcum]

Some points:

I predict there will be demand for QMK/ZMK-style macros (e.g. keyboard layers, double-tap keys, etc), and maybe even for hooking up an HDMI capture card and responding to the main computer's on-screen events with macros. For what it's worth, my raison d'etre for being interested in this project is because my employer doesn't allow any software to be installed. Normally I would use AutoHotkey/AutoIt to accomplish my automation tasks, but my employer's restrictions demand that automation has to be done outside the main computer. (right now, I'm not interested in implementing much beyond just record/playback macros, but I can see people being interested in more)

The LOC to implement a module system would be just as much to implement a record/playback system. I'm working on the record/playback now, and I don't think it takes much to implement it.

If someone else wanted to implement an extension system, I'd be willing to implement my change around it. I don't know that I'd implement an extension system myself though -- I've never implemented one, and am unsure about some of the implementation details.

My conclusion is that maybe it's easier for me to just maintain a fork alongside yours. I don't know, that's not the most preferable solution either, but maybe it is what it is.

(Also, it might be possible that someone wants to re-implement this in Python, which would make macros and the HDMI capture card much easier. This is another reason I don't think that spending time on an extension system here would be useful.)

[DeeNewcum]

The implementation is basically complete. This is the code. *shrug* If you'd rather not integrate it into the main project, I can maintain it as a parallel fork.

[Gadgetoid]

That's a lot more lightweight a macro setup than I was envisioning. I think the problem with bringing that upstream is twofold:

1. It's incomplete. Probably fine for a lot of people, but you can guarantee someone will come along and ask for X and Y and before you know it the entire codebase is an unmaintainable nightmare of macro recording.

2. pi400kb is a dump pipe to clone the attached USB HID keyboard over to USB OTG and trying to do anything more is probably going to end up incompatible with that goal in unexpected and very breaky ways.

3. Oh and a bonus no. 3 - between Keybow (https://github.com/pimoroni/keybow-firmware) and PMK (https://github.com/pimoroni/pmk-circuitpython) I probably don't need to be maintaining another macro keyboard platform.

P.S. - For disambiguating the keyboard input, there's a complete reference map of HID scancodes to input event codes here you could probably borrow: https://github.com/torvalds/linux/blob/master/drivers/hid/hid-input.c#L27-L44

And then input-event-codes.h provides all the K_ key codes: https://github.com/torvalds/linux/blob/master/include/uapi/linux/input-event-codes.h so you don't have to copy them verbatim into the source.

TLDR: Keep the fork. It's okay to have different goals!

[DeeNewcum]

And then input-event-codes.h provides all the K_ key codes

input-event-codes.h's values are different from the USB HID values. For instance, input-event-codes.h defines escape as 0x01, but in USB HID it's 0x29.

pi400kb is a dumb pipe to clone the attached USB HID keyboard over to USB OTG

It's a dumb pipe running on a very capable computer.

[Gadgetoid]

input-event-codes.h's values are different from the USB HID values. For instance, input-event-codes.h defines escape as 0x01, but in USB HID it's 0x29.

Yup that's why I linked the HID scancode to input code lookup here - https://github.com/torvalds/linux/blob/master/drivers/hid/hid-input.c#L27-L44

The raw scancodes don't correspond to any particular letter (it varies depending upon region), so storing constants for them is a recipe for angry Belgians.

[DeeNewcum]

so storing constants for them is a recipe for angry Belgians.

Ahh, I see. Thanks for the info.

[joeridillen]

Still this opens nice options of turning the pi into a inconspicuous HID-keyboard masked rubber ducky, not only grabbing real keyboard input, but sending prerecorded commands to the connected system as well f.i. X minutes after connection (or maybe even using Wi-Fi to do it live?).