-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: Logical or Physical Pixels? #108
Comments
Pugl intentionally does not have any such distinction in the public API, and if it does, it's a bug. If you ask for a view of 512x512 pixels, then you get 512x512 pixels. There are only pixels. There are no logical pixels, or physical pixels, or quasi pixels, or pseudo pixels, or any other kind of pixels some deranged mind in Cupertino thought up because they need to sell "Retina" without actually making any of the software changes necessary to properly support different DPIs. To be fair, Microsoft fucked this up egregiously too, if in a different way. They both have their reasons - well, reason singular (backwards compatibility for proprietary applications) - but that reason doesn't apply here, so there's no reason for pugl users to suffer that pain.
What exactly does "visually" mean? It sounds like you want it to mean, well, not 640x360 pixels? At the end of the day, you have to actually draw stuff, and I don't think asking for a window to be 640x360 pixels and actually getting 1280x720 is in any way simple. That's both confusing and error-prone, and in reality 99% of Pugl apps would simply be broken on MacOS if it were that way. A window that literally has 640x360 pixels in it is, well, 640x360 pixels big. That's the number of pixels you need to draw, that's the number of pixels that are visible on the screen, that's objectively the number of pixels. I guess that's "physical", but they're just... pixels. If you want the window to be bigger than that because the DPI is very high, well, then yep, you want the window to be bigger than that, so you have to... make the window bigger than that. So, yeah, you need the scale factor for that, but you need the scale factor (better the DPI, but we all fucked that one up too, so oh well) regardless. I don't think pretending pixels are anything but, well, pixels actually makes anything easier here. It just makes it worse. Needing to size things based on the DPI is inherently a part of dealing with variable DPI, unless you provide a mode where old apps that assume a fixed low DPI are automatically scaled up. That's why MacOS does this, forever punishing DPI-aware code to achieve it. Pugl doesn't have any legacy like that to support, so it has no such mode, so there's no need to punish users with made-up pixels. Whether the current API is actually adequate for this is another question. I'm certainly not attached to it and |
Just trying to help.... thank you that answers my question. It's all raw pixel sizing, perfect, now I know for sure. The docs aren't clear on that though. Visually means exactly what you said it does... the thing GUI programmers actually care about: the perceived size of widgets and windows the screen we're looking at. And I use the terms logical and physical just so you know what I'm talking about. It's impossible to consistently, from client code using pugl, to define a default size using So in short, I don't care that pugl doesn't try to hold my hand with scaling. But I DO want LVTK to be able to do so for it's users... and not have jumpy windows initially because scale factor is queue'd up in an event somewhere. |
I'll do some RND one of these days soon. Configure sounds right to me (or maybe even Realize) if scale has to come in an event. Will be interesting to see if that affects "jumpiness" |
Oh, and also. sorry. I don't want pugl tap dancing around so called logical pixels publicly & internally either. We're in agreement on that. But scale factor IS absolutely needed waaay early from a toolkit developer's perspective. |
I'm not yelling at you, I'm yelling at Apple :D It annoys the hell out of me how much of a mess this stuff is, across every single platform. X11, for its part, adopted the "let's just pretend the DPI is always 96 regardless of what the DPI actually is". It's effectively impossible to render text at a proper size on all three, it's all so embarrassing. Not sure about Wayland, I haven't done any scale stuff there yet. Anyway... Yeah, needing a view for puglGetScaleFactor is the main problem. IIRC it's actually accessible from the world on all the current platforms internally, the API is just wrong. Adding it to the configure event is necessary to support moving across displays and such (and just generally fits the idea that everything about the visual configuration of a view is in the configure event), but that would have the same problem, so it's not really an alternative so much as an addition. The tricky part is there there can be multiple displays, and they can have different scales, so conceptually there is no "world" scale factor at all. Pugl currently has no notion of a "screen" though. One would also be necessary for really doing fullscreen or mode switching or things like that properly as well. I fear the complexity of adding any sort of actual display API, but maybe just an integer display ID or something works. I haven't made it that far. Could also just ignore this part and add a |
Ok, good. I was worried there for a second :) I did actually set up a test case where the scale factor is delivered in a Configure event. I must admit, I can see your point now, This also helped me uncover areas in mac.m where over scaling or under scaling was happening depending on if TopLevel or Child view..... fuller report on that coming soon. My test apparatus lives inside Roboverb. The pugl code is this one: https://gitlab.com/lvtk/pugl/-/compare/main...scale-as-event?from_project_id=11851529 . Not sure how correct any of it is yet as I've only experimented on a Mac too. Based on what you're saying about the platforms having problems themselves.... Is worrying about scale + DPI just a non-issue? What about provide an assumed scale (what we have now) + a "quality" factor or something to hint that client code might need an adjustment scale THEY come up with? I've seen in more than one GUI toolkit where scaling is constantly adjusted to compensate for differing platforms immediately before initiating widget-tree rendering (in logical pixels :)) (not an expert in that area, but do actually read source code like a book some times). |
I also saw this: |
At least one configure event must always be delivered before actual exposure, so there shouldn't be any jumpiness at all as far as the window contents goes. I'm not sure about the window size itself, that can't be controlled so tightly. Dynamic resizing is certainly best avoided in general (I wish it didn't have to be in the API whatsoever, but that's a fantasy), so having both is still probably better, but... yeah, maybe just "you have to deal with this on configure" isn't so bad.
The whole point of the scale factor is that the applications doesn't set it. If an application wants to have further internal notions of scale (or zoom, or font size, or rotation, or whatever else), that's fine, but that's their problem and has nothing to do with Pugl. |
Also I ran into some problem or another on some platform or another trying to put the scale in |
Hmmm I'll keep playing around with it and see what I can break! Scale + "reconfigure" flag or return state so internal pugl can react. Code example on that one says it all. This is detecting a change in pugl's scale factor, then wanting a new size scaled up so widgets can work in user space. static PuglStatus configure (View& view, const PuglConfigureEvent& ev) {
// ....
auto& widget = view.widget;
if (view.pugl_scale != ev.scale) {
VIEW_DBG ("pugl: scale changed: " << ev.scale);
view.pugl_scale = ev.scale;
// The pugl frame needs adjusted to fit widget size in user coords. So
// request one and wait for the next configure.
VIEW_DBG ("pugl: resize frame for widget: " << widget.bounds().str()
<< " to PuglCoord " << (widget.bounds() * view.scale_factor()).str());
auto r = widget.bounds() * view.scale_factor(); // this correctly scales a rectangle.
puglSetSize (view.view, r.width, r.height);
return PUGL_BAD_CONFIGURATION; // instead of bad config, reconfigure wanted.
}
// ..... no re-config needed scale down and initiate widget drawing.
} |
An explicit reconfig wanted lets pugl know whether or not it should do whatever is does next, or handle a reconfigure. bad config could work too I guess, but thats more of a "find a graceful fallback" deal rather than a "standard operating procedure" reaction. |
Yeah, IDK about this one. I could also see a new |
That's what |
Returning to this on my Mac, what I currently see with The demo requests a default size of 512x512 (meaning physical pixels, or just pixels), and the window is initially that size, i.e., the cairo surface is 512x512 pixels. Expose events are as well, i.e., the full view expose events are 512x512. Mouse button, motion, and scroll events are reported relative to that, i.e., the center of the view is 256x256. This is all as expected. Do you have a case where Pugl itself is accepting or reporting inconsistent units? Of course, this demo isn't using the scale for anything, in the Pugl API 512 means 512 physical pixels on all platforms. If you want it to be twice that large, then you need to literally do that. Then it would be 1024x1024. tl;dr: I don't see a problem here. Regarding initial jumpiness, don't oscillate, choose the size you want from the start (puglGetScaleFactor works before the view is realized). |
... okay, well, not with the units exposed anyway. I think the thing that can't work here is changing the scale, so dragging a window to a higher DPI display and expecting it to be twice as many physical pixels large. I have no idea how that can work (for one, a window can be on several displays at once), but it doesn't sound like fun. It might be possible to make it work to detect that case and reconfigure yourself when necessary, but (as you discovered) it will need special handling. When using physical pixels everywhere, when a window is dragged to a different display, all Pugl can do is report what happened: the same old size in physical pixels, with a new position, and maybe a new scale factor because this display is different. If so, if the application knows the size it is at the current scale, it can somehow enqueue a request to enlarge/shrink, but it will have to be careful to track the scale it's adjusted to to not end up in disastrous loops. If Pugl used logical pixels, that problem of needing to reconfigure yourself goes away, but (in addition to having this complexity) then it would be expected that your underlying surface has changed in physical pixels, and that is an event that needs to be communicated and handled correctly. tl;dr: Adjusting scale for different displays dynamically doesn't work, and I'm not sure it can work with the current API, but it's probably a pretty involved thing to implement. I think it should be possible to, say, be twice as large on a system with a scale factor of 2 though. It's just a bit of a PITA to do, but it always is in some way or another 🤷 |
best solution I can think of with the current design using raw pixels and configure event, is to regularly call "get scale factor" at regular intervals and do something when it changes. |
related to this are full screen enter/leave events. on wasm/emscripten the dimensions of physical vs logical pixels are not always the same, browsers scale automatically as needed. with the exception of specific full screen modes (there is more than 1 yes). the client side can request a pixel-perfect rendering for the full screen mode, which changes in "scale factor" by no longer having the browser do automatic scaling but letting the client side handle raw pixels. do we have a way to notify of full screen enter/leave events? the scale factor changes could be handled in a similar fashion. |
Yeah... and if you click on the image, you can see that the screen capture is itself 2x resolution (can tell by it's over-sized appearance in the image viewer) |
I guess it depends on how you hook that up to something to draw. The current Pugl philosophy is there's just one kind of pixels and they're both the ones you draw to and get events about. I guess that unit would be "the thing you're drawing to a Canvas in" or something in the browser. I haven't put Pugl in a browser yet (although it'd be nice). I'm not super attached to the physical pixels philosophy or anything, but it seems by far the simplest thing, and using multiple units in some places is a change that would really need to be figured out. There is a mechanism for reporting fullscreen now, yes, but this is more of a window management hint thing. I'm not sure if it could be used here. Conceptually, I still think scale in a configure event makes sense, but it would need to be figured out exactly what everything means and how the app can react to it. Personally I won't lose any sleep over dynamically changing scale not being supported, but it'd be nice. I suspect the physical vs logical thing might be a red herring though. Is the actual tricky part here "how to maintain a window size based on some dynamic settings, like a scale factor"? A scale factor is one, but there could be many: a user preference in your own app, or maybe you have a text-based view of the world and want 80em width in the current font, or whatever. It seems like the same sort of problem regardless of what the factors are: you sized to The size (... and maybe also position?) could become virtual, but then screen-relative coordinates presumably do as well, and, uh... other to be determined things... and I feel like the problem just gets spread around even more. I have no idea how all of this should work beyond the crude simplifying assumption "get a scale factor at startup, and use that until exit". It's tricky whenever both users and code can resize things, even when there isn't a bunch of math going on. Maybe I should look at LVTK to see what the problem is specifically? I'd expect toolkits and such built on Pugl to just always use physical pixels, the only pixel unit in the Pugl API. There's also the scaling factor, and basing your size on that can be tricky. I think, currently, it should work fine to allocate a view, get the scale factor, use that to determine your general scale from then on, and set the view up and proceed to handle events and draw as usual. ... and handling changes is completely un-figured-out. I don't know if my deliberately simplified model of "Pugl-space" is the best, but the idea was:
On most platforms, this is the simple thing, it's just directly talking about real pixels all the time. On MacOS, Pugl itself has to do work to maintain the illusion. Without equality testing, this is a nightmare. Maybe if the scale factor is quantized it gets easier? Let's make life easy and say integer, then:
The last step needs to be figured out because trying to resize while handling a configure is... problematic. I don't understand what should happen with the configure event on MacOS when the window is dragged to a new screen. Does the size suddenly scale, or do you need to resize yourself, or...? We need to figure out consistent rules for what should happen in the case (based on what's actually possible on all the supported platforms). I imagined it would just be "everything stays the same, but the scale factor might change, deal with it". Maybe that doesn't work on MacOS, or even other places, I'm not sure. We want to minimize the chances of (the client code) "doing" a resize, I think, but I don't know what the platforms do, and I don't know how size "hints" (many of which are restrictions) interact with any of this. I think vaguely speaking what the API needs is a distinct time and place to catch this happening once and do everything that needs to be done (because it might include rebuilding renderers, reloading assets, resizing views, or whatever else), and the values and "meaning" of the sizes in a Relevant platform/implementation stuff I don't know: is it possible to detect a DPI or scale or whatever change reliably? what "magic" resizing happens, if any, when windows are moved to screens with different scales?
What does this mean from the POV of the Pugl API? What are the sizes? I would expect the scale factor of 2.0 on the lower resolution monitor would be massive. It sounds like the app is successfully living in a universe where everything is always at 2.0 scale in one way or another, so... yay? I actually have laptops with working HDMI again now, but haven't bothered to test any of this. |
It means no changes needed to support a mixed monitor setup... assuming fractional scaling is enabled.
Yeah me too. But apparently X11 uses a 2.0 scale for both and does it's own magic in the background. I don't think Mac/PC will be that easy though. Haven't tried this setup on those yet, or at least not for a few months and I don't recall what happens. |
Especially if there's magic scaling goes on somewhere, this sort of thing doesn't actually say anything about what's happening from the perspective of the program. This needs a lot of testing on diverse setups here, but not like... "it works for me" kind of testing, especially with several layers of complexity going on at once. We need to collect numbers and concrete stuff like:
Where "what happens" means:
The last two aren't currently user visible, but apparently something that can get weird too, so I guess one of the demo programs will have to be hacked to print them or something. My best guess for the above setup is that the scale is always 2.0 and sizes never change based on the, er, "actual" scale of the display a window is currently on. This implies that the windowing system itself downscales windows which are rendered at a higher resolution, which seems surprising to me, but hey, it's possible. ... it could also not be doing that and look exactly the same way, is the thing, especially since on top of whatever the system is doing, LVTK is doing things with scale on its own. Basically we need a big table of the above for all the platforms to make any sense of scaling. |
If I plug in my external display to this MBP, by default it also has a scale factor of 2 because MacOS sets it to run at half resolution (for "larger text") by default. Which is odd, but whatever, I can change it in settings to run at native resolution, and then it has a scale factor of 1.0, different than the internal display (which has 2.0). If I start the cairo demo on the external display with the When dragging across the DPI boundary from the low to the high res screen, from the POV of the Pugl program currently, the size doubles from 512x512 to 1024x1024 and the scale goes to 2.0. Going the other way, the opposite happens: size back down to 512x512, scale back down to 1.0. Drawing doesn't actually start happening at the new size until you release the drag. The Cairo surface is always made to the last configure size (this is Pugl internals), so that changes size, too, but that's currently using the backingScaleFactor of the mainScreen always, which seems wrong, but does seem to produce consistent behaviour (according to the current "everything is physical pixels" API) anyway, if with a bit of jank while in the process of dragging windows between displays. |
There's a bunch of issues with multiple display support in general, like... well, all coordinates. What's the position of a window? Since mouse events can be outside the view, their position is also meaningless if the view's is. On MacOS at least, there's a lot of transformations, and some are nonsense in the presence of multiple displays. Not sure about other platforms, IIRC on X11 you typically have one big virtual screen, which should work fine, no idea what Windows does. Multi-display support is a whole thing that basically just isn't there, without even adding different scales to the mix. They're related in the code because you can just search for all the places a screen or From an API perspective though, on MacOS, as far as I can tell, this would just fix positions so they make sense. The scaling going up and the window getting twice as large is what I expect from the Pugl API because that's what's "actually" happening with respect to physical pixels. It's currently impossible to have size constraints and deal with changing scale, though. That affects a lot of plugins (although it's generally better to be as flexible with size as possible), and I don't know how it should work. This is an objective flaw with the current API, so to speak: it can't work, no matter what you do. I think it either needs the introduction of "logical" pixels ala MacOS, or a reliable hook or event because only the client code can deal with it properly. I think I prefer the latter for two main reasons:
I just don't know how or if the second is possible. It has a lot to do with multi-display support, but other hooks might be necessary. We'd need to reliably detect any time the scale (or maybe other things that can affect size) changes, and communicate that in an event where the handler is allowed/expected to reset hints and such as necessary. |
Returned to tinkering with this, and increasingly feeling like multi-display/DPI support is just too much to fix right now in one shot. Probably best to figure out how to do the concrete thing of have the window size/etc hints based on the scale (and not dealing with the possibility of it changing for now). Poking through lv2kit, most of the confused things are around hints and 2 or 3 different notions of scale that get conflated, but it looks like it should be pretty easy to sort out ignoring the possibility of it changing for now (which seems to be reality on at least X11 anyway). |
Welp, I stayed away for a little to long. Yeah, there really is a lot to try and cover from inside Pugl itself. I agree, unless I read everything wrong, most things probably can be handled by the client code utilizing pugl. I liked the idea of starting a big table of system/setup behaviors in certain scenarios. Something like that will help a bunch finding commonalities and differences on the platforms. I'll say though... on windows.... even Chrome has big problems if you change screen resolution, or move it from window to the other. Could we maybe use the Pugl wiki to start filling in a table? A section with notes on how other applications are handling this too. |
Yeah, it's not easy to deal with changing things here. I've poked at it a bit but never come to something good yet, although it's been on the backburner for a while now. I still think that the meaning of the hints is the only thing really fundamentally in the way in the API, but I don't know what to do about it. I don't really care about moving between screens all that much at the moment, practically speaking, we need to sort out the initialization problem, which affects everyone. Multiple monitors with different display factors... meh. Would be nice.
I generally prefer to use Gitlab as the main, but whatever I guess, go for it. |
For these functions, what should users of the library expect: to use a logical or physical coordinate space? Should they all be the same?
I assume logical since that's the natural way to think about geometry in GUI's, but that isn't what Pugl does in reality. The reason I wanted a
puglGetScaleFactor
is because PuglEvents received in LVTK widgets are in Physical pixels and need scaledif != 1.0
. This could have changed in pugl, as I haven't tested LVTK with new code in pugl:main. Snippet of that code.. this is scaling to convert everything to Logical.But I think it's important to make it clear what we're dealing with. I always assumed the coordinates were supposed to be physical for some reason because of the above. Again, I haven't tried the main branch in LVTK so I don't know if it's relevant anymore. I'm honestly asking now, what are they supposed to be? physical or logical?
Especially for
puglSetSizeHint
. And I'm sure you are well aware: nobody wants calculate the scaling factor of monitors, backing drawing contexts, and what not just to set a default size. We just want to say "set the size to 640x360 and always manifest it 640x360 visually regardless of backend graphics setup and monitors used.So, with all that: the Cairo demo in
main
is setting a size hint defaulting @ 512x512. But when I run this on a 2x/4x monitor it displays as 256x256 in logical pixels. Running the same program on a 1x, normal, monitor it fills about half the vertical space which is what is to be expected. That's my thought process anyway.I'm on board for whatever, and maybe some of the details need documented, so it's clear the expectations about coordinate space. I'm not tied to
puglGetScaleFactor
either BTW, but if the public pugl API is operating in physical pixels, this information is essential to have early for the next layer in the GUI chain: the UI toolkit level.. maybe even before puglRealize... IDK.The text was updated successfully, but these errors were encountered: