Text interactions break on transformed layers

[CrazyCraftix]

Description

When working on a transformed layer, selecting a region of text by dragging on egui::TextEdit or egui::Label doesn't work properly.

Clicking on a transformed egui::TextEdit doesn't focus the widget. Holding the button for a bit before releasing works for some reason. Interestingly, this also happens when clicking the edge of the widget, even without transformation.

2024-03-15.21-49-13.mp4

Steps to reproduce

• use emilk/eframe_template@d168539
• update egui and eframe to bf7ffb9
• remove all ui code and add the following inside egui::CentralPanel:

use egui::{emath::TSTransform, Label, LayerId, Pos2, Rect, TextEdit, Vec2};

// let transform = TSTransform::IDENTITY;
// let transform = TSTransform::from_translation(egui::Vec2::new(50., 0.));
let transform = TSTransform::from_translation(egui::Vec2::new(150., 0.));
                                                                                   
let layer_id = LayerId::new(egui::Order::Middle, ui.id().with("child"));
ui.ctx().set_transform_layer(layer_id, transform);
                                                                                   
ui.with_layer_id(layer_id, |ui| {
    let mut rect = Rect::from_min_size(Pos2::new(20., 30.), Vec2::new(100., 20.));
    ui.put(rect, Label::new("this is a test label"));
                                                                                   
    rect.min.y += 20.;
    rect.max.y += 20.;
    static mut STR: String = String::new();
    ui.put(rect, TextEdit::singleline(unsafe { &mut STR }));
});

Thoughts

It seems like when handling these interactions, the transformation isn't applied properly.

For the focus issue, I think I have identified the relevant code snippet:

egui/crates/egui/src/response.rs

Lines 181 to 191 in bf7ffb9

	// We detect clicks/hover on a "interact_rect" that is slightly larger than
	// self.rect. See Context::interact.
	// This means we can be hovered and clicked even though `!self.rect.contains(pos)` is true,
	// hence the extra complexity here.
	if self.contains_pointer() {
	false
	} else if let Some(pos) = pointer.interact_pos() {
	!self.rect.contains(pos)
	} else {
	false // clicked without a pointer, weird
	}

In line 188, the untransformed rect is checked against the untransformed pointer position.
When the widget is transformed away, the click happens outside the original area and the focus is removed.
I tried accessing the transform from this function using self.ctx.memory(), however this freezes the app, probably because the memory is already being accessed at this point?

I don't know the code well enough yet to be able to fix this.

[Dampfwalze]

I also encountered this and I posted a comment on the PR that introduced the set_transform_layer method: #3906 (comment)

[QuantumEntangledAndy]

Rather then opening a new issue I will update here:

The relevant line of code causing this I believe is here

egui/crates/egui/src/widgets/text_edit/builder.rs

Lines 546 to 548 in 93d458b

	if let Some(pointer_pos) = ui.ctx().pointer_interact_pos() {
	if response.hovered() && text.is_mutable() {
	ui.output_mut(|o| o.mutable_text_under_cursor = true);

Changing it to something like this

if let Some(pointer_pos) = ui.ctx().pointer_interact_pos() {
    let t = ui.ctx().memory(|m| {
        m.layer_transforms
            .get(&ui.layer_id())
            .map(TSTransform::inverse)
            .unwrap_or_default()
    });
    let pointer_pos = t * pointer_pos;
    if response.hovered() && text.is_mutable() { 
         ui.output_mut(|o| o.mutable_text_under_cursor = true); 

should work

I'm a bit afk atm but I will see about opening a PR when I have time