Masking

[mike-spa]

Resolves: #26135
Resolves: #11570
Resolves: #22646

This PR implements a general infrastructure for masking any item drawn on the score. It should be noted that this is not achieved by drawing rectangles above/below the items (which is a hack that causes all sorts of problems, such as the ones described in #11570, and anyway is not a general solution), but rather by natively not drawing in the masked region.

Currently, this new system is applied to

• Barline VS Text. Any text-based item which crosses a barline will now cause the barline to be automatically masked. This forms part of the solution to Lyrics on first note of measure hanging into left barline #26135.
• Fret numbers VS String Lines in TABs. This resolves Guitar Tab fret numbers have a visible white box behind them #11570.

An option to control the first point has been added to the Barline style page.

Additionally as part of the solution to #26135, we've introduced a new option for Lyrics to avoid Barlines. This probably deserved a PR of its own cause it's not stricly related to masking, but the two things came up together so here they are. Obviously, this option is only relevant for the case of barlines across the staves.
The general option is in the Lyrics style page

and the local property override is in the Lyrics property page.

This is similar to what we've already done with Dynamics.

[cbjeukendrup]

Here's a quick review of especially the files from framework and below.

In general, I think this is the right solution. It's not very intuitive in my opinion; more ideal would be an item's own draw function would clear away the area behind that item. But that would be quite difficult to implement, and in the implementation I imagined, it would be impossible / even more complicated to erase only some elements from the background; which would be needed for example when we want to mask ties that cross time signatures, because then only the tie should be cut away, and not the staff lines. Also, it would depend too much on z order. So it seems your solution is the most viable one.

With any solution to this problem, there is the risk of performance impact, so we should do some testing for that.

Another thing to make sure is that it also works correctly with SVG export. Not sure if the SVG exporter supports clip paths or clipping at all yet.

[cbjeukendrup]

I'm afraid we're officially not allowed to use engraving:: in framework... @igorkorsukov what would be your advice here?

[mike-spa]

Yeah I was also unsure about this, cause I too didn't totally like the idea of passing a Shape down. I am going to refactor this and pass down a vector of rectangles instead (which basically is what a Shape is, anyway).

[igorkorsukov]

now the framework is already used in other products where there is no engraving (in Audacity), so it is impossible to use something from outside in the framework

To solve such problems, we can introduce a new type at the framework level and use it or convert to it.
Or in this case, Shape is a vector of RectF.

[mike-spa]

In general, I think this is the right solution. It's not very intuitive in my opinion; more ideal would be an item's own draw function would clear away the area behind that item. But that would be quite difficult to implement, and in the implementation I imagined, it would be impossible / even more complicated to erase only some elements from the background; which would be needed for example when we want to mask ties that cross time signatures, because then only the tie should be cut away, and not the staff lines. Also, it would depend too much on z order. So it seems your solution is the most viable one.

@cbjeukendrup yep, all correct. I should add that, because QPainterPath supports subtraction, my preferred solution would have been to clip the path that we're passing to the painter, rather than clip the painter itself. However, because we use our own implementation of PainterPath (incidentally, do you know the reason why we did this? why not just hide QPainterPath behind an interface and rely on that?) we don't have that functionality available. And, as you said, implementing general clipping ourselves would be extremely difficult. So the next best thing we can do is clip the painter.

With any solution to this problem, there is the risk of performance impact, so we should do some testing for that.

I am very confident that the drawing performance is unaffected by these changes. Both because only a tiny fraction of all the drawn objects will be actually clipped. But also because Qt "guarantees" that simple rectangular clipping is safe to use in performance-critical scenarios. For us, the expensive task is computing the masks, i.e. the MaskLayout functions. I'll do a quick check on that, but I'm sure we're fine.

Another thing to make sure is that it also works correctly with SVG export. Not sure if the SVG exporter supports clip paths or clipping at all yet.

That is a very good observation and in fact I've just tested it and it doesn't work with SVG. We'll just have to document that, I guess.

[cbjeukendrup]

However, because we use our own implementation of PainterPath (incidentally, do you know the reason why we did this? why not just hide QPainterPath behind an interface and rely on that?) we don't have that functionality available.

This was @igorkorsukov's choice; it's a logical choice, because for value types, like PainterPath, you can't really create interfaces. That's only done for types that are only passed by reference/pointers.

[bkunda]

Love it!

Only I can't seem to get the lyrics masking to work (even after a factory reset, and despite the new checkbox option in Style being enabled).

It does work for expression text, though:

Screen.Recording.2025-01-28.at.10.02.21.am.mov

Also a bug seems may have been introduced in this PR, which does not occur in the latest nightly: the note input cursor somehow becomes very opaque when the mouse cursor is moved:

mouse_cursor_issue.mov

Tested only on macOS so far, so I half wonder whether these issues may be OS-specific.

[igorkorsukov]

In general, I think this is the right solution. It's not very intuitive in my opinion; more ideal would be an item's own draw function would clear away the area behind that item. But that would be quite difficult to implement, and in the implementation I imagined, it would be impossible / even more complicated to erase only some elements from the background; which would be needed for example when we want to mask ties that cross time signatures, because then only the tie should be cut away, and not the staff lines. Also, it would depend too much on z order. So it seems your solution is the most viable one.

@mike-spa @cbjeukendrup
Just for understanding.
Why isn't it easier to solve this problem by simply defining the drawing order, first drawing the backgrounds elements, barlines, etc. below. And then drawing the overlying elements on top with their filled area with a color matching the background color. This would work universally for any elements, and the whole difficulty is in defining and drawing the elements in a given order, I think it's not difficult.

[igorkorsukov]

However, because we use our own implementation of PainterPath (incidentally, do you know the reason why we did this? why not just hide QPainterPath behind an interface and rely on that?) we don't have that functionality available.

This was @igorkorsukov's choice; it's a logical choice, because for value types, like PainterPath, you can't really create interfaces. That's only done for types that are only passed by reference/pointers.

Besides, if it’s difficult for us to make some function without Qt, then it will be difficult for others to do it too... And our neighboring project doesn’t have Qt.

[igorkorsukov]

And, as you said, implementing general clipping ourselves would be extremely difficult. So the next best thing we can do is clip the painter.

It would be good to find out whether our neighboring internal project can implement such clipping.

[mike-spa]

Only I can't seem to get the lyrics masking to work (even after a factory reset, and despite the new checkbox option in Style being enabled).

@bkunda that's because we've introduced a further option in the lyrics style page which by default is set to avoid barline collisions. If you deactivate that style option, or deactivate the individual property override for that single lyrics, it will let the collision happen and you'll see the masking.

Also a bug seems may have been introduced in this PR, which does not occur in the latest nightly: the note input cursor somehow becomes very opaque when the mouse cursor is moved:

Fixed now, thank you :)

[mike-spa]

Why isn't it easier to solve this problem by simply defining the drawing order, first drawing the backgrounds elements, barlines, etc. below. And then drawing the overlying elements on top with their filled area with a color matching the background color.

Because it isn't a very general solution @igorkorsukov. The main problems are:

1. The background is not necessarily a solid color.
   
2. The background can be transparent, in which this doesn't work anymore.
3. There can be situations where it's not possible to find a suitable drawing order. In this example, we want the key signature and time signature to mask the ties but not the barlines, but at the same time we need all of them to stay in front of the barlines. We can't achieve this with a rectangle.

This solution using the clipPath from QPainter is very simple for us so I think we should try to go with it. I am sure that masking is possible also in SVG, because if I convert the output PDF from Musescore into SVG (using an external tool), the barlines are masked correctly, so we probably just need to do a bit of research and find out how to do it.

Same for our neighbouring projects without Qt. Any vector-drawing tool should be capable of doing this I think.

For now, as this is a hyper-specialized engraving feature, I don't think it's a problem if the masking isn't exported to SVG or other non-Qt viewers.

[avvvvve]

Love to see it! Live masking while dragging was especially exciting to see in action:

Screen.Recording.2025-01-29.at.11.09.41.AM.mov

My main feedback here was already mentioned (SVG exported doesn't work), but if there's a workaround via PDF to SVG conversion, then I think we're okay.

Approving from my end, but this could definitely use some deeper QA testing.

---

If we're not planning on supporting proper SVG export for 4.5, here's a case we should be prepared to respond to with the PDF solution if we start hearing from users that we broke it:

The old SVG export solution (i.e. draw a white box behind the element) wasn't great either, but at least if you had the boxes there, you could use them as guides to manually clip the path in your vector editing program of choice.

Now, nothing gets exported beneath the masking elements, so if you wanted to cut bits of the line out, you won't have an easy, pre-placed guide to do so.

[bkunda]

Very happy with all this functionality, although I'd like a design review of the new options in the Style dialog ("lyrics" and "barlines") pages. It's not ideal that you have to interact with checkboxes in both places to get lyrics to pass through barlines with masking on.

But this is a minor detail. The affordances provided by this PR are truly excellent.

[mike-spa]

Update re SVG export @avvvvve @cbjeukendrup @igorkorsukov. As it turns out, the SVG standard does have an exact equivalent, which also happens to be called clipPath (see here for details), so it was quite easy to implement it (see the last commit). Now also our SVG export works perfectly with the new masking system :)

@cbjeukendrup it'd be great if you could give this a final round of code review so we can merge it soon.

[igorkorsukov]

It would be nice to delete

[Tantacrul]

To confirm, the masking options are off by default for older scores, on by default for new ones. (Although with Lyrics, you obviously need to switch off the 'avoid bar lines' first)?

Agree with @bkunda that it would probably be good to fix up those masking settings to try and find a way to make them a bit more intuitive to locate. Worth a checkbox in 'Properties' for relevant items? Not sure.

That aside, it works really nicely and looks great.

[Tantacrul]

When testing a little bit again, trying it out on a variety of bar line types. I've a few questions:

---

When I export as a PDF and open in Illustrator, each of the masks are simply white boxes overlaying the background. My understanding was that this wasn't the desired approach, so I'm wondering if it's an export issue or something else I'm missing, @mike-spa ?

What it looks like

Selecting the box

Deleting the box

---

Also, I'm not sure if this is desired but it doesn't work on the final barline (although it does work on when changing a regular barline to look like a final barline.)

---

Edit, I've changed the paper to a texture and it's obvious the solution is working properly in-app. With that in mind, the PDF export thing might not be the worst. It would obviously be better if it exactly matched the vectors in MuseScore 4 but it's really not so important. Curious to know exactly what the end-goal is for exported vectors.

---

Lastly, would we consider doing this for hairpins also?

---

Also, ditto @avvvvve's observation that cutaways don't work for SVG export, whether or not you choose a transparent BG.

[Tantacrul]

Curious to understand what the desired outcome is for exported SVGs and also PDF export.

[mike-spa]

When I export as a PDF and open in Illustrator, each of the masks are simply white boxes overlaying the background.

@Tantacrul I think that is mostly likely due to Illustrator itself and how it interprets the masks. I don't have Illustrator or other PDF editors, but if I convert Musescore's output PDF in SVG (using a random online tool), my SVG gets perfecly masked lines, not white rectangles, which tells me that the correct masking information is in the PDF. In any case, for PDF export we use 100% Qt so I wouldn't have any control over it anyway.

Also, ditto @avvvvve's observation that cutaways don't work for SVG export, whether or not you choose a transparent BG.

I think you're testing the older build, pls try out the latest one. I have now implemented masks in the SVG export (which, unlike PDF, we do ourselves so I can control it) and my output SVG looks like this. I'm using Inkscape to open all the SVGs btw.

Also, I'm not sure if this is desired but it doesn't work on the final barline

It's not about it being "final" but being at the end of the system, and that's intentional. I don't think the start- and end barline of a system should ever be masked (not least because it means that you're placing stuff in the page margins). Though @oktophonie should have last word on this really.

Lastly, would we consider doing this for hairpins also?

Well, I'd say certainly not now cause it would need yet another style/property option. As we develop this further (the really big one I'm looking forward to is masking ties...) it may be worth, as you and @bkunda mentioned, to think about something better, perhaps a style page specific to masking. But in general I think most of the time barlines aren't masked for other lines (though again @oktophonie territory).

[mike-spa]

although I'd like a design review of the new options in the Style dialog ("lyrics" and "barlines") pages. It's not ideal that you have to interact with checkboxes in both places to get lyrics to pass through barlines with masking on.

To confirm, the masking options are off by default for older scores, on by default for new ones. (Although with Lyrics, you obviously need to switch off the 'avoid bar lines' first)?
Agree with @bkunda that it would probably be good to fix up those masking settings to try and find a way to make them a bit more intuitive to locate. Worth a checkbox in 'Properties' for relevant items? Not sure.

@bkunda @Tantacrul

• Yes, masking is off by default on older scores (because people may have used custom rectangles and that kind of stuff and we don't want to mess it up unnecessarily). But it's on by default on new scores.
• The fact that lyrics collision-avoid barlines pertains to the layout of lyrics, so the relevant setting should be in the Lyrics style page. This is the exact same scheme we've used for Dynamics, where the general "Avoid barlines" option is in the Dynamics style page and there's a local override in the property panel.
• On the other hand, the masking of barlines is possible with all kinds of text, not just lyrics, so the option can't be anywhere else than the Barlines style page.

The fact that you need two separate options to get lyrics to cross and mask barlines is a non-issue because a) Normally barlines should not go across vocal staves (and in Musescore they won't, by default), so we're already dealing with an edge case. b) If you do use cross-stave barlines (which you shouldn't), then most of the time the best thing to do is collision-avoid them (hence why the setting is on by default). One would most likely disable collision-avoidance only for the individual lyrics that may be problematic. c) Masking will be on by default on the new files so you won't need to turn it on.

[Tantacrul]

@mike-spa

For PDF, the only thing that concerns me are cases like the one you mentioned to Igor above:

There can be situations where it's not possible to find a suitable drawing order. In this example, we want the key signature and time signature to mask the ties but not the barlines, but at the same time we need all of them to stay in front of the barlines. We can't achieve this with a rectangle.

---

If PDF's render white boxes for those situations, we might get into trouble. However, I can't know for sure until the thing is implemented. It's possible the PDF might be more 'clever' in how it distributes masks... but it's also possible we might get nasty white boxes blotting out barlines too.

[Tantacrul]

I think you're testing the older build, pls try out the latest one.

Excuse my ludidte-ness, but I'm trying out the build associated with this conversation in 'Checks' and SVG export is still showing the same problems as I mentioned above.

[avvvvve]

@mike-spa @Tantacrul In the build I downloaded today, I'm seeing that SVG export is masking things as expected on macOS!

---

I think there are some improvements to be made in terms of how masking is done, but not sure if they're easily solvable. See video:

Screen.Recording.2025-01-31.at.9.43.42.AM.mov

---

Also, if I export a PDF and open it directly in illustrator (as a PDF), it looks correct:

However, if I then export the open PDF file in Illustrator to SVG, then open the resulting SVG in any editor, the clipping is broken (this isn't really an issue though if our native SVG export works correctly):

[avvvvve]

Small request: In Styles, we move "Mask barlines when intersecting text" next to the other checkboxes, just beneath "Scale barlines to staff size"?