Exclusively applicable fix-its aren't testable

[gohanlon]

Description

I have a diagnostic with several fix-its, where only one can be applied. Here's the usage, and note that, while diagnostics is correct and (very!) useful, both fixes and expansion are nonsensical:

assertMacro {
  """
  @MemberwiseInit
  struct S {
    @Init(default: "Blob") var name = "Foo"
  }
  """
} diagnostics: {
  """
  @MemberwiseInit
  struct S {
    @Init(default: "Blob") var name = "Foo"
          ┬──────────────
          ╰─ 🛑 Custom 'default' can't be applied to already initialized variable
             ✏️ Remove 'default: "Blob"'
             ✏️ Remove '@Init(default: "Blob")'
             ✏️ Remove '= "Foo"'
  }
  """
} fixes: {
  """
  @MemberwiseInit
  struct S {
    @Init
  }
  """
} expansion: {
  """
  struct S {
    @Init

    internal init() {
    }
  }
  """
}

To show the actual fix-it behavior applied in Xcode, here's a theoretical array syntax format:

…
} fixes: {
  [
    """
    @MemberwiseInit
    struct S {
      @Init var name = "Foo"
    }
    """,
    """
    @MemberwiseInit
    struct S {
      var name = "Foo"
    }
    """,
    """
    @MemberwiseInit
    struct S {
      @Init(default: "Blob") var name: String
    }
    """,
  ]
} expansion: {
…

And, a theoretical dictionary syntax format:

} fixes: {
  [
    #"✏️ Remove 'default: "Blob"'"#:
    """
    @MemberwiseInit
    struct S {
      @Init var name = "Foo"
    }
    """,
    #"✏️ Remove '@Init(default: "Blob")'"#:
    """
    @MemberwiseInit
    struct S {
      var name = "Foo"
    }
    """,
    #"✏️ Remove '= "Foo"'"#:
    """
    @MemberwiseInit
    struct S {
      @Init(default: "Blob") var name: String
    }
    """,
  ]
} expansion: {

What do you think?

P.S.: swift-macro-testing is great—I'm heavily using/abusing it. Thank you!

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

Each fix-it should be applied to the original source independently, each producing a corresponding fixed-source string.

Actual behavior

It's assumed that all fix-its can always be applied on top of each other, accumulating into a single string.

Steps to reproduce

No response

swift-macro-testing version information

0.2.2

Destination operating system

macOS 14.1.2

Xcode version information

Xcode Version 15.0.1 (15A507)

Swift Compiler version information

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx14.0

[gohanlon]

Edited: The proposed "fixes" format below is now a simple string.

Whelp, my initial proposal is actually still too simplistic. To fully model the diagnostics domain would require another level of nesting:

assertMacro {
  """
  @MemberwiseInit(.public)
  public struct Person {
    var firstName = "Foo"
    var lastName: String
  }
  """
} diagnostics: {
  """
    @MemberwiseInit(.public)
    public struct Person {
      var firstName = "Foo"
      ┬────────────────────
      ╰─ 🛑 @MemberwiseInit(.public) would leak access to 'internal' property
         ✏️ Add '@Init(.public)'
         ✏️ Add 'public' access level
         ✏️ Add '@Init(.ignore)'
      var lastName = "Bar"
      ┬───────────────────
      ╰─ 🛑 @MemberwiseInit(.public) would leak access to 'internal' property
         ✏️ Add '@Init(.public)'
         ✏️ Add 'public' access level
         ✏️ Add '@Init(.ignore)' and an initializer
    }
  """
} fixes: {
  #"""
    var firstName = "Foo"
    ┬────────────────────
    ╰─ 🛑 @MemberwiseInit(.public) would leak access to 'internal' property
      ✏️ Add '@Init(.public)'
        @MemberwiseInit(.public)
        public struct Person {
          @Init(.public) var firstName = "Foo"
          var lastName: String
        }

      ✏️ Add 'public' access level
        @MemberwiseInit(.public)
        public struct Person {
          public var firstName = "Foo"
          var lastName: String
        }

      ✏️ Add '@Init(.ignore)'
        @MemberwiseInit(.public)
        public struct Person {
          @Init(.ignore) var firstName = "Foo"
          var lastName: String
        }

    var lastName = "Bar"
    ┬───────────────────
    ╰─ 🛑 @MemberwiseInit(.public) would leak access to 'internal' property
      ✏️ Add '@Init(.public)'
        @MemberwiseInit(.public)
        public struct Person {
          var firstName = "Foo"
          @Init(.public) var lastName: String
        }

      ✏️ Add 'public' access level
        @MemberwiseInit(.public)
        public struct Person {
          var firstName = "Foo"
          public var lastName: String
        }

      ✏️ Add '@Init(.ignore)'
        @MemberwiseInit(.public)
        public struct Person {
          var firstName = "Foo"
          @Init(.ignore) var lastName: String = <#value#>
        }
  """#
} expansion: {
  """
  public struct Person {
    var firstName = "Foo"
    var lastName: String

    public init() {}
  }
  """
}

Notes:

• The expansion closure string is the original expansion, not after having applied fix-its. That is, the expansion only reflects whether or not the MemberMacro returned any declarations, independent of whether it emitted diagnostics.
• This "fixes" format would be pretty intense for non-trivial cases. For cases where theres only a single fix-it among the diagnostics, the current simple "fixes" format could still be used.
• To help manage verbosity, the fixes could be represented as the diff between the original source and the fixed source, but the developer would then have to interpret a "diff of a diff" to fix failures.
• Perhaps the "fixes" could be disabled via assertMacro(includeFixes:)/withMacroTesting(includeFixes:) argument to gain some succinctness at the cost of some exhaustivity? Then the "fixes" can be skipped if they aren't interesting enough to warrant the space. For example, I have a large, exhaustive test case that's generated, and it will soon have three fix-its for every diagnostic. Those fix-its will be repetitive, and I have another set of tests that exercise the fix-its in particular (a simple test and many weird tests involving surrounding trivia).

[mbrandonw]

Hey @gohanlon, just wanted to thank you for looking into this issue and to let you know we have seen it! We've just been quite busy with some other things, but we hope to get back to it soon. Most likely in the near year.

[gohanlon]

No worries! I’ve implemented the proposal and will have a PR ready soon to advance our discussion when the time is right.

While wrapping up the PR, I found and fixed a bug exposed by the new format, where fix-its were misapplied. The issue was with (Swift’s) FixItApplier, replacing a parent instead of a child (like a code block with a variable declaration). To fix, I updated the FixItApplier with a recent version from Swift. Since that came up late in the PR work, I'll just include it in the PR for now.

[stephencelis]

@gohanlon Appreciate it! If it's not too much trouble, though, I think we'd take that FixItApplier fix as a separate PR, as it sounds like a no-brainer. That'll make it easier to review your changes.

[gohanlon]

Indeed, the FixItApplier fix can definitely land sooner than a big PR. I started on that work, just need a test or two and I'll create the PR!