string.IsNullOrEmpty is slower than pattern matching

[paulomorgado]

Just came across this:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices;

namespace StringTests
{
    class Program
    {
        static void Main(string[] args)
        {
            BenchmarkRunner.Run<Benchmark>();
        }
    }

    [MemoryDiagnoser]
    public class Benchmark
    {
        [Params(null, "", "text")]
        public string? Text;

        [Benchmark]
        public bool StringIsNullOrEmpty() => string.IsNullOrEmpty(this.Text);

        [Benchmark]
        public bool PatternMatching() => this.Text is { Length: 0 };

        [Benchmark]
        public bool AggressiveInliningPatternMatching() => StringExtensions.AggressiveInliningPatternMatching(this.Text);

        [Benchmark]
        public bool NoInliningPatternMatching() => StringExtensions.NoInliningPatternMatching(this.Text);
    }

    public static class StringExtensions
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static bool AggressiveInliningPatternMatching(string? text) => text is { Length: 0 };

        [MethodImpl(MethodImplOptions.NoInlining)]
        public static bool NoInliningPatternMatching(string? text) => text is { Length: 0 };
    }
}

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.21277
Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.200-preview.20601.7
[Host] : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT
DefaultJob : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT

Method	Text	Mean	Error	StdDev	Median
StringIsNullOrEmpty	?	0.5121 ns	0.1307 ns	0.3749 ns	0.4328 ns
PatternMatching	?	0.3755 ns	0.0480 ns	0.1231 ns	0.3387 ns
AggressiveInliningPatternMatching	?	0.5585 ns	0.0906 ns	0.2571 ns	0.4520 ns
NoInliningPatternMatching	?	0.9844 ns	0.0629 ns	0.1722 ns	0.9303 ns
StringIsNullOrEmpty		0.8629 ns	0.2261 ns	0.6597 ns	0.5986 ns
PatternMatching		0.0486 ns	0.0514 ns	0.1450 ns	0.0000 ns
AggressiveInliningPatternMatching		0.0621 ns	0.0617 ns	0.1700 ns	0.0000 ns
NoInliningPatternMatching		2.1394 ns	0.3309 ns	0.9387 ns	2.0965 ns
StringIsNullOrEmpty	text	0.6652 ns	0.0573 ns	0.1469 ns	0.6187 ns
PatternMatching	text	0.2546 ns	0.0996 ns	0.2889 ns	0.1639 ns
AggressiveInliningPatternMatching	text	0.0277 ns	0.0349 ns	0.1006 ns	0.0000 ns
NoInliningPatternMatching	text	3.6437 ns	0.4525 ns	1.3342 ns	3.3229 ns

Being on the ns scale and not on a dedicated system, these values might be meaningless.

Nevertheless, surprising.

[danmoseley]

sharplab

StringTests.Benchmark.StringIsNullOrEmpty()
    L0000: push ebp
    L0001: mov ebp, esp
    L0003: mov eax, [ecx+4]
    L0006: test eax, eax
    L0008: je short L0014
    L000a: cmp dword ptr [eax+4], 0
    L000e: jbe short L0014
    L0010: xor eax, eax
    L0012: jmp short L0019
    L0014: mov eax, 1
    L0019: pop ebp
    L001a: ret

StringTests.Benchmark.PatternMatching()
    L0000: mov eax, [ecx+4]
    L0003: test eax, eax
    L0005: je short L0012
    L0007: cmp dword ptr [eax+4], 0
    L000b: sete al
    L000e: movzx eax, al
    L0011: ret
    L0012: xor eax, eax
    L0014: ret

[danmoseley]

Oh, I didn't notice it did that (why?). But thanks, I guess there is no action to take here.

[paulomorgado]

@gfoidl, your example just proves that the implementation of string.IsNullOrEmpty is similar to pattern matching.

What I was comparing was using string.IsNullOrEmpty with using pattern matching.

this is the benchmark code with the amd64 JIT:

StringTests.Benchmark.StringIsNullOrEmpty()
    L0000: mov rax, [rcx+8]
    L0004: test rax, rax
    L0007: je short L0013
    L0009: cmp dword ptr [rax+8], 0
    L000d: jbe short L0013
    L000f: xor eax, eax
    L0011: jmp short L0018
    L0013: mov eax, 1
    L0018: ret

StringTests.Benchmark.PatternMatching()
    L0000: mov rax, [rcx+8]
    L0004: test rax, rax
    L0007: je short L0014
    L0009: cmp dword ptr [rax+8], 0
    L000d: sete al
    L0010: movzx eax, al
    L0013: ret
    L0014: xor eax, eax
    L0016: ret

string.IsNullOrEmpty is inlined, as I expected. I just didn't expect it to perform so bad in comparison.

But we are talking picoseconds here, anyway.

[GrabYourPitchforks]

Another note - if we're talking about benchmarking real-world code, it's sometimes deceiving when measuring a benchmark that returns bool. Instead, the benchmark should ideally exercise code paths that consume the bool using the pattern if (MyMethod()) { /* do something */ }. Looking at the raw codegen of string.IsNullOrEmpty in isolation shows that it's 2 bytes larger than the pattern matching sample, but check through the discussion on #4207 and you'll see that the "2 bytes larger" implementation actually results in better codegen in the caller when it's properly inlined.

[paulomorgado]

Another note - if we're talking about benchmarking real-world code, it's sometimes deceiving when measuring a benchmark that returns bool. Instead, the benchmark should ideally exercise code paths that consume the bool using the pattern if (MyMethod()) { /* do something */ }. Looking at the raw codegen of string.IsNullOrEmpty in isolation shows that it's 2 bytes larger than the pattern matching sample, but check through the discussion on #4207 and you'll see that the "2 bytes larger" implementation actually results in better codegen in the caller when it's properly inlined.

There's a comment in the source code of string.IsNullOrEmpty explaining that.

[stephentoub]

What I was comparing was using string.IsNullOrEmpty with using pattern matching.

this.Text is { Length: 0 } is equivalent to a (mythical) IsNotNullAndEmpty, not IsNullOrEmpty.

[danmoseley]

Answered by @gfoidl - but I can't mark a comment as answer.