Only some enum values are interpreted as bytes, and not string, in custom JsonConverter.

[Danielovich]

Description

Code is based on the documentation here:

https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-converters-how-to?pivots=dotnet-6-0#deserialize-inferred-types-to-object-properties

When trying to deserialize some JSON data with a with a certain enum value, a custom JsonConverter looks at the the enum value as a bytes array and not a string.

Changing the enum value to a different value makes the JsonConverter interpret it as a string.

The converter is set to return an object and will be used with other types than the reproduction steps puts forward. So it is meant to be given a range of different types other than QueryCondition2.

Reproduction Steps

namespace Reproduction
{
    using System;
    using System.ComponentModel;
    using System.Text.Json;
    using System.Text.Json.Serialization;
    using Xunit;

    public class Class1
    {
        [Fact]
        public void BlockLevelEnumQueryCondition()
        {
            var cond = new QueryCondition2("BlockLevel", BlockLevel.John);
            //var cond = new QueryCondition2("BlockLevel", BlockLevel.Johns);

            // Act
            var jsonDataSerialize = JsonSerializer.Serialize(cond, SomeClass.SerializerOption);
            var jsonDataDeserialize = JsonSerializer.Deserialize<QueryCondition2>(jsonDataSerialize, SomeClass.DeserializerOption);

            // Assert
            var queryConditions = jsonDataDeserialize;
            
            //Assert.Equal("Johns", queryConditions.Value); //Pass
            Assert.Equal("John", queryConditions.Value); //Fail
        }
    }

    public enum BlockLevel
    {
        John, //fails
        //Johns //Succeeds
    }

    public class AnObjectToInferredTypeConverter : JsonConverter<object>
    {
        public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) => reader.TokenType switch
        {
            JsonTokenType.String when reader.TryGetBytesFromBase64(out byte[] bytes) => bytes,
            JsonTokenType.String => reader.GetString()!,
            _ => JsonDocument.ParseValue(ref reader).RootElement.Clone()
        };

        public override void Write(
          Utf8JsonWriter writer,
          object objectToWrite,
          JsonSerializerOptions options) =>
          JsonSerializer.Serialize(writer, objectToWrite, objectToWrite.GetType());
    }

    public class SomeClass
    {

        [JsonIgnore]
        public static JsonSerializerOptions SerializerOption
        {
            get
            {
                var _serializerOptions = new JsonSerializerOptions
                {
                    DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
                    WriteIndented = true,
                    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
                };
                _serializerOptions.Converters.Add(new JsonStringEnumConverter());
                return _serializerOptions;
            }
        }

        [JsonIgnore]
        public static JsonSerializerOptions DeserializerOption
        {
            get
            {
                var _options = new JsonSerializerOptions
                {
                    DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
                    WriteIndented = true,
                    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
                };
                _options.Converters.Add(new JsonStringEnumConverter());
                _options.Converters.Add(new AnObjectToInferredTypeConverter());
                return _options;
            }
        }
    }


    public class QueryCondition2
    {
        [JsonConstructor]
        public QueryCondition2(string item, QueryOperator2 queryOperator, object value)
        {
            Item = item;
            QueryOperator = queryOperator;
            Value = value is long ? Convert.ToInt32(value) : value;
        }

        public QueryCondition2(string item, object value)
            : this(item, QueryOperator2.Equal, value)
        {
        }

        public string Item { get; }

        public QueryOperator2 QueryOperator { get; }
        public object Value { get; }
    }

    public enum QueryOperator2
    {
        [Description("=")]
        Equal
    }
}

Expected behavior

I expect the custom JsonConverter to act the same on the enum value, each time,

Actual behavior

The reproduction test BlockLevelEnumQueryCondition returns this on enum value "John"

Message: 
Assert.Equal() Failure
↓ (pos 0)
Expected: "John"
Actual: [38, 136, 103]
↑ (pos 0)

Regression?

No response

Known Workarounds

No response

Configuration

.NET 6
Windows 11 x64

Other information

No response

[eiriktsarpalis]

The issue seems to be with this code in your custom converter:

            JsonTokenType.String when reader.TryGetBytesFromBase64(out byte[] bytes) => bytes,
            JsonTokenType.String => reader.GetString()!,

Because John is a valid Base64 string, the object will deserialize as the decoded byte array. Because Johns isn't valid Base64 it will deserialize as the actual string.

[Danielovich]

There is no thing to my knowledge as an invalid base64 string. Why would John be any different than Johns?

I believe this has to do with the inner workings of how a base64 string is interpreted. Any 4 % 0 value will fail the test.

Try to put any 8 character value in there. All will fail too.

[bartonjs]

True base64 (vs base64url) always requires a multiple of 4 characters; so "John" is valid, "Johns" is not. Both base64s only have 64 valid characters, and neither of them considers ¡ a valid character, so "¡¡¡¡" is another example of an invalid base64 string.

In the grand universe of all possible strings, the "invalid base64" strings vastly outnumber the "valid base64" strings.

[Danielovich]

That is new to me. Thank you both for clarifying