Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/series/2.x' into feat-scala3-enu…
Browse files Browse the repository at this point in the history
…meration-support
  • Loading branch information
ThijsBroersen committed Apr 29, 2024
2 parents eeb0df0 + bc86948 commit b938414
Show file tree
Hide file tree
Showing 15 changed files with 457 additions and 34 deletions.
26 changes: 26 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,32 @@ banana.toJson
apple.toJson
```

Another way of changing type hint is using `@jsonHintNames` annotation on sealed class. It allows to apply transformation
to all type hint values in hierarchy. Same transformations are provided as for `@jsonMemberNames` annotation.

Here's an example:

```scala mdoc
import zio.json._

@jsonHintNames(SnakeCase)
sealed trait FruitKind

case class GoodFruit(good: Boolean) extends FruitKind

case class BadFruit(bad: Boolean) extends FruitKind

object FruitKind {
implicit val encoder: JsonEncoder[FruitKind] =
DeriveJsonEncoder.gen[FruitKind]
}

val goodFruit: FruitKind = GoodFruit(true)
val badFruit: FruitKind = BadFruit(true)

goodFruit.toJson
badFruit.toJson
```
## jsonDiscriminator


Expand Down
186 changes: 186 additions & 0 deletions docs/decoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,189 @@ implicit val decodeName: JsonDecoder[String Refined NonEmpty] =
```

Now the code compiles.

# Parsing custom JSON

In this section we show several approaches for decoding JSON that looks like:

```json
{
"01. symbol": "IBM",
"02. open": "182.4300",
"03. high": "182.8000"
}
```

Which we want to decode into the following case class:

```scala mdoc
final case class Quote(
symbol: String,
open: String,
high: String
)
```

All approaches have the same result:

```scala mdoc:fail
"""{"01. symbol":"IBM","02. open": "182.4300","03. high": "182.8000"}""".fromJson[Quote]
// >> Right(Quote(IBM,182.4300,182.8000))
```

## Approach 1: use annotation hints

In this approach we enrich the case class with annotations to tell the derived decoder which field names to use.
Obviously, this approach only works if we can/want to change the case class.

```scala mdoc:reset
import zio.json._

final case class Quote(
@jsonField("01. symbol") symbol: String,
@jsonField("02. open") open: String,
@jsonField("03. high") high: String
)

object Quote {
implicit val decoder: JsonDecoder[Quote] = DeriveJsonDecoder.gen[Quote]
}
```

## Approach 2: use an intermediate case class

Instead of hints, we can also put the actual field names in an intermediate case class. In our example the field names
are not valid scala identifiers. We fix this by putting the names in backticks:

```scala mdoc:reset
import zio.json._

final case class Quote(symbol: String, open: String, high: String)

object Quote {
private final case class JsonQuote(
`01. symbol`: String,
`02. open`: String,
`03. high`: String
)

implicit val decoder: JsonDecoder[Quote] =
DeriveJsonDecoder
.gen[JsonQuote]
.map { case JsonQuote(s, o, h) => Quote(s, o, h) }
}
```

## Approach 3: decode to JSON

In this approach we first decode to the generic `Json` data structure. This approach is very flexible because it can
extract data from any valid JSON.

Note that this implementation is a bit sloppy. It uses `toString` on a JSON node. The node is not necessarily a
String, it can be of any JSON type! So this might happily process JSON that doesn't match your expectations.

```scala mdoc:reset
import zio.json._
import zio.json.ast.Json

final case class Quote(symbol: String, open: String, high: String)

object Quote {
implicit val decoder: JsonDecoder[Quote] = JsonDecoder[Json]
.mapOrFail {
case Json.Obj(fields) =>
def findField(name: String): Either[String, String] =
fields
.find(_._1 == name)
.map(_._2.toString()) // ⚠️ .toString on any JSON type
.toRight(left = s"Field '$name' is missing")

for {
symbol <- findField("01. symbol")
open <- findField("02. open")
high <- findField("03. high")
} yield Quote(symbol, open, high)
case _ =>
Left("Not a JSON record")
}
}
```

## Approach 4: decode to JSON, use cursors

Here we also first decode to `Json`, but now we use cursors to find the data we need. Here we do check that the fields
are actually strings.

```scala mdoc:reset
import zio.json._
import zio.json.ast.{Json, JsonCursor}

final case class Quote(symbol: String, open: String, high: String)

object Quote {
private val symbolC = JsonCursor.field("01. symbol") >>> JsonCursor.isString
private val openC = JsonCursor.field("02. open") >>> JsonCursor.isString
private val highC = JsonCursor.field("03. high") >>> JsonCursor.isString

implicit val decoder: JsonDecoder[Quote] = JsonDecoder[Json]
.mapOrFail { c =>
for {
symbol <- c.get(symbolC)
open <- c.get(openC)
high <- c.get(highC)
} yield Quote(symbol.value, open.value, high.value)
}
}
```

# More custom decoder examples

Let's consider an `Animal` case class with a `categories` field that should be a list of strings. However, some
producers accidentally represent the categories as a comma-separated string instead of a proper list. We want to parse
both cases.

Here's a custom decode for our Animal case class:

```scala mdoc:reset
import zio.Chunk
import zio.json._
import zio.json.ast._

case class Animal(name: String, categories: List[String])

object Animal {
private val nameC = JsonCursor.field("name") >>> JsonCursor.isString
private val categoryArrayC = JsonCursor.field("categories") >>> JsonCursor.isArray
private val categoryStringC = JsonCursor.field("categories") >>> JsonCursor.isString

implicit val decoder: JsonDecoder[Animal] = JsonDecoder[Json]
.mapOrFail { c =>
for {
name <- c.get(nameC).map(_.value)
categories <- arrayCategory(c).map(_.toList)
.orElse(c.get(categoryStringC).map(_.value.split(',').map(_.trim).toList))
} yield Animal(name, categories)
}

private def arrayCategory(c: Json): Either[String, Chunk[String]] =
c.get(categoryArrayC)
.flatMap { arr =>
// Get the string elements, and sequence the obtained eithers to a single either
sequence(arr.elements.map(_.get(JsonCursor.isString).map(_.value)))
}

private def sequence[A, B](chunk: Chunk[Either[A, B]]): Either[A, Chunk[B]] =
chunk.partition(_.isLeft) match {
case (Nil, rights) => Right(rights.collect { case Right(r) => r })
case (lefts, _) => Left(lefts.collect { case Left(l) => l }.head)
}
}
```

And now, the Json decoder for Animal can handle both formats:
```scala mdoc
"""{"name": "Dog", "categories": "Warm-blooded, Mammal"}""".fromJson[Animal]
// >> Right(Animal(Dog,List(Warm-blooded, Mammal)))
"""{"name": "Snake", "categories": [ "Cold-blooded", "Reptile"]}""".fromJson[Animal]
// >> Right(Animal(Snake,List(Cold-blooded, Reptile)))
```
30 changes: 15 additions & 15 deletions project/NeoJmhPlugin.scala
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ object NeoJmhPlugin extends AutoPlugin {
override def projectSettings =
inConfig(Jmh)(
Defaults.testSettings ++ Seq(
run := (run in JmhInternal).evaluated,
run := (JmhInternal / run).evaluated,
neoJmhGenerator := "reflection",
neoJmhYourkit := Nil,
javaOptions ++= Seq(
Expand All @@ -71,11 +71,11 @@ object NeoJmhPlugin extends AutoPlugin {
)
) ++ inConfig(JmhInternal)(
Defaults.testSettings ++ Seq(
javaOptions := (javaOptions in Jmh).value,
envVars := (envVars in Jmh).value,
mainClass in run := Some("org.openjdk.jmh.Main"),
fork in run := true,
dependencyClasspath ++= (fullClasspath in Jmh).value,
javaOptions := (Jmh / javaOptions).value,
envVars := (Jmh / envVars).value,
run / mainClass := Some("org.openjdk.jmh.Main"),
run / fork := true,
dependencyClasspath ++= (Jmh / fullClasspath).value,
sourceGenerators += generateJmhSourcesAndResources.map { case (sources, _) =>
sources
},
Expand Down Expand Up @@ -106,23 +106,23 @@ object NeoJmhPlugin extends AutoPlugin {

def backCompatProjectSettings: Seq[Setting[_]] = Seq(
// WORKAROUND https://github.com/sbt/sbt/issues/3935
dependencyClasspathAsJars in NeoJmhPlugin.JmhInternal ++= (fullClasspathAsJars in NeoJmhKeys.Jmh).value
NeoJmhPlugin.JmhInternal / dependencyClasspathAsJars ++= (NeoJmhKeys.Jmh / fullClasspathAsJars).value
)

def generateBenchmarkSourcesAndResources: Def.Initialize[Task[(Seq[File], Seq[File])]] = Def.task {
val s = streams.value
val cacheDir = crossTarget.value / "jmh-cache"
val bytecodeDir = (classDirectory in Jmh).value
val bytecodeDir = (Jmh / classDirectory).value
val sourceDir = sourceManaged.value
val resourceDir = resourceManaged.value
val generator = (neoJmhGenerator in Jmh).value
val generator = (Jmh / neoJmhGenerator).value
val classpath = dependencyClasspath.value
val javaHomeV = (javaHome in Jmh).value
val outputStrategyV = (outputStrategy in Jmh).value
val workingDirectory = Option((baseDirectory in Jmh).value)
val connectInputV = (connectInput in Jmh).value
val envVarsV = (envVars in Jmh).value
val javaFlags = (javaOptions in Jmh).value.toVector
val javaHomeV = (Jmh / javaHome).value
val outputStrategyV = (Jmh / outputStrategy).value
val workingDirectory = Option((Jmh / baseDirectory).value)
val connectInputV = (Jmh / connectInput).value
val envVarsV = (Jmh / envVars).value
val javaFlags = (Jmh / javaOptions).value.toVector

val inputs: Set[File] = (bytecodeDir ** "*").filter(_.isFile).get.toSet
val cachedGeneration = FileFunction.cached(cacheDir, FilesInfo.hash) { _ =>
Expand Down
37 changes: 37 additions & 0 deletions zio-json-macros/shared/src/test/scala/zio/json/DeriveSpec.scala
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,28 @@ object DeriveSpec extends ZIOSpecDefault {
assert("""{"Child2":{}}""".fromJson[Parent])(isRight(equalTo(Child2()))) &&
assert("""{"type":"Child1"}""".fromJson[Parent])(isLeft(equalTo("(invalid disambiguator)")))
},
test("sum encoding with hint names") {
import examplesumhintnames._

assert("""{"child1":{}}""".fromJson[Parent])(isRight(equalTo(Child1()))) &&
assert("""{"child2":{}}""".fromJson[Parent])(isRight(equalTo(Child2()))) &&
assert("""{"type":"child1"}""".fromJson[Parent])(isLeft(equalTo("(invalid disambiguator)")))
},
test("sum alternative encoding") {
import examplealtsum._

assert("""{"hint":"Cain"}""".fromJson[Parent])(isRight(equalTo(Child1()))) &&
assert("""{"hint":"Abel"}""".fromJson[Parent])(isRight(equalTo(Child2()))) &&
assert("""{"hint":"Samson"}""".fromJson[Parent])(isLeft(equalTo("(invalid disambiguator)"))) &&
assert("""{"Cain":{}}""".fromJson[Parent])(isLeft(equalTo("(missing hint 'hint')")))
},
test("sum alternative encoding with hint names") {
import examplealtsumhintnames._

assert("""{"hint":"child1"}""".fromJson[Parent])(isRight(equalTo(Child1()))) &&
assert("""{"hint":"Abel"}""".fromJson[Parent])(isRight(equalTo(Child2()))) &&
assert("""{"hint":"Child1"}""".fromJson[Parent])(isLeft(equalTo("(invalid disambiguator)"))) &&
assert("""{"child1":{}}""".fromJson[Parent])(isLeft(equalTo("(missing hint 'hint')")))
}
)
)
Expand All @@ -59,6 +74,15 @@ object DeriveSpec extends ZIOSpecDefault {
case class Child2() extends Parent
}

object examplesumhintnames {
@jsonDerive
@jsonHintNames(SnakeCase)
sealed abstract class Parent

case class Child1() extends Parent
case class Child2() extends Parent
}

object exampleempty {
@jsonDerive
case class Empty(a: Option[String])
Expand All @@ -78,6 +102,19 @@ object DeriveSpec extends ZIOSpecDefault {
case class Child2() extends Parent
}

object examplealtsumhintnames {

@jsonDerive
@jsonDiscriminator("hint")
@jsonHintNames(SnakeCase)
sealed abstract class Parent

case class Child1() extends Parent

@jsonHint("Abel")
case class Child2() extends Parent
}

object logEvent {
@jsonDerive(JsonDeriveConfig.Decoder)
case class Event(at: Long, message: String, a: Seq[String] = Nil)
Expand Down
2 changes: 1 addition & 1 deletion zio-json/jvm/src/jmh/scala/zio/json/UUIDBenchmarks.scala
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class UUIDBenchmarks {
} yield s"$s1-$s2-$s3-$s4-$s5"

unparsedUUIDChunk = {
Unsafe.unsafeCompat { implicit u =>
Unsafe.unsafe { implicit u =>
zio.Runtime.default.unsafe.run(gen.runCollectN(10000).map(Chunk.fromIterable)).getOrThrow()
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ trait JsonDecoderPlatformSpecific[A] { self: JsonDecoder[A] =>
final def decodeJsonPipeline(
delimiter: JsonStreamDelimiter = JsonStreamDelimiter.Array
): ZPipeline[Any, Throwable, Char, A] = {
Unsafe.unsafeCompat { (u: Unsafe) =>
Unsafe.unsafe { (u: Unsafe) =>
implicit val unsafe: Unsafe = u

ZPipeline.fromPush {
Expand Down Expand Up @@ -121,7 +121,7 @@ trait JsonDecoderPlatformSpecific[A] { self: JsonDecoder[A] =>
throw new Exception(JsonError.render(trace))
}

Unsafe.unsafeCompat { (u: Unsafe) =>
Unsafe.unsafe { (u: Unsafe) =>
implicit val unsafe: Unsafe = u

runtime.unsafe.run(outQueue.offer(Take.single(nextElem))).getOrThrow()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ trait JsonEncoderPlatformSpecific[A] { self: JsonEncoder[A] =>
delimiter: Option[Char],
endWith: Option[Char]
): ZPipeline[Any, Throwable, A, Char] =
Unsafe.unsafeCompat { (u: Unsafe) =>
Unsafe.unsafe { (u: Unsafe) =>
implicit val unsafe: Unsafe = u

ZPipeline.fromPush {
Expand Down
Loading

0 comments on commit b938414

Please sign in to comment.