Design: expression type safety

[sargunv]

This conversation is ported over from an ephemeral Slack channel in order to preserve it. Original URL (will not live forever): https://osmus.slack.com/archives/C06U5MM097B/p1731958750167159

@westnordost:

while having a closer look, I noticed that many properties are strings that could be enums or sealed interfaces. E.g.

cap: Expression<String> = const("butt"),
join: Expression<String> = const("miter"),
miterLimit: Expression<Number> = const(2),
roundLimit: Expression<Number> = const(1.05),
could be
cap: Expression<String> = const(StrokeCap.Butt),
join: Expression<String> = const(StrokeJoin.Miter()),

...

sealed interface StrokeJoin {
  data object Bevel : StrokeJoin
  data class Round(val limit: Expression<Number> = const(1.05)) : StrokeJoin
  data class Miter(val limit: Expression<Number> = const(2.0)) : StrokeJoin
}

enum class StrokeCap { Butt, Round, Square }

Turns out, much of what I have documented in that mentioned API example is actually for these enums.
The below would be more Kotlin idiomatic than to use strings / string constants.

Although, I have no idea how to arrange that with Expressions. At least for the nested one (StrokeJoin). (edited)

@sargunv

Yeah the main hurdle is getting that to play nice with expression typing. If Expression is refactored to an interface, then enums could implement that like : Expression providing a string value. But using these expressions with other expression functions that take Expression as input or output would get awkward.

Alternatively we could have these enums just implement Expression, then the user could pass a const if they want but the enums would be available

Rn Expression is a class with internal constructor so having enums extend it isn't possible, but shouldn't be hard to refactor as a sealed interface instead, and internal impl for the existing stuff

Alternatively, keep typing as is and have const() accept these enums (and convert to string under the hood). That's what your code example above appears to do

@westnordost

That's what your code example above appears to do

Not quite, what I landed with is to pass the StrokeCap all the way through, let StrokeCap implement an interface IsValueConvertibleToString { fun valueToString(): String } and add is IsValueConvertibleToString -> JsonPrimitive(value.valueToString()) to utils.kt/normalizeJsonLike . But not sure about this and whether the solution(s) you wrote are cleaner. After all, it requires an implementation of ExpressionScope.const for every single type.
I could also just do

object StrokeCap {
  const val Butt = "butt"
  // etc.
}

for now, and finding a maybe better solution (i.e. proper types) can be done separately, later

Oh, actually, according to documentation, e.g. https://maplibre.org/maplibre-style-spec/layers/#line-cap does not support an expression

@sargunv

oh interesting, it still takes an Expression instance in the SDK but maybe only supports a constant value expresiion

@westnordost

I will look into that and either correct the maplibre-style documentation or SDK. (Or at least create an issue for that)
I'll also add at least the stubs for the text* parameters in SymbolLayer while I am looking through this file

@sargunv

I'm not sure how likely or how often it is that a property that accepts only a constant value is later updated to support expressions? Maybe accepting a (constant) expression is a hedge for forwards compatibility in case the property supports expressions later?

@westnordost

Mh well, but on the other hand, it wouldn't be nice to be greeted by an error (or the layer silently not working as intended) when one uses an expression where none is supported.
Actually, the same applies within expressions: There are different expression types, and all those properties each only support specific expressions.
Anyway, for the documentation PR I am doing, I will not change anything there.

@sargunv

Actually, the same applies within expressions: There are different expression types, and all those properties each only support specific expressions.

I have some rudimentary type safety here with return values of expressions (string, number, image, etc) using a generic type parameter, but it's awkward to use at times as Kotlin doesn't have union types and some expressions accept those, and also some (like get) can return anything. Some properties accept or don't accept interpolate expressions, some accept or don't accept feature state expressions. None of this can really be expressed well with Kotlin's type system.

The Android and iOS SDKs don't have any type safety for expressions, and I might have to go that route too?

I think a more advanced type system like Typescript's would be needed to effectively encode the rules of expressions, or a dedicated expression validator (long term idea, what if there was a new language for expressions with a dedicated validator, and this checker could be integrated into a compiler plugin for use with this library? Would also help with cleaner syntax)

@westnordost

I guess ¯_(ツ)_/¯. I was hoping you might know a solution to this, but maybe there just isn't a good one.

The expression syntax is the way it is because it is based on JSON - that base will be hard to change. In Tangram-ES, the style was defined in YAML and it was possible to instead of providing constant values for style properties, define a JavaScript function that returns some value. That was very powerful, although slightly weird. (I.e., over the top)

(from the point of view of the YAML, that injected JavaScript was just a string)

@sargunv

but maybe there just isn't a good one.

I think there is, but the solution doesn't lie in the Kotlin type system, and is a significant project of its own (so effort vs reward might be too high)

The expression syntax is the way it is because it is based on JSON - that base will be hard to change.

Yup, and that's the reason I think Typescript's type system would be suited to solving this (as opposed to Kotlin's). That type system was designed to handle all the wonky things that happen in JS/JSON APIs. Taken to an extreme, that type system is even turing complete and can do compile time validation of arbitrary LL(1) grammar on string literals. So like, maplibre-gl-js could have this level of compile time type safety in expressions, but we can't (directly).

But while Kotlin doesn't have a turing complete type system, it does have compiler plugins.

So imagine there's a standalone program (whether in TypeScript or something more conventional) that takes a JSON expression as input, and validates that the JSON represents a valid expression for a certain type of property (like stroke width). This program is essentially a parser and type checker for the expression "language", which is a subset of JSON.

And then there's a Kotlin compiler plugin that uses this validator to validate expressions at compile time.
So I could see a solution someday with an API like this, getting validated for type safety at compile time:

val a: Expression<StrokeWidth> = expr("[...]")
// or
LineLayer(lineWidth = expr("[...]"))

where the compiler plugin knows the constraints on a StrokeWidth ("returns Number, supports feature-state and interpolate") and passes those to the validator. And since this standalone validator is defining its own "language", it could have some sugar for more ergonomic expression syntax (transforming get("foo") to ["get", "foo"] inline)

Any such solution would have some complex hurdles, like supporting runtime parameters to expressions (to say, animate a color in Compose) or composing expressions (to say, encapsulate an interpolator to use across multiple properties). And I'm not sure how feasible those parts are, but it might be fun to explore someday in the future

But yeah, that's all putting a ton of effort into solving the problem of type safety for expressions, and probably isn't worth it, though would be a fun project 😅