Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add modernized data-treify into package? #14

Open
cartazio opened this issue Feb 16, 2023 · 23 comments
Open

add modernized data-treify into package? #14

cartazio opened this issue Feb 16, 2023 · 23 comments

Comments

@cartazio
Copy link
Contributor

https://github.com/cartazio/data-treify-ng/tree/main/src/Data I migrated Conals data Treify into using Data.Type.Equality and Type.Reflection, might be nice to make things a one stop shop once and for all?

thoughts: @andygill @RyanGlScott and others?

@RyanGlScott
Copy link
Member

I've never heard of data-treify before. I'm not necessarily opposed to the idea of incorporating something like this into data-reify, but I don't understand it well enough to say if this makes sense (or if this is something that I could reasonably maintain). Some questions that come to mind after glancing at the library:

  1. Why did this code come to be? The .cabal file mentions that data-reify was "tweaked it for typed syntax representations for use with GADTs", but it's not clear to me exactly what that means.
  2. Is data-treify a strictly more powerful interface than what data-reify offers? That is, does the former subsume the latter? Or are they solving different problems?
  3. What are some examples of actually using data-treify? The repo doesn't include any examples, which leaves me wondering about how it's actually put into practice. Having some examples or a test suite in data-treify would help me a lot to contextualize where it sits in relation to data-reify.

@cartazio
Copy link
Contributor Author

cartazio commented Feb 17, 2023 via email

@cartazio
Copy link
Contributor Author

so heres an example where data-reify fails, thats a self contained file

{-# LANGUAGE TypeFamilies, DeriveFunctor, DeriveFoldable, DeriveTraversable #-}
{-# LANGUAGE GADTs #-}
{-# OPTIONS_GHC -Wno-unused-imports #-}
module Main(main,Exp(..),ex1,ex2,ExpF(..)) where 

import Control.Applicative
import Data.Reify
import Data.Traversable
import Data.IntMap as IntMap
import Data.Foldable
import Data.Monoid

import Type.Reflection
import Data.Type.Equality
import Data.Kind (Type)

main :: IO ()
main = print "indexed"

data Exp :: Type -> Type where 
    ETrue :: Exp Bool 
    EFalse :: Exp Bool
    EFloat :: Float -> Exp Float 
    EEq :: Exp a -> Exp a -> Exp Bool 
    EAnd :: Exp Bool -> Exp Bool -> Exp Bool 
    EPlus :: Exp Float -> Exp Float -> Exp Float 

data ExpF :: Type -> Type where 
    EFTrue :: ExpF a 
    EFFalse :: ExpF a
    EFFloat :: Float -> ExpF a 
    EFEq :: ExpF a -> ExpF a -> ExpF a  
    EFAnd :: ExpF a -> ExpF a -> ExpF  a 
    EFPlus :: ExpF a -> ExpF a -> ExpF a 


ex1 :: Exp Float 
ex1 = EFloat 1 `EPlus` EFloat 2 

ex2 :: Exp Bool  --- example of implicit sharing 
ex2 = ex1 `EEq` ex1 

instance MuRef (Exp i) where
  type DeRef (Exp i) = ExpF
  mapDeRef f x = case x of 
    ETrue -> pure EFTrue
    EFalse -> pure EFFalse
    EFloat x -> pure $ EFFloat x 
    EEq a b -> liftA2 EFEq  (f a)   (f b) 

with the error being


  /Users/carter/WorkSpace/projects/active/data-reify/examples/indexed.hs: line 49, column 30:
    error:
    • Couldn't match type ‘u’ with ‘ExpF u’
      Expected: f (ExpF u)
        Actual: f u
      ‘u’ is a rigid type variable bound by
        the type signature for:
          mapDeRef :: forall (f :: * -> *) u.
                      Applicative f =>
                      (forall b. (MuRef b, DeRef (Exp i) ~ DeRef b) => b -> f u)
                      -> Exp i -> f (DeRef (Exp i) u)
        at examples/indexed.hs:45:3-10
    • In the second argument of ‘liftA2’, namely ‘(f a)’
      In the expression: liftA2 EFEq (f a) (f b)
      In a case alternative: EEq a b -> liftA2 EFEq (f a) (f b)
    • Relevant bindings include
        f :: forall b. (MuRef b, DeRef (Exp i) ~ DeRef b) => b -> f u
          (bound at examples/indexed.hs:45:12)
        mapDeRef :: (forall b.
                     (MuRef b, DeRef (Exp i) ~ DeRef b) =>
                     b -> f u)
                    -> Exp i -> f (DeRef (Exp i) u)
          (bound at examples/indexed.hs:45:3)
   |
49 |     EEq a b -> liftA2 EFEq  (f a)   (f b) 
   |                              ^^^

@RyanGlScott
Copy link
Member

Thanks for the examples!

Before diving into the specifics of data-treify in particular, I wanted to note that the DeRef instance in #14 (comment) seems a little strange. The DeRef instance for a data type is typically a "base functor" version of the same data type. That is, I would expect ExpF to have the same structure as Exp, but with all recursive occurrences of Exp replaced with the type parameter. For example, for the EEq data constructor:

EEq :: Exp a -> Exp a -> Exp Bool

I would have expected the corresponding data constructor in the ExpF side to look something like this:

EFEq :: a -> a -> ExpF a 

But instead, your example defines it as:

EFEq :: ExpF a -> ExpF a -> ExpF a

As far as I can tell, this is the reason why the MuRef instance fails to typecheck, as changing the definition of EFEq to the former makes it work.

That being said, this doesn't seem like the full story. One thing that ExpF doesn't do at all is mirror the GADT indexing of the original data type. For instance, the return type of EEq is:

EEq :: ... -> Exp Bool

That is, where are indexing the return type by Bool. On the other hand, the return type of EFEq doesn't have any indexing at all:

EFEq :: ... -> ExpF a

Is this what you'd expect? I'm genuinely asking here, as I have no prior experience with the idioms of data-treify.

@RyanGlScott
Copy link
Member

As far as whether data-reify subsumes data-treify or not, I think the answer is most likely "no". At least, I am very unclear on how you'd port Conal's example over to data-reify. In that example, the N base functor marks recursive occurrences of E in a very interesting way that doesn't seem to be possible with data-reify's version of MuRef/DeRef.

It's interesting that the kind of E is (Type -> Type) -> Type -> Type, as writing a MuRef instance for something of that kind feels very similar to writing an instance of TraversableFC from parameterized-utils. In fact, I was originally going to claim that data-reify's MuRef is to Traversable as data-treify's MuRef is to TravesableFC, but your example in #14 (comment) has kind Type -> Type, not (Type -> Type) -> Type -> Type.

This is all to say: I'm still not sure if I've fully understood all of the subtleties of what is going on in data-treify, but I feel like I'm getting closer.

@cartazio
Copy link
Contributor Author

yup my example was wrong , i forgot to replace the recursive instances with the parameter

EFEq :: a -> a -> ExpF a 

But instead, your example defines it as:

EFEq :: ExpF a -> ExpF a -> ExpF a

@cartazio
Copy link
Contributor Author

i'll cleanup the indexed type example a little bit more and include it with the other now mostly buildable examples as a new PR.

My conclusion so far (after you pointed out my mistake) is that

"it looks like you can use data reify on typed syntax, but you'll wind up needing to do some sort of existential or runtime checks". So i think i can write something that roughly looks like

implicit2explicitSharing :: Typable t => ExpNoLet a ->ExpWithLet a that internally is using a functor ExpF that caches a typeable dictionary at each constructor.

and then the conversion from the graph back to syntax would just be checking the indices one way or another, and a toplevel "sanity check" using the top level typeable instance. or something like that

@cartazio
Copy link
Contributor Author

yeah, the example i have here

instance MuRef (Exp i) where
  type DeRef (Exp i) = ExpF
  mapDeRef f z = case z of 
    ETrue -> pure EFTrue
    EFalse -> pure EFFalse
    EFloat x -> pure $ EFFloat x 
    EEq a b ->  EFEq <$>(f a) <*>   (f b) 
    EAnd a b -> EFAnd <$>  (f a) <*>  (f b )
    EPlus a b ->EFPlus <$> (f a)  <*> (f  b)

is basically a forgetful functor/type erasure

@cartazio
Copy link
Contributor Author

i guess this ticket is my misadventures in understanding how to use data-reify for typed syntax :)

@cartazio
Copy link
Contributor Author

cartazio commented Feb 18, 2023

my example isn't a port of the Conal style "parameterized over variable rep and indexed by expression type" because It seems like it (the type indices in the conal one) require Existential wrapping like

data WithTypeable (f :: Type -> Type) where 
   OnTyped :: Typeable a => f a -> WithTypeable f 

wrapping the "data-reifyed" sharing graph nodes

@RyanGlScott
Copy link
Member

I've had a chance to look at this code a little more closely, and one thing that I can't help but wonder is: why is Typeable used so pervasively in the API? I'm not really sure what benefits it provides. It's certainly not required to implement the API that data-reify provides, as it is simply using StableNames (without any Typeable business) under the hood. In fact, I wonder if it would be simpler to define MuRef like so:

class MuRef (h :: k -> Type) where
  type DeRef h :: (k -> Type) -> k -> Type

  mapDeRef :: forall m v. Applicative m
           => (forall a. h a -> m (        v a))
           -> (forall a. h a -> m (DeRef h v a))

This is much simpler, as now there is no need to make MuRef a multi-parameter type class that threads a ty type parameter around everywhere. This presentation of MuRef looks even closer to TraversableFC than before, which is another argument in its favor. (I consider data-reify's MuRef to be a very specialized version of Traversable, so it would be nice to have something that completes the analogy for TraversableFC.)

Similarly, I think we could simplify the machinery in Data.Reify.TGraph quite a bit. I think we could get rid of the Bind/V types and just define Graph like so:

data Graph (e :: k -> Type) =
  Graph
    [(Unique,Some (e (Const Unique)))]
    Unique

Where Some is defined as:

data Some :: (k -> Type) -> Type
  Some :: f x -> Some f

The use of Some is a requirement of working with GADTs, as we don't know ahead of time what the type index for each node in the graph will be. The rest of data-treify could be implemented in nearly the same way as data-reify with minimal changes involving Some and Const.

What do you think of this version, @cartazio? It's rather different from the original presentation in data-treify, but I think the benefits would be worth it. In particular, you'd no longer need to stick Typeable constraints everywhere in your data type definitions.

@cartazio
Copy link
Contributor Author

I like this design / idea! This actually seems like stab in the direction of a more principled api. (comparing some of the factoring in treify to the data-reify api did raise some eyebrows for me)

in the case of the Some, that would push any dictionary bits into the right hand side of type DeRef h= , correct? (the target nonrecursive functor)

i'm a teeny bit confused about your proposed Graph definition

data Graph (e :: k -> Type) =
  Graph
    [(Unique,Some (e (Const Unique)))]
    Unique

the kind of Const Unique is forall k . k -> Type right?
so the kind of e should perhaps be (k -> Type) -> Type ?

@RyanGlScott
Copy link
Member

the kind of Const Unique is forall k . k -> Type right?
so the kind of e should perhaps be (k -> Type) -> Type ?

Oops, that was a typo. I meant to write data Graph (e :: (k -> Type) -> k -> Type) ... there. That way, you can write this as the type signature for reifyNodes:

reifyGraph :: MuRef h => h a -> IO (Graph (DeRef h))

in the case of the Some, that would push any dictionary bits into the right hand side of type DeRef h= , correct? (the target nonrecursive functor)

I'm not quite sure what you mean by "dictionary bits". Are you asking what happens when GADT constructors have constraints in their type? That is something that this version of MuRef would continue to support. For example, here is what Conal's E example would look like in this version:

data E :: (Type -> Type) -> Type -> Type where
  Op   :: Op a -> E v a
  (:^) :: (Typeable a, Typeable b) =>
          E v (a -> b) -> E v a -> E v b

data N :: (Type -> Type) -> Type -> Type where
  ON  :: Op a -> N v a
  App :: (Typeable a, Typeable b) =>
         v (a -> b) -> v a -> N v b

instance MuRef (E v) where
  type DeRef (E v) = N
  mapDeRef _ (Op o)   = pure $ ON o
  mapDeRef k (f :^ a) = App <$> k f <*> k a

The Some is only needed because there are recursive occurrences of E in (:^) where the type index changes (e.g., E v (a -> b), and in order to represent all of these possible types in a list, you have to existentially close over the type index to make everything have the same type.

@cartazio
Copy link
Contributor Author

cartazio commented Feb 18, 2023

to be clear: i totally see that there shouldn't be a need for dictionaries in bottom up reconstruction, i'm just trying to make sure I see a clear way to get f:: Class c => ImplicitSharingExp a -> Maybe (ExplicitSharingExp a) definable

@cartazio
Copy link
Contributor Author

ok gotcha

@cartazio
Copy link
Contributor Author

this looks nice!

@cartazio
Copy link
Contributor Author

if the typeable dictionaries aren't needed, all the better!

@cartazio
Copy link
Contributor Author

cartazio commented Feb 18, 2023

(like, would SurfaceExp ty -> ExplicitSharingExp ty be definable on top of data-reify without Typeables somewhere? aka "prove" that the two programs have the same top level type?

@cartazio
Copy link
Contributor Author

and without any unsafe trust me casts? (though that would be pretty safe i think)

@RyanGlScott
Copy link
Member

would SurfaceExp a -> ExplicitSharingExp a be definable on top of data-reify without Typeables somewhere?

I'm not entirely sure. I presume that Exp is an example of how to do this? Admittedly, I haven't tried porting the entirety of that file over to the new design, so I think a first step towards answering your question would be to attempt this. That code heavily relies on Bind and V, but I do think a lot of it is isomorphic to Some (Const Unique).

The only part that might be tricky to do without Typeable is this bit of code:

   extract a' (Just (Bind (V _ a) n))
     | Just Refl <- a `testEquality` a' = n
     | otherwise                = error "bindsF: wrong type"

This is doing a type-safe cast using a TypeRep. I suspect that code could be rewritten to avoid using TypeRep, but I haven't tried doing so myself.

@cartazio
Copy link
Contributor Author

i'm just poking at the in-tree example, not presupposing its authoritative, but I think you're right, that's the hard bit :)

I"ll have a look see in a few days, I think the forgetful formulation will suffice for some of my near term stuff.

@cartazio
Copy link
Contributor Author

i guess the lin

   extract a' (Just (Bind (V _ a) n))
     | Just Refl <- a `testEquality` a' = n
     | otherwise                = error "bindsF: wrong type"

really comes down to "you'll need something that provides an a~b constraint.

full error

  /Users/carter/WorkSpace/projects/active/data-treify-ng/src/Exp.hs: line 159, column 52:
    error:
     Couldn't match type a1 with a'
      Expected: n (V TypeRep) a'
        Actual: n (V TypeRep) a1
      a1 is a rigid type variable bound by
        a pattern with constructor:
          Bind :: forall (f :: * -> *) (n :: (* -> *) -> * -> *) a.
                  Typeable a =>
                  V f a -> n (V f) a -> Bind f n,
        in an equation for extract
        at src/Exp.hs:158:22-35
      a' is a rigid type variable bound by
        the type signature for:
          extract :: forall a'.
                     TypeRep a' -> Maybe (Bind TypeRep n) -> n (V TypeRep) a'
        at src/Exp.hs:156:4-70
     In the expression: n
      In an equation for extract’:
          extract a' (Just (Bind (V _ a) n))
            | True = n
            | otherwise = error "bindsF: wrong type"
      In an equation for bindsF’:
          bindsF binds
            = \ (V i' a') -> extract a' (I.lookup i' m)
            where
                m :: I.IntMap (Bind TypeRep n)
                m = I.fromList [(i, b) | b@(Bind (V i _) _) <- binds]
                extract :: TypeRep a' -> Maybe (Bind TypeRep n) -> n (V TypeRep) a'
                extract _ Nothing = error "bindsF: variable not found"
                extract a' (Just (Bind (V _ a) n))
                  | True = n
                  | otherwise = error "bindsF: wrong type"
     Relevant bindings include
        n :: n (V TypeRep) a1 (bound at src/Exp.hs:158:35)
        a :: TypeRep a1 (bound at src/Exp.hs:158:32)
        a' :: TypeRep a' (bound at src/Exp.hs:158:12)
        extract :: TypeRep a' -> Maybe (Bind TypeRep n) -> n (V TypeRep) a'
          (bound at src/Exp.hs:157:4)
    |
159 |      | True {-Just Refl <- a `testEquality` a'-} = n
    |                                                    ^

@cartazio
Copy link
Contributor Author

i'll have a go at trying your suggestions on data-treify and share what i learn. I suspect that something with constraint kinds might be needed, where we default to no class constraints .. or maybe just some uses push a typeable constraint on the instance like maybe i just need

instance Typeable t => MuRef (f t) where 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants