Subtype relation for refined type fails

[thierry-st]

izumi.reflect.Tag subtype relationship is not faithful to compiled ones.

Here is a minimal repro (Scala 2 syntax to allow testing various versions):

import izumi.reflect.Tag

trait A {
  type T
}

trait AInt extends A {
  override type T = Int
}

object App extends App {
  implicitly[AInt <:< A { type T = Int }] // true
  println(Tag[AInt].tag <:< Tag[A { type T = Int }].tag) // false
}

Scastie link

It might be more efficient to tackle this issue and #469 both at once, but they seemed different enough to me to warrant different posts.

It might help to know that the issue is not present for Scala >= 3.2 and izumi-reflect =< 2.2.5. But the issue is present if violating any of these bounds (Scala 2 and/or izumi-reflect >= 2.3.0).

I will report soon the corresponding issue on the ZIO repo.

[thierry-st]

Here is the link to the corresponding ZIO issue: #9210

[pshirshov]

We would need to preserve full member information in order to support this. Doable but not easy. My primary concerns are tag size and performance.

[neko-kai]

@thierry-st Version 2.3.0 is the one that added structural type / refinement support to Scala 3 version. Before that, the comparison on Scala 3 was only between AInt and A, ignoring the refinement, that's why it succeeded on previous versions.

Currently type/value members are not present at all in izumi-reflect type model for concrete types - they only arise when comparisons between refinements happen.

Since there's no data about type T in AInt tag, the comparison with any refinement will fail.

Changing the design and adding type member data is possible, but not trivial.

(#469 is unfortunately not related)

[neko-kai]

@thierry-st Does this limitation of the model make some real-world usages too painful? If not, we'll probably not work on fixing this – although we will accept pull requests.

For now, I've added this issue to the list of known limitations.

Note that the more common opposite case, e.g. Aux pattern, where a refinement is a subtype of a concrete type, is supported.

[thierry-st]

Thanks very much for the clear explanations and the addition to known limitations!

As for painfulness of real-world usages, in the ZIO environment, the limitation can easily be overcome. So the only downside is cognitive cost (ZIO users need to be aware of this issue). Since nobody reacted so far on the ZIO issue, I'm inclined to leave it as is.

Just to let people know, I'm not going to attempt a PR for this, so feel free to go for it!

[pshirshov]

is cognitive cost

Unfortunately, it seems to be nearly impossible to implement a perfect RTTI at macro level, and the previous attempt made by the compiler team (scala-reflect) had many its own shortcomings. So, we may patch various leaks but it's not always feasible (and not always possible) to do so.

If we can't preserve the full type structures until run-time and bring the full compiler (or, at least, a significant part of the typer) there, discrepancies will always be there together with the associated cognitive cost.

Still, the library is good enough to just work in vast majority of the usecases. 🤷‍♀️

[jdegoes]

/bounty $750

[goshacodes]

@pshirshov Could you please help me understand FullDbInspector#buildFullDb?

It takes a type and constructs a Map of runtime representations of type and its parents to their parents, is it correct? Why do we need a mapping from parent to its parents here and not only our type to all its parents?
UPD: I got it - we need to check if something is subtype of our type. sorry for mention

[pshirshov]

It is intended to enumerate all the parents of a particular type up to the roots because we don't know which indirect parents may be passed to <:<. Of course it also extracts the relationships between parents and their own parents, I don't see how you may limit extraction. Although it does not attempt to extract all the parents of all the types referenced in the inheritance hierarchy (e.g. the generic parameters), the database gets too big if we do that.

[pshirshov]

As a starting point for this code I would recommend to look at the member extractors for refined types. The same logic stretched to all the types should allow you to approach the goal. You would need to alter the model, it might introduce incompatibilities but it's fine (most likely you may fix incompatibilities by assuming that your new collections are empty if they aren't present in the binary form). Probably this is a huge work to be done, previously I've made an attempt but quickly realized that much more time is required than I anticipated.

[goshacodes]

Thank you