Term.Resolved

[FlandiaYingman]

This is a draft PR that adds Term.Resolved to indicate that a term's symbol is resolved. This information can be used later in the lowering stage. For example, append .class to a ref to an overloaded module definition.

[FlandiaYingman]

I am facing some problems now:

• Now only Ref can expand to be appended with .class. But we should also do that to selections. (This is the reason that the current test cases fail.) However, this is particularly difficult for selections because there is already an existing symbol mechanism for selections. I tried a bit for a few hours, but I guess I am going into another refactorization hell 🤥...

[FlandiaYingman]

OK I managed to wrap selections with Term.Resolved. The next step is to remove the existing symbols in selections, and replace them with the symbols in Term.Resolved.

[FlandiaYingman]

Hello @CAG2Mark and @chengluyu, I need your help on this PR 🤧🥺, particularly regarding the lifter and the UCS/UPS normalization.

Summary on this PR

In this PR, we try to disambiguate Term.Ref's and Term.Sel's BlockMemberSymbol during resolution (disambiguation means to infer which overloaded definition does the term refer to in the specific context), by wrapping the terms that have to be disambiguated in a term Term.Resolved(t: Term, sym: DefinitionSymbol[?]). (DefinitionSymbol is the same as MemberSymbol, except that BlockMemberSymbol is not a DefinitionSymbol but a MemberSymbol).

A direct consequence is that, we now have enough information in the lowering/codegen stage to defer the process of adding the Sel(t, new Ident(".class") thing for a Ref or Sel referring to a overloaded term. Originally it was in the elaborator, and then with one of my previous PR it was moved to the resolver. Now in this PR, it will be moved to the JS builder, which is also the right place, as this was just a JS trick to compile overloaded definitions.

I changed Value.Ref to encode the disambiguation information (to be used later during code generation):

- case Ref(l: Local)
+ case Ref(l: Local, disamb: Opt[DefinitionSymbol[?]])

and Select to encode the disambiguated symbol information (we want to be as specific as possible):

- case class Select(qual: Path, name: Tree.Ident)(val symbol: Opt[FieldSymbol])
+ case class Select(qual: Path, name: Tree.Ident)(val symbol_SelectSymbol: Opt[DefinitionSymbol[?]])

Then, the JS builder automatically, based on the information encoded, decides whether or not to append the .class suffix to the generated code for a Ref or Select. The exact logic of appending .class is here:

    case Value.Ref(l, disamb) => l match
      case l: BlockMemberSymbol if {
        // TODO @Harry: refactor(ize) duplicated logic
        l.hasLiftedClass && 
        disamb.flatMap(d => d.asMod orElse d.asCls).exists(_ isnt ctx.builtins.Array)
      } =>
        doc"${getVar(l, l.toLoc)}.class"
      case _ =>
        getVar(l, r.toLoc)

    case s @ Select(qual, id) => 
      val dotClass = s.symbol_SelectSymbol match
        case S(ds) if {
          val v = ds.asBlkMember.exists(_.hasLiftedClass) && 
          (ds.asMod orElse ds.asCls).exists(_ isnt ctx.builtins.Array)
          v
        } => doc".class"
        case _ => doc""
      val name = id.name
      doc"${result(qual)}${
        if isValidFieldName(name)
        then doc".$name"
        else name.toIntOption match
          case S(index) => doc"[$index]"
          case N => doc"[${makeStringLiteral(name)}]"
      }${dotClass}"

What's the Problem

In the lifter and the UCS/UPS normalization, there is a bunch of customized logic of appending .class to a term. With this PR, they now append duplicate .class suffixes.

Also, seems like the compiler largely transforms the terms in the lifter and the UCS/UPS normalization process. Some encoded information is stripped out, but I couldn't find out the reason 😥.

UCS/UPS Normalization

I am taking EvenOddTree.mls as an example.

Looking at the resolved tree for the patterns:

:rt
pattern OddTree =
  A |
  Node(EvenTree, A, EvenTree) | Node(OddTree, A, OddTree) |
  Node(EvenTree, B, OddTree) | Node(OddTree, B, EvenTree)
pattern EvenTree =
  B |
  Node(EvenTree, A, OddTree) | Node(OddTree, A, EvenTree) |
  Node(EvenTree, B, EvenTree) | Node(OddTree, B, OddTree)
//│ Resolved tree:
//| ...
//│             left = Composition:
//│               polarity = true
//│               left = Constructor:
//│                 target = Resolved{sym=object:A,typ=object:A}:
//│                   t = Ref{sym=member:A} of member:A
//│                   sym = object:A
//│                 patternArguments = Nil
//│                 arguments = N
//│               right = Constructor:
//│                 target = Resolved{sym=term:class:Node.Node}:
//│                   t = Ref{sym=member:Node} of member:Node
//│                   sym = term:class:Node.Node

The references to A and Node are resolved correctly to object:A and term:class:Node.Node (which is the constructor of Node). But after lowering, the lowered tree becomes:

:lot
B is @compile EvenTree
//| Lowered tree:
//| ...
//│               path = Ref:
//│                 l = member:A
//│                 disamb = N
//| ...
//│               path = Select{sym=class:Node}:
//│                 qual = Ref:
//│                   l = member:Node
//│                   disamb = N
//│                 name = Ident of "class"

while we expect it to be

:lot
B is @compile EvenTree
//| Lowered tree:
//| ...
//│               path = Ref:
//│                 l = member:A
//│                 disamb = object:A
//| ...
//│               Ref:
//│                   l = member:Node
//│                   disamb = term:class:Node.Node

Would you provide some help on this case? Thanks Luyu 😉

Lifter

Taking lifter/ClassWithCompanion.mls as an example:

The C.empty in

fun foo(x) =
  class C(y) with
    fun get = [x, y]
  module C with
    val empty = C(123)
  C.empty

should be compiled to JS C.class.empty. The resolved tree indicates that it is correctly resolved that C actually refers to a module C:

//│         res = Resolved{sym=term:module:C.empty}:
//│           t = Sel{sym=member:empty}:
//│             prefix = Resolved{sym=module:C,typ=module:C}:
//│               t = Ref{sym=member:C} of member:C
//│               sym = module:C
//│             nme = Ident of "empty"
//│           sym = term:module:C.empty

Afte lowering, it is expected to be

//│         rhs = Select{sym=term:module:C.empty}:
//│           qual = Ref{disamb=module:C}:
//│             l = member:C
//│             disamb = S of module:C
//│           name = Ident of "empty"

However, if :lifter is specified, the inner reference magically loses its disambiguation information:

//│         rhs = Select{sym=term:module:C.empty}:
//│           qual = Ref:
//│             l = member:C
//│             disamb = N
//│           name = Ident of "empty"

Would you look into it? Thanks Mark 💓

[CAG2Mark]

Oh cool, something to do in tomorrow's conference when I am bored

[CAG2Mark]

You can try this commit @FlandiaYingman

CAG2Mark@f24b454

[chengluyu]

@FlandiaYingman 🥹 To be honest, I still don’t quite understand what you want to do. Let’s set a time on Discord to discuss it in person.

[FlandiaYingman]

Finally, now only 15 cases fail.

• Lifter (I applied Mark's patch, but a new problem arrives!)
• LLIR
• Handler
• WASM

[FlandiaYingman]

OK the reason why WASM failed is because I forgot to npm install. 🤡

[FlandiaYingman]

Now only 4 test cases fail!

• Lifter (asking Mark)
• Object Buffer

[FlandiaYingman]

Now only 2 test cases fail!

[LPTK]

👀

[FlandiaYingman]

What?

[FlandiaYingman]

Hoooooooooray!!! Thanks @CAG2Mark and @chengluyu 💞 for the help

I guess this PR is almost ready. I'll complete the cleanup tomorrow (today).

In this PR, the symbols in Term.Sel and Term.SynthSel were not removed, and the symbol resolution in the elaborator was also not completely moved to the resolver. I'll work on them when I get my courage back to touch on the symbols :<

But the good news (maybe) is that the two symbol resolution mechanism are completely separated. i.e., the lowering stage solely relies on the symbol resolved by the resolver. This shall facilitate the WASM backend and other backends (if any).

[FlandiaYingman]

It's ready for review~