`:incomplete` detection should properly consider where in the line I am

[Keno]

In the REPL, if I do

Core.eval(Base.Experimental, quote

and then hit enter, I get the opportunity to write code on a new line because the parser knows that the expression is incomplete.
However, if I write

Core.eval(Base.Experimental, quote)

even with the cursor after the quote, I get a syntax error.
This is a big problem with the latest REPL, because it inserts the close-paren automatically, so
typing the sequence that used to work now breaks.

[KristofferC]

How is that related?

[KristofferC]

The diff for this is quite trivial:

--- a/stdlib/REPL/src/REPL.jl
+++ b/stdlib/REPL/src/REPL.jl
@@ -1117,7 +1117,10 @@ end
 LineEdit.reset_state(hist::REPLHistoryProvider) = history_reset_state(hist)
 
 function return_callback(s)
-    ast = Base.parse_input_line(takestring!(copy(LineEdit.buffer(s))), depwarn=false)
+    buf = LineEdit.buffer(s)
+    pos = position(buf)
+    input = String(buf.data[1:pos])
+    ast = Base.parse_input_line(input, depwarn=false)
     return !(isa(ast, Expr) && ast.head === :incomplete)
 end

However, after using this, I feel we kind of need a commit line keybind. It is often you are in the middle of a function and want to commit what you have written. My fingers instinctively reached for Shift + Enter but I don't think there is a good way to detect this in a portable way in terminals. One option is to repurpose Alt+Enter to be commit line (since that will be way less useful after this) but maybe people will get upset by this...

[tecosaur]

but I don't think there is a good way to detect this in a portable way in terminals.

The best way is the Kitty Keyboard Protocol (KKP), which is pretty broadly supported at this point:

• Kitty (of course)
• Alacritty
• Ghostty
• Foot
• iTerm2
• rio
• WezTerm
• TuiOS (multiplexer)

The fish shell uses exactly this, for this purpose:

If the terminal supports shifted key codes from the kitty keyboard protocol, shift-enter now inserts a newline instead of executing the command line.
[v4 release notes]

We just need to do some more terminal querying to check support. We can do this in a blocking manner if we assume that all terminals support vt100 DA1 queries in a FIFO manner, otherwise since we can't safely read from stdin with a timeout we'd want to use a similar approach as I introduce in #59958.

See: https://sw.kovidgoyal.net/kitty/keyboard-protocol

[tecosaur]

BTW, there are more Kitty protocols that I do think it would be nice to support in a progressive-enhancement manner, e.g. notifications after long-running REPL execution completes

[KristofferC]

The best way is the Kitty Keyboard Protocol (KKP)

Yeah, that's why I had the word "portable" in there :P. We could do fancy stuff with that but something as basic as commit line should probably be available everywhere.

[tecosaur]

"works everywhere" compatibility is indeed worth aiming for. However, for non-essential functionality (you can still use the REPL without this), when there's a keybinding that most people are already used to (shift+enter) and we can support it on most terminals, it seems like that might be the way to go to me.

[tecosaur]

I also think there's an argument for us supporting KKP for other reasons, such as being able to distinguish esc being pressed from control sequences.