Smart quotes and CJK

[AlbertLei]

Pandoc seems to need an extra space or newline to turn straight quotes into curly quotes:

pandoc -f markdown -t html
我"好"了 我 "好" 了 I "am" ok I"am"ok
# <p>我”好”了 我 “好” 了 I “am” ok I”am”ok</p>

• In both 我"好"了 and I"am"ok, the quotes are turned into left curly quotes on both sides.

The output will be the same if I enable the +east_asian_line_breaks feature for markdown.

As a native Chinese user, I can confirm that the usual way is to write:
我“好”了. Ie, no space around quotes. For now, it looks like I have to type the curly quotes manually.

Is there a simple Lua filter that can save me from typing curly quotes here? It should work on all text elements (pars, lists, etc.) but skip elements like code blocks.

Or, would it make sense to request a new +east_asian_smart_quotes feature, like the existing +east_asian_line_breaks?

My pandoc version

• pandoc 3.8.2.1
• Features: +server +lua
• Scripting engine: Lua 5.4

[AlbertLei]

Some background info: In simplified Chinese (and Korean), the quotation marks share the same Unicode codepoints as European ones (“ and ”). source

In hindsight, the title is misleading. Traditional Chinese and Japanese use 『 』 and 「」 for quotation marks. I think those users have to manually type 「」 when writing markdown.

[jgm]

In English, at least, the space cues are crucial for figuring out which quotes are right and which are left (especially in the case of single quotes, which also occur as apostrophes). I am not sure what the conventions are in CJK. Is there a use of single quote at all? If so, is there a use as an apostrophe? Do quotation marks always come in matched pairs?

[AlbertLei]

I am not sure what the conventions are in CJK.

I can confirm all CJK languages generally do not use spaces around punctuation, including quotation marks, commas, periods, so on.

Is there a use of single quote at all?

Yes. In simplified Chinese and Korean, quotation marks are used in a way similar to English: double quotes for main quotations, single quotes for nested ones.

If so, is there a use as an apostrophe?

No. I can confirm apostrophes are for romanized texts only.

Do quotation marks always come in matched pairs?

Yes.

To summarize: simplified Chinese and Korean use quotation marks in a very similar way to English, except that there are no spaces around them.
(Japanese and traditional chinese use 『 』 and 「」 for quotation marks.)

[jgm]

If they are always in matched pairs, with no single apostrophes, then it could be possible to add something like east_asian_smart_quotes.

[AlbertLei]

About matched quotation pairs and no single apostrophes, I am 99% sure that's true for chinese.

Since I don’t write regularly in Japanese/Korean, I cannot be very certain about their usage.

[AlbertLei]

For the potential east_asian_smart_quotes extension to be truly useful, i wish it should handle some simple mixed usage of CJK and English quotation marks (and apostrophes).
For example:

• 二'二'（a' b） should be rendered as 二‘二’（a’ b）.
• Note there is no space between the second quotation mark and the full-width CJK left pathenthesis. So, the current pandoc -t html will render all three quotations as apostrophes.

[jgm]

Yes, the mixed-text case is going to be quite tricky.

[TomBener]

Quotation marks in Simplified Chinese are really annoying so I have to create a series filters to handle them in my project: https://github.com/TomBener/quarto-chinese

[AlbertLei]

Thx for the resource. I don't use Quarto now, but your project looks helpful.

I might try it next time I need to provide a docx format paper (though I hope that never happens)...

[tarleb]

To clarify another potential edge-case: does Simplified Chinese (or other another East Asian language) use quotes " to emulate the double prime symbol for inch ″, e.g., she is 5'9"? I'd assume it should be a very rare occurrence, given that imperial units are basically unused outside of North America and the UK, but raising this just to double check.

[AlbertLei]

It's very rare, if not impossible, for “double primes” to mean “inch” in Chinese context. At least I have never seen that usage in my life time.

In a Chinese context, if I see 5’9” without further information, my first assumption is that it is for measuring time (5m9s) or arc degrees.
That said, both uses are relatively uncommon outside of specific contexts like middle school math or sports timing.

[AlbertLei]

A possibly related problem concerns the markdown writer with smart extension:

echo a“b”c | pandoc -t native
# [ Para [ Str "a\8220b\8221c" ] ]   # the markdown reader is okay

echo a“b”c | pandoc -t markdown
# a"b"c   # the markdown writer is not okay

The output a"b"c seems problematic: it loses the curly quotes and it cannot be converted back to a“b”c:

echo a“b”c | pandoc -t markdown | pandoc -t html
# <p>a”b”c</p>  # <= note that both quotes are closing quotes

a“b”c may be a rare case for English writers.
But, as per previous posts, it's how chinese writers use curly quotes.