Improve word wrapping for CJK characters

Note about the change around line 9257 and 23372:

Before, the test for whitespace checks for can_wrap_before and
can_wrap_after simutaniously.  Now we separate these two checks, and
the logic needs to change a little bit.  However, when we don't enable
the new wrapping feature, 'can_wrap_after' is equivalent to
'IT_DISPLAYING_WHITESPACE' and 'can_wrap_before' is equivalent to
'!IT_DISPLAYING_WHITESPACE'.  And the new logic is equivalent with the
old one in that case.

Old logic:

    if (whitespace) /* Which means can wrap after && can't wrap
                       before.  */
      may_wrap = true;

    else if (may_wrap) /* aka (!whitespace && may_wrap)
      (set wrap point)  * aka (can't wrap after && can wrap before
      may_wrap = false  *      && may_wrap)
                        */

New logic:

    if (can_wrap_after)
      next_may_wrap = true
    else
      next_may_wrap = false;

    if (may_wrap && can_wrap_before)
      (set wrap point)

    /* Update may_wrap.  */
    may_wrap = next_may_wrap;

* src/xdisp.c (it_char_has_category, char_can_wrap_before)
(char_can_wrap_after): New functions.
(move_it_in_display_line_to, display_line): Replace calls to
'IT_DISPLAYING_WHITESPACE' with either 'char_can_wrap_before' or
'char_can_wrap_after'.
(word-wrap-by-category): New variable.

* lisp/cus-start.el (minibuffer-prompt-properties--setter): Add
'word-wrap-by-category' as a customizable variable.

* doc/emacs/display.texi (Visual Line Mode): Add a paragraph about the
new 'word-wrap-by-category' feature.
* etc/NEWS: Announce the change.

1 files changed, 21 insertions, 0 deletions

diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
index 75ef520d62a..d56a135b640 100644
--- a/doc/emacs/display.texi
+++ b/doc/emacs/display.texi
@@ -1808,6 +1808,27 @@ logical lines, so having a fringe indicator for each wrapped line
 would be visually distracting.  You can change this by customizing the
 variable @code{visual-line-fringe-indicators}.
 
+@vindex word-wrap-by-category
+@findex modify-category-entry
+@findex char-category-set
+@findex category-set-mnemonics
+  By default, Emacs only breaks lines after whitespace characters.
+That strategy produces bad results when CJK and Latin text are mixed
+together (because CJK characters don't use whitespace to separate
+words).  You can customize @code{word-wrap-by-category} to allow Emacs
+to break lines after any character with ``|'' category
+(@pxref{Categories,,, elisp, the Emacs Lisp Reference Manual}), which
+includes CJK characters.  Also, if this variable is set using
+Customize, Emacs automatically loads kinsoku.el.  When kinsoku.el is
+loaded, Emacs respects kinsoku rules when breaking lines.  That means
+characters with the ``>'' category don't appear at the beginning of a
+line (e.g., FULLWIDTH COMMA), and characters with the ``<'' category
+don't appear at the end of a line (e.g., LEFT DOUBLE ANGLE BRACKET).
+You can view the categories of a character by @code{char-category-set}
+and @code{category-set-mnemonics}, or type @kbd{C-u C-x =} with point
+on the character and look at the ``category'' section in the report.
+You can add categories to a character by @code{modify-category-entry}.
+
 @node Display Custom
 @section Customization of Display