support indented heredocs, add new "Reader macros" manual section

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

4 files changed, 81 insertions, 6 deletions

diff --git a/doc/GNUmakefile b/doc/GNUmakefile
index 53d9b35..bf81053 100644
--- a/doc/GNUmakefile
+++ b/doc/GNUmakefile
@@ -13,6 +13,8 @@ all: html info
 html info: $(PAGES) conf.py $(wildcard *.rst */*.rst)
 	sphinx-build -M $@ . _build
 
+reader.rst:
+
 $(PAGES) &: $(wildcard *.rst.in */*.rst.in) $(LISP)
 	$(SBCL) --eval "(mapc #'consfigurator::build-manual-rst \
 		              uiop:*command-line-arguments*)" --quit $(PAGES)
diff --git a/doc/index.rst b/doc/index.rst
index 2592b00..3412fe8 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -30,6 +30,7 @@ Consfigurator user's manual
    deployment
    data
    image
+   reader
 
 .. toctree::
    :maxdepth: 1
diff --git a/doc/news.rst b/doc/news.rst
index bc45564..da40c27 100644
--- a/doc/news.rst
+++ b/doc/news.rst
@@ -23,17 +23,32 @@ In summary, you should always be able to upgrade to a release which only
 increments ``patch``, but if either of the other two components have changed,
 you should review this document and see if your consfig needs updating.
 
-1.2.4 (unreleased)
+1.3.0 (unreleased)
 ------------------
 
 - New reader macro ``#>>EOF>>`` which is like ``#>EOF>`` except that it skips
   over the remainder of the current line and its newline.  This is more like
-  how heredocs work in other languages.  For the sake of future extension, the
-  remainder of the line after the ``#>>EOF>>`` should not contain anything
-  other than a single-line comment.
+  how heredocs work in other languages.
 
-  (This is not a breaking change because the existing implementation for
-  ``#>EOF>`` does not permit using terminators beginning with ``>``.)
+- Support for indented heredocs, where the indentation of the lines of the
+  heredoc is stripped.  This mode is activated by prefixing a tilde to the
+  heredoc terminator.  For example:
+
+  .. code-block:: none
+
+    (foo "argument 1" #>>~EOF>>
+	 My line 1.
+	 My line 2.
+	 EOF)
+
+  The function receives ``"My line 1.\nMy line 2."``
+
+  This is a minor breaking change because heredoc terminators may no longer
+  begin with a tilde.
+
+- New manual section "Reader macros" discussing Consfigurator's named
+  readtable, including some usage reservations for the sake of future
+  extension.
 
 - New tutorial, "Defining new properties".
 
diff --git a/doc/reader.rst b/doc/reader.rst
new file mode 100644
index 0000000..e6f9c3a
--- /dev/null
+++ b/doc/reader.rst
@@ -0,0 +1,57 @@
+Reader macros
+=============
+
+Loading Consfigurator defines the ``:CONSFIGURATOR`` named readtable.  Its
+original purpose was to define a few reader macros to make Lisp more readily
+usable for Unix systems administration, and as such it's helpful in consfigs.
+We now have the broader aim of providing a readtable that renders Lisp more
+useful for general Unix-style text manipulation.  To this end, the reader
+macros we define are all inspired by Perl syntax.
+
+Backwards compatibility
+-----------------------
+
+We don't expect to make incompatible changes to how these reader macros work,
+except to make them work more like their Perl equivalents.  With this in mind,
+some particular reservations are made for particular macros, as detailed below.
+
+``#?``: Regexps & interpolation
+-------------------------------
+
+Sharp-question mark is the well-known CL-INTERPOL_ reader macro.
+
+.. _CL-INTERPOL: https://edicl.github.io/cl-interpol/
+
+``#>EOF>`` and ``#>>EOF>>``: Heredocs
+-------------------------------------
+
+Following ``#>EOF>``, all characters are read into a string until the next
+literal ``EOF``.  You may use any string in place of ``EOF``, except that it
+must not begin with a tilde or contain any whitespace, and for the sake of
+future extension, it must not begin with a backwards slash or begin or end
+with single or double quotation marks.
+
+You can double up the ``>``, as in ``#>>EOF>>``, to skip the remainder of the
+line on which the ``#>>EOF>>`` appears, starting the heredoc at the beginning
+of the following line.  For the sake of future extension, the remainder of the
+line after the ``#>>EOF>>`` must not contain anything other than a single-line
+comment.
+
+The specification of the terminating string may be preceded by a tilde, as in
+``#>>~EOF>>``, to mean an indented heredoc:
+
+.. code-block:: none
+
+  (foo "argument 1" #>>~EOF>>
+       My line 1.
+       My line 2.
+       EOF)
+
+The function receives ``"My line 1.\nMy line 2.\n"``.
+
+See also
+--------
+
+- `perlop(1) <https://perldoc.perl.org/perlop>`_
+
+- `inferior-shell <https://cliki.net/inferior-shell>`_