1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- Created by GNU Texinfo 6.8, https://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<!-- This is the GNU Emacs Lisp Reference Manual
corresponding to Emacs version 29.0.50.
Copyright © 1990-1996, 1998-2023 Free Software Foundation, Inc.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with the
Invariant Sections being "GNU General Public License," with the
Front-Cover Texts being "A GNU Manual," and with the Back-Cover
Texts as in (a) below. A copy of the license is included in the
section entitled "GNU Free Documentation License."
(a) The FSF's Back-Cover Text is: "You have the freedom to copy and
modify this GNU manual. Buying copies from the FSF supports it in
developing GNU and promoting software freedom." -->
<title>Parser-based Font Lock (GNU Emacs Lisp Reference Manual)</title>
<meta name="description" content="Parser-based Font Lock (GNU Emacs Lisp Reference Manual)">
<meta name="keywords" content="Parser-based Font Lock (GNU Emacs Lisp Reference Manual)">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link href="index.html" rel="start" title="Top">
<link href="Index.html" rel="index" title="Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Font-Lock-Mode.html" rel="up" title="Font Lock Mode">
<link href="Multiline-Font-Lock.html" rel="prev" title="Multiline Font Lock">
<style type="text/css">
<!--
a.copiable-anchor {visibility: hidden; text-decoration: none; line-height: 0em}
a.summary-letter {text-decoration: none}
blockquote.indentedblock {margin-right: 0em}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
kbd {font-style: oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
span.nolinebreak {white-space: nowrap}
span.roman {font-family: initial; font-weight: normal}
span.sansserif {font-family: sans-serif; font-weight: normal}
span:hover a.copiable-anchor {visibility: visible}
ul.no-bullet {list-style: none}
-->
</style>
<link rel="stylesheet" type="text/css" href="./manual.css">
</head>
<body lang="en">
<div class="subsection" id="Parser_002dbased-Font-Lock">
<div class="header">
<p>
Previous: <a href="Multiline-Font-Lock.html" accesskey="p" rel="prev">Multiline Font Lock Constructs</a>, Up: <a href="Font-Lock-Mode.html" accesskey="u" rel="up">Font Lock Mode</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<span id="Parser_002dbased-Font-Lock-1"></span><h4 class="subsection">24.6.10 Parser-based Font Lock</h4>
<span id="index-parser_002dbased-font_002dlock"></span>
<p>Besides simple syntactic font lock and regexp-based font lock, Emacs
also provides complete syntactic font lock with the help of a parser.
Currently, Emacs uses the tree-sitter library (see <a href="Parsing-Program-Source.html">Parsing Program Source</a>) for this purpose.
</p>
<p>Parser-based font lock and other font lock mechanisms are not mutually
exclusive. By default, if enabled, parser-based font lock runs first,
replacing syntactic font lock, then the regexp-based font lock.
</p>
<p>Although parser-based font lock doesn’t share the same customization
variables with regexp-based font lock, it uses similar customization
schemes. The tree-sitter counterpart of <var>font-lock-keywords</var> is
<var>treesit-font-lock-settings</var>.
</p>
<span id="index-tree_002dsitter-fontifications_002c-overview"></span>
<span id="index-fontifications-with-tree_002dsitter_002c-overview"></span>
<p>In general, tree-sitter fontification works as follows:
</p>
<ul>
<li> A Lisp program (usually, part of a major mode) provides a <em>query</em>
consisting of <em>patterns</em>, each pattern associated with a
<em>capture name</em>.
</li><li> The tree-sitter library finds the nodes in the parse tree
that match these patterns, tags the nodes with the corresponding
capture names, and returns them to the Lisp program.
</li><li> The Lisp program uses the returned nodes to highlight the portions of
buffer text corresponding to each node as appropriate, using the
tagged capture names of the nodes to determine the correct
fontification. For example, a node tagged <code>font-lock-keyword</code>
would be highlighted in <code>font-lock-keyword</code> face.
</li></ul>
<p>For more information about queries, patterns, and capture names, see
<a href="Pattern-Matching.html">Pattern Matching Tree-sitter Nodes</a>.
</p>
<p>To setup tree-sitter fontification, a major mode should first set
<code>treesit-font-lock-settings</code> with the output of
<code>treesit-font-lock-rules</code>, then call
<code>treesit-major-mode-setup</code>.
</p>
<dl class="def">
<dt id="index-treesit_002dfont_002dlock_002drules"><span class="category">Function: </span><span><strong>treesit-font-lock-rules</strong> <em>&rest query-specs</em><a href='#index-treesit_002dfont_002dlock_002drules' class='copiable-anchor'> ¶</a></span></dt>
<dd><p>This function is used to set <var>treesit-font-lock-settings</var>. It
takes care of compiling queries and other post-processing, and outputs
a value that <var>treesit-font-lock-settings</var> accepts. Here’s an
example:
</p>
<div class="example">
<pre class="example">(treesit-font-lock-rules
:language 'javascript
:feature 'constant
:override t
'((true) @font-lock-constant-face
(false) @font-lock-constant-face)
:language 'html
:feature 'script
"(script_element) @font-lock-builtin-face")
</pre></div>
<p>This function takes a series of <var>query-spec</var>s, where each
<var>query-spec</var> is a <var>query</var> preceded by one or more
<var>:keyword</var>/<var>value</var> pairs. Each <var>query</var> is a
tree-sitter query in either the string, s-expression or compiled form.
</p>
<p>For each <var>query</var>, the <var>:keyword</var>/<var>value</var> pairs that
precede it add meta information to it. The <code>:language</code> keyword
declares <var>query</var>’s language. The <code>:feature</code> keyword sets the
feature name of <var>query</var>. Users can control which features are
enabled with <code>font-lock-maximum-decoration</code> and
<code>treesit-font-lock-feature-list</code> (described below). These two
keywords are mandatory.
</p>
<p>Other keywords are optional:
</p>
<table>
<thead><tr><th width="15%">Keyword</th><th width="15%">Value</th><th width="60%">Description</th></tr></thead>
<tr><td width="15%"><code>:override</code></td><td width="15%">nil</td><td width="60%">If the region already has a face, discard the new face</td></tr>
<tr><td width="15%"></td><td width="15%">t</td><td width="60%">Always apply the new face</td></tr>
<tr><td width="15%"></td><td width="15%"><code>append</code></td><td width="60%">Append the new face to existing ones</td></tr>
<tr><td width="15%"></td><td width="15%"><code>prepend</code></td><td width="60%">Prepend the new face to existing ones</td></tr>
<tr><td width="15%"></td><td width="15%"><code>keep</code></td><td width="60%">Fill-in regions without an existing face</td></tr>
</table>
<p>Lisp programs mark patterns in <var>query</var> with capture names (names
that starts with <code>@</code>), and tree-sitter will return matched nodes
tagged with those same capture names. For the purpose of
fontification, capture names in <var>query</var> should be face names like
<code>font-lock-keyword-face</code>. The captured node will be fontified
with that face.
</p>
<span id="index-treesit_002dfontify_002dwith_002doverride"></span>
<p>Capture names can also be function names, in which case the function
is called with 4 arguments: <var>node</var> and <var>override</var>, <var>start</var>
and <var>end</var>, where <var>node</var> is the node itself, <var>override</var> is
the override property of the rule which captured this node, and
<var>start</var> and <var>end</var> limits the region in which this function
should fontify. (If this function wants to respect the <var>override</var>
argument, it can use <code>treesit-fontify-with-override</code>.)
</p>
<p>Beyond the 4 arguments presented, this function should accept more
arguments as optional arguments for future extensibility.
</p>
<p>If a capture name is both a face and a function, the face takes
priority. If a capture name is neither a face nor a function, it is
ignored.
</p></dd></dl>
<dl class="def">
<dt id="index-treesit_002dfont_002dlock_002dfeature_002dlist"><span class="category">Variable: </span><span><strong>treesit-font-lock-feature-list</strong><a href='#index-treesit_002dfont_002dlock_002dfeature_002dlist' class='copiable-anchor'> ¶</a></span></dt>
<dd><p>This is a list of lists of feature symbols. Each element of the list
is a list that represents a decoration level.
<code>font-lock-maximum-decoration</code> controls which levels are
activated.
</p>
<p>Each element of the list is a list of the form <code>(<var>feature</var> …)</code><!-- /@w -->, where each <var>feature</var> corresponds to the
<code>:feature</code> value of a query defined in
<code>treesit-font-lock-rules</code>. Removing a feature symbol from this
list disables the corresponding query during font-lock.
</p>
<p>Common feature names, for many programming languages, include
<code>definition</code>, <code>type</code>, <code>assignment</code>, <code>builtin</code>,
<code>constant</code>, <code>keyword</code>, <code>string-interpolation</code>,
<code>comment</code>, <code>doc</code>, <code>string</code>, <code>operator</code>,
<code>preprocessor</code>, <code>escape-sequence</code>, and <code>key</code>. Major
modes are free to subdivide or extend these common features.
</p>
<p>Some of these features warrant some explanation: <code>definition</code>
highlights whatever is being defined, e.g., the function name in a
function definition, the struct name in a struct definition, the
variable name in a variable definition; <code>assignment</code> highlights
the whatever is being assigned to, e.g., the variable or field in an
assignment statement; <code>key</code> highlights keys in key-value pairs,
e.g., keys in a JSON object, or a Python dictionary; <code>doc</code>
highlights docstrings or doc-comments.
</p>
<p>For example, the value of this variable could be:
</p><div class="example">
<pre class="example">((comment string doc) ; level 1
(function-name keyword type builtin constant) ; level 2
(variable-name string-interpolation key)) ; level 3
</pre></div>
<p>Major modes should set this variable before calling
<code>treesit-major-mode-setup</code>.
</p>
<span id="index-treesit_002dfont_002dlock_002drecompute_002dfeatures"></span>
<p>For this variable to take effect, a Lisp program should call
<code>treesit-font-lock-recompute-features</code> (which resets
<code>treesit-font-lock-settings</code> accordingly), or
<code>treesit-major-mode-setup</code> (which calls
<code>treesit-font-lock-recompute-features</code>).
</p></dd></dl>
<dl class="def">
<dt id="index-treesit_002dfont_002dlock_002dsettings"><span class="category">Variable: </span><span><strong>treesit-font-lock-settings</strong><a href='#index-treesit_002dfont_002dlock_002dsettings' class='copiable-anchor'> ¶</a></span></dt>
<dd><p>A list of settings for tree-sitter based font lock. The exact format
of each setting is considered internal. One should always use
<code>treesit-font-lock-rules</code> to set this variable.
</p>
</dd></dl>
<p>Multi-language major modes should provide range functions in
<code>treesit-range-functions</code>, and Emacs will set the ranges
accordingly before fontifing a region (see <a href="Multiple-Languages.html">Parsing Text in Multiple Languages</a>).
</p>
</div>
<hr>
<div class="header">
<p>
Previous: <a href="Multiline-Font-Lock.html">Multiline Font Lock Constructs</a>, Up: <a href="Font-Lock-Mode.html">Font Lock Mode</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html" title="Index" rel="index">Index</a>]</p>
</div>
</body>
</html>
|
