Fast Command Line Syntax Highlighting with clp

clp's source code and installation instructions are available on Github and Sourcehut. It’s a work in progress hobby project, so expect bugs and quirks.

clp is a program that writes files to stdout with syntax highlighting. I built it to quickly fill fzf's preview window.

It's similar to bat but is simpler, faster, and easier to add languages that don't have existing Sublime syntax definitions.

When the --preview option is set in fzf, the preview command is kicked off in an external process and the results are displayed in the preview window, which in this case is clp.

fzf --preview 'clp {}'

Here's what we see when we execute the command:

clp obtains syntax highlighting information from Parsing Expressing Grammar (PEG) lexers provided by Scintillua.

The implementation is very simple; here's how clp works:

• shepherd input files to their respective lexer based on checks for filetype extension, MIME type, and more (thanks to Vis contributor Ian Nomar, who was hugely helpful in ironing out kinks in this project)
• lex the input file and produce a list of tokens
• iterate over the list of tokens:
  • for each token, it look up an ANSI color escape sequence corresponding to each token type, write it to stdout, and then write out the text that yielded the token to stdout

I hacked in an option to highlight line a line as well, since that's nice for file previewing.

Here's a performance comparison to similar programs taken from my machine, measuring the time it takes to highlight the sqlite3 amalgamation using hyperfine:

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
clp sqlite3.c	212.7 ± 0.9	211.5	214.7	1.00
bat --color=always sqlite3.c	3178.0 ± 109.6	3128.6	3488.7	14.94 ± 0.52
source-highlight sqlite3.c	4276.2 ± 12.6	4259.6	4297.7	20.10 ± 0.11

It's a pretty nice performance increase. It can also highlight around 150 programming languages, markup languages, input files, configuration files, and types of program output.

clp is currently limited to 16 colors; since the program itself can run on any system with a C99 compiler, I figured I'd keep its default configuration very basic so it can run in any 16 color terminal emulator out of the box.

Down the line I would like to make clp more easily theme-able and add some built-in themes. I'd also like to improve the aesthetics of line highlighting.

When I started the project, I had very little knowledge of Lua, Autotools, and C. Working on it was a very fun and satisfying way to get more familiar with all of them.

Overall, I'm happy with how clp is coming along and am excited to add more. It's fast, supports a ton of languages, is easily extendable, and very portable to boot.

Contributions are welcome, and if you'd like to add a lexer, you should submit it upstream to Scintillua. If it isn't accepted there, I will merge pretty much any lexer that does not try to mine for Bitcoin.

clp's source code and installation instructions are available on Github and Sourcehut

Update 8/20

Adding color themes

I've added the ability to change clp's colors by selecting different built-in color themes or creating your own.

To change your theme, create a configuration file in ~/.config/clp/clprc.lua to override the system default. It should look like this:

clprc = {}
clprc.theme = "ansi-16"
return clprc

The value of clprc.theme is the name of a colorscheme that clp will later require. The following themes are available out of the box:

3-bit ANSI theme

• ansi-16

Truecolor themes

• material
• codedark
• github_dark
• github_light
• gruvbox
• tokyonight
• sonokai
• everforest

The ANSI theme is enabled by default; it uses 16 color ANSI codes and should work on both light or dark terminals. The truecolors themes were exported from vim using export-colorscheme.nvim, which is a little neovim plugin I wrote recently.

You can create your own themes in ~/.config/clp/themes/. Like with clprc.lua, these should be Lua modules. Here's the definition of ansi-16:

local M = {}
M.theme = {
	['comment'] = {color="cyan"},
	['keyword'] = {color="blue"},
	['number'] = {color="red"},
	['string'] = {color="green"},
	['preprocessor'] = {color="magenta"},
	['type'] = {color="yellow"},
}
return M

The table keys on the left correspond to lexical tokens picked up by the lexer, and the right specifies the color assigned to that token when the text is written to stdout.

When the right hand value is a Lua object like above, clp will use 16 color ANSI codes to highlight the token.

To use truecolor, the right hand value should be a string in the form "#RRGGBB" using hex values, like here:

['comment'] = "#6a9955",

This will cause clp to emit 24 bit color codes upon encountering the token. This means it will only render if the terminal being used supports truecolor.

Throw a bunch of these definitions in a lua file in ~/.config/clp/themes/, then set clprc.theme to the name of the file (not including the file extension), and you should be good to go!

Acknowledgements

Many thanks to:

• Marc André Tanner & contributors for Vis
• Rich Felker for musl libc's awesome configure script and makefile
• orbitalquark for Scintillua
• Ian Nomar and Christopher Wellons for contributions & suggestions