Title : Common LISP awk macro for easy text file operations
Author: Solène
Date  : 04 February 2020
Tags  : awk lisp

I like Common LISP and I also like awk. Dealing with text files in Common LISP
is often painful. So I wrote a small awk like common lisp macro, which helps a
lot dealing with text files.

Here is the implementation, I used the uiop package for split-string function,
it comes with sbcl. But it's possible to write your own split-string or reused
the infamous split-str function shared on the Internet.


    (defmacro awk(file separator &body code)
      "allow running code for each line of a text file,
       giving access to NF and NR variables, and also to
       fields list containing fields, and line containing $0"
        `(progn
           (let ((stream (open ,file :if-does-not-exist nil)))
             (when stream
               (loop for line = (read-line stream nil)
                  counting t into NR
                  while line do
                    (let* ((fields (uiop:split-string line :separator ,separator))
                           (NF (length fields)))
                      ,@code))))))


It's interesting that the "do" in the loop could be replaced with a "collect",
allowing to reuse awk output as a list into another function, a quick example I
have in mind is this:

    ;; equivalent of awk '{ print NF }' file | sort | uniq
    ;; for counting how many differents fields long line we have
    (uniq (sort (awk "file" " " NF)))


Now, here are a few examples of usage of this macro, I've written the original
awk command in the comments in comparison:

    ;; numbering lines of a text file with NR
    ;; awk '{ print NR": "$0 }' file.txt
    ;;
    (awk "file.txt" " "
         (format t "~a: ~a~%" NR line))
    
    ;; display NF-1 field (yes it's -2 in the example because -1 is last field in the list)
    ;; awk -F ';' '{ print NF-1 }' file.csv
    ;;
    (awk "file.csv" ";"
         (print (nth (- NF 2) fields)))
    
    ;; filtering lines (like grep)
    ;; awk '/unbound/ { print }' /var/log/messages
    ;;
    (awk "/var/log/messages" " "
         (when (search "unbound" line)
           (print line)))
    
    ;; printing 4nth field
    ;; awk -F ';' '{ print $4 }' data.csv
    ;;
    (awk "data.csv" ";"
         (print (nth 4 fields)))