Steel Bank Common Lisp version 2.6.7

Namely, that "SB" stands for "Sanely Bootstrappable" - something that the CMU Common Lisp was unable to do

This is the first news I've seen on HN in weeks that I am genuinely excited about! I have several AVX-2 hobby projects in Common Lisp, and an AVX-512 machine. It's an unexpected surprise to read this morning that this very useful ISA is suddenly unlocked. I'll be trying it out right away.

(edit: Looks like they mean *compiler* support for AVX-512, but not SB-SIMD definitions (yet). So I believe the only way for end-users to call AVX-512 instructions right now is to write custom VOP's).

> "Or are these intrinsics you have to explicitly ask for?"

There are language SIMD types and you explicitly use SIMD functions that operate on them. I believe it's essentially the same idea as C intrinsics. You can for example write (interactively)

    (u32.8+ (make-u32.8 0 1 2 3 4 5 6 7)
            (u32.8 10))

    ;; => #<SB-EXT:SIMD-PACK-256   10   11   12   13   14   15   16   17>

    (loop with sum = (f32.8 0.0f0)
          for index below (* 8 (floor length 8)) by 8
          do (setq sum (f32.8+ (f32.8-aref array index)
                               sum))
          finally (return sum))

    VMOVUPS YMM1, [RDX+RCX*2+1]
    VADDPS YMM0, YMM1, YMM0

https://github.com/sbcl/sbcl/tree/master/contrib/sb-simd

On implementation level, it is a codegen layer. It uses a system of macros to generate instructions from a database of instructions. The database is specified manually:

https://github.com/sbcl/sbcl/tree/master/contrib/sb-simd/cod...

At compile-time, they are converted into "VOPs", i.e. intrinsic functions, which are used by the compiler to emit the actual machine instructions.

> can it auto-vectorize or anything like that?

Unfortunately, it can't.

> are these intrinsics you have to explicitly ask for?

Yes, more like higher-level intrinsics. You get quite some automation, but you are requesting manually what you need. More like a DSL, than pure intrinsics. This is how you can use it (as an example):

(defun count-lines-and-words-ascii (sap size ws-init-state) (declare (type fixnum size) (type (unsigned-byte 8) ws-init-state) (type sb-sys:system-area-pointer sap) (optimize (speed 3) (safety 0))) (loop with loop-end of-type fixnum = (logandc2 size 127) for i of-type fixnum from 0 below loop-end by 128

    with 0x0  of-type u8.32 = (u8.32 #x00)
    with 0x20 of-type u8.32 = (u8.32 #x20)
    with 0x0A of-type u8.32 = (u8.32 #x0A)

    with wa of-type u64.4 = (u64.4 0)
    with la of-type u64.4 = (u64.4 0)

    with ws-prev of-type u8.32 = (u8.32 ws-init-state)

    for c1 = (u8.32-sap-ref sap (+ i 0))
    for c2 = (u8.32-sap-ref sap (+ i 32))
    for c3 = (u8.32-sap-ref sap (+ i 64))
    for c4 = (u8.32-sap-ref sap (+ i 96))

    do
       (flet ((process-chunk (curr prev)
                (let* ((ctrl  (u8.32-sat- (u8.32- curr 9) 4))
                       (ws (u8.32-or (u8.32= ctrl 0x0) (u8.32= curr 0x20)))

                       (ws-shift (u8.32-alignr ws (u8.32-permute128 prev ws #x21) 15))
                       (wmask    (u8.32-andc1 ws ws-shift))

                       (lmask (u8.32= curr 0x0A)))
                  (values wmask lmask ws))))

         (multiple-value-bind (wm lm prev) (process-chunk c1 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c2 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c3 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c4 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev)))
    finally
       (return
         (loop for j from loop-end below size

               with words of-type fixnum    = (sum-lanes wa)
               with lines of-type fixnum    = (sum-lanes la)
               with prev-ws of-type boolean = (logbitp 31 (u8.32-movemask ws-prev))
               with tlines of-type fixnum   = 0
               with twords of-type fixnum   = 0
               for byte of-type fixnum      = (sb-sys:sap-ref-8 sap j)
               for curr-ws of-type boolean  = (or (= byte 32) (<= 9 byte 13))

               do
                  (when (= byte 10) (incf tlines))
                  (when (and (not curr-ws) prev-ws) (incf twords))
                  (setf prev-ws curr-ws)
               finally
                  (return (values (the fixnum (+ lines tlines))
                                  (the fixnum (+ words twords))
                                  nil))))))

- use SB-VM:NEW-ARENA to make a new arena

- use SB-VM:WITH-ARENA to redirect ordinary allocation into an existing arena like you would use WITH-OPEN-FILE or similar macros

The only real doc is this internals note, and it doesn't even cover NEW-ARENA which I guess is left as an exercise to the reader: https://github.com/sbcl/sbcl/blob/master/doc/internals-notes...

Personally I think/hope they would have earlier discoveries / more broad usage of the deterministic systems like nix etc, which are built upon functional principles of immutability etc.

Maybe the world would be guix/Hurd!

Somewhere, I have a copy of a commercial LISP for Windows which would compile to an executable --- apparently this sort of thing is still possible, but it's not widely known/used, and sadly Jean-Marie Hullot's "SOS Interface" for the Mac was co-opted to NeXTstep:

https://denninginstitute.com/itcore/userinterface/GUIHistory...

I'd dearly love to see a RAD (Rapid Application Development) tool using LISP w/ a nifty UI for working up a GUI which would compile to something easily deployed (maybe HTML5 Canvas and JavaScript) as a stand-alone/single-file web application?

We have a few services built in Clojure and we expose nrepl port on our pods in our SDEs, it's enormously helpful to test and debug things on the fly, without having to redeploy or restart anything. Without having to deal with state, caching, etc.

I imagine it would look a bit like Erlang's BEAM. No need to stop and start the application, but write a script to do hot updates of a live image.

https://ngnghm.github.io/

AWS already has lots of AMI and docker are essentially image based.

We would version whole images in something like git, and that's all.

I'm strongly convinced, having used Scheme, CL, Racket, Clojure that Lisp is doomed to be (mostly) a hobby language.

The very power of Lisp, macros, dynamic programming, reflection lead to a mess of an ecosystem where every single developer reinvents the wheel and nobody can understand yet another DSL invented by the next developer next to them.

Racket's theme of being a "programming language for building programming languages" is just the poster child of this naivety: I don't want even more friction.

What scales and works is simple and boring.

That's by the way why also Haskell has struggled forever. Beyond its dreadful DX, poor tooling and unacceptable compilation times, the language is just plagued by compiler extensions and every single developer reinventing its own abstractions.

There was the simple haskell movement to just standardize around a set of extensions and conventions, but nope, these languages unavoidably attract people that want to stay in the ivory tower.

Really, I love both lisps and haskell, but I would take a dreadful php over them every single day at work. No contest.

CCL isn't very actively maintained and currently doesn't have an ARM64 port, but otherwise continues to work fine. I believe one reason people use it is that it compiles a bit faster than SBCL, at least in part by doing less optimization.

I wanted something more 'algorithmic' - and more accurate.

So me and Claude build an sbcl-based Film Recommendation system. Type in a new film name, it goes to open film database OMDB, grabs the scores, and then, using films I have already rated and with a built-in tiny Neural Net, gives me a personal recommendation. It uses the OMDB data and an algo that weights those with my personal values (in half a dozen areas: overall, acting, cinematography, etc), along with a 'comments' box to note for friends / others.

The code is very well-done CL, heavily commented. The film & ratings database is stored as S-exprs, naturally, all in one file. I don't use a database as I only have a few hundred films, but maybe later I will. And that's the point -- this is a living chunk of code that I from time to time bolt in new features, or try new things (e.g. the UI is localhost:8080)

I fretted about having a 'real' project to really dig into CL for a long time, and finally, found a meaty-enough project that ends up being really useful & quite a learning (continuing learning) experience. I start it up inside emacs/sly like this:

  (ql:quickload '(:hunchentoot :dexador :yason))
  (load "s:/filmrec.lisp")
  (filmrec:start)

  (setf filmrec:*omdb-api-key* "fxxxx70")

  (filmrec:retrain)