Quicklisp implementation stats for 2017+

Here are some raw HTTP request stats for each CL implementation supported by Quicklisp from 2017-01-01 onward. Each request corresponds roughly to downloading a single project.
  • SBCL: 4,518,774 requests
  • Clozure CL: 488,057 
  • CLISP: 34,767
  • LispWorks: 25,700
  • ABCL: 23,913 
  • Allegro: 19,501
  • ECL: 19,027
  • CLASP: 3,335
  • CMUCL: 965
  • MKCL: 79
  • Scieneer: 0
I gathered this info to check Scieneer activity levels to plan future support. Turns out nobody with Scieneer has used Quicklisp since 2013. Bummer!


Download stats for January, 2018

Here are the raw Quicklisp download stats for January 2018.
20452  alexandria
17822  closer-mop
16236  cl-ppcre
15928  split-sequence
15762  babel
15131  iterate
14929  anaphora
14859  trivial-features
14580  let-plus
14136  trivial-gray-streams
13668  bordeaux-threads
12524  cffi
12215  trivial-garbage
11815  flexi-streams
11383  puri
11201  more-conditions
11063  nibbles
10266  usocket
 9545  utilities.print-items
 9499  trivial-backtrace
 9312  esrap
 9170  cl+ssl
 8940  chunga
 8812  cl-base64
 8446  chipz
 8417  cl-fad
 8107  drakma
 7212  cl-yacc
 6937  named-readtables
 6929  asdf-flv
 6647  fiveam
 6429  local-time
 6179  ironclad
 6103  parse-number
 5850  closure-common
 5844  cxml
 5743  log4cl
 5229  architecture.hooks
 5037  cl-json
 5023  parser.common-rules
 4773  plexippus-xpath
 4530  optima
 4323  lift
 4028  lparallel
 4002  cl-clon
 3882  cl-dot
 3594  cxml-stp
 3469  cl-interpol
 3421  xml.location
 3399  cl-unicode
 3363  utilities.print-tree
 3350  cl-store
 3311  fare-utils
 3305  fare-quasiquote
 3135  slime
 3109  inferior-shell
 3098  fare-mop
 2810  metabang-bind
 2557  trivial-utf-8
 2556  trivial-types
 2544  cl-utilities
 2288  uuid
 2244  cl-annot
 2234  cl-syntax
 2232  quri
 2128  trivial-indent
 2118  documentation-utils
 2035  cl-slice
 2027  plump
 2023  array-utils
 2013  gettext
 1998  static-vectors
 1983  symbol-munger
 1966  collectors
 1933  arnesi
 1932  md5
 1920  access
 1915  djula
 1898  cl-locale
 1893  cl-parser-combinators
 1833  fast-io
 1822  simple-date-time
 1804  hunchentoot
 1567  ieee-floats
 1537  yason
 1364  prove
 1312  asdf-system-connections
 1307  metatilities-base
 1304  cl-containers
 1192  osicat
 1138  monkeylib-binary-data
 1115  rfc2388
 1041  trivial-shell
  993  diff
  989  postmodern
  961  cl-custom-hash-table
  929  parse-float
  912  salza2
  882  cl-sqlite
  872  trivial-mimes


January 2018 Quicklisp dist update now available

New projects: 
  • bodge-blobs-support — Common utilities for loading/distributing foreign libraries — The Unlicense
  • bodge-chipmunk — Wrapper over chipmunk 2d physics library — MIT
  • chipmunk-blob — Chipmunk physics foreign library collection — MIT
  • cl-dct — Discrete cosine transform. — Apache-2.0
  • cl-grnm — Grid Restrained Nelder-Mead, a multivariate rootfinder. — MIT (See LICENSE.txt)
  • cl-skkserv — skkserv with Common Lisp — GPLv3
  • claw — Import c2ffi specs and generate CFFI wrappers — BSD-2-Clause
  • clinenoise — A trivial line-input library for VT-like terminals — BSD-2
  • clweb — CLWEB is a literate programming system for Common Lisp — Unspecified
  • dbd-oracle — ORACLE database driver for CL-DBI. — Lessor Lisp General Public License
  • dufy — Color library for Common Lisp — MIT
  • glsl-packing — calculate std140/std430 layout for a glsl UBO/SSBO — MIT
  • shadow — A lightweight system to help with defining and managing OpenGL shader programs. — MIT
  • westbrook — An RSS feed generator. — BSD
Updated projects: 3bgl-shader3d-matrices3d-vectorsarray-utilsasd-generatorasdf-vizaws-sign4beastccldoccellsceplcepl.sdl2cepl.sdl2-ttfchancerychirpchungacl-anacl-ansi-termcl-cognitocl-conllucl-csvcl-cutcl-dbicl-digraphcl-diskspacecl-dotcl-editdistancecl-flaccl-fondcl-gamepadcl-gobject-introspectioncl-gpiocl-hamcrestcl-ledgercl-lzmacl-mixedcl-monitorscl-mpg123cl-oclapicl-online-learningcl-openglcl-out123cl-pcgcl-random-forestcl-readlinecl-riffcl-sandboxcl-smtpcl-spidevcl-strcl-string-matchcl-stringscl-svgcl-unicodecl-wavclipcloser-mopclssclx-cursorcodexcommonqtcroatoancrypto-shortcutsdartsclhashtreedeedsdeferreddefpackage-plusdeploydissectdjuladocumentation-utilseazy-projectesrapfare-scriptsfast-httpflareflowforform-fiddlegamebox-mathgeneric-comparabilityglsl-specglsl-toolkitgraphhalftoneharmonyhu.dwim.perechumblerhunchensocketinquisitoriteratejosejskenzolacklambda-fiddlelasslegitlichat-protocollisp-chatlisp-unit2local-timelquerymaidenmcclimmgl-paxmk-string-metricsmodularizemodularize-hooksmodularize-interfacesnamed-readtablesninevehnorthoverlordpapyrusparachuteparser.common-rulespathname-utilspipingplumpportable-threadspostmodernqlotqmyndqtoolsqtools-uirandom-stateratifyredirect-streamremote-jsrutilsscalplserapeumsimple-inferiorssimple-rgbsimple-tasksskippy-renderersnoozesoftdrinkspinneretstaplestumpwmthe-cost-of-nothingtreestrivial-argumentstrivial-benchmarktrivial-file-sizetrivial-indenttrivial-main-threadtrivial-mimestrivial-thumbnailtrivial-timeouttrivial-updatetrivial-wsubiquitousunix-optsvarjoverbosewebsocket-driverwith-c-syntaxwuweixml.locationyaclml.

Removed projects: blackthorn-engine, gamebox-grids, linedit, software-evolution, squirl, tbnl.

The latest removed projects are due to author request or author neglect - if a project you need has been removed, maybe a new maintenance arrangement can be worked out, get in touch!

To get this update, use (ql:update-dist "quicklisp").



Build failure RSS feeds

Every day, I build each project that Quicklisp tracks. If a project fails, I try to track down the author and let them know. If a project chronically fails and there's no fix in sight, I drop the project from Quicklisp.

In an effort to help authors follow build status, I've made a system of RSS feeds that are updated with daily build failures. The feeds have the following patterns:

Each feed entry has info about where Quicklisp gets the project, what systems are failing, and an excerpt from the build log and backtrace. There are links to the full build log. 

I hope you find these reports useful. If you have any suggestions or questions, feel free to get in touch.


The Quicklisp local-projects mechanism

Quicklisp provides a lot of software, but there’s also a simple way to load things that Quicklisp doesn’t provide. That same mechanism can be used to override libraries Quicklisp does provide.

The local projects mechanism sets up a special directory that is automatically scanned for software to load. Here are a few quick examples.

Trying a library not in Quicklisp

First, imagine that you just heard about a great new library and want to try it. However, it’s not available through Quicklisp yet, only through a git repository on https://example.com/fun-project.git. One easy way to try it:
$ cd ~/quicklisp/local-projects
$ git clone https://example.com/fun-project.git
After the git command completes, and there is a fun-project subdirectory with a fun-project/fun-project.asd file present, the system is visible to ASDF and can be loaded either with ql:quickload or asdf:find-system. When loaded through ql:quickload, Quicklisp will automatically fetch and load any prerequisites if needed.

Overriding a library in Quicklisp

Second, imagine that you want to hack on a library that Quicklisp already provides. You don’t want to load and hack on the version from Quicklisp - that software is not under version control, and just represents a snapshot of the project at a particular point in time.

Once again, the procedure is to put the software in the ~/quicklisp/local-projects/ directory:
$ cd ~/quicklisp/local-projects/
$ git clone https://github.com/xach/vecto.git
After the git command completes, (ql:quickload "vecto") will load the library from local-projects rather than from the standard Quicklisp release.

How it works

The local-projects mechanism is relatively automatic. Here’s how it works underneath, and how to fix problems that might crop up.

ASDF has an extensible mechansim (the asdf:*system-definition-search-functions* variable) for searching for system files. Quicklisp extends this mechanism with a function that does the following, all in the context of the local-projectsdirectory.
  1. If there is no file named system-index.txt, it is created by scanning the directory tree for system files (matching "*.asd"). Each pathname is added to the file.
  2. If the system-index.txt file exists, but its timestamp is older than its containing directory, the directory is rescanned and the index recreated.
  3. The system-index.txt is searched for any entry with a pathname-name that matches the desired system name. If there’s a match, matching pathname is probed. If it still exists, it is returned. If it has disappeared, the system-index.txt is recreated as in step 1 and the search is retried.
  4. Otherwise the system search is deferred to the remaining ASDF system search functions.
When there are multiple system files with the same name in the directory tree, the one with the shortest full pathname name is returned. In the case of a pathname length tie, the one that is #'string< is returned.

Timestamp problems can sometimes crop up with step 2 above. For example, if you have a directory local-projects/my-project/ and you create local-projects/my-project/supporting-system.asd, the timestamp of local-projects/ is not updated and supporting-system.asd won’t be automatically added to the system index file.

There are a couple ways to force an update of the system index file. Within Lisp, you can use (ql:register-local-projects) to immediately regenerate system-index.txt. Outside of Lisp, you can use the touch command (or an equivalent) to update the timestamp of the local-projects directory, which will trigger a rebuild of the index on the next attempt to find systems..

Because of how the system index file is created (and recreated as needed), Quicklisp must have write access to the local-projects directory to make use of it.


The local-projects mechanism is configured through a special variable ql:*local-project-directories*. By default, it includes only the local-projects subdirectory in the Quicklisp install directory, but you can add or remove directories at any time to have more places scanned for systems.
To disable the local-projects mechanism entirely, set ql:*local-project-directories* to NIL.

Build failures with ASDF 3.3.1

SBCL 1.4.3 ships with ASDF 3.3.1, and a number of Quicklisp projects have build problems as a result. Linedit, mgl, micmac, cl-string-match, and others are affected.

Here is a build failure report for yesterday. (You can ignore the gendl failures - it's a special case.) If anyone has ways to fix these projects, please do so as soon as you can - otherwise they will be removed from the January Quicklisp dist update in a few weeks.


Download stats for December, 2017

Here are the raw Quicklisp download stats for December, 2017:

27247  alexandria
23604  closer-mop
21195  anaphora
20818  cl-ppcre
20690  split-sequence
20360  let-plus
20153  iterate
20032  babel
19888  trivial-features
18779  trivial-gray-streams
18077  bordeaux-threads
17215  cffi
16969  more-conditions
16966  trivial-garbage
16258  puri
16049  flexi-streams
15447  nibbles
14567  utilities.print-items
14366  usocket
13449  esrap
13366  chunga
13149  cl+ssl
12853  cl-base64
12701  chipz
12408  trivial-backtrace
12365  drakma
 9502  cl-fad
 9335  asdf-flv
 9270  cl-yacc
 8593  fiveam
 8050  parse-number
 7899  closure-common
 7893  cxml
 7878  log4cl
 7798  local-time
 7646  ironclad
 7621  architecture.hooks
 7347  named-readtables
 7343  parser.common-rules
 6783  plexippus-xpath
 6767  cl-json
 6708  lift
 6050  optima
 5425  lparallel
 5234  cl-clon
 5107  cxml-stp
 5031  xml.location
 4858  utilities.print-tree
 4855  cl-dot
 4430  cl-store
 4055  fare-quasiquote
 3963  fare-utils
 3816  inferior-shell
 3815  fare-mop
 3707  cl-unicode
 3432  cl-interpol
 3321  slime
 2919  trivial-utf-8
 2848  cl-utilities
 2830  metabang-bind
 2744  quri
 2628  uuid
 2415  trivial-types
 2375  cl-annot
 2372  cl-syntax
 2299  cl-slice
 2255  md5
 2247  trivial-indent
 2234  array-utils
 2229  plump
 2227  documentation-utils
 2226  static-vectors
 2219  gettext
 2107  symbol-munger
 2101  arnesi
 2092  collectors
 2087  access
 2086  fast-io
 2065  djula
 2056  cl-locale
 2051  cl-parser-combinators
 2014  hunchentoot
 1910  simple-date-time
 1844  ieee-floats
 1625  yason
 1352  rfc2388
 1293  monkeylib-binary-data
 1171  osicat
 1163  salza2
 1153  utilities.binary-dump
 1135  postmodern
 1044  trivial-shell
 1015  prove
  980  diff
  949  cl-who
  942  asdf-system-connections
  936  command-line-arguments
  933  cl-containers
  931  cl-custom-hash-table
  925  metatilities-base