Some problems when adding libraries to Quicklisp

Here are a few of the problems I encounter when trying to add a library to Quicklisp, as well as how to prevent or fix them.

Library does not build without warnings. As mentioned a little while ago, ql:quickload normally muffles warnings, even, unfortunately, for non-Quicklisp projects. The Quicklisp dist build environment does not muffle any warnings, and any that occur will break the build for the library. Make sure you use the :verbose t option to ql:quickload to see any warnings that crop up during compilation.

Library does not build at all. I think this happens when someone sees a library that seems cool, they find it is absent from Quicklisp, and they request its addition without trying it first. Please try it first! It's easy to try libraries: fetch the code, put it into ~/quicklisp/local-projects/, look for *.asd files, and use ql:quickload to load one. If it doesn't load, it may prove difficult for me to try to add it to Quicklisp. And if it doesn't have *.asd files, I can't add it to Quicklisp at all.

Library is missing critical metadata. Make sure the library has :author, :description, and :license in each ASDF system definition.

Library depends on another library that is not available in Quicklisp. It's fine to request the addition of multiple related libraries. It helps if you specify the order in which they need to be added to work.

Library system name conflicts with existing system. This happens sometimes when a library bundles its own private copy of a library already present in Quicklisp. In that case, it is usually best to unbundle the private copy, but I can also work around it on my end if necessary. Conflict also happens when someone just doesn't know that a system name is already in use. To check for conflicts, use (ql-dist:provided-systems t) to get a list of existing systems in Quicklisp.

Getting a library into Quicklisp

If there's a library you like, and you'd like to see it available to download and install via the standard Quicklisp dist, here's what to do.

First, make sure the license terms of the code allow for its redistribution. I can't add things with restrictive licenses or with licenses that are missing or unclear.

Second, make sure it works on more than just one implementation of Common Lisp. Quicklisp is for portable libraries. (In the future, I hope to make it easy to create separate new dists specifically for implementation-specific code, but even then, the main Quicklisp dist will be for portable libraries.)

As a side-effect of how I build and test the dist, it must also work on SBCL on Linux/AMD64. That means, unfortunately, that a great portable library that works on three different Windows CL implementations, but not on Linux, cannot be added. I hope to fix this limitation in the future.

Third, make sure the library has certain ASDF system definition metadata: :license, :author, and :description. It also should have a README file in some form or another. A note about the README: it should give at least short overview of what the library is for. "The Foo library is an implementation of Ruby's Snorfle in Common Lisp" is not a good overview; give me an idea of what it actually does, instead, e.g. "The Foo library fetches and parses movie showtime information." It's good to also provide information about how to report bugs and how to contact the author.

Fourth, make sure it builds with ASDF, rather than an external-to-Lisp build mechanism. I can't add libraries that require special configuration or action outside of ASDF. For example, if you have to edit a source file to configure library or resource directories before building, I can't add it to Quicklisp. If the library can be loaded with just (asdf:load-system ...), it's good.

Finally, let me know about it. I prefer to track requests via github's issue system, but you can also send me an email as well. It suffices to write something like "Please add the foo library, which is available from http://www.example.com/foo.tgz. The homepage is http://www.example.com/foo/."


It's important to note that I don't consider a library's quality or purpose when adding it to Quicklisp. It doesn't matter if you're submitting your own library. If you want it added, and it fits the above criteria, I will almost certainly add it.

There are a few exceptions: projects that require complicated or obscure foreign libraries, projects that can only be downloaded via some ad-laden link system like SourceForge, projects that use CVS, and anything else that makes it difficult for me to fetch or build the project.

When you open a github issue for a library, I'll occasionally update the issue's status. I will add issue comments if I have any problems building, or if any required bit of information (license, ASDF metadata, README) is missing.

Barring any problems, when the github issue for a library is closed, the latest Quicklisp dist has been released and it includes the new library. (Sometimes I mess this up, so if it seems like the library is still missing after a dist update, feel free to get in touch.)

How about updates? Many libraries do not need any extra work to get updated regularly in Quicklisp. For example, if a library can be downloaded from an URL like "http://example.com/cool-project/cool-project-latest.tgz", Quicklisp will detect when a new file is posted. For libraries downloaded from version control systems like git, updates are also automatically fetched. Only when a library uses a fixed URL per version is it necessary to open a github issue for updates.

Quicklisp dist updates happen about once per month. If the library is updated upstream, those updates will only be reflected after the next Quicklisp dist update. Each dist update freezes the state of the Quicklisp library "world" until the next monthly update.

If you'd like to see the process in action, watch the quicklisp-projects issue page for a month to see how things typically work.

If you have any questions about the process, feel free to get in touch.

Update: See also Some problems when adding libraries to Quicklisp, which explains how the above process can go wrong sometimes.

December 2014 download stats

Here are the top 100 downloads from Quicklisp for last month:

  3052   alexandria
  2743   cl-ppcre
  2619   babel
  2009   cffi
  1886   cl-fad
  1829   flexi-streams
  1757   trivial-features
  1670   bordeaux-threads
  1597   slime
  1552   closer-mop
  1541   trivial-gray-streams
  1523   chunga
  1390   trivial-garbage
  1267   cl+ssl
  1239   anaphora
  1152   usocket
  1106   hunchentoot
  1097   drakma
  1092   trivial-backtrace
  1085   cl-base64
  1083   iterate
  1077   local-time
  1011   split-sequence
   987   md5
   963   nibbles
   920   ironclad
   848   let-plus
   820   cl-unicode
   792   puri
   761   rfc2388
   736   cl-colors
   696   chipz
   690   quicklisp-slime-helper
   683   named-readtables
   653   metabang-bind
   635   cl-json
   626   cl-who
   624   cl-ansi-text
   561   parse-number
   556   prove
   548   cl-interpol
   533   postmodern
   506   clx
   501   yason
   443   log4cl
   440   lparallel
   436   trivial-utf-8
   422   salza2
   413   cl-csv
   406   optima
   392   osicat
   390   rt
   381   asdf-system-connections
   369   clack
   368   trivial-types
   368   parenscript
   358   iolib
   356   cl-containers
   356   cl-syntax
   347   metatilities-base
   340   cl-utilities
   340   cl-opengl
   323   cl-annot
   316   ieee-floats
   307   esrap
   294   uuid
   289   cl-sqlite
   281   vecto
   274   html-template
   273   cl-async
   271   zpb-ttf
   269   closure-common
   262   restas
   262   buildapp
   258   external-program
   257   command-line-arguments
   254   uiop
   250   cl-yacc
   249   caveman
   244   asdf-finalizers
   241   weblocks
   240   cl-async-future
   238   dynamic-classes
   233   cl-markdown
   232   zpng
   229   cxml
   226   static-vectors
   225   py-configparser
   224   uffi
   219   mcclim
   219   cl-vectors
   216   fiveam
   210   quri
   209   cl-closure-template
   208   cl-log
   208   cl-libevent2
   206   cl-abnf
   204   cl-dbi
   204   cl-db3
   203   ningle

I just started systematically managing Quicklisp HTTP logs, so I will soon present information like this on a regular basis.

January 2015 Quicklisp dist update now available

New projects:

• asdf-contrib — Extensions to ASDF — MIT
• asdf-flv — ASDF support for file-local variables. — GNU All Permissive
• chrome-native-messaging — A package to communicate with a Chrome extension as the native application — MIT License
• cl-ansi-term — library to output formatted text on ANSI-compliant terminals — GNU GPL v.3
• cl-binaural — Utilities to generate binaural sound from mono — GPL
• cl-hash-util — A simple and natural wrapper around Common Lisp's hash functionality. — MIT
• cl-hue — Client for Philips Hue light controller — Apache 2
• cl-junit-xml — Small library for writing junit XML files — MIT
• cl-mop — Simple, portable tools for dealing with CLOS objects. — Expat (MIT-style)
• cl-readline — Common Lisp bindings to GNU Readline library — GNU GPL v.3
• cl-slug — Small library to make slugs, mainly for URIs, from english and beyond. — LLGPL
• clim-widgets — small collection of clim widgets — BSD Simplified
• common-doc — A framework for representing and manipulating documents as CLOS objects. — MIT
• common-html — An HTML parser/emitter for CommonDoc. — MIT
• defclass-std — A shortcut macro to write DEFCLASS forms quickly. — LLGPL
• defenum — C++ and Java styled 'enum' in Common Lisp — MIT
• generic-comparability — CDR-8 implementation — LLGPL
• lev — libev bindings for Common Lisp — BSD 2-Clause
• lucerne — A Clack-based microframework. — MIT
• perlre — perl regular expression api - m// and s/// - for CL-PPCRE with CL-INTERPOL support — BSD Simplified --- the same as let-over-lambda
• trivial-update — tools for easy modification of places with any given function — MIT

Updated projects: arc-compat, avatar-api, blackbird, buildnode, caveman, cl-acronyms, cl-ana, cl-ansi-text, cl-async, cl-autowrap, cl-base58, cl-creditcard, cl-dbi, cl-gss, cl-inflector, cl-libuv, cl-marshal, cl-mock, cl-mustache, cl-opengl, cl-pass, cl-rabbit, cl-randist, cl-random, cl-sdl2, cl-virtualbox, cl-webkit, clack, clack-errors, clos-fixtures, closer-mop, clss, clx, colleen, com.informatimago, commonqt, corona, crane, css-selectors, datafly, eco, esrap, exponential-backoff, fast-http, fset, gbbopen, gendl, hdf5-cffi, hermetic, http-body, hu.dwim.stefil, integral, introspect-environment, iolib, jsown, lambda-fiddle, lisp-unit2, local-time, lol-re, ltk, mk-string-metrics, modularize, modularize-hooks, modularize-interfaces, new-op, osicat, plump, plump-tex, pp-toml, prove, pzmq, quri, rock, rutils, scalpl, sdl2kit, serapeum, sheeple, slime, spinneret, stumpwm, sxql, template, trivial-arguments, trivial-download, trivial-extract, trivial-features, trivial-garbage, weblocks, weblocks-utils, websocket-driver, woo, wookie, wuwei, xsubseq.

To get this update, use (ql:update-dist "quicklisp").

To install exactly this update, use (ql-dist:install-dist "http://beta.quicklisp.org/dist/quicklisp/2015-01-13/distinfo.txt" :replace t).

This Quicklisp update is supported by my employer, Clozure Associates. If you need commercial support for Quicklisp, or any other Common Lisp programming needs, it's available via Clozure Associates.