Dealing with "redundant" libraries

People sometimes submit libraries to Quicklisp that have significant functional overlap with other libraries. When someone wants to add a new library for functionality that is well-covered by other libraries, I don't always add it right away. There are a few questions I ask first.

First, are you aware that there are many other libraries that fit a similar need? If not, is it possible to use one of those instead? It is possible that an existing library is more complete and mature than a new library. And adding a new library can make it harder for someone else to choose a good option.

If you're aware of other libraries, is it possible that the essential new functionality your library provides can be incorporated into an existing library? 

If you're aware of other libraries, but still feel you need a new one, and it can't be incorporated into an existing library, I'll go ahead and add the library. 

Here are a few recent examples of that process playing out:
I don't reject libraries for lack of novelty. It's always possible that a new library becomes so clearly superior to all other options that it is the go-to library of the future. But it's best when that happens because you're aware of the existing options (and their strengths and limitations) rather than ignorant of them.


New feature: Quicklisp library bundles

Quicklisp library bundles are self-contained sets of systems that are exported from Quicklisp and loadable without involving Quicklisp.

I added the feature to the Quicklisp client on April 18th and announced it at ELS on the 20th. To get it, use (ql:update-client). It will be available when you restart for the next session.

I've wanted to provide this feature since the beginning of Quicklisp. I primarily pictured this as a kind of delivery tool, where your project should have access to a specific set of supporting libraries without the complexity of Quicklisp loaded in as well.

The interface is pretty simple: (ql:bundle-systems '("foo" "bar" "baz") :to "my-bundle/") will produce a file named "my-bundle/bundle.lisp" that, when loaded, will make "foo", "bar", and "baz" accessible via ASDF without involving Quicklisp at all. All the libraries on which "foo", "bar", and "baz" depend are also included in the bundle, recursively.

The bundle can be relocated without any issues. All its pathnames and indexes are relative to its bundle.lisp.

When bundle.lisp is loaded, the systems in the bundle take precedence over all other ASDF systems. Multiple bundles can be loaded at the same time; the most recently loaded one has the highest precedence. And a bundle can be loaded more than once. If loaded again, it will be moved to the top of the precedence list again.

Bundles also have a local-projects directory that adds some of the automagic from the Quicklisp local-projects feature. Bundle local-projects systems take precedence over the bundle's "built-in" systems.

Quicklisp library bundles are also documented! But they are only lightly tested. If you think library bundles might help you solve a problem, please give them a try and let me know if you have any problems or questions.


April 2015 Quicklisp dist update now available

This Quicklisp update is supported by my employer, Clozure Associates. If you need commercial support for Quicklisp, or any other Common Lisp programming needs, it's available via Clozure Associates.

New projects:
  • birch — A simple Common Lisp IRC client library — MIT
  • bytecurry.mocks — Tools to mock functions for unit tests — MIT
  • carrier — An async HTTP client — MIT
  • cl-cookie — HTTP cookie manager — BSD 2-Clause
  • cl-coveralls — Coverage tracker for Coveralls — BSD 2-Clause
  • cl-groupby — groupby: A higher order function named groupby as known from Scala. — MIT
  • cl-openstack-client — OpenStack client libraries — Apache-2.0
  • cl-poker-eval — 7-card hand poker evaluator — BSD
  • cl-shellwords — Common Lisp port of Ruby's shellwords.rb, for escaping and splitting strings to be passed to a shell. — MIT
  • cl-singleton-mixin — provides singleton-mixin class. — MIT
  • dexador — Yet another HTTP client for Common Lisp — MIT
  • duologue — High-level user interaction library for Common Lisp — MIT
  • dyna — Dyna is an AWS DynamoDB ORM for Common Lisp. — MIT
  • find-port — Find open ports programmatically. — MIT
  • fmt — Extensible format-like facility — MIT
  • folio2 — the folio2 functional-idioms system — Lisp Lesser GNU Public License
  • immutable-struct — Library that encourage the use of functional programming + pattern matching — LLGPL
  • jonathan — JSON encoder and decoder. — MIT
  • json-responses — Canned JSON responses for Hunchentoot — MIT
  • kebab — Common Lisp string,symbol,keyword PascalCase <=> camelCase <=> snake_case <=> kebab-case(lisp-case) converter. — LLGPL
  • lack — A minimal Clack — LLGPL
  • metap — Metap provides metaclass propagation along class inheritance structure. — MIT
  • nsort — Natural or Numeric Sort — BSD Simplified
  • plump-bundle — A binary storage format for Plump documents. — Artistic
  • proc-parse — Procedural vector parser — BSD 2-Clause
  • quadtree — Quadtree data structure in Common Lisp — MIT
  • quasiquote-2.0 — Writing macros that write macros. Effortless. — MIT
  • scriba — A markup format similar to Scribe. — MIT
  • should-test — Minimal yet feature-rich Common Lisp test framework. — MIT
  • string-escape — Emacs and Python style string escapes in #"..." form. — GPLv3
  • thorn — A CommonDoc extension for entering special characters. — MIT
  • trivia — NON-optimized pattern matcher compatible with OPTIMA, with extensible optimizer interface and clean codebase — LLGPL
  • trivia.balland2006 — Optimizer for Trivia based on (Balland 2006) — LLGPL
  • type-i — Type Inference Utility on Fundamentally 1-arg Predicates — LLGPL
  • type-r — Collections of accessor functions and patterns to access the elements in compound type specifier, e.g. `dimensions' in `(array element-type dimensions)' — LLGPL
  • unix-opts — minimalistic parser of command line arguments — MIT
Updated projects: 3bmd, access, antik, arrow-macros, buffalo, buildapp, chanl, cl-ana, cl-annot, cl-ansi-term, cl-async, cl-charms, cl-dbi, cl-dot, cl-factoring, cl-gobject-introspection, cl-grace, cl-indeterminism, cl-launch, cl-libyaml, cl-mlep, cl-mtgnet, cl-mw, cl-netstring-plus, cl-openid, cl-ply, cl-python, cl-quickcheck, cl-random, cl-readline, cl-reddit, cl-redis, cl-rlimit, cl-sdl2, cl-slug, cl-svg, cl-syntax, cl-tcod, cl-vectors, cl-voxelize, cl-yaml, clack, classimp, clavier, clhs, clim-widgets, clinch, clipper, clos-fixtures, closer-mop, clsql-helper, clx, codata-recommended-values, colleen, com.informatimago, common-doc, common-doc-plump, common-html, commonqt, croatoan, defclass-std, djula, drakma, eazy-project, esrap-liquid, fare-memoization, fast-http, fast-io, femlisp, gendl, graph, gsll, hdf5-cffi, hl7-client, hl7-parser, http-body, hu.dwim.util, hunchentoot, hyperluminal-mem, integral, interface, introspect-environment, js-parser, jsown, jwacs, lass, let-over-lambda, lfarm, linedit, lisp-interface-library, lisp-invocation, lisp-namespace, local-time, lucerne, magicffi, mcclim, media-types, mgl-pax, mk-string-metrics, nibbles, ningle, nst, plump, protobuf, qtools, quri, quux-time, racer, rutils, scalpl, scriptl, sdl2kit, serapeum, shellpool, simple-rgb, skippy, slime, st-json, staple, stumpwm, symbol-munger, trivial-backtrace, trivial-benchmark, trivial-debug-console, trivial-download, trivial-update, umlisp, verbose, vertex, vgplot, weblocks-stores, weft, workout-timer, xmls, zaws, zcdb, zpb-exif, zpng.

Removed projects: autoproject, brlapi, cambl, cl-couch, cl-ledger, hctsmsl, nekthuth, red-black.

To get these updates, use (ql:update-dist "quicklisp").



March 2015 download stats

Here are the top 100 download for last month:

 5853   alexandria
 4149   cl-ppcre
 3465   closer-mop
 3382   trivial-features
 3296   babel
 3086   named-readtables
 2977   cffi
 2814   flexi-streams
 2795   bordeaux-threads
 2785   trivial-gray-streams
 2784   cl+ssl
 2722   cl-fad
 2641   trivial-garbage
 2315   nibbles
 2313   usocket
 2277   chunga
 2251   anaphora
 2235   cl-base64
 2218   optima
 2142   split-sequence
 1942   ironclad
 1839   puri
 1803   fiveam
 1768   iterate
 1753   drakma
 1729   chipz
 1641   cl-colors
 1629   trivial-backtrace
 1586   local-time
 1559   md5
 1493   slime
 1486   let-plus
 1401   fare-utils
 1392   fare-quasiquote
 1340   cl-ansi-text
 1190   prove
 1180   hunchentoot
 1098   cl-unicode
 1072   trivial-types
 1053   rfc2388
 1021   metabang-bind
 1010   cl-utilities
  986   cl-interpol
  961   cl-syntax
  950   trivial-utf-8
  889   introspect-environment
  846   quicklisp-slime-helper
  827   cl-annot
  802   parse-number
  757   st-json
  716   quri
  710   osicat
  709   postmodern
  686   cl-marshal
  658   trivial-mimes
  654   xsubseq
  651   lparallel
  648   plump
  645   jsown
  641   asdf-system-connections
  614   ieee-floats
  613   trivial-indent
  605   metatilities-base
  604   uuid
  603   array-utils
  601   cl-containers
  596   cl-json
  590   lquery
  584   fast-http
  572   cl-sqlite
  567   salza2
  556   clss
  538   clack
  500   static-vectors
  495   clx
  490   command-line-arguments
  464   cl-markdown
  463   py-configparser
  462   dynamic-classes
  455   circular-streams
  446   asdf-finalizers
  446   zpb-ttf
  446   cl-log
  445   fast-io
  443   cl-abnf
  442   garbage-pools
  440   buildapp
  435   cl-mssql
  425   cl-who
  399   zpng
  397   esrap
  395   http-body
  394   vecto
  391   cl-csv
  389   cl-vectors
  388   iolib
  358   closure-common
  353   lisp-namespace
  351   cl-opengl
  351   cl-dbi