New projects in the next dist update

There will be a number of new projects in the next Quicklisp dist update, including asdf-install, cells (coming back, anyway), cl-geocode, cl-gpu, cl-iconv, cl-mssql, cl-mw, cl-org-mode, cl-popen, cl-proc, cl-quakeinfo, cl-uglify-js, csv-parser, curly, latex-table, proton, sb-vector-io, and spartns.

Yes, asdf-install.

The devil's guide to Quicklisp projects

Stuart Overton asked me "How can I make my projects Quicklisp-friendly?"

Here are some answers to the opposite question: How can you make your projects Quicklisp-hostile?

• Use a name that someone else is already using
• Use SourceForge
• Require extra steps to build, like running a shell script, make, ./configure, etc. outside of the asdf infrastructure
• Require libraries (especially foreign libraries) that are hard to find, hard to build, or both
• Bundle third-party libraries with your project; make sure they are either several versions out of date or locally modified in incompatible ways (stealth forking!)
• Make symlinks a critical part of the build process
• Your project name (cl-foo) should differ from its primary system name (foo, or even better, quux)
• System names should not be boring, so try using mixed-case or exotic characters for your system file names
• Make sure there are multiple forks of the project on github, the original one isn't the canonical one any more
• Depend on your own personal utility library, which is clearly better in many respects than the twelve other utility libraries already available
• Include none of README, INSTALL, LICENSE, COPYING
• Use your implementation's MOP and FFI directly instead of Closer to MOP and CFFI
• Make releases at most once or twice per decade
• Make your system definition be just a thin loader for your complicated loader script, which uses none of the defsystem's infrastructure or configuration

That's all I can think of for now...

"Old" notes

Here are some notes I wrote to myself from earlier this year. I had been kicking a Quicklisp-like idea around for a while, but these really got me excited about how the whole thing could come together without too much work. They don't reflect the current state of things but they're not far off.

So here's how QuickLisp could work.

  - archives (tarballs) are built from version control or release
    tarballs on a given date, probably the first day of the month

  - they are uploaded to
    http://a.quicklisp.org/source/<sha1>/<system>.tgz

    -
      e.g. http://a.quicklisp.org/source/53354a603e4dfa58a62f53ac871d62315fb9cf20/vecto.tgz

    - nah, just go with a date:

      - http://a.quicklisp.org/source/2010-05/vecto.tgz

  - a dependency file shows the system dependencies:

    - vecto: cl-vectors zpng zpb-ttf

    - ...

  - an index file maps system names to URLs:

    - vecto: http://a.quicklisp.org/2010-05/vecto.tgz

    - zpng: http://a.quicklisp.org/2010-05/zpng.tgz

  - there are default index/dependency files at
    http://a.quicklisp.org/index.txt and
    http://a.quicklisp.org/dependencies.txt 

  - publishing the latest index creates two files: the default index
    files, and year-month/index.txt year-month/dependencies.txt

    - so there's a metaindex.txt that points to previous versions of
      the index files

  - you can choose previous indexes if they work well for you, or if
    there's some kind of glitch with the defaults

  - no archive file is ever deleted

  - authors can provide their own system map and dependency files, so
    if jimbob wants to hook into the infrastructure, he provides an
    index file that points to his package, a dependency file that
    explains the dependencies, and the tarball itself. If there are a
    number of dependencies that aren't part of the main quicklisp
    archive, they can all be listed.

    - some way of prioritizing conflicting information...

  - metadata for archives: origin (git, darcs, cvs, svn, etc) and date
    fetched, system files contained within, etc

Note that I hadn't decided against WackyCaps for the name yet! The official Quicklisp name now is "Quicklisp", not "QuickLisp."

Client update, ASDF updated to 2.010

I've made some minor changes to the Quicklisp client (the part of Quicklisp that downloads and loads software).

• CMUCL rename-file behavior worked-around
• Windows CCL init file specified more correctly
• HTTP User-Agent string reflects the implementation and version better

I also updated the version of ASDF that comes with Quicklisp to 2.010. See Fare's announcement for details on what's new compared to 2.009.

You can get both of these updates by evaluating (ql:update-client). You will need to restart Lisp for the updates to take effect.

If you run into any problems during or after an update, please email me or post a message to the mailing list.

Volunteer moderators needed

The Quicklisp mailing list is generally unmoderated. However, to reduce spam, a new member's first submission is held in moderation and must be approved or rejected. I try to take care of moderation promptly, but my schedule doesn't always allow for it. I'm looking for a volunteer or two to help me out.

Your duty would be to check your mail for moderation notices and allow any non-spam messages through via the Google Groups moderation web interface. This isn't an editorial position - anything that isn't obvious spam should be allowed. The mail volume has been pretty low, with a handful of messages per week.

Ideal moderators would be in a different timezone than me and check email on weekends and holidays. Anyone in Europe, Asia, Australia, etc, feel like volunteering? If so, please drop me an email.

Project download statistics

Here are the top 50 projects downloaded from Quicklisp, ordered by download count:

    714 alexandria
    596 babel
    520 trivial-features
    503 cffi
    450 cl-ppcre
    423 trivial-gray-streams
    404 usocket
    403 flexi-streams
    398 bordeaux-threads
    393 slime
    386 cl+ssl
    371 chunga
    370 cl-base64
    361 cl-fad
    339 md5
    327 quicklisp-slime-helper
    323 trivial-backtrace
    321 rfc2388
    317 hunchentoot
    293 salza2
    289 puri
    285 closer-mop
    225 anaphora
    224 parenscript
    221 cl-who
    207 trivial-garbage
    201 iterate
    193 cl-vectors
    190 zpng
    177 asdf-system-connections
    174 zpb-ttf
    173 uffi
    173 metabang-bind
    170 split-sequence
    164 vecto
    163 cl-json
    162 cl-containers
    161 metatilities-base
    159 fare-utils
    156 weblocks
    156 fare-matcher
    148 drakma
    144 cl-cont
    143 closure-common
    140 moptilities
    138 f-underscore
    137 trivial-timeout
    136 metatilities
    135 clsql
    133 cxml

It's important to note that this doesn't distinguish between projects that were downloaded to satisfy some other project's dependency list and projects that were explicitly requested.

November Quicklisp dist update

I've pushed out a new version of the standard "quicklisp" dist. To get it, use (ql:update-all-dists). That command will show a summary of the changes and then perform the update. If you encounter any problems when updating, please email me or report it on the Quicklisp discussion group.

There are a few updates and changes I'd like to highlight.

The Weblocks version in the October Quicklisp dist had a performance issue that meant page display was very slow. I believe that issue has been fixed in the version in this update. If you try Weblocks with Quicklisp, please keep an eye out for any lingering performance problems.

Nikodemus recently updated alexandria to lock its package when using SBCL. That means certain kinds of definitions on alexandria symbols from outside the alexandria package signal an error. That triggered a cascade of updates in projects that depend on alexandria; a typical problem was unwitting redefinition of certain alexandria symbols like appendf and make-keyword.

cl-l10n is now included in Quicklisp. It no longer requires a shell script to fetch its Unicode data files. The data files are now part of a new CL project called cl-l10n-cldr. Thanks to Attila Lendvai for making this Quicklisp-friendly change.

There have been a few project removals as well. teepeedee2 does not work with the new Parenscript 2.2, and its author, John Fremlin, indicated he would not track Parenscript updates. Rather than include a project that doesn't build, I have left teepeedee2 out of the dist. Kenny Tilton's cells no longer compiles on SBCL because it relies on an Allegro-specific keyword for make-hash-table. He indicated he wouldn't make updates just to make it work on SBCL, so at the moment, it is not included in the dist. If you're happily using cells or teepeedee2 with Quicklisp, you can keep using them by not updating your dist. update Kenny just updated Cells and it builds again, so it'll be in the next dist update.

I missed a few project updates, including cl-proc and cl-clon. They will be included in the next dist update. If there are other projects you want to see included in Quicklisp, please create an issue for it on GitHub.

Finally, last week I put out a request for donations on twitter. The generosity of Quicklisp users has been remarkable, and I was able to purchase enough new computer bits and pieces to get a very fast Quicklisp build and test server. It's about twice as fast as doing builds on a Linux VM on my Macbook, and that improved performance helped me get this dist update up faster. A big thank you to everyone who has contributed! If haven't contributed but you'd like to, see the Donate button on www.quicklisp.org.

Updates coming within the week

I hope to get software update out within the week. Here's a preliminary report on the differences between the October dist and the November dist:

New projects:
    beirc-20101103-cvs
    chronicity-v0.2.6
    cl-bencode-20101103-git
    cl-crc64-20101103-git
    cl-l10n-cldr-20101103-darcs
    cl-l10n-20101103-darcs
    cl-sphinx-20101103-git
    cl-test-more-20101103-git
    cl-uglify-js-20101103-git
    cl-v4l2-20101103-git
    clfswm-20101103-svn
    css-lite-20101103-git
    gbbopen-20101103-svn
    hu.dwim.wui-20101103-darcs
    hunchentoot-auth-20101103-git
    hunchentoot-cgi-20101103-git
    hunchentoot-vhost-20101103-git
    loopless-20101103-git
    manardb-20101103-git
    nuclblog-20101103-git
    pileup-20101103-git
    restas-directory-publisher-20101103-git
    restas-20101103-git
    sb-cga-20101103-git
    xml-match-20101103-git

  Updated projects:
    alexandria-20101006-git -> alexandria-20101103-git
    aromyxo-20101006-git -> aromyxo-20101103-git
    babel-20101006-darcs -> babel-20101103-darcs
    blackthorn-engine-20101006-hg -> blackthorn-engine-20101103-hg
    chillax-20101006-git -> chillax-20101103-git
    cl-closure-template-20101006-git -> cl-closure-template-20101103-git
    cl-devil-20101006-git -> cl-devil-20101103-git
    cl-glfw-20101006-git -> cl-glfw-20101103-git
    cl-jpeg-20101006-http -> cl-jpeg-20101103-http
    cl-mongo-20101006-git -> cl-mongo-20101103-git
    cl-opengl-20101006-git -> cl-opengl-20101103-git
    cl-pdf-20101006-svn -> cl-pdf-20101103-svn
    cl-random-20101006-git -> cl-random-20101103-git
    cl-routes-20101006-git -> cl-routes-20101103-git
    cl-skip-list-20101006-git -> cl-skip-list-20101103-git
    cl-smtp-20101006-http -> cl-smtp-20101103-http
    cl-syslog-20101006-http -> cl-syslog-20101103-http
    cl-zmq-20101006-git -> cl-zmq-20101103-git
    clon-20101006-git -> clon-20101103-git
    closer-mop-20101006-darcs -> closer-mop-20101103-darcs
    closure-common-20101006-git -> closure-common-20101103-git
    clpython-20101006-git -> clpython-20101103-git
    cxml-stp-2008-11-30 -> cxml-stp-20101103-git
    cxml-20101006-git -> cxml-20101103-git
    dbus-20101006-git -> dbus-20101103-git
    eager-future-20101006-darcs -> eager-future-20101103-git
    f2cl-20101006-cvs -> f2cl-20101103-cvs
    fare-matcher-20101006-git -> fare-matcher-20101103-git
    fare-utils-20101006-git -> fare-utils-20101103-git
    folio-20101006-git -> folio-20101103-git
    glaw-20101006-git -> glaw-20101103-git
    glop-20101006-git -> glop-20101103-git
    gsll-20101006-git -> gsll-20101103-git
    hu.dwim.asdf-20101006-darcs -> hu.dwim.asdf-20101103-darcs
    hu.dwim.computed-class-20101006-darcs -> hu.dwim.computed-class-20101103-darcs
    hu.dwim.logger-20101006-darcs -> hu.dwim.logger-20101103-darcs
    hu.dwim.perec-20101006-darcs -> hu.dwim.perec-20101103-darcs
    hu.dwim.rdbms-20101006-darcs -> hu.dwim.rdbms-20101103-darcs
    hu.dwim.stefil-20101006-darcs -> hu.dwim.stefil-20101103-darcs
    hu.dwim.syntax-sugar-20101006-darcs -> hu.dwim.syntax-sugar-20101103-darcs
    hu.dwim.util-20101006-darcs -> hu.dwim.util-20101103-darcs
    hu.dwim.walker-20101006-darcs -> hu.dwim.walker-20101103-darcs
    iolib-0.7.1 -> iolib-0.7.3
    js-20101006-git -> js-20101103-git
    lift-20101006-git -> lift-20101103-git
    linedit-20101006-cvs -> linedit-20101103-git
    lispbuilder-20101006-svn -> lispbuilder-20101103-svn
    lla-20101006-git -> lla-20101103-git
    montezuma-20101006-svn -> montezuma-20101103-svn
    parenscript-2.1 -> parenscript-2.2
    parse-js-20101006-git -> parse-js-20101103-git
    plokami-1.4 -> plokami-20101103-git
    postmodern-20101006-git -> postmodern-20101103-git
    quicklisp-slime-helper-20101006-git -> quicklisp-slime-helper-20101103-git
    quickproject-20101006-git -> quickproject-20101103-git
    scribble-20101006-git -> scribble-20101103-git
    shuffletron-20101006-git -> shuffletron-20101103-git
    slime-20101006-cvs -> slime-20101103-cvs
    st-json-20101006-darcs -> st-json-20101103-git
    stefil-20101006-http -> stefil-20101103-darcs
    sw-mvc-20101006-git -> sw-mvc-20101103-git
    symbolicweb-20101006-git -> symbolicweb-20101103-git
    tpapp-utils-20101006-git -> tpapp-utils-20101103-git
    uffi-20101006-git -> uffi-20101103-git
    weblocks-20101006-hg -> weblocks-20101103-hg
    yason-20101006-svn -> yason-20101103-git
    zip-20101006-cvs -> zip-20101103-cvs

If there are projects you want to see added, please open a GitHub issue for them.