Quicklisp downtime today (December 4)

Quicklisp stores nearly all its files on Amazon's file service, S3. S3 is extremely reliable and highly available, and is a great value for sharing files with the world. I use my personal Amazon Web Services account to pay for it. I also use that Amazon account for hosting files for xach.com, wigflip.com, and other miscellaneous things.

Today I will make the transition from my personal Amazon account to a new, Quicklisp-only Amazon account. With that transition, it will be easier to use your many generous donations (thank you, Quicklisp users!) specifically for Quicklisp-related expenses.

The transition also means that the availability of beta.quicklisp.org might be interrupted over the next day or so. The quickstart file, quicklisp.lisp, will be unavailable for bootstrapping, and you will not be able to fetch any project archive files for installation.

Sorry for any inconvenience it causes. I started Quicklisp in part because website and version control outages made it hard to get Lisp software in a predictable, reliable way. I hope this short period of transition will be the last Quicklisp outage for a long time.

New projects in the next dist update

There will be a number of new projects in the next Quicklisp dist update, including asdf-install, cells (coming back, anyway), cl-geocode, cl-gpu, cl-iconv, cl-mssql, cl-mw, cl-org-mode, cl-popen, cl-proc, cl-quakeinfo, cl-uglify-js, csv-parser, curly, latex-table, proton, sb-vector-io, and spartns.

Yes, asdf-install.

The devil's guide to Quicklisp projects

Stuart Overton asked me "How can I make my projects Quicklisp-friendly?"

Here are some answers to the opposite question: How can you make your projects Quicklisp-hostile?

• Use a name that someone else is already using
• Use SourceForge
• Require extra steps to build, like running a shell script, make, ./configure, etc. outside of the asdf infrastructure
• Require libraries (especially foreign libraries) that are hard to find, hard to build, or both
• Bundle third-party libraries with your project; make sure they are either several versions out of date or locally modified in incompatible ways (stealth forking!)
• Make symlinks a critical part of the build process
• Your project name (cl-foo) should differ from its primary system name (foo, or even better, quux)
• System names should not be boring, so try using mixed-case or exotic characters for your system file names
• Make sure there are multiple forks of the project on github, the original one isn't the canonical one any more
• Depend on your own personal utility library, which is clearly better in many respects than the twelve other utility libraries already available
• Include none of README, INSTALL, LICENSE, COPYING
• Use your implementation's MOP and FFI directly instead of Closer to MOP and CFFI
• Make releases at most once or twice per decade
• Make your system definition be just a thin loader for your complicated loader script, which uses none of the defsystem's infrastructure or configuration

That's all I can think of for now...

"Old" notes

Here are some notes I wrote to myself from earlier this year. I had been kicking a Quicklisp-like idea around for a while, but these really got me excited about how the whole thing could come together without too much work. They don't reflect the current state of things but they're not far off.

So here's how QuickLisp could work.

  - archives (tarballs) are built from version control or release
    tarballs on a given date, probably the first day of the month

  - they are uploaded to
    http://a.quicklisp.org/source/<sha1>/<system>.tgz

    -
      e.g. http://a.quicklisp.org/source/53354a603e4dfa58a62f53ac871d62315fb9cf20/vecto.tgz

    - nah, just go with a date:

      - http://a.quicklisp.org/source/2010-05/vecto.tgz

  - a dependency file shows the system dependencies:

    - vecto: cl-vectors zpng zpb-ttf

    - ...

  - an index file maps system names to URLs:

    - vecto: http://a.quicklisp.org/2010-05/vecto.tgz

    - zpng: http://a.quicklisp.org/2010-05/zpng.tgz

  - there are default index/dependency files at
    http://a.quicklisp.org/index.txt and
    http://a.quicklisp.org/dependencies.txt 

  - publishing the latest index creates two files: the default index
    files, and year-month/index.txt year-month/dependencies.txt

    - so there's a metaindex.txt that points to previous versions of
      the index files

  - you can choose previous indexes if they work well for you, or if
    there's some kind of glitch with the defaults

  - no archive file is ever deleted

  - authors can provide their own system map and dependency files, so
    if jimbob wants to hook into the infrastructure, he provides an
    index file that points to his package, a dependency file that
    explains the dependencies, and the tarball itself. If there are a
    number of dependencies that aren't part of the main quicklisp
    archive, they can all be listed.

    - some way of prioritizing conflicting information...

  - metadata for archives: origin (git, darcs, cvs, svn, etc) and date
    fetched, system files contained within, etc

Note that I hadn't decided against WackyCaps for the name yet! The official Quicklisp name now is "Quicklisp", not "QuickLisp."

Client update, ASDF updated to 2.010

I've made some minor changes to the Quicklisp client (the part of Quicklisp that downloads and loads software).

• CMUCL rename-file behavior worked-around
• Windows CCL init file specified more correctly
• HTTP User-Agent string reflects the implementation and version better

I also updated the version of ASDF that comes with Quicklisp to 2.010. See Fare's announcement for details on what's new compared to 2.009.

You can get both of these updates by evaluating (ql:update-client). You will need to restart Lisp for the updates to take effect.

If you run into any problems during or after an update, please email me or post a message to the mailing list.

Volunteer moderators needed

The Quicklisp mailing list is generally unmoderated. However, to reduce spam, a new member's first submission is held in moderation and must be approved or rejected. I try to take care of moderation promptly, but my schedule doesn't always allow for it. I'm looking for a volunteer or two to help me out.

Your duty would be to check your mail for moderation notices and allow any non-spam messages through via the Google Groups moderation web interface. This isn't an editorial position - anything that isn't obvious spam should be allowed. The mail volume has been pretty low, with a handful of messages per week.

Ideal moderators would be in a different timezone than me and check email on weekends and holidays. Anyone in Europe, Asia, Australia, etc, feel like volunteering? If so, please drop me an email.

Project download statistics

Here are the top 50 projects downloaded from Quicklisp, ordered by download count:

    714 alexandria
    596 babel
    520 trivial-features
    503 cffi
    450 cl-ppcre
    423 trivial-gray-streams
    404 usocket
    403 flexi-streams
    398 bordeaux-threads
    393 slime
    386 cl+ssl
    371 chunga
    370 cl-base64
    361 cl-fad
    339 md5
    327 quicklisp-slime-helper
    323 trivial-backtrace
    321 rfc2388
    317 hunchentoot
    293 salza2
    289 puri
    285 closer-mop
    225 anaphora
    224 parenscript
    221 cl-who
    207 trivial-garbage
    201 iterate
    193 cl-vectors
    190 zpng
    177 asdf-system-connections
    174 zpb-ttf
    173 uffi
    173 metabang-bind
    170 split-sequence
    164 vecto
    163 cl-json
    162 cl-containers
    161 metatilities-base
    159 fare-utils
    156 weblocks
    156 fare-matcher
    148 drakma
    144 cl-cont
    143 closure-common
    140 moptilities
    138 f-underscore
    137 trivial-timeout
    136 metatilities
    135 clsql
    133 cxml

It's important to note that this doesn't distinguish between projects that were downloaded to satisfy some other project's dependency list and projects that were explicitly requested.