Quicklisp uses two text files for information about project releases (releases.txt) and systems (systems.txt). Whenever information about a system was needed (for example, where its system file can be found), both files were loaded completely, from scratch.
I made it that way because it was pretty easy. People immediately noticed that it was also pretty slow, especially when using (asdf:load-system "...") instead of (ql:quickload "..."). The penalty for frequent loading and reloading of the metadata got worse as the number of Quicklisp systems grew.
Today I released a client update that can load metadata from a fast on-disk hash table, a CDB file. The big flat files are converted to CDB once, as needed, and thereafter getting metadata is super-speedy. On my laptop, the improvement for lookups is about 100x; your results will depend on the speed of your disk.
To get the update, use (ql:update-client) and restart Lisp.
If this change causes you any trouble, please let me know via the Quicklisp mailing list.
UPDATE: This change causes trouble. I'm going to put out a fix as soon as I can. Please don't update; if you did update, you can revert with something like this:
cd ~/quicklisp/
wget http://beta.quicklisp.org/quickstart/quicklisp-2011111500.tgz
tar xzvf quicklisp-2011111500.tgz
rm -rf ~/.cache/common-lisp/
UPDATE 2: I can't reproduce the trouble I saw earlier. If you run into trouble, let me know. I really need cases I can reproduce to fix things.
UPDATE 3: Thanks to Mike Clarke, I found a serious problem with the CDB scheme. A fix will be available within a day or two.
Is the CDB conversion / query library integral part of quicklisp? It might be very useful for other projects.
ReplyDeleteI basically cut & pasted code from my ZCDB library into Quicklisp, so no, it is not tightly integrated.
ReplyDeleteI did not know about ZCDB, but I liked the format so much that I have adopted it for ECL's help file (the one that stores all the documentation strings from the library) and reimplemented it as a loadable module (require :ecl-cdb) respecting the ZCDB interface.
ReplyDelete