Fetching updates for all the projects in Quicklisp takes a long time. For each project it does a "git pull" or "svn update" or equivalent, depending on how the project is managed. It does this one at a time, serially.
There's no reason it couldn't do more than one update at a time. An ideal system would fetch some of the projects in parallel, taking care not to fetch multiple projects from the same site at a time. For example, there are dozens and dozens of github-hosted projects, but the Quicklisp controller should only have one github connection open at a time. Same with common-lisp.net, dwim.hu, bitbucket, and the same for other sites that host multiple projects.
It's possible that lparallel or lfarm could help with this task.
Are you up for adding this feature to quicklisp-controller? Hacks and glory await!
update Orivej Desh has done it! Thanks Orivej!
What about concurrent compilation? Given a system, launch a subprocess, compile the system, report back. If you're loading a system with three dependencies, compile them in parallel. Wait for the longest one. Hmm. Must look into it.
ReplyDelete