Sat, 14 May 2005
... to Lolo's small squid url rewrite hack: Download all packages with priority required, standard and important nighlty.
Netinstall base system in less than 10 minutes guaranteed...
As I bloged before: I'm here at a small development gathering in Guetersloh.
Here are people from quite different regions, and therefore with quite different
/etc/apt/sources.list files. So, what is the easiest way, to allow
them to get their packages from one fast mirror?
Interesting problem. Of course you could just tell them to point their sources list entries to the fastest mirror in your experience - there are even tools to find the fastest mirror like netselect-apt or apt-spy, but that isn't the most elegant solution, isn't it?
Of course you can just create a local mirror (for example with debmirror or with the script linked at http://www.debian.org/mirror/ftpmirror),
but that's a bit overkill, isn't it? Especially, if you have only one dsl-line.
And everyone still needs to change the
If you run a proxy like squid, you could just think, that often requested packages, will be in the cache, so you still let it be as it is. But if two people install the same package using different mirrors, it would still be downloaded and cached twice (since it would have different URLs).
A really elegant solution is to add a rewrite script to Squid, so that URLs from one mirror are transparently rewritten to your favourite mirror. Florian Lohoff just wrote hacked a nice script: Let's take the mirror list from Debian, to know which hosts we should rewrite (since we won't touch other server). Then take our Package-File and extract all file name from all packages, and now if someone wants to access http://ftp.foo.bar/debian/pool/main/b/baz_0.0-1_baz.deb this URI is rewritten if and only if ftp.foo.bar is in our list of known hosts AND if pool/main/b/baz_0.0-1_baz.deb is in our known packages. Hosts like security.debian.org and packages like baz_0.0-1ubuntu3.deb are ignored.
So you only need to force everyone to use the proxy (one trivial rule on your router), and now everyone does not even to touch their configuration, they can use (nearly) any mirror in their sources.list, and won't even notice, that they indeed downloading their packages from e.g. ftp.de2.debian.org. Of course you should tell them, what you have done ;)
And you should do some fine tuning, e.g. you might want to increase the max size of save objects in your squid config (to catch some bigger packages, too), and give your squid quite same space to storage his objects.
If you want to know details of this nice little hack, you should contact Flo directly, he did it, he knows it a bit better than me ;)
...took me 5 hours to get there. Well, it's quite nice here, the main train station is circa 10 minutes by foot from the Skolelinux Testzentrum in Germany, there is a nice pub 10m away, and if you need a small recovery time, you can take a small walk in a nearby park.
I only wished, it would take me less time to get here from my hometown near Frankfurt / Main.
This time it went quite bad: Beside the problem, that I couldn't book my train tickets in advance (I had a meeting, and didn't know, which train I would take), my train needed to be redirected (I think a suicide jumped before a previous train), and therefore I missed my connection, waiting for nearly an hour...
But hey, I finally arrived, and could join the already accumulated team, meeting old friends again, start to work on various topics, learn from them, and teach them a bit of what I know...