Getting started with Hunchentoot, a Common Lisp webserver

Update: There's a second, more interesting, article on the subject here.

Hunchentoot is a Common Lisp webserver by Edi Weitz. It can be used to serve static pages, but most people use it to write web applications in Lisp. There are a few other Lisp webservers, but it's probably the most capable, most compatible and best-maintained one at the moment.

This isn't a general Common Lisp tutorial. If you want such a thing, I would recommend the excellent Practical Common Lisp, which is available in book form or free online.

Hunchentoot will work on most Lisp implementations, though not clisp. If you're using SBCL on Windows, you should still be able to use it, but it will be single-threaded; so you can't really use it for production websites. It's probably okay for development. SBCL on Linux is probably the ideal (free) production environment; it's fast, threaded, and supports Unicode.

Assuming you have asdf-install, you can install hunchentoot by just doing:

(asdf-install:install :hunchentoot)

Hunchentoot has a list of functions associated with page names called *DISPATCH-TABLE*. It goes through this list one by one running the functions with the HTTP request; the function should return another function, which will be run with the request to produce a page as response. It comes with a few handy functions to match page names by prefix and by regular expression, but you can write your own if you like. A simple dispatch table looks like this:

(setq *dispatch-table*
      `(,(create-prefix-dispatcher "/test" 'test-page) ,(create prefix-dispatcher "/about" 'about-page)))

You can set dispatch-tables per server, but setting the global one is probably the easiest way to start off.

When the server is running, now, going to http://localhost:myport/test will run the test-page function. It should return a HTML string, which will be returned to the user. You can generate the HTML any way you like. There are a few HTML generators by the same author as the webserver; html-template fills values into a template (it's based on an old PERL library) while cl-who converts s-expressions into HTML.

Obviously, that in itself isn't enough to write a webapp. You want to be able to take input in the form of HTTP parameters, for instance. (parameter "paramname") will return the value of the (GET or POST) parameter called "paramname", or nil if it doesn't exist.

There's a lot more to it than that, of course, but that's probably enough to get started. Hunchentoot has most features common to normal webservers, and the documentation is quite helpful. Here's a quick example:

(defpackage :testserv
  (:use :cl
    :hunchentoot)
  (:export :start-server))

(in-package :testserv)

(setq *dispatch-table*
      `(,(create-prefix-dispatcher "/test" 'test-page)
     ,(create-prefix-dispatcher "/about" 'about-page)))

(defun test-page ()
  (let ((name (parameter "name")))
    (if name
    (format nil "Hi, <b>~a</b>" name)
    "Name: <form action='/test' method='get'><input type='text' name='name' /><input type='submit' name='submit' /></form>")))

(defun about-page ()
  "<h1>Hunchentoot Demo</h1>This is a very simple demonstration of the Hunchentoot webserver.")

To run this, just compile it, and do (testserv:start-server :port 8080), and go to http://localhost:8080/test

Update: There's a second, more interesting, article on the subject here.

Getting started with Hunchentoot, a Common Lisp webserver

Update: There's a second, more interesting, article on the subject here.

Hunchentoot is a Common Lisp webserver by Edi Weitz. It can be used to serve static pages, but most people use it to write web applications in Lisp. There are a few other Lisp webservers, but it's probably the most capable, most compatible and best-maintained one at the moment.

This isn't a general Common Lisp tutorial. If you want such a thing, I would recommend the excellent Practical Common Lisp, which is available in book form or free online.

Hunchentoot will work on most Lisp implementations, though not clisp. If you're using SBCL on Windows, you should still be able to use it, but it will be single-threaded; so you can't really use it for production websites. It's probably okay for development. SBCL on Linux is probably the ideal (free) production environment; it's fast, threaded, and supports Unicode.

Assuming you have asdf-install, you can install hunchentoot by just doing:

(asdf-install:install :hunchentoot)

Hunchentoot has a list of functions associated with page names called *DISPATCH-TABLE*. It goes through this list one by one running the functions with the HTTP request; the function should return another function, which will be run with the request to produce a page as response. It comes with a few handy functions to match page names by prefix and by regular expression, but you can write your own if you like. A simple dispatch table looks like this:

(setq *dispatch-table*
      `(,(create-prefix-dispatcher "/test" 'test-page) ,(create prefix-dispatcher "/about" 'about-page)))

You can set dispatch-tables per server, but setting the global one is probably the easiest way to start off.

When the server is running, now, going to http://localhost:myport/test will run the test-page function. It should return a HTML string, which will be returned to the user. You can generate the HTML any way you like. There are a few HTML generators by the same author as the webserver; html-template fills values into a template (it's based on an old PERL library) while cl-who converts s-expressions into HTML.

Obviously, that in itself isn't enough to write a webapp. You want to be able to take input in the form of HTTP parameters, for instance. (parameter "paramname") will return the value of the (GET or POST) parameter called "paramname", or nil if it doesn't exist.

There's a lot more to it than that, of course, but that's probably enough to get started. Hunchentoot has most features common to normal webservers, and the documentation is quite helpful. Here's a quick example:

(defpackage :testserv
  (:use :cl
    :hunchentoot)
  (:export :start-server))

(in-package :testserv)

(setq *dispatch-table*
      `(,(create-prefix-dispatcher "/test" 'test-page)
     ,(create-prefix-dispatcher "/about" 'about-page)))

(defun test-page ()
  (let ((name (parameter "name")))
    (if name
    (format nil "Hi, <b>~a</b>" name)
    "Name: <form action='/test' method='get'><input type='text' name='name' /><input type='submit' name='submit' /></form>")))

(defun about-page ()
  "<h1>Hunchentoot Demo</h1>This is a very simple demonstration of the Hunchentoot webserver.")

To run this, just compile it, and do (testserv:start-server :port 8080), and go to http://localhost:8080/test

Update: There's a second, more interesting, article on the subject here.

Update on the Wordpress plugin update situation

A few days ago I mentioned a potential privacy issue with Wordpress 2.3. Basically, when it checks for plugin updates, it sends your blog URL along; Wordpress (.org) then keeps a record of URLs used.

Little attention indeed was given to this, until it was slashdotted. Matt sprung into action, and hey, presto, Wordpress now has a privacy policy! The page describing the new version was also updated to tell users about plugins to disable the update system, though not simply to modify it to work without sending the blog URL, which does work at time of writing.

On the negative side, however, Matt now wants to banish the commoners from wp-hackers, the mailing list where the sending of the URLs was disclosed in the first place. I tend to suspect that the aim is to restrict it to Yes-men, though he mightn't find it all that easy; some very important Wordpress developers joined in questioning the necessity of the URL-sending.

No word on what the URL collection is actually for, as yet, by the way. The privacy policy does at least significantly reduce the list of evil things that could be done with it; in particular, they shouldn't be able to send lists of people and their plugins to advertising companies, or anything like that. The security issue remains; Wordpress's servers have been compromised before, and a list of plugins and version for every Wordpress blog out there could be terribly useful to hackers when plugin vulnerabilities are found.

The whole thing, though, is very much an example of the importance of openness. Now, when various competitors are appearing (Habari) or releasing new versions (Movable Type, Blogger), is the last time that Wordpress wants to go around being shady and alienating people.

Interestingly, also, a number of webhosts (including, apparently, Dreamhost) are already bundling plugins to disable the dubious functionality with Wordpress installs.

Excel 2007 fun

You have probably, by now, heard of the Excel 2007 bug which causes it to think that 850*77.1 is equal to 100,000, not 65,535. Now, lots of Microsofty and ex-Microsofty people, including Joel Spolsky, have explained that, of course, calculated using IEEE floating-point numbers, the result isn't quite 65,535, but none have explained why it is 100,000, that I've seen.

The answer, of course, is obvious. First, it is important to recognise that no-one in the whole world has bought a copy of Office 2007. I mean, why would they? Every new (Windows) version of Office, since at least '95, is effectively a cruel experiment on the users; for each new version they introduce at least one bizarre new user interface oddity. Office 95 had weird gradient title bars, which then made their way briefly into Windows 98 before vanishing. Office 97 had the now ubiquitous flat toolbar buttons which pop up when you move their mouse across them. Office 2000 had a very, very strange pseudo-MDI system; maybe all the windows were the same window, and maybe they weren't. I've never used 2003, but I'm sure it has some similar quirk. And, when it comes to it, a version of office a few iterations behind is quite good enough for most people. I believe take-up rates even of 2003 are a bit pathetic.

With Office 2007, though, they have outdone themselves. It has a really, really strange UI, stranger even than IE7's. I'm quite sure upgrading users would find it terribly confusing, if they existed, which, of course, they don't.

So who does have Office 2007? Why, pirates, of course! Software pirates always have to have the newest version, regardless of whether it's actually any good. And therein is the explanation for the 'special' treatment of ~65,535. Microsoft hopes to bankrupt the large pirates, who make a fortune out of selling copied software, by making their budget spreadsheets meaningless! Clever indeed.

(Obviously, I'm not actually serious about any of this, just because some 'special' people do need that sort of thing pointed out.)

On the prevalence of ad-blocking software

Recently, I noticed an interesting oddity in the statistics for my FindMeATune app. Google Analytics, which should only really report views by real browsers (with Javascript), is current reporting X page-views per day. Google AdSense, on the other hand, is reporting about 70% X page-views per day. Now, as just about every page on the site has ads, and Analytics reports very few people using spectacularly old browsers which can't show Google ads, this is interesting.



My take on the statistics is that many, many visitors are using ad-blocking software. Initially, I wondered was the issue that Google didn't have any ads for some visitors, but I have never recently seen it short of ads to show, and in any case, I think it counts its 'Public Service Ads' in impression statistics. Now, the site isn't in a particularly technologically sophisticated market demographic; I'm really quite surprised at how many people seem to have ad-blocking software.



Please note that I have nothing against people using ad-blocking software, though obviously it'd be nice to have the extra 30% revenue. I'm not one of these website operators who rants about theft of service when people use ad-blockers, though; I have no issue with people browsing the web as they prefer.



It will, however, be interesting to see what Google do if ad-blockers continue to become more common. AdSense is a big part of their revenue stream, after all. Will they, like they are doing with mobile ads now, allow publishers to insert ads into pages on the server side? Will they start using new techniques to fool the ad-blockers? Will Firefox with Google Toolbar start disabling ad-blockers, even?

So, do you have ads on any of your sites? Are you noticing similar discrepancies? What's your take on the phenomenon?

More sinister machinations from good ol' Wordpress

Via the wonderfully named and charmingly written Wordpress Wank:

Wordpress 2.3 is out today. And there was much rejoicing. Now, yesterday, someone on the Wordpress development mailing list noticed that the new plugin update notification server sends a list of your plugin names and versions, and your URL, to a server.

Cue confused core developers, much chatter, etc. It was, of course, Matt's doing, as with just about every silly development decision on Wordpress these past few years. I have mentioned previously, of course, the lovely not-at-all-connected-to-advertising BrowseHappy link.

And why, exactly, is the information collected? "...It could be useful in the future." Thanks, Matt. I'm sure that really reassures the users. He then advises people who don't trust Wordpress.org to fork the project. I think, though, there's a big difference between trusting Wordpress the open-source, reviewable project, and trusting Wordpress.org, that shadowy, Matt-dominated extension of Automattic.

There are actually a number of legitimate reasons for concern. First, if the data were ever leaked or stolen (bear in mind that Wordpress.org's webserver was compromised last year), then someone would have a list of plugin versions with URLs! This is a moderately big deal; all they'd then have to do is search plugin changelogs for security fixes, and they'd be able to produce a big list of vulnerable blogs!

And then, there's another issue. Matt hasn't said how this data will be used. Automattic notoriously isn't keen on terms of service; Wordpress.com only got them quite recently. Matt has previously been willing to engage in slightly shady advertising techniques (Wordpress.org spam link controversy, BrowseHappy ad with people using IE shoved towards BrowseHappy, those ads that no-one knew were there on Wordpress.com, etc.) From the plugins a user uses, you may be able to derive a fair bit of information about their hobbies and so forth. "Oh, this person has an animated cat in their an admin area and a plugin that displays the rugby scores!" Considering their history, I'd be reluctant to give Automattic any more personal information than absolutely necessary.

There are, by the way, patches to take out the URL-sending mentioned on that mailing list topic. If you are upgrading, I would strongly recommend you apply one.

We'll end with a little humour, I think, today. Take it away, Matt:

I would like to remind the participants of this thread that WP.org !=
Automattic, so to be fair to the members of both please distinguish
which you're referring to.

My run-in with blog advertising

Yesterday, Damien Mulley posted about the ethics of having advertising on your personal blog. It provoked a fair bit of discussion, mostly in defence of ads.

Now, I have no issue with people having the odd normal ad on their blog. It'll help them pay for their hosting, and I'd really rather it was their ads there than Automattic's, say; it's a little-known fact that Wordpress.com blogs show ads to infrequent users.

Where things start getting a little more dubious is where cretins like PayPerPost turn up. PayPerPost is a broker who allows advertisers to pay bloggers to post about their products. The blogger doesn't have to have used the product, there is no need to point out that the post is actually an ad, and advertisers are allowed require people to provide positive reviews. This, to my mind, is awful, and it would greatly diminish my faith in any blogger if I knew that they were using something like it. If I read a post along the lines of "I found the MaragretTron 8000 to be the ideal solution for levitating my cats", I expect it to be true (okay, maybe I don't expect THAT one to be true), not something that the blogger was bribed to say.

Anyway, oddly enough, this morning, I got an email from a maker of a weight-loss product asking would I like to post about it: "I was hoping you would consider mentioning the product in your blog, potentially as a story around innovative new tools for successful living healthfully. "

Well, I'll do my level best, even if possibly not in quite the way he intended!

The product (rel=nofollowed for your safety and comfort; I like to think of rel=nofollow as the condoms of the Internet, don't you?) is basically a weighing scales, as far as I can make out. With an infomercial by a fat person. Now, I don't know about you, but if I was attempting to lose weight, I would buy a weighing scales which isn't endorsed by a fat person.

Would I recommend that you buy it? Good lord no. Not only because I think it's a bit silly, but because the manufacturer sends out unsolicited emails looking for misleading advertising on blogs.

The current advertising policy for this blog, by the way, remains "no, thank you".

And I'm sorry, but I never write anything about "successful living healthfully", or any other such nonsense.

Facebook in Common Lisp

Well, yesterday I started writing a silly little demo application for Facebook, using cl-facebook. It isn't quite finished yet, but I'll probably get it up fairly soon.

While cl-facebook provides most necessary functionality and makes the whole thing far easier, it isn't quite, by itself, sufficient for creating Facebook-hosted applications. In particular, it doesn't handle Facebook's login and session stuff, so I had to do that myself, figuring out how it works from the (sparse) documentation and reading the source of the Facebook-supplied PHP library.

In addition, cl-facebook lacks a few of the functions supplied by the API. I may, possibly, release a patch for the library, and also some sort of simple framework to handle the whole scary login thing.

All in all, though, it was quite interesting, and not really all that difficult.

Stephen Fry, now with blog!

Mark informs me that the wonderful Stephen Fry now has a blog! His first post is a quite absurdly long ramble about smartphones, but fairly interesting nonetheless. I do hope he keeps it up.

Even better, he could do a podcast! Saturday Night Fry series two at last, perhaps?

The comments on that entry are interesting, actually, in that they are actually vaguely coherent; most celebrity blog comments are left by insane people; see Scott Adams' blog for ample example.

Also, the (brand new) blog really needs a little more configuration; it's a stock Wordpress install with post-id based URLs, and the telltale irritating pause on page loading that tells us that wp-cache or similar is not in use. I hope it doesn't die of over-popularity...

SILENCE, Cathedral!

One of my local cathedrals, I think St. Patrick's (it's the closer, and therefore louder, cathedral) has been indulging in a spot of change ringing for the last hour or so. I've always quite liked this from the distance, but this close it's a little distracting.

Also, I found yet another of those Deo Optimo Maximo Sub Invoc Someone churches on Francis Street. This one, in fact, has such a long spiel on it that the inscription is abbreviated to DOM SI or something.

DOM SI. Lord Metric System.

SSH Remote Forwarding - a quick guide

Use of ssh's local forwarding is common enough, for security or firewall evasion purposes. It lets you forward a local TCP port to a port elsewhere, tunnelled through your ssh server. Remote forwarding is a little more obscure, but still very useful.

My problem:

I am currently playing with writing Facebook applications. I'm using Common Lisp with Hunchentoot and cl-facebook as a development platform. It's nice to have this setup running on my laptop, as I can make use of Lisp's powerful debugging features, and load new program code with ease, but to test applications written for Facebook, you really need to have them on the Internet, so that Facebook can access them. With a DSL connection or similar, there would be no issue; I'd just forward an incoming port to my laptop from my router. However, as I've previously mentioned, I'm currently using a 3G mobile phone service modem to connect to the Internet. Three, my service provider, doesn't give users a unique IP, and it certainly doesn't allow them to open incoming ports.

Solution:

SSH remote forwarding allows you to have a port on a remote server forward to your local machine. As such, it's perfect for my needs; all I need to do is to forward a random port on my server to my Lisp webserver port (generally 8080 or similar) on my laptop. Sounds simple, eh?

Well, there's a little more to it than just that. The command "ssh -R 10000:localhost:8080 <myusername>@<myserver>" should do the trick... except that for vaguely recent versions of ssh, this will cause the listener on the server to bind to the loopback interface only, so it won't be accessible from the Internet. There are good, sensible security reasons for this, but it isn't what I want in this case.

A quick google would lead you to believe that adding the flag '-g' would be sufficient. In fact, at least in my case, it wasn't; you also have to tell the ssh server that it's allowed open ports on all interfaces.

To do this, you have to edit your sshd_config file, in my case /etc/ssh/sshd_config, and add the line "GatewayPorts yes". You then have to restart the sshd server.

Once you've done that, the command "ssh -4 -g -R 10000:localhost:8080 <myusername>@<myserver>" will indeed open a port on the server pointing back to your local machine. (EDIT: At least for a MacOS computer, you must specify IPV4, or things will break.)

This, at any rate, works for me; your milage may vary, especially if you're using a very old or unusual client or server. It should work with the big Windows ssh client, putty, though; it's an option in the GUI forwarding/tunnelling config page.

One other minor problem I was having was that some router along the way would close idle connections. The fix in this case is very simple; you just have to add "ServerAliveInterval 60" to your ssh_config file on the client side; in my case, this is /private/etc/ssh_config. This sends a packet or two every 60 seconds to let routers know that the connection is active.

I can't haz Web 2.0?

The following is Digg's error message for when it can't access its AJAX comments thing:

Yes, really. I think my Internet connection was acting up.

I was, incidentally, looking at a hilarious digg; it was an enthusiastic blog article about Wordpress 2.3; the (apparently popular and high-traffic) Wordpress site went down with database errors after a hundred diggs or so.

Peter Seibel in Google-annoying shocker

Peter Seibel, who wrote a very good freely available book on Common Lisp, wants people to link to it, using the phrase Common Lisp Tutorial; apparently the top Google searches for that currently are a bit awful.

Okay, then, but don't come crying to me when you're summoned in front of some Google Inquisition(beta).

Google Presentations

Google has just released a presentation thing as part of its Google Docs package. It's a little basic, but actually surprisingly good. It can import Powerpoint presentations, and its presenting mode is passable. If I had to put together a quick presentation, I'd probably consider using it. It wouldn't be great for big flashy corporate things, but I can certainly see its advantage for people who just want a simple presentation quickly.

MS Office, and the dreadful OpenOffice, look less attractive every day... I seem to remember, though, that when the whole Google Docs thing started, Google assured everyone that they wouldn't be competing with MS Office.

Three 3G broadband service

I recently moved out. Now, of course, I am ludicrously dependent on having an Internet connection at all times, so needed one in new flat. DSL wasn't an option; the building is quite new and telephone hasn't been connected yet. Cable ditto. I've used Irish Broadband's wireless Ripwave service before, and hope never to again. There's something similar called Clearwire, and I hear that it gives just as stellar performance. That left only the rather weird mobile phone Internet providers.

In recent months, a few companies have started providing broadband Internet in Ireland over the mobile phone networks, using HSDPA, a 3G technology which can allegedly provide up to 3.6 Mbit/sec down (Hah). The cheapest and most basic service is provided by confusingly-named Three; it costs 20 euro a month for a 10 gigabyte cap, but is 3G only; most of the others can also operate on an old GPRS network. Three claims 85% population coverage, and seems to have more or less full coverage of Dublin, which is, after all, what is important. 20 euro a month may sound absurdly expensive to those of you who live in countries with decent broadband, but believe me, in Ireland, land of the low-availability 50 euro DSL connection, it's pretty decent.

To get Three's broadband package, one must go into a shop and have a credit check. This would have been a wonderfully simple and quick procedure (though they do require a passport or driver's license and a utility bill) except that the shop I chose had a lovely ink-jet printer which could produce roughly a page per five minutes. It's a 12 month contract, and the access device (of which more later) costs 129 euro. There's a 14 day money-back thing, in case you're not satisfied with performance.

Setup is fairly easy. You insert the SIM card into the device. If you have Windows, you just connect the device and all is well. If, like me, you're using MacOS, you must first install a driver, which can be downloaded from Three's site. The function of the driver seems to be more or less to switch the device from disk mode (the USB drive contains the Windows drivers) and install a modem profile. You then set up a simple PPP connection, and it works. It seems to be more or less a high-speed USB serial device; setup on Linux involves a bit of magic to shift it into modem mode, followed by an ordinary PPP connection.

You can, and should, if you have a Mac, get a little bit of freeware software here. It lets you monitor the signal strength, tracks transfer usage and so on.

The device, bizarrely, seems to contain a battery. My guess at the reason for this is that 3G chipsets are generally designed for phones, and are expecting nice smooth battery power, not dodgy USB power, and using a rechargable battery was a cheap option. In any case, I have seen warnings that speeds for the first couple of hours after you get the device are not great, and my own experience goes along with this.

Once connected, speeds are actually generally reasonable. I've seen it get up to 1.5Mbit/sec, and normal speeds are in the 500Kbit/sec range; this is with not-very-good signal in Christchurch, where there are no doubt a lot of users. Latency is quite variable; about the lowest I've seen is 100 milliseconds to sites in Dublin. Interestingly, however, even when ping times are up in the 500s, ssh is generally quite responsive. I'm unsure as to whether they are prioritising ssh, de-prioritising ICMP, or something else.

And that brings me to another issue. Speeds to certain sites seem to be throttled at peak times. The other day it was crawlingly slow accessing most Google sites (Reader, AdSense, Analytics, etc.) but my blog, for instance, was moderately snappy. I suspect this may be part of the War on Youtube.

It also generally seems to take a while to establish a connection to a given site; once it's established, the connection is quite fast. In addition, downloads behave interestingly; the speed tends to ramp up over the first few seconds of the download. I suspect something to do with 3G adaptive capacity is going on.

Occasionally, if it loses the HSDPA signal, the device will switch into conventional 3G (the light changes from light blue to dark blue) and speeds, and especially latencies, do fall significantly.

I plan to try running a compressing HTTP proxy on a server I have, and seeing how it effects speed (and transfer usage). If successful, I will report back. A caching proxy on the client side might also be useful.

Another issue is sharing. Obviously, you can't just plug a random USB device into a random router and expect it to work. Apparently, it does work with a number of routers which have USB ports and replacable operating systems; as previously mentioned, it provides a fairly ordinary-looking USB serial device. Also, there's at least one router with semi-official support out-of-box. Sharing from a computer is also a possibility, and one I am going to try.

Finally, the thing is, of course, pretty portable. You can use it anywhere that Three's network exists; there's a more expensive version that has a lower transfer cap, but can roam on GPRS on other peoples' networks. I can see that this could be incredibly handy for people who travel a lot, or just want to pull out a laptop and check their mail from where-ever. You can also roam to Three networks in other countries for no extra charge.

So, would I recommend it? Well, it more or less fits my needs; browsing, email, IM and ssh. Fixed broadband would obviously be better, but fixed broadband is not available everywhere, and is generally a bit more expensive. It's certainly better than the horrible Ripwave service. However, it's certainly not for everyone; if you want to do lots of downloading and play online games it's not going to be great. Also, there have been various service problems; more discussion here and here. I think a certain amount of the criticism relates to unrealistic expectations, common in the Irish telecoms market. There are also issues with using SMTP and some VPNs, and there were major service disruptions during an upgrade last month. However, when it comes to it, it's my only sane choice for Internet access at the moment, so I don't think I'll be taking advantage of the 14 day money back clause.

Vital information left out of news story

From The Register:

A federal judge has given Sprint Nextel initial approval to pay $30m to settle a class-action lawsuit accusing it of colluding with other phone companies to overcharge customers. Qualified customers will receive prepaid calling cards from a cache worth $25m. Attorneys get the remaining $5m for fees and costs.

But the big question is, are the attorneys receiving a five million dollar prepaid calling card? If so, it it possible to exhaust such a thing before its expiration date?

From the QA department in work

In Soviet Activision, games test you!

(Sorry, I couldn't resist.)

Quick and easy SVN server

SVN (occasionally called subversion, mostly by the authors) is a popular revision control system. A revision control system is a system which allows you to keep track of development work on a software project; every time someone makes a change and commits it to the system, it is stored; thus it is made easy to revert to any given version, to see who has made what changes, and so on.

I've used various revision control systems for work, college and open source projects for the last five or six years; I seem to remember uploading my webcomic archival system to a CVS repository on Sourceforge over dial-up being a particularly painful experience.

However, I had never previously considered having one for personal use. Now, a revision control system is an absolute must for collaborative development; I'd be surprised if there are any development outfits which don't use one, at this point. The benefits for individual development are a bit less clear, but they are there. You can look back on changes you've made, create patches against release versions, you have a full backup of everything you've done, and so on. I decided, at any rate, to give one a go for my personal development projects.

I decided to go with SVN, as it's more or less what I'm used to. I realise that it is now, to an extent, old-hat; something called GIT has apparently gotten very big. I'm happy with it for the moment, though; I can always migrate. Now, there are surprisingly few SVN hosting setups, and most of them don't seem to be geared towards the 'personal project' market. They're pretty expensive, usually, and have all sorts of collaborative features. There are many open-source SVN hosts, of course; Google Code, Sourceforge, common-lisp.net and many, many, many others provide SVN hosting for open-source projects. These don't fit my needs, though; I want to keep track of personal projects, some of which are closed-source or not for release.

My only real option, then, was to set up my own server. I currently rent a server from a company called Hetzner; it hosts my blog, my reverse lyrics search engine, and various other bits and pieces, so I decided to put it there. Of course, depending on your needs, you could probably quite happily host an SVN server on any machine with a permanent Internet connection (dyndns or similar is your friend here), or even on a non-Internet connected machine if you do most of your development on one machine. I do develop on more than one computer, though, and it would be nice to be able to browse the repository from whereever.

The first step is, obviously, to install SVN on the server. If you're using a Debian-y Linux, just do 'apt-get install subversion'. Now, decide where you want to put your repository; it probably doesn't really matter. Create a user for the SVN server, and do, as that user, svnadmin create /var/svn (or whatever). You may need to mess with permissions a bit. This gives you an SVN repository.

Next, you need to set up a server. There are a number of options, but the simplest is probably to use the 'svnserve' server which comes with the SVN distribution. If you have hundreds of users this may not be a great idea (there's an Apache module which might be better there), but it's fine for personal use. This is really easy. Edit /var/svn/conf/svnserv.conf, setting anon-access to 'none' (or to 'read', if you want anyone to be able to check things out of your repository), and auth-access to 'write'. Set password-db to whatever you want to call your password file, and edit that file, putting in:

[users]
username = password

Then start the server as the SVN user, by doing "svnserve -d -r /var/svn". You should now be able to import things like so: "svn import dirtoimport svn://myserver.com/".

There are various security options, like encryption and higher granularity of permissions, but I'm not really that worried that someone is going to steal my precious source code, so I didn't bother with them.

The last, optional, step is a web-based repository browser. I find these terribly useful, but some people don't care for them. Their loss. I favour viewvc. It's pretty easy to install, and all you need is Python, the SVN Python bindings, and a webserver that does CGI. If speed is a big issue, you can use FastCGI, but realistically for a single-user system CGI is probably fine. Obviously, you should put a password on the directory with .htaccess or similar, unless you want everyone to be able to view your repository.

Quick introduction, but I hope it's useful, and I certainly would recommend that you use a repository for your personal projects; it's quite handy.

The downside to hosting your own blog

As I've previously mentioned, I recently moved from a Google-hosted Blogger blog to a self-hosted Movable Type one. Mostly, I'm pleased with the move. However, I just checked the bandwidth usage today; this blog apparently chews through about 150 megabytes a day! And it's not even a popular blog, and Feedburner reduces usage a bit; I'd hate to see figures for a multi-thousand subscriber one.

My reverse lyrics search engine thingy gets through 3.5 gigabytes a day, but it is, at least, a popular site, so it's understandable.

When I first had a blog (a Wordpress one, a couple of years ago), it got through about a hundred megabytes a month, on a gigabyte per month hosting plan; of course, no-one really read it. I'm just glad I'm not on that data plan any more!

Internet sharing for the 20th century

Today, I saw an apartment which has a couple of thickish cables, which I suspect to be CAT5, coming out the window and into others. It's not clear whether all the windows belong to the same apartment.

You'd think that they'd simply buy a wireless router.

In any case, they should be careful; the larger cable isn't far above head height for people walking below.

More fascinating tales from my life

I bought a pair of shoes today! This counts as news because I have an irrational fear of shops, especially clothes shops; shoe shops are a step worse because you actually have to ask for sizes and such. But for the first time in months I won't have to worry that puddles will attempt to enter through my sole, which is nice.

This is the strange thing about blogging; you can constantly go on about tedious nonsense, and people will STILL read it! I mean, I have about 80 or 90 readers, and my posts are generally on the same level as today's (I saw an odd packet of fags and bought new shoes). So what do I have to do to drive readers away? Set fire to them?

SECRET Cigarettes

On Saturday morning I chanced to be in a train station, and there, lying on the ground, was an empty box of "John Player Bright Blue" cigarettes. Now, John Player Blue is the brand of cigarettes Mags, a fictional character invented by the two Marks and myself, smokes; she has 40 of them with a pernod. I'd never seen these ones before, though. The box was a fetching blue colour, far prettier than normal cigarette boxes, which tend towards the metallic or brightly-coloured. They bear a logo which looks like the 'By Appointment to the Queen' one, but isn't; these are not the Queen's fags.

The interesting bit is this. The search phrase "John Player Bright Blue" brings up precisely five webpages on Google, all Irish. And it's not like they're something weird and new; one of the pages is a large PDF from 2005. It is a market that has a lot of flops, I suppose; what with restrictions on advertising, it is presumably difficult for any cigarette brand that turned up after the ban to gain much of a mindshare.

So there. This post will now corner FULLY A SIXTH of web searches for a cigarette that nobody smokes or talks about. Yay! I realise at this point that I should have taken a picture of the box, to prove I wasn't making up brands of recreational drug. This smaller picture, from one of the websites, will have to do.

Ewh

When, precisely, did it become socially acceptable to walk down a busy street in the middle of Dublin with your hand down the front of your tracksuit bottoms?

The things people search for (NSFW)

Not that much of my blog is safe for work at the best of times...

One of the nicest things about having a blog is getting to see exactly what people typed into google to get there. Below are a few highlights.

"deo optimo maximo sub invoc"
I mentioned the church on Westland Row at some point; it has this written on it. Nine people this month have searched for it.

escortireland
A prostitute rating site I stumbled across a few years back and made fun of. Tragically closed now, I think; it really was very funny.

ravished bride
Eek. No idea where this one cames from.

holiometer
what is holimeter
A device for measuring holiness, obviously. My blog is currently the only site on the web to mention it!

preteen sex
Argh. I suspect that this is actually finding a piece of comment spam on my blog somewhere.

"rs-24"+"mirv"
Rob's blog, for all your missile needs.

alexander abian
archimedes plutonium
Also, for all your mad scientist needs.

betty boob
What? Is this a variant on 'Betty Boop' that I don't want to know about?

crossdressing
drag queen
Google seems to have decided that I'm an authority.

fetysz gallery
I don't even know what that is.

free orgazime video no download
That's quite the most creative spelling of the word 'orgasm' that I've ever seen.

homeporn
safe granny porn
sexy teachers
moviesex
Internet people are weird. Really weird.

"in the twenty years since the chernobyl tragedy, the world's worst nuclear accident, there have been nearly [fill in alarmist and armageddonist factoid here]"
This text, rather wonderfully, appeared in a Greenpeace leaflet at some point.

lisp -emacs -autocad -autolisp
Surely typing 'common lisp' would have had much the same effect?

lockable toilet roll holders
I mentioned that we have these at work a while back. It wouldn't have occurred to me that they were the sort of things that people would actually want.

loli non nude
Can we have a 'you have to be less disgusting than this' rule for the Internet, please?

madeleine mccann and freemasons
Warning; Madeleine McCann may be a freemason. Do not, under any circumstances, shake hands with her.

mp::startup-idle-and-top-level-loops rsynnott
What?! 'mp::startup-idle-and-top-level-loops' is the incantation required to make CMUCL's multitasking work properly, but I can't understand why anyone was searching for that with my name...

mrs wobblebottom
I'm afraid I have not yet married.

newest penpal looking for male love free
What on earth?

nude webcams
The Victorians, of course, required their webcams to wear trousers.

pictures of how to use a condom safety
I'm afraid I don't think my webhost would be too enthusiastic. Anyway, the camera lens would crack.

strit poker
Is that like strip poker, but for dyslexic people?

telehouse plot
Ah, that'll be the freemasons again. I suspect that this actually referred to an alleged plot to blow up Telehouse, which houses a considerable fraction of the UK's Internet infrastructure.

transsexual woman -lgbt -feminism
I just want weird porn, damnit, none of your politics!

yay for killing babies
I don't think I like the Internet very much any more.

nude by accident
nude accident
What sort of accident, precisely?

The worst of it is that there are a few that are so horrid, I don't particularly want to mention them. The Internet is a strange place.

So, do you get interesting crazy perverts searching for your website? Why not detail your own incoming searches?

I'm a Complainer, or, don't mention Madeleine

Well, people do use the Internet in interesting ways! A while ago, I wrote a post in which I was a little sceptical of the unreasonable level of media attention that the Madeleine McCann disappearance case was receiving. Such coverage, by the way, had died down a bit, but is back with a vengeance now that the parents have been made official suspects.

Anyway, I got a number of angry comments from people offended by my article. Of course, people have a right to be offended and the whole issue of the media treatment of this sort of thing is a divisive one, but I couldn't help noticing how those people who complained made their way to my site. They found it by googling for blogs mentioning Madeleine McCann.

This seems, to me, strange behaviour; were they just touring blogs complaining? The complaints weren't particularly constructive or interesting; they were of the 'how dare you have a different opinion to mine' variety, and few of those who took the time to leave a comment could spell. It was also clear that none of them had actually bothered to read the article; they seemed to be working entirely off the title.

The title is from a Saturday Night Fry sketch in which a woman is interviewed; when asked what she did she responded: "I'm a complainer. I write letters of complaint to the BBC about things. I shall be complaining about this show if I hear it, I expect. Even if I don't."

Yahoo! apparently unaware of comment spam

As I mentioned yesterday, I have been recieving blog comment spam from a Yahoo-owned IP block. I complained to Yahoo, and they actually got back to me!

That was the good news. The bad news was that they said that they couldn't do anything unless I sent them email headers. For comment spam. Right.

Is it any wonder that they've lost to Google, really?

Horrible pervert reports media player bug

Someone recently reported a bug in one of Gnome's media players. Note the list of videos he was watching.

I hope that this is a deliberate spoof, but somehow I don't think it is.

Ewh. Never using Gnome again!

Apple shooting self in foot a little?

Today, Apple brought out the iPod Touch, effectively an iPhone sans phone. Wi-Fi, web-browser, and everything. Now, they've knocked the iPhone prices down $200 while they were at it, but still, for many people it would seem to make more sense to buy an iPod Touch and a nice-ish phone that can do 3G and Wi-Fi (and thus have the phone provide the iPod with wireless while they're at it) than to opt for the iPhone. Remember, at least in the US, the iPhone is only available on stupidly-expensive service packages.

For those who can't live without carrying around a few weeks of music, there's also the iPod Classic, sized at 160GB. Oh, and there's an iPod Nano that's let itself go a bit.

No word on a proper SSH client yet, though; without one, I'd be reluctant to consider the iPhone/iPod Touch.

Interesting source IP addresses for spam

Every day, I get plenty of comment spam. It doesn't generally make it through the filters, but I had a quick look at some of it today (link spam for drugs). Out of curiousity, I clicked on the IP address of one of the senders (69.147.76.36).

It brought me to the website of a company called AdInterax, now owned by Yahoo. They seem to own the block.

Now, I doubt the Yahoo subsidiary is actually sending the spam itself, so presumably some of their machines have been compromised. Either way, it really isn't good enough. I complained to Yahoo through one of their forms; we'll see how that goes.

Browse Hippy - no more ads for Wordpress.org site

I mentioned a little while ago that BrowseHappy.com, a site operated by Wordpress (.org? Automattic? Unclear...) had a Google affiliate ad for Firefox with Google Toolbar; the person who publishes said ads gets a dollar or so each time someone downloads and installs Firefox through them. As the site is linked to by every installation of Wordpress when accessed using Internet Explorer (in the administration area), and Matt, head of Automattic and maintainer of Wordpress, had vetoed the removal of said ad, I thought this was a little dodgy. The ad had been there for some time, by the way. The issue is also discussed here.

Anyway, the ad has now quietly disappeared, replaced, for some reason, by a picture of Firefox's tabs. Yay! I think that Matt should still let people know just what the money was used for, but it does at least remove the conflict of interests. Here's what Matt has to say on the issue (last comment); amazingly enough, the income from the ad was so low that it wasn't noticed.

I wonder will he be more amenable to removing the link from Wordpress now? Interestingly, he just blogged about how he likes IE7 today.

Actually, it turns out that I wasn't the first one to notice the attempt to drive advertising; someone commented on it a few months ago.

Another Irish Rail Oddity

I just noticed, re-reading my previous post on Irish Rail's approach to the Irish language (I only really saw because someone commented on it) that the no smoking signs mention a fine in pounds, not in euros. 

These are on trains, mind you, which were only really introduced when the euro was coming in, so there's really no excuse.

As an aside, the Luas (not a CIE-operated service) has a similarly weird approach. First, there are place-names translated in improbable ways; St. James' Hospital is written and pronounced by the machine as "Ospidéal St. Séamas", which isn't right. Second, the disembodied robotic voice which announces stations will occasionally abandon Irish altogether, simply repeating the English.

The joys of standardisation

I don't know if you remember, but at one time there were three different mutually incompatible types of 56kbit/sec modem. Sometimes, one modem could do more than one protocol, or could be changed to do another through firmware replacement.

Anyway, I just saw this post from Kibo about it. And he's right. They did indeed make different noises, and the v.90 did go 'boing, boing' in the middle of connecting. It was far more interesting than any modem noise which went before. The v.90 noise, incidentally, is what is always played in vaguely recent TV and movies when someone connects to the Internet. They don't have DSL on television.