Monday, April 25, 2016

Capacity units used by Amazon DynamoDB's Scan operation

Amazon DynamoDB's pricing model is based on provisioning a table with read and write capacity units per second. For basic operations, this is simple enough; a put takes 1 write capacity unit per kilobyte of data, rounded up, a get takes 0.5 read capacity units per 4 kilobytes of data, rounded up, or 1 unit if a strongly consistent get is required. It's made less clear by the varying block size per operation, and the way that prices for all this are quoted ($0.0065 for 10 units write capacity, $0.0065 for 50 units read capacity...), but it's at least explicit.

So, reading a 10 byte record costs 0.5 read capacity units, reading a 3999 byte record costs 0.5 read capacity units, reading a 4001 byte record costs 1 read capacity unit.

Things get a bit less clear when it comes to scan, which reads through the table in bulk. The documentation isn't at all clear on whether costing per record applies (with the rounding up behaviour seen in get), or costing per byte. The Internet doesn't seem to be too sure, either; there are stackoverflow posts etc. supporting both positions. It would make a huge difference to costs for applications with small records (and it's worth noting that the export data pipeline that Amazon recommends people use for backups uses scan...). If you have 1000 records of 10 bytes each, then if charging per record applies, that's 500 units, if charging by byte applies it's one-and-a-bit units. So I thought I'd find out for myself.

Here's a little test app (as an aside, have you ever seen a more obfuscated interface than the DynamoDB BatchGetItems one? It's not documented either, as far as I can see...)


Finished populating
For batchGet of 100 count of 10 byte records (1000 bytes), used [{TableName: ScanTestTable,CapacityUnits: 50.0,}] units
For scan of 100 count of 10 byte records (1000 bytes), used 0.5 units
Finished populating
For scan of 100 count of 100 byte records (10000 bytes), used 1.5 units
Finished populating
For scan of 100 count of 1000 byte records (100000 bytes), used 12.5 units

As expected, the batchGet (just a bunch of gets bundled into one request, in effect) behaves just like get, and costs half a capacity unit per record. The scan, however, does end up charging based on total byte size of the operation (rounded to 4kB), ignoring the number of records involved.

So, there you have it; scanning costs 0.5 read capacity units per 4 kB. This means that to provision a table to scan at 1MB/sec requires 125 capacity units, which costs $0.01625 per hour. Not too bad.

Thursday, February 11, 2016

An important message from Dear Leader Creighton

I have just been honoured with a glossy campaign leaflet from Lucinda Creighton, the glorious leader of Renua, in my letterbox. What has she to say?

Hard working mothers and fathers still pay at least half of what they earn in income tax! This is indeed startling news. Now, according to KPMG's handy tax calculator, if you're a couple, you start paying more than half what you earn at about the 770,000 euro mark (assuming you don't have a pension). So to all of you mothers and fathers out there earning less than that, sorry, you're not hard-working. Must do better. As to all of you who are neither mothers nor fathers, well, really, what more can be said?

It's good to know where Renua's definition of 'hard-working' falls, though.

Sunday, July 12, 2015


So after eight years on reddit, I've finally deleted my account:

I should have done it a lot sooner.

When I joined, reddit was a reasonably small nerdy link/discussion thing. But already the rot was spreading. Within a couple of years, well, this happened. It largely flew under my radar when it was still a thing, and by the time I heard about it, I thought "well, that's fucking horrific, but at least they got rid of it".

And I started looking outside my nerdy subreddits, and more at reddit as a whole, and, well, there were many more horrors. And, for whatever reason, I stayed, though increasingly, I was sticking to the areas inhabited mostly by, well, reasonably decent people, who were looking in disgust at the rest of it. 

And in the last year or so, things seemed to be getting a bit better, or so it seemed to me, or at least so I convinced myself. The opinion on the meta-subreddits had turned heavily against the racist, misogynistic, homophobic content. Some of the more visibly awful subreddits had been banned. But still, the community was fatally flawed; the 'default' subreddits, which all new users are automatically subscribed to, were astonishingly hateful.

And I think I, and many others, wanted to believe that this was a small minority of troubled people who would grow out of it.

And a few days ago, reddit got a new CEO. And this is what finally convinced me to question what I was still doing with an account at this website:

(For context, FPH was FatPeopleHate, a subreddit dedicated to hating fat people. It was banned not because, er, that's fucking awful, but because it was harassing people offsite. 'coontown' is a large white supremacist subreddit; it's not being banned because, hey, it's a well-behaved white supremacist subreddit.)

Yes, the white supremacist subreddits are fine to stick about, as long as the don't make too big a deal of harassing other users.

I should really have realised this a long time before. Reddit has a policy of 'free speech', of allowing almost any communities as long as they do not break a few extremely basic rules. Well, that's fine, it's their website. But seeing it said like this, by the new CEO, I don't know. I don't think I want anything further to do with this site.

As I say, I should have realised I needed to leave a long time ago. I wanted to believe that the people running the site were basically well-intentioned, though, if under-resourced and incompetent. I can certainly no longer believe this.

Reddit now contains what are likely the internet's largest white supremacist communities. It almost certainly contains the largest Internet misogyny communities; its MensRights and TheRedPill and similar subreddits are huge. It's the largest gathering place for the reactionary 'gamergate' movement. And perhaps more damaging, beyond these hardline subreddits, racism, misogyny and homophobia pervade the entire site. Sure, the more obscure subreddits, and the 'social justice'-oriented ones (on reddit, social justice is a pejorative term meaning 'not a Nazi') are largely clean of this, but it's all over all of the large subreddits. There are a lot of impressionable young people, who are using reddit and seeing all of this as normal.

I can no longer be a part of this. If you're a reddit user, you should really ask if you can, either.

I'll miss some of it; I'll miss ShitRedditSays, which existed to call attention to awful reddit stuff, and SubRedditDrama, which in its later years, in practice, largely did the same, and the badhistory and badscience and so on subreddits, which, well, mostly called out people using fake history and science on reddit to justify their racism and misogyny, and r/dentistry, which helped me with some of my teeth-related hypochondria and appears to be Nazi-free. But really, this is something I can no longer be a member in. I will have to find another way to waste my time, I suppose.

Sunday, May 24, 2015

Major Kira on Gamergate

From Star Trek DS9:

She's actually talking about the Ferengi, of course, but really, it's perfect for Angry Gamer Babies.

Thursday, November 27, 2014

Better living through TERRIFYING EYE SURGERY

So, for a long time, now, I've been suffering from strabismus. This causes double vision, makes stereoscopic vision largely impossible, and causes me a fair bit of discomfort when looking at things at a certain distance; lately this has particularly been a problem for computer monitors. It also makes me look a bit odd.

For a good while I didn't realise this was what the problem was; however, when I finally went to Specsavers for an updated prescription, they saw something was wrong and referred me to an ophthalmologist. In due course (about a year; I was lazy about driving the process along, because it was scary) this lead to, well, this (WARNING, unusually graphic images for a Wikipedia surgery article).

Before (note eye orientation):

After (note improved eye orientation, blood.

And here I'm appearing as a rather low-budget Borg with the eye-protection thingy that I must wear when sleeping.

End result; a bit of pain, a fair bit of discomfort (feels like there's something in my eye; this will persist until the sutures in my eyeball dissolve in a week)... and more-or-less working stereoscopic vision! Much though I was dreading it, it really didn't work out badly at all (so far; there's always the possibility that my eyes might just drop out or something, I suppose).

Thursday, October 2, 2014

Water, water, everywhere

I was just signing up for Irish Water, our exciting new paying-for-water thing.

From the application form:

Now, you might think that that means that if your phone number is, say, 087 123 4567, you should fill in '087 123 4567'. Let's try.

Okay, then. So, it's like this, then, is it?

Yes, it is.

Also, you have to click on the buttons twice.

Your water fees at work! I suppose on the plus side it's very shiny and Web 2.0; it just doesn't work properly.

Monday, February 10, 2014

Waters Under the Bridge

One odd thing about the coverage of the recent Iona/John Waters controversy is that it has been, pretty much entirely, Internet-led. Each new development has made the rounds of the blogs, Twitter, Broadsheet and maybe The Journal, before being grudgingly, if at all, picked up by the major media outlets.

Witness the horrifying John Waters tape released by Broadsheet. As far as I can see, none of the traditional media outlets have gone near it, but it's all over the Internet; there's a lot of public interest. Of course, a certain amount of this is "haw, haw, John Waters is even more terrible than we thought", but it really is quite important. This is a man who has a tremendous platform to talk about society, who is referred to as a 'public intellectual', who has a column in the Irish Times, who is a go-to whenever a right-wing opinion on a social issue is required. Until very recently, he was a government appointee to the Broadcasting Authority of Ireland. His views on gay people, and on women, as revealed on the tape, are absolutely relevant to these roles. The question of what words it is proper to use when discussing these opinions is also, it seems to me, fairly important.

He also, really, is entitled to a reply on this. It's so awful that if there's any extenuating circumstance, that needs to be made plain. Has anyone even asked him about it, at this point? It's really very strange.

Of course, there are conflicts of interest. The RTE is engaged in a dispute with their former regulator over the question of which apologies and settlements were and weren't offered, and Waters writes for the Irish Times (though, he has at time of writing vanished from their columnists page; Iona's Breda O'Brien also briefly disappeared, before returning triumphant with a lovely piece on stifling of dissent through unkind descriptions, finishing with a bit about "commissioned" children, presumably to emphasise that the stifling isn't working very well). I'm not sure what the Independent's excuse is, though.

Perhaps, the media hopes that this will all blow over, and that they can have him back writing his reactionary columns and ruminating pretentiously on other peoples' rights. I can see why they might want this. It is convenient to have someone to take on unpopular positions; we used to have great trouble getting prominent speakers to argue against gay rights in college debates, back in the day. I can't really see it working out for them, though, if that's what they want.

I may be being too optimistic, though. It's not like Waters' prior dubious opinions have torpedoed his public role; this is a man who has used the term 'feminazis', in the paper. Note the date, by the way; in 2002, Dermot Ahern appointed someone who uses the term 'feminazis' to the BCI, the predecessor of the BAI. Perhaps there's just something badly broken in how we handle our media in this country...

The Wisdom of the Waters

So, this has come to light, some important new information on the Gay Menace from John Waters, patron saint of not paying parking fines (seen here in his natural habitat). Have a read, it's utterly horrifying.

Yeah, that's totally a thing that will happen.

If there's anything which really makes a reasoned argument about gay marriage complete, it's implied incest. Thanks for that, Mr Waters.

Yeah, the Irish Times, the newspaper where John Waters is a columnist, is pretty much no-go for opinions like John Waters'. Important opinions like the fantasy child-stealing scenario outlined here, and arrangements to be made.

I'm... not actually sure what this bit is supposed to be about. Is he implying that us gays have a poor sense of fashion, or something?

Twitter is full of cries that Waters should give back the money, and even people assuming that he will have to do so. And, yet, there likely is no legal reason that he must. It's not like there was a trial where it was declared that John Waters is not homophobic, and this new evidence will re-open the matter. RTE chose to settle. It's not even as if they didn't have access to parts of this interview; while the most spectacularly awful stuff is absent, well, you'd think that this would be quite damning enough.

Please note that John Waters really likes the word 'satire'

If Waters has any decency, he will give the money to a neutral charity, or return it, and apologise to RTE, to O'Neill, and to the general public. Of course, if he had any decency, he wouldn't have taken the money in the first place. It's highly unlikely that he has to, though.

We really should be considering, in the aftermath of this, if our defamation laws are actually doing the job that they're supposed to do.

Bonus important insight from Waters:

Oh, no, that UKIP guy was right! Gays cause floods!

Saturday, February 8, 2014

The Homophobia Question

If you live in Ireland and haven't been actively avoiding all forms of media over the last couple of weeks,  you'll be aware of the current fuss over homophobia. Or, at least, not over homophobia itself, but over the word 'homophobia'. Of course, I could go over the whole question of the appropriateness of RTE's actions, and so forth, but that has more or less been done to death. I'm more interested in the calls, not just from the anti-gay-rights crowd, but from neutral parties and even some on the pro-gay-rights side, to eliminate the word from discussion of gay rights issues. This seems to me to be a grave mistake, and a dangerous concession to those opposed to equal rights.

Homophobia. It's a harsh, unpleasant word. It looks like a medical/psychological term, and indeed had its origins as one. And yet, in common usage, 'homophobia' is essentially to persons of non-majority sexuality and/or gender identity as 'sexism' is to women, and 'racism' is to ethnic minorities. 'Islamophobia' sounds nastier than 'anti-Semitism', with the word 'phobia' and the vague clinical feel, but it is, in general, just as offensive to call a person anti-Semitic as it is to call them Islamophobic. Xenophobia is sometimes actually used as a euphemism for racism, when talking about rabidly anti-immigrant far-right politicians, for instance. There's nothing special about the use of the 'phobia' postfix in describing certain bigotries; it's an accident of history.

Perhaps it would be better if a softer-sounding word had come to mean prejudice against gays, and a desire to deny them the rights enjoyed by the majority. Would the anti-civil-rights crowd prefer, for instance, to be called cis-hetero chauvinists, or gayists? However, it's about 50 years too late for that, now; 'homophobia' is well understood to be colloquially used in this way.

And yet, there is an apparent drive to define 'homophobia' to a stricter standard. If you propose, for instance, that black people should not be allowed to marry white people, then you will be called a racist, and few would have an issue with that, even if your reasoning is based on religion (as was the reasoning of some American proponents of the interracial marriage bans in the 60s) or on misunderstanding academic research (again, common in anti-racial-equality arguments), or on nebulous fears of the effects on society (again, not uncommon). Racism, here, extends to opposition to equal rights, even when the opposition is ostensibly based on something other than fear or hatred.

And this is about the narrowest definition of racism in common use. Even those people who argue that sexism and racism no longer exist in society because women and non-whites have legal equality (and there are people who will argue this) would typically have difficulty as seeing a proposed ban on inter-racial marriage, or a law removing the right of women to retain property in marriage, as anything other than racist and sexist respectively, regardless of the justifications.

I can see why some people, who oppose equality for gay people, would want to see the word 'homophobia' removed from discourse. It would lend a respectability to their wish to deny gay people civil rights over and above, say, the old attempts to deny women and racial minorities equal rights; it would make opposition to gay rights different and more credible in the eyes of the public.

It's hard to see the justification for doing so, though. What, after all, are their arguments? There's the religious argument, of course, but then that was used to oppose inter-racial marriage in America, too, and we don't treat that opposition as special and non-bigoted. There's the argument that marriage is for the production of children, but given that we allow infertile people to marry this one is lacking in credibility. There's the argument that gay marriage will somehow harm society, generally in a rather vague way, but this surely falls into the fear bracket. And there's the argument that homosexuals want to destroy marriage, for some reason, but this falls clearly into the fear bracket, and very paranoid at that.

I think part of the reason for this drive to avoid the h-word is that gay marriage is currently not legal, and thus opposition to it seems a little less obviously mean-spirited than, say, opposition to interracial marriage, or to married women being allowed to own property. You'll have significant difficulty today finding many people who claim to have opposed civil partnerships, and even fewer who'll admit to having opposed the legalisation of homosexuality, and yet both passed to considerable opposition. It is to the great advantage of opponents of gay marriage to be seen to be opposing it for principled reasons, and words which highlight that they're against civil rights damage this perception.

I'm not seeking to stop people voicing their opposition to gay marriage, and very few other people are, either. I'm just cautious of the attempt to set opposition to gay rights fundamentally apart from opposition to womens' and racial minorities' rights. If you wish to say, in public, that gay people want to destroy marriage, then more power to you, but you shouldn't expect this opinion to be held as somehow more reasonable and respectable than any other conspiracy theory about a minority group.

I don't think very many people in the anti-civil-rights camp really hate gays; for that matter, most racists don't hate black people. There is, clearly, however, a lot of fear, irrational fear of what granting equal rights would mean. I see no reason that this shouldn't be described as homophobia. It is, for some reason, terribly important to these people to deny me the rights that they (mostly; there are gay people opposed to gay marriage) have; I see no reason to sugar-coat their denial of my rights or the fear that drives them to it.

And the idea that the opponents of gay rights are being somehow silenced by the use of the word, or the allegations of 'heterophobia' (a word which should be treated with the same contempt as 'misandry' and 'anti-white bigotry') is absurd. Opponents of gay rights have a substantial platform in this country. If they choose to use it to say things which people criticise, this is not silencing.

Tuesday, January 14, 2014

The government's IT credibility problem

Today, Fine Gael’s website published this this (they changed the URL), a piece by Patrick O’Donovan, a Fine Gael TD on the Communications Committee.

Go read it; it’s really quite an amazing piece. I believe it’s supposed to be about the Silk Road, a seller of illegal materials, largely drugs, on the Tor network, and possibly also about Freedom Hosting, also on the Tor network and formerly one of the world’s larger distributors of child porn. It could even be about Tor itself.

The reason for my uncertainty is that it is utterly incoherent. It talks about open-source browsers, and “replacement” open-source browsers quickly appearing to continue the illegal trade. But this is nonsensical. The only non-open-source browser in common use today is the much-in-decline Internet Explorer; while Chrome and Safari are technically closed source, they are substantially open source. Firefox is entirely open source. And there’s nothing illegal about open source browsers. I can only imagine that by “open source browsers” he means “Tor network sites”.

Don’t get me wrong; I’m all for the recent shutdown of the Silk Road and Freedom Hosting. Freedom Hosting was indeed a big child porn distributor, and Silk Road’s operator was a very nasty piece of work.

I’m not even worried that the government will make bad legislation off the back of this. When it comes to it, the government will not be banning Google’s browser on the say-so of an obscure TD.

My issue is more the amazing carelessness. It would have taken O’Donovan five minutes of reading Wikipedia to, if not have a clear picture of what was going on, at least know better than to write what he did. The computer-machines seem to be a strange focal point of governmental cluelessness; while TDs writing on other subjects are hardly perfect, you’re not going to get James Reilly writing a piece advocating the use of radium to cure The Humours, or something, nor will you find Alan Shatter extolling the virtues of the Freeman on the Land philosophy. This isn’t the first time, though, that a TD has spouted complete nonsense about computers.

It makes it all the worse that O’Donovan is on the Communications Committee. You’d expect he could at least put in a little effort on what his job is supposed to be. I don’t really expect him to know this stuff, though it’d be a nice bonus, but you’d think he could look up what the words mean. I mean, what are we paying him for? Is this all a backbencher does, write nonsensical letters about something they half-remember from a tabloid?

It’s also, of course, embarrassing; you can’t really go on about the Knowledge Economy on the one hand and do this sort of thing on the other. Not really good enough, Fine Gael.