Ruby 1.9 and Unicode: The BOM Will Fuck Your Shit Up

So I’ve been playing around with the things mentioned in the title, and I found out something unfortunate when I moved a UTF-8 encoded file from a Ruby 1.9 machine to a Ruby 1.8 machine.

There’s this thing called a Byte Order Marker (BOM) that text editors use, apparently to remind themselves of the file’s UTF-8 encoding. I’m pretty sure it’s useless, because UTF-8 doesn’t actually have a variable byte order to keep track of, but there you go.

Basically, it’s 3 bytes that the text editor inserts at the beginning of a text file, and then hides from you. It might look like a plain text file, but it’s actually got 3 hidden bytes for no good reason. When you try to run it through the Ruby 1.8 interpreter, it’ll see 3 invalid characters on Line 1 and throw an error right away.

This sort of error message is pretty unhelpful, especially when you appear to have nothing at all on Line 1. You might enable visible whitespace: still nothing. You might try opening it in another text editor or IDE: you will likely still not see the problem, as the only program I’ve tried so far that doesn’t hide the BOM is NetBeans.

SciTE has two different UTF-8 encoding settings: UTF-8 and UTF-8 Cookie. In theory, the plain UTF-8 setting uses a Byte Order Marker, while UTF-8 Cookie setting doesn’t. In practice, the choice doesn’t seem to affect whether or not the Ruby interpreter chokes on the file, at least not with Ruby 1.8.

With 1.9 I’ve still had problems one or two times, but of the kind that could be fixed by closing the text editor, opening the file in NetBeans, removing the BOM, and restarting the text editor.

It’s not perfect, but at least it works now, even if it’s very slightly buggy.

Ruby 1.9 and Code Generation: How I Learned to Stop Worrying and Love Unicode

So I was working on this Ruby-based tool for generating Netbeans-compliant Swing app projects. Basically, I create a file that looks like this:

require 'java_swing'

Swing.app 'Project03AK', :subtitle => 'Laptop lending tracker',
:desc => 'This program keeps track of laptops borrowed by students.' do
  # Insert code here
end

I run this script, and it generates a Netbeans project with a main class that’s a Swing window, automatically centered and titled, the project and the window all have nice clean standardized names. Everything was going great until I got to the part where I started inserting comments in the generated Java code.

Basically, I have this Ruby script that inserts the arguments passed to Swing.app into a bunch of templates, and uses the resulting text to generate both the Java code and the related Netbeans project files. The problem here is that both Ruby and SciTE, my text editor, encode text in ASCII by default, whereas Netbeans encodes text in UTF-8.

That’s fine as long as Ruby is only generating code that uses the 26 english letters and regular english punctuation, but as soon as you start using things like àccéntêd characters, Netbeans interprets it as gibberish. I go to a French school, and my professors do not accept me handing in gibberish (except for VB code), so this is a problem.

If you don’t know/care about any of these encoding schemes or non-english characters, you need to read this. I did a few hours ago, and it helped me figure all of this out.

Basically, the solution is to install Ruby 1.9, which has Unicode support, and then go to File->Encoding->UTF-8 in SciTE. An é in the text editor will then be written to the generated Java files as a UTF-8 é, which will then be correctly interpreted as an é by Netbeans.

The Stoning of John Edwards

First off, probably a quarter to half of the people currently wringing their hands over Edwards’ extramarital affair have cheated, are cheating, or will cheat on their own spouse.

It’s not the kind of thing you’re supposed to talk about in polite company and it runs contrary to a lot of romantic/religious notions that people have, but monogamy is a fairly recent invention and is by no means natural. I’m just saying, keep in mind the ridiculous level of hypocrisy at work here.

Yes, he lied. Yes, it could have hurt his party’s chances, although it’s probably better that this distraction didn’t surface during the primaries. I do understand the disappointment of the people who defended him in good faith. You could even argue that people expect better-than-average behavior from their politicians, although the current gang of thugs and robber barons in power certainly speaks against that.

However, most of the sanctimonious commentators criticizing him would not have done any differently in his situation, and quite a few of them are in his situation.

Statistically, there is a significant portion of high-ranking politics/news people that have stories of infidelity of their own, so I’m assuming that we’ll soon see hundreds of these people come out and admit their own affairs in the name of honesty. Come on, who wants to go first?

*chirp!* *chirp!*

A disaster of Olympic proportions

So, despite the fact that they get to host the Olympics, the Chinese government apparently still doesn’t give a damn about little things like human rights, dignity, or freedom. How strange! Who could have ever predicted such a bizarre twist?

In addition to the abuse inflicted on the locals, for example destroying their houses, executing roughly 374 Chinese citizens during the Olympics, and deploying ridiculous amounts of surveillance and military strength, it looks like they’re also arresting foreign protesters, assaulting journalists, blocking Internet access even for foreign reporters, and just generally giving everyone a hard time.

Oh, this is a good one :

“We have laws regarding assembly and demonstrations, and we hope that foreigners will respect the laws of China.”

While we are aware of your laws, sir, there are reasons why we don’t respect them. It is because they are brutal and inhuman, in the image of the thugs, tyrants and monsters that create and enforce them.

We foreigners just happen to have a little more power to defy the crimes against humanity you call “laws” without getting murdered, that’s all.

Don’t worry, there will be more defiance and disrespect coming from foreigners, and soon.

Through the looking glass: surreal campaign ads

First, this laugably bad ad. It appears that the McCain campaign’s latest attack is “Obama wants to Photoshop his face onto Mount Rushmore and the Statue of Liberty”. Incredibly juvenile, but it would work if your average McCain voter knew what a Photoshop was.

Secondly, there’s this video, “The One”.

Twist ending : “Paid for by John McCain 2008″. Really? Are… are you sure, official McCain Youtube channel and JohnMcCain.com?

I see two possible messages McCain could be trying to get out here.

He might be trying to say “Vote for Obama”, which is bizarre and likely not a winning strategy for McCain. Then again, his campaign, his party, and a good portion of the news media appear to be under the influence of hallucinogenic substances.

At this point I wouldn’t be that surprised if Bush went on a puppy-kicking spree only to have McCain come out and defend him, blame it on Obama and the Liberal Media, and say that this type of situation shows he’s “ready to lead” and his opponent is “naïve”. Somehow, it would probably even work, and the media would spend the next two days asking questions like “Voting for McCain : good idea, or great idea?” and “The Liberal Media’s biased reporting: should we stop favoring Obama?”.

Anyways. The other message, the one McCain is more likely trying to push with the world-leader imagery and the glorification of his opponent, is something along the lines of “Obama is the Antichrist”. That might make sense in the mind of Americans who consider reading forwarded emails to be ‘research’, but those people already have lists of contradictory reasons why they’re afraid of him.

In any case, I’m pretty sure the Antichrist is actually supposed to get elected. If McCain was a good Christian and he really believed what he’s trying to scare people into believing, he’d have to make sure the Antichrist won the election or he’d be standing in the way of Prophecy.

On the other hand, if he was a good Christian he probably wouldn’t be ridiculing Moses to make a political attack.

Sorry, ISPs. You’ll have to deliver what you promise.

Unless you own a telecommunications company with a business model based on selling more bandwidth than you can deliver and then punishing your customers, this should be good news.

The FCC ruled against Comcast on Friday, saying that their interference with customers’ file transfers was a violation of federal policy. While it’s far from a guarantee of future network neutrality, it’s probably as good a precedent as we could have hoped for from this case.

Comcast has been given until the end of the year to get their act together and stop undermining the proper function of their service. While that is a lot further than the deadline I would have given them, the good news is that the Electronic Freedom Foundation has released the Switzerland Network Testing Tool. This means that Comcast’s customers should now be able to easily watch their ISP to make sure that they’re complying with the ruling.