Comments for Natural Code http://naturalcode.wordpress.com Code, science and politics. Mon, 20 Jul 2009 09:15:18 +0000 http://wordpress.com/ hourly 1 Comment on Ruby 1.9 and Unicode: The BOM Will Fuck Your Shit Up by Wiki-fan http://naturalcode.wordpress.com/2008/08/30/ruby-19-and-unicode-the-bom-will-fuck-your-shit-up/#comment-37 Wiki-fan Mon, 20 Jul 2009 09:15:18 +0000 http://naturalcode.wordpress.com/?p=125#comment-37 http://en.wikipedia.org/wiki/Byte-order_mark http://en.wikipedia.org/wiki/Byte-order_mark

]]> Comment on Ruby 1.9 and Unicode: The BOM Will Fuck Your Shit Up by Geoff Adams http://naturalcode.wordpress.com/2008/08/30/ruby-19-and-unicode-the-bom-will-fuck-your-shit-up/#comment-30 Geoff Adams Fri, 21 Nov 2008 19:56:03 +0000 http://naturalcode.wordpress.com/?p=125#comment-30 Actually, no, naturalcode is correct. While UTF-8 is indeed a variable-length encoding, there are no byte order issues. The specification clearly describes the order in which the bytes must appear in the stream, and that order does not depend on the endianness of the processing machine. Put another way, UTF-8 does not represent characters as mutli-byte numerical words, as UTF-16 does. It's a variable-length stream of bytes. The UTF-8 file cannot be interpreted differently if the BOM is 0xFE 0xFF or 0xFF 0xFE. I also don't understand why some editors put a BOM in UTF-8 files. I suppose that since the BOM is, itself, encoded as UTF-8 (and therefore doesn't even appear as a raw, two-byte 0xFE 0xFF or 0xFF 0xFE in the file), a processor can see the 0xEF 0xBB 0xBF sequence and recognize the file as UTF-8. But really... Actually, no, naturalcode is correct. While UTF-8 is indeed a variable-length encoding, there are no byte order issues. The specification clearly describes the order in which the bytes must appear in the stream, and that order does not depend on the endianness of the processing machine.

Put another way, UTF-8 does not represent characters as mutli-byte numerical words, as UTF-16 does. It’s a variable-length stream of bytes. The UTF-8 file cannot be interpreted differently if the BOM is 0xFE 0xFF or 0xFF 0xFE.

I also don’t understand why some editors put a BOM in UTF-8 files. I suppose that since the BOM is, itself, encoded as UTF-8 (and therefore doesn’t even appear as a raw, two-byte 0xFE 0xFF or 0xFF 0xFE in the file), a processor can see the 0xEF 0xBB 0xBF sequence and recognize the file as UTF-8. But really…

]]> Comment on Fire Katie Couric (hire a journalist instead) by McCain’s Big Lie about “The Surge”: Thank God Facts Don’t Matter in a Presidential Election, ey? « Reverend Manny and The Twilight Empire http://naturalcode.wordpress.com/2008/07/24/fire-katie-couric-hire-a-journalist-instead/#comment-29 McCain’s Big Lie about “The Surge”: Thank God Facts Don’t Matter in a Presidential Election, ey? « Reverend Manny and The Twilight Empire Wed, 01 Oct 2008 13:52:18 +0000 http://naturalcode.wordpress.com/?p=36#comment-29 [...] Now, let’s dig out some dates. (great breakdown from naturalcode.wordpress.com) When did it start? That’s right, the Sunni Awakening started a few months BEFORE the surge, and a few months before the troops ACTUALLY GOT THERE. Hm…. [...] [...] Now, let’s dig out some dates. (great breakdown from naturalcode.wordpress.com) When did it start? That’s right, the Sunni Awakening started a few months BEFORE the surge, and a few months before the troops ACTUALLY GOT THERE. Hm…. [...]

]]> Comment on Fire Katie Couric (hire a journalist instead) by Reverend Manny http://naturalcode.wordpress.com/2008/07/24/fire-katie-couric-hire-a-journalist-instead/#comment-28 Reverend Manny Wed, 01 Oct 2008 13:49:20 +0000 http://naturalcode.wordpress.com/?p=36#comment-28 FUCK YEAH!! It is un-fucking-believable that McCain is literally running on the fact that the Surge worked. Great, GREAT POST... i'm definitely linking to it... One Love, --Reverend Manny and the Twilight Empire FUCK YEAH!! It is un-fucking-believable that McCain is literally running on the fact that the Surge worked.

Great, GREAT POST… i’m definitely linking to it…

One Love,
–Reverend Manny and the Twilight Empire

]]> Comment on Ruby 1.9 and Unicode: The BOM Will Fuck Your Shit Up by James Bennett http://naturalcode.wordpress.com/2008/08/30/ruby-19-and-unicode-the-bom-will-fuck-your-shit-up/#comment-27 James Bennett Sun, 31 Aug 2008 04:26:40 +0000 http://naturalcode.wordpress.com/?p=125#comment-27 "I’m pretty sure it’s useless, because UTF-8 doesn’t actually have a variable byte order to keep track of" This is incorrect; UTF-8 can get away with a single byte for commonly-used characters in Western European scripts, but there are many more characters in Unicode than there are bits in a single byte. As a result, UTF-8 can use anywhere from one to four bytes to encode a particular Unicode character; hence UTF-8 is a variable-length encoding (as is UTF-16; only UTF-32 can guarantee that it uses the same number of bytes for every character) and can encounter byte-order issues which the BOM solves. “I’m pretty sure it’s useless, because UTF-8 doesn’t actually have a variable byte order to keep track of”

This is incorrect; UTF-8 can get away with a single byte for commonly-used characters in Western European scripts, but there are many more characters in Unicode than there are bits in a single byte. As a result, UTF-8 can use anywhere from one to four bytes to encode a particular Unicode character; hence UTF-8 is a variable-length encoding (as is UTF-16; only UTF-32 can guarantee that it uses the same number of bytes for every character) and can encounter byte-order issues which the BOM solves.

]]> Comment on Ruby 1.9 and Code Generation: How I Learned to Stop Worrying and Love Unicode by naturalcode http://naturalcode.wordpress.com/2008/08/24/ruby-19-and-code-generation-how-i-learned-to-stop-worrying-and-love-unicode/#comment-26 naturalcode Mon, 25 Aug 2008 19:54:11 +0000 http://naturalcode.wordpress.com/?p=122#comment-26 That makes sense. Probably the text editor has a France regional encoding and NetBeans uses a Canada French encoding, or something like that. That makes sense. Probably the text editor has a France regional encoding and NetBeans uses a Canada French encoding, or something like that.

]]> Comment on Ruby 1.9 and Code Generation: How I Learned to Stop Worrying and Love Unicode by guilhem http://naturalcode.wordpress.com/2008/08/24/ruby-19-and-code-generation-how-i-learned-to-stop-worrying-and-love-unicode/#comment-25 guilhem Mon, 25 Aug 2008 10:58:30 +0000 http://naturalcode.wordpress.com/?p=122#comment-25 I think what ak was saying is that : ASCII-7bit (ie ASCII) is a strict subset of UTF-8. Anything with an accent is not ASCII but any of the various regional encodings (like iso-8859-1 for france), which are sometimes referred to as just ASCII-8bit, but which are not compatible with each other. I think what ak was saying is that : ASCII-7bit (ie ASCII) is a strict subset of UTF-8. Anything with an accent is not ASCII but any of the various regional encodings (like iso-8859-1 for france), which are sometimes referred to as just ASCII-8bit, but which are not compatible with each other.

]]> Comment on Ruby 1.9 and Code Generation: How I Learned to Stop Worrying and Love Unicode by naturalcode http://naturalcode.wordpress.com/2008/08/24/ruby-19-and-code-generation-how-i-learned-to-stop-worrying-and-love-unicode/#comment-24 naturalcode Mon, 25 Aug 2008 04:20:49 +0000 http://naturalcode.wordpress.com/?p=122#comment-24 Doesn't look like it. This is what happens when my text editor is on its default settings: puts 'Café'.encoding >ASCII-8BIT Not only that, but when I do File->Encoding->UTF-8, the é and the last apostrophe become an Asian character. Once I erase it and type the é again, this is what happens: puts 'Café'.encoding >UTF-8 And NetBeans doesn't interpret the é properly unless I do it that way. Seems pretty clear-cut to me. I think if anyone is mixing up their charsets, it must be NetBeans or SciTE. Doesn’t look like it. This is what happens when my text editor is on its default settings:

puts ‘Café’.encoding
>ASCII-8BIT

Not only that, but when I do File->Encoding->UTF-8, the é and the last apostrophe become an Asian character. Once I erase it and type the é again, this is what happens:

puts ‘Café’.encoding
>UTF-8

And NetBeans doesn’t interpret the é properly unless I do it that way. Seems pretty clear-cut to me. I think if anyone is mixing up their charsets, it must be NetBeans or SciTE.

]]> Comment on Ruby 1.9 and Code Generation: How I Learned to Stop Worrying and Love Unicode by ak http://naturalcode.wordpress.com/2008/08/24/ruby-19-and-code-generation-how-i-learned-to-stop-worrying-and-love-unicode/#comment-23 ak Sun, 24 Aug 2008 22:37:29 +0000 http://naturalcode.wordpress.com/?p=122#comment-23 ASCII is a strict subset of UTF-8, i.e. every ASCII textfile is also a valid UTF-8 textfile, so you're obviously mixing it up with some other charset. ISO-8859-1(5) maybe? ASCII is a strict subset of UTF-8, i.e. every ASCII textfile is also a valid UTF-8 textfile, so you’re obviously mixing it up with some other charset. ISO-8859-1(5) maybe?

]]> Comment on The Stoning of John Edwards by naturalcode http://naturalcode.wordpress.com/2008/08/10/the-stoning-of-john-edwards/#comment-22 naturalcode Mon, 11 Aug 2008 16:06:43 +0000 http://naturalcode.wordpress.com/?p=115#comment-22 This article might give a little historical perspective. <a href="http://www.alternet.org/election08/94550/how_is_john_mccain%27s_affair_different_from_john_edwards%27/" rel="nofollow">How Is John McCain's Affair Different from John Edwards'?</a> <blockquote>We have this weird notion in America now that if a politician is caught in an affair that his career is done. We seem to be saying that what he did in his private life effects his policies or how he governs. But we all know that isn't true. We know that because almost all of our great presidents, and great leaders throughout history, have had numerous affairs. Obviously it didn't hurt how they governed at all.</blockquote> This article might give a little historical perspective.

How Is John McCain’s Affair Different from John Edwards’?

We have this weird notion in America now that if a politician is caught in an affair that his career is done. We seem to be saying that what he did in his private life effects his policies or how he governs. But we all know that isn’t true. We know that because almost all of our great presidents, and great leaders throughout history, have had numerous affairs. Obviously it didn’t hurt how they governed at all.

]]>