Ben's · Beta · LJ

Executive Summary

• Google Wave looks like a very nice concurrent rich text editor;
• I'm worried that it is being positioned to replace tools that are far better in their niche than Wave currently is;
• I'm worried that it may be able to replace tools that are better than Wave has the potential to be, and that it may therefore subtract [some] value from the internet as a communications medium.

Regression Bad, or, Hands Off My Email, or, Get Off My Lawn

I am very worried by Lars Rasmussen's description^[8] of how he believes Wave relates to email:

Q: This seems like this will replace email - but can it really replace all we love about email?

Lars: We think of email as an incredibly successful protocol. Google Wave is our suggestion for how this could work better. You can certainly store your own copy by way of the APIs and with the extensions. The model for ownership - it's a shared object, so how do you delete the object? Even though it's a shared object, no one can take it away from you without your consent. There will eventually be reversion to sync up with the cloud or you own servers. We're not planning on having spam in wave (laughs). Early on in email, spam wasn't really taken into account, so we benefit from that learning experience. We're planning on a feature so that you can't add me to your Wave without being on a white list.

Later on, Lars mentions that a bidirectional email-to-wave gateway might be difficult, but possible. The problem is that email includes a set of well-structured header information and the optional ability to include an arbitrary number of body elements, each which may contain highly structured information (e.g. MIME sections containing binary files), semistructured documents (e.g. HTML, Markdown, format=flowed, or other text formats which offer hints about enhanced rendering while not requiring it) or unstructured text; whereas Waves appear to not only strip away the requirement for this extra richness but also the ability to include it.

Accepting and rendering down rich structure while offering only text with primarily presentational markup makes for a unbalanced flow of information.

References:

1. @rcwoolley, personal correspondence
2. http://www.youtube.com/watch?v=v_UyVmITiYQ
3. http://www.youtube.com/watch?v=Sx3Fpw0XCXk
4. http://www.waveprotocol.org/whitepapers/internal-client-server-protocol
5. http://www.waveprotocol.org/whitepapers/operational-transform
6. http://www.maetico.com/everything-and-wave/
7. http://www.techcrunch.com/2009/05/28/google-wave-drips-with-ambition-can-it-fulfill-googles-grand-web-vision/
8. http://www.techcrunch.com/2009/05/28/live-with-the-google-wave-creators/
9. http://www.texttechnologies.com/2009/05/29/google-wave-finally-a-microsoft-killer/
10. http://twitter.com/gnomon/status/2011194686

Passion radiates. You can see it roll off people in waves, like haze from a locomotive engine. Like that heat, passion doesn't do much in a vacuum, and most of it gets wasted as entropy.

Manage it properly, though, and heat becomes flame, flame becomes fire. Fire can warm the needy, lift rockets to the heavens, and light the way through temples, tombs and libraries.

Fire brings light. Cast wide, it can bring knowledge and safety; focused coherently, in can carry words and thoughts to the other side of the world, or on into the far future.

Passion - gathered, directed, encoded - powers the world.

My passion is for the written word. I use it awkwardly, but I love it dearly. Thought, inspiration, hard-fought wisdom and brilliant insight live on and leap from person to person, across generations, as simple words. Used well, words uplift and ennoble us. They are the crowning achievement of humanity.

That gleaming potential is all too often tarnished by poor use. Bad language is wasteful and frustrating, like a stubborn sneeze or an unreachable itch. It can be mundane; it can be tragic. And it can be avoided.

Every word is an opportunity to do well, and to do good; to strike a chord and hear it richly resonate; to drop a stone in a pond and watch the ripples spread. Every word is a chance to stoke the embers of our passions and pass on a little warmth.

We can all choose to have our words be slow and calm; fierce and torrential; strong and bright and beautiful.

We can all do better.

We can all do better.

Optical fiber is a brilliant invention: it is a thread of spun glass. By the magic of total internal reflection, the refraction index of the material, and the wavelength(s) of the light fired down the pipe; by the engineering of dense wavelength multiplexing, and quantum-tunneling solid-state laser boosters set into the fiber every few kilometers; and, sometimes, by the hard work of cable-laying ship captains who stubbornly brave the high seas to thread this spun glass across the ocean floor, billions of bits flow from point to point across the world. Against all odds, not to mention the second law of thermodynamics, the world has managed to hew from the static of random potential the most powerful communications tool our civilization has ever seen.

Now. I don't like TV.

This Christmas I was given a generous gift: a Hauppauge HVR-950 USB TV tuner. I really, really wanted one of these. The widget is about the size of one of those slim Zippo lighters. I plug one end in to my USB hub and the other end to a coaxial cable that runs to a telescoping antenna of the same sort you'd find on a plastic radio. With the help of twelve kilobytes of closed-source firmware dynamically loaded into its onboard processor, I can tune into a whole set of channels streaming right out of the CN tower, not to mention what comes across the water from Buffalo. The CBC broadcasts an MPEG-2 stream with 1920 by 1080 lines of interlaced resolution; mplayer gives me a variety of options to upsample the spatial resolution with the help of the extra temporal resolution added by the interlacing of the fields.

The video stream is completely independent of the transmission technique, and my bargain-priced, extremely powerful quad-core processor makes short work of the digital signal; but the HVR-950 itself does some fascinating, insightful and difficult work to condense the vapour of that feeble radio transmission into something from which the video signal can be recovered.

The little widget gets warm when it's in use. The signal varies depending on what lies between the antenna and the window.

We have covered the planet with a web of spun glass and the dreams of a world light the fiber day and night; but that pride is completely different from the flare of inspiration sparked by the faintest of radio signals, coming through thin air from the other end of the world.

Dear LJ,

IOU:

• an explanation of why I recently bought an Intel D945GCLF Atom motherboard (short story: my laptop died spectacularly, G45-based motherboards won't hit retail channels for another two months, I can't wait that long for a new system, and I wanted a low-power standalone file server anyhow);
• an explanation, and perhaps explanatory pictures, of why the abovementioned motherboard is currently housed in a cardboard box tied with pink wrapping ribbon;
• how-to documents detailing how to get Damn Small Linux 4.4.2 playing nicely with..:
  1. a D-Link DGE-530T gigabit PCI network card (PCI ID 1186:4b01, rev 11);
  2. the built-in RealTek RTL8102E 10/100 network chip (PCI ID 10ec:8136, rev 02);
  3. the 82801G audio controller built into the ICH7 north bridge (PCI ID 8086:27d8, rev 01);
  4. Firefox 3;
• news about my recent awk hackery;
• an update on the job search.

From: "Ministers Prentice and Verner" <Minister.Industry@ic.gc.ca>
To: CanadianCitizens@everywhere.com
Subject: An Act to Amend the Copyright Act
Date: Thu, 12 Jun 2008 13:09:06 -0400

The Government of Canada has introduced Bill C-61, An Act to Amend the Copyright Act. The proposed legislation is a made-in-Canada approach that balances the needs of Canadian consumers and copyright owners, promoting culture, innovation and competition in the digital age.

What does Bill C-61 mean to Canadians?

Specifically, it includes measures that would:

• expressly allow you to record TV shows for later viewing; copy legally purchased music onto other devices, such as MP3 players or cell phones; make back-up copies of legally purchased books, newspapers, videocassettes and photographs onto devices you own; and limit the "statutory damages" a court could award for all private use copyright infringements;

• implement new rights and protections for copyright holders, tailored to the Internet, to encourage participation in the online economy, as well as stronger legal remedies to address Internet piracy;

• clarify the roles and responsibilities of Internet Service Providers related to the copyright content flowing over their network facilities; and

• provide photographers with the same rights as other creators.

What Bill C-61 does not do:

• it would not empower border agents to seize your iPod or laptop at border crossings, contrary to recent public speculation

What this Bill is not:

• it is not a mirror image of U.S. copyright laws. Our Bill is made-in-Canada with different exceptions for educators, consumers and others and brings us into line with more than 60 countries including Japan, France, Germany and Australia

Bill C-61 was introduced in the Commons on June 12, 2008 by Industry Minister Jim Prentice and Heritage Minister Josée Verner.

For more information, please visit the Copyright Reform Process website at www.ic.gc.ca/epic/site/crp-prda.nsf/en/home

Thank you for sharing your views on this important matter.

The Honourable Jim Prentice, P.C., Q.C., M.P.
Minister of Industry

The Honourable Josée Verner, P.C., M.P.
Minister of Canadian Heritage, Status of Women and Official Languages and Minister for La Francophonie

Current Mood:

disappointed

Because my cow-orkers just had to bring up the Dreaded Site™, and I just had to look at the Scheme version and note that, while at least it was present, the organization of the code in all three versions left a great deal to be desired. A few more moments of reading revealed that the output was wrong, too.

I'm sure you can see where this is going.

;; Gnomon - on Thu Apr 17 23:54:52 EDT 2008
;; sha1sum of lastname, firstname, newline:
;; 3952893c66ffbe071f266caa04b25161c1da30a1
;; Because I wanted a version that would Do
;; The Right Thing with capitals and plural
;; cases, and because mixing logic, display
;; and generation code does not satisfy me.

(define (iota n)
  (let loop ((l '()) (i 0))
    (if (> i n)
        l
        (loop (cons i l) (+ i 1)))))
;; See SRFI-1 for a better version of this;
;; my function is backward in several ways.
;; It can also be argued that SRFI-42 could
;; allow an implementation that consed up a
;; bit less temporary garbage. If SRFI-1 is
;; out, though, SRFI-42 is way out. Basics.

(define (plural? n . up)
  (let ((.. string-append)
        (num (number->string n))
        (bot " bottle")
        (ltr (if (null? up) "n" "N")))
    (case n ((0) (.. ltr "o more" bot "s"))
            ((1) (.. num bot))
            (else (.. num bot "s")))))
;; PLURAL? has to merge the logic for cases
;; when the string will be used at the head
;; of a sentence, requiring a capital; when
;; only one bottle is left, which needs the
;; singular form; and when none are left at
;; all, which requires a special case. It's
;; lucky that not all combinations of these
;; cases can be reached, allowing us to use
;; only three of the possible six branches.

(define (verse n)
  (let ((.. string-append)
        (top (plural? n 1))
        (mid (plural? n))
        (nxt (plural? (if (= n 0)
                          99
                          (- n 1))))
        (beer " of beer")
        (wall " on the wall")
        (actn (if (= n 0)
                  "Go to the store and buy some more, "
                  "Take one down and pass it around, ")))
    `(,(.. top beer wall ", " mid beer ".")
       .  ,(.. actn nxt  beer wall "."))))
;; VERSE conses a pair where the CAR stores
;; the first line of the verse, and the CDR
;; the second. There is no sense in using a
;; proper list; we would just waste the end
;; of it anyways. Also: yay quasiquotation!

(define (sing verse)
  (let ((n newline))
    (display (car verse)) (n)
    (display (cdr verse)) (n) (n)))
;; SING exploits the implicit BEGIN wrapped
;; around the body of a LET to DISPLAY both
;; lines of a verse in order. Also note the
;; pleasant symmetry of CAR and CDR that we
;; get to use because of our choice made up
;; above in VERSE to use a dotted pair as a
;; representation instead of a proper list.

(for-each sing (map verse (iota 99)))
;; MAP for effect, FOR-EACH for side effect

It's been submitted for inclusion, of course, with the following lines as an additional note to the site maintainers:

;; The code is long, the comments too;
;; A bit less wrong than version two;
;; No syntax-case, like version three;
;; Just one right place per step, you see.

I'm a little bit in love with the A+ version.

Current Mood:

tired

I'm going to start posting the random bits of awk code that I come up with.

Sorry.

For a project about which I'll be writing more in a bit, I needed the ability to quickly produce long strings - on the order of a few dozen, and in some case a few hundred, mebibytes. I went through several iterations before I came up with a version that I liked. That version is called rep6 in the below source code; in my personal library file I would just call it rep.

#!/bin/gawk -f 

# by gnomon
# sha1sum of printf "%s %s", firstname, lastname:
# 1110be40263bc427f7906f4e01c0cc218c6daf6e
# Tue Apr  8 21:11:32 EDT 2008
#
# posted to #awk in the hopes of garnering some feedback
# about style, ideas and performance. 

# GPL v2 (but it's not finished yet, so why bother?)

function rep1(s,n,      r) {
# O(n) allocate/appends
# 2 lines of code
# This is the simplest possible solution that will work:
# just repeatedly append the input string onto the value
# that will be passed back, decrementing the input count
# until it reaches zero.
        while (n-->0) r = r s;
        return r;
}

function rep2(s,n,      r) {
# O(1) + O(n) allocate/appends
# 3 lines of code
# The idea here is to generate a string of length n with
# sprintf and then use gsub to replace everything in the
# generated string with the input string. Hopefully this
# will take advantage of some smart internal workings of
# the regex engine to avoid repeated allocations.
# UPDATE: false hope. This is by far the slowest tactic.
        r = sprintf(("%0"n-1"s"),"0");
        gsub(/0/,s,r);
        return r;
}

function rep3(s,n,      c,n2,r,r2) {
# O(log n) allocate/appends UPDATE: this is a lie!
# 6 lines of code
# Here we trade for faster execution by sacrificing some
# code clarity. We keep a temporary string and a counter
# that we start off at 1, doubling both until we pass n;
# then we start over at 1, and we repeat until exactly n
# is reached. This may be ugly but it is faaaaaaast.
        for (;c < n; c += n2) {
                r2 = s;
                #for (n2=1; n2*2<(n-c); n2*=2) {
                #     APPEND++; r2=r2 r2}
                for (n2=1; n2*2<(n-c); n2*=2) r2=r2 r2;
                # APPEND++;
                r  = r r2;
        }
        return r;
}

function rep4(s,n) {
# O(1) allocate/appends
# 2 lines of code
# A feat - both stupid and naive! Is consing up a string
# to avoid repeated print operations really that bad, or
# can good buffering make a dumb idea actually workable?
# UPDATE: no, it can't. This still handily beats out the
# gsub version, but it's slower than rep1 and way slower
# than rep3. Moral: buffer not the unbufferable!
        while (n-->0) printf s;
        print "";
}

function rep5(s,n,      r,x) {
# O(log n) allocate/appends - *actually true* this time!
# 3 lines of code
# Rather than keeping track of a counter like rep3 does,
# here we just use the call stack and recurse. Turns out
# that I run out of RAM way faster than gawk runs out of
# call stack. There may be a good threshold at which the
# recursion can halt and we fall back to a simple append
# approach like rep1 - but really, why bother? This code
# is simple and fast. Also, the shortest possible string
# is one character, and 32 recursive calls turn that one
# character into 4,294,967,296; that is more than enough
# to fill all available RAM on most current machines. If
# it doesn't, the performance is likely to be limited by
# the speed of malloc() (and therefore sbrk()) anyhow. A
# quick test reveals that gawk 3.1.5 on a 32-bit x86 box
# with 512 megs of RAM can stack just short of 7e3 calls
# before segfaulting; dc -e'2 7000^f' prints a very long
# number indeed: 2108 decimal digits, to be precise, and
# it will presumably take a good long while before every
# normal machine has that many bytes of RAM. We're safe.
        #DEPTH++;
        if (n < 2) x = (n == 1);
        else r = rep5(s,(n-(x=(n%2==1)))/2);
        return (r r (x?s:""));
}

function rep6(str,num,  remain,result) {
# O(log n) allocate/appends - identical to rep5
# 7 lines of code
# This is a much cleaner version of the preceding block.
# For some reason I enjoy reading awk code bereft of the
# optional semicolons, but I enjoy writing it with; this
# is odd, but there it is. It'd be less embarassing if I
# showed only the below without any of the above, but it
# would be less valuable and less honest.
        #DEPTH++; # global var for counting recursions
        if (num < 2) {
                remain = (num == 1)
        } else {
                remain = (num % 2 == 1)
                result = rep6(str, (num - remain) / 2)
        }
        return result result (remain ? str : "")
}

BEGIN {
        #NUM = 10000000;     STR = "foo";
        NUM  = ARGV[1] + 0; STR = ARGV[2];
        delete ARGV[1];    delete ARGV[2];
        if        (ARGV[3] ~ /0/) {
                exit 0;
        } else if (ARGV[3] ~ /1/) {
                print rep1(STR,NUM);
        } else if (ARGV[3] ~ /2/) {
                print rep2(STR,NUM);
        } else if (ARGV[3] ~ /3/) {
                print rep3(STR,NUM);
                #print APPEND >> "/dev/stderr";
        } else if (ARGV[3] ~ /4/) {
                rep4(STR,NUM);
        } else if (ARGV[3] ~ /5/) {
                print rep5(STR,NUM);
                #print DEPTH >> "/dev/stderr";
        } else if (ARGV[3] ~ /6/) {
                print rep6(STR,NUM);
                #print DEPTH >> "/dev/stderr";
        }
        delete ARGV[3];
        exit 0;
}

# FIXME:
# - investigate whether some tactics work better when
#   consing small iterations of very large strings;
# * done: the version that starts at one, doubles until
#   overflow and then starts again at one
# - see if there's some way of making the gsub approach
#   work better: it's too fun to abandon outright;
# - is there some way to use fewer state variables in
#   rep3? The current version is really awkward;
# - is there any way to collapse or compound the r2 and
#   r assignments in rep3 to shave a line or two?
# * sure, but rep3 is a dead end anyway. Reusing the
#   value returned by a variable assignment to remove a
#   reference to it works, as in rep5, but the result
#   is a mess and a waste of effort. Still fun, though.

Current Mood:	tired
Current Music:	Postmaster, from Demographically Unsound by Tettix

I'm tapping this out in T9 on my E50 rather than using one of the two keyboards I usually have in my kit bag^[1]. It's not as goofy a setup as it sounds: one board is a ThinkOutside Stowaway that is smaller than the paperbacks I tote, and the other is an Apple Wireless which is thin enough to be a non-issue. I carry two because I keep the former paired with my phone and the latter with my N800, and I use the devices concurrently often enough that re-pairing is more onerous than taking the cargo space hit. Real-world space versus time optimization tradeoff, ladies and gents... But I digress.

I'm tapping away on my phone because there's something I want to get out of my increasingly murky brain before I get so tired that it disappears completely, and there's a good chance that'll happen before I get home.

The thing is this: mikeX, a fellow who hangs out in an IRC channel that I also frequent (#awk on irc.freenode.net), mentioned that he ran a performance comparison between a couple of constructions that ought to be equivalent:

• time gawk '/test/{print $2}' file.txt
• time mawk '/test/{print $2}' file.txt
• time grep test file.txt | awk '{print $2}'

The file^[2] was fifteen mebibytes of short lines with only one line, the last, containing the string "test". The first command turned out to be the slowest by far - about an order of magnitude. The second and third executed at roughly the same speed.

What makes this particularly interesting is that mikeX had just finished skimming through the same paper^[3] that I had coincidentally re-read just the night before - one which compares the merits of several classes of regular expression and regex engines - and concluded that the speed difference was due to a deficiency of the type discussed in the paper (basically, that grep, awk and others use a DFA-style matcher which offers sometimes huge speedups with patterns that do not require backreferences, and that Perl and several other languages use a backtracking matcher with horrific worst-case performance even in those situations when the simpler, hugely faster DFA could be used instead).

One the one hand, this is definitely not the case, since the pattern in question is a fixed string and therefore is both the best case for any matcher and devoid of the kind of backtracking that would highlight the bad behaviour described in the paper; on the other hand, something is causing that performance delta. So what gives?

I seem to recall that grep calls into play some kind of fast-path logic when the pattern is a fixed string (something tricky like Boyer-Moore? Something macho like exploiting CPU-specific string-scanning instructions? Something slightly less macho but more insightful, like taking into account cache fill timings?). I know that mawk is reputed to be a very fast version of the tool. I suspect that the matcher in gawk is quite sophisticated. I'm curious to see how Dmitry Zakharov's busybox awk applet ranks on the speed tests.

What I'm getting at, here, is that I'm about to do some code-diving that I think may yield some very educational results. I'll need some sleep first, though.

Hey, here's my stop.

1

OK, I'm cleaning this up after getting home. HTML editing and linking is way, way too painful to do with T9 and an impoverished text editor.

2

If you're curious enough to want to duplicate the test, you can generate your own test file with the following pipeline:

time awk 'function r(s,n){while(n-->0)print s};BEGIN{s=ARGV[1];n=ARGV[2];r(s,n);print "test case hello world";r(s,n);exit 0}' "Hello world, how are you" 300000 > test-file.txt

That shouldn't take longer than a second or two to run, and should generate a file with the md5sum 9cf078b42b15b6927d04b08c82699e16.

3

Regular Expression Matching Can Be Simple And Fast, Russ Cox, 2007-01

On the commute home today, I saw:

• A kid, not a day older than sixteen, in a secondhand jacket, sporting a red fitted cap covered in thin black squiggles with a stylized letter F above the bill. His chin crept back to halfway between his underbite and his adam's apple. He fiddled with his iPod as he sat down, then stowed it in his backpack. He pulled a dog-eared Penguin Classic out of his artfully distressed backpack - I couldn't resolve the title - and settled in. Five stops whipped by. It wasn't until he dotted the page that he noticed he was crying.
• Two large guys with identical haircuts and choice of clothing style. They breezed into the subway just behind a very blonde, very leggy lady wearing a very brief skirt indeed, and they all sat down together. The guys both wore very fashionable glasses and expensive clothing that didn't hide that they were both a good bit overweight. They grinned at each other and the rest of the subway and barely paid any attention to her attempts at making conversation. One nudged the other with his elbow and leaned in to crack a joke. They both laughed so hard for so long that they didn't see the lady get up to offer her seat to a swaying, grateful octogenerian.
• A lady in her late fifties dourly poking at the keypad of a Nokia N70 and convincing it to map her keypresses into ideographs. She filled two entire screens, then patiently waited until the subway reached a point on the track where it hit open air and GSM reception. She carefully examined the small cornerful of pixels that graphed her signal strength, waited for it to max out, then hit Send. Once the screen emptied, she stashed the handset in her coat pocket, chuckled, then broke out into a grin so wide that I couldn't help but return it.

To: CBC Radio (via)
From: [excised]
X-from-city: Toronto
X-from-province: Ontario

Dear CBC,

I am addressing the broadcasters and/or administrators, and not the show creators, right? Because if this *is* being read by a show creator, I'm talking to the wrong people. Could you forward it on to the guy with the corner office and the speakerphone? Much obliged. Coffee's on me next time.

Dear CBC,

Canadia 2056 troubles me. I've heard and read excellent things about it. A couple of my friends who got on the bandwagon early enough to catch the first season in its entirety tell me that it's excellent. The premise is right up my alley, and I grew up on Grant Naylor and Douglas Adams, after all.

Dear CBC, why do you make it so hard for new listeners to find this show?

There aren't any podcasts.

There aren't any sample episodes (not even the first one!).

There is a rather pricy CD set available, but it seems that the only place selling it is the online CBC Shop. The high price isn't a showstopper, but the fact that the Shop only accepts credit cards or gift cards which have been purchased with credit cards certainly is.

Dearest Canadian Broadcasting Corporation, I understand that there are guild and union complexities here that make distribution challenging. I really do understand it, and I don't mean to downplay how difficult it is to work these things out.

Loveliest and most rose-scented CBC, please also understand, though, that you've got a great show here, and a potentially rabid fan base. The standard deal in these situations is this: you make it possible for your fans to listen to the show, and we in return make it vastly popular, perhaps eventually raising it to the level of a cultural classic. Gobs of money get made, awards get handed out, everyone goes home happy. Isochronous broadcasting models simply don't reach the target audience, here. The good news is that there is a myriad, a plenitude, a veritable cornucopia of alternative distribution channels; and I absolutely guarantee that if there are any motivated fans out there, the show is one five-line crontabbed shell script from being redistributed without you - a win for audiences in only the shortest of terms.

O most inscrutable and confusing CBC - let's do this thing, shall we?

Thanks ever so much.

Slightly creepily yours,

Ben

To: CBC Radio (via)
From: [excised]
X-from-city: Toronto
X-from-province: Ontario

Dear Canadia site maintaners,

There may be a typo or wto on the front pge:

"New on the video page this week. Producer and director Greg De Clute speaks on the creation of an epsiode of the show."

"The first season of Canadia 2056 is now avaiable on CD from the CBC Shop."

I am as excited as everyone about the epsiodes now avaiable at the CBC Shop, but whiskey tango foxtrot - how did those zingers evaid the speelchecker?

Cordially your,

Bne

Current Mood:

annoyed

Dear internets, I owe you writings. Don't send the goons yet. My wife's been sick, my dog needed surgery, the recession has driven the price of keyboard presses so high... I see that you are unmoved.

Very well, then.

I'm frustrated with the BioShock. I want to buy it, but I can't; and I want it badly enough that my frustration has fuelled a letter rather than just a decision not to purchase the game.

The 2K Games BioShock forum has a thread on the topic. It seems that I am far from the only person who feels this way. Also, many of the people I asked for copyediting and proofreading advice mentioned that they would like to express support for my contention; it was not my intention to do anything but send this letter in by registered mail, representing only myself, but I certainly don't mind if others weigh in as well.

My plan is to print out the following next Friday (the 21st), modulo any good content advice received before then; to count up the number of people who respond to this post with anything containing the words "me too"; to include that number along with a link to this page as an addendum; and to send the whole mess to 2K Boston, 2K Games, and Take Two Interactive by registered mail. I'll probably toss over a copy to Elizabeth Tobey over at the Cult of Rapture, since she is keeping in touch with the fanbase.

I don't expect this to accomplish anything specific beyond venting my own frustration. I can hope, though.

Dear 2K Games,

Thank you for BioShock. It has lived up to the System Shock legacy, garnered glowing critical praise, and raised the bar for graphics, gameplay and writing. Congratulations: you've earned it!

On the other hand, BioShock has deeply disappointed me. I can't buy it, and I can't play it. Here's why.

My friends and I are avid, dedicated gamers. We have huge libraries encompassing classic adventures to the very latest console titles. Games are as important to us as books, movies and music; they are points in the history of the gaming industry and in our own lives. It's wonderful to dust off an old game and spend a rainy Sunday afternoon strolling down memory lane in the Great Underground Empire, or Britannia, or Citadel Station, or Hell.

Control software requires us to rely either on publisher benevolence for future replayability or on the skills of the game-cracking underground. I can trust only the latter, since the stance of the former has repeatedly been made crystal clear. I appreciate assurances of future unlocking, but only software crackers have delivered.

I use my computer for serious work, and I write software for personal, pedagogical and public purposes. A reliable, transparent and deterministic computer is a necessity for this, not a luxury. SecuROM makes even installing BioShock a non-starter.

I maintain many computers for my employer, family and friends, and myself; I know too well the headaches that SecuROM causes and I won't pay for that pain. I must refuse to support systems with it installed - reluctantly, since stranding friends is anathema, but refusal is the only reasonable action. SecuROM is not open or reviewed software. Its threat model includes the user performing actions on his or her own machine. It claims administrative control of the user's system. When opaque software does this, the computer can and must no longer be trusted.

I encourage my friends to do their own research, starting with your FAQ. Those who read up on the topic more often than not decide against purchasing your game.

Principle is also at stake. It is wrong and unfair that your ardent, honest fans must jump through hoops - managing activations, juggling discs, surrendering user choice about acceptable hardware and software - while those who play without paying, those whom you ought to discourage and inconvenience, instead enjoy a better experience.

Playing an illegitimate copy would perhaps satisfy my technical and historical objections, but it would neglect the equally important points of system and ethical integrity. I must emphasize that this is not an option.

Control software punishes only users who try to do the right thing. Regardless of intent, requiring that SecuROM be installed treats every user as a threat and a thief, and a lazy one, since cracked copies are widely available. It is technically painful; demonstrably ineffective; offensive and condescending; and it deeply compromises the playability of your game. This is not acceptable.

This is not acceptable.

I write regretfully, but with optimism. If I thought there were no chance of changing your mind, I would not have bothered. You can still do the right thing. I'll be the first in line to purchase BioShock when it is sold in an acceptable format, but neither I nor anyone I can influence will purchase games on your current unreasonable terms.

Your fan,

[me]

Scheme code, with no constant values longer than three decimal digits (i.e. AACS-Golf format):

(let ((^ expt))
  (* (+ -170
        (* 23 29 101 269
           (- (^ 2 11) 307)
           (- (^ 2 11) 175)
           (^ 3 4) (^ 7 3)
           (+ 89 (* (^ 2 7) (^ 3 8)))))
     (* 2 3 3 11 37 229 (- (^ 2 11) 855)
        (+ (^ 2 15) 645))))

In case anyone else wants to play AACS-Golf, the value you now want to factor is, in decimal:

92214563780560366130661709445338835634

Hexadecimal:

455FE10422CA29C4933F95052B792AB2

Segmented hexadecimal:

45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2

Binary octets:

01000101 01011111 11100001 00000100
00100010 11001010 00101001 11000100
10010011 00111111 10010101 00000101
00101011 01111001 00101010 10110010

Binary string:

01000101010111111110000100000100001000101100101000101001110001001001001100111111100101010000010100101011011110010010101010110010

The rules of AACS-Golf: using only the + (addition), - (subtraction), * (multiplication), / (division) and ^ (exponentiation) operators, and the ( and ) (parentheses) grouping operators, construct an expression which evaluates to the most recently compromised AACS processing key; no constant value may have more than N (base M) digits; score is given according to the number of constants which appear in the expression; lowest score wins.

My AACS-Golf entry for 3-base-10, above, nets me a score of 32. I'm sure someone else can do better. What's your score?

This is terrible, terrible code. Just stupendously terrible. The only reason I wrote it is because I was idly factoring by hand using dc(1) for about twenty minutes and got tired of it.

So please don't judge my coding, or Scheme, by the quality of the below. I know that the algorithm is braindead too.

(define (find-factors-seeded num . start-factors)
  (let ((bound (/ num 2))
        (decimation (apply * start-factors)))
    (let loop ((factors start-factors)
               (factor (apply max start-factors))
               (num (/ num decimation)))
      (cond ((> factor bound)
             (reverse factors))
            ((= 0 (remainder num factor))
             (begin
               (display factor)
               (display "\t")
               (display num)
               (newline)
               (loop (cons factor factors)
                     factor
                     (/ num factor))))
            (else
             (loop factors (+ 2 factor) num))))))

(define (factor-hd-dvd-processing-key)
; (find-factors-seeded
;  13256278887989457651018865901401704640
;  2 2 2 2 2 2 5 19 12043 216493)
  (find-factors-seeded
   13256278887989457651018865901401704640
   2 2 2 2 2 2 5 19 12043))

The Nintendo DS is a terrific piece of hardware.

First of all, it's a brilliant gaming device. The touch screen, though admittedly still ever-so-slightly tinged with gimmickry, has been used to great effect in a bunch of different games, and developers seem now to have a firm handle on when to use it and when to concentrate on the buttons instead. The games coming out for it are really, really excellent: I've put more hours into Castlevania: Dawn of Sorrow than I'd really care to admit, and Portrait of Ruin is edging its way up near Symphony of the Night and Metroid as one of my favourite side-scrollers of all time. The multiplayer aspects, too, are just superb. I've had bundles of fun playing New Super Mario Bros, Mario Kart and Metroid Prime: Hunters; the few times I've had the chance to play Tetris DS with more than three other people at the same time have been laugh riots; and playing over the internet is both surprisingly fluid and a whole lot of fun.

Second of all, the hardware is excellent and the hobbyist coder scene for the device is booming. Sure, there is the normal amount of game stealing taking place - I mean, that pall hangs over every console system - but there's so much original stuff being cranked out for the DS, and the games are so comparatively inexpensive, that there's really not much incentive to snarf ROMs. There is something to be said about being able to carry around a single card instead of a bagful of physical game cartridges, though, and that brings me to the point of this whole deal: aftermarket writeable carts.

Getting the DS to boot something other than an official game is a difficult process, or at least it used to be. There's a whole digital signature infrastructure in place as well as some other hardware- and software-verification shenanigans in place to prevent anything but official games from booting, but we all know exactly what these kinds of measures are worth: if motivated enthusiasts can physically touch the device, it's purely a matter of time before the thing is running Yet Another Tetris Clone and booting some variant of Linux.

Oh, wait, whaddya know?

First-generation attempts to boot user code on the DS were awkward: they involved faking out the verification routines with an actual game, then swapping in the user code using a few neat memory redirection techniques and a writeable cart in the secondary GameBoy Advance port without the DS firmware noticing. Crafty, but clunky.

Then came the PassMe and FlashMe tactics, the former which simply allowed booting of user code in the GBA port without requiring a real game cart and the latter which involved flashing the actual BIOS of the DS - inventive but risky.

Nowadays there are slick devices like the DS-X and the M3 DS Simply. The DS-X (around $120) looks like a normal DS cartridge with a pair of LEDs and a USB port on the top of it. Plug it into your computer and it shows up like a 512-megabyte flash drive, just waiting for you to copy over whatever programs or games you want to run on your DS; plug it into your DS and it will boot into a program launcher, ready to fire up whatever you've copied into its internal store. The M3 DS Simply (closer to $45) does exactly the same thing except that it lacks the LEDs and has no internal storage: instead, it accepts MicroSD cards (which, these days, can hold upward of two gigabytes apiece and which cost approximately Who Cares, They're So Cheap It Doesn't Matter).

Devices from the previous generation - the ones that slot into the secondary GameBoy Advance port on the DS and need a (Pass|Flash)Me to boot (like the Supercard Lite or M3 Perfect/Pro, usually around $40 these days) - are still useful for two reasons. One, if you opt for a full GBA-sized cart, you can also use it in your actual GBA hardware (which may not sound like a big win until you remember that the GBA has a serial port that the DS lacks, making it much easier to plug hobbyist hardware into the older system); two, earlier models accomplished their boot-and-switch tricks by having a small amount of internal RAM - usually about 32 megs - that they would copy code into before running, and this RAM can be used by DSLinux for its own nefarious ends... like running IRC clients, web browsers, SSH clients and so forth.

Hmmm, Forth... Perhaps even Scheme, too? Mwahahaha!

So for about eighty bucks - two games - plus the cost of a MicroSD card or two, you could be running all kinds of nifty programs and carrying around your entire collection of GBA and DS software, and all in a slick colour-matched package that makes your game machine look like it was made that way.

Oh boy oh boy oh boy!

(via jwz, of course)

A young man once found himself, as one usually does at the beginning of these stories, in the wrong place at the wrong time. He reacted in an atypical, coolheaded and quick-witted fashion, highlighting by turns his above-average physical prowess, attractive but not overwhelming good looks and his downright innate likability. Unfortunately his reasonable, even laudable actions drew him into a very complicated situation: there were more people who needed and asked for his help, a group of organized competents who unfortunately were motivated at cross purposes to the ones asking for help, and - as always - an attractive, feisty young lady who for some reason had managed to rack up a long string of romantic failures. It must have been her shining idealism, or perhaps just that she placed her career as a doctor, scientist, lawyer, professor or crusading mammal-saver above her own wants and needs.

Things went from bad to worse for the young man as his actions and inaccurately expressed and acted-upon belief in The Right Thing curried equal parts of favour from those who needed help and ire from those who wished that he would just butt the hell out. Involvement, curiousity, belligerence and self-righteousness raced to the boil in both groups. More and more important people got drawn in on both sides, and eventually towering spires of political, economic, electronic and pyrotechnic power were grabbed and swung by both sides like snapped-off pool cues. The fallout left the young man and woman bruised and aching, but their friends, families, psyches, hometowns or personal lives were permanently scarred or destroyed.

After a lull conveniently timed to last just long enough for the grieving process to progress from despair to incandescent rage, the young man and woman marshalled their resources and draw upon a couple of heretofore concealed or unavailable reserves of information, inspiration or firepower to mount one final offense against the enemy before Time Ran Out. They would have succeeded, too, except for just one or two subtle but vital flaws that, in their haste, they had neglected to iron out of their plan. Just when all hope was lost, however, the young woman revealed both the extent of her commitment to the cause and the depth of her feelings for the young man by sacrificing herself for him, incidentally providing just the opportunity that he needed to secure Final Victory against the antagonists and, in the ensuing chaos and upheaval, race off with her to the nearest medical facility.

He was too late. He had won the day, but the cost was terrible. All he could do was take solace in the fact that the young woman had wanted it to be this way, and that all of his actions, no matter the cost exacted upon his entourage by choice and by chance, were The Right Thing To Do At The Time.

The knowledge was cold comfort, though, and despite the crackling fire in his getaway cabin, his warm rug, his cup of tea and his purring lap cat, the winter seeped into his mood and his bones. He was no longer so young a man. He couldn't shake the feeling that things weren't supposed to end this way. Where was the triumph? Where were the fanfares, the necessarily secret but vastly powerful favours and debts owed to him by highly placed allies? He heaved a sigh, raised his steaming mug with both hands and inhaled, long and slow, a curl of steam that smelled of bergamot and home and her. He stood, weathering the cracking of his joints and a reproachful feline yowl, and paced over to the immense library that covered the entire wall of the cottage. He stared out the window for a while, studying snow blowing through moonlight.

Then he turned resolutely to the As and started looking for a new story.

1F 8B 08 00 13 CE 6C 45 00 03 65 94 4D 76 DB 30
0C 84 F7 3D 05 76 3E 17 44 42 26 2B 92 60 F9 13
45 3D 7D 07 94 D2 E6 BD 2E 12 DB 12 04 7E 98 19
C8 0B 7B E2 60 FF FB 68 D2 3B ED EC 86 36 2A B1
08 C5 57 A6 A2 27 05 49 35 96 37 0D A5 D1 38 16
1A 41 A8 C8 49 EF 79 91 DA 4F DC ED F8 E0 41 81
3F A4 BC 06 6D 22 85 6A D3 2A 2D 5D 24 9F 35 E1
41 F1 D6 23 A3 F3 AA B3 1F 87 48 A5 59 E9 8C 23
90 7E 48 A3 3D E2 4E 98 C5 37 94 4B E6 98 3A 31
79 BE E8 0C 31 09 75 6D 63 D1 04 C9 C4 CE 69 F3
0F 5C 6D 51 5B 1C 17 71 F1 E4 85 93 5D 5F 8D 0D
78 07 70 EC 7D 4A C7 F1 8E 0B 10 8A 47 3F B6 A1
FB B0 CF 30 33 17 E0 56 ED 3D 6E B8 97 2F EA C9
08 BB 0B E2 27 AE C4 0E 00 8C D0 3B E8 C0 BD 86
8E D4 45 A8 8A 56 AB 28 C0 ED EA 22 27 72 5A 86
7C 0E 88 E4 70 10 9D 22 07 C5 7D 09 9B A6 3B 2E
7C C3 B0 6F 1D 86 8F E2 83 74 82 A4 29 C6 C7 B8
90 30 2B D0 9E CE DC 50 2B 63 4D 1F 14 32 EE 4D
33 9D DA 0E D8 34 8C EB A1 F1 6A 16 48 59 84 9B
58 3D D3 6F CD 5B 94 C7 3B 30 99 A2 9B 38 9E 5D
EE E7 4E 9D C9 D3 CF 89 03 93 2C 7B 50 9A F1 93
1B 8C 28 23 36 81 36 20 C3 77 0C 9F 04 A4 49 91
93 59 1C A8 37 FB 7B FA D9 8C 0F C5 5B 6D B6 CD
BC E4 15 85 A2 F3 1D 56 0E E0 06 09 2C BF AC C9
B0 69 FF E5 C2 B7 08 35 4C EA E9 02 A4 D9 77 59
D7 41 03 7F 93 B4 F1 AD D3 E3 E4 F7 1C C2 B9 37
5C 6D 1A 01 DB 21 19 DC 40 78 1D C6 8C C5 23 02
A3 4B 02 65 C1 E3 0C 48 84 08 91 38 4C 47 4B 4F
FF 35 45 7E 23 2A D8 85 CC D7 76 4B F4 EA B7 3C
26 CC B3 2F 56 8C D0 00 29 81 BE 21 74 98 F1 E4
B2 1C F5 BA F2 12 21 0C BB E3 8E 22 AF 20 EE 2D
8A 61 58 7B D8 D0 14 2D DB DD CD AF 15 63 C2 CA
0D FA 5A 45 BB B1 14 B2 3A 01 47 83 09 EB AA 13
9C BC C5 94 EE A7 1D 5B 6E BF FD 0E 2D F6 91 F9
E9 CD 39 83 26 DA 24 01 B6 62 DD 6C A9 AF BA C4
EB 8B D0 12 58 61 E9 98 3C 22 B6 1B 08 BD B2 BB
57 0E 93 AE 29 CC 6C 5E 39 81 1C 07 0A 8D C9 69
FE 7A 4D E8 09 54 CD DA DA 7A 7F 2C 64 33 4A D7
8B 04 17 88 A7 8F AB FF 11 DD 01 3A 68 D9 C1 36
30 E3 25 1C FE C3 E5 B4 56 A1 23 30 60 7A B9 A4
50 F4 EB 88 75 1F E2 7B EC 6B E0 75 86 ED A6 E5
0E 8D 96 B8 30 4C 33 23 4B E3 84 20 06 50 FC 53
81 77 D4 2E 6E C0 B8 80 24 E2 90 73 AD 11 96 F4
F1 F3 F6 1A 11 E6 CD D4 81 50 01 63 F5 20 2F 68
3F F8 40 E0 87 65 FE 09 E0 1D FC 0A AD AC FA AF
0C A6 59 C4 3B D3 62 F3 65 E3 8A 93 C5 FF 44 AC
41 B3 23 9C F6 D4 8F 3F 95 FF 5F 91 9A 05 00 00

Current Mood:	stressed
Current Music:	Big Rude Jake - Buster Boy (Walk Tall)

I just had a lovely weekend, a very pleasant evening blitzing through about half of Steven Gould's Jumper (ill-gotten because I can't convince Chapters to order a copy and I haven't yet taken the time to apply for a library card - mea culpa), and a decent dinner. I've got a nice week shaping up and an excellent weekend to anticipate.

So why do I have this furnace of directionless rage boiling away inside? What the hell is the deal with that, anyhow? I keep hoping and trying to grow up and get to know myself well enough to figure these things out and deal with them, but I seem to be just ever so slightly better at inventing twisty mazes of internal logic than at navigating them.

Bah. Time to hermit for a bit until it passes, I guess.

Current Mood:

enraged

Hoo boy.