Keyboard clacking and the purr of a white noise generator.
Literally, the dog whining. In a more abstract sense, the looming uncertainty of an unknowable future.
Once more it is June. Again. And again I embark upon that epic effort of daily blogging, take three, wherein I call upon myself for a kind of rambling focus, picking from a list of daily topics, and with neither planning nor advance writing, strive to pepper this blog with the free-thought, free-writing wonder that is another one of Those 30 Posts in June. Today, that post just happens to be:
June 19th // Something You Are Listening To
The thing about spending most of your average weekday in a shared environment is that quite frequently you have little or no control over said environment.
Each day I sit myself down just before eight in the morning. Each day I lean back in my chair and settle in with a fresh cup of coffee. Each day I make my to-do list for the upcoming hours. And each day at about nine in the morning, at least for the last month or so without fail, the noise starts.
Construction. Jack-hammers. Saws. Hammering. Pounding. Crashing. Metal on pipes. Grinding. Power tools interfacing with wood and steel. Screws squeaking. Materials scraping. Construction.
As a result I work wearing headphones. I spend an hour here and there away from my desk with paper and pen. I wince. I flinch when the sounds seem like the ceiling is about to collapse. I hold my head in the palms of my hands when the rumble gives me a head ache. And I endure. Just endure.
And then I come here, to home, and hope for some quiet. Or, at the very least I try listening to something different.
I’ve been cleaning up my server. Long-story-short, my hosting provider abruptly informed me a couple weeks ago that they were discontinuing support (ie. turning off) an older version of the database that is hidden quietly behind — but vitally important for the operation of — this site.
So, I’ve been cleaning up my server.
It didn’t affect this blog, but when I crawled into the digital root cellar that is my server and had a look around it became abundantly clear that a couple years worth of old projects were due to be vaporized in a little less than a month if I ignored both the warning and the mess.
So, I’ve been cleaning up my server. (Which goes a long way to explaining the thinned trickle of posts lately, by the way!)
Long Lost Treasures
Of course, like anyone who’s been living somewhere for over a decade can attest, junk piles up. And a server, it would seem, is not much different. In the past I’ve apparently spent time building little sites that have served long lost purposes, or have flopped, or for whatever reason were stuffed into dusty folders and ignored for many years.
I’ve found some odd, interesting, and occasional cool stuff, let me tell you.
One of the interesting things that caught my eye and my attention this past few days was some work I was doing on trying to mess around with approximating a true random number generator.
Randomness is fun. And hard. There are loads of pseudo-random number generators, many of them built into programming languages and used by default in silly little applications that only need to give the appearance of randomness. For example, the header images on this blog that on each reload provides you, the reader, a slightly different look and layout for your viewing pleasure, makes use of the rand() function in MySQL or the rand() function in PHP, depending, to pick something marginally random from a slightly larger data set. Say, pick 4 feature articles from 10 potential candidates. Or, pick 1 header banner from 20 possible header banners. It only needs to seem random, and true randomness is not vital to the operation.
But true randomness, when true randomness is required, is important, too. Think about games of chance or when money or something else of value is on the line. Randomness matters.
So, at some point I had built a random number generator into the template of one of my blogs.
Noise and Code
It wasn’t so much a number generator, I suppose. It was a noise gathering tool. It was an entropy observation and recording system that gathered entropy from human interaction with the blog.
It was a noise gathering tool.
See, you are a random-like factor when it comes to the computer. When you click on something, load a page, click to another page, this is a kind of quasi-entropy. Entropy comes from many sources, and I’m sure a student of entropy maths (whatever that is) would argue that there are numerous flaws in my logic, but hundreds of visitors spread across random locations and a variety of influencing factors — coupled with the only moderate popularity of this particular website — means that, say, converting a few digits of the time-of-reload time-stamp to a binary string, jumbling that together with all the other bits of entropy, and outputting it as say, a series of four-digit binary numbers, gives something like this:
1100 1111 0000 0001 1101 1001 0010 1100 1001 0011
0010 0100 0100 1101 0100 1101 0001 0000 0010 0010
1000 0000 0110 1001 1100 1001 1100 1001 1001 0010
Or maybe this:
1110 0011 0001 1010 1000 0010 0110 1110 1101 1001
1111 0010 1110 1110 0111 0100 1001 1011 1110 1011
0011 0110 1100 1001 1011 0001 1011 1100 1011 0001
It’s quasi-random, and someone with more understanding of the topic and a bit better grasp of statistical mathematics could probably poke some holes in my method.
But in the meantime, I might just dust off this long lost bit of treasure code and make something interesting out of it. If nothing else, more randomness on this blog might not be a bad thing…
If you are curious about playing around with these numbers, the ever-changing output of this entropy file is available here. I’d be interested in your analysis… so comment, write, or whatever…
I could find all kinds of excuses for my idle lack of posts here, but the biggest and juciest of them all would be NaNoWriMo, of course. The month-long writing challenge kicked off a little more than two weeks ago, and — as of this writing — I’m managed to generate a thirty-thousand word gush of semi-respectable draft fiction (an this on top of accepting and prepping for a new job, looking after my daughter more often than normal, dealing with the brutal arrival of winter, and trying to polish of the last three of my outstanding contracts before the end of the month.)
Some would call that crazy. I call it keeping busy.
The curious thing about that writing-urge, particularly during an event such as NaNoWriMo — that, for the explanation of those out of the know, is the International challenge to voluntarily signup and write a 50K word novel draft across the month of November, fully, completely — is that we who participate in such silliness often find that the lingering stress of trying to cram in our daily writing tends to mellow out the rest of our daily obligations to a white noise of ‘just get it done-ness’ that, while important, is still just noise.
It’s a good thing I have an understanding and supportive spouse.
It also means that unneccessary writing — such as that one would find in blogs and other personal websites — is temporarily condemned to the written-word equivalent of the ‘fix-that-dripping-tap’ chore, the last thing one really wants to do — unless there is a worse alternative. That doesn’t make it an evil option per se, just an option that doesn’t happen to get done in the fractional amount of time regularly allotted in any given day to writing. There is an exception, however, it seems: when one reaches one’s point of writer’s block and one needs to write something besides the ongoing dialog or description that has been pounding through one’s fingers for the last nineteen days, and instead explain and justify why one is doing it in the first place.
You can read about — and the results of — my writing efforts at the novel’s website, www.vudupulse.com
I was merging onto the freeway near our house. The road was wet and the temperature was a degree or two below freezing. And you can probably see where this is going, right? I fishtailed a little bit, remembered by defensive driving skills, and regained control of the vehicle a few second before the adrenaline even had a chance to circulate all the way through to my heart. A split second no control across a bit of random ice. What a city.
“I wanted to talk to you about random numbers.” I said to Syntax Gull who was sitting nearby reading from a comic book.
“Why?” She asks.
“You sound like my daughter.” I frown, and mimic a two-year-old’s voice. “Why daddy? Why?”
Gull’s look is reproachful but she closes her comic, sets it delicately on the table, and tips her gaze in my direction. “Go on then. What do you want to discuss?”
“No.” Gull says flat, “You wanted to talk. Go.”
“Look.” I plead. “I didn’t want to upset you. It’s only that I’ve been picking away at some ideas and I’ve got no one around who really has that kind of expertise. I thought your interest in information… content… you know…”
Her expression drops a few degrees of sour and she seems a little less peeved as she prods me for more info. She sighs and says. “You’re working with random numbers then?”
I nod. “Trying to… it’s complicated.” I say. “I mean I’ve often used random number generators that have been built into various coding systems. And in one of my little scripts that I’ve set free on the site, well, it is essentially a little randomized database query.”
“The built in stuff… at least as I understand them… should be more than adequate to your purpose. No?”
“Oh, yeah. For that kind of stuff.” I agree. “Sure. I mean we’re not doing anything too complicated nor important for true randomness there. But it did get me thinking…” I shrug. “You know I was hitting the reload on the page over and over, and it started with a kind of test of the system, evolved into a trial to see if I could get certain entries to appear, and eventually…”
“You started noticing patterns?”
I smiled. “My little human brain noticing patterns amongst the noise of random entries? Not really so much patterns, as gaps, I guess.”
“Gaps?” Gull looked puzzled now. Her annoyance with my earlier comments obviously fading quickly away. “How so?”
“Well, it’s funny really.” I started. “You know I would add new entries to the system and then do a rough estaimate on how long… how many page reloads… until I should see it, right? But my math never really matched reality. An entry would either appear far too frequently or.. the other way around.”
“But isn’t that a case for randomness?” Gull smirked. “Doesn’t the lack of predictability lend ot the arguement that the system is random?”
“I don’t know.” I shrug. “But the point of the story is not how random those generators really are, but only that it got me thinking about how one might measure the randomness of a system.”
“This is leading somewhere, isn’t it?”
I nod, grinning. “So, last week I write a little script and plug it into the footer of one of my well-trafficked websites. That site gets about a hundred hits per day from all over the place. The script is fairly innocuous. It uses a few cycles of server processing power to generate a timestamp on every load of the page. The script then takes that timestamp, bites off the least significant digits at the tail end — the microseconds, or something — and spits out a short string of zeros and ones. Those zeros and ones are the binary representation of three numbers it’s clipped off the time. And then that little binary number is appended to a little file I’ve got stuffed away in a nearby directory. It all happens in the blink of an eye and every time the page is loaded.”
“For what purpose?” Gull furrows her brow.
“It’s noise. It’s meaningless numbers generated by the clicks and clacks of humans.” I say. “And it should be random… theoretically. Well, here’s an example of the output.” I call up the text file in the browser and copy a few dozen characters into a local word processor.”
11011010 11001100 11011111 01111001 11001100 00111001 10011111 01100101 00011101 10101100 11001100 00001110 00100000 11100001 01110110 10000000
“This is a bit of sample output from the Noise Collector, as I’ll call it.” I say. “I’ve broken it it into, let’s say arbitrary, eight-bit chunks.”
“Noise Collector? Sure.” Gull smirks. “Now what?”
Well, the trick is to see how numbers start appearing statistically in this data. But eight bit chunks becomes rather cumbersome for this kind of small sample size. A sample spread like that does have more meaning, but one starts looking at 256 data points and that gets complex for an early bit of work. I mean if I could collect ten thousand chunks of that size, well, maybe… but that’s step six or seven. Let’s call this step two.”
“What about step one?”
“Right. Well step one was a quick analysis of the distribution of zeros and ones. I collected about five thousand zeros and ones and found that the distribution was roughly fifty-fifty.”
“Is that good?” Gull asks.
“I don’t know. But it seems logical that in a random spread of data the distribution should be evenly split.”
“Okay.” Gull smiles. “So… in step two you gathered what again?”
“Oh. Well. I used the same data but I broke it into two bit chunks. So the data looks more like this…” I add some spaces to the data in the word processor.
11 01 10 10 11 00 11 00 11 01 11 11 01 11 10 01 11 00 11 00 00 11 10 01 10 01 11 11 01 10 01 01 00 01 11 01 10 10 11 00 11 00 11 00 00 00 11 10 00 10 00 00 11 10 00 01 01 11 01 10 10 00 00 00
“The result here is that we essentially get four options for randomized distribution… and I might add, the very first puzzling result I encountered.”
“So, essentially what I’ve done is I’ve created four options when one converts the binary to decimal.” I type:
00 equals 0
01 equals 1
10 equals 2
11 equals 3
“…and, if the distrubution holds true, what would you expect to see?” I ask.
Gull thinks for a moment and then says matter-of-factly. “One quarter across the board, I guess. Twenty-five percent for each, given the proper sample size.”
“I woud have argued that as well. And maybe there is something quirky or strange going on that I have yet to figure out, but I’ve done the stats on the whole set of about twenty-five hundred points as well as parcelled out sub-sets of that same data. I don’t get 25 percent all around. I get this.” I type again.
00 = 26% frequency
01 = 26% frequency
10 = 26% frequency
11 = 21% frequency
“That doesn’t make sense.” Gull furrows her brow at the screen. “What now?”
“I guess I have some more work to do.” I shrug.