Introducing.. dbexpect

I’ve been working on this for a little while now. dbexpect is an rspec-like DSL for automated unit testing ETL processes. For those not in the know that’s code that moves data. Data migrations, business intelligence data warehouses, nonsense like that.

The point is, it’s all focused around making it easy to set up data in a database, run your etl process, and then check expectations of what is now in the database. And it’s open source! Check it out

For the record the only other tool I’m aware of that’s targeted at doing this is Dbfit, which unfortunately last time I checked was pretty much end of life, and was a very clunky method for doing this kind of testing. Much more suited to testing code that is actually stored in the database.

Oh, I should quickly say many thanks to my employer C3 Products. For allowing us the time to go off and create what we’re interested in and then give it away for free.

CSS a go go

Ran into some weirdness just now while reading the basic but well put together intro to haskell

me: jump to 3.2.4
me: and see if you can spot that oh so rare bug that wouldn't be picked up by the compiler
mate: 3.2.4?
mate: heading?
me: search for 3.2.4
me: "Trees" is a bit general
mate: 0 hits
me: the fuck..


<h4 id="trees">Trees</h4>

No 3.2.4 mentioned anywhere in the html. Ok. Check the CSS.

.article #afterheader h4::before {
  content: counter(niv02)"."counter(niv03)"."counter(niv04)". ";

I don’t know whether to be impressed by this or horrified. On the one hand, aint CSS cool these days? On the other, I can’t rely on these section numbers, and at least in Chrome I can’t search for them or copy+paste them either. I think that section numbers probably fall more under content than structure.

Topological sort of a DOT Graph

Today I needed to get a sequential execution order for a system that I already had a DOT dependency graph of, which I knew contained no cycles.

Currently I don’t officially have access to.. well, anything resembling a development environment at all really, so I was faced with the prospect of doing this in vba. As my masochism is underdeveloped I haven’t actually tried this yet, but given I can barely read vba I’d estimate this task at at least a few hours, with a worst-case blowout of a day.

Obviously I didn’t do that, so below is the code to do it in perl, research time 15 minutes, development time 5 minutes. Let me say that again: 20 minutes, versus a few hours. Multiply that out by the amount of time you hire programmers for and how much you pay them and see how well your “risk of getting a virus mitigated by locking down all IT resources” strategy is stacking up.

use Graph::Reader::Dot;
use Graph;

my $reader = Graph::Reader::Dot->new();
my $graph = $reader->read_graph($ARGV[0]);

my @sorted_nodes = $graph->topological_sort;

foreach(@sorted_nodes) {
  print $_,"\n";

Tl;dr Let the people doing the work decide on the tools.

(I used perl instead of ruby because I couldn’t find a gem to read DOT files)

The idea that I can be presented with a problem, set out to logically solve it with the tools at hand, and wind up with a program that could not be legally used because someone else followed the same logical steps some years ago and filed for a patent on it is horrifying.
John Carmack

There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged.

One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world.

The other, of course, involves orcs.

@jeremydmiller who doesn’t know the original source

DRX - ruby object inspection

This is so freaking cool source

Steps to get running on ubuntu:

sudo apt-get install libtcltk-ruby tk-tile
sudo gem install drx
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'drx'
=> true
irb(main):003:0> 123.see
=> nil

Anyone who ever saw me use ‘ddd’ will understand why I love this.