Beau's tumblr

Introducing.. dbexpect

Sat, 22 Sep 2012 08:49:00 +1000

I’ve been working on this for a little while now. dbexpect is an rspec-like DSL for automated unit testing ETL processes. For those not in the know that’s code that moves data. Data migrations, business intelligence data warehouses, nonsense like that.

The point is, it’s all focused around making it easy to set up data in a database, run your etl process, and then check expectations of what is now in the database. And it’s open source! Check it out https://github.com/C3/dbexpect

For the record the only other tool I’m aware of that’s targeted at doing this is Dbfit, http://gojko.net/fitnesse/dbfit/ which unfortunately last time I checked was pretty much end of life, and was a very clunky method for doing this kind of testing. Much more suited to testing code that is actually stored in the database.

Oh, I should quickly say many thanks to my employer C3 Products. For allowing us the time to go off and create what we’re interested in and then give it away for free. http://www.c3products.com.au

CSS a go go

Tue, 10 Apr 2012 22:54:14 +1000

Ran into some weirdness just now while reading the basic but well put together intro to haskell http://yannesposito.com/Scratch/en/blog/Haskell-the-Hard-Way

me: jump to 3.2.4
me: and see if you can spot that oh so rare bug that wouldn't be picked up by the compiler
mate: 3.2.4?
mate: heading?
me: search for 3.2.4
me: "Trees" is a bit general
mate: 0 hits
me: the fuck..

So…

<h4 id="trees">Trees</h4>

No 3.2.4 mentioned anywhere in the html. Ok. Check the CSS.

.article #afterheader h4::before {
  content: counter(niv02)"."counter(niv03)"."counter(niv04)". ";
}

I don’t know whether to be impressed by this or horrified. On the one hand, aint CSS cool these days? On the other, I can’t rely on these section numbers, and at least in Chrome I can’t search for them or copy+paste them either. I think that section numbers probably fall more under content than structure.

Topological sort of a DOT Graph

Tue, 27 Jul 2010 23:09:52 +1000

Today I needed to get a sequential execution order for a system that I already had a DOT dependency graph of, which I knew contained no cycles.

Currently I don’t officially have access to.. well, anything resembling a development environment at all really, so I was faced with the prospect of doing this in vba. As my masochism is underdeveloped I haven’t actually tried this yet, but given I can barely read vba I’d estimate this task at at least a few hours, with a worst-case blowout of a day.

Obviously I didn’t do that, so below is the code to do it in perl, research time 15 minutes, development time 5 minutes. Let me say that again: 20 minutes, versus a few hours. Multiply that out by the amount of time you hire programmers for and how much you pay them and see how well your “risk of getting a virus mitigated by locking down all IT resources” strategy is stacking up.

use Graph::Reader::Dot;
use Graph;

my $reader = Graph::Reader::Dot->new();
my $graph = $reader->read_graph($ARGV[0]);

my @sorted_nodes = $graph->topological_sort;

foreach(@sorted_nodes) {
  print $_,"\n";
}

Tl;dr Let the people doing the work decide on the tools.

(I used perl instead of ruby because I couldn’t find a gem to read DOT files)

"The idea that I can be presented with a problem, set out to logically solve it with the tools at..."

Fri, 09 Jul 2010 11:22:37 +1000

“The idea that I can be presented with a problem, set out to logically solve it with the tools at hand, and wind up with a program that could not be legally used because someone else followed the same logical steps some years ago and filed for a patent on it is horrifying.”

- John Carmack

Oracle enhanced adapter for Rails 3

Tue, 22 Jun 2010 08:50:39 +1000

Oracle enhanced adapter for Rails 3:

It makes me very happy that someone continues to work on this.

Martin Fowler on team room layouts

Mon, 14 Jun 2010 22:54:08 +1000

Martin Fowler on team room layouts:

My favourite bit:

First make sure it is the right size for the team. While a team room should be open within itself, it should be closed to everyone else

Yes, a million times yes.

Interview with Jim Coplien

Tue, 01 Jun 2010 11:33:07 +1000

Interview with Jim Coplien:

I really enjoyed this interview (but I’m weird that way) with Jim Coplien. Goes over many many things, the rise of functional programming techniques, the prevalence of class-oriented programming over object-oriented programming, the DCI architecture and some general thoughts about Scrum and Agile and just plain old professionalism.

It’s also the first mention I’ve seen of DCI, which led me here, which is also very interesting.

It’s all much more articulate than my “it’s easy to construct and interact with the domain model, but where does the doing go?” refrain at least.

Apparently if I end with a question mark people can ‘answer’, but not otherwise. Stupidest feature ever by tumblr?

Flay for cucumber

Tue, 25 May 2010 17:52:14 +1000

Flay for cucumber:

Quick very hacky job to get flay to give some duplication metrics on cucumber files, gives ideas on where your features are repeating themselves and should be refactored into higher level steps or at least moved into Background’s and Outline’s.

Only tested on a regular cucumber install run from the root directory of a rails app, likely won’t work anywhere else.

"There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings..."

Sat, 01 May 2010 08:44:30 +1000

“

There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged.

One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world.

The other, of course, involves orcs.

”

- @jeremydmiller who doesn’t know the original source

DRX - ruby object inspection

Thu, 29 Apr 2010 11:12:13 +1000

This is so freaking cool source

Steps to get running on ubuntu:

sudo apt-get install libtcltk-ruby tk-tile
sudo gem install drx
irb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'drx'
=> true
irb(main):003:0> 123.see
=> nil
irb(main):004:0>

Anyone who ever saw me use ‘ddd’ will understand why I love this.

Weird history

Wed, 21 Apr 2010 17:52:41 +1000

Ada Lovelace - Commonly known as the first programmer

chiefly known for her work on Charles Babbage’s early mechanical general-purpose computer, the analytical engine. Her notes on the engine include what is recognized as the first algorithm intended to be processed by a machine;

fathered by

Lord Byron - Famous poet, “mad, bad and dangerous to know” who used his place in the house of lords to support the Luddites

His first speech before the Lords was loaded with sarcastic references to the “benefits” of automation, which he saw as producing inferior material as well as putting people out of work.

I wonder if daddy was proud? :-)

Behaviour of next in collect

Wed, 21 Apr 2010 10:32:00 +1000

Basically, anyone got a better way to do this? :

irb(main):003:0> [1,2,3,4,5,6].collect { |v| next if v == 3; v * 7 }.compact
=> [7, 14, 28, 35, 42]

Let me know on twitter if you have a better way.

I oversimplified my example, it’s actually more like:

objects.collect do |obj|
  begin
    QueryPredicate.new(obj.parser.parse_internal)
  rescue ParsingException => e
    next
  end
end.compact

Which prevents me using the suggestion of

[1,2,3,4,5,6].reject{|v| v == 3}.map{|v| v * 7}

as I don’t have any way of telling what can be rejected from the set. Still, I learnt the “reject” keyword, which makes me happy.

Suggested improvement from colleague:

objects.collect do |obj|
  begin
    QueryPredicate.new(obj.parser.parse_internal)
  rescue ParsingException => e
    nil
  end
end
objects.compact!

Yeah that’s a bit better

Podcast by DHH on planning, VC, and MBA's

Mon, 19 Apr 2010 23:25:54 +1000

Podcast by DHH on planning, VC, and MBA's:

David Heinemeier Hansson is the founder of ruby on rails, partner at 37signals, and co-author of Rework.

Pretty standard stuff for DHH really, planning is guessing etc etc, still it’s an interesting talk, particularly if you haven’t read/heard him before.

Always like hearing the “go small” spiel, applies in so many different places, two quotes from Warren Buffet:

“I don’t look to jump over 7-foot bars: I look around for 1-foot bars that I can step over. “
“If I was running $1 million today, or $10 million for that matter, I’d be fully invested. Anyone who says that size does not hurt investment performance is selling. The highest rates of return I’ve ever achieved were in the 1950s. I killed the Dow. You ought to see the numbers. But I was investing peanuts then. It’s a huge structural advantage not to have a lot of money. I think I could make you 50% a year on $1 million. No, I know I could. I guarantee that. “

I think that’s a pretty similar sentiment to what DHH says “one million dollars is a lot if it’s all going into your bank account”, and yet Warren Buffet has nothing to do with software (famously avoided the tech bubble), just happened to realise the same thing.

TDD/Refactoring/Least possible Videos

Mon, 19 Apr 2010 00:14:19 +1000

TDD/Refactoring/Least possible Videos:

If you have a spare couple of hours (and let’s be honest, now cricket season is over who doesn’t?) handy these videos area really worth watching. They’re sort’ve billed as an intro to TDD, which they are, but they also pretty well demonstrate all of the code side of development, test, code, refactor etc.

It’s a bit like getting a feel for what “Clean Code” is like just by sitting back and watching for a while.

Rails content_for stacks content

Fri, 16 Apr 2010 09:36:17 +1000

Worth remembering, the content_for helper appends subsequent calls rather than replacing.

So would render “Hello world!”.

Seems kind’ve insane to me, if I wanted that behaviour I could always add it to content_for rather than doing it by default.

Video presentation by two of the original SVN developers talking...

Tue, 13 Apr 2010 22:39:00 +1000

Video presentation by two of the original SVN developers talking about social good-practices with programming and the way different tools affects that.

Thoughts while watching:

“fail early, fail often” - Pretty much a theme throughout, and fair enough
code reviews - So many big name places seem to take this dead seriously
no code ownership - Code ownership always tends to be bad, liked the suggestions on how to lessen it though, ie
- assign bugs/features to people with a low knowledge of that area
- pair program bugs/features for that area low-knowledge/high-knowledge
big fish, small pond - No doubt it’s true that working with more skilled people than yourself is good for you, but it can be pretty hard just to recognise what you want to learn let alone find the people/environment that supports that
‘bus factor’ - heh, awesome name
git-rebase - I dunno that lots of people use this to rewrite history? I use it to avoid pointless noisy merge commits
documenting failure - Seems pretty obvious that doing this would be helpful, but as with all documentation I have no idea how to do it in a way that would be useful
subversion - I understand these guys fondness for svn, but it’s so far behind in so many areas, and I think github-like tools affectively alleviate their problems with distributed version control more than they acknowledged
pair programming - I think this is the one core practice detailed by XP that’s still seriously controversial. I have no opinion :-)

People who ask tangential societal questions after a talk drive me nuts. “really in reality” points are seriously annoying as well. “buyer beware”. Piss off, buyer is beware, just assume (as the presenter does) that the audience is as capable of understanding the realities of their own situation as you are.

Update to bash back button

Tue, 13 Apr 2010 16:12:00 +1000

My new .bash_aliases

alias cd="pushd >/dev/null
alias bd="pushd +1 >dev/null"
alias fd="pushd -0 >/dev/null"

My bash prompt now has a “back directory” command, *and* a “forward directory” command!

mongrel_rails, logrotate, and copytruncate

Mon, 12 Apr 2010 11:05:41 +1000

If using logrotate with rails remember to use the copytruncate option. This means that instead of renaming the old file and creating a new one, logrotate will copy the old file to the archive name, and truncate the current file. This means that mongrel_rails doesn’t lose its file handle and your logging doesn’t silently turn off :)

On The Value Of Fundamentals In Software Development

Tue, 06 Apr 2010 11:15:55 +1000

On The Value Of Fundamentals In Software Development

How I learned to let my workers lead

Fri, 02 Apr 2010 21:14:22 +1100

How I learned to let my workers lead:

Really interesting story on what is essentially the implementation of Agile/Lean at a manufacturing company.