Test Driven ETL Development

I did a screencast showing how my team uses Test Driven Development to develop ETL code.  Here is the link:

Test Driven ETL Development

The presentation is a bit long because I discuss some of the theory of what we do and demonstrate the entire end-to-end process.

Siri, Punctuated

I have been a little tired of dictating text messages to Siri and having them come across bland - having no punctuation to speak of.  I would ask "How are you?", and she would transcribe "How are you."  On a hunch I tried speaking the appropriate punctuation and she picked it right up.

I tried:

Me: "Where are you.  Where are you question mark.  Where are you exclamation point."
Siri: "Where are you.  Where are you?  Where are you!"

Just for fun, I also did these:

Me: "Are you quote happy unquote question mark."
Siri: 'Are you "happy"?'

Me: "I picked up the kids semicolon they are very whiny."
Siri: "I picked up the kids; they are very whiny."

Me: "Are you colon happy comma unhappy comma or sad question mark."
Siri: "Are you: happy, unhappy, or sad?"

Anyway - pretty cool stuff.

This is on iOS 7 - not sure if it works elsewhere.

Maven assemblies with file permissions (modes)

I have this maven project which packages up some shell scripts and batch files in a plugin.  On the client side, they get extracted into a local directory where they can be used for various functions.  Long story short, I was a bit irritated that on windows the batch files are treated as executables, but on Mac OS X and Linux, they were simply text files, and had to be launched with:

sh etlunit

Granted this is not a big deal, but I was annoyed anyway.  So, I Googled and found a directive to the maven assembly plugin which supports setting the file mode.

<fileSets>
<fileSet>
<directory>src/bin/</directory>
<outputDirectory>bin</outputDirectory>
<fileMode>0755</fileMode>
</fileSet>

  </fileSets>

I tried that, and for some reason now my bin directory (created by the plugin) was inaccessible - it's mode had changed inadvertently - to 0000.  On a hunch I added the directory mode as well:

<fileSets>
<fileSet>
<directory>src/bin/</directory>
<outputDirectory>bin</outputDirectory>
<fileMode>0755</fileMode>
<directoryMode>0755</directoryMode>
</fileSet>

  </fileSets>

And now I feel much better about the universe.  I can add the scripts to the path or use:

./etlunit

It's little victories like these that make the Open Source Software world so much fun . . .

Revenge, Justice, Mercy and Grace

[Disclaimer - this blog entry is about the Christian doctrine of grace]

I heard an illustration by Pastor Doyle today which contrasted these four ideas. I thought it was very powerful.

Suppose a man breaks into your home and brutally murders your only son. Your response can fall into these categories.

You hunt him down in anger and brutally murder him, inflicting the exact pain he caused your son.  This is Revenge.

You hunt him down, but once apprehended you turn him over to the authorities for punishment. Once pronounced guilty, he is executed according to the law.  This is Justice.

You hunt him down, this time you forgive him and leave him to live in his guilt and shame.  This is Mercy.

As before, you hunt this man down, and having detained him you extend forgiveness, but this time you invite him into your home to live in your son's bedroom, take his place at your table, and become your legal heir.  You love him as an adopted son and he is called by your name.  This is Grace.

The point drives home for me when I think of myself as the brutal murderer.  It's easy to think of oneself as the victim - the one extending forgiveness - but in God's sight we are all offenders differing only in degrees.

Romans 3:22-24

So, it is God who extends grace to us through his Son, Jesus - He is the only one who has the right to.

iOS 7 music and screen locks

Found my first really annoying iOS 7 bug. If I have music playing and the phone locks, when I unlock the phone the music jumps or starts over.

Especially annoying when shopping!

Time Machine Saved My Behind, Twice.

For the second time in two years, the drive on my iMac died.  The first one was a bad drive, I suspect the second was because I forgot to attach the heat sensor to the new drive.

Anyway, after a completely lost drive several years back I bought a Drobo and set up a RAID volume for TimeMachine - Apple's backup solution for Macs. So, when I woke up Tuesday morning to a clicking drive and a sad Mac, I only had to install a new drive, boot it into the recovery partition on my Drobo, and tell it to restore from the TimeMachine backup.    After 50 hours (!) my OS, computer accounts, my wife's photo library and my iTunes library featuring hundreds of movies, etc., were all restored.

Pretty cool.

So, I thought IntelliJ IDEA was screwed up . . .

My maven projects all started behaving badly.  No auto-completion, no auto-importing of classes, syntax highlighting wasn't working - I was getting really annoyed.  My initial thought was that the 12.1.5 update released last week must have been the culprit.

I was about to open a support case with JetBrains, when I noticed that an option called "Power Save Mode" was enabled (File menu, towards the bottom).  I turned that off, and presto!  Everything came back to normal.

Bottom line - don't use this feature.  It makes IDEA slightly better than a text editor - especially in comparison to what it normally does for you.