The Opiate of the Masses . . . ?

Karl Marx once said, in effect, that Religion is the opiate of the masses.  To be exact, this is the English translation of the quote according to Wikipedia:

"Religion is the sigh of the oppressed creature, the heart of a heartless world, and the soul of soulless conditions. It is the opium of the people."

I was thinking about this recently because I had an experience which brought this quote to mind, and about how off target this is - at least in our time.

Religion, or Faith?

He was wrong primarily in mistaking religion for faith.  It is tempting to give Marx the benefit of the doubt and assume that he understood the difference, but I don't believe that someone who has never experienced genuine faith in something greater than oneself can really know what it is.  Faith is not a drug to dull our senses and numb our emotions - it is a genuine devotion.

He further believed that he could substitute devotion to the collective - or more specifically the government institution - for a genuine faith.

Why is he wrong?  Because Opium induces a delirious high - perhaps helping the addict to release cares due to the drug.  True faith, however, yields hope - or, in the words of the Apostle Paul:

Galatians 5:22-23
But the fruit of the Spirit is love, joy, peace, forbearance, kindness, goodness, faithfulness, gentleness and self-control. Against such things there is no law. 
Galatians 5:21-24 (in Context) Galatians 5 (Whole Chapter)

If you compare this list to what Opium use produces, the comparison becomes laughable:

• Lying or other deceptive behavior
• Sudden worsening of performance in school or work, including expulsion or loss of jobs
• Loss of motivation and apathy toward future goals
• Withdrawal from friends and family, instead spending time with new friends with no natural tie
• Repeatedly stealing or borrowing money from loved ones, or unexplained absence of valuables
• Hostile behaviors toward loved ones, including blaming them for withdrawal or broken commitments
• Regular comments indicating a decline in self esteem or worsening body image

The true Opiate of our Times

So, if religion is not the opiate of our times, then what is?  I mentioned earlier that an experience I had brought this to mind.  I was feeling a little down at work (combination of stress and disillusionment with lack of ability to finish projects), and I put on some music - specifically Schubert's Winterreise.  I immediately felt an emotional ambivalence - the very effect I had hoped for.  I realized then how we use music in so many different ways in our world - from entertainment to mood alteration and even behavior modification.  Looking at it objectively, there are very few times in our lives when there is complete silence.  Music in the car, in the elevator, when we are on hold on the phone, in movies, blaring at sporting events in between any pause in action, etc.  In fact, situations which lack music seem lackluster and boring.  Clearly this pales in comparison to opium use, but that was Marx's choice, not mine, and I believe it is a more appropriate comparison than religion since the goal of music is similar, even if the results are dissimilar.

Our cultural infatuation with Music-like sounds (Most of the time not musical at all) was demonstrated to me when I went to an OU / Texas game at the Cotton Bowl.  Music was blaring the entire time there wasn't action directly in front of us - and it was horrible.  An acoustic assault on our senses.  Just because I love football, does that automatically mean I like AC/DC??!?!?  I am not sure the intention was even to entertain musically, but rather to incite an intensity and rush that is expected of a big-time football game.  It also aids some drunken fans in getting up for the stupid fights that sometimes occur afterwards.

The worst part is that I have spent a significant portion of my life as a musician, and music lover, which means in affect that I am both a dealer and a user.

Hi.  My name is Brad - I am an addict.

Exceptional JUnit Rules

I recently got aggravated at the JUnit capability for handling and testing expected exceptions.  Usually it looks like this:

@Test(expected = IllegalArgumentException.class)
public void testBadConditions()
{
}

That works just fine when first assembled, but what happens when that same exception class is thrown by a different piece of code, for a different reason?  The answer, the test passes.  This hit me when I was implementing a builder pattern in my etl-unit framework using a TDD approach, and my tests for validating parameters would pass, but when I added more validation the same exceptions were being thrown for different reasons (because I altered the order of validation) so the tests were still passing, but parts of code were being hidden from test cases and were no longer being covered.  When using TDD you tend to lean heavily on the unit tests, and things like this can be a big problem (tests passing when they should not).  I like to know right away when I have broken something.

I had a couple of options that I knew about:

1. I could use a different exception class (or subclass) for each validation rule.
2. I could use try/catch blocks in my tests and manually check the exception class and message.

Option (1) sucked, and in some cases I couldn't do it anyway, so I ruled that right out.  The second would allow me to use distinct messages to make them verifiable, but made for ugly tests (my opinion):

@Test

public void testBadConditions()

{

  try

  {

    causeAnError();

    // in case an exception is not thrown

    Assert.fail();

  }

  catch(IllegalArgumentException exc)

  {

     Assert.assertEquals("expected message", exc.getMessage());

  }

}

I didn't care too much for that - way too much noise in my tests.  I initially googled it and found, as expected, a bunch of irritating, condescending posts about why this is always bad and JUnit would never implement this.  The @Test annotation indeed does not support adding a message, but I have been using JUnit rules a bit recently and I hoped there might be a rule for this.  Suspecting that it would be called ExpectedException, I googled for that.  To my delight, I found it.  Here is an example of the above test using this rule:

@Rule public ExpectedException expectedException = ExpectedException.none();

@Test

public void testBadConditions()

{

    expectedException.expect(IllegalArgumentException.class);

    expectedException.expectMessage("expected message");

    causeAnError();

}

@Test

public void noExceptionThrownTest()

{

 ...

}

I really like the way this looks.  It is functionally equivalent to the previous example.  The declaration of the @Rule is initialized with ExpectedException.none() so that the default state before each test is not to expect an exception, and noExceptionThrownTest does not expect an exception as usual.  Then, at the start of the test, as with other JUnit rules, you tell the rule to expect an exception class and then give it the message to expect.

Other things you can do with this rule are specify a regular expression for the message, and assert an exception cause class type.

I'm not sure I like this better than having it in the @Test annotation, but I do like it way better than my alternatives.

Ignoring JUnit

In the past I have been against ignoring broken tests (usually accomplished by commenting-out the @Test annotation) - usually looking something like this:

//@Test
public void brokenTest()
{
  . . .
}

Generally when I have been involved in a project where this happens those tests get ignored for so long that they will eventually be thrown away.

Recently, however, overwhelming demand for this feature in my etl-unit testing framework (I.E., it was suggested by one developer and I liked it) meant I had to do some refactoring. When I was in the middle of  changing some code in a common module I wanted to get an idea for where I was in the bigger project.  I suspected JUnit had an @Ignore annotation and I was right.  So, instead of the above, this is what I had:

@Ignore
@Test
public void brokenTest()
{
  . . .
}

I liked this much more, for a few reasons:  (1) my IDE, Intellij IDEA, will run all the tests, but leave the ignored ones highlighted to call attention to them, (2) a simple search of the project will reveal everywhere this has been used, (3) this serves as a reminder to me of how much work I have left to do, and (4) I can do a full multi-module build to evaluate parts of implementation as I go.

I do prefer my own implementation for a few reasons.  The JUnit @Ignore annotation has an optional value attribute which can hold a reason for the ignore.  In etl-unit, the @Ignore annotation has a mandatory reason attribute which should contain a description of what is broken, etc (reason seems so much better to me - value is too vague).  Etlunit goes a step farther, however, and adds an optional reactivate-on attribute which holds a date after which time the test is no longer ignored, but rather than executing will throw an ERR_IGNORE_EXPIRED error, or a user-specified error or failure can be substituted, the latter being used when you would prefer for the test to fail rather than an error being generated.

I really like the reactivate date since these kinds of tests can easily be forgotten about, and when your continuous integration server builds start failing you will have a not-so-gentle reminder.

Life without Caffeine . . .

I had to give up caffeine, fizzy drinks, and a few other pleasant things a few months back because a Doctor told me I am getting old (I had no idea!), and that has revealed some things about myself I wasn't entirely ready for.  For one, I rely WAY to much on caffeine, and it wasn't easy to get it out of my system.  I was irritable for a few days - like I usually feel when I take in too much caffeine.  Secondly, without being able to just drink a cup of coffee (Bleh!!) or, better yet, a Doctor Pepper, to get me past the afternoon, I now have to manage my biology in an entirely different way.  For one, I have to sleep (What a concept!), and when I hit a dip in the afternoon it's a reminder that I am not getting enough sleep, and correcting that is no small task.  Gone are the late nights working after the kids go to bed in the solitude and serenity of the living room - a time which formerly was my most productive for working on open source projects and other things I didn't have to actively account for my time to anyone.

In my case, just stopping the caffeine was an unworkable solution - unless I am okay with my evening commute inevitably ending in a fatal accident due to exhaustion.  I feel so much better not ingesting all the sugar and caffeine and other toxic crap that soft drinks and coffee bring (I've never cared for energy drinks), but it is more than just a change in drinking habits - it's really a change in how you manage your body.

Be Systematic

Years ago when my children had cousins who were into such things, I heard a cheer (ad nauseam) which went like this:

Be, Aggressive!B-E, Aggressive!B-E AGG-R-E-SS-IVE Aggressive!

While being impressed that you could effectively get a 7-year old to spell 'Aggressive' by putting it into a catchy cheer, a parallel occurred to me from my own experiences.

Be, Systematic!
B-E, Systematic!
B-E Sys-t-e-matic Systematic!

Some background.  In my career, one of the most frustrating things I have encountered  happens when trying to brainstorm a problem.  It usually goes something like this (in a group setting):

Facilitator:  So, the widget is failing every day.  Any ideas?
Suspicious Engineer #1: I suspect it is the hoozit of the whatzit.
Overconfident Engineer #1: Naw - it can't be that - the hoozit is finely-tuned.
Facilitator: Any other ideas?
Suspicious Engineer #2: What about the Ides of March?  It has been known to cause problems before?
Overconfident Engineer #2: Not anything I have seen.
Self-proclaimed SME and overall dominant personality:  I think it has to be muppets in the server room.
Facilitator:  I'll bet you are right!
All:  Let's go get those muppets!

The big problem here is the complete lack of discipline in testing conclusions and challenging assumptions.  The facilitator did the whole team a disservice by allowing dominant personalities (I.E., those with who make the strongest assertions) to squelch good ideas just by saying it isn't likely to be true.  A good facilitation of this sort of meeting should collect both pieces of information:  the identification of the problem area, and the likelihood (priority, or rank) that it is the culprit.

Instead of just throwing ideas around like clay pigeons and letting people take shots at them, the facilitator should collect all ideas and rank them according to the order the team agrees on, then test each idea in order.  This works not only on teams, but especially in individual work.  I can't count how many times I have been told by an engineer that the problem was identified, as well as a fix, but once the fix was done they discovered that either (1) there was an additional problem that hadn't been identified, or (2) the analysis was wrong and the fix didn't work.  This is terribly frustrating for a team - thinking a problem is fixed only to have it pop up again.  It is even worse for the team when the fix is reported and deployed to production only to have another team (usually production support, or worse the business users) report the lack of fix back to the team.

In conclusion:  follow the scientific method.  Observe and analyze the situation, make a hypothesis, test that hypothesis, then commit to work.  Challenge yourself.  Don't believe your own conclusions (let alone others) until you can see it (as close as possible - some problems in computer science just can't be observed - such as a rare race condition).

Why you should do assertions the way I don't want to do them . . .

I have heard people say things like "In unit tests, each test should pass or fail for exactly one reason."  Practically, this means there should only be one assertion per method.  Yeah, right.  I've done the math.  That's a friggin' big pile of tests when you could have 20 assertions in one method!  And who wants to think up that many good test names (because runtest0 will only go so far)?  Am I right?

So, I coded a test like this, because there were multiple assertion validations with one condition, E.G.,

void testSomething()
{
 stage(data);
 assert(reason1);
 assert(reason2);
 ...
}

That worked fine, and met my goal of being green and saving the planet by reducing the number of carbon-footprint bearing test methods.  Then I refactored something that I knew would effect this code.  Unfortunately, I had to do a lot of creative commenting just to get the crap fixed because the entire test wouldn't pass until all the assertions passed.  What a pain.

So, I guess there is something to that after all . . .

Test Driven ETL Development

I did a screencast showing how my team uses Test Driven Development to develop ETL code.  Here is the link:

Test Driven ETL Development

The presentation is a bit long because I discuss some of the theory of what we do and demonstrate the entire end-to-end process.