Jared's Weblog

Main Page - Books - Links - Resources - About - Contact Me - RSS

Jared Richardson Most Popular
Help! I've Inherited Legacy Code
Testing Untestable Code
Continuous Integration... Why Bother?
Mock Client Testing
The Art of Work
Technical Idiot Savants
Targetted Skills Aquition vs Free Range Chickens
The Habits of Highly Effective Developers

Blog Archive

Tue, 15 Nov 2005

A Great Test

There are several characteristics a great test should have. I've discussed the topic recently and decided to create a blog entry about it. It will also be a part of a book I hope to start early next year.

By the way, bear in mind that I'm not talking about unit tests. While they do share some characteristics, they don't require external resources like external data.


First, the test (or it's caller) configures it's environment. If the test needs data in a database, wipe the old table, recreate the schema, and re-add the data. This can be done once per test run (as opposed to once per test) but leaving old data laying around is just asking for data integrity issues to cast doubt on the resutlts your tests returns.

Anything you can do to make your tests rock solid and relable, you should do. Recreating data for each test run is an important step. Use tools like dbUnit to easily reproduce your data.


A great test runs.

That is to say, nobody has to run it. Once a test suite is set in motion, nobody needs to push a button, start a server, click a dialog or run a script.

Nothing is more frustrating that seeing someone create an "automated" test that doesn't run without human intervention. Sometimes people get so close, but forget to stop short before the edge of the cliff.


Next, a great test passes or fails. A mediocre test has you look at log files to determine the pass or fail status.

Build enough smarts into your test that it can either pass or fail without your help. It already has enough smarts to test the code. Add just a bit more and let it know what the result is. This frees you from the boring chore of reading log files every morning.


A great test also fails from time to time. If it never fails, then it never catches a problem. You still get a lot of benefit from knowing your code is solid, but push the envelope with some of your tests.

There are few ways to move in this direction.

  1. You can target active code development. Put a few tests in place before the refactoring starts. These tests will ensure that your work doesn't change the code's function. This is great way to start your performance tuning, bug fixing, or feature adding work.
  2. Wrap tests around bugs as you find them and then test the code close by.
  3. Target code that's had historical problems. The odds are good that the routines that caused problems last year have undiscovered issues. Get test coverage in place to help uncover those unknown issues as well as keep the state of the code stable when addditional bug fixing occurs.


Finally, a great test cleans up after itself. It doesn't leave test droppings throughout the system that could corrupt the next test run or have unintended consequences.


A great test leaves behind enough of a message to determine where the failure occurred and what caused the problem. A great test returns more than a stack trace. It also returns a short message that tells you exactly what it looked for and didn't find.

Sure, someone could go code diving to find the failed test and attempt to decipher the intent of the author. But a great test doesn't require that. It tells you what happened.


This is a start... what more? What do you think about when you write a great test? What annoys you in a bad test? What test patterns trap the mediocre tests before they reach greatness?

Tell me.


posted at: 21:01 | path: | permanent link to this entry

Dancing Yoda: Update

I recently posted a link to a great video clip... Master Yoda break dancing. I've since learned that the Yoda clip is an Easter egg on the Revenge of the Sith DVD. You can find the directions to access the clip here.

The real story behind the egg is very interesting. The entire story is posted on Star Wars.com.

Apparenty Michel d'Annoville started the project as a surprise for the animation director... it turned out so well that when Lucas saw it, he insisted it go on the DVD.

But don't take my word for it. Read the entire story. :)



posted at: 20:14 | path: | permanent link to this entry

Run Linux on your Windows desktop

You may already know that VMWare is now giving away their virtual machine run time.

What does this mean? Think of Adobe and PDFs. They give away the PDF reader (Adobe Acrobat) but they sell the tools to create PDFs. This is the same concept. You can run a virtual machine with the VMware player but you can't create one.

But you don't have to create a virtual machine. You can just download one. :)

Check out the VMWare Virtual Machine Center. Just by registering you can get VMs for Red Hat Enterprise Linux, Novell's Suse, etc. You can get WebLogic and Websphere as well.

You can also download their Browser Appliance . It's a prebuilt Ubuntu Linux image with FireFox already installed. I'm running on my desktop at the moment and it's quite responsive.

One of the cool features that I didn't expect is how it preserves state. Initially you "boot" into your virutal machine, but when you exit it preserves the state of the memory. When you restart the virtual machine image, it brings you right back to the same spot. Did you leave the browser open? It'll still be open. Cool.

This looks like a really easy way to keep different web browsers around, different versions of an operating system, etc. I can see this being very useful for developers or testers who need another environment around... or just someone who wants to tinker a bit.


posted at: 19:26 | path: | permanent link to this entry

Sun, 13 Nov 2005

Keeping continuous integration relevant and fun

I found this blog entry this morning and I had to pass it on.

Some clever folks used the Ruby-DOOM API to create Doom maps based on the number of PMD warnings!

This is an idea that people should grab and run with... how many other tools could use something like this? CruiseControl? Cobertura? FindBugs?

It's amazing how much can be done with a some general knowledge, a little bit of specific knowledge and a dash of creativity.

What have you thought about creating this week? ;)



posted at: 09:17 | path: | permanent link to this entry

Fri, 11 Nov 2005

Dancing Yoda?

I've often wondered what Yoda was doing between the two trilogies, during that long exile. Now I know.

What else can I say? It's humor. You'll love it.

Yoda link

This is why they made Google Video!


posted at: 18:01 | path: | permanent link to this entry

Sat, 05 Nov 2005

Beyond Java: What's Next?

I just finished reading Beyond Java by Bruce Tate, author of several books including Bitter EJB, Better, Faster, Lighter Java and Spring: A Developer's Notebook. Let me start by saying that I really liked the book.

Who should read this book?

Anyone who writes Java for a living. In fact, anyone who works with code and also looks ahead of the game to see where things are headed next. Bruce has said that he's really good at seeing things from 10,000 feet high. This book gives you that type of high-level perspective but then it layers in practical application.

What's the big picture?

Bruce starts with the environment and situation that helped Java get started and then helped it thrive. He then talks about where Java is today, where it's headed, and how the situation parallels the early days of Java. He shows how the time is ripe for a new disruptive technology to emerge. Java continue to focus on the enterprise, and it's doing very well there, but the barrier to entry to for the new developer is getting ever higher. As Java gets more and more complex, the time is becoming more ripe for a new low-end replacement. Something that easy to use, easy to get started with and easy to become very productive with.

There's an good overview of a number of languages ranging from Perl to PHP to Smalltalk and where each one is positioned in the market. Bruce comes down hard on the side of Ruby, Rails and continuations... or at least hard on the side of the next big thing sharing a lot of the same traits those technologies bring to the table.

What's to like?

There are several parts of this book I really enjoyed.

I liked the historical overview of the Java language in it's early days. I worked in Smalltalk long enough ago to remember some of the history. The perspective of Microsoft's role in the rise of Java was also refreshingly honest instead of the obligatory Microsoft bashing so many books include.

There is an entire chapter devoted to introducing you to Ruby, another for Rails and a third for continuation servers. Each of the introduction chapters was excellent. Very well written and easy to follow. I've actually gone through the Ruby snippets in the book and been able to run them all. I consider the sign of a good book when I am pulled in enough to actually run the code samples.

What's not to like??

The book is about half history and half future with a strong bent towards Bruce's vision of the future. If you see what Bruce sees and agree, you'll love this book. If you don't agree with where Bruce sees the industry headed then I think you'll get annoyed by the Ruby focus. But I can't complain about anything else in the book. I really liked it.

What's the Summary?

What were the factors that led to Java's rise? Are those factors in play again to bring another technology to the forefront? If so, what will that new technology look like? How can I get a head start with the new technology?



posted at: 13:46 | path: | permanent link to this entry

Thu, 03 Nov 2005

Defect Driven Test Creation

Defect Driven Testing is a great way to start accumulating testing coverage on a product that's already in the field. It's a pretty simple idea that lots of people are already using under different names.

The steps are simple.

  1. Find a bug (also called a defect) in your product. Okay, I know this never happens to ~you~, but other people in your shop can use this. ;)
  2. Write an automated test that exposes the bug. This test should fail with a message that points out the defect.
  3. Take the test that fails and then work on your code until the test no longer fails. You know the code is fixed when the test stops failing. If the test doesn't fail anymore but the bug isn't completely fixed, fix the test you have already or add another one. This step is a lot like Test Driven Development.
  4. Add the test to your Continuous Integration system so that it will be run every time the code changes. This will keep the defect from sneaking back into the product. When the defect does show up, laugh loudly and stomp it out again.

That's it. It's pretty simple stuff but very effective. What are the key points here?

First, write the tests for the areas of the code that need tests. Whether this code is under active development or is just intricate, you'll be adding tests directly on top of the code with known issues. It's possible that other areas of your code need automated tests more, but you can address those areas when you're actually fixing bugs in those areas.

Second, you'll be adding tests in familiar code areas. I say it's familiar because you've been tracking down bugs in the area. This means you don't have to go learn a package to write tests for it. You're already invovled. Take advantage of that.

Third, bugs generally come back to haunt you. Many developers keep multiple copies of the code on their machines. When they fix a bug in one tree, they don't always get the fix into the other trees. Then they commit code in the other trees along with your pesky defect. Instant bug regression. You can "innoculate" your code against regressions by testing in a Continuous Integration system.

Also, when you add one test, be sure to play a little test Jazz. Variations on a theme. You'll often find more than one bug in an area.

Try out Defect Driven Testing the next time you have to track down a bug and squash it. You can build a very effective test suite in a short amount of time.

btw, related blog entries are Types of Tests, A Short JUnit Tutorial, and JUnit is not a Unit Testing framework. Sorry for the Java bias. I plan on adding more dotNet resources as time allows, but the ideas do apply.



posted at: 20:10 | path: | permanent link to this entry

No Fluff Just Stuff was awesome!

I spoke in both Atlanta and Reston's NFJS software symposiums and had a great time. I think I had a better time meeting and getting to know the other speakers than presenting... but presenting was a blast too.

I'm trying to give you a flavor for the types of people you'll run into and get to talk with if you get a chance to attend a NFJS. So here's a quick list of a few of the people I got to spend a few minutes with over the last two weekends, in no particular order.

I'm just going to stop at this point. The post would go too long if I told you about getting to talk with Dave Thomas, Mark Richards, Floyd Marinescu, Paul Duvall, Ted Neward, and others. Let me just say that attending a NFJS symposium will give you a chance to mingle with some very smart people and potentially pick their brains about the future of our industry. I consider myself very fortunate to a part of the conference.

Btw, those of you who were kind enough to fill out evaluation forms for me in Atlanta and Reston, thank you. I made the cut and should be on the tour next year. I appreciate all your feedback and hope to incorporate your comments into next year's talks.

posted at: 17:51 | path: | permanent link to this entry