Code and Unit Test

One thing I've noticed is that many test engineers, for coding and design, just hack something together so it can be thrown onto the test set. Sometimes it’s because of schedule pressure, sometimes it just more fun to touch hardware, sometimes it’s because it’s EE’s doing the coding with only one software class worth of training. In any case, this has got to stop!

Test Engineers (TE) doing software need to have some software discipline. They don’t need the same discipline as tactical or man-rated software development needs, but Test Engineers do need software discipline none the less. TE's need to do some unit testing, some stress testing, some fault path testing, Diagnostics levels, some issue like that.

Many TEs use the excuses of “our software is different” or “Our software is to hardware intensive” to avoid code testing. I say any 10 year old can give excuses, what about results? What is the quality of your software?

As I see it there are two basic types of software testing, internal testing and external testing testing.

First the external testing or unit testing. In the software world there is quite often a test group. The software developers develop software and throw it over the wall to testers. In the Test Engineering world, there's not always that luxury. That's the reason to have software self discipline. There's more to external testing but for now I'm limiting myself to unit testing.

Definition: Unit Test is the testing of a module or a logically grouped set of modules where inputs and outputs, limits, stress, path test occurs.

To test a unit(s) it can be as simple as a function calling the unit(s) being tested over and over with various inputs and then checking the output. If NI's TestStand is in use then a sequence is set up to call the module(s) under test and the output checked.

One tool that I use quite often is called CUnit. It's a unit testing framework and there is a whole range of _Unit type of frameworks. JUnit for Java, CPPUnit for C++, xUnit for MS Visual stuff (I.e. NI Measurement Studio) and some others I can't think of right now. A lot of this can be found at SourceForge.org, a great source of open source software.

Since test software is quite often hardware intensive, sometimes some hardware simulation is needed, and I use the word “simulation” in the weakest form. It's as simple as creating a dummy routine to replace the hardware instrument driver that returns the data you want to test with. I.e. good data, erroneous data, stressed data, etc.

To test, the driver routine would call the module being tested. The module being tested calls the driver simulation which returns a predetermined set of results which allows the module being tested to return a predetermined result. If the returned result is really what you expect, the test worked. Otherwise, the module needs to be fixed.

Overall, the point I'm making is not to hack something out and throw it on the hardware. There are options, whether you've thought of them or not. By The Way, this isn't a comprehensive list of ways to unit test. Use your imagination, I know you have one, you're an Engineer!

I just want people to step out of their little box and think “How can I make my code better?” and “What can I do to keep from giving incorrect results or give errors?”

Note: Some of these test methods are the part of Agile development called TDD, Test Driven Development (TDD). It just means your code should be driven by the tests. I didn't want to scare anyone with the word "Agile."

Zephyr - test management tool

Yea, this blog is about testing, usually about NI stuff, but competition is good for us all. Saying that, someone e-mailed me about a test management system called Zephyr. I read about it and it sounds like it has a lot of potential.

Zephyr is a complete test management system. It has collaboration tools, customizable dashboards for instant status, Test Desktops, with built in metrics, and test execution capability with defect tracking.

You can read about it's list of product features here and see an overview video here. There's also a limit person trial version here. It has some very good looking interfaces.



I have only read about the product and have not tried it out but it sounded like it had some very useful features. Since I have not actually used it I can only give you my impressions. Some things I liked were:

- Central repository for documents or at least links to the documents.
- Test/QA/Management Collaboration
- Instant update of test status
- It sounds like it was for Agile development with all the communication and collaboration, but I'm sure it can be used for any type of development.

The ability to manage widely distributed test systems is impressive and massively useful.

A couple of big questions, at least for me.

One, how well it plays with TestStand and other NI products. NI is the big dog on the block for test software and electronic, at least at Lockheed and to get a foothold it would need to complement our NI abilities.

Two, We have a lot of legacy code and equipment from NI that we don't want to lose. Since our previous tests cases are in TestStand written in CVI or LabVIEW, how well you can re-use your old tests? How well will it interact with the legacy code?

I'm probably download and try out the trial version but it would be in my spare time. Unfortunately, that wouldn't be until the end of the year.

All-in-all Zephyr looks like it has potential and I would like to hear from anyone who tries it.

Apologies

I want to apologize. I haven't been blogging much for a while due to my Mom being, first in the Hospital, and now in a nursing home.

Things are settling into more of a routine so I should be able to blog more, but she does come first.

Thanks