TestStudio vs. TestStand

I've been trying to learn about the Teradyne test sequencing product TestStudio. I wanted to compare it with NI's TestStand.

I'm going to say now that there is no comparison, it's not ready. TestStudio locks up so often that it's hard to focus on anything except the problems.

The main problem we're having is when we try to connect DLLs compiled by LabWindows CVI. About every third node (Test Step) that was set up to attach to a DLL would hangup TestStudio. The easier hangups only required TestStudio to be kill (via task manager) and restarted. Occasionally the PC would have to be reboot to continue working in TestStudio.

The Teradyne eknowledge site (the on-line help) was almost non-existent. I'm told by Teradyne that issues submitted by a company can only be see by people from that company. That means people can't see any product problems, issues, fixes, or comments . However, when we entered a problem in there on-line case tracking system, I did very quickly get a call to talk about it. Unfortunately I typically got excuses and that fixes we going to be in up coming versions.

My overall impression on TestStudio is that it's not ready. It's as if its an immature product only tested in on very specific uses. Quality control on the product seems to be left up to the user. The problem is that it's been around since about 2001 and a lot of these kinks should have been worked out.

Bottom line is that if you have a choice, go with TestStand. In my opinon TestStudio is not ready.

Sadness

I just want to let everyone know my Mother passed away last week from Pancreatic Cancer at the age of 86. She will be missed.

I'll continue my blog next week.

TestStudio vs TestStand

Teradyne has a software product called TestStudio that is a competitor with National Instruments TestStand. Since I've had my training class inTestStudio so I'm ready to review it. I waited for the training so I would have a better understanding.

I've noted before that the installation was difficult, each individual piece had to be installed separately so it kept me busy. I also had to install an older version of NI-VISA so that TestStudio would install and then re-install the latest.

The training was not very good. It was developed to be used with hardware available, which we don't have yet. This casued the instructor to skip several of the exercises and parts of exercises. A lot of the training material was vague in what you were suppose to do and there were quite a few error's in the material.

The product seemed very immature considering it's been around since at least 2001, possibly longer. During the training TestStudio kept poping up errors. One person had to change computers due to the unknown TestStudio problems. My TestStudio crashed and corrupted my project to the point I had to start over from scratch.

TestStudio is browser-based, of which I am not a big fan. But that is more of a personal preference and was one of the smaller issues. Overall, TestStudio has a lot of small Quirks. Some panels use "OK" buttons to save data, some the "X" in the corner and a few you use the File >> Save. A utility called Express Services, which is big in TestStudio, has to be specified at project creation, however, if a project is created without the Express Services option and it is decided Express Services are needed, you have to start over with a new project. Like many IE based programs, dialogue boxes can get covered up. TestStudio can appear to be closed eventhough it's not really due to a hidden popup. Also, the file browser does not always keep the location of the last browse.

One of the biggest annoyances is that when you add a new node (Test Step) an "are you sure" type of window pops up...every time! No option to not do have it pop-up is available.

Also, when passing parameters to DLL's from nodes (test steps) it is passed as char *. You can't just pass integers, doubles, etc. This limits the direct re-use of DLLs.

TestStudio's big feature appears to be setting up switch matrices between instruments and the UUT. It seems to do well at this but, since we didn't have hardware, we haven't had a chance to check this out.

Bottom line is, TestStudio is a poor second rate competetor to NI's TestStand. It's been around for several years, at least since about 2000, but seems to operate as a buggy and immature product.

Next week, I'm going to review more of TestStudio features (?).

NI Week is here!

It's finally the NI Week time of year and I am excited! I drove down early this morning. I found I couldn't sleep in spite of my need to get up early and be awake for the 3 hour drive down.

I've come on an Expo pass this year so my learning opportunities will be more limited than previous years but that doesn't dampen my desires to learn, network, and generally be nerdy...one of my favorite pastimes.

I am waiting for the Keynotes to start and thinking...WHO came up with the easy-listening, AC/DC re-mix of T.N.T. along with all the other sad remixes of good music. Basically, this my first disappoint in NI-Week in the last 3 years. I have a feeling I will get over it as the day goes on.

The NI Demonstrations were good but I really enjoyed Jeff K. future state of NI speech. I'm impressed evey year by his talks.

The Expo floor, of course, is very informational, lots of good information and potential uses. The unfortunate thing is, most of the information I gather will either be to new and different to be useful or will fall on deaf ears.

I am going to document what I find out and pass it on.

Again, this year, there is only one CVI demo and they won't let it run because it's to dangerous. It's a desructive test of a CD. Basically the CD crumbles. I guess CVI programmers are braver than LabVIEW programmers. I'm proposing a CVI controlled icamera and potato gun to take out the LabVIEW demo's...I don't think they'll go for it.

Loading TestStudio

I've tried to head into the comparison of TestStand and TestStudio with an open mind, be objective, filter out my biases. However, TestStudio has made this hard to do. Before I do a side-by-side comparison, I plan on learning a little about TestStudio.

We received and I loaded the TestStudio software along with some driver software for the hardware we were going to use. Once I began to explore TestStudio I looked in the help file for a Getting Started section, which I found. It was found to be lacking. It had short sentences like “Create One Library tree” but it had no links to how to accomplish this and no descriptions on how to do this. I searched around and figured this step out eventually, I'm pretty sure. But then the next sentence had the same shortcomings, short, to the point, with no "how" to do anything.

I decided to do it the good old fashion way, trial and error. Choose actions from the menu or right-click and choose actions from the list. I found how to create a step for an annoying “beep”...always fun. However, since TestStudio is browser based, I received a script error. I could not find the options menu item in the TestStudio browser, so I went to my Internet Explorer (IE) and Internet options to turn on scripting. Our IT department, in a fit of control and paranoia, had set no scripting and made it where ordinary users could not change it. While I'm not an ordinary user, I'm not going to do anything to my work computer that would get me in trouble...like give myself admin privs.

Luckily the “beep” node went in and worked in spite of the script error. Accomplishment! I decided to try attaching a node to a DLL. I got the same script error, along with a “Teradyne TestStudio DLL addin has encountered a problem and needs to close. We are sorry for the inconvenience.” message and an error pop up message with only a large red X, no text.

I gave up for the moment and was going to try to talk to the Teradyne guys to get some set up help. So I went back to work, fired up TestStand and immediately got a “Could not start” error. It worked last week, I loaded TestStudio, and now it doesn't work.

I had flashbacks to the Microsoft browser wars of the 90's when IE would cripple other browsers. I'm going to continue to learn, continue to try things, and try to continue to have an open mind.

NI Week...Woohoo

NI Week is almost upon us, it's the time of year to look forward to the new NI products and to network with all the NI people. I'm looking forward to the NI Week. I would be looking forward to all the seminar's and learning opportunities except that this year, some people at the place I work sees no immediate, short term benefit to conferences like this and AutoTestCon. Even in the long term it is hard to show measurable benefit. Since it's mostly secondary benefits, long term learning and networking, that is hard to measure, some groups will be poorly represented.

I am, however, going to to NI Week on an EXPO pass. I am excited about going and I am planning on making the most of my learning opportunities that I will have on my EXPO pass. I want to find out what various vendor's believe is the future of testing. I want to learn about the various products available to help us do our testing, I want to learn all I can...no matter what others think of conferences. I know there are learning opportunities, I know I am going to learn, I know I'm going to learn all I can, and I know it will help me in the future.

Overall, I'm excited about NI Week and all of it's opportunities.

Is Software Engineering relavent to Test Engineering

I've been trying to get a hold of Teradyne's TestStudio in order to compare it to TestStand. It's been about a month and a half since I've been trying to get a copy. Good news...sort of. I'm told my constant requests may have finially paid off. I'm told, Teradyne finally relented and is going to send us a preliminary copy of TestStudio. It may be in the building, it may be in the mail, so I haven't actually seen it. So, hopefully soon, I can do a comparison.

On to other things...

I've been thinking a lot about where Software Engineering fits into Test Engineering. While I understand the fact that it is an integral part of testing, I've noticed a prevailing thought, at least where I work, that software is secondary and trivial piece of the overall testing practice, "You can get anyone off the street to do the software." (qoute from a manager at work)

This thought was brought about when it seemed like I was the only one trying to get a preliminary copy of TestStudio. The tech lead dealing with getting TestStudio seemed unconcerned on getting the software tools while spending endless hours on getting exactly the right hardware. TestStudio was even picked as a software tool with almost zero input from people who deal mainly with software.

Due to the shrinking of many UUTs, a lot of tests are moving to software. Either the UUT is loaded with an Operational Test Program (OTP) or the UUT only has communications port, test software is becoming more and more important. People need more software expertise rather than less software software expertise.

I'm just frustrated knowing that most of the software decisions are being made by the software inept and I may not have a path up the technical ladder because of my leanings toward software.

If you're reading this, thanks for reading my vent.