TestStudio vs. TestStand

I've been trying to learn about the Teradyne test sequencing product TestStudio. I wanted to compare it with NI's TestStand.

I'm going to say now that there is no comparison, it's not ready. TestStudio locks up so often that it's hard to focus on anything except the problems.

The main problem we're having is when we try to connect DLLs compiled by LabWindows CVI. About every third node (Test Step) that was set up to attach to a DLL would hangup TestStudio. The easier hangups only required TestStudio to be kill (via task manager) and restarted. Occasionally the PC would have to be reboot to continue working in TestStudio.

The Teradyne eknowledge site (the on-line help) was almost non-existent. I'm told by Teradyne that issues submitted by a company can only be see by people from that company. That means people can't see any product problems, issues, fixes, or comments . However, when we entered a problem in there on-line case tracking system, I did very quickly get a call to talk about it. Unfortunately I typically got excuses and that fixes we going to be in up coming versions.

My overall impression on TestStudio is that it's not ready. It's as if its an immature product only tested in on very specific uses. Quality control on the product seems to be left up to the user. The problem is that it's been around since about 2001 and a lot of these kinks should have been worked out.

Bottom line is that if you have a choice, go with TestStand. In my opinon TestStudio is not ready.

TestStudio vs TestStand

Teradyne has a software product called TestStudio that is a competitor with National Instruments TestStand. Since I've had my training class inTestStudio so I'm ready to review it. I waited for the training so I would have a better understanding.

I've noted before that the installation was difficult, each individual piece had to be installed separately so it kept me busy. I also had to install an older version of NI-VISA so that TestStudio would install and then re-install the latest.

The training was not very good. It was developed to be used with hardware available, which we don't have yet. This casued the instructor to skip several of the exercises and parts of exercises. A lot of the training material was vague in what you were suppose to do and there were quite a few error's in the material.

The product seemed very immature considering it's been around since at least 2001, possibly longer. During the training TestStudio kept poping up errors. One person had to change computers due to the unknown TestStudio problems. My TestStudio crashed and corrupted my project to the point I had to start over from scratch.

TestStudio is browser-based, of which I am not a big fan. But that is more of a personal preference and was one of the smaller issues. Overall, TestStudio has a lot of small Quirks. Some panels use "OK" buttons to save data, some the "X" in the corner and a few you use the File >> Save. A utility called Express Services, which is big in TestStudio, has to be specified at project creation, however, if a project is created without the Express Services option and it is decided Express Services are needed, you have to start over with a new project. Like many IE based programs, dialogue boxes can get covered up. TestStudio can appear to be closed eventhough it's not really due to a hidden popup. Also, the file browser does not always keep the location of the last browse.

One of the biggest annoyances is that when you add a new node (Test Step) an "are you sure" type of window pops up...every time! No option to not do have it pop-up is available.

Also, when passing parameters to DLL's from nodes (test steps) it is passed as char *. You can't just pass integers, doubles, etc. This limits the direct re-use of DLLs.

TestStudio's big feature appears to be setting up switch matrices between instruments and the UUT. It seems to do well at this but, since we didn't have hardware, we haven't had a chance to check this out.

Bottom line is, TestStudio is a poor second rate competetor to NI's TestStand. It's been around for several years, at least since about 2000, but seems to operate as a buggy and immature product.

Next week, I'm going to review more of TestStudio features (?).

Loading TestStudio

I've tried to head into the comparison of TestStand and TestStudio with an open mind, be objective, filter out my biases. However, TestStudio has made this hard to do. Before I do a side-by-side comparison, I plan on learning a little about TestStudio.

We received and I loaded the TestStudio software along with some driver software for the hardware we were going to use. Once I began to explore TestStudio I looked in the help file for a Getting Started section, which I found. It was found to be lacking. It had short sentences like “Create One Library tree” but it had no links to how to accomplish this and no descriptions on how to do this. I searched around and figured this step out eventually, I'm pretty sure. But then the next sentence had the same shortcomings, short, to the point, with no "how" to do anything.

I decided to do it the good old fashion way, trial and error. Choose actions from the menu or right-click and choose actions from the list. I found how to create a step for an annoying “beep”...always fun. However, since TestStudio is browser based, I received a script error. I could not find the options menu item in the TestStudio browser, so I went to my Internet Explorer (IE) and Internet options to turn on scripting. Our IT department, in a fit of control and paranoia, had set no scripting and made it where ordinary users could not change it. While I'm not an ordinary user, I'm not going to do anything to my work computer that would get me in trouble...like give myself admin privs.

Luckily the “beep” node went in and worked in spite of the script error. Accomplishment! I decided to try attaching a node to a DLL. I got the same script error, along with a “Teradyne TestStudio DLL addin has encountered a problem and needs to close. We are sorry for the inconvenience.” message and an error pop up message with only a large red X, no text.

I gave up for the moment and was going to try to talk to the Teradyne guys to get some set up help. So I went back to work, fired up TestStand and immediately got a “Could not start” error. It worked last week, I loaded TestStudio, and now it doesn't work.

I had flashbacks to the Microsoft browser wars of the 90's when IE would cripple other browsers. I'm going to continue to learn, continue to try things, and try to continue to have an open mind.

Little about TestStudio

I had hoped to start comparing TestStudio to TestStand but with all our efforts to actually get a copy of TestStudio or much information about it have been thwarted by the people we're trying to buy it from. All the T's and C's haven't been finished up so we can't get an advanced copy. And, as I've said before, NI and their 30 day trial period pretty much set's the standard.

I could compare the Tech Support...but wait, I'm running into the same problem. We can't get into the e-knowledge until we actually purchaced there product. While I'm not saying the makers of TestStudio has anything to hide, what are they hiding? Again, NI sets the standard of customer support.

While I do understand their need for money, I just don't understand the business model that operates in a way contrary to helping the customer.

I'm just venting my frustration, after all the years of dealing with NI, their support, their mature products, it's frustrating to run into roadblocks like these TestStudio roadblocks.

Comparing TestStudio and TestStand..Not!

Comparing TestStand and TestStudio at this point is...impossible. At the momment, I'm still waiting on the official PO to go to Teradyne so that they will be so kind as to give us the software we're buying. We have sent them 10% of the PO for long lead items but that isn't enough insurance to actually get the software.

It's starting to wear thin, I just want to learn a little about this monstrosity called TestStudio. I am assuming it is a steaming pile of ... code. Until I actually get to see a copy of it, I'm not going to change my opinion.

Once I get to play with TestStudio I really am going to compare it to TestStand. Okay, I do have one bit of pre-knowledge of TestStudio that I can compare to TestStand. Any parameters passed out of TestStudio to a DLL or other code is only allowed to be passed in Character strings. TestStand does it right and passes the value in the form it needs (i.e. integers, doubles, pointers, etc).

Last week the score was TestStand 2, TestStudio -2. The score is now TestStand 3, TestStudio -3 for TestStands ability to pass parameters correctly and Teradynes unwillingness to allow people who are going to use it to actually use it.

TestStudio, Hmmm...

It's been a while, I've doing a lot of playing with my Genetic Algorithms program, writing a Neural Net program, and studying Chaos.

First, Congratulations to the Wendy, former Queen of CVI. She's moved on MIT, graduate school, and opportunities up there. Way to go!

Second, Welcome Adri K, the new Queen of CVI and all NI things Text. Everyone needs to go to NI Week and say hi to her or the NI CVI community.

Now to the meat of the subject. TestStudio, not TestStand but TestStudio.Teradyne's offering in the test sequencing arena. It seems to be cheap knock off of TestStand put out by Teradyne. This is the product we'll be using on our next test set. It was not my choice or the choice of anyone who will have to develop on it.

Since this choice has been made for use and I've been told to accept it, I want to write about comparisons between TestStudio and TestStand.

First comparison...none. Teradyne, who is apparently very big on keeping the purchaser of the product in the dark, will not allow us to have an evaluation copy or access the support site until they get their PO.

We are told it's great, or at least the only product that will run its proprietary switch manger. We're also told the switch manager is a .dll. .dll's will run under TestStand, so why is TestStudio the only way to run the switch manager?

I guess I'll have to wait until we actually get the software before I can compare it. Since TestStand allows a 30 day evaluation and TestStudio doesn't, plus we can't get to the support. Is Teradyne's product so bad they can't even allow people to evaluate it?

TestStand 2, TestStudio -2. Negative since I can't even get to any documentation except Propaganda sheet.