I've had some family and work issues lately but I'm going to continue this blog, it makes me thinks about what's important in testing.
Lately I've been working with National Instruments PXI 6229 Multi-function I/O board and NI DAQmx drivers. Using DAQmx has made me think about flexibility versus Usability. First, DAQmx is a departure from the standard method of get a handle to a device and use it. DAQmx functions wants a DAQmx task to be created and used. The task is associated with a set of operations, whether they're Digital I/O, Analog I/O, counter operations, A/D operations, etc. The task is then started and runs however long it takes to complete or how long is specified in one of the many operations setups.
This allows the general DAQmx routines to operate on any type of board. The problem is usability. Not all boards support all operations. A Digital I/O does not support counter or clock operations. Not all operations are as clearly unfit as those. Also, it's not known whether the function will work on the board until the software is run and the function returns an error. The PXI 6229 that allows clocked operations but doesn't have a clock which disallows any "clock" DAQmx functions. (Clock operation require using the counter or an external clock on a PFI line) Things like this are not obvious. Another usability problem is that there are many clock functions and it's hard to tell which one is needed. When trying to operate the [nonexistent] clock on a 6229 it's not clear if the function itself is not usable in the situation or if the board does not support it.
The examples NI provides aren't for all boards. The examples won't run on all boards. Some of the Digital I/O examples won't run on boards with Digital I/O.
Overall, the flexibility the DAQmx functions allow inhibit the usability of the functions. NI needs to apply the KISS principle, Keep It Simple Stupid.
Wednesday, September 26, 2007
Thursday, September 13, 2007
The test development environment
This is less about computer development environment and more about physical development environment. In the past I've been in the unfortunate situation of going off-site to develop. It went slow and I got phyically sick at least once a week. This situation is about to happen again.
I bring this up because a test set we are working on is being sent to the customer's site. In the earlier parts of the contract the tests will be developed here. Due to the lateness of the UUT's being developed, the test development process will be moved to their site, where the UUTs will be. They originally promised to send UUT's for us to verify our tests in April 2007 but changed the agreement, rather ignored the agreement and changed what we had to do. That meant the we have to send a test set up to there site and the use their UUT's. The same UUT's that their engineers will be using at the same time we would like use them. Or maybe we have the midnight to 6am shift to use the UUT's.
This winds up putting all the pressure on us. We're in a strange place working the test set. If we need technician help we have to find one of there technicians to help us. Same with tools, seating, and, most of all, UUT's. We will be at their mercy on when we can use the UUT's and guess who will be at the bottom of the priority list for use of the UUTs. (answer: us)
The only good thing about this situation is we will have at least some access to the UUT's to complete our test verification. Nothing else is good about it. We'll be out of our environment, staying in hotels, away from family, and eating out all the time. It gets old quick.
As for me, I'm only going up there occasionally, I can get my stuff done down here. Since I'm a single dad, it would be tough for me to go up there for 3 months like some of the others.
Have a great day
I bring this up because a test set we are working on is being sent to the customer's site. In the earlier parts of the contract the tests will be developed here. Due to the lateness of the UUT's being developed, the test development process will be moved to their site, where the UUTs will be. They originally promised to send UUT's for us to verify our tests in April 2007 but changed the agreement, rather ignored the agreement and changed what we had to do. That meant the we have to send a test set up to there site and the use their UUT's. The same UUT's that their engineers will be using at the same time we would like use them. Or maybe we have the midnight to 6am shift to use the UUT's.
This winds up putting all the pressure on us. We're in a strange place working the test set. If we need technician help we have to find one of there technicians to help us. Same with tools, seating, and, most of all, UUT's. We will be at their mercy on when we can use the UUT's and guess who will be at the bottom of the priority list for use of the UUTs. (answer: us)
The only good thing about this situation is we will have at least some access to the UUT's to complete our test verification. Nothing else is good about it. We'll be out of our environment, staying in hotels, away from family, and eating out all the time. It gets old quick.
As for me, I'm only going up there occasionally, I can get my stuff done down here. Since I'm a single dad, it would be tough for me to go up there for 3 months like some of the others.
Have a great day
Tuesday, September 4, 2007
NI and Education
I'm currently involved with the Grand Prairie Independent School District (ISD) to help develop academic programs for a Technical High School currently being planned and built. When we started I found out, to my dismay, that GPISD only had a simple Intro to Engineering class. They do have a programming class under the area of business and an electronics class under science. But nothing to lead people into engineering. I was appalled at the lack of Engineering curriculum.
Now, I'm interested in helping them get some engineering programs in place for the new technical high school. I was looking at what academic information NI has. They have an NI Academic area on there website. It is mainly LabVIEW based academics...as if that were a surprise.
They also have NI ELVIS which a LabVIEW based electronic is an integrated design and prototyping platform for engineering and science laboratories to teach concepts in measurements, circuits, controls, microcontroller, and embedded design. It is a bit pricey for high schools and I think targeted to colleges.
Hopefully, the GPISD can afford to put in place a good engineering program. Whether they can afford NI or not is yet to be seen.
Project lead the way is a good, somewhat canned engineering curriculum. There are other school districts that use Project Lead the way and GPISD will probably follow suite.
Well, have a good day.
Now, I'm interested in helping them get some engineering programs in place for the new technical high school. I was looking at what academic information NI has. They have an NI Academic area on there website. It is mainly LabVIEW based academics...as if that were a surprise.
They also have NI ELVIS which a LabVIEW based electronic is an integrated design and prototyping platform for engineering and science laboratories to teach concepts in measurements, circuits, controls, microcontroller, and embedded design. It is a bit pricey for high schools and I think targeted to colleges.
Hopefully, the GPISD can afford to put in place a good engineering program. Whether they can afford NI or not is yet to be seen.
Project lead the way is a good, somewhat canned engineering curriculum. There are other school districts that use Project Lead the way and GPISD will probably follow suite.
Well, have a good day.
Sunday, September 2, 2007
Requirements help from NI
NI has a product I'm looking at right now. It's called NI Requirements Gateway and it's to help keep track of requirements. It won't help deal with poorly written requirements like "The user interface shall be easy to use by anyone". It will help you track the requirements from the requirements document to the code. Whether the requirements are simply documented in MS Word or MS Excel, or kept in a requirements database like DOORS or Requisite Pro.
I'm dealing with requirements written in MS Word. Specialized MS Word tags (similar to tags like Heading 1) can be put in the requirements document. Then the complimentary tags can be placed in TestStand and down in the code called by the sequence step. NI Requirements Gateway then extracts the tags and follow them down to the code. I know it works with TestStand, LabWindows, and LabVIEW. All NI Products, and figuring NI is doing what they do, I bet it can flow down into other languages, like .dlls built in C++, though I'm not sure about that.
It can come up with a requirements traceability matrix, a table to easily trace the requirements from the document, to the code. It assumes the requirements are specific and not vague, general requirements like the one above. But for real requirement it helps prove the requirement was implemented.
Happy Requirements tracing.
I'm dealing with requirements written in MS Word. Specialized MS Word tags (similar to tags like Heading 1) can be put in the requirements document. Then the complimentary tags can be placed in TestStand and down in the code called by the sequence step. NI Requirements Gateway then extracts the tags and follow them down to the code. I know it works with TestStand, LabWindows, and LabVIEW. All NI Products, and figuring NI is doing what they do, I bet it can flow down into other languages, like .dlls built in C++, though I'm not sure about that.
It can come up with a requirements traceability matrix, a table to easily trace the requirements from the document, to the code. It assumes the requirements are specific and not vague, general requirements like the one above. But for real requirement it helps prove the requirement was implemented.
Happy Requirements tracing.
Wednesday, August 29, 2007
Requirements are Required
Test Requirements are required for proper testing. The project I'm currently on, more or less, have requirements. I say more or less because there are requirements written down, but they're not signed off. I, as a sub-contractor, and I'm trying to follow the requirements as written, even though it's not signed off.
Today I was working with the contractor, going over the user interface. Some questions came up about how the operator interface worked, he wanted things done differently. At one point I pointed out that I did it according to the requirements. The contractor I was showing it to said "I don't care what that [the requirements] says, I want it like this."
To change something they want is not the problem, it's the fact that it's written one way and going to be implemented a different way. We could held accountable later for not implimenting the requirements correctly.
I'm not an anal retentive person, I'm probably more loosy-goosey, but I also want to protect my self. I've been in the situation where a boss told me to do things one way instead of the way the documentation explained it. Later, the same boss called me out in a meeting on doing it incorrectly. Since then, I do it the way the documentation says and if someone wants it done differently, then change the documentation. That's when I'll change it.
Basically, to protect yourself, write down what's wanted (requirements), agree to them, then follow them. If someone wants changes, change the documentation, agree on cost and schedule, and then implement the changes.
Bottom line, CYA!
Today I was working with the contractor, going over the user interface. Some questions came up about how the operator interface worked, he wanted things done differently. At one point I pointed out that I did it according to the requirements. The contractor I was showing it to said "I don't care what that [the requirements] says, I want it like this."
To change something they want is not the problem, it's the fact that it's written one way and going to be implemented a different way. We could held accountable later for not implimenting the requirements correctly.
I'm not an anal retentive person, I'm probably more loosy-goosey, but I also want to protect my self. I've been in the situation where a boss told me to do things one way instead of the way the documentation explained it. Later, the same boss called me out in a meeting on doing it incorrectly. Since then, I do it the way the documentation says and if someone wants it done differently, then change the documentation. That's when I'll change it.
Basically, to protect yourself, write down what's wanted (requirements), agree to them, then follow them. If someone wants changes, change the documentation, agree on cost and schedule, and then implement the changes.
Bottom line, CYA!
Tuesday, August 28, 2007
"We've always done it that way" - argh
That is one statement that has always bugged me. National Instruments (NI) is cutting edge, forward thinking, and a has an overall good product. Now, mix doing testing the same old way and modern technology. What does that leave you? A good overall mess, but at least it's eas to develop.
Sometimes, contractors will layout requirements for products, like User Interfaces (UI), the way they've always laid them out. And no matter what arguments you use, the want what they want.
NI makes their products very flexible. The good and bad is that it makes it easy to develop a cumbersome, less flexible UI for stubborn customers.
UI's should be independent of the UUTs to be tested or test sequences run. They should allow the tester to add new test sequences and remove old test sequences without changing the UI. NI makes developing UIs with this philosophy easy. However, UIs that are heavily integrated with the test sequences can also be easily developed. While this is good for future work and continued employment, it's not good adding or removing new tests.
But we all do what we need to do to get the job we're given done. So thanks to NI, we can easily develop inflexible projects as well as flexible projects.
Have a great day.
Sometimes, contractors will layout requirements for products, like User Interfaces (UI), the way they've always laid them out. And no matter what arguments you use, the want what they want.
NI makes their products very flexible. The good and bad is that it makes it easy to develop a cumbersome, less flexible UI for stubborn customers.
UI's should be independent of the UUTs to be tested or test sequences run. They should allow the tester to add new test sequences and remove old test sequences without changing the UI. NI makes developing UIs with this philosophy easy. However, UIs that are heavily integrated with the test sequences can also be easily developed. While this is good for future work and continued employment, it's not good adding or removing new tests.
But we all do what we need to do to get the job we're given done. So thanks to NI, we can easily develop inflexible projects as well as flexible projects.
Have a great day.
Monday, August 27, 2007
More On Fault Insertion
How about "Fault Insertion, More On". In the past I've seen two different attitudes on fault insertion. The first test engineer was diligent at inserting and verifying faults he inserted. It took a while but, and the test lead, has confidence that if there are faults in a UUT, the test sequence will catch them and report them correctly.
Another test engineer, said he didn't have time to insert and verify faults because he had to get his test running. I thought making sure your test would catch faults was part of getting the tests running. Very recently, a fault was put into his testing (I probably shouldn't have pulled the break out box shorting plug) and I noticed it wasn't caught. He just got annoyed that someone was messing with his test.
Another time, his test crashed instead of catching a UUT error. When I say crashed I mean his code had a fatal error and could not continue. He just said "That shouldn't happen". He fixed the UUT so that it didn't cause the error (a broken wire in a cable) and didn't fix the error.
So, which tester do you think was embarrassed at sell off time with quality assurance there, watching, keeping track of problems. So who do you think was the Fault Insertion More On?
Another test engineer, said he didn't have time to insert and verify faults because he had to get his test running. I thought making sure your test would catch faults was part of getting the tests running. Very recently, a fault was put into his testing (I probably shouldn't have pulled the break out box shorting plug) and I noticed it wasn't caught. He just got annoyed that someone was messing with his test.
Another time, his test crashed instead of catching a UUT error. When I say crashed I mean his code had a fatal error and could not continue. He just said "That shouldn't happen". He fixed the UUT so that it didn't cause the error (a broken wire in a cable) and didn't fix the error.
So, which tester do you think was embarrassed at sell off time with quality assurance there, watching, keeping track of problems. So who do you think was the Fault Insertion More On?
Subscribe to:
Posts (Atom)
