Comparing TestStand and TestStudio at this point is...impossible. At the momment, I'm still waiting on the official PO to go to Teradyne so that they will be so kind as to give us the software we're buying. We have sent them 10% of the PO for long lead items but that isn't enough insurance to actually get the software.
It's starting to wear thin, I just want to learn a little about this monstrosity called TestStudio. I am assuming it is a steaming pile of ... code. Until I actually get to see a copy of it, I'm not going to change my opinion.
Once I get to play with TestStudio I really am going to compare it to TestStand. Okay, I do have one bit of pre-knowledge of TestStudio that I can compare to TestStand. Any parameters passed out of TestStudio to a DLL or other code is only allowed to be passed in Character strings. TestStand does it right and passes the value in the form it needs (i.e. integers, doubles, pointers, etc).
Last week the score was TestStand 2, TestStudio -2. The score is now TestStand 3, TestStudio -3 for TestStands ability to pass parameters correctly and Teradynes unwillingness to allow people who are going to use it to actually use it.
Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts
Monday, June 29, 2009
Sunday, November 23, 2008
NI Help
Since I work a lot with different instruments and don't have the luxury of completely learning the ins-and-out of one product, I rely on a companies help functions to get my job done.
Lately I've been integrating some Agilent RF equipment into a test set. I now have a much better appreciation of NI help. I've always felt like NI has some of the best technical help around, examples, and documentation but I've come to appreciate it even more.
I'm trying to use TestStand to control an automated test on an RF Signal Generator, RF Power Meter, and an RF Switch, all from Agilent but having trouble. Agilent really is trying to help but still have a really long way to go to get close to NI help. When I had problems I entered a couple of on-line help requests but received no responses. I had to call the local sales rep to get the e-mail of a tech rep. I have been e-mailing him and he has been helpful in a limited way. I did get the manual web sequence to control the RF Switch, but for automated tests, that doesn't mater.
The RF signal generator was easy because I've used it before and figured out it's quirks. The examples for the RF Power meter had nothing to do with actually reading power, hmmm. When I made calls to the driver functions, they came back with error messages that gave no hint at what was needed. The funcion panel help gave no hint at what the parameters were looking for. I wound up using some low level SCPI commands in combination with the driver functions. I couldn't get it to work with all SCPI or all driver functions.
As for the RF Switch, the examples seem to include calls to driver functions for many different switch types. For my RF switch, the driver functions generate a lot of "Not supported" error messages. But it was very hard to figure out what is supported or what calls I could make. Also, once I loaded the Agilent IO Drivers (which are required for the system to even see their switch) the Pickering switches disappeared from the PXI chassis, at least from a logical standpoint. I still haven't figured that one out.
It's been a week and a half trying to get the RF interconnections to work. I think I'm close but compared to the NI instruments, it's taken way to long.
Using Agilent instruments has given me more of an appreciation for how good NI help, discussion boards, phone help, and e-mail help really are. I especially want to thank Joel Garner, NI Sales Engineer extraordinaire, for his help.
Lately I've been integrating some Agilent RF equipment into a test set. I now have a much better appreciation of NI help. I've always felt like NI has some of the best technical help around, examples, and documentation but I've come to appreciate it even more.
I'm trying to use TestStand to control an automated test on an RF Signal Generator, RF Power Meter, and an RF Switch, all from Agilent but having trouble. Agilent really is trying to help but still have a really long way to go to get close to NI help. When I had problems I entered a couple of on-line help requests but received no responses. I had to call the local sales rep to get the e-mail of a tech rep. I have been e-mailing him and he has been helpful in a limited way. I did get the manual web sequence to control the RF Switch, but for automated tests, that doesn't mater.
The RF signal generator was easy because I've used it before and figured out it's quirks. The examples for the RF Power meter had nothing to do with actually reading power, hmmm. When I made calls to the driver functions, they came back with error messages that gave no hint at what was needed. The funcion panel help gave no hint at what the parameters were looking for. I wound up using some low level SCPI commands in combination with the driver functions. I couldn't get it to work with all SCPI or all driver functions.
As for the RF Switch, the examples seem to include calls to driver functions for many different switch types. For my RF switch, the driver functions generate a lot of "Not supported" error messages. But it was very hard to figure out what is supported or what calls I could make. Also, once I loaded the Agilent IO Drivers (which are required for the system to even see their switch) the Pickering switches disappeared from the PXI chassis, at least from a logical standpoint. I still haven't figured that one out.
It's been a week and a half trying to get the RF interconnections to work. I think I'm close but compared to the NI instruments, it's taken way to long.
Using Agilent instruments has given me more of an appreciation for how good NI help, discussion boards, phone help, and e-mail help really are. I especially want to thank Joel Garner, NI Sales Engineer extraordinaire, for his help.
Sunday, November 9, 2008
Misunderstood
I've found that the Test Engineering group is misunderstood, at least where I work. Most of the engineering groups have an identity. The Electrical Engineering group builds the hardware, the Mechanical Engineers builds the moving parts, Software Engineers write the software to control the hardware, and the Systems Engineers glue it all together.
It seems what Test Engineering does is a mystery to the other groups. We are called in late in projects because Project Managers don't know why they need us until they realize they need to make more than one widget (or whatever it is they're making). Then they realize their widget (or whatever it is they're building) was designed in a way that makes it incredibly hard to test.
It's too late but that's when they figure out they needed us to begin with.
We're trying to educate the programs on what we can do for them. We can do is:
- We do hardware and we do software.
- We develop systems (Test Systems)
- We can help them design their widget so it can be tested.
- We can make their tests automatic and repeatable.
- We can help with developmental testing
- We can make sure the widget is built correctly
- We can test your system in the field or on the production line
...Bottom line is we can help.
It seems what Test Engineering does is a mystery to the other groups. We are called in late in projects because Project Managers don't know why they need us until they realize they need to make more than one widget (or whatever it is they're making). Then they realize their widget (or whatever it is they're building) was designed in a way that makes it incredibly hard to test.
It's too late but that's when they figure out they needed us to begin with.
We're trying to educate the programs on what we can do for them. We can do is:
- We do hardware and we do software.
- We develop systems (Test Systems)
- We can help them design their widget so it can be tested.
- We can make their tests automatic and repeatable.
- We can help with developmental testing
- We can make sure the widget is built correctly
- We can test your system in the field or on the production line
...Bottom line is we can help.
Tuesday, September 23, 2008
Music and Teams
I was hanging out at the book store, as I tend to do since they have almost all that is needed in life; Coffee, lots of books to read, and free wifi. Some how, I accidentally wander out of the Computer book section, somehow missed the Science and Engineering section and wound up in the music book section. I browsed a book about Zappa for a while but stumbled a book call "Comfortably Numb: the inside story of Pink Floyd". It's a great book!
There were several points that seemed counter intuitive about teams until you realize that Pink Floyd produced many great albums (for the young people, an album is a vinyl disc that music was recorded on and played on a record player...with a needle) including The Dark side of the Moon, one of the best selling albums of all time.
First point, hiring people based on best team fit is over rated. Sometimes the band members were at each others throats. The point was they were all smart, all creative, and they had the drive to get the job done. They didn't let creative differences or personal difference get in the way of there goal.
Second point, some teams fall apart with dissension between team members, some teams refocus their experiences into creativity. Pink Floyd re-focused into writing great music while software teams can refocus into great software.
Third point, Creative people push themselves, and others, hard. Creative people will push themselves to bring their idea's to the forefront and sometimes will push others to get their idea's out there. They seem to push each other, sometimes in not so nice ways, to get people to complete there own ideas.
Fourth point, don't let technology overwhelm the ideas. This one may be a stretch but it has a good point. With Pink Floyd all the technology behind the shows sometimes got in the way of the music. Today, all the e-mail, IM, blogs, distract us from reaching the goal of developing our ideas.
Overall, the parts that I read were very enjoyable. I'm going to have to go back to the bookstore, hangout some more, and read the book. Or, hey, I could actually buy the book! We'll have to see.
There were several points that seemed counter intuitive about teams until you realize that Pink Floyd produced many great albums (for the young people, an album is a vinyl disc that music was recorded on and played on a record player...with a needle) including The Dark side of the Moon, one of the best selling albums of all time.
First point, hiring people based on best team fit is over rated. Sometimes the band members were at each others throats. The point was they were all smart, all creative, and they had the drive to get the job done. They didn't let creative differences or personal difference get in the way of there goal.
Second point, some teams fall apart with dissension between team members, some teams refocus their experiences into creativity. Pink Floyd re-focused into writing great music while software teams can refocus into great software.
Third point, Creative people push themselves, and others, hard. Creative people will push themselves to bring their idea's to the forefront and sometimes will push others to get their idea's out there. They seem to push each other, sometimes in not so nice ways, to get people to complete there own ideas.
Fourth point, don't let technology overwhelm the ideas. This one may be a stretch but it has a good point. With Pink Floyd all the technology behind the shows sometimes got in the way of the music. Today, all the e-mail, IM, blogs, distract us from reaching the goal of developing our ideas.
Overall, the parts that I read were very enjoyable. I'm going to have to go back to the bookstore, hangout some more, and read the book. Or, hey, I could actually buy the book! We'll have to see.
Sunday, August 24, 2008
OK, Now that we know the Super Intelligent Shades of the Color Green are going to take over the world, lets get on with less important things... like testing.
I've been working on processes and metrics for developing test sets for quite a while now (a couple of years). I love doing technical work so it has been tough doing mainly documentation. I have worked on some test programs part time while working on the processes, so it hasn't been totally tortuous.
I believe there is always a process of one type or another in developing test stations. The problem is some times the process is the POOMA (Pull Out Of My A**) process or maybe the "wing it" process. But then there's the other end of the process universe. This is where the process makes it nearly impossible to do a small project. Then, in the middle, there's the agile process which is suppose to be the processes that allow tasks to be done quickly and with the ability to easily adapt to changes. It's still not a cure all.
Even people who say they have no process, have a process, it's just not as straight forward as most or even seems coherent.
After working on the processes for a while I know that one process does not fit all projects even though that's what we try to do. I was suppose to document the process we go through to build test stations. Each project has been unique in it's own way, with some process steps that aren't in the process and skipping other steps that are in the process. Some projects that try to follow the process finds that outside groups don't follow the process which makes our process fail. There's many reason's process fail.
If the processes aren't working (not that people just don't want to do it), they need to be adjusted , in real-time rather than going through the year long process change process.
The project leads, the one's actually have to make the process happen, have to be the ones to adjust the process as needed. They need the lead way to adjust it as needed. The Process police also need to understand a single written process is not a panacea, it won't cure all. The processes need to be adjusted.
While I've been working on the processes, this is the dilemma I've been dealing with, having processes that don't imped getting things done but satisfies the Process Police.
I've been working on processes and metrics for developing test sets for quite a while now (a couple of years). I love doing technical work so it has been tough doing mainly documentation. I have worked on some test programs part time while working on the processes, so it hasn't been totally tortuous.
I believe there is always a process of one type or another in developing test stations. The problem is some times the process is the POOMA (Pull Out Of My A**) process or maybe the "wing it" process. But then there's the other end of the process universe. This is where the process makes it nearly impossible to do a small project. Then, in the middle, there's the agile process which is suppose to be the processes that allow tasks to be done quickly and with the ability to easily adapt to changes. It's still not a cure all.
Even people who say they have no process, have a process, it's just not as straight forward as most or even seems coherent.
After working on the processes for a while I know that one process does not fit all projects even though that's what we try to do. I was suppose to document the process we go through to build test stations. Each project has been unique in it's own way, with some process steps that aren't in the process and skipping other steps that are in the process. Some projects that try to follow the process finds that outside groups don't follow the process which makes our process fail. There's many reason's process fail.
If the processes aren't working (not that people just don't want to do it), they need to be adjusted , in real-time rather than going through the year long process change process.
The project leads, the one's actually have to make the process happen, have to be the ones to adjust the process as needed. They need the lead way to adjust it as needed. The Process police also need to understand a single written process is not a panacea, it won't cure all. The processes need to be adjusted.
While I've been working on the processes, this is the dilemma I've been dealing with, having processes that don't imped getting things done but satisfies the Process Police.
Saturday, June 21, 2008
Future of Test
One thing that a lot of test engineering companies are doing really well is gathering and analyzing failure data. But in the future the data could be gathered to a central repository and the fault data mined more efficiently, the way some companies data mine for business intelligence. Very likely using some AI techniques to analyze the data. Potentially some type of Neural Net, advanced statistical analysis, and Bayesian mathematics that accepts the failure information and gives the most likely fix. Well, that's what I hope since AI is my hobby. Both test set and Unit Under Tests (UUT) could be analyzed and information about future failures be determined in advanced from current failures.
When a test set or UUT failure occurs the failure information and analysis could be directed to the Test Engineer's pocket PC. He/She would be able to attach to the test set from where ever they are and trouble shoot most problems. Through the remote link some problems would be fixed and some would need to be coordinated with on-site technicians.
As for operating the tests, there could be a central operator planning, scheduling, and controlling all of the tests in a factory. The operator would have the various test screens on his/her screen to interact with the test software. There could be some people on the work floor to make sure everything gets hooked up correctly and the UUT's flow from one manufacture operation to the next.
I do believe wireless and the web will pay a much bigger role in future test. Remote operations will be more common and more advanced failure detection, reporting, and analysis will be the norm.
I also believe that AI will pay a bigger role in test...and not just because it's my hobby.
When a test set or UUT failure occurs the failure information and analysis could be directed to the Test Engineer's pocket PC. He/She would be able to attach to the test set from where ever they are and trouble shoot most problems. Through the remote link some problems would be fixed and some would need to be coordinated with on-site technicians.
As for operating the tests, there could be a central operator planning, scheduling, and controlling all of the tests in a factory. The operator would have the various test screens on his/her screen to interact with the test software. There could be some people on the work floor to make sure everything gets hooked up correctly and the UUT's flow from one manufacture operation to the next.
I do believe wireless and the web will pay a much bigger role in future test. Remote operations will be more common and more advanced failure detection, reporting, and analysis will be the norm.
I also believe that AI will pay a bigger role in test...and not just because it's my hobby.
Friday, June 13, 2008
Future of Test
At work, I've been dealing with innovation, future of test software, and even doing a little bit of real engineering work in my spare time.
With all this I keep wondering, what is the Future of test set? Of test software? How much will change before I retire? When can I retire (different topic)? Also, I was doing an install of some software and had some time to contemplate the future.
I know in the future the Engineers creed of "Smaller, Better, Faster" will be in effect. Also the concept of Cheaper is being pushed really hard. So what will happen in the future.
Smaller is easy. We've seen it happen for years in standards like VME going to VXI and to PXI as well as chassis like the compactRIO. With these different architectures we've also seen faster speeds. So in the next 10 years I'm sure there will be a sub-compactRIO or RIO-micro.
I'm pretty sure distributed systems will be more common. A system where there is a central control and information repository with connections to small "brick" testers that can be put just about anywhere, temperature chambers, remote less access able test area's. There will be wireless connections where possible so that even wires aren't needed. Some type of Ethernet (hardwire or wireless) will connect to the brick and the brick will control and communication with the units under test (UUT). All test results and operator interface will back at the central computer. The central computer will control several different tests at one time.
There will be more embedded test programs in the units under test (UUT), there will be less discrete interfaces. Since UUTs being tested are getting smaller (better faster) too, there will be less area for interfaces. Software expertise will be needed to develop Operational Test Programs (OTPs) to load into units to test them.
With distributed test systems and smaller test sets there will be more "soft" instrument front panels on the display screens and less physical instrument front panels. The soft instrument front panels on the screens would give an image of what the real instrument front panel would look like if it were real. They would control the various instruments manually when it's needed and would minimize when the device isn't being used. Touch screens would be great for this function so that the operator would still "touch" on the interface.
Of course, all this are just some possibilities for the future.
With all this I keep wondering, what is the Future of test set? Of test software? How much will change before I retire? When can I retire (different topic)? Also, I was doing an install of some software and had some time to contemplate the future.
I know in the future the Engineers creed of "Smaller, Better, Faster" will be in effect. Also the concept of Cheaper is being pushed really hard. So what will happen in the future.
Smaller is easy. We've seen it happen for years in standards like VME going to VXI and to PXI as well as chassis like the compactRIO. With these different architectures we've also seen faster speeds. So in the next 10 years I'm sure there will be a sub-compactRIO or RIO-micro.
I'm pretty sure distributed systems will be more common. A system where there is a central control and information repository with connections to small "brick" testers that can be put just about anywhere, temperature chambers, remote less access able test area's. There will be wireless connections where possible so that even wires aren't needed. Some type of Ethernet (hardwire or wireless) will connect to the brick and the brick will control and communication with the units under test (UUT). All test results and operator interface will back at the central computer. The central computer will control several different tests at one time.
There will be more embedded test programs in the units under test (UUT), there will be less discrete interfaces. Since UUTs being tested are getting smaller (better faster) too, there will be less area for interfaces. Software expertise will be needed to develop Operational Test Programs (OTPs) to load into units to test them.
With distributed test systems and smaller test sets there will be more "soft" instrument front panels on the display screens and less physical instrument front panels. The soft instrument front panels on the screens would give an image of what the real instrument front panel would look like if it were real. They would control the various instruments manually when it's needed and would minimize when the device isn't being used. Touch screens would be great for this function so that the operator would still "touch" on the interface.
Of course, all this are just some possibilities for the future.
Sunday, June 8, 2008
From Hack to Engineer
I talked about people being the most important tool of Test Engineering. People are the most important factor in any Engineering discipline. People, Engineers, are the constant though out development. No matter how good or thorough the processes are people still have to implement them. And if the processes are not good the people still have to develop the product.
There are several levels of skill that go along with doing the job of Engineer. A hack (not be be confused with hacker) learns as he goes, acts and then thinks, and cleans up his messes only when he absolutely has to, if he can't pass it on to someone else.
Then theirs the craftsman Engineer. He studies, he plans, uses his best practices and tools and takes pride in his work. But a craftsman is not to the level of Engineer because while, he develops a well crafted product, it lacks predictability and certainty.
Then theirs Engineers. With Engineers it's all about knowing instead of guessing. An Engineer doesn't estimate, he/she calculates. An Engineer doesn't hope, he knows. Estimates can formed based on what they devise. Engineers basically have consistency.
Consistency is what Engineers are about. They still need the hack characteristic of learning as they go and the craftsman characteristic of using best practices and tools.
The best way to get a good product is to have good Engineers working on the product. We also need to encourage and train the hack Engineer to become a Craftsman. We also need to encourage and train the craftsman Engineer to become a real Engineer.
There are several levels of skill that go along with doing the job of Engineer. A hack (not be be confused with hacker) learns as he goes, acts and then thinks, and cleans up his messes only when he absolutely has to, if he can't pass it on to someone else.
Then theirs the craftsman Engineer. He studies, he plans, uses his best practices and tools and takes pride in his work. But a craftsman is not to the level of Engineer because while, he develops a well crafted product, it lacks predictability and certainty.
Then theirs Engineers. With Engineers it's all about knowing instead of guessing. An Engineer doesn't estimate, he/she calculates. An Engineer doesn't hope, he knows. Estimates can formed based on what they devise. Engineers basically have consistency.
Consistency is what Engineers are about. They still need the hack characteristic of learning as they go and the craftsman characteristic of using best practices and tools.
The best way to get a good product is to have good Engineers working on the product. We also need to encourage and train the hack Engineer to become a Craftsman. We also need to encourage and train the craftsman Engineer to become a real Engineer.
Sunday, June 1, 2008
SQA - a necessary evil
Software quality is not a myth in Test Engineering…although it's not always accomplished. I know that around here it's not always acknowledged that Test Engineering has software so that the SQA people don't bug us to much. I believe that if everyone had a good software discipline, we wouldn't need SQA. The fact of the matter is, even most of the pure software engineers don't have that type of discipline.
If people are not monitored, at least occasionally, they tend to fall into the bad habits as schedules get tight or work loads increases. The code will degrade, maybe not into Chaos, but to at least crappy code. With a police presences (SQA) to keep the crowd of programmers from moving in the dangerous direction, it remains fun and good code is developed. Without over site, the results would be code chaos (or maybe just bad code). With the software police around, we move in the right direction.
Test Software, at least where I work, is not monitored by SQA like it should be. Typically the SQA people, or whomever sets up their budgets, don't budget enough for TE Software. The mission critical flight software gets the attention while test software is left alone.
Test Software needs to follow some standards, meet some criteria, needs to be verified to a certain level of "goodness", just like every other software. The standards are followed when we know someone is making sure we follow the standards.
The test software lead should demand the Software Quality group to do their job for test engineering, like for the other software groups. This is probably a departure from what most test engineer's want, but if the company wants solid software, then it needs to be done. If the SQA can't do the monitoring of the test software then the Test Engineering lead needs to do at least the basic monitoring. I know the lead doesn't have time but it needs to be done. The goal is good solid test software.
Basically, at the very least, the test lead needs to make the following is done:
Again, the goal is good code, the policing the engineers is how to get there.
If people are not monitored, at least occasionally, they tend to fall into the bad habits as schedules get tight or work loads increases. The code will degrade, maybe not into Chaos, but to at least crappy code. With a police presences (SQA) to keep the crowd of programmers from moving in the dangerous direction, it remains fun and good code is developed. Without over site, the results would be code chaos (or maybe just bad code). With the software police around, we move in the right direction.
Test Software, at least where I work, is not monitored by SQA like it should be. Typically the SQA people, or whomever sets up their budgets, don't budget enough for TE Software. The mission critical flight software gets the attention while test software is left alone.
Test Software needs to follow some standards, meet some criteria, needs to be verified to a certain level of "goodness", just like every other software. The standards are followed when we know someone is making sure we follow the standards.
The test software lead should demand the Software Quality group to do their job for test engineering, like for the other software groups. This is probably a departure from what most test engineer's want, but if the company wants solid software, then it needs to be done. If the SQA can't do the monitoring of the test software then the Test Engineering lead needs to do at least the basic monitoring. I know the lead doesn't have time but it needs to be done. The goal is good solid test software.
Basically, at the very least, the test lead needs to make the following is done:
Code reviews are done and problems found early
Make sure the requirements are met in the design and again in the code
He needs to make sure at least minimal style is met for reuse ability sake and for debugging.
He needs to make sure the code is checked and verified against the requirements.
Again, the goal is good code, the policing the engineers is how to get there.
Wednesday, May 28, 2008
Peer Reviews
"Peer reviews are a waste of time", "Peer reviews don't help", "We don't have time to peer review the", peer reviews are evil, blah, blah blah! I've heard this for years and all I can say to this is "Bologna!" A second or third pair of eyes on something is useful. Haven't you ever had a problem while debugging code and said "Hey Bob, can you take a look at this" and then either through your explanation or something Bob says you have an "a-ha" moment can get past the problem? A peer review is like trying to get to the "a-ha" moment before you realize you need to.
Statistics have shown that Peer Reviews do help if they're done right. (see the book “Best Kept Secrets of Peer Code Reviews” by Jason Cohen.
First, some good reasons to do peer reviews. The best reason is defects found early cost less to fix, simple economics. A second pair of eyes can help. Another reason is that less experienced programmers can learn from more experienced programmers at peer reviews. One note is that typically everybody can learn something from other peoples code.
Second, to shoot down some myths. People have a lot of reasons to not do Peer Reviews. Things like "my code is already good enough". To this I have to ask, have you ever had any problems with your code? If you answered No then you are better than any of the Rock Star programmers that I've heard about. Peer reviews are to get rid of problems before they get out.
How about “We don't have time for peer reviews.” Then how much time do you have to fix problems later? And how much will it cost?
Another myth is that peer reviews don't help. They don't help if you don't do them. They don't help if you don't really review the code. If you find just a few defects it can be worth doing it. My Mom and Dad used to say “Anything worth doing is worth doing right”.
There are alway excuses for not doing peer reviews, but no good reasons.
People have ego problems with peer reviews, too. There's the “Big Brother” effect where programmers feel like the peer review is to monitor their every move. This is not how it should be, it should be about removing defects and not about monitoring.
Earlier I said peer reviews help if they're done right. That means the code is really reviewed, the reviewers read over the code and look for potential problems, makes sure it meets the requirements, and is understandable. If this is done then defects are found and fixed. If the code is just glanced over, fewer defects are found, problems are found later and costs more to fix.
To close there are some truisms about Peer Reviews:
1) Hard code has more defects – the more complex the code, the more potential defects.
2) More time yields more defects – The more time spent reviewing the more defects found
3) It's all about the code – Review the code not the programmer
4) The more defects the better – Defects found early are cheaper to fix
The bottom line is a better product and that's what peer reviews should be all about. Happy reviewing!
Statistics have shown that Peer Reviews do help if they're done right. (see the book “Best Kept Secrets of Peer Code Reviews” by Jason Cohen.
First, some good reasons to do peer reviews. The best reason is defects found early cost less to fix, simple economics. A second pair of eyes can help. Another reason is that less experienced programmers can learn from more experienced programmers at peer reviews. One note is that typically everybody can learn something from other peoples code.
Second, to shoot down some myths. People have a lot of reasons to not do Peer Reviews. Things like "my code is already good enough". To this I have to ask, have you ever had any problems with your code? If you answered No then you are better than any of the Rock Star programmers that I've heard about. Peer reviews are to get rid of problems before they get out.
How about “We don't have time for peer reviews.” Then how much time do you have to fix problems later? And how much will it cost?
Another myth is that peer reviews don't help. They don't help if you don't do them. They don't help if you don't really review the code. If you find just a few defects it can be worth doing it. My Mom and Dad used to say “Anything worth doing is worth doing right”.
There are alway excuses for not doing peer reviews, but no good reasons.
People have ego problems with peer reviews, too. There's the “Big Brother” effect where programmers feel like the peer review is to monitor their every move. This is not how it should be, it should be about removing defects and not about monitoring.
Earlier I said peer reviews help if they're done right. That means the code is really reviewed, the reviewers read over the code and look for potential problems, makes sure it meets the requirements, and is understandable. If this is done then defects are found and fixed. If the code is just glanced over, fewer defects are found, problems are found later and costs more to fix.
To close there are some truisms about Peer Reviews:
1) Hard code has more defects – the more complex the code, the more potential defects.
2) More time yields more defects – The more time spent reviewing the more defects found
3) It's all about the code – Review the code not the programmer
4) The more defects the better – Defects found early are cheaper to fix
The bottom line is a better product and that's what peer reviews should be all about. Happy reviewing!
Saturday, May 24, 2008
Technical Debt
I was browsing around the web and found a interesting concept, Technical Debt. This was a concept brought up by Ward Cunningham in OOPSLA '92. It is being taken over by the Agile people under Agile Project Management . I think it's a good concept for test engineers to take into account, too. The Agile portion and blogs by Steve McConnell (Technical Debt and Technical Debt Decision making) discuss it from a software standpoint. But for test engineers, it can be more than just technical debt in the software. The articles don't go into much detail on exactly what technical deficit is.
Here is what I think. For software some items that would cause technical deficit would be issues:
- Hacked together code or code with a lot of short cuts
- Spaghetti code
- Code that is complex or hard to follow
- Incomplete or inadequate error checking.
- Code to be implemented later
- TBD's or incomplete requirements.
- Poor or Incomplete design
- Anything “owed” to the software that is deferred.
I was contemplating technical debt for the rest of test and hardware can most certainly have technical debt, too. Again, it would be anything owed to the hardware. Some of this would be:
- Incomplete schematics
- Unspecified connectors
- Incomplete grounding or shielding
- Incomplete wire lists
- Incomplete wire specifications where they were needed.
- Again, anything “owed” to the hardware
The mechanical portions could have technical debt as well. It would be things like:
- Incompatible of incomplete layout
- Unspecified mechanical connections,
- Other items that are needed but left unspecified.
(I not as much an expert on hardware or mechanical aspects of test but I'm sure a lot of the hardware people can add to these lists.)
The blogs by Steve McConnell on Technical Debt equate it to financial debt because they have a lot of the same issues. Technical debts, as well as financial debts, have to be taken care of at some point in time or they will bite you in the rear end. If they are not the hardware and mechanical technical debt can be disastrous at integration time or when test operators actually use the equipment. The Software technical debts can come back at anytime, like an unpaid bank loan. Some software debts may bite you at integration time, or worst validation time, or even worse, while the equipment is in use on a production line. To keep it in financial terms, this would be like a bankruptcy. You could get past the technical bankruptcy but your reputation could be ruined for a while.
Also, the hardware technical debts can cause the software have a technical deficit if a hardware debt has to be fixed in software. It's like the hardware defaulting on a technical loan co-signed by the software. This seems to happen quite often.
Overall, when technical debt is taken on, it needs to be taken into account for future releases. If there is to much, it can cause a burden on your future of your tests. If it's not taken care of it could bring everything crashing down, potentially during validation or production.
The scary thing is if you don't realize you're taking on technical debt you have bigger problems and your people need to be trained or replaced.
Here is what I think. For software some items that would cause technical deficit would be issues:
- Hacked together code or code with a lot of short cuts
- Spaghetti code
- Code that is complex or hard to follow
- Incomplete or inadequate error checking.
- Code to be implemented later
- TBD's or incomplete requirements.
- Poor or Incomplete design
- Anything “owed” to the software that is deferred.
I was contemplating technical debt for the rest of test and hardware can most certainly have technical debt, too. Again, it would be anything owed to the hardware. Some of this would be:
- Incomplete schematics
- Unspecified connectors
- Incomplete grounding or shielding
- Incomplete wire lists
- Incomplete wire specifications where they were needed.
- Again, anything “owed” to the hardware
The mechanical portions could have technical debt as well. It would be things like:
- Incompatible of incomplete layout
- Unspecified mechanical connections,
- Other items that are needed but left unspecified.
(I not as much an expert on hardware or mechanical aspects of test but I'm sure a lot of the hardware people can add to these lists.)
The blogs by Steve McConnell on Technical Debt equate it to financial debt because they have a lot of the same issues. Technical debts, as well as financial debts, have to be taken care of at some point in time or they will bite you in the rear end. If they are not the hardware and mechanical technical debt can be disastrous at integration time or when test operators actually use the equipment. The Software technical debts can come back at anytime, like an unpaid bank loan. Some software debts may bite you at integration time, or worst validation time, or even worse, while the equipment is in use on a production line. To keep it in financial terms, this would be like a bankruptcy. You could get past the technical bankruptcy but your reputation could be ruined for a while.
Also, the hardware technical debts can cause the software have a technical deficit if a hardware debt has to be fixed in software. It's like the hardware defaulting on a technical loan co-signed by the software. This seems to happen quite often.
Overall, when technical debt is taken on, it needs to be taken into account for future releases. If there is to much, it can cause a burden on your future of your tests. If it's not taken care of it could bring everything crashing down, potentially during validation or production.
The scary thing is if you don't realize you're taking on technical debt you have bigger problems and your people need to be trained or replaced.
Sunday, May 11, 2008
NI Week is Coming!!
NI Week is getting close!
August 5 - 2008
I went to last years NI week, and it was incredible!! It was a very valuable experience. I would love to go to this years but typically the company I work for won't let the same person go to something like this two years in a row. They tend to want to send different people every year. I'm going to try convince them I should go, I guess we'll see how it goes in the next few weeks.
Below is a link to the preliminary program for this year. By the way, a quote from me is on page 27.
View the NIWeek 2008 Preliminary Program
Testing using simulations
I've written a couple of times on unit test, however, I've heard the old Test Engineers mantra of “our software is different, you can't unit test it!”. I say Bull! I will agree that Test Equipment software is different but if you say “you can't unit test it” then you're lazy, or just don't know much about software.
The main reason TE software is different is because a lot of it runs against hardware, it calls hardware drivers to read information from hardware and control the hardware. It does take work but it's testable. Some situations where you don't have hardware to test with, can't induce all the error's you need to check, or just want a good software product, you need to unit test.
Some common failures that happen during the “Go” path testing are typically checked because they come up during regular development. But not all faults are checked. These off nominal paths are not easily checked with standard UUTs on a test set.
The easiest of these test methods is a Software Tool Code tester, such as KlocWork. This is because, after the tool is set up, you just run it and let it tell you about the potential (or certain) failures. I’m just starting to learn about KlocWork so I don’t know all the specifics at the moment. It appears to be able to capture a lot of the logic and path problems. It goes past the ability of LINT to verify standards and does checks along paths. I’m not sure how it works if you use LabWindows CVI functions or if that’s a non-problem.
Another way of testing is using simulation. Some easily available ways to do this are the simulation features in some IVI drivers, tools such as Agilent''s virtual rack, or with DAQmx and NI's Measurement &Automation Explorer (MAX). I haven't used virtual rack but it sounds like it will simulate instruments as if you were actually running tests with a rack of instruments. I don't know about the setup or operation of the, but the presentation I saw made it look like it could be useful. Virtual Rack IVI drivers
The simulation typically built into IVI drivers allow for instruments to be run in simulation mode. However, a more powerful simulation tool, at least for some NI instruments is the measurement explorer. For NI's DAQmx instruments, you can set up a simulated instrument and the test software operates as usual. The simulated instrument can set up to return various values, as needed. The main problem with this is that if an instrument is simulated using measurement explorer you have to go back to the measurement explorer to check it. In other words someone could take out a card and simulated an the software wouldn't know.
One brute force method of simulation is to comment out the call to and instrument driver and just set the return variable to a value. Without some software discipline, this can be dangerous. If one of these is left in the code then test won't return a valid answer. If this method is employed either a comment tag (I usually use //JAV) should be put in where the test is or a compiler directive should be used, like an #ifdef. I typically use this during developmental testing but I always make sure that, before the end of the day, all of these are out so that I don't forget about it before the next day.
My main point is to use tools that are readily available to make sure you put out the best product possible.
The main reason TE software is different is because a lot of it runs against hardware, it calls hardware drivers to read information from hardware and control the hardware. It does take work but it's testable. Some situations where you don't have hardware to test with, can't induce all the error's you need to check, or just want a good software product, you need to unit test.
Some common failures that happen during the “Go” path testing are typically checked because they come up during regular development. But not all faults are checked. These off nominal paths are not easily checked with standard UUTs on a test set.
Some tools for more comprehensive testing or fault testing would be:
IVI drivers simulation mode
Other simulations (I.e. DAQmx in MAX with Simulated drivers)
Inserting error data.
Software tool code testing KlocWork
The easiest of these test methods is a Software Tool Code tester, such as KlocWork. This is because, after the tool is set up, you just run it and let it tell you about the potential (or certain) failures. I’m just starting to learn about KlocWork so I don’t know all the specifics at the moment. It appears to be able to capture a lot of the logic and path problems. It goes past the ability of LINT to verify standards and does checks along paths. I’m not sure how it works if you use LabWindows CVI functions or if that’s a non-problem.
Another way of testing is using simulation. Some easily available ways to do this are the simulation features in some IVI drivers, tools such as Agilent''s virtual rack, or with DAQmx and NI's Measurement &Automation Explorer (MAX). I haven't used virtual rack but it sounds like it will simulate instruments as if you were actually running tests with a rack of instruments. I don't know about the setup or operation of the, but the presentation I saw made it look like it could be useful. Virtual Rack IVI drivers
The simulation typically built into IVI drivers allow for instruments to be run in simulation mode. However, a more powerful simulation tool, at least for some NI instruments is the measurement explorer. For NI's DAQmx instruments, you can set up a simulated instrument and the test software operates as usual. The simulated instrument can set up to return various values, as needed. The main problem with this is that if an instrument is simulated using measurement explorer you have to go back to the measurement explorer to check it. In other words someone could take out a card and simulated an the software wouldn't know.
One brute force method of simulation is to comment out the call to and instrument driver and just set the return variable to a value. Without some software discipline, this can be dangerous. If one of these is left in the code then test won't return a valid answer. If this method is employed either a comment tag (I usually use //JAV) should be put in where the test is or a compiler directive should be used, like an #ifdef. I typically use this during developmental testing but I always make sure that, before the end of the day, all of these are out so that I don't forget about it before the next day.
My main point is to use tools that are readily available to make sure you put out the best product possible.
Thursday, May 1, 2008
Code and Unit Testing
Last time I talked about Unit Testing or external types of test. Now some thoughts on internal testing.
IDEs (Integrated Development Environments) using basic debugging techniques (i.e. single step, view variables, break points, etc) are the front line of internal testing. They give developers insight in to what's going on in a program. A program is run, it doesn't work, you debug. But there are other ways, more powerful ways, of doing debugging internal to a module that aren't as time consuming.
First, there is an ASSERT statement in most C/C++ languages. In NI's CVI in the toolbox it is a DoAssert statement (include toolbox.h to use this). The DoAssert statement is used to help find problems during development that don't happen that often. Basically, it is a condition passed to the DoAssert (Eample: i>1) and if the condition evaluates to TRUE, execution continues. If the statement evaluates to FALSE, module information is printed and execution stops. It prints out the module name a number like the line number (remember __LINE__ is the current line number in the program) and a message, typically with some debug information.
I use ASSERT or DoAssert (CVI) to check for out of tolerance conditions that happen once in a blue moon (good 'ol southern saying that means “not very often”).
Another way of internal testing is to do logging. Some people only use logging as a last resort after problems are found but I advocate putting in logging as your coding, putting log statements in at potential problem spots. Compiler directives can control whether the logging is executed or not. (#ifdef and #endif)
I have some logging routines laying around that I always use. They use the vaprintf style of functions so the logging is more like a printf statement in C. The LogOpen function is called at the start that opens the log file. A LogClose function is called at the end to stop the logging. It does I/O during execution by flushing the buffer every time the LogData is called but that can be controlled with compiler directives. The flush can be turned off if you don't want the I/O delays.
And, like I started with, there is always the standard debugging tools. These are just a couple of ways to do debugging internal to the module. I just want people to step out of their little box and think “How can I make my code better?” and “What can I do to keep from giving incorrect results or give errors?”
Sunday, April 27, 2008
Code and Unit Test
One thing I've noticed is that many test engineers, for coding and design, just hack something together so it can be thrown onto the test set. Sometimes it’s because of schedule pressure, sometimes it just more fun to touch hardware, sometimes it’s because it’s EE’s doing the coding with only one software class worth of training. In any case, this has got to stop!
Test Engineers (TE) doing software need to have some software discipline. They don’t need the same discipline as tactical or man-rated software development needs, but Test Engineers do need software discipline none the less. TE's need to do some unit testing, some stress testing, some fault path testing, Diagnostics levels, some issue like that.
Many TEs use the excuses of “our software is different” or “Our software is to hardware intensive” to avoid code testing. I say any 10 year old can give excuses, what about results? What is the quality of your software?
As I see it there are two basic types of software testing, internal testing and external testing testing.
First the external testing or unit testing. In the software world there is quite often a test group. The software developers develop software and throw it over the wall to testers. In the Test Engineering world, there's not always that luxury. That's the reason to have software self discipline. There's more to external testing but for now I'm limiting myself to unit testing.
Definition: Unit Test is the testing of a module or a logically grouped set of modules where inputs and outputs, limits, stress, path test occurs.
To test a unit(s) it can be as simple as a function calling the unit(s) being tested over and over with various inputs and then checking the output. If NI's TestStand is in use then a sequence is set up to call the module(s) under test and the output checked.
One tool that I use quite often is called CUnit. It's a unit testing framework and there is a whole range of _Unit type of frameworks. JUnit for Java, CPPUnit for C++, xUnit for MS Visual stuff (I.e. NI Measurement Studio) and some others I can't think of right now. A lot of this can be found at SourceForge.org, a great source of open source software.
Since test software is quite often hardware intensive, sometimes some hardware simulation is needed, and I use the word “simulation” in the weakest form. It's as simple as creating a dummy routine to replace the hardware instrument driver that returns the data you want to test with. I.e. good data, erroneous data, stressed data, etc.
To test, the driver routine would call the module being tested. The module being tested calls the driver simulation which returns a predetermined set of results which allows the module being tested to return a predetermined result. If the returned result is really what you expect, the test worked. Otherwise, the module needs to be fixed.
Overall, the point I'm making is not to hack something out and throw it on the hardware. There are options, whether you've thought of them or not. By The Way, this isn't a comprehensive list of ways to unit test. Use your imagination, I know you have one, you're an Engineer!
I just want people to step out of their little box and think “How can I make my code better?” and “What can I do to keep from giving incorrect results or give errors?”
Note: Some of these test methods are the part of Agile development called TDD, Test Driven Development (TDD). It just means your code should be driven by the tests. I didn't want to scare anyone with the word "Agile."
Test Engineers (TE) doing software need to have some software discipline. They don’t need the same discipline as tactical or man-rated software development needs, but Test Engineers do need software discipline none the less. TE's need to do some unit testing, some stress testing, some fault path testing, Diagnostics levels, some issue like that.
Many TEs use the excuses of “our software is different” or “Our software is to hardware intensive” to avoid code testing. I say any 10 year old can give excuses, what about results? What is the quality of your software?
As I see it there are two basic types of software testing, internal testing and external testing testing.
First the external testing or unit testing. In the software world there is quite often a test group. The software developers develop software and throw it over the wall to testers. In the Test Engineering world, there's not always that luxury. That's the reason to have software self discipline. There's more to external testing but for now I'm limiting myself to unit testing.
Definition: Unit Test is the testing of a module or a logically grouped set of modules where inputs and outputs, limits, stress, path test occurs.
To test a unit(s) it can be as simple as a function calling the unit(s) being tested over and over with various inputs and then checking the output. If NI's TestStand is in use then a sequence is set up to call the module(s) under test and the output checked.
One tool that I use quite often is called CUnit. It's a unit testing framework and there is a whole range of _Unit type of frameworks. JUnit for Java, CPPUnit for C++, xUnit for MS Visual stuff (I.e. NI Measurement Studio) and some others I can't think of right now. A lot of this can be found at SourceForge.org, a great source of open source software.
Since test software is quite often hardware intensive, sometimes some hardware simulation is needed, and I use the word “simulation” in the weakest form. It's as simple as creating a dummy routine to replace the hardware instrument driver that returns the data you want to test with. I.e. good data, erroneous data, stressed data, etc.
To test, the driver routine would call the module being tested. The module being tested calls the driver simulation which returns a predetermined set of results which allows the module being tested to return a predetermined result. If the returned result is really what you expect, the test worked. Otherwise, the module needs to be fixed.
Overall, the point I'm making is not to hack something out and throw it on the hardware. There are options, whether you've thought of them or not. By The Way, this isn't a comprehensive list of ways to unit test. Use your imagination, I know you have one, you're an Engineer!
I just want people to step out of their little box and think “How can I make my code better?” and “What can I do to keep from giving incorrect results or give errors?”
Note: Some of these test methods are the part of Agile development called TDD, Test Driven Development (TDD). It just means your code should be driven by the tests. I didn't want to scare anyone with the word "Agile."
Friday, April 18, 2008
Zephyr - test management tool
Yea, this blog is about testing, usually about NI stuff, but competition is good for us all. Saying that, someone e-mailed me about a test management system called Zephyr. I read about it and it sounds like it has a lot of potential.
Zephyr is a complete test management system. It has collaboration tools, customizable dashboards for instant status, Test Desktops, with built in metrics, and test execution capability with defect tracking.
You can read about it's list of product features here and see an overview video here. There's also a limit person trial version here. It has some very good looking interfaces.
I have only read about the product and have not tried it out but it sounded like it had some very useful features. Since I have not actually used it I can only give you my impressions. Some things I liked were:
- Central repository for documents or at least links to the documents.
- Test/QA/Management Collaboration
- Instant update of test status
- It sounds like it was for Agile development with all the communication and collaboration, but I'm sure it can be used for any type of development.
The ability to manage widely distributed test systems is impressive and massively useful.
A couple of big questions, at least for me.
One, how well it plays with TestStand and other NI products. NI is the big dog on the block for test software and electronic, at least at Lockheed and to get a foothold it would need to complement our NI abilities.
Two, We have a lot of legacy code and equipment from NI that we don't want to lose. Since our previous tests cases are in TestStand written in CVI or LabVIEW, how well you can re-use your old tests? How well will it interact with the legacy code?
I'm probably download and try out the trial version but it would be in my spare time. Unfortunately, that wouldn't be until the end of the year.
All-in-all Zephyr looks like it has potential and I would like to hear from anyone who tries it.
Zephyr is a complete test management system. It has collaboration tools, customizable dashboards for instant status, Test Desktops, with built in metrics, and test execution capability with defect tracking.
You can read about it's list of product features here and see an overview video here. There's also a limit person trial version here. It has some very good looking interfaces.
I have only read about the product and have not tried it out but it sounded like it had some very useful features. Since I have not actually used it I can only give you my impressions. Some things I liked were:
- Central repository for documents or at least links to the documents.
- Test/QA/Management Collaboration
- Instant update of test status
- It sounds like it was for Agile development with all the communication and collaboration, but I'm sure it can be used for any type of development.
The ability to manage widely distributed test systems is impressive and massively useful.
A couple of big questions, at least for me.
One, how well it plays with TestStand and other NI products. NI is the big dog on the block for test software and electronic, at least at Lockheed and to get a foothold it would need to complement our NI abilities.
Two, We have a lot of legacy code and equipment from NI that we don't want to lose. Since our previous tests cases are in TestStand written in CVI or LabVIEW, how well you can re-use your old tests? How well will it interact with the legacy code?
I'm probably download and try out the trial version but it would be in my spare time. Unfortunately, that wouldn't be until the end of the year.
All-in-all Zephyr looks like it has potential and I would like to hear from anyone who tries it.
Apologies
I want to apologize. I haven't been blogging much for a while due to my Mom being, first in the Hospital, and now in a nursing home.
Things are settling into more of a routine so I should be able to blog more, but she does come first.
Thanks
Things are settling into more of a routine so I should be able to blog more, but she does come first.
Thanks
Monday, March 10, 2008
Test Engneers
Most people dealing with Automated Test Equipment (ATE) call themselves “test engineer’s” (TE) or test people. However, since there is typically no degree’s in test engineering, they get degrees in Electrical engineering or software engineering or computer science and engineering and then are recruited in the “test group”. I’ve seen most TE’s come out of the group with EE degrees, a lot of SW Engineers don’t like to touch hardware. But then a lot of EE’s feel that doing software makes them dirty or is FM (Freaking Magic) and only took minimal, how to code classes in college.
Since most TE’s are out of EE the software portion of ATE is “hacked” out (the old-school meaning of hack, to bang out code until it works) the code is less than optimal. In other-words, it works for the normal case and little else.
Some people who have software degrees or have been doing software for a long time work toward having more software discipline in there ATE code, but it still tends to be lacking in most area’s. Software tools tend to go unused or limited use, some believe an IDE is all that is needed.
There’s more to developing software than just an IDE and some time. Tools are available to help develop code. I think software tools aren’t used in test due to a lack of understanding, lack of knowledge, and lack of patience.
Since most TE’s are out of EE the software portion of ATE is “hacked” out (the old-school meaning of hack, to bang out code until it works) the code is less than optimal. In other-words, it works for the normal case and little else.
Some people who have software degrees or have been doing software for a long time work toward having more software discipline in there ATE code, but it still tends to be lacking in most area’s. Software tools tend to go unused or limited use, some believe an IDE is all that is needed.
There’s more to developing software than just an IDE and some time. Tools are available to help develop code. I think software tools aren’t used in test due to a lack of understanding, lack of knowledge, and lack of patience.
Thursday, September 13, 2007
The test development environment
This is less about computer development environment and more about physical development environment. In the past I've been in the unfortunate situation of going off-site to develop. It went slow and I got phyically sick at least once a week. This situation is about to happen again.
I bring this up because a test set we are working on is being sent to the customer's site. In the earlier parts of the contract the tests will be developed here. Due to the lateness of the UUT's being developed, the test development process will be moved to their site, where the UUTs will be. They originally promised to send UUT's for us to verify our tests in April 2007 but changed the agreement, rather ignored the agreement and changed what we had to do. That meant the we have to send a test set up to there site and the use their UUT's. The same UUT's that their engineers will be using at the same time we would like use them. Or maybe we have the midnight to 6am shift to use the UUT's.
This winds up putting all the pressure on us. We're in a strange place working the test set. If we need technician help we have to find one of there technicians to help us. Same with tools, seating, and, most of all, UUT's. We will be at their mercy on when we can use the UUT's and guess who will be at the bottom of the priority list for use of the UUTs. (answer: us)
The only good thing about this situation is we will have at least some access to the UUT's to complete our test verification. Nothing else is good about it. We'll be out of our environment, staying in hotels, away from family, and eating out all the time. It gets old quick.
As for me, I'm only going up there occasionally, I can get my stuff done down here. Since I'm a single dad, it would be tough for me to go up there for 3 months like some of the others.
Have a great day
I bring this up because a test set we are working on is being sent to the customer's site. In the earlier parts of the contract the tests will be developed here. Due to the lateness of the UUT's being developed, the test development process will be moved to their site, where the UUTs will be. They originally promised to send UUT's for us to verify our tests in April 2007 but changed the agreement, rather ignored the agreement and changed what we had to do. That meant the we have to send a test set up to there site and the use their UUT's. The same UUT's that their engineers will be using at the same time we would like use them. Or maybe we have the midnight to 6am shift to use the UUT's.
This winds up putting all the pressure on us. We're in a strange place working the test set. If we need technician help we have to find one of there technicians to help us. Same with tools, seating, and, most of all, UUT's. We will be at their mercy on when we can use the UUT's and guess who will be at the bottom of the priority list for use of the UUTs. (answer: us)
The only good thing about this situation is we will have at least some access to the UUT's to complete our test verification. Nothing else is good about it. We'll be out of our environment, staying in hotels, away from family, and eating out all the time. It gets old quick.
As for me, I'm only going up there occasionally, I can get my stuff done down here. Since I'm a single dad, it would be tough for me to go up there for 3 months like some of the others.
Have a great day
Wednesday, August 29, 2007
Requirements are Required
Test Requirements are required for proper testing. The project I'm currently on, more or less, have requirements. I say more or less because there are requirements written down, but they're not signed off. I, as a sub-contractor, and I'm trying to follow the requirements as written, even though it's not signed off.
Today I was working with the contractor, going over the user interface. Some questions came up about how the operator interface worked, he wanted things done differently. At one point I pointed out that I did it according to the requirements. The contractor I was showing it to said "I don't care what that [the requirements] says, I want it like this."
To change something they want is not the problem, it's the fact that it's written one way and going to be implemented a different way. We could held accountable later for not implimenting the requirements correctly.
I'm not an anal retentive person, I'm probably more loosy-goosey, but I also want to protect my self. I've been in the situation where a boss told me to do things one way instead of the way the documentation explained it. Later, the same boss called me out in a meeting on doing it incorrectly. Since then, I do it the way the documentation says and if someone wants it done differently, then change the documentation. That's when I'll change it.
Basically, to protect yourself, write down what's wanted (requirements), agree to them, then follow them. If someone wants changes, change the documentation, agree on cost and schedule, and then implement the changes.
Bottom line, CYA!
Today I was working with the contractor, going over the user interface. Some questions came up about how the operator interface worked, he wanted things done differently. At one point I pointed out that I did it according to the requirements. The contractor I was showing it to said "I don't care what that [the requirements] says, I want it like this."
To change something they want is not the problem, it's the fact that it's written one way and going to be implemented a different way. We could held accountable later for not implimenting the requirements correctly.
I'm not an anal retentive person, I'm probably more loosy-goosey, but I also want to protect my self. I've been in the situation where a boss told me to do things one way instead of the way the documentation explained it. Later, the same boss called me out in a meeting on doing it incorrectly. Since then, I do it the way the documentation says and if someone wants it done differently, then change the documentation. That's when I'll change it.
Basically, to protect yourself, write down what's wanted (requirements), agree to them, then follow them. If someone wants changes, change the documentation, agree on cost and schedule, and then implement the changes.
Bottom line, CYA!
Subscribe to:
Posts (Atom)
