Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
verification strategy with no specs
77 views
Skip to first unread message
alb
May 8, 2013, 5:13:40 AM5/8/13
to
Dear all,
I've been appointed to review and verify a vhdl project of about 25k
lines of code, *without* a specification document!
There are various scattered notes/docs which describes somehow some
details (*not all*), but there's no description of what the individual
parts should do, even though there are only 4 types of FPGA in the system.
It seems unbelievable to me that they got there without any spec but
this is something I cannot change. My main question here is to
understand if there exist strategies to face such type of situations and
which one is more effective.
I've started looking at the craziness at some implementation level (all
code is practically uncommented!), but I'm at the level of firing a
question to the designers for each line of code just to understand the
reason behind!
I know it sounds like a 'rescue' plan, but if anyone can point me to
some - preferably documented - direction I would greatly appreciate.
Al
--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
I've been appointed to review and verify a vhdl project of about 25k
lines of code, *without* a specification document!
There are various scattered notes/docs which describes somehow some
details (*not all*), but there's no description of what the individual
parts should do, even though there are only 4 types of FPGA in the system.
It seems unbelievable to me that they got there without any spec but
this is something I cannot change. My main question here is to
understand if there exist strategies to face such type of situations and
which one is more effective.
I've started looking at the craziness at some implementation level (all
code is practically uncommented!), but I'm at the level of firing a
question to the designers for each line of code just to understand the
reason behind!
I know it sounds like a 'rescue' plan, but if anyone can point me to
some - preferably documented - direction I would greatly appreciate.
Al
--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
John Speth
May 8, 2013, 12:50:47 PM5/8/13
to
On 5/8/2013 2:13 AM, alb wrote:
> Dear all,
>
> I've been appointed to review and verify a vhdl project of about 25k
> lines of code, *without* a specification document!
>
> There are various scattered notes/docs which describes somehow some
> details (*not all*), but there's no description of what the individual
> parts should do, even though there are only 4 types of FPGA in the system.
>
> It seems unbelievable to me that they got there without any spec but
> this is something I cannot change. My main question here is to
> understand if there exist strategies to face such type of situations and
> which one is more effective.
>
> I've started looking at the craziness at some implementation level (all
> code is practically uncommented!), but I'm at the level of firing a
> question to the designers for each line of code just to understand the
> reason behind!
>
> I know it sounds like a 'rescue' plan, but if anyone can point me to
> some - preferably documented - direction I would greatly appreciate.
Here's what you do: Get an audience, scratch your head and rub your
> Dear all,
>
> I've been appointed to review and verify a vhdl project of about 25k
> lines of code, *without* a specification document!
>
> There are various scattered notes/docs which describes somehow some
> details (*not all*), but there's no description of what the individual
> parts should do, even though there are only 4 types of FPGA in the system.
>
> It seems unbelievable to me that they got there without any spec but
> this is something I cannot change. My main question here is to
> understand if there exist strategies to face such type of situations and
> which one is more effective.
>
> I've started looking at the craziness at some implementation level (all
> code is practically uncommented!), but I'm at the level of firing a
> question to the designers for each line of code just to understand the
> reason behind!
>
> I know it sounds like a 'rescue' plan, but if anyone can point me to
> some - preferably documented - direction I would greatly appreciate.
beard while looking pensive. Then declare with authority "It appears
the product meets all stated specifications".
Seriously, you'll need to write the specs before you can test it. It
sounds like that's what you're doing anyway. I sympathize with you. It
must be a huge undertaking if you actually expect to successfully
complete it.
JJS
alb
May 8, 2013, 4:42:27 PM5/8/13
to
Hi John,
On 08/05/2013 18:50, John Speth wrote:
[]
to get an higher level description of the various components in order to
define interfaces, functionality and performances.
Maybe I should size the effort and come with a proposal to the group in
order to make it effective. I'm not sure how long will it take to write
the specs from scratch and it certainly adds time to the already tight
schedule (of course!).
A very diffused sentiment in my environment comes from the false belief
that every piece of hardware or software is buggy and in the end you can
still live with it, so why bother with all these specs and
verifications? They seem to see a 'verification plan' as another
bureaucratic wall to tear down or to dodge quickly filling a couple of
formal tests required by the funding agencies.
Sometimes, I must say, I feel like tilting at windmills...
On 08/05/2013 18:50, John Speth wrote:
[]
>> I've started looking at the craziness at some implementation level (all
>> code is practically uncommented!), but I'm at the level of firing a
>> question to the designers for each line of code just to understand the
>> reason behind!
[]
>> code is practically uncommented!), but I'm at the level of firing a
>> question to the designers for each line of code just to understand the
>> reason behind!
> Here's what you do: Get an audience, scratch your head and rub your
> beard while looking pensive. Then declare with authority "It appears
> the product meets all stated specifications".
I wish I had the kind of beard to allow me such a statement!
> beard while looking pensive. Then declare with authority "It appears
> the product meets all stated specifications".
>
> Seriously, you'll need to write the specs before you can test it. It
> sounds like that's what you're doing anyway. I sympathize with you. It
> must be a huge undertaking if you actually expect to successfully
> complete it.
So your suggestion is to sit down with the current designer(s) and try
> Seriously, you'll need to write the specs before you can test it. It
> sounds like that's what you're doing anyway. I sympathize with you. It
> must be a huge undertaking if you actually expect to successfully
> complete it.
to get an higher level description of the various components in order to
define interfaces, functionality and performances.
Maybe I should size the effort and come with a proposal to the group in
order to make it effective. I'm not sure how long will it take to write
the specs from scratch and it certainly adds time to the already tight
schedule (of course!).
A very diffused sentiment in my environment comes from the false belief
that every piece of hardware or software is buggy and in the end you can
still live with it, so why bother with all these specs and
verifications? They seem to see a 'verification plan' as another
bureaucratic wall to tear down or to dodge quickly filling a couple of
formal tests required by the funding agencies.
Sometimes, I must say, I feel like tilting at windmills...
Andy
May 9, 2013, 3:13:44 PM5/9/13
to
I agree, without a specification, you cannot verify function.
However, there are established design standards, regardless of function, that are prescribed to avoid certain, most often unintended, consequences and hazards.
The design can be reviewed per those standards with little or no knowledge of what the circuit is supposed to do.
Also, revising the circuit in response to the review findings will likely require some knowledge of what it was supposed to do.
I gather that more is known about what the system of FPGAs is supposed to do, than is known about what the individual FPGAs are supposed to do. If this is the case, you may have to devise some tests to exercise the intended system behavior, while instrumenting the interfaces between FPGAs to gather information on what is happening. Together with reviewing the code, one could presumably come up with a "spec" for what all of the interfaces of each FPGA are supposed to do. And from that, a functional spec can be written for each FPGA.
Such a specification will be much tighter than necessary, since you only have visibility to what they actually did, not what they really needed to do. Slight variations to what they did might still work, but it is impossible to know if you don't know what 'works' means.
Andy
However, there are established design standards, regardless of function, that are prescribed to avoid certain, most often unintended, consequences and hazards.
The design can be reviewed per those standards with little or no knowledge of what the circuit is supposed to do.
Also, revising the circuit in response to the review findings will likely require some knowledge of what it was supposed to do.
I gather that more is known about what the system of FPGAs is supposed to do, than is known about what the individual FPGAs are supposed to do. If this is the case, you may have to devise some tests to exercise the intended system behavior, while instrumenting the interfaces between FPGAs to gather information on what is happening. Together with reviewing the code, one could presumably come up with a "spec" for what all of the interfaces of each FPGA are supposed to do. And from that, a functional spec can be written for each FPGA.
Such a specification will be much tighter than necessary, since you only have visibility to what they actually did, not what they really needed to do. Slight variations to what they did might still work, but it is impossible to know if you don't know what 'works' means.
Andy
Rob Anderson
Jun 30, 2013, 8:51:10 PM6/30/13
to
The thing to do is run the test code, capture the output and assume it is a litmus test. Use DIFF on it.
The problem is that if you change things, eg. reset or clocking, the output will not be identical. So change things slowly and keep it identical.
0 new messages