PVC Distribution Milestone 2 parallel test

[Lynn Foster]

Hi all,

I'm kicking off a separate thread to track the parallel test between NRP value equation results and the spreadsheet results. 

Please note that the level of detail may be a bit painful, so feel free not to follow it!

Also please note that the spirit of this is to get the value equation working in NRP.  Doing a parallel run at this stage is useful in creating a new software feature.  And the code for the value equation is quite complex, although this particular distribution is not complex because it is not tracking through a value flow more than one level.  Probably a good thing.  Anyway, this exercise has nothing to do with proving someone is right or wrong, I am myself interested to find any bugs or mis-conceptions in NRP.

I am kicking this off now to let people know that I did change the 2 contributions that Daniel had recorded to programming instead of mechanical design.  (This was already changed in the spreadsheet, and I needed to have a valid comparison.)

I also see in my sandbox run that there are some events that used minutes rather than hours, which I am pretty positive is incorrect.  So I will fix that in the data. 

If anyone doesn't want me to mess with your live data, please let me know!!!!!  I will report all changes so people can disagree if I make a specific mistake.  I can always easily change things back.

For Jim and anyone else who wants to connect with me in figuring this out, I can supply more detail either here or offline.  No rush, just when you want to work on it.

Thanks,

Lynn

[Lynn Foster]

Update:

1. The recording of minutes rather than hours isn't a problem - both NRP and the spreadsheet are disregarding because in both cases, the quantity is just multiplied by a $/hr in the equation.  So I didn't change any of that data.  (Tibi, you might want to change them in NRP just for the record, in case the units are ever used for something.)

2. The total number of hours recorded in the NRP calculation is 192.37, vs. total 165.57 in the spreadsheet.  But we both have 72 events coming in from the filtering.  I found one issue in the spreadsheet that could account for this.  On the Contributions per Member M2 tab, the spreadsheet is not including all the columns in the total hours per person.  It is using D:G rather than D:I range. 

Let's fix the spreadsheet and then see where we are at.  On an encouraging note, Daniel's distribution does match and he doesn't have any hours outside of the D:G range.  But we'll see, there could easily be other issues.

I could fix the spreadsheet, and will if you think that is best.  But I'm a bit uncomfortable doing that, since I didn't create it.  Tibi, your preference?

--
Go to SENSORICA home
https://sites.google.com/site/sensoricahome/home
Go to our Working Space
https://sites.google.com/site/sensoricahome/home/working-space
--
You received this message because you are subscribed to SENSORICA group.
To post to this group, send email to Sens...@googlegroups.com
To unsubscribe from this group, send email to
Sensorica+...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/Sensorica?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups "SENSORICA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to Sensorica+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lynn Foster]

OK I remembered that I can just make a copy of the spreadsheet, so I did so and fixed the error last identified.  It didn't make any difference in the amount distributed, but did make the total hours add up to what we have in NRP. 

So, continuing.....  Found another one, and this is it!  The spreadsheet sheet Contributions of Members M2, rows 32-41, are not calculating correctly.  I'm not a spreadsheet wiz so I don't know what the formulas used are attempting to do.  But when I simplified them to use the budgeted dollars directly, I got answers that totaled correctly to the budgeted dollars (which the spreadsheet doesn't), and also match the NRP calculation.

Yay!  Glad to have that solved, thought it might take more digging.

Here is my modified spreadsheet: https://docs.google.com/spreadsheets/d/1fvtuYGA5X2rt1IMCawIxlZLGJSia6kahVjXFmHHTEf4/edit#gid=2044565645.  Note this is not a totally fixed spreadsheet, just changed what I needed to debug the problem.

Attached is the new numbers from the spreadsheet.  And they match NRP if you run my test value equation  in the sandbox.

[Bob Haugen]

Tibi, you have not responded to this.

Lynn spent several hours on Sunday debugging and fixing your
spreadsheet. It is now Wednesday.

I am sure you are busy, but so are we.

Yesterday, you paid Daniel off the old uncorrected version of the spreadsheet:
https://docs.google.com/spreadsheets/d/1YNgwrjoOQgGWGP3QJbNrfgFbSKue1FCgeQdFdlUsBVg/edit#gid=2044565645

Doesn't matter for Daniel, because his total distribution is the same
in the corrected version. But it will matter a lot for other people.

And it also matters for honest P2P collaboration. Lynn will elaborate.

> --
> Go to SENSORICA's project page
> http://www.sensorica.co/home/what-we-do/projects/pv-characterization

> ---
> You received this message because you are subscribed to the Google Groups

> "PV characterization project forum" group.

> To unsubscribe from this group and stop receiving emails from it, send an

> email to pv-characterization-pr...@googlegroups.com.

> To post to this group, send email to

> pv-characterizat...@googlegroups.com.
> Visit this group at
> http://groups.google.com/group/pv-characterization-project-forum.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pv-characterization-project-forum/CACvmXVyc%2BYpvbc6SS1DKPNXBm0mAZ_4mCzgFoSr062C%3DsfYLjg%40mail.gmail.com.

[Lynn Foster]

Tibi, here is what Bob meant by honest P2P collaboration.  This goes on all the time and significantly undermines our efforts with the software.  So here is the example, meant to try to help you understand how to make the situation better.

Following are the series of statements you made in the original thread (payday Milestone 2) discussing the spreadsheet vs NRP distribution logic:

> I did it in the spreadsheet, since the value equation engine cannot support the way data was logged, for this budgeted project, a forward-looking value equation. We haven't reached the level of abstraction that can support all the different use cases with the Value Equation. More conceptual work needs to be done...

>Lynn, we can select specific processes to input time contributions data into the value equation, but we can't go further, to select specific work types within a specific process.
In the milestone 1, we put all the R&D into one bucket, electronics, software and mechanics, because they had the same $/h rate. Milestone 2 is different, we need to consider these differently, and they were logged in the same process.

>1. might require some programming, we might not be able to solve it with the filtering methods we have now.

> My idea is that it might not be possible to map the value equation formula from the spreadsheet into the NRP-VAS without extra programming. It all depends on the flexibility of the crunching engine behind the value equation in the NRP-VAS. Spreadsheets are pretty versatile already...

All of these turned out to be false, in fact NRP handled all of it fine.  So, you could have said a couple things in the last few days:

1. Lynn, thanks for spending the time to debug my spreadsheet, it made quite a difference to people's income.

2. All the concerns I expressed about how the NRP handles this were wrong, I'm glad to see it can handle all of this.

Or you could not have expressed your statements (in the usual disparaging but very certain tone) in the first place, since you DID NOT KNOW and you WERE TOTALLY WRONG!

Honestly, it is really discouraging for us to work in this environment!  I hope this helps you (and others) to understand the dynamic we are trying to fix.  It's hard on the health and really a waste of valuable time to deal with this type of dynamic.

Lynn

[Tiberius Brastaviceanu]

2. The total number of hours recorded in the NRP calculation is 192.37, vs. total 165.57 in the spreadsheet.  But we both have 72 events coming in from the filtering.  I found one issue in the spreadsheet that could account for this.  On the Contributions per Member M2 tab, the spreadsheet is not including all the columns in the total hours per person.  It is using D:G rather than D:I range.  

Good catch Lynn, error corrected.  

To view this discussion on the web visit https://groups.google.com/d/msgid/pv-characterization-project-forum/CACvmXVwOmjyuuZErZH-e%3Dxf-REmyK2PV5Nw8_42UrvRRdrUPXA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

t!b! 

co-founder of SENSORICA, 

an open, decentralized and self-organizing

value network (an open enterprise)

founder of Multitude Project

Google Profile

Facebook Tiberius Brastaviceanu

Twitter  @TiberiusB

[Tiberius Brastaviceanu]

So, continuing.....  Found another one, and this is it!  The spreadsheet sheet Contributions of Members M2, rows 32-41, are not calculating correctly.  I'm not a spreadsheet wiz so I don't know what the formulas used are attempting to do.  But when I simplified them to use the budgeted dollars directly, I got answers that totaled correctly to the budgeted dollars (which the spreadsheet doesn't), and also match the NRP calculation.

There was a bug there too, two columns were interted, pointing to the wrong budget allocation in the Summary spreadsheet. Fixed in the spreadsheet.  

[Abran Khalid]

Thanks for all the hard work you put in, Tibi, Lynn and Bob.I am sure finding out the issues by manually going through spreadsheets would not have been a pleasant task. 

Best, 

Abran

[Tiberius Brastaviceanu]

Addressing Bob's and Lynn's concerns about proper collaboration. 

I just read this email. Even if it was sent 4 days ago, I did not have time to open it until now, a few minutes ago. It was actually brought to my attention by Jim. I would have sent thanks, but I feel that you took all that pleasure away from me... I do it anyway. Thank you Lynn for debugging the spreadsheet. 

The results of the value equation in the NRP does match the results in the spreadsheet now, after the bugs have been fixed. 

Still, my concerns about the transparency of what goes on in the value equation are valid. Every time we have a discrepancy between the two methods we will need to check them both. At this point, we have a black box, because one needs to read the source code of the NRP-VAS in order to do a formal comparison. Moreover, this transparency is needed in an open environment anyway. 

Another note about collaboration

I am still recovering from the damage caused by the discussion that Bob initiated on our mailing list, starting with "we don't trust Tibi", founded on a misinterpretation of an email I sent to him and a few others. This was an irresponsible move with great potential of destroying the social fabric of SENSORICA. I am collaborative, but a bit dry and I keep it technical. I am sorry... but that's how it is. This explains my tone. 

[Bob Haugen]

On Thu, Sep 24, 2015 at 12:36 AM, Tiberius Brastaviceanu
<tiberius.br...@gmail.com> wrote:
> Still, my concerns about the transparency of what goes on in the value
> equation are valid. Every time we have a discrepancy between the two methods
> we will need to check them both. At this point, we have a black box, because
> one needs to read the source code of the NRP-VAS in order to do a formal
> comparison. Moreover, this transparency is needed in an open environment
> anyway.

Your concerns are valid, but also apply to your spreadsheet, which you
could not understand yourself.

I wrote a bit about the tradeoffs between spreadsheets and the NRP
value equation here:
https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.43lilqbuyu5c

And tried to improve the documentation of what is going on in the NRP
value equation in that whole document, with more about algorithm and
math starting here:
https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.ykk2hzjh847n

You had talked about wanting to express the whole value equation in
mathematical notation, and that section should give you some ideas
about what you would need to do to accomplish that desire. Short
version: some of it's ~possibly~ doable, some possibly not.

[Tiberius Brastaviceanu]

Your concerns are valid, but also apply to your spreadsheet, which you
could not understand yourself.

?? 

 

I wrote a bit about the tradeoffs between spreadsheets and the NRP
value equation here:
https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.43lilqbuyu5c

I think everybody wants to handle value equations in the NRP-VAS, including myself. As VE-s are new in the NRP, we continue to run them in parallel in Spreadsheet. 

 

And tried to improve the documentation of what is going on in the NRP
value equation in that whole document, with more about algorithm and
math starting here:
https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.ykk2hzjh847n

It would be nice to see the formalism in the NRP, next to the value equation, after a value equation is implemented. 

 

You had talked about wanting to express the whole value equation in
mathematical notation, and that section should give you some ideas
about what you would need to do to accomplish that desire. Short
version: some of it's ~possibly~ doable, some possibly not.

I think we need to explore what's doable together and implement as much as possible. 

[Bob Haugen]

On Thu, Sep 24, 2015 at 9:10 AM, Tiberius Brastaviceanu
<tiberius.br...@gmail.com> wrote:
>>
>> Your concerns are valid, but also apply to your spreadsheet, which you
>> could not understand yourself.
>
> ??

You could not detect the mistakes in your own spreadsheet. It took
Lynn several hours to find and (probably) fix them.

In other words, a spreadsheet can pretty easily reach a level of
complexity where it is no longer transparent. How many other
Sensoricans would have wanted to do that?

>> I wrote a bit about the tradeoffs between spreadsheets and the NRP
>> value equation here:
>>
>> https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.43lilqbuyu5c
>
>
> I think everybody wants to handle value equations in the NRP-VAS, including
> myself. As VE-s are new in the NRP, we continue to run them in parallel in
> Spreadsheet.

As we're said, we totally agree. Thank you for doing those parallel
tests, they are very valuable.

>>
>> And tried to improve the documentation of what is going on in the NRP
>> value equation in that whole document, with more about algorithm and
>> math starting here:
>>
>> https://docs.google.com/document/d/1hg0q7zNyuM1OM-8OOoZrkCPpB_EUH8YGLLnfoDFopy4/edit#heading=h.ykk2hzjh847n
>
>
> It would be nice to see the formalism in the NRP, next to the value
> equation, after a value equation is implemented.
>
>>
>> You had talked about wanting to express the whole value equation in
>> mathematical notation, and that section should give you some ideas
>> about what you would need to do to accomplish that desire. Short
>> version: some of it's ~possibly~ doable, some possibly not.
>
>
> I think we need to explore what's doable together and implement as much as
> possible.

We do want to continue to improve the transparency of the value
equations. All of the diagnostic stuff that we've added to the
sandbox page has been aimed at helping you (and us) match the results
to your spreadsheets. And we want to continue in that direction.

The hard part of presenting the value equation in a mathematical
formalism is the graph traversals. For example, in the PV project, if
the 3D printer is used, that requires a fairly specialized graph
traversal to find the people who contributed to buying the printer,
and also the materials used.

Same for any consumables.

Our direction in the future, however, is gestating in the Open Vocab
project, which is based on Linked Open Data, which is a graph
formalism. That's still a research project. I'd guess a couple of
years away from becoming an operational ecosystem tool. But somewhere
along that path, the graph traversals should be expressable in formal
notation.

[Bob Haugen]

Another idea for transparency: logs:
http://mathbabe.org/2015/09/24/interrogating-algorithms/

"auditable and hard-to-tamper-with logs of how the software is running"