Plot EER point on DET curve (bob.measure.plot.det)

[Omar C]

Hi I would like to know how to plot an EER point in a DET curve:

import bob
import matplotlib.pyplot as mpl
bob.measure.plot.det(...) #call this as many times as you need
#AFTER you plot the DET curve, just set the axis in this way:
mpl.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])

It returns something like:

https://drive.google.com/file/d/0B4Jac-wNMDxHR01MX2xaT2VMdHM/view?usp=sharing

I would like to obtain a picture like:

Where the EER point is plot. Any suggestions on how to achieve this?

Regards,

Omar

[Tiago Freitas Pereira]

Hi Omar,

It is very easy.

The trick is just to put in the correct scale.

Follow attached an example

Cheers

Tiago

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Amir Mohammadi]

Hi,

It would be great to have this in the documentation.

Omar do you want do it?

Thanks,

Amir

On Thu, Feb 2, 2017 at 11:50 AM Tiago Freitas Pereira <tiagofr...@gmail.com> wrote:

Hi Omar,

It is very easy.

The trick is just to put in the correct scale.

Follow attached an example

Cheers

Tiago

On Thu, Feb 2, 2017 at 11:23 AM, Omar C <omar.cost...@gmail.com> wrote:

Hi I would like to know how to plot an EER point in a DET curve:

import bob
import matplotlib.pyplot as mpl
bob.measure.plot.det(...) #call this as many times as you need
#AFTER you plot the DET curve, just set the axis in this way:
mpl.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])

It returns something like:

https://drive.google.com/file/d/0B4Jac-wNMDxHR01MX2xaT2VMdHM/view?usp=sharing

I would like to obtain a picture like:

Where the EER point is plot. Any suggestions on how to achieve this?

Regards,

Omar

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

[Amir Mohammadi]

Right now,

We accept patches by email until we figure out something better:
http://alblue.bandlem.com/2011/12/git-tip-of-week-patches-by-email.html
https://git-scm.com/docs/git-send-email

Best,

Amir

[Tiago Freitas Pereira]

Hi

Well, maybe the documentation is not extremely explicit, but you can find this information about the scale of the data here (http://pythonhosted.org/bob.measure/py_api.html#bob.measure.det)

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Omar C]

Thank you Tiago this works perfectly! 

On Thursday, February 2, 2017 at 12:50:03 PM UTC+2, Tiago Freitas Pereira wrote:

Hi Omar,

It is very easy.

The trick is just to put in the correct scale.

Follow attached an example

Cheers

Tiago

On Thu, Feb 2, 2017 at 11:23 AM, Omar C <omar.cost...@gmail.com> wrote:

Hi I would like to know how to plot an EER point in a DET curve:

import bob
import matplotlib.pyplot as mpl
bob.measure.plot.det(...) #call this as many times as you need
#AFTER you plot the DET curve, just set the axis in this way:
mpl.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])

It returns something like:

https://drive.google.com/file/d/0B4Jac-wNMDxHR01MX2xaT2VMdHM/view?usp=sharing

I would like to obtain a picture like:

Where the EER point is plot. Any suggestions on how to achieve this?

Regards,

Omar

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Omar C]

Yes I can volunteer for this if necessary!

[Omar C]

Hi All,

Does anybody know why this approach does not work for the attached program and scores? Thank you!

[Omar C]

Tiago please see last post on the thread. It does not work for a special case. you might be able to provide more insight about the problem.

On Thursday, February 2, 2017 at 12:50:03 PM UTC+2, Tiago Freitas Pereira wrote:

Hi Omar,

It is very easy.

The trick is just to put in the correct scale.

Follow attached an example

Cheers

Tiago

On Thu, Feb 2, 2017 at 11:23 AM, Omar C <omar.cost...@gmail.com> wrote:

Hi I would like to know how to plot an EER point in a DET curve:

import bob
import matplotlib.pyplot as mpl
bob.measure.plot.det(...) #call this as many times as you need
#AFTER you plot the DET curve, just set the axis in this way:
mpl.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])

It returns something like:

https://drive.google.com/file/d/0B4Jac-wNMDxHR01MX2xaT2VMdHM/view?usp=sharing

I would like to obtain a picture like:

Where the EER point is plot. Any suggestions on how to achieve this?

Regards,

Omar

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Omar C]

Hi all, I have tried different scaling configurations and this is not working is this a problems with the source code itself?

[Amir Mohammadi]

Hi Omar,

Thank you for giving us the scores.

In your case, the point you are plotting is on the EER point but the DET curve is wrong (not wrong weird).

Your problem is because most of your negatives scores are very close to 0:

>>> scores  = numpy.load('fusion_all_200_datatset.npy')
... positives = scores[0]
... negatives = scores[1]
# The negatives are mostly 0
>>> sum(negatives < 9e-16)
51911
>>> sum(negatives < 9e-17)
51525
>>> sum(negatives < 9e-18)
51029
>>> sum(negatives < 9e-20)
49675
>>> sum(negatives < 9e-22)
47543
>>> sum(negatives < 9e-30)
27487
>>> sum(negatives < 9e-60)
0

I have opened an issue for your problem here: https://gitlab.idiap.ch/bob/bob.measure/issues/26

But we'll try to discuss this here as you cannot comment there.

Best,

Amir

On Fri, Feb 3, 2017 at 3:41 PM Omar C <omar.cost...@gmail.com> wrote:

Hi all, I have tried different scaling configurations and this is not working is this a problems with the source code itself?

[Omar C]

Hi Amir, thank you for opening the ticket. Is there any provisional solution to this before is fixed in the main branch?

[Amir Mohammadi]

Hi Omar,

I posted some temporary code on the issue which you can use to plot your DET curves for now:
https://gitlab.idiap.ch/bob/bob.measure/issues/26#note_13247

Best,

Amir

On Sat, Feb 4, 2017 at 10:14 AM Omar C <omar.cost...@gmail.com> wrote:

Hi Amir, thank you for opening the ticket. Is there any provisional solution to this before is fixed in the main branch?

[Omar C]

Hi Amir thank you for providing a fix to this, The algorithm works for some of the scores I have but fails as well for others. Could you please take a look at the DET using the following scores:

https://drive.google.com/file/d/0B4Jac-wNMDxHRktEd2hoUmhZT0E/view?usp=sharing

It fails to put the EER at the right location.

Regards,

Omar

[Omar C]

Anyone had the chance to check this?

Cheers,
Omar

[Omar C]

Still not a fix for this guys, can anyone shine some light on this?

Cheers

[Amir Mohammadi]

Hi everyone,

After 9 months ... I found some time to look into this problem.

Omar provided two set of scores for which the det curves were not working.

The issue for the first set of scores (available in the issue) was reported here:
https://gitlab.idiap.ch/bob/bob.measure/issues/26

A workaround was provided then and now a fix:
https://gitlab.idiap.ch/bob/bob.measure/merge_requests/44/

I looked into the second set of scores that Omar provided but the problem

was not because of bob.measure but it was simply because of the way EER

point was plotted on the DET curve.

The second set of scores are here:
https://drive.google.com/file/d/0B4Jac-wNMDxHRktEd2hoUmhZT0E/view

and I have attached a DET curve and the code to plot the det curve.

The problem was that at the EER point since the number of positives were very low

the FAR and FRR were 0.009908256880733945, 0.0

so the EER point would be on the x-axis.

Best,

Amir

--

[Amir Mohammadi]

Sorry attached the wrong DET plot.