ROC curve analysis

[govand ahmed]

Can wizard pro do ROC curve analysis
regards

[Evan Miller]

Hi Govand,

Wizard does not support ROC curve analysis. I'm not sure what this
would mean in the context of Wizard's models since all of the
parameters are estimated from the data. If you had a specific
application in mind I'd be happy to hear more about it.

Evan

On Tue, Mar 12, 2013 at 4:11 AM, govand ahmed <govand...@gmail.com> wrote:
> Can wizard pro do ROC curve analysis
> regards
>

> --
> You received this message because you are subscribed to the Google Groups
> "Wizard User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to wizard-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Evan Miller
http://www.evanmiller.org/

[Evan Miller]

Hello again,

I looked into ROC curves and found a nice way to integrate them into
Wizard's Model view for binary-outcome variables. You can see an
example here:

http://wizard.evanmiller.org/roc.png

All the best,

Evan

[Jan Veldsink]

Hello Evan,

I just updated to your new version.. and can not find the ROC option... 

Could you please show where or under what conditions ROC coms up as an option.

With kind regards,

Jan

Op dinsdag 12 maart 2013 10:11:46 UTC+1 schreef govand ahmed het volgende:

[Evan Miller]

Hi Jan,

The ROC curve option appears after you create a logit or probit
(two-outcome) model. The option will appear in the top-right corner of
the residual scatterplot view. Do you see it?

Evan

[Jan Veldsink]

Evan,

Thanks Yes I can see it .. looks great... great feature...

Would you consider a four quadrant representation of the ROC graph?

With kind regards

Jan

Op dinsdag 12 maart 2013 10:11:46 UTC+1 schreef govand ahmed het volgende:

Can wizard pro do ROC curve analysis
regards

[Evan Miller]

Hi Jan,

Glad you like the new feature, and thanks for the suggestion about a
four-quadrant representation. I'll think about how to integrate that
view in a future version.

Evan

[hiren....@doctors.org.uk]

Hi Evan

enjoying getting to use your software. great work.

i could do with some advice on ROC analysis. when i export the values to Excel, the cutoff values are strange. my data ranges from -6 to 15.4, yet the cutoff values generated are from 0.1 to 0.4??

not sure what i'm doing wrong here. could i send you my data file to have a look at??

many thanks

Hiren

[Evan Miller]

Hi,

The cutoff values refer to the logistic probability -- so 0.1 means roughly "Classify as X if the probability of being X is at least 10%". Does that make sense?

Evan

To unsubscribe from this group and stop receiving emails from it, send an email to wizard-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[peter....@bluepearlvet.com]

Hi Evan

Just found this old thread . I love your software and the ROC curves are great.

Re your comment above, this may be a simple statistical question, but is there any way that I can go back to extract the true cut off values for any given sensitivity/specificity pairing?

Thank you!

Peter

For more options, visit https://groups.google.com/d/optout.

[Evan Miller]

Hi Peter,

You can obtain the cutoff values from the xls export -- control-click the image and choose Export Data as Excel.

Evan

To unsubscribe from this group and stop receiving emails from it, send an email to wizard-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[peter....@bluepearlvet.com]

Thanks, I did find that function but I am looking for a way to convert the probability cutoff back to an absolute value for determining a cut off for a test. My inability to solve this problem probably reflects my lack of understanding of the underlying statistics but I would appreciate any tips!

Peter

[Evan Miller]

Hi Peter,

I see! The logistic formula is P=1/(1+exp(-x)). Solving for x...

P = 1/(1+exp(-x))

1+exp(-x)=1/P

exp(-x)=1/P - 1

-x = log(1/P - 1)

x = -log(1/P - 1)

Does that give you the values you expect?

Evan

To unsubscribe from this group and stop receiving emails from it, send an email to wizard-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[peter....@bluepearlvet.com]

Unfortunately not. However, I was able to solve the problem by copying the column containing all the individual test results, deleting all the duplicates to give me a list of all possible test outcomes, and pasting that into a new column in the ROC Excel output.

Here is a snippet from the middle of the table as an example (second column added) - it would be great if this was included in the output - it is very useful data for setting a cutoff for a medical test based on optimizing sensitivity and specificity. Thanks!

[Daniel Burchette]

Hi. Any progress on this feature? I am struggling with the same thing and unfortunately, your screenshot seems to have gone.

I am trying to identify the numerical value that represents the best cut off for a numerical variable (Blood Test) that predicts a categorical outcome (Diagnosis - Yes, No). When I export the ROC curve analysis into Excel, the 'cut-off' values are not comparable to the numerical values I need to find a cut off for.

Kind regards,

Dan