CI upper and lower bounds are identical

[Michael McMurray]

I apologize if this has been covered elsewhere, a quick search didn't reveal anything.

We have been happily using the browser app to calculate effect sizes and associated confidence intervals, and only one sample out of many seems to generate a problem: the upper and lower bounds of the 95% CI are the same, hence the CI plotted is absent (presumably because it is zero). Attached is the text of the Results. The first dataset had no problem, like all the others we've analyzed; the second set is where the problem occurs.

Any ideas?

Thanks in advance,

Michael

[Michael McMurray]

And here are the values for that dataset  and the "control"

Control:

72.167

59.75

37

38.417

34.417

54.583

57.25

57.25

71.333

69.083

65.333

58.417

40.833

44.25

63.5

38.667

39.75

51.333

41.5

34.333

33.917

41.667

26.667

33.667

35.5

41.417

37.75

37.417

61

34.917

54.833

54

35.25

34.167

55.25

48.667

62

64.167

37.833

33.083

57.167

35.75

51.417

38.167

52.083

33.083

35.167

39.083

33.917

33.083

54.333

59.167

38.5

54.083

51.25

58.167

40.75

43.833

49.333

50.25

46.5

41.583

58.333

63.25

56.833

43.833

63.5

48.417

50.083

65

65.333

48.333

41.667

44

69.083

62.5

44.333

56.333

62.917

48.833

43.083

73.667

53.833

100.667

51.25

53.25

39

43.083

37.167

48.417

38.5

39.083

45.417

46.5

76.417

80.5

63.917

45.333

50.5

40.25

Experimental:

53.667

46.667

37.417

44.75

52.25

49.667

39.333

55

54.417

47.75

36

36.667

39

50.417

47.417

36.75

56.917

46.083

37.833

30.75

47.5

34.5

34.083

51.75

50.5

61.25

40

45.417

65.083

40.5

44.167

48.583

46.083

37.417

47.417

40

56.333

54.5

43.583

41.5

54.25

48.25

38.167

33.417

36.5

36.25

36.25

44.333

43.583

58.75

39

38.667

36.5

38.083

99.417

52.583

50.75

50.333

39.333

51.583

48.25

40.333

53.5

55

40.167

34.917

36.5

57.333

37.5

37.167

49.583

70.083

38.917

38.417

44.167

41.083

50.667

44.417

44.417

42.833

35.917

36.417

31.417

44.167

37.667

45.583

47

49.833

53.917

38.667

45

40.25

34

34.75

55.75

51.917

39.333

36.917

38.25

46.583

[Michael McMurray]

Finally, I should note that the problem only occurs when we seek to determine differences in the median. If we use the mean, no issue (i.e., a reasonable CI is generated).

[Lewis Martin]

I can't comment on the workings of estimationstats, but this has occurred for me in the past with the median operation, because every bootstrap sample had the same median. Although I just tested your data with some code (below) and did not see that problem.

Just thought I'd note - the mean can be estimated by fitting a gaussian, and the uncertainty in the estimate of the gaussian centre (by monte carlo estimation or something) is the uncertainty in the mean. If you can do that, then just swap out the gaussian for the laplace distribution, because that distribution models the median.

python code:

import numpy as np
ctrl = np.loadtxt('ctrl.txt')
expr = np.loadtxt('expr.txt')

def simple_bstrap(sample, function = np.median, n=10000):
    return function(np.random.choice(sample, size=(n,len(sample))), axis=1)

median_diffs = simple_bstrap(expr) - simple_bstrap(ctrl)

srtd = median_diffs[median_diffs.argsort()]
print('Difference in median 95% CI:', (srtd[250], srtd[9750]))

----

Difference in median 95% CI: (-9.457999999999998, 1.0)

--
You received this message because you are subscribed to the Google Groups "estimationstats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to estimationsta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/estimationstats/03278592-71e2-4848-a51f-5145e1385cfbn%40googlegroups.com.

[Adam Claridge-Chang]

We haven't had the bandwidth to address this, but I wanted to update with this note from github issues.

https://github.com/ACCLAB/dabestr/issues/105

Not sure if this explains the above problems, but it appears the medians function has a flaw that needs correcting.