permutation testing in other metrics stored in dt.G

[Jonatan Ottino]

Hi Chris and everybody,

I was trying to 'adapt' the following code to compute centrality closeness to test instead other graph metrics stored in 'dt.G' (after creating graphs and tidying things up).

The original code from p. 108 says the following:

clocent.diffs.perm <- function(g, densities) {

  meas <- lapply(g, function(x) t(sapply(x, function(y) centr_clo(y)$res)))

  meas.diff <- sapply(seq_along(V(g[[1]][[1]])), function(x)

    brainGraph:::auc_diff(densities, cbind(meas[[1]][, x], meas[[2]][, x])))

  tmp <- as.data.table(t(meas.diff))

  setnames(tmp, 1:ncol(tmp), V(g[[1]][[1]])$name)

  return(tmp)

}

perms.clocent <- brainGraph_permute(densities, 

                                    all.dat.resids, 

                                    perms = myPerms[1:1e2, ],

                                    level = 'other', 

                                    .function = clocent.diffs.perm, 

                                    auc = T)

summary(perms.clocent, alt = 'two.sided')

Now, I'd like to compute other graph metrics not included in brainGraph_permute by default (for instance, E.local). I was trying to run:

clocent.diffs.perm <- function(g, densities) {

  meas <- lapply(g, function(x) t(sapply(x, function(y) NA)))

  meas[[1]] <- dt.G$E.global[dt.G$Group == 'Patient']

  meas[[2]] <- dt.G$E.global[dt.G$Group == 'Control'] 

  meas.diff <- meas[[1]] - meas[[2]]

  tmp <- as.data.table(t(meas.diff))

  return(tmp)

}

perms.clocent <- brainGraph_permute(densities, 

                                    all.dat.resids, 

                                    perms = myPerms,

                                    level = 'other', 

                                    .function = clocent.diffs.perm, 

                                    auc = T)

When trying to get the summary statistics, I get the following error:

summary(perms.clocent, alt = 'two.sided')

Error in `[.data.table`(result.dt, , `:=`(p, (sum(abs(get(measure)) >=  : 

  by=c(...), key(...) or names(...) must evaluate to 'character'

When turning the 'auc' option to FALSE, I get instead:

Error in setkeyv(x, cols, verbose = verbose, physical = physical) : 

  some columns are not in the data.table: densities

I think I messed up with the densities part. In the original function, 'densities' is well-defined as its purpose is to be further used in auc_diff. 

I've searched throughout the forum for previous messages asking something similar with very little luck. Is this the right way to pass to brainGraph_permute other metrics though?

By the way, I'm using:

R version 3.6.1 (2019-07-05)

brainGraph version 2.7.3.

Best,

Jonatan

[Chris Watson]

Your function will not work because it has to calculate the metric of interest (e.g., E.local) from each set of permuted graphs. You should only change 1 line from the example function:

In the line with "centr_clo(y)$res" it can just be

meas <- lapply(g, function(x) t(sapply(x, efficiency, use.parallel=FALSE)))

Note that this will probably take a long time, so first try it out with a small number of permutations (say around 50).

The "use.parallel=FALSE" is needed because that code is already called from within a "foreach" loop.

You might also need to add "weights=NA" before that argument, as well.

Chris

P.S. In the upcoming version of the package, this will no longer be necessary as there will be code for just about every vertex-level metric of interest. I am still hoping to finish up that release soon but it perpetually falls behind.

--
You received this message because you are subscribed to the Google Groups "brainGraph-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brainGraph-he...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/16444d3a-bd0d-4f7c-b96d-c8e75bf6b641%40googlegroups.com.

[Jonatan Ottino]

sorry Chris perhaps I explained myself wrong but I was hoping to use the local efficiency and the other measures stored within dt.G just as you did in your paper from 2017 (10.1002/brb3.83).

I've seen in there that you performed permutation testing over not hard-coded metrics such as the number of hubs or E.local (this last one per hemisphere, though).

Best,

J

To unsubscribe from this group and stop receiving emails from it, send an email to brainGr...@googlegroups.com.

[Chris Watson]

I understand, but you still cannot use "dt.G" for that. When subjects are permuted, then 2 new groups are created; the graphs are then newly generated from these groups, at all specified densities. This is not the same as, for example, permuting the observed values of graph metrics because the graphs will almost certainly have different topology. So you should still try to use the code I suggested in my previous reply.

If you need more clarification/explanation, please let me know.

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to brainGraph-he...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/4310d08e-f74d-444d-8644-6fcbb6dd47e7%40googlegroups.com.

[Jonatan Ottino-González]

Thank you Chris, the code from your previous response works perfectly fine but is returning the local efficiency at the vertex level. I'm still trying to calculate E.local at the graph-level (not vertex). I'll investigate more and let you know whether I finally made it. Looking forward to that update.

J

To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/4310d08e-f74d-444d-8644-6fcbb6dd47e7%40googlegroups.com.

[Chris Watson]

In that case, I believe the function should be something like this:

my.function <- function(g, densities, auc) {
meas <- sapply(g, sapply, function(y) mean(efficiency(y, use.parallel=FALSE)))
if (isTRUE(auc)) {
tmp <- data.table(densities=1, E.local=auc_diff(densities, meas))
} else {
tmp <- data.table(densities=densities, E.local=meas[, 1] - meas[, 2], key='densities')
}
return(tmp)
}

You should try this in a small number of permutations to see if it works
correctly.

Chris

On Tue, Apr 07, 2020 at 10:25 AM, Jonatan Ottino-González <jonata...@gmail.com> wrote:

> from: "Jonatan Ottino-González" <jonata...@gmail.com>
> date: Tue, Apr 07 08:25 AM -07:00 2020
> to: brainGraph-help <brainGr...@googlegroups.com>
> reply-to: brainGr...@googlegroups.com
> subject: Re: [brainGraph-help] permutation testing in other metrics stored in dt.G

>>>>> <https://groups.google.com/d/msgid/brainGraph-help/16444d3a-bd0d-4f7c-b96d-c8e75bf6b641%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .

>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "brainGraph-help" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to brainGr...@googlegroups.com <javascript:>.

>>> To view this discussion on the web visit

>>> https://groups.google.com/d/msgid/brainGraph-help/4310d08e-f74d-444d-8644-6fcbb6dd47e7%40googlegroups.com
>>> <https://groups.google.com/d/msgid/brainGraph-help/4310d08e-f74d-444d-8644-6fcbb6dd47e7%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .

>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups "brainGraph-help" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to brainGraph-he...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/3d269be5-b15a-4554-b585-d0f57eee7eb8%40googlegroups.com.

[Jonatan Ottino-González]

Hi Chris,

after feeding the function to: 

perms.eff <- brainGraph_permute(densities, 

                                    all.dat.resids, 

                                    perms = myPerms[1:1e2, ],

                                    level = 'other', 

                                    .function = my.function, 

                                    auc = T)

I get the following error:

Error in { : task 1 failed - "argument "auc" is missing, with no default"

then, I removed the auc argument from the code as well as the if/else statements just to see if working that way. The first part seemed to work well:

my.function <- function(g, densities) {

  meas <- sapply(g, sapply, function(y) mean(efficiency(y, use.parallel = F)))

  tmp <- data.table(densities = 1, E.local = brainGraph:::auc_diff(densities, meas))

  return(tmp)

}

but when it comes to get the summaries with:

summary(perms.eff, alt = 'two.sided')

I got this:

Error in summary.brainGraph_permute(perms.eff, alt = "two.sided") : 

  measure %in% names(permDT) is not TRUE

I've already tried renaming E.local as 'mod' or some other metric that it is within the names that the package expects, but it returns this error:

Error in get(atlas) : invalid first argument

J

> To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/3d269be5-b15a-4554-b585-d0f57eee7eb8%40googlegroups.com.

[Chris Watson]

You need to call the function (brainGraph_permute) with "atlas=x" where "x" is
the name of your atlas. Try that and I think it should work.

Chris

On Wed, Apr 08, 2020 at 12:15 AM, Jonatan Ottino-González <jonata...@gmail.com> wrote:

> from: "Jonatan Ottino-González" <jonata...@gmail.com>

> date: Tue, Apr 07 10:15 PM -07:00 2020

> to: brainGraph-help <brainGr...@googlegroups.com>
> reply-to: brainGr...@googlegroups.com
> subject: Re: [brainGraph-help] permutation testing in other metrics stored in dt.G
>

>> jonata...@gmail.com <javascript:>> wrote:
>>
>> > from: "Jonatan Ottino-González" <jonata...@gmail.com <javascript:>>

>> > date: Tue, Apr 07 08:25 AM -07:00 2020

>> > to: brainGraph-help <brainGr...@googlegroups.com <javascript:>>
>> > reply-to: brainGr...@googlegroups.com <javascript:>

>> https://groups.google.com/d/msgid/brainGraph-help/3d269be5-b15a-4554-b585-d0f57eee7eb8%40googlegroups.com.

>>
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups "brainGraph-help" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to brainGraph-he...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/brainGraph-help/66b81906-6fe1-4ee6-9c3b-035a39869772%40googlegroups.com.

[Hollie]

Hi both, 

I am also interested in computing graph level local efficiency and came across this thread. Following the steps above I encountered the same errors, however when adding "atlas=x" into the call I get the following: 

Error in brainGraph_permute(densities, resids, perms = myPerms[1:100, : unused argument (atlas = "dk") Just wondering if you also encountered this and were able to resolve it Jonatan? 

Thanks a bunch, 

Hollie