Output p-values #33

IrinaVanzhula · 2022-05-18T02:35:34Z

Hello Claudia,

I am finding that many of the p-values for edge or node centrality comparisons are whole numbers, either zeros or 1s. Is there a way to have the actual p-value in the out put and should I assume that p-value of zero means its < .001?
Thank you!

pinusm · 2022-05-18T07:09:16Z

Hey,
Not Claudia, but I've seen this as well.
This might be related to the number of iterations you used.
Could you provide the function call that produced this?

cvborkulo · 2022-05-19T11:47:47Z

Hi Irina,

It shouldn't be zero. Or did you control for multiple testing? Or, as pinusm suggests, have few iterations?

IrinaVanzhula · 2022-06-01T02:51:21Z

Hi both, here is the function I used. I did control for multiple testing and have many iterations. So what do those zeros mean in this case?

NCT1 <- NCT(Male, Female, weighted = TRUE, test.edges = FALSE, estimator = myGlassoSpearman,
it=5000, p.adjust.methods= "bonferroni", test.centrality=TRUE,
centrality=c("strength","expectedInfluence"))

IrinaVanzhula · 2022-06-01T03:08:33Z

Here is part of my output. Also apologies for delayed response. For some reason I am not getting email notifications.

cvborkulo · 2022-06-21T12:39:53Z

Hi Irina, As I can see in your code, you apply a Bonferroni correction. If you would not get zeros, I assume. Could you try? But I also see zeros for M and S. With 5000 iterations, I would not expect that. Could you send me your data + code? Or a part of your data, as long as I can reproduce your issue. You can send it to ***@***.******@***.***>. Also, use the updated version on github. I just updated it. Thank you! Claudia From: IrinaV ***@***.***> Date: Wednesday, 1 June 2022 at 05:08 To: cvborkulo/NetworkComparisonTest ***@***.***> Cc: Claudia van Borkulo ***@***.***>, Comment ***@***.***> Subject: Re: [cvborkulo/NetworkComparisonTest] Output p-values (Issue #33) Here is part of my output. Also apologies for delayed response. For some reason I am not getting email notifications. NETWORK INVARIANCE TEST Test statistic M: 0.129854 p-value 0 GLOBAL STRENGTH INVARIANCE TEST Global strength per group: 18.63467 16.82085 Test statistic S: 1.813817 p-value 0 strength expectedInfluence R1 1.0000 0.0000 R4 0.0000 1.0000 R5 1.0000 0.3600 R7 1.0000 1.0000 R8 0.3744 1.0000 R9 0.0000 0.0000 R12 0.0000 0.0000 R14 0.0000 0.0288 — Reply to this email directly, view it on GitHub<#33 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABPVHTAOT3LYT4J6FM3LYRLVM3H3ZANCNFSM5WG2VDVQ>. You are receiving this because you commented.Message ID: ***@***.***>

cvborkulo · 2022-10-11T08:31:48Z

Hi Irina, Curious to see your code as well! Best, Claudia Outlook voor iOS<https://aka.ms/o0ukef> downloaden

…

________________________________ Van: Michael Pinus ***@***.***> Verzonden: Wednesday, May 18, 2022 9:09:31 AM Aan: cvborkulo/NetworkComparisonTest ***@***.***> CC: Subscribed ***@***.***> Onderwerp: Re: [cvborkulo/NetworkComparisonTest] Output p-values (Issue #33) Hey, Not Claudia, but I've seen this as well. This might be related to the number of iterations you used. Could you provide the function call that produced this? — Reply to this email directly, view it on GitHub<#33 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABPVHTADAQZHHK5SWV6N7S3VKSJSXANCNFSM5WG2VDVQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

AlexCocker · 2022-11-15T05:41:48Z

I have noticed the same issue as Irina with my data

cvborkulo · 2022-11-24T09:44:50Z

Hi Alex, I did not hear back from Irina. Could you provide code with which I can reproduce this issue?

AlexCocker · 2022-11-29T05:23:00Z

Hi Claudia, thank you for the help, the code is below. If you need me to email you the excel file I imported the data from please let me know

##############

remove(list = ls())
library(readxl)
library(psych)
library(NetworkComparisonTest)

PriyaDataTNLCD <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=2,na = "."))
DataTNLCD <- PriyaDataTNLCD[,5:29]
PriyaDataPTNLCD <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=3,na = "."))
DataPTNLCD <- PriyaDataPTNLCD[,5:29]
Cellnames1 <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=4,na = "."))

corTNLCD <- cor(DataTNLCD, use = "pairwise.complete.obs", method = "spearman")
corPTNLCD <- cor(DataPTNLCD, use = "pairwise.complete.obs", method = "spearman")

NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "bonferroni", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness"))

NCTTNLvsPTNLCD

NETWORK INVARIANCE TEST
Test statistic M: 0.7141979
p-value 0

GLOBAL STRENGTH INVARIANCE TEST
Global strength per group: 0 21.25911
Test statistic S: 21.25911
p-value 0

EDGE INVARIANCE TEST

              Var1               Var2 p-value

26 Neutro% HLA-DR+%CD14+CD16+ 1
51 Neutro% HLA-DR+%CD14+CD16- 0
52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1
76 Neutro% HLA-DR+%CD14-CD16+ 1

cvborkulo · 2022-12-01T11:03:54Z

Hi Alex, I see that you used the Bonferroni correction but that should yield a 0 for M and S. What does your output look like when you do not correct for multiple testing? You could also email me the data or a part of it, as long as it reproduces the issue. Best, Claudia Op di 29 nov. 2022 om 06:23 schreef AlexCocker ***@***.***>:

…

Hi Claudia, thank you for the help, the code is below. If you need me to email you the excel file I imported the data from please let me know ############## remove(list = ls()) library(readxl) library(psych) library(NetworkComparisonTest) PriyaDataTNLCD <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=2,na = ".")) DataTNLCD <- PriyaDataTNLCD[,5:29] PriyaDataPTNLCD <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=3,na = ".")) DataPTNLCD <- PriyaDataPTNLCD[,5:29] Cellnames1 <- as.data.frame(read_excel("/Users/alexcocker/Desktop/2022-11-13 - Priya Flow Data.xlsx",sheet=4,na = ".")) corTNLCD <- cor(DataTNLCD, use = "pairwise.complete.obs", method = "spearman") corPTNLCD <- cor(DataPTNLCD, use = "pairwise.complete.obs", method = "spearman") NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "bonferroni", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness")) NCTTNLvsPTNLCD NETWORK INVARIANCE TEST Test statistic M: 0.7141979 p-value 0 GLOBAL STRENGTH INVARIANCE TEST Global strength per group: 0 21.25911 Test statistic S: 21.25911 p-value 0 EDGE INVARIANCE TEST Var1 Var2 p-value 26 Neutro% HLA-DR+%CD14+CD16+ 1 51 Neutro% HLA-DR+%CD14+CD16- 0 52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1 76 Neutro% HLA-DR+%CD14-CD16+ 1 — Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPVHTHL5BXTF6ZZCDB4WT3WKWHMBANCNFSM5WG2VDVQ> . You are receiving this because you commented.Message ID: ***@***.***>

AlexCocker · 2022-12-01T18:13:51Z

Without correction, or with FDR based correction, I still see the same issue:

No Correction

NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "none", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness"))

NCTTNLvsPTNLCD

NETWORK INVARIANCE TEST
Test statistic M: 0.7141979
p-value 0

GLOBAL STRENGTH INVARIANCE TEST
Global strength per group: 0 21.25911
Test statistic S: 21.25911
p-value 0

EDGE INVARIANCE TEST

              Var1               Var2 p-value

26 Neutro% HLA-DR+%CD14+CD16+ 1
51 Neutro% HLA-DR+%CD14+CD16- 0
52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1
76 Neutro% HLA-DR+%CD14-CD16+ 1

FDR Correction

NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "BH", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness"))

NCTTNLvsPTNLCD

NETWORK INVARIANCE TEST
Test statistic M: 0.7141979
p-value 0

GLOBAL STRENGTH INVARIANCE TEST
Global strength per group: 0 21.25911
Test statistic S: 21.25911
p-value 0

EDGE INVARIANCE TEST

              Var1               Var2 p-value

26 Neutro% HLA-DR+%CD14+CD16+ 1
51 Neutro% HLA-DR+%CD14+CD16- 0
52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1
76 Neutro% HLA-DR+%CD14-CD16+ 1

cvborkulo · 2022-12-01T18:17:00Z

And just to be sure, you use the latest updated package from GitHub? Op do 1 dec. 2022 om 19:14 schreef AlexCocker ***@***.***>

…

Without correction, or with FDR based correction, I still see the same issue: No Correction NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "none", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness")) NCTTNLvsPTNLCD NETWORK INVARIANCE TEST Test statistic M: 0.7141979 p-value 0 GLOBAL STRENGTH INVARIANCE TEST Global strength per group: 0 21.25911 Test statistic S: 21.25911 p-value 0 EDGE INVARIANCE TEST Var1 Var2 p-value 26 Neutro% HLA-DR+%CD14+CD16+ 1 51 Neutro% HLA-DR+%CD14+CD16- 0 52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1 76 Neutro% HLA-DR+%CD14-CD16+ 1 FDR Correction NCTTNLvsPTNLCD <- NCT(corTNLCD, corPTNLCD, it = 10000, p.adjust.methods= "BH", test.edges=TRUE, edges="all", test.centrality=TRUE, centrality=c("strength","betweenness")) NCTTNLvsPTNLCD NETWORK INVARIANCE TEST Test statistic M: 0.7141979 p-value 0 GLOBAL STRENGTH INVARIANCE TEST Global strength per group: 0 21.25911 Test statistic S: 21.25911 p-value 0 EDGE INVARIANCE TEST Var1 Var2 p-value 26 Neutro% HLA-DR+%CD14+CD16+ 1 51 Neutro% HLA-DR+%CD14+CD16- 0 52 HLA-DR+%CD14+CD16+ HLA-DR+%CD14+CD16- 1 76 Neutro% HLA-DR+%CD14-CD16+ 1 — Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPVHTDA2BB4WL6UEBSEYCLWLDTGTANCNFSM5WG2VDVQ> . You are receiving this because you commented.Message ID: ***@***.***>

AlexCocker · 2022-12-01T18:52:37Z

This is the session info. I did update the NCT package before trying to run this but it may not have worked

Session Info

sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS 13.0.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] NetworkComparisonTest_2.2.1 psych_2.2.9 readxl_1.4.1

loaded via a namespace (and not attached):
[1] minqa_1.2.4 colorspace_2.0-3 deldir_1.0-6 class_7.3-20
[5] htmlTable_2.4.1 corpcor_1.6.10 base64enc_0.1-3 rstudioapi_0.14
[9] proxy_0.4-27 mice_3.14.0 lavaan_0.6-12 IsingFit_0.3.1
[13] fansi_1.0.3 mvtnorm_1.1-3 codetools_0.2-18 splines_4.1.2
[17] R.methodsS3_1.8.2 mnormt_2.1.1 doParallel_1.0.17 knitr_1.40
[21] glasso_1.11 networktools_1.5.0 Formula_1.2-4 polynom_1.4-1
[25] nloptr_2.0.3 broom_1.0.1 cluster_2.1.4 png_0.1-7
[29] R.oo_1.25.0 compiler_4.1.2 backports_1.4.1 assertthat_0.2.1
[33] Matrix_1.5-1 fastmap_1.1.0 cli_3.4.1 htmltools_0.5.3
[37] tools_4.1.2 igraph_1.3.5 gtable_0.3.1 glue_1.6.2
[41] reshape2_1.4.4 dplyr_1.0.10 Rcpp_1.0.9 carData_3.0-5
[45] cellranger_1.1.0 vctrs_0.4.2 gdata_2.18.0.1 nlme_3.1-159
[49] iterators_1.0.14 eigenmodel_1.11 xfun_0.33 stringr_1.4.1
[53] lme4_1.1-30 lifecycle_1.0.2 weights_1.0.4 gtools_3.9.3
[57] candisc_0.8-6 MASS_7.3-58.1 scales_1.2.1 heplots_1.3-9
[61] parallel_4.1.2 NetworkToolbox_1.4.2 smacof_2.1-5 RColorBrewer_1.1-3
[65] pbapply_1.5-0 gridExtra_2.3 ggplot2_3.3.6 IsingSampler_0.2.1
[69] rpart_4.1.16 latticeExtra_0.6-30 stringi_1.7.8 foreach_1.5.2
[73] plotrix_3.8-2 e1071_1.7-11 checkmate_2.1.0 bootnet_1.5
[77] boot_1.3-28 mgm_1.2-13 shape_1.4.6 rlang_1.0.6
[81] pkgconfig_2.0.3 lattice_0.20-45 purrr_0.3.4 htmlwidgets_1.5.4
[85] tidyselect_1.1.2 plyr_1.8.7 magrittr_2.0.3 R6_2.5.1
[89] snow_0.4-4 generics_0.1.3 nnls_1.4 Hmisc_4.7-1
[93] DBI_1.1.3 pillar_1.8.1 foreign_0.8-83 withr_2.5.0
[97] survival_3.4-0 abind_1.4-5 nnet_7.3-18 tibble_3.1.8
[101] car_3.1-0 interp_1.1-3 wordcloud_2.6 fdrtool_1.2.17
[105] utf8_1.2.2 ellipse_0.4.3 jpeg_0.1-9 grid_4.1.2
[109] qgraph_1.9.2 data.table_1.14.2 pbivnorm_0.6.0 digest_0.6.29
[113] tidyr_1.2.1 R.utils_2.12.0 stats4_4.1.2 munsell_0.5.0
[117] glmnet_4.1-4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Output p-values #33

Output p-values #33

IrinaVanzhula commented May 18, 2022

pinusm commented May 18, 2022

cvborkulo commented May 19, 2022

IrinaVanzhula commented Jun 1, 2022

IrinaVanzhula commented Jun 1, 2022

cvborkulo commented Jun 21, 2022 via email

cvborkulo commented Oct 11, 2022 via email

AlexCocker commented Nov 15, 2022

cvborkulo commented Nov 24, 2022

AlexCocker commented Nov 29, 2022

cvborkulo commented Dec 1, 2022 via email

AlexCocker commented Dec 1, 2022

cvborkulo commented Dec 1, 2022 via email

AlexCocker commented Dec 1, 2022

Output p-values #33

Output p-values #33

Comments

IrinaVanzhula commented May 18, 2022

pinusm commented May 18, 2022

cvborkulo commented May 19, 2022

IrinaVanzhula commented Jun 1, 2022

IrinaVanzhula commented Jun 1, 2022

NETWORK INVARIANCE TEST

cvborkulo commented Jun 21, 2022 via email

cvborkulo commented Oct 11, 2022 via email

AlexCocker commented Nov 15, 2022

cvborkulo commented Nov 24, 2022

AlexCocker commented Nov 29, 2022

cvborkulo commented Dec 1, 2022 via email

AlexCocker commented Dec 1, 2022

No Correction

FDR Correction

cvborkulo commented Dec 1, 2022 via email

AlexCocker commented Dec 1, 2022

Session Info