Significant edges different between networks (i.e. after edge invariance test) even though Network Invariance Test non-significant

[maxbri94]

Hi :)

Thank you for creating this great statistical test and package, really helpful!

I have a conceptual questions about evaluating the NCT results. I'm estimating two networks of depression symptoms post-intervention, one for a control group and one for the intervention group.

`network_model_T1_SCCM <- estimateNetwork(ana_data_T1_SCCM,
default = "EBICglasso",
tuning = 0.5)

network_model_T1_TAU <- estimateNetwork(ana_data_T1_TAU,
default = "EBICglasso",
tuning = 0.5)

NCT_SCCMTAUT1 <- NCT(network_model_T1_SCCM, network_model_T1_TAU, it=1000, test.edges = TRUE, abs=TRUE)
summary(NCT_SCCMTAUT1)
`

I get the following, non-significant results for network invariance and global strength invariance:

NETWORK INVARIANCE TEST
Test statistic M: 0.2550473
p-value 0.3326673

GLOBAL STRENGTH INVARIANCE TEST
Global strength per group: 3.663337 3.700252
Test statistic S: 0.036915
p-value 0.8591409

However, when I check out individual edges, I do actually find individual significant edge differences between the networks, e.g....

37 phq9_interest phq9_appetite 0.018981019 0.19195629
48 phq9_insomnia phq9_guilt 0.048951049 0.15244224
67 phq9_noenergy phq9_psychomotor 0.003996004 0.25504734

It was my understanding that a significant networ invariance test, being an omnibus test, indicates that there is at least one significant edge difference, i.e. I shouldn't be finding significant results in the edge invariance test.

What am I missing? Is this still a reasonable result or does this indicate mistakes in my analysis? Or did I misunderstand the network invariance test?

Thank you in advance!

Best,

Max

[KarolineHuth]

Hi Max,

the NCT is a bootstrap test of various measures one might be interested in. It is not an omnibus test as you might know from an Anova or so.

It could happen that the overall test is non-significant while individual edges do differ.

• the overall test examines the overall strength of connection, i.e., the sum of all edge weights
• the individual edge test merely compares one edge between two groups (and does so for all edges)

For example, if edge 1-2 is strong in group 1 but weak in group 2, edge 1-2 is different in the two groups. However, say edge 2-3 is strong in group 2 but weak in group 1, the overall edge strengths in the two networks might even out and the overall strength of connectivity does not differ.

I hope this helps!

[maxbri94]

Hi Karoline,

thank you so much for your response :)

I'm still a little bit confused. I understand that the overall strength of the network of two networks might be the same (i.e. as tested by the global strength invariance test) but individual edges may be different. However, in my question, I was more trying to understand how the network invariance strength can be non-significant while individual edges are significant.

In the 2023 paper by van Borkulo, the following passage - according to my reading - insinuates that a significant network invariance test can only happen if there is at least one significant edge difference between networks:

"Second, we inspected the omnibus test on invariance of network structure to investigate whether there are any differences in edges. Results indicated that there was at least one edge different: M = 0.24, p = 0.003. Consequently, we performed exploratory post hoc testing of all edges to investigate which edge(s) differed between networks. Results indicated that there are four edges that differed significantly, namely ins – hyp (E = 0.240, p = 0.002), int – hyp (E = 0.028, p = 0.016), dep and ins (E = 0.111, p = 0.028), and dep and ret (E = 0.075, p = 0.042)."

I hope my question is now clear. How do you then think do I get significant edges without a significant invariance of network structure test? Or if I misunderstood something, also happy to hear :)

THank you so much for your help!