Conditional Independence Test

[zgong001]

The attached are entropy for one variable and for two variables.

[zgong001]

The attached is the Independence Test without any parent.

[zgong001]

The entropy of two different parent sets giving 2 variables are the following. The following is for sprinkler data set.

Suppose X = 5, Y = 6, W = 0 , Z = 1, W ' = 2, Z ' = 3 (5, 6, 0, 1, 2, 3 are column number)

p-value of ( X ind Y | W, Z) =0.5702, p-value of ( X ind Y | W ' , Z ' ) = 0.9399

log Score ( X , Pa (x) = { W, Z } = -35.4498 , log Score ( X , Pa (x) = { W ' , Z ' } = -54.4876

entrop (Nxwz) =1.565021 entrop (Nxw'z') = 1.42160

log Score ( Y , Pa (x) = { W, Z } )= -71.5613 , log Score ( X , Pa (x) = { W ' , Z ' } )=-72.1485

entrop (Nywz) =1.899676 entrop (Nyw'z') = 1.563514

entrop (Nywz) = 1.841074 entrop (Nyw'z') = 1.678443

[zgong001]

Suppose X = 6, Y = 7, W = 1 , Z = 2, W ' = 3, Z ' = 4 (7, 6, 1, 2, 3,4 are column numbers)

p-value of ( X ind Y | W, Z) =0.0003027, p-value of ( X ind Y | W ' , Z ' ) = 0.000283

log Score ( X , Pa (x) = { W, Z } =-71.3621 , log Score ( X , Pa (x) = { W ' , Z ' } = -57.7773

entrop (Nxwz) = 1.422716 entrop (Nxw'z') = 1.661798

log Score ( Y , Pa (x) = { W, Z } )= -73.1975 , log Score ( X , Pa (x) = { W ' , Z ' } )=-72.8787

entrop (Nywz) =1.448039 entrop (Nyw'z') = 1.804677

[zgong001]

The attached is the test result about the relationship between p-value and score, and relationship between score and entropy.

The yellow row in the sheet "p-value violation" does not following the rule:

if p-value of ( X ind Y | W, Z) =0.0003027 greater than ( > ) p-value of ( X ind Y | W ' , Z ' ) , then

log Score ( X , Pa (X) = { W, Z } less than ( < ) log Score ( X , Pa (X) = { W ' , Z ' } and

log Score ( Y , Pa (Y) = { W, Z } less than ( < ) log Score ( Y , Pa (Y) = { W ' , Z ' } .



The blue row in the sheet "entropy violation" does not following the rules:

if log Score ( X , Pa (X) = { W, Z } less than ( < ) log Score ( X , Pa (X) = { W ' , Z ' } then

entropy (Nxwz) less than ( < ) entropy (Nxw'z') .

if log Score ( Y , Pa (Y) = { W, Z } less than ( < ) log Score ( Y , Pa (Y) = { W ' , Z ' } then

entropy (Nywz) less than ( < ) entropy (Nyw'z').

[zgong001]

The attached is the updated test result about the relationship between p-value and score, and relationship between score and entropy.
The parent sets ( (w,z) and (w',z') are mutual exclusive. They have no intersection.

The yellow rows in the sheet "p-value violation" are p-value violation case.
The orange rows in the sheet "entropy violation" are entropy violation case.

[zgong001]

The attached is the detailed process of calculating the entropy.

[cwyoo]

The attached is the detailed process of calculating the entropy.

Please use conditional entropy and also present the corresponding variable scores as well.

[zgong001]

The attached is the detailed process of calculating the conditional entropy H((4,7)|(0,1)) and H((4,7)|(2,3)).

[zgong001]

The attached is the detailed process of calculating the conditional entropy H( 4| (0,1) ), H( 4 | (2,3) ), H( 7| (0,1) ) and H( 7| (2,3) ).