IDL Analyst Reference Guide: Categorical and Discrete Data Analysis

IMSL_EXACT_NETWORK

The IMSL_EXACT_NETWORK function computes Fisher exact probabilities and a hybrid approximation of the Fisher exact method for a two-way contingency table using the network algorithm.

Syntax

Result = IMSL_EXACT_NETWORK(table [, APPROX_PARAMS=array]
[, /DOUBLE] [, /NO_APPROX] [, P_VALUE=variable] [, PROB_TABLE=variable] [, WK_PARAMS=array])

Return Value

The p-value for independence of rows and columns. The p-value represents the probability of a more extreme table where "extreme" is taken in the Neyman-Pearson sense. The p-value is "two-sided".

Arguments

table

Two-dimensional array containing the observed counts in the contingency table.

Keywords

APPROX_PARAMS

One-dimensional array of size 3. Approx_Params(0) is the expected value used in the hybrid approximation to Fisher's exact test algorithm for deciding when to use asymptotic probabilities when computing path lengths. Approx_Params(1) is the percentage of remaining cells that must have estimated expected values greater than Approx_Params(0) before asymptotic probabilities can be used in computing path lengths. Approx_Params(2) is the minimum cell estimated value allowed for asymptotic chi-squared probabilities to be used.

Asymptotic probabilities are used in computing path lengths whenever Approx_Params(1) or more of the cells in the table have estimated expected values of Approx_Params(0) or more, with no cell having expected value less than Approx_Params(2). See the Discussion section for details.

Defaults: Approx_Params(0) = 5.0

Approx_Params(1) = 80.0

Approx_Params(2) = 1.0

DOUBLE

If present and nonzero, double precision is used.

NO_APPROX

If present and nonzero, the Fisher exact test is used and Approx_Param is ignored.

P_VALUE

Named variable into which the p-value for independence of rows and columns is stored. The p-value represents the probability of a more extreme table where "extreme" is in the Neyman-Pearson sense. The P_Value is "two-sided". The p-value is also returned in functional form (see Returned Value).

A table is more extreme if its probability (for fixed marginals) is less than or equal to Prob_Table.

PROB_TABLE

Named variable into which the probability of the observed table occurring given that the null hypothesis of independent rows and columns is true is stored.

WK_PARAMS

One-dimensional array of size 3. The network algorithm requires a large amount of workspace. Some of the workspace requirements are well-defined, while most of the workspace requirements can only be estimated. The estimate is based primarily on table size.

The IMSL_EXACT_ENUM function allocates a default amount of workspace suitable for small problems. If the algorithm determines that this initial allocation of workspace is inadequate, the memory is freed, a larger amount of memory allocated (twice as much as the previous allocation), and the network algorithm is re-started. The algorithm allows for up to Wk_Params(2) attempts to complete the algorithm.

Because each attempt requires computer time, it is suggested that Wk_Params(0) and Wk_Params(1) be set to some large numbers (like 1,000 and 30,000) if the problem to be solved is large. It is suggested that Wk_Params(1) be 30 times larger than Wk_Params(0). Although IMSL_EXACT_ENUM will eventually work its way up to a large enough memory allocation, it is quicker to allocate enough memory initially.

The known (well-defined) workspace requirements are as follows: Define f_·· = SSf_ij equal to the sum of all cell frequencies in the observed table, nt = f_·· + 1, mx = max (n_rows, n_columns), mn = min (n_rows, n_columns), t₁ = max (800 + 7mx, (5 + 2mx) (n_rows + n_columns + 1) ), and t₂ = max(400 + mx, + 1, n_rows + n_columns + 1) where n_rows = N_ELEMENTS(table(*,0)) and n_columns = N_ELEMENTS(table(0,*)).

The following amount of integer workspace is allocated: 3mx + 2mn + t₁.

The following amount of real workspace is allocated: nt + t₂.

The remainder of workspace that is required must be estimated and allocated based on Wk_Params(0) and Wk_Params(1). The amount of integer workspace allocated is 6n (Wk_Params(0) + Wk_Params(1)). The amount of real workspace allocated is n (6*Wk_Params(0) + 2* Wk_Params(1)). Variable n is the index for the attempt, 1 < n £ Wk_Params(2).

Defaults: Wk_Params(0) = 100

Wk_Params(1) = 3000

Wk_Params(2) = 10

Discussion

The IMSL_EXACT_NETWORK function computes Fisher exact probabilities or a hybrid algorithm approximation to Fisher exact probabilities for an r by c contingency table with fixed row and column marginals (a marginal is the number of counts in a row or column), where r = n_rows and c = n_columns. Let f_ij denote the count in row i and column j of a table, and let f_i and f_·j denote the row and column marginals. Under the hypothesis of independence, the (conditional) probability of the fixed marginals of the observed table is given by:

where f_·· is the total number of counts in the table. P_f corresponds to output keyword Prob_Table.

A "more extreme" table X is defined in the probabilistic sense as more extreme than the observed table if the conditional probability computed for table X (for the same marginal sums) is less than the conditional probability computed for the observed table. Note that this definition can be considered "two-sided" in the cell counts.

Example

This example demonstrates various methods of computing chi-squared p-value with respect to accuracy. As seen in the output of this example, the Fisher exact probability and the usual asymptotic chi-squared probability (generated using IMSL_CONTINGENCY) can be different.

.RUN  
PRO print_results, p, p2, p3, p4  
   PRINT, 'Asymptotic Chi-Squared p-value'  
   PRINT, 'p-value =', p  
   PRINT, 'Network Algorithm with Approximation'  
   PRINT, 'p-value =', p2  
   PRINT, 'Network Algorithm without Approximation'  
   PRINT, 'p-value =', p3  
   PRINT, 'Total Enumeration Method'  
   PRINT, 'p-value =', p4  
END  
  
table = TRANSPOSE([[20, 20, 0, 0, 0], [10, 10, 2, 2, 1], $  
   [20, 20, 0, 0, 0]])  
p  = IMSL_CONTINGENCY(table)  
p2 = IMSL_EXACT_NETWORK(table)  
p3 = IMSL_EXACT_NETWORK(table, /NO_APPROX)  
p4 = IMSL_EXACT_ENUM(table)  
print_results, p, p2, p3, p4  
  
Asymptotic Chi-Squared p-value  
p-value =    0.0322604  
Network Algorithm with Approximation  
p-value =    0.0601165  
Network Algorithm without Approximation  
p-value =    0.0598085  
Total Enumeration Method  
p-value =    0.0597294