Does random permutation tests for the fixed effects in a REML analysis (R.W. Payne).
Options
PRINT = string tokens |
Controls printed output (prwald, criticalwald, ownstatistics, monitoring); default prwa, crit |
NTIMES = scalar |
Number of permutation samples to make; default 99 |
NRETRIES = scalar |
Maximum number of extra samples to take when some REML analyses fail to converge; default NTIMES |
BLOCKSTRUCTURE = formula |
Model formula defining any blocking to consider during the randomization; default |
EXCLUDE = factors |
Factors in the block formula whose levels are not to be randomized |
SEED = scalar |
Seed for random number generation; default 0 continues an existing sequence or, if none, selects a seed automatically |
WMETHOD = string token |
Controls which Wald statistics are used (add, drop); default add |
OWNMETHOD = string token |
Type of test required for own statistics (twosided, greaterthan, lessthan); default twos |
CIPROBABILITY = scalar |
Probability level for the confidence interval for own statistics; default 0.95 |
Parameters
SAVE = REML save structures |
Specifies the (REML) save structure of the original analysis; default * uses the SAVE structure from the most recent REML analysis |
WALD = pointers |
Wald statistics saved in a pointer with a variate for each term |
PRWALD = pointers |
Critical values for Wald statistics saved in a pointer with a scalar for each term |
CRITICALWALD = pointers |
Saves a pointer with variates for the 5%, 1% and 0.1% significance levels containing the corresponding critical values for the fixed terms, obtained from the quantiles of the Wald statistics from the permuted data sets |
NNOTCONVERGED = scalars |
Saves the number of permutations whose REML analysis failed to converge |
OWNDATA = pointers |
Data required to calculate own statistics |
OWNOBSERVEDVALUES = variates |
Saves observed values of the own statistics |
OWNPROBABILITIES = variates |
Saves probabilities for the own statistics |
OWNESTIMATES = variates |
Saves estimates for the own statistics |
OWNSES = variates |
Saves standard errors for the own statistics |
OWNLOWERCIS = variates |
Saves lower values of the confidence intervals for the own statistics |
OWNUPPERCIS = variates |
Saves upper values of the confidence intervals for the own statistics |
OWNSTATISTICS = pointers |
Saves the own statistics obtained from the permutation samples, in a pointer with a variate for each statistic |
Description
VPERMTEST performs a random permutation test for fixed effects in a REML analysis. The SAVE parameter can supply the save structure from the original analysis; if this is not set, the tests are done for the most recent REML analysis.
The test probabilities are calculated by taking the proportion of Wald statistics in the permutation samples that are larger than the observed Wald statistic of each fixed term. (As a result these should not suffer from the bias that is found in the probabilities for the Wald statistics themselves, which tend to be too low.) The WMETHOD option controls whether the Wald statistics are obtained from the table where terms are added sequentially (the default), or from the table where suitable terms are dropped from the full fixed model. Note that, if you use the table where terms are dropped, the only terms that can be tested are those that are not marginal to any other term in the fixed model: for example, the main effect A cannot be tested if the model contains an interaction, such as A.B.
The NTIMES option defines how many random permutations to perform; by default there are 99 (as well as the “null” permutation where the data keep their original order). The NRETRIES option specifies the maximum number of extra samples to take when some REML analyses fail to converge; the default is to use the same number as specified by NTIMES. The SEED option allows you to specify the seed to use for the random-number generator that is used to construct the permutation samples. The default, SEED=0, continues the sequence of random numbers from a previous generation or, if this is the first use of the generator in this run of Genstat, it initializes the seed automatically. If NTIMES exceed the maximum possible number of permutations for the data, an “exact” test is performed in which every permutation is used once. This is feasible only for small datasets. There are n! (n factorial) permutations of n units: 3!=6, 4!=24, 5!=120, 6!=720, 7!=5040, 8!=40320, and so on. The NNOTCONVERGED parameter can save the number of samples whose analyses did not converge, in a scalar.
If the data are from a designed experiment, you may need to use the BLOCKSTRUCTURE option to specify a block model to define how to do the randomization. The EXCLUDE option can then restrict the randomization so that one or more of the factors in the block model is not randomized. See the RANDOMIZE directive for further details.
Output is controlled by the PRINT option, with settings:
prwald to print the probabilities calculated from the distribution of the Wald statistics;
criticalwald to print a table giving estimated critical values for each of the Wald statistics, formed from the permutation samples;
ownstatistics to print the estimates, standard errors and confidence intervals for the own statistics, and
monitoring to monitor the progress of the test.
The Wald statistics from the permutation tests can be saved, in a pointer with a variate for each of the FIXEDTERMS, using the WALD parameter. The probabilities calculated from the tests can be saved, in a pointer with a scalar for each of the FIXEDTERMS, using the PRWALD parameter.
You can define your own statistics to be assessed by the test. They are calculated by a procedure _VPERMownstatistics, which is called by VPERMTEST following the REML analysis of each permutation sample. Its use is shown in the VPERMTEST example, which can be modified to calculate your own statistics instead. The information required by _VPERMownstatistics to do the calculations is supplied, in a pointer, by the OWNDATA parameter. The OWNMETHOD option specifies the type of test to be made. The default, twosided tests whether the statistics differ from zero. The greaterthan setting tests whether they are greater than zero, and the lessthan setting tests whether they are less than zero. Standard errors and confidence intervals are also calculated, The CIPROBABILITY option specifies the probability for the confidence intervals (default 0.95). The OWNOBSERVEDVALUES parameter can save a variate containing the values of the own statistics from the original data set. The OWNPROBABILITIES can save a variate containing the probabilities from the tests. The OWNESTIMATES can save a variate containing the bootstrap estimates of the statistics (calculated as the mean of the values obtained from the bootstrap samples) The OWNSES can save a variate containing standard errors of bootstrap estimates. The OWNLOWERCIS and OWNUPPERCIS parameters can save variates containing the lower and upper values, respectively, of the confidence intervals. Finally, the OWNSTATISTICS can save the values of the own statistics obtained from the bootstrap samples, in a pointer with a variate for each statistic.
The maximum number of iterations (MAXCYCLE) and number of blocks of internal memory to be (WORKSPACE) to be used in the REML analyses can be set by a call to the VAOPTIONS procedure before you use VPERMTEST.
Options: PRINT, NTIMES, NRETRIES, BLOCKSTRUCTURE, EXCLUDE, SEED, WMETHOD, OWNMETHOD, CIPROBABILITY.
Parameters: SAVE, WALD, PRWALD, CRITICALWALD, NNOTCONVERGED, OWNDATA, OWNOBSERVEDVALUES, OWNPROBABILITIES, OWNESTIMATES, OWNSES, OWNLOWERCIS, OWNUPPERCIS, OWNSTATISTICS.
See also
Directives: REML, VCOMPONENTS.
Procedures: VAOPTIONS, VBOOTSTRAP, VRPERMTEST.
Commands for: REML analysis of linear mixed models.
Example
CAPTION 'VPERMTEST example',!t('Split plot design, see the',\
'Guide to Genstat, Part 2, Section 4.2.1.'); STYLE=meta,plain
SPLOAD [PRINT=*] '%gendir%/data/Oats.gsh'
VCOMPONENTS [FIXED=variety*nitrogen] RANDOM=blocks/wplots/subplots
REML [PRINT=model,comp,Wald] yield
VPERMTEST [NTIMES=99; SEED=15405]
" Own statistics: test differences (in the full model) between variety
Marvellous and each of the other two varieties "
PROCEDURE [WORDLENGTH=long] '_VPERMownstatistics'
PARAMETER NAME=\
'DATA', "(I: pointer) information required to calculate the statistics"\
'STATISTICS';"(O: variate) estimated statistics"\
MODE=p; TYPE='pointer','variate'
" insert commands to calculate the statistics "
VKEEP DATA[1]; MEANS=means
VARIATE vmeans; VALUES=means
VARIATE [NVALUES=DATA[2]] STATISTICS
CALCULATE STATISTICS = vmeans$[DATA[3]] - vmeans$[DATA[4]]
ENDPROCEDURE "_VPERMownstatistics"
TEXT [VALUES='Marvellous - Victory','Marvellous - Golden rain'] Contrast
POINTER [VALUES=variety,Contrast,!(3,3),!(1,2)] Owninfo
VPERMTEST [PRINT=#,ownstatistics; NTIMES=99; SEED=251015] OWNDATA=Owninfo