Methodology Used to Estimate the Bayes
Significance Level for the Null Hypothesis of
No Change between the 2016-2017 and 2017-2018
Population Percentages
Comparisons between the state population percentages displayed in Tables 1 to 31 in this document are based on the 2016 through 2018 National Surveys on Drug Use and Health (NSDUHs). The moving average state estimates for the overlapping 2016-2017 and 2017-2018 time periods were obtained from independent applications of the survey-weighted hierarchical Bayes (SWHB) methodology; that is, the 2017-2018 models were fit independently of the previously fitted 2016-2017 models. This independent analysis approach was followed because there was no desire to revise the previously published 2016-2017 estimates. The same fixed predictor variables were used in the 2016-2017 and 2017-2018 models, but annual updates were made when more current versions were available.1 The age group-specific fixed predictor variables were defined at five levels (namely, the person level, census block group level, tract level, county level, and state level). Also, each age group model had 51 state-level random effects and 250 "within-state" area-level random effects.1
To estimate change in state population percentages, let 
and 
denote the 2016-2017 and 2017-2018 prevalence rates, respectively, for state-s and age group-a. The change between and is defined in terms of the log-odds ratio (
) as opposed to the simple difference because the posterior distribution of the is closer to Gaussian than the posterior distribution of the simple difference (
). The is defined as
, D
where ln denotes the natural logarithm. The p value given is computed to test the null hypothesis of no change (i.e., 
or equivalently 
). An estimate of is given by
, D
where the 
represents the previously published 2016-2017 state estimates and the 
represents the 2017-2018 state estimates. To compute the variance of 
, that is, 
, let 
and 
, then
, D
where 
denotes the covariance between 
and 
. This covariance is defined in terms of the associated correlation as follows:
. D
Note that 
and 
used here to calculate are the same variances used in calculating the previously published 2016-2017 Bayesian confidence intervals and the 2017-2018 Bayesian confidence intervals, respectively.
The correlation between and was obtained by simultaneously modeling the 2016, 2017, and 2018 NSDUH data. Note that these models used the same predictors employed in the 2017-2018 small area estimation model for all 3 years. This simultaneous modeling approach was adopted based on the results of the validation study2 conducted for measuring change in the 1999-2000 and 2000-2001 state population percentages. For this simultaneous model, four age groups (12 to 17, 18 to 25, 26 to 34, and 35 or older) by 3 years (2016, 2017, and 2018), that is, 12 subpopulation-specific models, were fitted, each with its own set of fixed and random effects. In this case, the general covariance matrices for the state and within-state random effects were 12 × 12 matrices corresponding to the 12 element (age group × year) vectors of random effects. Note that the survey-weighted, Bernoulli-type log likelihood employed in the SWHB methodology was appropriate for this simultaneous model because the 12 age group × year subpopulations were nonoverlapping. The correlation 
was approximated by the correlation calculated using the posterior distributions of 
and 
from the simultaneous model.
To calculate the p value for testing the null hypothesis of no difference (
), it is assumed that the posterior distribution of 
is normal with 
and 
. With the null value (
), the Bayes p value or significance level for the null hypothesis of no difference is 
, where 
is a standard normal random variate, 
, and 
denotes the absolute value of 
. This Bayesian significance level (or p value) for the null value of , say 
, is defined following Rubin3 as the posterior probability for the collection of the values that are less likely or have smaller posterior density 
than the null (no change) value . That is, 
. With the posterior distribution of approximately normal, 
is given by the above expression.
