Comparison of the 2008-2010 and 2010-2012 Substate Estimates
This discussion focuses on the methodology used to conduct statistical tests of significance for comparing National Survey on Drug Use and Health (NSDUH) substate estimates between 2008-2010 and 2010-2012. Estimates shown in Tables 1 to 26 earlier in this document are based on small area estimation (SAE) procedures, and the methodology used to compare these estimates is the same methodology used to compare the State small area estimates. For details, see Section A.13 of Appendix A in the 2009-2010 State SAE report.1 However, because some substate regions within five States (Louisiana, Maine, Nevada, South Dakota, and Tennessee) were defined differently for the two time periods (i.e., 2008-2010 and 2010-2012), significance test results (based on the Bayes posterior probability of no change: p value) could not be produced for these substate regions. Note that all regions in South Dakota changed across that time period. In this document's tables, a "--" is used to denote substate regions that are not defined consistently across the two time periods. For more details about the substate regions that changed definitions, see the "2010-2012 NSDUH Substate Region Definitions" at https://www.samhsa.gov/data/sites/default/files/substate2k12-StateTabs/NSDUHsubstateStateTabsTOC2012.htm.
Comparisons of substate estimates between 2008-2010 and 2010-2012 were made only for persons aged 12 or older except for the underage alcohol use outcome in Table 11 and the underage binge alcohol use outcome in Table 12 (where comparisons were made for persons aged 12 to 20) and for the four mental health outcomes in Tables 23 through 26 where comparisons were made for persons aged 18 or older (serious mental illness, any mental illness, serious thoughts of suicide, and major depressive episode outcomes).
To determine whether differences between substate estimates over two time periods were statistically significant, let 
and 
denote the 2008-2010 and 2010-2012 prevalence rates, respectively, for substate-s. The change between and is defined in terms of the log-odds ratio 
as opposed to the simple difference because the posterior distribution of the is closer to Gaussian than the posterior distribution of the simple difference, 
. The is defined as follows:
, D
where ln denotes the natural logarithm.
The p value given in the comparison tables is computed to test the null hypothesis of no change; that is, 
or equivalently 
. An estimate of is given by
, D
where the 
are the 2008-2010 substate estimates presented in the "2008-2010 NSDUH Substate Region Estimates: Excel Tables" at https://www.samhsa.gov/data/sites/default/files/Substate2k10-NSDUHsubstateStates2010/NSDUHsubstate51StateTOC2010.htm, and the 
are the 2010-2012 substate estimates presented in the "2010-2012 NSDUH Substate Region Estimates: Excel Tables" at https://www.samhsa.gov/data/sites/default/files/substate2k12-StateTabs/NSDUHsubstateStateTabsTOC2012.htm. Note that in the 2008-2010 substate Excel tables, the estimates and corresponding confidence intervals (CIs) for the CenterPoint region in North Carolina were suppressed for all outcomes. However, the less stringent criteria for minimum sample sizes used for the 2010-2012 regions allowed for the publication of the 2008-2010 estimates in these comparison tables.
To compute the posterior variance of 
, that is, 
, let 
and 
,
then 
, where 
denotes the covariance between 
and 
. This covariance is defined in terms of the associated correlation, as follows:
. D
Note that the 
and 
used here to calculate 
are the same variances used in calculating the 2008-2010 Bayesian CIs and the 2010-2012 CIs, respectively. Thus, the posterior variances can be calculated using the published Bayesian CIs, 
and 
, as follows:
, D
where 
and 
.
The correlation between and was obtained by simultaneously modeling the 2008-2009, 2010, and the 2011-2012 NSDUH data using a method similar to obtaining the correlation for producing State-level estimates of . For details, see Section A.13 of Appendix A in the 2009-2010 report.2
To calculate the p value for testing the null hypothesis of no difference (
) it was assumed that the posterior distribution of 
is normal with 
and 
. With the null value of () the Bayes p value or posterior probability of no difference is 
, where Z is a standard normal random variate, 
, and 
denotes the absolute value of z.
