2.4 | Statistical analyses
The percentage of structures meeting performance targets (ranking < 3) for integrity, erosion, and deposition was evaluated annually. If a structure received a failed (4) rating for integrity and function, it was excluded from the statistical analysis in subsequent years. The maximum ranking across years was used to summarize the overall effectiveness of instream structures for the project. The percentage of structures meeting the performance criteria was evaluated by year to investigate performance over time. The maximum ranking was also used to calculate the proportion of structures by type that failed to meet performance criteria for integrity, erosion, and deposition. Percentages were used for comparison to the project goal that 90% of instream structures be stable and functional three years after the completion of construction.
Cumulative link mixed models (CLMM) were fit to the ordinal rankings for integrity, erosion, and deposition to evaluate performance over time and between structure types using the ‘ordinal’ package (Christensen, 2022) in RStudio (Posit team, 2022). Integrity was treated as categorical response variable with four levels (Table 2), while rankings for erosion and deposition were collapsed into three levels: minor (ranking = 0-1), moderate (ranking = 2-3), and severe (ranking = 4-5). Structure type was included as a categorical predictor with eight levels (Table 1), and year was included as a categorical predicator with six levels. As structure type did not change across years, an interaction between structure type and year was not included in the models, but structure number was included as a random effect. Likelihood ratio tests were performed for each model to test for significant effects, and the ‘emmeans’ package (Lenth et al., 2023) was used to obtain estimated marginal means and perform pairwise comparisons for any significant effects. We defined statistical significance for all test as p< 0.10 to protect against Type II error.
Ordinal regression was also used to investigate the effects of log-vane slope and channel morphology on the maximum rankings for log-vane integrity, erosion, and deposition across years. Cumulative link models (CLM) were fit to the log-vane data using integrity, erosion, and deposition as ordinal response variables with the ‘ordinal’ package in RStudio (Christensen, 2022). Log-vane slope and angle of departure were treated as continuous factors, while preexisting morphology was included as a categorical factor with two levels (riffle or pool). Linear regression was used to test for correlation between log-vane slope and angle of departure. As the relationship between slope and angle of departure was significant (β = -129.4, p = 0.049, multiple R-squared = 0.095), only slope and morphology were included as additive factors in the CLMs for log-vane integrity, erosion, and deposition. Similar to the CLMMs, likelihood ratio tests were used to test for significant effects, and threshold coefficients were reported for all CLMs and CLMMs.
Residual pool depths were analyzed by structure type, years since construction, and preexisting morphology (riffle or pool). As the distribution for RPD values did not meet the assumption of normality, the change in RPD between each survey was calculated for each pool. We then used the ‘lme4’ package (Bates, Mächler, Bolker, & Walker, 2015) in RStudio to fit linear mixed-effect models with the change in RPD as the continuous response variable. As the variance in the response variable was not equal for riffles and pools (F = 0.327, p< 0.0001), separate models were used to evaluate the change in RPD for riffles and pools. The linear mixed-effects models included two categorical fixed effects: structure type (n = 7) and years since construction (n = 5). The two-way interaction between the independent variables was included in the models, and pool number was included as a random effect due to the repeated measures design. ANOVA was used to test for significant interaction and main effects, and the ‘emmeans’ package (Lenth et al., 2023) was used to perform pairwise comparisons for any significant effects.
Structure types for the RPD analysis included boulder clusters, boulder vanes, log toe, log vanes, and no structure, in addition to structure combinations of boulder vane/boulder toe and log vane/log toe. As construction occurred over two years (2013-2014), the years since construction was used for the RPD analysis rather than the actual year of the topographic survey. The changes in RPD between pre-construction and as-built conditions were represented by year 0, then the first post-runoff survey for each pool became year 1, and so on and so forth. Pools that were developed during construction in 2013 had five years of subsequent surveys (2014-2018 or years 1-5), while pools built during 2014 had four years of subsequent surveys (2015-2018 or years 1-4). As not all pools had five years of subsequent surveys following construction, year 5 was excluded from the models to improve balance in the dataset. Additionally, the structure types of boulder vane, boulder vane/boulder toe combination, and log vane/log toe combination all had sample sizes equal to or less than one for riffle habitats (Table 3). As such, those structure types were excluded from the model for riffles.