CFE Tree Cover Study (2008-2017-2021)
Bradley Saul
2026-06-04
Study Design
The design was based on the random point sampling described in Kaspar et al. (2017). 2500 points each were selected in the rural buffer and county owned lands, plus 1500 points in the county outside the buffer and county lands. Using a custom web application, users labeled each point as TREE, NOT TREE, or UNSURE based on NC Onemap orthoimagery from 2008, 2017, and 2021. This is the same point sample used in the 2019 analysis, now extended with a third year of imagery (2021). For power calculations, we assumed the study areas had 65% tree cover in 2008. Based on a type 1 error rate of 0.05, the study was designed to have 80% statistical power to detect a change in tree cover of ~3% within both the rural buffer and county lands.
The figure below shows all the points selected in the study. The purple points are the rural buffer; the red are county-owned lands; and the green are points in the county excluding the rural buffer or county-owned lands.
The application measured intra- and interrater reliability by displaying points that had previously been identified by the same user or another user with a probablity of 0.15.
Study Results
8 people participated in the identification of points. In total, 9896 identifications were made, and 714 points had at least one identification for all three years (11% of the available points).
| Area | 2008 | 2017 | 2021 | All years | Proportion of available points |
|---|---|---|---|---|---|
| Orange County owned property | 1215 | 1211 | 1216 | 267 | 0.11 |
| County excluding rural buffer and OC property | 722 | 704 | 739 | 150 | 0.10 |
| Rural buffer | 1250 | 1237 | 1196 | 297 | 0.12 |
| Overall | 3187 | 3152 | 3151 | 714 | 0.11 |
Rater Reliability
Intrarater
| # of times | # of point/years |
|---|---|
| 1 | 9803 |
| 2 | 45 |
| 3 | 1 |
In the 46 point/years where a given user identified the same point/year more than once, the following table shows the number of point/years and the proportion of those point/years were each user agreed on all their identifications of that point/year.
| User | Number of point/years | Proportion of Agreement |
|---|---|---|
| A | 12 | 0.58 |
| B | 23 | 0.96 |
| F | 1 | 1.00 |
| I | 7 | 1.00 |
| J | 3 | 1.00 |
| Pattern | n |
|---|---|
| TTU | 1 |
| TU | 5 |
Interrater
| # of different users | # of point/years |
|---|---|
| 1 | 9136 |
| 2 | 349 |
| 3 | 5 |
Excluding the point/years considered for intrarater reliablity, the proportion of point/years where there was complete interrater reliability was 0.65.
| Patterns | n |
|---|---|
| NT | 64 |
| NTT | 1 |
| NU | 18 |
| TTU | 1 |
| TU | 41 |
Paired Analysis
Handling Disagreements
Based on the intra- and interrater reliablity assessments, which generally showed good agreement, the following algorithm was chosen for point/years with disagreement:
- the majority rules in cases where more than half of the identifications (by any user and/or multiple by the same user) were a particular value; e.g., “NNT” \(\mapsto\) “N”; “TTU” \(\mapsto\) “T”; “TUU” \(\mapsto\) “U”; etc.
- all others are set to “U”
Patterns of change across all three years
The following table shows the observed
2008 -> 2017 -> 2021 tree cover pattern in each of
the three areas in our study after applying the voting algorithm
described above.
| Pattern | OC property | County Excluding RB and OC prop | Rural Buffer |
|---|---|---|---|
| U -> T -> U | 0.093 | 0.075 | 0.109 |
| N -> U -> U | 0.086 | 0.105 | 0.096 |
| U -> U -> T | 0.077 | 0.083 | 0.083 |
| T -> U -> U | 0.058 | 0.069 | 0.076 |
| U -> T -> T | 0.055 | 0.055 | 0.066 |
| U -> U -> N | 0.076 | 0.079 | 0.064 |
| U -> N -> U | 0.068 | 0.085 | 0.058 |
| T -> T -> U | 0.041 | 0.038 | 0.051 |
| T -> U -> T | 0.046 | 0.034 | 0.044 |
| U -> U -> U | 0.032 | 0.038 | 0.038 |
| N -> N -> U | 0.068 | 0.051 | 0.034 |
| N -> U -> N | 0.066 | 0.058 | 0.034 |
| T -> T -> T | 0.026 | 0.023 | 0.033 |
| N -> T -> U | 0.025 | 0.013 | 0.032 |
| U -> N -> N | 0.044 | 0.041 | 0.032 |
| N -> U -> T | 0.024 | 0.028 | 0.024 |
| N -> N -> N | 0.035 | 0.030 | 0.018 |
| N -> T -> T | 0.010 | 0.011 | 0.018 |
| U -> T -> N | 0.010 | 0.019 | 0.018 |
| T -> U -> N | 0.011 | 0.014 | 0.014 |
| U -> N -> T | 0.014 | 0.010 | 0.013 |
| T -> N -> U | 0.010 | 0.018 | 0.011 |
| N -> N -> T | 0.004 | 0.004 | 0.009 |
| T -> T -> N | 0.007 | 0.011 | 0.008 |
| T -> N -> T | 0.005 | 0.003 | 0.007 |
| T -> N -> N | 0.004 | 0.002 | 0.006 |
| N -> T -> N | 0.005 | 0.002 | 0.005 |
Most often a “U” indicates that the user was unable to make an identification because an image failed to load rather than being uncertain about an identification. That is, a “U” identification is unlikely to depend on the actual state of a point or the user. Hence, we assume the “U” identifications are missing completely at random. For each pairwise comparison below, we exclude points with a “U” in either year of that pair (so each comparison uses all points for which both of its years could be identified).
Estimated change in tree cover
The following table shows the estimated change in tree cover for each study area and time period with an adjusted Wald 95% confidence interval for matched pairs (Agresti and Min 2005; Scherer 2018). A positive estimate indicates a net gain in tree cover over the period. Each row uses the subset of points with a non-missing (non-“U”) identification in both years of that period.
| Area | Period | N (paired points) | Estimate (95% CI) |
|---|---|---|---|
| Orange County owned property | 2008 -> 2017 | 516 | 0.089 (0.0469, 0.1308) |
| Orange County owned property | 2008 -> 2021 | 526 | 0.068 (0.0261, 0.1103) |
| Orange County owned property | 2017 -> 2021 | 472 | 0.006 (-0.0346, 0.0473) |
| County excluding rural buffer and OC property | 2008 -> 2017 | 268 | 0.011 (-0.0478, 0.0701) |
| County excluding rural buffer and OC property | 2008 -> 2021 | 286 | 0.073 (0.0082, 0.1376) |
| County excluding rural buffer and OC property | 2017 -> 2021 | 275 | -0.072 (-0.1286, -0.0158) |
| Rural buffer | 2008 -> 2017 | 504 | 0.134 (0.0848, 0.1840) |
| Rural buffer | 2008 -> 2021 | 478 | 0.104 (0.0513, 0.1571) |
| Rural buffer | 2017 -> 2021 | 508 | -0.006 (-0.0497, 0.0379) |
The figure below shows the same estimates graphically. Intervals that do not cross the dashed line at zero are statistically significant at the 0.05 level: a positive estimate is a net gain in tree cover, a negative estimate a net loss.
Summary
Extending the original 2008-2017 study with a third year of imagery (2021), tree cover increased on Orange County owned property and in the rural buffer surrounding Chapel Hill and Carrboro, with the gains concentrated in the 2008-2017 period and little change from 2017-2021. The rest of the county, by contrast, was flat from 2008-2017 but shows a statistically significant decline of roughly 7 percentage points from 2017-2021; over the full 2008-2021 span all three areas still show a net increase. As discussed in the appendix, the 2008-2017 estimates (and hence the full-span estimates) are likely inflated by a labeling bias against the 2008 imagery, so the increases over periods that include 2008 should be read as upper bounds; the 2017-2021 decline in the rest of the county does not share that caveat.
Appendix: Comparison to the 2019 study
This study reuses the same point sample as the original 2019 analysis, which labeled 2008 and 2017 imagery. The tables below compare the two studies on the number of observers, the number of identifications, and the estimated change in tree cover over the period they share, 2008 -> 2017. To make the comparison fair, the 2019 estimates here are recomputed with the same matched-pairs method and 95% confidence level used throughout this report (the original 2019 report inadvertently used a 5% confidence level).
| 2019 study | 2026 study | |
|---|---|---|
| Image years | 2008, 2017 | 2008, 2017, 2021 |
| Participants | 7 | 8 |
| Identifications | 16294 | 9896 |
| Area | 2019 study | 2026 study |
|---|---|---|
| Orange County owned property | 0.030 (0.0189, 0.0406) | 0.089 (0.0469, 0.1308) |
| Rural buffer | 0.040 (0.0259, 0.0543) | 0.134 (0.0848, 0.1840) |
| County excluding rural buffer and OC property | 0.032 (0.0123, 0.0514) | 0.011 (-0.0478, 0.0701) |
Label agreement on the common points
The two studies disagree on the 2008 -> 2017 change estimate even though they labeled the same imagery. To separate coverage (which points each study identified) from labeling (how each study labeled them), we restrict both studies to the points that are paired – non-“U” in both 2008 and 2017 – in both studies, and re-estimate the change using each study’s own labels on that shared set.
| Area | N (paired points) | 2019 labels | 2026 labels |
|---|---|---|---|
| Orange County owned property | 463 | 0.037 (0.0153, 0.0578) | 0.090 (0.0463, 0.1344) |
| Rural buffer | 460 | 0.037 (0.0036, 0.0700) | 0.139 (0.0869, 0.1902) |
| County excluding rural buffer and OC property | 246 | 0.012 (-0.0366, 0.0608) | 0.008 (-0.0537, 0.0698) |
| Overall | 1169 | 0.032 (0.0130, 0.0501) | 0.092 (0.0623, 0.1221) |
Restricting to the common points does not reconcile the estimates, so the difference between the studies is driven by labeling rather than coverage. The table below shows, on those same points, how often the two studies agreed on a point’s label and the resulting tree-cover proportion under each study’s labels.
| Year | Agreement | % tree (2019) | % tree (2026) |
|---|---|---|---|
| 2008 | 0.785 | 0.600 | 0.443 |
| 2017 | 0.856 | 0.631 | 0.536 |
The disagreement is asymmetric: the 2026 raters labeled fewer points as TREE, and disproportionately so for 2008, which depresses the 2008 baseline more than 2017 and inflates the apparent 2008 -> 2017 increase. This is consistent with – and larger than – the bias toward labeling 2008 TREE points as NOT TREE noted in the 2019 validation, and it would tend to inflate rather than attenuate the change estimate. The more thoroughly identified 2019 labels are therefore likely the better estimate of the true 2008 -> 2017 change for the county-owned and rural-buffer areas.