Comment Log Display

Over the last year, we have continually requested the data used to
validate the DNDC model and assess model uncertainty and bias. Thus
far, Board staff have provided a list of publications and a single
graph showing DNDC model-generated emissions vs. field measured
emissions. In order for us, other researchers and the public to be
able to review the model validation, a detailed description of the
specific parameters used in the graphed model runs are needed.
These data include the measured soil and environmental parameters
that were used and the parameters that were taken from databases
for the full set of 87 sites. All of these data must be available,
since they would be needed to generate the summary graph and to
support the assertions about bias made by Board staff. We have
repeatedly, over the course of 2014, been assured that that this
information would be made publically available, yet it has not yet
been made available. 

We recognize the heavily conservative decisions taken in the
adjustments of the model runs using Monte Carlo simulations, and
the small number of credits expected to be generated by this
Protocol. Still, releasing these data in a form that can be
reviewed is important for a number of reasons.

Firstly, this is the first time under the Compliance Offset Program
that emissions reductions will be estimated using a process-based
biogeochemical model. In the future, the acceptance of such a model
hinges on the precedent established by the approach taken in this
protocol. Fully transparent documentation of the DNDC model’s
validation is thus essential because it sets a precedent for future
protocols.

Secondly, there may be very real and substantive questions that
hinge on how DNDC was run in these calibrations. For example, in
the rice project protocol developed by the Climate Action Reserve
for the voluntary market, limits on the range of Soil Organic
Carbon (SOC) percentages for rice fields were included because
there were not enough data points and model runs from a full range
of SOCs to know whether DNDC was accurate beyond a specific range.
When used to estimate emissions, DNDC is calibrated to each field’s
growing conditions, and thus the model is tuned to those specific
conditions. (It has been criticized for being overly tuned and thus
hard to apply to new sets conditions where it has not yet been
validated.) Thus, knowing all the environmental conditions (i.e.,
the ranges of parameters used in the model), and the source data
for those parameters and outputs from the model runs is critical to
assessing the model’s validity for use in estimating emissions
reduction in the conditions under which it will be used in
California’s offset program. 

Finally, we note that the October 2014 draft protocol included
separate uncertainty deduction factors for the different rice
growing regions, under the reasonable assumption that the model
might perform differently in such different climatic and soil
characteristic ranges as California and the mid-South. We have
since seen that the proposed latest version of the protocol
referred to on February 20th, 2015 will include a single factor for
all regions. We have also heard that this change is supported by
statistical analyses made available to us by Dr. Bill Salas from
Applied GeoSolutions, who found that there are no significant
region-specific terms in the regression model nor any statistically
significant different residuals in the model between different
regions, suggesting that no region-specific uncertainty deductions
are warranted. Essentially, Dr. Salas’s analysis showed that there
is no statistical improvement in regressing the predicted vs.
observed emissions on a region-by-region basis rather than doing it
as a whole.  Such detailed analysis is essential to understanding
the bases on which decisions have been made. Dr. Salas’ description
was sent to us personally (and was not made publically available).

What we have not seen is whether there are any variations in model
performance based on project-type. Specifically, has the DNDC model
been validated separate and specifically for AWD projects? Do we
know whether DNDC captures CH4 emissions from cycles of drying and
wetting the same way that it captures such emissions from rice
fields grown under baseline conditions? Assessments of model
uncertainty and bias for specific project-types that support the
assertion of model validity for those specific project types should
be made available to the public.

What has been provided by Board staff to date – the list of the
scientific publications that report the field measurements that
were used to validate the model -- is only half the information.
It’s only one of the axes on the graph. We request that, before
finalizing the protocol and sending it to the Board for adoption,
the public be given a chance to assess the assertion of model
validity in its entirety based on all the information ARB is using
to make its own assessments.

Lastly, regarding N2O emissions, we believe that it is important to
address the potential significant increases (spikes) in N2O
emissions from AWD projects that dry fields too quickly after N
fertilizer application. There appears to be a lack of consensus
among experts about whether the DNDC model can accurately estimate
N2O emissions under such conditions. This means that even though
N2O emissions increases, modeled by DNDC, are included as debits in
the calculation of emissions reductions the full extent of N2O
emissions pulses may be underestimated. The current draft Protocol
does not address this issue. We urge ARB to convene a discussion of
the issue of post-fertilizer application spikes from rice
cultivated under an AWD regime, and DNDC’s performance at modeling
such spikes, among scientists working on this issue. In particular,
such a discussion should address the period of time after N
fertilizer application during which high spikes might be
anticipated and should seek potential solutions in the protocol to
address this concern – such as required delays after fertilizer
application before the first drying period in an AWD project.
Finally, and in addition, because of the salience of the potential
for N2O spikes in this period, we urge that guidance should be
provided to project proponents on ways to minimize the risk of N2O
spikes when conducting an AWD project.