calibtools¶

This module implements features for calibrating model parameters.

Module calibtools implements the following members:

TypeParameter Type variable.

TypeRule Type variable.

TypeRule1 Type variable.

TypeRule2 Type variable.

TargetFunction Protocol class for the target function required by class CalibrationInterface.

Adaptor Protocol class for defining adoptors required by Replace objects.

SumAdaptor Adaptor, which calculates the sum of the values of multiple Rule objects and assigns it to the value(s) of the target Parameter object.

FactorAdaptor Adaptor, which calculates the product of the value of the parent Replace object and the value(s) of a given reference Parameter object and assigns it to the value(s) of the target Parameter object.

Rule Base class for defining calibration rules.

Replace Rule class, which simply replaces the current model parameter value(s) with the current calibration parameter value.

Add Rule class, which adds its calibration delta to the original model parameter value(s).

Multiply Rule class for multiplying the original model parameter value(s) by its calibration factor.

CalibrationInterface Interface for the coupling of HydPy to optimisation libraries like NLopt.

RuleIUH A Rule, class specialised for IUH parameters.

ReplaceIUH A RuleIUH class for replacing IUH parameter values with the current calibration parameter values.

MultiplyIUH A RuleIUH class for replacing IUH parameter values with the current calibration parameter values, applied on the original IUH values as factors.

CalibSpec Helper class for specifying the properties of a single calibration parameter.

CalibSpecs Collection class for handling CalibSpec objects.

make_rules() Conveniently create multiple Rule objects at once.

class hydpy.auxs.calibtools.TargetFunction(*args, **kwargs)[source]¶

Bases: Protocol

Protocol class for the target function required by class CalibrationInterface.

The target functions must calculate and return a floating-point number reflecting the quality of the current parameterisation of the models of the current project. Often, as in the following example, the target function relies on objective functions as nse(), applied on the time series of the Sim and Obs sequences handled by the HydPy object:

>>> from hydpy import HydPy, nse, TargetFunction
>>> class Target(TargetFunction):
...     def __init__(self, hp):
...         self.hp = hp
...     def __call__(self):
...         return sum(nse(node=node) for node in self.hp.nodes)
>>> target = Target(HydPy())

See the documentation on class CalibrationInterface for more information.

class hydpy.auxs.calibtools.Adaptor(*args, **kwargs)[source]¶

Bases: Protocol

Protocol class for defining adoptors required by Replace objects.

Often, one calibration parameter (represented by one Replace object) depends on other calibration parameters (represented by other Replace objects) or other “real” parameter values. Please select an existing or define a new adaptor and assign it to a Replace object to introduce such dependencies.

See class SumAdaptor or class FactorAdaptor for concrete examples.

class hydpy.auxs.calibtools.SumAdaptor(*rules: Rule[Parameter])[source]¶

Bases: Adaptor

Adaptor, which calculates the sum of the values of multiple Rule objects and assigns it to the value(s) of the target Parameter object.

Class SumAdaptor helps to introduce “larger than” relationships between calibration parameters. A common use case is the time of concentration of different runoff components. For example, the time of concentration of base flow should be larger than the one of direct runoff. Accordingly, when modelling runoff concentration with linear storages, the recession coefficient of direct runoff should be larger. Principally, we could ensure this during a calibration process by defining two Rule objects with fixed non-overlapping parameter ranges. For example, we could search for the best direct runoff delay between 1 and 5 days and the base flow delay between 5 and 100 days. We demonstrate this for the recession coefficient parameters K and K4 of application model hland_v1 (assuming the nonlinearity parameter Alpha to be zero):

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import Replace, SumAdaptor
>>> k = Replace(name="k",
...             parameter="k",
...             value=2.0**-1,
...             lower=5.0**-1,
...             upper=1.0**-1,
...             parameterstep="1d",
...             model="hland_v1")
>>> k4 = Replace(name="k4",
...             parameter="k4",
...             value=10.0**-1,
...             lower=100.0**-1,
...             upper=5.0**-1,
...             parameterstep="1d",
...             model="hland_v1")

To allow for non-fixed non-overlapping ranges, we can prepare a SumAdaptor object, knowing both our Rule objects, assign it the direct runoff-related Rule object, and, for example, set its lower boundary to zero:

>>> k.adaptor = SumAdaptor(k, k4)
>>> k.lower = 0.0

Calling method apply_value() of the Replace objects makes our SumAdaptor object apply the sum of the values of all of its Rule objects:

>>> control = hp.elements.land_dill.model.parameters.control
>>> k.apply_value()
>>> with pub.options.parameterstep("1d"):
...     control.k
k(0.6)

class hydpy.auxs.calibtools.FactorAdaptor(rule: Rule[Parameter], reference: Type[Parameter] | Parameter | str, mask: BaseMask | str | None = None)[source]¶

Bases: Adaptor

Adaptor, which calculates the product of the value of the parent Replace object and the value(s) of a given reference Parameter object and assigns it to the value(s) of the target Parameter object.

Class FactorAdaptor helps to respect dependencies between model parameters. If you, for example, aim at calibrating the permanent wilting point (PWP) of model lland_v1, you need to make sure it always agrees with the maximum soil water storage (WMax). Especially, one should avoid permanent wilting points larger than total porosity. Due to the high variability of soil properties within most catchments, it is no real option to define a fixed upper threshold for PWP. By using class FactorAdaptor, you can instead calibrate a multiplication factor. Setting the bounds of such a factor to 0.0 and 0.5, for example, would result in PWP values ranging from zero up to half of WMax for each respective response unit.

To show how class FactorAdaptor works, we select another use-case based on the Lahn example project prepared by function prepare_full_example_2():

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()

hland_v1 calculates the “normal” potential snow-melt with the degree-day factor CFMax. For glacial zones, it also calculates a separate potential glacier-melt with the additional degree-day factor GMelt. Suppose we have CFMax readily available for the different hydrological response units of the Lahn catchment. We might find it useful to calibrate GMelt based on the spatial pattern of CFMax. Therefore, we first define an Replace rule for parameter GMelt:

>>> from hydpy import Replace, FactorAdaptor
>>> gmelt = Replace(name="gmelt",
...                 parameter="gmelt",
...                 value=2.0,
...                 lower=0.5,
...                 upper=2.0,
...                 parameterstep="1d",
...                 model="hland_v1")

Second, we initialise a FactorAdaptor object based on target rule gmelt and our reference parameter CFMax and assign it our rule object:

>>> gmelt.adaptor = FactorAdaptor(gmelt, "cfmax")

The Dill subcatchment, like the whole Lahn basin, does not contain any glaciers. Hence it defines (identical) CFMax values for the zones of type FIELD and FOREST but must not specify any value for GMelt:

>>> control = hp.elements.land_dill.model.parameters.control
>>> control.cfmax
cfmax(field=4.55853, forest=2.735118)
>>> control.gmelt
gmelt(nan)

Next, we call method apply_value() of the Replace object to apply the FactorAdaptor object on all relevant GMelt instances of the Lahn catchment:

>>> gmelt.adaptor(control.gmelt)

The string representation of the GMelt instance of the Dill catchment indicates nothing happened:

>>> control.gmelt
gmelt(nan)

However, inspecting the individual values of the respective response units reveals the multiplication was successful:

>>> from hydpy import print_values
>>> print_values(control.gmelt.values)
9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236, 9.11706,
5.470236, 9.11706, 5.470236, 9.11706, 5.470236

Calculating values for response units that do not require these values can be misleading. We can improve the situation by using the masks provided by the respective model; in our example, mask Glacier. To make this clearer, we set the first six response units to ZoneType GLACIER:

>>> from hydpy.models.hland_v1 import *
>>> control.zonetype(GLACIER, GLACIER, GLACIER, GLACIER, GLACIER, GLACIER,
...                  FIELD, FOREST, ILAKE, FIELD, FOREST, ILAKE)

We now can assign the SumAdaptor object to the direct runoff-related Replace object and, for example, set its lower boundary to zero:

Now we create a new FactorAdaptor object, handling the same parameters but also the Glacier mask:

>>> gmelt.adaptor = FactorAdaptor(gmelt, "cfmax", "glacier")

To see the results of our new adaptor object, we change the values both of our reference parameter and our rule object:

>>> control.cfmax(field=5.0, forest=3.0, glacier=6.0)
>>> gmelt.value = 0.5

The string representation of our target parameter shows that the glacier-related day degree factor of all glacier zones is now half as large as the snow-related one:

>>> gmelt.apply_value()
>>> control.gmelt
gmelt(3.0)

Note that all remaining values (for zone types FIELD, FOREST, and ILAKE are still the same. This intended behaviour allows calibrating, for example, hydrological response units of different types with different rule objects:

>>> print_values(control.gmelt.values)
3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 9.11706, 5.470236, 9.11706, 5.470236,
9.11706, 5.470236

Bases: ABC, Generic[TypeParameter]

Base class for defining calibration rules.

Each Rule object relates one calibration parameter with some model parameters. We select the class Replace as a concrete example for the following explanations and use the Lahn example project, which we prepare by calling function prepare_full_example_2():

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()

We define a Rule object supposed to replace the values of parameter FC of application model lland_v1. Note that argument name is the rule’s name, whereas the argument parameter is the parameter’s name:

>>> from hydpy import Replace
>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                model="hland_v1")

The following string representation shows us the complete list of available arguments:

>>> rule
Replace(
    name="fc",
    parameter="fc",
    value=100.0,
    lower=-inf,
    upper=inf,
    keyword=None,
    parameterstep=None,
    model="hland_v1",
    selections=("complete",),
)

The initial value of parameter FC is 206 mm:

>>> fc = hp.elements.land_lahn_1.model.parameters.control.fc
>>> fc
fc(206.0)

We can modify it by calling method apply_value():

>>> rule.apply_value()
>>> fc
fc(100.0)

You can change and apply the value at any time:

>>> rule.value = 200.0
>>> rule.apply_value()
>>> fc
fc(200.0)

Sometimes, one must differentiate between the original value to be calibrated and the actually applied value. Therefore, (only) the Replace class allows for defining custom “adaptors”. Prepare an Adaptor function and assign it to the relevant Replace object (see the documentation on class SumAdaptor or FactorAdaptor for more realistic examples):

>>> rule.adaptor = lambda target: target(2.0 * rule.value)

Now, our rule does not apply the original but the adapted calibration parameter value:

>>> rule.apply_value()
>>> fc
fc(400.0)

Use method reset_parameters() to restore the original states of the affected parameters (“original” here means at the time of initialisation of the Rule object):

>>> rule.reset_parameters()
>>> fc
fc(206.0)

Some parameter types support defining their values via custom keywords. FC, for example, allows setting the values of multiple zones of the same land-use type via keyword arguments such as forest:

>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                keyword="forest",
...                model="hland_v1")
>>> rule.apply_value()
>>> fc
fc(field=206.0, forest=100.0)

The value of parameter FC is not time-dependent. Therefore, any parameterstep information given to its Rule object is ignored (note that we pass an example parameter object of type FC instead of the string fc this time):

>>> Replace(name="fc",
...         parameter=fc,
...         value=100.0,
...         model="hland_v1",
...         parameterstep="1d")
Replace(
    name="fc",
    parameter="fc",
    value=100.0,
    lower=-inf,
    upper=inf,
    keyword=None,
    parameterstep=None,
    model="hland_v1",
    selections=("complete",),
)

For time-dependent parameters, the rule queries the current global parameterstep value if you do not specify one explicitly (note that we pass the parameter type PercMax and the module hland_v1 this time):

>>> from hydpy.models import hland_v1
>>> from hydpy.models.hland.hland_control import PercMax
>>> rule = Replace(name="percmax",
...                parameter=PercMax,
...                value=5.0,
...                model=hland_v1)

The Rule object internally handles, to avoid confusion, a copy of parameterstep.

>>> from hydpy import pub
>>> pub.options.parameterstep = None
>>> rule
Replace(
    name="percmax",
    parameter="percmax",
    value=5.0,
    lower=-inf,
    upper=inf,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("complete",),
)
>>> rule.apply_value()
>>> percmax = hp.elements.land_lahn_1.model.parameters.control.percmax
>>> with pub.options.parameterstep("1d"):
...     percmax
percmax(5.0)

Alternatively, you can pass a parameter step size yourself:

>>> rule = Replace(name="percmax",
...                parameter="percmax",
...                value=5.0,
...                model="hland_v1",
...                parameterstep="2d")
>>> rule.apply_value()
>>> with pub.options.parameterstep("1d"):
...     percmax
percmax(2.5)

Missing parameter step-size information results in the following error:

>>> Replace(name="percmax",
...         parameter="percmax",
...         value=5.0,
...         model="hland_v1")
Traceback (most recent call last):
...
RuntimeError: While trying to initialise the `Replace` rule object `percmax`, the following error occurred: Rules which handle time-dependent parameters require information on the parameter timestep size.  Either assign it directly or define it via option `parameterstep`.

With the following definition, the Rule object queries all Element objects handling hland_v1 instances from the global Selections object pub.selections:

>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                model="hland_v1")
>>> rule.elements
Elements("land_dill", "land_lahn_1", "land_lahn_2", "land_lahn_3")

Alternatively, you can specify selections by passing themselves or their names (the latter requires them to be a member of pub.selections):

>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                selections=[pub.selections.headwaters, "nonheadwaters"])
>>> rule.elements
Elements("land_dill", "land_lahn_1", "land_lahn_2", "land_lahn_3")

Without using the model argument, you must ensure the selected elements handle the correct model instance yourself:

>>> Replace(name="fc",
...         parameter="fc",
...         value=100.0)
Traceback (most recent call last):
...
RuntimeError: While trying to initialise the `Replace` rule object `fc`, the following error occurred: Model `musk_classic` of element `stream_dill_lahn_2` does not define a control parameter named `fc`.

>>> Replace(name="fc",
...         parameter="fc",
...         value=100.0,
...         model="musk_classic",
...         selections=[pub.selections.headwaters, "nonheadwaters"])
Traceback (most recent call last):
...
ValueError: While trying to initialise the `Replace` rule object `fc`, the following error occurred: Object `Selections("headwaters", "nonheadwaters")` does not handle any `musk_classic` model instances.

name: str¶: The name of the Rule object.

parametername: str¶: The name of the addressed Parameter objects.

keyword: str | None¶: The name of the addressed keyword argument or, for a positional argument, None.

upper: float¶

Upper boundary value.

No upper boundary corresponds to plus inf.

lower: float¶

Lower boundary value.

No lower boundary corresponds to minus inf.

selections: Tuple[str, ...]¶: The names of all relevant Selection objects.

elements: Elements¶: The Element objects, which handle the relevant target Parameter instances.

parametertype: Type[TypeParameter]¶: The type of the addressed Parameter objects.

property value: float¶

The calibration parameter value.

Property value ensures that the given value adheres to the defined lower and upper boundaries:

>>> from hydpy import Replace
>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                lower=50.0,
...                upper=200.0,
...                model="hland_v1")

>>> rule.value = 0.0
>>> rule.value
50.0

With option warntrim enabled (the default), property value also emits a warning like the following:

>>> from hydpy.core.testtools import warn_later
>>> with pub.options.warntrim(True), warn_later():
...     rule.value = 300.0
UserWarning: The value of the `Replace` object `fc` must not be smaller than `50.0` or larger than `200.0`, but the given value is `300.0`.  Applying the trimmed value `200.0` instead.
>>> rule.value
200.0

abstract apply_value() → None[source]¶

Apply the current value to the relevant Parameter objects.

To be overridden by the concrete subclasses.

reset_parameters() → None[source]¶

Reset all relevant parameter objects to their original states.

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import Replace
>>> rule = Replace(name="fc",
...                parameter="fc",
...                value=100.0,
...                model="hland_v1")
>>> fc = hp.elements.land_lahn_1.model.parameters.control.fc
>>> fc
fc(206.0)
>>> fc(100.0)
>>> fc
fc(100.0)
>>> rule.reset_parameters()
>>> fc
fc(206.0)

property parameterstep: Period | None¶

The parameter step size relevant to the related model parameter.

For non-time-dependent parameters, property parameterstep is (usually) None.

assignrepr(prefix: str, indent: int = 0) → str[source]¶: Return a string representation of the actual Rule object prefixed with the given string.

Bases: Rule[Parameter]

Rule class, which simply replaces the current model parameter value(s) with the current calibration parameter value.

See the documentation on class Rule for further information.

adaptor: Adaptor | None = None¶

An optional function object for customising individual calibration strategies.

See the documentation on the classes Rule, SumAdaptor, and FactorAdaptor for further information.

apply_value() → None[source]¶

Apply the current value to the relevant Parameter objects.

See the documentation on class Rule for further information.

Bases: Rule[Parameter]

Rule class, which adds its calibration delta to the original model parameter value(s).

Please read the examples of the documentation on class Rule first. Here, we modify some of these examples to show the unique features of class Add.

The first example deals with the non-time-dependent parameter FC. The following Add object adds its current value to the parameter’s original values:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import Add
>>> rule = Add(name="fc",
...            parameter="fc",
...            value=100.0,
...            model="hland_v1")
>>> fc = hp.elements.land_lahn_1.model.parameters.control.fc
>>> fc
fc(206.0)
>>> rule.apply_value()
>>> fc
fc(306.0)

When specifying the keyword field, the Add rule modifies the field capacity of zones of type FIELD only:

>>> fc(206.0)
>>> rule = Add(name="fc",
...            parameter="fc",
...            value=100.0,
...            keyword="field",
...            model="hland_v1")
>>> rule.apply_value()
>>> fc
fc(field=306.0, forest=206.0)

The second example deals with the time-dependent parameter CFMax and shows that everything works even when the actual parameterstep (2 days) differs from the current simulationstep (1 day):

>>> rule = Add(name="cfmax",
...            parameter="cfmax",
...            value=2.0,
...            model="hland_v1",
...            parameterstep="2d")
>>> cfmax = hp.elements.land_lahn_1.model.parameters.control.cfmax
>>> cfmax
cfmax(field=5.0, forest=3.0)
>>> rule.apply_value()
>>> cfmax
cfmax(field=6.0, forest=4.0)

This time, we modify the FOREST zones only:

>>> cfmax(field=5.0, forest=3.0)
>>> rule = Add(name="cfmax",
...            parameter="cfmax",
...            value=2.0,
...            keyword="forest",
...            model="hland_v1",
...            parameterstep="2d")
>>> rule.apply_value()
>>> cfmax
cfmax(field=5.0, forest=4.0)

In the third example, we modify the scalar parameter NmbSegments by its optional keyword argument lag:

>>> rule = Add(name="lag",
...            parameter="nmbsegments",
...            value=1.0,
...            keyword="lag",
...            model="musk_classic",
...            parameterstep="2d")
>>> nmbsegments = hp.elements.stream_lahn_1_lahn_2.model.parameters.control.nmbsegments
>>> nmbsegments
nmbsegments(lag=0.583)
>>> rule.apply_value()
>>> nmbsegments
nmbsegments(lag=2.583)

apply_value() → None[source]¶: Apply the current (adapted) value to the relevant Parameter objects.

Bases: Rule[Parameter]

Rule class for multiplying the original model parameter value(s) by its calibration factor.

Please read the examples of the documentation on class Rule first. Here, we modify some of these examples to show the unique features of class Multiply.

The first example deals with the non-time-dependent parameter FC. The following Multiply object multiplies the parameter’s original values by its current calibration factor:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import Add
>>> rule = Multiply(name="fc",
...                 parameter="fc",
...                 value=2.0,
...                 model="hland_v1")
>>> fc = hp.elements.land_lahn_1.model.parameters.control.fc
>>> fc
fc(206.0)
>>> rule.apply_value()
>>> fc
fc(412.0)

When specifying the keyword field, the Multiply rule modifies the field capacity of zones of type FIELD only:

>>> fc(206.0)
>>> rule = Multiply(name="fc",
...            parameter="fc",
...            value=2.0,
...            keyword="field",
...            model="hland_v1")
>>> rule.apply_value()
>>> fc
fc(field=412.0, forest=206.0)

The second example deals with the time-dependent parameter CFMax and shows that everything works even when the actual parameterstep (2 days) differs from the current simulationstep (1 day):

>>> rule = Multiply(name="cfmax",
...                 parameter="cfmax",
...                 value=2.0,
...                 model="hland_v1",
...                 parameterstep="2d")
>>> cfmax = hp.elements.land_lahn_1.model.parameters.control.cfmax
>>> cfmax
cfmax(field=5.0, forest=3.0)
>>> rule.apply_value()
>>> cfmax
cfmax(field=10.0, forest=6.0)

This time, we modify the FOREST zones only:

>>> cfmax(field=5.0, forest=3.0)
>>> rule = Multiply(name="cfmax",
...                 parameter="cfmax",
...                 value=2.0,
...                 keyword="forest",
...                 model="hland_v1",
...                 parameterstep="2d")
>>> cfmax
cfmax(field=5.0, forest=3.0)
>>> rule.apply_value()
>>> cfmax
cfmax(field=5.0, forest=6.0)

In the third example, we modify the scalar parameter NmbSegments by its optional keyword argument lag:

>>> rule = Multiply(name="lag",
...            parameter="nmbsegments",
...            value=2.0,
...            keyword="lag",
...            model="musk_classic",
...            parameterstep="2d")
>>> nmbsegments = hp.elements.stream_lahn_1_lahn_2.model.parameters.control.nmbsegments
>>> nmbsegments
nmbsegments(lag=0.583)
>>> rule.apply_value()
>>> nmbsegments
nmbsegments(lag=1.166)

apply_value() → None[source]¶: Apply the current (adapted) value to the relevant Parameter objects.

class hydpy.auxs.calibtools.CalibrationInterface(hp: HydPy, targetfunction: TargetFunction)[source]¶

Bases: Generic[TypeRule1]

Interface for the coupling of HydPy to optimisation libraries like NLopt.

Essentially, class CalibrationInterface is supposed for the structured handling of multiple objects of the different Rule subclasses. Hence, please read the documentation on class Rule before continuing, on which we base the following explanations.

We work with the Lahn example project again:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()

First, we create a CalibrationInterface object. Initially, it needs to know the relevant HydPy object and the target or objective function (here, we define the target function sloppily via the lambda statement; see the documentation on the protocol class TargetFunction for a more formal definition and further explanations):

>>> from hydpy import CalibrationInterface, nse
>>> ci = CalibrationInterface(
...     hp=hp,
...     targetfunction=lambda: sum(nse(node=node) for node in hp.nodes))

Next, we use function make_rules(), which creates one Replace rule related to parameter FC and another one related to parameter PercMax in one step, and add them via method add_rules():

>>> from hydpy import Replace
>>> from hydpy.auxs.calibtools import make_rules
>>> ci.add_rules(*make_rules(rule=Replace,
...                          names=["fc", "percmax"],
...                          parameters=["fc", "percmax"],
...                          values=[100.0, 5.0],
...                          keywords=[None, None],
...                          lowers=[50.0, 1.0],
...                          uppers=[200.0, 10.0],
...                          parametersteps="1d",
...                          model="hland_v1"))

>>> print(ci)
CalibrationInterface
>>> ci
Replace(
    name="fc",
    parameter="fc",
    value=100.0,
    lower=50.0,
    upper=200.0,
    keyword=None,
    parameterstep=None,
    model="hland_v1",
    selections=("complete",),
)
Replace(
    name="percmax",
    parameter="percmax",
    value=5.0,
    lower=1.0,
    upper=10.0,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("complete",),
)

Adding rules later does not remove already available ones. For demonstration, we add one for calibrating parameter Coefficients of application model musk_classic via its keyword damp:

>>> len(ci)
2
>>> ci.add_rules(Replace(name="damp",
...                      parameter="coefficients",
...                      value=0.2,
...                      lower=0.0,
...                      upper=0.5,
...                      keyword="damp",
...                      selections=["complete"],
...                      model="musk_classic"))
>>> len(ci)
3

All rules are available via attribute and keyword access:

>>> ci.fc
Replace(
    name="fc",
    parameter="fc",
    value=100.0,
    lower=50.0,
    upper=200.0,
    keyword=None,
    parameterstep=None,
    model="hland_v1",
    selections=("complete",),
)

>>> ci.FC
Traceback (most recent call last):
...
AttributeError: The actual calibration interface does neither handle a normal attribute nor a rule object named `FC`.

>>> ci["damp"]
Replace(
    name="damp",
    parameter="coefficients",
    value=0.2,
    lower=0.0,
    upper=0.5,
    keyword="damp",
    parameterstep=None,
    model="musk_classic",
    selections=("complete",),
)

>>> ci["Damp"]
Traceback (most recent call last):
...
KeyError: 'The actual calibration interface does not handle a rule object named `Damp`.'

The following properties return consistently sorted information on the handles Rule objects:

>>> ci.names
('fc', 'percmax', 'damp')
>>> ci.keywords
(None, None, 'damp')
>>> ci.values
(100.0, 5.0, 0.2)
>>> ci.lowers
(50.0, 1.0, 0.0)
>>> ci.uppers
(200.0, 10.0, 0.5)

All tuples reflect the current state of all rules:

>>> ci.damp.value = 0.3
>>> ci.values
(100.0, 5.0, 0.3)

For the following examples, we perform a simulation run and assign the values of the simulated time series to the observed series:

>>> conditions = hp.conditions
>>> hp.simulate()
>>> for node in hp.nodes:
...     node.sequences.obs.series = node.sequences.sim.series
>>> hp.conditions = conditions

As the agreement between the simulated and the “observed” time series is perfect for all four gauges, method calculate_likelihood() returns the highest possible sum of four nse() values and also stores it under the attribute result:

>>> from hydpy import round_
>>> round_(ci.calculate_likelihood())
4.0
>>> round_(ci.result)
4.0

When performing a manual calibration, it might be convenient to use method apply_values(). To explain how it works, we first show the values of the relevant parameters of some randomly selected model instances:

>>> stream = hp.elements.stream_lahn_1_lahn_2.model
>>> stream.parameters.control
nmbsegments(lag=0.583)
coefficients(damp=0.0)
>>> land = hp.elements.land_lahn_1.model
>>> land.parameters.control.fc
fc(206.0)
>>> land.parameters.control.percmax
percmax(1.02978)

Method apply_values() of class CalibrationInterface calls method apply_value() of all handled Rule objects, performs some preparations (for example, it derives the values of the secondary parameters), executes a simulation run, calls method calculate_likelihood(), and returns the result:

>>> result = ci.apply_values()
>>> stream.parameters.control
nmbsegments(lag=0.583)
coefficients(damp=0.3)
>>> land.parameters.control.fc
fc(100.0)
>>> land.parameters.control.percmax
percmax(5.0)

Due to the changes in our parameter values, our simulation is not “perfect” anymore:

>>> round_(ci.result)
1.605136

Use method reset_parameters() to restore the initial states of all affected parameters:

>>> ci.reset_parameters()
>>> stream.parameters.control
nmbsegments(lag=0.583)
coefficients(damp=0.0)
>>> land = hp.elements.land_lahn_1.model
>>> land.parameters.control.fc
fc(206.0)
>>> land.parameters.control.percmax
percmax(1.02978)

Now we get the same “perfect” efficiency again:

>>> hp.simulate()
>>> round_(ci.calculate_likelihood())
4.0
>>> hp.conditions = conditions

Note the perform_simulation argument of method apply_values(), which allows changing the model parameter values and updating the HydPy object only without triggering a simulation run (and to calculate and return a new likelihood value):

>>> ci.apply_values(perform_simulation=False)
>>> stream.parameters.control
nmbsegments(lag=0.583)
coefficients(damp=0.3)
>>> land.parameters.control.fc
fc(100.0)
>>> land.parameters.control.percmax
percmax(5.0)

Optimisers, like those implemented in NLopt, often provide their new parameter estimates via vectors. Method perform_calibrationstep() accepts such vectors and updates the handled Rule objects accordingly. After that, it performs the same steps as described for method apply_values():

>>> round_(ci.perform_calibrationstep([100.0, 5.0, 0.3]))
1.605136

>>> stream.parameters.control
nmbsegments(lag=0.583)
coefficients(damp=0.3)

>>> land.parameters.control.fc
fc(100.0)
>>> land.parameters.control.percmax
percmax(5.0)

Method perform_calibrationstep() writes intermediate results into a log file, if available. Prepare it beforehand via method prepare_logfile():

>>> with TestIO():
...     ci.prepare_logfile(logfilepath="example_calibration.log",
...                        objectivefunction="NSE",
...                        documentation="Just a doctest example.")

To continue “manually”, we now can call method update_logfile() to write the lastly calculated efficiency and the corresponding calibration parameter values to the log file:

>>> with TestIO():   
...     ci.update_logfile()
...     with open("example_calibration.log") as file_:
...         print(file_.read())
# Just a doctest example.

NSE           fc    percmax damp
parameterstep None  1d      None
1.605136      100.0 5.0     0.3

To prevent (automatic) calibration runs from crashing due to IO problems, method update_logfile() raises warnings instead of errors in such cases and logs the inwritten data internally:

>>> import os
>>> from hydpy.core.testtools import warn_later
>>> with TestIO(), warn_later():
...     ci._logfilepath = "dirname1/filename.log"
...     ci.update_logfile()
UserWarning: While trying to update the logfile `dirname1/filename.log`, the following problem occured: [Errno 2] No such file or directory: 'dirname1/filename.log'.

On subsequent calls, it tries to write both the previously logged and the new data:

>>> with TestIO():   
...     os.makedirs("dirname1", exist_ok=True)
...     ci.update_logfile()
...     with open("dirname1/filename.log") as file_:
...         print(file_.read())
1.605136 100.0 5.0 0.3
1.605136 100.0 5.0 0.3

Call method finalise_logfile() to ensure the CalibrationInterface object does not withhold data after the end of a calibration run. If you do so, it sleeps until it gets the chance to write the logged data and warns you about this problem from time to time (we demonstrate this by mocking the warn() function and, to keep our test example awake, the sleep() function):

>>> with TestIO():
...     ci._logfilepath = "dirname2/filename.log"
...     ci.update_logfile()
Traceback (most recent call last):
...
UserWarning: While trying to update the logfile `dirname2/filename.log`, the following problem occured: [Errno 2] No such file or directory: 'dirname2/filename.log'.
>>> from unittest import mock
>>> with TestIO():
...     with mock.patch("time.sleep") as mocked:
...         mocked.side_effect = Exception("time.sleep actually called")
...         ci.finalise_logfile()
Traceback (most recent call last):
...
UserWarning: Trying to finalise logfile `dirname2/filename.log` failed 1 times.
>>> with TestIO():
...     with mock.patch("warnings.warn"), mock.patch("time.sleep") as mocked:
...         mocked.side_effect = Exception("time.sleep actually called")
...         ci.finalise_logfile()
Traceback (most recent call last):
...
Exception: time.sleep actually called
>>> with TestIO():   
...     os.makedirs("dirname2", exist_ok=True)
...     ci.finalise_logfile()
...     with open("dirname2/filename.log") as file_:
...         print(file_.read())
1.605136 100.0 5.0 0.3

>>> ci._logfilepath = "example_calibration.log"

For automatic calibration, one needs a calibration algorithm like the following, which checks the lower and upper boundaries and the initial values of all Rule objects:

>>> def find_max(function, lowers, uppers, inits):
...     best_result = -999.0
...     best_parameters = None
...     for values in (lowers, uppers, inits):
...         result = function(values)
...         if result > best_result:
...             best_result = result
...             best_parameters = values
...     return best_parameters

Now we can assign method perform_calibrationstep() to this oversimplified optimiser, which then returns the best examined calibration parameter values:

>>> with TestIO():
...     find_max(function=ci.perform_calibrationstep,
...              lowers=ci.lowers,
...              uppers=ci.uppers,
...              inits=ci.values)
(200.0, 10.0, 0.5)

The log file now contains one line for our old result and three lines for the results of our optimiser:

>>> with TestIO():   
...     with open("example_calibration.log") as file_:
...         print(file_.read())
# Just a doctest example.

NSE           fc    percmax damp
parameterstep None  1d      None
1.605136      100.0 5.0     0.3
-0.710211     50.0  1.0     0.0
2.313934      200.0 10.0    0.5
1.605136      100.0 5.0     0.3

Class CalibrationInterface also provides method read_logfile(), which automatically selects the best calibration result. Therefore, it needs to know that the highest result is the best, which we indicate by setting argument maximisation to True:

>>> with TestIO():
...     ci.read_logfile(logfilepath="example_calibration.log", maximisation=True)
>>> ci.fc.value
200.0
>>> ci.percmax.value
10.0
>>> ci.damp.value
0.5
>>> round_(ci.result)
2.313934
>>> round_(ci.apply_values())
2.313934

On the contrary, if we set argument maximisation to False, method read_logfile() returns the worst result in our example:

>>> with TestIO():
...     ci.read_logfile(logfilepath="example_calibration.log", maximisation=False)
>>> ci.fc.value
50.0
>>> ci.percmax.value
1.0
>>> ci.damp.value
0.0
>>> round_(ci.result)
-0.710211
>>> round_(ci.apply_values())
-0.710211

To prevent errors due to different parameter step-sizes, method read_logfile() raises the following error whenever it detects inconsistencies:

>>> ci.percmax.parameterstep = "2d"
>>> with TestIO():
...     ci.read_logfile(logfilepath="example_calibration.log",maximisation=True)
Traceback (most recent call last):
...
RuntimeError: The current parameterstep of the `Replace` rule `percmax` (`2d`) does not agree with the one documentated in log file `example_calibration.log` (`1d`).

Method read_logfile() reports inconsistent rule names as follows:

>>> ci.remove_rules(ci.percmax)
>>> with TestIO():
...     ci.read_logfile(logfilepath="example_calibration.log",maximisation=True)
Traceback (most recent call last):
...
RuntimeError: The names of the rules handled by the actual calibration interface (damp and fc) do not agree with the names in the header of logfile `example_calibration.log` (damp, fc, and percmax).

The last consistency check is optional. Set argument check to False to force method read_logfile() to query all available data instead of raising an error:

>>> ci.add_rules(Replace(name="beta",
...                      parameter="beta",
...                      value=2.0,
...                      lower=1.0,
...                      upper=4.0,
...                      selections=["complete"],
...                      model="hland_v1"))
>>> ci.fc.value = 0.0
>>> ci.damp.value = 0.0
>>> with TestIO():
...     ci.read_logfile(
...         logfilepath="example_calibration.log",
...         maximisation=True,
...         check=False,
...     )
>>> ci.beta.value
2.0
>>> ci.fc.value
200.0
>>> ci.damp.value
0.5

conditions: Dict[str, Dict[str, Dict[str, float | ndarray[Any, dtype[float64]]]]]¶

The conditions of the given HydPy object.

CalibrationInterface queries the conditions during its initialisation and uses them later to reset all relevant conditions before each new simulation run.

result: float | None¶: The last result, as calculated by the target function.

add_rules(*rules: TypeRule1) → None[source]¶

Add some Rule objects to the actual CalibrationInterface object.

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import CalibrationInterface
>>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: None)
>>> from hydpy import Replace
>>> ci.add_rules(Replace(name="fc",
...                      parameter="fc",
...                      value=100.0,
...                      model="hland_v1"),
...              Replace(name="percmax",
...                      parameter="percmax",
...                      value=5.0,
...                      model="hland_v1"))

Note that method add_rules() might change the number of Element objects relevant for the CalibrationInterface object:

>>> damp = Replace(name="damp",
...                parameter="coefficients",
...                value=0.2,
...                keyword="damp",
...                model="musk_classic")
>>> len(ci._elements)
4
>>> ci.add_rules(damp)
>>> len(ci._elements)
7

get_rule(name: str, type_: Type[TypeRule2] | None = None) → TypeRule1 | TypeRule2[source]¶

Return a Rule object (of a specific type).

Method get_rule() is a more typesafe alternative to simple keyword access. Besides the name of the required Rule object, pass its subclass to convince your IDE (and yourself) that the returned rule follows this more specific type:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import Add, CalibrationInterface, make_rules, nse, Replace
>>> ci = CalibrationInterface(
...     hp=hp,
...     targetfunction=lambda: sum(nse(node=node) for node in hp.nodes))
>>> ci.add_rules(*make_rules(rule=Replace,
...                          names=["fc", "percmax"],
...                          parameters=["fc", "percmax"],
...                          values=[100.0, 5.0],
...                          keywords=["forest", None],
...                          lowers=[50.0, 1.0],
...                          uppers=[200.0, 10.0],
...                          parametersteps="1d",
...                          model="hland_v1"))

>>> ci.get_rule("fc", Replace).name
'fc'

>>> ci.get_rule("Fc", Replace).name
Traceback (most recent call last):
...
RuntimeError: The actual calibration interface does not handle a rule object named `Fc`.

>>> ci.get_rule("fc", Replace).name
'fc'

>>> ci.get_rule("fc", Add).name
Traceback (most recent call last):
...
RuntimeError: The actual calibration interface does not handle a rule object named `fc` of type `Add`.

remove_rules(*rules: str | TypeRule1) → None[source]¶

Remove some Rule objects from the actual CalibrationInterface object.

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import CalibrationInterface
>>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: None)
>>> from hydpy import Replace
>>> ci.add_rules(Replace(name="fc",
...                      parameter="fc",
...                      value=100.0,
...                      model="hland_v1"),
...              Replace(name="percmax",
...                      parameter="percmax",
...                      value=5.0,
...                      model="hland_v1"),
...              Replace(name="damp",
...                      parameter="coefficients",
...                      value=0.2,
...                      keyword="damp",
...                      model="musk_classic"))

You can remove each rule either by passing itself or its name (note that method remove_rules() might change the number of Element objects relevant for the CalibrationInterface object):

>>> len(ci._elements)
7
>>> fc = ci.fc
>>> fc in ci
True
>>> "damp" in ci
True
>>> ci.remove_rules(fc, "damp")
>>> fc in ci
False
>>> "damp" in ci
False
>>> len(ci._elements)
4

Trying to remove a non-existing rule results in the following error:

>>> ci.remove_rules("fc")
Traceback (most recent call last):
...
RuntimeError: The actual calibration interface object does not handle a rule object named `fc`.

prepare_logfile(logfilepath: str, objectivefunction: str = 'result', documentation: str | None = None) → None[source]¶

Prepare a log file.

Use argument objectivefunction to describe the TargetFunction used for calculating the efficiency and argument documentation to add some information to the header of the logfile.

See the main documentation on class CalibrationInterface for further information.

update_logfile() → None[source]¶

Update the current log file, if available.

See the main documentation on class CalibrationInterface for further information.

finalise_logfile() → None[source]¶

Update the current log file if method update_logfile() was not entirely successful in doing so.

See the main documentation on class CalibrationInterface for further information.

read_logfile(logfilepath: str, maximisation: bool, check: bool = True) → None[source]¶

Read the log file with the given file path.

See the main documentation on class CalibrationInterface for further information.

property names: Tuple[str, ...]¶

The names of all handled Rule objects.

See the main documentation on class CalibrationInterface for further information.

property values: Tuple[float, ...]¶

The values of all handled Rule objects.

See the main documentation on class CalibrationInterface for further information.

property keywords: Tuple[str | None, ...]¶

The (optional) target keywords of all handled Rule objects.

See the main documentation on class CalibrationInterface for further information.

property lowers: Tuple[float, ...]¶

The lower boundaries of all handled Rule objects.

See the main documentation on class CalibrationInterface for further information.

property uppers: Tuple[float, ...]¶

The upper boundaries of all handled Rule objects.

See the main documentation on class CalibrationInterface for further information.

property selections: Tuple[str, ...]¶

The names of all Selection objects addressed at least one of the handled Rule objects.

See the documentation on function make_rules() for further information.

property parametertypes: Tuple[Tuple[Type[Parameter], str | None], ...]¶

The types of all Parameter objects addressed by at least one of the handled Rule objects.

See the documentation on function make_rules() for further information.

apply_values(perform_simulation: bool = True) → float | None[source]¶

Apply all current calibration parameter values on all relevant parameters.

Set argument perform_simulation to False to only change the actual parameter values and update the HydPy object without performing a simulation run.

See the main documentation on class CalibrationInterface for further information.

reset_parameters() → None[source]¶

Reset all relevant parameters to their original states.

See the main documentation on class CalibrationInterface for further: information.

calculate_likelihood() → float[source]¶

Apply the defined TargetFunction and return the result.

See the main documentation on class CalibrationInterface for further information.

perform_calibrationstep(values: Iterable[float], *args: Any, **kwargs: Any) → float[source]¶

Update all calibration parameters with the given values, update the HydPy object, perform a simulation run, and calculate and return the achieved efficiency.

See the main documentation on class CalibrationInterface for further information.

Print the current calibration parameter values in a table format.

The following examples combine the base examples of the documentation on class CalibrationInterface and class ReplaceIUH, so please make sure to understand them before proceeding.

We again use the Lahn example project but replace the musk_classic model instances with those of application model arma_v1, which allows discussing some special cases concerning the handling of RuleIUH:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import prepare_model
>>> for element in hp.elements.river:
...     element.model = prepare_model("arma_v1")
...     element.model.parameters.control.responses([[], [1.0]])
...     element.model.parameters.update()

We pass a (useless) dummy target function to the CalibrationInterface object:

>>> from hydpy import CalibrationInterface
>>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: 1.0)

Regarding hland_v1, we intend to calibrate the parameters FC and PercMax with different values for the selections headwaters and nonheadwaters:

>>> from hydpy import CalibSpec, CalibSpecs, make_rules, Replace
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="fc", default=100.0, lower=50.0, upper=200.0),
...     CalibSpec(name="percmax", default=5.0, lower=1.0, upper=10.0, parameterstep="1d"))
>>> ci.add_rules(*make_rules(rule=Replace,
...                          calibspecs=calibspecs,
...                          model="hland_v1",
...                          selections=("headwaters", "nonheadwaters"),
...                          product=True))

Regarding arma_v1, we cannot calibrate the values of parameter Responses in a meaningful way. So instead, we use the LinearStorageCascade as a meta-model and calibrate its parameters k and n:

>>> from hydpy import LinearStorageCascade, ReplaceIUH
>>> k = ReplaceIUH(name="k_global",
...                target="k",
...                parameter="responses",
...                value=2.0,
...                lower=1.0,
...                parameterstep="1d",
...                selections=("streams",))
>>> n = ReplaceIUH(name="n_global",
...                target="n",
...                parameter="responses",
...                value=4.0,
...                lower=1.0,
...                upper=100.0,
...                selections=("streams",))
>>> name2lsc = {element.name: LinearStorageCascade(k=1.0, n=1.0)
...             for element in hp.elements.river}
>>> k.add_iuhs(**name2lsc)
>>> n.add_iuhs(**name2lsc)
>>> ci.add_rules(k, n)

We change the values of two Rule objects related to hland_v1 to clarify that all values appear in the correct table cells:

>>> ci["fc_headwaters"].value = 200.0
>>> ci["percmax_nonheadwaters"].value = 10.0

By default, method print_table() prints the values of all handled Rule objects. It varies the target control parameters on the first axis and the target selections on the second axis. Row two and three contain the (identical) lower and upper boundary values corresponding to the respective control parameters:

>>> ci.print_table()  
              lower  upper  headwaters  nonheadwaters  streams
k->Responses  1.0   inf     /           /              2.0
n->Responses  1.0   100.0   /           /              4.0
FC            50.0   200.0  200.0       100.0          /
PercMax       1.0    10.0   5.0         10.0           /

For non-identical boundary values, method print_table() prints fill values in the relevant cells. Besides this, the following example shows how to define alternative titles for the boundary value columns:

>>> ci["fc_headwaters"].lower = 60.0
>>> ci["percmax_nonheadwaters"].upper = 20.0
>>> ci.print_table(bounds=("min", "max"))  
              min    max    headwaters  nonheadwaters  streams
k->Responses  1.0    inf    /          /               2.0
n->Responses  1.0    100.0  /          /               4.0
FC              /    200.0  200.0      100.0           /
PercMax       1.0    /      5.0        10.0            /

Pass None to argument bounds to omit writing any boundary value column:

>>> ci.print_table(bounds=None)  
              headwaters  nonheadwaters  streams
k->Responses  /           /              2.0
n->Responses  /           /              4.0
FC            200.0       100.0          /
PercMax       5.0         10.0           /

The next example shows how to change the tabulated target parameters and selections. Method print_table() uses the (given alternative) fill value for each parameter-selection-combination not met by any of the available Rule objects. For RuleIUH-related parameters, we must specify both the control parameter (as a type, in our example Responses) and the meta-parameter (as a string, in our example k) within a tuple:

>>> from hydpy.models.hland.hland_control import CFlux, PercMax
>>> from hydpy.models.arma.arma_control import Responses
>>> ci.print_table(  
...     parametertypes=(PercMax, CFlux, (Responses, "k")),
...     selections=("streams", "headwaters"),
...     bounds=None,
...     fillvalue="-")
              streams  headwaters
PercMax       -        5.0
CFlux         -        -
k->Responses  2.0      -

Note that the value of the same calibration parameter might appear multiple times when targeting multiple Selection objects:

>>> ci["fc_headwaters"].selections = ("headwaters", "streams")
>>> ci.print_table(bounds=None)  
              headwaters  nonheadwaters  streams
k->Responses  /           /              2.0
n->Responses  /           /              4.0
FC            200.0       100.0          200.0
PercMax       5.0             10.0               /

class hydpy.auxs.calibtools.RuleIUH(*, name: str, target: str, parameter: Type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶

Bases: Rule[arma_control.Responses]

A Rule, class specialised for IUH parameters.

RuleIUH serves as a base class only. Please see the concrete implementation ReplaceIUH for further information.

update_parameters: bool = True¶

Flag indicating whether method apply_value() should calculate the coefs and pass them to the relevant model parameter or not.

Set this flag to False for the first ReplaceIUH object when another handles the same elements and is applied afterwards.

target: str¶: Name of the addressed property of the relevant IUH subclass.

add_iuhs(**iuhs: IUH) → None[source]¶

Add one IUH object for each relevant Element object.

See the main documentation on class ReplaceIUH for further information.

reset_parameters() → None[source]¶

Reset all relevant parameter objects to their original states.

See the main documentation on class ReplaceIUH for further information.

class hydpy.auxs.calibtools.ReplaceIUH(*, name: str, target: str, parameter: Type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶

Bases: RuleIUH

A RuleIUH class for replacing IUH parameter values with the current calibration parameter values.

Usually, it is not a good idea to calibrate the AR and MA coefficients of parameters like Responses of model arma_v1 individually. Instead, we need to calibrate the few coefficients of the underlying IUH objects, which calculate the ARMA coefficients. Class ReplaceIUH helps to accomplish this task.

Note

Class ReplaceIUH is still under development. For example, it does not address the possibility of different ARMA coefficients related to different discharge thresholds. Hence, the usage of class ReplaceIUH might change in the future.

So far, there is no example project containing arma_v1 models instances. Therefore, we generate a simple one consisting of two Element objects only:

>>> from hydpy import Element, prepare_model, Selection
>>> element1 = Element("element1", inlets="in1", outlets="out1")
>>> element2 = Element("element2", inlets="in2", outlets="out2")
>>> complete = Selection("complete", elements=[element1, element2])
>>> element1.model = prepare_model("arma_v1")
>>> element2.model = prepare_model("arma_v1")

We focus on class TranslationDiffusionEquation in the following. First, we create two separate instances and use them to calculate the response coefficients of both arma_v1 instances:

>>> from hydpy import TranslationDiffusionEquation
>>> tde1 = TranslationDiffusionEquation(u=5.0, d=15.0, x=1.0)
>>> tde2 = TranslationDiffusionEquation(u=5.0, d=15.0, x=2.0)
>>> element1.model.parameters.control.responses(tde1.arma.coefs)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276),
                  (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0,
                   -0.000001, 0.0, 0.0, 0.0, 0.0)))
>>> element2.model.parameters.control.responses(tde2.arma.coefs)
>>> element2.model.parameters.control.responses
responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004),
                  (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))

Next, we define one ReplaceIUH for modifying parameter u and another one for changing d:

>>> from hydpy import ReplaceIUH
>>> u = ReplaceIUH(name="U",
...                target="u",
...                parameter="responses",
...                value=5.0,
...                lower=1.0,
...                upper=10.0,
...                selections=[complete])
>>> d = ReplaceIUH(name="D",
...                target="d",
...                parameter="responses",
...                value=15.0,
...                lower=5.0,
...                upper=50.0,
...                selections=[complete])

We add and thereby connect the Element and TranslationDiffusionEquation objects to both ReplaceIUH objects via method add_iuhs():

>>> u.add_iuhs(element1=tde1, element2=tde2)
>>> d.add_iuhs(element1=tde1, element2=tde2)

Note that method add_iuhs() enforces to add all IUH objects at ones to avoid inconsistencies that might be hard to track later:

>>> d.add_iuhs(element1=tde1)
Traceback (most recent call last):
...
RuntimeError: While trying to add `IUH` objects to the `ReplaceIUH` rule `D`, the following error occurred: The given elements (element1) do not agree with the complete set of relevant elements (element1 and element2).

By default, each ReplaceIUH object triggers the calculation of the ARMA coefficients during the execution of its method apply_value(), which can be a waste of computation time if we want to calibrate multiple IUH coefficients. To save computation time in such cases, set option update_parameters to False for all except the lastly executed ReplaceIUH objects:

>>> u.update_parameters = False

Now, changing the value of rule U and calling method apply_value() does not affect the coefficients of both Responses parameters:

>>> u.value = 10.0
>>> u.apply_value()
>>> tde1
TranslationDiffusionEquation(d=15.0, u=10.0, x=1.0)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276),
                  (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0,
                   -0.000001, 0.0, 0.0, 0.0, 0.0)))
>>> tde2
TranslationDiffusionEquation(d=15.0, u=10.0, x=2.0)
>>> element2.model.parameters.control.responses
responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004),
                  (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))

On the other side, calling method apply_value() of rule D does activate the freshly set value of rule D and the previously set value of rule U, as well:

>>> d.value = 50.0
>>> d.apply_value()
>>> tde1
TranslationDiffusionEquation(d=50.0, u=10.0, x=1.0)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.811473, -0.15234, -0.000256, 0.000177),
                  (0.916619, -0.670781, 0.087185, 0.007923)))
>>> tde2
TranslationDiffusionEquation(d=50.0, u=10.0, x=2.0)
>>> element2.model.parameters.control.responses
responses(th_0_0=((0.832237, -0.167205, 0.002007, 0.000184),
                  (0.836513, -0.555399, 0.037628, 0.014035)))

Use method reset_parameters() to restore the original ARMA coefficients:

>>> d.reset_parameters()
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276),
                  (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0,
                   -0.000001, 0.0, 0.0, 0.0, 0.0)))
>>> element2.model.parameters.control.responses
responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004),
                  (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))

apply_value() → None[source]¶

Apply all current calibration parameter values to all relevant IUH objects and eventually update the related parameter’s ARMA coefficients.

See the main documentation on class ReplaceIUH for further information.

class hydpy.auxs.calibtools.MultiplyIUH(*, name: str, target: str, parameter: Type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶

Bases: RuleIUH

A RuleIUH class for replacing IUH parameter values with the current calibration parameter values, applied on the original IUH values as factors.

Please read the documentation on class ReplaceIUH first, from which we take the following test configuration:

>>> from hydpy import Element, prepare_model, Selection
>>> element1 = Element("element1", inlets="in1", outlets="out1")
>>> element2 = Element("element2", inlets="in2", outlets="out2")
>>> complete = Selection("complete", elements=[element1, element2])
>>> element1.model = prepare_model("arma_v1")
>>> element2.model = prepare_model("arma_v1")

>>> from hydpy import TranslationDiffusionEquation
>>> tde1 = TranslationDiffusionEquation(u=5.0, d=15.0, x=1.0)
>>> tde2 = TranslationDiffusionEquation(u=5.0, d=15.0, x=2.0)
>>> element1.model.parameters.control.responses(tde1.arma.coefs)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276),
                  (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0,
                   -0.000001, 0.0, 0.0, 0.0, 0.0)))
>>> element2.model.parameters.control.responses(tde2.arma.coefs)
>>> element2.model.parameters.control.responses
responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004),
                  (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))

Initialising MultiplyIUH works exactly as for ReplaceIUH, except for the semantic difference that value, lower, and upper now represent factors:

>>> from hydpy import MultiplyIUH
>>> u = MultiplyIUH(name="U",
...                 target="u",
...                 parameter="responses",
...                 value=2.0,
...                 lower=1.0,
...                 upper=4.0,
...                 selections=[complete])
>>> d = MultiplyIUH(name="D",
...                 target="d",
...                 parameter="responses",
...                 value=0.5,
...                 lower=0.2,
...                 upper=2.0,
...                 selections=[complete])

>>> u.add_iuhs(element1=tde1, element2=tde2)
>>> d.add_iuhs(element1=tde1, element2=tde2)
>>> u.update_parameters = False

The following examples demonstrate that the current calibration values actually as factors, applied to the original values of the relevant IUH properties:

>>> u.value = 3.0
>>> u.apply_value()
>>> d.value = 1.0/3.0
>>> d.apply_value()
>>> tde1
TranslationDiffusionEquation(d=5.0, u=15.0, x=1.0)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.0, 0.0),
                  (0.933333, 0.066667)))
>>> tde2
TranslationDiffusionEquation(d=5.0, u=15.0, x=2.0)
>>> element2.model.parameters.control.responses
responses(th_0_0=((0.0, 0.0),
                  (0.866667, 0.133333)))

>>> u.value = 1.0
>>> u.apply_value()
>>> d.value = 1.0
>>> d.apply_value()
>>> tde1
TranslationDiffusionEquation(d=15.0, u=5.0, x=1.0)
>>> element1.model.parameters.control.responses
responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276),
                  (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0,
                   -0.000001, 0.0, 0.0, 0.0, 0.0)))
>>> tde2
TranslationDiffusionEquation(d=15.0, u=5.0, x=2.0)
>>> element2.model.parameters.control.responses
responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004),
                  (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))

add_iuhs(**iuhs: IUH) → None[source]¶

Add one IUH object for each relevant Element object.

See the main documentation on class ReplaceIUH for further information.

apply_value() → None[source]¶

Apply all current calibration parameter values to all relevant IUH objects and eventually update the related parameter’s ARMA coefficients.

See the main documentation on class MultiplyIUH for further information.

class hydpy.auxs.calibtools.CalibSpec(*, name: str, default: float, keyword: None | None = None, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None)[source]¶

Bases: object

Helper class for specifying the properties of a single calibration parameter.

So far, class CalibSpec does not provide much functionality besides checking upon initialisation that the given default and boundary values are consistent:

>>> from hydpy import CalibSpec
>>> CalibSpec(name="par1", default=1.0)
CalibSpec(name="par1", default=1.0)

>>> CalibSpec(name="par1", default=1.0, keyword="key1")
CalibSpec(name="par1", default=1.0, keyword="key1")

>>> CalibSpec(name="par1", default=1.0, lower=2.0)
Traceback (most recent call last):
...
ValueError: The following values given for calibration parameter `par1` are not consistent: default=1.0, lower=2.0, upper=inf.

>>> CalibSpec(name="par1", default=1.0, upper=0.5)
Traceback (most recent call last):
...
ValueError: The following values given for calibration parameter `par1` are not consistent: default=1.0, lower=-inf, upper=0.5.

>>> CalibSpec(name="par1", default=1.0, lower=0.0, upper=2.0)
CalibSpec(name="par1", default=1.0, lower=0.0, upper=2.0)

Use the parameterstep argument for time-dependent calibration parameters:

>>> CalibSpec(name="par1", default=1.0/3.0, lower=1.0/3.0, upper=1.0/3.0,
...           parameterstep="1d")
CalibSpec(
    name="par1", default=0.333333, lower=0.333333, upper=0.333333, parameterstep="1d"
)

See the documentation on class CalibSpecs for further information.

name: str¶: Name of the calibration parameter.

default: float¶: The default value of the calibration parameter.

keyword: str | None¶: The (optional) target keyword of the calibration parameter.

lower: float¶: Lower bound of the allowed calibration parameter value.

upper: float¶: Upper bound of the allowed calibration parameter value.

parameterstep: Period | None¶: The parameter step size to be set before applying the defined calibration parameter values.

class hydpy.auxs.calibtools.CalibSpecs(*parspecs: CalibSpec)[source]¶

Bases: object

Collection class for handling CalibSpec objects.

The primary purpose of class CalibSpecs is to handle multiple CalibSpec objects and to make all their attributes accessible in the same order. See property names as one example. Note that all such properties are sorted in the order or the attachment of the different CalibSpec objects:

>>> from hydpy import CalibSpec, CalibSpecs
>>> calibspecs = CalibSpecs(
...     CalibSpec(
...         name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d"
...     ),
...     CalibSpec(name="second", default=1.0, keyword="kw2", lower=0.0),
...     CalibSpec(name="first",default=2.0, upper=2.0))
>>> calibspecs
CalibSpecs(
    CalibSpec(name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d"),
    CalibSpec(name="second", default=1.0, keyword="kw2", lower=0.0),
    CalibSpec(name="first", default=2.0, upper=2.0),
)

You can query and remove CalibSpec objects via keyword and attribute access:

>>> print(calibspecs)
CalibSpecs("third", "second", "first")

>>> third = calibspecs["third"]
>>> third in calibspecs
True
>>> del calibspecs["third"]
>>> third in calibspecs
False
>>> calibspecs["third"]
Traceback (most recent call last):
...
KeyError: 'The current `CalibSpecs` object does not handle a `CalibSpec` object named `third`.'
>>> del calibspecs["third"]
Traceback (most recent call last):
...
KeyError: 'The current `CalibSpecs` object does not handle a `CalibSpec` object named `third`.'

>>> second = calibspecs.second
>>> "second" in calibspecs
True
>>> del calibspecs.second
>>> "second" in calibspecs
False
>>> calibspecs.second
Traceback (most recent call last):
...
AttributeError: The current `CalibSpecs` object does neither handle a `CalibSpec` object nor a normal attribute named `second`.
>>> del calibspecs.second
Traceback (most recent call last):
...
AttributeError: The current `CalibSpecs` object does not handle a `CalibSpec` object named `second`.

>>> len(calibspecs)
1

Now we can re-append the previously removed CalibSpec objects (and thereby bring the order of attachment in agreement with the CalibSpec names):

>>> calibspecs.append(second, third)
>>> for calibspec in calibspecs:
...     print(calibspec)
first
second
third

append(*calibspecs: CalibSpec) → None[source]¶

Append one or more CalibSpec objects.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> first = CalibSpec(name="first", default=1.0, lower=0.0)
>>> second = CalibSpec(name="second",default=2.0, keyword="kw2", upper=2.0)
>>> calibspecs = CalibSpecs()
>>> calibspecs.append(first)
>>> calibspecs.append(second, third)
>>> calibspecs
CalibSpecs(
    CalibSpec(name="first", default=1.0, lower=0.0),
    CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0),
    CalibSpec(name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d"),
)

property names: Tuple[str, ...]¶

The names of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(CalibSpec(name="first", default=1.0, lower=0.0),
...                         CalibSpec(name="second",default=2.0, upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.names
('first', 'second', 'third')

property defaults: Tuple[float, ...]¶

The default values of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="first", default=1.0, lower=0.0),
...     CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.defaults
(1.0, 2.0, 3.0)

property keywords: Tuple[str | None, ...]¶

The (optional) target keywords of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="first", default=1.0, lower=0.0),
...     CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.keywords
(None, 'kw2', None)

property lowers: Tuple[float, ...]¶

The lower boundary values of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="first", default=1.0, lower=0.0),
...     CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.lowers
(0.0, -inf, -10.0)

property uppers: Tuple[float, ...]¶

The upper boundary values of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="first", default=1.0, lower=0.0),
...     CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.uppers
(inf, 2.0, 10.0)

property parametersteps: Tuple[Period | None, ...]¶

The parameter steps of all CalibSpec objects in the order of attachment.

>>> from hydpy import CalibSpec, CalibSpecs
>>> third = CalibSpec(
...     name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d")
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="first", default=1.0, lower=0.0),
...     CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0))
>>> calibspecs.append(third)
>>> calibspecs.parametersteps
(None, None, Period("1d"))

Conveniently create multiple Rule objects at once.

Please see the main documentation on class CalibrationInterface first, from which we borrow the general setup:

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()
>>> from hydpy import CalibrationInterface, make_rules, nse
>>> ci = CalibrationInterface(
...     hp=hp,
...     targetfunction=lambda: sum(nse(node=node) for node in hp.nodes))

Here, we show only the supplemental features of function make_rules() in some brevity.

Function make_rules() checks that all given sequences have the same length:

>>> from hydpy import Replace
>>> make_rules(rule=Replace,
...            names=["fc", "percmax"],
...            parameters=["fc", "percmax"],
...            values=[100.0, 5.0],
...            keywords=["forest", None],
...            lowers=[50.0, 1.0],
...            uppers=[200.0],
...            parametersteps="1d",
...            model="hland_v1")
Traceback (most recent call last):
...
ValueError: When creating rules via function `make_rules`, all given sequences must be of equal length.

The separate handling of the specifications of all calibration parameters is error-prone. You can bundle all specifications within a CalibSpecs object instead and pass them at once for more safety and convenience:

>>> from hydpy import CalibSpec, CalibSpecs
>>> calibspecs = CalibSpecs(
...     CalibSpec(name="fc", default=100.0, keyword="forest", lower=50.0, upper=200.0),
...     CalibSpec(name="percmax", default=5.0, lower=1.0, upper=10.0, parameterstep="1d"))
>>> make_rules(rule=Replace,
...            calibspecs=calibspecs,
...            parametersteps="1d",
...            model="hland_v1")[1]
Replace(
    name="percmax",
    parameter="percmax",
    value=5.0,
    lower=1.0,
    upper=10.0,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("complete",),
)

You are free also to use the individual arguments (e.g. names) to override the related specifications defined by the CalibSpecs object:

>>> make_rules(rule=Replace,
...            calibspecs=calibspecs,
...            names=[name.upper() for name in calibspecs.names],
...            parametersteps="1d",
...            model="hland_v1")[1]
Replace(
    name="PERCMAX",
    parameter="percmax",
    value=5.0,
    lower=1.0,
    upper=10.0,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("complete",),
)

Function make_rules() raises the following error if you neither pass a CalibSpecs object nor the complete list of individual calibration parameter specifications:

>>> make_rules(rule=Replace,
...            names=["fc", "percmax"],
...            parameters=["fc", "percmax"],
...            values=[100.0, 5.0],
...            keywords=["forest", None],
...            lowers=[50.0, 1.0],
...            parametersteps="1d",
...            model="hland_v1")
Traceback (most recent call last):
...
TypeError: When creating rules via function `make_rules`, you must pass a `CalibSpecs` object or provide complete information for the following arguments: names, parameters, values, keywords, lowers, and uppers.

You can run function make_rules() in “product mode”, meaning that its execution results in distinct Rule objects for all combinations of the given calibration parameters and selections:

>>> make_rules(rule=Replace,
...            calibspecs=calibspecs,
...            model="hland_v1",
...            selections=("headwaters", "nonheadwaters"),
...            product=True)
[Replace(
    name="fc_headwaters",
    parameter="fc",
    value=100.0,
    lower=50.0,
    upper=200.0,
    keyword="forest",
    parameterstep=None,
    model="hland_v1",
    selections=("headwaters",),
), Replace(
    name="percmax_headwaters",
    parameter="percmax",
    value=5.0,
    lower=1.0,
    upper=10.0,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("headwaters",),
), Replace(
    name="fc_nonheadwaters",
    parameter="fc",
    value=100.0,
    lower=50.0,
    upper=200.0,
    keyword="forest",
    parameterstep=None,
    model="hland_v1",
    selections=("nonheadwaters",),
), Replace(
    name="percmax_nonheadwaters",
    parameter="percmax",
    value=5.0,
    lower=1.0,
    upper=10.0,
    keyword=None,
    parameterstep="1d",
    model="hland_v1",
    selections=("nonheadwaters",),
)]

Trying to run in “product mode” without defining the target selections results in the following error message:

>>> make_rules(rule=Replace,
...            calibspecs=calibspecs,
...            parametersteps="1d",
...            model="hland_v1",
...            product=True)
Traceback (most recent call last):
...
TypeError: When creating rules via function `make_rules` in "product mode" (with the argument `product` being `True`), you must supply all target selection objects via argument `selections`.

Table of Contents

Previous topic

Next topic

This Page

Versions

calibtools¶