calibtools¶
This module implements features for calibrating model parameters.
Module calibtools implements the following members:
TypeParameterType variable.
TypeRuleType variable.
TypeRule1Type variable.
TypeRule2Type variable.
TargetFunctionProtocol class for the target function required by classCalibrationInterface.
AdaptorProtocol class for defining adoptors required byReplaceobjects.
SumAdaptorAdaptor, which calculates the sum of the values of multipleRuleobjects and assigns it to the value(s) of the targetParameterobject.
FactorAdaptorAdaptor, which calculates the product of the value of the parentReplaceobject and the value(s) of a given referenceParameterobject and assigns it to the value(s) of the targetParameterobject.
RuleBase class for defining calibration rules.
ReplaceRuleclass, which simply replaces the current model parameter value(s) with the current calibration parameter value.
AddRuleclass, which adds its calibration delta to the original model parameter value(s).
MultiplyRuleclass for multiplying the original model parameter value(s) by its calibration factor.
CalibrationInterfaceInterface for the coupling of HydPy to optimisation libraries like NLopt.
ReplaceIUHARuleIUHclass for replacingIUHparameter values with the current calibration parameter values.
MultiplyIUHARuleIUHclass for replacingIUHparameter values with the current calibration parameter values, applied on the originalIUHvalues as factors.
CalibSpecHelper class for specifying the properties of a single calibration parameter.
CalibSpecsCollection class for handlingCalibSpecobjects.
make_rules()Conveniently create multipleRuleobjects at once.
- class hydpy.auxs.calibtools.TargetFunction(*args, **kwargs)[source]¶
Bases:
ProtocolProtocol class for the target function required by class
CalibrationInterface.The target functions must calculate and return a floating-point number reflecting the quality of the current parameterisation of the models of the current project. Often, as in the following example, the target function relies on objective functions as
nse(), applied on the time series of theSimandObssequences handled by theHydPyobject:>>> from hydpy import HydPy, nse, TargetFunction >>> class Target(TargetFunction): ... def __init__(self, hp): ... self.hp = hp ... def __call__(self): ... return sum(nse(node=node) for node in self.hp.nodes) >>> target = Target(HydPy())
See the documentation on class
CalibrationInterfacefor more information.
- class hydpy.auxs.calibtools.Adaptor(*args, **kwargs)[source]¶
Bases:
ProtocolProtocol class for defining adoptors required by
Replaceobjects.Often, one calibration parameter (represented by one
Replaceobject) depends on other calibration parameters (represented by otherReplaceobjects) or other “real” parameter values. Please select an existing or define a new adaptor and assign it to aReplaceobject to introduce such dependencies.See class
SumAdaptoror classFactorAdaptorfor concrete examples.
- class hydpy.auxs.calibtools.SumAdaptor(*rules: Rule[Parameter])[source]¶
Bases:
AdaptorAdaptor, which calculates the sum of the values of multiple
Ruleobjects and assigns it to the value(s) of the targetParameterobject.Class
SumAdaptorhelps to introduce “larger than” relationships between calibration parameters. A common use case is the time of concentration of different runoff components. For example, the time of concentration of base flow should be larger than the one of direct runoff. Accordingly, when modelling runoff concentration with linear storages, the recession coefficient of direct runoff should be larger. Principally, we could ensure this during a calibration process by defining twoRuleobjects with fixed non-overlapping parameter ranges. For example, we could search for the best direct runoff delay between 1 and 5 days and the base flow delay between 5 and 100 days. We demonstrate this for the recession coefficient parametersKandK4of application modelhland_96(assuming the nonlinearity parameterAlphato be zero):>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import Replace, SumAdaptor >>> k = Replace(name="k", ... parameter="k", ... value=2.0**-1, ... lower=5.0**-1, ... upper=1.0**-1, ... parameterstep="1d", ... model="hland_96") >>> k4 = Replace(name="k4", ... parameter="k4", ... value=10.0**-1, ... lower=100.0**-1, ... upper=5.0**-1, ... parameterstep="1d", ... model="hland_96")
To allow for non-fixed non-overlapping ranges, we can prepare a
SumAdaptorobject, knowing both ourRuleobjects, assign it the direct runoff-relatedRuleobject, and, for example, set its lower boundary to zero:>>> k.adaptor = SumAdaptor(k, k4) >>> k.lower = 0.0
Calling method
apply_value()of theReplaceobjects makes ourSumAdaptorobject apply the sum of the values of all of itsRuleobjects:>>> control = hp.elements.land_dill_assl.model.parameters.control >>> k.apply_value() >>> with pub.options.parameterstep("1d"): ... control.k k(0.6)
- class hydpy.auxs.calibtools.FactorAdaptor(rule: Rule[Parameter], reference: type[Parameter] | Parameter | str, mask: BaseMask | str | None = None)[source]¶
Bases:
AdaptorAdaptor, which calculates the product of the value of the parent
Replaceobject and the value(s) of a given referenceParameterobject and assigns it to the value(s) of the targetParameterobject.Class
FactorAdaptorhelps to respect dependencies between model parameters. If you, for example, aim at calibrating the permanent wilting point (PWP) of modellland_dd, you need to make sure it always agrees with the maximum soil water storage (WMax). Especially, one should avoid permanent wilting points larger than total porosity. Due to the high variability of soil properties within most catchments, it is no real option to define a fixed upper threshold forPWP. By using classFactorAdaptor, you can instead calibrate a multiplication factor. Setting the bounds of such a factor to 0.0 and 0.5, for example, would result inPWPvalues ranging from zero up to half ofWMaxfor each respective response unit.To show how class
FactorAdaptorworks, we select another use-case based on the Lahn example project prepared by functionprepare_full_example_2():>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2()
hland_96calculates the “normal” potential snow-melt with the degree-day factorCFMax. For glacial zones, it also calculates a separate potential glacier-melt with the additional degree-day factorGMelt. Suppose we haveCFMaxreadily available for the different hydrological response units of the Lahn catchment. We might find it useful to calibrateGMeltbased on the spatial pattern ofCFMax. Therefore, we first define anReplacerule for parameterGMelt:>>> from hydpy import Replace, FactorAdaptor >>> gmelt = Replace(name="gmelt", ... parameter="gmelt", ... value=2.0, ... lower=0.5, ... upper=2.0, ... parameterstep="1d", ... model="hland_96")
Second, we initialise a
FactorAdaptorobject based on target rule gmelt and our reference parameterCFMaxand assign it our rule object:>>> gmelt.adaptor = FactorAdaptor(gmelt, "cfmax")
The dill_assl subcatchment, like the whole Lahn basin, does not contain any glaciers. Hence it defines (identical)
CFMaxvalues for the zones of typeFIELDandFORESTbut must not specify any value forGMelt:>>> control = hp.elements.land_dill_assl.model.parameters.control >>> control.cfmax cfmax(field=4.55853, forest=2.735118) >>> control.gmelt gmelt(nan)
Next, we call method
apply_value()of theReplaceobject to apply theFactorAdaptorobject on all relevantGMeltinstances of the Lahn catchment:>>> gmelt.adaptor(control.gmelt)
The string representation of the
GMeltinstance of the Dill catchment indicates nothing happened:>>> control.gmelt gmelt(nan)
However, inspecting the individual values of the respective response units reveals the multiplication was successful:
>>> from hydpy import print_vector >>> print_vector(control.gmelt.values) 9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236
Calculating values for response units that do not require these values can be misleading. We can improve the situation by using the masks provided by the respective model; in our example, mask
Glacier. To make this clearer, we set the first six response units toZoneTypeGLACIER:>>> from hydpy.models.hland_96 import * >>> control.zonetype(GLACIER, GLACIER, GLACIER, GLACIER, GLACIER, GLACIER, ... FIELD, FOREST, ILAKE, FIELD, FOREST, ILAKE)
We now can assign the
SumAdaptorobject to the direct runoff-relatedReplaceobject and, for example, set its lower boundary to zero:Now we create a new
FactorAdaptorobject, handling the same parameters but also theGlaciermask:>>> gmelt.adaptor = FactorAdaptor(gmelt, "cfmax", "glacier")
To see the results of our new adaptor object, we change the values both of our reference parameter and our rule object:
>>> control.cfmax(field=5.0, forest=3.0, glacier=6.0) >>> gmelt.value = 0.5
The string representation of our target parameter shows that the glacier-related day degree factor of all glacier zones is now half as large as the snow-related one:
>>> gmelt.apply_value() >>> control.gmelt gmelt(3.0)
Note that all remaining values (for zone types
FIELD,FOREST, andILAKEare still the same. This intended behaviour allows calibrating, for example, hydrological response units of different types with different rule objects:>>> print_vector(control.gmelt.values) 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 9.11706, 5.470236, 9.11706, 5.470236, 9.11706, 5.470236
- class hydpy.auxs.calibtools.Rule(*, name: str, parameter: type[TypeParameter] | TypeParameter | str, value: float, lower: float = -inf, upper: float = inf, keyword: str | None = None, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
Bases:
ABC,Generic[TypeParameter]Base class for defining calibration rules.
Each
Ruleobject relates one calibration parameter with some model parameters. We select the classReplaceas a concrete example for the following explanations and use the Lahn example project, which we prepare by calling functionprepare_full_example_2():>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2()
We define a
Ruleobject supposed to replace the values of parameterFCof application modellland_dd. Note that argument name is the rule’s name, whereas the argument parameter is the parameter’s name:>>> from hydpy import Replace >>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96")
The following string representation shows us the complete list of available arguments:
>>> rule Replace( name="fc", parameter="fc", value=100.0, lower=-inf, upper=inf, keyword=None, parameterstep=None, model="hland_96", selections=("complete",), )
The initial value of parameter
FCis 206 mm:>>> fc = hp.elements.land_lahn_marb.model.parameters.control.fc >>> fc fc(206.0)
We can modify it by calling method
apply_value():>>> rule.apply_value() >>> fc fc(100.0)
You can change and apply the value at any time:
>>> rule.value = 200.0 >>> rule.apply_value() >>> fc fc(200.0)
Sometimes, one must differentiate between the original value to be calibrated and the actually applied value. Therefore, (only) the
Replaceclass allows for defining custom “adaptors”. Prepare anAdaptorfunction and assign it to the relevantReplaceobject (see the documentation on classSumAdaptororFactorAdaptorfor more realistic examples):>>> rule.adaptor = lambda target: target(2.0 * rule.value)
Now, our rule does not apply the original but the adapted calibration parameter value:
>>> rule.apply_value() >>> fc fc(400.0)
Use method
reset_parameters()to restore the original states of the affected parameters (“original” here means at the time of initialisation of theRuleobject):>>> rule.reset_parameters() >>> fc fc(206.0)
Some parameter types support defining their values via custom keywords.
FC, for example, allows setting the values of multiple zones of the same land-use type via keyword arguments such as forest:>>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... keyword="forest", ... model="hland_96") >>> rule.apply_value() >>> fc fc(field=206.0, forest=100.0)
The value of parameter
FCis not time-dependent. Therefore, anyparameterstepinformation given to itsRuleobject is ignored (note that we pass an example parameter object of typeFCinstead of the string fc this time):>>> Replace(name="fc", ... parameter=fc, ... value=100.0, ... model="hland_96", ... parameterstep="1d") Replace( name="fc", parameter="fc", value=100.0, lower=-inf, upper=inf, keyword=None, parameterstep=None, model="hland_96", selections=("complete",), )
For time-dependent parameters, the rule queries the current global
parameterstepvalue if you do not specify one explicitly (note that we pass the parameter typePercMaxand the modulehland_96this time):>>> from hydpy.models import hland_96 >>> from hydpy.models.hland.hland_control import PercMax >>> rule = Replace(name="percmax", ... parameter=PercMax, ... value=5.0, ... model=hland_96)
The
Ruleobject internally handles, to avoid confusion, a copy ofparameterstep.>>> from hydpy import pub >>> pub.options.parameterstep = None >>> rule Replace( name="percmax", parameter="percmax", value=5.0, lower=-inf, upper=inf, keyword=None, parameterstep="1d", model="hland_96", selections=("complete",), ) >>> rule.apply_value() >>> percmax = hp.elements.land_lahn_marb.model.parameters.control.percmax >>> with pub.options.parameterstep("1d"): ... percmax percmax(5.0)
Alternatively, you can pass a parameter step size yourself:
>>> rule = Replace(name="percmax", ... parameter="percmax", ... value=5.0, ... model="hland_96", ... parameterstep="2d") >>> rule.apply_value() >>> with pub.options.parameterstep("1d"): ... percmax percmax(2.5)
Missing parameter step-size information results in the following error:
>>> Replace(name="percmax", ... parameter="percmax", ... value=5.0, ... model="hland_96") Traceback (most recent call last): ... RuntimeError: While trying to initialise the `Replace` rule object `percmax`, the following error occurred: Rules which handle time-dependent parameters require information on the parameter timestep size. Either assign it directly or define it via option `parameterstep`.
With the following definition, the
Ruleobject queries allElementobjects handlinghland_96instances from the globalSelectionsobject pub.selections:>>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96") >>> rule.elements Elements("land_dill_assl", "land_lahn_kalk", "land_lahn_leun", "land_lahn_marb")
Alternatively, you can specify selections by passing themselves or their names (the latter requires them to be a member of pub.selections):
>>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... selections=[pub.selections.headwaters, "nonheadwaters"]) >>> rule.elements Elements("land_dill_assl", "land_lahn_kalk", "land_lahn_leun", "land_lahn_marb")
When not using the model argument, you must ensure the selected elements handle the correct model instance:
>>> Replace(name="fc", ... parameter="fc", ... value=100.0) Traceback (most recent call last): ... RuntimeError: While trying to initialise the `Replace` rule object `fc`, the following error occurred: No (sub)model of element `stream_dill_assl_lahn_leun` defines a control parameter named `fc`.
“Empty” rule objects are always considered erroneous:
>>> Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="musk_classic", ... selections=[pub.selections.headwaters, "nonheadwaters"]) Traceback (most recent call last): ... ValueError: While trying to initialise the `Replace` rule object `fc`, the following error occurred: Object `Selections("headwaters", "nonheadwaters")` does not handle any `musk_classic` model instances.
All mentioned functionalities also work for submodels:
>>> rule = Replace(name="soilmoisturelimit", ... parameter="soilmoisturelimit", ... value=0.8, ... model="evap_aet_hbv96") >>> submodel = hp.elements.land_lahn_marb.model.aetmodel >>> soilmoisturelimit = submodel.parameters.control.soilmoisturelimit >>> soilmoisturelimit soilmoisturelimit(0.9) >>> rule.apply_value() >>> soilmoisturelimit soilmoisturelimit(0.8)
We encourage explicitly defining the model type when working with complex submodel combinations so as not to calibrate different but equally named parameters accidentally:
>>> rule = Replace(name="fc", ... parameter="fc", ... value=0.8, ... model="evap_aet_hbv96") Traceback (most recent call last): ... RuntimeError: While trying to initialise the `Replace` rule object `fc`, the following error occurred: Model `evap_aet_hbv96` of element `land_dill_assl` does not define a control parameter named `fc`.
We consider name clashes like the following made-up example unlikely but still carry out additional runtime type checks as a precaution:
>>> control = hp.elements.land_lahn_marb.model.parameters.control >>> control.soilmoisturelimit = control.fc >>> rule = Replace(name="?", ... parameter="soilmoisturelimit", ... value=0.8, ... selections=[pub.selections.headwaters]) Traceback (most recent call last): ... RuntimeError: While trying to initialise the `Replace` rule object `?`, the following error occurred: Parameter types are inconsistent: `hydpy.models.hland.hland_control.FC` vs `hydpy.models.evap.evap_control.SoilMoistureLimit`.
- keyword: str | None¶
The name of the addressed keyword argument or, for a positional argument,
None.
- element2parameters: dict[Element, list[TypeParameter]]¶
The
Elementobjects and their related parameter objects.
- property elements: Elements¶
The
Elementobjects, which handle the relevant targetParameterinstances.
- property value: float¶
The calibration parameter value.
Property
valueensures that the given value adheres to the defined lower and upper boundaries:>>> from hydpy import Replace >>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... lower=50.0, ... upper=200.0, ... model="hland_96")
>>> rule.value = 0.0 >>> rule.value 50.0
With option
warntrimenabled (the default), propertyvaluealso emits a warning like the following:>>> from hydpy.core.testtools import warn_later >>> with pub.options.warntrim(True), warn_later(): ... rule.value = 300.0 UserWarning: The value of the `Replace` object `fc` must not be smaller than `50.0` or larger than `200.0`, but the given value is `300.0`. Applying the trimmed value `200.0` instead. >>> rule.value 200.0
- abstractmethod apply_value() None[source]¶
Apply the current value to the relevant
Parameterobjects.To be overridden by the concrete subclasses.
- reset_parameters() None[source]¶
Reset all relevant parameter objects to their original states.
>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import Replace >>> rule = Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96") >>> fc = hp.elements.land_lahn_marb.model.parameters.control.fc >>> fc fc(206.0) >>> fc(100.0) >>> fc fc(100.0) >>> rule.reset_parameters() >>> fc fc(206.0)
- parameterstep¶
The parameter step size relevant to the related model parameter.
For non-time-dependent parameters, property
parameterstepis (usually)None.
- class hydpy.auxs.calibtools.Replace(*, name: str, parameter: type[TypeParameter] | TypeParameter | str, value: float, lower: float = -inf, upper: float = inf, keyword: str | None = None, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
-
Ruleclass, which simply replaces the current model parameter value(s) with the current calibration parameter value.See the documentation on class
Rulefor further information.- adaptor: Adaptor | None = None¶
An optional function object for customising individual calibration strategies.
See the documentation on the classes
Rule,SumAdaptor, andFactorAdaptorfor further information.
- class hydpy.auxs.calibtools.Add(*, name: str, parameter: type[TypeParameter] | TypeParameter | str, value: float, lower: float = -inf, upper: float = inf, keyword: str | None = None, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
-
Ruleclass, which adds its calibration delta to the original model parameter value(s).Please read the examples of the documentation on class
Rulefirst. Here, we modify some of these examples to show the unique features of classAdd.The first example deals with the non-time-dependent parameter
FC. The followingAddobject adds its current value to the parameter’s original values:>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import Add >>> rule = Add(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96") >>> fc = hp.elements.land_lahn_marb.model.parameters.control.fc >>> fc fc(206.0) >>> rule.apply_value() >>> fc fc(306.0)
When specifying the keyword field, the
Addrule modifies the field capacity of zones of typeFIELDonly:>>> fc(206.0) >>> rule = Add(name="fc", ... parameter="fc", ... value=100.0, ... keyword="field", ... model="hland_96") >>> rule.apply_value() >>> fc fc(field=306.0, forest=206.0)
The second example deals with the time-dependent parameter
CFMaxand shows that everything works even when the actualparameterstep(2 days) differs from the currentsimulationstep(1 day):>>> rule = Add(name="cfmax", ... parameter="cfmax", ... value=2.0, ... model="hland_96", ... parameterstep="2d") >>> cfmax = hp.elements.land_lahn_marb.model.parameters.control.cfmax >>> cfmax cfmax(field=5.0, forest=3.0) >>> rule.apply_value() >>> cfmax cfmax(field=6.0, forest=4.0)
This time, we modify the
FORESTzones only:>>> cfmax(field=5.0, forest=3.0) >>> rule = Add(name="cfmax", ... parameter="cfmax", ... value=2.0, ... keyword="forest", ... model="hland_96", ... parameterstep="2d") >>> rule.apply_value() >>> cfmax cfmax(field=5.0, forest=4.0)
In the third example, we modify the scalar parameter
NmbSegmentsby its optional keyword argument lag:>>> rule = Add(name="lag", ... parameter="nmbsegments", ... value=1.0, ... keyword="lag", ... model="musk_classic", ... parameterstep="2d") >>> nmbsegments = hp.elements.stream_lahn_marb_lahn_leun.model.parameters.control.nmbsegments >>> nmbsegments nmbsegments(lag=0.583) >>> rule.apply_value() >>> nmbsegments nmbsegments(lag=2.583)
- keyword: str | None¶
The name of the addressed keyword argument or, for a positional argument,
None.
- class hydpy.auxs.calibtools.Multiply(*, name: str, parameter: type[TypeParameter] | TypeParameter | str, value: float, lower: float = -inf, upper: float = inf, keyword: str | None = None, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
-
Ruleclass for multiplying the original model parameter value(s) by its calibration factor.Please read the examples of the documentation on class
Rulefirst. Here, we modify some of these examples to show the unique features of classMultiply.The first example deals with the non-time-dependent parameter
FC. The followingMultiplyobject multiplies the parameter’s original values by its current calibration factor:>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import Add >>> rule = Multiply(name="fc", ... parameter="fc", ... value=2.0, ... model="hland_96") >>> fc = hp.elements.land_lahn_marb.model.parameters.control.fc >>> fc fc(206.0) >>> rule.apply_value() >>> fc fc(412.0)
When specifying the keyword field, the
Multiplyrule modifies the field capacity of zones of typeFIELDonly:>>> fc(206.0) >>> rule = Multiply(name="fc", ... parameter="fc", ... value=2.0, ... keyword="field", ... model="hland_96") >>> rule.apply_value() >>> fc fc(field=412.0, forest=206.0)
The second example deals with the time-dependent parameter
CFMaxand shows that everything works even when the actualparameterstep(2 days) differs from the currentsimulationstep(1 day):>>> rule = Multiply(name="cfmax", ... parameter="cfmax", ... value=2.0, ... model="hland_96", ... parameterstep="2d") >>> cfmax = hp.elements.land_lahn_marb.model.parameters.control.cfmax >>> cfmax cfmax(field=5.0, forest=3.0) >>> rule.apply_value() >>> cfmax cfmax(field=10.0, forest=6.0)
This time, we modify the
FORESTzones only:>>> cfmax(field=5.0, forest=3.0) >>> rule = Multiply(name="cfmax", ... parameter="cfmax", ... value=2.0, ... keyword="forest", ... model="hland_96", ... parameterstep="2d") >>> cfmax cfmax(field=5.0, forest=3.0) >>> rule.apply_value() >>> cfmax cfmax(field=5.0, forest=6.0)
In the third example, we modify the scalar parameter
NmbSegmentsby its optional keyword argument lag:>>> rule = Multiply(name="lag", ... parameter="nmbsegments", ... value=2.0, ... keyword="lag", ... model="musk_classic", ... parameterstep="2d") >>> nmbsegments = hp.elements.stream_lahn_marb_lahn_leun.model.parameters.control.nmbsegments >>> nmbsegments nmbsegments(lag=0.583) >>> rule.apply_value() >>> nmbsegments nmbsegments(lag=1.166)
- keyword: str | None¶
The name of the addressed keyword argument or, for a positional argument,
None.
- class hydpy.auxs.calibtools.CalibrationInterface(hp: HydPy, targetfunction: TargetFunction)[source]¶
Bases:
Generic[TypeRule1]Interface for the coupling of HydPy to optimisation libraries like NLopt.
Essentially, class
CalibrationInterfaceis supposed for the structured handling of multiple objects of the differentRulesubclasses. Hence, please read the documentation on classRulebefore continuing, on which we base the following explanations.We work with the Lahn example project again:
>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2()
First, we create a
CalibrationInterfaceobject. Initially, it needs to know the relevantHydPyobject and the target or objective function (here, we define the target function sloppily via the lambda statement; see the documentation on the protocol classTargetFunctionfor a more formal definition and further explanations):>>> from hydpy import CalibrationInterface, nse >>> ci = CalibrationInterface( ... hp=hp, ... targetfunction=lambda: sum(nse(node=node) for node in hp.nodes))
Next, we use function
make_rules(), which creates oneReplacerule related to parameterFCand another one related to parameterPercMaxin one step, and add them via methodadd_rules():>>> from hydpy import Replace >>> from hydpy.auxs.calibtools import make_rules >>> ci.add_rules(*make_rules(rule=Replace, ... names=["fc", "percmax"], ... parameters=["fc", "percmax"], ... values=[100.0, 5.0], ... keywords=[None, None], ... lowers=[50.0, 1.0], ... uppers=[200.0, 10.0], ... parametersteps="1d", ... model="hland_96"))
>>> print(ci) CalibrationInterface >>> ci Replace( name="fc", parameter="fc", value=100.0, lower=50.0, upper=200.0, keyword=None, parameterstep=None, model="hland_96", selections=("complete",), ) Replace( name="percmax", parameter="percmax", value=5.0, lower=1.0, upper=10.0, keyword=None, parameterstep="1d", model="hland_96", selections=("complete",), )
Adding rules later does not remove already available ones. For demonstration, we add one for calibrating parameter
Coefficientsof application modelmusk_classicvia its keyword damp:>>> len(ci) 2 >>> ci.add_rules(Replace(name="damp", ... parameter="coefficients", ... value=0.2, ... lower=0.0, ... upper=0.5, ... keyword="damp", ... selections=["complete"], ... model="musk_classic")) >>> len(ci) 3
All rules are available via attribute and keyword access:
>>> ci.fc Replace( name="fc", parameter="fc", value=100.0, lower=50.0, upper=200.0, keyword=None, parameterstep=None, model="hland_96", selections=("complete",), )
>>> ci.FC Traceback (most recent call last): ... AttributeError: The actual calibration interface does neither handle a normal attribute nor a rule object named `FC`...
>>> ci["damp"] Replace( name="damp", parameter="coefficients", value=0.2, lower=0.0, upper=0.5, keyword="damp", parameterstep=None, model="musk_classic", selections=("complete",), )
>>> ci["Damp"] Traceback (most recent call last): ... KeyError: 'The actual calibration interface does not handle a rule object named `Damp`.'
The following properties return consistently sorted information on the handles
Ruleobjects:>>> ci.names ('fc', 'percmax', 'damp') >>> ci.keywords (None, None, 'damp') >>> ci.values (100.0, 5.0, 0.2) >>> ci.lowers (50.0, 1.0, 0.0) >>> ci.uppers (200.0, 10.0, 0.5)
All tuples reflect the current state of all rules:
>>> ci.damp.value = 0.3 >>> ci.values (100.0, 5.0, 0.3)
For the following examples, we perform a simulation run and assign the values of the simulated time series to the observed series:
>>> conditions = hp.conditions >>> hp.simulate() >>> for node in hp.nodes: ... node.sequences.obs.series = node.sequences.sim.series >>> hp.conditions = conditions
As the agreement between the simulated and the “observed” time series is perfect for all four gauges, method
calculate_likelihood()returns the highest possible sum of fournse()values and also stores it under the attribute result:>>> from hydpy import round_ >>> round_(ci.calculate_likelihood()) 4.0 >>> round_(ci.result) 4.0
When performing a manual calibration, it might be convenient to use method
apply_values(). To explain how it works, we first show the values of the relevant parameters of some randomly selected model instances:>>> stream = hp.elements.stream_lahn_marb_lahn_leun.model >>> stream.parameters.control nmbsegments(lag=0.583) coefficients(damp=0.0) >>> land = hp.elements.land_lahn_marb.model >>> land.parameters.control.fc fc(206.0) >>> land.parameters.control.percmax percmax(1.02978)
Method
apply_values()of classCalibrationInterfacecalls methodapply_value()of all handledRuleobjects, performs some preparations (for example, it derives the values of the secondary parameters), executes a simulation run, calls methodcalculate_likelihood(), and returns the result:>>> result = ci.apply_values() >>> stream.parameters.control nmbsegments(lag=0.583) coefficients(damp=0.3) >>> land.parameters.control.fc fc(100.0) >>> land.parameters.control.percmax percmax(5.0)
Due to the changes in our parameter values, our simulation is not “perfect” anymore:
>>> round_(ci.result) 1.638413
Use method
reset_parameters()to restore the initial states of all affected parameters:>>> ci.reset_parameters() >>> stream.parameters.control nmbsegments(lag=0.583) coefficients(damp=0.0) >>> land = hp.elements.land_lahn_marb.model >>> land.parameters.control.fc fc(206.0) >>> land.parameters.control.percmax percmax(1.02978)
Now we get the same “perfect” efficiency again:
>>> hp.simulate() >>> round_(ci.calculate_likelihood()) 4.0 >>> hp.conditions = conditions
Note the perform_simulation argument of method
apply_values(), which allows changing the model parameter values and updating theHydPyobject only without triggering a simulation run (and to calculate and return a new likelihood value):>>> ci.apply_values(perform_simulation=False) >>> stream.parameters.control nmbsegments(lag=0.583) coefficients(damp=0.3) >>> land.parameters.control.fc fc(100.0) >>> land.parameters.control.percmax percmax(5.0)
Optimisers, like those implemented in NLopt, often provide their new parameter estimates via vectors. Method
perform_calibrationstep()accepts such vectors and updates the handledRuleobjects accordingly. After that, it performs the same steps as described for methodapply_values():>>> round_(ci.perform_calibrationstep([100.0, 5.0, 0.3])) 1.638413
>>> stream.parameters.control nmbsegments(lag=0.583) coefficients(damp=0.3)
>>> land.parameters.control.fc fc(100.0) >>> land.parameters.control.percmax percmax(5.0)
Method
perform_calibrationstep()writes intermediate results into a log file, if available. Prepare it beforehand via methodprepare_logfile():>>> with TestIO(): ... ci.prepare_logfile(logfilepath="example_calibration.log", ... objectivefunction="NSE", ... documentation="Just a doctest example.")
To continue “manually”, we now can call method
update_logfile()to write the lastly calculated efficiency and the corresponding calibration parameter values to the log file:>>> with TestIO(): ... ci.update_logfile() ... with open("example_calibration.log") as file_: ... print(file_.read()) # Just a doctest example. NSE fc percmax damp parameterstep None 1d None 1.638413 100.0 5.0 0.3
To prevent (automatic) calibration runs from crashing due to IO problems, method
update_logfile()raises warnings instead of errors in such cases and logs the inwritten data internally:>>> import os >>> from hydpy.core.testtools import warn_later >>> with TestIO(), warn_later(): ... ci._logfilepath = "dirname1/filename.log" ... ci.update_logfile() UserWarning: While trying to update the logfile `dirname1/filename.log`, the following problem occured: [Errno 2] No such file or directory: 'dirname1/filename.log'.
On subsequent calls, it tries to write both the previously logged and the new data:
>>> with TestIO(): ... os.makedirs("dirname1", exist_ok=True) ... ci.update_logfile() ... with open("dirname1/filename.log") as file_: ... print(file_.read()) 1.638413 100.0 5.0 0.3 1.638413 100.0 5.0 0.3
Call method
finalise_logfile()to ensure theCalibrationInterfaceobject does not withhold data after the end of a calibration run. If you do so, it sleeps until it gets the chance to write the logged data and warns you about this problem from time to time (we demonstrate this by mocking thewarn()function and, to keep our test example awake, thesleep()function):>>> with TestIO(): ... ci._logfilepath = "dirname2/filename.log" ... ci.update_logfile() Traceback (most recent call last): ... UserWarning: While trying to update the logfile `dirname2/filename.log`, the following problem occured: [Errno 2] No such file or directory: 'dirname2/filename.log'. >>> from unittest import mock >>> with TestIO(): ... with mock.patch("time.sleep") as mocked: ... mocked.side_effect = Exception("time.sleep actually called") ... ci.finalise_logfile() Traceback (most recent call last): ... UserWarning: Trying to finalise logfile `dirname2/filename.log` failed 1 times. >>> with TestIO(): ... with mock.patch("warnings.warn"), mock.patch("time.sleep") as mocked: ... mocked.side_effect = Exception("time.sleep actually called") ... ci.finalise_logfile() Traceback (most recent call last): ... Exception: time.sleep actually called >>> with TestIO(): ... os.makedirs("dirname2", exist_ok=True) ... ci.finalise_logfile() ... with open("dirname2/filename.log") as file_: ... print(file_.read()) 1.638413 100.0 5.0 0.3
>>> ci._logfilepath = "example_calibration.log"
For automatic calibration, one needs a calibration algorithm like the following, which checks the lower and upper boundaries and the initial values of all
Ruleobjects:>>> def find_max(function, lowers, uppers, inits): ... best_result = -999.0 ... best_parameters = None ... for values in (lowers, uppers, inits): ... result = function(values) ... if result > best_result: ... best_result = result ... best_parameters = values ... return best_parameters
Now we can assign method
perform_calibrationstep()to this oversimplified optimiser, which then returns the best examined calibration parameter values:>>> with TestIO(): ... find_max(function=ci.perform_calibrationstep, ... lowers=ci.lowers, ... uppers=ci.uppers, ... inits=ci.values) (200.0, 10.0, 0.5)
The log file now contains one line for our old result and three lines for the results of our optimiser:
>>> with TestIO(): ... with open("example_calibration.log") as file_: ... print(file_.read()) # Just a doctest example. NSE fc percmax damp parameterstep None 1d None 1.638413 100.0 5.0 0.3 -0.722221 50.0 1.0 0.0 2.347116 200.0 10.0 0.5 1.638413 100.0 5.0 0.3
Class
CalibrationInterfacealso provides methodread_logfile(), which automatically selects the best calibration result. Therefore, it needs to know that the highest result is the best, which we indicate by setting argument maximisation toTrue:>>> with TestIO(): ... ci.read_logfile(logfilepath="example_calibration.log", maximisation=True) >>> ci.fc.value 200.0 >>> ci.percmax.value 10.0 >>> ci.damp.value 0.5 >>> round_(ci.result) 2.347116 >>> round_(ci.apply_values()) 2.347116
On the contrary, if we set argument maximisation to
False, methodread_logfile()returns the worst result in our example:>>> with TestIO(): ... ci.read_logfile(logfilepath="example_calibration.log", maximisation=False) >>> ci.fc.value 50.0 >>> ci.percmax.value 1.0 >>> ci.damp.value 0.0 >>> round_(ci.result) -0.722221 >>> round_(ci.apply_values()) -0.722221
To prevent errors due to different parameter step-sizes, method
read_logfile()raises the following error whenever it detects inconsistencies:>>> ci.percmax.parameterstep = "2d" >>> with TestIO(): ... ci.read_logfile(logfilepath="example_calibration.log",maximisation=True) Traceback (most recent call last): ... RuntimeError: The current parameterstep of the `Replace` rule `percmax` (`2d`) does not agree with the one documentated in log file `example_calibration.log` (`1d`).
Method
read_logfile()reports inconsistent rule names as follows:>>> ci.remove_rules(ci.percmax) >>> with TestIO(): ... ci.read_logfile(logfilepath="example_calibration.log",maximisation=True) Traceback (most recent call last): ... RuntimeError: The names of the rules handled by the actual calibration interface (damp and fc) do not agree with the names in the header of logfile `example_calibration.log` (damp, fc, and percmax).
The last consistency check is optional. Set argument check to
Falseto force methodread_logfile()to query all available data instead of raising an error:>>> ci.add_rules(Replace(name="beta", ... parameter="beta", ... value=2.0, ... lower=1.0, ... upper=4.0, ... selections=["complete"], ... model="hland_96")) >>> ci.fc.value = 0.0 >>> ci.damp.value = 0.0 >>> with TestIO(): ... ci.read_logfile( ... logfilepath="example_calibration.log", ... maximisation=True, ... check=False, ... ) >>> ci.beta.value 2.0 >>> ci.fc.value 200.0 >>> ci.damp.value 0.5
- conditions: dict[str, dict[str, dict[str, dict[str, float | ndarray[tuple[int, ...], dtype[float64]]]]]]¶
The
conditionsof the givenHydPyobject.CalibrationInterfacequeries the conditions during its initialisation and uses them later to reset all relevant conditions before each new simulation run.
- add_rules(*rules: TypeRule1) None[source]¶
Add some
Ruleobjects to the actualCalibrationInterfaceobject.>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import CalibrationInterface >>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: None) >>> from hydpy import Replace >>> ci.add_rules(Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96"), ... Replace(name="percmax", ... parameter="percmax", ... value=5.0, ... model="hland_96"))
Note that method
add_rules()might change the number ofElementobjects relevant for theCalibrationInterfaceobject:>>> damp = Replace(name="damp", ... parameter="coefficients", ... value=0.2, ... keyword="damp", ... model="musk_classic") >>> len(ci._elements) 4 >>> ci.add_rules(damp) >>> len(ci._elements) 7
- get_rule(name: str, type_: type[TypeRule2] | None = None) TypeRule1 | TypeRule2[source]¶
Return a
Ruleobject (of a specific type).Method
get_rule()is a more typesafe alternative to simple keyword access. Besides the name of the requiredRuleobject, pass its subclass to convince your IDE (and yourself) that the returned rule follows this more specific type:>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import Add, CalibrationInterface, make_rules, nse, Replace >>> ci = CalibrationInterface( ... hp=hp, ... targetfunction=lambda: sum(nse(node=node) for node in hp.nodes)) >>> ci.add_rules(*make_rules(rule=Replace, ... names=["fc", "percmax"], ... parameters=["fc", "percmax"], ... values=[100.0, 5.0], ... keywords=["forest", None], ... lowers=[50.0, 1.0], ... uppers=[200.0, 10.0], ... parametersteps="1d", ... model="hland_96"))
>>> ci.get_rule("fc", Replace).name 'fc'
>>> ci.get_rule("Fc", Replace).name Traceback (most recent call last): ... RuntimeError: The actual calibration interface does not handle a rule object named `Fc`.
>>> ci.get_rule("fc", Replace).name 'fc'
>>> ci.get_rule("fc", Add).name Traceback (most recent call last): ... RuntimeError: The actual calibration interface does not handle a rule object named `fc` of type `Add`.
- remove_rules(*rules: str | TypeRule1) None[source]¶
Remove some
Ruleobjects from the actualCalibrationInterfaceobject.>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import CalibrationInterface >>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: None) >>> from hydpy import Replace >>> ci.add_rules(Replace(name="fc", ... parameter="fc", ... value=100.0, ... model="hland_96"), ... Replace(name="percmax", ... parameter="percmax", ... value=5.0, ... model="hland_96"), ... Replace(name="damp", ... parameter="coefficients", ... value=0.2, ... keyword="damp", ... model="musk_classic"))
You can remove each rule either by passing itself or its name (note that method
remove_rules()might change the number ofElementobjects relevant for theCalibrationInterfaceobject):>>> len(ci._elements) 7 >>> fc = ci.fc >>> fc in ci True >>> "damp" in ci True >>> ci.remove_rules(fc, "damp") >>> fc in ci False >>> "damp" in ci False >>> len(ci._elements) 4
Trying to remove a non-existing rule results in the following error:
>>> ci.remove_rules("fc") Traceback (most recent call last): ... RuntimeError: The actual calibration interface object does not handle a rule object named `fc`.
- prepare_logfile(logfilepath: str, objectivefunction: str = 'result', documentation: str | None = None) None[source]¶
Prepare a log file.
Use argument objectivefunction to describe the
TargetFunctionused for calculating the efficiency and argument documentation to add some information to the header of the logfile.See the main documentation on class
CalibrationInterfacefor further information.
- update_logfile() None[source]¶
Update the current log file, if available.
See the main documentation on class
CalibrationInterfacefor further information.
- finalise_logfile() None[source]¶
Update the current log file if method
update_logfile()was not entirely successful in doing so.See the main documentation on class
CalibrationInterfacefor further information.
- read_logfile(logfilepath: str, maximisation: bool, check: bool = True) None[source]¶
Read the log file with the given file path.
See the main documentation on class
CalibrationInterfacefor further information.
- property names: tuple[str, ...]¶
The names of all handled
Ruleobjects.See the main documentation on class
CalibrationInterfacefor further information.
- property values: tuple[float, ...]¶
The values of all handled
Ruleobjects.See the main documentation on class
CalibrationInterfacefor further information.
- property keywords: tuple[str | None, ...]¶
The (optional) target keywords of all handled
Ruleobjects.See the main documentation on class
CalibrationInterfacefor further information.
- property lowers: tuple[float, ...]¶
The lower boundaries of all handled
Ruleobjects.See the main documentation on class
CalibrationInterfacefor further information.
- property uppers: tuple[float, ...]¶
The upper boundaries of all handled
Ruleobjects.See the main documentation on class
CalibrationInterfacefor further information.
- property selections: tuple[str, ...]¶
The names of all
Selectionobjects addressed at least one of the handledRuleobjects.See the documentation on function
make_rules()for further information.
- property parametertypes: tuple[tuple[type[Parameter], str | None], ...]¶
The types of all
Parameterobjects addressed by at least one of the handledRuleobjects.See the documentation on function
make_rules()for further information.
- apply_values(perform_simulation: bool = True) float | None[source]¶
Apply all current calibration parameter values on all relevant parameters.
Set argument perform_simulation to
Falseto only change the actual parameter values and update theHydPyobject without performing a simulation run.See the main documentation on class
CalibrationInterfacefor further information.
- reset_parameters() None[source]¶
Reset all relevant parameters to their original states.
- See the main documentation on class
CalibrationInterfacefor further information.
- See the main documentation on class
- calculate_likelihood() float[source]¶
Apply the defined
TargetFunctionand return the result.See the main documentation on class
CalibrationInterfacefor further information.
- perform_calibrationstep(values: Iterable[float], *args: Any, **kwargs: Any) float[source]¶
Update all calibration parameters with the given values, update the
HydPyobject, perform a simulation run, and calculate and return the achieved efficiency.See the main documentation on class
CalibrationInterfacefor further information.
- print_table(*, parametertypes: Sequence[type[Parameter] | tuple[type[Parameter], str | None]] | None = None, selections: Sequence[str] | None = None, bounds: tuple[str, str] | None = ('lower', 'upper'), fillvalue: str = '/', sep: str = '\t', file_: TextIO | None = None) None[source]¶
Print the current calibration parameter values in a table format.
The following examples combine the base examples of the documentation on class
CalibrationInterfaceand classReplaceIUH, so please make sure to understand them before proceeding.We again use the Lahn example project but replace the
musk_classicmodel instances with those of application modelarma_rimorido, which allows discussing some special cases concerning the handling ofRuleIUH:>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import prepare_model >>> for element in hp.elements.river: ... element.model = prepare_model("arma_rimorido") ... element.model.parameters.control.responses([[], [1.0]]) ... element.model.parameters.update()
We pass a (useless) dummy target function to the
CalibrationInterfaceobject:>>> from hydpy import CalibrationInterface >>> ci = CalibrationInterface(hp=hp, targetfunction=lambda: 1.0)
Regarding
hland_96, we intend to calibrate the parametersFCandPercMaxwith different values for the selections headwaters and nonheadwaters:>>> from hydpy import CalibSpec, CalibSpecs, make_rules, Replace >>> calibspecs = CalibSpecs( ... CalibSpec(name="fc", default=100.0, lower=50.0, upper=200.0), ... CalibSpec(name="percmax", default=5.0, lower=1.0, upper=10.0, parameterstep="1d")) >>> ci.add_rules(*make_rules(rule=Replace, ... calibspecs=calibspecs, ... model="hland_96", ... selections=("headwaters", "nonheadwaters"), ... product=True))
Regarding
arma_rimorido, we cannot calibrate the values of parameterResponsesin a meaningful way. So instead, we use theLinearStorageCascadeas a meta-model and calibrate its parameterskandn:>>> from hydpy import LinearStorageCascade, ReplaceIUH >>> k = ReplaceIUH(name="k_global", ... target="k", ... parameter="responses", ... value=2.0, ... lower=1.0, ... parameterstep="1d", ... selections=("streams",)) >>> n = ReplaceIUH(name="n_global", ... target="n", ... parameter="responses", ... value=4.0, ... lower=1.0, ... upper=100.0, ... selections=("streams",)) >>> name2lsc = {element.name: LinearStorageCascade(k=1.0, n=1.0) ... for element in hp.elements.river} >>> k.add_iuhs(**name2lsc) >>> n.add_iuhs(**name2lsc) >>> ci.add_rules(k, n)
We change the values of two
Ruleobjects related tohland_96to clarify that all values appear in the correct table cells:>>> ci["fc_headwaters"].value = 200.0 >>> ci["percmax_nonheadwaters"].value = 10.0
By default, method
print_table()prints the values of all handledRuleobjects. It varies the target control parameters on the first axis and the target selections on the second axis. Row two and three contain the (identical) lower and upper boundary values corresponding to the respective control parameters:>>> ci.print_table() lower upper headwaters nonheadwaters streams k->Responses 1.0 inf / / 2.0 n->Responses 1.0 100.0 / / 4.0 FC 50.0 200.0 200.0 100.0 / PercMax 1.0 10.0 5.0 10.0 /
For non-identical boundary values, method
print_table()prints fill values in the relevant cells. Besides this, the following example shows how to define alternative titles for the boundary value columns:>>> ci["fc_headwaters"].lower = 60.0 >>> ci["percmax_nonheadwaters"].upper = 20.0 >>> ci.print_table(bounds=("min", "max")) min max headwaters nonheadwaters streams k->Responses 1.0 inf / / 2.0 n->Responses 1.0 100.0 / / 4.0 FC / 200.0 200.0 100.0 / PercMax 1.0 / 5.0 10.0 /
Pass
Noneto argument bounds to omit writing any boundary value column:>>> ci.print_table(bounds=None) headwaters nonheadwaters streams k->Responses / / 2.0 n->Responses / / 4.0 FC 200.0 100.0 / PercMax 5.0 10.0 /
The next example shows how to change the tabulated target parameters and selections. Method
print_table()uses the (given alternative) fill value for each parameter-selection-combination not met by any of the availableRuleobjects. ForRuleIUH-related parameters, we must specify both the control parameter (as a type, in our exampleResponses) and the meta-parameter (as a string, in our examplek) within atuple:>>> from hydpy.models.hland.hland_control import CFlux, PercMax >>> from hydpy.models.arma.arma_control import Responses >>> ci.print_table( ... parametertypes=(PercMax, CFlux, (Responses, "k")), ... selections=("streams", "headwaters"), ... bounds=None, ... fillvalue="-") streams headwaters PercMax - 5.0 CFlux - - k->Responses 2.0 -
Note that the value of the same calibration parameter might appear multiple times when targeting multiple
Selectionobjects:>>> ci["fc_headwaters"].selections = ("headwaters", "streams") >>> ci.print_table(bounds=None) headwaters nonheadwaters streams k->Responses / / 2.0 n->Responses / / 4.0 FC 200.0 100.0 200.0 PercMax 5.0 10.0 /
- class hydpy.auxs.calibtools.RuleIUH(*, name: str, target: str, parameter: type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
Bases:
Rule[arma_control.Responses]A
Rule, class specialised forIUHparameters.RuleIUHserves as a base class only. Please see the concrete implementationReplaceIUHfor further information.- update_parameters: bool = True¶
Flag indicating whether method
apply_value()should calculate thecoefsand pass them to the relevant model parameter or not.Set this flag to
Falsefor the firstReplaceIUHobject when another handles the same elements and is applied afterwards.
- add_iuhs(**iuhs: IUH) None[source]¶
Add one
IUHobject for each relevantElementobject.See the main documentation on class
ReplaceIUHfor further information.
- reset_parameters() None[source]¶
Reset all relevant parameter objects to their original states.
See the main documentation on class
ReplaceIUHfor further information.
- class hydpy.auxs.calibtools.ReplaceIUH(*, name: str, target: str, parameter: type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
Bases:
RuleIUHA
RuleIUHclass for replacingIUHparameter values with the current calibration parameter values.Usually, it is not a good idea to calibrate the AR and MA coefficients of parameters like
Responsesof modelarma_rimoridoindividually. Instead, we need to calibrate the few coefficients of the underlyingIUHobjects, which calculate the ARMA coefficients. ClassReplaceIUHhelps to accomplish this task.Note
Class
ReplaceIUHis still under development. For example, it does not address the possibility of different ARMA coefficients related to different discharge thresholds. Hence, the usage of classReplaceIUHmight change in the future.So far, there is no example project containing
arma_rimoridomodels instances. Therefore, we generate a simple one consisting of twoElementobjects only:>>> from hydpy import Element, prepare_model, Selection >>> element1 = Element("element1", inlets="in1", outlets="out1") >>> element2 = Element("element2", inlets="in2", outlets="out2") >>> complete = Selection("complete", elements=[element1, element2]) >>> element1.model = prepare_model("arma_rimorido") >>> element2.model = prepare_model("arma_rimorido")
We focus on class
TranslationDiffusionEquationin the following. First, we create two separate instances and use them to calculate the response coefficients of botharma_rimoridoinstances:>>> from hydpy import TranslationDiffusionEquation >>> tde1 = TranslationDiffusionEquation(u=5.0, d=15.0, x=1.0) >>> tde2 = TranslationDiffusionEquation(u=5.0, d=15.0, x=2.0) >>> element1.model.parameters.control.responses(tde1.arma.coefs) >>> element1.model.parameters.control.responses responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276), (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0, -0.000001, 0.0, 0.0, 0.0, 0.0))) >>> element2.model.parameters.control.responses(tde2.arma.coefs) >>> element2.model.parameters.control.responses responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004), (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))
Next, we define one
ReplaceIUHfor modifying parameteruand another one for changingd:>>> from hydpy import ReplaceIUH >>> u = ReplaceIUH(name="U", ... target="u", ... parameter="responses", ... value=5.0, ... lower=1.0, ... upper=10.0, ... selections=[complete]) >>> d = ReplaceIUH(name="D", ... target="d", ... parameter="responses", ... value=15.0, ... lower=5.0, ... upper=50.0, ... selections=[complete])
We add and thereby connect the
ElementandTranslationDiffusionEquationobjects to bothReplaceIUHobjects via methodadd_iuhs():>>> u.add_iuhs(element1=tde1, element2=tde2) >>> d.add_iuhs(element1=tde1, element2=tde2)
Note that method
add_iuhs()enforces to add allIUHobjects at ones to avoid inconsistencies that might be hard to track later:>>> d.add_iuhs(element1=tde1) Traceback (most recent call last): ... RuntimeError: While trying to add `IUH` objects to the `ReplaceIUH` rule `D`, the following error occurred: The given elements (element1) do not agree with the complete set of relevant elements (element1 and element2).
By default, each
ReplaceIUHobject triggers the calculation of the ARMA coefficients during the execution of its methodapply_value(), which can be a waste of computation time if we want to calibrate multipleIUHcoefficients. To save computation time in such cases, set optionupdate_parameterstoFalsefor all except the lastly executedReplaceIUHobjects:>>> u.update_parameters = False
Now, changing the value of rule U and calling method
apply_value()does not affect the coefficients of bothResponsesparameters:>>> u.value = 10.0 >>> u.apply_value() >>> tde1 TranslationDiffusionEquation(d=15.0, u=10.0, x=1.0) >>> element1.model.parameters.control.responses responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276), (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0, -0.000001, 0.0, 0.0, 0.0, 0.0))) >>> tde2 TranslationDiffusionEquation(d=15.0, u=10.0, x=2.0) >>> element2.model.parameters.control.responses responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004), (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))
On the other side, calling method
apply_value()of rule D does activate the freshly set value of rule D and the previously set value of rule U, as well:>>> d.value = 50.0 >>> d.apply_value() >>> tde1 TranslationDiffusionEquation(d=50.0, u=10.0, x=1.0) >>> element1.model.parameters.control.responses responses(th_0_0=((0.811473, -0.15234, -0.000256, 0.000177), (0.916619, -0.670781, 0.087185, 0.007923))) >>> tde2 TranslationDiffusionEquation(d=50.0, u=10.0, x=2.0) >>> element2.model.parameters.control.responses responses(th_0_0=((0.832237, -0.167205, 0.002007, 0.000184), (0.836513, -0.555399, 0.037628, 0.014035)))
Use method
reset_parameters()to restore the original ARMA coefficients:>>> d.reset_parameters() >>> element1.model.parameters.control.responses responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276), (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0, -0.000001, 0.0, 0.0, 0.0, 0.0))) >>> element2.model.parameters.control.responses responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004), (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))
- keyword: str | None¶
The name of the addressed keyword argument or, for a positional argument,
None.
- element2parameters: dict[Element, list[TypeParameter]]¶
The
Elementobjects and their related parameter objects.
- apply_value() None[source]¶
Apply all current calibration parameter values to all relevant
IUHobjects and eventually update the related parameter’s ARMA coefficients.See the main documentation on class
ReplaceIUHfor further information.
- class hydpy.auxs.calibtools.MultiplyIUH(*, name: str, target: str, parameter: type[arma_control.Responses] | arma_control.Responses | str, value: float, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, model: types.ModuleType | str | None = None)[source]¶
Bases:
RuleIUHA
RuleIUHclass for replacingIUHparameter values with the current calibration parameter values, applied on the originalIUHvalues as factors.Please read the documentation on class
ReplaceIUHfirst, from which we take the following test configuration:>>> from hydpy import Element, prepare_model, Selection >>> element1 = Element("element1", inlets="in1", outlets="out1") >>> element2 = Element("element2", inlets="in2", outlets="out2") >>> complete = Selection("complete", elements=[element1, element2]) >>> element1.model = prepare_model("arma_rimorido") >>> element2.model = prepare_model("arma_rimorido")
>>> from hydpy import TranslationDiffusionEquation >>> tde1 = TranslationDiffusionEquation(u=5.0, d=15.0, x=1.0) >>> tde2 = TranslationDiffusionEquation(u=5.0, d=15.0, x=2.0) >>> element1.model.parameters.control.responses(tde1.arma.coefs) >>> element1.model.parameters.control.responses responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276), (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0, -0.000001, 0.0, 0.0, 0.0, 0.0))) >>> element2.model.parameters.control.responses(tde2.arma.coefs) >>> element2.model.parameters.control.responses responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004), (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))
Initialising
MultiplyIUHworks exactly as forReplaceIUH, except for the semantic difference that value, lower, and upper now represent factors:>>> from hydpy import MultiplyIUH >>> u = MultiplyIUH(name="U", ... target="u", ... parameter="responses", ... value=2.0, ... lower=1.0, ... upper=4.0, ... selections=[complete]) >>> d = MultiplyIUH(name="D", ... target="d", ... parameter="responses", ... value=0.5, ... lower=0.2, ... upper=2.0, ... selections=[complete])
>>> u.add_iuhs(element1=tde1, element2=tde2) >>> d.add_iuhs(element1=tde1, element2=tde2) >>> u.update_parameters = False
The following examples demonstrate that the current calibration values actually as factors, applied to the original values of the relevant
IUHproperties:>>> u.value = 3.0 >>> u.apply_value() >>> d.value = 1.0/3.0 >>> d.apply_value() >>> tde1 TranslationDiffusionEquation(d=5.0, u=15.0, x=1.0) >>> element1.model.parameters.control.responses responses(th_0_0=((0.0, 0.0), (0.933333, 0.066667))) >>> tde2 TranslationDiffusionEquation(d=5.0, u=15.0, x=2.0) >>> element2.model.parameters.control.responses responses(th_0_0=((0.0, 0.0), (0.866667, 0.133333)))
>>> u.value = 1.0 >>> u.apply_value() >>> d.value = 1.0 >>> d.apply_value() >>> tde1 TranslationDiffusionEquation(d=15.0, u=5.0, x=1.0) >>> element1.model.parameters.control.responses responses(th_0_0=((0.906536, -0.197555, 0.002128, 0.000276), (0.842788, -0.631499, 0.061685, 0.015639, 0.0, 0.0, 0.0, -0.000001, 0.0, 0.0, 0.0, 0.0))) >>> tde2 TranslationDiffusionEquation(d=15.0, u=5.0, x=2.0) >>> element2.model.parameters.control.responses responses(th_0_0=((1.298097, -0.536702, 0.072903, -0.001207, -0.00004), (0.699212, -0.663835, 0.093935, 0.046177, -0.00854)))
- add_iuhs(**iuhs: IUH) None[source]¶
Add one
IUHobject for each relevantElementobject.See the main documentation on class
ReplaceIUHfor further information.
- apply_value() None[source]¶
Apply all current calibration parameter values to all relevant
IUHobjects and eventually update the related parameter’s ARMA coefficients.See the main documentation on class
MultiplyIUHfor further information.
- class hydpy.auxs.calibtools.CalibSpec(*, name: str, default: float, keyword: None | None = None, lower: float = -inf, upper: float = inf, parameterstep: timetools.PeriodConstrArg | None = None)[source]¶
Bases:
objectHelper class for specifying the properties of a single calibration parameter.
So far, class
CalibSpecdoes not provide much functionality besides checking upon initialisation that the given default and boundary values are consistent:>>> from hydpy import CalibSpec >>> CalibSpec(name="par1", default=1.0) CalibSpec(name="par1", default=1.0)
>>> CalibSpec(name="par1", default=1.0, keyword="key1") CalibSpec(name="par1", default=1.0, keyword="key1")
>>> CalibSpec(name="par1", default=1.0, lower=2.0) Traceback (most recent call last): ... ValueError: The following values given for calibration parameter `par1` are not consistent: default=1.0, lower=2.0, upper=inf.
>>> CalibSpec(name="par1", default=1.0, upper=0.5) Traceback (most recent call last): ... ValueError: The following values given for calibration parameter `par1` are not consistent: default=1.0, lower=-inf, upper=0.5.
>>> CalibSpec(name="par1", default=1.0, lower=0.0, upper=2.0) CalibSpec(name="par1", default=1.0, lower=0.0, upper=2.0)
Use the parameterstep argument for time-dependent calibration parameters:
>>> CalibSpec(name="par1", default=1.0/3.0, lower=1.0/3.0, upper=1.0/3.0, ... parameterstep="1d") CalibSpec( name="par1", default=0.333333, lower=0.333333, upper=0.333333, parameterstep="1d" )
See the documentation on class
CalibSpecsfor further information.
- class hydpy.auxs.calibtools.CalibSpecs(*parspecs: CalibSpec)[source]¶
Bases:
objectCollection class for handling
CalibSpecobjects.The primary purpose of class
CalibSpecsis to handle multipleCalibSpecobjects and to make all their attributes accessible in the same order. See propertynamesas one example. Note that all such properties are sorted in the order or the attachment of the differentCalibSpecobjects:>>> from hydpy import CalibSpec, CalibSpecs >>> calibspecs = CalibSpecs( ... CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d" ... ), ... CalibSpec(name="second", default=1.0, keyword="kw2", lower=0.0), ... CalibSpec(name="first",default=2.0, upper=2.0)) >>> calibspecs CalibSpecs( CalibSpec(name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d"), CalibSpec(name="second", default=1.0, keyword="kw2", lower=0.0), CalibSpec(name="first", default=2.0, upper=2.0), )
You can query and remove
CalibSpecobjects via keyword and attribute access:>>> print(calibspecs) CalibSpecs("third", "second", "first")
>>> third = calibspecs["third"] >>> third in calibspecs True >>> del calibspecs["third"] >>> third in calibspecs False >>> calibspecs["third"] Traceback (most recent call last): ... KeyError: 'The current `CalibSpecs` object does not handle a `CalibSpec` object named `third`.' >>> del calibspecs["third"] Traceback (most recent call last): ... KeyError: 'The current `CalibSpecs` object does not handle a `CalibSpec` object named `third`.'
>>> second = calibspecs.second >>> "second" in calibspecs True >>> del calibspecs.second >>> "second" in calibspecs False >>> calibspecs.second Traceback (most recent call last): ... AttributeError: The current `CalibSpecs` object does neither handle a `CalibSpec` object nor a normal attribute named `second`. >>> del calibspecs.second Traceback (most recent call last): ... AttributeError: The current `CalibSpecs` object does not handle a `CalibSpec` object named `second`.
>>> len(calibspecs) 1
Now we can re-append the previously removed
CalibSpecobjects (and thereby bring the order of attachment in agreement with theCalibSpecnames):>>> calibspecs.append(second, third) >>> for calibspec in calibspecs: ... print(calibspec) first second third
- append(*calibspecs: CalibSpec) None[source]¶
Append one or more
CalibSpecobjects.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> first = CalibSpec(name="first", default=1.0, lower=0.0) >>> second = CalibSpec(name="second",default=2.0, keyword="kw2", upper=2.0) >>> calibspecs = CalibSpecs() >>> calibspecs.append(first) >>> calibspecs.append(second, third) >>> calibspecs CalibSpecs( CalibSpec(name="first", default=1.0, lower=0.0), CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0), CalibSpec(name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d"), )
- property names: tuple[str, ...]¶
The names of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs(CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second",default=2.0, upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.names ('first', 'second', 'third')
- property defaults: tuple[float, ...]¶
The default values of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs( ... CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.defaults (1.0, 2.0, 3.0)
- property keywords: tuple[str | None, ...]¶
The (optional) target keywords of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs( ... CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.keywords (None, 'kw2', None)
- property lowers: tuple[float, ...]¶
The lower boundary values of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs( ... CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.lowers (0.0, -inf, -10.0)
- property uppers: tuple[float, ...]¶
The upper boundary values of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs( ... CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.uppers (inf, 2.0, 10.0)
- property parametersteps: tuple[Period | None, ...]¶
The parameter steps of all
CalibSpecobjects in the order of attachment.>>> from hydpy import CalibSpec, CalibSpecs >>> third = CalibSpec( ... name="third", default=3.0, lower=-10.0, upper=10.0, parameterstep="1d") >>> calibspecs = CalibSpecs( ... CalibSpec(name="first", default=1.0, lower=0.0), ... CalibSpec(name="second", default=2.0, keyword="kw2", upper=2.0)) >>> calibspecs.append(third) >>> calibspecs.parametersteps (None, None, Period("1d"))
- hydpy.auxs.calibtools.make_rules(*, rule: type[TypeRule], calibspecs: CalibSpecs | None = None, names: Sequence[str] | None = None, parameters: Sequence[parametertools.Parameter | str] | None = None, values: Sequence[float] | None = None, keywords: Sequence[str | None] | None = None, lowers: Sequence[float] | None = None, uppers: Sequence[float] | None = None, parametersteps: Sequence1[timetools.PeriodConstrArg | None] = None, model: types.ModuleType | str | None = None, selections: Iterable[selectiontools.Selection | str] | None = None, product: bool = False) list[TypeRule][source]¶
Conveniently create multiple
Ruleobjects at once.Please see the main documentation on class
CalibrationInterfacefirst, from which we borrow the general setup:>>> from hydpy.core.testtools import prepare_full_example_2 >>> hp, pub, TestIO = prepare_full_example_2() >>> from hydpy import CalibrationInterface, make_rules, nse >>> ci = CalibrationInterface( ... hp=hp, ... targetfunction=lambda: sum(nse(node=node) for node in hp.nodes))
Here, we show only the supplemental features of function
make_rules()in some brevity.Function
make_rules()checks that all given sequences have the same length:>>> from hydpy import Replace >>> make_rules(rule=Replace, ... names=["fc", "percmax"], ... parameters=["fc", "percmax"], ... values=[100.0, 5.0], ... keywords=["forest", None], ... lowers=[50.0, 1.0], ... uppers=[200.0], ... parametersteps="1d", ... model="hland_96") Traceback (most recent call last): ... ValueError: When creating rules via function `make_rules`, all given sequences must be of equal length.
The separate handling of the specifications of all calibration parameters is error-prone. You can bundle all specifications within a
CalibSpecsobject instead and pass them at once for more safety and convenience:>>> from hydpy import CalibSpec, CalibSpecs >>> calibspecs = CalibSpecs( ... CalibSpec(name="fc", default=100.0, keyword="forest", lower=50.0, upper=200.0), ... CalibSpec(name="percmax", default=5.0, lower=1.0, upper=10.0, parameterstep="1d")) >>> make_rules(rule=Replace, ... calibspecs=calibspecs, ... parametersteps="1d", ... model="hland_96")[1] Replace( name="percmax", parameter="percmax", value=5.0, lower=1.0, upper=10.0, keyword=None, parameterstep="1d", model="hland_96", selections=("complete",), )
You are free also to use the individual arguments (e.g. names) to override the related specifications defined by the
CalibSpecsobject:>>> make_rules(rule=Replace, ... calibspecs=calibspecs, ... names=[name.upper() for name in calibspecs.names], ... parametersteps="1d", ... model="hland_96")[1] Replace( name="PERCMAX", parameter="percmax", value=5.0, lower=1.0, upper=10.0, keyword=None, parameterstep="1d", model="hland_96", selections=("complete",), )
Function
make_rules()raises the following error if you neither pass aCalibSpecsobject nor the complete list of individual calibration parameter specifications:>>> make_rules(rule=Replace, ... names=["fc", "percmax"], ... parameters=["fc", "percmax"], ... values=[100.0, 5.0], ... keywords=["forest", None], ... lowers=[50.0, 1.0], ... parametersteps="1d", ... model="hland_96") Traceback (most recent call last): ... TypeError: When creating rules via function `make_rules`, you must pass a `CalibSpecs` object or provide complete information for the following arguments: names, parameters, values, keywords, lowers, and uppers.
You can run function
make_rules()in “product mode”, meaning that its execution results in distinctRuleobjects for all combinations of the given calibration parameters and selections:>>> make_rules(rule=Replace, ... calibspecs=calibspecs, ... model="hland_96", ... selections=("headwaters", "nonheadwaters"), ... product=True) [Replace( name="fc_headwaters", parameter="fc", value=100.0, lower=50.0, upper=200.0, keyword="forest", parameterstep=None, model="hland_96", selections=("headwaters",), ), Replace( name="percmax_headwaters", parameter="percmax", value=5.0, lower=1.0, upper=10.0, keyword=None, parameterstep="1d", model="hland_96", selections=("headwaters",), ), Replace( name="fc_nonheadwaters", parameter="fc", value=100.0, lower=50.0, upper=200.0, keyword="forest", parameterstep=None, model="hland_96", selections=("nonheadwaters",), ), Replace( name="percmax_nonheadwaters", parameter="percmax", value=5.0, lower=1.0, upper=10.0, keyword=None, parameterstep="1d", model="hland_96", selections=("nonheadwaters",), )]
Trying to run in “product mode” without defining the target selections results in the following error message:
>>> make_rules(rule=Replace, ... calibspecs=calibspecs, ... parametersteps="1d", ... model="hland_96", ... product=True) Traceback (most recent call last): ... TypeError: When creating rules via function `make_rules` in "product mode" (with the argument `product` being `True`), you must supply all target selection objects via argument `selections`.