variabletools

This module implements general features for defining and working with model parameters and sequences.

Features more specific to either parameters or sequences are implemented in modules parametertools and sequencetools, respectively.

Module variabletools implements the following members:

hydpy.core.variabletools.INT_NAN: int = -999999

Surrogate for nan, which is available for floating-point values but not for integer values.

hydpy.core.variabletools.trim(self: Variable, lower=None, upper=None) → None[source]

Trim the value(s) of a Variable instance.

Usually, users do not need to apply the trim() function directly. Instead, some Variable subclasses implement their own trim methods relying on function trim(). Model developers should implement individual trim methods for their Parameter or Sequence_ subclasses when their boundary values depend on the actual project configuration (one example is soil moisture; its lowest possible value should possibly be zero in all cases, but its highest possible value could depend on another parameter defining the maximum storage capacity).

For the following examples, we prepare a simple (not fully functional) Variable subclass, making use of function trim() without any modifications. Function trim() works slightly different for variables handling float, int, and bool values. We start with the most common content type, float:

>>> from hydpy.core.variabletools import trim, Variable
>>> class Var(Variable):
...     NDIM = 0
...     TYPE = float
...     SPAN = 1.0, 3.0
...     trim = trim
...     initinfo = 2.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess

First, we enable the printing of warning messages raised by function trim():

>>> from hydpy import pub
>>> pub.options.warntrim = True

When not passing boundary values, function trim() extracts them from class attribute SPAN of the given Variable instance, if available:

>>> var = Var(None)
>>> var.value = 2.0
>>> var.trim()
>>> var
var(2.0)

>>> var.value = 0.0
>>> var.trim()
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `0.0` and `1.0`, respectively.
>>> var
var(1.0)

>>> var.value = 4.0
>>> var.trim()
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `4.0` and `3.0`, respectively.
>>> var
var(3.0)

In the examples above, outlier values are set to the respective boundary value, accompanied by suitable warning messages. For minimal deviations (defined by function get_tolerance()), which might be due to precision problems only, outliers are trimmed but not reported:

>>> var.value = 1.0 - 1e-15
>>> var == 1.0
False
>>> trim(var)
>>> var == 1.0
True

>>> var.value = 3.0 + 1e-15
>>> var == 3.0
False
>>> var.trim()
>>> var == 3.0
True

Use arguments lower and upper to override the (eventually) available SPAN entries:

>>> var.trim(lower=4.0)
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `3.0` and `4.0`, respectively.

>>> var.trim(upper=3.0)
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `4.0` and `3.0`, respectively.

Function trim() interprets both None and nan values as if no boundary value exists:

>>> import numpy
>>> var.value = 0.0
>>> var.trim(lower=numpy.nan)
>>> var.value = 5.0
>>> var.trim(upper=numpy.nan)

You can disable function trim() via option trimvariables:

>>> with pub.options.trimvariables(False):
...     var.value = 5.0
...     var.trim()
>>> var
var(5.0)

Alternatively, you can omit the warning messages only:

>>> with pub.options.warntrim(False):
...     var.value = 5.0
...     var.trim()
>>> var
var(3.0)

If a Variable subclass does not have (fixed) boundaries, give it either no SPAN attribute or a tuple containing None values:

>>> del Var.SPAN
>>> var.value = 5.0
>>> var.trim()
>>> var
var(5.0)

>>> Var.SPAN = (None, None)
>>> var.trim()
>>> var
var(5.0)

The above examples deal with a 0-dimensional Variable subclass. The following examples repeat the most relevant examples for a 2-dimensional subclass:

>>> Var.SPAN = 1.0, 3.0
>>> Var.NDIM = 2
>>> var.shape = 1, 3
>>> var.values = 2.0
>>> var.trim()

>>> var.values = 0.0, 1.0, 2.0
>>> var.trim()
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `0.0, 1.0, 2.0` and `1.0, 1.0, 2.0`, respectively.
>>> var
var(1.0, 1.0, 2.0)

>>> var.values = 2.0, 3.0, 4.0
>>> var.trim()
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `2.0, 3.0, 4.0` and `2.0, 3.0, 3.0`, respectively.
>>> var
var(2.0, 3.0, 3.0)

>>> var.values = 1.0-1e-15, 2.0, 3.0+1e-15
>>> var.values == (1.0, 2.0, 3.0)
array([[False,  True, False]])
>>> var.trim()
>>> var.values == (1.0, 2.0, 3.0)
array([[ True,  True,  True]])

>>> var.values = 0.0, 2.0, 4.0
>>> var.trim(lower=numpy.nan, upper=numpy.nan)
>>> var
var(0.0, 2.0, 4.0)

>>> var.trim(lower=[numpy.nan, 3.0, 3.0])
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `0.0, 2.0, 4.0` and `0.0, 3.0, 3.0`, respectively.

>>> var.values = 0.0, 2.0, 4.0
>>> var.trim(upper=[numpy.nan, 1.0, numpy.nan])
Traceback (most recent call last):
...
UserWarning: For variable `var` at least one value needed to be trimmed.  The old and the new value(s) are `0.0, 2.0, 4.0` and `1.0, 1.0, 4.0`, respectively.

For Variable subclasses handling float values, setting outliers to the respective boundary value might often be an acceptable approach. However, this is often not the case for subclasses handling int values, which often serve as option flags (e.g. to enable/disable a certain hydrological process for different land-use types). Hence, function trim() raises an exception instead of a warning and does not modify the wrong int value:

>>> Var.TYPE = int
>>> Var.NDIM = 0
>>> Var.SPAN = 1, 3

>>> var.value = 2
>>> var.trim()
>>> var
var(2)

>>> var.value = 0
>>> var.trim()
Traceback (most recent call last):
...
ValueError: The value `0` of parameter `var` of element `?` is not valid.
>>> var
var(0)
>>> var.value = 4
>>> var.trim()
Traceback (most recent call last):
...
ValueError: The value `4` of parameter `var` of element `?` is not valid.
>>> var
var(4)

>>> from hydpy import INT_NAN
>>> var.value = 0
>>> var.trim(lower=0)
>>> var.trim(lower=INT_NAN)

>>> var.value = 4
>>> var.trim(upper=4)
>>> var.trim(upper=INT_NAN)

>>> Var.SPAN = 1, None
>>> var.value = 0
>>> var.trim()
Traceback (most recent call last):
...
ValueError: The value `0` of parameter `var` of element `?` is not valid.
>>> var
var(0)

>>> Var.SPAN = None, 3
>>> var.value = 0
>>> var.trim()
>>> var.value = 4
>>> var.trim()
Traceback (most recent call last):
...
ValueError: The value `4` of parameter `var` of element `?` is not valid.

>>> del Var.SPAN
>>> var.value = 0
>>> var.trim()
>>> var.value = 4
>>> var.trim()

>>> Var.SPAN = 1, 3
>>> Var.NDIM = 2
>>> var.shape = (1, 3)
>>> var.values = 2
>>> var.trim()

>>> var.values = 0, 1, 2
>>> var.trim()
Traceback (most recent call last):
...
ValueError: At least one value of parameter `var` of element `?` is not valid.
>>> var
var(0, 1, 2)
>>> var.values = 2, 3, 4
>>> var.trim()
Traceback (most recent call last):
 ...
ValueError: At least one value of parameter `var` of element `?` is not valid.
>>> var
var(2, 3, 4)

>>> var.values = 0, 0, 2
>>> var.trim(lower=[0, INT_NAN, 2])

>>> var.values = 2, 4, 4
>>> var.trim(upper=[2, INT_NAN, 4])

For bool values, defining outliers does not make much sense, which is why function trim() does nothing when applied to variables handling bool values:

>>> Var.TYPE = bool
>>> var.trim()

If function trim() encounters an unmanageable type, it raises an exception like the following:

>>> Var.TYPE = str
>>> var.trim()
Traceback (most recent call last):
...
NotImplementedError: Method `trim` can only be applied on parameters handling floating-point, integer, or boolean values, but the "value type" of parameter `var` is `str`.

>>> pub.options.warntrim = False
hydpy.core.variabletools.get_tolerance(values)[source]

Return some “numerical accuracy” to be expected for the given floating-point value(s).

The documentation on function trim() explains also function get_tolerance(). However, note the special case of infinite input values, for which function get_tolerance() returns zero:

>>> from hydpy.core.variabletools import get_tolerance
>>> import numpy
>>> get_tolerance(numpy.inf)
0.0
>>> from hydpy import round_
>>> round_(get_tolerance(
...     numpy.array([1.0, numpy.inf, 2.0, -numpy.inf])), 16)
0.000000000000001, 0.0, 0.000000000000002, 0.0
class hydpy.core.variabletools.FastAccess[source]

Bases: object

Used as a surrogate for typed Cython classes handling parameters or sequences when working in pure Python mode.

class hydpy.core.variabletools.Variable(subvars: SubVariables)[source]

Bases: object

Base class for Parameter and Sequence_.

The subclasses are required to provide the class attributes NDIM and TYPE, defining the dimensionality and the type of the values to be handled by the subclass, respectively. Class attribute INIT is optional and should provide a suitable default value.

Class Variable implements methods for arithmetic calculations, comparisons and type conversions. See the following examples on how to do math with HydPys Parameter and Sequence_ objects.

We start with demonstrating the supported mathematical operations on 0-dimensional Variable objects handling float values:

>>> import numpy
>>> from hydpy.core.variabletools import FastAccess, Variable
>>> class Var(Variable):
...     NDIM = 0
...     TYPE = float
...     initinfo = 0.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess
>>> var = Var(None)

You can perform additions both with other Variable objects and with ordinary number objects:

>>> var.value = 2.0
>>> var + var
4.0
>>> var + 3.0
5.0
>>> 4.0 + var
6.0
>>> var += 1
>>> var
var(3.0)
>>> var += -1.0
>>> var
var(2.0)

In case something went wrong, all math operations return errors like the following:

>>> var = Var(None)
>>> var + 1.0
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: While trying to add variable `var` and `float` instance `1.0`, the following error occurred: For variable `var`, no value has been defined so far.

In general, the examples above are valid for the following binary operations:

>>> var.value = 3.0
>>> var - 1
2.0
>>> 7.0 - var
4.0
>>> var -= 2.0
>>> var
var(1.0)

>>> var.value = 2.0
>>> var * 3
6.0
>>> 4.0 * var
8.0
>>> var *= 0.5
>>> var
var(1.0)

>>> var.value = 3.0
>>> var / 2
1.5
>>> 7.5 / var
2.5
>>> var /= 6.0
>>> var
var(0.5)

>>> var.value = 3.0
>>> var // 2
1.0
>>> 7.5 // var
2.0
>>> var //= 0.9
>>> var
var(3.0)

>>> var.value = 5.0
>>> var % 2
1.0
>>> 7.5 % var
2.5
>>> var %= 3.0
>>> var
var(2.0)

>>> var.value = 2.0
>>> var**3
8.0
>>> 3.0**var
9.0
>>> var **= 4.0
>>> var
var(16.0)

>>> var.value = 5.0
>>> divmod(var, 3)
(1.0, 2.0)
>>> divmod(13.0, var)
(2.0, 3.0)

Additionally, we support the following unary operations:

>>> var.values = -5.0
>>> +var
-5.0
>>> -var
5.0
>>> abs(var)
5.0
>>> ~var
-0.2
>>> var.value = 2.5
>>> import math
>>> math.floor(var)
2
>>> math.ceil(var)
3
>>> bool(var)
True
>>> int(var)
2
>>> float(var)
2.5
>>> var.value = 1.67
>>> round(var, 1)
1.7

You can apply all the operations discussed above (except float and int) on Variable objects of arbitrary dimensionality:

>>> Var.NDIM = 1
>>> Var.TYPE = float
>>> var.shape = (2,)
>>> var.values = 2.0
>>> var + var
array([4., 4.])
>>> var + 3.0
array([5., 5.])
>>> [4.0, 0.0] + var
array([6., 2.])
>>> var += 1
>>> var
var(3.0, 3.0)

>>> var.values = 3.0
>>> var - [1.0, 0.0]
array([2., 3.])
>>> [7.0, 0.0] - var
array([ 4., -3.])
>>> var -= [2.0, 0.0]
>>> var
var(1.0, 3.0)

>>> var.values = 2.0
>>> var * [3.0, 1.0]
array([6., 2.])
>>> [4.0, 1.0] * var
array([8., 2.])
>>> var *= [0.5, 1.0]
>>> var
var(1.0, 2.0)

>>> var.values = 3.0
>>> var / [2.0, 1.0]
array([1.5, 3. ])
>>> [7.5, 3.0] / var
array([2.5, 1. ])
>>> var /= [6.0, 1.]
>>> var
var(0.5, 3.0)

>>> var.values = 3.0
>>> var // [2.0, 1.0]
array([1., 3.])
>>> [7.5, 3.0] // var
array([2., 1.])
>>> var //= [0.9, 1.0]
>>> var
var(3.0, 3.0)

>>> var.values = 5.0
>>> var % [2.0, 5.0]
array([1., 0.])
>>> [7.5, 5.0] % var
array([2.5, 0. ])
>>> var %= [3.0, 5.0]
>>> var
var(2.0, 0.0)

>>> var.values = 2.0
>>> var**[3.0, 1.0]
array([8., 2.])
>>> [3.0, 1.0]**var
array([9., 1.])
>>> var **= [4.0, 1.0]
>>> var
var(16.0, 2.0)

>>> var.value = 5.0
>>> divmod(var, [3.0, 5.0])
(array([1., 1.]), array([2., 0.]))
>>> divmod([13.0, 5.0], var)
(array([2., 1.]), array([3., 0.]))

>>> var.values = -5.0
>>> +var
array([-5., -5.])
>>> -var
array([5., 5.])
>>> abs(var)
array([5., 5.])
>>> ~var
array([-0.2, -0.2])
>>> var.value = 2.5
>>> import math
>>> math.floor(var)
array([2, 2])
>>> math.ceil(var)
array([3, 3])
>>> var.values = 1.67
>>> round(var, 1)
array([1.7, 1.7])
>>> bool(var)
True
>>> int(var)
Traceback (most recent call last):
...
TypeError: The variable `var` is 1-dimensional and thus cannot be converted to a scalar int value.
>>> float(var)
Traceback (most recent call last):
...
TypeError: The variable `var` is 1-dimensional and thus cannot be converted to a scalar float value.

Indexing is supported (for consistency reasons, even for 0-dimensional variables):

>>> Var.NDIM = 0
>>> var.value = 5.0
>>> var[0] += var[0]
>>> var[:]
10.0
>>> var[1]
Traceback (most recent call last):
...
IndexError: While trying to access the value(s) of variable `var` with key `1`, the following error occurred: The only allowed keys for 0-dimensional variables are `0` and `:`.

>>> Var.NDIM = 1
>>> var = Var(None)
>>> var.shape = (5,)
>>> var.value = 2.0, 4.0, 6.0, 8.0, 10.0
>>> var[0]
2.0
>>> var[-1]
10.0
>>> var[1:-1:2] = 2.0 * var[1:-1:2]
>>> var
var(2.0, 8.0, 6.0, 16.0, 10.0)
>>> var[:] = "test"
Traceback (most recent call last):
...
ValueError: While trying to set the value(s) of variable `var` with key `slice(None, None, None)`, the following error occurred: could not convert string to float: 'test'

Comparisons with Variable objects containing multiple values return a single boolean value. Two objects are equal if all of their value pairs are equal, and they are unequal if at least one of their value pairs is unequal:

>>> var.shape = (2,)
>>> var.values = 1.0, 3.0
>>> var == [0.0, 2.0], var == [1.0, 2.0], var == [1.0, 3.0]
(False, False, True)
>>> var != [0.0, 2.0], var != [1.0, 2.0], var != [1.0, 3.0]
(True, True, False)

While either the == or the != operator returns True (but not both), this must not be the case for the operator pairs <`and `>= as well as > and <=:

>>> var < 2.0, var < 3.0, var < 4.0
(False, False, True)
>>> var <= 2.0, var <= 3.0, var <= 4.0
(False, True, True)
>>> var >= 0.0, var >= 1.0, var >= 2.0
(True, True, False)
>>> var > 0.0, var > 1.0, var > 2.0
(True, False, False)

Comparing wrongly shaped values does work for == and != but results in errors for the other operations:

>>> var.values = 2.0
>>> var == [2.0], var != [2.0]
(True, False)
>>> var == [2.0, 2.0, 2.0], var != [2.0, 2.0, 2.0]
(False, True)
>>> var < [2.0], var <= [2.0], var >= [2.0], var > [2.0]
(False, True, True, False)
>>> var < [2.0, 2.0, 2.0]   
Traceback (most recent call last):
...
ValueError: While trying to compare variable `var` of element `?` with object `[2.0, 2.0, 2.0]` of type `list`, the following error occurred: operands could not be broadcast together with shapes (2,) (3,)...

You can compare different Variable objects directly with each other:

>>> from copy import deepcopy
>>> var < var, var < deepcopy(var)
(False, False)
>>> var <= var, var <= deepcopy(var)
(True, True)
>>> var == var, var == deepcopy(var)
(True, True)
>>> var != var, var != deepcopy(var)
(False, False)
>>> var >= var, var >= deepcopy(var)
(True, True)
>>> var > var, var > deepcopy(var)
(False, False)

When asking for impossible comparisons, trim() raises error like the following:

>>> var < "text"
Traceback (most recent call last):
...
TypeError: While trying to compare variable `var` of element `?` with object `text` of type `str`, the following error occurred: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Note that, in contrast to the usual numpy array comparison, we ignore all single comparison results between two nan values:

>>> from numpy import nan
>>> var.shape = (3,)
>>> var.values = 1.0, 2.0, nan
>>> var < [2.0, 3.0, nan], var < [1.0, 2.0, nan], var < [2.0, nan, nan], var < [2.0, 3.0, 4.0]
(True, False, False, False)
>>> var <= [1.0, 3.0, nan], var <= [1.0, 1.0, nan], var <= [1.0, nan, nan], var <= [1.0, 3.0, 5.0]
(True, False, False, False)
>>> var == [1.0, 2.0, nan], var == [1.0, 1.0, nan], var == [1.0, nan, nan], var == [1.0, 2.0, 3.0]
(True, False, False, False)
>>> var != [1.0, 1.0, nan], var != [1.0, 2.0, nan], var != [1.0, nan, nan], var != [1.0, 2.0, 3.0]
(True, False, True, True)
>>> var >= [1.0, 1.0, nan], var >= [1.0, 3.0, nan], var <= [1.0, nan, nan], var <= [1.0, 3.0, 5.0]
(True, False, False, False)
>>> var > [0.0, 1.0, nan], var > [0.0, 2.0, nan], var < [0.0, nan, nan], var < [0.0, 1.0, 2.0]
(True, False, False, False)

Hence, when all entries of two compared objects are nan, we consider these objects equal:

>>> var.values = nan
>>> var < [nan, nan, nan], var <= [nan, nan, nan], var == [nan, nan, nan], var != [nan, nan, nan], var >= [nan, nan, nan], var > [nan, nan, nan]
(False, True, True, False, True, False)
>>> Var.NDIM = 0
>>> var = Var(None)
>>> var.shape = ()
>>> var.value = nan
>>> var < nan, var <= nan, var == nan, var != nan, var >= nan, var > nan
(False, True, True, False, True, False)

The len() operator does not work for 0-dimensional variables:

>>> Var.NDIM = 0
>>> var = Var(None)
>>> var.shape = ()
>>> len(var)
Traceback (most recent call last):
...
TypeError: The `len` operator was applied on `var`, but this variable is 0-dimensional and thus unsized.  Consider using the `numberofvalues` property instead.

For higher-dimensional variables, len always returns the length of the first dimension:

>>> Var.NDIM = 1
>>> var = Var(None)
>>> var.shape = (5,)
>>> len(var)
5
>>> Var.NDIM = 3
>>> var = Var(None)
>>> var.shape = (2, 1, 4)
>>> len(var)
2

Variable objects are hashable based on their id() value to avoid avoiding confusion when adding different but equal objects into one set or dict object. The following examples show this behaviour by making deep copies of existing Variable objects:

>>> Var.NDIM = 0
>>> var1 = Var(None)
>>> var1.value = 5.0
>>> varset = set([var1])
>>> var1 in varset
True
>>> var1.value = 7.0
>>> var1 in varset
True
>>> var2 = deepcopy(var1)
>>> var1 == var2
True
>>> var2 in varset
False

>>> Var.NDIM = 1
>>> var1 = Var(None)
>>> var1.shape = (2,)
>>> var1.value = 3.0, 5.0
>>> varset = set([var1])
>>> var1 in varset
True
>>> var1[1] = 7.0
>>> var1 in varset
True
>>> var2 = deepcopy(var1)
>>> var1 == var2
True
>>> var2 in varset
False

Enabling option reprcomments adds the respective docstring header to the string representation of a variable:

>>> Var.NDIM = 0
>>> Var.__doc__ = "header.\n\nbody\n"
>>> var = Var(None)
>>> var.value = 3.0
>>> from hydpy import pub
>>> pub.options.reprcomments = True
>>> var
# header.
var(3.0)

>>> pub.options.reprcomments = False
>>> var
var(3.0)

During initialisation, each Variable subclass tries to extract its unit from its docstring:

>>> type("Var", (Variable,), {"__doc__": "Discharge [m³/s]."}).unit
'm³/s'

For missing or poorly written docstrings, we set unit to “?”:

>>> type("Var", (Variable,), {}).unit
'?'
>>> type("Var", (Variable,), {"__doc__": "Discharge ]m³/s[."}).unit
'?'
>>> type("Var", (Variable,), {"__doc__": "Discharge m³/s]."}).unit
'?'
NDIM: int
TYPE: Type
SPAN: Tuple[int | float | bool | None, int | float | bool | None] = (None, None)
INIT: int | float | bool | None = None
NOT_DEEPCOPYABLE_MEMBERS: Tuple[str, str, str, str] = ('subvars', 'subpars', 'subseqs', 'fastaccess')
strict_valuehandling: bool = True
name: str

Name of the variable in lower case letters.

unit: str

Unit of the variable.

mask
abstract property initinfo: Tuple[float | int | bool | pointerutils.Double, bool]

To be overridden.

property value

The actual parameter or sequence value(s).

First, we prepare a simple (not fully functional) Variable subclass:

>>> from hydpy.core.variabletools import Variable
>>> class Var(Variable):
...     NDIM = 0
...     TYPE = float
...     initinfo = 3.0, True
...     _CLS_FASTACCESS_PYTHON = FastAccess

Without making use of default values (see below), trying to query the actual value of a freshly initialised Variable object results in the following error:

>>> var = Var(None)
>>> var.value
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: For variable `var`, no value has been defined so far.

Property value tries to normalise assigned values and raises an error, if not possible:

>>> var.value = 3
>>> var.value
3.0

>>> var.value = ["2.0"]
>>> var.value
2.0

>>> var.value = 1.0, 1.0
Traceback (most recent call last):
...
ValueError: While trying to set the value(s) of variable `var`, the following error occurred: 2 values are assigned to the scalar variable `var`.
>>> var.value
2.0

>>> var.value = "O"
Traceback (most recent call last):
...
TypeError: While trying to set the value(s) of variable `var`, the following error occurred: The given value `O` cannot be converted to type `float`.
>>> var.value
2.0

The above examples deal with a 0-dimensional variable handling float values. The following examples focus on a 2-dimensional variable handling int values:

>>> from hydpy import INT_NAN
>>> Var.NDIM = 2
>>> Var.TYPE = int
>>> Var.initinfo = INT_NAN, False

For multidimensional objects, assigning new values required defining their shape first:

>>> var = Var(None)
>>> var.value
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: Shape information for variable `var` can only be retrieved after it has been defined.

>>> var.value = 2
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: While trying to set the value(s) of variable `var`, the following error occurred: Shape information for variable `var` can only be retrieved after it has been defined.

>>> var.shape = (2, 3)
>>> var.value
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: For variable `var`, no values have been defined so far.

>>> var.value = 2
>>> var.value
array([[2, 2, 2],
       [2, 2, 2]])

>>> var.value = 1, 2
Traceback (most recent call last):
...
ValueError: While trying to set the value(s) of variable `var`, the following error occurred: While trying to convert the value(s) `(1, 2)` to a numpy ndarray with shape `(2, 3)` and type `int`, the following error occurred: could not broadcast input array from shape (2,) into shape (2,3)
>>> var.value
array([[2, 2, 2],
       [2, 2, 2]])

>>> var.shape = (0, 0)
>>> var.shape
(0, 0)
>>> var.value   
array([], shape=(0, 0), dtype=int...)
property values

Alias for value.

property shape: Tuple[int, ...]

A tuple containing the actual lengths of all dimensions.

Note that setting a new shape results in a loss of the actual values of the respective Variable object.

First, we prepare a simple (not fully functional) Variable subclass:

>>> from hydpy.core.variabletools import Variable
>>> class Var(Variable):
...     NDIM = 1
...     TYPE = float
...     initinfo = 3.0, True
...     _CLS_FASTACCESS_PYTHON = FastAccess

Initially, the shape of a new Variable object is unknown:

>>> var = Var(None)
>>> var.shape
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: Shape information for variable `var` can only be retrieved after it has been defined.

For multidimensional objects, assigning shape information (as a tuple of int values) prepares the required array automatically. Due to the initinfo surrogate of our test class, the entries of this array are 3.0:

>>> var.shape = (3,)
>>> var.shape
(3,)
>>> var.values
array([3., 3., 3.])

For the initinfo flag (second tuple entry) being False, the array is still prepared but not directly accessible to the user:

>>> import numpy
>>> Var.initinfo = numpy.nan, False
>>> var = Var(None)

>>> var.shape = (3,)
>>> var.shape
(3,)
>>> var.values
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: For variable `var`, no values have been defined so far.

>>> var.fastaccess.var
array([nan, nan, nan])

Property shape tries to normalise assigned values and raises errors like the following, if not possible:

>>> var.shape = "x"
Traceback (most recent call last):
...
TypeError: While trying create a new numpy ndarray for variable `var`, the following error occurred: 'str' object cannot be interpreted as an integer
>>> from hydpy import attrready
>>> attrready(var, "shape")
False
>>> var.fastaccess.var

>>> var.shape = (1,)
>>> attrready(var, "shape")
True

>>> var.shape = (2, 3)
Traceback (most recent call last):
...
ValueError: Variable `var` is 1-dimensional, but the given shape indicates `2` dimensions.
>>> attrready(var, "shape")
False
>>> var.fastaccess.var

0-dimensional Variable objects inform the user about their shape but do not allow to change it for obvious reasons:

>>> class Var(Variable):
...     NDIM = 0
...     TYPE = int
...     initinfo = 3, True
...     _CLS_FASTACCESS_PYTHON = FastAccess

>>> var = Var(None)
>>> var.shape
()
>>> var.value
Traceback (most recent call last):
...
hydpy.core.exceptiontools.AttributeNotReady: For variable `var`, no value has been defined so far.

>>> var.shape = ()
>>> var.shape
()
>>> var.value
3
>>> var.shape = (2,)
Traceback (most recent call last):
...
ValueError: The shape information of 0-dimensional variables as `var` can only be `()`, but `(2,)` is given.

With a False initinfo flag, the default value is still readily prepared after initialisation but not directly accessible to the user:

>>> from hydpy import INT_NAN
>>> Var.initinfo = INT_NAN, False
>>> var = Var(None)
>>> var.shape
()
>>> var.shape = ()
>>> attrready(var, "value")
False
>>> var.fastaccess.var
-999999

>>> var.value = 6
>>> var.value
6

>>> var.shape = ()
>>> var.fastaccess.var
-999999
property numberofvalues: int

The total number of values handles by the variable according to the current shape.

We create an incomplete Variable subclass for testing:

>>> from hydpy.core.variabletools import FastAccess, Variable
>>> class Var(Variable):
...     TYPE = float
...     initinfo = 0.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess
>>> var = Var(None)

0-dimensional variables always handle precisely one value:

>>> Var.NDIM = 0
>>> var = Var(None)
>>> var.shape = ()
>>> var.numberofvalues
1

For higher-dimensional variables, numberofvalues is the cumulative product of the individual dimensons lengths:

>>> Var.NDIM = 1
>>> var = Var(None)
>>> var.shape = (5,)
>>> var.numberofvalues
5
>>> Var.NDIM = 3
>>> var = Var(None)
>>> var.shape = (2, 1, 4)
>>> var.numberofvalues
8

As long as the shape of a higher-dimensional variable is undefined, numberofvalues is zero:

>>> var = Var(None)
>>> var.numberofvalues
0
verify() → None[source]

Raise a RuntimeError if at least one of the required values of a Variable object is None or nan.

The descriptor mask() defines which values are considered necessary. For Variable subclasses defining nan as their INIT value, method verify() assumes that nan are not problematic.

Examples on a 0-dimensional Variable:

>>> from hydpy.core.variabletools import Variable
>>> class Var(Variable):
...     NDIM = 0
...     TYPE = float
...     initinfo = 0.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess
>>> var = Var(None)
>>> import numpy
>>> var.shape = ()
>>> var.value = 1.0
>>> var.verify()
>>> var.value = numpy.nan
>>> var.verify()
Traceback (most recent call last):
...
RuntimeError: For variable `var`, 1 required value has not been set yet: var(nan).

>>> var.INIT = numpy.nan
>>> var.verify()

Examples on a 2-dimensional Variable:

>>> Var.NDIM = 2
>>> var = Var(None)
>>> var.shape = (2, 3)
>>> var.value = numpy.ones((2,3))
>>> var.value[:, 1] = numpy.nan
>>> var.verify()
Traceback (most recent call last):
...
RuntimeError: For variable `var`, 2 required values have not been set yet: var([[1.0, nan, 1.0], [1.0, nan, 1.0]]).

>>> Var.mask = var.mask
>>> Var.mask[0, 1] = False
>>> var.verify()
Traceback (most recent call last):
...
RuntimeError: For variable `var`, 1 required value has not been set yet: var([[1.0, nan, 1.0], [1.0, nan, 1.0]]).

>>> Var.mask[1, 1] = False
>>> var.verify()
property valuevector: Vector

The values of the actual Variable object, arranged in a 1-dimensional vector.

For a 1-dimensional variable object, property valuevector returns the original values without any modification:

>>> from hydpy.models.hland import *
>>> simulationstep("1d")
>>> parameterstep("1d")
>>> nmbzones(3)
>>> sclass(2)
>>> states.sm.values = 1.0, 2.0, 3.0
>>> from hydpy import print_values
>>> print_values(states.sm.valuevector)
1.0, 2.0, 3.0

For all other variables, valuevector raises the following error by default:

>>> states.uz.valuevector
Traceback (most recent call last):
...
NotImplementedError: Variable `uz` does not implement a method for converting its values to a 1-dimensional vector.

If considered appropriate, model developers should override valuevector for individual multidimensional variables, to support methods like average_values(), which rely on 1-dimensional data. One example is the state sequence SP of base model hland, which handles values for individual zones (second axis) and snow classes (first axis). Here we decided to let valuevector return the sums of all snow classes for each zone so that the content of the returned vector agrees with the contents of most 1-dimensional sequences of hland:

>>> states.sp = [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
>>> print_values(states.sp.valuevector)
2.5, 3.5, 4.5
property refweights: parametertools.Parameter | Vector[float]

Reference to a Parameter object or a simple vector that defines weighting coefficients (e.g. fractional areas) for applying function average_values(). Must be overwritten by subclasses when required.

average_values(*args, **kwargs) → float[source]

Average the actual values of the Variable object.

For 0-dimensional Variable objects, the result of method average_values() equals value. The following example shows this for the poorly defined class SoilMoisture:

>>> from hydpy.core.variabletools import Variable
>>> class SoilMoisture(Variable):
...     NDIM = 0
...     TYPE = float
...     refweigths = None
...     availablemasks = None
...     initinfo = None
...     _CLS_FASTACCESS_PYTHON = FastAccess
...     value = 200.0
>>> sm = SoilMoisture(None)
>>> sm.average_values()
200.0

When the dimensionality of this class is increased to one, applying method average_values() results in the following error:

>>> SoilMoisture.NDIM = 1
>>> import numpy
>>> SoilMoisture.shape = (3,)
>>> SoilMoisture.value = numpy.array([200.0, 400.0, 500.0])
>>> sm.average_values()
Traceback (most recent call last):
...
AttributeError: While trying to calculate the mean value of variable `soilmoisture`, the following error occurred: Variable `soilmoisture` does not define any weighting coefficients.

So model developers have to define another (in this case 1-dimensional) Variable subclass (usually a Parameter subclass) and make the relevant object available via property refweights:

>>> class Area(Variable):
...     NDIM = 1
...     shape = (3,)
...     value = numpy.array([1.0, 1.0, 2.0])
...     initinfo = None
...     _CLS_FASTACCESS_PYTHON = FastAccess
>>> area = Area(None)
>>> SoilMoisture.refweights = property(lambda self: area)
>>> sm.average_values()
400.0

In the examples above, all single entries of values are relevant, which is the default case. However, subclasses of Variable can define an alternative mask, allowing to make some entries irrelevant. Assume, for example, that our SoilMoisture object contains three single values, each one associated with a specific hydrological response unit (hru). To indicate that soil moisture is undefined for the third unit (maybe because it is a water area), we set the third entry of the verification mask to False:

>>> from hydpy.core.masktools import DefaultMask
>>> class Soil(DefaultMask):
...     @classmethod
...     def new(cls, variable, **kwargs):
...         return cls.array2mask([True, True, False])
>>> SoilMoisture.mask = Soil()
>>> sm.average_values()
300.0

Alternatively, method average_values() accepts additional masking information as positional or keyword arguments. Therefore, the corresponding model must implement some alternative masks, which are provided by property availablemasks. We mock this property with a new Masks object, handling one mask for flat soils (only the first hru), one mask for deep soils (only the second hru), and one mask for water areas (only the third hru):

>>> class FlatSoil(DefaultMask):
...     @classmethod
...     def new(cls, variable, **kwargs):
...         return cls.array2mask([True, False, False])
>>> class DeepSoil(DefaultMask):
...     @classmethod
...     def new(cls, variable, **kwargs):
...         return cls.array2mask([False, True, False])
>>> class Water(DefaultMask):
...     @classmethod
...     def new(cls, variable, **kwargs):
...         return cls.array2mask([False, False, True])
>>> from hydpy.core import masktools
>>> class Masks(masktools.Masks):
...     CLASSES = (FlatSoil,
...                DeepSoil,
...                Water)
>>> SoilMoisture.availablemasks = Masks()

One can pass either the mask classes themselves or their names:

>>> sm.average_values(sm.availablemasks.flatsoil)
200.0
>>> sm.average_values("deepsoil")
400.0

Both variants can be combined:

>>> sm.average_values(sm.availablemasks.deepsoil, "flatsoil")
300.0

The following error happens if the general mask of the variable does not contain the given masks:

>>> sm.average_values("flatsoil", "water")
Traceback (most recent call last):
...
ValueError: While trying to calculate the mean value of variable `soilmoisture`, the following error occurred: Based on the arguments `('flatsoil', 'water')` and `{}` the mask `CustomMask([ True, False,  True])` has been determined, which is not a submask of `Soil([ True,  True, False])`.

Applying masks with custom options is also supported. One can change the behaviour of the following mask via the argument complete:

>>> class AllOrNothing(DefaultMask):
...     @classmethod
...     def new(cls, variable, complete):
...         if complete:
...             bools = [True, True, True]
...         else:
...             bools = [False, False, False]
...         return cls.array2mask(bools)
>>> class Masks(Masks):
...     CLASSES = (FlatSoil,
...                DeepSoil,
...                Water,
...                AllOrNothing)
>>> SoilMoisture.availablemasks = Masks()

Again, one can apply the mask class directly (but note that one has to pass the relevant variable as the first argument):

>>> sm.average_values(   
...     sm.availablemasks.allornothing(sm, complete=True))
Traceback (most recent call last):
...
ValueError: While trying to...

Alternatively, one can pass the mask name as a keyword and pack the mask’s options into a dict object:

>>> sm.average_values(allornothing={"complete": False})
nan

You can combine all variants explained above:

>>> sm.average_values("deepsoil", flatsoil={}, allornothing={"complete": False})
300.0
property availablemasks: Masks

For ModelSequence objects, a Masks object provided by the corresponding Model object; for NodeSequence object, a suitable DefaultMask.

>>> from hydpy.examples import prepare_full_example_2
>>> hp, pub, TestIO = prepare_full_example_2()

>>> hp.elements["land_dill"].model.parameters.control.fc.availablemasks
complete of module hydpy.models.hland.hland_masks
land of module hydpy.models.hland.hland_masks
upperzone of module hydpy.models.hland.hland_masks
snow of module hydpy.models.hland.hland_masks
soil of module hydpy.models.hland.hland_masks
field of module hydpy.models.hland.hland_masks
forest of module hydpy.models.hland.hland_masks
ilake of module hydpy.models.hland.hland_masks
glacier of module hydpy.models.hland.hland_masks
sealed of module hydpy.models.hland.hland_masks
noglacier of module hydpy.models.hland.hland_masks

>>> hp.nodes.dill.sequences.sim.availablemasks
defaultmask of module hydpy.core.masktools
get_submask(*args, **kwargs) → CustomMask[source]

Get a sub-mask of the mask handled by the actual Variable object based on the given arguments.

See the documentation on method average_values() for further information.

property commentrepr: List[str]

A list with comments for making string representations more informative.

With the reprcomments option disabled, commentrepr is empty.

class hydpy.core.variabletools.MixinFixedShape[source]

Bases: object

Mixin class for defining variables with a fixed shape.

SHAPE: Tuple[int, ...]
property shape: Tuple[int, ...]

Variables that mix in MixinFixedShape are generally initialised with a fixed shape.

We take parameter BV of base model lstream and sequence WaterLevel of base model exch as examples:

>>> from hydpy import prepare_model
>>> prepare_model("lstream").parameters.control.bv.shape
(2,)
>>> waterlevel = prepare_model("exch").sequences.factors.waterlevel
>>> waterlevel.shape
(2,)

If we try to set a new shape, MixinFixedShape responds with the following exceptions:

>>> waterlevel.shape = 2
Traceback (most recent call last):
...
AttributeError: The shape of variable `waterlevel` cannot be changed but this was attempted for element `?`.

See the documentation on property shape of class Variable for further information.

hydpy.core.variabletools.sort_variables(values: Iterable[Type[TypeVariable_co] | Tuple[Type[TypeVariable_co], T]]) → Tuple[Type[TypeVariable_co] | Tuple[Type[TypeVariable_co], T], ...][source]

Sort the given Variable subclasses by their initialisation order.

When defined in one module, the initialisation order corresponds to the order within the file:

>>> from hydpy import classname, sort_variables
>>> from hydpy.models.hland.hland_control import Area, NmbZones, ZoneType
>>> from hydpy import classname
>>> for var in sort_variables([NmbZones, ZoneType, Area]):
...     print(classname(var))
Area
NmbZones
ZoneType

When defined in multiple modules, alphabetical sorting of the modules’ filepaths takes priority:

>>> from hydpy.models.lland.lland_control import NHRU, Albedo
>>> for var in sort_variables([NmbZones, ZoneType, Area, NHRU, Albedo]):
...     print(classname(var))
Area
NmbZones
ZoneType
NHRU
Albedo

Function sort_variables() also supports sorting tuples. Each first entry must be a Variable subclass:

>>> for var, idx in sort_variables([(NmbZones, 1), (ZoneType, 2), (Area, 3)]):
...     print(classname(var), idx)
Area 3
NmbZones 1
ZoneType 2

>>> for var, idx in sort_variables([(NmbZones, 1), (ZoneType, 2), (Area, 3)]):
...     print(classname(var), idx)
Area 3
NmbZones 1
ZoneType 2

sort_variables() does not remove duplicates:

>>> for var, idx in sort_variables([(Area, 3), (ZoneType, 2), (Area, 1), (Area, 3)]):
...     print(classname(var), idx)
Area 1
Area 3
Area 3
ZoneType 2
class hydpy.core.variabletools.SubVariables(master: TypeGroup_co, cls_fastaccess: Type[TypeFastAccess_co] | None = None)[source]

Bases: Generic[TypeGroup_co, TypeVariable_co, TypeFastAccess_co]

Base class for SubParameters and SubSequences.

Each subclass of class SubVariables is thought for handling a certain group of Parameter or Sequence_ objects. One specific example is subclass InputSequences, collecting all InputSequence objects of a specific hydrological model.

For the following examples, we first prepare a (not fully functional) Variable subclass:

>>> from hydpy.core.variabletools import FastAccess, SubVariables, Variable
>>> class TestVar(Variable):
...     NDIM = 0
...     TYPE = float
...     initinfo = 0.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess

Out test SubVariables subclass is thought to handle only this single Variable subclass, indicated by putting it into the tuple class attribute CLASSES:

>>> class SubVars(SubVariables):
...     CLASSES = (TestVar,)
...     name = "subvars"
...     _CLS_FASTACCESS_PYTHON = FastAccess

After initialisation, SubVariables objects reference their master object (either a Parameters or a Sequences object), passed to their constructor. However, in our simple test example, we just passed a string instead:

>>> subvars = SubVars("test")
>>> subvars.vars
'test'

The string representation lists all available variables and, with the option reprcomments enabled, an additional informative header:

>>> subvars
testvar(?)
>>> from hydpy import pub
>>> pub.options.reprcomments = True
>>> subvars
# SubVars object defined in module variabletools,
# handling the following variables:
testvar(?)
>>> pub.options.reprcomments = False

Class SubVariables provides attribute access to the handled Variable objects and protects Variable objects from accidental overwriting:

>>> subvars.testvar = 3.0
>>> subvars.testvar
testvar(3.0)

Trying to query not available Variable objects (or other attributes) results in the following error message:

>>> subvars.wrong
Traceback (most recent call last):
...
AttributeError: Collection object `subvars` does neither handle a variable nor another attribute named wrong.

Class SubVariables protects only the handled Variable objects from overwriting with unplausible data:

>>> subvars.vars = "wrong"
>>> subvars.vars
'wrong'

>>> subvars.testvar = "wrong"
Traceback (most recent call last):
...
ValueError: While trying to set the value(s) of variable `testvar`, the following error occurred: 5 values are assigned to the scalar variable `testvar`.

Alternatively, you can item-access a variable:

>>> subvars["testvar"]
testvar(3.0)

>>> subvars["wrong"]
Traceback (most recent call last):
...
AttributeError: Collection object `subvars` does not handle a variable named `wrong`.

Class SubVariables supporte iteration and the application of the len() operator:

>>> for variable in subvars:
...     print(variable.name)
testvar
>>> len(subvars)
1
vars: TypeGroup_co
abstract property name: str

To be overridden.

hydpy.core.variabletools.to_repr(self: Variable, values, brackets: bool = False) → str[source]

Return a valid string representation for the given Variable object.

Function to_repr() is thought for internal purposes only, more specifically for defining string representations of subclasses of class Variable like the following:

>>> from hydpy.core.variabletools import to_repr, Variable
>>> class Var(Variable):
...     NDIM = 0
...     TYPE = int
...     initinfo = 1.0, False
...     _CLS_FASTACCESS_PYTHON = FastAccess
>>> var = Var(None)
>>> var.value = 2
>>> var
var(2)

The following examples demonstrate all covered cases. Note that option brackets allows choosing between a “vararg” and an “iterable” string representation for multidimensional variables:

>>> print(to_repr(var, 2))
var(2)

>>> Var.NDIM = 1
>>> var = Var(None)
>>> var.shape = 3
>>> print(to_repr(var, range(3)))
var(0, 1, 2)
>>> print(to_repr(var, range(3), True))
var([0, 1, 2])
>>> print(to_repr(var, range(30)))
var(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
    19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
>>> print(to_repr(var, range(30), True))
var([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
     19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29])

>>> Var.NDIM = 2
>>> var = Var(None)
>>> var.shape = (2, 3)
>>> print(to_repr(var, [range(3), range(3, 6)]))
var(0, 1, 2,
    3, 4, 5)
>>> print(to_repr(var, [range(3), range(3, 6)], True))
var([[0, 1, 2],
     [3, 4, 5]])
>>> print(to_repr(var, [range(30), range(30, 60)], True))
var([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
      19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
     [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45,
      46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]])
>>> print(to_repr(var, [range(30), range(30, 60)]))
var(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
    19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
    30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
    47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59)