HydPy-Conv-IDW-ED (inverse distance weighted interpolation with external drift)

conv_idw_ed extends conv_idw by taking an additional statistical relationship between the interpolated variable and an arbitrary independent variable into account. A typical use case is elevation-dependent temperature interpolation. You can understand the approach of conv_idw_ed as a simplification of External Drift Kriging.

The algorithm works as follows (we take elevation-dependent temperature interpolation as an example):

• Estimate the linear regression coefficients to model the relationship between air temperature and elevation at the available stations for the current simulation time step (or take the given default values).

• Use the resulting linear model to predict the temperature at the stations and the interpolation target points.

• Calculate the residuals (differences between predictions and measurements) at all stations.

• Interpolate the residuals based on the same inverse distance weighting approach as applied by application model conv_idw.

• Combine the predicted temperature values and the interpolated residuals to gain the final interpolation results at all target points.

For perfect correlations between temperature and elevation, the interpolated values depend only on elevation. If there is no correlation, the interpolated values depend solely on spatial proximity to the stations.

Integration tests

Note

When new to HydPy, consider reading section Integration Tests first.

We start the following explanations with repeating the examples documented for application model conv_idw. Hence, we first define identical test settings (please see the documentation on application model conv_idw for more information):

>>> from hydpy import Element, Node, pub
>>> pub.timegrids = "2000-01-01", "2000-01-04", "1d"

>>> from hydpy.models.conv_idw_ed import *
>>> parameterstep()

>>> in1, in2, in3 = Node("in1"), Node("in2"), Node("in3")
>>> element = Element("conv",
...                   inlets=(in1, in2, in3),
...                   outlets=["out1", "out2", "out3", "out4"])

>>> inputcoordinates(in1=(0.0, 3.0),
...                  in2=(2.0, -1.0),
...                  in3=(4.0, 2.0))
>>> outputcoordinates(out1=(0.0, 3.0),
...                   out2=(3.0, -2.0),
...                   out3=(1.0, 2.0),
...                   out4=(1.0, 1.0))
>>> power(2.0)
>>> maxnmbinputs(3)

>>> element.model = model
>>> from hydpy.core.testtools import IntegrationTest
>>> test = IntegrationTest(element)
>>> test.dateformat = "%Y-%m-%d"
>>> with pub.options.checkseries(False):
...     in1.sequences.sim.series = 1.0, nan, nan
...     in2.sequences.sim.series = 3.0, 2.0, nan
...     in3.sequences.sim.series = 4.0, nan, nan

Next, we prepare the additional parameters of conv_idw_ed. Most importantly, we define the values of the independent variable for all input and output nodes:

>>> inputheights(in1=0.0,
...              in2=2.0,
...              in3=4.0)
>>> outputheights(out1=0.0,
...               out2=1.0,
...               out3=2.0,
...               out4=3.0)

In our example calculations, we use three input nodes. However, we first set the number of data-points that must be available to estimate the linear model to four, which of course is impossible. Hence, conv_idw_ed will always use the default values. We set both of them to zero and thus effectively disable the “external drift” functionality so that conv_idw_ed works exactly like conv_idw:

>>> minnmbinputs(4)
>>> defaultconstant(0.0)
>>> defaultfactor(0.0)

Under the given configuration, conv_idw_ed reproduces the results of the test examples documented for application model conv_idw precisely:

>>> test()
|       date |           inputs | actualconstant | actualfactor |           inputpredictions |                outputpredictions |           inputresiduals |                 outputresiduals |                 outputs | in1 | in2 | in3 | out1 | out2 | out3 | out4 |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2000-01-01 | 1.0  3.0     4.0 |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | 1.0  3.0             4.0 | 1.0  3.0  1.75              2.4 | 1.0  3.0  1.75      2.4 | 1.0 | 3.0 | 4.0 |  1.0 |  3.0 | 1.75 |  2.4 |
| 2000-01-02 | nan  2.0     nan |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | nan  2.0             nan | 2.0  2.0   2.0              2.0 | 2.0  2.0   2.0      2.0 | nan | 2.0 | nan |  2.0 |  2.0 |  2.0 |  2.0 |
| 2000-01-03 | nan  nan     nan |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | nan  nan             nan | nan  nan   nan              nan | nan  nan   nan      nan | nan | nan | nan |  nan |  nan |  nan |  nan |

>>> maxnmbinputs(2)
>>> test()
|       date |           inputs | actualconstant | actualfactor |           inputpredictions |                outputpredictions |           inputresiduals |                          outputresiduals |                          outputs | in1 | in2 | in3 | out1 |     out2 |     out3 | out4 |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2000-01-01 | 1.0  3.0     4.0 |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | 1.0  3.0             4.0 | 1.0  3.105263  1.545455              2.0 | 1.0  3.105263  1.545455      2.0 | 1.0 | 3.0 | 4.0 |  1.0 | 3.105263 | 1.545455 |  2.0 |
| 2000-01-02 | nan  2.0     nan |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | nan  2.0             nan | nan       2.0       nan              2.0 | nan       2.0       nan      2.0 | nan | 2.0 | nan |  nan |      2.0 |      nan |  2.0 |
| 2000-01-03 | nan  nan     nan |            0.0 |          0.0 | 0.0  0.0               0.0 | 0.0  0.0  0.0                0.0 | nan  nan             nan | nan       nan       nan              nan | nan       nan       nan      nan | nan | nan | nan |  nan |      nan |      nan |  nan |

Now we enable the “external drift” functionality by providing non-zero default values. Hence, conv_idw_ed employs the same linear model in all simulation time steps:

>>> maxnmbinputs(3)
>>> defaultfactor(1.0)
>>> defaultconstant(2.0)
>>> test()
|       date |           inputs | actualconstant | actualfactor |           inputpredictions |                outputpredictions |             inputresiduals |                       outputresiduals |                    outputs | in1 | in2 | in3 | out1 | out2 |    out3 | out4 |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2000-01-01 | 1.0  3.0     4.0 |            2.0 |          1.0 | 2.0  4.0               6.0 | 2.0  3.0  4.0                5.0 | -1.0  -1.0            -2.0 | -1.0  -1.1  -1.15625             -1.2 | 1.0  1.9  2.84375      3.8 | 1.0 | 3.0 | 4.0 |  1.0 |  1.9 | 2.84375 |  3.8 |
| 2000-01-02 | nan  2.0     nan |            2.0 |          1.0 | 2.0  4.0               6.0 | 2.0  3.0  4.0                5.0 |  nan  -2.0             nan | -2.0  -2.0      -2.0             -2.0 | 0.0  1.0      2.0      3.0 | nan | 2.0 | nan |  0.0 |  1.0 |     2.0 |  3.0 |
| 2000-01-03 | nan  nan     nan |            2.0 |          1.0 | 2.0  4.0               6.0 | 2.0  3.0  4.0                5.0 |  nan   nan             nan |  nan   nan       nan              nan | nan  nan      nan      nan | nan | nan | nan |  nan |  nan |     nan |  nan |

If we set the required number of data-points to three (two would also be fine), conv_idw_ed estimates the parameters of the linear model (ActualFactor and ActualConstant) in the first time step on its own, and falls back to the default values in the remaining time steps:

>>> minnmbinputs(2)
>>> test()
|       date |           inputs | actualconstant | actualfactor |                     inputpredictions |                               outputpredictions |                      inputresiduals |                                 outputresiduals |                       outputs | in1 | in2 | in3 | out1 |  out2 |     out3 | out4 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2000-01-01 | 1.0  3.0     4.0 |       1.166667 |         0.75 | 1.166667  2.666667          4.166667 | 1.166667  1.916667  2.666667           3.416667 | -0.166667  0.333333       -0.166667 | -0.166667  0.258333  -0.096354         0.033333 | 1.0  2.175  2.570312     3.45 | 1.0 | 3.0 | 4.0 |  1.0 | 2.175 | 2.570312 | 3.45 |
| 2000-01-02 | nan  2.0     nan |            2.0 |          1.0 |      2.0       4.0               6.0 |      2.0       3.0       4.0                5.0 |       nan      -2.0             nan |      -2.0      -2.0       -2.0             -2.0 | 0.0    1.0       2.0      3.0 | nan | 2.0 | nan |  0.0 |   1.0 |      2.0 |  3.0 |
| 2000-01-03 | nan  nan     nan |            2.0 |          1.0 |      2.0       4.0               6.0 |      2.0       3.0       4.0                5.0 |       nan       nan             nan |       nan       nan        nan              nan | nan    nan       nan      nan | nan | nan | nan |  nan |   nan |      nan |  nan |

The final example shows that everything works as expected for the edge cases of perfect and missing correlation. In the first time step, there is a perfect (negative) correlation at the input nodes. Hence, all residuals (calculated for the input nodes and interpolated for the output nodes) are zero. In the second time step, there is no correlation. Hence, all values predicted by the linear model are the same. The third time step shows that conv_idw_ed handles the special (but not uncommon) case of missing variability in the input values well:

>>> in1.sequences.sim.series = 4.0, 0.0, 0.0
>>> in2.sequences.sim.series = 3.0, 1.0, 0.0
>>> in3.sequences.sim.series = 2.0, 0.0, 0.0
>>> test()
|       date |           inputs | actualconstant | actualfactor |                     inputpredictions |                               outputpredictions |                      inputresiduals |                                 outputresiduals |                      outputs | in1 | in2 | in3 | out1 | out2 |     out3 | out4 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 2000-01-01 | 4.0  3.0     2.0 |            4.0 |         -0.5 |      4.0       3.0               2.0 |      4.0       3.5       3.0                2.5 |       0.0       0.0             0.0 |       0.0       0.0        0.0              0.0 | 4.0   3.5       3.0      2.5 | 4.0 | 3.0 | 2.0 |  4.0 |  3.5 |      3.0 |  2.5 |
| 2000-01-02 | 0.0  1.0     0.0 |       0.333333 |          0.0 | 0.333333  0.333333          0.333333 | 0.333333  0.333333  0.333333           0.333333 | -0.333333  0.666667       -0.333333 | -0.333333  0.516667  -0.192708         0.066667 | 0.0  0.85  0.140625      0.4 | 0.0 | 1.0 | 0.0 |  0.0 | 0.85 | 0.140625 |  0.4 |
| 2000-01-03 | 0.0  0.0     0.0 |            0.0 |          0.0 |      0.0       0.0               0.0 |      0.0       0.0       0.0                0.0 |       0.0       0.0             0.0 |       0.0       0.0        0.0              0.0 | 0.0   0.0       0.0      0.0 | 0.0 | 0.0 | 0.0 |  0.0 |  0.0 |      0.0 |  0.0 |

class hydpy.models.conv_idw_ed.Model

Bases: BaseModel

HydPy-Conv-IDW-ED (inverse distance weighted interpolation with external drift).

The following “inlet update methods” are called in the given sequence at the beginning of each simulation step:

• Pick_Inputs_V1 Pick the input from all inlet nodes.

The following “run methods” are called in the given sequence during each simulation step:

• Calc_ActualConstant_ActualFactor_V1 Calculate the linear regression coefficients ActualConstant and ActualFactor for modelling the relationship between Inputs (dependent variable) and Heights (independent variable).

• Calc_InputPredictions_V1 Predict the values of the input nodes based on a linear model.

• Calc_OutputPredictions_V1 Predict the values of the output nodes based on a linear model.

• Calc_InputResiduals_V1 Determine the residuals at the input nodes.

• Calc_OutputResiduals_V1 Perform a simple inverse distance weighted interpolation based on the residuals previously determined for the input nodes.

• Calc_Outputs_V3 Calculate the values of the output nodes by combining the initial predictions with the interpolated residuals.

The following “outlet update methods” are called in the given sequence at the end of each simulation step:

• Pass_Outputs_V1 Pass the output to all outlet nodes.

The following “additional methods” might be called by one or more of the other methods or are meant to be directly called by the user:

• Return_Mean_V1 Return the arithmetic mean value for the given vector.

• Interpolate_InverseDistance_V1 Perform a simple inverse distance weighted interpolation.

DOCNAME: DocName = ('Conv-IDW-ED', 'inverse distance weighted interpolation with external drift')

REUSABLE_METHODS: ClassVar[tuple[type[ReusableMethod], ...]] = ()

class hydpy.models.conv_idw_ed.ControlParameters(master: Parameters, cls_fastaccess: type[FastAccessParameter] | None = None, cymodel: CyModelProtocol | None = None)

Bases: SubParameters

Control parameters of model conv_idw_ed.

The following classes are selected:

• InputCoordinates() Coordinates of the inlet nodes [?].

• OutputCoordinates() Coordinates of the outlet nodes [?].

• InputHeights() The height (above sea level or anything else) of the input nodes [?].

• OutputHeights() The height (above sea level or anything else) of the output nodes [?].

• MaxNmbInputs() The maximum number of input locations to be taken into account for interpolating the values of a specific output location [-].

• MinNmbInputs() The minimum number of inputs for performing a statistical analysis [-].

• DefaultConstant() Default or fallback value for the constant of the linear regression model [?].

• DefaultFactor() Default or fallback value for the factor of the linear regression model [?].

• Power() Power parameter for calculating inverse distance weights [-].

class hydpy.models.conv_idw_ed.DerivedParameters(master: Parameters, cls_fastaccess: type[FastAccessParameter] | None = None, cymodel: CyModelProtocol | None = None)

Bases: SubParameters

Derived parameters of model conv_idw_ed.

The following classes are selected:

• NmbInputs() The number of inlet nodes [-]

• NmbOutputs() The number of outlet nodes [-]

• Distances() Distances of the inlet nodes to each outlet node [?].

• ProximityOrder() Indices of the inlet nodes in the order of their proximity to each outlet node [-].

• Weights() Weighting coefficients of the inlet nodes corresponding to their proximity to each outlet node and parameter Power [-].

class hydpy.models.conv_idw_ed.FluxSequences(master: Sequences, cls_fastaccess: type[TypeFastAccess_co] | None = None, cymodel: CyModelProtocol | None = None)

Bases: FluxSequences

Flux sequences of model conv_idw_ed.

The following classes are selected:

• Inputs() The (unmodified) values supplied by the input nodes [?].

• ActualConstant() The actual value for the constant of the linear regression model [?].

• ActualFactor() The actual value for the factor of the linear regression model [?].

• InputPredictions() The values of the input nodes predicted by a regression model [?].

• OutputPredictions() The values of the output nodes predicted by a regression model [?].

• InputResiduals() The exact residuals of a regression model calculated for the input nodes [?].

• OutputResiduals() The guessed residuals of a regression model interpolated for the input nodes [?].

• Outputs() The final interpolation results estimated for the output nodes [?].

class hydpy.models.conv_idw_ed.InletSequences(master: Sequences, cls_fastaccess: type[TypeFastAccess_co] | None = None, cymodel: CyModelProtocol | None = None)

Bases: InletSequences

Inlet sequences of model conv_idw_ed.

The following classes are selected:

• Inputs() Inputs [?].

class hydpy.models.conv_idw_ed.OutletSequences(master: Sequences, cls_fastaccess: type[TypeFastAccess_co] | None = None, cymodel: CyModelProtocol | None = None)

Bases: OutletSequences

Outlet sequences of model conv_idw_ed.

The following classes are selected:

• Outputs() Outputs [?].