bncopula

cop_plot

Created on 5-12-2019

@author: Sjoerd Gnodde

copulabayesnet.cop_plot.cop_2_param(Copula2d, type_copula='pdf', plot_method='3d', zclip=6, elev=15.0, azim=280, x_name=None, y_name=None, title=None, save_fig=False, save_path='pred_target.png')

Plots a 2-parameter copula, either on a 2d or 3d plane, either the pdf or the cdf.

Parameters

Copula2dbncopula.Copula2d Object

A copula object

type_copulastr (optional)

Default value: “pdf” The capacity of the copula: either ‘pdf’ or ‘cdf’

plot_methodstr (optional)

Default value: 3d. Plot either 2d or 3d.

zclipfloat (optional)

Default value: 6. The maximum z-value to plot

elevfloat (optional)

Default value: 15. From which elevation to watch the 3d plot

azimfloat (optional)

Default value: 280. The angle to watch the 3d plot.

x_namestr (optional)

The name of the x-axis. When None, tries to find name in the copula.

y_namestr (optional)

The name of the y-axis. When None, tries to find name in the copula.

titlestr (optional)

Title of the figure.

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

copulabayesnet.cop_plot.cumulative_plot(data_dict, start_date=None, end_date=None, ylabel='Cumulative precipitation (mm)', title='Comparision cumulative precipitation', save_fig=False, save_path='../../figures/cumulative.png')

Make a cumulative plot of for example the precipitation.

Parameters

data_dictdictionary with pandas.Series

The data. Key is name of the data

start_datedatetime.DateTime (optional)

The start date. Takes all to end date when not specified.

end_datedatetime.DateTime (optional)

The end date. Takes all data from start date when not specified.

ylabelstr (optional)

The ylabel of the plot. Default value: “Cumulative precipitation (mm)”

titlestr (optional)

The title of the plot. Default value: “Comparision cumulative precipitation”

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

copulabayesnet.cop_plot.different_cop(x, y, level, type_copula='pdf', plot_method='3d', zclip=6, elev=15.0, azim=280, x_name='$u_1$', y_name='$u_2$', title=None, save_fig=False, save_path='pred_target.png')

Plots any 2-parameter copula, either on a 2d or 3d plane, either the pdf or the cdf.

Parameters

Copula2dbncopula.Copula2d Object

A copula object

type_copulastr (optional)

Default value: “pdf” The capacity of the copula: either ‘pdf’ or ‘cdf’

plot_methodstr (optional)

Default value: 3d. Plot either 2d or 3d.

zclipfloat (optional)

Default value: 6. The maximum z-value to plot

elevfloat (optional)

Default value: 15. From which elevation to watch the 3d plot

azimfloat (optional)

Default value: 280. The angle to watch the 3d plot.

x_namestr (optional)

The name of the x-axis. When None, tries to find name in the copula.

y_namestr (optional)

The name of the y-axis. When None, tries to find name in the copula.

titlestr (optional)

Title of the figure.

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

copulabayesnet.cop_plot.different_cop_video(x, y, level, type_copula='pdf', plot_method='3d', zclip=6, elev=15.0, azim=280, x_name='$u_1$', y_name='$u_2$', title=None, save_video=False, save_fig=False, save_path='pred_target.png')

Plots any 2-parameter copula, either on a 2d or 3d plane, either the pdf or the cdf.

Parameters

Copula2dbncopula.Copula2d Object

A copula object

type_copulastr (optional)

Default value: “pdf” The capacity of the copula: either ‘pdf’ or ‘cdf’

plot_methodstr (optional)

Default value: 3d. Plot either 2d or 3d.

zclipfloat (optional)

Default value: 6. The maximum z-value to plot

elevfloat (optional)

Default value: 15. From which elevation to watch the 3d plot

azimfloat (optional)

Default value: 280. The angle to watch the 3d plot.

x_namestr (optional)

The name of the x-axis. When None, tries to find name in the copula.

y_namestr (optional)

The name of the y-axis. When None, tries to find name in the copula.

titlestr (optional)

Title of the figure.

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

copulabayesnet.cop_plot.double_hist(data_uncond, data_cond, data_name='Discharge $(m^3/s)$', title='Histogram conditioned and unconditioned variables', bins=16, save_fig=False, save_path='../../figures/double_hist.png')

Make a histogram of data and conditionalized expectation.

Parameters

data_uncondnumpy.array, list or pandas.Series

Unconditional data

data_condnumpy.array, list or pandas.Series

Conditional data

titlestr (optional)

Title of the figure

binsint or sequence or str (optional)

Default value: 16 If an integer is given, bins + 1 bin edges are calculated and returned, consistent with numpy.histogram. If bins is a sequence, gives bin edges, including left edge of first bin and right edge of last bin. In this case, bins is returned unmodified. All but the last (righthand-most) bin is half-open. In other words, if bins is: then the first bin is [1, 2) (including 1, but excluding 2) and the second [2, 3). The last bin, however, is [3, 4], which includes 4. Unequally spaced bins are supported if bins is a sequence. With Numpy 1.11 or newer, you can alternatively provide a string describing a binning strategy, such as ‘auto’, ‘sturges’, ‘fd’, ‘doane’, ‘scott’, ‘rice’ or ‘sqrt’, see numpy.histogram.

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

copulabayesnet.cop_plot.pred_target(pred, target, confidence_interval=None, dates=None, data_name='Water level (m)', xlabel='Date', title='Prediction and actual water level', save_fig=False, save_path='pred_target.png', conflabel='Confidence interval')

Plot the prediction and target

Parameters

prednumpy.array, list or pandas.Series

The predicted values

targetnumpy.array, list or pandas.Series

The observations

confidence_intervalnumpy.array (optional)

The confidence interval for this item, as timestep:lower,upper

dateslist or pandas.Series of datetime.datetime objects (optional)

xticks, when given. Otherwise xticks are all the whole numbers

data_namestr (optional)

Label on the y-axis

xlabelstr (optional)

Label on the x-axis

titlestr (optional)

Title of the figure

save_figbool (optional)

Whether or not to save the figure.

save_pathstr (optional)

Path where to save the figure to

conflabelstr (optional)

label of the confidence index

data_preprocessor

Created on Thu Nov 28 15:11:19 2019

Filtering, testing water balance

@author: Sjoerd Gnodde

class copulabayesnet.data_preprocessor.CorrMatrix(loc=None)

Bases: object

Preprocess the data

…

Attributes

data : dictionary with pandas dataframes

Methods

load_file()

Load a single file into the dictionary

class copulabayesnet.data_preprocessor.DataPreProc

Bases: object

Preprocess the data

…

Attributes

data_dictdictionary with pandas.DataFrames with the data

Formatted as: name data : dataframe

sourcedictonary

The dictionary with the source of the data: name data : path file

Methods

basic_filter(zval=1.5, name_list=None, inplace=True)

Do a basic, general filter over the given data. Method is the Standard score:

https://en.wikipedia.org/wiki/Standard_score (also called z-score)

When single column, removes rows. When multiple columns, turns into NaN.

Does not return anything, but saves it to object.

combine_knmi_decades(new_name=None, name_list=None)

Combine multiple decades of KNMI data

load_from_gui()

Load a data source from a basic GUI

load_from_name_and_path(path, name, data_type=’dtype2’, parameters=None)

Load a file with a given path and name. Saves it in object

load_multiple_from_gui(n)

load_source(path=None)

Load the data source from the documents

save_source(path=None)

Save the data source to a file given in the box

RH_filter(name_list=None, inplace=True)

basic_filter(zval=1.5, name_list=None, inplace=True)

Do a basic, general filter over the given data. Method is the Standard score:

https://en.wikipedia.org/wiki/Standard_score (also called z-score)

When single column, removes rows. When multiple columns, turns into NaN.

Does not return anything, but saves it to object.

Parameters

zvalfloat (optional)

Default The maximum z-score of items. Higher than this value will be removed

nameslist of str (optional)

The items you want to filter

inplacebool (optional)

Default value: True. When True, replaces old data. Else, saves with new name; with ‘F’ appended to old name.

combine_knmi_decades(new_name=None, name_list=None)

Combine multiple decades of KNMI data

Parameters

new_name: str

The name of the new frame

name_listlist of strings (optional)

Select the dataframes to combine

Returns

new_datapandas.DataFrame

dataframe with the combined KNMI data

load_from_gui()

Load a data source from a basic GUI

load_from_name_and_path(path, name, data_type='dtype2', parameters=None, lentest=False, lenmin=10)

Load a file with a given path and name. Saves it in object

Parameters

pathstr

Path of the file

namestr

Give the file a usefule name

parameterslist of str or None (optional)

What parameters to load. When None, loads all.

Returns

new_datapandas.DataFrame

KNMI data in pandas dataframe

load_multiple_from_gui(n)

load_source(path=None)

Load the data source from the documents

Parameters

pathstr or None (optional)

path of file to load. If None, asks for path with box

save_source(path=None)

Save the data source to a file given in the box

Parameters

pathstr or None (optional)

path of file to save. If None, asks for path with box

data_processor

Created on Thu Nov 28 15:08:34 2019

Taking montly maxima, averages, selecting data prior to

@author: Sjoerd Gnodde

class copulabayesnet.data_processor.DataProc(data_dict)

Bases: object

Further process the data

Give it a data_dict from a DataPreProc() object

…

Attributes

data_dictdictionary with pandas.DataFrames with the data

Formatted as: name data : dataframe

Methods

load_processed_file(path=None, **kwargs)

Load data that already has been preproecessed and processed, to a DataFrame.

merge(name_list=None, timestep=’month’, type_data=’mean’, combination=’inner’, export_name=’merged’)

Merge dataframes

resample_adv()

Resamples with back_factors

resample_basic(name_list=None, timestep=’M’, resamp_meth=’mean’, inplace=False, new_names=None)

Resample basic: get one number per given timestep

spatially_combine_data(new_name, lambdas, column=None, name_list=None)

Combine multiple spatial data sources with factors

Alphabetically order the weights!

to_array(name=None)

Return a certain DataFrame as an array

to_csv(name_list=None, index=False)

Save one or multiple dataframes to csv with a gui (when no names are passed)

load_processed_file(path=None, name=None, **kwargs)

Load data that already has been preproecessed and processed, to a DataFrame.

Parameters

pathstring or None (optional)

The location of the file. If None, loads with a gui.

**kwargs

Keyword arguments of pd.read_csv

Returns

new_datapandas.DataFrame

The loaded DataFrame, to check whether it loaded correctly.

merge(name_list=None, timestep='month', type_data='mean', combination='inner', export_name='merged')

resample_basic(name_list=None, timestep='M', resamp_meth='mean', inplace=False, new_names=None)

Resample basic: get one number per given timestep

Parameters

name_listlist of strings or None (optional)

Select the dataframes to combine. When None, select with GUI.

timestepstr (optional)

Default value: ‘M’ for month. Can also be any other timestep.

resamp_meth: str (optional)

The resample method. Default value: ‘mean’. Can also be ‘max’, ‘min’ or ‘sum’.

inplacebool (optional)

Wheter to replace the old dataset

resample_save_dates(name=None, new_name=None, resamp_meth='max', column='Value')

Resample basic: get one number per given timestep

Only possible for month

Parameters

namestr or None (optional)

Select the dataframe to combine. When None, select with GUI.

new_namestr or None (optional)

New name to save it to. When None, adds ‘M’ to old name

resamp_meth: str (optional)

The resample method. Default value: ‘max’. Can also be ‘min’.

columnstr (optional)

The name of the column to take from the dataframe Defaul value: ‘Value’

resample_temporal_average_with_date(subsetname, datesname, locname='location', daysback=datetime.timedelta(days=7), timeadded=datetime.timedelta(seconds=43200), lag=datetime.timedelta(days=1), column='NDVI', drop_first=False, datestartname='datestart', dateendname='dateend')

Resamples relatively to certain dates. Input averages and returns averages.

Parameters

subsetname

The name of the DataFrame/Series with the values

datesnamestr

The name of the DataFrame with the dates

locnamestr (optional)

The column name of the timestamp of the time to subset on Default value: ‘location’

daysbackdatetime.timedelta object (optional)

Time to substract from date (how long should the mean go back) Default value: dt.timedelta(days=7)

timeaddeddatetime.timedelta object (optional)

Time to add to date (for example when date is taken as 0:00h) Default value: dt.timedelta(hours=12)

lagdatetime.timedelta object (optional)

Time not to take into account before discharge moment. Positive = backwards in time. Default value: dt.timedelta(days=1)

columnstr (optional)

When the to subset dataset is a DataFrame instead of Series, the name of the column to use. Default value: ‘Value’

drop_firstbool (optional)

Drop the first value when true. This is useful when first peak is in the beginning of the first month. Default value: False

Returns

new_dfpandas.DataFrame

DataFrame with sumsampled values

resample_with_dates(subsetname, datesname, resamp_meth='sum', locname='location', daysback=datetime.timedelta(days=7), timeadded=datetime.timedelta(seconds=43200), lag=datetime.timedelta(days=1), column='Value', drop_first=True)

Resamples relatively to certain dates

Parameters

subsetname

The name of the DataFrame/Series with the values

datesnamestr

The name of the DataFrame with the dates

resamp_methstr (optional)

The method to resample with. Can be ‘sum’ or ‘mean’. Default value: ‘sum’

locnamestr (optional)

The column name of the timestamp of the time to subset on Default value: ‘location’

daysbackdatetime.timedelta object (optional)

Time to substract from date (how long should the mean go back) Default value: dt.timedelta(days=7)

timeaddeddatetime.timedelta object (optional)

Time to add to date (for example when date is taken as 0:00h) Default value: dt.timedelta(hours=12)

lagdatetime.timedelta object (optional)

Time not to take into account before discharge moment. Positive = backwards in time. Default value: dt.timedelta(days=1)

columnstr (optional)

When the to subset dataset is a DataFrame instead of Series, the name of the column to use. Default value: ‘Value’

drop_firstbool (optional)

Drop the first value when true. This is useful when first peak is in the beginning of the first month. Default value: True

Returns

new_dfpandas.DataFrame

DataFrame with sumsampled values

spatially_combine_data(new_name, lambdas, column=None, name_list=None)

Combine multiple spatial data sources with factors

Alphabetically order the weights!

Parameters

new_namestr

Name for the new dataframe

lambdasList or numpy.Array of floats

The weights

columnstr (optional)

Which column to combine. !! When None, it can cause errors when certain columns are not float.

name_listList

Which dataframes to combine. If None, use gui to select.

Returns

new_dfpandas.DataFrame

The combined dataframe

to_array(name=None)

Return a certain DataFrame as an array.

Parameters

namestr (optional)

Name of the DataFrame to return as numpy.ndarray

Returns

array_datanumpy.ndarray

The numpy.ndarray of the data

to_csv(name_list=None, index=False)

Save one or multiple dataframes to csv with a gui (when no names are passed)

Parameters

name_listList of strings (optional)

Names of the dataframes you want to save

indexbool (optional)

Default value: False Save the index as well when True.

class copulabayesnet.data_processor.WatBal(area)

Bases: object

Collect all the water balance factors and calculate balance, plot etc.

Make sure the data is already resampled.

Attributes

datapandas.Dataframe

The data of the water balance

positivedictionary

For every column, whether it should be added or substracted True = positive

unitstr

The unit that is used

areafloat

The area of the catchment

Methods

add_discharge(data, unit=’m3’, name=’Discharge’)

Add discharge to the DataFrame

add_evap(data, unit=’mm’, name=’Evaporation’)

Add evaporation to the DataFrame

add_extra(data, positive=False, name=’Extra’)

Add an extra influence to the DataFrame

add_gw(data, n=0.25, level=True, unit=’cm’, name=’Groundwater difference’)

Add groundwater to the DataFrame

add_prec(data, unit=’mm’, name=’Precipitation’)

Add precipitation to the DataFrame

add_wwtp(amount, unit=’mm’, name=’WWTP’)

Add an uniform forcing: same per month

calculate_balance(per_month=False, rel_axis=’q_in’)

Calculate the water balance

create_average_year()

Average per month

height(unit=’mm’)

Turn the unit into 1d

load_csv(path=None)

Load csv as data. Overwrites other dataframe.

minimize()

Remove columns with NaN

plot(avg_year_plt=True, title=’Water balance’, save_fig=False, save_path=’Water Balance.png’)

Plot

remove_force(name)

Drop a force from the dataframe

save_positive(path=None)

Save dataframe to csv.

to_csv(path=None)

Save dataframe to csv.

volume()

Turn the unit into volume

add_discharge(data, unit='m3', name='Discharge')

Parameters

unitstr (optional)

The unit of the data. Default value: “mm” Can also be “m” or “m3”

namestr (optional)

The name of this data. Default value: “Discharge”

add_extra(data, positive=False, name='Extra')

Add an extra influence to the DataFrame

Parameters

datapandas.Series

Series object with the data

namestr (optional)

The name of this data. Default value: “Extra”

add_gw(data, n=0.25, level=True, unit='cm', name='Groundwater difference')

Add groundwater to the DataFrame

Parameters

datapandas.Series

Series object with the data

nfloat (optional)

Default value = 0.25 Effective porosity

levelbool (optional)

Whether or not a water level is given, in stead of a difference.

unitstr (optional)

The unit of the data. Default value: “cm” Can also be “m” or “mm”.

namestr (optional)

The name of this data. Default value: “Groundwater difference”

add_pot_evap(data, unit='mm', name='Potential evaporation')

Add evaporation to the DataFrame

Parameters

datapandas.Series

Series object with the data

unitstr (optional)

The unit of the data. Default value: “mm” Can also be “m” or “cm”.

namestr (optional)

The name of this data. Default value: “Evaporation”

add_prec(data, unit='mm', name='Precipitation')

Add precipitation to the DataFrame

Parameters

datapandas.Series

Series object with the data

unitstr (optional)

The unit of the data. Default value: “mm” Can also be “m” or “cm”.

namestr (optional)

The name of this data. Default value: “Precipitation”

add_wwtp(amount, unit='mm', name='WWTP')

Add an uniform forcing: same per month

Parameters

amountfloat

The WWTP amount per month

unitstr (optional)

The unit of the data. Default value: “mm” Can also be “m” or “m3”

namestr (optional)

The name of this data. Default value: “WWTP”

budyko_curve(plot=True, title='Budyko curve', max_x=1.8, save_figure=False, extra_sources=None, save_path='../figures/budyko curve.png')

Actual evaporation based on the water balance

Parameters

plotbool (optional)

Default value: True Plot a figure or not

titlestr (optional)

Default value: “Budyko curve” The title of the plot

max_xfloat (optional)

Length of the plot. Default value: 2

Returns

arid_indexfloat

Aridity index

evap_indexfloat

Evaporation index

calculate_balance(per_month=False, rel_axis='q_in')

Parameters

per_monthbool (optional)

Water balance per month. Default value: False

rel_axisstr (optional)

Default value: ‘q_in’. To what parameter the difference is compared. Either ‘q_in’, ‘q_out’ or ‘del_s’.

Returns

difffloat

Difference water balance

rel_difffloat

Relative difference water balance

create_average_year()

Average per month

height(unit='mm')

Turn the unit into 1d

Parameters

unitstr (optional)

Default value: “mm” The unit to convert to.

load_csv(path=None)

Load csv as data. Overwrites other dataframe.

Parameters

pathstr (optional)

The path of the file

minimize()

Remove columns with NaN

plot(avg_year_plt=True, title='Water balance', save_fig=False, save_path='Water Balance.png')

Plot the water balance

plot_pot_evap_prec(title='Precipitation and evaporation', save_fig=False, save_path='Precipitation and evaporation.png')

Plot the water balance with only evaporation and precipiation

Avg_year should have been made

remove_force(name)

Drop a force from the dataframe

Parameters

namestr

The name of the item to be removed.

save_positive(path=None)

Save dataframe to csv.

Parameters

pathstr (optional)

The path of the file

to_csv(path=None)

Save dataframe to csv.

Parameters

pathstr (optional)

The path of the file

volume()

Turn the unit into volume

copulabayesnet.data_processor.back_factors(n=7, power=0.5)

Calculate the factors

Parameters

nint (optional)

Default value: 7 The number of days to take into account.

powerfloat (optional)

Default value: 0.5 The power in the formula

Returns

factorsnumpy.array of floats

The factors of the values. Sum of the factors = 1

copulabayesnet.data_processor.back_values(data, date, n=7, power=0.5)

copulabayesnet.data_processor.select_date()

copulabayesnet.data_processor.water_balance(self, disch_all, rain_all, evap_all, gw_all, rwzi_single_month=0, average_year=False, plot_figure=False, data_unit='mm/month', save_figure=False, save_path='../figures/overall_water_balance.png')

Return and plot the water balance

! When average_year==True, make shure that all items have the same timeframe.

Parameters

disch_allnumpy.array or pandas.DataFrame

The discharge measurements

rain_allnumpy.array or pandas.DataFrame

The precipiation measurements

evap_allnumpy.array or pandas.DataFrame

The evaporation data

gw_allnumpy.array or pandas.DataFrame

The groundwater DIFFERENCE data

rwzi_single_monthfloat

Discharge of the WWTP (RWZI in Dutch) in a single month. Default value: 0.

average_yearbool

When True, averages over every January, February etc. Default value: False

plot_figurebool

Plot the figure. Default value: False

data_unitstr

The unit of the data Default value: “mm/month”

save_figurebool

Whether to save the figure Default value: False

save_pathstr

Save the figure to this path.

Returns

total_difffloat

The total water balance difference

rel_total_difffloat

The total water balance difference divided by the sum of the discharge

diffnumpy.array or pandas.DataFrame

The difference per timestep

rel_diffnumpy.array or pandas.DataFrame

The relative difference per timestep

spatial_tools

Created on Thu Dec 12 10:21:22 2019

@author: GNOS

copulabayesnet.spatial_tools.inv_dist_weight(distances, b)

Inverse distance weight

Parameters

distancesnumpy.array of floats

Distances to point of interest

bfloat

The parameter of the inverse distance weight. The higher, the higher the influence of closeby stations.

Returns

lambdasnumpy.array of floats

The lambda parameters of the stations