Snakemake Rules ===== .. warning:: **Outdated Documentation** This page is currently **not up to date** and reflects an older commit (e6f69605067650bd949dfd66ae139d4e2ffa02a0) of the project. The content may be inaccurate or missing recent changes. Updates are planned and will appear in future releases. Please refer to the project repository for the latest information. This chapter documents the scripts executed via the snakemake routine. The workflow management tool snakemake works by defining different rules which require input files and produce outputfiles. Snakemake automatically determines which rules have to be executed to produce a desired output file. In this chapter, the different rules are explained. All rules are defined in the :file:``snakefile`` in the top level of the repository. All rules execute python files. They are documented in the next chapter and linked here. downloadCordex ------------ **Output:** ``config["data_dir"] + {domain}/original/{model}/{climate\_variable}\_m{model}\_rcp{rcp}\_{year}\_{toTemporal}.nc`` **Script:** :ref:`downloadCordex` * downloads cordex data using wget scripts renameCordex ------------ **Output:** ``config["data_dir"] + '/{domain}/original/{model}/{climate\_variable}\_m{model}\_rcp{rcp}\_{year}\_{toTemporal}.nc`` **Script:** :ref:`renameCordex` * only used if downloadCordex is False in config * renames existing cordex data to be used in *cd2es* downloadERA5 ------------ **Output:** ``config["data_dir"] + /{domain}/observed/{climate\_variable\_ERA5}\_ERA5\_{year}.nc`` **Script:** :ref:`downloadEra` * downloads ERA5 reanalysis data for regression and bias adaption * you need to be registered at the copernicus climate data store and your credits must either be in a .cdsapirc (see Getting started 7.) or you must enable giveCdsCredentialsInConfig in config and provide your data in the config build_topology ------------ **Output:** ``results/{domain}/topo.nc`` **Script:** :ref:`buildTopo` * build a topology file which is used to map all cordex climate files to * borders are chosen via the choice of domain * the spatial resolution can be influenced by changing the geography/x_size or y_size values in the config process_cordex_data ------------ **Input:** ``results/{domain}/topo.nc``, ``config["data_dir"] + {domain}/original/{model}/{climate\_variable}\_m{model}\_rcp{rcp}\_{year}\_{toTemporal}.nc`` **Output:** ``config["data_dir"]+/{domain}/remap/{model}/{climate\_variable}\_m{model}\_rcp{rcp}\_{year}\_{toTemporal}.nc`` **Script:** :ref:`process` * takes the topology file and a cordex climate file as input * remaps cordex data from rotated coordinate system to rectangular coordinate system, interpolates if necessary and removes the 29.02. in leap years * *cd2es* cannot work with cordex data where time is given in date time 360 * you need to use different models calculate_quantiles ------------ **Input:** ``config["data_dir"] + {domain}/remap/{model}/{climate_variable}_m{model}_rcp{rcp}_{year}_{toTemporal}.nc`` **Output:** ``config["data_dir"] + {domain}/remap/{model}/{climate_variable}_m{model}_rcp{rcp}_{toTemporal}_quantiles.nc`` **Script:** :ref:`calcQ` * takes historic model data for all years given in the config under ``bias_adaption`` between ``yearHistStart`` and ``yearHistEnd`` as input * calculates the 100 quantiles for each grid point for the given climate variable (necessary for bias adaption) calculate_quantiles_observed ------------ **Input:** ``config["data_dir"] + {domain}/observed/{climate_variable}_observed_{year}_{toTemporal}.nc`` **Output:** ``config["data_dir"] + {domain}/observed/{climate_variable}_observed_{toTemporal}_quantiles.nc`` **Script:** :ref:`calcQ` * same as ``calculate_quantiles``, only for reanalysis data bias_adopt ------------ **Input:** ``config["data_dir"] + {domain}/remap/{model}/{climate_variable}_m{model}_rcp{rcp}_{toTemporal}_quantiles.nc``, ``config["data_dir"] + {domain}/observed/{climate_variable}_observed_{toTemporal}_quantiles.nc``, ``config["data_dir"]+/{domain}/remap/{model}/{climate\_variable}\_m{model}\_rcp{rcp}\_{year}\_{toTemporal}.nc`` **Output:** ``results/{domain}/{technology}\_m{model}\_rcp{rcp}\_{year}\_ba.csv`` **Script:** :ref:`biasAdopt` * takes processed cordex files and historic climate model and reanalysis quantile files as input * bias adopts the climate file using quantile delta mapping * used to correct biases in climate data * more information: https://doi.org/10.1175/JCLI-D-14-00754.1 build_{technology}_profile ------------ **Input:** ``config["data_dir"]+/{domain}/remap/{model}/sfcWind_m{model}_rcp{rcp}_{year}_1h.nc`` **Output:** ``results/{domain}/{technology}\_m{model}\_rcp{rcp}\_{year}\_notAgg.nc`` **Script:** :ref:`buildSolar`, :ref:`buildWind`, :ref:`buildCsp`, :ref:`buildHydro`, :ref:`buildDemand`, :ref:`buildOT`, :ref:`buildCL` * builds capacity factor time series for different technologies 'csp', 'pv', 'wind' (onshore), 'wind_offshore', 'hydro_production' (only Europe), 'demand' (only Europe) and tpp(CoolingType)_p\{Plant\} with cooling type in [CL, OT] (closed loop, once through * efficiency of thermal power plants) * takes a processed cordex (or ERA5) climate data file as input * uses parameters given in config * hydro requires additional data on power plant location * at the moment only data for Europe is available, therefore it does not work for other continents * all rules appear twice, once for cordex data, once for ERA5 demand_regression ------------ **Input:** ``results/{{domain}}/tas_m{{model}}_rcp{{rcp}}_{year}_d_aggregated.csv`` **Output:** ``results/{domain}/demand_regression_m{model}_rcp{rcp}.csv`` **Script:** :ref:`demandReg` * takes already aggregated cordex temperature files as input * calculates quadratic regressions for historic demand data vs. historic temperatures * requires historical demand data * at the moment only data for Europe is available, therefore it does not work for other continents * if demandRegression/makePlots is true in the config, plots of the regression functions are created in the folder :file:``demandPlots`` calibrate_hydro_power plants ------------ **Input:** ``config["data_dir"]+/{{domain}}/remap/{{model}}/mrro_m{{model}}_rcp{{rcp}}_{year}_d.nc`` **Output:** ``results/{domain}/power plant_database_m{model}_rcp{rcp}.csv`` **Script:** :ref:`calbHydro` * takes processed cordex runoff (mrro) files as input * uses the power plant database for europe to calculate average run offs in the past and match it to the installed capacity of the existing hydro power plants calculate_turbine_curve ------------ **Output:** ``results/{domain}/turbine_curve_{onOffshore}.txt`` **Script:** :ref:`calcCurve` * calculated an on* or offshore turbine curve based on the information given in the config below on/offwind * the curve is calculated for each wind speed between 0 and 40 m/s in 0.01 m/s steps * to simplify later calculations, the resulting curve is fitted with an :math:``exp(-a \cdot x^2) \cdot (b \cdot x^3+c \cdot x^2+d \cdot x)`` function * the fit parameters are saved in the output file and used for further calculations calculate_OT_flow ------------ **Input:** ``config["data_dir"]+/{{domain}}/remap/{{model}}/mrro_m{{model}}_rcp{{rcp}}_{year}_d.nc`` **Output:** ``results/{domain}/OT_flow_m{model}_rcp{rcp}.nc`` **Script:** :ref:`calcOTFlow` * takes processed cordex runoff (mrro) files as input * calculates historic water availability for once through power plants aggregate_files_dataDirect, aggregate_tech_files etc. ------------ **Script:** :ref:`aggFiles` * geographically aggregates climate data files or capacity factor files either to countries (if no custom bus map use is specified in config) or to the geography given in the custom bus map build_backbone_input ------------ **Input:** various time series files **Output:** ``results/{domain}/backbone\_m{model}\_rcp{rcp}\_{year}.xlsx`` **Script:** :ref:`buildBB` * inserts time series into a backbone (https://doi.org/10.3390/en12173388 ) model * basic backbone input data can be constructed with another tool (https://gitlab.ruhr-uni-bochum.de/ee/backbone-tools)