Usage

Installation

In this chapter, we explain how to install cd2es. On Linux, the installation is pretty straightforward. Since the amount of data and computational effort can be quite large, we recommend using it on an HPC. Windows compatibility is no longer supported. If you want to use it on Windows, we recommend running it in a Linux subsystem (which you need anyways) entirely. At the moment, there is no compatibility for MacOS that has been tested. Read the following instructions carefully.

Prerequisites

1. Download and install Java (https://www.oracle.com/java/technologies/downloads/)

2. Download and install Git (https://git-scm.com/downloads)

3. On Windows: you need the Linux subsystem WSL

   • Install WSL: execute wsl --install

   • To activate WSL, it is necessary to have virtualization activated in BIOS (https://support.microsoft.com/en-us/windows/enable-virtualization-on-windows-c5578302-6e43-4b4b-a449-8ced115f58e1).

   • You have to restart your computer after installing WSL. After restart, a window with Ubuntu should open and show that the Linux subsystem is being installed. Wait for the completion of the installation. Create a new user and set a password.

4. If you want to download ERA5 climate data automatically, you need to register at the Copernicus climate data storage (https://cds.climate.copernicus.eu/). Then follow the instructions on their website to set up their API: https://cds.climate.copernicus.eu/how-to-api and accept the terms of use for the ERA5 land data (log in to CDS and scroll down here: https://cds.climate.copernicus.eu/datasets/reanalysis-era5-land).

Recommended method pixi

Using pixi as a package manager is the highly recommended method.

Installation process pixi

1. Open Linux terminal

2. Install pixi (https://pixi.prefix.dev/latest/installation/)

3. Clone the repository using Git by typing git clone https://gitlab.ruhr-uni-bochum.de/ee/cd2es.git {folderName}. Replace {folderName} with the folder where you want the software to be saved.

4. Navigate to the folder of the repository using cd pathToYourRepository.

5. Activate the project environment using pixi shell (has to be done only once)

6. You should be ready to go now!

Basic usage principle pixi

The tool is based on the workflow management tool Snakemake. You use it by going to the repository folder in a console. You configure the tool by using a config.yaml file. Therefore, first, copy the config_default.yaml to the same folder, rename it to config.yaml and adapt it to your purposes. In the beginning, it is enough to change the data_dir, desired energy system model (ESM), and the information given under “scenario” until line 21. The rest of the config contains details on how to download CORDEX climate data and ERA5 reanalysis data (for regression and bias adaptation) and data for the conversion from climate data to energy system input data (like wind turbine hub height).

• Navigate to the folder of the repository using cd pathToYourRepository.

Then, there are two ways of using the tool:

1. Using the command:

   pixi run snakemake -j7 --resources mem_mb=10000 api_slots_cds=2 api_slots_esgf=2
   

   • With this command, the Snakemake routine will produce all files as specified in the config under scenario.

   • The command -j is required and specifies the number of cores Snakemake can use. Adopt it to your computer.

   • --resources mem_mb=10000 is used to restrict the execution of RAM-intensive operations. If you have a lot of RAM, you can increase the number. If you get errors like “Unable to allocate memory,” you should decrease the number.

   • With api_slots_cds=2, the number of parallel downloads for ERA5 datasets is limited, while api_slots_esgf=2 applies to CORDEX.

2. If you want to produce only a single file, you can use the command:

   snakemake -j7 nameOfDesiredFile --resources mem_mb=10000 api_slots_cds=2 api_slots_esgf=2
   

   • Replace “nameOfDesiredFile” with the path and name of the file you want to produce.

   • The scenario part of the config is irrelevant when using the tool in this fashion, as you define all input parameters by giving the name of the desired file with its wildcards.

   • The other parts (e.g., data_dir or cordexParameters) are still relevant.

   • You can see which files are available by looking into Snakemake Rules.

Legacy method conda (deprecated)

Warning: Using conda as a package manager is no longer supported or maintained. Please consider using pixi instead.

Installation process conda (deprecated)

1. Download and install Miniconda (https://docs.conda.io/projects/miniconda/en/latest/index.html).

2. Make sure that Miniconda and Git are added to the PATH variable (either by checking the box during the installation or by adding it later manually). You can check if it was successful by typing git/conda into a terminal. If the result is not “command not found,” you were successful.

3. Open the shell Git Bash (other shells, especially Windows PowerShell, do not work).

4. Clone the repository using Git by typing git clone https://gitlab.ruhr-uni-bochum.de/ee/cd2es.git {folderName}. Replace {folderName} with the folder where you want the software to be saved.

5. Navigate to the folder of the repository using cd pathToYourRepository.

6. Go into the folder install by using cd install.

7. Run the first install script with bash install.sh to set up all dependencies for cd2es.

8. You should be ready to go now!

The setup files do the following:

• Install Mamba for easier package management

• Install conda environment cd2es with all necessary packages

• On Windows only:

  • Download wget for Windows

  • Install CDO (climate data processing tool) in the WSL subsystem

• On Linux: install CDO (climate data processing tool)

Manual installation conda (deprecated)

In case the bash scripts for installation fail, we provide the necessary information to do a manual installation.

1. Initialize conda with the command conda init bash.

2. Install Mamba with the command conda install mamba -n base -c conda-forge.

3. Set up the Python environment using the environment file environment.yaml with the command: mamba create -n cd2es -f environment.yaml.

4. Activate the environment with conda activate cd2es.

5. Install CDO (tool necessary for working with CORDEX climate data).

   • On Linux: sudo apt-get install cdo

   • On Windows: you need the Linux subsystem WSL

     • Install WSL: execute wsl --install

     • Open WSL

     • Install CDO in WSL with sudo apt-get install cdo

     • If you receive the message unable to locate package, run sudo apt update.

     • To activate WSL, it is necessary to have virtualization activated in BIOS (https://support.microsoft.com/en-us/windows/enable-virtualization-on-windows-c5578302-6e43-4b4b-a449-8ced115f58e1).

6. If you want to download CORDEX climate data automatically, you need wget.

   • On Linux, wget should already be installed. If not, use sudo apt-get install wget.

   • On Windows, you can get wget via https://eternallybored.org/misc/wget/. Copy wget.exe to the folder C:/Windows/System32.

   • You also need to install Java (https://www.oracle.com/java/technologies/downloads/).

Basic usage principle conda (deprecated)

The tool is based on the workflow management tool Snakemake. You use it by going to the repository folder in a terminal (preferably Git Bash). You configure the tool by using a config.yaml file. Therefore, first, copy the config_default.yaml to the same folder, rename it to config.yaml and adapt it to your purposes. In the beginning, it is enough to change the data_dir and the information given under “scenario” until line 21. The rest of the config contains details on how to download CORDEX climate data and ERA5 reanalysis data (for regression and bias adaptation) and data for the conversion from climate data to energy system input data (like wind turbine hub height).

Activate the environment with conda activate cd2es. Then, there are two ways of using the tool:

• Using the command: snakemake -j7 --resources mem_mb=10000

  • With this command, the Snakemake routine will produce all files as specified in the config under scenario.

  • The command -j is required and specifies the number of cores Snakemake can use. Adopt it to your computer.

  • --resources mem_mb=10000 is used to restrict the execution of RAM-intensive operations. If you have a lot of RAM, you can increase the number. If you get errors like “Unable to allocate memory,” you should decrease the number.

  • If you want to produce only a single file, you can use the command snakemake -j7 nameOfDesiredFile --resources mem_mb=10000 while replacing “nameOfDesiredFile” with the path and name of the file you want to produce.

  • The scenario part of the config is irrelevant when using the tool in this fashion, as you define all input parameters by giving the name of the desired file with its wildcards.

  • The other parts (e.g., data_dir or cordexParameters) are still relevant.

  • You can see which files are available by looking into Snakemake Rules.