Usage ===== .. _installation: Installation ------------ In this chapter, we explain how to install *cd2es*. On Linux, the installation is pretty straightforward. Since the amount of data and computational effort can be quite large, we recommend using it on an HPC. Windows compatibility is no longer supported. If you want to use it on Windows, we recommend running it in a Linux subsystem (which you need anyways) entirely. At the moment, there is no compatibility for MacOS that has been tested. Read the following instructions carefully. .. _prereq: Prerequisites ~~~~~~~~~~~~~ #. Download and install Java (https://www.oracle.com/java/technologies/downloads/) #. Download and install Git (https://git-scm.com/downloads) #. On Windows: you need the Linux subsystem WSL * Install WSL: execute ``wsl --install`` * To activate WSL, it is necessary to have virtualization activated in BIOS (https://support.microsoft.com/en-us/windows/enable-virtualization-on-windows-c5578302-6e43-4b4b-a449-8ced115f58e1). * You have to restart your computer after installing WSL. After restart, a window with Ubuntu should open and show that the Linux subsystem is being installed. Wait for the completion of the installation. Create a new user and set a password. #. If you want to download ERA5 climate data automatically, you need to register at the Copernicus climate data storage (https://cds.climate.copernicus.eu/). Then follow the instructions on their website to set up their API: https://cds.climate.copernicus.eu/how-to-api and accept the terms of use for the ERA5 land data (log in to CDS and scroll down here: https://cds.climate.copernicus.eu/datasets/reanalysis-era5-land). Recommended method ``pixi`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Using ``pixi`` as a package manager is the highly recommended method. Installation process ``pixi`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ #. Open Linux terminal #. Install ``pixi`` (https://pixi.prefix.dev/latest/installation/) #. Clone the repository using Git by typing ``git clone https://gitlab.ruhr-uni-bochum.de/ee/cd2es.git {folderName}``. Replace ``{folderName}`` with the folder where you want the software to be saved. #. Navigate to the folder of the repository using ``cd pathToYourRepository``. #. Activate the project environment using ``pixi shell`` (has to be done only once) #. You should be ready to go now! Basic usage principle ``pixi`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The tool is based on the workflow management tool Snakemake. You use it by going to the repository folder in a console. You configure the tool by using a :file:`config.yaml` file. Therefore, first, copy the :file:`config_default.yaml` to the same folder, rename it to :file:`config.yaml` and adapt it to your purposes. In the beginning, it is enough to change the ``data_dir``, desired energy system model (ESM), and the information given under "scenario" until line 21. The rest of the config contains details on how to download CORDEX climate data and ERA5 reanalysis data (for regression and bias adaptation) and data for the conversion from climate data to energy system input data (like wind turbine hub height). * Navigate to the folder of the repository using ``cd pathToYourRepository``. Then, there are two ways of using the tool: #. Using the command: .. code-block:: python pixi run snakemake -j7 --resources mem_mb=10000 api_slots_cds=2 api_slots_esgf=2 * With this command, the Snakemake routine will produce all files as specified in the config under ``scenario``. * The command ``-j`` is required and specifies the number of cores Snakemake can use. Adopt it to your computer. * ``--resources mem_mb=10000`` is used to restrict the execution of RAM-intensive operations. If you have a lot of RAM, you can increase the number. If you get errors like "Unable to allocate memory," you should decrease the number. * With ``api_slots_cds=2``, the number of parallel downloads for ERA5 datasets is limited, while ``api_slots_esgf=2`` applies to CORDEX. #. If you want to produce only a single file, you can use the command: .. code-block:: python snakemake -j7 nameOfDesiredFile --resources mem_mb=10000 api_slots_cds=2 api_slots_esgf=2 * Replace "nameOfDesiredFile" with the path and name of the file you want to produce. * The ``scenario`` part of the config is irrelevant when using the tool in this fashion, as you define all input parameters by giving the name of the desired file with its wildcards. * The other parts (e.g., ``data_dir`` or ``cordexParameters``) are still relevant. * You can see which files are available by looking into :doc:`descriptionOfRules`. Legacy method ``conda`` (deprecated) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Warning: Using ``conda`` as a package manager is no longer supported or maintained. Please consider using ``pixi`` instead. Installation process ``conda`` (deprecated) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ #. Download and install Miniconda (https://docs.conda.io/projects/miniconda/en/latest/index.html). #. Make sure that Miniconda and Git are added to the ``PATH`` variable (either by checking the box during the installation or by adding it later manually). You can check if it was successful by typing ``git``/``conda`` into a terminal. If the result is not "command not found," you were successful. #. Open the shell Git Bash (other shells, especially Windows PowerShell, do not work). #. Clone the repository using Git by typing ``git clone https://gitlab.ruhr-uni-bochum.de/ee/cd2es.git {folderName}``. Replace ``{folderName}`` with the folder where you want the software to be saved. #. Navigate to the folder of the repository using ``cd pathToYourRepository``. #. Go into the folder ``install`` by using ``cd install``. #. Run the first install script with ``bash install.sh`` to set up all dependencies for *cd2es*. #. You should be ready to go now! The setup files do the following: * Install Mamba for easier package management * Install conda environment ``cd2es`` with all necessary packages * On Windows only: * Download ``wget`` for Windows * Install CDO (climate data processing tool) in the WSL subsystem * On Linux: install CDO (climate data processing tool) Manual installation ``conda`` (deprecated) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In case the bash scripts for installation fail, we provide the necessary information to do a manual installation. #. Initialize conda with the command ``conda init bash``. #. Install Mamba with the command ``conda install mamba -n base -c conda-forge``. #. Set up the Python environment using the environment file ``environment.yaml`` with the command: ``mamba create -n cd2es -f environment.yaml``. #. Activate the environment with ``conda activate cd2es``. #. Install CDO (tool necessary for working with CORDEX climate data). * On Linux: ``sudo apt-get install cdo`` * On Windows: you need the Linux subsystem WSL * Install WSL: execute ``wsl --install`` * Open WSL * Install CDO in WSL with ``sudo apt-get install cdo`` * If you receive the message ``unable to locate package``, run ``sudo apt update``. * To activate WSL, it is necessary to have virtualization activated in BIOS (https://support.microsoft.com/en-us/windows/enable-virtualization-on-windows-c5578302-6e43-4b4b-a449-8ced115f58e1). #. If you want to download CORDEX climate data automatically, you need ``wget``. * On Linux, ``wget`` should already be installed. If not, use ``sudo apt-get install wget``. * On Windows, you can get ``wget`` via https://eternallybored.org/misc/wget/. Copy ``wget.exe`` to the folder ``C:/Windows/System32``. * You also need to install Java (https://www.oracle.com/java/technologies/downloads/). Basic usage principle ``conda`` (deprecated) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The tool is based on the workflow management tool Snakemake. You use it by going to the repository folder in a terminal (preferably Git Bash). You configure the tool by using a :file:`config.yaml` file. Therefore, first, copy the :file:`config_default.yaml` to the same folder, rename it to :file:`config.yaml` and adapt it to your purposes. In the beginning, it is enough to change the ``data_dir`` and the information given under "scenario" until line 21. The rest of the config contains details on how to download CORDEX climate data and ERA5 reanalysis data (for regression and bias adaptation) and data for the conversion from climate data to energy system input data (like wind turbine hub height). Activate the environment with ``conda activate cd2es``. Then, there are two ways of using the tool: * Using the command: ``snakemake -j7 --resources mem_mb=10000`` * With this command, the Snakemake routine will produce all files as specified in the config under ``scenario``. * The command ``-j`` is required and specifies the number of cores Snakemake can use. Adopt it to your computer. * ``--resources mem_mb=10000`` is used to restrict the execution of RAM-intensive operations. If you have a lot of RAM, you can increase the number. If you get errors like "Unable to allocate memory," you should decrease the number. * If you want to produce only a single file, you can use the command ``snakemake -j7 nameOfDesiredFile --resources mem_mb=10000`` while replacing "nameOfDesiredFile" with the path and name of the file you want to produce. * The ``scenario`` part of the config is irrelevant when using the tool in this fashion, as you define all input parameters by giving the name of the desired file with its wildcards. * The other parts (e.g., ``data_dir`` or ``cordexParameters``) are still relevant. * You can see which files are available by looking into :doc:`descriptionOfRules`.