.. _data-sources: Data sources ============ An eDisGo study combines two kinds of input data: the **grid topology** (where the lines, transformers, loads and generators are) and **scenario / time-series data** (how much they consume or feed in, and which flexibilities exist). This page explains where each comes from and how to access it. .. contents:: :local: :depth: 1 Grid topology — ding0 --------------------- eDisGo works on synthetic medium- and low-voltage grids generated by `ding0 `_. ding0 produces a complete, realistic MV/LV grid for each German *MV grid district*; a grid is identified by that district number (e.g. ``32377``). A ding0 grid is a directory of CSV files (``buses.csv``, ``lines.csv``, ``loads.csv``, ``generators.csv``, ``transformers.csv``, ``transformers_hvmv.csv``, ``storage_units.csv``, ``switches.csv``, ``network.csv``; the CSVs may also sit in a ``topology/`` subfolder). It is loaded by passing the directory to the :class:`~edisgo.edisgo.EDisGo` constructor: .. code-block:: python from edisgo import EDisGo edisgo = EDisGo(ding0_grid="path/to/ding0_grid", legacy_ding0_grids=False) * ``legacy_ding0_grids`` **defaults to** ``True``. Set it to ``False`` for the **current** (egon-data compatible) grid format; ``True`` is only for older ding0 grids. Omitting it on a current grid triggers the legacy code path (different column renames) and mis-imports the grid. Internally the constructor calls :meth:`~edisgo.edisgo.EDisGo.import_ding0_grid`, which you can also invoke directly on an existing :class:`~edisgo.edisgo.EDisGo` object. Where to get ding0 grids: * **Zenodo** — ready-made example grids are published on `Zenodo (record 10405129) `_; always pick the latest record. * **Generate your own** — follow the `ding0 documentation `_. Scenario & time-series data — OpenEnergy Platform / egon-data ------------------------------------------------------------- Everything scenario-dependent — future generator parks, load profiles, heat-pump COP and heat demand, demand-side-management potential, home-battery and electric-vehicle data — is loaded from the **egon-data** dataset hosted on the `OpenEnergy Platform (OEP) `_. The relevant import methods are :meth:`~edisgo.edisgo.EDisGo.import_generators`, :meth:`~edisgo.edisgo.EDisGo.import_electromobility`, :meth:`~edisgo.edisgo.EDisGo.import_heat_pumps`, :meth:`~edisgo.edisgo.EDisGo.import_dsm`, :meth:`~edisgo.edisgo.EDisGo.import_home_batteries` and :meth:`~edisgo.edisgo.EDisGo.set_time_series_active_power_predefined`. Most of these select a scenario with the ``scenario`` argument — ``"eGon2035"`` or ``"eGon100RE"``. Two differ: :meth:`~edisgo.edisgo.EDisGo.import_generators` takes ``generator_scenario`` (``"eGon2035"``/``"eGon100RE"`` for current ding0 grids, or ``"nep2035"``/``"ego100"`` for legacy grids), and :meth:`~edisgo.edisgo.EDisGo.set_time_series_active_power_predefined` selects its sources per component type (e.g. ``fluctuating_generators_ts="oedb"``). In eDisGo's API the OEP data source is referred to as ``"oedb"`` (OpenEnergy DataBase) for historical reasons. OEP database (standard) ~~~~~~~~~~~~~~~~~~~~~~~~~ The OEP is the default data source. For most import methods, when the ``engine`` argument is omitted a default OEP engine is created automatically: .. code-block:: python edisgo.set_time_series_active_power_predefined( fluctuating_generators_ts="oedb", # load feed-in profiles from the OEP ) .. note:: :meth:`~edisgo.edisgo.EDisGo.import_electromobility` is an exception: with ``data_source="oedb"`` it does **not** create a default engine, so you must pass an explicit ``engine``. **OEP token.** To authenticate, place a file named ``OEP_TOKEN.txt`` containing your personal access token in the directory ``edisgo/config/`` (or set the environment variable ``OEP_TOKEN``). A token can be requested at https://openenergyplatform.org/. Using a token is optional but recommended, as anonymous access is rate-limited. Custom PostgreSQL database (egon-data) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can also connect to your own egon-data PostgreSQL database via an SSH tunnel. Connecting through :func:`~edisgo.io.db.engine` always uses an SSH tunnel (``ssh=True`` with a complete ``ssh-tunnel`` section); for a tunnel-less local database you have to build a SQLAlchemy engine yourself. The database keys follow the egon-data CLI-flag style and must be prefixed with ``--``. Create a YAML configuration file: .. code-block:: yaml egon-data: --database-host: mydb.example.org --database-port: 5432 --database-name: egon_data --database-user: my_user --database-password: my_password ssh-tunnel: # required when ssh=True ssh-host: my.ssh.server ssh-user: ubuntu ssh-pkey: ~/.ssh/id_rsa pgres-host: localhost then create an engine with :func:`~edisgo.io.db.engine` and pass it to any import method: .. code-block:: python from edisgo.io.db import engine eng = engine(path="path/to/database_config.yaml", ssh=True) edisgo.import_heat_pumps(scenario="eGon2035", engine=eng) .. list-table:: :header-rows: 1 :widths: 25 45 30 * - Access type - Description - Authentication * - OEP (default) - Official egon-data source on the OpenEnergy Platform - OEP token (recommended) * - Custom PostgreSQL - Your own egon-data database via an SSH tunnel - YAML config file with ``--`` credentials + ``ssh-tunnel`` section Load profiles — demandlib -------------------------- Where no measured or scenario profile is available, standard load profiles for the residential, commercial, agricultural and industrial sectors are generated with the oemof `demandlib `_. This is selected with ``conventional_loads_ts="demandlib"`` in :meth:`~edisgo.edisgo.EDisGo.set_time_series_active_power_predefined`. Electric-vehicle data — SimBEV / TracBEV ---------------------------------------- When not taken from the OEP, electric-vehicle data can be imported from a directory with :meth:`~edisgo.edisgo.EDisGo.import_electromobility` using ``data_source="directory"`` and the ``charging_processes_dir`` / ``potential_charging_points_dir`` arguments. The data is produced by `SimBEV `_ (charging processes: standing times, charging demand per vehicle) and `TracBEV `_ (potential charging-point locations). The import is tied to specific tool versions — SimBEV commit ``3083c5a`` and TracBEV commit ``14d864c``; data from other versions is not guaranteed to parse. See :ref:`electromobility-methodology` and the :doc:`../tutorials/electromobility_example` notebook. Configuration data ------------------- Defaults for grid expansion (equipment, costs, allowed voltage deviations, load factors), worst-case assumptions and database tables live in the package configuration files. They are documented in :ref:`default_configs` and the equipment data in :ref:`equipment`.