Skip to content

pandas_openscm.db.interfaces#

Interfaces used throughout the db (database) module

Classes:

Name Description
OpenSCMDBDataBackend

Backend for (de-)serialising data
OpenSCMDBIndexBackend

Backend for (de-)serialising the index (and file map)

OpenSCMDBDataBackend #

Bases: Protocol

Backend for (de-)serialising data

Designed to be used with OpenSCMDB

Methods:

Name Description
load_data

Load a data file
save_data

Save data to disk

Attributes:

Name Type Description
ext str

Extension to use with data files saved by this backend.
preserves_index bool

Whether this backend preserves the index of data upon (de-)serialisation
Source code in src/pandas_openscm/db/interfaces.py 
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
@runtime_checkable
class OpenSCMDBDataBackend(Protocol):
    """
    Backend for (de-)serialising data

    Designed to be used with [OpenSCMDB][(m)]
    """

    ext: str
    """
    Extension to use with data files saved by this backend.
    """

    preserves_index: bool
    """
    Whether this backend preserves the index of data upon (de-)serialisation
    """

    @staticmethod
    def load_data(data_file: Path) -> pd.DataFrame:
        """
        Load a data file

        This is a low-level method
        that just handles the specifics of loading the data from disk.
        Working out the path from which to load the data
        should happen in higher-level functions.

        Parameters
        ----------
        data_file
            File from which to load the data

        Returns
        -------
        :
            Loaded data

        Notes
        -----
        This just loads the data directly from disk.
        If the data had a `pd.MultiIndex` when it was saved,
        this may or not be restored.
        It is up to the user
        to decide whether to do any `pd.MultiIndex` restoration or not,
        based on their use case and the value of `self.preserves_index`.
        We do not make this choice as converting back to a
        `pd.MultiIndex` can be a very expensive operation,
        and we want to give the user control over any such optimisations.
        """

    @staticmethod
    def save_data(
        data: pd.DataFrame,
        data_file: Path,
    ) -> None:
        """
        Save data to disk

        This is a low-level method
        that just handles the specifics of serialising the data to disk.
        Working out what to save and in what path
        should happen in higher-level functions.

        Parameters
        ----------
        data
            Data to save

        data_file
            File in which to save the data
        """

ext instance-attribute #

ext: str

Extension to use with data files saved by this backend.

preserves_index instance-attribute #

preserves_index: bool

Whether this backend preserves the index of data upon (de-)serialisation

load_data staticmethod #

load_data(data_file: Path) -> DataFrame

Load a data file

This is a low-level method that just handles the specifics of loading the data from disk. Working out the path from which to load the data should happen in higher-level functions.

Parameters:

Name Type Description Default
data_file Path

File from which to load the data

 required

Returns:

Type Description
DataFrame

Loaded data
Notes

This just loads the data directly from disk. If the data had a pd.MultiIndex when it was saved, this may or not be restored. It is up to the user to decide whether to do any pd.MultiIndex restoration or not, based on their use case and the value of self.preserves_index. We do not make this choice as converting back to a pd.MultiIndex can be a very expensive operation, and we want to give the user control over any such optimisations.

Source code in src/pandas_openscm/db/interfaces.py 
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
@staticmethod
def load_data(data_file: Path) -> pd.DataFrame:
    """
    Load a data file

    This is a low-level method
    that just handles the specifics of loading the data from disk.
    Working out the path from which to load the data
    should happen in higher-level functions.

    Parameters
    ----------
    data_file
        File from which to load the data

    Returns
    -------
    :
        Loaded data

    Notes
    -----
    This just loads the data directly from disk.
    If the data had a `pd.MultiIndex` when it was saved,
    this may or not be restored.
    It is up to the user
    to decide whether to do any `pd.MultiIndex` restoration or not,
    based on their use case and the value of `self.preserves_index`.
    We do not make this choice as converting back to a
    `pd.MultiIndex` can be a very expensive operation,
    and we want to give the user control over any such optimisations.
    """

save_data staticmethod #

save_data(data: DataFrame, data_file: Path) -> None

Save data to disk

This is a low-level method that just handles the specifics of serialising the data to disk. Working out what to save and in what path should happen in higher-level functions.

Parameters:

Name Type Description Default
data DataFrame

Data to save

 required
data_file Path

File in which to save the data

 required
Source code in src/pandas_openscm/db/interfaces.py 
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
@staticmethod
def save_data(
    data: pd.DataFrame,
    data_file: Path,
) -> None:
    """
    Save data to disk

    This is a low-level method
    that just handles the specifics of serialising the data to disk.
    Working out what to save and in what path
    should happen in higher-level functions.

    Parameters
    ----------
    data
        Data to save

    data_file
        File in which to save the data
    """

OpenSCMDBIndexBackend #

Bases: Protocol

Backend for (de-)serialising the index (and file map)

Designed to be used with OpenSCMDB

Methods:

Name Description
load_file_map

Load the file map
load_index

Load the index
save_file_map

Save the file map to disk
save_index

Save the index to disk

Attributes:

Name Type Description
ext str

Extension to use with index files saved by this backend.
preserves_index bool

Whether this backend preserves the pd.MultiIndex upon (de-)serialisation
Source code in src/pandas_openscm/db/interfaces.py 
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
@runtime_checkable
class OpenSCMDBIndexBackend(Protocol):
    """
    Backend for (de-)serialising the index (and file map)

    Designed to be used with [OpenSCMDB][(m)]
    """

    ext: str
    """
    Extension to use with index files saved by this backend.
    """

    preserves_index: bool
    """
    Whether this backend preserves the `pd.MultiIndex` upon (de-)serialisation
    """

    @staticmethod
    def load_file_map(file_map_file: Path) -> pd.DataFrame:
        """
        Load the file map

        This is a low-level method
        that just handles the specifics of loading the index from disk.
        Working out the path from which to load the file map
        should happen in higher-level functions.

        Parameters
        ----------
        file_map_file
            File from which to load the file map

        Returns
        -------
        :
            Loaded file map

        Notes
        -----
        This returns a [pd.DataFrame][pandas.DataFrame].
        It is up to the user to cast this to a [pd.Series][pandas.Series]
        if they wish.
        """

    @staticmethod
    def load_index(index_file: Path) -> pd.DataFrame:
        """
        Load the index

        This is a low-level method
        that just handles the specifics of loading the index from disk.
        Working out the path from which to load the index
        should happen in higher-level functions.

        Parameters
        ----------
        index_file
            File from which to load the index

        Returns
        -------
        :
            Loaded index

        Notes
        -----
        This just loads the index directly from disk.
        If the index had a `pd.MultiIndex` when it was saved,
        this may or not be restored.
        It is up to the user
        to decide whether to do any `pd.MultiIndex` restoration or not,
        based on their use case and the value of `self.preserves_index`.
        We do not make this choice as converting back to a
        `pd.MultiIndex` can be a very expensive operation,
        and we want to give the user control over any such optimisations.
        """

    def save_file_map(
        self,
        file_map: pd.Series[Path],  # type: ignore # pandas confused about what it supports
        file_map_file: Path,
    ) -> None:
        """
        Save the file map to disk

        This is a low-level method
        that just handles the specifics of serialising the file map to disk.
        Working out what to save and in what path
        should happen in higher-level functions.

        Parameters
        ----------
        file_map
            File map to save

        file_map_file
            File in which to save the file map
        """

    def save_index(
        self,
        index: pd.DataFrame,
        index_file: Path,
    ) -> None:
        """
        Save the index to disk

        This is a low-level method
        that just handles the specifics of serialising the index to disk.
        Working out what to save and in what path
        should happen in higher-level functions.

        Parameters
        ----------
        index
            Index to save

        index_file
            File in which to save the index
        """

ext instance-attribute #

ext: str

Extension to use with index files saved by this backend.

preserves_index instance-attribute #

preserves_index: bool

Whether this backend preserves the pd.MultiIndex upon (de-)serialisation

load_file_map staticmethod #

load_file_map(file_map_file: Path) -> DataFrame

Load the file map

This is a low-level method that just handles the specifics of loading the index from disk. Working out the path from which to load the file map should happen in higher-level functions.

Parameters:

Name Type Description Default
file_map_file Path

File from which to load the file map

 required

Returns:

Type Description
DataFrame

Loaded file map
Notes

This returns a pd.DataFrame. It is up to the user to cast this to a pd.Series if they wish.

Source code in src/pandas_openscm/db/interfaces.py 
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
@staticmethod
def load_file_map(file_map_file: Path) -> pd.DataFrame:
    """
    Load the file map

    This is a low-level method
    that just handles the specifics of loading the index from disk.
    Working out the path from which to load the file map
    should happen in higher-level functions.

    Parameters
    ----------
    file_map_file
        File from which to load the file map

    Returns
    -------
    :
        Loaded file map

    Notes
    -----
    This returns a [pd.DataFrame][pandas.DataFrame].
    It is up to the user to cast this to a [pd.Series][pandas.Series]
    if they wish.
    """

load_index staticmethod #

load_index(index_file: Path) -> DataFrame

Load the index

This is a low-level method that just handles the specifics of loading the index from disk. Working out the path from which to load the index should happen in higher-level functions.

Parameters:

Name Type Description Default
index_file Path

File from which to load the index

 required

Returns:

Type Description
DataFrame

Loaded index
Notes

This just loads the index directly from disk. If the index had a pd.MultiIndex when it was saved, this may or not be restored. It is up to the user to decide whether to do any pd.MultiIndex restoration or not, based on their use case and the value of self.preserves_index. We do not make this choice as converting back to a pd.MultiIndex can be a very expensive operation, and we want to give the user control over any such optimisations.

Source code in src/pandas_openscm/db/interfaces.py 
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
@staticmethod
def load_index(index_file: Path) -> pd.DataFrame:
    """
    Load the index

    This is a low-level method
    that just handles the specifics of loading the index from disk.
    Working out the path from which to load the index
    should happen in higher-level functions.

    Parameters
    ----------
    index_file
        File from which to load the index

    Returns
    -------
    :
        Loaded index

    Notes
    -----
    This just loads the index directly from disk.
    If the index had a `pd.MultiIndex` when it was saved,
    this may or not be restored.
    It is up to the user
    to decide whether to do any `pd.MultiIndex` restoration or not,
    based on their use case and the value of `self.preserves_index`.
    We do not make this choice as converting back to a
    `pd.MultiIndex` can be a very expensive operation,
    and we want to give the user control over any such optimisations.
    """

save_file_map #

save_file_map(
    file_map: Series[Path], file_map_file: Path
) -> None

Save the file map to disk

This is a low-level method that just handles the specifics of serialising the file map to disk. Working out what to save and in what path should happen in higher-level functions.

Parameters:

Name Type Description Default
file_map Series[Path]

File map to save

 required
file_map_file Path

File in which to save the file map

 required
Source code in src/pandas_openscm/db/interfaces.py 
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def save_file_map(
    self,
    file_map: pd.Series[Path],  # type: ignore # pandas confused about what it supports
    file_map_file: Path,
) -> None:
    """
    Save the file map to disk

    This is a low-level method
    that just handles the specifics of serialising the file map to disk.
    Working out what to save and in what path
    should happen in higher-level functions.

    Parameters
    ----------
    file_map
        File map to save

    file_map_file
        File in which to save the file map
    """

save_index #

save_index(index: DataFrame, index_file: Path) -> None

Save the index to disk

This is a low-level method that just handles the specifics of serialising the index to disk. Working out what to save and in what path should happen in higher-level functions.

Parameters:

Name Type Description Default
index DataFrame

Index to save

 required
index_file Path

File in which to save the index

 required
Source code in src/pandas_openscm/db/interfaces.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
def save_index(
    self,
    index: pd.DataFrame,
    index_file: Path,
) -> None:
    """
    Save the index to disk

    This is a low-level method
    that just handles the specifics of serialising the index to disk.
    Working out what to save and in what path
    should happen in higher-level functions.

    Parameters
    ----------
    index
        Index to save

    index_file
        File in which to save the index
    """