lomas_server.data_connector package

Submodules

lomas_server.data_connector.data_connector module

class lomas_server.data_connector.data_connector.DataConnector(metadata: Metadata)[source]

Bases: ABC

Overall access to sensitive data.

df: DataFrame | None = None
get_metadata() → Metadata[source]

Get the metadata for this dataset.

Returns:

The metadata object.

Return type:

Metadata

abstract get_pandas_df() → DataFrame[source]

Get the data in pandas dataframe format.

Returns:

The pandas dataframe for this dataset.

Return type:

pd.DataFrame

get_polars_lf() → LazyFrame[source]

Get the data in polars lazyframe format.

Returns:

The polars lazyframe for this dataset.

Return type:

pl.LazyFrame

lomas_server.data_connector.data_connector.get_column_dtypes(metadata: Metadata) → tuple[dict[str, str], list[str]][source]

Extracts and returns the column types from the metadata.

Parameters:

metadata (Metadata) – The metadata.

Returns:

dict: The dictionary of the column type.

list: The list of columns of datetime type

Return type:

Tuple[Dict[str, str], List[str]]

lomas_server.data_connector.factory module

lomas_server.data_connector.factory.data_connector_factory(dataset_name: str, admin_database: AdminDatabase, private_db_credentials: dict[int, PrivateDBCredentials]) → DataConnector[source]

Returns the appropriate dataset class based on dataset storage location.

Parameters:

  • dataset_name (str) – The dataset name.

  • admin_database (AdminDatabase) – An initialized instance of AdminDatabase.

Raises:

InternalServerException – If the dataset type does not exist.

Returns:

The DataConnector instance for this dataset.

Return type:

DataConnector

lomas_server.data_connector.factory.get_dataset_credentials(private_db_credentials: dict[int, PrivateDBCredentials], db_type: PrivateDatabaseType, credentials_name: str) → PrivateDBCredentials[source]

Search the list of private database credentials and.

returns the one that matches the database type and credentials name.

Parameters:

  • private_db_credentials (Sequence[PrivateDBCredentials]) – The list of private database credentials.

  • db_type (PrivateDatabaseType) – The type of the database.

Raises:

InternalServerException – If the credentials are not found.

Returns:

The matching credentials.

Return type:

PrivateDBCredentials

lomas_server.data_connector.in_memory_connector module

class lomas_server.data_connector.in_memory_connector.InMemoryConnector(metadata: Metadata, dataset_df: DataFrame)[source]

Bases: DataConnector

DataConnector for a dataset created from an in-memory pandas DataFrame.

get_pandas_df() → DataFrame[source]

Get the data in pandas dataframe format.

Returns:

pandas dataframe of dataset (a copy)

Return type:

pd.DataFrame

lomas_server.data_connector.path_connector module

class lomas_server.data_connector.path_connector.PathConnector(metadata: Metadata, dataset_path: str)[source]

Bases: DataConnector

DataConnector for dataset located at constant path.

Path can be local or remote (http).

get_pandas_df() → DataFrame[source]

Get the data in pandas dataframe format.

Raises:

InternalServerException – If the file format is not supported.

Returns:

pandas dataframe of dataset

Return type:

pd.DataFrame

lomas_server.data_connector.s3_connector module

class lomas_server.data_connector.s3_connector.S3Connector(metadata: Metadata, credentials: DSS3Access)[source]

Bases: DataConnector

DataConnector for dataset in S3 storage.

get_pandas_df() → DataFrame[source]

Get the data in pandas dataframe format.

Raises:

InternalServerException – If the dataset cannot be read.

Returns:

pandas dataframe of dataset

Return type:

pd.DataFrame

Module contents