lomas_server package

Subpackages

• lomas_server.admin_database package
  • Submodules
  • lomas_server.admin_database.admin_database module
    • AdminDatabase
      • AdminDatabase.does_dataset_exist()
      • AdminDatabase.does_user_exist()
      • AdminDatabase.get_and_set_may_user_query()
      • AdminDatabase.get_dataset_field()
      • AdminDatabase.get_dataset_metadata()
      • AdminDatabase.get_epsilon_or_delta()
      • AdminDatabase.get_initial_budget()
      • AdminDatabase.get_remaining_budget()
      • AdminDatabase.get_total_spent_budget()
      • AdminDatabase.get_user_previous_queries()
      • AdminDatabase.has_user_access_to_dataset()
      • AdminDatabase.prepare_save_query()
      • AdminDatabase.save_query()
      • AdminDatabase.set_may_user_query()
      • AdminDatabase.update_budget()
      • AdminDatabase.update_delta()
      • AdminDatabase.update_epsilon()
      • AdminDatabase.update_epsilon_or_delta()
    • dataset_must_exist()
    • user_must_exist()
    • user_must_have_access_to_dataset()
  • lomas_server.admin_database.constants module
  • lomas_server.admin_database.factory module
  • lomas_server.admin_database.mongodb_database module
    • AdminMongoDatabase
      • AdminMongoDatabase.does_dataset_exist()
      • AdminMongoDatabase.does_user_exist()
      • AdminMongoDatabase.get_and_set_may_user_query()
      • AdminMongoDatabase.get_dataset_field()
      • AdminMongoDatabase.get_dataset_metadata()
      • AdminMongoDatabase.get_epsilon_or_delta()
      • AdminMongoDatabase.get_user_previous_queries()
      • AdminMongoDatabase.has_user_access_to_dataset()
      • AdminMongoDatabase.save_query()
      • AdminMongoDatabase.set_may_user_query()
      • AdminMongoDatabase.update_epsilon_or_delta()
    • check_result_acknowledged()
  • lomas_server.admin_database.utils module
    • database_factory()
    • get_mongodb()
    • get_mongodb_url()
  • lomas_server.admin_database.yaml_database module
    • AdminYamlDatabase
      • AdminYamlDatabase.does_dataset_exist()
      • AdminYamlDatabase.does_user_exist()
      • AdminYamlDatabase.get_and_set_may_user_query()
      • AdminYamlDatabase.get_dataset_field()
      • AdminYamlDatabase.get_dataset_metadata()
      • AdminYamlDatabase.get_epsilon_or_delta()
      • AdminYamlDatabase.get_user_previous_queries()
      • AdminYamlDatabase.has_user_access_to_dataset()
      • AdminYamlDatabase.save_current_database()
      • AdminYamlDatabase.save_query()
      • AdminYamlDatabase.set_may_user_query()
      • AdminYamlDatabase.update_epsilon_or_delta()
  • Module contents
• lomas_server.administration package
  • Subpackages
    • lomas_server.administration.tests package
      • Submodules
      • lomas_server.administration.tests.test_streamlit_app module
      • Module contents
  • Module contents
• lomas_server.data_connector package
  • Submodules
  • lomas_server.data_connector.data_connector module
  • lomas_server.data_connector.factory module
  • lomas_server.data_connector.in_memory_connector module
  • lomas_server.data_connector.path_connector module
  • lomas_server.data_connector.s3_connector module
  • Module contents
• lomas_server.dp_queries package
  • Subpackages
    • lomas_server.dp_queries.dp_libraries package
      • Submodules
      • lomas_server.dp_queries.dp_libraries.diffprivlib module
      • lomas_server.dp_queries.dp_libraries.factory module
      • lomas_server.dp_queries.dp_libraries.opendp module
      • lomas_server.dp_queries.dp_libraries.smartnoise_sql module
      • lomas_server.dp_queries.dp_libraries.smartnoise_synth module
      • lomas_server.dp_queries.dp_libraries.utils module
      • Module contents
  • Submodules
  • lomas_server.dp_queries.dp_querier module
    • DPQuerier
      • DPQuerier.cost()
      • DPQuerier.query()
  • lomas_server.dp_queries.dummy_dataset module
    • get_dummy_dataset_for_query()
    • make_dummy_dataset()
  • Module contents
• lomas_server.routes package
  • Submodules
  • lomas_server.routes.routes_admin module
  • lomas_server.routes.routes_dp module
  • lomas_server.routes.utils module
  • Module contents
• lomas_server.tests package
  • Submodules
  • lomas_server.tests.constants module
  • lomas_server.tests.test_api module
  • lomas_server.tests.test_api_diffprivlib module
  • lomas_server.tests.test_api_smartnoise_synth module
  • lomas_server.tests.test_collection_models module
  • lomas_server.tests.test_dummy_generation module
    • TestMakeDummyDataset
      • TestMakeDummyDataset.test_boolean_column()
      • TestMakeDummyDataset.test_cardinality_column()
      • TestMakeDummyDataset.test_datetime_column()
      • TestMakeDummyDataset.test_float_column()
      • TestMakeDummyDataset.test_int_column()
      • TestMakeDummyDataset.test_nullable_column()
      • TestMakeDummyDataset.test_seed()
      • TestMakeDummyDataset.test_unknown_column()
  • lomas_server.tests.test_mongodb_admin module
  • lomas_server.tests.test_mongodb_admin_cli module
  • Module contents
• lomas_server.utils package
  • Submodules
  • lomas_server.utils.config module
    • Config
      • Config.admin_database
      • Config.dataset_store
      • Config.develop_mode
      • Config.dp_libraries
      • Config.model_computed_fields
      • Config.model_config
      • Config.model_fields
      • Config.server
      • Config.submit_limit
    • ConfigLoader
      • ConfigLoader.get_config()
      • ConfigLoader.load_config()
      • ConfigLoader.set_config()
    • DBConfig
      • DBConfig.db_type
      • DBConfig.model_computed_fields
      • DBConfig.model_config
      • DBConfig.model_fields
    • DPLibraryConfig
      • DPLibraryConfig.model_computed_fields
      • DPLibraryConfig.model_config
      • DPLibraryConfig.model_fields
      • DPLibraryConfig.opendp
    • DatasetStoreConfig
      • DatasetStoreConfig.ds_store_type
      • DatasetStoreConfig.model_computed_fields
      • DatasetStoreConfig.model_config
      • DatasetStoreConfig.model_fields
    • LRUDatasetStoreConfig
      • LRUDatasetStoreConfig.max_memory_usage
      • LRUDatasetStoreConfig.model_computed_fields
      • LRUDatasetStoreConfig.model_config
      • LRUDatasetStoreConfig.model_fields
    • MongoDBConfig
      • MongoDBConfig.address
      • MongoDBConfig.db_name
      • MongoDBConfig.model_computed_fields
      • MongoDBConfig.model_config
      • MongoDBConfig.model_fields
      • MongoDBConfig.password
      • MongoDBConfig.port
      • MongoDBConfig.username
    • OpenDPConfig
      • OpenDPConfig.contrib
      • OpenDPConfig.floating_point
      • OpenDPConfig.honest_but_curious
      • OpenDPConfig.model_computed_fields
      • OpenDPConfig.model_config
      • OpenDPConfig.model_fields
    • Server
      • Server.host_ip
      • Server.host_port
      • Server.log_level
      • Server.model_computed_fields
      • Server.model_config
      • Server.model_fields
      • Server.reload
      • Server.time_attack
      • Server.workers
    • TimeAttack
      • TimeAttack.magnitude
      • TimeAttack.method
      • TimeAttack.model_computed_fields
      • TimeAttack.model_config
      • TimeAttack.model_fields
    • YamlDBConfig
      • YamlDBConfig.db_file
      • YamlDBConfig.model_computed_fields
      • YamlDBConfig.model_config
      • YamlDBConfig.model_fields
    • get_config()
  • lomas_server.utils.query_examples module
  • Module contents

Submodules

lomas_server.app module

lomas_server.app.lifespan(app: FastAPI) → AsyncGenerator

Lifespan function for the server.

This function is executed once on server startup, yields and finishes running at server shutdown.

Server initialization is performed (config loading, etc.) and the server state is updated accordingly. This can have potential side effects on the return values of the “depends” functions, which check the server state.

async lomas_server.app.middleware(request: Request, call_next: Callable[[Request], Response]) → Response

Adds delays to requests response to protect against timing attack

lomas_server.constants module

class lomas_server.constants.AdminDBType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Types of administration databases

MONGODB: str = 'mongodb'

YAML: str = 'yaml'

class lomas_server.constants.ConfigKeys(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Keys of the configuration file

DATASET_STORE: str = 'dataset_store'

DATASET_STORE_TYPE: str = 'ds_store_type'

DB: str = 'admin_database'

DB_TYPE: str = 'db_type'

DB_TYPE_MONGODB: str = 'mongodb'

DEVELOP_MODE: str = 'develop_mode'

DP_LIBRARY: str = 'dp_libraries'

LRU_DATASET_STORE_MAX_SIZE: str = 'max_memory_usage'

MONGODB_ADDR: str = 'address'

MONGODB_PORT: str = 'port'

RUNTIME_ARGS: str = 'runtime_args'

SERVER: str = 'server'

SETTINGS: str = 'settings'

SUBMIT_LIMIT: str = 'submit_limit'

TIME_ATTACK: str = 'time_attack'

class lomas_server.constants.DPLibraries(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Name of DP Library used in the query

DIFFPRIVLIB = 'diffprivlib'

OPENDP = 'opendp'

SMARTNOISE_SQL = 'smartnoise_sql'

class lomas_server.constants.DatasetStoreType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Types of classes to handle datasets in memory

BASIC: str = 'basic'

LRU: str = 'LRU_cache'

class lomas_server.constants.OpenDPDatasetInputMetric(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Type of opendp input metric for datasets see https://docs.opendp.org/en/stable/api/python/opendp.metrics.html see https://github.com/opendp/opendp/blob/main/rust/src/metrics/mod.rs

CHANGE_ONE_DISTANCE = 'ChangeOneDistance'

HAMMING_DISTANCE = 'HammingDistance'

INSERT_DELETE_DISTANCE = 'InsertDeleteDistance'

INT_DISTANCE = 'u32'

SYMMETRIC_DISTANCE = 'SymmetricDistance'

class lomas_server.constants.OpenDPMeasurement(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Type of divergence for opendp measurement see https://docs.opendp.org/en/stable/api/python/opendp.measurements.html

FIXED_SMOOTHED_MAX_DIVERGENCE = 'fixed_smoothed_max_divergence'

MAX_DIVERGENCE = 'max_divergence'

SMOOTHED_MAX_DIVERGENCE = 'smoothed_max_divergence'

ZERO_CONCENTRATED_DIVERGENCE = 'zero_concentrated_divergence'

class lomas_server.constants.PrivateDatabaseType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Type of Private Database for the private data

PATH = 'PATH_DB'

S3 = 'S3_DB'

class lomas_server.constants.TimeAttackMethod(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Possible methods against timing attacks

JITTER = 'jitter'

STALL = 'stall'

lomas_server.mongodb_admin module

lomas_server.mongodb_admin.add_dataset(db: Database, dataset_name: str, database_type: str, metadata_database_type: str, dataset_path: str | None = '', metadata_path: str | None = '', s3_bucket: str | None = '', s3_key: str | None = '', endpoint_url: str | None = '', aws_access_key_id: str | None = '', aws_secret_access_key: str | None = '', metadata_s3_bucket: str | None = '', metadata_s3_key: str | None = '', metadata_endpoint_url: str | None = '', metadata_aws_access_key_id: str | None = '', metadata_aws_secret_access_key: str | None = '') → None

Set a database type to a dataset in dataset collection.

Parameters:

• db (Database) – mongo database object

• dataset_name (str) – Dataset name

• database_type (str) – Type of the database

• metadata_database_type (str) – Metadata database type

• dataset_path (str) – Path to the dataset (for local db type)

• metadata_path (str) – Path to metadata (for local db type)

• s3_bucket (str) – S3 bucket name

• s3_key (str) – S3 key

• endpoint_url (str) – S3 endpoint URL

• aws_access_key_id (str) – AWS access key ID

• aws_secret_access_key (str) – AWS secret access key

• metadata_s3_bucket (str) – Metadata S3 bucket name

• metadata_s3_key (str) – Metadata S3 key

• metadata_endpoint_url (str) – Metadata S3 endpoint URL

• metadata_aws_access_key_id (str) – Metadata AWS access key ID

• metadata_aws_secret_access_key (str) – Metadata AWS secret access key

Raises:

ValueError – If the dataset already exists or if the database type is unknown.

Returns:

None

lomas_server.mongodb_admin.add_dataset_to_user(db: Database, user: str, dataset: str, epsilon: float, delta: float) → None

Add dataset with initialized budget values to list of datasets that the user has access to. Will not add if already added (no error will be raised in that case).

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to check

• dataset (str) – name of the dataset to add to user

• epsilon (float) – epsilon value for initial budget of user

• delta (float) – delta value for initial budget of user

Raises:

ValueError – _description_

Returns:

None

lomas_server.mongodb_admin.add_datasets_via_yaml(db: Database, yaml_file: str | Dict, clean: bool, overwrite_datasets: bool, overwrite_metadata: bool) → None

Set all database types to datasets in dataset collection based on yaml file.

Parameters:

• db (Database) – mongo database object

• yaml_file (Union[str, Dict]) – if str: a path to the YAML file location if Dict: a dictionnary containing the collection data

• clean (bool) – Whether to clean the collection before adding.

• overwrite_datasets (bool) – Whether to overwrite existing datasets.

• overwrite_metadata (bool) – Whether to overwrite existing metadata.

Raises:

ValueError – If there are errors in the YAML file format.

Returns:

None

lomas_server.mongodb_admin.add_user(db: Database, user: str) → None

Add new user in users collection with initial values for all fields set by default.

Parameters:

• db (Database) – mongo database object

• user (str) – username to be added

Raises:

• ValueError – If the user already exists.

• WriteConcernError – If the result is not acknowledged.

Returns:

None

lomas_server.mongodb_admin.add_user_with_budget(db: Database, user: str, dataset: str, epsilon: float, delta: float) → None

Add new user in users collection with initial values for all fields set by default.

Parameters:

• db (Database) – mongo database object

• user (str) – username to be added

• dataset (str) – name of the dataset to add to user

• epsilon (float) – epsilon value for initial budget of user

• delta (float) – delta value for initial budget of user

Raises:

ValueError – _description_

Returns:

None

lomas_server.mongodb_admin.add_users_via_yaml(db: Database, yaml_file: str | Dict, clean: bool, overwrite: bool) → None

Add all users from yaml file to the user collection

Parameters:

• db (Database) – mongo database object

• yaml_file (Union[str, Dict]) – if str: a path to the YAML file location if Dict: a dictionnary containing the collection data

• clean (bool) – boolean flag True if drop current user collection False if keep current user collection

• overwrite (bool) – boolean flag True if overwrite already existing users False errors if new values for already existing users

Returns:

None

lomas_server.mongodb_admin.check_dataset_and_metadata_exist(enforce_true: bool) → Callable

Creates a wrapper function that raises a ValueError if the supplied user does not already exist in the user collection.

lomas_server.mongodb_admin.check_user_exists(enforce_true: bool) → Callable

Creates a wrapper function that raises a ValueError if the supplied user does (not) exist in the user collection depending on the enforce_true parameter.

Parameters:

enforce_true (bool) – If set to True, the wrapper will enforce the user is already in the database. If set to False, it will enforce the user is NOT in the database.

Returns:

The wrapper function that enforces user presence

(or absence) before calling the suplied function.

Return type:

Callable

lomas_server.mongodb_admin.check_user_has_dataset(enforce_true: bool) → Callable

Creates a wrapper function that raises a ValueError if the supplied user has access (or not) to the supplied dataset depending on the enforce_true parameter.

Parameters:

enforce_true (bool) – If set to true, the wrapper function will enforce the user has access to the dataset. If set to False, the wrapper function will enforce the user has NOT access to the specified dataset.

Returns:

The wrapper function that asserts user access (or not)

to the provided dataset.

Return type:

Callable

lomas_server.mongodb_admin.del_dataset(db: Database, dataset: str) → None

Delete dataset from dataset collection.

Parameters:

• db (Database) – mongo database object

• dataset (str) – Dataset name to be deleted

Returns:

None

lomas_server.mongodb_admin.del_dataset_to_user(db: Database, user: str, dataset: str) → None

Remove if exists the dataset (and all related budget info) from list of datasets that user has access to.

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to which to delete a dataset

• dataset (str) – name of the dataset to remove from user

Returns:

None

lomas_server.mongodb_admin.del_user(db: Database, user: str) → None

Delete all related information for user from the users collection.

Parameters:

• db (Database) – mongo database object

• user (str) – username to be deleted

Returns:

None

lomas_server.mongodb_admin.drop_collection(db: Database, collection: str) → None

Delete collection.

Parameters:

• db (Database) – mongo database object

• collection (str) – Collection name to be deleted.

Returns:

None

lomas_server.mongodb_admin.get_list_of_datasets(db: Database) → list

Get the list of all dataset is ‘datasets’ collection

Parameters:

db (Database) – mongo database object

Returns:

list of names of all datasets

Return type:

dataset_names (list)

lomas_server.mongodb_admin.get_list_of_datasets_from_user(db: Database, user: str) → list

Get the list of all datasets from the user

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to show archives

Returns:

list of names of all users

Return type:

user_datasets (list)

lomas_server.mongodb_admin.get_list_of_users(db: Database) → list

Get the list of all users is ‘users’ collection

Parameters:

db (Database) – mongo database object

Returns:

list of names of all users

Return type:

user_names (list)

lomas_server.mongodb_admin.set_budget_field(db: Database, user: str, dataset: str, field: str, value: float) → None

Set (for some reason) a budget field to a given value if given user exists and has access to given dataset.

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to set budget to

• dataset (str) – name of the dataset to set budget to

• field (str) – one of ‘epsilon’ or ‘delta’

• value (float) – value to set as epsilon or delta

Returns:

None

lomas_server.mongodb_admin.set_may_query(db: Database, user: str, value: bool) → None

Set (for some reason) the ‘may query’ field to a given value if given user exists.

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to enable/disable

• value (bool) – may query value (True or False)

Returns:

None

lomas_server.mongodb_admin.show_archives_of_user(db: Database, user: str) → List[dict]

Show all previous queries from a user

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to show archives

Returns:

list of previous queries from the user

Return type:

archives (List)

lomas_server.mongodb_admin.show_collection(db: Database, collection: str) → list

Show a collection

Parameters:

• db (Database) – mongo database object

• collection (str) – Collection name to be shown.

Returns:

None

lomas_server.mongodb_admin.show_dataset(db: Database, dataset: str) → dict

Show a dataset from dataset collection.

Parameters:

• db (Database) – mongo database object

• dataset (str) – name of the dataset to show

Returns:

informations about the dataset

Return type:

dataset_info (dict)

lomas_server.mongodb_admin.show_metadata_of_dataset(db: Database, dataset: str) → dict

Show a metadata from metadata collection.

Parameters:

• db (Database) – mongo database object

• dataset (str) – name of the dataset of the metadata to show

Returns:

informations about the metadata

Return type:

metadata (dict)

lomas_server.mongodb_admin.show_user(db: Database, user: str) → dict

Show a user

Parameters:

• db (Database) – mongo database object

• user (str) – username of the user to show

Returns:

all information of user from ‘users’ collection

Return type:

user (dict)

lomas_server.mongodb_admin_cli module

lomas_server.uvicorn_serve module

Module contents