Lomas Client Side: Using Smartnoise-Synth

This notebook showcases how researcher could use the Secure Data Disclosure system. It explains the different functionnalities provided by the lomas-client client library to interact with the secure server.

The secure data are never visible by researchers. They can only access to differentially private responses via queries to the server.

Each user has access to one or multiple projects and for each dataset has a limited budget with \(\epsilon\) and \(\delta\) values.

Step 1: Install the library

To interact with the secure server on which the data is stored, Dr.Antartica first needs to install the library lomas-client on her local developping environment.

It can be installed via the pip command:

[ ]:

# !pip install lomas_client

Or using a local version of the client

[2]:

import sys
import os
sys.path.append(os.path.abspath(os.path.join('..')))

[3]:

from lomas_client import Client
import numpy as np

Step 2: Initialise the client

Once the library is installed, a Client object must be created. It is responsible for sending sending requests to the server and processing responses in the local environment. It enables a seamless interaction with the server.

To create the client, Dr. Antartica needs to give it a few parameters: - a url: the root application endpoint to the remote secure server. - user_name: her name as registered in the database (Dr. Alice Antartica) - dataset_name: the name of the dataset that she wants to query (PENGUIN)

She will only be able to query on the real dataset if the queen Icergina has previously made her an account in the database, given her access to the PENGUIN dataset and has given her some epsilon and delta credit (as is done in the Admin Notebook for Users and Datasets management).

[4]:

APP_URL = "http://lomas_server"
USER_NAME = "Dr. Antartica"
DATASET_NAME = "PENGUIN"
client = Client(url=APP_URL, user_name = USER_NAME, dataset_name = DATASET_NAME)

And that’s it for the preparation. She is now ready to use the various functionnalities offered by lomas-client.

Step 3: Metadata and dummy dataset

Getting dataset metadata

Dr. Antartica has never seen the data and as a first step to understand what is available to her, she would like to check the metadata of the dataset. Therefore, she just needs to call the get_dataset_metadata() function of the client. As this is public information, this does not cost any budget.

This function returns metadata information in a format based on SmartnoiseSQL dictionary format, where among other, there is information about all the available columns, their type, bound values (see Smartnoise page for more details). Any metadata is required for Smartnoise-SQL is also required here and additional information such that the different categories in a string type column column can be added.

[5]:

penguin_metadata = client.get_dataset_metadata()
penguin_metadata

[5]:

{'max_ids': 1,
 'row_privacy': True,
 'censor_dims': False,
 'columns': {'species': {'type': 'string',
   'cardinality': 3,
   'categories': ['Adelie', 'Chinstrap', 'Gentoo']},
  'island': {'type': 'string',
   'cardinality': 3,
   'categories': ['Torgersen', 'Biscoe', 'Dream']},
  'bill_length_mm': {'type': 'float', 'lower': 30.0, 'upper': 65.0},
  'bill_depth_mm': {'type': 'float', 'lower': 13.0, 'upper': 23.0},
  'flipper_length_mm': {'type': 'float', 'lower': 150.0, 'upper': 250.0},
  'body_mass_g': {'type': 'float', 'lower': 2000.0, 'upper': 7000.0},
  'sex': {'type': 'string',
   'cardinality': 2,
   'categories': ['MALE', 'FEMALE']}},
 'rows': 344}

Step 3: Create a Synthetic Dataset keeping all default parameters

We want to get a synthetic model to represent the private data.

Therefore, we use a Smartnoise Synth Synthesizers.

Let’s list the potential options. There respective paramaters are then available in Smarntoise Synth documentation here.

[8]:

from snsynth import Synthesizer
Synthesizer.list_synthesizers()

[8]:

['mwem', 'dpctgan', 'patectgan', 'mst', 'pacsynth', 'dpgan', 'pategan', 'aim']

AIM: Adaptive Iterative Mechanism

We start by executing a query on the dummy dataset without specifying any special parameters for AIM (all optional kept as default). Also only works on categorical columns so we select “species” and “island” columns to create a synthetic dataset of these two columns.

[9]:

res_dummy = client.smartnoise_synth_query(
    synth_name="aim",
    epsilon=1.0,
    delta=0.0001,
    select_cols = ["species", "island"],
    dummy=True,
)
res_dummy['query_response'].head()

[9]:

	species	island
0	Adelie	Dream
1	Gentoo	Torgersen
2	Adelie	Dream
3	Adelie	Dream
4	Gentoo	Torgersen

The algorithm works and returned a synthetic dataset. We now estimate the cost of running this command:

[10]:

res_cost = client.estimate_smartnoise_synth_cost(
    synth_name="aim",
    epsilon=1.0,
    delta=0.0001,
    select_cols = ["species", "island"],
)
res_cost

[10]:

{'epsilon_cost': 1.0, 'delta_cost': 0.0001}

Executing such a query on the private dataset would cost 1.0 epsilon and 0.0001 delta. Dr. Antartica decides to do it with now the flag dummmy to False and specifiying that the wants the aim synthesizer model in return (with return_model = True).

NOTE: if she does not set the parameter return_model = True, then it is False by default and she will get a synthetic dataframe as response directly.

[11]:

res = client.smartnoise_synth_query(
    synth_name="aim",
    epsilon=1.0,
    delta=0.0001,
    select_cols = ["species", "island"],
    dummy=True,
    return_model = True
)
res['query_response']

/usr/local/lib/python3.11/site-packages/mbi/__init__.py:15: UserWarning: MixtureInference disabled, please install jax and jaxlib
  warnings.warn('MixtureInference disabled, please install jax and jaxlib')

[11]:

<snsynth.aim.aim.AIMSynthesizer at 0x75edc6e1ffd0>

She can now get the model and sample results with it. She choose to sample 10 samples.

[12]:

synth = res['query_response']
synth.sample(10)

[12]:

	species	island
0	Gentoo	Biscoe
1	Gentoo	Dream
2	Gentoo	Dream
3	Adelie	Torgersen
4	Chinstrap	Dream
5	Adelie	Dream
6	Adelie	Torgersen
7	Chinstrap	Biscoe
8	Chinstrap	Torgersen
9	Chinstrap	Torgersen

She now wants to specify some specific parameters to the AIM model. Therefore, she needs to set some parameters in synth_params based on the Smartnoise-Synth documentation here. She decides that she wants to modify the max_model_size to 50 (the default was 80) and tries on the dummy.

[13]:

res_dummy = client.smartnoise_synth_query(
    synth_name="aim",
    epsilon=1.0,
    delta=0.0001,
    select_cols = ["species", "island"],
    dummy=True,
    return_model = True,
    synth_params = {"max_model_size": 50}
)
res_dummy['query_response']

[13]:

<snsynth.aim.aim.AIMSynthesizer at 0x75edc6883650>

[14]:

synth = res_dummy['query_response']
synth.sample(5)

[14]:

	species	island
0	Gentoo	Dream
1	Adelie	None
2	Adelie	Biscoe
3	Gentoo	Torgersen
4	Chinstrap	Torgersen

Now that the workflow is understood for AIM, she wants to experiment with various synthesizer on the dummy.

MWEM: Multiplicative Weights Exponential Mechanism

She tries MWEM on all columns with all default parameters. As return_model is not specified she will directly receive a synthetic dataframe back.

[15]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mwem",
    epsilon=1.0,
    dummy=True,
)
res_dummy['query_response'].head()

[15]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Gentoo	Torgersen	42.25	22.5	155.0	2250.0	FEMALE
1	Chinstrap	Biscoe	63.25	13.5	235.0	5250.0	FEMALE
2	Gentoo	Torgersen	42.25	22.5	155.0	2750.0	FEMALE
3	Chinstrap	Torgersen	63.25	13.5	235.0	5250.0	FEMALE
4	Chinstrap	Dream	63.25	13.5	235.0	5250.0	FEMALE

She now specifies 3 columns and some parameters explained here.

[16]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mwem",
    epsilon=1.0,
    select_cols = ["species", "island", "sex"],
    synth_params = {"measure_only": False, "max_retries_exp_mechanism": 5},
    dummy=True,
)
res_dummy['query_response'].head()

[16]:

	species	island	sex
0	Chinstrap	Torgersen	FEMALE
1	Gentoo	Torgersen	FEMALE
2	Gentoo	Torgersen	FEMALE
3	Gentoo	Torgersen	FEMALE
4	Chinstrap	Biscoe	FEMALE

Finally it MWEM, she wants to go more in depth and create her own data preparation pipeline. Therefore, she can use Smartnoise-Synth “Data Transformers” explained here and send her own constraints dictionnary for specific steps. This is more for advanced user.

By default, if no constraints are specified, the server creates its automatically a data transformer based on selected columns, synthesizer and metadata.

Here she wants to add a clamping transformation on the continuous columns before training the synthesizer. She add the bounds based on metadata.

[18]:

bl_bounds = penguin_metadata["columns"]["bill_length_mm"]
bd_bounds = penguin_metadata["columns"]["bill_depth_mm"]
bl_bounds, bd_bounds

[18]:

({'type': 'float', 'lower': 30.0, 'upper': 65.0},
 {'type': 'float', 'lower': 13.0, 'upper': 23.0})

[19]:

from snsynth.transform import BinTransformer, ClampTransformer, ChainTransformer, LabelTransformer

my_own_constraints = {
    "bill_length_mm": ChainTransformer(
        [
            ClampTransformer(lower = bl_bounds["lower"] + 10, upper = bl_bounds["upper"] - 10),
            BinTransformer(bins = 20, lower = bl_bounds["lower"] + 10, upper = bl_bounds["upper"] - 10),
        ]
    ),
    "bill_depth_mm": ChainTransformer(
        [
            ClampTransformer(lower = bd_bounds["lower"] + 2, upper = bd_bounds["upper"] - 2),
            BinTransformer(bins=20, lower = bd_bounds["lower"] + 2, upper = bd_bounds["upper"] - 2),
        ]
    ),
    "species": LabelTransformer(nullable=True)
}

[20]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mwem",
    epsilon=1.0,
    select_cols = ["bill_length_mm", "bill_depth_mm", "species"],
    constraints = my_own_constraints,
    dummy=True,
)
res_dummy['query_response'].head()

[20]:

	bill_length_mm	bill_depth_mm	species
0	48.625	16.05	Chinstrap
1	51.625	18.75	Gentoo
2	51.625	18.75	Gentoo
3	51.625	18.75	Gentoo
4	44.875	15.15	Adelie

Also a subset of constraints can be specified for certain columns and the server will automatically generate those for the missing columns.

[21]:

my_own_constraints = {
    "bill_length_mm": ChainTransformer(
        [
            ClampTransformer(lower = bl_bounds["lower"] + 10, upper = bl_bounds["upper"] - 10),
            BinTransformer(bins = 20, lower = bl_bounds["lower"] + 10, upper = bl_bounds["upper"] - 10),
        ]
    )
}

In this case, only the bill_length will be clamped.

[22]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mwem",
    epsilon=1.0,
    select_cols = ["bill_length_mm", "bill_depth_mm", "species"],
    constraints = my_own_constraints,
    dummy=True,
)
res_dummy['query_response'].head()

[22]:

	bill_length_mm	bill_depth_mm	species
0	41.125	20.5	Chinstrap
1	53.875	20.5	Adelie
2	49.375	21.5	Chinstrap
3	53.875	20.5	Adelie
4	41.125	22.5	Chinstrap

MST: Maximum Spanning Tree

She now experiments with MST. As the synthesizer is very needy in terms of computation, she selects a subset of column for it. See MST here.

[23]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mst",
    epsilon=1.0,
    select_cols = ["species", "sex"],
    dummy=True,
)
res_dummy['query_response'].head()

[23]:

	species	sex
0		FEMALE
1	Adelie	FEMALE
2	Gentoo	MALE
3	Gentoo	MALE
4	Adelie

She can also specify a specific number of samples to get (if return_model is not True):

[24]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mst",
    epsilon=1.0,
    select_cols = ["species", "sex"],
    nb_samples = 4,
    dummy=True,
)
res_dummy['query_response']

[24]:

	species	sex
0	Adelie	FEMALE
1	Chinstrap
2	Adelie	MALE
3	Adelie	FEMALE

And a condition on these samples. For instance, here, she only wants female samples.

[25]:

res_dummy = client.smartnoise_synth_query(
    synth_name="mst",
    epsilon=1.0,
    select_cols = ["sex", "species"],
    nb_samples = 4,
    condition = "sex = FEMALE",
    dummy=True,
)
res_dummy['query_response']

[25]:

	sex	species
0		Gentoo
1		Gentoo
2		Chinstrap
3		Gentoo

DPCTGAN: Differentially Private Conditional Tabular GAN

She now tries DPCTGAN. A first warning let her know that the random noise generation for this model is not cryptographically secure and if it is not ok for her, she can decode to stop using this synthesizer. Then she does not get a response but an error 422 with an explanation.

[26]:

res_dummy = client.smartnoise_synth_query(
    synth_name="dpctgan",
    epsilon=1.0,
    dummy=True,
)
res_dummy

/code/lomas_client/utils.py:37: UserWarning: Warning:dpctgan synthesizer random generator for noise and shuffling is not cryptographically secure. (pseudo-rng in vanilla PyTorch).
  warnings.warn(

Server error status 422: {"ExternalLibraryException":"Error fitting model: sample_rate=5.0 is not a valid value. Please provide a float between 0 and 1. Try decreasing batch_size in synth_params (default batch_size=500).","library":"smartnoise_synth"}

The default parameters of DPCTGAN do not work for PENGUIN dataset. Hence, as advised in the error message, she decreases the batch_size (also she chekcks the documentation here.

[27]:

res_dummy = client.smartnoise_synth_query(
    synth_name="dpctgan",
    epsilon=1.0,
    synth_params = {"batch_size": 50},
    dummy=True,
)
res_dummy['query_response'].head()

[27]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Gentoo	Torgersen	47.809876	17.548668	188.346262	5152.039871	MALE
1	Gentoo	Torgersen	45.505892	15.785036	215.527822	4079.444863	FEMALE
2	Chinstrap	Biscoe	39.293022	19.115118	176.019657	3719.095856	MALE
3	Adelie	Dream	47.029771	17.322245	202.441730	4394.811036	MALE
4	Gentoo	Dream	44.393938	17.215382	200.788893	4653.589007	FEMALE

[ ]:

PATEGAN: Private Aggregation of Teacher Ensembles

Unfortunatelly, she is not able to train the pategan synthetizer on the PENGUIN dataset. Hence, she must try another one.

[28]:

res_dummy = client.smartnoise_synth_query(
    synth_name="pategan",
    epsilon=1.0,
    dummy=True,
)
res_dummy

Server error status 422: {"ExternalLibraryException":"pategan not reliable with this dataset.","library":"smartnoise_synth"}

PATECTGAN: Conditional tabular GAN using Private Aggregation of Teacher Ensembles

[29]:

res_dummy = client.smartnoise_synth_query(
    synth_name="patectgan",
    epsilon=1.0,
    dummy=True,
)
res_dummy['query_response'].head()

[29]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Adelie	Torgersen	44.929629	17.136712	206.450379	3677.639037	MALE
1	Chinstrap	Biscoe	41.911177	15.958753	219.224480	3503.649727	MALE
2	Chinstrap	Biscoe	39.371106	18.538643	215.860750	6183.452576	MALE
3	Gentoo	Dream	39.778699	19.392393	220.403337	6291.852415	MALE
4	Gentoo	Torgersen	41.940040	18.712894	209.865149	3379.273981	FEMALE

[30]:

res_dummy = client.smartnoise_synth_query(
    synth_name="patectgan",
    epsilon=1.0,
    select_cols = ["island", "bill_length_mm", "body_mass_g"],
    synth_params = {
        "embedding_dim": 256,
        "generator_dim": (128, 128),
        "discriminator_dim": (256, 256),
        "generator_lr": 0.0003,
        "generator_decay": 1e-05,
        "discriminator_lr": 0.0003,
        "discriminator_decay": 1e-05,
        "batch_size": 500
    },
    nb_samples = 100,
    dummy=True,
)
res_dummy['query_response'].head()

[30]:

	island	bill_length_mm	body_mass_g
0	Torgersen	48.482846	4627.040502
1	Biscoe	36.627252	4458.971380
2	Biscoe	50.437214	4419.834249
3	Torgersen	57.515079	4114.015296
4	Dream	37.914762	3729.208380

DPGAN: DIfferentially Private GAN

For DPGAN, there is the same warning as for DPCTGAN with the cryptographically secure random noise generation.

[31]:

res_dummy = client.smartnoise_synth_query(
    synth_name="dpgan",
    epsilon=1.0,
    dummy=True,
)
res_dummy['query_response'].head()

/code/lomas_client/utils.py:37: UserWarning: Warning:dpgan synthesizer random generator for noise and shuffling is not cryptographically secure. (pseudo-rng in vanilla PyTorch).
  warnings.warn(

[31]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Chinstrap	Torgersen	47.418710	23.000000	192.233215	4478.892826	MALE
1	Adelie	Biscoe	45.057986	17.836150	195.967093	5085.548207	MALE
2	Adelie	Dream	46.495445	17.600734	195.489978	3655.695215	MALE
3	Gentoo	Torgersen	58.334219	19.706083	250.000000	7000.000000	FEMALE
4	Adelie	Biscoe	65.000000	16.195120	206.065695	4478.947320	FEMALE

One final time she samples with conditions:

[32]:

res_dummy = client.smartnoise_synth_query(
    synth_name="dpgan",
    epsilon=1.0,
    condition = "body_mass_g > 5000",
    dummy=True,
)
res_dummy['query_response'].head()

/code/lomas_client/utils.py:37: UserWarning: Warning:dpgan synthesizer random generator for noise and shuffling is not cryptographically secure. (pseudo-rng in vanilla PyTorch).
  warnings.warn(

[32]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Adelie	Torgersen	43.485997	19.496153	180.303064	6134.536535	MALE
1	Gentoo	Torgersen	63.116608	21.453384	189.907485	7000.000000	MALE
2	Adelie	Dream	65.000000	20.504224	189.021906	5361.825123	MALE
3	Adelie	Dream	65.000000	20.310123	193.809981	5012.155779	FEMALE
4	Adelie	Torgersen	64.386925	16.037792	188.511754	6332.941920	MALE

And now on the real dataset

[33]:

res_dummy = client.smartnoise_synth_query(
    synth_name="dpgan",
    epsilon=1.0,
    condition = "body_mass_g > 5000",
    nb_samples = 10,
    dummy=False,
)
res_dummy['query_response']

/code/lomas_client/utils.py:37: UserWarning: Warning:dpgan synthesizer random generator for noise and shuffling is not cryptographically secure. (pseudo-rng in vanilla PyTorch).
  warnings.warn(

[33]:

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex
0	Adelie	Torgersen	43.332719	16.996525	197.196980	6542.874247	FEMALE
1	Gentoo	Biscoe	53.666000	19.943369	184.627506	5256.182522	FEMALE
2	Adelie	Torgersen	47.092630	21.298139	193.409859	5405.979574	FEMALE
3	Chinstrap	Biscoe	54.154047	15.847898	192.380197	6358.633310	MALE
4	Adelie	Biscoe	44.494324	16.248922	188.783114	5486.318067	FEMALE
5	Gentoo	Biscoe	45.828992	16.119691	209.792230	6048.840851	FEMALE
6	Chinstrap	Biscoe	45.586326	17.164572	196.064838	6345.135689	FEMALE
7	Gentoo	Biscoe	52.858690	18.386937	190.213206	5387.799114	MALE
8	Chinstrap	Biscoe	65.000000	17.667682	203.667950	7000.000000	FEMALE
9	Chinstrap	Biscoe	65.000000	19.445511	195.644815	5514.548540	FEMALE

Step 6: See archives of queries

She now wants to verify all the queries that she did on the real data. It is possible because an archive of all queries is kept in a secure database. With a function call she can see her queries, budget and associated responses.

[34]:

previous_queries = client.get_previous_queries()

Let’s check the last query

[36]:

last_query = previous_queries[-1]
last_query

[36]:

{'user_name': 'Dr. Antartica',
 'dataset_name': 'PENGUIN',
 'dp_librairy': 'smartnoise_synth',
 'client_input': {'dataset_name': 'PENGUIN',
  'synth_name': 'dpgan',
  'epsilon': 1.0,
  'delta': None,
  'select_cols': [],
  'synth_params': {},
  'nullable': True,
  'constraints': '',
  'return_model': False,
  'condition': 'body_mass_g > 5000',
  'nb_samples': 10},
 'response': {'requested_by': 'Dr. Antartica',
  'query_response':      species     island  bill_length_mm  bill_depth_mm  flipper_length_mm  \
  0     Adelie  Torgersen       43.332719      16.996525         197.196980
  1     Gentoo     Biscoe       53.666000      19.943369         184.627506
  2     Adelie  Torgersen       47.092630      21.298139         193.409859
  3  Chinstrap     Biscoe       54.154047      15.847898         192.380197
  4     Adelie     Biscoe       44.494324      16.248922         188.783114
  5     Gentoo     Biscoe       45.828992      16.119691         209.792230
  6  Chinstrap     Biscoe       45.586326      17.164572         196.064838
  7     Gentoo     Biscoe       52.858690      18.386937         190.213206
  8  Chinstrap     Biscoe       65.000000      17.667682         203.667950
  9  Chinstrap     Biscoe       65.000000      19.445511         195.644815

     body_mass_g     sex
  0  6542.874247  FEMALE
  1  5256.182522  FEMALE
  2  5405.979574  FEMALE
  3  6358.633310    MALE
  4  5486.318067  FEMALE
  5  6048.840851  FEMALE
  6  6345.135689  FEMALE
  7  5387.799114    MALE
  8  7000.000000  FEMALE
  9  5514.548540  FEMALE  ,
  'spent_epsilon': 1.0,
  'spent_delta': 0.00015673368198174188},
 'timestamp': 1724943669.4523578}

[ ]: