Simulations

Tools for benchmarking aggregation rules

class embedded_voting.experiments.aggregation.RandomWinner[source]

Returns a random winner. Mimics a Rule. .. rubric:: Examples

>>> np.random.seed(42)
>>> generator = make_generator()
>>> ratings = generator(7)
>>> rule = RandomWinner()
>>> rule(ratings).winner_
4
>>> rule(ratings).winner_
3
class embedded_voting.experiments.aggregation.SingleEstimator(i)[source]

Returns the best estimation of one given agent. Mimics a Rule. :param i: Index of the selected agents. :type i: int

Examples

>>> np.random.seed(42)
>>> generator = make_generator()
>>> ratings = generator(7)
>>> rule = SingleEstimator(10)
>>> ratings[10, :]
Ratings([ 1.2709017 ,  0.03209107,  1.98196138,  1.12347711, -1.55465272,
         -0.72448238,  0.63366952])
>>> rule(ratings).winner_
2
embedded_voting.experiments.aggregation.evaluate(list_agg, truth, testing, training, pool=None)[source]

Run a sim. :param list_agg: Rules to test. :type list_agg: list :param truth: Ground truth of testing values (n_tries X n_candidates). :type truth: ndarray :param testing: Estimated scores (n_agents X n_tries X n_candidates). :type testing: ndarray :param training: Training scores (n_agents X training_size). :type training: ndarray :param pool: Use parallelism. :type pool: Pool, optional.

Returns:Efficiency of each algorithm.
Return type:ndarray

Examples

>>> np.random.seed(42)
>>> n_training = 10
>>> n_tries = 100
>>> n_c = 20
>>> generator = make_generator()
>>> training = generator(n_training)
>>> testing = generator(n_tries*n_c).reshape(generator.n_voters, n_tries, n_c)
>>> truth = generator.ground_truth_.reshape(n_tries, n_c)
>>> list_agg = make_aggs(order=default_order+['Rand'])
>>> with Pool() as p:
...     res = evaluate(list_agg=list_agg[:-1], truth=truth, testing=testing, training=training, pool=p)
>>> ', '.join( f"{a.name}: {r:.2f}" for a, r in zip(list_agg, res) )
'MA: 0.94, PL+: 0.89, EV+: 0.95, EV: 0.94, AV: 0.90, PV: 0.86, RV: 0.85, Single: 0.82, PL: 0.78'
>>> res = evaluate(list_agg=list_agg, truth=truth, testing=testing, training=training)
>>> ', '.join( f"{a.name}: {r:.2f}" for a, r in zip(list_agg, res) )
'MA: 0.94, PL+: 0.89, EV+: 0.95, EV: 0.94, AV: 0.90, PV: 0.86, RV: 0.85, Single: 0.82, PL: 0.78, Rand: 0.49'
embedded_voting.experiments.aggregation.f_max(ratings_v, history_mean, history_std)[source]
Parameters:
  • ratings_v (ndarray) – Score vector.
  • history_mean (float) – Observed mean.
  • history_std (float) – Observed standard deviation
Returns:

The positive part of the normalized scores.
Return type:

ndarray

Examples

>>> f_max(10, 5, 2)
2.5
>>> f_max(10, 20, 10)
0.0
embedded_voting.experiments.aggregation.f_renorm(ratings_v, history_mean, history_std)[source]
Parameters:
  • ratings_v (ndarray) – Score vector.
  • history_mean (float) – Observed mean.
  • history_std (float) – Observed standard deviation
Returns:

The scores with mean and std normalized.
Return type:

ndarray

Examples

>>> f_renorm(10, 5, 2)
2.5
>>> f_renorm(10, 20, 10)
-1.0
embedded_voting.experiments.aggregation.make_aggs(groups=None, order=None, features=None, group_noise=1, distinct_noise=0.1)[source]

Crafts a list of aggregator rules. :param groups: Sizes of each group (for the Model-Aware rule). :type groups: list of int :param order: Short names of the aggregators to return. :type order: list, optional :param features: Features correlations (for the Model-Aware rule). Default to independent groups. :type features: ndarray, optional :param group_noise: Feature noise intensity. :type group_noise: float, default=1.0 :param distinct_noise: Distinct noise intensity. :type distinct_noise: float, default=0.1

Returns:Aggregators.
Return type:list

Examples

>>> list_agg = make_aggs()
>>> [agg.name for agg in list_agg]
['MA', 'PL+', 'EV+', 'EV', 'AV', 'PV', 'RV', 'Single', 'PL']
embedded_voting.experiments.aggregation.make_generator(groups=None, truth=None, features=None, feat_noise=1, feat_f=None, dist_noise=0.1, dist_f=None)[source]
Parameters:
  • groups (list of int) – Sizes of each group.
  • truth (TruthGenerator, default=N(0, 1)) – Ground truth generator.
  • features (ndarray) – Features correlations.
  • feat_noise (float, default=1.0) – Feature noise intensity.
  • feat_f (method, default to normal law) – Feature noise distribution.
  • dist_noise (float, default=0.1) – Distinct noise intensity.
  • dist_f (method, default to normal law) – Distinct noise distribution.
Returns:

Provides grounds truth and estimates.
Return type:

Generator

Examples

>>> np.random.seed(42)
>>> generator = make_generator()
>>> ratings = generator(2)
>>> truth = generator.ground_truth_
>>> truth[0]
0.4967141530112327
>>> ratings[:, 0]
Ratings([1.22114616, 1.09745525, 1.1986587 , 1.09806092, 1.09782972,
         1.16859892, 0.95307467, 0.97191091, 1.08817394, 1.04311958,
         1.17582742, 1.05360028, 1.00317232, 1.29096757, 1.12182506,
         1.15115551, 1.00192787, 1.08996442, 1.15549495, 1.02930333,
         2.05731381, 0.20249691, 0.23340782, 2.01575631])
>>> truth[1]
-0.13826430117118466
>>> ratings[:, 1]
Ratings([ 1.73490024,  1.51804687,  1.58119528,  1.73370001,  1.78786054,
          1.73115071,  1.70244906,  1.68390351,  1.56616168,  1.64202946,
          1.66795001,  1.81972611,  1.74837571,  1.53770987,  1.74642228,
          1.67550566,  1.64632168,  1.77518151,  1.81711384,  1.8071419 ,
         -0.23568328, -1.22689647,  0.71740695, -1.26155344])