Paper ID: 59
Title: Truthful Univariate Estimators

Review #1
=====
Summary of the paper (Summarize the main claims/contributions of the paper.): The paper considers the problem of constructing efficient *truthful* estimators for the mean of a distribution. The underlying assumption is that n parties hold a number w_i drawn from a distribution D. The goal is to devise an estimator h(w_1,...,w_n) for the mean of D, such no party can push the estimator closer to its own value by reporting a different number from what it actually has. The basic observation behind this paper is that the naive estimator (the average) is not truthful. Another basic point is that at least for symmetric distributions, the median is a truthful estimator for the mean.   The paper proves several results. The main are: 1. There are estimators that are strictly better than the median 2. For bounded distributions, they characterize the class of truthful estimators that are optimal 

Clarity - Justification: The paper is well written

Significance - Justification: I found the problem under study quite natural and interesting. The results obtained are nice and close to optimal. The proofs seem non-trivial (though I'm not an expert in this field).

Detailed comments. (Explain the basis for your ratings while providing constructive feedback.): See significance section.

=====

Review #2
=====
Summary of the paper (Summarize the main claims/contributions of the paper.): The paper addresses the problem of finding the optimal truthful estimator of the mean when the data is gathered from a population of strategic agents. Each agent receives a private sample from the distribution D and their goal is to pull the mean as close as possible to their own observed value. Authors study two cases: 1) D is symmetric; 2) D is asymmetric with bounded support. They characterize the dominance (in terms of mean squared error) relationships among truthful estimators and identify the truthful estimators that are optimal.

Clarity - Justification: The paper is well-written and addresses a simple yet interesting statistical problem. Authors do provide a working example (setting the temperature in a building) and a broad motivation for their problem (machine learning when data is collected from strategic agent). However, I think it would be much more convincing if they could present a concrete setting of economic significance where their results could be applied.  

Significance - Justification: The results presented in the paper rely heavily of an earlier powerful result by Moulin which provides a full characterization of truthful estimators in terms of generalized medians. Still, I think authors ask an interesting question that can potentially be applied to other settings.

Detailed comments. (Explain the basis for your ratings while providing constructive feedback.): Overall, I enjoyed reading the paper, and I think the paper can have a broad audience in ICML.

=====

Review #3
=====
Summary of the paper (Summarize the main claims/contributions of the paper.): The paper analyzes truthful estimators of the mean of a population. 

Clarity - Justification: Well written paper with clear objective.

Significance - Justification: The main result provided by the authors is that the best truthful estimator for symmetric distributions is the median when the number of samples is odd and the median with a "phantom point" when the number of samples is even.  The authors provide a more complicated characterization of the optimal truthful estimator of the mean when the distribution is not symmetric.  Whereas the math on this results is correct, I do not see an interest in the machine learning community for this type of work. This is due to the following reasons:  1) Distributions are hardly ever symmetric 2) If distributions are not symmetric then clearly any generalized median will remain a bad estimator.  3) This paper would have been more interesting if we were considering a regression problem instead of simply estimating the mean. 

Detailed comments. (Explain the basis for your ratings while providing constructive feedback.): The math is OK  but I do not think this paper will be significant to the ICML community. 

=====