The paper studies a feedback form framework for preference learning, called ranked choices. In this setting, participants rank their top k (k≥1) choices from an individualized display set. As such, this setting generalizes many commonly studied feedback structures, such as discrete choices. The authors introduce a distance-based (Mallows-type) ranking model using a new distance function termed reverse major index (RMJ), which can be used to learn participant preferences from their ranked choices. Despite the requirement to sum over all permutations, the ranking model yields simple expressions of ranked choice probabilities, enabling effective inference of model parameters from data with theoretically proven consistency. Through comprehensive numerical studies on several data sets, the authors showcase the model’s efficiency for parameter estimation and favorable generalization power, especially under limited information.
This paper also provides a Prescriptive application of the study: Suppose a company aims to learn customer preferences over a range of product alternatives and identify the most preferred one with high probability. For every k, which represents the length of the ranked list in the customer feedback, the company can sequentially decide on the display set and request customers’ top k ranked choices. The authors study the interplay between feedback sophistication (represented by k) and its information efficiency. Under a sequential experimental design framework, they characterize the (asymptotic) sample complexity under the optimal display set offering procedure. They find that, although the information efficiency always increases with k, a small value of k=2 is already close (and sometimes equal) to that of full-ranking feedback.
