Evaluation consistency in iGAs: User contradictions as cycles in partial-ordering graphs
11 January 2006Llorà, X., Alías, F. , Formiga, L., Sastry, K., Goldberg, D. E. (2005). IlliGAL Report No. 2005022. University of Illinois at Urbana Champaign. [Full paper - PDF] [Full paper - PS].
Abstract:
Active interactive genetic algorithms (aiGAs) rely on actively optimizing synthetic fitness functions. In interactive genetic algorithms (iGAs) framework, user evaluations provide the necessary input for synthesizing a reasonably accurate surrogate fitness function that models user evaluations or, in other words, his/her decision preferences. User evaluations collected via tournament selection only provide partial-ordering relations between solutions. Active iGAs assemble a partial-ordering graph of user evaluations. In such a directed graph, any contradictory evaluation provided by the user introduces a cycle in the graph. This property is explored in this paper to measure the consistency of the evaluations provided by the user along the evolutionary process. The consistency measures are applied to a real-world problem, the weight tuning of the cost function involved in corpus-based text-to-speech synthesis. Results show the usefulness of such measures to identify inconsistent users during the evolutionary tuning process, and successfully the number of evaluations required by more than half.
Related Posts:
- Combating user fatigue in iGAs: Partial ordering, support vector machines, and synthetic fitness
- Efficient interactive weight tuning for TTS synthesis: Reducing user fatigue by improving user consistency
- Analyzing active interactive genetic algorithms using visual analytics
- Bayesian optimization algorithm, decision graphs, and Occam’s razor
- Evaluation relaxation using substructural information and linear estimation
No comments yet
