Kumara Sastry » Publications

Enhancing the efficiency of the extended compact genetic algorithm

Kumara Sastry — Tue, 20 May 2008 03:27:37 +0000

Duque, T., Goldberg, D. E., Sastry, K. (2008). IlliGAL Report No. 2008006. University of Illinois at Urbana-Champaign, Urbana IL. [Full Paper - PDF] [Full Paper - PS].

Abstract:
Evolutionary Algorithms are largely used search and optimization procedures that, when properly designed, can solve intractable problems in tractable polynomial time. Efficiency enhancements are used to turn them from tractable to practical.

In this paper we show preliminary results of two efficiency enhancements proposed for the Extended Compact Genetic Algorithm. First, a model building enhancement was used to reduce the complexity of the process from O(n³) to O(n²), speeding up the algorithm by 1000 times on a 4096 bits problem. Then, local-search hybridization was used to reduce the population size by at least 32 times, reducing the memory and running time required by the algorithm. These results draw the first steps toward a competent and efficient Genetic Algorithm.

Linkage learning, rule representation, and the χ-ary extended compact classifier system

Kumara Sastry — Sun, 06 Apr 2008 05:39:48 +0000

Llorà, X., Sastry, K., Lima, C. F., Lobo, F. G., Goldberg, D. E. (2008). IlliGAL Report No. 2008005. University of Illinois at Urbana-Champaign, Urbana IL. [Full Paper - PDF] [Full Paper - PS].

Abstract:
This paper reviews a competent Pittsburgh LCS that automatically mines important substructures of the underlying problems and takes problems that were intractable with first-generation Pittsburgh LCS and renders them tractable. Specifically, we propose a ?-ary extended compact classifier system which uses (1) a competent genetic algorithm (GA) in the form of ?-ary extended compact genetic algorithm, and (2) a niching method in the form restricted tournament replacement, to evolve a set of maximally accurate and maximally general rules. Besides showing that linkage exist on the multiplexer problem, and that ?eCCS scales exponentially with the number of address bits (building block size) and quadratically with the problem size, this paper also explores non-traditional rule encodings. Gene expression encodings, such as the Karva language, can also be used to build ?eCCS probabilistic models. However, results show that the traditional ternary encoding { 0,1,#} presents a better scalability than the gene expression inspired ones.

Speeding online synthesis via enforced selecto-recombination

Kumara Sastry — Sun, 06 Apr 2008 05:32:39 +0000

Saruwatari, S., Llorà, X., Yasui, N. I., Tamura, H., Sastry, K., Goldberg, D. E. (2008). IlliGAL Report No. 2008004. University of Illinois at Urbana-Champaign, Urbana IL. [Full Paper - PDF] [Full Paper - PS].

Abstract:
Brainstorming has been greatly used as a method to generate a large number of ideas by variety of each participant’s knowledge. However, brainstorming does not always work well because of spatial, communication limitations. Moreover, brainstorming techniques present limited scalability. Meanwhile, genetics algorithms have been mostly regarded as an engineering or technological tool. However, the innovation intuition suggests that genetic algorithms may be also regarded as models of human innovation and creativity. This paper focuses on online creativity sessions. Modeling those creative efforts using selecto-recombinative mechanism can provide three times more novel ideas, increase the posting frequency by a 2.6 factor, and help overcome superficiality on online communications by favoring synthetic thinking.

Improving small population performance under noise with viral infection + tropism

Kumara Sastry — Sun, 06 Apr 2008 05:28:10 +0000

Sato, Y., Goldberg, D. E., Sastry, K. (2008). IlliGAL Report No. 2008002. University of Illinois at Urbana-Champaign, Urbana IL. [Full Paper - PDF] [Full Paper - PS].

Abstract:
In this paper we report on the effect of viral infection with tropism on the formation of building blocks in genetic operations. In previous research, we applied genetic algorithms to the analysis of time-series signals with noise. We demonstrated the possibility of reducing the number of required entities and improving the rate of convergence when searching for a solution by having some of the host chromosomes harbor viruses with a tropism function. Here, we simulate problems having both multimodality and deceptiveness features and problems that include noise as test functions, and show that viral infection with tropism can increase the proportion of building blocks in the population when it cannot be assumed that a necessary and sufficient number of entities are available to find a solution. We show that this capability is especially noticeable in problems that include noise.

Real-coded ECGA for solving decomposable real-valued optimization problems

Kumara Sastry — Sun, 06 Apr 2008 04:33:28 +0000

Li, M., Goldberg, D. E., Sastry, K., Yu, T.-L. (2007). Proceedings of the 2007 IEEE Congress on Evolutionary Computation (CEC 2007). 2194–2201. [Full Paper].

Abstract:
This paper presents the real-coded extended compact genetic algorithms (rECGA) for decomposable real-valued optimization problems. Mutual information among real-valued variables is employed to measure variables interaction or dependency, and the variables clustering and aggregation algorithms are proposed to identify the substructures of a problem through partitioning variables. Then, mixture Gaussian probability density function is estimated to model the promising individuals for each substructure, and the sampling of multivariate Gaussian probability density function is done by adopting Cholesky decomposition. Finally, experiments on decomposable test functions are conducted. The results show that the rECGA is able to correctly identify the substructure of decomposable problems with linear or nonlinear correlations, and achieves a good scalability.

Influence of selection and replacement strategies on linkage learning in BOA

Kumara Sastry — Sun, 06 Apr 2008 04:28:54 +0000

Lima, C. F., Pelikan, M., Goldberg, D. E., Lobo, F. G., Sastry, K., Hauschild, M. (2007). Proceedings of the 2007 IEEE Congress on Evolutionary Computation (CEC 2007). 1083–1090. [Full Paper].

Abstract:
The Bayesian optimization algorithm (BOA) uses Bayesian networks to learn linkages between the decision variables of an optimization problem. This paper studies the influence of different selection and replacement methods on the accuracy of linkage learning in BOA. Results on concatenated m-k deceptive trap functions show that the model accuracy depends on a large extent on the choice of selection method and to a lesser extent on the replacement strategy used. Specifically, it is shown that linkage learning in BOA is more accurate with truncation selection than with tournament selection. The choice of replacement strategy is important when tournament selection is used, but it is not relevant when using truncation selection. On the other hand, if performance is our main concern, tournament selection and restricted tournament replacement should be preferred. These results aim to provide practitioners with useful information about the best way to tune BOA with respect to structural model accuracy and overall performance.

A simple real-coded extended compact genetic algorithm

Kumara Sastry — Sun, 06 Apr 2008 04:24:33 +0000

Fossati, L., Lanzi, P. L., Sastry, K., Goldberg, D. E., Gomez, O. (2007). Proceedings of the 2007 IEEE Congress on Evolutionary Computation (CEC 2007). 342–348. [Full Paper].

Abstract:
This paper presents a simple real-coded estimation of distribution algorithm (EDA) design using ?-ary extended compact genetic algorithm (?ECGA) and discretization methods. Specifically, the real-valued decision variables are mapped to discrete symbols of user-specified cardinality using discretization methods. The ?ECGA is then used to build the probabilistic model and to sample a new population based on the probabilistic model. The effect of alphabet cardinality and the selection pressure on the scalability of the real-coded ECGA (rECGA) method is investigated. The results show that the population size required by rECGA—to successfully solve a class of additivelyseparable problems—scales sub-quadratically with problem size and the number of function evaluations scales sub-cubically with problem size. The proposed rECGA is simple, making it amenable for further empirical and theoretical analysis. Moreover, the probabilistic models built in the proposed realcoded ECGA are readily interpretable and can be easily visualized. The proposed algorithm and the results presented in this paper are first step towards conducting a systematic analysis of real-coded EDAs and towards developing a design theory for development of scalable and robust real-coded EDAs.

Discovering building blocks for human based genetic algorithms

Kumara Sastry — Sun, 06 Apr 2008 04:01:19 +0000

Ueda, T., Yasui, N. I., Llorà, X., Sastry, K. Goldberg, D. E. (2008). Smart Systems Engineering: Computational Intelligence in Architecting Complex Engineering Systems (ANNIE 2007). [First Runner Up, Theoretical Developments in Computational Intelligence] [Preprint: IlliGAL Report No. 2007020].

Abstract:
The push for rapid innovation and creativity in this Internet age places a premium on eective integration of both human and computer-generated knowledge. One of the key components of a distributed and scalable environment for accomplishing this integration called DISCUS is the human-based genetic algorithm (HBGA)–a GA where humans perform genetic operations. This paper takes the first step towards designing a competent HBGA, which can enable humans to innovate quickly, reliably, and accurately. Specifically, this paper proposes a methodology for discovering building blocks from text documents including reports, chat, transcripts and e-mail. The proposed method has been applied to simple test problems and to a news article set. The results show that the proposed BB-identification methodology is eective and enables humans to eectively exchange the BBs for rapid innovation.

Using previous models to bias structural learning in the hierarchical BOA

Kumara Sastry — Sun, 06 Apr 2008 03:27:19 +0000

Hauschild, M., Pelikan, M., Sastry, K., Goldberg, D. E. (2008). MEDAL Report No. 2008003. University of Missouri at St. Louis. [Full Paper - PDF].

Abstract:
Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. While the primary goal of applying EDAs is to discover the global optimum or at least its accurate approximation, besides this, any EDA provides us with a sequence of probabilistic models, which in most cases hold a great deal of information about the problem. Although using problem-specific knowledge has been shown to significantly improve performance of EDAs and other evolutionary algorithms, this readily available source of problem-specific information has been practically ignored by the EDA community. This paper takes the first step towards the use of probabilistic models obtained by EDAs to speed up the solution of similar problems in future. More specifically, we propose two approaches to biasing model building in the hierarchical Bayesian optimization algorithm (hBOA) based on knowledge automatically learned from previous hBOA runs on similar problems. We show that the proposed methods lead to substantial speedups and argue that the methods should work well in other applications that require solving a large number of problems with similar structure.

iBOA: The incremental Bayesian optimization algorithm

Kumara Sastry — Sun, 06 Apr 2008 03:16:16 +0000

Pelikan, M., Sastry, K., Goldberg, D. E. (2008). Medal Report No. 2008002. University of Missouri at St. Louis. [Full Paper - PDF].

Abstract:

This paper proposes the incremental Bayesian optimization algorithm (iBOA), which modifies standard BOA by removing the population of solutions and using incremental updates of the Bayesian network. iBOA is shown to be able to learn and exploit unrestricted Bayesian networks using incremental techniques for updating both the structure as well as the parameters of the probabilistic model. This represents an important step toward the design of competent incremental estimation of distribution algorithms that can solve difficult nearly decomposable problems scalably and reliably.