Steven Gustafson

Conventional reinforcement learning (RL) allows an agent to learn policies via environmental rewards only, with a long and slow learning curve at the beginning stage. On the contrary, human learning is usually much faster because prior and general knowledge and multiple information resources are utilized. In this paper, we propose a \textbf{P}lanner-\textbf{A}ctor-\textbf{C}ritic architecture for hu\textbf{MAN}-centered planning and learning (\textbf{PACMAN}), where an agent uses its prior… Show more

Conventional reinforcement learning (RL) allows an agent to learn policies via environmental rewards only, with a long and slow learning curve at the beginning stage. On the contrary, human learning is usually much faster because prior and general knowledge and multiple information resources are utilized. In this paper, we propose a \textbf{P}lanner-\textbf{A}ctor-\textbf{C}ritic architecture for hu\textbf{MAN}-centered planning and learning (\textbf{PACMAN}), where an agent uses its prior, high-level, deterministic symbolic knowledge to plan for goal-directed actions, while integrates Actor-Critic algorithm of RL to fine-tune its behaviors towards both environmental rewards and human feedback. This is the first unified framework where knowledge-based planning, RL, and human teaching jointly contribute to the policy learning of an agent. Our experiments demonstrate that PACMAN leads to a significant jump start at the early stage of learning, converges rapidly and with small variance, and is robust to inconsistent, infrequent and misleading feedback. Show less

(Extended Abstract in AAMAS. Long version: https://arxiv.org/abs/1905.02168)

Machine learning is becoming an essential part of developing solutions for many industrial applications, but the lack of interpretability hinders wide industry adoption to rapidly build, test, deploy and validate machine learning models, in the sense that the insight of developing machine learning solutions are not structurally encoded, justified and transferred. In this paper we describe
Maana… Show more

(Extended Abstract in AAMAS. Long version: https://arxiv.org/abs/1905.02168)

Machine learning is becoming an essential part of developing solutions for many industrial applications, but the lack of interpretability hinders wide industry adoption to rapidly build, test, deploy and validate machine learning models, in the sense that the insight of developing machine learning solutions are not structurally encoded, justified and transferred. In this paper we describe
Maana Meta-learning Service, an interpretable and interactive automated machine learning service residing in Maana Knowledge Platform that performs machine-guided, user assisted pipeline search and hyper-parameter tuning and generates structured knowledge about decisions for pipeline profiling and selection. The service is shipped with Maana Knowledge Platform and is validated using benchmark dataset. Furthermore, its capability of deriving knowledge from pipeline search facilitates various inference tasks and transferring to similar data science projects. Show less

In this paper we investigate an alternative knowledge representation and learning strategy for the automated machine learning (AutoML) task. Our approach combines a symbolic planner with reinforcement learning to evolve programs that process data and train machine learning classifiers. The planner, which generates all feasible plans from the initial state to the goal state, gives preference first to shortest programs and then later to ones that maximize rewards. The results demonstrate the… Show more

In this paper we investigate an alternative knowledge representation and learning strategy for the automated machine learning (AutoML) task. Our approach combines a symbolic planner with reinforcement learning to evolve programs that process data and train machine learning classifiers. The planner, which generates all feasible plans from the initial state to the goal state, gives preference first to shortest programs and then later to ones that maximize rewards. The results demonstrate the efficacy of the approach for finding good machine learning pipelines, while at the same time showing that the representation can be used to infer new knowledge relevant for the problem instances being solved. These insights can be useful for other automatic programming approaches, like genetic programming (GP) and Bayesian optimization pipeline learning, with respect to representation and learning strategies. Show less

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep… Show more

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options.This framework features a planner -- controller -- meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components cross-fertilize each other and eventually converge to an optimal symbolic plan along with the learned subtasks, bringing together the advantages of long-term planning capability with symbolic knowledge and end-to-end reinforcement learning directly from a high-dimensional sensory input. Experimental results validate the interpretability of subtasks, along with improved data efficiency compared with state-of-the-art approaches. Show less

This work addresses challenges arising from extracting entities from textual data, including the high cost of data annotation, model accuracy, selecting appropriate evaluation criteria, and the overall quality of annotation. We present a framework that integrates Entity Set Expansion (ESE) and Active Learning (AL) to reduce the annotation cost of sparse data and provide an online evaluation method as feedback. This incremental and interactive learning framework allows for rapid annotation and… Show more

This work addresses challenges arising from extracting entities from textual data, including the high cost of data annotation, model accuracy, selecting appropriate evaluation criteria, and the overall quality of annotation. We present a framework that integrates Entity Set Expansion (ESE) and Active Learning (AL) to reduce the annotation cost of sparse data and provide an online evaluation method as feedback. This incremental and interactive learning framework allows for rapid annotation and subsequent extraction of sparse data while maintaining high accuracy.
We evaluate our framework on three publicly available datasets and show that it drastically reduces the cost of sparse entity annotation by an average of 85% and 45% to reach 0.9 and 1.0 F-Scores respectively. Moreover, the method exhibited robust performance across all datasets. Show less

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may not be robust to domain uncertainties and changes. In this paper we present a unified framework {\em PEORL} that integrates symbolic planning with hierarchical reinforcement… Show more

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may not be robust to domain uncertainties and changes. In this paper we present a unified framework {\em PEORL} that integrates symbolic planning with hierarchical reinforcement learning (HRL) to cope with decision-making in a dynamic environment with uncertainties.
Symbolic plans are used to guide the agent's task execution and learning, and the learned experience is fed back to symbolic knowledge to improve planning. This method leads to rapid policy search and robust symbolic plans in complex domains. The framework is tested on benchmark domains of HRL. Show less

In this chapter, we explore how Genetic Programming can assist and augment the expert-driven process of developing data-driven models. In our use case, modelers must develop hundreds of models that represent individual properties of parts, components, assets, systems and meta-systems like power plants. Each of these models is developed with an objective in mind, like estimating the useful remaining life or detecting anomalies. As such, the modeler uses their expert judgment, as well as… Show more

In this chapter, we explore how Genetic Programming can assist and augment the expert-driven process of developing data-driven models. In our use case, modelers must develop hundreds of models that represent individual properties of parts, components, assets, systems and meta-systems like power plants. Each of these models is developed with an objective in mind, like estimating the useful remaining life or detecting anomalies. As such, the modeler uses their expert judgment, as well as available data to select the most appropriate method. In this initial paper, we examine the most basic example of when the experts select a kind of regression modeling approach and develop models from data. We then use that captured domain knowledge from their processes, as well as end models to determine if Genetic Programming can augment, assist and improve their final results. We show that while Genetic Programming can indeed find improved solutions according to an error metric, it is much harder for Genetic Programming to find models that do not increase complexity. Also, we find that one approach in particular shows promise as a way to incorporate domain knowledge. Show less

With the advent of Big Data technologies, organizations can efficiently store and analyze more data than ever before. However, extracting maximal value from this data can be challenging for many reasons. For example, datasets are often not stored using human-understandable terms, making it difficult for a large set of users to benefit from them. Further, given that different types of data may be best stored using different technologies, datasets that are closely related may be stored separately… Show more

With the advent of Big Data technologies, organizations can efficiently store and analyze more data than ever before. However, extracting maximal value from this data can be challenging for many reasons. For example, datasets are often not stored using human-understandable terms, making it difficult for a large set of users to benefit from them. Further, given that different types of data may be best stored using different technologies, datasets that are closely related may be stored separately with no explicit linkage. Finally, even within individual data stores, there are often inconsistencies in data representations, whether introduced over time or due to different data producers. These challenges are further compounded by frequent additions to the data, including new raw data as well as results produced by large-scale analytics. Thus, even within a single Big Data environment, it is often the case that multiple rich datasets exist without the means to access them in a unified and cohesive way, often leading to lost value. This paper describes the development of a Big Data management infrastructure with semantic technologies at its core to provide a unified data access layer and a consistent approach to analytic execution. Semantic technologies were used to create domain models describing mutually relevant datasets and the relationships between them, with a graphical user interface to transparently query across datasets using domain-model terms. This prototype system was built for GE Power & Water's Power Generation Products Engineering Division, which has produced over 50TB of gas turbine and component prototype test data to date. The system is expected to result in significant savings in productivity and expenditure. Show less

Clinical documents are vital resources for radiologists to have a better understanding of patient history. The use of clinical documents can complement the often brief reasons for exams that are provided by physicians in order to perform more informed diagnoses. With the large number of study exams that radiologists have to perform on a daily basis, it becomes too time-consuming for radiologists to sift through each patient's clinical documents. It is therefore important to provide a capability… Show more

Clinical documents are vital resources for radiologists to have a better understanding of patient history. The use of clinical documents can complement the often brief reasons for exams that are provided by physicians in order to perform more informed diagnoses. With the large number of study exams that radiologists have to perform on a daily basis, it becomes too time-consuming for radiologists to sift through each patient's clinical documents. It is therefore important to provide a capability that can present contextually relevant clinical documents, and at the same time satisfy the diverse information needs among radiologists from different specialties. In this work, we propose a knowledge-based semantic similarity approach that uses domain-specific relationships such as part-of along with taxonomic relationships such as is-a to identify relevant radiology exam records. Our approach also incorporates explicit relevance feedback to personalize radiologists information needs. We evaluated our approach on a corpus of 6,265 radiology exam reports through study sessions with radiologists and demonstrated that the retrieval performance of our approach yields an improvement of 5% over the baseline. We further performed intra-class and inter-class similarities using a subset of 2,384 reports spanning across 10 exam codes. Our result shows that intra-class similarities are always higher than the inter-class similarities and our approach was able to obtain 6% percent improvement in intra-class similarities against the baseline. Our results suggest that the use of domain-specific relationships together with relevance feedback provides a significant value to improve the accuracy of the retrieval of radiology exam reports. Show less

The Contract Search Tool is a semantic search platform that enables effective analysis of complex, long-term contractual service agreement for machines such as gas turbines. The approach we developed can effectively identify paragraphs of text for specific legal concepts. Then the key content can be decomposed and organized by the semantics model that captures key elements of the concepts and links to specific paragraphs. This is achieved by performing semantic text analysis to capture… Show more

The Contract Search Tool is a semantic search platform that enables effective analysis of complex, long-term contractual service agreement for machines such as gas turbines. The approach we developed can effectively identify paragraphs of text for specific legal concepts. Then the key content can be decomposed and organized by the semantics model that captures key elements of the concepts and links to specific paragraphs. This is achieved by performing semantic text analysis to capture implicitly-stated provisions and the definitions of provisions, and relevant information is returned in an organized manner. The tool can be applied to increase productivity of legal review, share legal knowledge with service managers, and reduce legal risk in contract review process. Show less

This study examines how extracting relationships from data can lead to very different social networks. The chapter uses online message board data to define a relationship between two authors. After applying a threshold on the number of communications between members, the authors further constrain relationships to be supported by each member in the relationship also having a relationship to the same third member: the triangle constraint. By increasing the number of communications required to… Show more

This study examines how extracting relationships from data can lead to very different social networks. The chapter uses online message board data to define a relationship between two authors. After applying a threshold on the number of communications between members, the authors further constrain relationships to be supported by each member in the relationship also having a relationship to the same third member: the triangle constraint. By increasing the number of communications required to have a valid relationships between members, they see very different social networks being constructed. Authors find that the subtle design choices that are made when extracting relationships can lead to different networks, and that the variation itself could be useful for classifying and segmenting nodes in the network. For example, if a node is ‘central’ across different approaches to extracting relationships, one could assume with more confidence that the node is indeed ‘central’. Lastly, the chapter studies how future communication occurs between members and their ego-networks from prior data. By increasing the communication requirements to extract valid relationships, it is seen how future communication prediction is impacted and how social network design choices could be better informed by understanding these variations. Show less

Marketing is most successful when people spread the message within their social network. The Internet can serve as an approximation of the spread of messages, particularly marketing campaigns, to both measure marketing effectiveness and provide data for influencing future efforts. However, to measure the network of web sites spreading the marketing message potentially requires a massive amount of data collection over a long period of time. Additionally, collecting data from the Internet is very… Show more

Marketing is most successful when people spread the message within their social network. The Internet can serve as an approximation of the spread of messages, particularly marketing campaigns, to both measure marketing effectiveness and provide data for influencing future efforts. However, to measure the network of web sites spreading the marketing message potentially requires a massive amount of data collection over a long period of time. Additionally, collecting data from the Internet is very noisy and can create a false sense of precision. Therefore, we propose to use ego-centric network sampling to both reduce the amount of data required to collect as well as handle the inherent uncertainty of the data collected. In this the book chapter, we study whether the proposed ego-centric network sampling accurately captures the network structure.We use the Stanford-Berkeley network to show that the approach can capture the underlying structure with a minimal amount of data. Show less

The journal and in particular the resource reviews have been running for 10 years. There are a number of activities being planned to celebrate. However it is a good time to revisit our original and updated goals again [(Langdon, Genet Progrm Evolvable Mach 1(1/2):165–169 (2000); Langdon and Gustafson, Genet Program Evolvable Mach 6(2):221–228 (2005)], compare them with what the journal has achieved and make new plans. "Books" section onwards gives up to date statistics on the genetic… Show more

The journal and in particular the resource reviews have been running for 10 years. There are a number of activities being planned to celebrate. However it is a good time to revisit our original and updated goals again [(Langdon, Genet Progrm Evolvable Mach 1(1/2):165–169 (2000); Langdon and Gustafson, Genet Program Evolvable Mach 6(2):221–228 (2005)], compare them with what the journal has achieved and make new plans. "Books" section onwards gives up to date statistics on the genetic programming and evolvable hardware literature and electronic resources. Show less

It is approximately 50 years since the first computational experiments were conducted in what has become known today as the field of Genetic Programming (GP), twenty years since John Koza named and popularised the method, and ten years since the first issue appeared of the Genetic Programming & Evolvable Machines journal. In particular, during the past two decades there has been a significant range and volume of development in the theory and application of GP, and in recent years the field has… Show more

It is approximately 50 years since the first computational experiments were conducted in what has become known today as the field of Genetic Programming (GP), twenty years since John Koza named and popularised the method, and ten years since the first issue appeared of the Genetic Programming & Evolvable Machines journal. In particular, during the past two decades there has been a significant range and volume of development in the theory and application of GP, and in recent years the field has become increasingly applied. There remain a number of significant open issues despite the successful application of GP to a number of challenging real-world problem domains and progress in the development of a theory explaining the behavior and dynamics of GP. These issues must be addressed for GP to realise its full potential and to become a trusted mainstream member of the computational problem solving toolkit. In this paper we outline some of the challenges and open issues that face researchers and practitioners of GP. We hope this overview will stimulate debate, focus the direction of future research to deepen our understanding of GP, and further the development of more powerful problem solving algorithms. Show less

This paper presents a light-weight information retrieval and analysis architecture that addresses the complex task of gathering, combining, and storing documents to enable indepth analysis. The growing interest in mining the Internet for conversation topics, opinions, and influencers has resulted in many free and commercial products. At the heart of such capability are two core technologies: information retrieval and text mining. While search engines and technologies like RSS make gathering… Show more

This paper presents a light-weight information retrieval and analysis architecture that addresses the complex task of gathering, combining, and storing documents to enable indepth analysis. The growing interest in mining the Internet for conversation topics, opinions, and influencers has resulted in many free and commercial products. At the heart of such capability are two core technologies: information retrieval and text mining. While search engines and technologies like RSS make gathering information easier, they, like text mining, still require a significant amount of consideration when applying them in mission critical situations. For example, different search engines retrieve irrelevant results, and it is difficult to impossible to know that all relevant results have been found. Also, doing significant analysis of such documents will usually require the fusion of other information sources - a task that most search engines, at least, do not support. We have developed a system and architecture for light-weight document and information retrieval to enable focused and deep analysis of text, authors and publishers, and the networks that they form between each other through citations and other reference and co-occurrence analysis. While it is both intuitive and obvious that such a system is necessary for in-depth analysis, it is nontrivial as to how to construct such a system out of the many moving pieces, data sources and technologies. We show both the architecture, discuss the decisions steps, and demonstrate analysis that are enabled by the system. Show less

We are interested in the variance of social networks when using supporting evidence to define “friend” relationships, particularly in online social networks. Of related interest, relevant to this study, is the impact of various network sampling methods as well as the ability to capture the true structure of a network given incomplete or inaccurate data. Using empirical social network data, we explore the effect of requiring friendship relationships to be supported with communications between… Show more

We are interested in the variance of social networks when using supporting evidence to define “friend” relationships, particularly in online social networks. Of related interest, relevant to this study, is the impact of various network sampling methods as well as the ability to capture the true structure of a network given incomplete or inaccurate data. Using empirical social network data, we explore the effect of requiring friendship relationships to be supported with communications between members, where friendship between members must also be corroborated with friendships to a third member. Ultimately, we hope to identify substantial relationships between members that may be capable of influencing behavior. A hypothetical scenario of measuring the vitality of an online community allows us to assess the effect on pertinent network metrics. Results indicate some amount of stability in certain measurements, but enough variance is present to suggest that in empirical networks, the presence of key nodes and edges reduces the robustness previously measured in random networks. Most significantly, the study demonstrates a possible way to identify the robustness of relationships within networks, as well as identify high-level groupings of communities as stable under different relationships, by increasing the amount of 'evidence' required to create ties between members, irrespective of the strength of the ties. Show less

Marketing is most successful when people spread the message within their social network. The Internet can serve as an approximation of the spread of messages, particularly marketing campaigns, to both measure marketing effectiveness and provide data for influencing future efforts. However, to measure the network of Web sites spreading the marketing message potentially requires a massive amount of data collection over a long period of time. Additionally, collecting data from the Internet is very… Show more

Marketing is most successful when people spread the message within their social network. The Internet can serve as an approximation of the spread of messages, particularly marketing campaigns, to both measure marketing effectiveness and provide data for influencing future efforts. However, to measure the network of Web sites spreading the marketing message potentially requires a massive amount of data collection over a long period of time. Additionally, collecting data from the Internet is very noisy and can create a false sense of precision. Therefore, we propose to use ego-centric network sampling to both reduce the amount of data required to collect as well as handle the inherent uncertainty of the data collected. In this paper, we study whether the proposed ego-centric network sampling accurately captures the network structure. We use the Stanford-Berkeley network to show that the approach can capture the underlying structure with a minimal amount of data. Show less

Generalization is a very important issue in Machine Learning. In this paper, we present a new idea for improving Genetic Programming generalization ability. The idea is based on a dynamic two-layered selection algorithm and it is tested on a real-life drug discovery regression application. The algorithm begins using root mean squared error as fitness and the usual tournament selection. A list of individuals called ``repulsors'' is also kept in memory and initialized as empty. As an individual… Show more

Generalization is a very important issue in Machine Learning. In this paper, we present a new idea for improving Genetic Programming generalization ability. The idea is based on a dynamic two-layered selection algorithm and it is tested on a real-life drug discovery regression application. The algorithm begins using root mean squared error as fitness and the usual tournament selection. A list of individuals called ``repulsors'' is also kept in memory and initialized as empty. As an individual is found to overfit the training set, it is inserted into the list of repulsors. When the list of repulsors is not empty, selection becomes a two-layer algorithm: individuals participating to the tournament are not randomly chosen from the population but are themselves selected, using the average dissimilarity to the repulsors as a criterion to be maximized. Two kinds of similarity/dissimilarity measures are tested for this aim: the well known structural (or edit) distance and the recently defined subtree crossover based similarity measure. Although simple, this idea seems to improve Genetic Programming generalization ability and the presented experimental results show that Genetic Programming generalizes better when subtree crossover based similarity measure is used, at least for the test problems studied in this paper. Show less

In evolutionary algorithms, distance metrics between solutions are often useful for many aspects of guiding and understanding the search process. A good distance measure should reflect the capability of the search: if two solutions are found to be close in distance, or similarity, they should also be close in the search algorithm sense, i.e., the variation operator used to traverse the search space should easily transform one of them into the other. This paper explores such a distance for… Show more

In evolutionary algorithms, distance metrics between solutions are often useful for many aspects of guiding and understanding the search process. A good distance measure should reflect the capability of the search: if two solutions are found to be close in distance, or similarity, they should also be close in the search algorithm sense, i.e., the variation operator used to traverse the search space should easily transform one of them into the other. This paper explores such a distance for genetic programming syntax trees. Distance measures are discussed, defined and empirically investigated. The value of such measures is then validated in the context of analysis (fitness-distance correlation is analyzed during population evolution) as well as guiding search (results are improved using our measure in a fitness sharing algorithm) and diversity (new insights are obtained as compared with standard measures). Show less

To analyse various properties of the search process of genetic programming it is useful to quantify the distance between two individuals. Using operator-based distance measures can make this analysis more accurate and reliable than using distance measures which have no relationship with the genetic operators. This paper extends a recent definition of a distance measure based on subtree crossover for genetic programming. Empirical studies are presented that show the suitability of this measure… Show more

To analyse various properties of the search process of genetic programming it is useful to quantify the distance between two individuals. Using operator-based distance measures can make this analysis more accurate and reliable than using distance measures which have no relationship with the genetic operators. This paper extends a recent definition of a distance measure based on subtree crossover for genetic programming. Empirical studies are presented that show the suitability of this measure to dynamically calculate the fitness distance correlation coefficient during the evolution, to construct a fitness sharing system for genetic programming and to measure genotypic diversity in the population. These experiments confirm the accuracy of the new measure and its consistency with the subtree crossover genetic operator. Show less

Problem solving using multi-agent robotic systems has received significant attention in recent research. Complex strategies are required to organize and control these systems. Biological-inspired methodologies are often employed to bypass this complexity, e.g. self-organization. However, another line of research is to understand the relationship between low-level behaviors and complex high-level strategies. In this paper, we focus on understanding the interference caused in multi-robotic… Show more

Problem solving using multi-agent robotic systems has received significant attention in recent research. Complex strategies are required to organize and control these systems. Biological-inspired methodologies are often employed to bypass this complexity, e.g. self-organization. However, another line of research is to understand the relationship between low-level behaviors and complex high-level strategies. In this paper, we focus on understanding the interference caused in multi-robotic systems for the problem of search and tagging. Given a set of targets that must be found and tagged by a set of robots, what are the effects of scaling the number of robots and sensor ranges? Intuitively, increasing robot numbers, or sensor strength would seem beneficial. However, experience suggests that path and sensor interference caused by increased robots, increased targets, and sensor range will be harmful. The following investigation uses several abstract models to elucidate the issues of robot scaling and sensor noise. Show less

This paper presents an investigation of a novel model for parallel evolutionary algorithms (EAs) based on the biological concept of species. In EA population search, new species represent solutions that could lead to good solutions but are disadvantaged due to their dissimilarity from the rest of the population. The Speciating Island Model (SIM) attempts to exploit new species when they arise by allocating them to new search processes executing on other islands (other processors). The long term… Show more

This paper presents an investigation of a novel model for parallel evolutionary algorithms (EAs) based on the biological concept of species. In EA population search, new species represent solutions that could lead to good solutions but are disadvantaged due to their dissimilarity from the rest of the population. The Speciating Island Model (SIM) attempts to exploit new species when they arise by allocating them to new search processes executing on other islands (other processors). The long term goal of the SIM is to allow new species to diffuse throughout a large (conceptual) parallel computer network, where idle and unimproving processors initiate a new search process with them. In this paper, we focus on the successful identification and exploitation of new species and show that the SIM can achieve improved solution quality as compared to a canonical parallel EA. Show less

This paper reports an improvement to genetic programming (GP) search for the symbolic regression domain, based on an analysis of dissimilarity and mating. GP search is generally difficult to characterise for this domain, preventing well motivated algorithmic improvements. We first examine the ability of various solutions to contribute to the search process. Further analysis highlights the numerous solutions produced during search with no change to solution quality. A simple algorithmic… Show more

This paper reports an improvement to genetic programming (GP) search for the symbolic regression domain, based on an analysis of dissimilarity and mating. GP search is generally difficult to characterise for this domain, preventing well motivated algorithmic improvements. We first examine the ability of various solutions to contribute to the search process. Further analysis highlights the numerous solutions produced during search with no change to solution quality. A simple algorithmic enhancement is made that reduces these events and produces a statistically significant improvement in solution quality. We conclude by verifying the generalisability of these results on several other regression instances. Show less

This paper reports on the development of a novel island model for evolutionary algorithms, which is intrinsically parallel and intended to better utilise resources and outlier solutions encountered during search. Outliers serve as seeds for new islands using a similar evolutionary algorithm or a local search procedure. In this initial study, we examine a definition of outliers and demonstrate the ability to obtain improvements using outliers and a simple local search method.

This paper explores distance measures based on genetic operators for genetic programming using tree structures. The consistency between genetic operators and distance measures is a crucial point for analytical measures of problem difficulty, such as fitness distance correlation, and for measures of population diversity, such as entropy or variance. The contribution of this paper is the exploration of possible definitions and approximations of operator-based edit distance measures. In… Show more

This paper explores distance measures based on genetic operators for genetic programming using tree structures. The consistency between genetic operators and distance measures is a crucial point for analytical measures of problem difficulty, such as fitness distance correlation, and for measures of population diversity, such as entropy or variance. The contribution of this paper is the exploration of possible definitions and approximations of operator-based edit distance measures. In particular, we focus on the subtree crossover operator. An empirical study is presented to illustrate the features of an operator-based distance. This paper makes progress toward improved algorithmic analysis by using appropriate measures of distance and similarity. Show less

This paper introduces the Tree-String problem for genetic programming and related search and optimisation methods. To improve the understanding of optimisation and search methods, we aim to capture the complex dynamic created by the interdependencies of solution structure and content. Thus, we created an artificial domain that is amenable for analysis, yet representative of a wide-range of real-world applications. The Tree-String problem provides several benefits, including: the direct control… Show more

This paper introduces the Tree-String problem for genetic programming and related search and optimisation methods. To improve the understanding of optimisation and search methods, we aim to capture the complex dynamic created by the interdependencies of solution structure and content. Thus, we created an artificial domain that is amenable for analysis, yet representative of a wide-range of real-world applications. The Tree-String problem provides several benefits, including: the direct control of both structure and content objectives, the production of a rich and representative search space, the ability to create tunably difficult and random instances and the flexibility for specialisation. Show less

The journal and in particular the resource reviews have been running for ten years.
There are a number of activities being planned to celebrate. However it is a good time to revisit
our original and updated goals again [1, 2], compare them with what the journal has achieved and
make new plans. Section 2 onwards gives up to date statistics on the genetic programming and
evolvable hardware literature and electronic resources.

This paper examines the sampling of unique structures and behaviours in genetic programming. A novel description of behaviour is used to better understand the solutions visited during genetic programming search. Results provide new insight about deception that can be used to improve the algorithm and demonstrate the capability of genetic programming to sample different large tree structures during the evolutionary process.

Population measures for genetic programs are defined and analysed in an attempt to better understand the behaviour of genetic programming. Some measures are simple, but do not provide sufficient insight. The more meaningful ones are complex and take extra computation time. Here we present a unified view on the computation of population measures through an information hypertree (iTree). The iTree allows for a unified and efficient calculation of population measures via a basic tree traversal.

A vast number of very successful applications of Global-Local Search Hybrids have been reported in the literature in the last years for a wide range of problem domains. The majority of these papers report the combination of highly specialized pre-existing local searchers and usually purpose-specific global operators (e.g. genetic operators in an Evolutionary Algorithm).In this paper we concentrate on one particular class of Global-Local Search Hybrids, Memetic Algorithms (MAs), and we describe… Show more

A vast number of very successful applications of Global-Local Search Hybrids have been reported in the literature in the last years for a wide range of problem domains. The majority of these papers report the combination of highly specialized pre-existing local searchers and usually purpose-specific global operators (e.g. genetic operators in an Evolutionary Algorithm).In this paper we concentrate on one particular class of Global-Local Search Hybrids, Memetic Algorithms (MAs), and we describe the implementation of ``self-generating'' mechanisms to produce the local searches the MA uses. This implementation is tested in two problems, NK-Landscape Problems and the Maximum Contact Map Overlap Problem (MAX-CMO). Show less

This paper examines measures of diversity in genetic programming. The goal is to understand the importance of such measures and their relationship with fitness. Diversity methods and measures from the literature are surveyed and a selected set of measures are applied to common standard problem instances in an experimental study. Results show the varying definitions and behaviours of diversity and the varying correlation between diversity and fitness during different stages of the evolutionary… Show more

This paper examines measures of diversity in genetic programming. The goal is to understand the importance of such measures and their relationship with fitness. Diversity methods and measures from the literature are surveyed and a selected set of measures are applied to common standard problem instances in an experimental study. Results show the varying definitions and behaviours of diversity and the varying correlation between diversity and fitness during different stages of the evolutionary process. Populations in the genetic programming algorithm are shown to become structurally similar while maintaining a high amount of behavioural differences. Conclusions describe what measures are likely to be important for understanding and improving the search process and why diversity might have different meaning for different problem domains. Show less

This paper investigates the relationship between code growth and problem difficulty in genetic programming. The symbolic regression problem domain is used to investigate this relationship using two different types of increased instance difficulty. Results are supported by a simplified model of genetic programming and show that increased difficulty induces higher selection pressure and less genetic diversity, which both contribute toward an increased rate of code growth.

In this chapter we concentrate on one particular class of Global-Local Search Hybrids, Memetic Algorithms (MAs), and we describe the implementation of “self-assembling” mechanisms to produce the local searches the MA uses. To understand the context in which self-assembling is applied we discuss some important aspects of Memetic theory and how these concepts could be harnessed to implement more competitive MAs. Our implementation is tested in two problems, Maximum Contact Map Overlap Problem… Show more

In this chapter we concentrate on one particular class of Global-Local Search Hybrids, Memetic Algorithms (MAs), and we describe the implementation of “self-assembling” mechanisms to produce the local searches the MA uses. To understand the context in which self-assembling is applied we discuss some important aspects of Memetic theory and how these concepts could be harnessed to implement more competitive MAs. Our implementation is tested in two problems, Maximum Contact Map Overlap Problem (MAX-CMO) and the NK-Landscape Problems.

Three lessons can be drawn from this paper:

Memetic theory provides a rich set of metaphors and insights that can be harnessed within optimisation algorithms as to provide better search methods.

The optimization of solutions can be done simultaneously with the self-assembling of local search strategies which can then be exploited by the Memetic Algorithm (or other metaheuristic)

Local search strategies that are evolved to supply building blocks can greatly improve the quality of the search obtained by the Memetic Algorithm and do not seem to suffer from premature convergence (an ubiquitous problem for global-local hybrids). Show less

Tree initialisation techniques for genetic programming (GP) are examined in [4],[3], highlighting a bias in the standard implementation of the initialisation method Ramped Half-n-Half (RHH) [1]. GP trees typically evolve to random shapes, even when populations were initially full or minimal trees [2]. In canonical GP, unbalanced and sparse trees increase the probability that bigger subtrees are selected for recombination, ensuring code growth occurs faster and that subtree crossover will have… Show more

Tree initialisation techniques for genetic programming (GP) are examined in [4],[3], highlighting a bias in the standard implementation of the initialisation method Ramped Half-n-Half (RHH) [1]. GP trees typically evolve to random shapes, even when populations were initially full or minimal trees [2]. In canonical GP, unbalanced and sparse trees increase the probability that bigger subtrees are selected for recombination, ensuring code growth occurs faster and that subtree crossover will have more difficultly in producing trees within specified depth limits. The ability to evolve tree shapes which allow more legal crossover operations, by providing more possible crossover points (by being bushier), and control code growth is critical. The GP community often uses RHH [4]. The standard implementation of the RHH method selects either the grow or full method with 0.5 probability to produce a tree. If the tree is already in the initial population it is discarded and another is created by grow or full. As duplicates are typically not allowed, this standard implementation of RHH favours full over grow and possibly biases the evolutionary process. Show less

This paper presents an analysis of increased
diversity in genetic programming. A selection strategy
based on genetic lineages is used to increase genetic di-
versity. A genetic lineage is defined as the path from an
individual to individuals which were created from its ge-
netic material. The method is applied to three problem
domains: Artificial Ant, Even-5-Parity and symbolic re-
gression of the Binomial-3 function. We examine how in-
creased diversity affects… Show more

This paper presents an analysis of increased
diversity in genetic programming. A selection strategy
based on genetic lineages is used to increase genetic di-
versity. A genetic lineage is defined as the path from an
individual to individuals which were created from its ge-
netic material. The method is applied to three problem
domains: Artificial Ant, Even-5-Parity and symbolic re-
gression of the Binomial-3 function. We examine how in-
creased diversity affects problems differently and draw
conclusions about the types of diversity which are more
important for each problem. Results indicate that di-
versity in the Ant problem helps to overcome deception,
while elitism in combination with diversity is likely to
benefit the Parity and regression problems. Show less

n this paper we implement a Self-Generating Memetic Algorithm for the Maximum Contact Map Overlap Problem (MAX-CMO). We demonstrate how the optimization of solutions can be done simultaneously with the discovering of useful local search strategies. In turn, the evolved local searchers act as suppliers of building blocks for the evolutionary algorithm.

This paper presents a survey and comparison of significant diversity measures in the genetic programming literature. This study builds on previous work by the authors to gain a deeper understanding of the conditions under which genetic programming evolution is successful. Three benchmark problems (Artificial Ant, Symbolic Regression and Even-5-Parity) are used to illustrate different diversity measures and to analyse their correlation with performance. Results show that measures of population… Show more

This paper presents a survey and comparison of significant diversity measures in the genetic programming literature. This study builds on previous work by the authors to gain a deeper understanding of the conditions under which genetic programming evolution is successful. Three benchmark problems (Artificial Ant, Symbolic Regression and Even-5-Parity) are used to illustrate different diversity measures and to analyse their correlation with performance. Results show that measures of population diversity based on edit distances and phenotypic diversity suggest that successful evolution occurs when populations converge to a similar structure but with high fitness diversity. Show less

This report represents an initial investigation into the use
of genetic programming to solve the N-prisoners puzzle. The puzzle has
generated a certain level of interest among the mathematical community.
We believe that this puzzle presents a significant challenge to the field
of evolutionary computation and to genetic programming in particular.
The overall aim is to generate a solution that encodes complex decision
making. Our initial results demonstrate that genetic… Show more

This report represents an initial investigation into the use
of genetic programming to solve the N-prisoners puzzle. The puzzle has
generated a certain level of interest among the mathematical community.
We believe that this puzzle presents a significant challenge to the field
of evolutionary computation and to genetic programming in particular.
The overall aim is to generate a solution that encodes complex decision
making. Our initial results demonstrate that genetic programming can
evolve good solutions. We compare these results to engineered solutions
and discuss some of the implications. One of the consequences of this
study is that it has highlighted a number of research issues and direc-
tions and challenges for the evolutionary computation community. We
conclude the article by presenting some of these directions which range
over several areas of evolutionary computation, including multi-objective
fitness, coevolution and cooperation, and problem representations. Show less

This paper presents a survey and comparison of the significant diversity measures in the genetic programming literature. The overall aim and motivation behind this study is to attempt to gain a deeper understanding of genetic programming dynamics and the conditions under which genetic programming works well. Three benchmark problems (Artificial Ant, Symbolic Regression and Even5-parity) are used to illustrate different diversity measures and to analyse their correlation with performance. The… Show more

This paper presents a survey and comparison of the significant diversity measures in the genetic programming literature. The overall aim and motivation behind this study is to attempt to gain a deeper understanding of genetic programming dynamics and the conditions under which genetic programming works well. Three benchmark problems (Artificial Ant, Symbolic Regression and Even5-parity) are used to illustrate different diversity measures and to analyse their correlation with performance. The results show that diversity is not an absolute indicator of performance and that phenotypic measures appear superior to genotypic ones. Finally we conclude that interesting potential exists with tracking ancestral lineages. Show less

We present an adaptation of the standard genetic program (GP) to hierarchically decomposable, multi-agent learning problems. To break down a problem that requires cooperation of multiple agents, we use the team objective function to derive a simpler, intermediate objective function for pairs of cooperating agents. We apply GP to optimize first for the intermediate, then for the team objective function, using the final population from the earlier GP as the initial seed population for the next… Show more

We present an adaptation of the standard genetic program (GP) to hierarchically decomposable, multi-agent learning problems. To break down a problem that requires cooperation of multiple agents, we use the team objective function to derive a simpler, intermediate objective function for pairs of cooperating agents. We apply GP to optimize first for the intermediate, then for the team objective function, using the final population from the earlier GP as the initial seed population for the next. This layered learning approach facilitates the discovery of primitive behaviors that can be reused and adapted towards complex objectives based on a shared team goal. Show less

This presentation is concerned with discussing the applicability of genetic pro-
gramming to automatic timetabling. Given that evolutionary algorithms and
genetic algorithms have been successfully investigated for solving timetabling
problems for many years[1][2][3][4][5], we are interested as to whether genetic
programming[6], an evolutionary algorithm similar in some respects to genetic
algorithms, would be suitable and benecial for timetabling. The authors do
not know the… Show more

This presentation is concerned with discussing the applicability of genetic pro-
gramming to automatic timetabling. Given that evolutionary algorithms and
genetic algorithms have been successfully investigated for solving timetabling
problems for many years[1][2][3][4][5], we are interested as to whether genetic
programming[6], an evolutionary algorithm similar in some respects to genetic
algorithms, would be suitable and benecial for timetabling. The authors do
not know the answer to this question but intend to explore and discuss it in
the presentation. The eld of genetic programming mainly diers from genetic
algorithms in the representation (programs vs. bit strings) and the size of in-
dividuals/genomes (variable vs. xed). The direct representation of timetables
is straightforward as a bit string and genetic algorithms are a well established
timetabling approach. Two questions are then raised when considering genetic
programming for timetabling: 1) Why would we want a program to represent a
timetable instead of a bit string? 2) Why would it be benecial to use a vari-
able length representation instead of xed length? We consider three possible
approaches for exploring genetic programming methods for a timetabling prob-
lem. Experiment design is underway to determine the feasibility of the following
proposals. Show less

A vast number of very successful applications of Memetic algorithms (MAs) have been reported in the literature in the last years for a wide range of problem domains. The majority of the papers dealing with MAs are the result of the combination of highly specialised preexisting local searchers and usually purpose-speci c genetic operators. Moreover, those algorithms require a considerable eort devoted to the tuning of the local search and evolutionary parts of the algorithm. We have demonstrated… Show more

A vast number of very successful applications of Memetic algorithms (MAs) have been reported in the literature in the last years for a wide range of problem domains. The majority of the papers dealing with MAs are the result of the combination of highly specialised preexisting local searchers and usually purpose-speci c genetic operators. Moreover, those algorithms require a considerable eort devoted to the tuning of the local search and evolutionary parts of the algorithm. We have demonstrated in our previous work (see references below), that given a range of possible local search strategies available to a Memetic Algorithm, the optimal choice of which one must be used is not only problem and instance dependent but also tightly related to the state of the search process itself. We also showed that it is indeed possible to produce Memetic Algorithms that adapt on-the-y to those situations for a variety of problem domains. In this paper we continue our studies of the design of robust Memetic Algorithms by introducing the concept of \self-generating" Memetic Algorithms. As mentioned above the success of a Memetic Algorithm depends on the pre-existence of powerful local searchers. Here we allow the Memetic Algorithm to create its local searchers and to co-evolve the behaviours it needs to successfully solve a problem. Show less

We present an adaptation of the standard genetic program (GP) t o hierarchically decomposable, multi-agent learning problems. To break down a problem that requires cooperation of multiple agents, we use the team objective function to derive a simpler, intermediate objective function for pairs of cooperating agents. W e apply GP to optimize first for the intermediate, then for the team objective function, using the final population from the earlier GP as the initial seed population for the… Show more

We present an adaptation of the standard genetic program (GP) t o hierarchically decomposable, multi-agent learning problems. To break down a problem that requires cooperation of multiple agents, we use the team objective function to derive a simpler, intermediate objective function for pairs of cooperating agents. W e apply GP to optimize first for the intermediate, then for the team objective function, using the final population from the earlier GP as the initial seed population for the next. This layered learning approach facilitates the discovery of primitive behaviors that can be reused and adapted towards complex objectives based on a shared team goal. We use this method to evolve agents to play a subproblem of robotic soccer (keep -away soccer). Finally, we show how layered learning GP evolves better agents than standard GP, including GP with automatically defined functions, and how the problem decomposition results in a significant learning-speed increase.
Show less

We present an alternative to standard genetic programming (GP) that applies layered learning techniques to decompose a problem. GP is applied to subproblems sequentially, where the population in the last generation of a subproblem is used as the initial population of the next subproblem. This method is applied to evolve agents to play keep-away soccer, a subproblem of robotic soccer that requires cooperation among multiple agents in a dynnamic environment. The layered learning paradigm allows… Show more

We present an alternative to standard genetic programming (GP) that applies layered learning techniques to decompose a problem. GP is applied to subproblems sequentially, where the population in the last generation of a subproblem is used as the initial population of the next subproblem. This method is applied to evolve agents to play keep-away soccer, a subproblem of robotic soccer that requires cooperation among multiple agents in a dynnamic environment. The layered learning paradigm allows GP to evolve better solutions faster than standard GP. Results show that the layered learning GP outperforms standard GP by evolving a lower fitness faster and an overall better fitness. Results indicate a wide area of future research with layered learning in GP. Show less

The goal of this research is to apply genetic implementations of algorithms for selection, partitioning,andsynthesis of attributes in largescale data mining problems. Domain knowledge about these operators has been shown to reduce the number of fitness evaluations for candidate attributes. We report results on genetic optimization of attribute selection problems and current work on attribute partitioning, synthesis specifications, and the encoding of domain knowledge about operators in a… Show more

The goal of this research is to apply genetic implementations of algorithms for selection, partitioning,andsynthesis of attributes in largescale data mining problems. Domain knowledge about these operators has been shown to reduce the number of fitness evaluations for candidate attributes. We report results on genetic optimization of attribute selection problems and current work on attribute partitioning, synthesis specifications, and the encoding of domain knowledge about operators in a fitness function. The purpose of this approach is to reduce overfitting in inductive learning and produce more general genetic versions of existing searchbased algorithms (or wrappers) for KDD performance tuning [KS98, HG00]. Several GA implementations of alternative attribute synthesis algorithms are applied to concept learning problems in military and commercial KDD applications. One of these, Jenesis,isdeployed on several network-of-workstation clusters. It is shown to achieve strongly improved test set accuracy, compared to unwrapped decision tree learning and search-based wrappers [KS98]. Show less

A KDD-based architecture should serve as a good framework to learn an improved strategy of ball control for intelligent soccer playing agents. Current work on using genetic algorithms to improve large scale data mining has been successful and provides an architecture for implementing future systems. This architecture is well suited for genetic programming and it is proposed that it can be extended to such applications. By learning "real-world" strategies, the performance of robot agents can be… Show more

A KDD-based architecture should serve as a good framework to learn an improved strategy of ball control for intelligent soccer playing agents. Current work on using genetic algorithms to improve large scale data mining has been successful and provides an architecture for implementing future systems. This architecture is well suited for genetic programming and it is proposed that it can be extended to such applications. By learning "real-world" strategies, the performance of robot agents can be improved and become more similar to that of their biological counterparts. Layered learning is used to learn high-level behaviors that encompass previously learned low-level behaviors. Three phases of implementation are presented, with the goal of reducing the difference between soccer agent learning and human soccer learning.

Genetic Programming for Strategy Learning in Soccer Playing Agents: A KDD-Based Architecture - ResearchGate. Available from: http://www.researchgate.net/publication/228681511_Genetic_Programming_for_Strategy_Learning_in_Soccer_Playing_Agents_A_KDD-Based_Architecture [accessed Apr 23, 2015]. Show less