Discovering all most specific sentences
Authors:
Open Access
- 1 June 2003
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Database Systems
- Vol. 28 (2) , 140-174
Abstract
Data mining can be viewed, in many instances, as the task of computing a representation of a theory of a model or a database, in particular by finding a set of maximally specific sentences satisfying some property. We prove some hardness results that rule out simple approaches to solving the problem.The a priori algorithm is an algorithm that has been successfully applied to many instances of the problem. We analyze this algorithm, and prove that is optimal when the maximally specific sentences are "small". We also point out its limitations.We then present a new algorithm, the Dualize and Advance algorithm, and prove worst-case complexity bounds that are favorable in the general case. Our results use the concept of hypergraph transversals. Our analysis shows that the a priori algorithm can solve the problem of enumerating the transversals of a hypergraph, improving on previously known results in a special case. On the other hand, using results for the general case of the hypergraph transversal enumeration problem, we can show that the Dualize and Advance algorithm has worst-case running time that is sub-exponential to the output size (i.e., the number of maximally specific sentences).We further show that the problem of finding maximally specific sentences is closely related to the problem of exact learning with membership queries studied in computational learning theory.Keywords
This publication has 25 references indexed in Scilit:
- Depth first generation of long patterns2000
- Mining frequent patterns without candidate generation2000
- On the Complexity of Dualization of Monotone Disjunctive Normal FormsJournal of Algorithms, 1996
- Oracles and Queries That Are Sufficient for Exact LearningJournal of Computer and System Sciences, 1996
- Identifying the Minimal Transversals of a Hypergraph and Related ProblemsSIAM Journal on Computing, 1995
- Complexity of Identification and Dualization of Positive Boolean FunctionsInformation and Computation, 1995
- Algorithms for inferring functional dependencies from relationsData & Knowledge Engineering, 1994
- Design by example: An application of Armstrong relationsJournal of Computer and System Sciences, 1986
- Generalization as searchArtificial Intelligence, 1982
- The Complexity of Enumeration and Reliability ProblemsSIAM Journal on Computing, 1979