Towards Better Quality Specification Miners - LO, David (Institutional Knowledge at Singapore Management University, 2006)
Softwares are often built without speciﬁcation. Tools to automatically extract speciﬁcation from software are needed and many techniques have been proposed. One type of these speciﬁcations – temporal API speciﬁcation – is often speciﬁed in the form of automaton (i.e., FSA/PFSA). There have been many work on mining software temporal speciﬁcation using dynamic analysis techniques; i.e., analysis of software program traces. Unfortunately, the issues of scalability, robustness and accuracy of these techniques have not been comprehensively addressed. In this paper, we describe a framework that enables assessments of the performance of a speciﬁcation miner in generating temporal speciﬁcation of software through traces recorded from its API interaction. Our framework requires the temporal speciﬁcation produced by the miner to be expressed as probabilistic ﬁnite state automaton (PFSA). The framework accepts a user-deﬁned simulator PFSA and a speciﬁcation miner. It produces quality assurance measures on the speciﬁcation generated by the miner. We investigate metrics used in these measures by adapting techniques found in artiﬁcial intelligence, program analysis, bioinformatics and data mining to the software speciﬁcation domain. Extensive experiments on two speciﬁcation miners have been performed to evaluate the effectiveness of the proposed quality assurance measures.