Validation of Intelligence in Large Rule-Based Systems with Common Sense. Model-Based Validation

• Main Author: William Jarrold
• Other Authors: The Pennsylvania State University CiteSeerX Archives
• Format: Text
• Language: English
• Published: 2001
• Subjects: North Atlantic
• Online Access: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.1077.5793; http://www.aaai.org/Papers/Symposia/Spring/2001/SS-01-04/SS01-04-009.pdf

Abstract The following describes work in progress on a testing system for a large common sense knowledge base. How this testing system is relevant to the requirements of validation of intelligence is described. Four kinds of validation tests are explained: challenge tests, regression tests, knowledge base integrity tests, and ruminationbased tests. The ontology of these tests and its accompanying software package is intended for use with Cyc, a large rule based system with common sense and natural language capabilities. Part I: Commonsense, Large Rule-Based Systems and Validation This work starts with the postulate that for at least some application domains, an approach to validation that is less rigorous than traditional formal software verification techniques is appropriate. Cyc, I shall argue, is in such a domain. Cyc is a large rule-based system being constructed at Cycorp (www.cyc.com). At current count Cyc contains over 1.2 million assertions which relate more than 100,000 concepts to each other. Each such assertion is expressed in the representation language of Cyc, called CycL, a variant of first order predicate calculus with enhancements (see http://www.cyc.com/cycl.html for details). Intelligence Validation for Commonsense Systems In order to verify or validate a software system, one needs as complete and precise a specification of system requirements as possible. The more stringent the verification desired, the more precise the requirement specification needs to be. Such completeness and precision is problematic if one's application domain is commonsense reasoning. As the following use cases indicate, commonsense reasoning requirements involve inherent vagueness or ambiguity. Yet, precisely what concept is referred to by the English word "France" in (a)? For example, does "France" denote (1) a body of land bordering on Spain, the North Atlantic and the Mediterranean, (2) an amalgam of regional lifestyles, culinary practices, artistic traditions etc or (3) the people who are citizens of France? In ...