SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available. We introduce SCOPE (Schema Construction and Ontology-induction Pipeline Evaluation), a train-text-only benchmark for corpus-to-schema induction and optional schema fusion from raw text, built from 24 public information extraction sources (15 RE and 9 EE) normalized into evaluation-only gold schema graphs; its core event-extraction target covers event types and within-event argument roles, with inter-event links reported separately. We present SCION (Schema Construction and Induction with Ontology Normalization), an auditable reference pipeline rather than a new extraction architecture; it constructs candidate spaces from train text and restricts naming, merging, filtering, validation, and conservative fusion to candidate-linked evidence under strict JSON contracts. On the SCOPE core suite, SCION-lite attains the highest F1 among released source-schema references, Text2Onto-style, LLM-only, and matched extract-then-aggregate baselines under Literal, Fuzzy, Continuous, and Graph schema-graph metrics, while the compact open-model SCION-RL variant reduces reliance on proprietary LLM schema engineers. These results are reported against normalized typed-edge targets rather than as claims that induced schemas surpass human ontology design; the release includes evidence-linked outputs, parse/fallback logs, candidate retention/merging logs, run manifests, code, and benchmark packages at https://github.com/wandugu/paper_scion.

Many AI systems that read documents need a clear map of what kinds of things, events, and relationships they should look for. Today, this map is often written by experts in advance, which can be slow, costly, and hard to keep consistent across different fields. This paper introduces SCOPE, a collection of 24 public text datasets that lets researchers test whether a system can build such a map from raw training text alone. It also introduces SCION, a reference system that builds the map by collecting evidence from text, proposing only items supported by that evidence, checking the result automatically, and recording why each item was kept or merged. In experiments, SCION produced maps that matched the reference answers better than several alternatives, including direct use of general-purpose AI models and older text-mining methods. A smaller open-model version of the system also reduces dependence on commercial AI services. The goal is not to replace human domain experts, but to make the early stages of designing and maintaining document-reading systems cheaper, more reproducible, and easier to audit.