Evaluating Semantic Artifact Continuity for Noisy Social Text: RDF Materialization, SHACL Conformance, and Model Input Evidence

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Noisy social text can be transformed into semantic representations, but evaluating the resulting artifacts is difficult when source records, ontology activations, graph resources, conformance reports, and model inputs are considered separately. The evaluation problem is whether their connections remain inspectable as these artifacts are generated and used. This study examines the above continuity using 1614 Reddit posts about artificial intelligence and technology. Ontology activations produce sibling Resource Description Framework (RDF) and numeric representations linked through stable identifiers. Explicit correspondence checks and controlled fault tests examine these connections, while Shapes Constraint Language (SHACL) checks assess conformance to declared graph constraints. Existing verification checks detect 15 of 22 injected continuity faults. The remaining seven are detected only by supplementary diagnostics. Separate human assessment of 120 validation texts evaluates concept presence and local polarity against the source text, distinguishing extraction quality from artifact continuity. Downstream classification provides secondary evidence of modest mean macro-F1 gains in agreement with inherited lexical labels, varying by model across three seeds on one fixed split. The contribution is an evaluation of governed semantic artifact continuity at the post and variable levels, making the connections among these artifacts and their failure conditions clear and inspectable.

This publication has 31 references indexed in Scilit: