Steps for Making Legacy Data Semantic and AI-Ready
Frank Guerino
Hi,
Given this community’s interests in semantics and ontologies, and given how it’s been to find this kind of information, I thought I’d share attempts to document instructions for what it means to make legacy “non-semantic” data compliant for A.I. consumption and use by making it “semantic” (all because AI works from Natural Language)
For each step, I’ve tried to provide descriptions and examples. Coverage includes semantic IDs semantic attributes, semantic relationships, semantic ontologies, rules, etc.
The goal is to, over time, continuously improve and further detail these steps.
As we all work to transition our enterprise data to and for AI, I hope community members find this all useful, and I welcome any ideas for improvements.
Steps for Making Legacy Data Semantic for Consumption and Use by AI …
Step # | Step | What the Step Means |
1 | Description: Identify where meaning, identity, relationships, definitions, evidence, and authority are missing, ambiguous, or unreliable in legacy data — and treat these gaps as a governed backlog that must be paid down before AI can reason over the data safely. (This is more of an assessment step.)
Example 1: An enterprise inventories its legacy CRM, ERP, and case-management systems and discovers that Customer Status carries eight different meanings across the estate, none of them documented — recording this as a Knowledge Debt item to be reconciled before any AI-facing publication.
Example 2: A healthcare payer catalogs undocumented codes, orphaned foreign keys, unlabeled derived fields, and expired business rules across its claims platform, ranks them by AI-use risk, and assigns owners to remediate the highest-severity items first. | |
2 | Establish a multidisciplinary operating model for semantic conversion | Description: Assemble the cross-functional roles, responsibilities, decision rights, review cadences, and governance forums that will define, produce, validate, and sustain semantic representations — recognizing that no single team owns meaning, identity, relationships, rules, and lineage alone. (This is more of a skills establishment step.)
Example 1: An enterprise establishes a Semantic Conversion Council with named participation from Data Governance, Enterprise Architecture, Business Domain Stewards, AI Engineering, Security, and Compliance — meeting on a defined cadence with documented decision authority.
Example 2: A financial services firm defines Responsible, Accountable, Consulted, and Informed assignments for each conversion activity: business stewards own definitions, data engineers own extraction and lineage, ontology stewards own predicates and rules, and a governance forum approves publication to AI retrieval services. |
3 | Define the Semantic Layer, Ontology, rules, and meaning model | Description: Establish the governed vocabulary, Ontology, Taxonomy, rules, constraints, and metadata that tell AI what enterprise data means and how it should be interpreted — including the Noun Types, predicates, and validation rules the downstream conversion work will follow. (This is a design step.)
Example 1: An enterprise defines whether Customer, Client, and Account Holder are approved synonyms or distinct concepts, preventing AI from treating them inconsistently across systems.
Example 2: A healthcare payer defines the governed meanings and relationships among Member, Subscriber, Dependent, Plan, Benefit, Claim, Provider, and Authorization before allowing AI to reason across them. |
4 | Description: Keep source-system keys, codes, and identifiers so every semantic representation traces back to the original record, system of record, and integration context — then add stable, human-readable Semantic IDs alongside them so the same objects are addressable, understandable, and reusable across AI, systems, and humans. (This is a design step.)
Example 1: A customer record from a legacy CRM keeps its original CUSTOMER_ID = 104582 and receives a stable Semantic ID such as customer.acme-manufacturing, so analysts can reconcile the enriched record back to the source and AI can address the customer by a natural-language-friendly identifier.
Example 2: A healthcare payer preserves the original claim number, source table, batch ID, and ingestion timestamp for a claim, and adds the Semantic ID claim.2026-104582-inpatient-authorization for AI retrieval and reasoning.
Example 3: An application internally identified as APP_0931 retains that original identifier for lineage and receives the Semantic ID application.claims-intake-portal for retrieval, governance reporting, and cross-inventory analysis. | |
5 | Description: Translate opaque field names, codes, flags, and derived values into governed business terms with clear definitions, context, constraints, and controlled meanings — so AI interprets each attribute the same way an informed business reader would. (This is an implementation step.)
Example 1: A database column named CTR is mapped to the Semantic Attribute Customer, with a definition explaining whether it refers to a customer identifier, a customer count, or a customer category.
Example 2: A field named STAT_CD = A is converted into Lifecycle Status = Active, with the allowed values, source code mapping, effective date, and governing definition retained. | |
6 | Description: Use foreign keys, shared values, lineage, integrations, reports, documentation, configurations, event records, and human knowledge to identify and validate meaningful relationships before they are represented semantically. (This is an implementation step.)
Example 1: A team discovers that an application uses a database by combining connection strings, configuration files, query logs, and a database administrator's confirmation.
Example 2: A customer-to-product relationship is inferred from shared identifiers in orders, billing records, and support tickets, then validated by a business steward before publication. | |
7 | Description: Convert the discovered technical connections into readable business statements that explain how two objects relate, such as "Application supports Capability" or "Customer is managed by Person." (This is an implementation step.)
Example 1: A foreign-key relationship between APPLICATION.CAPABILITY_ID and CAPABILITY.ID becomes the readable statement, "Claims Intake Portal supports Claims Processing."
Example 2: A vendor-to-contract join becomes, "Acme Software is governed by Contract CT-2026-104," rather than remaining an unexplained pair of database keys. | |
8 | Description: Apply governed Ontology elements and repeatable rules to control naming, mapping, interpretation, relationship creation, validation, and approval across the conversion process — turning the definitions established in Step 3 into operational enforcement. (This is an implementation step.)
Example 1: A rule for defining semantic relationships states that a Foreign Key that represents a Person, in a Column that represents a Business Owner, in a row that represents an Application, all gets translated into a semantic relationship such as "Person Jane Doe is the Business Owner for Application XYZ."
Example 2: A rule states that only applications with an approved production status may be linked to live customer-facing capabilities.
Example 3: An Ontology defines that a Regulation may impose Regulatory Obligations, and that a Control may satisfy an Obligation only when supporting evidence and an effective date are present. | |
9 | Prepare Semantic Instance Documents for AI retrieval and reasoning | Description: Assemble each important data instance into a complete, readable document object that contains its identity, attributes, traits, relationships, lineage, governance, and retrieval context (i.e., Person Jane Doe gets her own Natural Language document object that fully describes her semantically). (This is an implementation step.)
Example 1: A complete application document is generated containing its Semantic ID, owner, lifecycle status, business capabilities, vendors, technologies, data stores, risks, controls, lineage, and source references.
Example 2: A customer document combines approved identity data, active products, service history, preferences, consent restrictions, and related contracts into one governed representation for AI retrieval. |
10 | Enrich, index, and publish semantic representations for AI use | Description: Add retrieval metadata, lineage, sensitivity, source identifiers, relationship context, and refresh information, then publish the semantic representations to approved search, vector, or retrieval services. (This is an implementation step.)
Example 1: Semantic application documents are enriched with sensitivity, ownership, effective dates, source links, and refresh timestamps before being indexed in an enterprise search or vector platform.
Example 2: Policy and control documents are published to an AI retrieval service only after adding jurisdiction, applicability, approval status, version, retention class, and authoritative-source metadata. |
11 | Manage refresh, drift, lineage, validation, and governance over time | Description: Continuously synchronize semantic representations with source data and business meaning, detect drift, revalidate changes, preserve lineage, govern access, and retire obsolete content. (This is more of a governance and maintenance step.)
Example 1: When an application owner, supported capability, or production status changes, the semantic representation is regenerated, revalidated, and reindexed automatically.
Example 2: A nightly drift process detects that a source code definition changed from Active to Active or Pending Closure, flags the semantic mapping for steward review, and prevents the old meaning from being treated as authoritative. |
My best,
Frank
--
Frank Guerino, Principal
The International Foundation for Information Technology (IF4IT)
http://www.if4it.com
1.908.294.5191 (M)
Guerino1_Skype (S)
Alex Shkotin
Hi Frank,
A grandiose program. I remembered one approach, discussed it with Deepseek, and we came up with the following description [1].
I'd call it knowledge reengineering.
Could it be useful in your program?
Alex
[1] https://chat.deepseek.com/share/5fpebbbwfjjd7y0t2v
Certainly. Here is a **brief summary in English** of your proposed two-step approach:
---
**Proposed Approach: From Tabular Data to Facts for LLM Processing**
- **Step 1 (Algorithmic / Deterministic):**
Instead of using an LLM to interpret raw tables, write specific, rule-based scripts or parsers for each table type (or class of tables). These algorithms extract structured data from Excel, CSV, relational databases, etc., and convert them into uniform, machine-readable **facts** — i.e., quantifier-free propositions containing only object/process identifiers and their attributes.
- **Step 2 (LLM-based):**
Feed these extracted facts (not the raw tables) into a large language model. Since facts are semantically explicit and informationally equivalent to the original tables, the LLM can focus on higher-level reasoning: answering questions, generating summaries, checking consistency, or making inferences — without struggling to parse table structures.
---
**Key advantages of this separation:**
| Aspect | Benefit |
|--------|---------|
| **Speed & Cost** | Algorithmic extraction is deterministic, fast, and cheap (no API calls or heavy GPUs needed for Step 1). |
| **Reliability** | No hallucination risk during data extraction — results are reproducible and precise. |
| **Semantic Clarity** | LLM receives clean, meaningful statements, reducing context noise and improving reasoning quality. |
| **Modularity** | Each table class gets a tailored extractor; the LLM remains task-agnostic and reusable. |
---
**Research alignment:**
This hybrid pattern is gaining traction in fact-checking, information extraction, and table-based QA. While many studies use LLMs for *both* extraction and reasoning, a growing body of work advocates for *distilling* extraction into code or heuristics, reserving LLMs solely for knowledge-intensive reasoning — exactly as you propose.
--
All contributions to this forum are covered by an open-source license.
For information about the wiki, the license, and how to subscribe or
unsubscribe to the forum, see http://ontologforum.org/info
---
You received this message because you are subscribed to the Google Groups "ontolog-forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ontolog-foru...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ontolog-forum/E8C66531-5F73-421E-8366-D0FDA496F2E4%40if4it.com.
Frank Guerino
Hi Alex,
My apologies for the delayed response. I wanted to spend time understanding it before responding.
I believe your feedback provides strong improvements. The DeepSeek synthesis captures an essential architectural pattern that clearly fits.
What you and DeepSeek describe is effectively the execution engine for carrying out several of the core implementation steps in the program, specifically bridging raw data into explicit semantic representations without introducing extraction errors or unnecessary token costs.
How I believe this feedback refines the highlighted steps…
- Deterministic vs. Probabilistic Separation (Steps 5 & 10):
- Extraction (Step 5): Using deterministic parsers, scripts, or declarative mappings (e.g., RML) to convert raw tables into uniform, quantifier-free propositions eliminates hallucination risk during ingestion and cuts GPU/API overhead.
- Reasoning (Step 10): Feeding clean, explicit semantic fact sets to the LLM allows the model to devote its entire context window to high-value tasks — such as logical reasoning, cross-domain inference, summarization, and consistency checking — rather than parsing table geometry or column heuristics.
- Target Schema & Rule Enforcement (Steps 3 & 8):
- The rule-based extractors operating in Step 5 operationalize the Ontology, Noun Types, Predicates, and Mapping Rules established upfront in Step 3. Defining these target proposition schemas early gives data engineers a clear specification to write toward.
Updated Steps Framework — NOTE: I haven’t fully vetted it yet but below is what I believe to be the updated table with [Alex: ...] annotations highlighting where I intend to integrate your feedback.
Step # | Step | What the Step Means & Examples |
Description: Identify where meaning, identity, relationships, definitions, evidence, and authority are missing, ambiguous, or unreliable in legacy data — and treat these gaps as a governed backlog that must be paid down before AI can reason over the data safely. (Assessment step) Example: An enterprise inventories its legacy CRM, ERP, and case-management systems and discovers that Customer Status carries eight different undocumented meanings across the estate—recording this as a Knowledge Debt item to be reconciled before AI publication. |
Description: Assemble the cross-functional roles, responsibilities, decision rights, review cadences, and governance forums that will define, produce, validate, and sustain semantic representations. (Organization step) Example: A financial services firm establishes RACI assignments where business stewards own definitions, data engineers own extraction/lineage, ontology stewards own predicates/rules, and a governance forum approves publication to AI retrieval services. |
Description: Establish the governed vocabulary, Ontology, Taxonomy, rules, constraints, and target proposition models that tell AI what enterprise data means [Alex: provides the target proposition schema for both rule-based extractors and downstream AI]. (Design step) Example: A healthcare payer defines the governed meanings and target fact structures for Member, Subscriber, Claim, and Authorization before allowing AI to reason across them. |
Description: Keep source-system keys/codes for lineage tracing, then add stable, human-readable Semantic IDs alongside them so objects are uniquely addressable across AI, systems, and humans. (Design step) Example: A record keeps CUSTOMER_ID = 104582 and receives a stable Semantic ID such as customer.acme-manufacturing, allowing full lineage reconciliation while letting AI address the entity cleanly. | ||
5 | Implement rule-based, deterministic extraction for attributes and traits | Description: [Alex: Write specific, rule-based scripts, parsers, or declarative mapping code (e.g., RML) for each table type to extract raw data without LLMs. Translate opaque field names and codes into uniform, quantifier-free semantic propositions.] (Implementation step — Hybrid/Deterministic) Example: [Alex: A script reads CTR = 104582 and STAT_CD = A from a table and deterministically emits the explicit propositions: [Customer: customer.acme-manufacturing] [hasIdentifier] [104582] and [hasLifecycleStatus] [Active], eliminating hallucination risk during extraction.] |
Description: Use foreign keys, shared values, lineage, integrations, configurations, and human knowledge to identify and validate meaningful relationships before they are represented semantically. (Implementation step) Example: A customer-to-product relationship is inferred from shared identifiers across orders, billing, and support tickets, then validated by a business steward before publication. |
Description: Convert technical joins and foreign keys into explicit, readable semantic facts using business-approved predicates, ensuring full semantic clarity before AI ingestion. (Implementation step) Example: [Alex: A foreign-key join between APPLICATION.CAPABILITY_ID and CAPABILITY.ID is deterministically translated into the statement: [Application: application.claims-intake-portal] [supportsCapability] [Capability: capability.claims-processing].] |
Description: Enforce ontology constraints and operational rules [Alex: during deterministic extraction to control naming, mapping, validation, and automated approval]. (Governance step) Example: A rule dictates that a Person foreign key in a Business Owner column automatically converts to the triple predicate isBusinessOwnerFor, provided the Person record is active. |
Description: Aggregate the extracted uniform propositions, identities, attributes, relationships, lineage, and governance metadata into complete, self-contained semantic instance documents or natural language representations optimized for AI context windows. (Implementation step) Example: A complete application document is compiled into a single explicit text object containing its Semantic ID, owner, lifecycle status, supported capabilities, vendor dependencies, and underlying data stores. |
Description: Add retrieval metadata, lineage, sensitivity tags, and refresh timestamps, then index the semantic documents. [Alex: Feed these explicit fact representations into LLMs for higher-level reasoning, answering, consistency checking, and inference—reserving the LLM solely for knowledge-intensive reasoning.] (AI Execution step) Example: Policy and control semantic documents are indexed for an AI reasoning agent to evaluate regulatory compliance across multi-system workflows without the LLM needing to parse raw SQL schemas. |
Description: Continuously synchronize semantic representations with source data changes, [Alex: re-run deterministic parsers when source schemas evolve], detect semantic drift, and maintain end-to-end data lineage. (Maintenance step) Example: A nightly drift process detects an unmapped status code in a legacy system, flags the gap in the Knowledge Debt backlog, and prevents unverified data from reaching the AI layer. |
Thanks again. I believe your feedback definitely improves the framework.
My best,
Frank
--
Frank Guerino, Principal
The International Foundation for Information Technology (IF4IT)
http://www.if4it.com
1.908.294.5191 (M)
To view this discussion visit https://groups.google.com/d/msgid/ontolog-forum/CAFxxROTzvUJwSTZFAp1F-in6LXWmg4%3DQNUtbLKUNuUzzjGRZZw%40mail.gmail.com.
Alex Shkotin
Frank,
Great!
Next week, I'm going to take a closer look at data logic. That is, the part of formal ontology that can be called factology.
After all, the simplest definition of ontology is theory plus facts.
Then we can see where in your steps we can write "theory" instead of "ontology." This will be closer to mathematics.
Your "execution engine" reminded me of https://arxiv.org/abs/2509.09775. Have a look.
Best wishes,
Alex
To view this discussion visit https://groups.google.com/d/msgid/ontolog-forum/38F759D1-B741-43F7-A03C-4D07999D7ABA%40if4it.com.
John F Sowa
Alex Shkotin
John,
It's always interesting to discuss the subtleties of formalization. Formalization is a powerful technology. Or, as Dana Scott said, "formalization is an experimental science". see https://link.springer.com/chapter/10.1007/978-3-642-46490-4_4
There was just an interesting exchange of views on the expressive power of ZFC in the [FOM] https://mathweb.ucsd.edu/~sbuss/FOM/. And Larry Paulson wrote:
"I used to keep pinned up the following quotation by Gödel (from an obscure book review, 1935):
Thus we are led to conclude that, although everything mathematical is formalisable, it is nevertheless impossible to formalise all of mathematics in a single formal system, a fact that intuitionism has asserted all along."
A theory must be very mature for its formalization to yield anything useful. If theoretical knowledge is chaotic, then formalization will only reveal that.
Alex
--
All contributions to this forum are covered by an open-source license.
For information about the wiki, the license, and how to subscribe or
unsubscribe to the forum, see http://ontologforum.org/info
---
You received this message because you are subscribed to the Google Groups "ontolog-forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ontolog-foru...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ontolog-forum/bc027143ea174426beb20bbf99f280a4%40923bac8a6c9540b9ade7d4f015086095.