Onboard AutomationML (AML) data to CDF¶
Prerequisite:
- Installed Neat, see Installation
- Launched a notebook environment.
- Familiar with the
NeatSession
object, see introduction - Access to
NeatEngine
.
In this tutorial, we will:
- extract a knowledge graph from a
aml
topology file and transform it for further usage in CDF - infer a data model based on the transformed knowledge graph
- and finally push both the data model and instances from knowledge graph to CDF
Read more about aml here: https://www.automationml.org/
Extract knowledge graph from AML topology file¶
We will start by instantiating a NeatSession
and read the data from a local folder.
Make sure you have an AML file in the folder. In case of this tutorial we are using a AML file provided by DISC initiative.
from cognite.neat import NeatSession, get_cognite_client
# Note that we use Oxigraph in this example, this will not work in a CDF notebook
# to make it work in a CDF notebook replace set storage="memory"
neat = NeatSession(get_cognite_client(".env"), storage="oxigraph")
Found .env file in repository root. Loaded variables from .env file. Neat Engine 2.0.3 loaded.
Process of extracting graph from an aml file can take a bit of time, so be patient.
neat.read.xml.aml("data.aml")
Prunes nodes of specific rdf types: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1631/1631 [00:00<00:00, 40111.82it/s] Pruning the graph of triples where object is a node that is not found in graph.: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1631/1631 [00:00<00:00, 4397.65it/s]
If we now check content of the graph
object, we will see that it contains a knowledge graph extracted from the aml file:
neat
Instances
Overview:
- 1 named graphs
- Total of 3 unique types
- 973 instances in default graph
default graph:
Type | Occurrence | |
---|---|---|
0 | ExternalInterface | 858 |
1 | InternalElement | 67 |
2 | InternalLink | 48 |
Provenance:
- Initialize graph store as OxigraphStore
- Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
- Prunes the graph of triples where the object is a node that is not found in the graph.
- Attaches a target property from a target node that is connected to a source node.
- Prunes nodes of specific rdf types
- Pruning the graph of triples where object is a node that is not found in graph.
neat.show.instances();
instances.html
Infer Data Model¶
We can infer a data model from data in the NeatSession
by calling .infer()
.
neat.infer()
No issues found
neat
Data Model
type | Logical Data Model |
---|---|
intended for | Information Architect |
name | Inferred Model |
external_id | NeatInferredDataModel |
version | v1 |
classes | 3 |
properties | 45 |
Instances
Overview:
- 1 named graphs
- Total of 3 unique types
- 973 instances in default graph
default graph:
Type | Occurrence | |
---|---|---|
0 | ExternalInterface | 858 |
1 | InternalElement | 67 |
2 | InternalLink | 48 |
Provenance:
- Initialize graph store as OxigraphStore
- Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
- Prunes the graph of triples where the object is a node that is not found in the graph.
- Attaches a target property from a target node that is connected to a source node.
- Prunes nodes of specific rdf types
- Pruning the graph of triples where object is a node that is not found in graph.
We now convert verified data model to be DMS compliant
neat.convert()
Rules converted to dms.
Success: NEAT(verified,logical,neat_space,NeatInferredDataModel,v1) → NEAT(verified,physical,neat_space,NeatInferredDataModel,v1)
Inspect Data Model¶
First we will set data model id to ("aml_playground", "AML", "topology_v3.0"), where tuple represents (space, external_id, version) of data model
We use the .show
to visualize the data model and see how data model objects are connected, the resulting interactive data model visualization should be similar to this one:
neat.show.data_model()
http_purl.org_cognite_neat_data-model_verified_physical_neat_space_NeatInferredDataModel_v1.html
neat.set.data_model_id(("aml_playground_2", "AML", "topology_v3.0"))
Success: NEAT(verified,physical,neat_space,NeatInferredDataModel,v1) → NEAT(verified,physical,aml_playground_2,AML,topology_v3.0)
neat
Data Model
aspect | physical |
---|---|
intended for | DMS Architect |
name | Inferred Model |
space | aml_playground_2 |
external_id | AML |
version | topology_v3.0 |
views | 3 |
containers | 3 |
properties | 45 |
Instances
Overview:
- 1 named graphs
- Total of 3 unique types
- 973 instances in default graph
default graph:
Type | Occurrence | |
---|---|---|
0 | ExternalInterface | 858 |
1 | InternalElement | 67 |
2 | InternalLink | 48 |
Provenance:
- Initialize graph store as OxigraphStore
- Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
- Prunes the graph of triples where the object is a node that is not found in the graph.
- Attaches a target property from a target node that is connected to a source node.
- Prunes nodes of specific rdf types
- Pruning the graph of triples where object is a node that is not found in graph.
- Added dict to urn:x-rdflib:default named graph
- Upsert prefixes to the name graph {named_graph}
- Added dict to urn:x-rdflib:default named graph
- Upsert prefixes to the name graph {named_graph}
Publishing Data Model to CDF¶
Now we are ready to publish this to CDF.
neat.to.cdf.data_model()
You can inspect the details with the .inspect.outcome.data_model(...) method.
name | unchanged | |
---|---|---|
0 | spaces | 1 |
1 | containers | 3 |
2 | views | 3 |
3 | data_models | 1 |
4 | nodes | 0 |
Populating Data Model¶
Neat keeps track of the data, so we can immidiately populate this data model with the original data
neat.to.cdf.instances()
INFO | 2025-01-22 15:05:22,473 | Staring DMSLoader and will process 3 views. INFO | 2025-01-22 15:05:22,482 | Starting ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0') 1/3.
Loading ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0'): 0%| …
INFO | 2025-01-22 15:05:22,826 | Finished ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0'). INFO | 2025-01-22 15:05:23,795 | Starting ViewId(space='aml_playground_2', external_id='InternalElement', version='topology_v3.0') 2/3. INFO | 2025-01-22 15:05:24,905 | Finished ViewId(space='aml_playground_2', external_id='InternalElement', version='topology_v3.0'). INFO | 2025-01-22 15:05:25,074 | Starting ViewId(space='aml_playground_2', external_id='InternalLink', version='topology_v3.0') 3/3. INFO | 2025-01-22 15:05:25,086 | Finished ViewId(space='aml_playground_2', external_id='InternalLink', version='topology_v3.0').
You can inspect the details with the .inspect.outcome.instances(...) method.
name | created | changed | |
---|---|---|---|
0 | ExternalInterface | 858 | 0 |
1 | InternalElement | 65 | 2 |
2 | edge | 972 | 0 |
3 | InternalLink | 0 | 48 |