Onboard AutomationML (AML) data to CDF¶

Prerequisite:

• Installed Neat, see Installation
• Launched a notebook environment.
• Familiar with the NeatSession object, see introduction
• Access to NeatEngine.

In this tutorial, we will:

• extract a knowledge graph from a aml topology file and transform it for further usage in CDF
• infer a data model based on the transformed knowledge graph
• and finally push both the data model and instances from knowledge graph to CDF

Read more about aml here: https://www.automationml.org/

Extract knowledge graph from AML topology file¶

We will start by instantiating a NeatSession and read the data from a local folder. Make sure you have an AML file in the folder. In case of this tutorial we are using a AML file provided by DISC initiative.

In [1]:

from cognite.neat import NeatSession, get_cognite_client

from cognite.neat import NeatSession, get_cognite_client

In [2]:

# Note that we use Oxigraph in this example, this will not work in a CDF notebook
# to make it work in a CDF notebook replace set storage="memory"

neat = NeatSession(get_cognite_client(".env"), storage="oxigraph")

# Note that we use Oxigraph in this example, this will not work in a CDF notebook # to make it work in a CDF notebook replace set storage="memory" neat = NeatSession(get_cognite_client(".env"), storage="oxigraph")

Found .env file in repository root. Loaded variables from .env file.
Neat Engine 2.0.3 loaded.

Process of extracting graph from an aml file can take a bit of time, so be patient.

In [3]:

neat.read.xml.aml("data.aml")

neat.read.xml.aml("data.aml")

Prunes nodes of specific rdf types: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1631/1631 [00:00<00:00, 40111.82it/s]
Pruning the graph of triples where object is a node that is not found in graph.: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1631/1631 [00:00<00:00, 4397.65it/s]

If we now check content of the graph object, we will see that it contains a knowledge graph extracted from the aml file:

In [5]:

neat

neat

Out[5]:

Instances

Overview:

• 1 named graphs
• Total of 3 unique types
• 973 instances in default graph

default graph:

	Type	Occurrence
0	ExternalInterface	858
1	InternalElement	67
2	InternalLink	48

Provenance:

• Initialize graph store as OxigraphStore
• Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
• Prunes the graph of triples where the object is a node that is not found in the graph.
• Attaches a target property from a target node that is connected to a source node.
• Prunes nodes of specific rdf types
• Pruning the graph of triples where object is a node that is not found in graph.

If we run notebook locally we can preview a part of knowledge graph. The entire knowledge graph for this specific file we used look like this (while you will see only fragment of it in notebook):

In [7]:

neat.show.instances();

neat.show.instances();

instances.html

Infer Data Model¶

We can infer a data model from data in the NeatSession by calling .infer().

In [6]:

neat.infer()

neat.infer()

Out[6]:

No issues found

In [7]:

neat

neat

Out[7]:

Data Model

type	Logical Data Model
intended for	Information Architect
name	Inferred Model
external_id	NeatInferredDataModel
version	v1
classes	3
properties	45

Instances

Overview:

• 1 named graphs
• Total of 3 unique types
• 973 instances in default graph

default graph:

	Type	Occurrence
0	ExternalInterface	858
1	InternalElement	67
2	InternalLink	48

Provenance:

• Initialize graph store as OxigraphStore
• Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
• Prunes the graph of triples where the object is a node that is not found in the graph.
• Attaches a target property from a target node that is connected to a source node.
• Prunes nodes of specific rdf types
• Pruning the graph of triples where object is a node that is not found in graph.

We now convert verified data model to be DMS compliant

In [8]:

neat.convert()

neat.convert()

Rules converted to dms.

Out[8]:

Success: NEAT(verified,logical,neat_space,NeatInferredDataModel,v1) → NEAT(verified,physical,neat_space,NeatInferredDataModel,v1)

Inspect Data Model¶

First we will set data model id to ("aml_playground", "AML", "topology_v3.0"), where tuple represents (space, external_id, version) of data model

We use the .show to visualize the data model and see how data model objects are connected, the resulting interactive data model visualization should be similar to this one:

In [9]:

neat.show.data_model()

neat.show.data_model()

http_purl.org_cognite_neat_data-model_verified_physical_neat_space_NeatInferredDataModel_v1.html

Out[9]:

In [10]:

neat.set.data_model_id(("aml_playground_2", "AML", "topology_v3.0"))

neat.set.data_model_id(("aml_playground_2", "AML", "topology_v3.0"))

Out[10]:

Success: NEAT(verified,physical,neat_space,NeatInferredDataModel,v1) → NEAT(verified,physical,aml_playground_2,AML,topology_v3.0)

In [11]:

neat

neat

Out[11]:

Data Model

aspect	physical
intended for	DMS Architect
name	Inferred Model
space	aml_playground_2
external_id	AML
version	topology_v3.0
views	3
containers	3
properties	45

Instances

Overview:

• 1 named graphs
• Total of 3 unique types
• 973 instances in default graph

default graph:

	Type	Occurrence
0	ExternalInterface	858
1	InternalElement	67
2	InternalLink	48

Provenance:

• Initialize graph store as OxigraphStore
• Extracted triples to named graph urn:x-rdflib:default using AmlExtractor
• Prunes the graph of triples where the object is a node that is not found in the graph.
• Attaches a target property from a target node that is connected to a source node.
• Prunes nodes of specific rdf types
• Pruning the graph of triples where object is a node that is not found in graph.
• Added dict to urn:x-rdflib:default named graph
• Upsert prefixes to the name graph {named_graph}
• Added dict to urn:x-rdflib:default named graph
• Upsert prefixes to the name graph {named_graph}

Publishing Data Model to CDF¶

Now we are ready to publish this to CDF.

In [28]:

neat.to.cdf.data_model()

neat.to.cdf.data_model()

You can inspect the details with the .inspect.outcome.data_model(...) method.

Out[28]:

	name	unchanged
0	spaces	1
1	containers	3
2	views	3
3	data_models	1
4	nodes	0

Populating Data Model¶

Neat keeps track of the data, so we can immidiately populate this data model with the original data

In [29]:

neat.to.cdf.instances()

neat.to.cdf.instances()

INFO | 2025-01-22 15:05:22,473 | Staring DMSLoader and will process 3 views.
INFO | 2025-01-22 15:05:22,482 | Starting ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0') 1/3.

Loading ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0'):   0%|     …

INFO | 2025-01-22 15:05:22,826 | Finished ViewId(space='aml_playground_2', external_id='ExternalInterface', version='topology_v3.0').
INFO | 2025-01-22 15:05:23,795 | Starting ViewId(space='aml_playground_2', external_id='InternalElement', version='topology_v3.0') 2/3.
INFO | 2025-01-22 15:05:24,905 | Finished ViewId(space='aml_playground_2', external_id='InternalElement', version='topology_v3.0').
INFO | 2025-01-22 15:05:25,074 | Starting ViewId(space='aml_playground_2', external_id='InternalLink', version='topology_v3.0') 3/3.
INFO | 2025-01-22 15:05:25,086 | Finished ViewId(space='aml_playground_2', external_id='InternalLink', version='topology_v3.0').

You can inspect the details with the .inspect.outcome.instances(...) method.

Out[29]:

	name	created	changed
0	ExternalInterface	858	0
1	InternalElement	65	2
2	edge	972	0
3	InternalLink	0	48

In [ ]: