Use SHACL shape as input for code generation

[lecoqlibre]

Currently we are generating source code from a UML model. This solution is costly to maintain and not user friendly. It needs to be done using a graphical tool that is hard to install and debug (Eclipse + Acceleo). The learning curve of Acceleo is high. Plus we are mainly using this solution to generate accessors and mutators (getters, setters, adders and removers).

For all these reasons I would like to propose to replace the UML model by SHACL shapes. SHACL Shapes Constraint Language is a language for validating RDF graphs against a set of conditions. I did some tests writing a simple generator in TypeScript and it worked well. Below is an example of an Enterprise shape in Turtle that allowed me to generate source code:

@prefix : <#>.
@prefix dfc: <https://github.com/datafoodconsortium/ontology/releases/latest/download/DFC_BusinessOntology.owl#>.
@prefix sh: <http://www.w3.org/ns/shacl#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>. 

:EnterpriseShape a sh:NodeShape;
    sh:targetClass dfc:Enterprise;
    sh:closed false;

    sh:property [
	sh:path dfc:name;
        sh:message "The name of the enterprise.";
        sh:datatype xsd:string;
        sh:maxCount 1;
	];

    sh:property [
	sh:path dfc:description;
        sh:message "The description of the enterprise.";
        sh:datatype xsd:string;
	];

    sh:property [
	sh:path dfc:date;
        sh:message "The date of the enterprise.";
        sh:datatype xsd:dateTime;
    ];

    sh:property [
	sh:path dfc:hasAddress;
        sh:message "The different addresses of the enterprise.";
        sh:datatype dfc:Address;
    ];

    sh:property [
	sh:path dfc:hasPhoneNumber;
        sh:message "The different phone numbers of the enterprise.";
        sh:datatype dfc:PhoneNumber;
   ];

    sh:property [
	sh:path dfc:hasSocialMedia;
        sh:message "The different social medias of the enterprise.";
        sh:datatype dfc:SocialMedia;
	];

    sh:property [
	sh:path dfc:logo;
        sh:message "The different logos of the enterprise.";
        sh:datatype xsd:anyURI;
	];

    sh:property [
	sh:path dfc:orders;
        sh:message "The different orders ordered by the enterprise.";
        sh:datatype dfc:Order;
	];

    sh:property [
	sh:path dfc:requests;
        sh:message "To be defined.";
        sh:datatype dfc:FunctionalProduct;
	];

    sh:property [
	sh:path dfc:owns;
        sh:message "To be defined.";
        sh:datatype dfc:PhysicalProduct;
	];

    sh:property [
	sh:path dfc:email;
        sh:message "To be defined.";
        sh:datatype xsd:string;
	];

    sh:property [
	sh:path dfc:websitePage;
        sh:message "To be defined.";
        sh:datatype xsd:anyURI;
	];

    sh:property [
	sh:path dfc:VATnumber;
        sh:message "To be defined.";
        sh:datatype xsd:string;
        sh:maxCount 1;
	];

    sh:property [
	sh:path dfc:defines;
        sh:message "To be defined.";
        sh:datatype dfc:CustomerCategory;
	];

    sh:property [
	sh:path dfc:hasMainContact;
        sh:message "To be defined.";
        sh:datatype dfc:Person;
        sh:maxCount 1;
	];

    sh:property [
	sh:path dfc:supplies;
        sh:message "To be defined.";
        sh:datatype dfc:SuppliedProduct;
	];

    sh:property [
	sh:path dfc:manages;
        sh:message "To be defined.";
        sh:datatype dfc:CatalogItem;
	];

    sh:property [
	sh:path dfc:proposes;
        sh:message "To be defined.";
        sh:datatype dfc:TechnicalProduct;
	];

    sh:property [
	sh:path dfc:transforms;
        sh:message "To be defined.";
        sh:datatype dfc:AsPlannedLocalTransformation;
	];

    sh:property [
	sh:path dfc:maintains;
        sh:message "To be defined.";
        sh:datatype dfc:Catalog;
	];

    sh:property [
	sh:path dfc:affiliates;
        sh:message "To be defined.";
        sh:datatype dfc:Person;
	];

    sh:property [
	sh:path dfc:coordinatedBy;
        sh:message "To be defined.";
        sh:datatype dfc:Coordination;
	].

As you see it is very simple to use and it's just RDF! We would also use shapes to do data validation.

It is not possible to generate source code from an ontology itself as an ontology is wide more open and some constraints may not be expressed.

[RaggedStaff]

As discussed on today's call, I agree that SHACL would be a simpler & more appropriate way to express the ontology/standard to UML.

I have 2 principal concerns:

1. If we change the way we generate the code, there is a significant risk we break existing implementations. We can mitigate this by ensuring we have thorough test suites (perhaps based on the existing test suites for the connector libraries) and ensuring all tests are passed
2. There will be a cost involved in implementing this & (re)creating the 3 existing connectors (Typescript, PHP & Ruby). We need clarity on what that cost will be so we can either agree or seek funding to cover that cost.

[RachL]

We also spoke about the impact on platforms currently using the connectors. We would need to take into account the cost for them to migrate, unless we are able to generate the same code.

[mkllnk]

This is a much better solution. I like that it's just RDF and we are not introducing any new language to the DFC space here. But I'm not quite sure about the next step. Is it useful to aim for one code generator that generates libraries in all languages from the same model? Or would it be much simpler to have a Ruby library that uses the SHACL shapes to generate Ruby code and a separate PHP library that generates PHP code?

We just need a simple templating tool that reads SHACL. If you wrote one in Typescript already, maybe we should just use that to generate all the languages? And that can be automated on CI. Libraries can be released and published easily. The only manual work would be in writing SHACL.

For the transition, we have pretty good test coverage in Ruby. And we can compare the generated code easily.

[mkllnk]

@lecoqlibre, do you think that you could generate some of the SHACL from the ontology and then add to it separately?

Looking at this example:

:EnterpriseShape a sh:NodeShape;
    sh:targetClass dfc:Enterprise;
    sh:closed false;

    sh:property [
	sh:path dfc:name;
        sh:message "The name of the enterprise.";
        sh:datatype xsd:string;
        sh:maxCount 1;
	];

• The Enterprise/Organisation class is in the ontology.
• The name property is in the ontology.
• Is the data type defined? I guess we know if it's another class or a primitive.
• Is the max count in the ontology? I thought so..

For Ruby, that would be enough information to write a basic connector library. We don't even need types, even though they are useful.

I'm just wondering how we can keep the ontology as source of truth.

[lecoqlibre]

It's possible to generate some SHACL from the ontology but I think it is not a good option. In fact I think we are expecting things from our ontology that it can not give. I think that our ontology can not be the source of truth. It is too much open as an ontology should be. It is not made for that. An ontology is just a vocabulary and it does not tell how that vocabulary should be used. We are missing contextualization.

Contexts should be defined by the protocol. If we want one source of truth I think it should be the protocol that both defines the ontology and the way it can be used (contexts). The ontology file must be derived from the protocol, not the inverse.

Today the ontology team encodes the ontology but do not defines any context. The reason is that contexts can not be encoded into an ontology. It is not the right tool. We need to define contexts in our protocol. That explains why even if we have on ontology implementers still don't know what to do with it.

So for me SHACL shapes should be defined in the protocol, by hand.

[lecoqlibre]

To clarify, I'm not saying that SHACL shapes should replace the ontology. They are complementary. I think we want to have:

• a protocol: to define interactions between platforms and allow implementations;
• an ontology: to define the semantics and allow reasoning;
• SHACL shapes: to define constraints an allow data validation and code generation. They should replace the UML data model.

I was saying that there are some technical things we discussed about during the ontology meetings that we were not able to express in the ontology. We have to define them in the protocol because on ontology is not self-sufficient.