Questions about OPC UA PubSub|OPC UA Standard|Forum|OPC Foundation

Hi community!

In our company, Equinor, we have studied, used and dreamed about OPC UA for the last 5 years and we look at it as a key connectivity framework between OT and IT today and into the future. We have also the last two years established a common data platform in Equinor called Omnia (based on Microsoft Azure) that among other things contains an IoT platform that is able to handle timeseries data streamed from (currently) on-premises Historians.

We were eager on the release of the OPC UA PubSub specification because of the many advantages it provides in data exchanges between OT and cloud applications/services. That communication model goes hand in hand with event-driven architecture patterns used in many IoT platforms. My experience with IIoT and data-intensive platforms so far is that data (e.g. timeseries data published by an OPC UA publisher) often needs to be accessed and processed in many different ways, and in the cloud it is rarely one single technology that can satisfy all the needs at the same time. For data-intensive systems it not unusual to use a combination of several different datastores, indexes, caches, processing systems/layers etc. and implement mechanisms for moving data from one component to another - and having data published to a broker is the first important step and the entry point into the rest of the cloud services.

I have some questions/reflections that I would love to hear your input on.

• Some of the questions I get is “Why do we need OPC UA PubSub?” and “What benefits do we get by implementing OPC UA PubSub?” – and now I talk about OPC UA PubSub used for connectivity between OT and IT (cloud connectivity). The background for these questions is that several “IoT Gateways” in the market, released long before OPC UA Part 14, is offering functionality to connect to any OPC UA server (OPC UA client stack), subscribe to selected variables and publish them further to the cloud, either Microsoft Azure, Amazon AWS and/or Google Cloud Platform using either their SDKs / APIs or native AMQP/MQTT libraries. From a technical point of view, the data is flowing fine from A to B with (usually) a vendor-specific JSON encoding. Why shall these IoT Gateways then support OPC UA PubSub? I have tried to reflect around that.
  • Security is one benefit addressed in OPC UA Part 14. OPC UA PubSub defines an end-to-end security independent of the transport protocol. But to my understanding, Part 14 also says that this end-to-end security is only possible with UADP NetworkMessages and binary message encoding. For JSON encoded messages you must rely on security mechanisms provided by AMQP/MQTT and the message broker. For cloud connectivity, JSON is heavily used and if somebody is using a binary format it is often Protocols Buffers (Protobuf), the open binary format by Google. What are your reflections about this? Do you think UADP will be commonly supported in programming languages equally to formats like JSON, Protobuf? We work closely with Microsoft and their OPC UA open source contributions are impressive, but there are currently no services in the cloud with inherent knowledge about UADP – but that may change in the future.
  • Another benefit I see is that OPC UA PubSub at least defines a standard overall schema. It defines JSON objects for DataSetMessage, NetworkMessage and DataSetMetadata with mandatory and optional fields. OPC UA PubSub is not defining the payload object itself (which I guess is up to per application) in the DataSetMessage, but it defines an overall message structure that at least will ensure a syntactic interoperability to some level.
  • The configuration is also a key benefit addressed in OPC UA Part 14. The specification says that a publisher typically will be an OPC UA server (the owner of the information) and a subscriber is often an OPC UA client, but it also says that this is not a requirement. In the products I have seen and tested, the OPC UA publisher was a standalone application with an embedded OPC UA client stack that subscribed (client/server) to a relevant dataset in a third-party OPC UA server, constructs PubSub messages and makes it available to the publisher for publishing. The OPC UA PubSub information model for configuration promotes doing the configuration of publishers and subscribers using the OPC UA client/server communication model which means the OPC UA publisher must be an OPC UA server as well (?). This type of configuration has several benefits, you know that regardless of the vendor of the OPC UA PubSub application, the configuration and parameters are consistent and have the same meaning, and you can ideally use other third-party OPC UA clients supporting the PubSub configuration information model for configuration. This supports portability, i.e. sharing configuration across implementations without modifications. Do you have any reflections on this? The specification also says that publishers and subscribers may be configurable through vendor-specific engineering tools – and then the mentioned benefits will be less I guess.
  • I also like the fact that OPC UA PubSub is not bound to a particular messaging system and encoding. It means that the core features and requirements can be mapped to new emerging transport protocols / messaging systems and encodings as they come. I see this as a benefit.
  • Do you have any other feedback to the questions I get, “Why do we need OPC UA PubSub?” and “What benefits do we get by implementing OPC UA PubSub?” for cloud connectivity?
• As mentioned, the DataSetMessage payload itself is application-specific. This means that even if the OPC UA server with the originating dataset has an information model according to a companion specification, there are some uncertainty / freedom on what shall be included in the DataSetMessages payload. How shall the payload JSON object be structured, and what shall it consist of? Of how much of the information model in the source will you get through DataSetMessages and DataSetMetadataMessages? What shall you model as DataSet and what you shall model as DataSetMetadata? I have seen examples where metadata is put into the DataSetMessage instead of DataSetMetadataMessage (no implementation of DataSetMetaData) and examples where you have both. Do you think OPC UA companion specifications also will give some requirements / guidelines on how to construct the payload objects themselves if you want to publish from an OPC UA server with a companion standard? Or do you think we as an end user shall put some requirements on this?
• In your opinion, what does it mean to be OPC UA PubSub compliant? What exactly shall a publisher or OPC UA server implement and support to be "OPC UA PubSub compliant"? Can an publisher getting data from a non-OPC source that only constructs OPC UA PubSub messages be considered an OPC UA PubSub compliant publisher? I don’t find many OPC UA PubSub profiles at https://apps.opcfoundation.org.....reporting/, only a couple on the Transport Category.
• OPC UA Part 14 says that a DataSetClass acts as a template declaring the content of a DataSet. Do you know what this means in practice?
• Do you know any other companies out there that have created OPC UA PubSub products? Or end users using OPC UA PubSub for cloud connectivity today that we can share experiences with?
• There are some “DDS vs OPC UA PubSub” opinions out there that makes me a bit uncertain as well. I don’t know so much about DDS, but I understand they are in the PubSub business. I read that OPC Foundation and OMG was going to collaborate (https://opcfoundation.org/news.....standards/), but I have not read any news about this the last years. And OMG has made some kind of OPC UA / DDS Gateway specification (https://www.omg.org/spec/DDS-O.....DDS-OPCUA/). What are your reflections on this? I am a bit uncertain if DDS is the right choice for connectivity between cloud and OT.

Best Regards,

Steffan Sørenes

Equinor