Log In
Home
11/04/2016
Offline
Hello
I´m pretty new to opcua but could implement allready a little testserver.
Now I have some questions regarding the load that will happen in a server and in the network.
The background is: We have a software model that will update it´s states in a certain time, say min. 100ms. This is however not a constant. This software model is of medium complexity, say 50 objects in a 3-4 leveled tree hierarchy, 1000 values approx. alltogether. But there may exist 10 or more of these models parallel.
The software model updates or calculates the real object data not at once, but over the min. cyclus time, let´s say it takes ~90ms in this example. However, only the final state of all values are of interest as they represent a unique dataset (the values themselves, not the state of a value). This is very important to understand. The client should treat this final dataset, not any intermediate states. The client should in now way mix up different value data from a previous real world measurment cycle with the current one!
Now, how can i control opcua to mirror to clients only the values when the final real world state of all values is reached?
I know about monitoring, but this is a polling, and that is not what I want as it can cause unpredictable load. What I understood is that the client even configures the poll rates, guess what will happen... .
On the other hand the server can write directly values but this may result in hundreds of individual messages over network, definitly not a good concept too.
Is there any possibillity to update the adressspace with new values without transmission, and then at the end trigger the client to read all in one message or transmit automatically changes with one message to the client? Server controlled?
Is it at least possible to control the point of value updates grouped by node objects?
Thank you for info
Stefan
Moderators-Specifications
Moderators-Companion
Moderators-Implementation
Moderators-Certification
Moderators-COM
02/24/2014
Offline
Stefan
I'm not sure what your model looks like, but when I have a group of values that need to be reported together I usually create structure (datatype) and then build a variable type that has the structure as the value. Then the server software can just report the structured value (with a single time stamp) when the calculation cycle is complete. The client can just subscribe for the value, and it will only report when it changes (so no polling or special reporting needed).
I know from your description that this might not be great since you have a complex structure of objects (3-4 levels deep) and many values (1000 values), but the the structure can be as complex as you want - i.e. it can be nested structures. If the individual values need to be exposed the variable Type can have children that are the sub-structures. Again it would be up to the server as to when it decides the values of any sub-structures need to be reported.
Does the model need to be able to report the intermediate values for debugging or testing? the underlining server could move the data in the reporting variables when the cycle is complete could be another option ( the running calculation variables could just be internal variables).
A last option would be to provide the values as an custom EventType where the event is trigger only when the calculation is complete. the client would have to subscribe for the event. A generic client might not be able to process this type of event, i.e. it might take a client that understands the custom event.
Hopefully this helps
Paul
Paul Hunkar - DSInteroperability
Just as a clarification:
What you call polling is actually publishing (best known as long polling). I.e. the Client places a "Publish Request" but the Server does not respond until it has interesting data to publish.
Since the publishing rate is also negotiated between Client and Server the reporting frequency is pretty much under control of the Server.
11/04/2016
Offline
paul.hunkar@dsinteroperability.com said
StefanI'm not sure what your model looks like, but when I have a group of values that need to be reported together I usually create structure (datatype) and then build a variable type that has the structure as the value. Then the server software can just report the structured value (with a single time stamp) when the calculation cycle is complete. The client can just subscribe for the value, and it will only report when it changes (so no polling or special reporting needed).
I know from your description that this might not be great since you have a complex structure of objects (3-4 levels deep) and many values (1000 values), but the the structure can be as complex as you want - i.e. it can be nested structures. If the individual values need to be exposed the variable Type can have children that are the sub-structures. Again it would be up to the server as to when it decides the values of any sub-structures need to be reported.
Does the model need to be able to report the intermediate values for debugging or testing? the underlining server could move the data in the reporting variables when the cycle is complete could be another option ( the running calculation variables could just be internal variables).
A last option would be to provide the values as an custom EventType where the event is trigger only when the calculation is complete. the client would have to subscribe for the event. A generic client might not be able to process this type of event, i.e. it might take a client that understands the custom event.
Hopefully this helps
Paul
HI Paul
I´ll dig into that. Structures would be the next step. I also thought about an Event, however this may take just some more time where I can not update - or in worst case start processing - my next cycle data in case. I´d prefer to be active as Server - or let´s say data Provider, as the classic Client-Server concept does not apply for opc imo, except for the Network connection.
@Karl-Heinz
What I understood there are even 2 Kind of pollings. The publish requests (where I don´t know how many come in, how many get queued, get I flooded, is there allways one so my data moves out as fast as possible - all together not a good concept ), and the Monitoring Loop (sampling rate and reporting rate) in the Server which polls regullary for data updates - e.g. it will call datasource callbacks, where I have to write the new value. As the later one is configured by the Client it may cause unpredictable load on my side. I still have to find out how to Limit this rate, which is maybe sdk specific. While the Monitoring concept may be ok for a classical primitive sensor observer (e.g. Motherboard temperature watcher), i consider it terrible for my purpose.
BR
Stefan
11/04/2016
Offline
May i just ask one question: If the Server updates himself a value by writing, not in a datasource callback, not monitored, just straight ahead - Is the value immediatly mirrored to Clients by a message or only the next time the Client starts a specific read (however) on that value?
Moderators-Specifications
Moderators-Companion
Moderators-Implementation
Moderators-Certification
Moderators-COM
02/24/2014
Offline
Stefan,
Just to clarify a few points - Are you using an SDK? and is it for embedded devices or one for larger servers?
So most SDK's handle the client interface aspect of the server - you have to write the code that handle updating values (i.e. getting the current value). Depending on the type of SDK how this is done can vary quite a bit.
A client can get values from a server in two manners reading or subscribing.
In reading the server receives the read request and answers it, most SDK will just report the current value back, in many case it does not even enter into application code (i.e. code you wrote).
In subscribing the client specifies a sampling rate and a publishing rate - but the server can overwrite this and propose a different rate. The sampling rate is the maximum updates rate - i.e. a sampling rate of 10 Milliseconds mean you will not get values more often then 1 every 10 milliseconds. Sampling rates are set on individual monitored Items. The publishing rate is how often values are sent back to the client (multiple values can be batched together and returned in a packet). The publish interval applies to all monitored items in a subscription. In most SDK's the major portion of the work for subscriptions is done by the SDK, but the server vendor does get to control when value are marked as updated and queue to be sent to the client. Since the client has to initiate all communication, the client sends a publish request down to the server which the server just holds it until it determines something need to be returned to the client. For subscriptions clients can set filters (i.e. value has to change by so much, or get a new timestamp or any of several other filters), which affect what is returned.
So to answer your question - it depends on the SDK you are using and how the internal points are handled and if the client has is reading or subscribing.
Paul
Paul Hunkar - DSInteroperability
11/04/2016
Offline
Hello Paul
I use a SDK but care only about the Server. My real Job is to collect and Prozess the real life object data. OPC should be used just as Interface for datalogging - but not at random times, as explained it should do this as a dataset to describe a well defined real life object state. Furthermore the datacollector (who has to control a set of Clients connected to several different data Provider, say Server or process Station, so actually we have a 1 "MasterClient" to n Server condition in opc) must grab from different Servers and assign all data to one virtual object.
So I would actually Need a Kind of PartID identifieing each data. I´m wondering why opc does not have a 64bit unique identifier by default. Timestamp is critical as it may vary for a huge amount of data even inside one Server, let alone 10 different servers. Of course i can set up my own attribs somehow but this makes everything more complex.
My fear is that opc will slow down my real Job, processing the object data. So I Need control for the way the Client is asking for data. I would prefer to control or offer the way how data can be grabbed as Server, or even Trigger the data sending. Because only me knows when the best Point to do so is reached.
Best regards
Stefan
11/04/2016
Offline
Just to think further: What would be cool is if the Server could define a Kind of changeset. So the Client would be able in the callbacks to identify all data belonging to the changeset. In best case, the api will collect on Receiver side all changeset data and may even wait until all changeset data was received. Then the Client gets a callback with the Array of all changed items in the changeset, so he can check the dependencies or release an Action only once at the end. Number of items in changeset with changeset index or Count would do also.
Moderators-Specifications
Moderators-Companion
Moderators-Implementation
Moderators-Certification
Moderators-COM
02/24/2014
Offline
Stefan,
I look back over this thread and I'm not sure where you are headed. You have talked about a client and a server, I'm not actually sure of what you are building and how best to answer your questions. Hopefully this general description can help.
From a server point of view:
A server exposes an address space - every node is uniquely identified by a Nodeid, the NodeId can be a GUID, Integer,String or bytestring, it is up to the server implementer to decide what is used, the NodeId also contains a namespace index, which provide information about who provided the nodeid (i.e. defined as part of a standard, or by the server etc.) the key point is that the NodeId will uniquely identify every node in a server and in a system.
A server address space is composed of Objects, Variables (both are considered Nodes) and References. Objects organize the nodes, Variable have values and reference show relationships between the various nodes.
The server must obtain values for the variables. Sometimes the value is generated by the server in internal code, some times the value is obtained by the server from some underlining system using some other protocol or communication method. In either case a timestamp is associated with when the value was obtained (ServerTimestamp), along with a status. A second Timestamp (sourceTiimestamp) may also exist that indicates when the value was generated (which might be different then when it was obtained, if the underlining system has timestamps).
The server when it is servicing requests for data has to determine if it needs to go to an underlining system and get a value or if the underlining system always reports back values and they are just current in the system. Internally generated values are can behave in the same manner, in that the value could be generated on some cycle and the cached value is just current or it might be that the generated values are only generated when someone is accessing them (i.e. the value is generated as part of the access). This is up to the server implementer, since typically they have a much better idea of how the underlining data is obtained and or generated (does the underling system need to be polled, does it report by exception etc.)
There are two types of data requests in the current release of OPC UA: Read and Subscribe.
In a Read the server just reports what it thinks is the current value (this may have required a read from an underlining system or it might just be the cached value - it is up to the server vendor to determine what is current). The client does provide a maxAge, which is a guide to the server that the server timestamp should not be older than what is requested. i.e. if the maxAge is 500 milli second the value has to have been obtained by the server withing the last 500 milliseconds or the server has to know that the value is the current value (if the underlining system reports all changes, the server can assume the value is current even if it is older - it would just get an updated timestamp.) Reading is usually used for onetime gets of values.
In a subscribe, the server reports changes (possibly via a filter or via a trigger). Subscriptions have a number of settings associated with them:
- sampling rates which are used to restrict how often something is reported (a monitored item) - i.e. if a value changes every millisecond, but a sampling rate is 1 second then the client does not want to see a 1000 values it just wants the value at most every second. Server may restrict both the maximum or minimum sampling rate that can be provided and in some systems the server may also restrict the sampling rates to some fixed intervals, since those are the only intervals available from the underlining system. i.e. my polling rate from the underlining system is 100 milli-seconds, so my sampling rate can not be smaller then 100 millisecond and must be multiple of it.
- queue size which say how many values to report at one time (send me 1000 values or just send me 1 value)
- Publishing rate which is used to group together items to report. The client may want values every milli-second, but it would like 1000 grouped together every second, instead of 1000 notification in a second. the publishing interval also groups together values from multiple monitored items.
- there are several other parameters that can affect this like DiscardOldest, but I'm not going to cover all of them
For subscriptions - monitored items report changes only - not every monitored item reports only items that have a change i.e. if nothing changes in a publishing interval there is nothing to report. if every point changes then every point is reported.
Subscriptions also support a trigger capability - a subscription can be created with a number of monitored items, then a seperate items can be configured as a trigger, the subscription would not report changes until the trigger changes at which point all items in the queue for the subscription are reported. If the queue size is 1 then only the most recent value is report. maybe this would work for what you are trying/
From a client point of view:
the client can read values, the read just gets the value from the variable and returns it. This is useful when getting properties of some other items that don't change or don't change often.
The client can subscribe for values, the subscription can report all changes in values or have some filter or trigger to restrict what is reported.
Pub/Sub
We are currently working on a pub/sub extension to OPC UA and I'm wondering if that might be a better fit for what you are looking for? the source publish data and subscriber just listen for it. Multiple subscribers can get the same packet. The packet can be a delta packet (changes only) or a key frame packet - all of the values in the group. The publisher decides what is sent out the subscriber just looks at the packet and keeps what it wants out of the packet. This is new and will be released beginning of next year, prototypes are available.
Paul
Paul Hunkar - DSInteroperability
11/04/2016
Offline
Thx for your Long reply and excuse my late answer.
I know the read and subscribe model, and it does not fit my Need for real time.
As far as I understood (please correct me), even Events signalled in the Server are notified in the Client only when a subscription Intervall ellapsed (monitored) or the Client may read it (unlikely).
So I want to break everything down into a simple question:
Is there any mechanism that a Server signals something for the Client(s), and the Client is informed about this immediatly (as fast as normal delay times due to Network Transmission, Encoding/decoding etc. allow) ???
BR
Stefan
Moderators-Specifications
Moderators-Companion
Moderators-Implementation
Moderators-Certification
Moderators-COM
02/24/2014
Offline
Stefan,
The subscription from a server can also be very fast (millisecond range). A client can configure a 1 milli second update rate (assuming the server supports it) and then any changes will be delivered every milli-second, which is typically more than enough for most applications, but this still may not be fast enough for what you are looking for I'm guessing
I think you are looking for the new Pub/Sub extension. A Publisher publishes data and subscriber just listen for it (servers are usually the publisher). This feature is coming out end of the month as an RC. Pub/Sub when coupled with TSN will be able to support real-time control. Without the TSN it is very fast, but not always the same slot i.e. a message could vary depending on network load for how long it takes to be delivered, TSN eliminates the network load variability. The UDP transport for Pub/Sub is targeted for Controller to Controller communication or Controller to Robot communication. With TSN it would be real-time, without it would fast enough for real-time but deliver variability could cause issues in some systems. The TSN option will not be available until later 2017 (TSN is IEEE Time Sensitive Networking)
The same information model can be shared between Client/Server or Pub/Sub
[note: pub/sub has other targets (such as cloud application) and other transports (AMQP, MQTT...) also, but the UDP model is what it sounds like you are looking for]
Paul
Paul Hunkar - DSInteroperability
11/04/2016
Offline
Hello Paul
That sounds promising, though I don´t understand the details for the moment and I don´t understand why it must be so complex. And until it is realized in the SDK´s around, time will pass.
Fact: you own the socket, and you own the protocol, so you can send what you want. Actually a server can send something without an input of a client - just a matter of protocol that the client identifies it and reads it.
So why can´t you make the current events kind of real time? I set them as server, they are sent out to subscribers immediatly. That´s all. Now the client can read immediatly
If server could trigger somehow point of sampling (e.g. for data with sampling rate set to INFINITE), this would work also easily
Anyway, looking forward for Pub/Sub
thx for information
Stefan
Moderators-Specifications
Moderators-Companion
Moderators-Implementation
Moderators-Certification
Moderators-COM
02/24/2014
Offline
Stefan,
Prototypes are available for Pub/Sub, so you can start looking at it already (depending on what SDK you are using)
For client / server the client must initiate something - i.e. have a command that the server responds to - which is how the publish mechanism works in the subscription model. The client sets up a subscription, defines the rates, but if a server allow a very short rate (milli-seconds or less) it can effectively send something as soon as it is ready to send - the client parks publishes at the server and the server just responds to it. So if you are writing both sides and can define a short enough interval it appear as if the server sends a response to the client as soon as anything changes in the server.
Paul
Paul Hunkar - DSInteroperability
1 Guest(s)
Usage Policy