| Re: [milo-dev] Netty LEAK message |
I am using ed0bed2 at 2017/09/25. I'm sorry, but in our circumstance
I can not immediately update it to current.
Also, as a result of running under the conditions of yesterday,
the following log was output. I have seen the inconsistency of
the sequence number of ChunkDecoder#decode() several times before.
Since it was reproduced in about one day, it became easier to
investigate.
---
2017-12-12 04:27:14,764+JST �d�å org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder decode - expected sequence number 113958 but received 113959
2017-12-12 04:27:14,794+JST �d�å org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder decode - 4d534746620000000d000000a80000 0027bd010027bd010001003a030500 0020000000dd61516bc8c0d6d4aae9 b95ade66e0a7e5503e531c86dbea43 1f1813a4125645e06aec10b672d301 11bd010000000000ffffffffa86100 0000000000000000
2017-12-12 04:27:14,794+JST �d�å org.eclipse.milo.opcua.stack.server.handlers. UaTcpServerSymmetricHandler lambda$1 - Error decoding symmetric message: expected sequence number 113958 but received 113959
UaException: status=Bad_SecurityChecksFailed, message=expected sequence number 113958 but received 113959
at org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder. decode(ChunkDecoder.java:115)
at org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder. decodeSymmetric(ChunkDecoder. java:66)
at org.eclipse.milo.opcua.stack.server.handlers. UaTcpServerSymmetricHandler. lambda$1( UaTcpServerSymmetricHandler. java:176)
at org.eclipse.milo.opcua.stack.core.channel. SerializationQueue.lambda$1( SerializationQueue.java:58)
at org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$ PollAndExecute.run( ExecutionQueue.java:107)
at java.util.concurrent.Executors$RunnableAdapter. call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask. java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker( 2017-12-12 04:27:15,159+JST �d�å io.netty.util.internal.ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
logging.Slf4JLogger error - LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/ reference-counted-objects.html for more information.
Recent access records: 0
Created at:
io.netty.buffer.AbstractByteBufAllocator. compositeDirectBuffer( AbstractByteBufAllocator.java: 215)
io.netty.buffer.AbstractByteBufAllocator. compositeDirectBuffer( AbstractByteBufAllocator.java: 210)
io.netty.buffer.AbstractByteBufAllocator. compositeBuffer( AbstractByteBufAllocator.java: 185)
org.eclipse.milo.opcua.stack.core.util.BufferUtil. compositeBuffer(BufferUtil. java:36)
org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder. decode(ChunkDecoder.java:70)
org.eclipse.milo.opcua.stack.core.channel.ChunkDecoder. decodeSymmetric(ChunkDecoder. java:66)
org.eclipse.milo.opcua.stack.server.handlers. UaTcpServerSymmetricHandler. lambda$1( UaTcpServerSymmetricHandler. java:176)
org.eclipse.milo.opcua.stack.core.channel. ---SerializationQueue.lambda$1( SerializationQueue.java:58)
org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$ PollAndExecute.run( ExecutionQueue.java:107)
java.util.concurrent.Executors$RunnableAdapter. call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask. java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)
<kevinherron@xxxxxxxxx> wrote, Mon, 11 Dec 2017 04:38:14 -0800
> > > > 2017-12-07 23:34:06,351+JST �x�� org.eclipse.milo.opcua.stack.
> What version of Milo is this based on?
>
> I ran ExampleServer with only 32MB allocated, 3 clients, 67 items, 50ms
> publishing and sampling interval, and left it overnight but haven't seen
> any indication of a leak.
>
> I'm using current HEAD (941736c) in master branch.
>
> On Sun, Dec 10, 2017 at 8:44 PM, Shigeru Ishida <ishida_shigeru@xxxxxxxxxxx>
> wrote:
>
> > Hi Kevin,
> >
> > -Dio.netty.leakDetection.maxRecords=1000
> > -Dio.netty.leakDetection.level=paranoid
> >
> > It started with the above setting. I hope to see some logs
> > within this year.
> >
> > Thank you very much.
> >
> > Regards,
> >
> > --Shigeru
> >
> > <kevinherron@xxxxxxxxx> wrote, Sun, 10 Dec 2017 20:05:57 -0800
> >
> > > Hmm. I don't see anything obvious right now.
> > >
> > > Try setting -Dio.netty.leakDetection.maxRecords=1000 (or more) as a JVM
> > > startup parameter.
> > >
> > > If it's not a production system, you might also set
> > > -Dio.netty.leakDetection.level=paranoid to hopefully reproduce the issue
> > > faster...
> > >
> > > On Sun, Dec 10, 2017 at 5:14 PM, Shigeru Ishida <
> > ishida_shigeru@xxxxxxxxxxx>
> > > wrote:
> > >
> > > > The following log was output just before Netty LEAK log. The log level
> > of
> > > > the simulation server is set to INFO. Before this log, it is the log
> > just
> > > > after startup. Also after Netty LEAK log, "java.lang.OutOfMemoryError:
> > > > Java heap space"
> > > > was output for a while.
> > > >
> > > > In addition, I have changed SubscriptionModel.java of 2017/09/25
> > version
> > > > slightly, so I will send it in case.
> > > >
> > > > ---
> > > > 2017-12-07 23:36:48,242+JST �x�� org.eclipse.milo.opcua.stack.> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel$
> > > > ScheduledUpdate.lambda$0(SubscriptionModel.java:133)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel$
> > > > ScheduledUpdate$$Lambda$1225/1616440616.apply(Unknown Source)
> > > > at java.util.stream.ReferencePipeline$3$1.accept(
> > > > ReferencePipeline.java:193)
> > > > at java.util.ArrayList$ArrayListSpliterator.
> > > > forEachRemaining(ArrayList.java:1374)
> > > > at java.util.stream.AbstractPipeline.copyInto(
> > > > AbstractPipeline.java:481)
> > > > at java.util.stream.AbstractPipeline. wrapAndCopyInto(
> > > > AbstractPipeline.java:471)
> > > > at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(
> > > > ReduceOps.java:708)
> > > > at java.util.stream.AbstractPipeline.evaluate(
> > > > AbstractPipeline.java:234)
> > > > at java.util.stream.ReferencePipeline.collect(
> > > > ReferencePipeline.java:499)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel$
> > > > ScheduledUpdate.run(SubscriptionModel.java:134)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel$
> > > > $Lambda$1224/129353204.accept(Unknown Source)
> > > > at java.util.concurrent.CopyOnWriteArrayList.forEach(
> > > > CopyOnWriteArrayList.java:890)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel.
> > > > reschedule(SubscriptionModel.java:108)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel.
> > > > lambda$0(SubscriptionModel.java:72)
> > > > at org.eclipse.milo.opcua.sdk.server.util.SubscriptionModel$
> > > > $Lambda$1219/791525489.run(Unknown Source)
> > > > at org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$
> > > > PollAndExecute.run(ExecutionQueue.java:107)
> > > > at java.util.concurrent.Executors$RunnableAdapter.
> > > > call(Executors.java:511)
> > > > at java.util.concurrent.FutureTask.run(FutureTask. java:266)
> > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > ThreadPoolExecutor.java:1142)
> > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > ThreadPoolExecutor.java:617)
> > > > at java.lang.Thread.run(Thread.java:748)
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:38:03,743+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: Java heap space
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:38:37,462+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:39:30,428+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: Java heap space
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:39:55,693+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:40:59,601+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:41:20,030+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:41:32,946+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: Java heap space
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:41:37,620+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable > > > > 2017-12-07 23:42:28,122+JST �x�� org.eclipse.milo.opcua.stack.
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > >
> > > > > > 2017-12-07 23:42:37,978+JST �d��¥ io.netty.util.internal.> > > > core.util.ExecutionQueue$PollAndExecute run - Uncaught Throwable
> > during
> > > > execution.
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > ---
> > > > ============================================================
> > > > ==================
> > > > /*
> > > > * Copyright (c) 2016 Kevin Herron
> > > > *
> > > > * All rights reserved. This program and the accompanying materials
> > > > * are made available under the terms of the Eclipse Public License
> > v1.0
> > > > * and Eclipse Distribution License v1.0 which accompany this
> > distribution.
> > > > *
> > > > * The Eclipse Public License is available at
> > > > * http://www.eclipse.org/legal/epl-v10.html
> > > > * and the Eclipse Distribution License is available at
> > > > * http://www.eclipse.org/org/documents/edl-v10.html .
> > > > */
> > > >
> > > > package org.eclipse.milo.opcua.sdk.server.util;
> > > >
> > > > import java.math.RoundingMode;
> > > > import java.util.Collections;
> > > > import java.util.Iterator;
> > > > import java.util.List;
> > > > import java.util.Map;
> > > > import java.util.Set;
> > > > import java.util.concurrent.CompletableFuture;
> > > > import java.util.concurrent.ExecutorService;
> > > > import java.util.concurrent.ScheduledExecutorService;
> > > > import java.util.concurrent.TimeUnit;
> > > > import java.util.stream.Collectors;
> > > >
> > > > import org.eclipse.milo.opcua.sdk.server.DiagnosticsContext;
> > > > import org.eclipse.milo.opcua.sdk.server.OpcUaServer;
> > > > import org.eclipse.milo.opcua.sdk.server.api.AttributeManager;
> > > > import org.eclipse.milo.opcua.sdk.server.api.AttributeManager.
> > ReadContext;
> > > > import org.eclipse.milo.opcua.sdk.server.api.DataItem;
> > > > import org.eclipse.milo.opcua.sdk.server.api.MonitoredItem;
> > > > import org.eclipse.milo.opcua.stack.core.AttributeId;
> > > > import org.eclipse.milo.opcua.stack.core.types.builtin.DataValue;
> > > > import org.eclipse.milo.opcua.stack.core.types.builtin.unsigned.
> > UInteger;
> > > > import org.eclipse.milo.opcua.stack.core.types.enumerated.
> > > > TimestampsToReturn;
> > > > import org.eclipse.milo.opcua.stack.core.types.structured. ReadValueId;
> > > > import org.eclipse.milo.opcua.stack.core.util.ExecutionQueue;
> > > >
> > > > import com.google.common.collect.Lists;
> > > > import com.google.common.collect.Maps;
> > > > import com.google.common.math.DoubleMath;
> > > >
> > > > public class SubscriptionModel {
> > > >
> > > > private final Set<DataItem> itemSet = Collections.newSetFromMap(
> > > > Maps.newConcurrentMap());
> > > >
> > > > private final List<ScheduledUpdate> schedule =
> > > > Lists.newCopyOnWriteArrayList();
> > > >
> > > > private final ExecutorService executor;
> > > > private final ScheduledExecutorService scheduler;
> > > > private final ExecutionQueue executionQueue;
> > > >
> > > > private final OpcUaServer server;
> > > > private final AttributeManager attributeServices;
> > > >
> > > > public SubscriptionModel(OpcUaServer server, AttributeManager
> > > > attributeServices) {
> > > > this.server = server;
> > > >
> > > > this.attributeServices = attributeServices;
> > > >
> > > > executor = server.getExecutorService();
> > > > scheduler = server.getScheduledExecutorService();
> > > >
> > > > executionQueue = new ExecutionQueue(executor);
> > > > }
> > > >
> > > > public void onDataItemsCreated(List<DataItem> items) {
> > > > executionQueue.submit(() -> {
> > > > itemSet.addAll(items);
> > > > reschedule();
> > > > });
> > > > }
> > > >
> > > > public void onDataItemsModified(List<DataItem> items) {
> > > > executionQueue.submit(this::reschedule);
> > > > }
> > > >
> > > > public void onDataItemsDeleted(List<DataItem> items) {
> > > > executionQueue.submit(() -> {
> > > > itemSet.removeAll(items);
> > > > reschedule();
> > > > });
> > > > }
> > > >
> > > > public void onMonitoringModeChanged(List<MonitoredItem> items) {
> > > > executionQueue.submit(this::reschedule);
> > > > }
> > > >
> > > > private void reschedule() {
> > > > Map<Double, List<DataItem>> bySamplingInterval =
> > itemSet.stream()
> > > > .filter(DataItem::isSamplingEnabled)
> > > > .collect(Collectors.groupingBy(DataItem::
> > > > getSamplingInterval));
> > > >
> > > > List<ScheduledUpdate> updates = bySamplingInterval.keySet().
> > > > stream()
> > > > .map(samplingInterval -> {
> > > > List<DataItem> items = bySamplingInterval.get(
> > > > samplingInterval);
> > > >
> > > > return new ScheduledUpdate(samplingInterval, items);
> > > > })
> > > > .collect(Collectors.toList());
> > > >
> > > > schedule.forEach(ScheduledUpdate::cancel);
> > > > schedule.clear();
> > > > schedule.addAll(updates);
> > > > schedule.forEach(ScheduledUpdate::run);
> > > > }
> > > >
> > > > private class ScheduledUpdate implements Runnable {
> > > >
> > > > private volatile boolean cancelled = false;
> > > >
> > > > private final long samplingInterval;
> > > > private final List<DataItem> items;
> > > >
> > > > private long lastExecuteTimestamp = 0L;
> > > > private long scheduledLastExecuteTimestamp = 0L;
> > > >
> > > > private ScheduledUpdate(double samplingInterval, List<DataItem>
> > > > items) {
> > > > this.samplingInterval = DoubleMath.roundToLong(
> > samplingInterval,
> > > > RoundingMode.UP);
> > > > this.items = items;
> > > > }
> > > >
> > > > private void cancel() {
> > > > cancelled = true;
> > > > }
> > > >
> > > > @Override
> > > > public void run() {
> > > > List<PendingRead> pending = items.stream()
> > > > .map(item -> new PendingRead(item.getReadValueId()))
> > > > .collect(Collectors.toList());
> > > >
> > > > List<ReadValueId> ids = pending.stream()
> > > > .map(PendingRead::getInput)
> > > > .collect(Collectors.toList());
> > > >
> > > > CompletableFuture<List<DataValue>> future = new
> > > > CompletableFuture<>();
> > > >
> > > > ReadContext context = new ReadContext(
> > > > server, null, future, new DiagnosticsContext<>());
> > > >
> > > > future.thenAcceptAsync(values -> {
> > > > Iterator<DataItem> ii = items.iterator();
> > > > Iterator<DataValue> vi = values.iterator();
> > > >
> > > > while (ii.hasNext() && vi.hasNext()) {
> > > > DataItem item = ii.next();
> > > > DataValue value = vi.next();
> > > >
> > > > TimestampsToReturn timestamps =
> > > > item.getTimestampsToReturn();
> > > >
> > > > if (timestamps != null) {
> > > > UInteger attributeId = item.getReadValueId().
> > > > getAttributeId();
> > > >
> > > > value = (AttributeId.Value.isEqual(attributeId))
> > ?
> > > > DataValue.derivedValue(value, timestamps) :
> > > > DataValue.derivedNonValue(value,
> > timestamps);
> > > > }
> > > >
> > > > item.setValue(value);
> > > > }
> > > >
> > > > if (!cancelled) {
> > > > lastExecuteTimestamp =
> > System.currentTimeMillis();
> > > > if (scheduledLastExecuteTimestamp == 0) {
> > > > scheduledLastExecuteTimestamp =
> > > > lastExecuteTimestamp;
> > > > }
> > > > long nextSamplingInterval =
> > > > getNextSamplingInterval(samplingInterval, lastExecuteTimestamp,
> > > > scheduledLastExecuteTimestamp);
> > > > scheduler.schedule(this, nextSamplingInterval,
> > > > TimeUnit.MILLISECONDS);
> > > > scheduledLastExecuteTimestamp =
> > lastExecuteTimestamp +
> > > > nextSamplingInterval;
> > > > }
> > > > }, executor);
> > > >
> > > > executor.execute(() -> attributeServices.read(context, 0d,
> > > > TimestampsToReturn.Both, ids));
> > > > }
> > > >
> > > > private long getNextSamplingInterval(long samplingInterval,
> > long
> > > > lastExecuteTimestamp, long scheduledLastExecuteTimestamp) {
> > > > long diffFromScheduledTimestamp = lastExecuteTimestamp
> > -
> > > > scheduledLastExecuteTimestamp;
> > > > long rest = diffFromScheduledTimestamp %
> > samplingInterval;
> > > > return (samplingInterval - rest);
> > > > }
> > > > }
> > > >
> > > > }
> > > > ============================================================
> > > > ==================
> > > >
> > > > <kevinherron@xxxxxxxxx> wrote, Fri, 8 Dec 2017 04:59:17 -0800
> > > >
> > > > > Were there any other error messages in the logs when it happened or
> > just
> > > > > this?
> > > > >
> > > > > On Fri, Dec 8, 2017 at 12:10 AM, Shigeru Ishida <
> > > > ishida_shigeru@xxxxxxxxxxx>
> > > > > wrote:
> > > > >
> > > > > > Hi Kevin,
> > > > > >
> > > > > > About the simulation server developed by milo, when running with
> > java
> > > > vm
> > > > > > option
> > > > > > -Dio.netty.leakDetectionLevel=advanced, about 10 days passed and
> > Netty
> > > > > > LEAK
> > > > > > error message appeared.
> > > > > >
> > > > > > The main subscription/monitoring conditions are as follows.
> > > > > >
> > > > > > - Publishing interval 200 (msec)
> > > > > > - Sampling interval 100 (msec)
> > > > > > - NodeId count 350
> > > > > > - Number of concurrent clients 6
> > > > > >
> > > > > > It is running on Apache Felix 4.6.1 (OSGi), and the main
> > Milo/Netty you
> > > > > > are using
> > > > > > is as follows.
> > > > > >
> > > > > > - Milo at 2017.09.25.
> > > > > > - Netty 4.1.4
> > > > > >
> > > > > > Do you know what caused the problem?
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > --Shigeru
> > > > > >
> > > > > > ---
> > > > logging.Slf4JLogger
> > > > > > error - LEAK: ByteBuf.release() was not called before it's
> > > > > > garbage-collected. See http://netty.io/wiki/
> > > > reference-counted-objects.html
> > > > > > for more information.
> > > > > > WARNING: 543 leak records were discarded because the leak record
> > count
> > > > is
> > > > > > limited to 4. Use system property io.netty.leakDetection.maxRecords
> > to
> > > > > > increase the limit.
> > > > > > Recent access records: 5
> > > > > > #5:
> > > > > > io.netty.buffer.AdvancedLeakAwareByteBuf.
> > internalNioBuffer(
> > > > > > AdvancedLeakAwareByteBuf.java:735)
> > > > > > io.netty.buffer.CompositeByteBuf. internalNioBuffer(
> > > > > > CompositeByteBuf.java:1444)
> > > > > > io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:
> > 585)
> > > > > > io.netty.buffer.AbstractByteBuf.toString(
> > > > > > AbstractByteBuf.java:1213)
> > > > > > io.netty.buffer.SwappedByteBuf.toString(
> > > > SwappedByteBuf.java:992)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeString(BinaryDecoder.java:144)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeNodeId(BinaryDecoder.java:213)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > ReadValueId.decode(ReadValueId.java:88)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > MonitoredItemCreateRequest.decode( MonitoredItemCreateRequest.
> > java:80)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeArray(BinaryDecoder.java:453)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > CreateMonitoredItemsRequest.decode( CreateMonitoredItemsRequest.
> > > > java:94)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeMessage(BinaryDecoder.java:388)
> > > > > > org.eclipse.milo.opcua.stack.server.handlers.
> > > > > > UaTcpServerSymmetricHandler.lambda$1( UaTcpServerSymmetricHandler.
> > > > java:179)
> > > > > > org.eclipse.milo.opcua.stack.core.channel.
> > > > > > SerializationQueue.lambda$1(SerializationQueue.java:58)
> > > > > > org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$
> > > > > > PollAndExecute.run(ExecutionQueue.java:107)
> > > > > > java.util.concurrent.Executors$RunnableAdapter.
> > > > > > call(Executors.java:511)
> > > > > > java.util.concurrent.FutureTask.run(FutureTask. java:266)
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > > > ThreadPoolExecutor.java:1142)
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > > > ThreadPoolExecutor.java:617)
> > > > > > java.lang.Thread.run(Thread.java:748)
> > > > > > #4:
> > > > > > io.netty.buffer.AdvancedLeakAwareByteBuf. nioBufferCount(
> > > > > > AdvancedLeakAwareByteBuf.java:705)
> > > > > > io.netty.buffer.CompositeByteBuf. nioBufferCount(
> > > > > > CompositeByteBuf.java:1426)
> > > > > > io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:
> > 583)
> > > > > > io.netty.buffer.AbstractByteBuf.toString(
> > > > > > AbstractByteBuf.java:1213)
> > > > > > io.netty.buffer.SwappedByteBuf.toString(
> > > > SwappedByteBuf.java:992)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeString(BinaryDecoder.java:144)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeNodeId(BinaryDecoder.java:213)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > ReadValueId.decode(ReadValueId.java:88)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > MonitoredItemCreateRequest.decode( MonitoredItemCreateRequest.
> > java:80)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeArray(BinaryDecoder.java:453)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > CreateMonitoredItemsRequest.decode( CreateMonitoredItemsRequest.
> > > > java:94)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeMessage(BinaryDecoder.java:388)
> > > > > > org.eclipse.milo.opcua.stack.server.handlers.
> > > > > > UaTcpServerSymmetricHandler.lambda$1( UaTcpServerSymmetricHandler.
> > > > java:179)
> > > > > > org.eclipse.milo.opcua.stack.core.channel.
> > > > > > SerializationQueue.lambda$1(SerializationQueue.java:58)
> > > > > > org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$
> > > > > > PollAndExecute.run(ExecutionQueue.java:107)
> > > > > > java.util.concurrent.Executors$RunnableAdapter.
> > > > > > call(Executors.java:511)
> > > > > > java.util.concurrent.FutureTask.run(FutureTask. java:266)
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > > > ThreadPoolExecutor.java:1142)
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > > > ThreadPoolExecutor.java:617)
> > > > > > java.lang.Thread.run(Thread.java:748)
> > > > > > #3:
> > > > > > io.netty.buffer.AdvancedLeakAwareByteBuf. getInt(
> > > > > > AdvancedLeakAwareByteBuf.java:189)
> > > > > > io.netty.buffer.CompositeByteBuf._getInt(
> > > > > > CompositeByteBuf.java:802)
> > > > > > io.netty.buffer.AbstractByteBuf.readInt(
> > > > AbstractByteBuf.java:767)
> > > > > > io.netty.buffer.SwappedByteBuf.readInt(
> > > > SwappedByteBuf.java:581)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeInt32(BinaryDecoder.java:94)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeString(BinaryDecoder.java:134)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeNodeId(BinaryDecoder.java:213)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > ReadValueId.decode(ReadValueId.java:88)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > MonitoredItemCreateRequest.decode( MonitoredItemCreateRequest.
> > java:80)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeArray(BinaryDecoder.java:453)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > CreateMonitoredItemsRequest.decode( CreateMonitoredItemsRequest.
> > > > java:94)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeMessage(BinaryDecoder.java:388)
> > > > > > org.eclipse.milo.opcua.stack.server.handlers.
> > > > > > UaTcpServerSymmetricHandler.lambda$1( UaTcpServerSymmetricHandler.
> > > > java:179)
> > > > > > org.eclipse.milo.opcua.stack.core.channel.
> > > > > > SerializationQueue.lambda$1(SerializationQueue.java:58)
> > > > > > org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$
> > > > > > PollAndExecute.run(ExecutionQueue.java:107)
> > > > > > java.util.concurrent.Executors$RunnableAdapter.
> > > > > > call(Executors.java:511)
> > > > > > java.util.concurrent.FutureTask.run(FutureTask. java:266)
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > > > ThreadPoolExecutor.java:1142)
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > > > ThreadPoolExecutor.java:617)
> > > > > > java.lang.Thread.run(Thread.java:748)
> > > > > > #2:
> > > > > > io.netty.buffer.AdvancedLeakAwareByteBuf. getShort(
> > > > > > AdvancedLeakAwareByteBuf.java:165)
> > > > > > io.netty.buffer.CompositeByteBuf._getShort(
> > > > > > CompositeByteBuf.java:754)
> > > > > > io.netty.buffer.AbstractByteBuf.readShort(
> > > > > > AbstractByteBuf.java:707)
> > > > > > io.netty.buffer.SwappedByteBuf.readShort(
> > > > SwappedByteBuf.java:541)
> > > > > > io.netty.buffer.SwappedByteBuf. readUnsignedShort(
> > > > > > SwappedByteBuf.java:551)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeNodeId(BinaryDecoder.java:213)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > ReadValueId.decode(ReadValueId.java:88)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > MonitoredItemCreateRequest.decode( MonitoredItemCreateRequest.
> > java:80)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeArray(BinaryDecoder.java:453)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > CreateMonitoredItemsRequest.decode( CreateMonitoredItemsRequest.
> > > > java:94)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeMessage(BinaryDecoder.java:388)
> > > > > > org.eclipse.milo.opcua.stack.server.handlers.
> > > > > > UaTcpServerSymmetricHandler.lambda$1( UaTcpServerSymmetricHandler.
> > > > java:179)
> > > > > > org.eclipse.milo.opcua.stack.core.channel.
> > > > > > SerializationQueue.lambda$1(SerializationQueue.java:58)
> > > > > > org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$
> > > > > > PollAndExecute.run(ExecutionQueue.java:107)
> > > > > > java.util.concurrent.Executors$RunnableAdapter.
> > > > > > call(Executors.java:511)
> > > > > > java.util.concurrent.FutureTask.run(FutureTask. java:266)
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > > > ThreadPoolExecutor.java:1142)
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > > > ThreadPoolExecutor.java:617)
> > > > > > java.lang.Thread.run(Thread.java:748)
> > > > > > #1:
> > > > > > io.netty.buffer.AdvancedLeakAwareByteBuf. getByte(
> > > > > > AdvancedLeakAwareByteBuf.java:153)
> > > > > > io.netty.buffer.CompositeByteBuf._getByte(
> > > > > > CompositeByteBuf.java:747)
> > > > > > io.netty.buffer.AbstractByteBuf.readByte(
> > > > AbstractByteBuf.java:689)
> > > > > > io.netty.buffer.SwappedByteBuf.readByte(
> > > > SwappedByteBuf.java:531)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeNodeId(BinaryDecoder.java:199)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > ReadValueId.decode(ReadValueId.java:88)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)
> > > > > > org.eclipse.milo.opcua.stack.core.types.structured.
> > > > > > MonitoredItemCreateRequest.decode( MonitoredItemCreateRequest.
> > java:80)
> > > > > > org.eclipse.milo.opcua.stack.core.serialization.binary.
> > > > > > BinaryDecoder.decodeSerializable( BinaryDecoder.java:406)