<BEA-000802><ExecuteRequest failed
java.lang.RuntimeException: MaxThreads constraint ‘weblogic.unicast.ForwardingWorkManager’ queue for workManager ‘weblogic.unicast.ForwardingWorkManager’ exceeded maximum capacity of ‘8192’ elements. Max threads constraint count is set to 1.
This can be addressed by switching to Unicast. Its fixed in 12.1.3 and there is patch in previous versions – 16909588.
<BEA-000113><Removing xxx from cluster view due to PeerGone.>
Enable the following debug and raise a ticket with Oracle Support.
-Dweblogic.debug.DebugCluster=true
-Dweblogic.debug.DebugClusterAnnouncements=true
-Dweblogic.debug.DebugReplication=true
-Dweblogic.debug.DebugReplicationDetails=true
-Dweblogic.debug.DebugUnicastMessaging=true
-Dweblogic.debug.DebugClusterFragments=true
-Dweblogic.debug.DebugClusterHeartbeats=true
-Dweblogic.debug.DebugHttpSessions=true
The debug can able be enabled using weblogic admin utility.
java weblogic.Admin -url t3://localhost:7001 – username weblogic -password weblogic SET -type ServerDebug -property DebugCluster true
You can set the value is false if you want to turn off the debug.
<BEA-003108><Unicast receive error : java.io.EOFException
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readLong(DataInputStream.java:416)
at java.io.ObjectInputStream$BlockDataInputStream.readLong(ObjectInputStream.java:3206)
at java.io.ObjectInputStream.readLong(ObjectInputStream.java:1043)
at weblogic.cluster.HeartbeatMessage.readExternal(HeartbeatMessage.java:55)
This error generally comes if debug has not been appplied on ALL JVM’s.
<BEA-002939><The maximum thread constraint ClusterMessaging has been reached 17 times for the last 195 seconds.>
We see below thread dump in the logs
“weblogic.cluster.MessageReceiver” #200 daemon prio=5 os_prio=0 tid= nid= in Object.wait() [TID]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at weblogic.cluster.messaging.internal.server.UnicastFragmentSocket.receive(UnicastFragmentSocket.java:107)
- locked (a java.util.ArrayList)
at weblogic.cluster.FragmentSocketWrapper.receive(FragmentSocketWrapper.java:91)
at weblogic.cluster.ClusterMessagesManager.run(ClusterMessagesManager.java:496)
at java.lang.Thread.run(Thread.java:748)
This happens when we use cluster broadcast.
We need to apply Patch 29591900
<BEA-000442><The connection attempt was rejected because the incoming protocol is not enabled on channel “xxxxx”.>
You can try using the default unicast channel instead of custom network channel. Also ensure you have the necessary patches.
<BEA-003107><Lost 2 unicast message(s).>
You can consider enabling message ordering.
<BEA-000112><Removing xxx jvmid:-1351871841684449850S:xxx:[-1,-1,3172,3172,-1,-1,-1]:xxx:xxxxxxx from cluster view due to timeout.>
You can try enabling message ordering as suggested above.