WebLogic Clustering Issues

<BEA-000802><ExecuteRequest failed
java.lang.RuntimeException: MaxThreads constraint ‘weblogic.unicast.ForwardingWorkManager’ queue for workManager ‘weblogic.unicast.ForwardingWorkManager’ exceeded maximum capacity of ‘8192’ elements. Max threads constraint count is set to 1.

This can be addressed by switching to Unicast. Its fixed in 12.1.3 and there is patch in previous versions – 16909588.

<BEA-000113><Removing xxx from cluster view due to PeerGone.>

Enable the following debug and raise a ticket with Oracle Support.

-Dweblogic.debug.DebugCluster=true
-Dweblogic.debug.DebugClusterAnnouncements=true
-Dweblogic.debug.DebugReplication=true
-Dweblogic.debug.DebugReplicationDetails=true
-Dweblogic.debug.DebugUnicastMessaging=true
-Dweblogic.debug.DebugClusterFragments=true
-Dweblogic.debug.DebugClusterHeartbeats=true
-Dweblogic.debug.DebugHttpSessions=true

<BEA-003108><Unicast receive error : java.io.EOFException
java.io.EOFException

at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readLong(DataInputStream.java:416)
at java.io.ObjectInputStream$BlockDataInputStream.readLong(ObjectInputStream.java:3206)
at java.io.ObjectInputStream.readLong(ObjectInputStream.java:1043)
at weblogic.cluster.HeartbeatMessage.readExternal(HeartbeatMessage.java:55)

This error generally comes if debug has not been appplied on ALL JVM’s.

<BEA-002939><The maximum thread constraint ClusterMessaging has been reached 17 times for the last 195 seconds.>

We see below thread dump in the logs

“weblogic.cluster.MessageReceiver” #200 daemon prio=5 os_prio=0 tid= nid= in Object.wait() [TID]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at weblogic.cluster.messaging.internal.server.UnicastFragmentSocket.receive(UnicastFragmentSocket.java:107)

  • locked (a java.util.ArrayList)
    at weblogic.cluster.FragmentSocketWrapper.receive(FragmentSocketWrapper.java:91)
    at weblogic.cluster.ClusterMessagesManager.run(ClusterMessagesManager.java:496)
    at java.lang.Thread.run(Thread.java:748)

This happens when we use cluster broadcast.

We need to apply Patch 29591900

<BEA-000442><The connection attempt was rejected because the incoming protocol is not enabled on channel “xxxxx”.>

You can try using the default unicast channel instead of custom network channel. Also ensure you have the necessary patches.

<BEA-003107><Lost 2 unicast message(s).>

You can consider enabling message ordering.

<BEA-000112><Removing xxx jvmid:-1351871841684449850S:xxx:[-1,-1,3172,3172,-1,-1,-1]:xxx:xxxxxxx from cluster view due to timeout.>

You can try enabling message ordering as suggested above.

Leave a Reply