When there is a distributed transaction involved in message processing, it is very important that all the resources involved in the transaction are reachable without any latency. When this is not the case and there is an issue with the communication between the resources, there can be issues of message processing which might result in transaction timeouts, rollbacks and other runtime errors.
Take a scenario of the below architecture:
A SLSB client/Application sends a message of 2MB size to weblogic Queue, that message is received by the MDB deployed on the weblogic server. That message is then processed, retrieves some information from databases. After the processing is complete, the messages is then displayed by the MDB.
First of all we need to identify where is the message size causing an issue. Is it from the queue end, message bridge end (if used), or is it the MDB that is not able to consume the messages. Enable the debug flags as required.
When these messages are large in size and out of the size limit of the queue or the protocol by which the message is being transferred, below errors can occur:
1) weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: ‘10000080’ bytes exceeds the configured maximum of: ‘10000000’ bytes for protocol : ‘t3’.
Solution : Try to increase the message size from the admin console for the protocol by:
– Queue-> Configuration-> Threshholds and Quotas-> Maximum Message Size
– MAX message size in Server-> Admin/managed server-> Protocals-> General
– Setting the system property -Dweblogic.MaxMessageSize=20000000
Note : The message size would vary as per the application requirement.
2) <BEA-489003> <Caught Exception: com.bea.wli.variables.XmlObjectRuntimeException: Error Creating the XmlObject
com.bea.wli.variables.XmlObjectRuntimeException: Error Creating the XmlObject
at com.bea.wli.variables.ProcessXML.underlyingXmlObject(ProcessXML.java:759)
at com.bea.xml.FilterXmlObject.xmlText(FilterXmlObject.java:164)
at com.bea.wli.broker.Message.toString(Message.java:264)
– This is seen when the message processing includes retrieving some data from the database using SQL Queries but the messages size exceeds the maximum size of a document stored in the SQL Document Store.
Solution: – Increase the values of weblogic.wli.DocumentMaxInlineSize and weblogic.wli.DocumentMaxInMemorySize in wli-config.properties.
– Increase the WLI transaction timeout in the wlw-config.xml.
– Set the parameters XASetTransactionTimeout=”true” and XATransactionTimeout=”xxx” as perferred.
3)java.io.IOException: Could not connect to SQL Document Store
at com.bea.wli.variables.ProcessXML.getLoaderStream(ProcessXML.java:302)
at com.bea.wli.variables.ProcessXML.getTokenStreamData(ProcessXML.java:338)
at com.bea.wli.variables.XmlObjectVariableFactory.getXmlObject(XmlObjectVariableFactory.java:359)
Caused by: com.bea.wli.store.DocumentStoreException: Could not connect to SQL Document Store
at com.bea.wli.store.impl.SQLStore$SQLStoreConnection.<init>(SQLStore.java:3162)
– Check if there is a network issue and the SQL store is available. Take 5-6 thread dumps at the interval of 10 seconds each to check if there are any stuck threads while connecting to the SQL store.
4) java.sql.SQLException: Unexpected exception while enlisting XAConnection java.sql.SQLException: Transaction rolled
back: Transaction timed out after 600 seconds
– Now if the message size of 2mb is being processed successfully and if you are facing issue with a larger message say 3mb, you might see the above transaction timeout error in the log files. This means issue is with the message processing taking a long time.
– If the whole sending and receiving of messages from application is one single transaction, check where is the latency. Is the communication between all the resources happening properly or is there a network issue. Check if all the information is being retrieved from the database on time or is it taking long.
If you find any other issue or reason for the JMS transaction rollbacks or delay in message processing, feel free to comment and let us know about it, we will take a look and try to resolve the issue.
Best Regards.
Hi Divya,
thanks for sharing your valuable information 🙂
Recently i have encountered the first scenario which you have mentioned here above.
My question is what could be the common causes when we face below error in admin server logs and its causing admin server to be down impacting not able to access admin console.Is that because one of the queue which is configured is trying to deliver large message and its exceeded of max accepted volume of the queue size or there might be some other reason i can suspect.
Error : 1st scenario.
Hi Satish,
If the aggregate size of the messages pushed to a consumer is larger than the current protocol’s maximum message size (default size is 10 MB and is configured on a per WebLogic Server instance basis using the console and on a per client basis using the Dweblogic.MaxMessageSize command line property), the message delivery fails.
You can try setting the parameter -Dweblogic.MaxMessageSize in the server startup script and restart the server.
Cheers,
Anandraj
Hi,
I haven’t seen errors like the above but have had issues with Base64 streams extracted from a database clob where the stream is cropped before the end when viewing the payload in the weblogic server, but only if the files are greater than a certain size, however it can manage to get numerous files in a single payload. Could this be the same issue as the above?