Monthly Archive: December 2010

Different Out Of Memory Issues

* Exception in thread “CompilerThread1” java.lang.OutOfMemoryError: requested 793020 bytes for Chunk::new. Out of swap space?

Out of memory while reading in symbol table of /apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl
( 0)  0xc8461230     [/apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 1)  0xc80a5fec     [/apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 2)  0xc7f00420     [/apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 3)  0xc7f00ca0     [/apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 4)  0xc8368d08     [/apps/bea/weblogic92/jdk1.5.0.18/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 5)  0xc005b2e4   __pthread_body + 0x44  [/usr/lib/libpthread.1]
( 6)  0xc0065574   __pthread_start + 0x14  [/usr/lib/libpthread.1]
Java out of memory messages are marked with pid: 13828 in /var/adm/syslog/syslog.log.

Possible causes:
– not enough swap space left, or
– kernel parameter MAXDSIZ is very small.

Solution:
Although it appears that an OutOfMemoryError is thrown this apparent exception is reported by the HotSpot VM code when an allocation from the native heap failed and the native heap may be close to exhaustion. The message indicates the size (in bytes) of the request that failed and also indicates what the memory is required for. In some cases the reason will be shown but in most cases the reason will be the name of a source module reporting the allocation failure. If an OutOfMemoryError with this error is thrown it may require using utilities on the operating system to diagnose the issue further. Examples of issues that may not be related to the application are when the operating system is configured with insufficient swap space, or when there is another process on the system that is consuming all memory resources. If neither of these issues is the cause then it is possible that the application is failed due to native leak; for example, application or library code is continuously allocating memory but is not releasing it to the operating system.
For more information: http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf

The recommendation for swap space size in the Solaris is that swap should be configured about 30% of physical RAM.

The following link has suggested a workaround to add ‘-XX:+UseDefaultStackSize -Xss256K’ parameter.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4916142

Add: -XX:CodeCacheMinimumFreeSpace=2M -XX:+ReservedCodeCacheSize=64M -XX:PermSize=128m -XX:MaxPermSize=384m (As per your other JVM settings)

Sun team needs to be involved for this issue. Usually such issues are solved by sun support.

=============================

* java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:574)
at weblogic.work.RequestManager.createThreadAndExecute(RequestManager.java:271)
at weblogic.work.RequestManager.executeIt(RequestManager.java:245)
at weblogic.work.ServerWorkManagerImpl.schedule(ServerWorkManagerImpl.java:142)

Solution:
1) Set kernel parameter maxdsiz to a higher value
2) Reduce the current heap size.
3) Check the kernel values:  ulimit -a
4) If the NPROC soft limit is lower than the hard limit, increase it as needed: ulimit -u <new value>.
Check the Operating System documentation to make changes permanent at the OS configuration files.
5) Need to reduce the JVM stack size and the OS stack size both. Set the -xss in java options and set ulimit -s on the OS level.

=============================

* java.lang.OutOfMemoryError: PermGen space
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.getConstructor(Unknown Source)

Solution: Increase the max permgen space -XX:MaxPermSize=256m

There can be a leak in the permgen objects. If tuning parameters do not resolve the issue, we need to use the memory leak detector tools and find out which instances in the permgen space are not getting cleared.

=============================

* java.lang.OutOfMemoryError: allocLargeObjectOrArray – Object size: 372032, Num elements: 372012
* java.lang.outofmemoryerror: nativeGetNewTLA

Solution: -XXtlasize:128k -XXlargeobjectlimit:128k

If this does not solve the issue, we need to check in the application code for the large objects being created and not being destroyed. Take JRA Recording (Oracle JRockit) or use JConsole and memory leak detector tools (JMAP, JHAT) for analysis on the

=============================

* java.lang.OutOfMemoryError: Java Heap Space

Solution: First thing that needs to be checked is the gc logs. Need to check whether the garbage collection is happening properly. If the heap keeps gradually increasing even after full gc, tune the gc algorithms and check if the behavior is the same. Use memory leak detector tools for both sun jdk and JRockit to check which instances from the application are not getting destroyed.

If this is not the case and the application genuinely needs more memory, increase the heap size by using the parameters:
Example: -Xms2048m -Xmx2048m

=============================

* java.lang.StackOverflowError

Solution: Stack over flow error is usually generated due to a recursive call made by the application (infinite recursion), or its because of an attempt to allocate more memory on the stack than will fit. This is usually the result of creating local array variables that are far too large for the current stack.

For the first possibility, we need to check the application code as to where is the recursive call being made.
For the second possibility, we can increase the JVM stack size by the parameter : Example: -Xss512K

=============================

java.lang.OutOfMemoryError: GC overhead limit exceeded.
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:3209) 

Try adding this jvm option
-XX:-UseGCOverheadLimit

 

Integrate YourKit Java Profiler with a standalone Weblogic Server

YourKit Java Profiler is a CPU and memory profiler that makes it easy to solve wide range of CPU- and memory-related performance problems. It features automatic leak detection, powerful tools for the analysis of memory distribution, an object heap browser, comprehensive memory tests as part of your JUnit testing process, extremely low profiling overhead, transparent deobfuscation support, and integration with Eclipse, JBuilder, IntelliJ IDEA, NetBeans, and JDeveloper IDEs
Ref: – http://freshmeat.net/projects/yjp/
Download the trail version from here
http://www.yourkit.com/download/index.jsp

Steps to integrate it with Weblogic Server

Integrate with standalone J2EE Server

Select Weblogic 9/10

Browse to the location where the startup script is.

Select the startup script; this will create the startup script for the Profiler

Start your server using that script
C:\bea103\user_projects\domains\First_Domain\bin\startWebLogic_with_yjp

You will see it in the running JVM’s list. Click on it.

You should be able to see the Memory usage stats

Garbage collection activity

Class Loading activity

I am yet to explore the tool and see how I can use it for the maximum benefit.
Looks to me it combines the feature of gcviewer, eclipse memory analyzer and JProbe.
“Good to have” tool in a support engineers armory! 😉

Creating Datasource using WLST

connect(‘weblogic’,’weblogic’,’t3://localhost:7001′)
edit()
startEdit()
cd(‘/’)
cmo.createJDBCSystemResource(‘PegaRULES’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES’)
cmo.setName(‘PegaRULES’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDataSourceParams/PegaRULES’)
set(‘JNDINames’,jarray.array([String(‘jdbc/PegaRULES’)], String))
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDriverParams/PegaRULES’)
cmo.setUrl(‘jdbc:sqlserver://localhost:1433’)
cmo.setDriverName(‘com.microsoft.sqlserver.jdbc.SQLServerDriver’)
cmo.setPassword(‘pega612’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCConnectionPoolParams/PegaRULES’)
cmo.setTestTableName(‘SQL SELECT 1\r\n\r\n’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDriverParams/PegaRULES/Properties/PegaRULES’)
cmo.createProperty(‘user’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDriverParams/PegaRULES/Properties/PegaRULES/Properties/user’)
cmo.setValue(‘pega612’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDriverParams/PegaRULES/Properties/PegaRULES’)
cmo.createProperty(‘databaseName’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDriverParams/PegaRULES/Properties/PegaRULES/Properties/databaseName’)
cmo.setValue(‘prpc612’)
cd(‘/JDBCSystemResources/PegaRULES/JDBCResource/PegaRULES/JDBCDataSourceParams/PegaRULES’)
cmo.setGlobalTransactionsProtocol(‘OnePhaseCommit’)
cd(‘/SystemResources/PegaRULES’)
set(‘Targets’,jarray.array([ObjectName(‘com.bea:Name=AdminServer,Type=Server’)], ObjectName))
activate()
dumpStack()
exit()

JBoss Command Line Utility Twiddle

Recently I had faced a situation where I was forced to invoke a method on a particular “MBean” 10-15 times. I had completed this task manually from “JMX-Console”, but I didn’t feel it was elegant way of executing the task.

So I had started doing research to find out if there is better way of doing this task. Finally I found a utility called as “twiddle” which basically comes with every JBoss installation. In this article, I am just sharing information about what I had learned. There are many more complex tasks which can be performed with this utility.


Variables
:-

JBOSS_HOME :- This is the location where JBoss is installed.

Prerequisites :-

We need to set “JAVA_HOME” variable before executing twiddle. This can be done in two ways.
Solution 1:-

In the Profile :- In “bash profile”(I use ‘bash’ profile, same can be done with other profiles), so go to “HOME” directory( enter “cd” at command prompt), open “.bash_profile(make sure there is “dot” before “bash_profile”, this is secret file) and enter below lines.

JAVA_HOME=<location where “java” is installed>
export JAVA_HOME

Save the file and enter “source .bash_profile”. “source” command reads commands from the file and executes in current shell environment.

Solution 2 :-

Update “twiddle.sh”(Take backup before editing the file).
Add below lines

JAVA_HOME=<location where “java” is installed>
export JAVA_HOME
echo “VAS JAVA_HOME : ${JAVA_HOME}”

Note :- Solution “2” is better than “1” because solution 1 might effect other applications too, but solution “2” will effect only “twiddle.sh”

Validation :-

Enter “echo ${JAVA_HOME}” , this should return the location where the JBoss is installed. If this variable returns “empty” value, then this variable is not set properly, please check above steps.


Execution
:-

Go to the directory “JBOSS_HOME/bin” and execute “./twiddle.sh”. It should display the help options.

Help :-

./twiddle.sh –help-commands :- displays all the commands we can invoke on this utility.
./twiddle.sh –H<command> :- will provide additional information on particular command, such as syntax of the commands, options that can be passed to the command.

Example :-
./twiddle.sh -Hinvoke

Syntax :-

./twiddle.sh -s <serverUrl> -u <userName> -p <password> <command> [ command_arguments

serverUrl :- <protocol>://<serverName>:<port>
protocol :- jnp ( This is optional)
serverName :- Since we will be executing the command from the box where JBoss is installed, we can either give “localhost” or actual server name.
port :- 1099.  This the port where Jboss Naming service is listening. If you are using out of box Jboss installation then this will be the port. But if you changed this and you don’t remember the port, then either check the file “JBOSS_HOME/server/<serverName>/conf/bindingservice.beans/META-INF/bindings-jboss-beans.xml” or log on to “http://<serverName>:<port>/jmx-console” and click on “jboss” on left column and then click “service=Naming” and the value for the attribute “port”.

Commands :-

jsr77 :- displays jsr77 related information.
xmbean :- print out mbean metadata as an “xmbean” descriptor. In this file you can find all attributes and operations on a particular mbean.
info :- get the metdata for an MBean.
get :-  get value for an mbean attribute.
invoke :- invoke an operation on an MBean
create :- create an MBean
setattrs :- set the values of one or more MBean attributes
unregister :- unregister one or more MBeans
queryMethod :- query list of matchin methods on MBeans
listDomains :-  list all the domains.
query :-  query list of matching MBeans
set :- set the value for one MBean attribute
serverinfo :- get the information about MBean Server.


Examples :-

jsr77 :-

Displays the “jsr77” information for this particular server.

./twiddle.sh -s jnp://localhost:1099 -u admin -p admin jsr77

xmbean :-

Displays information of the MBean as “xmbean” descriptor.

./twiddle.sh -s jnp://localhost:1099 -u admin -p admin xmbean jboss:service=Naming

info :-

Query attribute of a particular bean.

./twiddle.sh -s jnp://localhost:1099 -u admin -p admin xmbean jboss:service=Naming

get :-

Get value of a particular ‘attribute’.

./twiddle.sh -s localhost:1099 -u admin -p admin get ‘jboss:service=Naming’ Name

Get value of a multiple ‘attributes’. The attributes must be separated by ‘Space’.

./twiddle.sh -s localhost:1099 -u admin -p admin get ‘jboss:service=Naming’ Name State

invoke :-

Invoke operation on a particular mbean.

./twiddle.sh -s localhost:1099 -u admin -p admin invoke jboss.system:type=ServerInfo listThreadCpuUtilization

Note :- There are multiple additional options, I didn’t explore them as of now.

create :-

Create an MBean.

I didn’t explore this command, I need to do research on this command a little more. I will update this article once I found more on this command.

setattrs :-

Set values of one or more bean attributes.

In below command we set values of two attributes “BackLog” and “JNPServerSocketFactoryBean” to “100” and “null” respectively.

./twiddle.sh -s localhost:1099 -u admin -p admin setattrs ‘jboss:service=Naming’ Backlog 100 JNPServerSocketFactory null

unregister :-

Unregister a one or more Mbeans.

Again I didn’t get much chance to explore this command, will update on this on this command once I am done with research.

queryMethod :-

Query all methods on all Mbeans.

./twiddle.sh -s localhost:1099 -u admin -p admin queryMethod list

Query methods on few mbeans

./twiddle.sh -s localhost:1099 -u admin -p admin queryMethod -f ‘jboss:*’ list

listDomains :-

List all the domains.

./twiddle.sh -s localhost:1099 -u admin -p admin listDomains

query :-

Display all Mbeans on the server.

./twiddle.sh -s localhost:1099 -u admin -p admin query ‘*:*’

Display MBeans on particular domain. In below command we display in the domain “jboss:j2ee”.

./twiddle.sh -s localhost:1099 -u admin -p admin query ‘jboss.j2ee:*’

set :-

Set the value on particular attribute.

./twiddle.sh -s localhost:1099 -u admin -p admin set ‘jboss.system:service=Logging,type=Log4jService’ RefreshPeriod 60

serverinfo :-

Display information about MBean server.

./twiddle.sh -s localhost:1099 -u admin -p admin serverinfo -l

Number of MBeans on the server.

./twiddle.sh -s localhost:1099 -u admin -p admin serverinfo -c

Default Domain.

./twiddle.sh -s localhost:1099 -u admin -p admin serverinfo -d

Few Important Useful Operations :- We can do many operations with “Twiddle”, below are just few examples.

Deploying an Application :-

./twiddle.sh -s localhost:1099 -u admin -p admin invoke ‘jboss.service:MainDeployer’ deploy “file:///tmp/VASVijay/VASApp.war”

Undeploying an Application :-

./twiddle.sh -s localhost:1099 -u admin -p admin invoke ‘jboss.service:MainDeployer’ undeploy “file:///tmp/VASVijay/VASApp.war”

Redeploying an Application :-

./twiddle.sh -s localhost:1099 -u admin -p admin invoke ‘jboss.service:MainDeployer’ redeploy “file:///tmp/VASVijay/VASApp.war”

Take ThreadDump :-

./twiddle.sh -s localhost:1099 -u admin -p admin invoke ‘jboss.system:type=ServerInfo’ listThreadDump
Get Server Info :-

./twiddle.sh -s localhost:1099 -u admin -p admin get ‘jboss.system:type=ServerInfo’

Issues :-

Twiddle will be throwing exception like below, this is a know nexception. This is because “twiddle” is executed on separate “JVM” from the “JBoss JVM”. There is patch for this at “https://issues.jboss.org/browse/JBAS-4323”

10:16:57,158 ERROR [Twiddle] Exec failed java.io.NotSerializableException: org.jboss.resource.connectionmanager.TxConnectionManager
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at java.util.ArrayList.writeObject(ArrayList.java:570)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461)

Pending Tasks :-

Exploring below commands and options :-

Options :- “-c” option. I was not able to find out correct way to execute this option, so I am checking out with Redhat on this option. If any one of you know how to use this option, please let me know. Thanks in advance for sharing the solution.

Commands :- “create” and “unregister” commands,  I need to write code to explore these. I will soon add examples to execute these commands, once I test these commands successfully.

References :-

This article wouldn’t had been possible without ‘google’ and many wonderful articles written by experts. Few of them are below. I referenced many sites for this utility, I couldn’t remember all site names, If I didn’t mention any particular site name whose example I had used,  its just that I don’t remember the site name, I sincerely apologize for not mentioning the site name.  Please let know if any one feels I had missed to add any particular site in the reference, I will add the site to the references.

http://community.jboss.org/wiki/Twiddle

http://nagpals.com/jboss-examples-of-twiddle

I hope that you all liked this article, if there any issues or suggestions or if you don’t like anything in the article,  please comment.

For some more Twiddle examples, refer the below post.

http://weblogic-wonders.com/weblogic/2011/02/13/twiddle-utility-examples/

Thanks,

Wonders-Team,

Dream, Learn, Share and Inspire !

WebLogic SNMP to monitor JMS Server

Configuring WebLogic SNMP to monitor JMS Server attributes using Counter Monitor.

SNMP: It is basically Internet Protocol used to manage the devices over the IP based Network. The devices could be servers ,routers, printers, switches etc.

WebLogic Server software includes the ability to communicate with enterprise-wide management systems using Simple Network Management Protocol (SNMP). The WebLogic Server SNMP capability enables you to integrate management of WebLogic Servers into an SNMP-compliant management system. Such a system provides a single view of the various software and hardware resources of a complex, distributed system.

The below article demonstrates of how WebLogic SNMP can be used to monitor the JMSServerRuntime Mbean attributes using Counter Monitor.

In the below example , we would check whether the  ‘MessagesCurrentCount‘ of the JMS Server exceeds the specified threshold and generate trap notifications accordingly.

Before we configure the SNMP Counter Monitor, make sure the JMS Server and Queue is properly configured.

Refer the below article to configure the JMS Queue.

http://weblogic-wonders.com/weblogic/2010/11/30/weblogic-jms-feature-using-a-queue/

The steps to configure Counter Monitor are as below.

1. Creating SNMP agent.

2. Creating SNMP Counter Monitor

3. Creating SNMP Trap Destination.

4. Verifying the SNMP Alerts.

1. Creating SNMP Agent:

a. Login into the Administration Console, navigate to Diagnostics → SNMP.

Click on New tab to create a Server SNMP Agent

b. Click on the newly created SNMP Agent → Enable the SNMP agent and you can modify the default SNMP agent attributes like SNMP ports etc.

c.Target the SNMP Agent to the AdminServer .

2. Create SNMP Counter Monitor:

Counter Monitor:

A counter monitor observes attribute values that are returned as an Integer object type.

You can specify that a trap is generated if an attribute is beyond the bounds of a threshold value. You can also specify that if a value exceeds a threshold, the monitor increases the threshold by an offset value. Each time the observed attribute exceeds the new threshold, the threshold is increased by the offset value, up to a maximum allowable threshold that you specify.

There are other different monitors that can be set along side Counter Monitor.

String Monitor: A string monitor observes changes in attributes that are expressed as String Objects.

For further reference on the same refer the below post.

http://weblogic-wonders.com/weblogic/2010/12/17/weblogic-snmp-for-string-monitor/

Gauge Monitor: A Gauge Monitor indicates whether  an attribute is operating within a specific range.

For further reference on the same refer the below post.

http://weblogic-wonders.com/weblogic/2010/12/19/weblogic-snmp-to-monitor-jdbc-connection-pool/

a.Click on Counter Monitor tab from the console.

b. From the drop down list of Mbeans, select  ‘JMSServerRuntime‘ Mbean

c. Select the MessagesCurrentCount‘ attribute from the drop down list.

d. Enter the threshold value for MessagesCurrentCount attribute exceeding which will generate SNMP Traps.

e. Click on the newly created Counter Monitor and specify which JMSServer Mbean you want to monitor (JMSServer-0), polling interval of the MBeans.

f. Target the created Counter Monitor to the AdminServer.

3. Creating SNMP Trap Destination:

a. Click SNMP Agent –>  Trap Destinations –>  New.

b. Specify the Trap Destination attributes like Host and Ports.

4. Verify the SNMP Traps:

To verify that a WebLogic Server SNMP Agent is generating and sending notifications, we can use the WebLogic command line utility SnmpTrapMonitor.

a. Open a command prompt and set the classpath to the weblogic.jar file under the %WLS_HOME%/lib folder.

set classpath=%classpath%;%WLS_HOME%\server\lib\weblogic.jar

b. Run the weblogic command line utility weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor to receive notifications.

Make sure that you specify the same listen port for this Trap Monitor as the trap destination that you created for the SNMP Agent.

For example, if your trap destination specifies a listen port of 171, enter the following command: java

java weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor -p 171

The SNMP setup is complete. The config.xml file would like below.

**********************************************************************

 

<snmp-counter-monitor>

<name>SNMPCounterMonitor-0</name>

<enabled-server>AdminServer</enabled-server>

<monitored-m-bean-type>JMSServerRuntime</monitored-m-bean-type>

<monitored-m-bean-name>JMSServer-0</monitored-m-bean-name>

<monitored-attribute-name>MessagesCurrentCount</monitored-attribute-name>

<polling-interval>10</polling-interval>

<threshold>10</threshold>

<offset>0</offset>

<modulus>0</modulus>

</snmp-counter-monitor>

 

**********************************************************************

If the MessagesCurrentCount value exceeds the threshold value of the JMSServer would generate Traps on to the listener as below.

From the command line utility:

**********************************************************************

java weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor -p 171

— Snmp Trap Received —
Version        : v1
Source         : UdpEntity:127.0.0.1:161
Community      : public
Enterprise     : enterprises.140.625
TrapOID        : enterprises.140.625.0.75
RawTrapOID     : 1.3.6.1.4.1.140.625.0.75
Trap Objects   : {
{ enterprises.140.625.100.5=Mon Dec 20 15:42:45 IST 2010 }
{ enterprises.140.625.100.10=AdminServer }
{ enterprises.140.625.100.55=jmx.monitor.counter.threshold }
{ enterprises.140.625.100.60=10 }
{ enterprises.140.625.100.65=11 }
{ enterprises.140.625.100.70=com.bea:Location=AdminServer,Name=JMSServer-0,ServerRuntime=AdminServer,Type=JMSServerRuntime }
{ enterprises.140.625.100.75=JMSServerRuntime }
{ enterprises.140.625.100.80=MessagesCurrentCount }
}
**********************************************************************

From the Admin  Server console or server logs.

<Notice> <SNMP> <BEA-320931> <The SNMP trap version is 1>

For further references:

http://download.oracle.com/docs/cd/E13222_01/wls/docs103/snmpman/index.html

Best Regards,

Wonders Team. 🙂

WebLogic SNMP to monitor JDBC Connection Pool

Configuring WebLogic SNMP to monitor JBDC Connection Pool attributes using Gauge Monitor.

SNMP: It is basically Internet Protocol used to manage the devices over the IP based Network. The devices could be servers ,routers, printers, switches etc.

WebLogic Server software includes the ability to communicate with enterprise-wide management systems using Simple Network Management Protocol (SNMP). The WebLogic Server SNMP capability enables you to integrate management of WebLogic Servers into an SNMP-compliant management system. Such a system provides a single view of the various software and hardware resources of a complex, distributed system.

The below article demonstrates of how to configure a simple SNMP Gauge monitor to monitor the JDBC Connection pool attribute “ActiveConnectionsHighCount”  value within a specified range and generate the SNMP traps accordingly.

For example, we can check whether the  ActiveConnectionsHighCount of the Connection Pool operates within a range of Max and Min threshold values and generate trap notifications accordingly.

The steps are as below.

1. Creating SNMP agent.

2. Creating SNMP Gauge Monitor

3. Creating SNMP Trap Destination.

4. Verifying the SNMP Alerts.

1. Creating SNMP Agent:

a. Login into the Administration Console, navigate to Diagnostics → SNMP.

Click on New tab to create a Server SNMP Agent

b. Click on the newly created SNMP Agent → Enable the SNMP agent and you can modify the default SNMP agent attributes like SNMP ports etc.

c.Target the SNMP Agent to the Managed Server MS1.

2. Creating SNMP Gauge Monitor:

Gauge Monitor:

A gauge monitor observes changes in MBean attributes that are expressed as integers or floating-point.

You can specify that a notification is generated if an attribute is beyond the bounds of a high or low threshold value.

There are other different Monitors that can be set along side Gauge Monitor.

String Monitor:  A string monitor observes changes in attributes that are expressed as String Objects.

For further reference on the same refer the below post.

http://weblogic-wonders.com/weblogic/2010/12/17/weblogic-snmp-for-string-monitor/

a.Click on Gauge Monitor tab from the console.

b. From the drop down list of Mbeans, select  ‘JDBCConnectionPoolRuntime‘ Mbean

c. Select the ActiveConnectionsHighCount attribute from the drop down list.

d. Enter the threshold values for Max and Min number to specify the range for ActiveConnectionsHighCount attribute.

e. Click on the newly created Gauge Monitor and specify which DataSource Mbean you want to monitor (DS), polling interval of the MBeans.

f. Target the created Counter Monitor to the Managed server MS1.

3. Creating SNMP Trap Destination:

a. Click SNMP Agent –>  Trap Destinations –>  New.

b. Specify the Trap Destination attributes like Host and Ports.

4. Verify the SNMP Traps:

To verify that a WebLogic Server SNMP Agent is generating and sending notifications, we can use the WebLogic command line utility SnmpTrapMonitor.

a. Open a command prompt and set the classpath to the weblogic.jar file under the %WLS_HOME%/lib folder.

set classpath=%classpath%;%WLS_HOME%\server\lib\weblogic.jar

b. Run the weblogic command line utility weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor to receive notifications.

Make sure that you specify the same listen port for this Trap Monitor as the trap destination that you created for the SNMP Agent.

For example, if your trap destination specifies a listen port of 171, enter the following command: java

java weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor -p 171

The SNMP setup is complete. The config.xml file would like below.

**********************************************************************

 

<snmp-gauge-monitor>

<name>SNMPGaugeMonitor-0</name>

<enabled-server>MS1</enabled-server>

<monitored-m-bean-type>JDBCConnectionPoolRuntime</monitored-m-bean-type>

<monitored-m-bean-name>JDBC Data Source-0</monitored-m-bean-name>

<monitored-attribute-name>ActiveConnectionsHighCount</monitored-attribute-name>

<polling-interval>10</polling-interval>

<threshold-high>20</threshold-high>

<threshold-low>10</threshold-low>

</snmp-gauge-monitor>

 

**********************************************************************

If the ActiveConnectionsHighCount value exceeds / falls below the threshold values of the DataSource Connection Pool would generate Traps on to the listener as below.

From the command line utility:

**********************************************************************

java weblogic.diagnostics.snmp.cmdline.Manager SnmpTrapMonitor -p 171

RawTrapOID     : 1.3.6.1.4.1.140.625.0.75
Trap Objects   : {
{ enterprises.140.625.100.5=Sun Dec 19 20:52:36 IST 2010 }
{ enterprises.140.625.100.10=MS1 }
{ enterprises.140.625.100.55=jmx.monitor.gauge.low }
{ enterprises.140.625.100.60=2 }
{ enterprises.140.625.100.65=0 }
{ enterprises.140.625.100.70=com.bea:Name=JDBC Data Source-0,ServerRuntime=MS
1,Type=JDBCConnectionPoolRuntime }
{ enterprises.140.625.100.75=JDBCConnectionPoolRuntime }
{ enterprises.140.625.100.80=ActiveConnectionsHighCount }
}
**********************************************************************

From the Managed server console or server logs.

<Notice> <SNMP> <BEA-320931> <The SNMP trap version is 1>

For further references:

http://download.oracle.com/docs/cd/E13222_01/wls/docs103/snmpman/index.html

Best Regards,

Wonders Team. 🙂