com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte

com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte

The above exception is encountered during server startup. This happens when SerializedSystemIni.dat present under domainyour_domainsecurity folder is modified.

To troubleshoot this issue, we need to do the following.

Check if the Domain is in Production Mode or Development Mode.

If its in Development, take a backup of your config.xml first.
Then change the values of the following elements of config.xml to clear text

credential-encrypted (at 2 places) and node-manager-password-encrypted.

If you have a datasource configured, or a jms resource, make sure to change the encrypted values in those configurations file as well.

If its a Production domain, then change it to Development by changing the production mode to false in the startup script and manually remove the entry for the production mode in the config.xml.

set PRODUCTION_MODE=false

If still your issue is not resolved, fell free to post a comment and I will try to help you out.

32 comments

  1. I installed weblogic 10.3 on AIX 6 with jdk 64 , when i try to start the server i got the below error , any help please.

    <The realm "myrealm" failed to be loaded: weblogic.security.service.SecurityServiceException: java.lang.ExceptionInInitializerError.
    weblogic.security.service.SecurityServiceException: java.lang.ExceptionInInitializerError
    at weblogic.security.service.CSSWLSDelegateImpl.initializeServiceEngine(CSSWLSDelegateImpl.java:342)
    at weblogic.security.service.CSSWLSDelegateImpl.initialize(CSSWLSDelegateImpl.java:221)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.InitializeServiceEngine(CommonSecurityServiceManagerDelegateImpl.java:1783)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealm(CommonSecurityServiceManagerDelegateImpl.java:442)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadRealm(CommonSecurityServiceManagerDelegateImpl.java:840)
    Truncated. see log file for complete stacktrace

    Caused By: java.lang.ExceptionInInitializerError
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:222)
    at com.octetstring.vde.util.guid.GuidGenerator.nextGuidInBytes(GuidGenerator.java:125)
    at com.octetstring.vde.util.guid.Guid.(Guid.java:84)
    at com.octetstring.vde.backend.standard.BackendStandard.add(BackendStandard.java:379)
    at com.octetstring.vde.backend.BackendHandler.add(BackendHandler.java:460)
    Truncated. see log file for complete stacktrace

    Caused By: java.lang.NullPointerException
    at com.octetstring.vde.util.guid.GuidParamGenerator.generateNodeID(GuidParamGenerator.java:47)
    at com.octetstring.vde.util.guid.GuidStateManager.initializeGUIDParameters(GuidStateManager.java:59)
    at com.octetstring.vde.util.guid.GuidStateManager.(GuidStateManager.java:30)
    at com.octetstring.vde.util.guid.GuidStateManager.(GuidStateManager.java:23)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    Truncated. see log file for complete stacktrace
    >

    Stopping Derby server…
    Derby server stopped.

    1. Hi Hussain,

      Is this a fresh installation or do you have any application deployed on it? The issue seems to be with the jdk you are using. Let me know the complete version of your jdk, also can you try using a different jdk vendor and check if the issue still persists?

      If that does not work, paste the stack trace from the server log so that we can check for some more information.

      Best Regards,
      Divya

  2. Check if the servers host names are correctly specified in your /etc/hosts file.
    Also if you have configured SQL/rdbms authenticators while creating the domain, make sure to run the rdbms sql file in your db.

  3. hi
    I modified the config file as you said. but i am getting the following exception

    weblogic.application.ModuleException:
    at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:290)
    at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(Modu
    leListenerInvoker.java:199)
    at weblogic.application.internal.flow.DeploymentCallbackFlow$1.next(Depl
    oymentCallbackFlow.java:507)
    at weblogic.application.utils.StateMachineDriver.nextState(StateMachineD
    river.java:41)
    at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(Dep
    loymentCallbackFlow.java:149)
    Truncated. see log file for complete stacktrace

    Caused By: weblogic.common.resourcepool.ResourceSystemException: Cannot load dri
    ver class: com.microsoft.jdbc.sqlserver.SQLServerDriver
    at weblogic.jdbc.common.internal.JDBCUtil.parseException(JDBCUtil.java:3
    01)
    at weblogic.jdbc.common.internal.ConnectionEnvFactory.loadDriver(Connect
    ionEnvFactory.java:75)
    at weblogic.jdbc.common.internal.ConnectionEnvFactory.(ConnectionE
    nvFactory.java:131)
    at weblogic.jdbc.common.internal.ConnectionPool.initPooledResourceFactor
    y(ConnectionPool.java:686)
    at weblogic.common.resourcepool.ResourcePoolImpl.start(ResourcePoolImpl.
    java:230)
    Truncated. see log file for complete stacktrace

    could you please help me out of this??
    >

    1. Hi Nisha,

      Form the stack trace, it appear that WLS is not able to find the driver class

      Caused By: weblogic.common.resourcepool.ResourceSystemException: Cannot load dri
      ver class: com.microsoft.jdbc.sqlserver.SQLServerDriver

      Can u put the sql driver in weblogic server classpath.

      You can do it by editing the startWeblogic.cmd

      set CLASSPATH = driver.jar;%CLASSPATH%

      Let me know if you still face issues.

      Cheers!
      Faisal

  4. Hello,

    I am having a similar issue as the initial post here where we get the PaddingException. I followed the instructions I believe and got the service to start in Development mode when I put the unencrypted values in the config.xml file. However, I need to get this back into a production mode and the values in the config.xml file are not being re-encrypted when the server starts in Development mode. How do I force that to happen?

    Thanks,

    1. Hi Mike,

      The usage of clear Text passwords is supported in only the development environment. However it does not accept the clear text passwords in the production environment.
      From WLS 9.x onwards a server re-start does not re-encrypt the passwords.
      You would require to use the encrypt utility provided by weblogic to encrypt the passwords first and manually edit the config.xml files with the encrypted passwords.

      The weblogic encrypt utility can be used as below.
      1. Open a command prompt and set the classpath by running the setDomainEnv script present under the Domain’s bin folder
      (for instance c:\bea103\user_projects\domains\mydomain\bin)
      2. Execute java weblogic.security.Encrypt command which will prompt for the password and will print the encrypted value in stdout.

      The following are some sample output from running the utility

      java weblogic.security.Encrypt
      Password:
      {3DES}9HWsf87pJTw=

      Note : The encrypt utility should be run from the Domain folder as it uses the domain’s password salt file(SerializedSystemIni.dat) for encrypting the clear text string literals.

      If you are not in the domain directory structure you can pass the domain folder using the -Dweblogic.RootDirectory parameter.

      java -Dweblogic.RootDirectory=c:\BEA103\user_projects\domains\mydomain weblogic.security.Encrypt weblogic123
      {3DES}/1GqbotdSXzy/Ec70btnhA==

      Note: Please take a back up of the config.xml before editing it manually.

      For further references:

      http://download.oracle.com/docs/cd/E11035_01/wls100/admin_ref/utils.html#wp1209592

      Cheers,
      Wonders Team. 🙂

  5. I am an not a java expert and have tried many times to ensure my CLASSPATH is properly set but I get the following when I try to run what you recommend:

    /opt/wam/bea/jrockit_160_05/bin/java weblogic.security.Encrypt
    Exception in thread “Main Thread” java.lang.NoClassDefFoundError: weblogic/security/Encrypt

    I am sure it is something I am not setting properly but I dont know where that would be. I am in my domain’s base directory when I run this.

    Thanks

    1. Mike,

      You can set the CLASSPATH by running the setDomainEnv.sh script under the $BEA-HOME/user_projects/domains/domainname/bin folder

      Alternatively you can try to execute the below command.

      java -Dweblogic.RootDirectory=”” -cp “”
      weblogic.security.Encrypt

      For Example:

      java -Dweblogic.RootDirectory=”/opt/wam/bea/user_projects/domains/MyDomain” -cp “/opt/wam/bea/wlserver_10.3/server/lib/weblogic.jar” weblogic.security.Encrypt

      Password: test
      {3DES}1wDhQ8IVr/g=

      Do let us know, how it goes.

      Cheers,
      Anandraj

  6. Thank you very much, that -cp command did the trick. I was setting my CLASSPATH using the setDomainEnv.sh command but that was not enough.

    All is back together now, again much appreciated.

  7. Hi Anand,

    Under which directory-path should I run/execute the command ?

    I’ve the same error , “unable to start AdminServer”

    Error –
    <The realm "myrealm" failed to be loaded: weblogic.security.service.SecurityServiceException: java.lang.ExceptionInInitializerError

    thanks
    kiran

    1. This error occurs when the WebLogic server is not able to load and initialize the security Realm.

      This is a very generic error and could be caused due to improper security configuration.

      Could you please provide the details about your environment?

      Regards,
      Anandraj

  8. Hi Everyone,

    I am struggling with this error for the past few days, your help is highly appreciated. I am trying to deploy an EAR file in the weblogic server version 10.3.3.0 but it is failing with the below error,

    <Failure occurred in the execution of deployment request with ID '1338842649393' for task '0'. Error is: 'com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte.'
    com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte.
    at com.rsa.jsafe.JA_PKCS5Padding.a(Unknown Source)
    at com.rsa.jsafe.JG_BlockCipher.decryptFinal(Unknown Source)
    at weblogic.security.internal.encryption.JSafeEncryptionServiceImpl.decryptBytes(JSafeEncryptionServiceImpl.java:113)
    at weblogic.security.internal.encryption.JSafeEncryptionServiceImpl.decryptString(JSafeEncryptionServiceImpl.java:173)
    at weblogic.security.internal.encryption.ClearOrEncryptedService.decrypt(ClearOrEncryptedService.java:96)
    Truncated. see log file for complete stacktrace

    I tried all the suggestions mentioned in the above trail, no improvement still getting the same response, I did the following

    1. Specified cleartext password in the config.xml
    2. Restarted the server in the development mode.
    3. No encrypted pwd stored in the config.xml after the restart.
    4. Generated the encrypted pwd using the weblogic encrypt utility.
    5. Specified the encrypted pwd in the config.xml and restarted the server in the production mode

    I am stuck and cant proceed further, anyone please help me out in this. Your support is highly appreciated.

    1. Hi,

      Keep only clear text entries in config.xml for now and see if it works.. also make sure the server is in dev mode..

      Do you have a backup of your config.xml

      What is the size of SerializedSystemIni.dat file, is it corruted

  9. Thanks Admin for your quick response,

    I tried your suggesstion, no luck and still same response. I ensured that, config.xml contains cleartext password and started the server in development mode.

    Yes, I do have the backup of config.xml and size of

    SerializedSystemIni.dat – 64
    DefaultAuthenticatorInit.ldift – 4114
    DefaultRoleMapperInit.ldift – 2398
    XACMLRoleMapperInit.ldift – 22654

    Not sure I how i can confirm that my SerializedSystemIni.dat file is corrupted or not.

    FYI, I am referring to weblogic version 10.3.3.0 and i am able to generate the encrypted password using eweblogic’s encrypt utility. Also each time I am getting different encrypted password for a one constant cleartext password.

    Any other suggestions? Please let me know if you need any details from me to look at this issue further.

    Please help me out in this. Thanks in advance.

  10. I had resolved the issue and the problem is, the EAR file which i tried to deploy has reference to 3DES encrypted value whereas weblogic 10.3.3.0 supports only AES encryption/decryption and hence the deplyoment fails while decrypting the 3DES encrypted value by the weblgic. The same has been corrected now, thx for your support.

  11. Hi Tiruppathi
    Can you please tell what step you did. I am stuck at the same point as you.

  12. i am getting below exception: can you please help me out:

    this is fresh installation weblogic version 10.3.1

    <The realm "myrealm" failed to be loaded: weblogic.security.service.SecurityServiceException: java.lang.ExceptionInInitializerError.
    weblogic.security.service.SecurityServiceException: java.lang.ExceptionInInitializerError
    at weblogic.security.service.CSSWLSDelegateImpl.initializeServiceEngine(CSSWLSDelegateImpl.java:342)
    at weblogic.security.service.CSSWLSDelegateImpl.initialize(CSSWLSDelegateImpl.java:221)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.InitializeServiceEngine(CommonSecurityServiceManagerDelegateImpl.java:1783)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealm(CommonSecurityServiceManagerDelegateImpl.java:442)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadRealm(CommonSecurityServiceManagerDelegateImpl.java:840)
    Truncated. see log file for complete stacktrace
    java.lang.ExceptionInInitializerError
    at com.octetstring.vde.util.guid.GuidGenerator.nextGuidInBytes(GuidGenerator.java:120)
    at com.octetstring.vde.util.guid.Guid.(Guid.java:84)
    at com.octetstring.vde.backend.standard.BackendStandard.add(BackendStandard.java:379)
    at com.octetstring.vde.backend.BackendHandler.add(BackendHandler.java:460)
    at com.octetstring.vde.util.LDIF.importLDIF(LDIF.java:279)
    Truncated. see log file for complete stacktrace
    java.lang.NullPointerException
    at com.octetstring.vde.util.guid.GuidParamGenerator.generateNodeID(GuidParamGenerator.java:47)
    at com.octetstring.vde.util.guid.GuidStateManager.initializeGUIDParameters(GuidStateManager.java:59)
    at com.octetstring.vde.util.guid.GuidStateManager.(GuidStateManager.java:30)
    at com.octetstring.vde.util.guid.GuidStateManager.(GuidStateManager.java:23)
    at com.octetstring.vde.util.guid.GuidGenerator.nextGuidInBytes(GuidGenerator.java:125)
    Truncated. see log file for complete stacktrace
    >

  13. Hi! I can’t fix my problem, can someone help ?

    I work with Weblogic8.15, and I always have the invalid pad byte
    exception.

    Here is my config.xml (I changed URL with “xxx”) :

    and if I change all CredentielEncrypted by “weblogic” I got this second error :

    [Fatal Error] :1:1: Content is not allowed in prolog.
    ***************************************************************************
    The WebLogic Server did not start up properly.
    Exception raised: ‘weblogic.management.configuration.ConfigurationException: error parsing configuration file – with nested exception:
    [org.xml.sax.SAXParseException: Content is not allowed in prolog.]’
    Reason: weblogic.management.configuration.ConfigurationException: error parsing configuration file – with nested exception:
    [org.xml.sax.SAXParseException: Content is not allowed in prolog.]
    ***************************************************************************

    Please help!

    1. We might have to do a screenshare for this.
      You can add me on skype : faiz6692

      Thanks,
      Faisal

  14. Woot. I was getting <Server subsystem failed. Reason: java.lang.AssertionError: java.lang.reflect.InvocationTargetException. Somehow, I found this page and realized that my SerializedSystemIni.dat had been incorrectly replaced. After restoring from last night's backup, everything worked.

    Thanks!
    Iggy

  15. HI i am getting below error while trying to start my managed server

    <Server failed. Reason:

    There are 1 nested errors:

    weblogic.security.service.SecurityServiceRuntimeException: [Security:090399]Security Services Unavailable
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(Unknown Source)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(Unknown Source)
    at weblogic.security.service.SecurityServiceManager.initialize(Unknown Source)
    at weblogic.security.SecurityService.start(SecurityService.java:141)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

    Please help me in this regards as i applied all the suggested solutions

    1. Is your admin server up? rename the ldap folder of the managed server and try to start up the managed server again.. please make sure admin is running.

      1. Yes my admin is up, I tried all possible ways and even copied boot.properties and security directory also but still i am not able start my managed servers, it is giving below error:

        <Server failed. Reason:

        There are 1 nested errors:

        weblogic.security.service.SecurityServiceRuntimeException: [Security:090399]Security Services Unavailable
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(Unknown Source)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(Unknown Source)
        at weblogic.security.service.SecurityServiceManager.initialize(Unknown Source)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

  16. Hi I got the following error during startup nodemanager ,any help to appreciated..
    Thanks in advance..

    SEVERE: Fatal error in node manager server
    weblogic.security.internal.encryption.EncryptionServiceException
    at weblogic.security.internal.encryption.JSafeEncryptionServiceImpl.decryptBytes(JSafeEncryptionServiceImpl.java:139)
    at weblogic.security.internal.encryption.JSafeEncryptionServiceImpl.decryptString(JSafeEncryptionServiceImpl.java:187)
    at weblogic.security.internal.encryption.ClearOrEncryptedService.decrypt(ClearOrEncryptedService.java:96)
    at weblogic.nodemanager.server.Encryptor.decrypt(Encryptor.java:165)
    at weblogic.nodemanager.server.SSLConfig$KeyStoreConfig.getCustomIdentityPrivateKeyPassPhrase(SSLConfig.java:281)
    at weblogic.security.utils.KeyStoreConfigurationHelper.getIdentityPrivateKeyPassPhrase(KeyStoreConfigurationHelper.java:166)
    at weblogic.nodemanager.server.SSLConfig.loadKeyStoreConfig(SSLConfig.java:167)
    at weblogic.nodemanager.server.SSLConfig.(SSLConfig.java:102)
    at weblogic.nodemanager.server.NMServer.init(NMServer.java:186)
    at weblogic.nodemanager.server.NMServer.(NMServer.java:148)
    at weblogic.nodemanager.server.NMServer.main(NMServer.java:380)
    at weblogic.NodeManager.main(NodeManager.java:31)
    + set +x

  17. Hi Faizal,
    Even Iam facing similar issue while starting Admin Server. I have SSO enabled and in Production mode. Do you think I have to modify config.xml bit differently ?? Because i have modified the 3 (encrypted pwd places) in the config.xml with weblogic password. But now Iam getting a different error as below

    Feb 2, 2016 12:27:53 AM oracle.security.jps.internal.credstore.ssp.CsfWalletManager openWallet
    WARNING: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.

    <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
    weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1398)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1018)
    at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
    at weblogic.security.SecurityService.start(SecurityService.java:141)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    Truncated. see log file for complete stacktrace
    Caused By: oracle.security.jps.JpsRuntimeException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
    at oracle.security.jps.internal.policystore.PolicyDelegationController.(PolicyDelegationController.java:167)
    at oracle.security.jps.internal.policystore.JavaPolicyProvider.(JavaPolicyProvider.java:369)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    Truncated. see log file for complete stacktrace
    Caused By: oracle.security.jps.JpsException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
    at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPDPService(PolicyUtil.java:2855)
    at oracle.security.jps.internal.policystore.PolicyUtil.getPDPService(PolicyUtil.java:3099)
    at oracle.security.jps.internal.policystore.PolicyDelegationController.(PolicyDelegationController.java:164)
    at oracle.security.jps.internal.policystore.JavaPolicyProvider.(JavaPolicyProvider.java:369)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    Truncated. see log file for complete stacktrace
    Caused By: oracle.security.jps.service.credstore.CredStoreException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
    at oracle.security.jps.internal.credstore.ssp.CsfWalletManager.openWallet(CsfWalletManager.java:184)
    at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.doInit(SspCredentialStore.java:239)
    at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.(SspCredentialStore.java:153)
    at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.(SspCredentialStore.java:139)
    at oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider.getInstance(SspCredentialStoreProvider.java:115)
    Truncated. see log file for complete stacktrace
    >

    Please advise and help me !

    Regards/
    KK

Comments are closed.