JPS-01050: Opening of wallet based credential store failed.

Issue:
Unable to start services like Foundation,Planning .
Description:
On viewing the logs in below location
E:\Oracle\Middleware\user_projects\domains\EPMServer\servers\FoundationServices0
You find below error .

Logs:
Caused By: oracle.security.jps.JpsRuntimeException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: End of stream reached before all 41559 expected bytes could be read. Only received 0 bytes before the end of the stream was reached
    at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:170)
    at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:383)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at java.lang.Class.newInstance0(Class.java:355)
    at java.lang.Class.newInstance(Class.java:308)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1343)
    at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1024)
    at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:879)
    at weblogic.security.SecurityService.start(SecurityService.java:141)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)


Cause:
Corruption of cwallet.sso . One of the causes is disk space on the installed drive and temp space is full.

Solution:
1.Go to E:\Oracle\Middleware\user_projects\domains\EPMSystem\config\fmwconfig
2.Observe that the cwallet.sso is either 0kb or 1kb .
3.Rename it to cwallet.sso.bkup
4.Restart the services now.

Lateral Damage:
You may lose any non default keystores saved in the cwallet.sso

Read More

Recovery Week! HFM Application recovery

This week was full of recovery of crashed , destroyed HFM applications.
Situation
This was Go Live week and history has proved before all important days in the calendar a catastrophy should happen and we are no exception.

How we recovered our HFM applications and then did multiple test cases and here are the results.

Case 1: We restored the entire cold backup on our database side and restarted the services . This worked fine. We were able to recover the applications but sadly this worked only during test case and no during the actual replacement.


Case 2:
Our main application crashed and luckily someone had made a duplicate copy of the application. We duplicated the crashed application, then deleted it. Then from the backup copy we duplicated it and renamed it to the actual application.


Case 3:
Our main application crashed. This happened with a different client . They were not able to open the application after migration. So I had a backup of HYP_HFM schema (the schema in which Hyperion HFM is configured) restored it and then restored the metadata from the .ads file backup I had.


Case 4:

We did not test this case but you can restore the application using the LCM backups you have.

Read More

OutOfMemoryError: PermGen space

Error: OutOfMemoryError: PermGen space

Description: This error usually occurs after you have fixed this error "java.lang.OutOfMemoryError: Java heap space" . Commonly occurs in Solaris environment. 


Reason:  The permanent generation space is less hence we need to add more space. 

Solution:

Do this for any managed server that is throwing below error like Planning,Calculation manager etc

1. If it is Unix environment , go to instance home ex : /app/Oracle/Middleware/user_projects/epmsystem1/bin/deploymentScripts folder

2.  Here you will find a lot of scripts for Foundation,Planning,EAS etc.
3. Back the file up  and change the values to a higher value and restart the services
 -Xms1024m -xmx4096m -XX:PermSize=256m -XX:MaxPermSize=512





Refer to this guide to https://blogs.oracle.com/pa/resource/Oracle_EPM_11_1_2_3_Tuning_Guide_v2.pdf
 

Read More