Wednesday, June 24, 2015

Authorization Error while doing offline deployment in IBM BPM PS

Problem :: Facing Error while doing offline deployment in IBM BPM PS servers. Error is 'You are not authorized to make changes to items in this context'.

We were doing offline deployment with very well established commands and commands which we use in other environments and working from long time.


Work Around ::
We raised PMR for same and  after some investigation IBM team came back with details that there was problem in Database an entry for particular application and authorization.  
Table - LSW_ACL_ENTRY
We need to check the entry for our project in this table which we can do by comparing below columns
LSW_ACL_ENTRY -> PO_ID
LSW_PROJECT     -> Project_Id

And then check what is the User_id or group_id in the respective column of LSW_ACL_ENTRY. in our case user_id was 1 which means tw_autor user was having access to that processApp.

IBM team was not sure why it was like that as it should be user_id 3 which is tw_admin who should have access to application.

Now, catch is we were using tw_admin credentials to login to wsadmin console before doing deployment. and this user was not having access to application. at same time we cannot do deployment using tw_author as it is not an admin user.

As a workaround , We have added tw_author user in tw_admins group and completed our deployment using tw_author user which worked.

IBM Team is still not sure why this thing happened and they said they will investigate and come back with permanent solution as adding tw_author in tw_admins group is not a good idea.

I will update this post if we find any permanent solution.

Monday, June 1, 2015

IBM BPM PC servre throwing OOM error and High CPU usage after 7.5.1.2 upgrade from 7.5.1.1

Problem::
Getting OutofMemory error and High CPU usage of applicaiton server just after upgrading to IBM BPM 7.5.1.2
We have raised PMR to IBM for same and solution we got was as below.

Solution ::
After upgrading BPM 7.5.1.1 to 7.5.1.2 all snapshots will be some kind 
of 'fixed'. Hence, those are loaded in the heap and could allocate a   
lot of memory, depending on the amount. This operation is called       
UserAttributeDefinitionRepair and is an one time operation. So when it 
is completed once, it is not executed again. If the system crashes     
during this operation, BPM tries to re-run it again and will probably  
fail again.                                                            
                                                                        
One can verify, that this process is running, doing the following: To  
check if the 'UserAttributeDefinitionRepair' is still running, please  
check table LSW_SYSTEM and look for the PROPKEY                        
'UserAttributeDefinitionRepair'. If its PROPVALUE is not COMPLETED,    
than the process is not finished yet.                                  
                                                                        
In order to regulate the number of loaded snapshots one can adjust the 
Branch Context Cache Size. We suggested to use the following action    
plan:                                                                  
                                                                        
1) Change the branch-context-max-cache-size to 64 and monitor the      
system, if OOM occurs (testing now)                                    
2) Change the branch-context-max-cache-size to 32 and monitor the      
system, if OOM occurs,                                                 
3) Install the recommended fixes for BPMSnapshotCleanup and dependency 
ifixes on top of 7.5.1.2.                                              
 - 7512 tab                                                             
4) Run BPMSnapshotCleanup command as instructed to clean up unneeded   
 snapshots.                                                            
main.doc/managinglib/topic/managing_snapshots_i.html?cp=SSFPJS_7.5.1   
5) After clean up database, start server and monitor the system.       
                                                                        
Steps 1 and 2 might be the most crucial. Reducing the cache size causes
to be not that much Snapshots be loaded in the heap at once. As a      
result, the whole process will take longer, but it will also not fail. 
This value can be adjusted after the process is once finished (see     
above: PROPVALUE shows COMPLETED then).                                
                                                                        
Steps 3 to 5 are recommended in order to remove old, unused snapshots, 
that influence the whole process. Keeping the system clean is a good   
approach in general. Those cleaning operations are just available on   
BPM 7.5.1.2 and not 7.5.1.1. See:                                      
ofmemory-regarding-poversions/                                         
                                                                        
In order to prevent the whole problem, you can use the following       
process for further migrations from BPM 7.5.1.1 to BPM 7.5.1.2:        
                                                                        
1) Identify the process apps and toolkits which have the largest numbers
of snapshots by running DB queries. Take note.                         
2) Repeat the following steps to each process app or toolkit will be   
 cleaned.                                                              
    a) Create a new snapshot;                                          
    b) Export snapshot created in a);                                   
    c) Archive and delete process app or toolkit;                      
    d) Re-import snapshot exported in b).                              
                                                                        
    Note: If multiple versions of toolkit snapshots are referenced,    
please do the following:                                               
    a1) Export all toolkit snapshots to keep one by one;               
    b1) Archive and delete the toolkit;                                 
    c1) Import toolkit snapshot in the desired sequence, import the    
oldest snapshot first. Do NOT import them with dependent process apps  
since it may potentially messed up the toolkit snapshot levels.