Oracle Server - Enterprise Edition - Version 8.0.3.0 and later
Goal:
Discuss what can be expected when implementing Dead Connection Detection (DCD) along with Database Resource Limits with user Profiles, inactive sessions and the effects on V$SESSION, V$PROCESS and OS Processes
Fix:
Dead Connection Detection (DCD)
These are previously valid connections with the database but the connection between the client and server processes has terminated abnormally. Examples of a dead connection:
- A user reboots/turn off their machine without logging off or disconnecting from the database.
- A network problem prevents communication between the client and the server.
In the cases, the shadow process running on the server and the session in the database may not terminate.
Implemented by
* adding SQLNET.EXPIRE_TIME =
with DCD is enabled, the server-side process sends a small 10-byte packet to the client process after the duration of the time interval specified in minutes by the SQLNET.EXPIRE_TIME parameter.
If the client side connection is still connected and responsive, the client sends a response packet back to the database server, resetting the timer.. and another packet will be sent when next interval expires.
If the client fails to respond to the DCD probe packet
* The server-side process is marked as dead connection and
* PMON performs the clean up of the database processes / resources\
* The client OS processes are terminated.
Database Resource Limits (using user profiles)
Implemented by
* Setting RESOURCE_LIMIT = TRUE in the database startup parameter file (spfile or pfile)
* Creating or modifying existing user profiles (DBA_PROFILES) to have one or more resource limit
* Assigning a profile to a user whose resources are wished to be limited
It could happen that if the idle_time has been set on the DEFAULT profile, this can lead to an MTS dispatchers being set to 'sniped' and then getting 'cleaned up' via the shell script. The removal of the dispatcher will result in other sessions 'dying' .In that case, If you are to implement resource limits, may be advisable to create new profiles
that be assigned to users and not to change the characteristics of DEFAULT.
Alternatively, if you do change DEFAULT, ensure that all the properties that you
have affected have been fully tested in a development environment.
When a resource limit is exceeded (for example IDLE_TIME) ... PMON does the following
* Mark the V$SESSION as SNIPED
* Clean up the database resources for the session
* Remove the V$SESSION entry
For this case we will see that :
* PMON has cleaned up the V$SESSION entries .. but both the OS processes and the V$PROCESS entries will still exist
* SQLNET will continue to be able to send the 10 byte packet successfully until the session is logged off
This condition can be a major problem as
* The database exhausts PROCESSES and gives ORA-20 maximum number of processes
* The OS can become exhausted due to the unneeded resources consumed by the abandoned processes
The SYMPTOMS of this condition are
* The database view V$PROCESS will have no corresponding V$SESSION entry
* An OS process / thread still exists for the SNIPED session
The solutions to this scenario can are to cleanup the OS processes ... after which the $PROCESS entries should be removed automatically
Methods to cleanup OS processes:
* UNIX : kill -x ... the OS process at the OS level (typically kill -9)
* UNIX: if using a dedicated server, use the following shell script to kill the shadow process (script has been tested on Solaris, AIX, Tru64 and HPUX):
#!/bin/sh
tmpfile=../tmp/tmp.$$
sqlplus -S ' / as sysdba' <
spool $tmpfile
set echo off;
set feed off;
select p.spid from v\$process p,v\$session s
where s.paddr=p.addr
and s.status='SNIPED';
spool off
EOF
for x in `cat $tmpfile | egrep "^[0123456789]+"`
do
kill -9 $x
done
rm $tmpfile
On occasions we see conditions where a database session has a V$SESSION.STATUS = SNIPED ... and the entry never goes away . This condition can be achieved by implementing Database Resource Limits + Profiles without DCD and allow the database session to exceed the limit in the profile
Summary of what was discussed here:
1) DCD initiates clean up of OS and database processes that have disconnected / terminated abnormally
2) DCD will not initiate clean up sessions that are still connected ... but are idle / abandoned / inactive
3) Database Resource Limits + user Profiles clean up database resources for user sessions that exceed resource limits
4) Database Resource Limits + user Profiles will not clean up OS processes
5) If DCD and Database Resource Limits + user Profiles are used in combination .. Dead Connections OS and Database Resources will be cleaned up
6) IDLE / ABANDONED / INACTIVE sessions OS processes will not be cleaned up even if DCD and Database Resource Limits + user Profiles are used in combination ... these must be cleaned up manually
