SELECT /* well-paid people */ e.last_name, e.department_id, e.salary
FROM employees e
WHERE e.salary> (SELECT AVG(salary) FROM EMPLOYEES
WHERE department_id=e.department_id);

This is correlated subquery and we can ask how many times the subquery will be executed?
We will see that the answer is not a number.
2.Second solution can be:
SELECT /* well-paid people */ e.last_name, e.department_id,
e.salary ,d.average
FROM employees e,
(SELECT department_id, AVG(salary) average FROM employees GROUP BY department_id ) d
WHERE e.department_id=d.department_id AND e.salary>d.average ;
Now we examine the execution plans (using my sqlid stroed procedure, see an other blog entry .
Let’s the execution plains:
EXEC sqlid(‘well-paid people’,'typical’)

For the second solution, please consider the following figure:

We can observe – although- the plan hash values are different, actually the content of them are identical!
In other words Oracle executed a query transformation for the second SELECT statement and used the semantically same plan for both quires.
Let’s suppose that we prohibit the query transformation for the first SELECT statement:
SELECT /*+ NO_QUERY_TRANSFORMATION*/
e.last_name, e.department_id, e.salary
FROM employees e
WHERE e.salary> (SELECT AVG(salary) FROM EMPLOYEES
WHERE department_id=e.department_id);

We can realize that this plan is more expensive than previous one.
Be careful with hints!
Sometimes we can demolish the server’s good plan with our hints!

Oracle developers sometimes delete their subprograms, views, tables etc accidentally.
From Oracle 10g we have opportunity to salvage them from the undo segment.
Note that – by default – the Oracle does not guarantee the successful salvage in 100%.
(See later in this article or the documentation fore more detail)

The FLASHBACK statement – with wide range of  functionalities – ensures the possibility to restore earlier state of objects or completely recreate them.
Flashback queries appeared in Oracle 9i first.
This is well known by the DBA’s and developers.
Maybe not so popular that we can execute flashback queries against the Data Dictionary.
Average users (ie. Not SYS) can not do it without necessary object privileges.
Note the FLASHBACK ANY TABLE  system privilege is not enough for these queries!

Let’s see an example:
1. HR user created earlier a function, this was the content of the tax.sql:
CREATE OR REPLACE FUNCTION TAX(P_AMOUNT IN NUMBER)
RETURN NUMBER
IS
M  NUMBER;
BEGIN
IF P_AMOUNT <8000 THEN M:=0.08;
ELSIF P_AMOUNT <18000 THEN M:=0.25;
ELSE  M:=0.3;
END IF;
RETURN P_AMOUNT*M;
END;
/

Although HR has the original version of the function, but he modified his function in the database many times  and he did not track these modifications in the text file (in our example tax.sql).
HR user deleted his function accidentally:

DROP FUNCTION TAX;

A couple minutes later he realized that it was a mistake and his version of the function in tax.sql is too old! He asked the DBA to help him.
The DBA connected to the database instance as SYS and issued the following GRANT statement:

GRANT FLASHBACK ON USER_SOURCE TO HR;

The DBA wanted to be proactive and  helpful, so he or she  issued the second  SQL statement:

GRANT FLASHBACK ON USER_VIEWS TO HR;

Now HR will be able to issue a flashback query against the subset (only for USER_SOURCE and USER_VIEWS)  of the Data Dictionary:

SELECT TEXT FROM USER_SOURCE AS OF TIMESTAMP SYSTIMESTAMP- INTERVAL ‘&MIN’ MINUTE
WHERE NAME=’TAX’;

TEXT
————————————
FUNCTION TAX(P_AMOUNT IN NUMBER)
RETURN NUMBER
IS
M  NUMBER;
BEGIN
IF P_AMOUNT <8000 THEN M:=0.08;
ELSIF P_AMOUNT <18000 THEN M:=0.25;
ELSE  M:=0.3;
END IF;
RETURN P_AMOUNT*M;
END;

11 rows selected.

If HR redirects the output of the above query in a spool file, he can recreate  the latest version of his function.
Of course HR can issue the same select against the USER_VIEWS.

This method useful If:

1. The Oracle object is a text type object (subprograms, views, directories, context objects, …)
2. The DROP statement was not “too old”. This means, if there was a big traffic in the database, many transactions used the undo segment in latest couple of hours or even minutes)
3. The DBA issued similar statements to this:
   ALTER SYSTEM SET UNDO_RETENTION = 3600;
   ALTER TABLESPACE UNDOTBS1 RETENTION GUARANTEE ;

Window aggregate functions operate on an ordered set of rows and for each row they return an aggregate value for a window rooted at that row.
The window can be unbounded, partially unbounded or bounded. The size of the window can be specified by physically (eg. With number of rows, ROWS) or logically (typically with date/time values, RANGE).

There are several keywords available:

CURRENT ROW – specifies that window starts or ends at the current row

UNBOUNDED PRECEDING – specifies that the window starts at the first row of the partition.

UNBOUNDED FOLLOWING – specifies that the window ends at the last row of the partition.
If we don’t specify explicitly the one of the boundaries the CURRENT ROW is the default.

Example 1:

We want to create a cummulated aggregation (sometimes called running aggregates) for the salaries for the whole table and for each department:

SELECT department_id,employee_id, last_name, salary, SUM(salary) OVER (ORDER BY salary DESC RANGE UNBOUNDED PRECEDING)  AS cum_range,
SUM(salary) OVER (PARTITION BY department_id ORDER BY salary DESC RANGE UNBOUNDED PRECEDING) AS cum_range_for_depts,
SUM(salary) OVER (ORDER BY salary desc ROWS UNBOUNDED PRECEDING)  cum_rows,
SUM(salary) OVER (PARTITION BY department_id ORDER BY salary DESC ROWS UNBOUNDED PRECEDING)  cum_rows_for_depts
FROM employees
--ORDER BY department_id ;

Example 2:

Now we define an increasing, decreasing and fix sized window for each department:

SELECT department_id, last_name, salary,
SUM(salary) OVER (PARTITION BY department_id ORDER BY salary DESC
ROWS UNBOUNDED PRECEDING)  AS cum_sum,
SUM(salary) OVER (PARTITION BY department_id ORDER BY salary DESC
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)  till_current,
AVG(salary) OVER (PARTITION BY department_id ORDER BY salary DESC
RANGE UNBOUNDED PRECEDING) cum_avg , AVG(salary)
OVER ( PARTITION BY department_id ORDER BY salary DESC RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) avg_sal_rest,
AVG(salary)   OVER (PARTITION BY department_id) AS dept_avg
FROM employees
/

Example 3:

We want to select to 5 most popular products and running total for each quarter in each year.

SELECT CASE WHEN t.rank=1 THEN t.year END year, t.quarter, t.product, t.amount, t.quantity, t.rsum_for_quarter, t.rank rank_for_amount
FROM(SELECT t.calendar_year year,t.calendar_quarter_desc quarter, p.prod_name product, SUM(s.amount_sold) amount, SUM(SUM(s.amount_sold))
OVER (PARTITION BY t.calendar_year,t.calendar_quarter_desc
ORDER BY SUM(s.amount_sold) DESC RANGE UNBOUNDED PRECEDING) rsum_for_quarter, SUM(s.quantity_sold) quantity,
RANK()  OVER (PARTITION BY t.calendar_year , t.calendar_quarter_desc
ORDER BY SUM(s.amount_sold) DESC ) rank
FROM sh.sales s, sh.times t, sh.products p
WHERE s.time_id=t.time_id AND s.prod_id=p.prod_id
GROUP BY t.calendar_year,t.calendar_quarter_desc,p.prod_name) t
WHERE t.rank<5;
In the SUM(SUM(s.amount_sold)) expression the inner SUM is standard group function, which is the argument of the outer SUM that is an analytical function.

Example 4:
Many times we want to see each element’s contribution (rate) for the total value.
Also called percentages or sharing value.
In the following example we compute the percentages of each  salary of subtotal (department level) of grand total:

SELECT department_id, last_name, salary,
RATIO_TO_REPORT(salary)
OVER (PARTITION BY department_id)  AS rate_for_dept,
RATIO_TO_REPORT(salary) OVER ()   AS rate_for_total
FROM employees;

Example 5:
Now we want to compare each employee’s salary to the lowest anf highest salaries of his/her department using the FIRST_VALUE and LAST_VALUE functions.

SELECT t.*,t.salary/t.lowest_sal  rate_for_lowest_sal,
t.salary/t.highest_sal AS rate_for_highest_sal
FROM ( SELECT department_id , last_name, salary,
FIRST_VALUE(salary)
OVER ( PARTITION BY department_id ORDER BY salary
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS lowest_sal,
LAST_VALUE(salary) OVER ( PARTITION BY department_id ORDER BY salary
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED   FOLLOWING) AS highest_sal
FROM employees ORDER BY department_id) t
/

Example 6:
Many times we need to compute moving values(MAX,SUM,AVG). In this example we calculate moving sum for better understanding:

SELECT last_name, hire_date, salary,
SUM(salary)  OVER (ORDER BY hire_date RANGE BETWEEN NUMTOYMINTERVAL(3,'MONTH') PRECEDING AND NUMTOYMINTERVAL(3,'MONTH') FOLLOWING) AS mov_sum_3_months,
SUM(salary)  OVER (ORDER BY hire_date ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) AS mov_sum_3_rows
FROM employees;

Observe the difference between RANGE and ROWS.
With RANGE we specified a six months window, while with ROWS we created a 7 row width window.

To be continued…

The traditional SQL statements and functions do not provide strong support for analytic tasks. Basic business intelligence calculations such as moving averages, rankings and lead/lag comparisons have required extensive programming outside of standard SQL , often with performance challenges.

From Oracle 8.1.6 Oracle provides powerful new family of functions which are useful for analyzing data warehouses or simple big data sets.

Oracle has created four families of analytic functions

• Ranking family: RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST
• Window Aggregate family: AVG, SUM, MIN, MAX, COUNT, VARIANCE and STDDEV (typically with fix sized window)
• Reporting Aggregate family: AVG, SUM, MIN, MAX, COUNT, VARIANCE and STDDEV , RATIO_TO_REPORT (typically with varying sized window)
• LAG/LEAD family: LAG  and LEAD

It is important to understand the processing order of the functionalyties of the analytic SQL statements

• The joins will be executed
• Evaluating the WHERE, GROUP BY, HAVING clauses
• Partitions created
• Analytic functions applied to each row in each partition
• ORDER BY is processed to allow for precise output ordering.

In the following examples we will use the the sample schemas (HR,SH) which are available in all Oracle database.

Ranking family:

Example 1:

Let’s suppose we want rank the salaries in the employees table.

Issuing the following SELECT

SELECT ROWNUM,last_name,salary from
(SELECT * FROM EMPLOYEES ORDER BY salary DESC) t
WHERE ROWNUM<=&counter;

for counter=2  :

1 KING               24000

2 KOCHHAR      17000

for counter=3  :

 1 KING               24000

2 KOCHHAR      17000

3 DE HAAN        17000

Which are the ranks for Kochhar and De Haan?
Solution:

SELECT last_name,salary,
RANK() OVER( ORDER BY salary DESC) normal_rank,
DENSE_RANK() OVER( ORDER BY salary desc) dense_rank
FROM employees;

Example 2:
Let’s see the top 5 best products regarding the revenue for each year:

SELECT t.*
FROM (select t.calendar_year year , p.prod_name product, SUM(s.amount_sold), SUM(s.quantity_sold),
RANK()  OVER (PARTITION BY t.calendar_year ORDER BY SUM(s.amount_sold) DESC ) rank
FROM sh.sales s, sh.times t ,sh.products p
WHERE s.time_id=t.time_id AND s.prod_id=p.prod_id
GROUP BY t.calendar_year,p.prod_name) t
WHERE t.rank<=5;

Example 3:

We can retrieve the 4 best products regarding the revenue for each product category:

SELECT t.*
FROM (select p.prod_category,t.calendar_quarter_desc quarter  , ROUND(SUM(s.amount_sold)) amount ,SUM(s.quantity_sold) quantity,
RANK()  OVER (PARTITION BY P.prod_category
ORDER BY SUM(s.amount_sold) DESC ) rank
FROM sh.sales s, sh.times t ,sh.products p
WHERE s.time_id=t.time_id AND s.prod_id=P.prod_id
GROUP BY p.prod_category,calendar_quarter_desc ) t
WHERE t.rank<=4
/

Example 4:
Now we consider the cumulative distribution of the salaries for each departments:

SELECT department_id, last_name, last_name, salary,
CUME_DIST() OVER(PARTITION BY department_id ORDER BY  Salary)
FROM employees;

Example 5:
We create ranks and subtotals  for amounts in  each year:

SELECT t.*
FROM (select t.calendar_year year , p.prod_name  product , SUM(s.amount_sold) amount,SUM(s.quantity_sold),
RANK()  OVER (PARTITION BY t.calendar_year ORDER BY SUM(s.amount_sold) DESC ) -1 rank,GROUPING(p.prod_name) grp_id
FROM sh.sales s, sh.times t ,sh.products p
WHERE s.time_id=t.time_id AND s.prod_id=p.prod_id
GROUP BY ROLLUP ( t.calendar_year,p.prod_name)) t
WHERE t.rank<=5
ORDER BY year,grp_id,amount DESC
/

Example 6:
The ROW_NUMBER function – which is not equivalent with neither ROWNUM nor RANK – assigns a unique number, in the sequence defined by the ORDER BY clause, to each row within the partition.

SELECT  ROW_NUMBER() OVER(ORDER BY department_id) row_number , ROWNUM ,
ROW_NUMBER() OVER( PARTITION BY department_id ORDER BY salary DESC) dept_row_number,
department_id,last_name,salary,
RANK() OVER( PARTITION BY department_id ORDER BY salary DESC) normal_rank,
DENSE_RANK() OVER( PARTITION BY department_id ORDER BY salary DESC) dense_rank
FROM employees
/
To be continued…

Tracing  a  client

Let’s suppose there are two client or application user (JOEL and LASZLO).
They use the same application, but different modules and doing different activities.
Via connection pool they use –implicitly – the apps Oracle user account.
LASZLO is responsible for REPORTING using the EDUCATION module.
JOEL is manager working with SME REPORTING in MANAGEMENT MODULE.
LASZLO is working in EDUCATION_DEV modul  using  DEVELOPING activity
JOEL wants to observe LASZLO’s activity tracing LASZLO  as a client user:
We can imitate with DBMS_SESSION and DBMS_APPLICATION_INFO packages:

In SQL*Plus APPS executed the following PL/SQL program:
EXEC DBMS_SESSION.SET_IDENTIFIER(‘LASZLO’)
EXEC DBMS_APPLICATION_INFO.SET_MODULE(‘EDUCATION_DEV’,'DEVELOPING’)

In an other SQL*Plus instance APPS executed the following PL/SQL program:
EXEC DBMS_SESSION.SET_IDENTIFIER(‘LASZLO’)
EXEC DBMS_APPLICATION_INFO.SET_MODULE(‘EDUCATION’,'REPORTING’)

In SQL Devepoler SYS executed the following PL/SQL program:
EXEC DBMS_SESSION.SET_IDENTIFIER(‘JOEL’)
EXEC DBMS_APPLICATION_INFO.SET_MODULE(‘MANAGEMENT’,'SME REPORTING’)

We want to trace LASZLO’S activity
The trace files will be created under this directory:

Now  you can execute (as SYS) the following PL/SQL program:
EXEC DBMS_MONITOR.CLIENT_ID_TRACE_ENABLE(‘LASZLO’)
If you want to set the statistics level for that session
create AFTER LOGON trigger, like this:
CREATE OR REPLACE TRIGGER logon_trig
AFTER LOGON  ON  hr.SCHEMA
BEGIN
EXECUTE IMMEDIATE ‘ALTER SESSION SET STATISTICS_LEVEL=ALL’;
END;
/

LASZLO executed  the following query  as DEVELOPING using  apps Oracle user :

SELECT /* LASZLO_DEVELOPING */ last_name,e.salary
FROM hr.employees e
WHERE
E.SALARY>(select avg(d.salary) FROM hr.employees d
WHERE d .department_id=e.department_id);

LASZLO executed  the following query  as REPORTING  using  apps Oracle user :

SELECT /* LASZLO_REPORT */ last_name,e.salary
FROM hr.employees e
WHERE E.SALARY>(select avg(d.salary)
FROM hr.employees d
WHERE  d.department_id=e.department_id);

Now  you can execute (as SYS) the following PL/SQL program:
EXEC DBMS_MONITOR.CLIENT_ID_TRACE_DISABLE(‘LASZLO’)
Under the trace directory you will find the following files: orcl_ora_9508.trc and orcl_ora_1752.trc.

But you don’t have to know them!

It is enough to run the TRCSESS utility which collects the necessary information from the trace files and produce  a new one:

trcsess output=laszlo.trc clientid=LASZLO

This file contains very detailed information. If You need a more readable text file, please run the tkprof utility in  command  line
tkprof LASZLO.trc  LASZLO.txt

Part from laszlo.txt (LASZLO_DEVELOPING)

Part from laszlo.txt (LASZLO_REPORT)

Tracing  a  session

This method is applicable, if the session was created by an c/s application.
Very easy to identify which is the target  session:

Let us suppose that  the session is identified by
SID=420 and SERIAL#=2103.
The HR user’s  Operating system process identifier (SPID) is 6684.
You can find the directory of the trace file which will be created later:

Now  you can execute (as SYS) the following PL/SQL program:
EXEC DBMS_MONITOR.session_trace_enable(420,2103,TRUE,TRUE,’ALL_EXECUTIONS’)

3. argument (waits)  TRUE, wait information is present in the trace
4. argument (binds)  TRUE, bind information is present in the trace
If you want to set the statistics level for that session
create an AFTER LOGON trigger, like this:
CREATE OR REPLACE TRIGGER logon_trig
AFTER LOGON  ON  hr.SCHEMA
BEGIN
EXECUTE IMMEDIATE
‘ALTER SESSION SET STATISTICS_LEVEL=ALL’;
END;
/

Connect and execute the following query as HR :

You can turn off the tracing for the same session:

EXEC DBMS_MONITOR.session_disable(420,2103)

Under the trace directory you will find the following file: orcl_ora_6684.trc where 6684 is HR’s SPID.
This file contains very detailed information.
If You need a more readable text file, please run the tkprof utility in  command  line
tkprof orcl_ora_6684.trc hr_sqlplus2.txt
Detail from the file:

Execute the sqlid stored procedure
(which can be found in “Trace your Oracle session” blog)
as SYS:

EXEC hr.sqlid(‘MAX(SUM(salary))’)

Many times developers and/or DBA’s want to trace their own and/or other session’s  SQL and PL/SQL activities.
There are many possibilities to solve this problem especially, if you  want to trace your  session.
Very easy to identify which is your session:

Let us suppose that  your session is identified by SID=403 and SERIAL#=1242.
Your Operating system process identifier (SPID) is 8564
You can find the directory of the trace file which will be created later:

Modify your session:
ALTER SESSION SET SQL_TRACE=TRUE;
ALTER SESSION SET  STATISTICS_LEVEL=ALL;

Execute the following query:

Under the trace directory You will find the following file: orcl_ora_8564.trc where 8564 is your SPID.
This file contains very detailed information.
If You need a more readable text file, please run the tkprof utility in  command  line
tkprof orcl_ora_8564.trc hr_sqlplus.txt

Detail from the text file:

Executing  a suitable PL/SQL stored procedure, called  SQLID:

You want to find your SELECT statement’s execution plan with plan statistics (due to statistics level is ALL!)
EXEC SQLID(‘MAX(SUM(salary))’)

SQL developers frequently hear and realize that the subqueries
are not efficient, avoid using them.
Indeed, many times those SELECT statements, which contains subquires
in the WHERE or HAVING clause are not very efficient.

But this not a general rule!

I'd like to demonstrate that subquieres with IN operator and
correlated subquires can be very efficient.

Let see the next example:

SELECT last_name, department_id, job_id FROM employees
WHERE department_id IN
(SELECT department_id FROM departments WHERE location_id = 1700);
Here is the execution plan for the SELECT statement:

SQL_ID  gm32t8f65882q, child number 0
-------------------------------------
SELECT last_name, department_id, job_id FROM   employees WHERE
department_id IN (SELECT department_id FROM departments WHERE  location_id = 1700)
Plan hash value: 1021246405
--------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name              | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |                   |       |       |     6 (100)|          |
|   1 |  NESTED LOOPS                |                   |       |       |            |          |
|   2 |   NESTED LOOPS               |                   |    37 |   814 |     6   (0)| 00:00:01 |
|*  3 |    TABLE ACCESS FULL         | DEPARTMENTS       |     4 |    20 |     4   (0)| 00:00:01 |
|*  4 |    INDEX RANGE SCAN          | EMP_DEPARTMENT_IX |    10 |       |     0   (0)|          |
|   5 |   TABLE ACCESS BY INDEX ROWID| EMPLOYEES         |    10 |   170 |     1   (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter("LOCATION_ID"=1700)
4 - access("E"."DEPARTMENT_ID"="D"."DEPARTMENT_ID")
====================================================================================================
Let us see the second solution and the execution plan!

SELECT last_name, e.department_id, job_id
FROM employees e,departments d
WHERE e.department_id =d.department_id and location_id = 1700;

SQL_ID  gq10rm3ypv7v2, child number 0
-------------------------------------
SELECT last_name, e.department_id, job_id FROM   employees
e,departments d WHERE  e.department_id =d.department_id and
location_id = 1700
Plan hash value: 1021246405
--------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name              | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |                   |       |       |     6 (100)|          |
|   1 |  NESTED LOOPS                |                   |       |       |            |          |
|   2 |   NESTED LOOPS               |                   |    37 |   814 |     6   (0)| 00:00:01 |
|*  3 |    TABLE ACCESS FULL         | DEPARTMENTS       |     4 |    20 |     4   (0)| 00:00:01 |
|*  4 |    INDEX RANGE SCAN          | EMP_DEPARTMENT_IX |    10 |       |     0   (0)|          |
|   5 |   TABLE ACCESS BY INDEX ROWID| EMPLOYEES         |    10 |   170 |     1   (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter("LOCATION_ID"=1700)
4 - access("E"."DEPARTMENT_ID"="D"."DEPARTMENT_ID")
====================================================================================================

We can realize that both quires use the same execution plan (The plan Hash values are same!)
so they are same efficient!