Lesson for the day

With cost based optimizer if you don’t collect statistics properly then it means that you are screwed.

This is what I did.

Connected to:
Oracle9i Enterprise Edition Release 9.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.1.0 - Production

I created the following table.

SQL> create table hx_index_test_tbl(
2 no integer not null,
3 name varchar2(50) not null,
4 old_value varchar2(100),
5 new_value varchar2(100));

Table created.

Next I added these two indexes to the table.

SQL> create index hx_index_test_tbl_idx on hx_index_test_tbl(name,no);

Index created.

SQL> create index hx_index_test_tbl_idx_001 on hx_index_test_tbl(name);

Index created.

Then I populated some data.

SQL> insert into hx_index_test_tbl
2 select row_number() over(partition by name order by 1) no, name, rpad('*',100,'*') old_value, rpad('*',100,'*') new_value
3 from (select 'A'mod(level,100) name from dual connect by level <=1000);

1000 rows created.

SQL> commit;

Commit complete.

Got the statistics gathered.

SQL> exec dbms_stats.gather_table_stats(user,'hx_index_test_tbl',method_opt=>'for all indexed columns');

PL/SQL procedure successfully completed.

SQL> set serveroutput on
SQL> set autotrace traceonly explain
SQL> select * from hx_index_test_tbl where name = 'A0';

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=5 Card=10 Bytes=1000)
1 0 TABLE ACCESS (FULL) OF 'HX_INDEX_TEST_TBL' (Cost=5 Card=10 Bytes=1000)

Oops its going for a full scan. It’s not considering my index. I want it to pick up the index HX_INDEX_TEST_TBL_IDX_001.

SQL> select * from hx_index_test_tbl where name = 'A0' and no = 1;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=100)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'HX_INDEX_TEST_TBL' (Cost=2 Card=1 Bytes=100)
2 1 INDEX (RANGE SCAN) OF 'HX_INDEX_TEST_TBL_IDX' (NON-UNIQUE) (Cost=1 Card=1)

But when I add no in the where clause its considering the index why is it so???

The reason is very simple. See how I have gathered statistics.

dbms_stats.gather_table_stats(user,'hx_index_test_tbl',method_opt=>'for all indexed columns');

I forgot to gather for Index of this table. Now let me gather for the index of this table also.

SQL> EXEC dbms_stats.gather_table_stats(user,'hx_index_test_tbl',method_opt=>'for all indexed columns', cascade=>TRUE);

PL/SQL procedure successfully completed.

Now see what happens...

SQL> select * from hx_index_test_tbl where name = 'A0';

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=10 Bytes=1000)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'HX_INDEX_TEST_TBL' (Cost=2 Card=10 Bytes=1000)
2 1 INDEX (RANGE SCAN) OF 'HX_INDEX_TEST_TBL_IDX_001' (NON-UNIQUE) (Cost=1 Card=10)


So if you don’t gather stats properly with cost based optimizer you will get weird execution plans.

String List

Being a PLSQL developer many a times I have been in a situation where I would get list of values as input in a string and have to check if they exist in a particular table. The values would be separated by a delimiter. Let’s consider the below example

String Input: ‘ORACLE, SQL SERVER, DB2, MYSQL, SYBASE’

I have to use this string to see if these values exist in the table.

One possible way is to build a dynamic SQL. Just pad single quotes before and after each value in the string and build a SQL with the IN clause. But I want to do it without a dynamic SQL. How is this possible?

I came up with this SQL which splits this string and gives me the values in the string as rows.

Here is the SQL…

SQL> var a varchar2(100)

SQL> exec :a := 'ORACLE, SQL SERVER, DB2, MYSQL, SYBASE'

PL/SQL procedure successfully completed.

SQL>  SELECT TRIM(SUBSTR(:a, INSTR(','||:a,',',1,level), INSTR(:a||',',',',1,level) - INSTR(','||:a,',',1,level))) my_list
  2        FROM dual, (SELECT length(replace(translate(lower(:a)||',','abcdefghijklmnopqrstuvwxyz1234567890',' '),' ','')) cnt
  3                                    FROM dual)
  4  CONNECT BY level <= cnt
  5  /

MY_LIST
--------------------------------------------------------------------------------
ORACLE
SQL SERVER
DB2
MYSQL
SYBASE

So what have I done here? Let me split it into steps for your understanding.

1. I am identifying the possible letters that would appear in the values and translating them into blank space. Following command does that for me.

SQL> SELECT translate(lower(:a)||',','abcdefghijklmnopqrstuvwxyz1234567890',' ') val
   2       FROM dual;

VAL
--------------------------------------------------------------------------------
 ,  , , ,  ,

Now you can see I don’t have anything other than comma (The delimiter).

2. Now I am going to remove all the blank space from the output that I got in step 1.

SQL> SELECT replace(translate(lower(:a)||',','abcdefghijklmnopqrstuvwxyz1234567890',' '),' ','') val         
   2      FROM dual;

VAL
--------------------------------------------------------------------------------
,,,,,

Now the length of the output will give me the total number of values in the list. Note that I have padded a extra comma to the string as the number of delimiter is always less than the number of values by one.

3. So the third step is obviously to count the number of comma in the above input and determine total number of values in the string list.

SQL> SELECT length(replace(translate(lower(:a)||',','abcdefghijklmnopqrstuvwxyz1234567890',' '),' ','')) cnt
  2       FROM dual
  3  /

       CNT
----------
            5

4. Next step is to generate the string list for 5 times in rows using the CONNECT BY clause.

SQL> SELECT level, :a my_list
  2       FROM dual, (SELECT length(replace(translate(lower(:a)||',','abcdefghijklmnopqrstuvwxyz1234567890',' '),' ','')) cnt
  3                               FROM dual)
  4  CONNECT BY level <= cnt
  5  /

     LEVEL MY_LIST
---------- ---------------------------------------------------------------------------------------------------------------
         1 ORACLE, SQL SERVER, DB2, MYSQL, SYBASE
         2 ORACLE, SQL SERVER, DB2, MYSQL, SYBASE
         3 ORACLE, SQL SERVER, DB2, MYSQL, SYBASE
         4 ORACLE, SQL SERVER, DB2, MYSQL, SYBASE
         5 ORACLE, SQL SERVER, DB2, MYSQL, SYBASE

5. Now its simple with the help of SUBSTR and INSTR I can split each values as shown below.

SQL>  SELECT TRIM(SUBSTR(:a, INSTR(','||:a,',',1,level), INSTR(:a||',',',',1,level) - INSTR(',':a,',',1,level))) my_list
  2        FROM dual, (SELECT length(replace(translate(lower(:a)',','abcdefghijklmnopqrstuvwxyz1234567890',' '),' ','')) cnt
  3                            FROM dual)
  4  CONNECT BY level <= cnt
  5  /

MY_LIST
--------------------------------------------------------------------------------
ORACLE
SQL SERVER
DB2
MYSQL
SYBASE

So now I can use this SQL to convert any value list into rows and use in my SQL statements.

And if you can use Regular expression things can be much more easier.

Check this out.

SQL>  SELECT trim(regexp_substr(:a,'[^,]+', 1, level)) my_list
  2     FROM dual
  3  CONNECT BY level <= NVL( LENGTH( REGEXP_REPLACE( :a, '[^,]+', NULL ) ), 0 ) + 1
  4  /

MY_LIST
--------------------------------------------------------------------------------
ORACLE
SQL SERVER
DB2
MYSQL
SYBASE

NOTE: This might need have to be customized based on the given requirement.

Code reusability in oracle pl/sql

Again today when checking out tom’s blog I lead me to this post in Tayler Muth’s Blog. This sounds to be an interesting topic. So I thought for a while and this is what came up to my mind…

Code reusability in oracle pl/sql

First to start with pl/sql is procedural language which is not object oriented. Once said that it implies that this language is not familiar with the concept of inheritance. I have been using pl/sql for quite some time. It’s a great programming language. It would be even better if it had the class concept in it.

Class concept would come handy when we want to build a framework which can be utilized over and over.

Currently I am working on a masking utility. We have different masking algorithms say…

Algorithm 1, Algorithm 2, Algorithm 3 etc.

The general framework of this masking is something like this.

1. Get the field from user which needs to be masked.
2. Get the master table for that field.
3. Load the distinct data for the field from master table into a staging table
4. Apply your masking logic on the staging table.
5. Find all the transaction tables that have this field in it.
6. Using the staging table update all the transaction table.

Here this is a very good candidate for class concept where I can have a parent class which will have the entire frame work of what I have given in six steps. And I can have one child class for each algorithm and in each child class I can just over write the step four where I can apply my masking algorithm. But unfortunately we don’t have it. Hence I went for package, Use some CASE statements and hooked in my algorithms and made it as optimal as possible. I basically hate writing lot of code. If Oracle can introduce OOP(Object Oriented Programming) in PL/SQL it would be of great use and even more flexible.

Read Only Table.

I saw this post on Richard foote's blog which pointed me to this otn thread.

So i gave it a try. This is what i did.

Say I have a table as given below in production.

SQL> CREATE TABLE hx_my_read_only_table (no INTEGER CHECK(NO>0), name VARCHAR2(100));

Table created.

SQL> INSERT INTO hx_my_read_only_table VALUES(1,'KARTHICK');

1 row created.

SQL> INSERT INTO hx_my_read_only_table VALUES(2,'VIMAL');

1 row created.

SQL> INSERT INTO hx_my_read_only_table VALUES(3,'VIJAY');

1 row created.

SQL> COMMIT;

Commit complete.

SQL> SELECT * FROM hx_my_read_only_table;

NO NAME
---------- --------------------------------------------------------------------------------
1 KARTHICK
2 VIMAL
3 VIJAY

And I want to make it read only. I will use ACCESS CONTROL to achieve it.

SQL> CREATE CONTEXT hx_my_context USING hx_my_pkg;

Context created.

SQL> CREATE OR REPLACE PACKAGE hx_my_pkg
2 AS
3 PROCEDURE hx_set_context (
4 hx_p_fld VARCHAR2,
5 hx_p_value VARCHAR2);
6
7 FUNCTION hx_get_val (
8 hx_p_fld varchar2) RETURN NUMBER;
9
10 FUNCTION hx_get_key(
11 p_schema in varchar2,
12 p_object in varchar2) RETURN VARCHAR2;
13 END hx_my_pkg;
14 /
Package created.
SQL> CREATE OR REPLACE PACKAGE BODY hx_my_pkg
2 AS
3 PROCEDURE hx_set_context (
4 hx_p_fld VARCHAR2,
5 hx_p_value VARCHAR2)
6 IS
7 BEGIN
8 DBMS_SESSION.SET_CONTEXT('HX_MY_CONTEXT', hx_p_fld, hx_p_value);
9 END;
10
11 FUNCTION hx_get_val (
12 hx_p_fld varchar2) RETURN NUMBER
13 IS
14 BEGIN
15 RETURN SYS_CONTEXT('HX_MY_CONTEXT', hx_p_fld);
16 END;
17
18 FUNCTION hx_get_key(
19 p_schema in varchar2,
20 p_object in varchar2) RETURN VARCHAR2
21 IS
22 lPredicate VARCHAR2 (2000);
23 BEGIN
24 lPredicate := 'NO = SYS_CONTEXT(''HX_MY_CONTEXT'', ''NO'')';
25
26 RETURN lPredicate;
27 END;
28
29 END hx_my_pkg;
30 /

Package body created.

SQL> BEGIN
2 DBMS_RLS.ADD_POLICY (USER, 'HX_MY_READ_ONLY_TABLE', 'HX_READ_ONLY_POLICY',USER, 'hx_my_pkg.hx_get_key', 'INSERT, UPDATE, DELETE' );
3 END;
4 /

PL/SQL procedure successfully completed.

SQL> BEGIN
2 hx_my_pkg.hx_set_context('NO',0);
3 END;
4 /

PL/SQL procedure successfully completed.

SQL> UPDATE hx_my_read_only_table
2 SET NAME = 'RICHARD'
3 WHERE NO = 1;

0 rows updated.

SQL> DELETE FROM hx_my_read_only_table;

0 rows deleted.

SQL> INSERT INTO hx_my_read_only_table VALUES(4,'RICHARD');

1 row created.

I was expecting this one not to insert record. I have to see what the problem is.

SQL> ROLLBACK;

Rollback complete.

Mean while I restricted the insert by with this trigger. I personally hate triggers but....

SQL> CREATE OR REPLACE TRIGGER hx_my_trigger BEFORE INSERT ON hx_my_read_only_table FOR EACH ROW
2 BEGIN
3 IF :NEW.NO <> hx_my_pkg.hx_get_val('NO')
4 THEN
5 RAISE_APPLICATION_ERROR(-20001, 'Insert Not possible...');
6 END IF;
7 END;
8 /

Trigger created.

SQL> INSERT INTO hx_my_read_only_table VALUES(4,'RICHARD');

INSERT INTO hx_my_read_only_table VALUES(4,'RICHARD')
*
ERROR at line 1:
ORA-20001: Insert Not possible...
ORA-06512: at "SYSADM.HX_MY_TRIGGER", line 4
ORA-04088: error during execution of trigger 'SYSADM.HX_MY_TRIGGER'

So i have Restricted INSERT, UPDATE, DELETE on this table.

SQL> SELECT * FROM HX_MY_READ_ONLY_TABLE;

NO NAME
---------- --------------------------------------------------------------------------------
1 KARTHICK
2 VIMAL
3 VIJAY

The bridge is falling down.

Few days back tom wrote this in his blog. I felt there is lot of truth in it and was very happy to hear it from him. Today when I came into office the first mail I came across had these pictures.

This one is an unacceptable construction flaw. The mail said it has happened in Korea. Such thing may be very… very… rare event in construction business. But here in software we see lots of bridges falling down every day. After seeing this I have started to worry even more about what he said.

DML Error Logging in 10gR2

This is a cool stuff in oracle. I just tried it few days back and I was exited so thought of sharing it here.

Have you ever stuck up with a bulk insert failing on a check constraint? In such a situation we spend most of our time trying to find out the culprit rows. Oracle has come up with a great stuff by which such rows can be logged separately without disturbing the valid rows.

To put it straight by using the DML Error Logging feature you can just pick up all the invalid rows and put it separately and let the valid records go into the target table.Following is a small demonstration of this feature.

This is the source table from which the data is going to be selected.

SQL> CREATE TABLE SOURCE(NO NUMBER, NAME VARCHAR2(100))
2 /

Table created.

SQL> INSERT INTO SOURCE SELECT LEVEL, 'KARTHICK' FROM DUAL CONNECT BY LEVEL <=15
2 /

15 rows created.

This is the target table into which the data from the source table is going to be inserted.

SQL> CREATE TABLE TARGET(NO NUMBER, NAME VARCHAR2(100))
2 /

Table created.

I am creating a check constraint as follows

SQL> ALTER TABLE TARGET ADD CONSTRAINT SOURCE_CHK CHECK(NO <10)
2 /

Table altered.

Now let’s try to insert the source table data into target table.

SQL> INSERT INTO TARGET SELECT * FROM SOURCE;
INSERT INTO TARGET SELECT * FROM SOURCE
*
ERROR at line 1:
ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated

As we have a check constraint on column TARGET.NO which says NO should be less than 10, if we have any row which does not satisfy this condition then the entire insert fails. But I want the valid rows to be inserted into my target table and all the invalid rows to be logged separately. I am going to achieve this using the DML Error Logging technique.

To achieve that first we need to create a Error Log Table. The following command helps us in doing that.

SQL> BEGIN
2 DBMS_ERRLOG.create_error_log (dml_table_name => 'TARGET');
3 END;
4 /

PL/SQL procedure successfully completed.

This creates a new error log table for our TARGET table. The error log table is described below

SQL> desc err$_TARGET;

Name Null? Type
----------------------------------------- -------- ----------------------------

ORA_ERR_NUMBER$ NUMBER
ORA_ERR_MESG$ VARCHAR2(2000)
ORA_ERR_ROWID$ ROWID
ORA_ERR_OPTYP$ VARCHAR2(2)
ORA_ERR_TAG$ VARCHAR2(2000)
NO VARCHAR2(4000)
NAME VARCHAR2(4000)

Now we need to use LOG ERROR INTO syntax in our INSERT INTO statement to log the error.

SQL> INSERT INTO TARGET SELECT * FROM SOURCE LOG ERRORS INTO err$_TARGET ('INSERT') REJECT LIMIT UNLIMITED
2 /

9 rows created.

Now you can observe that of the 15 rows 9 rows that satisfy the check constraints are inserted into the TARGET table.So the invalid records will be logged into the Error Log table that we have created above. Let’s have a look into the error log table.

SELECT * FROM ERR$_TARGET

ORA_ERR_NUMBER$ ORA_ERR_MESG$ ORA_ERR_ROWID$ ORA_ERR_OPTYP$ ORA_ERR_TAG$ NO NAME
--------------------------------------------------------------------------------------------------------------------------------------------
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 10 KARTHICK
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 11 KARTHICK
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 12 KARTHICK
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 13 KARTHICK
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 14 KARTHICK
2290 ORA-02290: check constraint (SYSADM.SOURCE_CHK) violated I INSERT 15 KARTHICK

Now which this information we can easily start our trouble shooting to figure out why the rows failed the check constraint.

Do i need to repel?

I saw this posting by tom today. I felt like commenting it. And this is what i wrote.

It is not most of the time "I don't know the answer" but its "I don't know the question".

What I want?

Almost and always in the programming world it’s not what I want, it’s what they want me to want. Unclear requirements we see it every day don't we.

Why such questions?

The inability to question the authority is what leads to such questions. Some of the questions asked may not be looking for the answer but may be in the phase of trying to understand what is needed. I know it’s a wrong (very... very...) approach. But in the practical world that’s the way it is.

Don’t you here it?

Do what I say...
Don't do it that way, Do it my way...

Do I need to repel?

Is it right to say what you ask for just does not make any sense to me. If I tell that I know the next question (if the person sitting opposite to you is a genuine person) would be "Then tell me how it should be?” Do we really have answer at that moment is what matters. If we have then we may find a way out. But most of the time we know that something is wrong but we don’t know what is right. To find the right thing we need time. And that is what most of them don’t have with them. For all this as I said earlier the person should be genuine. What if he or she is a bossy person? It’s simple..."You are screwed up".

So I feel such question is born not because "I don’t know what I want" but because "I don’t know what they want me to want".