Oracle SQL
  • LICENSE

Tag Archives: SQL*Plus

:1 and SP2-0553: Illegal variable name “1”.

Posted on September 7, 2021 by Sayan Malakshinov Posted in SQL, SQL*Plus, SQL*PLus tips, SqlCL 2,231 Page views Leave a comment

You may know that some applications generate queries with bind variables’ names like :1 or :”1″, and neither SQL*Plus nor SQLCl support such variables:

SQLPlus:

SQL> var 1 number;
SP2-0553: Illegal variable name "1".

SQLCL:

SQL> var 1 number;
ILLEGAL Variable Name "1"

So we can’t run such queries as-is, but, obviously, we can wrap them into anonymous PL/SQL blocks and even create a special script for that:

Continue reading→
bind variable sql SQL*Plus SqlCL

SQL*Plus tips #9: Reading traces and incident files

Posted on May 28, 2021 by Sayan Malakshinov Posted in diagnostic event 10046, oracle, SQL*Plus, SQL*PLus tips, trace, troubleshooting 2,631 Page views Leave a comment

@tracefile_read_last_by_mask filemask [regexp] [ignore_regexp]
– finds last trace by filemask and filters rows by regexp and filters out rows by ignore_regexp:

@ tracefile_by_mask.sql [mask] – finds and shows last 10 trace files by mask

Continue reading→
SQL*Plus sqlplus traces

Format SQL or PL/SQL directly in Oracle database

Posted on December 23, 2020 by Sayan Malakshinov Posted in java, oracle, PL/SQL, SQL, SQL*Plus, SQL*PLus tips 2,954 Page views Leave a comment

Obviously we can format/beautify SQL or PL/SQL code using external tools, but sometimes it would be more convenient to format it directly in database, for example if we want to see few different sql_text’s from v$sqlarea. And thanks to Oracle SqlCL and Oracle SQL Developer, we can easily use oracle.dbtools.app.Format function from their Java library dbtools-common.jar, so if you use SqlCL or SQL Developer, you can use the same formatting options.

1. load appropriate java library into Oracle

You may have already installed SQLDeveloper or SqlCL on your database server, just check $ORACLE_HOME/sqldeveloper or $ORACLE_HOME/sqcl directories. If – not, you need to download appropriate SqlCL version that matches your java version in Oracle. For 12.2 – 19.8 you can download latest SqlCL 20.3. In fact we need just dbtools-common.jar from lib directory. I put it into $ORACLE_HOME/sqlcl/lib directory on the server and load it using loadjava:

Continue reading→
beautifier pl/sql formatter sql format SQL*Plus sqlformatter

Top time-consuming predicates from ASH

Posted on May 13, 2019 by Sayan Malakshinov Posted in oracle, query optimizing, SQL, statistics, troubleshooting 1,870 Page views Leave a comment

Sometimes it might be useful to analyze top time-consuming filter and access predicates from ASH, especially in cases when db load is spread evenly enough by different queries and top segments do not show anything interesting, except usual things like “some tables are requested more often than others”.
Of course, we can start from analysis of SYS.COL_USAGE$: col_usage.sql

col_usage.sql

[sourcecode language=”sql”]
col owner format a30
col oname format a30 heading "Object name"
col cname format a30 heading "Column name"
accept owner_mask prompt "Enter owner mask: ";
accept tab_name prompt "Enter tab_name mask: ";
accept col_name prompt "Enter col_name mask: ";

SELECT a.username as owner
,o.name as oname
,c.name as cname
,u.equality_preds as equality_preds
,u.equijoin_preds as equijoin_preds
,u.nonequijoin_preds as nonequijoin_preds
,u.range_preds as range_preds
,u.like_preds as like_preds
,u.null_preds as null_preds
,to_char(u.timestamp, ‘yyyy-mm-dd hh24:mi:ss’) when
FROM
sys.col_usage$ u
, sys.obj$ o
, sys.col$ c
, all_users a
WHERE a.user_id = o.owner#
AND u.obj# = o.obj#
AND u.obj# = c.obj#
AND u.intcol# = c.col#
AND a.username like upper(‘&owner_mask’)
AND o.name like upper(‘&tab_name’)
AND c.name like upper(‘&col_name’)
ORDER BY a.username, o.name, c.name
;
col owner clear;
col oname clear;
col cname clear;
undef tab_name col_name owner_mask;
[/sourcecode]

[collapse]

But it’s not enough, for example it doesn’t show predicates combinations. In this case we can use v$active_session_history and v$sql_plan:

Top 50 predicates

[sourcecode language=”sql”]
with
ash as (
select
sql_id
,plan_hash_value
,table_name
,alias
,ACCESS_PREDICATES
,FILTER_PREDICATES
,count(*) cnt
from (
select
h.sql_id
,h.SQL_PLAN_HASH_VALUE plan_hash_value
,decode(p.OPERATION
,’TABLE ACCESS’,p.OBJECT_OWNER||’.’||p.OBJECT_NAME
,(select i.TABLE_OWNER||’.’||i.TABLE_NAME from dba_indexes i where i.OWNER=p.OBJECT_OWNER and i.index_name=p.OBJECT_NAME)
) table_name
,OBJECT_ALIAS ALIAS
,p.ACCESS_PREDICATES
,p.FILTER_PREDICATES
— поля, которые могут быть полезны для анализа в других разрезах:
— ,h.sql_plan_operation
— ,h.sql_plan_options
— ,decode(h.session_state,’ON CPU’,’ON CPU’,h.event) event
— ,h.current_obj#
from v$active_session_history h
,v$sql_plan p
where h.sql_opname=’SELECT’
and h.IN_SQL_EXECUTION=’Y’
and h.sql_plan_operation in (‘INDEX’,’TABLE ACCESS’)
and p.SQL_ID = h.sql_id
and p.CHILD_NUMBER = h.SQL_CHILD_NUMBER
and p.ID = h.SQL_PLAN_LINE_ID
— если захотим за последние 3 часа:
— and h.sample_time >= systimestamp – interval ‘3’ hour
)
— если захотим анализируем предикаты только одной таблицы:
— where table_name=’&OWNER.&TABNAME’
group by
sql_id
,plan_hash_value
,table_name
,alias
,ACCESS_PREDICATES
,FILTER_PREDICATES
)
,agg_by_alias as (
select
table_name
,regexp_substr(ALIAS,’^[^@]+’) ALIAS
,listagg(ACCESS_PREDICATES,’ ‘) within group(order by ACCESS_PREDICATES) ACCESS_PREDICATES
,listagg(FILTER_PREDICATES,’ ‘) within group(order by FILTER_PREDICATES) FILTER_PREDICATES
,sum(cnt) cnt
from ash
group by
sql_id
,plan_hash_value
,table_name
,alias
)
,agg as (
select
table_name
,’ALIAS’ alias
,replace(access_predicates,’"’||alias||’".’,’"ALIAS".’) access_predicates
,replace(filter_predicates,’"’||alias||’".’,’"ALIAS".’) filter_predicates
,sum(cnt) cnt
from agg_by_alias
group by
table_name
,replace(access_predicates,’"’||alias||’".’,’"ALIAS".’)
,replace(filter_predicates,’"’||alias||’".’,’"ALIAS".’)
)
,cols as (
select
table_name
,cols
,access_predicates
,filter_predicates
,sum(cnt)over(partition by table_name,cols) total_by_cols
,cnt
from agg
,xmltable(
‘string-join(for $c in /ROWSET/ROW/COL order by $c return $c,",")’
passing
xmltype(
cursor(
(select distinct
nvl(
regexp_substr(
access_predicates||’ ‘||filter_predicates
,'("’||alias||’"\.|[^.]|^)"([A-Z0-9#_$]+)"([^.]|$)’
,1
,level
,’i’,2
),’ ‘)
col
from dual
connect by
level<=regexp_count(
access_predicates||’ ‘||filter_predicates
,'("’||alias||’"\.|[^.]|^)"([A-Z0-9#_$]+)"([^.]|$)’
)
)
))
columns cols varchar2(400) path ‘.’
)(+)
order by total_by_cols desc, table_name, cnt desc
)
select
table_name
,cols
,sum(cnt)over(partition by table_name,cols) total_by_cols
,access_predicates
,filter_predicates
,cnt
from cols
where rownum<=50
order by total_by_cols desc, table_name, cnt desc;
[/sourcecode]

[collapse]

As you can see it shows top 50 predicates and their columns for last 3 hours. Despite the fact that ASH stores just sampled data, its results are representative enough for high-load databases.
Just few details:

  • Column “COLS” shows “search columns”, and total_by_cols – their number of occurrences
  • I think it’s obvious, that this info is not unambiguous marker of the problem, because for example few full table scans can misrepresent the statistics, so sometimes you will need to analyze such queries deeper (v$sqlstats,dba_hist_sqlstat)
  • We need to group data by OBJECT_ALIAS within SQL_ID and plan_hash_value, because in case of index access with lookup to table(“table access by rowid”) some predicates are in the row with index access and others are in the row with table access.

Depending on the needs, we can modify this query to analyze ASH data by different dimensions, for example with additional analysis of partitioning or wait events.

oracle query optimization SQL*Plus troubleshooting

SQL*Plus tips #8: How to read the output of dbms_output without “serveroutput on”

Posted on December 10, 2017 by Sayan Malakshinov Posted in oracle, SQL*Plus, SQL*PLus tips 2,581 Page views 1 Comment

When “serveroutput” is enabled, SQL*Plus executes “BEGIN DBMS_OUTPUT.GET_LINES(:LINES, :NUMLINES); END;” after each command.
That’s why I don’t like when it is always enabled: it adds extra calls and round-trips and it is inconvenient when I want to get a plan of the last executed query:

SQL> set serverout on;
SQL> select * from dual;

D
-
X

SQL> select * from table(dbms_xplan.display_cursor('','','allstats last'));

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------
SQL_ID  9babjv8yq8ru3, child number 0

BEGIN DBMS_OUTPUT.GET_LINES(:LINES, :NUMLINES); END;

NOTE: cannot fetch plan for SQL_ID: 9babjv8yq8ru3, CHILD_NUMBER: 0
      Please verify value of SQL_ID and CHILD_NUMBER;
      It could also be that the plan is no longer in cursor cache (check v$sql_plan)

So usually I switch “serveroutput” on only if needed, but sometimes I can forget to enable it. In such cases I use very simple script that reads the output using dbms_output.get_lines and prints it using refcursor:
https://github.com/xtender/xt_scripts/blob/master/output_print.sql

When you set “serveroutput on“, SQL*Plus also executes “dbms_output.enable” and if you set “serverout off” it executes “dbms_output.disable”, that’s why my glogin.sql contains “call dbms_output.enable(1e6);” and you need to execute it after each “set serverout off” if you want to use this script.

oracle SQL*Plus

SQL*Plus tips #7: How to find the current script directory

Posted on June 26, 2015 by Sayan Malakshinov Posted in oracle, SQL*Plus, SQL*PLus tips 3,665 Page views 4 Comments

You know that if we want to execute another script from the current script directory, we can call it through @@, but sometimes we want to know the current path exactly, for example if we want to spool something into the file in the same directory.
Unfortunately we cannot use “spool @spoolfile”, but it is easy to find this path, because we know that SQL*Plus shows this path in the error when it can’t to find @@filename.

So we can simply get this path from the error text:

rem Simple example how to get path (@@) of the current script.
rem This script will set "cur_path" variable, so we can use &cur_path later.
 
set termout off
spool _cur_path.remove
@@notfound
spool off;
 
var cur_path varchar2(100);
declare 
  v varchar2(100);
  m varchar2(100):='SP2-0310: unable to open file "';
begin v :=rtrim(ltrim( 
                        q'[
                            @_cur_path.remove
                        ]',' '||chr(10)),' '||chr(10));
  v:=substr(v,instr(v,m)+length(m));
  v:=substr(v,1,instr(v,'notfound.')-1);
  :cur_path:=v;
end;
/
set scan off;
ho (rm _cur_path.remove 2>&1  | echo .)
ho (del _cur_path.remove 2>&1 | echo .)
col cur_path new_val cur_path noprint;
select :cur_path cur_path from dual;
set scan on;
set termout on;
 
prompt Current path: &cur_path

I used here the reading file content into variable, that I already showed in the “SQL*Plus tips. #1”.
UPDATE: I’ve replaced this script with a cross platform version.

Also I did it with SED and rtrim+ltrim, because 1) I have sed even on windows; and 2) I’m too lazy to write big PL/SQL script that will support 9i-12c, i.e. without regexp_substr/regexp_replace, etc.
But of course you can rewrite it without depending on sed, if you use windows without cygwin.

PS. Note that “host pwd” returns only directory where SQL*Plus was started, but not executed script directory.

Download latest version

SQL*Plus

Just a couple of screenshots of sqlplus+rlwrap+cygwin+console

Posted on October 16, 2014 by Sayan Malakshinov Posted in oracle, SQL*Plus 2,362 Page views 2 Comments

I previously wrote that I peeped the idea about showing the session information in terminal title from Timur Akhmadeev’s screenshots, and Timur wrote:

I’m using (a bit modified) Tanel Poder’s login.sql available in his TPT scripts library: http://tech.e2sn.com/oracle-scripts-and-tools

Scripts:
Tanel’s i.sql
My title.sql and on_login.sql
View screenshots

SQL*Plus

Just another SQL beautifier

Posted on October 30, 2013 by Sayan Malakshinov Posted in oracle, SQL*Plus, SQL*PLus tips 3,264 Page views 6 Comments

Previously i wrote beautifier in perl, but it was not so good, so i decided to write it in java using popular BlancoSQLFormatter library.
So you can download it now: https://orasql.org/scripts/SQLBeautifier.jar
Usage:

java -jar SQLBeautifier.jar your_file.sql

or

echo select * from dual | java -jar SQLBeautifier.jar

You certainly can conveniently use it within sql*plus with script like that:

set timing off head off termout off
col qtext format a150
prompt ################################  Original query text:  ################################################;
#spool &_SPOOLS/to_format.sql
spool to_format.sql
select
    coalesce(
        (select sql_fulltext from v$sqlarea a where a.sql_id='&1')
    ,   (select sql_text from dba_hist_sqltext a where a.sql_id='&1' and dbid=(select dbid from v$database))
    ) qtext
from dual
;
spool off

prompt ################################  Formatted query text #################################################;
#host perl inc/sql_format_standalone.pl &_SPOOLS/to_format.sql
host java -jar SQLBeautifier.jar to_format.sql
prompt ################################  Formatted query text End #############################################;
set termout on head on

Example:
beautifier2

Links:

  • Download SQLBeautifier.jar
  • Source code(Git repo)
  • BlancoSQLFormatter library
  • SQL*Plus script example
SQL*Plus

SQL*Plus tips #6: Colorizing output

Posted on May 22, 2013 by Sayan Malakshinov Posted in oracle, SQL*Plus, SQL*PLus tips 5,921 Page views 4 Comments

If you have seen a colored scripts like a fish from “Session Snapper v.4” by Tanel Poder or OraLatencyMap by Luca Canali, you may be also want to colorize your scripts.
I’ve created the script for this purposes with a set of predefined substitution variables.
Just download colors.sql and try this script:

@colors.sql;
prompt ::: &amp;_C_RED ***  TEST PASSED  *** &amp;_C_RESET :::
prompt ::: &amp;_C_RED *** &amp;_C_BLINK TEST PASSED &amp;_C_BLINK_OFF *** &amp;_C_RESET :::

You will get something like this:
sqltips6
All these variables have a prefix “_C” and “_CB” for background colors.

Do not forget to close text attributes, for example: &_C_BOLD; bold text &_C_BOLD_OFF; enables “bold” attribute and disables it after “bold text”.

Full list of the predefined variables:

Predefined variables

Description Variable
_C_RESET Reset formatting – Turn off all attributes
_C_BOLD
_C_BOLD_OFF
Set bright mode on/off
_C_UNDERLINE
_C_UNDERLINE_OFF
Set underline mode on/off
_C_BLINK
_C_BLINK_OFF
Set blink mode on/off
_C_REVERSE
_C_REVERSE_OFF
Exchange foreground and background colors
_C_HIDE
_C_HIDE_OFF
Hide text (foreground color would be the same as background)
_C_BLACK
_C_RED
_C_GREEN
_C_YELLOW
_C_BLUE
_C_MAGENTA
_C_CYAN
_C_WHITE
_C_DEFAULT
Font colors
_CB_BLACK
_CB_RED
_CB_GREEN
_CB_YELLOW
_CB_BLUE
_CB_MAGENTA
_CB_CYAN
_CB_WHITE
_CB_DEFAULT
Background colors

[collapse]

In addition, I want to show a simple example of printing histograms.
We can print a simple histogram using the following query:

-- loading colors variables:
@inc/colors;
-- set max length of bar:
def _max_length=80;
-- columns formatting:
col bar format a&_max_length;
-- clear screen:
prompt &_CLS
with t as (-- it's just a test values for example:
            select level id
                 , round(dbms_random.value(1,100)) val
            from dual
            connect by level<=10
          )
select t.*
      -- bar length is just " (value / max_value) * max_length" in symbols:
      ,floor( val * &_max_length / max(val)over()
            ) as bar_length
      -- generating of bar:
      ,lpad( chr(176)
            ,ceil(val * &_max_length / max(val)over())
            ,chr(192)
           ) as bar
from t;
sqltips6-1

And now we can colorize it:

Colorized script

[sourcecode language="sql"]
-- loading colors variables:
@inc/colors;
-- set max length of bar:
def _max_length=100;
-- column formatting
col bar format a&amp;amp;_max_length;
-- clear screen:
prompt &amp;amp;_CLS

-- test query which prints histogram(or may be simply bars?):
with t as (-- it's just a test values for example:
            select level id
                 , round(dbms_random.value(1,100)) val
            from dual
            connect by level&amp;lt;=10
          )
select
       id
      ,val
      , case
           when pct &amp;gt;= 0.9 then '&amp;amp;_C_RED'
           when pct &amp;lt;= 0.4 then '&amp;amp;_C_GREEN'
           else '&amp;amp;_C_YELLOW'
        end
        -- string generation:
      ||lpad( chr(192)
             ,ceil(pct * &amp;amp;_max_length)-9 -- color - 5 chars and reset - 4
             ,chr(192)
            )
      ||'&amp;amp;_C_RESET'
       as bar
from (
     select
        t.*
       ,val / max(val)over() as pct -- as a percentage of max value:
     from t
     ) t2
/
[/sourcecode]

[collapse]
sqltips6-2
SQL*Plus

SQL*Plus tips #5: sql_text/sql_fulltext formatting(sql beatifier)

Posted on April 29, 2013 by Sayan Malakshinov Posted in oracle, SQL*Plus, SQL*PLus tips 7,903 Page views 13 Comments

Sometimes I get tired of watching unformatted query text from v$sqlarea, dba_hist_sqltext in SQL*Plus, so I decided to include automatic query formatting in my scripts.
I thought that there are many libraries for such purposes on languages which i know, and it’s true, but it turned out that many of them are not appropriate for Oracle.
So I took the most appropriate – perl module SQL::Beautify and corrected it. Now i can share my first solution.

How it looks:
sql_textf

What you need to do it:
1. If you on Windows and you have not install Oracle RDBMS or cygwin, you need to install perl. It can be done for example with ActivePerl or StrawberryPerl, but i recommend cygwin
2. You need to save sql_format_standalone.pl within $SQL_PATH/inc directory.

inc/sql_format_standalone.pl

[sourcecode language=”sql”]
package OraTokenizer;

use warnings;
use strict;

use 5.006002;

use Exporter;

our @ISA = qw(Exporter);

our @EXPORT_OK= qw(tokenize_sql);

our $VERSION= ‘0.24’;

my $re= qr{
(
(?:–)[\ \t\S]* # single line comments
|
(?:<>|<=>|>=|<=|==|=|!=|!|<<|>>|<|>|\|\||\||&&|&|-|\+|\*(?!/)|/(?!\*)|\%|~|\^|\?)
# operators and tests
|
\(\+\)
# oracle join
|
[\[\]\(\),;.] # punctuation (parenthesis, comma)
|
\’\'(?!\’) # empty single quoted string
|
\"\"(?!\"") # empty double quoted string
|
"(?>(?:(?>[^"\\]+)|""|\\.)*)+"
# anything inside double quotes, ungreedy
|
`(?>(?:(?>[^`\\]+)|“|\\.)*)+`
# anything inside backticks quotes, ungreedy
|
‘(?>(?:(?>[^’\\]+)|”|\\.)*)+’
# anything inside single quotes, ungreedy.
|
/\*[\ \t\r\n\S]*?\*/ # C style comments
|
(?:[\w:@]+(?:[.\$](?:\w+|\*)?)*)
# words, standard named placeholders, db.table.*, db.*
|
\n # newline
|
[\t\ ]+ # any kind of white spaces
)
}smx;

sub tokenize_sql {
my ( $query, $remove_white_tokens )= @_;

my @query= $query =~ m{$re}smxg;

if ($remove_white_tokens) {
@query= grep( !/^[\s\n\r]*$/, @query );
}

return wantarray ? @query : \@query;
}

sub tokenize {
my $class= shift;
return tokenize_sql(@_);
}

1;

=pod

=head1 NAME

OraTokenizer – A simple SQL tokenizer.

=head1 VERSION

0.20

=head1 SYNOPSIS

use OraTokenizer qw(tokenize_sql);

my $query= q{SELECT 1 + 1};
my @tokens= OraTokenizer->tokenize($query);

# @tokens now contains (‘SELECT’, ‘ ‘, ‘1’, ‘ ‘, ‘+’, ‘ ‘, ‘1’)

@tokens= tokenize_sql($query); # procedural interface

=head1 DESCRIPTION

OraTokenizer is a simple tokenizer for SQL queries. It does not claim to be
a parser or query verifier. It just creates sane tokens from a valid SQL
query.

It supports SQL with comments like:

— This query is used to insert a message into
— logs table
INSERT INTO log (application, message) VALUES (?, ?)

Also supports C<”>, C<""> and C<\’> escaping methods, so tokenizing queries
like the one below should not be a problem:

INSERT INTO log (application, message)
VALUES (‘myapp’, ‘Hey, this is a ”single quoted string”!’)

=head1 API

=over 4

=item tokenize_sql

use OraTokenizer qw(tokenize_sql);

my @tokens= tokenize_sql($query);
my $tokens= tokenize_sql($query);

$tokens= tokenize_sql( $query, $remove_white_tokens );

C<tokenize_sql> can be imported to current namespace on request. It receives a
SQL query, and returns an array of tokens if called in list context, or an
arrayref if called in scalar context.

=item tokenize

my @tokens= OraTokenizer->tokenize($query);
my $tokens= OraTokenizer->tokenize($query);

$tokens= OraTokenizer->tokenize( $query, $remove_white_tokens );

This is the only available class method. It receives a SQL query, and returns an
array of tokens if called in list context, or an arrayref if called in scalar
context.

If C<$remove_white_tokens> is true, white spaces only tokens will be removed from
result.

=back

=head1 ACKNOWLEDGEMENTS

=over 4

=item

Evan Harris, for implementing Shell comment style and SQL operators.

=item

Charlie Hills, for spotting a lot of important issues I haven’t thought.

=item

Jonas Kramer, for fixing MySQL quoted strings and treating dot as punctuation character correctly.

=item

Emanuele Zeppieri, for asking to fix OraTokenizer to support dollars as well.

=item

Nigel Metheringham, for extending the dollar signal support.

=item

Devin Withers, for making it not choke on CR+LF in comments.

=item

Luc Lanthier, for simplifying the regex and make it not choke on backslashes.

=back

=head1 AUTHOR

Copyright (c) 2007, 2008, 2009, 2010, 2011 Igor Sutton Lopes "<IZUT@cpan.org>". All rights
reserved.

This module is free software; you can redistribute it and/or modify it under
the same terms as Perl itself.

=cut

###################################

package OraBeautify;

use strict;
use warnings;

our $VERSION = 0.04;

use Carp;

# Keywords from SQL-92, SQL-99 and SQL-2003.
use constant KEYWORDS => qw(
ABSOLUTE ACTION ADD AFTER ALL ALLOCATE ALTER AND ANY ARE ARRAY AS ASC
ASENSITIVE ASSERTION ASYMMETRIC AT ATOMIC AUTHORIZATION AVG BEFORE BEGIN
BETWEEN BIGINT BINARY BIT BIT_LENGTH BLOB BOOLEAN BOTH BREADTH BY CALL
CALLED CASCADE CASCADED CASE CAST CATALOG CHAR CHARACTER CHARACTER_LENGTH
CHAR_LENGTH CHECK CLOB CLOSE COALESCE COLLATE COLLATION COLUMN COMMIT
CONDITION CONNECT CONNECTION CONSTRAINT CONSTRAINTS CONSTRUCTOR CONTAINS
CONTINUE CONVERT CORRESPONDING COUNT CREATE CROSS CUBE CURRENT CURRENT_DATE
CURRENT_DEFAULT_TRANSFORM_GROUP CURRENT_PATH CURRENT_ROLE CURRENT_TIME
CURRENT_TIMESTAMP CURRENT_TRANSFORM_GROUP_FOR_TYPE CURRENT_USER CURSOR
CYCLE DATA DATE DAY DEALLOCATE DEC DECIMAL DECLARE DEFAULT DEFERRABLE
DEFERRED DELETE DEPTH DEREF DESC DESCRIBE DESCRIPTOR DETERMINISTIC
DIAGNOSTICS DISCONNECT DISTINCT DO DOMAIN DOUBLE DROP DYNAMIC EACH ELEMENT
ELSE ELSEIF END EPOCH EQUALS ESCAPE EXCEPT EXCEPTION EXEC EXECUTE EXISTS
EXIT EXTERNAL EXTRACT FALSE FETCH FILTER FIRST FLOAT FOR FOREIGN FOUND FREE
FROM FULL FUNCTION GENERAL GET GLOBAL GO GOTO GRANT GROUP GROUPING HANDLER
HAVING HOLD HOUR IDENTITY IF IMMEDIATE IN INDICATOR INITIALLY INNER INOUT
INPUT INSENSITIVE INSERT INT INTEGER INTERSECT INTERVAL INTO IS ISOLATION
ITERATE JOIN KEY LANGUAGE LARGE LAST LATERAL LEADING LEAVE LEFT LEVEL LIKE
LIMIT LOCAL LOCALTIME LOCALTIMESTAMP LOCATOR LOOP LOWER MAP MATCH MAX
MEMBER MERGE METHOD MIN MINUTE MODIFIES MODULE MONTH MULTISET NAMES
NATIONAL NATURAL NCHAR NCLOB NEW NEXT NO NONE NOT NULL NULLIF NUMERIC
OBJECT OCTET_LENGTH OF OLD ON ONLY OPEN OPTION OR ORDER ORDINALITY OUT
OUTER OUTPUT OVER OVERLAPS PAD PARAMETER PARTIAL PARTITION PATH POSITION
PRECISION PREPARE PRESERVE PRIMARY PRIOR PRIVILEGES PROCEDURE PUBLIC RANGE
READ READS REAL RECURSIVE REF REFERENCES REFERENCING RELATIVE RELEASE
REPEAT RESIGNAL RESTRICT RESULT RETURN RETURNS REVOKE RIGHT ROLE ROLLBACK
ROLLUP ROUTINE ROW ROWS PIVOT UNPIVOT XMLTABLE XMLSEQUENCE XMLQUERY
SAVEPOINT SCHEMA SCOPE SCROLL SEARCH SECOND SECTION SEQUENCE
SELECT SENSITIVE SESSION SESSION_USER SET SETS SIGNAL SIMILAR SIZE SMALLINT
SOME SPACE SPECIFIC SPECIFICTYPE SQL SQLCODE SQLERROR SQLEXCEPTION SQLSTATE
SQLWARNING START STATE STATIC SUBMULTISET SUBSTRING SUM SYMMETRIC SYSTEM
SYSTEM_USER TABLE TABLESAMPLE TEMPORARY TEXT THEN TIME TIMESTAMP
TIMEZONE_HOUR TIMEZONE_MINUTE TINYINT TO TRAILING TRANSACTION TRANSLATE
TRANSLATION TREAT TRIGGER TRIM TRUE UNDER UNDO UNION UNIQUE UNKNOWN UNNEST
UNTIL UPDATE UPPER USAGE USER USING VALUE VALUES VARCHAR VARYING VIEW WHEN
WHENEVER WHERE WHILE WINDOW WITH WITHIN WITHOUT WORK WRITE YEAR ZONE
);

sub new {
my ($class, %options) = @_;

my $self = bless { %options }, $class;

# Set some defaults.
$self->{query} = ” unless defined($self->{query});
$self->{spaces} = 4 unless defined($self->{spaces});
$self->{space} = ‘ ‘ unless defined($self->{space});
$self->{break} = "\n" unless defined($self->{break});
$self->{wrap} = {} unless defined($self->{wrap});
$self->{keywords} = [] unless defined($self->{keywords});
$self->{rules} = {} unless defined($self->{rules});
$self->{uc_keywords} = 0 unless defined $self->{uc_keywords};

push @{$self->{keywords}}, KEYWORDS;

# Initialize internal stuff.
$self->{_level} = 0;

return $self;
}

# Add more SQL.
sub add {
my ($self, $addendum) = @_;

$addendum =~ s/^\s*/ /;

$self->{query} .= $addendum;
}

# Set SQL to beautify.
sub query {
my ($self, $query) = @_;

$self->{query} = $query if(defined($query));

return $self->{query};
}

# Beautify SQL.
sub beautify {
my ($self) = @_;

$self->{_output} = ”;
$self->{_level_stack} = [];
$self->{_new_line} = 1;

my $last;

$self->{_tokens} = [ OraTokenizer->tokenize($self->query, 1) ];

while(defined(my $token = $self->_token)) {
my $rule = $self->_get_rule($token);

# Allow custom rules to override defaults.
if($rule) {
$self->_process_rule($rule, $token);
}

elsif($token eq ‘(‘) {
$self->_add_token($token);
$self->_new_line;
push @{$self->{_level_stack}}, $self->{_level};
$self->_over unless $last and uc($last) eq ‘WHERE’;
}

elsif($token eq ‘)’) {
$self->_new_line;
$self->{_level} = pop(@{$self->{_level_stack}}) || 0;
$self->_add_token($token);
$self->_new_line;
}

elsif($token eq ‘,’) {
$self->_add_token($token);
$self->_new_line;
}

elsif($token eq ‘;’) {
$self->_add_token($token);
$self->_new_line;

# End of statement; remove all indentation.
@{$self->{_level_stack}} = ();
$self->{_level} = 0;
}

elsif($token =~ /^(?:SELECT|FROM|WHERE|HAVING)$/i) {
$self->_back unless $last and $last eq ‘(‘;
$self->_new_line;
$self->_add_token($token);
$self->_new_line if($self->_next_token and $self->_next_token ne ‘(‘);
$self->_over;
}

elsif($token =~ /^(?:GROUP|ORDER|LIMIT)$/i) {
$self->_back;
$self->_new_line;
$self->_add_token($token);
}

elsif($token =~ /^(?:BY)$/i) {
$self->_add_token($token);
$self->_new_line;
$self->_over;
}

elsif($token =~ /^(?:UNION|INTERSECT|EXCEPT)$/i) {
$self->_new_line;
$self->_add_token($token);
$self->_new_line;
}

elsif($token =~ /^(?:LEFT|RIGHT|INNER|OUTER|CROSS)$/i) {
$self->_back;
$self->_new_line;
$self->_add_token($token);
$self->_over;
}

elsif($token =~ /^(?:JOIN)$/i) {
if($last and $last !~ /^(?:LEFT|RIGHT|INNER|OUTER|CROSS)$/) {
$self->_new_line;
}

$self->_add_token($token);
}

elsif($token =~ /^(?:AND|OR)$/i) {
$self->_new_line;
$self->_add_token($token);
$self->_new_line;
}

else {
$self->_add_token($token, $last);
}

$last = $token;
}

$self->_new_line;

$self->{_output};
}

# Add a token to the beautified string.
sub _add_token {
my ($self, $token, $last_token) = @_;

if($self->{wrap}) {
my $wrap;

if($self->_is_keyword($token)) {
$wrap = $self->{wrap}->{keywords};
}
elsif($self->_is_constant($token)) {
$wrap = $self->{wrap}->{constants};
}

if($wrap) {
$token = $wrap->[0] . $token . $wrap->[1];
}
}

my $last_is_dot =
defined($last_token) && $last_token eq ‘.’;

if(!$self->_is_punctuation($token) and !$last_is_dot) {
$self->{_output} .= $self->_indent;
}

# uppercase keywords
$token = uc $token
if $self->_is_keyword($token) and $self->{uc_keywords};

$self->{_output} .= $token;

# This can’t be the beginning of a new line anymore.
$self->{_new_line} = 0;
}

# Increase the indentation level.
sub _over {
my ($self) = @_;

++$self->{_level};
}

# Decrease the indentation level.
sub _back {
my ($self) = @_;

–$self->{_level} if($self->{_level} > 0);
}

# Return a string of spaces according to the current indentation level and the
# spaces setting for indenting.
sub _indent {
my ($self) = @_;

if($self->{_new_line}) {
return $self->{space} x ($self->{spaces} * $self->{_level});
}
else {
return $self->{space};
}
}

# Add a line break, but make sure there are no empty lines.
sub _new_line {
my ($self) = @_;

$self->{_output} .= $self->{break} unless($self->{_new_line});
$self->{_new_line} = 1;
}

# Have a look at the token that’s coming up next.
sub _next_token {
my ($self) = @_;

return @{$self->{_tokens}} ? $self->{_tokens}->[0] : undef;
}

# Get the next token, removing it from the list of remaining tokens.
sub _token {
my ($self) = @_;

return shift @{$self->{_tokens}};
}

# Check if a token is a known SQL keyword.
sub _is_keyword {
my ($self, $token) = @_;

return ~~ grep { $_ eq uc($token) } @{$self->{keywords}};
}

# Add new keywords to highlight.
sub add_keywords {
my $self = shift;

for my $keyword (@_) {
push @{$self->{keywords}}, ref($keyword) ? @{$keyword} : $keyword;
}
}

# Add new rules.
sub add_rule {
my ($self, $format, $token) = @_;

my $rules = $self->{rules} ||= {};
my $group = $rules->{$format} ||= [];

push @{$group}, ref($token) ? @{$token} : $token;
}

# Find custom rule for a token.
sub _get_rule {
my ($self, $token) = @_;

values %{$self->{rules}}; # Reset iterator.

while(my ($rule, $list) = each %{$self->{rules}}) {
return $rule if(grep { uc($token) eq uc($_) } @$list);
}

return undef;
}

sub _process_rule {
my ($self, $rule, $token) = @_;

my $format = {
break => sub { $self->_new_line },
over => sub { $self->_over },
back => sub { $self->_back },
token => sub { $self->_add_token($token) },
push => sub { push @{$self->{_level_stack}}, $self->{_level} },
pop => sub { $self->{_level} = pop(@{$self->{_level_stack}}) || 0 },
reset => sub { $self->{_level} = 0; @{$self->{_level_stack}} = (); },
};

for(split /-/, lc $rule) {
&{$format->{$_}} if($format->{$_});
}
}

# Check if a token is a constant.
sub _is_constant {
my ($self, $token) = @_;

return ($token =~ /^\d+$/ or $token =~ /^([‘"`]).*\1$/);
}

# Check if a token is punctuation.
sub _is_punctuation {
my ($self, $token) = @_;

return ($token =~ /^[,;.]$/);
}

1;

=pod

=head1 NAME

OraBeautify – Beautify SQL statements by adding line breaks indentation

=head1 SYNOPSIS

my $sql = OraBeautify->new;

$sql->query($sql_query);

my $nice_sql = $sql->beautify;

=head1 DESCRIPTION

Beautifies SQL statements by adding line breaks indentation.

=head1 METHODS

=over 4

=item B<new>(query => ”, spaces => 4, space => ‘ ‘, break => "\n", wrap => {})

Constructor. Takes a few options.

=over 4

=item B<query> => ”

Initialize the instance with a SQL string. Defaults to an empty string.

=item B<spaces> => 4

Number of spaces that make one indentation level. Defaults to 4.

=item B<space> => ‘ ‘

A string that is used as space. Default is an ASCII space character.

=item B<break> => "\n"

String that is used for linebreaks. Default is "\n".

=item B<wrap> => {}

Use this if you want to surround certain tokens with markup stuff. Known token
types are "keywords" and "constants" for now. The value of each token type
should be an array with two elements, one that is placed before the token and
one that is placed behind it. For example, use make keywords red using terminal
color escape sequences.

{ keywords => [ "\x1B[0;31m", "\x1B[0m" ] }

=item B<uc_keywords> => 1|0

When true (1) all SQL keywords will be uppercased in output. Default is false (0).

=back

=item B<add>($more_sql)

Appends another chunk of SQL.

=item B<query>($query)

Sets the query to the new query string. Overwrites anything that was added with
prior calls to B<query> or B<add>.

=item B<beautify>

Beautifies the internally saved SQL string and returns the result.

=item B<add_keywords>($keyword, $another_keyword, \@more_keywords)

Add any amount of keywords of arrays of keywords to highlight.

=item B<add_rule>($rule, $token)

Add a custom formatting rule. The first argument is the rule, a string
containing one or more commands (explained below), separated by dashes. The
second argument may be either a token (string) or a list of strings. Tokens are
grouped by rules internally, so you may call this method multiple times with
the same rule string and different tokens, and the rule will apply to all of
the tokens.

The following formatting commands are known at the moment:

=over 4

=item B<token> – insert the token this rule applies to

=item B<over> – increase indentation level

=item B<back> – decrease indentation level

=item B<break> – insert line break

=item B<push> – push current indentation level to an internal stack

=item B<pop> – restore last indentation level from the stack

=item B<reset> – reset internal indentation level stack

=back

B<push>, B<pop> and B<reset> should be rarely needed.

B<NOTE>:
Custom rules override default rules. Some default rules do things that
can’t be done using custom rules, such as changing the format of a token
depending on the last or next token.

B<NOTE>:
I’m trying to provide sane default rules. If you find that a custom
rule of yours would make more sense as a default rule, please create a ticket.

=back

=head1 BUGS

Needs more tests.

Please report bugs in the CPAN bug tracker.

This module is not complete (known SQL keywords, special formatting of
keywords), so if you want see something added, just send me a patch.

=head1 COPYRIGHT

Copyright (C) 2009 by Jonas Kramer. Published under the terms of the Artistic
License 2.0.

=cut

########################################
package main;
use strict;

open (SQL, "<", $ARGV[0]) || die (‘File not found!’);
my $query = join("\n",<SQL>);

my $beautifier = OraBeautify->new;
$beautifier -> add_keywords(qw{
pivot unpivot
model dimension measures rules
xmltable xmlsequence columns});

$beautifier->query(
$query,
spaces => 4,
space => ‘ ‘,
break => "\n",
wrap => {‘$’,’$’}
);

my $nice_sql = $beautifier->beautify;

print $nice_sql ."\n";
__END__

[/sourcecode]

[collapse]

3. Create empty directory “tmp” within $SQL_PATH
4. Now you can use it for example like i did it in sql_textf.sql:
sql_textf.sql

[sourcecode language=”sql”]
set timing off head off
col qtext format a150
prompt ################################ Original query text: ################################################;
spool tmp/to_format.sql
select
coalesce(
(select sql_fulltext from v$sqlarea a where a.sql_id=’&1′)
, (select sql_text from dba_hist_sqltext a where a.sql_id=’&1′ and dbid=(select dbid from v$database))
) qtext
from dual
;
spool off

prompt ################################ Formatted query text #################################################;
host perl inc/sql_format_standalone.pl tmp/to_format.sql
prompt ################################ Formatted query text End #############################################;
set termout on head on
[/sourcecode]

[collapse]

I also use it in other scripts, like sqlid.sql:

[sourcecode language=”sql”]
@inc/input_vars_init;

REM ############### COMMON FORMATTING #######################
col SQL_ID for a13
col sql_child_number head CH# for 999
col SQL_PROFILE head PROFILE for a19
—————————————–
— params check:
set termout off timing off
def _sqlid=&1
col _child new_val _child noprint
select
case
when translate(‘&2′,’x0123456789′,’x’) is null
then nvl(‘&2′,’%’)
else ‘%’
end "_CHILD"
from dual;
—————————————–
set termout on

prompt ####################################################################################################;
prompt # Show SQL text, child cursors and execution stats for SQLID &1 child &2
prompt ####################################################################################################;

REM ################### SHOW SQL TEXT ############################
@sql_textf &_sqlid

REM ################### SHOW V$SQL ##############################
col proc_name for a30
col P_schema for a20

select
s.sql_id
,s.CHILD_NUMBER sql_child_number
,s.address parent_handle
,s.child_address object_handle
,s.PLAN_HASH_VALUE plan_hv
,s.hash_value hv
,s.SQL_PROFILE sql_profile
,decode(s.EXECUTIONS,0,0, s.ELAPSED_TIME/1e6/s.EXECUTIONS) elaexe
,s.EXECUTIONS cnt
,s.FETCHES fetches
,s.END_OF_FETCH_COUNT end_of_fetch_count
,s.FIRST_LOAD_TIME first_load_time
,s.PARSE_CALLS parse_calls
,decode(s.executions,0,0, s.DISK_READS /s.executions) disk_reads
,decode(s.executions,0,0, s.BUFFER_GETS /s.executions) buffer_gets
,decode(s.executions,0,0, s.DIRECT_WRITES /s.executions) direct_writes
,decode(s.executions,0,0, s.APPLICATION_WAIT_TIME/1e6/s.executions) app_wait
,decode(s.executions,0,0, s.CONCURRENCY_WAIT_TIME/1e6/s.executions) concurrency
,decode(s.executions,0,0, s.USER_IO_WAIT_TIME /1e6/s.executions) io_wait
,decode(s.executions,0,0, s.PLSQL_EXEC_TIME /1e6/s.executions) plsql_t
,decode(s.executions,0,0, s.java_exec_time /1e6/s.executions) java_exec_t
,s.ROWS_PROCESSED row_processed
,s.OPTIMIZER_MODE opt_mode
,s.OPTIMIZER_COST cost
,s.OPTIMIZER_ENV_HASH_VALUE env_hash
,s.PARSING_SCHEMA_NAME P_schema
,decode(s.executions,0,0, s.CPU_TIME/1e6/s.executions) CPU_TIME
,s.PROGRAM_ID
,(select object_name from dba_objects o where o.object_id=s.PROGRAM_ID) proc_name
,s.PROGRAM_LINE# proc_line
from v$sql s
where
sql_id = (‘&_sqlid’)
and child_number like ‘&_child’
order by
sql_id,
hash_value,
child_number
/
REM ##################### END V$SQL ##############################

REM ################### PLSQL OBJECT ##############################
col owner for a10
col object_name for a30
col text for a120

select
a.SQL_ID,a.SQL_PROFILE
,p.owner,p.object_name
,s.line
,rtrim(s.text,chr(10)) text
from
v$sqlarea a
left join dba_procedures p
on a.PROGRAM_ID=p.OBJECT_ID
left join dba_source s
on p.owner=s.owner
and p.OBJECT_NAME=s.name
and s.line between a.PROGRAM_LINE#-5 and a.PROGRAM_LINE#+5
where a.SQL_ID=’&_sqlid’
/
REM ################### EXECUTIONS IN SQL_MONITOR ######################
@if "’&_O_RELEASE’>’11.2’" then

col error_message for a40
@rtsm/execs "&_sqlid" "&_child"

/* end if */
REM ########################### clearing ############################
col SQL_PROFILE clear
col owner clear
col object_name clear
col text clear
col error_message clear
@inc/input_vars_undef;
[/sourcecode]

[collapse]

SQL*Plus
photo Sayan Malakshinov

Oracle ACE Pro Oracle ACE Pro Alumni

DEVVYOracle Database Developer Choice Award winner

Oracle performance tuning expert

UK / Cambridge

LinkedIn   Twitter
sayan@orasql.org

Recent Posts

  • Oracle Telegram Bot
  • Partition Pruning and Global Indexes
  • Interval Search: Part 4. Dynamic Range Segmentation – interval quantization
  • Interval Search Series: Simplified, Advanced, and Custom Solutions
  • Interval Search: Part 3. Dynamic Range Segmentation – Custom Domain Index

Popular posts

Recent Comments

  • Oracle SQL | Interval Search: Part 4. Dynamic Range Segmentation – interval quantization on Interval Search: Part 3. Dynamic Range Segmentation – Custom Domain Index
  • Oracle SQL | Interval Search: Part 4. Dynamic Range Segmentation – interval quantization on Interval Search: Part 2. Dynamic Range Segmentation – Simplified
  • Oracle SQL | Interval Search: Part 4. Dynamic Range Segmentation – interval quantization on Interval Search: Optimizing Date Range Queries – Part 1
  • Oracle SQL | Interval Search Series: Simplified, Advanced, and Custom Solutions on Interval Search: Part 2. Dynamic Range Segmentation – Simplified
  • Oracle SQL | Interval Search: Part 2. Dynamic Range Segmentation – Simplified on Interval Search: Part 3. Dynamic Range Segmentation – Custom Domain Index

Blogroll

  • Alex Fatkulin
  • Alexander Anokhin
  • Andrey Nikolaev
  • Charles Hooper
  • Christian Antognini
  • Coskan Gundogar
  • David Fitzjarrell
  • Igor Usoltsev
  • Jonathan Lewis
  • Karl Arao
  • Mark Bobak
  • Martin Bach
  • Martin Berger
  • Neil Chandler
  • Randolf Geist
  • Richard Foote
  • Riyaj Shamsudeen
  • Tanel Poder
  • Timur Akhmadeev
  • Valentin Nikotin
  • 1
  • 2
  • Next
©Sayan Malakshinov. Oracle SQL