Ralf Bensmann

Galileo-Videotraining Einstieg in Java und twitter4j

Wed, 23 May 2012 03:53:00 -0700

Im Videotraining Einstieg in Java wird als Projekt ein Twitter-Client unter Verwendung der twitter4j Bibliothek erstellt. Als ich das Training im November 2010 aufgenommen habe war die aktuelle Version 2.1.7-SNAPSHOT. Leider wird diese Version nicht mehr wie im Video gezeigt zum Download angeboten.

Wer das Training mit der Version absolvieren möchte, findet die Bibliothek auf der DVD im Verzeichnis Uebungsmaterial/Sourcecode/Kapitel 9-11/workspace/Training/lib/twitter4j-core-2.1.7-SNAPSHOT.jar oder kann sie hier herunterladen.

Permalink | Leave a comment »

Deleting lots of tables in SQL Server

Thu, 26 Apr 2012 08:53:25 -0700

Today I had to drop 2.604 tables from an SQL Server database… too much to tell the customer to “do it by hand” using Management Studio ;–)

So here is a little SQL to query tables and drop them in a batch:

-- Temporary table to hold a list of tables to drop
create table #dropthem (id int identity, name nvarchar(255))
insert into #dropthem select name from sys.tables where name like 'some_tables_butnotall%'
-- Query temporary table for tables to drop
declare @id int, @table nvarchar(100), @stmt nvarchar(200)
select top 1 @id = id, @table = name from #dropthem
while @@rowcount = 1
begin
    set @stmt = N'DROP TABLE ' + @table
    print @stmt
    exec(@stmt)
    select top 1 @id = id, @table = name from #dropthem where id > @id
end
drop table #dropthem

Hope that helps.

Permalink | Leave a comment »

Single or Double Quotation Marks in Groovy

Tue, 20 Sep 2011 10:11:00 -0700

In Groovy you can choose between single quotation marks ' or double quotation marks " to represent a string. As opposed to Java, where single quotation marks denote a single character and double quotation marks denote a string.

Groovy provides groovy.lang.GString which you can think of a java.lang.String on steroids, as it has templating built-in.

For example, the value of a variable x should be included in a string, you can write:

def x = 1
println "Variable x has a value of ${x}"

Whereas in Java you had to write:

void test() {
    int x = 1;
    System.out.println("Variable x has a value of " + x);
}

Or by using a StringBuilder or StringBuffer:

void test() {
    int x = 1;
    StringBuilder builder = new StringBuilder();
    builder.append("Variable x has a value of ").append(x);
    System.out.println(builder.toString());
}

Difference

But what’s the difference in Groovy? With single marks a java.lang.String is created – no templating available – and with double quotation marks a GString can be used. So using GString where String is enough you will do unnecessary things and maybe loose performance.

Look at these examples:

Single quotation marks

println ''.getClass()
class java.lang.String

Double quotation marks, no template –> no GString

println "".getClass()
class java.lang.String

Single quotation marks with template (bad idea)

def x = 1
println '${x}'.getClass()
class java.lang.String

Double quotation marks with template

def x = 1
println "${x}".getClass()
class org.codehaus.groovy.runtime.GStringImpl

Permalink | Leave a comment »

Securing Oracle Listener

Mon, 04 Jul 2011 02:29:31 -0700

When an Oracle database is deployed on an internet host, you can use Oracle’s builtin security options to allow access only from certain clients. In this case I didn’t want to setup complex firewall rules or use a SSH tunnel.

Add the following settings to $TNS_ADMIN/sqlnet.ora and list all allowed IPs in tcp.invited_nodes:

tcp.nodelay = yes
tcp.validnode_checking = yes
tcp.invited_nodes = (127.0.0.1, 123.45.167.289, 123.45.167.290)

Restart the listener:

oracle:~ $ lsnrctl stop && lsnrctl start

The Oracle SQL*Net listener is listening on port 1521 and accepts connections from any:

root:~ # netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode       PID/Program name
tcp        0      0 123.45.167.289:1521     0.0.0.0:*               LISTEN      1001       18401       5319/tnslsnr    
tcp        0      0 123.45.167.289:22       0.0.0.0:*               LISTEN      0          6881        3316/sshd       
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      0          7049        3403/master     
udp6       0      0 ::1:23824               :::*                                1001       18904       5503/ora_pmon_RCAT
udp6       0      0 :::27665                :::*                                1001       20679       5531/ora_mmon_RCAT
udp6       0      0 :::47978                :::*                                1001       21789       5751/ora_mmon_AOC
udp6       0      0 ::1:23157               :::*                                1001       20226       5722/ora_pmon_AOC

To protect the pmon and mmon processes listening on UDP6 ports, I use a simple firewall rule to drop packets received on the outside interface eth0:

root:~ # ip6tables -A INPUT -i eth0 -p udp -j DROP
root:~ # ip6tables -L -v -n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       udp      eth0   *       ::/0                 ::/0      

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Permalink | Leave a comment »

Show actual git branch in prompt

Sat, 02 Jul 2011 08:18:00 -0700

To show the actual git branch of a project when you are in bash put the following function into your profile (e.g. .bash_profile):

get_git_branch() {
  local br=$(git branch 2> /dev/null | grep "*" | sed 's/* //g')
  [ -n "$br" ] && echo " @$br"
}

and set your prompt accordingly:

export PS1='\\u@\\h:\\w$(get_git_branch) \\$ '

(Replace double-slashes above with single-slashes!)

For example, go into a project and see the shell prompt expands with the actual branch:

rbe@rbemac3:~ $ cd project/odisee/
rbe@rbemac3:~/project/odisee @master $

Create a new branch:

rbe@rbemac3:~/project/odisee @master $ git branch -a
* master
  remotes/github/master
  remotes/origin/master
rbe@rbemac3:~/project/odisee @master $ git checkout -b rel2
M   grootils
Switched to a new branch 'rel2'

And again, the prompt changed:

rbe@rbemac3:~/project/odisee @rel2 $

HTH.

Permalink | Leave a comment »

Using Oracle LogMiner

Fri, 13 May 2011 02:07:00 -0700

I just gave an Oracle backup & recovery workshop for a customer, talking about Oracle in general, DataPump and Recovery Manager (RMAN). At the end we came across LogMiner to find a certain SCN of a DDL statement. As I did not use LogMiner for a while, lets review this great tool.

LogMiner allows you to look inside your redo logs and review DML and DDL statements.

Setting up LogMiner

Check if you have configured a directory where Oracle can read and write (this are not the directories created with CREATE DIRECTORY):

SQL> show parameter utl_file_dir

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
utl_file_dir                         string

Value is empty? Set Oracle parameter UTL_FILE_DIR in (server) parameter file, bounce the instance and run utlfile.sql:

$ sqlplus / as sysdba


SQL> alter system set utl_file_dir = '/tmp' scope = spfile;
SQL> shutdown
SQL> startup
SQL> @?/rdmbs/admin/utlfile

Next you need to setup LogMiner dictionary four your instance to map object ids to names and provide human-readable insight to the logs — if you don’t build it you see something like this:

SQL> select sql_redo from v$logmnr_contents;
SQL_REDO
--------------------------------------------------------------------------------
set transaction read write;
insert into UNKNOWN.Objn:25128(Col[1]) values (HEXTORAW('c102'));
commit;

instead of:

SQL> select sql_redo from v$logmnr_contents;
SQL_REDO
--------------------------------------------------------------------------------
set transaction read write;
insert into test1 values (1);
commit;


$ sqlplus / as sysdba

Build the dictionary with DBMS_LOGMNR_D.BUILD:

SQL> exec dbms_logmnr_d.build(dictionary_filename => 'dictionary.ora', dictionary_location => '/tmp');

PL/SQL procedure successfully completed.

LogMiner views

What data dictionary views do we have got?

SQL> select view_name from all_views where view_name like '%LOGMNR%';

VIEW_NAME
------------------------------
V_$LOGMNR_CALLBACK
V_$LOGMNR_CONTENTS
V_$LOGMNR_DICTIONARY
V_$LOGMNR_DICTIONARY_LOAD
V_$LOGMNR_LATCH
V_$LOGMNR_LOGFILE
V_$LOGMNR_LOGS
V_$LOGMNR_PARAMETERS
V_$LOGMNR_PROCESS
V_$LOGMNR_REGION
V_$LOGMNR_SESSION
V_$LOGMNR_STATS
V_$LOGMNR_TRANSACTION
DBA_LOGMNR_LOG
DBA_LOGMNR_PURGED_LOG
DBA_LOGMNR_SESSION

13 rows selected.

I will just use V$LOGMNR_CONTENTS in this blog entry. See V$LOGMNR_CONTENTS in Oracle Database Reference 11g Release 2 (11.2) for an explanation of all columns in this view.

A simple session with LogMiner

Take a look at the available logfile by querying V$LOGFILE and V$ARCHIVED_LOG:

SQL> select sequence#,name,status,deleted from v$archived_log;

 SEQUENCE# NAME                                                                        S DEL
---------- --------------------------------------------------------------------------- - ---
         7 /u04/app/oracle/orafra/PROD/archivelog/2011_05_11/o1_mf_1_7_6wnn7thl_.arc   A NO
         8 /u04/app/oracle/orafra/PROD/archivelog/2011_05_11/o1_mf_1_8_6wnt0o61_.arc   A NO
         9 /u04/app/oracle/orafra/PROD/archivelog/2011_05_11/o1_mf_1_9_6wnt1fr5_.arc   A NO
[...]
        40 /u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_40_6wsqps23_.arc  A NO
        41 /u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_41_6wsqsccn_.arc  A NO
        42 /u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_42_6wsqsgxv_.arc  A NO

SQL> select f.group#,l.sequence#,f.member from v$logfile f inner join v$log l on f.group# = l.group#;

    GROUP#  SEQUENCE# MEMBER
---------- ---------- ----------------------------------------------------------------------
         1         43 /u02/app/oracle/oradata/PROD/onlinelog/o1_mf_1_6wlfqxho_.log
         1         43 /u02/app/oracle/oradata/PROD/onlinelog/o1_mf_1_6wlfqxww_.log
         2         42 /u02/app/oracle/oradata/PROD/onlinelog/o1_mf_2_6wlfr060_.log
         2         42 /u02/app/oracle/oradata/PROD/onlinelog/o1_mf_2_6wlfr21y_.log

We can see that logs from sequence #7 up to #42 are archived and they are NOT DELeted, so they are available for analysis. The actual log sequence #43 is available in redo log group #1.

Start LogMiner

At first add the logfiles you want to analyze, take care: first logfile is added with option DBMS_LOGMNR.NEW, others with DBMS_LOGMNR.ADDFILE. I want to analyze the last four log sequences (3 archived, 1 online log sequence):

SQL> exec dbms_logmnr.add_logfile(logfilename => '/u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_40_6wsqps23_.arc', options => dbms_logmnr.new);

PL/SQL procedure successfully completed.

SQL> exec dbms_logmnr.add_logfile(logfilename => '/u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_41_6wsqsccn_.arc', options => dbms_logmnr.addfile);

PL/SQL procedure successfully completed.

SQL> exec dbms_logmnr.add_logfile(logfilename => '/u04/app/oracle/orafra/PROD/archivelog/2011_05_13/o1_mf_1_42_6wsqsgxv_.arc', options => dbms_logmnr.addfile);

PL/SQL procedure successfully completed.

SQL> exec dbms_logmnr.add_logfile(logfilename => '/u02/app/oracle/oradata/PROD/onlinelog/o1_mf_1_6wlfqxho_.log', options => dbms_logmnr.addfile);

PL/SQL procedure successfully completed.

Start a session using the previously created dictionary logfile for the instance:

SQL> exec dbms_logmnr.start_logmnr(dictfilename => '/tmp/dictionary.ora');


SQL> select scn, sql_redo from v$logmnr_contents where ... ;

Alternatively you can start a session and give some contraints on time or SCN:

SQL> exec dbms_logmnr.start_logmnr(dictfilename => '/tmp/dictionary.ora'
  2  , starttime => to_date('01-Feb-2011 08:00:00', 'DD-MON-YYYY HH:MI:SS')
  3  , endtime => to_date('01-Feb-2011 10:00:00', 'DD-MON-YYYY HH:MI:SS'));

LogMiner shows only DML statements and transaction control from redo and undo. DDL (i.e. DROP TABLE) is a DML on data dictionary items! To find DDL statements you have to query for DML against SYS.TAB$ for example.

SQL> column seg_name format a25 trunc
SQL> col seg_name format a10
SQL> col seg_owner format a20
SQL> col table_space format a15
SQL> col operation format a11
SQL> select
        seg_name
        , seg_type
        , seg_owner
        , table_name
        , table_space
        , operation
        , scn
    from
        v$logmnr_contents
    where
        operation != 'INTERNAL'
        and seg_name in ('COL$','OBJ$','TAB$')
    group by
        seg_name
        , seg_type
        , seg_owner
        , table_name
        , table_space
        , operation
        , scn
    order by
        scn;

SEG_NAME     SEG_TYPE SEG_OWNER            TABLE_NAME                       TABLE_SPACE     OPERATION          SCN
---------- ---------- -------------------- -------------------------------- --------------- ----------- ----------
OBJ$                2 SYS                  OBJ$                             SYSTEM          INSERT          407518
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407521
OBJ$                2 SYS                  OBJ$                             SYSTEM          INSERT          407524
OBJ$                2 SYS                  OBJ$                             SYSTEM          INSERT          407528
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407535
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407539
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407543
OBJ$                2 SYS                  OBJ$                             SYSTEM          INSERT          407594
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407596
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407598
OBJ$                2 SYS                  OBJ$                             SYSTEM          UPDATE          407600

End your session with DBMS_LOGMNR.END_LOGMNR or just start a new one with DBMS_LOGMNR.START_LOGMNR.

SQL> exec dbms_logmnr.end_logmnr;

I can’t find a dropped table with LogMiner

Maybe you’re using the Flashback feature and the table was moved into recycle bin:

SQL> show parameter flashback

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_flashback_retention_target        integer     1440

SQL> show parameter recyclebin

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
recyclebin                           string      on

The recyclebin is turned on. So dropped tables will not be dropped, they are renamed:

SQL> show recyclebin
ORIGINAL NAME    RECYCLEBIN NAME                OBJECT TYPE  DROP TIME
---------------- ------------------------------ ------------ -------------------
TEST             BIN$oyQvIAiLn73gQCi8yGgVVw==$0 TABLE        2011-05-13:09:23:36

Recover it using the FLASHBACK TABLE command:

SQL> flashback table ttt to before drop;

Finding a log sequence by time

To find certain statements you need to locate the needed logfile(s). At first you may know the time when a statement was issued. The data dictionary views V$ARCHVIED_LOG and V$BACKUP_REDOLOG give a lot of information on what log sequence and SCNs (system change numbers) belong to which time frame:

SQL> set linesize 132
SQL> col name format a70
SQL> alter session set nls_date_format = 'DD.MM.YY HH24:MI';
SQL> select name, sequence#, first_change#, first_time from v$archived_log;

[...]

NAME                                                                    SEQUENCE# FIRST_CHANGE# FIRST_TIME
---------------------------------------------------------------------- ---------- ------------- --------------
                                                                               35        270339 15.02.11 15:04
                                                                               36        270523 15.02.11 15:08
                                                                               37        275453 15.02.11 15:26
                                                                               38        278957 15.02.11 15:28
                                                                               39        283483 15.02.11 16:37
                                                                               40        283681 15.02.11 16:43
/u02/app/oracle/fra/AOC/archivelog/2011_02_16/o1_mf_1_41_6or9x348_.arc         41        283705 15.02.11 16:43
/u02/app/oracle/fra/AOC/archivelog/2011_02_16/o1_mf_1_42_6orb7r4w_.arc         42        337380 16.02.11 20:38
/u02/app/oracle/fra/AOC/archivelog/2011_02_17/o1_mf_1_43_6otb8y65_.arc         43        337619 16.02.11 20:43
/u02/app/oracle/fra/AOC/archivelog/2011_02_18/o1_mf_1_44_6ox4wn28_.arc         44        397223 17.02.11 14:56
/u02/app/oracle/fra/AOC/archivelog/2011_02_18/o1_mf_1_45_6ox4xqth_.arc         45        450667 18.02.11 16:43

39 rows selected.

This query shows us that log sequences from 41 to 45 are available on disk in the FRA (flash recovery area). Our online redo logs are:

SQL> set linesize 132
SQL> col member format a70
SQL> alter session set nls_date_format = 'DD.MM.YY HH24:MI:SS';
SQL> select l.group#, sequence#, first_change#, first_time, min(member) member
     from v$log l inner join v$logfile f on l.group# = f.group#
     group by l.group#, sequence#, first_change#, first_time;

     GROUP#  SEQUENCE# FIRST_CHANGE# FIRST_TIME        MEMBER
 ---------- ---------- ------------- ----------------- -------------------------------------------------------
          2         46        450705 18.02.11 16:43:51 /u02/app/oracle/fra/AOC/onlinelog/o1_mf_2_6om6m588_.log
          1         45        450667 18.02.11 16:43:15 /u02/app/oracle/fra/AOC/onlinelog/o1_mf_1_6om6m1tr_.log

Use LogMiner like shown above to take a look into the log. Now I want to see the statements, look at the column SQL_REDO in V$LOGMNR_CONTENTS:

SQL> select sql_redo from v$logmnr_contents where scn=407518;

SQL_REDO
---------------------------------------------------------------------------------------------------------------------------
commit;
set transaction read write;
insert into "SYS"."SCHEDULER$_EVENT_LOG"("LOG_ID","LOG_DATE","TYPE#","NAME","OWNER","CLASS_ID","OPERATION","STATUS","USER_NAME","CLI
ENT_ID","GUID","DBID","FLAGS","CREDENTIAL","DESTINATION","ADDITIONAL_INFO") values ('99',TO_TIMESTAMP_TZ('12-MAY-11 10.00.06.423812
PM +02:00'),'66','DRA_REEVALUATE_OPEN_FAILURES','SYS','12166','RUN','SUCCEEDED',NULL,NULL,NULL,NULL,'0',NULL,NULL,NULL);

insert into "SYS"."SCHEDULER$_JOB_RUN_DETAILS"("LOG_ID","LOG_DATE","REQ_START_DATE","START_DATE","RUN_DURATION","INSTANCE_ID","SESSI
ON_ID","SLAVE_PID","CPU_USED","ERROR#","ADDITIONAL_INFO","CREDENTIAL","DESTINATION") values ('99',TO_TIMESTAMP_TZ('12-MAY-11 10.00.0
6.536282 PM +02:00'),NULL,TO_TIMESTAMP_TZ('12-MAY-11 10.00.03.047298 PM EUROPE/VIENNA'),TO_DSINTERVAL('+000 00:00:03'),'1','160,747'
,'22112',TO_DSINTERVAL('+000 00:00:00.08'),'0',NULL,NULL,NULL);

insert into "SYS"."OBJ$"("OBJ#","DATAOBJ#","OWNER#","NAME","NAMESPACE","SUBNAME","TYPE#","CTIME","MTIME","STIME","STATUS","REMOTEOWN
ER","LINKNAME","FLAGS","OID$","SPARE1","SPARE2","SPARE3","SPARE4","SPARE5","SPARE6") values ('13098',NULL,'0','ORA$AT_OS_OPT_SY_7','
1',NULL,'66',TO_DATE('12-MAY-11', 'DD-MON-RR'),TO_DATE('12-MAY-11', 'DD-MON-RR'),TO_DATE('12-MAY-11', 'DD-MON-RR'),'1',NULL,NULL,'0'
,NULL,'6','65535','0',NULL,NULL,NULL);

If you need to backout that change, query the column SQL_UNDO:

SQL_UNDO
---------------------------------------------------------------------------------------------------------------------------
delete from "SYS"."SCHEDULER$_EVENT_LOG" where "LOG_ID" = '99' and "LOG_DATE" = TO_TIMESTAMP_TZ('12-MAY-11 10.00.06.423812 PM +02:00
') and "TYPE#" = '66' and "NAME" = 'DRA_REEVALUATE_OPEN_FAILURES' and "OWNER" = 'SYS' and "CLASS_ID" = '12166' and "OPERATION" = 'RU
N' and "STATUS" = 'SUCCEEDED' and "USER_NAME" IS NULL and "CLIENT_ID" IS NULL and "GUID" IS NULL and "DBID" IS NULL and "FLAGS" = '0
' and "CREDENTIAL" IS NULL and "DESTINATION" IS NULL and ROWID = 'AAABbfAACAAAAzOAAO';

delete from "SYS"."SCHEDULER$_JOB_RUN_DETAILS" where "LOG_ID" = '99' and "LOG_DATE" = TO_TIMESTAMP_TZ('12-MAY-11 10.00.06.536282 PM
+02:00') and "REQ_START_DATE" IS NULL and "START_DATE" = TO_TIMESTAMP_TZ('12-MAY-11 10.00.03.047298 PM EUROPE/VIENNA') and "RUN_DURA
TION" = TO_DSINTERVAL('+000 00:00:03') and "INSTANCE_ID" = '1' and "SESSION_ID" = '160,747' and "SLAVE_PID" = '22112' and "CPU_USED"
 = TO_DSINTERVAL('+000 00:00:00.08') and "ERROR#" = '0' and "ADDITIONAL_INFO" IS NULL and "CREDENTIAL" IS NULL and "DESTINATION" IS
NULL and ROWID = 'AAABbjAACAAAAztAAI';

delete from "SYS"."OBJ$" where "OBJ#" = '13098' and "DATAOBJ#" IS NULL and "OWNER#" = '0' and "NAME" = 'ORA$AT_OS_OPT_SY_7' and "NAM
ESPACE" = '1' and "SUBNAME" IS NULL and "TYPE#" = '66' and "CTIME" = TO_DATE('12-MAY-11', 'DD-MON-RR') and "MTIME" = TO_DATE('12-MAY
-11', 'DD-MON-RR') and "STIME" = TO_DATE('12-MAY-11', 'DD-MON-RR') and "STATUS" = '1' and "REMOTEOWNER" IS NULL and "LINKNAME" IS NU
LL and "FLAGS" = '0' and "OID$" IS NULL and "SPARE1" = '6' and "SPARE2" = '65535' and "SPARE3" = '0' and "SPARE4" IS NULL and "SPARE
5" IS NULL and "SPARE6" IS NULL and ROWID = 'AAAAASAABAAAEg1AAL';

Optimize

It’s best to create a (temporary) table of the contents of v$logmnr_contents to save time and performance when querying the logs:

SQL> create table mylog as select ... from v$logmnr_contents;

Take care

It’s very important to end the LogMiner session with DBMS_LOGMNR.END_LOGMNR, as you will get an ORA-600 error when logging off:

SQL> exec dbms_logmnr.end_logmnr();

Trying to use LogMiner concurrently from another session will give a ORA-01306: dbms_logmnr.start_logmnr() must be invoked before selecting from v$logmnr_contents.

See MOS Note 62508.1 “The LogMiner Utility” for details.

Recover the database using RMAN

After finding the correct log sequence or SCN use it with RMAN’s UNTIL-clause like this:

RMAN> recover database until scn 220123;

RMAN> recover database until logseq 43;

Resources

19 Using LogMiner to Analyze Redo Log Files in Oracle Database Utilities 11g Release 2 (11.2)

Permalink | Leave a comment »

Installing Java 6 and GlassFish 3.1 on FreeBSD 8

Thu, 28 Apr 2011 05:19:00 -0700

I’m in the process of creating a HA cluster based on FreeBSD, GEOM, HAST and ZFS. For our customers we need Java and GlassFish as well.

The servers are set up with FreeBSD 8.2.

Installing Oracle JDK 1.6

You need to download several files manually as licenses must be accepted:

Place all these files under /usr/ports/distfiles.

To install the jdk16 port, download some files from Oracle via command line:

# cd /usr/ports/distfiles
# curl -v -L -o /usr/ports/distfiles/jdk-6u3-fcs-src-b05-jrl-24_sep_2007.jar http://www.java.net/download/jdk6/6u3/promoted/b05/jdk-6u3-fcs-src-b05-jrl-24_sep_2007.jar
# curl -v -L -o /usr/ports/distfiles/jdk-6u3-fcs-bin-b05-jrl-24_sep_2007.jar http://www.java.net/download/jdk6/6u3/promoted/b05/jdk-6u3-fcs-bin-b05-jrl-24_sep_2007.jar
# curl -v -L -o /usr/ports/distfiles/jdk-6u3-fcs-mozilla_headers-b05-unix-24_sep_2007.jar http://www.java.net/download/jdk6/6u3/promoted/b05/jdk-6u3-fcs-mozilla_headers-b05-unix-24_sep_2007.jar

Attention: installing the port jdk16 will also automagically install diablo-jdk16.

Tweak the ports

As the port assumes an older version of the Timzeone Updater, edit the Makefiles in /usr/ports/java/jdk16 and /usr/ports/java/diablo-jdk16. In my case I could download the version 1.3.38_2011e from Oracle:

TZUPDATE_VERSION=       1_3_38
TZUPDATE_TZVERSION=     2011e

Add the SHA256 hash for tzupdater-1_3_38-2011e.zip to the distinfo files:

# cd /usr/ports/distfiles
# sha256 tzupdater-1_3_38-2011e.zip >> /usr/ports/java/jdk16/distinfo
# sha256 tzupdater-1_3_38-2011e.zip >> /usr/ports/java/diablo-jdk16/distinfo

Important: first cd into /usr/ports/disfiles and the issue sha256 as only the filename (without the path) must be present in distinfo.

Install the jdk16 port

# cd /usr/ports/java/jdk16
# make install

Verify that Java was installed to /usr/local/jdk1.6.0:

# which java
/usr/local/bin/java

Start Java with --version:

# java -version
java version "1.6.0_03-p4"
Java(TM) SE Runtime Environment (build 1.6.0_03-p4-root_28_apr_2011_10_26-b00)
Java HotSpot(TM) Client VM (build 1.6.0_03-p4-root_28_apr_2011_10_26-b00, mixed mode)

Yeeha!

Installing GlassFish 3.1

Manually download the ZIP-version from Oracle GlassFish Downloads.

Unzip that distribution:

# cd /opt
# unzip ogs-3.1.zip

Next create a GlassFish domain:

# cd glassfish3
# bin/asadmin create-domain domain1

Watch the domain being created:

Enter admin user name [Enter to accept default "admin" / no password]> 
Using port 4848 for Admin.
Using default port 8080 for HTTP Instance.
Using default port 7676 for JMS.
Using default port 3700 for IIOP.
Using default port 8181 for HTTP_SSL.
Using default port 3820 for IIOP_SSL.
Using default port 3920 for IIOP_MUTUALAUTH.
Using default port 8686 for JMX_ADMIN.
Using default port 6666 for OSGI_SHELL.
Using default port 9009 for JAVA_DEBUGGER.
Distinguished Name of the self-signed X.509 Server Certificate is:
[CN=skunk1,OU=GlassFish,O=Oracle Corporation,L=Santa Clara,ST=California,C=US]
Distinguished Name of the self-signed X.509 Server Certificate is:
[CN=skunk1-instance,OU=GlassFish,O=Oracle Corporation,L=Santa Clara,ST=California,C=US]
No domain initializers found, bypassing customization step
Domain domain1 created.
Domain domain1 admin port is 4848.
Domain domain1 allows admin login as user "admin" with no password.
Command create-domain executed successfully.

Also enable secure admin to use remote administration, i.e. with NetBeans:

# cd glassfish3
# bin/asadmin enable-secure-admin
Command enable-secure-admin executed successfully.

Finally start your domain:

# cd glassfish3
# bin/asadmin start-domain domain1


Waiting for domain1 to start ...
Successfully started the domain : domain1
domain  Location: /root/glassfish3/glassfish/domains/domain1
Log File: /root/glassfish3/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.

I could not use the web based administration console as after starting it up by pointing the browser to http://freebsd:4848 an ClassNotFoundException showed up in GlassFish’s log:

# cd glassfish3
# less glassfish/domains/domain1/logs/server.log

[#|2011-04-28T11:14:02.467+0200|INFO|oracle-glassfish3.1|javax.enterprise.resource.webcontainer.jsf.config|_ThreadID=20;_ThreadName=Thread-1;|Unsanitized stacktrace from failed start...
com.sun.faces.config.ConfigurationException: 
  Source Document: jndi:/__asadmin/WEB-INF/faces-config.xml
  Cause: Unable to find class 'com.sun.webui.jsf.faces.UIComponentELResolver'

Googling around, this seems to be fixed by installing a newer version of Java 6. Maybe another port like openjdk6 will do the trick? Anyways, I can use GlassFish with NetBeans (Server tab) under FreeBSD now.

There’s also some documentation on GlassFish v2 and FreeBSD 8.0: http://freebie.miraclenet.co.th/server/fbsd_glassfish.html.>

Permalink | Leave a comment »

How to Install MySQL 5.5 on Mac OS X

Sat, 26 Mar 2011 02:02:52 -0700

Short tutorial on how to install MySQL 5.5 on Mac OS X 10.6.

Point your browser to the MySQL download page and download the “Mac OS X 10.6 64-bit DMG-Archive”. The DMG (Disk IMage) may open automatically depending on your configuration. If it does not, double-click it.

Double-click on the mysql*.pkg to start installation of the MySQL server:

Read important information:

Accept license agreement:

Accept standard location and install MySQL:

Hooray!

Allow the installer to modify your system by entering your password:

Now install the MySQL preferences pane (will go into System Prefences) which lets you start and stop the MySQL server:

Choose who can use the pane:

Open System Preferences, clock on MySQL and you can start/stop MySQL server or choose whether MySQL should start automatically at system startup:

If you want to start MySQL automatically at system startup, you should install the startup item:

HTH.

Permalink | Leave a comment »

Set JVM Options for IntelliJ on Mac OS X

Thu, 24 Mar 2011 05:59:00 -0700

To tune IntelliJ’s settings when starting the Java virtual machine, go to /Applications/IntelliJ IDEA 10.app/Contents and edit Info.plist:

$ sudo bash
# cd /Applications/IntelliJ IDEA 10.app/Contents
# vi Info.plist

Look for VMOptions and put the right options into the `` tag below it.

VMOptions

-Xms512m -Xmx768m -XX:MaxPermSize=512m -ea -agentlib:yjpagent=disablej2ee,disablecounts,disablealloc,sessionname=IntelliJIdea10 -Xbootclasspath/a:../lib/boot.jar

Permalink | Leave a comment »

Objects, Pointers and Primitives in Objective-C

Sat, 05 Mar 2011 03:48:00 -0800

Objective-C adds lots of things to C — you have C constructs like structs, unions and pointers, but also objects, protocols, reference counting and garbage collection and a lot more.

An object and its primitive counterpart

I came across using classes like NSNumber and method declarations using a primitive like int or long. A variable “holding an” or referring to an object is actually a pointer, the type id is a generic pointer type.

Imagine the following prototype:

- (void) fun3: (long) j;

And a call to it:

// Create NSNumber object from integer value
NSNumber* num = [NSNumber numberWithInt: 123];
// Call fun3 with NSNumber object as parameter
[self fun3: num];

Does this work? Whats the result?

In Java there’s auto-boxing and auto-unboxing: auto-boxing is the process of automatically wrapping a primitive int variable or literal in an java.lang.Integer object and auto-unboxing is vice versa. So the following is ok:

public void autobox() {
    Integer i = 1; // 1 gets wrapped as new Integer(1)
    int j = i;     // j will contain 1; same as i.intValue()
}
public void autobox(int i) {
    int j = i; // Will contain value of i; same as i.intValue()
}
autobox(new Integer(23));

There’s not auto-boxing in Objective-C 2.0. So what will happen with the code above? It will compile and run! What happens is that (long) j will contain a pointer to (a copy of the original pointer num) the same NSNumber object at num.

- (void) fun3: (long) j {
    // Prints j = 1199616, the address-of the copied(!) parameter
    NSLog(@"j = %u", j);
    // Cast object at pointer j to NSNumber
    NSNumber* n = (NSNumber*) j;
    // Prints 123
    NSLog(@"n intValue = %i", [n intValue]);
    // Query the type of what j points to: NSCFNumber
    NSLog(@"class of (id) j = %@", [((id) j) class]);
    NSLog(@"%i", [((id) j) intValue]); // Will print 123, too
    NSLog(@"%i", ((id) j)); // Will print 1199616, too
}

So a more correct variant is to write parameter j as (long*) j as it’s a pointer.

Using int instead of long as parameter type in fun3 above will result in a compiler warning

warning: cast to pointer from integer of different size

as pointers are 64 bits wide (sizeof(j) == 8 * 8 = 64, on my MacBook x86_64 system) and int is 32 bits (sizeof(int) = 4 * 8 = 32).

The solution is to be sure what types are used and write the function as:

- (void) fun3: (NSNumber*) n {
    // Prints 123
    NSLog(@"n intValue = %i", [n intValue]);
}

Releasing an object: dangling pointers

When an object is released either through a message like release or through garbage collection a call to fun3 will still succeed! But: the “valid” pointer will refer to an invalid/unusable object or memory location!

// Create NSNumber object from integer value
NSNumber* num = [NSNumber numberWithInt: 123];
// Release object
[num release];
// Call fun3 with pointer to invalid NSNumber object
[self fun3: num];

// BOOM: in fun3 the pointer refers to an invalid object
- (void) fun3: (long) j {
    // Prints j = 1199616, the address-of the copied(!) parameter
    NSLog(@"j = %u", j);
    // BOOM: the following will fail as the pointer refers to released object
    NSNumber* n = (NSNumber*) j; // Cast object at pointer j to NSNumber
}

This will produce a EXC_BAD_ACCESS signal within fun3 and your program may crash.

Take care!

Resources

Permalink | Leave a comment »

Useful Oracle Shell Aliases and SQL*Plus Settings

Tue, 22 Feb 2011 02:36:00 -0800

Inspired by Christian’s blog entry about rlwrap (german), I decided to share my functions and settings for Bourne Again shell and Oracle SQL*Plus.

Install rlwrap

rlwrap is needed to provide more history and ease of navigation at the SQL*Plus prompt. To install rlwrap — the readline wrapper — type on Mac OS X:

$ sudo port install rlwrap

Or Debian, Ubuntu, …

$ sudo aptitude install rlwrap

Or on Solaris using Blastwave:

$ sudo pkgutil install CSWrlwrap

.bash_login or .bashrc

Create a login profile for the oracle user in .bash_login or .bashrc and export some of Oracle’s environment variables. The shell prompt PS1 is set to include the actual ORACLE_SID — use apostrophe to include evnrionment variable in a prompt. Add some shell functions and aliases to provide ease of use:

setsid will set the ORACLE_SID including any needed environment variables like ORACLE_ADMIN
setbase sets ORACLE_BASE to the parameter; use with sethome!
sethome sets ORACLE_HOME to $ORACLE_BASE/product/ plus its parameter
sqlplus wraps SQL*Plus in rlwrap
sysdba calls sqlplus / as sysdba, already wrapped using rlwrap
sysoper calls sqlplus / as sysoper, already wrapped using rlwrap

The script:

# Oracle
_setpath() {
    if [ $# -gt 1 ]; then
        p=$(echo $PATH | sed "s#$2##")
    else
        p=$PATH
    fi
    export PATH=$1/bin:$p
}
_oracle_settnsadmin() {
    export TNS_ADMIN=$1
    _oracle_showenv
} 
_oracle_showenv() {
    env | grep -Ee "PATH|ORACLE|TNS|NLS" | sort
}
_oracle_setsid() {
    export ORACLE_SID=$1
    export ORACLE_ADMIN=$ORACLE_BASE/admin/$ORACLE_SID
    _oracle_showenv
}
_oracle_setbase() {
    export ORACLE_BASE=$1
    _oracle_showenv
    echo "** ATTENTION: Please set ORACLE_HOME with sethome too"
}
_oracle_sethome() {
    OLDORACLE_HOME=$ORACLE_HOME
    export ORACLE_HOME=$ORACLE_BASE/product/$1
    export ORACLE_ADMIN=$ORACLE_BASE/admin/$ORACLE_SID
    export TNS_ADMIN=$ORACLE_HOME/network/admin
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib
    _setpath $OLDORACLE_HOME $ORACLE_HOME
    _oracle_showenv
}
alias setsid='_oracle_setsid'
alias setbase='_oracle_setbase'
alias sethome='_oracle_sethome'
alias settns='_oracle_settnsadmin'
alias sqlplus='rlwrap sqlplus'
alias sysdba='sqlplus / as sysdba'
alias sysoper='sqlplus / as sysoper'
alias rman='rlwrap rman'
# Customize here
export PS1='\\u@\\h:\\w ($ORACLE_SID) $ ' # Please don't use double-backslashes, use a single backslash!
setbase /u01/app/oracle >/dev/null
sethome 11.2.0/db_1 >/dev/null
setsid BLA1

Attention please don’t use double-backslashes in PS1 above, my blog just can’t display it correctly. Customize it for your system (e.g. LD_LIBRARY_PATH) or path in sethome.

Each command which changed the environment variable will print all variables belonging to Oracle in a sorted manner.

Example for setbase:

oracle@arg0:~ (BLA1) $ setbase /u01/app/oracle
LD_LIBRARY_PATH=/u01/app/oracle/product/11.2.0/lib
ORACLE_ADMIN=/u01/app/oracle/admin/AOC
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/11.2.0
ORACLE_SID=AOC
PATH=/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
PS1=\u@\h:\w ($ORACLE_SID) $ 
TNS_ADMIN=/u01/app/oracle/product/11.2.0/db_1/network/admin
** ATTENTION: Please set ORACLE_HOME with sethome too

Example for sethome:

oracle@arg0:~ (BLA1) $ sethome 11.2.0/db_1
LD_LIBRARY_PATH=/u01/app/oracle/product/11.2.0/db_1/lib
ORACLE_ADMIN=/u01/app/oracle/admin/AOC
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_SID=AOC
PATH=/u01/app/oracle/product/11.2.0/db_1/bin:/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
PS1=\u@\h:\w ($ORACLE_SID) $ 
TNS_ADMIN=/u01/app/oracle/product/11.2.0/db_1/network/admin

As you can see, ORACLE_BASE, ORACLE_HOME, ORACLE_ADMIN, TNS_ADMIN and PATH were adjusted. When you change the SID:

oracle@arg0:~ (BLA1) $ setsid RCAT
LD_LIBRARY_PATH=/u01/app/oracle/product/11.2.0/db_1/lib
ORACLE_ADMIN=/u01/app/oracle/admin/RCAT
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_SID=RCAT
PATH=/u01/app/oracle/product/11.2.0/db_1/bin:/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
PS1=\u@\h:\w ($ORACLE_SID) $ 
TNS_ADMIN=/u01/app/oracle/product/11.2.0/db_1/network/admin
oracle@arg0:~ (RCAT) $

the variables ORACLE_SID and ORACLE_ADMIN have changed.

Easy installation

Download my bash profile using wget or curl:

$ wget http://files.art-of-coding.eu/oracle/bashprofile.txt

Or include it directly into your .bash_profile like this:

$ wget -O - http://files.art-of-coding.eu/oracle/bashprofile.txt | cat >> .bash_profile

Example:

$ wget -O - http://files.art-of-coding.eu/oracle/bashprofile.txt | cat >> .bash_profile
--2011-02-22 11:26:27--  http://files.art-of-coding.eu/oracle/bashprofile.txt
Resolving files.art-of-coding.eu... 178.236.6.33
Connecting to files.art-of-coding.eu|178.236.6.33|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 824 [text/plain]
Saving to: `STDOUT'

100%[==========================================================================================>] 824         --.-K/s   in 0s      

2011-02-22 11:26:27 (70.3 MB/s) - `-' saved [824/824]

SQL*Plus prompt

Define a SQL*Plus prompt to see more than just SQL>. You can choose to create a login.sql which must be in the working directory where you execute sqlplus or put a glogin.sql in $ORACLE_HOME/sqlplus/admin:

set sqlprompt "_USER'@'_CONNECT_IDENTIFIER _PRIVILEGE> "

You’ll now see where and what you are in SQL*Plus:

$ sysdba


SQL*Plus: Release 11.2.0.1.0 Production on Tue Feb 22 11:20:30 2011

Copyright (c) 1982, 2009, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning option

SYS@RCAT AS SYSDBA>

You can even add the actual date and time:

set termout off
alter session set nls_date_format = 'DD.MM. HH24:MI:SS';
set sqlprompt "_DATE _USER'@'_CONNECT_IDENTIFIER _PRIVILEGE> "
set termout on


$ sysdba
22.02. 11:22:17 SYS@RCAT AS SYSDBA>

Resources

Oracle University about SQL*Plus' Forgotten History

Permalink | Leave a comment »

Oracle Database Parameters

Sun, 20 Feb 2011 10:04:00 -0800

Since Oracle 9i there are much more dynamic parameters which can be changed in runtime. So it’s a good thing to use server parameter files (SPFILEs) instead of plain text parameter files (PFILEs). I will use MYDB as instance name in this blog.

If you don’t use a SPFILE just create one using:

SQL> create spfile from pfile;

and bounce the instance:

SQL> shutdown
SQL> startup

The original $ORACLE_HOME/dbs/initMYDB.ora still exists but won’t be used as long as a server parameter file exists. The SPFILE is named $ORACLE_HOME/dbs/spfileMYDB.ora.

After using a SPFILE and changing parameters using ALTER SYSTEM in runtime, you can create a PFILE with the actual parameters by typing:

SQL> create pfile from spfile;

You can query these data dictionary views to get informations about Oracle’s parameters:

V$OBSOLETE_PARAMETER shows information about obsolete parameters
V$PARAMETER displays the initialization parameters that are used in the current session
V$PARAMETER2 like V$PARAMETER, but every value of a parameter appears in a row
V$PARAMETER_VALID_VALUES shows a list of valid values for parameters
V$SPPARAMETER displays contents of server parameter file
V$SYSTEM_PARAMETER displays the initialization parameters that are currently in effect for the instance
V$SYSTEM_PARAMETER2 like V$SYSTEM_PARAMETER, but every value of a parameter appears in a row

V$PARAMETER

The columns *_MODIFIABLE show whether a parameter value is modifiable with ALTER SYSTEM (ISSYS_MODIFIABLE), in the session through ALTER SESSION (ISSES_MODIFIABLE) or if every instance can have a different value (ISINSTANCE_MODIFIABLE).

ISSYS_MODIFIABLE tells us whether we can ALTER SYSTEM that parameter or not:

IMMEDIATE — can be altered regardless of what type of parameter file was used to start the instance and changes take effect immediately,
DEFERRED — can be altered regardless of what type of parameter file was used to start the instance and change will take effect in subsequent sessions,
FALSE — can only be changed when a SPFILE is used, you need to bounce the instance.

ISMODIFIED shows if you have changed the value after instance startup. Query WHERE ismodified != 'FALSE' to see them: the value MODIFIED means altered by ALTER SESSION, SYSTEM_MOD says altered by ALTER SYSTEM.

To show some parameters I reduced the list below (342 total records!) to most used ones. These parameters are from 11g:

SQL> set linesize 132
SQL> set pages 9000
SQL> select name, isdefault, isses_modifiable, issys_modifiable, isinstance_modifiable, ismodified
  2  from v$parameter order by name;

NAME                           ISDEFAULT ISSES ISSYS_MOD ISINS ISMODIFIED
------------------------------ --------- ----- --------- ----- ----------
asm_diskgroups                 TRUE      FALSE IMMEDIATE TRUE  FALSE
asm_diskstring                 TRUE      FALSE IMMEDIATE TRUE  FALSE
asm_power_limit                TRUE      TRUE  IMMEDIATE TRUE  FALSE
audit_file_dest                TRUE      FALSE DEFERRED  TRUE  FALSE
background_dump_dest           TRUE      FALSE IMMEDIATE TRUE  FALSE
buffer_pool_keep               TRUE      FALSE FALSE     FALSE FALSE
buffer_pool_recycle            TRUE      FALSE FALSE     FALSE FALSE
cluster_database               TRUE      FALSE FALSE     FALSE FALSE
compatible                     TRUE      FALSE FALSE     FALSE FALSE
control_files                  FALSE     FALSE FALSE     FALSE FALSE
core_dump_dest                 TRUE      FALSE IMMEDIATE TRUE  FALSE
cursor_sharing                 TRUE      TRUE  IMMEDIATE TRUE  FALSE
cursor_space_for_time          TRUE      FALSE FALSE     FALSE FALSE
db_16k_cache_size              TRUE      FALSE IMMEDIATE TRUE  FALSE
db_2k_cache_size               TRUE      FALSE IMMEDIATE TRUE  FALSE
db_32k_cache_size              TRUE      FALSE IMMEDIATE TRUE  FALSE
db_4k_cache_size               TRUE      FALSE IMMEDIATE TRUE  FALSE
db_8k_cache_size               TRUE      FALSE IMMEDIATE TRUE  FALSE
db_block_buffers               TRUE      FALSE FALSE     FALSE FALSE
db_block_checking              TRUE      TRUE  IMMEDIATE TRUE  FALSE
db_block_checksum              TRUE      FALSE IMMEDIATE TRUE  FALSE
db_block_size                  FALSE     FALSE FALSE     FALSE FALSE
db_cache_advice                TRUE      FALSE IMMEDIATE TRUE  FALSE
db_cache_size                  TRUE      FALSE IMMEDIATE TRUE  FALSE
db_create_file_dest            FALSE     TRUE  IMMEDIATE TRUE  FALSE
db_create_online_log_dest_1    FALSE     TRUE  IMMEDIATE TRUE  FALSE
db_create_online_log_dest_2    FALSE     TRUE  IMMEDIATE TRUE  FALSE
db_domain                      TRUE      FALSE FALSE     FALSE FALSE
db_file_multiblock_read_count  TRUE      TRUE  IMMEDIATE TRUE  FALSE
db_file_name_convert           TRUE      TRUE  FALSE     FALSE FALSE
db_flashback_retention_target  TRUE      FALSE IMMEDIATE FALSE FALSE
db_keep_cache_size             TRUE      FALSE IMMEDIATE TRUE  FALSE
db_name                        FALSE     FALSE FALSE     FALSE FALSE
db_recovery_file_dest          FALSE     FALSE IMMEDIATE FALSE FALSE
db_recovery_file_dest_size     FALSE     FALSE IMMEDIATE FALSE FALSE
db_writer_processes            TRUE      FALSE FALSE     FALSE FALSE
dbwr_io_slaves                 TRUE      FALSE FALSE     FALSE FALSE
diagnostic_dest                FALSE     FALSE IMMEDIATE TRUE  FALSE
disk_asynch_io                 TRUE      FALSE FALSE     FALSE FALSE
fast_start_io_target           TRUE      FALSE IMMEDIATE TRUE  FALSE
fast_start_mttr_target         TRUE      FALSE IMMEDIATE TRUE  FALSE
filesystemio_options           TRUE      FALSE FALSE     FALSE FALSE
hash_area_size                 TRUE      TRUE  FALSE     FALSE FALSE
instance_name                  FALSE     FALSE FALSE     FALSE FALSE
java_pool_size                 TRUE      FALSE IMMEDIATE TRUE  FALSE
job_queue_processes            TRUE      FALSE IMMEDIATE TRUE  FALSE
large_pool_size                TRUE      FALSE IMMEDIATE TRUE  FALSE
log_archive_config             TRUE      FALSE IMMEDIATE TRUE  FALSE
log_archive_dest               TRUE      FALSE IMMEDIATE TRUE  FALSE
log_archive_dest_1             TRUE      TRUE  IMMEDIATE TRUE  FALSE
log_archive_dest_2             TRUE      TRUE  IMMEDIATE TRUE  FALSE
log_archive_dest_state_1       TRUE      TRUE  IMMEDIATE TRUE  FALSE
log_archive_dest_state_2       TRUE      TRUE  IMMEDIATE TRUE  FALSE
log_archive_duplex_dest        TRUE      FALSE IMMEDIATE TRUE  FALSE
log_archive_format             TRUE      FALSE FALSE     FALSE FALSE
log_archive_max_processes      TRUE      FALSE IMMEDIATE TRUE  FALSE
log_archive_min_succeed_dest   TRUE      TRUE  IMMEDIATE TRUE  FALSE
log_archive_start              TRUE      FALSE FALSE     FALSE FALSE
log_buffer                     TRUE      FALSE FALSE     FALSE FALSE
log_checkpoint_interval        TRUE      FALSE IMMEDIATE TRUE  FALSE
log_checkpoint_timeout         TRUE      FALSE IMMEDIATE TRUE  FALSE
log_file_name_convert          TRUE      FALSE FALSE     FALSE FALSE
memory_max_target              TRUE      FALSE FALSE     FALSE FALSE
memory_target                  FALSE     FALSE IMMEDIATE TRUE  FALSE
open_cursors                   TRUE      FALSE IMMEDIATE TRUE  FALSE
open_links                     TRUE      FALSE FALSE     FALSE FALSE
open_links_per_instance        TRUE      FALSE FALSE     FALSE FALSE
optimizer_dynamic_sampling     TRUE      TRUE  IMMEDIATE TRUE  FALSE
optimizer_features_enable      TRUE      TRUE  IMMEDIATE TRUE  FALSE
optimizer_index_caching        TRUE      TRUE  IMMEDIATE TRUE  FALSE
optimizer_index_cost_adj       TRUE      TRUE  IMMEDIATE TRUE  FALSE
optimizer_mode                 TRUE      TRUE  IMMEDIATE TRUE  FALSE
os_roles                       TRUE      FALSE FALSE     FALSE FALSE
pga_aggregate_target           TRUE      FALSE IMMEDIATE TRUE  FALSE
processes                      FALSE     FALSE FALSE     FALSE FALSE
query_rewrite_enabled          TRUE      TRUE  IMMEDIATE TRUE  FALSE
recyclebin                     TRUE      TRUE  DEFERRED  TRUE  FALSE
remote_login_passwordfile      FALSE     FALSE FALSE     FALSE FALSE
service_names                  TRUE      FALSE IMMEDIATE TRUE  FALSE
session_cached_cursors         TRUE      TRUE  FALSE     FALSE FALSE
session_max_open_files         TRUE      FALSE FALSE     FALSE FALSE
sessions                       TRUE      FALSE FALSE     FALSE FALSE
sga_max_size                   TRUE      FALSE FALSE     FALSE FALSE
sga_target                     TRUE      FALSE IMMEDIATE TRUE  FALSE
shared_pool_reserved_size      TRUE      FALSE FALSE     FALSE FALSE
shared_pool_size               TRUE      FALSE IMMEDIATE TRUE  FALSE
sort_area_retained_size        TRUE      TRUE  DEFERRED  TRUE  FALSE
sort_area_size                 TRUE      TRUE  DEFERRED  TRUE  FALSE
spfile                         TRUE      FALSE IMMEDIATE TRUE  FALSE
statistics_level               TRUE      TRUE  IMMEDIATE TRUE  FALSE
streams_pool_size              TRUE      FALSE IMMEDIATE TRUE  FALSE
timed_os_statistics            TRUE      TRUE  IMMEDIATE TRUE  FALSE
timed_statistics               TRUE      TRUE  IMMEDIATE TRUE  FALSE
trace_enabled                  TRUE      FALSE IMMEDIATE TRUE  FALSE
undo_management                FALSE     FALSE FALSE     FALSE FALSE
undo_retention                 TRUE      FALSE IMMEDIATE TRUE  FALSE
undo_tablespace                FALSE     FALSE IMMEDIATE TRUE  FALSE
user_dump_dest                 TRUE      FALSE IMMEDIATE TRUE  FALSE
utl_file_dir                   FALSE     FALSE FALSE     FALSE FALSE

[...]

342 rows selected.

Resources

Permalink | Leave a comment »

Duplicate an Oracle Database with RMAN

Sat, 19 Feb 2011 04:04:00 -0800

It’s easy to duplicate a database with RMAN. Since Oracle 10g it will even create the duplicate database for you — you only have to create the instance.

Setup auxiliary instance

Create admin directory under $ORCLE_BASE/admin and password file
Register duplicate database in TNS and listener, $TNS_ADMIN/tnsnames.ora and $TNS_ADMIN/listener.ora
Create a the parameter file init.ora to start the instance, don’t forget to set DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT

I assume OFA (Optimal Flexible Architecture) directory naming here:

ORACLE_BASE = /u01/app/oracle
ORACLE_HOME = $ORACLE_BASE/product/10.2.0
TNS_ADMIN = $ORACLE_HOME/network/admin

and usage of OMF (Oracle Managed Files) and the FRA (flash recovery area):

SQL> show parameter file_dest

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest                  string      /u02/app/oracle/oradata
db_recovery_file_dest                string      /u02/app/oracle/fra
db_recovery_file_dest_size           big integer 2G

Step 1

$ mkdir /u01/app/oracle/admin/DUPDB

Step 2

Edit $TNS_ADMIN/tnsnames.ora and add the duplicate database:

ORIGDB.bensmann.com =
  (DESCRIPTION =
    (ADDRESS =
          (PROTOCOL = TCP)
          (HOST = localhost)
          (PORT = 1521)
    )
    (CONNECT_DATA = (SID = ORIGDB)
    )
  )

DUPDB.bensmann.com =
  (DESCRIPTION =
    (ADDRESS =
          (PROTOCOL = TCP)
          (HOST = localhost)
          (PORT = 1521)
    )
    (CONNECT_DATA = (SID = DUPDB)
    )
  )

Add an entry to the Oracle listener, for example:

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = ORIGDB.bensmann.com)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)
      (SID_NAME = ORIGDB)
    )
    (SID_DESC =
      (GLOBAL_DBNAME = DUPDB.bensmann.com)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)
      (SID_NAME = DUPDB)
    )
  )

and restart the listener:

$ lsnrctl stop
$ lsnrctl start

Step 3

Create the parameter file /u01/app/oracle/product/10.2.0/dbs/initDUPDB.ora. Take care and look for DUPDB string in the values and change them to reflect your naming.

This is for 10g:

db_name='DUPDB'
instance_name='DUPDB'
sga_target=150m
control_files=('controlDUPDB01.ctl', 'controlDUPDB02.ctl')
db_block_size=8192
db_create_file_dest='/u02/app/oracle/oradata'
db_create_online_log_dest_1='/u02/app/oracle/oradata'
db_create_online_log_dest_2='/u02/app/oracle/fra'
db_recovery_file_dest='/oracle/app/oradata'
db_recovery_file_dest_size=2G
background_dump_dest='/u01/app/oracle/admin/DUPDB/ddump'
core_dump_dest='/u01/app/oracle/admin/DUPDB/cdump'
user_dump_dest='/u01/app/oracle/admin/DUPDB/udump'
processes=50
undo_management='AUTO'
undo_tablespace='UNDOTBS'
remote_login_passwordfile=EXCLUSIVE

and set the parameters DB|LOG_FILE_NAME_CONVERT to reflect changes in names for logfiles datafiles. This is neccessary as the control files restored from backup contain paths to the original locations — and you want to access new files and don’t want to access or even overwrite the original ones. It’s very important if the duplicate database is on the same system as the original database ;–)

The values in *_FILE_NAME_CONVERT consist of a simple list containing a pair of mappings: OLD_PATH, NEW_PATH[, OLD_PATH, NEW_PATH ...] .

db_file_name_convert=('/u02/app/oracle/oradata/ORIGDB', '/u02/app/oracle/oradata/DUPDB', '/u02/app/oracle/fra/ORIGDB', '/u02/app/oracle/fra/DUPDB')
log_file_name_convert=('/u02/app/oracle/oradata/ORIGDB', '/u02/app/oracle/oradata/DUPDB', '/u02/app/oracle/fra/ORIGDB', '/u02/app/oracle/fra/DUPDB')

For 11g replace sga_target = 150m with memory_target = 180m and all three *_dump_dest parameters with diagnostic_dest = '/u01/app/oracle/admin/DUPDB'.

To be able to login AS SYSDBA via TNS, create a password file. This file has to be named orapwDUPDB and must be in directory $ORACLE_HOME/dbs. The password must be exactly like the one for the user SYS:

$ orapwd file=$ORACLE_HOME/dbs/orapwDUPDB password=oracle

Duplicate

To duplicate a database, run the DUPLICATE TARGET DATABASE command. RMAN will automatically create the auxiliary database itself, the only thing needed is to start the Oracle instance in NOMOUNT mode before.

Take care of your ORACLE_SID and RMAN CONNECT strings.

$ export ORACLE_SID=DUPDB
$ sqlplus / as sysdba
SQL> startup nomount

Duplicate the database ORIGDB to DUPDB:

$ rman target sys/oracle@ORIGDB catalog rman/rman@rcat auxiliary /
RMAN> duplicate target database to DUPDB;

You can re-un the command from time to time to synchronize the duplicate database (e.g. in a script).

Resources

Permalink | Leave a comment »

Dramatic Effect of MBRC

Fri, 11 Feb 2011 02:59:00 -0800

The Oracle parameter db_file_multiblock_read_count, often shorted “MBRC”, is a mighty one. If set to the wrong value your database may suffer from bad I/O. It’s up to you to test and find the right value for it.

I made some tests for a customer in the last days with their 10g Release 2 database on Solaris 10 SPARC. Here are my results.

Timings

Table T1:

MBRC = 128: 148.330 rows / 28.03 seconds = 5291,8301819 rows per second

(25209 rows were added between these tests, system itself was not modified)
MBRC = 32: 173.539 rows / 19.49 seconds = 8904,0020523 rows per second

Speedup: 8904,0020523 / 5291,8301819 * 100 = 168,25940641%, so it’s a gain of I/O performance of 68%!

Facts

Tests were made on the same system, with same database, table/query and I/O subsystem.

The storage is a Sun StorageTek 6180 with 32 FC-AL disks with a size of 300 GB. The datafiles are on a RAID-5 with 256 kB stripe size/depth and and stripe width of 6 giving a maximum possible single I/O of 256 kB * 6 = 1536 kB. The database block size is 8192 bytes. Total maximum Oracle I/O with a MBRC of 32 is 32 * 8 kB = 256 kB, with a MBRC of 128 = 128 * 8 kB = 1024 kB.

So at first it seems that a MBRC of 128 will totally fit into a single I/O using all 6 disks in the RAID. But if your data is smaller (e.g. the most I/O will fit on a single disk as it is <= 256 kB in this case) and not all disks will be used, it’s not useful to have such a high MBRC.

Read Chapter 8 I/O of Oracle Database Performance Tuning Guide, see section 8.2.1.3, table 8-2 “Minimum Stripe Depth” to find that the minium stripe size/depth should be twice as Oracle block size.

Finding big tables

To find big tables just query the table dba_tables from data dictionary. This statement will list the biggest tables with more than a million rows at the end, so it’s easy to use in SQL*Plus.

SELECT
   owner
   , table_name
   , num_rows counter
FROM
   dba_tables
WHERE
   owner NOT IN ('SYS', 'SYSMAN', 'CTXSYS', 'MDSYS', 'ORDDATA', 'XDB')
   AND owner NOT LIKE 'APEX%'
   AND num_rows > 1000000
ORDER BY
   num_rows, table_name
;

Finding long table scans

There are a lot of statistics about table scans. An interesting one is “table scans (long tables)” as it counts table scans for tables with more than 5 database blocks:

SQL> select name, value from v$sysstat where name = 'table scans (long tables)';

NAME                             VALUE
-------------------------------- -----------
table scans (long tables)        1821

Use this view to find whether your system does lots of full table scans or not.

Testing different values for MBRC

You can test the effect of MBRC yourself by creating an empty table and setting table statistics for it to reflect that it has one million rows:

SQL> create table test1 as select * from all_objects where 1 = 0;

Table created.

SQL> exec dbms_stats.set_table_stats(user, 'test1', numrows => 100000, numblks => 10000);

PL/SQL procedure successfully completed.

Use set autotrace traceonly explain in SQL*Plus and test select * from with different MBRC settings in the session. With MRBC = 1:

SQL> set autotrace traceonly explain
SQL> alter session set db_file_multiblock_read_count = 1;

Session altered.

SQL> select * from test1;

Execution Plan
----------------------------------------------------------
Plan hash value: 4122059633

---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |   100K|  9765K| 10006   (1)| 00:02:38 |
|   1 |  TABLE ACCESS FULL| TEST1 |   100K|  9765K| 10006   (1)| 00:02:38 |
---------------------------------------------------------------------------

And with MBRC = 64, for example:

SQL> alter session set db_file_multiblock_read_count = 64;

Session altered.

SQL> set autotrace traceonly explain
SQL> select * from test1;

Execution Plan
----------------------------------------------------------
Plan hash value: 4122059633

---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |   100K|  9765K|  1415   (1)| 00:00:23 |
|   1 |  TABLE ACCESS FULL| TEST1 |   100K|  9765K|  1415   (1)| 00:00:23 |
---------------------------------------------------------------------------

A script to test different settings of DB_FILE_MULTIBLOCK_READ_COUNT

I wrote a PL/SQL procedure TEST_MBRC to test different settings:

CREATE OR REPLACE PROCEDURE test_mbrc(
    p_table IN VARCHAR2
    , p_mbrc_start IN PLS_INTEGER DEFAULT 1
    , p_mbrc_end IN PLS_INTEGER DEFAULT 128
    , p_flush_buffer_cache IN BOOLEAN DEFAULT FALSE
)
AS
    v_count PLS_INTEGER;      -- number of records
    v_start_time PLS_INTEGER; -- time in 100th of a second
    v_timediff BINARY_DOUBLE; -- time in 100th of a second
    v_besttime BINARY_DOUBLE; -- time in 100th of a second
    v_bestmbrc PLS_INTEGER;   -- best value of MBRC
    v_throughput PLS_INTEGER; -- read rows per second
BEGIN
    v_bestmbrc := 0;
    v_besttime := 9999;
    FOR v_mbrc IN p_mbrc_start .. p_mbrc_end LOOP
        -- Flush buffer cache?
        IF p_flush_buffer_cache THEN
            EXECUTE IMMEDIATE 'ALTER SYSTEM FLUSH BUFFER_CACHE';
        END IF;
        -- Alter MBRC in session
        EXECUTE IMMEDIATE 'ALTER SESSION SET db_file_multiblock_read_count = ' || v_mbrc;
        -- Start time
        v_start_time := dbms_utility.get_time();
        DBMS_OUTPUT.PUT('MBRC = ' || TO_CHAR(v_mbrc, '999'));
        -- Select
        EXECUTE IMMEDIATE 'SELECT /*+ FULL(t) NOPARALLEL(t) NOCACHE(t) */ COUNT(*) FROM ' || p_table || ' t' INTO v_count;
        -- Time difference; 100th of a second
        v_timediff := dbms_utility.get_time() - v_start_time;
        IF v_timediff < v_besttime THEN
            v_besttime := v_timediff;
            v_bestmbrc := v_mbrc;
        END IF;
        IF v_timediff > 0 THEN
            v_throughput := v_count / v_timediff;
        ELSE
            v_throughput := v_count;
        END IF;
        -- Convert 100th of a second to second for output
        DBMS_OUTPUT.PUT_LINE(
            '... took' || TO_CHAR(v_timediff / 100, '999990.99') || ' seconds'
            || ' to read' || TO_CHAR(v_count, '99999999') || ' rows'
            || ', throughput =' || TO_CHAR(v_throughput, '9999990.99') || ' rows per second'
        );
    END LOOP;
    DBMS_OUTPUT.PUT_LINE('Best MBRC is ' || v_bestmbrc);
END;
/

This script will issue a select /*+ full(t) noparallel(t) nocache(t) */ count(*) from table t against a specified table, display all timings for that statement with different MBRCs and the best MBRC/time:

SQL> exec test_mbrc('test1000', 1, 128, false);

The parameter p_flush_buffer_cache enables a feature to flush the buffer cache before each select.

System statistics, the CBO and the MBRC

The following has been tested on a Linux server with 2 SATA disks in a software RAID-1.

To get the maximum possible MBRC for your system set db_file_multiblock_read_count to a extremly high value and check what Oracle did with it:

SQL> set autotrace off
SQL> alter session set db_file_multiblock_read_count = 32768;
SQL> show parameter multiblock

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_file_multiblock_read_count        integer     4096

On my Linux server the maximum is 4096.

If db_file_multiblock_read_count is set to zero or not set, Oracle will determine it on its own and the CBO (cost based optimizer) will use a default of MBRC = 8.

Use DBMS_STATS.GATHER_SYSTEM_STATS('START') and DBMS_STATS.GATHER_SYSTEM_STATS('STOP') to gather system statistics over a period of time:

SQL> exec dbms_stats.gather_system_stats('start');

PL/SQL procedure successfully completed.

Create a test table with e.g. 3 million rows:

SQL> create table test1000 (id number,name varchar2(100));

Table created.

SQL> declare
  2  begin
  3    for i in 1 .. 3000000 loop
  4      execute immediate 'insert into test1000 values (:1, :2)' using i, 'blabla ' || i;
  5    end loop;
  6  end;
  7  /

PL/SQL procedure successfully completed.

Now test different values for MBRC, note the different times and throughput:

SQL> set serveroutput on size 1000000
SQL> set linesize 132

SQL> exec test_mbrc('test1000');
MBRC =    1... took  1.66 seconds to read  3000000 rows, throughput =   18072.00 rows per second
MBRC =    2... took  1.32 seconds to read  3000000 rows, throughput =   22727.00 rows per second
MBRC =    3... took  1.12 seconds to read  3000000 rows, throughput =   26786.00 rows per second
[...]
MBRC =    4... took  1.24 seconds to read  3000000 rows, throughput =   24194.00 rows per second
[...]
MBRC =    8... took  1.18 seconds to read  3000000 rows, throughput =   25424.00 rows per second
[...]
MBRC =   16... took  1.20 seconds to read  3000000 rows, throughput =   25000.00 rows per second
[...]
MBRC =   24... took  1.17 seconds to read  3000000 rows, throughput =   25641.00 rows per second
[...]
MBRC =   32... took  1.03 seconds to read  3000000 rows, throughput =   29126.00 rows per second
[...]
MBRC =   64... took  0.99 seconds to read  3000000 rows, throughput =   30303.00 rows per second
[...]
MBRC =   82... took  0.95 seconds to read  3000000 rows, throughput =   31579.00 rows per second
[...]
MBRC =  128... took  1.18 seconds to read  3000000 rows, throughput =   25424.00 rows per second
Best MBRC is 82

PL/SQL procedure successfully completed.

Attention: if you run this script many times, you may see varying results, even on a idle system! So test different settings of MBRC and compare to Oracle’s statistics:

SQL> exec dbms_stats.gather_system_stats('stop');

PL/SQL procedure successfully completed.

The data dictionary table SYS.AUX_STATS$ shows the result:

SQL> col pname format a20
SQL> col pval2 format a40
SQL> select * from sys.aux_stats$;

SNAME                          PNAME                PVAL1      PVAL2
------------------------------ -------------------- ---------- ----------------------------------------
SYSSTATS_INFO                  STATUS                          COMPLETED
SYSSTATS_INFO                  DSTART                          02-10-2011 13:44
SYSSTATS_INFO                  DSTOP                           02-10-2011 13:50
SYSSTATS_INFO                  FLAGS                         1
SYSSTATS_MAIN                  CPUSPEEDNW             1421,591
SYSSTATS_MAIN                  IOSEEKTIM                13,718
SYSSTATS_MAIN                  IOTFRSPEED                 4096
SYSSTATS_MAIN                  SREADTIM                 70,082
SYSSTATS_MAIN                  MREADTIM                  1,822
SYSSTATS_MAIN                  CPUSPEED                   1419
SYSSTATS_MAIN                  MBRC                          4
SYSSTATS_MAIN                  MAXTHR                 85218304
SYSSTATS_MAIN                  SLAVETHR               14120960

13 rows selected.

Oracle says:

IOSEEKTIM: Average I/O seek time of 13 ms
IOTFRSPEED: Storage performes maximum of 4096 bytes/ms (note: per milliseconds)
SREADTIM: Average single block read time of 70 ms
MREADTIM: Average multiblock read time of 1,8 ms
MBRC of 4 will be used
MAXTHR: Maxmium I/O throughput of 85.218.304 bytes/sec = 85 MB/sec

See Chapter 14 Manging Optimizer Statistics, table 14-2 in Oracle Database Performance Tuning Guide 10g Release 2 for an explanation of all values.

Alternatively you can use DBMS_STATS.GATHER_SYSTEM_STATS('interval', interval => N) to compute statistics for a certain amount of time. N is the number of minutes.

So Oracle determined a MBRC of 4 and will use it now — regardless of the setting of db_file_multiblock_read_count! Setting db_file_multiblock_read_count manually has no effect:

SQL> alter system set db_file_multiblock_read_count = 82;

System altered.

SQL> set autotrace trace
SQL> select /*full(t)*/ count(*) from test1000 t;


Execution Plan
----------------------------------------------------------
Plan hash value: 2018862720

-----------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Cost (%CPU)| Time     |
-----------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     1 |  3737   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |          |     1 |            |          |
|   2 |   TABLE ACCESS FULL| TEST1000 |  3000K|  3737   (1)| 00:00:45 |
-----------------------------------------------------------------------


Statistics
----------------------------------------------------------
    132  recursive calls
      0  db block gets
   9898  consistent gets
   9880  physical reads
      0  redo size
    533  bytes sent via SQL*Net to client
    523  bytes received via SQL*Net from client
      2  SQL*Net roundtrips to/from client
      3  sorts (memory)
      0  sorts (disk)
      1  rows processed

It shows the same time as the MBRC = 4 from system statistics:

SQL> alter system set db_file_multiblock_read_count = 4;

System altered.

SQL> set autotrace trace
SQL> select /*full(t)*/ count(*) from test1000 t;


Execution Plan
----------------------------------------------------------
Plan hash value: 2018862720

-----------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Cost (%CPU)| Time     |
-----------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     1 |  3737   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |          |     1 |            |          |
|   2 |   TABLE ACCESS FULL| TEST1000 |  3000K|  3737   (1)| 00:00:45 |
-----------------------------------------------------------------------


Statistics
----------------------------------------------------------
    132  recursive calls
      0  db block gets
   9898  consistent gets
   9880  physical reads
      0  redo size
    533  bytes sent via SQL*Net to client
    523  bytes received via SQL*Net from client
      2  SQL*Net roundtrips to/from client
      3  sorts (memory)
      0  sorts (disk)
      1  rows processed

After deleting system statistics with DBMS_STATS.DELETE_SYSTEM_STATS:

SQL> exec dbms_stats.delete_system_stats;

PL/SQL procedure successfully completed.

you can play around with it and compare the value from the PL/SQL procedure TEST_MBRC and Oracle’s value:

SQL> alter system set db_file_multiblock_read_count = 82;

System altered.

SQL> set autotrace trace
SQL> select /*full(t)*/ count(*) from test1000 t;

Execution Plan
----------------------------------------------------------
Plan hash value: 2018862720

-----------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Cost (%CPU)| Time     |
-----------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     1 |  1785   (2)| 00:00:22 |
|   1 |  SORT AGGREGATE    |          |     1 |            |          |
|   2 |   TABLE ACCESS FULL| TEST1000 |  3000K|  1785   (2)| 00:00:22 |
-----------------------------------------------------------------------


Statistics
----------------------------------------------------------
    132  recursive calls
      0  db block gets
   9898  consistent gets
   9880  physical reads
      0  redo size
    533  bytes sent via SQL*Net to client
    523  bytes received via SQL*Net from client
      2  SQL*Net roundtrips to/from client
      3  sorts (memory)
      0  sorts (disk)
      1  rows processed


SQL> alter system set db_file_multiblock_read_count = 4;

System altered.

SQL> set autotrace trace
SQL> select /*full(t)*/ count(*) from test1000 t;

Execution Plan
----------------------------------------------------------
Plan hash value: 2018862720

-----------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Cost (%CPU)| Time     |
-----------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     1 |  3737   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |          |     1 |            |          |
|   2 |   TABLE ACCESS FULL| TEST1000 |  3000K|  3737   (1)| 00:00:45 |
-----------------------------------------------------------------------


Statistics
----------------------------------------------------------
    132  recursive calls
      0  db block gets
   9898  consistent gets
   9880  physical reads
      0  redo size
    533  bytes sent via SQL*Net to client
    523  bytes received via SQL*Net from client
      2  SQL*Net roundtrips to/from client
      3  sorts (memory)
      0  sorts (disk)
      1  rows processed

And even 32 will do a good job:

SQL> alter system set db_file_multiblock_read_count = 32;

System altered.

SQL> set autotrace trace
SQL> select /*full(t)*/ count(*) from test1000 t;


Execution Plan
----------------------------------------------------------
Plan hash value: 2018862720

-----------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Cost (%CPU)| Time     |
-----------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     1 |  1937   (2)| 00:00:24 |
|   1 |  SORT AGGREGATE    |          |     1 |            |          |
|   2 |   TABLE ACCESS FULL| TEST1000 |  3000K|  1937   (2)| 00:00:24 |
-----------------------------------------------------------------------


Statistics
----------------------------------------------------------
    132  recursive calls
      0  db block gets
   9898  consistent gets
   9880  physical reads
      0  redo size
    533  bytes sent via SQL*Net to client
    523  bytes received via SQL*Net from client
      2  SQL*Net roundtrips to/from client
      3  sorts (memory)
      0  sorts (disk)
      1  rows processed

You certainly have to test it on your own!

Set them all

This script will set the MBRC in all of your Oracle database instances:

#!/bin/ksh
# Create SQL script to set MBRC
cat > setmbrc.sql <

Resources

Permalink | Leave a comment »

Configuring JDK 6 on Mac OS X

Mon, 07 Feb 2011 09:43:00 -0800

I needed to install Oracle JDeveloper and WebLogic Server, but the installer did not find an existing Java installation.

After some research I saw that the directory structure of the JDK is not compatible to the “normal” setup. After some research on the web, it’s easy to setup a JAVA_HOME environment variable, create a link for a certain Java archive and let the installer recognize the JDK correctly.

First, open a terminal and go to your Java 6 installation at /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home:

rbemac2:~ rbe$ cd /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
rbemac2:Home rbe$

Next create a directory jre with a subdirectory lib in it and go into it (you have to be root for this, so use sudo):

rbemac2:Home rbe$ sudo mkdir -p jre/lib
Password:
rbemac2:Home rbe$ cd jre/lib
rbemac2:lib rbe$

Then create a link from classes.jar to rt.jar:

rbemac2:lib rbe$ sudo ln -s ../../../Classes/rt.jar classes.jar

It should look like this:

rbemac2:lib rbe$ ls -l
total 8
lrwxr-xr-x  1 root  wheel  28  3 Feb 12:43 rt.jar -> ../../../Classes/classes.jar

Last but not least create a environment.plist in a directory .MacOSX (note the notation, lower and upper case letters) under your home directory and put the following in it:

rbemac2:~ rbe$ cat /Users/rbe/.MacOSX/environment.plist



  
    JAVA_HOME
    /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home

You have to log out and in again to activate this setting. Open a terminal and go to $JAVA_HOME.

rbemac2:~ rbe$ echo $JAVA_HOME
/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
rbemac2:~ rbe$ cd $JAVA_HOME
rbemac2:Home rbe$

After these steps, the installer will recognize your JDK and perform the installation.

Permalink | Leave a comment »

Java Training für Einsteiger

Sat, 05 Feb 2011 02:50:00 -0800

Mein neues Java-Training für Einsteiger (erschienen Dezember 2010 bei Galileo Press, ISBN-10 3-8362-1568-3, ISBN-13 978-3-8362-1568-8) ist auf Platz 1 der Bestseller-Liste bei Amazon im Bereich Software/Java!

(Gesehen gestern, 04.02.2011)

Das Videotraining ist erhältlich bei Amazon, Lehmanns, Libri, buecher.de, buch24.de, edv-buchversand.de, bilandia.de, weiteren Händlern und bei eBay.

Ich hoffe es gefällt allen Käufern!

Das Training

Steigen Sie ein in die Programmierung mit Java! Ihr Trainer Ralf Bensmann erklärt Ihnen anschaulich und praxisorientiert alle Grundlagen von Java. Lernen Sie direkt am Bildschirm, wie Sie Streams, Netzwerke, Servlets und Datenbanken bis hin zu einem eigenen Twitter-Client programmieren. Sehen Sie einem Profi beim Live-Coding zu und erfahren Sie, wie Sie selber mit Java objektorientiert programmieren und eigene Projekte realisieren. Aus dem Inhalt:

JDK und Eclipse installieren
Datentypen, Operatoren und Kontrollstrukturen in Java
Objektorientierung
Java Collections Framework
Generics
Exceptions und Assertions
Debugging mit Eclipse
Streams, Reader und Writer
Grafische Benutzeroberflächen mit Swing
Datenbanken
Parallele Ausführung
Networking und Servlets
Einen Twitter-Client programmieren

Dieses Video-Training hat 83 Lektionen und eine Gesamtspielzeit von 8 Stunden. Dieses Video-Training ist lauffähig ohne Installation auf folgenden Systemen:

Windows
Mac
Linux

In jedem Fall brauchen Sie ein DVD-Laufwerk, Lautsprecher und eine Auflösung des Monitors von mindestens 1024 x 768 Pixeln.

Sie möchten endlich lernen, mit Java selbstständig zu programmieren? Dann ist dieses Training genau das Richtige für Sie! Ralf Bensmann erklärt Ihnen Java von Grund auf anhand vieler praktischer Beispiele zum Nachprogrammieren und vermittelt Ihnen wertvolle Tipps und Tricks aus der eigenen Berufspraxis. Sie lernen Film für Film alle Konzepte von Java kennen, inklusive Objektorientierte Programmierung und Implementierung von Standard-Algorithmen. Ausführlich widmet sich Ihr Trainer auch der Programmierung grafischer Oberflächen mit Swing. Sie lernen alles über Streams, Networking und Servlets und programmieren selber Ihre ersten Anwendungen in Java.

Aus dem Inhalt:

Datentypen und Kontrollstrukturen in Java
Operatoren
Objektorientierung
Entwurfsmuster
Generische Klassen
Algorithmen
Streams, Reader und Writer
Grafische Benutzeroberflächen mit Swing
Weboberflächen mit Servlets
Networking
Twitter-Client

Dieses Video-Training hat 100 Lektionen und eine Gesamtspielzeit von 8 Stunden.

Dieses Video-Training ist lauffähig ohne Installation auf folgenden Systemen: * Windows 7, Vista und XP

In jedem Fall brauchen Sie ein DVD-Laufwerk, Lautsprecher und eine Auflösung des Monitors von mindestens 1024 x 768 Pixeln.

Weitere Links

Problelektionen bei Galileo, etwas nach unten scrollen
Eintrag im Katalog der DNB — Deutschen Nationalbibliothek.

Permalink | Leave a comment »

Apple MacBook Pro and NVIDIA Graphics

Wed, 02 Feb 2011 01:29:00 -0800

UPDATE 2011-08-28

After many tries (re-installing Mac OS X with Apple Care, exchange the logic board through a support partner) to get the MBP stable, I went to Genius Bar at an Apple Store and they said “Let us check the system, we call you”. Ok, I would give it a try as someone told me that going to an Apple store is the better way.

When I was at home I decided “No, I bought another MBP a few months after the defect one, because I just have to work, so even if they repair it, I cannot use it anymore!”. So I called Apple and explained my case to them — it took a little while ;–) Solution: they exchange the defect MBP with a brand-new iMac 27", Intel 3,4 GHz i7, 8 GB RAM, AMD Radeon 2 GB, 1 TB hard disk!

Why the hell did this took so long? Why didn’t they exchange the defect MBP a lot earlier?

The original post

Ok, I am an Apple fan. Two MacBook Pro, two iPhones and an iPod classic and really happy with it.

But the last experience with my new MacBook Pro (bought 11/2010) was annoying. It had system freezes every day: from one per day to several times per day or per hour.

My 15-inch MBP has:

Intel Core i7 with 2 cores at 2,66 GHz
8 GB RAM
500 GB 7200rpm hard drive and
two graphics chips:
1. Intel HD Graphics with 256 MB shared RAM and a
2. NVIDIA 330M with 512 MB dedicated RAM
and the Hi-Res 1680x1050 display.

Symptoms

When the external monitor is not attached and the MBP switches automatically between the both graphic cards, it freezes sporadically. When a beamer is connected using a Mini DisplayPort to VGA adapter and the beamer (DisplayPort) is disconnected, the system freezes immediately (even with my solution described below).

In my (special?) case it helped to attach an external monitor to get the system rock stable as usual for weeks I’ve got a 24-inch BenQ W2400V connected through the Mini DisplayPort to DVI adapter.

Even reinstalling Mac OS X 10.6.4 (my MBP was delivered with this version) without installing third-party software (I use Parallels Desktop and VirtualBox which install their own kernel extensions) did not help. Please try this yourself before calling Apple Care or your service partner.

:–(

There must be a flaw in the NVIDIA driver, every time the system freezes one of the following happened:

Everything is unaccessible and unusable, just the mouse pointer can be moved
Screen is showing a solid color, yellow in most cases
In rare situations there were some screen flickering and random shapes

Log

The log shows a NVDA(OpenGL): Channel Exception with different subtypes of errors like Watchdog Timeout Error, SW Notify Error or FBIState Timeout Error:

Sometimes a kernel panic is logged after rebooting through power cycle. It seemed to happen after symptom #2 (see above):

Finding a (temporary?) solution

I found a lot of threads in the Apple forum:

and on non-Apple pages:

For me the best tip was to download a free software called “gfxCardStatus” from here. It seems to help to turn off the NVIDIA chip and use the Intel card only and always:

Install gfxCardStatus
Open preferences and ensure that it will started at boot

Reboot
Open the menu and select “Intel only”

Choose NVIDIA only if you use an external monitor, as it can be used with it only. Hope your’s will be stable as mine: mobile = “Intel only”, desktop w/ external monitor = “NVIDIA only”.

It’s really a pity to be unable to use the NVIDIA graphics card, but I can do my work. Now I will go and ask Apple for a solution.

Tools

Permalink | Leave a comment »

Clojure: When an Agent Calls Another Agent

Sat, 29 Jan 2011 00:25:13 -0800

Agents in Clojure are functions that are applied “in the background” — a thread. Any function in Clojure implements java.lang.Runnable. To see what happens when an agent calls another agent, I set up two agents agt1 and agt2, their state is just an integer:

rbemac2:~ rbe$ clj
Clojure 1.2.0


user=> (def agt1 (agent 0))
#'user/agt1
user=> (def agt2 (agent 1))
#'user/agt2

Now just increment the number by 1:

user=> (send agt1 (fn [val] (inc val)))
#7f401d28: 1>
user=> (send agt2 (fn [val] (inc val)))
#20d12eea: 2>

As expectd agt1 has the value 1 and agt2 the value 2. If agt1 sends an update to agt2, what is the result?

user=> (send agt1 (fn [val] (send agt2 inc)))
#7f401d28: #20d12eea: 2>>

They both have the value 3

user=> @agt1
#20d12eea: 3>
user=> @agt2
3

because agt2 == 2 + 1 and agt1 refers to agt2. So dereferencing the agent agt1 returns the agt2 with an actual value of 3. You can now “double-deref” agt1 to get the value:

user=> @@agt1
3

Take care and return a result of the original value of the agent when you don’t want to apply the result to agt1:

user=> (def agt1 (agent 0))
#'user/agt1
user=> (def agt2 (agent 1))
#'user/agt2
user=> (send agt1 (fn [val] (inc val)))
#1420c906: 1>
user=> (send agt2 (fn [val] (inc val)))
#3a0b53e: 2>
user=> (send agt1 (fn [val] (send agt2 inc) val))
#1420c906: 1>
user=> @agt1
1
user=> @agt2
3

Permalink | Leave a comment »

Alte Lebensweisheit

Thu, 20 Jan 2011 12:36:00 -0800

Die Jugend von heute liebt Luxus, hat schlechte Manieren und verachtet Autorität. Sie widersprechen ihren Eltern, legen die Beine übereinander und tyrannisieren ihre Lehrer.

Sokrates, 470 v.Chr.

Permalink | Leave a comment »

Comparing Strings With Metrics in Haskell

Thu, 20 Jan 2011 07:20:00 -0800

I wrote a module for String distance metrics two years ago in Haskell. You find this code at github.com.

The module

module Hompare
(
jaroWinkler
, levenshtein
, cologneMethod
)
where

import Prelude
import Data.Char (toUpper)

defines two global helper functions, intListToInt to convert a list of integers into a number, e.g. [1, 2, 3, 4] to 1234:

1
2
3

-- Convert list of integer to one integer: [1,2,3,4] to 1234.
intListToInt :: [Integer] -> Integer
intListToInt = foldl (\a x -> x + a * 10) 0

and commonPrefix to determine the common prefix shared by two strings:

-- Common prefix of two strings; up to 4 characters
commonPrefix :: String -> String -> String
commonPrefix "" "" = ""
commonPrefix "" _ = ""
commonPrefix _ "" = ""
commonPrefix s1 s2
	| s1 == s2 = take 4 s1
	| otherwise = cp s1 s2 []
	where
		cp s1 s2 acc
			| length acc == 4 = acc
			| head s1 == head s2 = head s1 : cp (tail s1) (tail s2) acc
			| otherwise = acc

The module exports three functions to compare strings with metrics: jaroWinkler, levenshtein and cologneMethod. The function cologneMethod implements the Kölner Verfahren which is designed for the german language, while Jaro Winkler and Levenshtein can be used in general.

-- Compute Jaro distance between two strings
jaro :: String -> String -> Double
jaro "" _ = 0.0
jaro _ "" = 0.0
jaro s1 s2
	| s1 == s2 = 1.0
	| s1 /= s2 =
		let
			-- Length of both strings
			l1 = length s1
			l2 = length s2
			-- Index s2
			z2 = zip [1..] s2
			m = foldl (++) [] [charMatch p ((max l1 l2) `div` 2) z2 | p <- zip [1..] s1]
			ml = length m
			t = sum [realToFrac (transposition p z2) / 2.0 | p <- m]
			ml1 = realToFrac ml / realToFrac l1
			ml2 = realToFrac ml / realToFrac l2
			mtm = (realToFrac ml - t) / realToFrac ml
		in
			(1 / 3) * (ml1 + ml2 + mtm)
		where
			-- [] of matching characters for 1 character
			charMatch (p,q) far list = filter (\(x,y) -> x >= p - far && x <= p + far && y == q) list
			-- # of transpositions for 1 character
			transposition (p,q) list = length $ filter (\(x,y) -> p /= x && q == y) list

-- Compute Winkler distance between two strings on top of Jaro distance
winkler :: String -> String -> Double -> Double
winkler "" _ _ = 0.0
winkler _ "" _ = 0.0
winkler s1 s2 jaro
	| s1 == s2 = 1.0
	| s1 /= s2 =
		let
			l = length $ commonPrefix s1 s2
			p = 0.1
		in
			jaro + ((realToFrac l * p) * (1.0 - jaro))

-- Compute Jaro-Winkler distance between two strings
jaroWinkler :: String -> String -> Double
jaroWinkler "" _ = 0.0
jaroWinkler _ "" = 0.0
jaroWinkler s1 s2
	| s1 == s2 = 1.0
	| s1 /= s2 = winkler s1 s2 $ jaro s1 s2

-- Levenshtein distance between two strings
levenshtein :: String -> String -> Int
levenshtein "" "" = 0
levenshtein s1 "" = length s1
levenshtein "" s2 = length s2
levenshtein s t = d !! (length s) !! (length t)
	where
		d = [[distance m n | n <- [0..length t]] | m <- [0..length s]]
		distance i 0 = i
		distance 0 j = j
		distance i j =
			minimum [d !! (i - 1) !! j + 1,
			         d !! i !! (j - 1) + 1,
				 d !! (i - 1) !! (j - 1) + (if s !! (i - 1) == t !! (j - 1) then 0 else 1)]

-- Cologne Method: compute 'Kölner Verfahren' for a string
cologneMethod :: String -> Integer
cologneMethod "" = 0
cologneMethod s = intListToInt $ foldr cons [] $ filter (> 0) $ cologne [toUpper x | x <- s] '\NUL'
	where
		-- Construct list of Ints while adjacent elements would never be the same
		cons :: Integer -> [Integer] -> [Integer]
		cons x [] = [x]
		cons x acc
			| x == head acc = acc
			| otherwise = x:acc
		--
		cologne :: String -> Char -> [Integer]
		cologne "" _ = [0]
		cologne s p = cm (head s) p (s !! 1) : cologne (tail s) (head s)
		-- Return code for character
		cm :: Char -> Char -> Char -> Integer
		-- TODO UTF-8 cm 'Ä' p n = cm 'AE' p n
		-- TODO UTF-8 cm 'Ö' p n = cm 'OE' p n
		-- TODO UTF-8 cm 'Ü' p n = cm 'UE' p n
		-- TODO UTF-8 cm 'ß' p n = cm 'SS' p n
		cm c p n
			| c `elem` "AEIJOUY" = 0
			| c == 'H' = 0
			| c == 'B' = 1
			| c == 'P' && n /= 'H' = 1
			| c `elem` "DT" && not (p `elem` "CSZ") = 2
			| c `elem` "FVW" = 3
			| c == 'P' && n == 'H' = 3
			| c `elem` "GKQ" = 4
			| c == 'C' && p == '\NUL' && n `elem` "AHKLOQRUX" = 4
			| c == 'C' && p /= '\NUL' && n `elem` "AHKOQUX" && not (p `elem` "SZ") = 4
			| c == 'X' && not (n `elem` "CKQ") = 48
			| c == 'L' = 5
			| c `elem` "MN" = 6
			| c == 'R' = 7
			| c `elem` "SZ" = 8
			| c == 'C' && n `elem` "SZ" = 8
			| c == 'C' && p == '\NUL' && not (n `elem` "AHKLOQRUX") = 8
			| c == 'C' && not (n `elem` "AHKOQUX") = 8
			| c `elem` "DT" && p `elem` "CSZ" = 8
			| c == 'X' && n `elem` "CKQ" = 8
			| otherwise = 0

To test these functions, here is a little program:

module TestHompare
where

import Text.Printf
import Control.Exception
import Control.Concurrent (threadDelay)
import System.CPUTime
import Hompare

time :: IO t -> IO t
time a = do
    start <- getCPUTime
    v <- a
    --threadDelay 1000000
    end <- getCPUTime
    let diff = (fromIntegral (end - start)) / (10^12)
    printf "Computation time: %d - %d = %0.3f sec\n" end start (diff :: Double)
    return v

main = do
	time $ print $ jaroWinkler "MARTHA" "MARHTA"
	time $ print $ jaroWinkler "DIXON" "DICKSONX"
	time $ print $ cologneMethod ""
	time $ print $ cologneMethod "Wikipedia"
	time $ print $ cologneMethod "Muller-Ludenscheidt"

Permalink | Leave a comment »