chris goes xp

CSV files in MS Excel and Oracle Open Office

2011-02-24T12:29:00.019+01:00

My input file is created in MS Excel, it is 1 row of data with 3 columns (in the second column there is a newline character):

When I save this file as .csv in excel I get following structure:

COL1,COL2,COL3
³ó ?????,"2 äöü ??? lines
In 1 field ???",+123456

File is corrupted

When you open this file in Excel no multibytes and even lost "+" in phone number but the new line character properly inserted.

When I save the input Excel file as Unicode Text .txt the Unicode (MS Unicode ucs-2le) all the issues with .csv will be resolved (while saving).

COL1 COL2 COL3

łóś ещьул "2 äöü ыва lines
In 1 field ещь" +123456

Unfortunately while opening such file with Text Import Wizard:

In third step you can change type to Text (to rescue phone number):

After clicking "Finish" the result is dissapointing:

Multibytes and phone number is properly shown, but the newline is broken.

When you change the extension of the previous Unicode File from .txt to .csv and double click on it to open it with Excel you will have new line and multibytes but the phone will be corrupted:

When you try to open the same Unicode .csv file with Oracle Open Office:
File -> Open -> .csv file
In Import Wizard you set character set to Unicode (it's MS Unicode), delimiter to Tab and column types to Text (to rescue phone number):

Success story:

The question is:
"Why excel cannot properly open a CSV file created by itself?"

Or maybe there is a workaround (without modyfing the CSV file) to do so?

Oracle Open Office FTW!!!

validating emails in oracle db using java stored function => and testing from ruby

2010-10-22T15:01:00.013+02:00

1. email validation function is written in java (flexibility) and compiled as stored procedure in oracle db (performance)

1a) write EmailValidations.java:

import java.lang.String;
import java.util.regex.Pattern;
import java.util.regex.Matcher;

class EmailValidations {
  public static int is_valid(String email) {
    if (email == null || email == "") return 0;

    String upper_email = email.toUpperCase();

    Pattern p = Pattern.compile("^[\\w\\.-]*[^\\.]@([\\w\\-]+\\.)+[A-Z]{2,4}$");
    Matcher m = p.matcher(upper_email);
    return m.matches() ? 1: 0;
  }
}

1b) load and compile java class into your oracle db:

$ loadjava -u scott/tiger@emeadb11 -v -resolve EmailValidations.java
--> replace scott/tiger@emeadb11 with your valid username/password@tnsname_entry

2a). create package that will contain your function

create or replace PACKAGE             "MISC_PKG"
IS
FUNCTION is_email_valid (email_address in varchar) RETURN number;
END MISC_PKG;
/

2b). create package body:

create or replace
PACKAGE BODY             "MISC_PKG" AS

FUNCTION is_email_valid (email_address in varchar) RETURN number
AS LANGUAGE JAVA
NAME 'EmailValidations.is_valid(java.lang.String) return int';

END MISC_PKG;
/

3) test with ruby-plsql

3a) spec_helper.rb that will contain your credentials and include rspecs and ruby-plsql gems

require 'rubygems'
require 'rspec'
require 'ruby-plsql'

DB_USER = "scott"
DB_PWD  = "tiger"
DB_SRVR = "emeadb11"

plsql.connection = OCI8.new DB_USER, DB_PWD, DB_SRVR
p "connected"

plsql.connection.autocommit = false

RSpec.configure do |config|
  # config.filter_run :focus => true
  config.after(:each) do
    plsql.rollback
  end
end

3b) write ruby specs spec_is_email_valid.rb

require File.dirname(__FILE__) + '/spec_helper'

describe "check is_email_valid function" do
  
  it "should return 1 for valid emails" do
    plsql.misc_pkg.is_email_valid("c123@o2.pl").should == 1 
    plsql.misc_pkg.is_email_valid("c@o2.pl").should == 1 
  end
  
  it "should return 0 for invalid emails" do
    plsql.misc_pkg.is_email_valid("c123@o2,pl").should == 0 
    plsql.misc_pkg.is_email_valid("c@o2pl").should == 0 
  end

  it "should return 0 as in email_err_reason" do
    plsql.misc_pkg.is_email_valid("S.J@GMAIL").should == 0
    plsql.misc_pkg.is_email_valid("SY.STORT").should == 0
    plsql.misc_pkg.is_email_valid("01516081607").should == 0
  end
    
    
  it "should return 0 when invalid characters in it" do
    plsql.misc_pkg.is_email_valid("J.AMS.@CATS.ORG").should == 0
    plsql.misc_pkg.is_email_valid("STNJ@ISON.CO.ZA/JACO@VTR.CO.ZA").should == 0
  end
  
  it "should return 0 when null email" do
    plsql.misc_pkg.is_email_valid("").should == 0
    plsql.misc_pkg.is_email_valid(nil).should == 0
  end
end

4) you can test it with:

$ rspec spec_is_email_valid.rb 
"connected"
.....

Finished in 5.64 seconds
5 examples, 0 failures

You can easily extend your tests once the change was done to your is_valid() java function.

For more information on testing pl/sql functions from ruby see excelent blog (author of ruby-plsl):
http://blog.rayapps.com/2009/11/27/oracle-plsql-unit-testing-with-ruby/

1und1.de domain with heroku rails app

2010-08-29T23:30:00.006+02:00

1. assuming you have an app on heroku appname.heroku.com
2. change CNAME entry for DNS on 1und1.de
- Domains
- mark your domainat1und1.de and select DNS -> Einstellungen Bearbeiten
- for CNAME entry put: appname.heroku.com
- apply and wait couple of hours until it’s populated
3. in your heroku app you need to add an add-on “Custom Domains” (Basic is free)

$ heroku addons:add custom_domains
Adding custom_domains to ... FAILED
!   Please verify your account before installing this addon
!   For more information, see http://docs.heroku.com/billing
!   Verify now at http://heroku.com/verify

as you can see you need to verify your account before (credit card needed -> no charge applies)

when it’s done:

$ heroku addons:add custom_domains:basic
Adding custom_domains:basic to ... done

$ heroku domains:add www.domainat1und1.de
Added www.domainat1und1.de as a custom domain name to appname.heroku.com
$ heroku domains:add domainat1und1.de
Added domainat1und1.de as a custom domain name to appname.heroku.com

FULL instructions under: http://docs.heroku.com/custom-domains

oracle db - how to identify compilation errors

2009-10-20T12:59:00.006+02:00

select * from user_errors;

SELECT RTRIM(INITCAP(e.type))||' '||e.name||': Line '||
      TO_CHAR(e.line)||' Column '||TO_CHAR(e.position) linecol,
      s.text sourceline,RPAD(' ',(e.position - 1))||'^' errpos,
      e.text error
FROM   user_source s,user_errors e
WHERE  e.type = s.type
AND    e.name = s.name
AND    e.line = s.line
ORDER BY e.name,e.sequence

The latter one will give you the exact line where the error occcurs

encrypting partitions with LUKS in Linux

2009-09-23T16:08:00.004+02:00

with these commands we encrypt a full partition located under /dev/sde1
(UUID=7a89446f-c43f-41b1-bc2b-99c74c64cdeb)

In the below steps I use dcfldd (which is much more user friendly than dd and gives you progress status) and cryptsetup.

sudo apt-get install dcfldd cryptsetup

sudo dcfldd if=/dev/urandom of=/dev/sde1 statusinterval=10 bs=10M conv=notrunc
sudo cryptsetup luksFormat /dev/sde1
sudo cryptsetup luksOpen /dev/sde1 maxtor_encrypted
sudo mkfs.ext3 /dev/mapper/maxtor_encrypted
sudo mkdir /mnt/maxtor_enc
sudo mount /dev/mapper/maxtor_encrypted /mnt/maxtor_enc -t ext3 -o noatime,nodiratime

it is safer to use UUIDs instead of /dev/sde1 because even if order of the drives changes (USB drives, or because of your BIOS) you always can uniquely identify the one with the unique UUID.
You can get the UUIDs of all of your disks' partitions by running

$ blkid

Now let's write to scripts for mounting and umounting our encrypted drive:

chris@emeadb:~/bin$ cat mycryptmount
#!/bin/bash
export PATH=/usr/local/bin/:/bin/:/usr/bin/
echo "mounting encrypted drive"

df | grep maxtor_enc && sudo /home/chris/bin/mycryptumount
echo "maxtor_enc umounted"

sudo cryptsetup luksOpen /dev/disk/by-uuid/7a89446f-c43f-41b1-bc2b-99c74c64cdeb maxtor_encrypted
sudo mount /dev/mapper/maxtor_encrypted /mnt/maxtor_enc -t ext3 -o noatime,nodiratime

chris@emeadb:~/bin$ cat mycryptumount
#!/bin/bash
export PATH=/usr/local/bin/:/bin/:/usr/bin/

sudo umount /dev/mapper/maxtor_encrypted
sudo cryptsetup luksClose maxtor_encrypted

chmod +x both of them

now you are able to mount with

$ mycryptmount

and umount with

$ mycrypumount

Ada - jedne z pierwszych prob efektywnego poruszania sie

2009-09-13T15:01:00.000+02:00

ruby oracle oci8 UTF8 corruption

2009-05-28T21:06:00.003+02:00

I have a small script oci1.rb:

require 'oci8' 
conn = OCI8.new("test","test","test_db") 
File.open("out.txt","wb") do |out| 
  tabs = conn.exec('SELECT * from test_utf8') do |r| 
     out.write(r.join(',')) 
     out.write("\n") 
  end 
end 
conn.logoff

that runs perfectly on windows producing from my test_db:

polish,łóśżć 
russsian,фывафыва 
german,äöü

when I run exactly the same on my linux box:

polish,loszc 
russsian,???????? 
german,aou

I tested then a small script that reads and writes UTF8 on linux to be sure my linux terminal can display UTF8
(read_write_utf8.rb):

File.open("utf8_out.txt","wb") do |out| 
  File.open("utf8.txt","r").each do |line| 
    out.write(line) 
  end 
end

and that works good!

chris@emeadb:~/work/ruby/oci$ ruby read_write_utf8.rb 
chris@emeadb:~/work/ruby/oci$ cat utf8.txt 
polish,łóśżć 
russsian,фывафыва 
german,äöü 
chris@emeadb:~/work/ruby/oci$ cat utf8_out.txt 
polish,łóśżć 
russsian,фывафыва 
german,äöü

Finally the issue was with my NLS_LANG variable that was not set in the Linux box on the account I used for running the script.

I tried first to set it up in the ruby script itself:

ENV['NLS_LANG']='AMERICAN_AMERICA.UTF8'

that did not work.

the solution was to set it in the shell.

export NLS_LANG=AMERICAN_AMERICA.UTF8

you can set it in your .bashrc (depending on the shell you use) to make it default for the account.

debian change from dhcp to static IP

2009-04-22T16:20:00.005+02:00

One would think to achieve the update from the DHCP assigned IP to the static one, the only thing to do is to update your /etc/network/interfaces

from:

iface eth0 inet dhcp

to:

iface eth0 inet static
        address 10.165.248.252
        netmask 255.255.248.0
        gateway 10.165.248.1
        broadcast 10.165.248.255
        network 10.165.248.0

and restart:

/etc/init.d/networking restart

(optionally you can adjust your /etc/resolv.conf (to update DNS servers, and search paths)

The changed IP is immediately visible.

How surprised I was when after couple of hours the IP was changed back to the one from the DHCP pool. (resolv.conf was then automatically updated from the DHCP server)

Solution was to kill the dhcp3-client. (and change again the resolv.conf and restart networking)

ddl over database link in oracle 10gR2

2009-03-30T17:42:00.007+02:00

recently I encountered a requirement to create a table on remote database DB_TARGET (working in DB_SOURCE)

for example:

create table table_a@dblink_to_db_target; --> does not work
-- ORA-02021: DDL operations are not allowed on a remote database

whereas

select 'hello' from dual@dblink_to_db_target; --> works

This is a workaround I used:
on TARGET_DB:

CREATE OR REPLACE 
FUNCTION prepare_table 
   ( table_name in varchar2) return varchar2 
   IS 
    err_num NUMBER; 
    err_msg VARCHAR2(100); 
    sqlstmt varchar2(4000) := ''; 
    tbl varchar2(30) := ''; 
BEGIN 
    if instr(table_name,'.') > 0 then 
        tbl := substr(table_name,instr(table_name,'.')+1); 
        sqlstmt := 'CREATE TABLE ' || tbl || ' as select * from ' || 
table_name; 
        execute immediate sqlstmt; 
        return 'table ' || tbl || ' successfully created'; 
    else 
        return 'table in owners'' schema'; 
    end if; 
EXCEPTION 
    WHEN others THEN 
        err_msg := SUBSTR(SQLERRM, 1, 100); 
        return 'error when creating ' || tbl || '-' || err_msg || '-' 
|| sqlstmt; 
END; 
/

this function creates a table specified as a table_name parameter only
if the "." is in the name => goal is to create a table in my_schema
from other user's schema.
this function works fine when calling this on DB_TARGET:

SQL> set serveroutput on; 
SQL> declare 
  2  result varchar2(300) := ''; 
  3  begin 
  4  result := prepare_table('other_user.table_a'); 
  5  dbms_output.put_line(result); 
  6  end; 
  7  / 
table table_a successfully created 

PL/SQL procedure successfully completed.

but then surprisingly to me:
when I try to call this function from DB_SOURCE:

SQL> set serveroutput on; 
SQL> declare 
  2  result varchar2(300) := ''; 
  3  begin 
  4  result := prepare_table@dblink_to_db_target('other_user.table_a'); 
  5  dbms_output.put_line(result); 
  6  end; 
  7  / 
error when creating gcd_countries-ORA-02064: distributed operation not 
supported-CREATE TABLE table_a as select * from other_user.table_a 

PL/SQL procedure successfully completed.

I found a workaround with a wrapper procedure on DB_TARGET:

CREATE OR REPLACE 
PROCEDURE prepare_table_proc(table_name in varchar2) 
   IS 
   result varchar2(300) := ''; 
BEGIN 
    result := prepare_table(table_name); 
    dbms_output.put_line('success'); 
EXCEPTION 
    WHEN others THEN 
        dbms_output.put_line('err'); 
END; 
/

when I call it from DB_SOURCE like this:

SQL> call prepare_table_proc@dblink_to_db_target('other_user.table_a'); 
Call completed.

And the table_a is created on DB_TARGET.

when we look closer why the function itself did not work we see in the description of the error message:
ORA-02064: distributed operation not supported
Cause: One of the following unsupported operations was attempted:
1. array execute of a remote update with a subquery that
references a dblink, or
2. an update of a long column with bind variable and an update of
a second column with a subquery that both references a dblink and a
bind variable, or
3. a commit is issued in a coordinated session from an RPC
procedure call with OUT parameters or function call.

in my case point 3. yields and hence the error message.

Oracle documentation comes with help in:
http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_sql.htm#sthref6148

it works only as a procedure and we cannot return a value to the invoking DB_SOURCE anyway.

maybe there's a better way...
right now I am happy with the workaround

why you should always use aliases in SQL (oracle)

2009-02-18T09:14:00.004+01:00

let's create for this example following 2 tables (a and b):

create table a (org_id number, org_name varchar2(1)); 
insert into a values(1,'a'); 
insert into a values(2,'b'); 
insert into a values(3,'c');
commit;

create table b (name varchar2(1)); 
insert into b values('a'); 
insert into b values('b');
commit;

Now let's run this:

select * from a 
where a.org_id in (select org_id from b); 
-- returns all 3 rows from a 

select * from a 
where a.org_id not in (select org_id from b); 
-- returns 0 rows

The tricky part is, you might want the org_id in the subquery to be taken from b (which does not exist) and hence expect an INVALID IDENTIFIER error message as running only subquery:

select org_id from b 
        * 
ORA-00904: "ORG_ID": invalid identifier

But the initial query is correlated and actually the org_id in the subquery comes from outer table a and not from b.

That's why a good practice would be always to use aliases as prefixes to the columns.

In that way we can rewrite the query that will now raise an exception as expected:

select * from a 
where a.org_id not in (select b.org_id from b); 
-- ORA-00904: "B"."ORG_ID": invalid identifier

vim converting between MS Unicode UCS-2LE and UTF-8

2009-01-22T09:25:00.006+01:00

Many times at work I need to convert between 2 different encodings from Unicode:
1. MS Unicode -> generated e.g. by Excel when Saving data as Unicode text
2. UTF8 encoding (standard)

Requirement VIM is compiled with multi_byte support.
check if in VIM normal mode

:echo has('multi_byte')

returns 1

In VIM normal mode to open a file with certain encoding:

:e ++enc=<encoding> <filename>

To save in different encoding:

:w ++enc=<encoding> <filename>

e.g.
open in ucs-2le -> save in utf-8

:e ++enc=ucs-2le /tmp/file_ucs2le.csv
:w ++enc=utf-8 /tmp/file_utf8.csv

It works even if your fileencoding in your .vimrc (_vimrc on Windows) is different and VIM normally would recognize as different encoding from the one you want.

++enc sets fileencoding and not encoding (filencoding is needed when opening or saving the file)

More on Unicode in VIM under
http://vim.wikia.com/wiki/Working_with_Unicode

compiling VIM 7.2 with GTK2 under debian etch

2009-01-20T10:12:00.004+01:00

To install GTK2 is not an issue since we can use the apt-get install libgtk2.0 libgtk2.0-dev to install gtk2 with dev libraries.

The issue I encountered was when I tried to compile VIm 7.2 with GTK2, which could not be found:

checking for GTK - version >= 2.2.0... no
checking for GTK - version >= 1.1.16... yes; found version 1.2.10

Then I decided to compile GTK2 on my own.

I failed when trying to install GTK2.14 with GLIB2.18.4 because some function definitions changed.

Finally the versions I successfully installed were:

- gtk2.8.20
- glib2.9.6
- pangoo1.11
- atk1.25.2
- cairo1.8.6
- pixman0.13.2

then to compile VIM i used (/opt/gtk2 is the prefix where I installed just compiled gtk2):

export PKG_CONFIG_PATH="/opt/gtk2/lib/pkgconfig"
export LD_LIBRARY_PATH=/opt/gtk2/lib
export CPPFLAGS="-I/opt/gtk2/include"
export LDFLAGS="-L/opt/gtk2/lib"
 ./configure --prefix=/opt/vim/ --enable-perlinterp --enable-rubyinterp --enable-multibyte --enable-gui --with-features=big

Finally GTK2 found:

checking for GTK - version >= 2.2.0... yes; found version 2.8.20

happy VImming :)

accessing Github from behind HTTP proxy

2008-12-16T11:24:00.005+01:00

When you need to pull or push from Github (git://) and you're behind corporate HTTP firewall, you might find below instructions helpful:

1. set environmental variables (in .bashrc or .bash_profile)

export GIT_SSH="${HOME}/bin/http-ssh"
export GIT_PROXY_COMMAND="${HOME}/bin/http-cmd"

2. prepare http-ssh and http-cmd in your ${HOME}/bin/ directory (or any other of your choice -> just adjuct the environmental variables accordingly)

#!/bin/sh
# Filename: http-ssh
ssh -o ProxyCommand="${HOME}/bin/http-cmd %h %p" $@

#!/bin/sh
# Filename: http-cmd
/usr/bin/connect -H http://<your_http_proxy>:<port> $@

now you can freely use git command.

sftp from behind firewall via HTTP proxy in Ruby

2008-12-09T16:39:00.004+01:00

Same as the previous Post using ruby:

require 'rubygems'
require 'net/ssh'
require 'net/sftp'
require 'net/ssh/proxy/http'

base_dir = "/some_base_directory"

proxy_host = 'IP_or_FQDN_of_your_proxy'
proxy_port = port_of_your_proxy

proxy = Net::SSH::Proxy::HTTP.new(proxy_host,proxy_port)

Net::SSH.start('destination_IP_or_FQDN','username',
                {:proxy         => proxy,
                 :password      => 'password'}) do |ssh|

        ssh.sftp.connect do |sftp|
                # do something
        end
end

ssh sftp from behind firewall via HTTP proxy

2008-12-09T16:18:00.006+01:00

If you work in corporate network that is behind firewall.

if you try this:

chris@emeadb:~$ sftp username@xx.xx.xx.xx
Connecting to xx.xx.xx.xx...
ssh: connect to host xx.xx.xx.xx port 22: No route to host
Couldn't read packet: Connection reset by peer

With help comes connect-proxy
Install it with (in debian like systems):

sudo apt-get install connect-proxy

then configure it:

vi ~/.ssh/config

adjust to your needs the content:

Host *
        ProxyCommand connect-proxy -H <IP or FQDN of your proxy>:<port> %h %p

enjoy:

chris@emeadb:~$ sftp username@xx.xx.xx.xx
Connecting to xx.xx.xx.xx...
Secure SFTP Serverusername@xx.xx.xx.xx's password:

enabling/disabling ARCHIVELOG in Oracle 10g

2008-11-27T09:29:00.008+01:00

to check your current mode:

SELECT log_mode FROM v$database;

LOG_MODE
------------
ARCHIVELOG

to enable:

SHUTDOWN;
STARTUP MOUNT EXCLUSIVE;
ALTER DATABASE ARCHIVELOG;
ALTER DATABASE OPEN;

to disable:

SHUTDOWN;
STARTUP MOUNT EXCLUSIVE;
ALTER DATABASE NOARCHIVELOG;
ALTER DATABASE OPEN;

If you receive this error message:

SQL> alter database noarchivelog;
alter database noarchivelog
*
ERROR at line 1:
ORA-38774: cannot disable media recovery - flashback database is enabled

It means flashback database is set, we need to disable flashback first

SQL> alter database flashback off;

Database altered.

SQL> alter database noarchivelog;

Database altered.

SQL> alter database open;

Database altered.

SQL> SELECT log_mode FROM v$database;

LOG_MODE
------------
NOARCHIVELOG

oracle archiver hangs because space_used = space_limit

2008-10-30T14:46:00.005+01:00

Here is how to solve the issue without extending DB_RECOVERY_FILE_DEST_SIZE:

rman target /
RMAN> delete archivelog until time 'SYSDATE - 1';
or
RMAN> delete archive all;

RMAN> delete expired archivelog all;

sql loader and tab separated multibyte fields

2008-09-04T10:59:00.004+02:00

when you use following part in your control file:

fields terminated by X'0009' optionally enclosed by X'0022'

you probably notice that the multi tab characters got swelled to one tab character, which causes that null value fields are gone, and the next non-null fields got shifted in that way, that there are no null values between 2 non-null value fields in db.

The solution to this issue is removing the "optionally enclosed ..." statement.

10g hangs because of Archiver is unable to archive a redo log because the output device

2008-07-24T10:09:00.006+02:00

the most probable option is that there is no place for your archive logs.
If you use flash_recovery_area, increase the size there:

$ sqlplus /nolog

SQL> connect sys as sysdba;
Enter password: 
Connected.
SQL> select * from v$recovery_file_dest;

if SPACE_USED is the same or close to SPACE_LIMIT it's definitely this reason (no place to store archive logs)

increase the size of flash_recovery area:

SQL> alter system set db_recovery_file_dest_size = 50g scope=BOTH;

this will set the size of the flash_recovery_area to 50g immediately, and after restart as well (setting the parameter to SPFILE)

If your database is not opened, try to shut it down first.

SQL> shutdown immediate

if that takes long time then

SQL> shutdown abort
SQL> startup nomount

do the stuff written above

SQL> alter database open;

sqlplus enabling arrow keys for command history

2008-07-16T13:13:00.005+02:00

It is very annoying when you in sqlplus under Linux cannot use ARROW keys to choose between executed commands.
there is a workaround for that, it's name is rlwrap:

sudo apt-get install rlwrap

then you can execute:

rlwrap sqlplus /nolog

or create an alias on sqlplus to be executed as rlwrap sqlplus.

thanks,
chris

burn big files >4GB under Linux

2008-06-12T17:46:00.004+02:00

1. prepare a DVD size file:

dd if=/dev/zero of=dvd_file.iso bs=1024 count=4589800
mkudffs dvd_file.iso
mkdir iso 
mount -o loop -t udf dvd_file.iso iso
cp your_big_file_to_burn iso
umount iso

2. burn it

cdrecord -verbose -dao -driveropts=burnfree dvd_file.iso

creating image of NTFS drive under Linux - ntfsclone

2008-06-11T12:46:00.008+02:00

use rescucd (www.sysresccd.org with ntfsclone in ver.2.0) for this task.

0. mount backup medium on /mnt

1. save NTFS image:

ntfsclone --save-image --output /mnt/winxp.img /dev/sda2

2. restore NTFS image from image file:

ntfsclone --restore-image --overwrite /dev/sda2 /mnt/winxp.img

UPDATE:
if your target backup drive is limited to some smaller file size (like 4GB) you might want to compress and split the image into multiple files:
e.g. (all one line)

ntfsclone -s -o - /dev/sda2 |gzip|split -b 2000m -  /mnt/winxp.img.gz_

this will create 2GB big multiple files for that image

To restore: (assuming 3 files were created aa, ab, ac)

cat /mnt/winxp.img.gz_aa /mnt/winxp.img.gz_ab /mnt/winxp.img.gz_ac  |gunzip -c|ntfsclone --restore-image --overwrite /dev/sda2 -

partimage to backup restore drive images

2008-06-11T09:52:00.007+02:00

prerequisites:
1. partimage (www.partimage.org here 0.6.7)
2. rescuecd (www.sysresccd.org here 1.0.3) and burn it on CD
3. sfdisk
4. 2 drives with same size (or the one that you will be restoring to bigger size) + the backup HDD with some file system on it (in my case ext3 to hold big data files)

HowTo Backup:
1. attach the backup drive (can be USB HDD, stick, whatever that will save your images - in my case I use 300GB USB drive)
2. Boot the rescuecd
3. to find out the location of the drive in my case /dev/sdb1, use command

fdisk -l

- if this is linux fs drive with ext3 , use following command:

mount -t ext3 /dev/sdb1 /mnt

4. save original MBR with dd

dd if=/dev/sda of=backup-sda.mbr count=1 bs=512

5. save all the extended partition entries

sfdisk -d /dev/sda > backup-sda.sf

IMPORTANT: save both files from 4. and 5. in safe places (they will be needed to restore the structure of the original to the new disk)

6. run partimage command

7. mark the partition that you want to back up (e.g. /dev/sda3)
8. in the Image file to create/use enter the path to the backup disk
e.g.

/mnt/sda3_partimage

9. in Action to be done:
mark - Save partition into a new image file
10. press F5 to go to Next Page
11. Choose required Compression level (e.g. gz)
12. Image split mode if the partitions you save are big, it is wise to split them into the file that fit on CD or DVD (I enter here 4500MB)
After continuing with F5 you give a description and the process starts, generating files:
sda3_partimage.000, sda3_partimage.001, etc
be careful these file are gzip'ed file, please rename them to sda3_partimage.00X.gz for X in 0,1,2..

How To Restore:
1. attach the backup drive containing your partimage files and mbr and extended partition entries
2. attach empty drive (in my case /dev/sda)
3. Boot the rescuecd
4. to find out the location of the backup drive in my case /dev/sdb1, use command

fdisk -l

- if this is linux fs drive with ext3 , use following command:

mount -t ext3 /dev/sdb1 /mnt

5. restore MBR and extended partition entries:

- dd if=/mnt/backup-sda.mbr of=/dev/sda
- sfdisk /dev/sda <> check with fdisk -l if all partitions made on /dev/sda

6. make sure backup file names on backup drive (/mnt) are renamed to gz, then gunzip them:

gunzip sda3_patimage.000.gz (will create much bigger sda3_partimage.000 file),     
... or     
gunzip sda3_partimage.*.gz

7. run partimage command
8. Partition to save/resotre (in my case sda3)
9. Image file: enter the path to your gunziped image files /mnt/sda3_partimage.000
even if you have more than 1 file, always put the location of the 000 file
10. Action to be done: pick Restore partition from an image file After continuing with F5 you should get restored partition on /dev/sda3
11. format SWAP partition (in my case /dev/sda5 -> check your fdisk -l)

mkswap -c /dev/sda5

12. Install GRUB boot-loader

mkdir /new_disk
mount /dev/sda3 /new_disk
grub-install --root-directory=/new_disk /dev/sda

re-installing GRUB

2008-04-06T20:19:00.005+02:00

Situation:
You have linux and windows on your hdd,
As it happened you needed to reinstall M$ Windows, after that operation your GRUB is not visible after reboot.

Resolution:
1. run any of the Linux LiveCD distros (I used Ubuntu)
2. in Console type (as root or sudo):

->  grub

3. in Grub app (I had my linux partition on /dev/sda7 which is mapped to hd0,6 in GRUB):

->  root (hd0,6)
->  setup (hd0,6)

4. reboot
5. you're done if everything went fine -> I hope so :)

thanks,
chris

pierwsze proby przemieszczania sie :)

2007-11-24T23:22:00.001+01:00

Oracle to_date function dates from 20th century

2007-09-17T15:32:00.000+02:00

there is a very useful function in Oracle DB to_date. But when you run:

select to_date('78/06/01','YY/MM/DD') from dual;

you might be expecting a date '1-Jun-1978' (might differ depending on your local nls settings), but you get surprisingly: '1-Jun-2078'.
Helpful is parameter RR instead of YY.
RR returns years from 21st century when given value is in range 0-49
and returns years from 20th century when given value is in range 50-99

For most cases you probably want:

select to_date('78/06/01','RR/MM/DD') from dual;

thanks
chris

Mencoder converting files

2007-09-01T01:50:00.000+02:00

Ever wanted to play a file that your favourite mplayer does not want to play?
Or it shows other difficulties?

Recently I got some tech movies in .flv (flash) format.
Mplayer1.0.1rc1 plays them, but you cannot seek in those files.
Problematic could be as well .mov files.
For the .flv files you can checkout the latest version of mplayer from the subversion.
You can though convert to a format that mplayer accepts without problems (like .avi)

mencoder input.flv -oac pcm -ovc lavc -o output.avi
mencoder input.mov -oac pcm -ovc lavc -o output.avi

Happy watching....
chris

Pidgin more instances

2007-08-29T15:34:00.000+02:00

Why would you have a need to use more instances of IM pidgin, when you can use in one instance more protocols, etc?

For me the reason was:
- to log only Buddies messages and not IRC chats (they grow fast)
- to flash the Pidgin window only when I receive the messages from Buddies and not from IRC chats

To achieve it create a small bat file that will run second, third, ... instance of the Pidgin

set PIDGIN_MULTI_INST=1
c:\programms\pidgin2.1\pidgin -c c:\.pidgin-alt

where you need to apply paths where you installed your pidgin (in my case in programms\pidgin2.1\) after -c parameter is the path to the directory that will hold all settings for the second, third, etc... instance if this directory does not exist, it will be created)

running that bat file will run in a seperate instance and there you can make independent settings.

happy instant messaging.
chris

for my honey

2007-08-28T18:30:00.000+02:00

    To make it clear
    there is only one
    and it is not beer,
    She smiles and cries
    but never lies
    as I hold that line
    I'm glad that she's mine.

    I code some stuff,
    I drink, have fun
    but she's the one working so hard.
    She waits for me,
    wiping her neck,
    Wait honey I'll be right back!!!

RMagick install from source in Debian Etch

2007-08-28T13:23:00.000+02:00

Installing RMagick from sources (6.3.5)

1. Download the latest imagemagick from http://www.imagemagick.org/script/install-source.php#unix

- compile with follwoing options (assuming you don't need it for perl nor C++)


./configure --prefix=/opt/imagemagick --enable-shared --disable-static --without-perl --without-magick-plus-plus

of course you can change prefix or remove it when you want it to be isntalled by default

./make
sudo make install

Make sure /opt/imagemagick/includes/lib is in your LD_LIBRARY_PATH
and /opt/imagemagick/bin in your PATH

2. Download the latest RMagick from http://rubyforge.org/projects/rmagick/

./configure
./make

If at this stage you get errors like "unable to read font '(null)'":
- check first if your ImageMagick installation was successful running display command
- if running display you get "unable to find font .... helvetica ..." then you need to install fonts on your system
- if display runs fine but you still get the above error from one of ruby test scripts setting fonts options in ./configure of imagemagick might help
- try to configure again ImageMagick with following options:

./configure --prefix=/opt/imagemagick --enable-shared --disable-static --without-perl --without-magick-plus-plus  --with-gs_font-dir=/usr/share/fonts/type1/gsfonts --with-windows-font-dir=/home/chris/win_fonts

(applying appropriate paths)

./make
sudo make install

- compile again RMagick
- still not running? ...

I hope that helps
chris

Oracle - table changes how to notice

2007-08-28T11:43:00.000+02:00

Couple of days ago I had a real problem how to provide always the most recent report with the products hierarchy. The first thought was to write a kind of a hash function to get just a hash stamp of the whole table and compare against the new one. The problem arose when it came to implementing the idea.
There is no built way to get a hash value of a whole table.

Suddenly I thought: "OK, let's check if the number of rows is different, then it would mean the content of the table changed".

I walked that way.
Replacing the table I always create first the new one with some different name and if everything goes fine, I replace the name to the right one at the end.

I have a production table called prods_hier and the new one in the background called prods_hier_bak

the following lines of code do the comparing:


changed:=false
select count(*) into no_of_rows_new from prods_hier;
select count(*) into no_of_rows_bak from prods_hier_bak;
if no_of_rows_new <> no_of_rows_bak then
    changed := true;
else -- the same no_of_rows
    select count(*) into no_of_changed_rows from
    ( select * from prods_hier
      minus
      select * from prods_hier_bak
    );
    if no_of_changed_rows <> 0 then
        changed := true;
    end if;
end if;