accessing Github from behind HTTP proxy

When you need to pull or push from Github (git://) and you're behind corporate HTTP firewall, you might find below instructions helpful:

1. set environmental variables (in .bashrc or .bash_profile)

export GIT_SSH="${HOME}/bin/http-ssh"
export GIT_PROXY_COMMAND="${HOME}/bin/http-cmd"

2. prepare http-ssh and http-cmd in your ${HOME}/bin/ directory (or any other of your choice -> just adjuct the environmental variables accordingly)

#!/bin/sh
# Filename: http-ssh
ssh -o ProxyCommand="${HOME}/bin/http-cmd %h %p" $@

#!/bin/sh
# Filename: http-cmd
/usr/bin/connect -H http://<your_http_proxy>:<port> $@

now you can freely use git command.

sftp from behind firewall via HTTP proxy in Ruby

Same as the previous Post using ruby:

require 'rubygems'
require 'net/ssh'
require 'net/sftp'
require 'net/ssh/proxy/http'

base_dir = "/some_base_directory"

proxy_host = 'IP_or_FQDN_of_your_proxy'
proxy_port = port_of_your_proxy

proxy = Net::SSH::Proxy::HTTP.new(proxy_host,proxy_port)

Net::SSH.start('destination_IP_or_FQDN','username',
                {:proxy         => proxy,
                 :password      => 'password'}) do |ssh|

        ssh.sftp.connect do |sftp|
                # do something
        end
end

ssh sftp from behind firewall via HTTP proxy

If you work in corporate network that is behind firewall.

if you try this:

chris@emeadb:~$ sftp username@xx.xx.xx.xx
Connecting to xx.xx.xx.xx...
ssh: connect to host xx.xx.xx.xx port 22: No route to host
Couldn't read packet: Connection reset by peer

With help comes connect-proxy
Install it with (in debian like systems):

sudo apt-get install connect-proxy

then configure it:

vi ~/.ssh/config

adjust to your needs the content:

Host *
        ProxyCommand connect-proxy -H <IP or FQDN of your proxy>:<port> %h %p

enjoy:

chris@emeadb:~$ sftp username@xx.xx.xx.xx
Connecting to xx.xx.xx.xx...
Secure SFTP Serverusername@xx.xx.xx.xx's password:

enabling/disabling ARCHIVELOG in Oracle 10g

to check your current mode:

SELECT log_mode FROM v$database;

LOG_MODE
------------
ARCHIVELOG

to enable:

SHUTDOWN;
STARTUP MOUNT EXCLUSIVE;
ALTER DATABASE ARCHIVELOG;
ALTER DATABASE OPEN;

to disable:

SHUTDOWN;
STARTUP MOUNT EXCLUSIVE;
ALTER DATABASE NOARCHIVELOG;
ALTER DATABASE OPEN;

If you receive this error message:

SQL> alter database noarchivelog;
alter database noarchivelog
*
ERROR at line 1:
ORA-38774: cannot disable media recovery - flashback database is enabled

It means flashback database is set, we need to disable flashback first

SQL> alter database flashback off;

Database altered.

SQL> alter database noarchivelog;

Database altered.

SQL> alter database open;

Database altered.

SQL> SELECT log_mode FROM v$database;

LOG_MODE
------------
NOARCHIVELOG

oracle archiver hangs because space_used = space_limit

Here is how to solve the issue without extending DB_RECOVERY_FILE_DEST_SIZE:

rman target /
RMAN> delete archivelog until time 'SYSDATE - 1';
or
RMAN> delete archive all;

RMAN> delete expired archivelog all;

sql loader and tab separated multibyte fields

when you use following part in your control file:

fields terminated by X'0009' optionally enclosed by X'0022'

you probably notice that the multi tab characters got swelled to one tab character, which causes that null value fields are gone, and the next non-null fields got shifted in that way, that there are no null values between 2 non-null value fields in db.

The solution to this issue is removing the "optionally enclosed ..." statement.

10g hangs because of Archiver is unable to archive a redo log because the output device

the most probable option is that there is no place for your archive logs.
If you use flash_recovery_area, increase the size there:

$ sqlplus /nolog

SQL> connect sys as sysdba;
Enter password: 
Connected.
SQL> select * from v$recovery_file_dest;

if SPACE_USED is the same or close to SPACE_LIMIT it's definitely this reason (no place to store archive logs)

increase the size of flash_recovery area:

SQL> alter system set db_recovery_file_dest_size = 50g scope=BOTH;

this will set the size of the flash_recovery_area to 50g immediately, and after restart as well (setting the parameter to SPFILE)

If your database is not opened, try to shut it down first.

SQL> shutdown immediate

if that takes long time then

SQL> shutdown abort
SQL> startup nomount

do the stuff written above

SQL> alter database open;

sqlplus enabling arrow keys for command history

It is very annoying when you in sqlplus under Linux cannot use ARROW keys to choose between executed commands.
there is a workaround for that, it's name is rlwrap:

sudo apt-get install rlwrap

then you can execute:

rlwrap sqlplus /nolog

or create an alias on sqlplus to be executed as rlwrap sqlplus.

thanks,
chris

burn big files >4GB under Linux

1. prepare a DVD size file:

dd if=/dev/zero of=dvd_file.iso bs=1024 count=4589800
mkudffs dvd_file.iso
mkdir iso 
mount -o loop -t udf dvd_file.iso iso
cp your_big_file_to_burn iso
umount iso

2. burn it

cdrecord -verbose -dao -driveropts=burnfree dvd_file.iso

creating image of NTFS drive under Linux - ntfsclone

use rescucd (www.sysresccd.org with ntfsclone in ver.2.0) for this task.

0. mount backup medium on /mnt

1. save NTFS image:

ntfsclone --save-image --output /mnt/winxp.img /dev/sda2

2. restore NTFS image from image file:

ntfsclone --restore-image --overwrite /dev/sda2 /mnt/winxp.img

UPDATE:
if your target backup drive is limited to some smaller file size (like 4GB) you might want to compress and split the image into multiple files:
e.g. (all one line)

ntfsclone -s -o - /dev/sda2 |gzip|split -b 2000m -  /mnt/winxp.img.gz_

this will create 2GB big multiple files for that image

To restore: (assuming 3 files were created aa, ab, ac)

cat /mnt/winxp.img.gz_aa /mnt/winxp.img.gz_ab /mnt/winxp.img.gz_ac  |gunzip -c|ntfsclone --restore-image --overwrite /dev/sda2 -

partimage to backup restore drive images

prerequisites:
1. partimage (www.partimage.org here 0.6.7)
2. rescuecd (www.sysresccd.org here 1.0.3) and burn it on CD
3. sfdisk
4. 2 drives with same size (or the one that you will be restoring to bigger size) + the backup HDD with some file system on it (in my case ext3 to hold big data files)

HowTo Backup:
1. attach the backup drive (can be USB HDD, stick, whatever that will save your images - in my case I use 300GB USB drive)
2. Boot the rescuecd
3. to find out the location of the drive in my case /dev/sdb1, use command

fdisk -l

- if this is linux fs drive with ext3 , use following command:

mount -t ext3 /dev/sdb1 /mnt

4. save original MBR with dd

dd if=/dev/sda of=backup-sda.mbr count=1 bs=512

5. save all the extended partition entries

sfdisk -d /dev/sda > backup-sda.sf

IMPORTANT: save both files from 4. and 5. in safe places (they will be needed to restore the structure of the original to the new disk)

6. run partimage command

7. mark the partition that you want to back up (e.g. /dev/sda3)
8. in the Image file to create/use enter the path to the backup disk
e.g.

/mnt/sda3_partimage

9. in Action to be done:
mark - Save partition into a new image file
10. press F5 to go to Next Page
11. Choose required Compression level (e.g. gz)
12. Image split mode if the partitions you save are big, it is wise to split them into the file that fit on CD or DVD (I enter here 4500MB)
After continuing with F5 you give a description and the process starts, generating files:
sda3_partimage.000, sda3_partimage.001, etc
be careful these file are gzip'ed file, please rename them to sda3_partimage.00X.gz for X in 0,1,2..

How To Restore:
1. attach the backup drive containing your partimage files and mbr and extended partition entries
2. attach empty drive (in my case /dev/sda)
3. Boot the rescuecd
4. to find out the location of the backup drive in my case /dev/sdb1, use command

fdisk -l

- if this is linux fs drive with ext3 , use following command:

mount -t ext3 /dev/sdb1 /mnt

5. restore MBR and extended partition entries:

- dd if=/mnt/backup-sda.mbr of=/dev/sda
- sfdisk /dev/sda <> check with fdisk -l if all partitions made on /dev/sda

6. make sure backup file names on backup drive (/mnt) are renamed to gz, then gunzip them:

gunzip sda3_patimage.000.gz (will create much bigger sda3_partimage.000 file),     
... or     
gunzip sda3_partimage.*.gz 

7. run partimage command
8. Partition to save/resotre (in my case sda3)
9. Image file: enter the path to your gunziped image files /mnt/sda3_partimage.000
even if you have more than 1 file, always put the location of the 000 file
10. Action to be done: pick Restore partition from an image file After continuing with F5 you should get restored partition on /dev/sda3
11. format SWAP partition (in my case /dev/sda5 -> check your fdisk -l)

mkswap -c /dev/sda5

12. Install GRUB boot-loader

mkdir /new_disk
mount /dev/sda3 /new_disk
grub-install --root-directory=/new_disk /dev/sda

re-installing GRUB

Situation:
You have linux and windows on your hdd,
As it happened you needed to reinstall M$ Windows, after that operation your GRUB is not visible after reboot.

Resolution:
1. run any of the Linux LiveCD distros (I used Ubuntu)
2. in Console type (as root or sudo):

->  grub

3. in Grub app (I had my linux partition on /dev/sda7 which is mapped to hd0,6 in GRUB):

->  root (hd0,6)
->  setup (hd0,6)

4. reboot
5. you're done if everything went fine -> I hope so :)

thanks,
chris