Arrow keys in vi

I recently installed Mandriva 2007 Spring on a couple of machines. One of the annoyances was the fact I could not use my arrow keys in vi like I used to. To resolve this, type the command:

echo set nocompatible >> ~/.vimrc

That's all there is to it.

Read more!

My backup scripts

I wanted to automate my backups of select files. I wanted my backup to do a monthly full backup, and a weekly incremential. I also wanted a rolling backup, so anything over the last 2 months would be overwritten or deleted.

Here is my monthly backup script, I saved as /etc/cron.monthly/monthly.bu:
#!/bin/sh
# Monthly backup script
# Makes a full backup

# Change the 2 variables below to fit your computer/backup
DIRECTORIES="/root /etc /var/log" # directories to backup
BACKUPDIR=/server/gonzobu # where to store the backups

# You should not have to change anything below here
MO=`date +%b` # Month name e.g. Apr
LOGFILE=$BACKUPDIR/$HOSTNAME.log # Log file
TIMEFILE=$BACKUPDIR/lastbackup # File to remember last backup

# Monthly full backup
tar --totals -cvf $BACKUPDIR/$HOSTNAME-$MO.tar $DIRECTORIES &> $LOGFILE

# Display settings for mailed in report by cron daemon.
echo Monthly full backup completed.
# Show stats and file names backed up, not top level directories
grep -v "not dumped" $LOGFILE | grep -v "/$"
# For stats only, comment line above, and uncomment line below
# grep "Total bytes" $LOGFILE

# Update lastbackup file
touch $TIMEFILE

# Clean up old backups (set to keep the last 2 months of backups)
find $BACKUPDIR -mtime +68 -delete


And here is my weekly backup I saved as /etc/cron.weekly/weekly.bu:
#!/bin/sh
# Weekly backup script
# Makes a incremental backup

# Change the 2 variables below to fit your computer/backup
DIRECTORIES="/root /etc /var/log" # directories to backup
BACKUPDIR=/server/gonzobu # where to store the backups

# You should not have to change anything below here
DM=`date +%d%b` # Date and Month e.g. 27Sep
LOGFILE=$BACKUPDIR/$HOSTNAME.log # Log file
TIMEFILE=$BACKUPDIR/lastbackup # File to remember last backup

# Check if first time run
if [ -f $TIMEFILE ]; then
NEWER="-N $TIMEFILE"
else
NEWER=""
fi

# Weekly incremental backup
tar $NEWER --totals -cvf $BACKUPDIR/$HOSTNAME-$DM.tar $DIRECTORIES &> $LOGFILE

# Display settings for mailed in report by cron daemon
echo Weekly incremental backup completed.
# Show stats and filesnames backed up, not top level directories
grep -v "not dumped" $LOGFILE | grep -v "/$"
# For stats only, comment line above and uncomment line below
# grep "Total bytes" $LOGFILE

# Update lastbackup file
touch $TIMEFILE


To make these work, first you need to create the backup directory in your file system. Then change the variables in each file to match your system (files to backup, and where to put the backups). Be sure to make both scripts executable. That's all there is to it.

To explain why I did the 68 days at the end of the monthly script, it's because I wanted to save a full 2 months. 31 days + 30 days + 1 full week (in case the week ends before the cron job runs).

Read more!

Use all 5 buttons on your mouse

I have a Logitech MX400 Laser mouse. It has a tilt scroll wheel, and two side buttons. I have these two side buttons mapped in Guild Wars for targeting. So here is how you get them working.

Edit /etc/X11/xorg.conf. Find the section "InputDevice". Add the bold items below.

Section "InputDevice"
Identifier "Mouse1"
Driver "mouse"
Option "Protocol" "ExplorerPS/2"
Option "Device" "/dev/mouse"
Option "Buttons" "7"
Option "ZAxisMapping" "4 5"
Option "Emulate3Buttons" "false"
Option "ButtonMapping" "1 2 3 6 7"
EndSection

I know it says 7 buttons. That's cause Linux counts the scroll wheel movements as "buttons". However, you notice that the ZAxisMapping is set for buttons 4 and 5 (Up and Down on scroll wheel).

Restart X with a Ctrl-Alt-Backspace. That's it! If anyone has any idea how to get the sideways tilting scrolling to work, please post a comment.

Read more!

Image a drive with dd

I ended up removing Linux from the laptop cause I was running out of space on the Windows partition, and wanted to re-install Windows, since I was having problems with it (Windows, not Linux). So I installed XP with SP2. I then patched it up fully, installed Firefox, Zone Alarm, and Clamwin. The used portion of the drive was about 3GB.

To create an image of this setup, I decided to try the dd command from a Linux LiveCD. First step was to move the data to the beginning of the disk. Running Windows defrag a few times is supposedly all you need. From my experience, the visual after the defrag shows some data is left in the middle of the drive. If you have an idea about this, please post a comment.

Next I booted up to Kanotix, (my favorite LiveCD), and ran the commands:
su
mkdir /mnt/smb
mount -t smbfs -o lfs,rw,username=jbess //10.0.0.5/Backups /mnt/smb

lfs is needed because your image may exceed the 2GB default limit. Unless you want to make small chunks of your image, this is what you must use. The 10.0.0.5 is my Mandriva server running Samba and ProFTPd.

Next, mount the drive to be imaged (for me I just clicked on the drive on the desktop that Kanotix automatically put there), and then run df to check the used drive space. I had 3013344KB used, so I rounded to 3100MB, or 3.1GB. After you get the numbers, be sure to unmount the drive, or the next step may have problems.

To compress the image, I used gzip. I typed:
dd if=/dev/hdc bs=1M count=3100 | gzip > /mnt/smb/laptop_SP2.img.gz

Be sure to replace the count with how many megabytes are used on the drive. I ended up with a 1.8GB file on my server. That's about a 42% compression, which is less than what the man page for gzip states. It took 70 minutes at 766KB/s (according to the stats). Quite slow. Too slow actually. I thought it was samba being slow. So I tried ftp.

From the ftp> prompt, I typed:
put "|dd if=/dev/hdc bs=1M count=3100 | gzip" laptop_SP2.img.gz

The | tells ftp to run a shell command, in this case, dd. This was a little better at around 1.6MB/s, more than twice as fast. During both samba and ftp I was monitoring the transfers with KNemo traffic plotter. I noticed there was dips or lulls in traffic being sent every few seconds, and lasted for several seconds. I thought that maybe running it through gzip was slowing the stream down, and with no buffer in place, the data transfered stopped, which increased the time. So I then took out gzip. Wow, I was impressed!

The samba share command was:
dd if=/dev/hdc bs=1M count=3100> /mnt/smb/laptop_SP2.img
Took 15 minutes at 3.5MB/s on average. I tried again with FTP.

With FTP, the command was:
put "|dd if=/dev/hdc bs=1M count=3100" laptop_SP2.img
That finished in 8.5 minutes, with a 6.4MB/s transfer rate.

Curious, I googled and found this article about zip program comparisons:
http://www.linuxjournal.com/article/8051

According to the article rzip give the fastest speed and compression. I checked and rzip is NOT installed on the Kanotix LiveCD, but the next best one was, lzop. I ran my tests again.

smb (samba):
dd if=/dev/hdc bs=1M count=3100 | lzop > /mnt/smb/laptop_SP2.img
Time - 23 Minutes, Rate - 2.3MB/s

ftp:
put "|dd if=/dev/hdc bs=1M count=3100 | lzop" laptop_SP2.img.lzo
Time - 20 Minutes, Rate - 2.7MB/s

Using lzop got me a lower compression, at 33%, but at two-thirds faster speed compared to gzip when using samba, and just under two times faster with FTP. Here are the full comparisons:

SMB - No Compression, 3.5 MB/s
SMB - gzip (42% compression), 766 KB/s
SMB - lzop (33% compression), 2.3 MB/s
FTP - No Compression, 6.4 MB/s
FTP - gzip (42% compression), 1.6 MB/s
FTP - lzop (33% compression), 2.7 MB/s

My conclusions? Faster is better in my situation. I want an image quickly. I can always zip the image later if I need the space. If I do need to zip it, I will use gzip, or install rzip and use that. For my setup, FTP is king in the transfer speed. If you do want to compress the image file, and have a decent transfer speed, I suggest the lzop methods, and obviously FTP is faster. My preferred method: FTP with no compression.

This is the results of my setup. Your numbers will probably vary. Do some testing yourself. Post a comment if you do with your results. I will write a post about restoring an image later on. There are lots of articles already if you google it, but basically you just reverse the commands.

Edit: upon attempting to restore this image, I found some problems, mostly boot record stuff. I am sure this works, just need to play with it some more when I get a chance.

Read more!

Software RAID

The primary purpose of my server is a file server. While it has other purposes, that is what it was originally built for. Earlier this year we bought a 250 GB HD from Woot. After filling it more than half way, I got to thinking about backups. With Black Friday just passing, the wife took a trip to Best Buy, where there was a 250 GB HD for $60. A good deal. And at first I was going to to just use rsync to keep the files mirrored, I looked into RAID 1, which accomplishes the same task, but in real time. Here is my setup:

• 30GB HD with OS installed on IDE 0, set as Master. (/dev/hda)
  
• (2) 250GB HD for file storage, one on IDE 0 as slave, the other on IDE 1 as Master (dev/hdb & c)
  
• CD-ROM on IDE 1 as slave. (/dev/hdd)
  

Here is how I set it up:

1. Install raidtools:
1. urpmi raidtools
   
2. Backup drive data. I was lucky and my wife had just enough space on her HD to be a temporary storage. It took close to 4 hours to transfer the data over a samba connection on a 100MB connection. Backing up the data is a MUST since you will be changing partition tables and re-formating the drives.
3. Install second hard drive, and make sure BIOS recognizes it. I made sure the two 250 GB drives were on separate IDE channels.
4. After the backup was verified, I changed the file system type on both drives. Type these commands:
1. fdisk /dev/hdb
2. t (to change file system type)
3. fd (which is Linux Raid Auto)
4. w (to write the new partition table)
5. Repeat the same for /dev/hdc
6. Copy the template raid config files:
1. cp /usr/share/doc/raidtools/raid1.conf.sample /etc/raidtab
7. Edit the new raidtab with vi. Make it look like this:

   # My raid-1 configuration
   raiddev /dev/md0
   raid-level 1
   nr-raid-disks 2
   nr-spare-disks 0
   chunk-size 32
   
   device /dev/hdb1
   raid-disk 0
   
   device /dev/hdc1
   raid-disk 1

8. Initialize the RAID:
1. mkraid /dev/md0
9. Check to make sure it's working:
1. cat /proc/mdstat
2. Should see no error messages
10. Format new RAID partition
1. mkfs -t ext3 -b 4096 /dev/md0
11. Now everything should be set, last step is change fstab to mount the new RAID array:
1. /dev/md0 /server ext3 defaults 1 2
12. Type mount -a and you now have access to the RAID array.
    

Note: /proc/mdstat is your key to the status of your RAID array. Check it if you suspect any problems later on.

Read more!