Keith's Tech Notes

cygwin hints

2010-12-10T02:08:00.002Z

Setting the home directory to Document and Settings

mkpasswd -l -p "$(cygpath -H)" > /etc/passwd

Cygwin specific utils
http://cygwin.com/cygwin-ug-net/using-utils.html

Why ZFS?

2008-07-30T16:05:00.005+01:00

What is ZFS?
ZFS is a POSIX compliant file system and Volume Manager developed by Sun Microsystems. The source code is in the public domain and has been ported to FreeBSD and its dirivatives (Apple amongst others).

ZFS supports error detection and recovery while the system is in use.

Personally, I've been very concerned about the integrity of data on disks for over 20 years. It seems I've spent my professional life waiting for ZFS.

The Problem with Hardware RAID
Hardware is great! It provides an OS independent solution to RAID management. All the OS sees is a single device adn the hardware takes the load, manages parallel writes and the like. Throw in a higher level RAID and you're protected against one or two disk failures. I mean it's great.

That's until you do suffer failure. If you suffer disk failure and you have a hot swappable setup then that'll do. But if you don't, you can spend a long time without service, as I found out recently. It takes time to rebuild the array.

If you suffer controller failure, you'll need a compatible controller to read the disks. That may not be as easy as it sounds, and you'll loose ALL your data if you guess wrong.

The ZFS boys highlight one further fault
There's a problem on RAID5 with partial writes. Disk writes are not atomic, the data and parity writes are unlikely to occure at exactly the same time. If you suffer power failure during a write, you'll have a corrupt track.

One further advantage
The ZFS file system uses the same snapshot mechanism as the ZFS Volume Manager. That improves the system's efficiency. Admittedly, this only reverses an introduced inefficiency.

Back to Why ZFS?
ZFS as a software Volume Management system can implement a RAID5 equivalent called RAIDZ (and RAIDZ2 for RAID6). This can survive hardware failures. And it corrects the known RAID5 partial write problem.

Even without RAID, ZFS can take spare disks on a volume, or disk pool in ZFS speak. The ZFS filesystem can allegedly maintain more than one copy of files specified by a count. However, I'm yet to see this work.

An Introduction ZFS on FreeBSD

2008-07-21T13:35:00.013+01:00

What is ZFS?
ZFS is a somewhat fault tolerant file system integrated with a volume manager developed by Sun Microsystems. The source code has been placed in the public domain. This made the code available and it has been ported to FreeBSD and Darwin.

ZFS writes checksum information along with each block of data is writes. This allows it to detect errors when the data is read.

Resources
Here's a summary of ZFS from the authors.
http://www.opensolaris.org/os/community/zfs/docs/zfs_last.pdf

The FreeBSD/ZFS wiki
http://wiki.freebsd.org/ZFS

This guide describes how to install FreeBSD 7.0 on a ZFS filesystem.
https://www.ish.com.au/solutions/articles/freebsdzfs

For more wisdon on ZFS see:
http://blog.thefrog.net/2008/04/more-zfs-on-freebsd.html

What is FreeBSD?
FreeBSD is an old UNIX variant. It's the BSD derived project optimised for the PC platform.

FreeBSD is free to access, free to use, and free to modify and reuse. It's stable and reliable. ZFS is an experimental feature within FreeBSD 7.0, i.e. ZFS is neither stable nor reliable in FreeBSD 7.0.

What is Darwin?
Darwin is the operating system used by Apple. Apple actuall runs Mach 3.2. But you can ignore that unless you're writing device drivers. The next layer up is based on FreeBSD and called Darwin. The upper layer is concerned with the GUI.

A Few Concepts
ZPOOL
When DRAM is added to a computer, it's added to a memory pool and available to all applications. Similarly, the designers of ZFS liked the notion of a disk pool. Storage virtual devices are used to make up a pool. The pool size can be increased by adding more storage. That's where the analogy ends however as you can't really take storage unit out of a pool (unless it's a spare).

RAIDZ
This is a software RAID system that addresses a problem in RAID5 by providing a slight optimisation, it uses a variable block size rather than RAID5's fixed equivalent. It's part of the ZFS storage system and is managed as a storage virtual device.

VDEV
Virtual Storage devices are exactly that, virtual storage. Each actual storage device can differ in type. For example, ramdisk, hard disk, flash memory, RAIDZ and so on. All of these are managed by the virtual storage system, VDEV.

Rails Scaffolding

2008-05-23T12:51:00.003+01:00

Scaffolding in Ruby on Rails is a wizard that generates a Model, View, Controller and database migration.

How do you do it?
Generate a new Rails project
rails project

Go to the project directory
cd project

Run the scaffold. Tell it to create an object called Account that contains an id, name and updated_at fields.
script/generate scaffold account name:string updated_at:timestamp

Migrate the database to the next version, in this case, version 1.
rake db:migrate

Run the webserver so you can interact with it through a browser at address: http://localhost:3000/
script/server

And that's it!

Building a Ruby on Rails website

2007-12-29T01:31:00.001Z

Overview
This guide describes how to build a commercial quality ruby web server using free components Debian GNU/Linux, Apache 2 and Ruby on Rails.

A good reference book is Ruby By Example by Kevin Baird.

Installing the Operating System
The very first piece of software to begin with is the Operating System.
Please follow this link. Installing Debian GNU/Linux

Installing Apache 2
Install Apache 2 by typing:
apt-get install apache2 apache2-doc

Install the Ruby module and Embedded Ruby by typing:
apt-get install libapache2-mod-ruby liberuby ruby eruby ruby-dev

Apache 2 will be installed. The Apache 2 configuration directory is /etc/apache2
The standard web server directory, DocumentRoot, is /var/www

Configuring Apache 2 for Ruby
First, we configure Apache 2 to interpret .rhtml and .rcss files using Embedded Ruby.
cd /etc/apache2/

Create file:
mods-available/mod_ruby.conf

<IfModule mod_ruby.c>
     RubyRequire apache/ruby-run
     RubyRequire apache/eruby-run
     <Files *.rcss>
           AddType text/css .rcss
           AddType application/x-httpd-ruby *.rb
           SetHandler ruby-object
           RubyHandler Apache::ERubyRun.instance
     </Files>
     <Files *.rhtml>
           AddType text/html .rhtml
           AddType application/x-httpd-ruby *.rb
           SetHandler ruby-object
           RubyHandler Apache::ERubyRun.instance
     </Files>
     RubyRequire auto-reload
</IfModule>

Make a soft link to this file in the mods-enabled directory.
cd /etc/apache2/mods-enabled
ln -s ../mods-available/mod_ruby.conf

Second, we configure a web site to use Ruby CGI and Embedded Ruby. We use Apache's virtual server support to create a Virtual Server. We'll assume:
The virtual server's name is jecht
The server's IP address is 192.168.0.34
The server's port is 80

Create file:
sites-available/jecht

NameVirtualHost 192.168.0.34:80
<VirtualHost 192.168.0.34:80>
     ServerAdmin kbw@jecht
     ServerName jecht:80
     ServerAlias jecht
     DocumentRoot /var/www/jecht/
     ErrorLog /var/log/apache2/error.log
     CustomLog /var/log/apache2/access.log combined

     ScriptAlias /cgi-bin/ /var/www/jecht/cgi-bin/
     <Directory /var/www/jecht-basic/cgi-bin/>
           AllowOverride None
           Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
           Order allow,deny
           Allow from all
     </Directory>
     RubyRequire apache/ruby-run
     RubyRequire apache/eruby-run
     <Files *.rcss>
           AddType text/css .rcss
           AddType application/x-httpd-ruby *.rb
           SetHandler ruby-object
           RubyHandler Apache::ERubyRun.instance
     </Files>
     <Files *.rhtml>
           AddType text/html .rhtml
           AddType application/x-httpd-ruby *.rb
           SetHandler ruby-object
           RubyHandler Apache::ERubyRun.instance
     </Files>
     RubyRequire auto-reload
</virtualhost>

Make a soft link to this file in the sites-enabled directory.
cd /etc/apache2/sites-enabled
ln -s ../sites-available/jecht

Testing Embedded Ruby
Embedded Ruby is Ruby code that is embedded HTML. Apache 2 will run the Ruby code when rendering the page.

Create the Embedded Ruby directory for the site:
mkdir -p /var/www/jecht/mod_ruby

Create the Embedded Ruby file to be served:
/var/www/jecht/mod_ruby/demo.rhtml


<!DOCTYPE html
     PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
     <title>Mod Ruby</title>
     <style> code {
             background-color: #ddf;
             color: #f00;
             padding: 0.3em;
     } </style>
</head>
<body>
     <h1>Embedded Ruby
     <p>Welcome to <em><%= ENV['SERVER_NAME'] %></em> at <%= Time.now %>.</p>
</body>
</html>

View the site from:
http://jecht/mod_ruby/demo.rhtml

Understanding PC Hard Disk Partitions

2007-12-28T00:16:00.001Z

Computer hard disks are often divided. The names used for these divisions in the PC world is partition. In the Unix world, these divisions are called slices.

The beginning of the PC hard disk contains a table which describes the partitions on the hard disk. The table is only large enough for four entries. In MS-DOS and Windows, each partition becomes a drive letter; well almost.

Partitions under DOS/Windows
MS-DOS introduced a convention that allows more than four partitions to be available. The idea is the original four partitions are called Primary partitions. A new kind of partion was introduced, called the Extended partition. An Extended partition may be further divided into a large number of other partitions, called Logical partitions.

So, in MS-DOS or Windows, if you wanted a disk split into a 8G C: drive, a 10G D: drive and a 16G E: drive, the partition would be as follows:
Partition table entry 1: 8G Primary partition
Partition table entry 2: 26G Extended partition
The Extended partition would have a 10G Logical partition and a 16G Logical partion.

Partitions under Linux
It seems Linux was designed to coexist with DOS/Windows. Linux uses hd to prefix IDE/ATA hard disk names. The first ATA hard disk will be hda, the second hdb and so on.
If the partition is a primary partition, it'll have the table index. So the first primary partition on the first ATA hard disk will be hda1.
The Extended partion takes number five and the Logical partitions six onwards.
So, Linux, if you wanted the first ATA hard disk split into a 8G C: drive, a 10G D: drive and a 16G E: drive, the partition would be as follows.
hda1: 8G
hda5: 26G
hda6: 10G
hda7: 16G

Partitions under BSD
BSD and Unix predate DOS and Windows and partitions. A BSD disk divisions are called slices. A slice is futher divided into partitions. These are not to be confused with DOS/Windows/Linux partitions.
Different disk types have different names. That's were Linux gets the idea from. The ATA disks on my FreeBSD system are ad, where the ATA disks on my OpenBSD system are wd.

Again, using my FreeBSD system as an example, it has partitions:
ad0 - ATA disk1
ad0s1 - ATA disk1, slice 1
ad0s1a - ATA disk1, slice 1, partition 1
ad0s1b - ATA disk1, slice 1, partition 2
ad0s1c
ad0s1d - ATA disk1, slice 1, partition 3
ad0s1e - ATA disk1, slice 1, partition 4
ad0s1f - ATA disk1, slice 1, partition 5
ad0s1g - ATA disk1, slice 1, partition 6

And ad0s1c?, partition C is a partition that represents the entire slice. In this instance, it's the same as ad0s1.

Summary
DOS/Windows and Linux use Primary partition, Extended partition and Logical partition.
BSD and Unix use slices and partitions.

Installing Debian GNU/Linux

2007-12-28T00:12:00.001Z

Why Debian?
Debian is an early GNU/Linux distribution that has a rigorous development cycle. The installations are highly customisable and can be made more fit for purpose, much like a BSD distribution.

If you want to get started with Linux and you just want to play about with it, you could try Ubuntu. But if you want to build a production server, I'd recomend Debian.

Installing
Etch is the latest stable Debian release, 4.0r3. Download Debian etch from the Debian install page.

Burn the ISO disks to (rewriteable) CDs and boot from disk 1.
Follow the installation instructions. The installer will allow you to select the kind of packages to be installed. Alternatively, you can deselect all options and go with a minimal instllation.

Installing Packages
Once Debian is installed, you can use APT to install individual packages.

To search for the package that contains sudo, type:
apt-cache search sudo

To install the sudo package, type:
apt-get install sudo

Easy right?

Bringing The System Up to Date
Use aptitude to get the latest list of packages.
apt-get update

Use aptitude to upgrade the system.
apt-get upgrade