g4u - Harddisk Image Cloning for PCs

Floppy verion

• Download
• Write the images to seperate floppy disks. Be sure to label them.
  • Under Unix use cat g4u-2.1-1.fs >/dev/diskette
    The floppy device may have a different name. Examples:
    • NetBSD: /dev/fd0a
    • Solaris: /dev/diskette
    • Linux: /dev/fd0

  • for DOS, use rawrite.exe.
  • for Windows use rawr32.zip.

- OR -

CD version:

• Download bootable ISO CD image
  If you downloaded the g4u-2.*.iso.zip file, unpack it to get g4u-2.*.iso
• Consult your CDROM writing software (Nero, DiskJuggler, WinOnCD, cdrecord, ...) 's manual on how to write the ISO file to a bootable CDROM.

4. Using G4U

• On the "sourcing system"
  Since g4u does not use the filesystem information when accessing the disk, It does NOT use the allocation tables. This means that it will transfer ALL sectors whether they are in use or NOT!
  It is best to write a constant pattern filling up ALL unallocated space with the "sourcing" system active.
  Since g4u compresses the output, sectors with the same data pattern take very little space.

• On the "storage" FTP server
  1. Create an user-account called install, and protect it with a password like secret-g4u.

  2. Prepare a hosts file containing the hostname of HDimg-store and the FTP server's IP address.
     For example:
     
     192.168.1.3 HDimg-store
     This permits the g4u commands to reference the ftpserver as HDimg-store which allows changing the FTP server without changing any commands run for the save.
  3. Prepare a save.sh file containing commands simular to :
     

     disks
     export DISKS=`disks`
     parts $DISKS
     
     echo continue?
     read continue
     date
     date >>log
     export yymmdd=`date +%y%m%d`
     GZIP=8 uploadpart HDimg-store  template-system.wd0e-$yymmdd.gz wd0e
     date
     date >>log
     echo "put log log-$yymmdd.txt" | ftp install@HDimg-store
     

     
     You must use the correct network interface name and disk partition names depending on the "sourcing" system.
     You may have to boot g4u on the sourcing system in order to determine them. See Image Creation

  4. Verify that the install user can put a file by using an ftp client locally on the ftp server.
     Note: These are the commands used by g4u.
     -n tells ftp to ignore .netrc autologin.
     127.0.0.1 is the FTP server itself
     

     ftp -n
     open 127.0.0.1 
     user install  secret-g4u
     bin 
     put hosts
     put save.sh
     ls
     bye

     If you have a permission problem check the FTP server's /etc/shells or what ever access control is implemented.

     If it is necessary to use different user-account, g4u utilities accept: login@server on the command line.

  5. Before booting G4U on the "sourcing" system
     Verify that there is a DHCP server that assigns IP addresses and other information ( netmask, default gateway, name server) needed to access the FTP server.
     • unix: keyin: ifconfig -a expect something like:

       rtk0: flags=8843>UP,BROADCAST,RUNNING,SIMPLEX*,MULTICAST> mtu 1500
address: 00:50:fc:c0:33:a8
media: Ethernet autoselect (none)
status: active
inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
       This is the important information
       lo0: flags=8008>LOOPBACK,MULTICAST> mtu 33192

     • windows keyin: ipconfig /all expect:

       Host Name . . . . . : sourcehost
       ...
       Ethernet adapter Local Area Connection:
       ...
       

       IP Address. . . . . :	This is the  important  information  192.168.0.4
       Subnet Mask . . . . :	255.255.255.0
       Default Gateway . . :	192.168.0.1
       DHCP Server . . . . :	192.168.0.1

       ...
       

       - or -
       
     • be prepared to set the TCP/IP values manually.

• Image creation
  • Boot the g4u CD or boot-floppy on the "sourcing" system.
    Observe the linux kernel boot and the devices found .

    NetBSD/i386 ustarfs Primary Bootstrap
    << NetBSD/i386 BIOS Boot, Revision 3.2
    << Memory: 639/187328 k
    Press return to boot now, any other key for boot menu
    booting fd0a:netbsd - starting in 0
    
    isa0 at pcib0
    ...more boot messages...
    wskbd0 at ...
    ...
    
    Running MAKEDEV all ... 

    Next the DHCP request:
    ...DHCP messages...
    Finally you'll get a description of possible commands, and
    the shell prompt
    g4u>

    keyin:

    ftp install@ftpserver.IPaddress*
    get hosts /etc/hosts
get save.sh
quit

  • List the disks recognized by g4u by using the disks command.
    On a IDE based PC they will be:

    wd0 PrimaryIDE.master	wd1 PrimaryIDE.slave
    wd2 SecondaryIDE.master	wd3 SecondaryIDE.slave

    upload Entire harddisk*: to transfer the first harddisk to the FTP server keyin: (if you make a typo use ^u to erase the line, ^H to backspace ) uploaddisk HDimg-store source-system.wd0.051231.gz wd0 For the second IDE disk, use : wd1 . For SCSI use: sd0 . The image will be compressed with gzip, so it is recommended to use the .gz filename suffix for the filename to be created on the FTP server. To use a different account name use user-account@HDimg-store.

    - or -

    upload One partition at a time: List the partitions on the source disk using: parts disk where disk is one of the disks printed by disks . (ie: wd0, wd1, sd0 ). Partitions are defined with letters starting from a. These are BSD-partitions, which have little in common with DOS MBR partitions. Yes it is confusing. Examples of partition names: wd0e or sd0f Keyin: chmod a+x save.sh save.sh

    (for minimum compression)*

    GZIP=1
    Defaults:
    • disk+partition=wd0d
    • user=install
    • GZIP=9 8 may be much better

  • Depending on your network, CPU, harddisk hardware and contents, image creation can take several hours! You should expect to see

    220 ftp server here
    331 Password required
    230 user logged in
    Remote system is ....
    200 Type set to I.
    remote: filename
    227 entering Passive mode (nnn,nnn,nnn,nnn,nnn)
    125 Data connection already open; transfer starting.
    
         nnnMB*         mm.mm MB/s*

    eventually a summry will be displayed

    
    nnnnnn+0 records out
    090988808908908098 bytes transfered in ssss. secs (09890809bytes/sec)
    
    0+783738 records in
    0+783738 records out
    090988808908908098 bytes transfered in ssss. secs (09890809bytes/sec)
    226 transfer complete.
    9999999999 bytes send in mm:ss (1.8 KB/s)
    221 A vert er zain
    
       nnnMB        mm.mm MB/s
    

  • Check that your FTP server's install account now has the image file.
  • You can boot the source machine now.

    No filesystems were mounted or modified.

  4.3 Image deployment

  1. Boot the g4u CD or floppy on the TARGET system to the shell prompt.
  2. Download the image.

     Entire harddisk: Keyin: slurpdisk your.ftp.server.com filename.gz. This will log into the FTP server's install account, retrieve the image, uncompress it and write it back to /dev/wd0. For account name other than install prefix the server's address with account@

     - or -

     One partition: Keyin: slurppart your.ftp.server.comIP filename.gz wd0e Note that the partition information is on the TARGET MBR, which MUST be the same as before image creation. , check the start sector and size values given by parts. For an image that includes the MBR, do a full backup with "uploaddisk".

  3. Reboot the target machine

  4.4 Copying a disk locally

  If you just want to copy one local disk to another local disk with no network & server involved, the copydisk command is what you want.
  For example to copy the first IDE disk to the second IDE disk, use
  copydisk wd0 wd1
  
  for SCSI disks use
  copydisk sd0 sd1.
  Beware! All data on the target disk will be erased!

  A list of disks as found during system startup can be found by using the disks command.

  4.5 Copying a partition locally

  If you want to only copy one local partition to another local partition (similar to what uploadpart and slurppart do, just without the network and FTP in between), this can be done with the copypart command. It takes two partition names as arguments, and copies the contents of one partition to the other. As an example to copy first local partition wd0e to the second one wd0f, run:
  copypart wd0e wd0f
  A list of disks can be found using the disks command, to list all the partitions on a disk use the 'parts' command. Partitions have the form of wd0d, w1e, sd1f.

  Be aware that the partitions to copy should have identical size (down to the sector), else bad things will happen. When copying a 'big' partition into a 'small' one, g4u won't thrash the data behind the 'small' partition, but of course the copy is not complete either. Take special note that that case could happen when you restore a copy made that way, and which went fine when you first copied your small working partition to your big backup partition!

  4.6 Programs available after booting g4u.

  In order to keep the size of g4u to a minimun ONLY these programs are included:
  

  cat, chmod, date, pwd, rm, sh
disklabel, init, mknod,
mount, mount_ffs, mount_cd9660, mount_mfs, mount_kernfs
mbrlabel, parts, newfs
route, dhclient, ifconfig
less, sed, more, umount
dd, ftp, gzip, gunzip, sleep, progress
reboot, halt, sync


  not available: ping, traceroute,vi, df(use parts), mkdir, cut

1. Booting g4u in bochs
2. Device detection
3. Welcome to g4u!
4. Some random g4u commands
5. Uploading a disk image with uploaddisk
6. Restoring with slurpdisk

5. FAQs and hints on disk cloning

5.1 Supported filesystems

One of the questions arising a lot is "what filesystems does g4u support". The answer is: "all of them". g4u reads the disk bit by bit, starting from byte #0 to the end. It includes any MBR, boot record, partition table and the partitions themselves without further investigating the structure of the data stored in these partitions.

5.2 Supported Operating Systems

The question on operating systems that can be deployed with g4u is the same as for the filesystems: any. Given the image-approach again, g4u is able to handle any operating system. Systems that were cloned successfully include NetBSD, Linux, Novell Netware 4.11 and 5.1, Solaris/x86, Windows NT, 2000 and XP.

By moving the harddisks to a PC, g4u can even be used to deploy operating systems for non-PC based SCSI machines running HP-UX, Irix, Solaris, AIX etc.

5.3 Supported Hardware

The system running g4u itself can have IDE, SATA, SCSI or RAID disks with various controllers (Adaptec, ...) as well as wide range of PCMCIA, Cardbus, ISA and PCI network cards. Please see the g4u kernel config for the full list of supported hardware.

If you're unsure if your hardware is supported, simply boot g4u and see if your network card gets listed by "ifconfig -a" and if your disks get listed by the "disks" command. If not, adding relevant parts of "dmesg" output (from g4u; press space bar to scroll down) is required for analysis if you ask for help. See "Reporting problems" for more information.

5.4 A word on disk sizes

The question how g4u deals with different disk sizes arises a lot too. The general answer is, g4u works best with identical disk sizes & geometry. Putting an image from a small disk on a big disk works, putting an image from a big disk to a small disk is likely to cause problems.

If you cannot avoid preparing an image on a big disk that'll get deployed to a small disk later, make sure the "extra" space is not occupied by a active partition or filesystem, else data loss is very likely to occur!

If you intend to deploy a "small" image to a "big" disk, the extra space that's not covered by g4u can be used for creating a partition and a filesystem. You will have to do that on your own, e.g. using your operating systems' post installation steps.

5.5 Changing compression level

By default, images uploaded to the FTP server are compressed with gzip 9. This saves as much disk space as possible, but also takes a long time because of the CPU requirments at this level.
You can reduce the gzip level for "uploaddisk" by setting the GZIP environment variable:

 GZIP=8 uploaddisk your.ftp.server.com filename.gz wd0

You can change compression levels between 1 (fast, little compression) and 9 (slow, maximum compression). Of course you can specify all the usual options to uploaddisk.

Why NOT always use the highest compression?
Depending on the data, sourcing CPU performance and network activity, different compressions will transfer faster. Many CPUs cannot run GZIP 9 fast enough to keep the network active. This results in a cycle of CPU busy - network idle, burst of network activity, then CPU busy - network idle...
Reducing the compression will increase the network activity but the CPU will dispatch packets more frequently resulting in a reduction of the total time.
Using GZIP 1, packets may be dispatched so fast that they interfere with other network traffic ( remember this is going to take a long time).This will also cause the file on the "storage" system to be as large as the partition (or disk) on the "sourcing system"

Some users have reported a dramatic reduction in total time using gzip 8 without significant increase in the size of the resulting image.

You will need to tune the compression to your configuration and needs.

5.6 List of recognized disks

During startup of g4u, all devices recognized are listed, but very fast. To get a list of recognized disks, use the 'disks' command:

# disks
wd0 at pciide0 channel 0 drive 0: 
wd0: drive supports 16-sector pio transfers, lba addressing
wd0: 6149 MB, 13328 cyl, 15 head, 63 sec, 512 bytes/sect x 12594960 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (using DMA data transfers)

The above example shows a 6GB IDE harddisk.

5.7 Problems with images greater than 2GB

Do you experience g4u aborting file transfers after the image has grown to 2 GB on the FTP server? The problem here is not g4u, but most likely your FTP server. Some older Linux distributions are known to only allow files of up to 2GB filesize, and even if there is a Linux 2.4 kernel running, that's no guarantee for a properly working server. Make sure that your ftp daemon is upto date, or install a decent operating system.

So far, whatever FTP server comes with NetBSD, Solaris and Windows 2000 has been used without problems.

5.8 Can you add feature XXX?

I got requests for adding many features to g4u:
• using TFTP
• using SSH/scp
• using NFS
• adding a X or curses based GUI
• writing images to CDROM / deployment from CDROM
• bzip2 compression

After moving to a multi-floppy set for g4u, some of these features may be added in the future, while others (X...) are not likely. Stay tuned!

5.9 Problems with network performance

If upload performance is weak (less than 5.00 MB/s on a 100BaseT Ethernet switch) even with a small compression level or a fast CPU and the harddisk is idle means the network configuration needs adjustment.
3/09/06 10BaseT laptopNIC averages 1.6MB/s
A common problem in switched Ethernet is a duplex mismatch between the NIC and the switch. In NetBSD, the default is to negotiate speed and duplex automatically. Other settings can be set manually.

Forcing 100BaseTX/Full-duplex:

 ifconfig -a
    fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> MTU 1500
          address: ..:..:..:..:.. 
          media: Ethernet autoselect (100baseTX full-duplex)
          status: active
          inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
    lo0: flags=8008 mtu 33192     
 

 ifconfig eee0   media 100BaseTX mediaopt Full-duplex
    dfxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> MTU 1500
          address: ..:..:..:..:.. 
          media: Ethernet 100baseTX full-duplex
          status: active
          inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255

Using autonegotiation (default):

 ifconfig fxp0 media auto
 ifconfig -a
    fxp0: flags=[...] 
        media: Ethernet autoselect (100baseTX)

For more information, please see the ifconfig(8) manpage as well as the Auto-Negotiation Valid Configuration Table featuring "Why Can't the Speed and Duplex Be Hardcoded On Only One Link Partner?".

5.10 Reducing the image size

People complain that the image resulting from g4u is very big. This is normal as g4u clones the whole disks with all blocks, not attributing if they contain any valid data or if they are empty/unused. To find empty/unused blocks (and not clone them), g4u would need intimate understanding of the contained filesystem, which is different again for each filesystem - Windows FAT, Linux Ext2/3/ReiserFS/..., BSD FFS, Solaris UFS, etc. Given both tight space limitations on the floppy as well as shortage on filesystem documentation and implementations available, teaching g4u to ignore empty blocks is not likely to happen.

But there is an easy way to circumvent the problem: use the native operating system's understanding (and implementation) of the filesystem, and make sure it prepares empty/unused blocks in a way so they don't contain random garbage data but values which can be compressed easily by g4u, thus resulting in small image sizes.

Effectively, you just fill up the disk's unused blocks with zero-bytes. Open file for writing, stuff in 0-bytes until the disk is full, then close the file and remove it. The result is that all unused blocks were used by the file, and filled with data that g4u can then compress easily. Usually the operating system will just mark the blocks as unused, without changing the actual data content.

Using this technique on a 20GB disk that had 6GB Solaris 8/x86 and the rest Windows 2000 Workstation shrunk the image from ~6GB compressed to ~2GB compressed. You can probably imagine the effect of this on deployment time too. :)

To perform the filling of unused data blocks with zero-bytes, there are several ways, depending on what operating system you use on your computer, and what software you have available:

Standard Unix:

This works on any Unix variant - Linux, NetBSD, Solaris, etc.:

dd if=/dev/zero of=/0bits bs=20971520   # bs=20m
rm /0bits

Windows Perl solution:

This one needs perl installed. In a command shell, type:

cd /d c:\
c:\win-preclone.pl c:

Click here to download the win-preclone.pl perl script.

Windows Pascal solution:

This pascal program was contributed by Matthias Jordan [mjordan at code-fu dot de]:

• nullfile-1.02.exe
• nullfile-1.02.pas

The programs are provided here without warranty.

Windows "Erasor":

This freeware program can erase your disk in a safe way, but it can also be told to just write a pattern of all-0-bits to the disk. Grab it at www.heidi.ie/eraser. Thanks to Stephen Krans [s040 at krans dot org] for the hint!

Windows "onboard" solution:

Aparently Windows XP comes with a tool to do some harddisk encryption that can also be used to write 0-bytes to the disk. To do so, run the following command: cipher /W:C: for drive C:. You will need to abort (Control+C) after the first round, else it will write random data after filling the disk nicely with 0-bytes.

Run any of these right before shutting down the operating system to create an image with g4u, and see the size difference.

5.11 Manual IP configuration

Sometimes you may not want or be able to use DHCP. Like if you are not the net admin.

dhclient > 0
more 0

Can't bind to dhcp address: address already in use
Please make sure there is no other dhcp server running and that there is no entry for the dhcp or bootp in /etc/indetd.conf . Also make sure your are not runnig HP JetAdmin sortware, which includes a bootp server

The items that need configuring are the

• an available IP address for the g4u PC. 192.168.0.222 is used in this example
• the network mask of the network segment the g4u PC is on, usually: 255.255.255.0 .
• IP address of the router. 192.168.0.1 in this example (if the HDimgStore is not on the same subnet)
  Values should be determined by using the information BEFORE shutting down the g4u PC.

  After booting g4u,

  1. Determine the g4u PC's NIC device name:
     ifconfig -a
     It will be in the form ex0, tlp0, but not lo0.
     (Note unlike Linux, NetBSD doesn't call ethernet cards eth0)

  2. Configure the network device's IP number and netmask.

     ifconfig xx0 192.168.0.222 netmask 255.255.255.0

  3. Last, you may want to make the default router known unless your FTP server is in the same IP subnet as the machine you want to use g4u on. Let's say the default router's IP address is then the command to enter is:

     route add default 192.168.0.1

  That's all - simple, huh? Just remember that g4u is still Unix! After these steps, you should be able to use g4u just as if it used DHCP.

  5.12 Extracting the g4u kernel

  I've been asked how to boot g4u from harddisk (using e.g. grub). The idea is to extract the kernel from the boot floppy, and hand that to grub (or whatever bootloader you want - anyone got docs on a PXE setup?). Here's how to extract the kernel, named "netbsd":

  % ( cat g4u-2.1-1.fs | dd bs=512 skip=16 ; \
  ?   cat g4u-2.1-2.fs | dd bs=512 skip=16 \
  ? ) | tar vxf -
  -r--r--r--  1 feyrer   netbsd        53948 Nov  3 23:08 boot
  -rw-rw-r--  1 feyrer   netbsd      1479905 Nov  3 23:08 netbsd
  

  Note that the kernel ("netbsd") is actually still compressed, which is fine for the NetBSD bootloader and probably GRUB, but just in case, you may want to uncompress it:

  % file netbsd
  netbsd: gzip compressed data, was "netbsd-INSTALL_G4U", from Unix
  % mv netbsd netbsd.gz
  % gunzip netbsd.gz 
  % ls -la netbsd
  -rw-rw-r--  1 feyrer  wheel  5523084 Dec  7 18:08 netbsd
  % file netbsd
  netbsd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped 

PXE to netboot g4u). Here's how to extract the kernel, named "netbsd":

% ( cat g4u-2.3-1.fs | dd bs=512 skip=16 ; \
?   cat g4u-2.3-2.fs | dd bs=512 skip=16 ; \
?   cat g4u-2.3-3.fs | dd bs=512 skip=16 \

? ) | tar vxf -
-r--r--r--  1 feyrer   netbsd        53948 Nov  3 23:08 boot
-rw-rw-r--  1 feyrer   netbsd      1479905 Nov  3 23:08 netbsd

Note that the kernel ("netbsd") is actually still compressed, which is fine for the NetBSD bootloader and probably GRUB, but just in case, you may want to uncompress it:

% file netbsd
netbsd: gzip compressed data, was "netbsd-INSTALL_G4U", from Unix
% mv netbsd netbsd.gz
% gunzip netbsd.gz 

% ls -la netbsd
-rw-rw-r--  1 feyrer  wheel  5523084 Dec  7 18:08 netbsd
% file netbsd
netbsd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped 

5.13 Netbooting g4u via PXE

In order to netboot g4u via PXE, you first need to extract the kernel from the g4u floppies, and then follow the steps that describe a netboot (diskless boot) of NetBSD/i386, see the NetBSD documentation for various ways to do this.

A writeup of the necessary steps for g4u is available in the list archives, either in french language by Jean-Christophe Guis or in an english language translation by Steve Clement.

Another writeup that documents how to configure Fedora Core 5 to act as PXE server is available from Aaron Axelsen.

5.14 What FTP server software to use?

When you're uploading or downloading images to or from your FTP server, and you see a line like

 553 Cannot send file larger than 4 gigabytes 

scroll by, you can assume that that line (and any others with a number at the start) originates from your FTP server, and it is thus not g4u that's buggy but your FTP server software that has a problem.

Some known working FTP server programs are:

• NetBSD's ftpd(8) that comes in the default installation
• GuildFTPd on Windows XP
• The FTP server that comes with Microsoft Windows 2000 and 2003 Server (there seem to be problems with any FTP servers that come with non-"Server"-Versions of MS Windows...)
• vsftp on your favourite Linux distribution (reported working: Fedora 2, Debian Sarge)
• Cerberus FTP
• Novell Netware's FTP server (reported working: Netware 6)
• TYPSoft FTP Server on a USB memory stick with Windows XP
• Filezilla Server on Windows 98SE and XP
• Mac OS X and Mac OS X Server's FTP daemon
• Bullet Proof FTP Server v. 2.3.1 (Build 26)
• (More? Let me know!)

5.15 Non-standard applications

g4u was originally made to setup a cluster of PCs. Since then, it has been used for several other types of hardware and application areas. I'd like to collect some of them here:
• Copied a dual drive Tivo
• Saved Novell NetWare server disks
• Copied a Nokia IP330 Checkpoint Firewall 1 boxes
• Install several clusters of firewalls, compute machines, school workstations, etc.
• ...
Please send me mail if you've used g4u to clone something funny, cool, unusal, geeky, etc.!

6. Support and reporting problems

6.1 Support

The following ways of getting support for g4u exist:
• Mailing list: g4u-announce@feyrer.de
  This list will contain announces about g4u only.
  • Subscribe or change your settings via the web, or
  • send a mail with "subscribe yourpassword" to g4u-announce-request@feyrer.de to subscribe. Please replace "yourpassword" with your personal, secret list password.
  • Look at the g4u-announce list archive.

• Mailing list: g4u-help@feyrer.de
  This list is intended for questions and answers about g4u.
  • Subscribe or change your settings via the web, or
  • send a mail with "subscribe yourpassword" to g4u-help-request@feyrer.de to subscribe. Please replace "yourpassword" with your personal, secret list password.
  • Look at the g4u-help list archive.
• Orkut "g4u" community:

  There's a "g4u" community at Orkut which I've created as a forum for g4u. As Orkut is too slow these days (Dec 2004), I won't follow discussion on Orkut a lot, and recommend using the above-mentioned mailing lists for getting timely response.

• Optional(!) commercial license are available, see the "Commercial license" page for more information.

6.2 Reporting problems

If you have trouble with g4u and want to report a problem, PLEASE add the following information. It's impossible to help you without knowing details on your systems & setup:
• What g4u version do you use
• What exact command(s) did you use
• What exact output did you get (all of it, 1:1, no interpretation on your own)
• What does "disks" say? "parts"?
• If you have some problems with some network card or disk driver, please include the relevant parts from "dmesg" output (scroll down with 'space')
• What operating system, filesystem type and FTP server software do you use on your FTP server?

7. Rebuilding from source

G4u is based on NetBSD boot floppy disk set. It consists of a custom kernel and custom bootfloppy, both stripped down to an absolute minimum to fit on two 1.44MB floppy disk or a CDROM. There are also the "copydisk", "uploaddisk" and "slurpdisk" scripts.

To rebuild the images:

• Note that for rebuilding, no root permissions are needed any more! Below, the "#" prompt shows action needed as root, "%" is the prompt for commands executed as user.

• Get a i386/PC machine running NetBSD 2.0. Cross-building from other Unix-like operating systems may work too, but is untested as far as g4u is concerned.

• Install mkisofs (part of the cdrecord package), e.g. either as precompiled binary:

       # PKG_PATH=ftp://ftp.NetBSD.org/pub/NetBSD/packages/2.0/i386/All 
       # export PKG_PATH
       # pkg_add -v cdrecord 

  or, if you have pkgsrc installed:

       # cd /usr/pkgsrc/sysutils/cdrecord
       # make install 

• Get NetBSD-current source from ~May 17th 2005 as /usr/src:

       % su 
       # mkdir /usr/cvs
       # chown $USER /usr/cvs
       # ln -s cvs/src /usr/src
       # exit
       % cd /usr/cvs
       % env CVS_RSH=ssh cvs -d anoncvs@anoncvs.netbsd.org:/cvsroot co -D 20050517 src 

• To be on the safe side, rebuild a full release (regression testing and more):

       % ./build.sh -N 1 -U release 

• Remember where to find the new compiler - adjust if necessary:

       % setenv T /usr/src/obj.i386/tooldir.NetBSD-2.0-i386/bin 

• Get the g4u 2.1 sources

• Unpack the g4u sources:

       % cd /usr/src
       % tar plzvxf .../g4u-2.1.tgz

• Apply patches:

       % cd /usr/src/sys/dev/ata
       % patch <wd.c.patch-g4u
       % cd /usr/src/sys/dev/scsipi
       % patch <sd.c.patch-g4u
       % cd /usr/src/sys/kern
       % patch <subr_prf.c.patch-g4u
      

• Build the G4U floppy & ISO-image:

       % cd /usr/src/distrib/i386/floppies/g4u 
       % $T/nbmake-i386 

• After that, you'll find "g4u1.fs", "g4u1.fs" and "g4u.iso" in /usr/src/distrib/i386/floppies/g4u:

       % pwd
       /usr/cvs/src/distrib/i386/floppies/g4u
       % ls -l g4u.*
       -rw-rw-r--   1 feyrer  netbsd  3309568 Nov  3 23:09 g4u.iso
       -rw-rw-r--   1 feyrer  netbsd  1474560 Nov  3 23:08 g4u1.fs
       -rw-rw-r--   1 feyrer  netbsd  1474560 Nov  3 23:08 g4u2.fs 

8. Links & Ressources

A tutorial for installing M0n0wall with g4u:

Roberto Pereyra provided me with instructions on how to install the M0n0wall firewall with g4u. Cool to see them provide their own disk images!

Harvesting GUIDs / UUIDs for RIS prestaged computer accounts

``One of the most time consuming and overly repetitive tasks in supporting an organization with a large number of computers is handling turnover and new machine orders. A typical setup consisted of copying down the MAC addresses and destination hostnames on a piece of paper, typing them up and emailing them to someone else, update the dhcp server, etc., etc. This project is to reduce the ammount of human intervention needed to set up a machine.'' And as a bonus, it's also based on NetBSD! :) [link]

The iR110/150 (sparc64) dd Network Backup Page:

People have asked me for a UltraSPARC (sparc64) based version of g4u. Well here's one: The iR110/150 dd Network Backup Page documents scripts that do disk backups of an UltraSPARC system to a remote NFS server. Bonus to the author for using Solaris! :-)

Yet Another Ghost Installer (YAGI):

Daniel Ettle's diploma thesis which I supervised, about a network based installer at Toshiba Regensburg Operations.

Symantec Ghost

Commercial product that didn't do what I needed and which led me to create g4u. Still, check it our if you want non-image based cloning. If you think they're too expensive, remember that I don't want to starve too. :)

9. History

2.1	Changes: Added serial drivers (com on isapnp and isa) back for those people that need serial console (a few asked...), added support for IT Express and ATI IXP IDE and Promise SATA150 controllers, many bugfixes and enhancements from NetBSD-current, including EHCI-based USB improvements, and "dmesg" now hints at pressing space/CR to scroll down [20050622]
2.0	Yes, that's 2.0 - really! I got so fed up with the constand lack of space on the g4u floppy version that I finally decided to end the never-ending madness of removing drivers to make it fit. Instead, there are two versions of g4u now: a CDROM-version (can also be used on DVD at no extra costs), and a TWO-floppy-version (consisting of, well, TWO floppies, instead of the one used so far)! When booting the floppy version, it will load the first floppy, then ask for the second one. Press RETURN, and it will load the second floppy. (The splitting onto more than one floppy was made possible by NetBSD's fine infrastructure which is used for g4u, and which allows booting from a multi-volume tar-archive). The floppy and CDROM version still share the same codebase - there's still a limit of 2.88MB now, but this allows me adding back all the drivers I had to remove in the past: The wscons console driver, which allows using USB keyboards SATA drives (updated) I2O devices CS8900 (ISA) Ethernet Alteon ACEnic gigabit Ethernet PCMCIA and Cardbus drivers for SCSI, IDE and a *lot* of ethernet and wireless network cards. Fiber Channel drivers for your SAN! Hardware RAID drivers: Adaptec AAC family, AMI/LSI Logic MegaRAID, Compaq PCI array controllers, ICP-Vortex GDT and Intel RAID, Mylex DAC960 & DEC SWXCR family and 3ware Escalade RAID controllers. Whee! Wireless LAN cards: Aironet PC4500/PC4800, Atheros 5210/5211/5212, ADMtek ADM8211, Intersil Prism Mini-PCI. Token Ring drivers for both PCMCIA and ISA IBM/3COM TROPIC cards Besides all these drivers which were previously removed from g4u and now added back, be sure to check out the full list of supported hardware, including more wavelan cards, gigabit ethernet and SATA drives! Another minor addition made in this release is the 'copypart' program, which can be used to, well, copy single partitions locally. See Copying a partition locally for more information. So, this all sounds like christmas? That's a bit away, but maybe you want to donate a bit so I can buy myself and my beloved some nice present (and maybe even one that allows me to do a port of g4u to an Apple PowerBook G4? *hint, hint*)? ;) Those that want to do some good not only to me but to themselves and their beloved too can now aquire an optional commercial license for g4u. g4u will remain free to use for both private and commercial users, but I've received many requests for commercial licenses, and here they are! Last but not least, I'm fed up with the slowness of Orkut, and using the infrastructure from Source Forge, I've created two mailing lists, one for g4u related announces, and one for general questions on g4u. See the support section for more information. [20041207]
Still 1.17	Today I had to discover that someone used g4u as a source for g4l, but instead of giving proper credit and respecting the licensing and copyright on g4u, all traces were removed that the core work was done by me. Instead, the author put his name in and his work under a different license. I'm not happy about this, and ask people to discourage such actions as well as to despise the the author of g4l (cowardly calling himself only "nme"). Read more about this in my blog.
1.17	Major bugfix release which intends to address all the flaws in previous releases (see release notes for 1.15 and 1.16)! Besides the Freshmeat g4u page, interested parties can now find a g4u community with forum etc. on Orkut - drop me a mail if you want/need an invitation or have any questions (just as before :)! [20040718]
Back to 1.15!	There's a critical flaw in slurpdisk and other restore facilities in 1.16, which render g4u 1.16 unusable. The QA team has been fired, and we're back to 1.15. Sorry! :( [20040702]
1.16	OK, let's see if I got things right this time: Fixed copydisk, which I broke while adding the progress bar code. Document default GZIP level when calling uploaddisk and uploadpart with no argument. Shaved more space from the config by going for the generic PCIIDE driver (only; please let me know if this made things a lot slower and if it killed SATA-support esp. if enabling IDE in your BIOS won't help - and send me a SATA-equipped machine to test this in the future! :-), removed 'rnd', gave the kernel a fixed device to start from (the ramdisk), removed COMPAT_*. Replaced console driver - you may need to turn on support for USB keyboards in your BIOS! [20040630]
still 1.15	There's a bug in the "copydisk" command of g4u 1.15, please use g4u 1.14 if you need the copydisk command.
still 1.15	I have a donations page now!
1.15	Finally: Improved progress report! No more dots, volume transferred and current throughput is printed instead. Other changes: 'help' command to re-print help screen; update help screen a bit and made the prompt look a bit friendlier. Consistent GZIP handling. Building as non-root is now fully working (Thanks NetBSD!). No more Token Ring drivers, sorry, ran into space issues again. Thank for everyone who has donated so far, I'm working on a donations page. FWIW, donations of books from Amazon are welcome as well, see my wishlist: [20040412]
1.14	After some minor *cough* bugs in 1.13, here's 1.14. No more "file system full" etc., no functional changes either. Added a few screenshots to the documentation. Keep the donations coming! :-) [20040218]
Still 1.13	Source update! The sources released for 1.13 had a small problem (missing gz_compress, etc. while linking), I've updated the g4u-1.13.tgz source archive. Proper checksums are 2226011164 (CRC, cksum(1)) and fa2e2f00b079be1a9cf1c81d625896b1 (MD5). [20040122]
1.13	13's a bad sign, and version 1.13 means so for g4u. Due to my job situation I'm not sure if I can continue working on g4u in the future. Your donations can motivate to do so - paypal@feyrer.de is waiting for you. Today, g4u is mentioned in in one place with commercial products like PowerQuests's DriveImage and Norton's Ghost. g4u is free software and I hoped upon it's users to support it via donations. If I just got $10 from everyone sending me questions about g4u, that'd be more than enough. Fact is that only 1 person has donated back some money so far. Here's your chance to contribute back to the Open Source community - join in! Griping aside, V1.13 is a wrapup based on NetBSD-current as of Jan 14th 2004, esp. made after many people hat troubles booting 1.12. New drivers are for Adaptec 29320, 39320 (aic790x) SCSI (ahd), Broadcom 4401 10/100 Ethernet (bge), DECchip 21x4x and clones Ethernet (tlp), Intel 8254x gigabit (wm), Intel IGP01E1000 MII driver (igphy), Intel i31244 and SiI SATALink SATA controllers. See the g4u kernel config file for all the details. Matthias Jordan has also sent me an update of his "nullfile" to v1.02 which I've added to the g4u webpage. Due to some things growing again, I had to remove all Wireless LAN and most PCI RAID controller drivers. Maybe it's time to start special g4u versions for laptops (with pcmcia, cardbus, usb, wavelan), servers (with raid, ...) in addition to the current version of g4u. Let me know if you're interested! [20040111]
1.13pre2	Prerelease of 1.13 ISO after many people had problems booting the 1.12 ISO. No new features, but two administrative things to note: First, I've lost my job on Jan 1st 2004 due to the Bavarian government killing jobs in academics. That job was the main motivation behind creating g4u, and development of g4u beyond that date is uncertain. I plan to do at least one more release to wrap up things, but that'll need some more time to decide what drivers to include (not). Second, I have created a paypal account paypal@feyrer.de, if you want to donate money, buy me a pizza or coke or just SUPPORT FREE SOFTWARE, do so! (Judging by the feedback I got both in personal mail and in public forums, g4u seems to be a major alternative to some commercial products these days. I'd be happy if the people using it could give some money back to make it possible to continue working on the project! Yes, this is a plea for help!)
1.12	Fix broken dd commmand ("out of memory!"). [20030820]
1.11	New commands "uploadpart" and "slurppart" to save and restore partitions. Partition information is not changed and taken from MBR. Command 'disks' to list disks found, command 'parts' to list (BSD!) partitions on a given disk (wd0, ...). "uploaddisk" now has a default of rwd0d.gz for the imagename, so just "uploaddisk server" should work (just as "slurpdisk server" does already). [20030819]
1.10	Bugfix release: make "GZIP=1 uploaddisk ..." work properly; document how to use a different account than 'install'. Document how to get the image size down by filling unused disk blocks with zero-bytes. [20030603]
1.9	Maintenance update for NetBSD-current as of today; "dmesg" is now page-wise (|more); "disks" lists disks recognized on boot; allow setting GZIP=-1 to get lower compression; increased NMBCLUSTERS for usb2ethernet; Cloning from local disk to local disk with "clonedisk"; Lots of doc updates. [20030522]
1.8	The Pumpkin Release! Sync with NetBSD 1.6. Now includes drivers for USB, RAID cards, Gigabit Ethernet and Token Ring. As a Halloween special, g4u now also comes as ISO image file in addition to a floppy image, so it can be booted from a CD instead of a floppy. Happy Halloween! :) [20021031]
1.7	Update to accomodate latest drivers and kernel features from NetBSD 1.5.2. Fixed download of the .fs files from the web server. Added documentation about disk sizes. [20010919]
1.6	Maintenance release to use latest drivers from NetBSD 1.5.1. Due to increased size, the Token Ring driver was removed, sorry. Greeting message of g4u with instructions was improved a bit. Web site now has a copy of the kernel config, for documentation of supported device drivers. [20010516]
1.5	This release adds support for easy cloning of SCSI disks by giving "sd0" as the third argument to slurpdisk and uploaddisk. Also, support for i386 and i486 CPUs was added. [20010515]
1.4	Maintenance release. [20010306]
	Played with GIMP to create the Animation next to the TOC. Thanks to the XBill artists for the OS images! Various doc updates. [20010306]
	Updated the web page a bit to answer various questions on supported filesystems and operating systems I got. Also add TOC. [20010305]
1.3	Another bugfix release. This should get to a working state soon (i.e. now) [20010302]
1.2	Bugfix release, the ramdisk was busted so that many commands just didn't work (ftp, ...) [20010301]
1.1	First public release, updated to NetBSD 1.5 [20010228]
1.0	Internal version based on NetBSD 1.4 that's been in use for two years in internal operation.

10. Copying, licenses & donations

Please see the package author's page for details http://www.feyrer.de/g4u
aka http://fbim.fh-regensburg.de/~feyrer/g4u/


g4u is distributed under the standard BSD license: