!doctype html> rsync network file transfer/ mirror 230210

rsync

Severely tersified by DGG, original Here

High performance, file copying tool, used locally or to/from another host, for backup and mirroring. Can perform most ftp tasks.

Local side is refered to as the "client" and the remote as the "server".
rsyncd, the (optional) daemon is always a server. A remote server can be either a daemon or a remote-shell spawned process.

Local: rsync [option] src[dest]

Use remote shell: (when rsyncd is not running on host) notice single ':'
Pull: rsync [option] [user@]host:src[dest]
Push: rsync [option] src[/][user@]host:dest
To copy a directory to a different name,
use a trailing slash on the source directory

Use rsync daemon at remote host. (notice ::)
Pull: rsync [option] [user@]host::src[dest]
rsync [option] rsync://[user@]host[:port]/src[dest]
Push: rsync [option] src[user@]host::dest
rsync [option] src… rsync://[user@]host[:port]/dest

Single file example to host not using remote rsyncd. Useful when transfering a large file (SD card image) when ftp quits without completing the transfer. Anticipating interruption of transfer and being able to continue later, including having the log saved.

rsync  --verbose --verbose  --compress --progress  \
              --log-file=rsync$yymmdd.log \
                                             SD-16GB.200421.gz/ user@host:
Retrieve entire directory: (First with dry-run to see what is going to happen)
ruser=DaRemote$UNAME
rhost=DaRemoteSHot
rsync --verbose --verbose --compress --progress --archive --xattrs --acls --log-file=/tmp/rsync.$yymmdd.log \
                --update   --dry-run                                                                          \
                                        $ruser@$rhost:/home/$ruser/Documents/     . # dot is destination

Display updates from a repository:

rsync --recursive user@fromHost:directory/sub . --list-only 

Copies locally (perhaps network shares) if neither the source or destination path are a host.

Most basic transfer options:
--compress Compress in flight data ( no effect on destination) .
--archive Recursion, presering basic attributes (permissions, owner, type symbolic links ).
--atimes Preserve access times
--crtimes Preserve creation times.
--xattrs Preserve eXtended attributes.
--acls Preserve Access Control Lists.
--update If existing file is current or newer, skip transfer.
--log-file=rsync$yymmdd.log log ( .profile includes export yymmdd=`date +"%y%m%d"`
--hard-links preserve hard links, see symbolic links
--exclude-from=rsync-excludes exclude patterns (one per line).

Is your shell clean? abort message.

Examples
There is a lot to know about rsync ( and I only know a little), here's what you HAVE TO know.

If a single source is specified without a destination, the files are listed in an output format similar to ls -l.


The "=" is optional; whitespace can be used.
--no-option Turn off implied options
For example --archive is a synonym for -recursive --links etc. follow it with --no-links to not recreate links.

Example: --no-devices, --no-perms have different defaults in various circumstances
Example: --no-whole-file, --no-blocking-io, --no-dirs.

The order of the options is important. The result of
--no--recursive --archive is that --recursive would end up being turned on, the opposite of
--archive --no--recursive.

--list-only Use to specify more than one source (explicititly or with wild-card expanded by the shell into multiple args) for listing, otherwise second source would be intrepreted as the destination.
rsync --list-only uname@Real-World-Systems.com:*.txt # list all txt files on that host's root directory
-rw-r----- 160 2017/05/18 20:31:30 apacheNotes.txt
-rw-r--r-- 1490 2018/10/22 16:57:32 pepper-181022.txt
-rw-r----- 3679 2017/12/29 19:04:47 smacproPart4.txt

rsync --list-only /Volumes/big/*.MOV # list all the MOV files on that volume/dirctory
-rw-r--r-- 308842544 2018/05/16 07:56:53 CamperGrandTour_2064.MOV
-rw-r--r-- 12014106 2018/06/23 15:40:36 DoorClosingWithGap188.MOV
-rw-r--r-- 185538306 2017/02/18 13:29:27 Hummers_0655.MOV
-rw-r--r-- 19434966 2019/01/19 09:28:38 IMG_0332.MOV
--recursive is not effective
-n
--dry-run
No transfers, produces a similar report to a real run.
-v
--verbose
By default, rsync works silently.
Gives information about files being transferred and a brief summary at the end.
No mention is made if a file had its attributes changed.

--verbose --verbose Includes files are being skipped, files over max-size,
hiding file xxx because of patterni x .

Filenames are displayed in --out-format of %n%L by default, ( name and if it is a link, where it points).

-q
--quiet
decreases information reported.
Useful when invoking from cron.
Use with --verbose and/or --itemize-changes to see what a command is going to do.
options affecting report
-i
--itemize-changes
Increases verbosity. Default --out-format='%i %n%L'.
Repeating the option causes unchanged files to be output,.

.d        var/
.f        var/logPrune.sh 
%i string YXcstpoguax, where
Y . item is not being updated (attributes may be modified).
< file is being sent.
> file is being received
c change/creation is occurring for the item
(such as the creation of a directory or the changing of a symlink, etc.).
h hard link to another item (requires --hard-links).
* itemized-output area contains a message (example: "deleting").
X file-type
f file
d directory
L symlink
D device
S special file (examples: named sockets and fifos).

The other letters in the the associated attribute being updated
. no change.
+ new item ++++++++++
space identical item
?unknown attribute
 Reasons for transfer
c regular file has a different checksum (requires --checksum) or
symlink, device, or special file has a changed value.
s size of a regular file
t time of last modification (requires --times).
Ttime of last modification will be set to the transfer time
if a file/symlink/device is updated without --times or
   a symlink is changed and the receiver can't set its time.
p permissions are different (requires --perms).
o owner is different (requires --owner and super-user privileges).
g group is different (requires --group and the authority to set the group).
u future use.
a ACL (Access Control List) information changed.
x extended attribute information changed.
When deleting files, %i will output the string *deleting.
--out-format=format string of field specifiers.
Default with --verbose is %n%L (name and, if the item is a link, where it points). See "log format" in rsyncd.conf .

Each file, dir, etc. that gets updated in a significant way (a transferred file, a recreated symlink/device, or a touched directory) will be output.
(e.g. With --itemize-changes the logging of names includes any item changed.

The out-formated string is output prior to a file's transfer.
If a transfer-statistic option is requested, logging is done at the end of the file's transfer. When this late logging is in effect and --progress is specified, the name of the file being transferred is output prior to its progress information followed by the out-format output.

--log-file=file For a non-daemon transfer logging with a default %i %n%L. Example: Remote side logs what is happening:
rsync --archive --verbose --rsync-path="rsync --log-file=/tmp/rlog" src/ dest/
Useful to diagnose connection closing.
--log-file-format=format "transfer logging" unless the string is empty, then logging is turned off.
See log format in rsyncd.conf .
Default with --log-file %i %n%L i.e. name link.
-h
--human-readable
K, M, or G. units are K (1000), M (1000*1000), and G (1000*1000*1000);
-h -h , units are powers of 1024 .
--stats output statistics
Number of files: 88417
Number of files transferred: 0
Total file size: 5008416760 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 2710302
File list generation time: 1.156 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 20
Total bytes received: 2710322

sent 20 bytes  received 2710322 bytes  285299.16 bytes/sec
total size is 5008416760  speedup is 1847.89
--progress Implies --verbose
While transferring a regular file, updates a progress line like :
782448  63%  110.64kB/s    0:00:04
The transfer might finish in :04 seconds

Statistics can be misleading with the delta-transfer algorithm. For example, if the file is a log previously sent the reported rate will drop dramatically when the receiver gets to the new data, and the transfer will take much longer to finish than predicted.

When a file transfer finishes, the progress line is replaced with a summary line like :

1238099 100%  146.38kB/s    0:00:08  (xfer#5, to-check=169/396)
This was the 5th regular file transfered, 169 more files for the receiver to check out of the 396 files
Handling file match
Default: skip files with the same size and modification timestamp.
-I
--ignore-times
 
--size-only Useful when a system which does not preserve timestamps well.
--modify-window=seconds timestamps considered equal if they differ by no more than the seconds.
Default 0 (exact match).
FAT filesystem represents times with a 2-second resolution --modify-window=1 allows times to differ by up to 1 second.
-u
--update
Preserve a destination with a newer modification time.

Does not affect the copying of symlinks or other special files.

-c
--checksum
compare checksums for files with matching sizes.
Performance degradation affecting both sides, requires reading all the data in the files.

transfer handling

--append ONLY for files known to "growing", for example logs. Probably inapproiate for transfering entire directories.
No check is made to ensure that the begining of source and destination are the same.
The update occurs inplace.
No transfer occurs if the source file is not larger. The file's attributes ( permissions, ownership, etc.) are not updated.
Does not affect the updating of non-regular files.

Important for transfers which need to be restarted.

--append-verify --append, only if checksums of the beginning of the files match.
--inplace Writes directly to the destination file.
Without this a tempfile is created and moved when its transfer is complete preserving the previous version in the event of an aborted transfer.

Useful when

  • There is insufficient free space on the destination file system to contain the existing file and the new temp version.
  • transfers with large files that may to be interrupted.
  • transfers include large files with block-based changes or appended data,
  • systems are disk bound, not network bound.
    Effects:
  1. During the transfer, file will be inconsistent
  2. source files being updated will not be transfered.
  3. destination files being read cannot be updated.
  4. If the transfer is interrupted or fails, file may be left in an inconsistent state
  5. A file that does not have write permissions can not be updated, and
  6. The efficiency of the delta-transfer algorithm may be reduced if some data in the destination file is overwritten before it can be copied to a position later in the file
    Combine with --backup, since rsync will use the backup file as the basis file for the transfer to prevent this.

WARNING: Do not use this to update files being accessed by others.

Implies --partial (an interrupted transfer does not delete the file),
Conflicts with --partial-dir and --delay-updates.

--partial Keep partially transfered file.
Default: delete partially transferred file
--partial-dir=partdir directory holding partial data, Implies --partial.

If partdir is an absolute path it is consideered an existing working directory.

Do not use an absolute path unless
(1) there are no files in the transfer having the same name (since all the updated files will be put into a single directory )
and
(2) there are no mount points in the hierarchy (the delayed updates will fail if they can't be renamed into place).

If partdir is a relative path, it will be made and excluded preventing sending partdir files on the sending side, and prevent untimely deletion of partdir files on the receiving side.
An example: the above --partial-dir option would add the equivalent of -f '-p .rsync-partial/' at the end of any other filter rules.

Add an exclude/hide/protect rule for the partdir to

  • override default.
  • the auto-added rule may be ineffective at the end of other rules, To clean-up partdirs, specify --delete-after and add a "risk" filter rule, eg. -f 'R .rsync-partial/'.

    Avoid using --delete-before or --delete-du ring unless the left-over partdir data is not needed during the current run.)

    The partial-dir should not be writable by other users and avoid using /tmp.

    Set the partdir to $RSYNC_PARTIAL_DIR. Setting this in $RSYNC_PARTIAL_DIR does not enable --partial only where partial files go when --partial is specified. For instance, instead of using --partial-dir=.rsync-tmp along with --progress, Set $RSYNC_PARTIAL_DIR=.rsync-tmp and then use -P to use that for partial transfers.
    --partial does not use this with:

    • --inplace
    • --delay-updates
    .

    For the purposes of "refuse options" setting, --partial-dir does NOT imply --partial. a refusal of --partial can be used to disallow the overwriting of destination files with a partial transfer, while still allowing the safer idiom provided by --partial-dir.

    If --whole-file is specified (or implied), partial-dir files are removed.

  • -P equivalent to --partial --progress. Useful for a long transfer that may be interrupted.
    --delay-updates puts the temporary file in a holding directory until the end of the transfer, then all the files are moved into place in an attempt to make the updating of the files more atomic.
    By default the files are placed into .~tmp~/ in each file's destination directory. Change the directory with --partial-dir.

    Conflicts with --inplace and --append.

    Requires enough free disk space to hold an additional copy of all the updated files.

    See the "atomic-rsync" perl script in the "support" subdir for an update algorithm that is more atomic (it uses --link-dest and a parallel hierarchy of files).

    -W
    --whole-file
    delta-transfer is not used.
    The transfer may be faster if this option is used when the bandwidth between the source and destination machines is higher than the bandwidth to disk (example: when the "disk" is actually a networked filesystem).
    Default when source and destination are local paths, if no batch writing option is in effect.
    -S
    --sparse
    Handle sparse files efficiently so they take up less space on the destination.
    Conflicts with --inplace.
     
    -R
    --relative
    Use relative paths.
    The full path names specified on the command line are sent to the server rather than just the last parts of the filenames.
    Useful when sending several different directories at the same time.
    For example:
    > rsync -av /foo/bar/baz.c remote:/tmp/ 
    Creates /tmp/baz.c.

    > rsync -avR /foo/bar/baz.c remote:/tmp/ 
    creates /tmp/foo/bar/baz.c preserving its full path.

    These extra path elements are called "implied directories" ("foo" and the "foo/bar" directories in the above example).
    rsync sends these implied directories as real directories in the file list, even if a path element is really a symlink on the sending side. This prevents unexpected behaviors when copying the full path of a file that you didn't realize had a symlink in its path.
    To duplicate a server-side symlink, include both the symlink via its path, and referent directory via its real path.

    To limit the amount of path information sent as implied directories for each path, insert a dot and a slash into the source path.

    >rsync -avR /foo/./bar/baz.c remote:/tmp/
    Creates /tmp/bar/baz.c on the remote machine. (Note that the dot must be followed by a slash, so "/foo/." would not
    --no-implied-dirs affects the default behavior of --relative .
    The attributes of the implied directories from the source names are not included in the transfer. This means that the corresponding path elements on the destination system are left unchanged if they exist, and any missing implied directories are created with default attributes. This allows these implied path elements to have big differences, such as being a symlink to a directory on the receiving side.

    For instance, if a command-line arg or a files-from entry told rsync to transfer the file "path/foo/file", the directories "path" and "path/foo" are implied when --relative is used. If "path/foo" is a symlink to "bar" on the destination system, the receiving rsync would ordinarily delete "path/foo", recreate it as a directory, and receive the file into the new directory. With --no-implied-dirs, the receiving rsync updates "path/foo/file" using the existing path elements, which means that the file ends up being created in "path/bar".
    Another way to accomplish this link preservation is to use the --keep-dirlinks option (which will also affect symlinks to directories in the rest of the transfer).

    -b
    --backup
    Existing destination files are renamed before files are transferred or deleted.
    (watch free space on destination).
    --backup-dir =DIR With --backup store all backups in DIR on the receiving side. This can be used for incremental backups.
    --suffix=suffix if no --backup-dir was specified Default suffix is ~
    with --backup-dir no suffix is appended by default.

    without --backup-dir,

    1. --omit-dir-times is implied, and
    2. if --delete is in effect (without --delete-excluded), rsync will add a "protect" filter-rule for the backup suffix to the end of all existing excludes (e.g. -f "P *~").
    Prevents previously backed-up files from being deleted.
    With filter rules, insert own exclude/protect rule higher up in the list so that it has a high enough priority to be effective (e.g., if your rules specify a trailing inclusion/exclusion of '*', the auto-added rule would never be reached).
    -d
    --dirs
    Tell the sending side to include any directories that are encountered. Unlike --recursive, a directory's contents are not copied unless the directory name specified is "." or ends with a trailing slash (e.g. ".", "dir/.", "dir/", etc.). Without this option or --recursive , rsync will skip all directories it encounters (and output a message to that effect for each one).
    Specifing both --dirs and --recursive, --recursive takes precedence.

    --dirs is implied by --files-from or --list-only (including an implied --list-only usage) if wasn't specified (so that directories are seen in the listing).
    Specify --no-dirs to turn this off.

    Link handling

    skip to attribute handling
    -l
    --links
    recreate symlinks on the destination.
    -L
    --copy-links
    for symlinks, the item that they point to (the referent) is copied.
    --copy-unsafe-links copy the referent of symlinks that point outside the copied tree.
    Absolute symlinks are treated like ordinary files, and as are symlinks in the source path itself when --relative is used.
    Has no additional effect if --copy-links was also specified.
    --safe-links ignore any symbolic links which point outside the copied tree and absolute symlinks
    With --relative may give unexpected results.
    -k
    --copy-dirlinks
    sending side treats a symlink to a directory as though it were a real directory.
    Useful if you don't want symlinks to non-directories to be affected, as they would be using --copy-links.

    Without this , if the sending side has replaced a directory with a symlink to a directory, the receiving side will delete anything that is in the way of the new symlink, including a directory hierarchy (as long as --force or --delete is in effect).

    See --keep-dirlinks for an analogous option for the receiving side.

    -K
    --keep-dirlinks
    receiving side treats a symlink to a directory as though it were a real directory, only if it matches a real directory from the sender.
    Without this the receiver's symlink would be deleted and replaced with a real directory.

    For example, transfer a directory "foo" that contains a file "file", but "foo" is a symlink to directory "bar" on the receiver.
    Without --keep-dirlinks, the receiver deletes symlink "foo", recreates it as a directory, and receives the file into the new directory.
    With --keep-dirlinks, the receiver keeps the symlink and "file" ends up in "bar". caution: when using --keep-dirlinks, you must trust all the symlinks in the copy!
    If it is possible for an untrusted user to create their own symlink to any directory, the user could then (on a subsequent copy) replace the symlink with a real directory and affect the content of whatever directory the symlink references.
    For backup copies, use a bind mount instead of a symlink to modify your receiving hierarchy.

    See also --copy-dirlinks for an analogous option for the sending side.

    Attributes, Meta-data, i.e. file permissions, Access Control Lists, owner, group …

    -p
    --perms
    set the destination permissions to be the same as the source permissions.
    See --chmod to modify what are considered to be the source permissions.

    With --no--perms:

  • Existing files retain permissions, ( --executability might change the execute permission.)
  • New files Have access permissions set to the source permissions masked with the receiving directory's default permissions (either the receiving process's umask, or the permissions specified via the destination directory's default AccessControlList.
    Special mod bits are disabled except where a new directory inherits setgid from its parent.

    To give new files the destination-default permissions (while leaving existing files unchanged), use --no--perm and --chmod=ugo=rwX

    rsync alias --compress --no--perms --no--group --chmod=ugo=rwX

    then use : rsync -avZ src/ dest/

  • --chmod apply comma-separated chmod strings to the permission of the files in the transfer.
    as if they were the permissions supplied by the sender.
    Can seem to have no effect on existing files if --perms is not enabled.

    An item that should only apply to a directory is prefixed with D', or to a file if prefixed with 'F'.
    For example:

    --chmod=Dg+s,ug+w,Fo-w,+X
    Multiple --chmods are permitted.

    See --perms and --executability for how the resulting permission value can be applied to the files in the transfer.

    -E
    --executability
    preserve the executability (or nonexecutability) of regular files when --perms is not enabled.
    A regular file is considered to be executable if at least one 'x' is turned on in its permissions. When an existing destination file's executability differs from that of the corresponding source file, rsync modifies the destination file's permissions as follows:
  • To make a file non-executable, rsync turns off all its 'x' permissions.
  • To make a file executable, rsync turns on each 'x' permission that has a corresponding 'r' permission enabled.
  • -A
    --acls
    update the ACLs, implies --perms.
    See --fake-super to backup and restore ACLs that are not compatible.
    -X
    --xattrs
    update the remote extended attributes.
    When done by super-user, copies all namespaces except system.*.
    A normal user only copies the user.* namespace.
    To to backup and restore non-user namespaces as a normal user, see --fake-super .
    -o
    --owner
    Preserve owner (other than receiving user)
    Default: the owner is set to the user on the receiving side.

    only if the receiver is super-user (see --super and --fake-super ).
    Caution: The preservation of ownership will associate matching names but if it falls back to using the ID number (see --numeric-ids it will surely be incorrect!!.

    -g
    --group
    Preserve group
    If the receiving program is not running as the super-user (or if --no-super was specified), only groups that the invoking user on the receiving side is a member of will be preserved.
    Without this option, the group is set to the default group of the invoking user on the receiving side.

    The preservation of group information will associate matching names by default, but may fall back to using the ID number

    -t
    --times
    preserve modification times (implied with --archive).
    If times are not preserved (in addition to corrupting the modification time on the destination) the next time a transfer is performed, the evaluation which excludes files that have not been modified will be incorrect. The deltatransfer algorithm will minimize the update.
    -O
    --omit-dir-times
    omit directories when preserving modification times (see --times).
    Suggested if NFS is sharing the directories on the receiving side.
    Implied with --backup without --backup-dir.
     
    --super receiving side attempts to execute super-user activities even if it wasn't run by the super-user.
    Include: preserving users via the --owner option,
    preserving all groups (not just the current user's groups) via --groups , and
    copying devices via --devices .
    Useful for systems that allow such activities without being the super-user and
    ensuring errors if the receiving side isn't being run as the super-user.
    To turn off super-user activities, the super-user can use --no-super.
    --fake-super simulate super-user activities by saving/restoring the privileged attributes via special extended attributes.
    Includes the file's owner and group (if it is not the default), the file's device info (device & special files are created as empty text files), and any permission bits that won't be allowed to be set on the real file (e.g. the real file gets u-s,g-s,o-t for safety) or that would limit the owner's access (since the real super-user can always access/change a file, the files we create can always be accessed/changed by the creating user).
    Handles ACLs (if --acls was specified) and non-user extended attributes (if --xattrs was specified).
    backup data without using a super-user, and to store ACLs from incompatible systems.
    Only affects the side where the option is used. To affect the remote side of a remote-shell connection, specify an rsync path:
     rsync -av --rsync-path="rsync --fake-super" /src/ host:/dest/
    There is only one "side" in a local copy, this affects both the sending and receiving of files. Specify a copy using "localhost" to avoid this, possibly using the lsh shell script (from the support directory) as a substitute for an actual remote shell (see --rsh).

    overridden by both --super and --no-super.

    See fake super in rsyncd.conf.

    -x
    --one-file-system
    avoid crossing a filesystem boundary when recursing.
    This does not limit the ability to specify items to copy from multiple filesystems, just rsync's recursion through the hierarchy of each directory that the user specified, and also the analogous recursion on the receiving side during deletion.
    A "bind" mount to the same device is considered being on the same filesystem.

    If repeated, rsync omits all mount-point directories.
    Otherwise, it includes an empty directory at each mount-point it encounters (using the attributes of the mounted directory because those of the underlying mount-point directory are inaccessible).

    If collapse symlinks (via --copy-links or --copy-unsafe-links), a symlink to a directory on another device is treated like a mount-point. Symlinks to non-directories are unaffected by this option.

    --existing,
    --ignore-non-existing
    skip creating files and directories that do not exist on the destination.
    If combined with the --ignore-existing , no files will be updated (useful to delete extraneous files).

    This is a transfer rule, not an exclude, it doesn't affect the data that goes into the file-lists, and thus it doesn't affect deletions. It just limits the files that the receiver requests to be transferred.

    --ignore-existing skip updating files that already exist on the destination (this does NOT ignore existing directories, or nothing would get done). See also --existing. This is a transfer rule, not an exclude, so it doesn't affect the data that goes into the file-lists, and doesn't affect deletions.
    It limits the files that the receiver requests to be transferred. Useful for doing backups using --link-dest when continung a n interrupted backup. Since a --link-dest run is copied into a new directory hierarchy (when it is used properly), using --ignore existing will ensure that the already-handled files don't get tweaked (which avoids a change in permissions on the hard-linked files).
    This does mean that this option is only looking at the existing files in the destination hierarchy itself.
    Deleting Files on the receiving side
    This occurs when files on the sending side are deleted or moved to a different directory.
    If this run is expected to be a backup, this results with files deleted locally will not be recoverable.
    --delete For directories being synchronized (i.e. updating those previously sent using dir or dir/ ) files no longer on the sending side are deleted from the receiving side. (Not effective for files using a wildcard for the directory's contents which was expanded to a list of files.)
    excluded files are not deleted unless --delete-excluded or mark the rules as only matching on the sending side See include/exclude modifiers .

    Deletions will also occur when --dirs is enabled for directories whose contents are being copied.

    Including --dry-run shows files that would be deleted without deleting them.

    --delete-during(default)
    --delete-before
    --delete-after
    --delete-delay
    May be a better choice than after)
    --prune-empty-dirs
    , and --delay-updates.
    (use --del --delete-during to request this deletion mode explicitly).
    Incremental recursion can be disabled using --no-inc-recursive.

    If the sending side detects I/O errors, deletion of any files will be disabled,
    useful with NFS errors on the sending side)

    May be combined with one of the --delete-when as well as --delete-excluded.
    See --delete-delay and --delete-after.

    --delete-before Deletions are done before any transfer .
    See --delete.

    Helpful when the destination filesystem is low on space.
    Introduces a delay before the start of the transfer which might cause the transfer to timeout
    Forces non-incremental recursion that requires a scan of the files for the transfer into memory at once
    (see --recursive).

    --delete-during
    --del
    deletions are done as the transfer occurs.
    The per-directory delete scan is done before each directory is checked for updates
    --delete-delay File deletions are done after the transfer.
    Useful when combined with --delay-updates and/or --fuzzy, and is more efficient than using --delete-after (can behave differently, since --delete-after computes the deletions in a separate pass after all updates are done).
    If the number of removed files is very large --delete-after (unless --recursive is doing an incremental scan). See --delete.
    --delete-after file deletions are done after the transfer has completed.
    Useful when sending new per-directory merge files as a part of the transfer and you want their exclusions to take effect for the delete phase of the current transfer.
    Forces non-incremental recursion that requires scanning all the file names in the transfer into memory at once.
    --delete-excluded Include deleting files on the receiving side that are excluded.

    See the FILTER RULES section for a way to make individual exclusions behave this way on the receiver, and for a way to protect files from --delete-excluded

    --force Delete a directory when it is to be replaced by a file, even if the directory contains files. Only relevant if deletions are not active (see --delete for details).
    --ignore-errors --delete deletes files even when there are I/O errors.Danger
    --max-delete=N MAXimun number of files or directories deleted, once exceeded exit with an error code 25 .

    --max-delete=0 warning without removing

    --remove-source-files Remove from the sending side files successfully MOVED .
     
    options controling files to be transfered
    -a
    --archive
    synonym for -recursive --links --perms --times --group --owner --devices --specials
    ACL and eXtended attributes are not be preserved ed
    When --files-from is specified --recursive is not implied.

    Does not preserve hardlinks, ---hard-links.

    If -archive follows --compress it will reenable the "--no-" options

    The side-effects of the --files-from option are NOT positional, as it affects the default state of several options and slightly changes the meaning of --archive.

    -r
    --recursive
    copy directories recursively. See --dirs .
    -f
    --filter=rule
    add rules to selectively include/exclude files. Useful in combination with a recursive transfer.
    Multiple --filter options are permitted.
    If rule contains whitespace, quote it.
    An underscore can be used to replace the space that separates a rule from its arg. See FILTER RULES
    --max-size=bytes Supress transferring files larger than bytes.
    may be a fractional (e.g. --max-size=1.5m).
    A transfer rule limits the files that the receiver requests to be transferred
    not an exclude, doesn't affect the file-lists or deletions.
    Suffixes (case insensitive)
    • KB (1,000) : K KiB kilibyte (1024):
    • MB ( 1,000,000) :M MiB megibyte (1024*1024)
    • GB (1,000 MB) :G GiB gibibyte (1024*1024*1024)
    • +1 or -1 to offset by one byte.

    Examples:
    --max-size=1.5mb-1 is 1,499,999 bytes
    --max-size=2g+1 is 2,147,483,649 bytes.

    --min-size=mb Skip small files,i.e. those less than mb. small, junk files.
    -F
    -F-F
    Use .rsync-filter sprinkled through the hierarchy rules to filter the files and
    and exclude them.
    --exclude=pattern
    --exclude-from=file file contains exclude patterns (one per line).

    Blank lines and lines starting with ; or # are ignored.
    If FILE is -, the list will be read from standard input.
    -C
    --cvs-exclude
    Exclude files that shouldn't be transfered, similar to CVS
    These are marked perishable see filter rules
    1. The exclude list initialized with:
       RCS   SCCS   CVS   CVS.adm   RCSLOG   cvslog.*   tags   TAGS   .make.state   .nse_depinfo   *~   #*   .#*   ,*   _$*   *$   *.old   *.bak   *.BAK   *.orig   *.rej   .del-*   *.a   *.olb   *.o   *.obj   *.so   *.exe   *.Z   *.elc   *.ln   core   .svn/   .git/   .bzr/
    2. Files in $CVSIGNORE (delimited by whitespace). See CVS
    3. Files listed in $HOME/.cvsignore
    4. Files in a directory containing a .cvsignore
    5. Files matching a pattern in .cvsignore (delimited by whitespace).

    When combining --filter rules with -C CVS excludes are appended regardless of where the -C was placed on the commandline.
    This makes them a lower priority than any rules specified explicitly.
    To control where these get inserted into filter rules, omit -C as a command-line option and use a combination of --filter=:C and --filter=-C (either on the command-line or by putting the ":C" and "-C" rules into a filter file with other rules). :C turns on the per-directory scanning for the .cvsignore file. -C does a one-time import of the CVS excludes.

    --include=pattern
    --include-from= file as with --exclude-from
    --files-from=file Specify the exact list of files (as read from file or - for standard input).
    Adjusts the default behavior to make transferring the files and directories easier:
  • --relative is implied, presering path information
  • --dirs is implied, creating directories
  • --archive does not imply --recursive so specify it.

    These side-effects change the default state of rsync.
    The position of --files-from on the command line has no bearing on how other options are parsed (i.e. --archive works the same before or after --files-from, as does --no---relative ).
    The filenames read from file are relative to the source dir. Leading slashes are removed and no ".." references are allowed to go higher than the source dir. For example:

    rsync -a --files-from=rsyncfiles /usr remote:/backup
    With "bin" (or "/bin") in rsyncfiles the /usr/bin will be created as /backup/bin on the destination.
    With "bin/" (note the trailing slash), the immediate contents of the directory would also be sent (without explicitly mentioned in the file
    In either case, if --recursive was explicitly specified that dir's entire hierarchy would be transferred. The effect of --relative (enabled by default) is to duplicate only the path info that is read from the file -- it does not force the duplication of the source-spec path (/usr in this case).

    --files-from can be read from the remote host by using host:" prefix of the file which must match one end of the transfer. Using ":" means "use the remote end of the transfer". For example:

    > rsync -a --files-from=:/path/file-list src:/ /tmp/copy
    Copy files in /path/file-list on the remote "src" host.

    If --iconv and --protect-args are specified the filenames will be translated from the sending host's charset to the receiving host's charset.

  • -0
    --from0
    rules/filenames are terminated by a null ('00') ,
    This affects --exclude-from, --include-from, --files-from, and any merged files specified in a --filter rule. Does not affect --cvs-exclude
    -s
    --protect-args
    sends filenames and most options without allowing the shell to interpret them. This means that spaces are not split in names, and any non-wildcard special characters are not translated (such as ~, $, ;, &, etc.).
    Wildcards are expanded by rsync (not the shell).

    if used with --iconv, the args related to the remote side will also be translated to the remote character-set. The translation happens before wild-cards are expanded. See --files-from .

    -T
    --temp-dir=dir
    Use dir as a scratch directory when creating temporary copies of the files transferred on the receiving side.
    Default: create each temporary file in the same directory as the associated destination file.

    Useful when there is not enough free space to hold a copy of the largest file in the destination filesystem. This requires rsync to it copy the file into place when the file is transfered.

    Combined with --delay-updates , which will ensure that all copied files get put into subdirectories in the destination hierarchy, awaiting the end of the transfer.
    If there is not enough room to duplicate all the arriving files on the destination filesystem, use --partial-dir with a relative path; because it is OK to stash off a copy of a single file in a subdir in the destination hierarchy, rsync will use the partial-dir as a staging area to bring over the copied file, and then rename it into place from there. (Specifying a --partial-dir with an absolute path does not have this side-effect.)

    -y
    --fuzzy
    basis file for any destination file that is missing. The current algorithm looks in the same directory as the destination file for either a file that has an identical size and modified-time, or a similarly-named file. If found, rsync uses the fuzzy basis file to try to speed up the transfer.

    Use of --delete might get rid of any potential fuzzy-match files, so either use --delete-after or specify some filename exclusions to prevent this.

    --compare-dest=DIR use DIR on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination directory). If a file is found in DIR that is identical to the sender's file, the file will NOT be transferred to the destination directory. This is useful for creating a sparse backup of just files that have changed from an earlier backup.

    multiple --compare-dest directories may be provided, which will cause rsync to search the list in the order specified for an exact match. If a match is found that differs only in attributes, a local copy is made and the attributes updated. If a match is not found, a basis file from one of the DIRs will be selected to try to speed up the transfer.

    If DIR is a relative path, it is relative to the destination directory. See also --copy-dest and --link-dest.

    --copy-dest=dir like --compare-dest, also copy unchanged files found in dir using a local copy.
    Useful for doing transfers to a new destination while leaving existing files intact, and then doing a flash-cutover when all files have been successfully transferred.

    Multiple --copy-dest directories may be provided, which will cause rsync to search the list in the order specified for an unchanged file. If a match is not found, a basis file from one of the DIRs will be selected to try to speed up the transfer.

    If DIR is a relative path, it is relative to the destination directory. See also --compare-dest and --link-dest.

    --link-dest=DIR like --copy-dest, but unchanged files are hard linked from DIR to the destination directory.
    The files must be identical in all preserved attributes (e.g. permissions, possibly ownership) in order for the files to be linked together. An example:
    rsync -av --link-dest=$PWD/prior_dir host:src_dir/ new_dir/
    If file's aren't linking, double-check their attributes. Also check if some attributes are getting forced outside of rsync's control, such a mount option that squishes root to a single user, or mounts a removable drive with generic ownership (such as OS X's "Ignore ownership on this volume" option).

    multiple --link-dest directories may be provided, which will cause rsync to search the list in the order specified for an exact match. If a match is found that differs only in attributes, a local copy is made and the attributes updated. If a match is not found, a basis file from one of the DIR will be selected to try to speed up the transfer.

    works best when copying into an empty destination hierarchy, as rsync treats existing files as definitive (so it never looks in the link-dest dirs when a destination file already exists), and as malleable (so it might change the attributes of a destination file, which affects all the hardlinked versions).

    combine this option with --ignore-times, rsync will not link any files together because it only links identical files together as a substitute for transferring the file, never as an additional check after the file is updated.

    If DIR is a relative path, it is relative to the destination directory. See also --compare-dest and --copy-dest.

    -B
    --block-size=blksz
    forces the block size in delta-transfer algorithm to a fixed value.
    Selected based on the size of each file being updated. See technical report.
     
    -e
    --rsh=rmsh
    remote shell for communication. default: ssh to use rsh on a local network.

    With [user@]host::module/path, the remote shell to run an rsync daemon on the remote host, and all data will be transmitted through that remote shell connection, not a socket connection to a running rsync daemon on the remote host.
    See using rsync-daemon features via a remote-shell connection

    Command-line arguments are permitted in rmsh provided that is presented to rsync as a single argument.
    Use spaces (not tabs or other whitespace) to separate rmsh into the command and arguments.
    Use apostrophe and/or quotes to preserve spaces in an argument (not backslashes).
    Doubling an apostrophe inside a apostrophe'd string results in apostrophe ( likewise for quotes ) (attention to which quotes the shell is parsing and which quotes rsync is parsing).

    Examples:

     -e 'ssh -p 2234'
     -e 'ssh -o "ProxyCommand nohup ssh firewall nc -w1 %h %p"' 
    
    ssh uses .ssh/config for site specific options.

    $RSYNC_RSH specifies remote shell program, which accepts the same range of values as -e.
    See --blocking-io which is affected by this .

    --devicescharacter and block device files are created on remote system.
    receiving rsync must be super-user (see also the --super and --fake-super options).
    --specials transfer special files such as named sockets and fifos.
    -D equivalent to --devices --specials
    --rsync-path=fullName When rsync is not in the remote path
    Example: --rsync-path=/usr/local/bin/rsync .
    fullName is run with a shell and can be any program, script, or command sequence , so long as it does not corrupt the standard-in & standard-out that rsync is using to communicate. To set a different default directory on the remote for use with --relative. example:
                      rsync -avR --rsync-path="cd /a/b && rsync" host:c/d /e/  
    -z
    --compress
    compresses data as it is sent. ( better compression than a compressing transport )
    --skip-compress=suf[[/…] Override the list of file suffixes that will not be compressed.
    An empty string to indicates that no file should be skipped.

    Simple character-class matching is supported: each must consist of a list of letters inside the square brackets (e.g. no special classes, such as "[:alpha:]", are supported).

    asterisk (*) and question-mark (?) have no special meaning.

    Example specifies 6 suffixes to skip (since 1 of the 5 rules matches 2 suffixes):

    --skip-compress=gz/jpg/mp[34]/7z/bz2
    Default:
    gz/zip/z/rpm/deb/iso/bz2/t[gb]z/7z/mp[34]/mov/avi/ogg/jpg/jpeg/

    A copy from a daemon rsync will adds skipped suffixes to its list of non-compressing files (and its list may be configured to a different default).

    --compress-level=n compression level. If n is non-zero, --compress is implied.
    --password-file=file for accessing an rsync daemon, must not be world readable.

    When accessing a daemon using a remote shell, this option only comes into effect after the remote shell finishes its authentication (i.e. if a password is specified in the config file).

    For shell transfers use "Public key authentication".

    --numeric-ids Use numeric user and group IDs rather than using user and group names. DANGER!

    By default username and groupname are used to establish ownership.
    If a user or group on the source system does not match the destination, the numeric ID from the source system is used.

    If on the source system, user jsmith is user ID 507 and on destination is user ID 509 and bsmith is user ID 507, files on the the destination will belong to bsmith !

    See the comments on the "use chroot" setting in the rsyncd.conf manpage for information on how the chroot setting affects rsync's ability to look up the names of the users and groups and what you can do about it.
    The special user and groups ID 0 are never mapped names even if the --numeric-ids option is not specified.
    Only applicable when transfering files with various owners.

    Options related to networking
    --bwlimit=kbps BandWidth limit, kilobytes per second. Only effective with files greater than 3 MB. Zero: no limit.
    --timeout=seconds If no data is transferred for seconds rsync exits with rc=12. Default:0, no timeout.bad
    --contimeout=seconds Wait seconds for a connection to the rsync daemon, ( host::dir) afterwards exit with rc=35.
    --address address Specific IP address (or hostname) to bind to for virtual hosting. See daemon mode .
    --port=port Only useful connecting with an rsync daemon using non default port. Default:873.
    URL for connection via shell can include a port.
    -4, --ipv4
    -6, --ipv6
    Only affects sockets that rsync has direct control over, such as the outgoing socket when directly contacting an rsync daemon
    --sockopts tune systems to the utmost degree. See rsync docs.
    --blocking-io when launching a remote shell transport.
    If the remote shell is rsh or remsh defaults to using blocking I/O.
    -8
    --8-bit-output
    leave all high-bit characters unescaped in the output (report? like ugly filenames?). Control characters (except tabs) are always escaped, regardless of this option's setting.

    Output a literal backslash (\) and a hash (#), and 3 octal digits. For example, a newline would output as "\#012". A literal backslash that is in a filename is not escaped unless it is followed by a hash and 3 digits (0-9).

    -m
    --prune-empty-dirs
    receiving rsync removes empty directories from the file-list, including nested directories that have no non-directory children.
    Useful for avoiding the creation of a bunch of useless directories when the sending rsync is recursively scanning a hierarchy of files using include/exclude/filter rules.

    the use of transfer rules, such as the --min-size option, does not affect what goes into the file list, and thus does not leave directories empty, even if none of the files in a directory match the transfer rule.

    Because the file-list is actually being pruned, also affects what directories get deleted when a delete is active. However, keep in mind that excluded files and directories can prevent existing items from being deleted due to an exclude both hiding source files and protecting destination files. See the perishable filter-rule option for how to avoid this.

    prevent the pruning of certain empty directories from the file-list by using a global "protect" filter.
    For instance, this option would ensure that the directory "emptydir" was kept in the file-list:

    --filter 'protect emptydir/'

    Here's an example that copies all .pdf files in a hierarchy, only creating the necessary destination directories to hold the .pdf files, and ensures that any superfluous files and directories in the destination are removed (note the hide filter of non-directories being used instead of an exclude): rsync -avm --del --include='*.pdf' -f 'hide,! */' src/ dest TO retain superfluous destination files, the "--include='*/' --exclude='*'"

    --write-batch=FILE create a file that can later be applied to another identical destination with --read-batch . See the "BATCH MODE" section for details, and also the --only-write-batch option.
    --only-write-batch=file like --write-batch, except that no updates are made on the destination system when creating the batch allowing transporting changes to the destination system via some other means and then apply the changes via --read-batch.

    Write the batch directly to portable media: if this media fills before the end of the transfer, apply that partial transfer to the destination and repeat the process to get the rest of the changes (this will use a partially updated destination system while the multi-update cycle is happening).

    you only save bandwidth when pushing changes to a remote system because this allows the batched data to be diverted from the sender into the batch file without having to flow over the wire to the receiver (when pulling, the sender is remote, and thus can't write the batch).

    --read-batch=FILE Apply changes stored in , a file previously generated by --write-batch.
    If file is -, the batch data will be read from standard input. See the "BATCH MODE" section for details.
    --protocol=NUM Force an older protocol version to be used. This is useful for creating a batch file that is compatible with an older version of rsync. For instance, if rsync 2.6.4 is being used with --write-batch , but rsync 2.6.3 is what will be used to run the --read-batch option, you should use "--protocol=28" when creating the batch file to force the older protocol version to be used in the batch file (assuming you can't upgrade the rsync on the reading system).
    --iconv=convert_spec convert filenames between character sets
    Use "." for character-set via the locale setting.
    --iconv=local,remote For example: --iconv=utf8,iso88591. This order ensures that the option will stay the same whether pushing or pulling files.
    --no-iconv or a convert_spec of "-" to turn off conversion.
    Default is site-specific, and can be affected via $RSYNC_ICONV

    For a list of charset names local iconv library supports, run "iconv --list".

    The following list contain all the coded character …
      , 8859_1, 8859_2, 8859_3, 8859_4, 8859_5, 8859_6, 8859_7, 8859_8, 8859_9, 10646-1:1993, 10646-1:1993/UCS4,
      ANSI_X3.4-1968, ANSI_X3.4-1986, ANSI_X3.4, ANSI_X3.110-1983, ANSI_X3.110, ARABIC, ARABIC7, ARMSCII-8, ASCII, 
    …
     US-ASCII, US, UTF-7, UTF-8, UTF-16, UTF-16BE, UTF-16LE, UTF-32, UTF-32BE, UTF-32LE, UTF7, UTF8, …
    

    With --protect-args filenames for transfer on the command-line are translated. See --files-from .

    No conversion of names in filters (including include/exclude files) is done!.
    Specify extra include/exclude rules if there are filename differences on the two sides.

    With --iconv the daemon uses the charset specified in its "charset" configuration

    --info=flags fine-grained control over the information output . An individual flag name may be followed by a level number, with 0 meaning to silence that output, 1 being the default output level, and higher numbers increasing the output of that flag (for those that support higher levels). Use --info=help to see all the available flag names, what they output, and what flag names are added for each increase in the verbose level.
    Some examples: rsync -a --info=progress2 src/ dest/ rsync -avv --info=stats2,misc1,flist0 src/ dest/ Note that --info=name's output is affected by the --out-format and --itemize-changes (-i) options. See those options for more information on what is output and when.
    --debug=flags fine-grained control over the debug output. An individual flag name may be followed by a level number, with 0 meaning to silence that output, 1 being the default output level, and higher numbers increasing the output of that flag (for those that support higher levels).
    --debug=help displays flag names, what they output, and what flag names are added for each increase in the verbose level.
    Examples: rsync -avvv --debug=none src/ dest/ rsync -avA --del --debug=del2,acl src/ dest/

    some debug messages will only be output when --msgs2stderr is specified, especially those pertaining to I/O and buffer debugging. This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if one or more flags needed to be send to the server and the server was too old to understand them). See also the "max verbosity" caveat above when dealing with a daemon.

    --msgs2stderr All output to stderr rather than to send messages to the client side via the protocol (which normally outputs info messages via stdout). Intended for debugging in order to avoid changing the data sent via the protocol, since the extra protocol data can change what is being tested. The option does not affect the remote side of a transfer without using --remote-option -- e.g. -M--msgs2stderr. Daemon connection does not have a stderr channel to send messages back to the client side, so if you are doing any daemon-transfer debugging using this option, you should start up a daemon using --no-detach so that you can see the stderr output on the daemon side. This option has the side-effect of making stderr output get line-buffered so that the merging of the output of 3 programs happens in a more readable manner. -
    --checksum-seed=num 4 byte checksum seed is included in each block and file checksum calculation.
    By default the checksum seed is generated by the server and defaults to the current time() .
    used to set a specific checksum seed, for repeatable block and file checksums, or in the case where the user wants a more random checksum seed.
    Setting NUM to 0 use the default of time() .
    --help
    --version


    Daemon Options

    --daemon Run as a daemon. accessed using a client with host::module or rsync://host/module/.

    If standard input is a socket then rsync will assume that it is being run via inetd, otherwise
    it will detach from the current terminal and become a background daemon, reading rsyncd.conf on each connect made by a client. config

    "exported" directories (modules) must be specified in rsyncd.conf

    --address addr allows to specify a specific IP address (or hostname) to bind to.
    By default rsync will bind to the wildcard address when run as a daemon with --daemon . This makes virtual hosting possible in conjunction with --config . See "address" global in rsyncd.conf
    --bwlimit=kbps BandWith limit. for daemon sends.
    The client can specify a smaller value, but their requested value will be rounded down if they try to exceed it. See the client version of this option (above) for some extra details.
    --log-file=file over-ride setting in the config file.
    --log-file-format=FORMAT override the FORMAT in the config file.
    --config=file alternate config file. default /etc/rsyncd.conf
    When running over a remote shell and the remote user is not the super-user use rsyncd.conf in the current directory ($HOME).
    --no-detach nor become a background job. a service on Cygwin, When rsync is run under a debugger. No effect if rsync is run from inetd or sshd.
    --port=port Default: 873 is priviledged so used over 1000 for non root user.
    See port rsyncd.conf.1.html,
    --sockopts overrides the socket setting in rsyncd.conf
    -v
    --verbose
    increases the amount of information the daemon logs during its startup phase.
    After the client connects, the verbosity level will be controlled by the options the client used and the "max verbosity" setting in the module's config section.
    -4
    --ipv4
    -6
    --ipv6
    prefered transport when creating the incoming sockets
    -h
    --help --daemon,
    output a short help page describing the options available for starting daemon.
    --no-motd suppresses the Message-Of-The-Day (MOTD) text, and the list of modules that the daemon sends in response to the "rsync dahost::"


    The client can connect to a host using either:

    In daemon mode an exported directory is called a module which has specific permissions and restrictions.
    This allows control over which directories are transferable.

    To list the modules available
    > rsync hostname::

    To list the files in a module :
    > rsync hostname::module

    Example

     > rsync jupiter::rsync
    drwxr-xr-x         102 2010/11/19 18:40:23 001_PHOTOS
    drwxr-xr-x         238 2010/11/25 15:46:02 p8_2006 

    Once rsync starts it traps sigINT which means the usual ^c will not interrupt rsync.
                      Use ^\ which is sigQUIT or another screen to quit with

    kill -quit `/bin/ps | /usr/bin/grep " rsync" |/usr/bin/head -1|/usr/bin/cut -f2 -d" "`

    Example

    Backup files from a web server to the local system, updating files previously saved,
    Monitoring progress and error (saving a log to a file for later review). Uses the remote shell transport as indicated by the single colon (:) between the hostname and the directory. This means that rsync will login and start rsync at the other end. This precludes having the rsync daeom running.

     echo "++++++++++++++++++" >> rsync.err  # initalize rsync.err so tail is ok
     tail -f rsync.err &     # continue to show errors as rsync runs                         
     rsync --progress --archive --hard-links --compress --update     --stats               \
                      --log-file=`date +rsync_%y%m%d.log` --log-format='%t %o %B %l %M %f' \
           uname@webSite.com:public_html backupDir   2>> rsync.err 

    Expect to see

    uname@webSite.com password:
    Enter the password which will not show ( If the password shows rsync has exited see diagnose).

    Use ^Z to push the job into the backround.

    The progress of rsync will be displayed.

    2011/10/26 21:14:30 recv rwx------ 20480 2011/10/26-21:14:10 jupiter/downloads
    2011/10/26 21:14:30 recv rw------- 4520453 2011/09/23-13:00:56 jupiter/downloads/ACPIspec40a.pdf
         4520453 100%    2.70MB/s    0:00:01 (xfer#1, to-check=47541/88416)
    2011/10/26 21:14:31 recv rw-r--r-- 1364852 2011/10/18-10:18:34 jupiter/downloads/nuketheappstore.zip
         1364852 100%    1.42MB/s    0:00:00 (xfer#2, to-check=47246/88416)
    …
    
    When the transfer is complete
    Number of files: 88416
    Number of files transferred: 2
    Total file size: 5004066643 bytes
    Total transferred file size: 5885305 bytes
    Literal data: 5885305 bytes
    Matched data: 0 bytes
    File list size: 2710269
    File list generation time: 1.120 seconds
    File list transfer time: 0.000 seconds
    Total bytes sent: 70
    Total bytes received: 7497967
    
    sent 70 bytes  received 7497967 bytes  714098.76 bytes/sec
    total size is 5004066643  speedup is 667.38
    
    

     fg to bring the tail process to the foreground.
     then ^C to interrupt the tail.

    time rsync --verbose --progress gername@webtems.com:* . --xattrs --acls

    Symbolic links

    Symbolic links are directory entries that contain a string which is the name of the file they link to(referant), perhaps in another directory or even another filesystem.

    If the referant file is not included then the symlink might require the referent to be included.
    If the symlink refers to a directory on another filesystem the referent probably should not be included!

    During any given transfer some symlinks might need to be handled differently from others!

    default: not transferred, outputs: "skipping non-regular" file.
    --links are recreated with the same target on the destination
    --archive implies --links.
    --copy-links are "collapsed" by copying their referent.(requires additional space)

    safe and unsafe symbolic links:
    Symbolic links are unsafe if they are absolute symlinks (start with /), empty, or if they contain enough ".." components to ascend from the directory being copied.

    Example: web site mirror wishes to ensure that the module that is copied does not include symbolic links to /etc/passwd in the public section of the site.

    1. --copy-unsafe-links will cause any links to be copied as the file they point to on the destination.
    2. --safe-links will cause unsafe links to be omitted altogether.
      (specify --links for --safe-links be effective.)

    In order of precedence
    (from most likely to cause trouble, i.e. skip referent file,
     to most likely to cause excesive data)
    default ignored.
    --copy-links become files.
    safe symlinksunsafe symlinks
    --links
    --archive
    --copy-unsafe-links
    are duplicated become files
    --copy-unsafe-links skipedbecome files, noisily.
    --links --safe-links
    --archive
    duplicated skiped.
    --links
    --archive
    Duplicate all

    Problems

    The startup script(.profile, .bashrc, …) or remote shell is producing output. This could include motd (Message Of The Day), lastlog date as well as host fingerprint produced with the ssh -o VisualHostKey=yes

    To diagnose this problem run:

    /usr/bin/ssh uname@remotehost /bin/true > out.dat
    if [ -s out.dat ] ; then echo "-- should be size 0; here it is:";cat out.dat
    else                     echo "No output(good). Check the firewall rules";fi

    Shell startup scripts (ex: .cshrc, .profile) which produce output must be renamed temporarily.

    Filter patterns

    specify -vv to show why each file is included or excluded.

    Environment Variables

    CVSIGNORE supplements any ignore patterns in .cvsignore files. --cvs-exclude
    RSYNC_ICONV defaults --iconv
    RSYNC_RSHoverrides shell used as the transport . Command line options are permitted after the command name, as in -e.
    RSYNC_PROXY redirects client to use a web proxy when connecting, specify:hostname:port pair.
    RSYNC_PASSWORD runs authenticated connections without intervention.
    Does not supply a password to a remote shell transport such as ssh.
    USER or LOGNAME defaults username sent to daemon. default: nobody.
    HOMEdefault directory for .cvsignore.

    Examples

    > sudo  rsync --progress --archive --hard-links --compress \
                         meuname@host::sources mysrcs >>0_0rsync.log &
    
    > tail -f 0_0rsynd.log   # this shows output
    receiving file list ... 
    46108 files to consider
    …
    iPhoto/iPod Photo Cache/F50/T9226.ithmb
          671600 100%  636.14kB/s    0:00:01 (xfer#7321, to-check=7/46108)
    iPhotoHidden/iPhoto
          443076 100%    6.50MB/s    0:00:00 (xfer#7322, to-check=5/46108)
    lt/Originals/0104_BirdsAtHome/OurOwl_5063.AVI
        74781936 100%    9.78MB/s    0:00:07 (xfer#7323, to-check=0/46108)
    
    sent 161377 bytes  received 7744458181 bytes  7114946.77 bytes/sec
    total size is 25579631224  speedup is 3.30
    
    

    Installing

    Download install config and start rsync

    1. MS windows version from SourceForge
    Uncompress/utar it: tar -xvf rsync-3.0.7.tar.gz

    Now it's in rsync-3.0.7/


    rsync --daemon --help    
    rsync  version 3.0.7  protocol version 30
    Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
    Web site: http://rsync.samba.org/
    Capabilities:
        64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
        socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, no ACLs, xattrs, iconv, no symtimes
    
    rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
    are welcome to redistribute it under certain conditions.  See the GNU
    General Public Licence for details.
    
    Usage: rsync daemon [OPTION]...
    
         --address=ADDRESS       bind to the specified address (like when  the server has multiple addresses)
         --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
         --config=FILE           default: /etc/rsyncd.conf 
         --no-detach             do not detach from the parent
         --port=PORT             port number (Instead of 873 check firewall)
         --log-file=FILE         
         --log-file-format=FMT   
         --sockopts=OPTIONS      custom TCP options like:
     -v, --verbose               increase verbosity
     -4, --ipv4                  prefer 
     -6, --ipv6                  
    
    
    When not invoking rsync as a daemon, avoid daemon-specific options.

    FILES

    /etc/rsyncd.conf or rsyncd.conf

    Genuine documentation: Samba.org/rsync

    EXIT VALUES

           0      Success 
           1      Syntax or usage error 
           2      Protocol incompatibility 
           3      Errors selecting input/output files, dirs 
           4      Requested action not supported: an attempt was made  to  manipulate  64-bit files on a platform that cannot support them; or an
                  option was specified that is supported by the client and not  by the server.  
           5      Error starting client-server protocol 
           6      Daemon unable to append to log-file 
           10     Error in socket I/O 
           11     Error in file I/O 
           12     Error in rsync protocol data stream 
           13     Errors with program diagnostics 
           14     Error in IPC code 
           20     Received SIGUSR1 or SIGINT 
           21     Some error returned by waitpid() 
           22     Error allocating core memory buffers 
           23     Partial transfer due to error 
           24     Partial transfer due to vanished source files 
           25     The --max-delete limit stopped deletions 
           30     Timeout in data send/receive 
           35     Timeout waiting for daemon connection
    

    Error Messages

    rsync error: unexplained error (code 255) at /AppleInternal/Library/BuildRoots/810eba08-405a-11ed-86e9-6af958a02716/Library/Caches/com.apple.xbs/Sources/rsync/rsync/rsync.c(244) [sender=2.6.9]
    Not helpful since this refers to the source .

    ENVIRONMENT VARIABLES

    CVSIGNORE supplements any ignore patterns .cvsignore . See --cvs-exclude
    RSYNC_ICONV default --iconv setting
    RSYNC_RSH override the default shell used as the transport for rsync. Command line options are permitted after the command name, as in -e .
    RSYNC_PROXY client use a web proxy when connecting to a hostname:port
    RSYNC_PASSWORD allow running authenticated connections to an daemon without user intervention. This does not supply a password to a remote shell transport such as ssh; , consult the remote shell's documentation.
    USER or LOGNAME used to determine the default username sent to an rsync daemon. Default: nobody
    HOME location of user's default .cvsignore .

    FileS

    /etc/rsyncd.conf or rsyncd.conf

    See

    rsyncd.conf(5)

    Bugs/unexpected behavories

    Times are transferred as *nix time_t values

    When transferring to FAT filesystems rsync may re-sync unmodified files. See --modify-window .

    File permissions, devices, etc. are transferred as native numerical values
    see --delete

    Version

    As of 05/05/23 the current version is Rsync version 3.3.0pre1 released
    This man page is for 3.0.7 ; as of 11/22/17 the current version is 3.1.2

    Internal Options

    --server and --sender are used internally by rsync, and should never be typed by a user under normal circumstances. Some awareness of these options may be needed in certain scenarios, such as when setting up a login that can only run an rsync command. For instance, the support directory of the rsync distribution has an example script named rrsync (for restricted rsync) that can be used with a restricted ssh login.

    Go to the /http://rsync.samba.org/ for the Credits, Thanks and Author..

    Mailing lists.samba.org

    Interesting notes from : How Rsync Works A Practical Overview

    1. The sender creates the file list including pathnames, ownership, mode, permissions, size, modtime and checksum (If --checksum has been specified).
      The list is transmitted as it is created.
    2. both sides sort the list and files are then refered to by index.
    3. sender transfers idname tables for users and groups
    4. The receiver traverses the existing file directory, comparing files to the filelist, creating directorys and identifing entries that will not need to be transfered
    5. The receiver creates block checksums of files needing to be transfered and provides them to the sender
    6. The sender generates a "rolling checksum" of a file and determins if a block already exists on the receiver. This provides for dynamic block creation on the sender which is effective when updates insert/delete information into the file. Consider the case of the addition of 1 byte near the beginning of the file. Fixed block algorithm to determine that all blockd would need to be transfered. 31 Dec 2009 rsync(1)

      versions

      It is unfortunate that many hosting services do not maintain current versions of many programs.
      As of 10/20/21 the current version is 3.2.3 20-08-0776 protocol 3q1
      Mac os
      Montery 12.2 comes with 2.6.9 protocol version29 rsync version 3.0.9 protocol version 30 (10 years old, 7 releases behind) . Darwin Denniss-MBP.germans 15.6.0 Darwin Kernel Version 15.6.0: Jun 21 2018 El Captian & Montery!
      GoDaddy 
      rsync version 3.0.6 protocol version 30 (5/8/09 8 ½ years old) Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others. 64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints, socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, ACLs, xattrs, iconv, no symtimes
      
      
      midphase
      rsync  version 3.1.2  protocol version 31 
      Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
          64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
          socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, ACLs, xattrs, iconv, symtimes, preallocnew
      
      
      onion omega
      LEDE
      WRT
       rsync  version 3.1.2  protocol version 31
      Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
      Web site: http://rsync.samba.org/
      Capabilities:
          64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
          no socketpairs, hardlinks, symlinks, no IPv6, batchfiles, inplace, append, no ACLs, no xattrs, iconv, symtimes, prealloc
      
      
      

      user: It may be helpful to have Public key: log in without giving a password.

      A very complete "cronable" script script, ftp rename profiles, ftp restore profiles