Nearly two years passed and there are still people out there who don't know 'ControlMaster' which was introduced in
OpenSSH 4.0.
ControlMaster is set to no by default.
To quote man 5 ssh_config:
Enables the sharing of multiple sessions over a single network
connection. When set to ``yes'' ssh will listen for connections
on a control socket specified using the ControlPath argument.
Additional sessions can connect to this socket using the same
ControlPath with ControlMaster set to ``no'' (the default).
These sessions will try to reuse the master instance's network
connection rather than initiating new ones, but will fall back to
connecting normally if the control socket does not exist, or is
not listening.
Setting this to ``ask'' will cause ssh to listen for control con-
nections, but require confirmation using the SSH_ASKPASS program
before they are accepted (see ssh-add(1) for details). If the
ControlPath can not be opened, ssh will continue without connecting
to a master instance.
This is pretty useful. Set ControlMaster to 'auto' (or autoask) if you want to use an existing
socket, create a new one if you can't connect to an existing socket or there is none existing.
This has the nice (depends on how paranoit you are) side effect that you gain speed and if you login via password
on a server, you don't have to type your password every while you login because you can use
an already authenticated socket.
cat >> ~/.ssh/config << EOF
Host *
ControlPath ~/.ssh/master-%r@%h:%p
ControlMaster auto
EOF
Be sure to check the permissions of the ControlPath so noone else can use your existing socket.