###############################################################################
#                                                                             #
#  TakTuk, a middleware for adaptive large scale parallel remote executions   #
#  deployment. Perl implementation, copyright(C) 2006 Guillaume Huard.        #
#                                                                             #
#  This program is free software; you can redistribute it and/or modify       #
#  it under the terms of the GNU General Public License as published by       #
#  the Free Software Foundation; either version 2 of the License, or          #
#  (at your option) any later version.                                        #
#                                                                             #
#  This program is distributed in the hope that it will be useful,            #
#  but WITHOUT ANY WARRANTY; without even the implied warranty of             #
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the              #
#  GNU General Public License for more details.                               #
#                                                                             #
#  You should have received a copy of the GNU General Public License          #
#  along with this program; if not, write to the Free Software                #
#  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA #
#                                                                             #
#  Contact: Guillaume.Huard@imag.fr                                           #
#           ENSIMAG - Laboratoire ID                                          #
#           51 avenue Jean Kuntzmann                                          #
#           38330 Montbonnot Saint Martin                                     #
#                                                                             #
###############################################################################

=pod TakTuk communication layer interface documentation

=begin html

<center><h1>USER MANUAL</h1></center>

=end html

=head1 NAME

TakTukComm - Interface to C<taktuk(1)> communication facilities

=head1 SYNOPSIS

B<Perl interface:>

  $taktuk::error;
  taktuk::error_msg($);

  taktuk::get($);

  taktuk::send(%);
  taktuk::recv(%);

B<C interface:>

  #include <taktuk.h>

  const char *taktuk_error_msg(int msg_code);

  int taktuk_init_threads();
  int taktuk_leave_threads();

  int taktuk_get(char *field, long *value);

  int taktuk_multi_send(const char *dest, const void *buffer,
                                                         size_t length);
  int taktuk_multi_sendv(const char *dest, const struct iovec *iov,
                                                            int iovcnt);

  int taktuk_send(long dest, const void *buffer, size_t length);
  int taktuk_sendv(long dest, const struct iovec *iov, int iovcnt);

  int taktuk_recv(long *from, void *buffer, size_t *length, int timeout);
  int taktuk_recvv(long *from, const struct iovec *iov, int iovcnt,
                                                           int timeout);

  int taktuk_wait_message(long *from, size_t *size, int timeout);

  int taktuk_read ( void *buffer, size_t length );
  int taktuk_readv( const struct iovec *iov, int iovcnt );

=head1 DESCRIPTION

The B<TakTuk> communication layer interfaces provide a way for programs
executed using the C<taktuk(1)> to exchange data. It is based on a simple
send/receive model using multicast-like sends and optionally timeouted
receives.  This is only designed to be a control facility, in
particular this is not a high performance communication library. This interface
is available both for the Perl and the C langages.

WARNING: the B<TakTuk> communication interface is not process-safe : it is
probably a very bad idea to use point-to-point communication in
more than one process related to a single logical peer (these processes might
be several local commands or forked processes). Nevertheless, the C interface
is thread-safe (it uses posix mutexes in its critical sections). These issues
are likely to be addressed in future B<TakTuk> versions.

=head1 PERL INTERFACE

The Perl communication interface for B<TakTuk> is made of two functions that
can be called by scripts executed using the C<taktuk_perl> command of
B<TakTuk> or using the B<TakTuk> package provided with the B<TakTuk>
distribution.
These two functions are:

=over

=item B<taktuk::get($);>

gets some information from B<TakTuk>. Currently available informations are
'rank' and 'count'. This is a better way to get these informations than
environment variables as its takes into account renumbering that might occur
after process spawn.

=item B<taktuk::send(%);>

sends a scalar to a single peer or a set specification (see C<taktuk(1)> for
informations about set specifications).
The two mandatory fields in the arguments are C<to> (with a set specification)
and C<body>. Returns an undefined value upon error.

=item B<taktuk::recv(%);>

blocks until the reception of a message. Returns a list of three elements: the
logical number of the destination of the message, the logical number of its
source and the message itself.
Accepts an optional C<timeout> argument with a numeric value.
Returns an empty list upon error.

=back

When an error occur, both of these functions set the variable C<$taktuk::error>
to the numeric code of the error that occured. A textual description of the
error is provided by the function C<taktuk::error_msg($)> that takes the error
code as an argument.

Error codes are the following :

=over

=item taktuk::ESWRIT

a call to C<taktuk::syswrite> failed. This is due to a C<syswrite> error
different than C<EAGAIN>. The code should be accessible using C<errno>.

=item taktuk::EFCLSD

the communication channel to the B<TakTuk> engine has been closed. This
typically occur when shutting down the logical network (using Ctrl-C on root
node for instance).

=item taktuk::ESREAD (C<taktuk::recv> only)

a call to C<sysread> failed (the code should be accessible using C<errno>).

=item taktuk::EARGTO (C<taktuk::send> only)

C<to> field missing in the arguments.

=item taktuk::EARGBD (C<taktuk::send> only)

C<body> field missing in the arguments.

=item taktuk::ETMOUT (C<taktuk::recv> only)

The call to C<taktuk::recv> timeouted. This only occur when giving a C<timeout>
field as C<taktuk::recv> argument.

=item taktuk::EINVST (C<taktuk::send> only)

The set specification given as a destination to the C<taktuk::send> function is
not correct.

=back

=head1 C INTERFACE

Any program using B<TakTuk> C communication interface has to link his program
to the C<taktuk> and C<pthread> libraries (the C<taktuk> library is provided
with the distribution).
Headers of communication functions and constants definitions can be found in
C<taktuk.h> also provided with the distribution.

The communication functions are:

B<miscellaneous functions>

=over

=item int taktuk_init_threadsZ<>();

=item int taktuk_leave_threadsZ<>();

functions to be called once in the process respectively before threads creation
and after threads destruction if you want B<TakTuk> C interface to be
thread-safe.

=item int taktuk_get(char *field, long *result);

gets some information from B<TakTuk> and places it into result. Currently
available informations are "rank" and "count". This is a better way to get
these informations than environment variables as its takes into account
renumbering that might occur after process spawn.

=back

B<multicast send functions>

=over

=item int taktuk_multi_send(const char *dest, const void *buffer, size_t length);

=item int taktuk_multi_sendv(const char *dest, const struct iovec *iov, int iovcnt);

C<taktuk_multi_send> sends the content of C<buffer> made of C<length> bytes to
the set of target destinations C<dest> (nul terminated characters string, see
C<taktuk(1)> for informations about set specifications).
C<taktuk_multi_sendv> is the vector variant of C<taktuk_multi_send> (similar to
C<writev> system function).

=back

B<single host send/recv functions>

=over

=item int taktuk_send(long dest, const void *buffer, size_t length);

=item int taktuk_sendv(long dest, const struct iovec *iov, int iovcnt);

sends the content of C<buffer> made of C<length> bytes to C<dest>.
C<taktuk_sendv> is the vector variant of C<taktuk_send> (similar to
C<writev> system function).

=item int taktuk_recv(long *from, void *buffer, size_t *length, int timeout);

=item int taktuk_recvv(long *from, const struct iovec *iov, int iovcnt, int timeout);

blocks until the reception of a message.  If a regular message is received,
C<taktuk_recv> sets the value of its arguments: C<from> to the logical number
of the sender, C<buffer> to the data received wich is made of
C<length> bytes.  If timeout is non nul, C<taktuk_recv()> might receive a
timeout notification.  In this case, C<taktuk_recv()> returns the TAKTUK_ETMOUT
error code.
C<taktuk_recvv> is the vector variant of C<taktuk_recv> (similar to
C<readv> system function).

WARNING: the buffer size should be sufficient to receive all the data of the
matching send. If this is not the case, the program is likely to end up
abruptly with a segmentation fault.

=back

B<low-level recv functions>

=over

=item int taktuk_wait_message(long* from, size_t *size, int timeout);

=item int taktuk_read (void* buffer, size_t length);

=item int taktuk_readv(const struct iovec *iov, int iovcnt);

C<taktuk_wait_message> waits for a taktuk message to be available and sets
C<from>  to the logical number of the sender and C<size> to the size of the
received message.
It must be followed by a call to either C<taktuk_read> or C<taktuk_readv> to
read the data (using the size given by C<taktuk_wait_message>).
As other B<TakTuk> receive functions, this function might return the
TAKTUK_ETMOUT error code if C<timeout> is not nul and expires before the
reception.

If you don't know in advance the size of the data being sent to you, you can
use these lower level functions. Actually, C<taktuk_recv> is equivalent to a
call to C<taktuk_wait_message> followed by errors checks on buffer size and a
call to C<taktuk_read>. This is the same for C<taktuk_recvv>.

=back

When an error occur, all of these functions return one of the following numeric
error code.  A textual description of the error is provided by the function
C<taktuk_error_msg()> that takes the error code as an argument.

Error codes are the following :

=over

=item TAKTUK_ESWRIT

a call to C<write(2)> failed. The code should be accessible using C<errno>.

=item TAKTUK_EFCLSD

the communication channel to the B<TakTuk> engine has been closed. This
typically occur when shutting down the logical network (using Ctrl-C on root
node for instance).

=item TAKTUK_ESREAD (receives only)

a call to C<read(2)> failed. The code should be accessible using C<errno>.

=item TAKTUK_ETMOUT (receives only)

The call to C<taktuk_recv()> (or its vectorized equivalent) or to
C<taktuk_wait_message> timeouted.  This only occur when giving a non nul
C<timeout> value to these functions.

=item TAKTUK_EALLOC

An internal memory allocation failure occured.

=item TAKTUK_EIBUFF

A buffer of incorrect size has been given to store all the data read by a
receive function.

=item TAKTUK_ENOCON

The connector communication channels have not been found. This typically occur
when launching a B<TakTuk> program without using B<TakTuk>.

=item TAKTUK_EINVAL (get only)

The field given to C<taktuk_get> is not a valid B<TakTuk> field.

=item TAKTUK_EMTXNM (init threads only)

No memory to allocate a new mutex

=item TAKTUK_EMTXAG (init threads only)

Resources temporarily unavailable for new mutex allocation.

=back

Other error codes are internal B<TakTuk> errors which should strongly suggest a
bug in B<TakTuk>. They have also a textual description that is returned by
C<taktuk_error_msg>.

=head1 SEE ALSO

C<tatkuk(1)>

=head1 AUTHOR

The original concept of B<TakTuk> has been proposed by Cyrille Martin in his PhD thesis. People involved in this work include Jacques Briat, Olivier Richard, Thierry Gautier and Guillaume Huard.

The author of the version 3 (perl version) and current maintainer of the package is Guillaume Huard.

=head1 COPYRIGHT

The C<taktukcomm> communication interface library is provided under the terms
of the GNU General Public License version 2 or later.

=cut

