InteGrade: Object-Oriented Grid Middleware v0.3 RC1

IG-Deployer: Taktuk powered version v0.1

Deployment Manual

Last updated: July 26th 2008.

For Further information see the original Installation manual at http://integrade.incubadora.fapesp.br/portal/files/docs/install.txt

-----------------------------------------------------------------------------
INTRODUCTION

IG-Deployer is automatic deployment tool for Integrade. It is based on a tool 
for deploying parallel remote executions of commands to a large set of remote 
nodes called Taktuk(http://taktuk.gforge.inria.fr/). For further information 
about IG-Deployer(http://integrade.incubadora.fapesp.br/portal/files/docs/nascimento-wscad.pdf).

This document describes the procedures for the deployment of InteGrade, using 
IG-deployer. It's intended to be used by users not familiar with InteGrade, 
not the technologies it leverages. For information about this  subject  or 
any  problems,  questions,  musings, or  suggestions regarding  InteGrade, 
please  contact  the InteGrade  support team  at 
integrade-support@incubadora.fapesp.br or consult the InteGrade portal (http://integrade.incubadora.fapesp.br).

-----------------------------------------------------------------------------
SETTING UP THE ENVIRONMENT

Integrade v0.3 RC1 has many dependencies, and currently only works in GNU/Linux
platform. Some of the dependencies are included in this version of IG-Deployer, 
but others are required as pre-requisites of this tool. For instance and ease 
of notation, we will split the machines that are part of the installation set 
into two groups, the spanning node, composed by the node where the ig-deployer 
will be installed, and the grid, that is the rest of the machines that will 
receive Integrade. Frequently, the spanning node will also be part of the grid.

In order to begin a proper deployment of Integrade, this set of packages must be 
available in all grid machines, with an special exception for PERL, the only 
dependency of the spanning node. This exception of dependencies allow the deployer 
to run hypothetically in any operational system with a perl implementation and ssh
client. The pre-requisites are:

- PERL v5.8.8 or greather.

- GCC  4.0  or greater  -  The GNU  C  Compiler.   
  Debian and Fedora packages: gcc-4.0,  gcc-4.0-base  

- G++  4.0 or  greater -  The GNU  C++ Compiler.   
  Debian  and Fedora package: g++-4.0
  
- GNU Make  - The popular  UNIX-based build tool.  
  Debian and Fedora package: make

- JDK 1.5  or greater  - The Java  Development Kit  v1.5.0, including
  both the  compiler and runtime  environment for the  Java Language.
  Available  at  http://java.sun.com/javase/downloads/index_jdk5.jsp

The deployer also requires SecureShell (ssh) connection without password in all 
machines as explained here 
(http://bashcurescancer.com/setting_up_ssh_keys_for_access_without_password.html)

It already contains the main integrade dependencies:

- Ant 1.5 or greater  - A Java-based platform-independent build tool.

- JacORB  2.2.3 -  A Java-based  open-source CORBA  ORB that  runs on
  cluster manager machines. 
  
- OiLPack (incl.  Lua-5.0-Plus, Compat-Lua-5.1-r5, LuaSocket-2.0, and
  OiL 0.3.1) - A software bundle containing the compiler  and runtime
  for the Lua scripting language,  a library  for  building Lua-based
  socket applications, and the  OiL lightweight CORBA ORB, which runs
  on resource  provider nodes.
  
Assume you have all this packages you can download and unpack the IG-deployer 
with the following steps:

$ wget http://integrade.incubadora.fapesp.br/portal/software/deployer.tar.gz
$ tar zxvf ig-deployer.tar.gz

or optionally checkout from SVN:

$ svn co https://incubadora.fapesp.br/svn/integrade/trunk/deployer

To see if everything worked well just type:

$ cd deployer
$ ./deployer check

You should see:

Welcome to ig-deployer v0.1. Powered by TakTuk.
default.yml > GRM ubuntu of ubuntu in /usr/local/ig with java /usr/lib/jvm/java-6-sun/
-----------------------------------------------------------------------------
KNOWN ISSUES

- If the deps folder has the same path as the users home directory, the installation
results in a problem

- Ig-Deployer does not return the corrects returns values in many cases

- The target "start" blocks its execution, it starts the integrade but it never knows 
that the start processes finishes.

- Sometimes the main service, the GRM, starts after some LRM, this is a problem in a set of large nodes because some machines will not be able to register to their GRM, and will not be part of the grid. This only happens if the set of nodes are larger than 1.

- Openssl Dependency is not being compiled nor properly linked.



-----------------------------------------------------------------------------
YAML FILES (default.yml)

The syntax of the configuration files are very simple. It based on a human 
friendly data serialization standar called YAML (YAML Ain't  Markup Language) 
available at (http://www.yaml.org/). There are 5 things that must be specified 
on the configuration file.

GRM: The node where the Global Resource Manager must be installed. The grm 
may be seen as the Integrade server, and one special machine must be chosen to be GRM.

PATH: The default path of installation for all machines. 

NFS: The path for the NFS installation. If this is null the dependencies will 
be installed in every single machine, if a path is given, it will assume that 
all machines have access to the dependency folder and will only install on the GRM.

JAVA: The path where the java dependency where installed.

MACHINES: A list of machines, ending with "two dots".


An example should be:

---
grm: ubuntu
shared: /global/integrade
path: /usr/local/ig
java: /usr/local/jvm
ubuntu:
suse:
fedora:

With this configuration, Integrade will be installed in 3 machines, ubuntu, 
suse and fedora. All the machines will have Integrade at /usr/local/ig 
(the folder must exists before installation and must have write 
permissions for the user running ig-deployer). All the dependencies will 
be installed in ubuntu, at /global/integrade and it's assumed that all the 
other machines will have access to that folder. It will also assume that 
/usr/lib/jvm/java-6-sun/ contains the path for the Java Virtual Machine. Note
that you don't need the shared information if you're going to replicate dependencies
into all target nodes or if you want to install only into one machine.

-----------------------------------------------------------------------------
COMMANDS

The syntax of ig-deployer is the following: 

deployer COMMAND [TOPOLOGY]

Where COMMAND is mandatory and is one of the commands introduced later in this 
document, TOPOLOGY stands for the configuration file named TOPOLOGY.yml, 
notice that the topology name must be given without the .yml extension, and 
if you supply no topology name, the default.yml topology will be assumed.

-----------------------------------------------------------------------------
"ALL" COMMAND

The all command runs the sequence of commands CHECK, PING, CLEAN, PACK, DEPLOY, INSTALL, START.

-----------------------------------------------------------------------------
"CHECK" COMMAND
   This command performs a test of syntax against the given YAML file, to 
   guarantee that file represents a valid topology. A valid topology must  
   include a GRM, at least one LRM, a path to install the files and a java path.
   
-----------------------------------------------------------------------------
"PACK" COMMAND
   The command pack searches and packs two folders. The first one is Integrade, 
   that must be at ../integrade. The second one is the dependency package that is 
   inside the folder packs/dependency. 
   After this step two files will be generated in /pack folder, the deps.tar.gz 
   and the deploy.tar.gz. If one of this files exists at the time of command 
   invocation, the respective file is not created.
   
-----------------------------------------------------------------------------
"DEPLOY" COMMAND
   Sends the files deploy.tar.gz and deps.tar.gz for remote hosts.
-----------------------------------------------------------------------------
"INSTALL" COMMAND
   Unpack and build all the dependencies. The result of this command is a 
   complete installation of Integrade. It depends of previous executions of 
   the commands PACK and DEPLOY.
   
-----------------------------------------------------------------------------
"START" COMMAND
   Stop all services, start the grid GRM and them all the clients.
-----------------------------------------------------------------------------
"STOP" COMMAND
   Stop all services.
-----------------------------------------------------------------------------
"PING" COMMAND
   Try to connect in all hosts in order to verify existence and functionality of them.
-----------------------------------------------------------------------------
"CLEAN" COMMAND
   Erases local deployment files deps.tar.gz and deploy.tar.gz. It also removes remote 
   files needed for installation.
-----------------------------------------------------------------------------
"UNINSTALL" COMMAND
   Removes InteGrade from the entire grid.
-----------------------------------------------------------------------------
