Professional Documents
Culture Documents
Mendel Linux Howto
Mendel Linux Howto
MENDEL Software
The Linux source distribution for Mendels Accountant can be downloaded from:
http://sourceforge.net/projects/mendelsaccount . To install the software, as root user run the
following commands:
# tar xvfz mendel_v1.4.7.tar.gz
# cd mendel_v1.4.7
# ./configure
# make
# make install
Note, in the above statements, replace mendel_v1.4.7.tar.gz and mendel_v1.4.7 with the
specific name of the Linux version you downloaded. The configure script will check to make
sure all the needed software requirements are installed on your system and are compatible
with Mendels Accountant. It will also prompt the user for where the files should be installed,
and how much memory should be allocated for Mendel. By default, the configure script will
configure Mendels Accountant to run using (a) the C engine (which is slower than the
Fortran version), (b) without multi-tribe support, and (c) without a queuing system (which
will allow a Linux user or users to submit as many jobs as they like, and will queue the jobs
and schedule them when resources are available). The only third-party software package that
is required for the default install of Mendels Account is Gnuplot. To be more specific, the
only third-party software that does not come pre-installed on typical Linux systems e.g. both
Apache web server and Perl is also required but typically is provided with most Linux
distributions. In case the user wants to run with multi-tribe support, or use it in conjuction
with a queuing system, the following instructions are provided as a reference manual.
TORQUE
Intel Fortran
Compiler
Supported
version
>= 4.2.3
>= 2.0
>= 5.8.8
>= 1.0.5
Packaged with
Fedora
Yes
Yes
Yes
No
1.2
>= 8.0
No
No
Website
http://gnuplot.info
http://apache.org
http://perl.org
www.mcs.anl.gov/research/proje
cts/mpich2
http://ClusterResources.com
http://software.intel.com/enus/intel-compilers/
In addition, if using Linux as a MENDEL client (i.e. run the software on a Linux
system), the Konqueror web browser must be used (Firefox will not work correctly).
Install the Konqueror web browser, by installing kdebase package.
Gnuplot
Gnuplot 4.2 must be installed with support for PNG. The easiest way to install Gnpulot is by
using yum as root:
# yum install gnuplot
If yum is not installed, the source code can be download from http://gnuplot.info. Then
compile and install. After installation, please test that it is indeed 4.2 by using the following
command:
# gnuplot --version
Pre 4.2 versions will encounter some errors when plotting MENDEL results. Also to be able
to create the needed PNG files for the plots, Gnuplot 4.2 and later uses a library libgd
(www.libgd.org), which in turn uses a library called libpng (http://sf.net/projects/libpng).
Therefore, if gnuplot is compiled manually (instead of using rpm or yum), both libgd and
libpng must first be installed. The proper order of installing these packages should be as
follows:
1. first install libpng from http://sourceforge.net/projects/libpng (click Download,
libpng-stable,
then
libpng-1.2.37.tar.gz)
or
try
the
direct
link:
http://internap.dl.sourceforge.net/sourceforge/libpng/libpng-1.2.37.tar.gz
2. Then, install libgd (www.libgd.org) -- click Downloads, then click .tar.gz under
"Download the latest" or try the direct link: http://www.libgd.org/releases/gd2.0.35.tar.gz
3. After libgd is installed, make gnuplot in the following manner:
a. # cd gnuplot-4.2.3
b. # ./configure --with-png=/usr/local/lib
c. # make
d. # make install
Perl
Perl is usually installed at the time when Linux is installed. You can test if it is already
installed by typing:
# perl --version
If it is not installed, you can install one of three ways, listed from easiest to most difficult:
1. yum install perl
2. rpm i perl-5.8.8-10.i386.rpm (using from Linux installation CD)
3. downloading from perl.org and compiling
Also, for the ability to import case parameters from other servers, perl-libwww must be
installed. This can be installed using the following command:
# yum install perl-libwww-perl
MPICH
MPICH stands for Message Passing Interface Chameleon. The latest version, MPICH2, can
be downloaded from http://www.mcs.anl.gov/research/projects/mpich2/. Download the platform
labeled Source Code (1.0.8p1) mpich2-1.0.8p1.tar.gz. To install type the following as root:
# tar xvfz mpich2-1.0.8p1.tar.gz
# cd mpich2-1.0.8p1
# ./configure
# make
# make install
There are more instructions below for configuring MPICH.
automatically check to see if MPICH2 has already been installed, and will check to make sure
that it is setup correctly.
system)
software
and
is
also
freely
available
for
download
via
first download and install the Torque package according to the instructions which follow, then
run ./configure --with-torque, then make, then make install.
Installing Torque. After downloading torque, unpack it as root user:
# cd /usr/local/src
# tar xvfz /mydir//torque.tgz
# cd /usr/local/src/torque
# ./configure --with-default-server=localhost
Note:
If
just
standalone
machine
(not
cluster),
localhost
This will compile Torque for your specific system, as well as install some necessary programs
for schedule management such as qmgr, qsub, qstat, pbsnodes, pbs_mom,
which are all called via MENDELs perl modules. The torque.setup script will create a
default queue called batch. These Torque programs are briefly described here:
qsub - program used for submitting new MENDEL jobs to the queue
pbsnodes used to control status of compute nodes. Pbsnodes will mark the nodes
as either free, down, or offline.
On some systems, the user apache as executed through the web server will not be able to find
the default web server in /var/spool/torque/server_name. To get around this
problem, the following line should be added to the Apache configuration file
/etc/httpd/conf/httpd.conf:
SetEnv PBS_DEFAULT my.server.edu
After modifying httpd.conf, the Apache web server must be restarted by either of the
following commands: sytem-config-services or chkconfig httpd restart.
After setting up, you should also add the file /var/spool/torque/mom_priv/config to contain
the following contents. For example, it should look like the following for the PBS server
named hs84 with a compute node named localhost:
$pbsserver
hs84
pbs_server
$logevent
255
$clienthost
localhost
/etc/hosts file).
NFS Mounting
If you are running on a cluster computer with separate nodes networked together, all the
nodes must share a common directory for storing the MENDEL data files. Since apache only
has write permissions under /var/www/html, it is logical to export this directory via NFS.
In order to export this file, the superuser (or root user) must modify the /etc/exports
file to include the following line:
/var/www
10.0.0.0/255.255.255.0(rw,sync,no_root_squash)
/home
10.0.0.0/255.255.255.0(rw,sync,no_root_squash)
/usr/local
10.0.0.0/255.255.255.0(sync)
The 10.0.0.0 must be changed to the match the local subnetwork you are using, as defined in
the file /etc/hosts. For some systems, this number may be 192.168.*.0 where * represents a
number anywhere between 0 and 255. For more information about exporting file systems via
NFS, type man exportfs. Once this file has been modified, the root user must run the
command exportfs a to update the NFS system. Also, one must make sure the NFS
service is running. This can be started several ways. The easiest way, if running Fedora
Linux, is by executing the following program as root user: system-config-services.
Select NFS and click start, and make sure to click Save, so that NFS will automatically be
started everytime the system is booted. Alternatively, this NFS can be started via the
following command:
# chkconfig nfs --levels 345 on
Exporting the directories from the head node is only half the battle. The next step is to
include instructions for mounting the exported directory on each of the compute nodes. This
is accomplished by adding the following line to /etc/fstab for each of the compute
nodes:
192.168.1.1:/var/www (ro,sync)
Before this directory can be mounted, /var/www must be created and completely empty on
every compute node. If /var/www already exists on the compute nodes, it can be renamed
to /var/www.local and a new /var/www created via mkdir /var/www.
Security
One thing that will cause much frustration in getting things working on the server is Linuxs
internal security firewall.
following ports must be opened: SSH, WWW (HTTP), NFS. It is recommended during the
setup process, that the firewall be disabled until once everything is setup, then re-enabled and
tested to make sure everything still works with the firewall. The easiest way to manage open
these ports and enable/disable the firewall is by using the program: system-configsecuritylevel in Fedora.
Once the changes are made and saved, the new settings are
Setting up rsh-server
MPI uses remote shell (rsh) to communicate between nodes. Since rsh in the past has been
used by many hackers in the past to break into Unix/Linux systems, Linux systems such has
Fedora have made it not trivial to setup an rsh-server. Included here are the basic instructions
for setting up the rsh-server. The rsh-server is handled by xinetd (if xinetd has not been
installed it must be installed) which only launches the programs when they are needed (rather
than having them running constantly in the background). Note, this must be setup and
running on each compute node.
# yum install xinetd
# yum install rsh-server
# cd /etc/xinetd.d
The following files must be modified: rexec, rsh, rlogin, kexec, klogin, ksh. For each of these
files one line must be changed in the following manner:
# disabled = yes
Once these changes are made and saved, xinetd must be restarted using the following
command:
# /etc/init.d/xinetd restart
Note, this rsh-server and the corresponding instructions above must be executed on every
compute server in the cluster on the system.
# cd /var/www
# echo MPD_SECRETWORD=nr76-k2x > .mpd.conf
# chown apache.apache .mpd.conf
# chmod 600 .mpd.conf
The MPD_SECRETWORD should be a unique password. This identical file (either by NFS
mounting /var/www or copying the file to each node) should exist in /var/www on all nodes
of the cluster.
For MPICH2, a ring of MPD (multi-purpose daemons) must be started for user apache. Doing
this requires allowing user apache to login with shell access.
On Ubuntu/Debian systems, Apache uses the user www-data, which should already be
setup on your system. You may check /etc/passwd to verify that the following line should
exists:
www-data:x:33:33:www-data:/var/www:/bin/sh
On Fedora/CentOS/Redhat systems,the file /etc/passwd must be modified by root user on
every node and the following line should be changed from:
apache:x:48:48:Apache:/var/www:/sbin/nologin
to:
apache:x:48:48:Apache:/var/www:/bin/bash
Then, for security reasons, a password should be given to apache. This is accomplished by
typing the following command as root user:
# passwd apache
Modifying mpdlib.py
A modification must be made to the file mpdlib.py (by default this is installed in
/usr/local/bin) to allow user apache to start use MPD. This file is modified by copying the
following two lines:
elif hasattr(os,'getuid')
and
os.getuid() == 0:
# if ROOT
parmsRCFilename = os.path.abspath('/etc/mpd.conf')
and pasting them just after the previous line, and making the following modifications:
elif
hasattr(os,'getuid')
and
os.getuid()
==
48:
if
apache
parmsRCFilename = os.path.abspath('/var/www/.mpd.conf')
If you are using a Debian/Ubuntu system, instead of using uid 48 above, you will need to use
33, the uid of the user www-data. You may want to check /etc/passwd, and double check
that www-data is using a uid of 33. Try running id www-data to get the uid of user wwwdata.
If the machine is not a cluster, but a single node (standalone server), one may simply type:
# mpdboot
where mpd.hosts (which needs to be created by the root user) is very similar to
/var/spool/torque/server_priv/nodes, containing a list of nodes you want to start (but without
np= statements). For example:
localhost
c01
c02
c03
The process of starting this ring of daemons is usually not trivial and may be the source of
much frustration. mpdboot assumes all the pbs_moms are running on all the compute nodes,
no nodes are down, and communication is working properly between all nodes. If it works
correctly, it should not give any output or error messages when executing the mpdboot
command. Moreover, this MPD ring can be verified by typing: mpdtrace which should output
a list of nodes in the ring. This is the easiest way to start an MPD ring. Alternatively, an
MPD ring can be started by executing an mpd command on each node separately. See
mpd --help" or mpdhelp for more instructions.
This last line should then be copied into the file /var/www/html/mendel/.htpasswd, which
only contains several of these lines with usernames, and encrypted passwords for different
users.
In order for this file to work, the AllowOverride option for the /var/www/html directory
(in the file /etc/httpd/conf/httpd.conf) must be set to All is shown below:
<Directory "/var/www/html">
Options Indexes Includes FollowSymLinks
#AllowOverride None
AllowOverride All
Allow from all
Order allow,deny
</Directory>
Once changes are made to httpd.conf, the Apache web server must be restarted. This can be
When testing whether the new security is going to work in a browser, if the website has
already been accessed via the browser, the web browser must be exited, and then restarted.
Troubleshooting
Note: Security-Enhanced Linux (SELinux) may prevent MENDEL operating correctly. There
should be a way to adjust the fine-grained conrols to allow MENDEL to work properly. If
you are using a Fedora sytem, you can run the command system-config-securitylevel. The
easy fix is to change the state of SELinux from Enforcing to Permissive mode.
I get 'Unauthorized Request' to every qmgr command:
The $PBS_HOME/server_name needs to be the same name as that associated with your
primary interface.
network interface (e.g. eth0 or eth1). Then make sure /etc/hosts file has the entry you are
specifying as PBS_SERVER after that IP address.
I get the error message qsub: Bad UID for job execution:
This message may have to do with the fact that pbs_mom is not correctly running or that
the node is down. You can run pbsnodes command and see if the node is down. You can
manually change the state of the node by starting qmgr and typing at the qmgr prompt
set node localhost state = free (assuming the compute node is named localhost).
localhost
localhost.localdomain
make sure pbs_server and pbs_sched are running on the head node, and
pbs_mom is running on each compute node
Also, check the run log /var/log/messages to see if there are any error
messages why the node went offline.
Also, if you are using a firewall, make sure that ports 15,000 15,004 are open.
Please
visit
http://www.clusterresources.com/pages/resources/documentation/common-
APPENDIX
/etc/rc.d/init.d/pbs_server
#!/bin/sh
#
# pbsserv: This script will start and stop the PBS server
#
# description: PBS is a batch versitle batch system for SMPs and
clusters
#
### BEGIN INIT INFO
# Provides: pbsserv
# Required-Start: $network
$remote_fs
# Required-Stop: $network
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Description: Start PBS server
### END INIT INFO
case "$1" in
start)
echo -n "Starting PBS Server"
/usr/local/sbin/pbs_server
;;
stop)
echo -n "Stopping PBS Server"
/usr/local/bin/qterm -t quick
;;
status)
echo -n "Checking for PBS server: "
checkproc /usr/local/sbin/pbs_server
;;
try-restart)
$0 status >/dev/null &&
$0 restart
;;
restart)
echo "Restarting PBS server"
$0 stop
$0 start
;;
*)
echo "Usage: $0 {start|stop|status|restart|reload}"
exit 1
esac
/etc/rc.d/init.d/pbs_sched
#!/bin/sh
#
# pbssched: This script will start and stop the PBS Scheduler
#
# description: PBS is a batch versitle batch system for SMPs and
clusters
#
### BEGIN INIT INFO
# Provides: pbssched
# Required-Start: $network pbsserv $remote_fs
# Required-Stop: $network
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Description: Start PBS Sceduler
### END INIT INFO
case "$1" in
start)
echo -n "Starting the PBS Scheduler"
/usr/local/sbin/pbs_sched
;;
stop)
echo -n "Stopping the PBS Scheduler"
killall pbs_sched
;;
status)
echo -n "Checking for PBS Scheduler: "
checkproc /usr/local/sbin/pbs_sched
;;
try-restart)
## Stop the service and if this succeeds (i.e. the
## service was running before), start it again.
$0 status >/dev/null &&
$0 restart
/etc/rc.d/init.d/pbs_mom
#!/bin/sh
#
# pbsmom: This script will start and stop the PBS mom
#
# description: PBS is a batch versitle batch system for SMPs and
clusters
#
$remote_fs
# Required-Stop: $network
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Description: Start PMB MOM
### END INIT INFO
case "$1" in
start)
echo -n "Starting PBS Mom"
/usr/local/sbin/pbs_mom
;;
stop)
echo -n "Stopping PBS Mom"
killall pbs_mom
;;
try-restart)
$0 status >/dev/null &&
$0 restart
$0 stop
$0 start
rc_status
;;
*)
echo "Usage: $0 {start|stop|status|restart|try-restart}"
exit 1
esac