infrastructure: first cut at updates for git
This is a first cut at updating various infrastructure for git. There will definitely be more commits; some of the scripts require committed/pushed code (e.g., the various make-tarball scripts). So it's not possible to know if we got it right without committing/pushing.
Этот коммит содержится в:
Jeff Squyres
родитель
db2e5bf27c
Коммит
99c39991cd
0
.gitignore_global → .gitignore
поставляемый
0
.gitignore_global → .gitignore
поставляемый
1205
.hgignore_global
1205
.hgignore_global
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
@ -10,7 +10,7 @@ dnl Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
||||
dnl University of Stuttgart. All rights reserved.
|
||||
dnl Copyright (c) 2004-2005 The Regents of the University of California.
|
||||
dnl All rights reserved.
|
||||
dnl Copyright (c) 2006-2010 Cisco Systems, Inc. All rights reserved.
|
||||
dnl Copyright (c) 2006-2014 Cisco Systems, Inc. All rights reserved.
|
||||
dnl Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved.
|
||||
dnl Copyright (c) 2009 IBM Corporation. All rights reserved.
|
||||
dnl Copyright (c) 2009 Los Alamos National Security, LLC. All rights
|
||||
@ -34,7 +34,7 @@ opal_show_subtitle "OPAL Configuration options"
|
||||
# Is this a developer copy?
|
||||
#
|
||||
|
||||
if test -d .svn -o -d .hg -o -d .git; then
|
||||
if test -d .git; then
|
||||
OPAL_DEVEL=1
|
||||
else
|
||||
OPAL_DEVEL=0
|
||||
|
||||
@ -183,7 +183,7 @@ m4_ifdef([project_oshmem],
|
||||
# override/fixup the version numbers set by AC_INIT, since on
|
||||
# developer builds, there's no good way to know what the version is
|
||||
# before running configure :(. We only use the base version number
|
||||
# (ie, no svn r numbers) for the version set in AC_INIT. This will
|
||||
# (i.e., no git hash) for the version set in AC_INIT. This will
|
||||
# always match reality because we add the VERSION file (the only way
|
||||
# to change the major.minor.release{greek}) into the configure
|
||||
# dependencies. PACKAGE_VERSION the AC_DEFINE doesn't change once set
|
||||
|
||||
17
contrib/dist/build-server/README.txt
поставляемый
17
contrib/dist/build-server/README.txt
поставляемый
@ -28,20 +28,17 @@ at the moment, so this brief README will have to do for now.
|
||||
# ...etc.
|
||||
|
||||
- crontab.txt: the cron jobs that are running as "mpiteam" on
|
||||
eddie.osl.iu.edu as of 8 Aug 2012. They show the CLI options to
|
||||
mtt.open-mpi.org as of 1 Oct 2014. They show the CLI options to
|
||||
several of these scripts.
|
||||
|
||||
- openmpi-nightly-tarball.sh: script used to make the nightly tarballs
|
||||
and copy them to the live web tree
|
||||
|
||||
- openmpi-update-www.open-mpi.org.sh: this is the script fired by
|
||||
cron to basically run "svn up" on the live www.open-mpi.org web
|
||||
cron to basically run "git pull" on the live www.open-mpi.org web
|
||||
site, in local directory /l/osl/www/www.open-mpi.org (this is a
|
||||
network mount, actually)
|
||||
|
||||
- openmpi-test-tarball.sh: I don't remember what this is. I suspect
|
||||
it isn't used anymore.
|
||||
|
||||
- openmpi-release.sh: use this script to make official Open MPI
|
||||
tarball releases. Give it the path in the OMPI SVN repo to make the
|
||||
tarball from. For example:
|
||||
@ -60,13 +57,3 @@ at the moment, so this brief README will have to do for now.
|
||||
of these old directories. It's set to fire by
|
||||
openmpi-nightly-tarball.sh, and removes any busted nightly build
|
||||
directories older than 28 days.
|
||||
|
||||
- openmpi-nightly-test-build.sh: I don't remember what this is. I
|
||||
suspect it isn't used anymore.
|
||||
|
||||
- openmpi-nightly-tmpbranch-tarball.sh: I don't remember what this is.
|
||||
I suspect it isn't used anymore.
|
||||
|
||||
- openmpi-ft-cr-tarball.sh: I suspect this was used by Josh to make
|
||||
tarballs of his FT work when it was off on a branch. I suspect it
|
||||
isn't used anymore.
|
||||
|
||||
132
contrib/dist/build-server/crontab.txt
поставляемый
132
contrib/dist/build-server/crontab.txt
поставляемый
@ -1,6 +1,132 @@
|
||||
0 21 * * * $HOME/scripts/openmpi-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
#########################
|
||||
# MPI Testing Tool (MTT)
|
||||
# - Original plan
|
||||
#
|
||||
# Everyday
|
||||
# - 23:00 - 1:00 Update MTT Statistics [Database]
|
||||
# - 1:00 - 3:00 Update MTT Statistics [Contribution]
|
||||
# - 3:00 - 3:05 Update Contrib Graph
|
||||
# - 4:00 - 4:30 Maintain internal PostgreSQL stats (Day)
|
||||
# - 4:30 - 8:30 Update Static MTT Visualization Extension (Disabled due to heavy load)
|
||||
#
|
||||
# Saturday
|
||||
# - 1:00 - 12:00 Backup the MTT database
|
||||
# - 14:30 - 18:30 Update Static MTT Visualization Extension
|
||||
#
|
||||
# Sunday
|
||||
# - 4:00 - 4:30 Maintain internal PostgreSQL stats (Week)
|
||||
# - 4:00 - 4:30 Maintain internal PostgreSQL stats (Month) [1st sunday only]
|
||||
# - 5:00 - 5:30 Check for Integrity Violations (week)
|
||||
#
|
||||
#########################
|
||||
|
||||
|
||||
#########################
|
||||
# Update MTT Statistics [Database]
|
||||
# - everyday at 11:00
|
||||
# - runs in about 25 - 30 min
|
||||
#########################
|
||||
0 23 * * * /l/osl/www/mtt.open-mpi.org/cron/mtt-update-stats-db-only.pl
|
||||
|
||||
#########################
|
||||
# Update MTT Statistics [Contribution]
|
||||
# - everyday at 1 am
|
||||
# - runs in about 2 hours
|
||||
#########################
|
||||
0 1 * * * /l/osl/www/mtt.open-mpi.org/cron/mtt-update-stats.pl
|
||||
|
||||
#########################
|
||||
# Update Contrib Graphs
|
||||
# - 3 am Everyday
|
||||
# - Runs in < 1 min
|
||||
# "-l" limits to just last year of data. (slightly faster to run)
|
||||
#########################
|
||||
0 3 * * * /l/osl/www/mtt.open-mpi.org/cron/create-contrib-graph.pl -l
|
||||
30 3 * * * /l/osl/www/mtt.open-mpi.org/cron/create-contrib-graph.pl
|
||||
|
||||
|
||||
|
||||
#########################
|
||||
# Maintain internal PostgreSQL stats (Week)
|
||||
# - Sunday at 4 am (except when Monthly is running)
|
||||
# - runs in about 30 min
|
||||
# JJH: 11/7/2011 - Since Monthly disabled, run every week
|
||||
#########################
|
||||
##0 4 * * 0 if [ `date +\%d` -ge 8 ]; then /l/osl/www/mtt.open-mpi.org/cron/periodic-maintenance.pl -week 1> /dev/null 2> /dev/null; fi
|
||||
0 4 * * 0 /l/osl/www/mtt.open-mpi.org/cron/periodic-maintenance.pl -week 1> /dev/null 2> /dev/null
|
||||
|
||||
#########################
|
||||
# Maintain internal PostgreSQL stats (Year)
|
||||
# - Dec. 1 at 4 am
|
||||
# - runs in about 6 hours (?? Verify)
|
||||
#########################
|
||||
0 4 1 12 * /l/osl/www/mtt.open-mpi.org/cron/periodic-maintenance.pl -year 1> /dev/null 2> /dev/null
|
||||
|
||||
|
||||
#########################
|
||||
# Email Reports
|
||||
# - Tues - Thursday at 9 am (Past 12 hours)
|
||||
# - Tues - Thursday at 9 pm (Past 24 hours)
|
||||
# - Monday at 9 am (Past 60 hours)
|
||||
#########################
|
||||
0 9 * * 2-4 /l/osl/www/mtt.open-mpi.org/cron/alerts.php -f /l/osl/www/mtt.open-mpi.org/cron/morning.ini 1> /dev/null 2> /dev/null
|
||||
0 21 * * 2-4 /l/osl/www/mtt.open-mpi.org/cron/alerts.php -f /l/osl/www/mtt.open-mpi.org/cron/evening.ini 1> /dev/null 2> /dev/null
|
||||
0 9 * * 1 /l/osl/www/mtt.open-mpi.org/cron/alerts.php -f /l/osl/www/mtt.open-mpi.org/cron/monday.ini 1> /dev/null 2> /dev/null
|
||||
|
||||
|
||||
|
||||
#######################################################################
|
||||
# Build nightly tarballs
|
||||
#######################################################################
|
||||
0 21 * * * $HOME/scripts/netloc-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
1 21 * * * $HOME/scripts/hwloc-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
5 21 * * * $HOME/scripts/openmpi-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
|
||||
|
||||
#######################################################################
|
||||
# Update web sites
|
||||
#######################################################################
|
||||
*/15 * * * * $HOME/scripts/openmpi-update-www.open-mpi.org.sh < /dev/null > $HOME/tmp/git-www-ompi.log 2>&1
|
||||
*/10 * * * * $HOME/scripts/forum-update-meetings.mpi-forum.org.sh < /dev/null > $HOME/tmp/svn-up-forum.log 2>&1
|
||||
|
||||
|
||||
#=====================================================================
|
||||
#=====================================================================
|
||||
# Outdated / unused scripts
|
||||
#=====================================================================
|
||||
#=====================================================================
|
||||
|
||||
#########################
|
||||
# Maintain internal PostgreSQL stats (Day)
|
||||
# - every weekday at 4 am
|
||||
# - runs in about 10 - 20 min
|
||||
#
|
||||
# JJH: 12/20/2011 Disable this to try to help with memory issues in the db
|
||||
#########################
|
||||
#0 4 * * 1-5 /l/osl/www/mtt.open-mpi.org/cron/periodic-maintenance.pl -day 1> /dev/null 2> /dev/null
|
||||
|
||||
#########################
|
||||
# Maintain internal PostgreSQL stats (Month)
|
||||
# - 1st Sunday of the Month at 4 am
|
||||
# - runs in about 21 hours (not 20 min)
|
||||
# JJH: 11/7/2011 - Disabled due to long runtime causing submission errors
|
||||
#########################
|
||||
#0 4 * * 0 if [ `date +\%d` -lt 8 ]; then /l/osl/www/mtt.open-mpi.org/cron/periodic-maintenance.pl -month 1> /dev/null 2> /dev/null; fi
|
||||
|
||||
#30 20 * * * $HOME/scripts/openmpi-nightly-tmpbranch-tarball.sh </dev/null >/dev/null 2>&1
|
||||
0 4 * * * $HOME/scripts/lammpi-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
#0 0 * * * $HOME/scripts/openmpi-ft-cr-tarball.sh </dev/null >/dev/null 2>&1
|
||||
30 * * * * $HOME/scripts/openmpi-update-www.open-mpi.org.sh < /dev/null > $HOME/tmp/svn-up-ompi.log 2>&1
|
||||
|
||||
# Don't build any more LAM/MPI tarballs!
|
||||
#0 4 * * * $HOME/scripts/lammpi-nightly-tarball.sh </dev/null >/dev/null 2>&1
|
||||
|
||||
# As of 15 Sep 2014, we're 100% Gitdub. Leaving this (commented out) entry for
|
||||
# the time being, just in case gitdub implodes, and we need a hint on how to go
|
||||
# back to the old way.
|
||||
#*/5 * * * * /u/mpiteam/scripts/github-send-commit-mails.pl /u/mpiteam/git/hwloc-debian /u/mpiteam/git/docs /u/mpiteam/git/ompi-www /u/mpiteam/git/orcm /u/mpiteam/git/mtt
|
||||
# Removed: /u/mpiteam/git/scon (defunct)
|
||||
# Removed: /u/mpiteam/git/netloc (now using gitdub)
|
||||
|
||||
# Keep OMPI svn:ignore properties in sync with .gitignore_global and
|
||||
# .hgignore_global
|
||||
# Oct 2014: once we switched to GitHub, this was no longer necessary.
|
||||
#0 1 * * * $HOME/scripts/openmpi-nightly-githgignore.sh
|
||||
|
||||
77
contrib/dist/build-server/openmpi-ft-cr-tarball.sh
поставляемый
77
contrib/dist/build-server/openmpi-ft-cr-tarball.sh
поставляемый
@ -1,77 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
#####
|
||||
#
|
||||
# Configuration options
|
||||
#
|
||||
#####
|
||||
|
||||
# e-mail address to send results to
|
||||
results_addr=jjhursey@open-mpi.org
|
||||
|
||||
# svn repository uri
|
||||
code_uri=http://svn.open-mpi.org/svn/ompi
|
||||
|
||||
# where to put built tarballs
|
||||
#outputroot=/l/osl/www/www.open-mpi.org/nightly
|
||||
outputroot=/home/mpiteam/openmpi/testing
|
||||
|
||||
# where to find the build script
|
||||
script_uri=${code_uri}/trunk/contrib/nightly/create_tarball.sh
|
||||
|
||||
# The tarballs to make
|
||||
#if [ $# -eq 0 ] ; then
|
||||
dirs="tmp/jjhursey-ft-cr-stable"
|
||||
ver=trunk
|
||||
#else
|
||||
# dirs=$@
|
||||
# ver=trunk
|
||||
#fi
|
||||
|
||||
# Build root - scratch space
|
||||
build_root=/home/mpiteam/openmpi/testing-tarball-build-root
|
||||
|
||||
export PATH=$HOME/local/bin:$PATH
|
||||
export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
|
||||
|
||||
#####
|
||||
#
|
||||
# Actually do stuff
|
||||
#
|
||||
#####
|
||||
|
||||
# load the modules configuration
|
||||
#JJH File No longer exists: . /etc/profile.d/00-modules.sh
|
||||
. /etc/profile.d/modules.sh
|
||||
module use ~/modules
|
||||
module load sowing
|
||||
|
||||
# get our nightly build script
|
||||
mkdir -p $build_root
|
||||
cd $build_root
|
||||
script=`basename $script_uri`
|
||||
wget --quiet --no-check-certificate --tries=10 $script_uri -O $script
|
||||
if test ! $? -eq 0 ; then
|
||||
echo "wget of Open MPI nightly tarball create script failed."
|
||||
if test -f $script ; then
|
||||
echo "Using older version of $script for this run."
|
||||
else
|
||||
echo "No build script available. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
chmod +x $script
|
||||
|
||||
# Loop making them
|
||||
for dir in $dirs; do
|
||||
# ver=`basename $dir`
|
||||
|
||||
module load "autotools/ompi-$ver"
|
||||
|
||||
./$script \
|
||||
$build_root/ft-cr-${ver} \
|
||||
$results_addr \
|
||||
$code_uri/$dir \
|
||||
$outputroot/ft-cr-$ver >/dev/null 2>&1
|
||||
module unload "autotools"
|
||||
done
|
||||
64
contrib/dist/build-server/openmpi-nightly-tarball.sh
поставляемый
64
contrib/dist/build-server/openmpi-nightly-tarball.sh
поставляемый
@ -10,13 +10,16 @@
|
||||
results_addr=testing@open-mpi.org
|
||||
|
||||
# svn repository uri
|
||||
code_uri=http://svn.open-mpi.org/svn/ompi
|
||||
master_code_uri=https://github.com/open-mpi/ompi.git
|
||||
master_raw_uri=https://raw.github.com/open-mpi/ompi
|
||||
release_code_uri=https://github.com/open-mpi/ompi.git
|
||||
release_raw_uri=https://raw.github.com/open-mpi/ompi
|
||||
|
||||
# where to put built tarballs
|
||||
outputroot=/l/osl/www/www.open-mpi.org/nightly
|
||||
|
||||
# where to find the build script
|
||||
create_script=/contrib/nightly/create_tarball.sh
|
||||
script_uri=contrib/nightly/create_tarball.sh
|
||||
|
||||
# helper scripts dir
|
||||
script_dir=/u/mpiteam/scripts
|
||||
@ -24,7 +27,7 @@ script_dir=/u/mpiteam/scripts
|
||||
# The tarballs to make
|
||||
if [ $# -eq 0 ] ; then
|
||||
# We're no longer ever checking the 1.0 - 1.4 branches anymore
|
||||
dirs="/trunk /branches/v1.7 /branches/v1.6"
|
||||
branches="master v1.8 v1.6"
|
||||
else
|
||||
dirs=$@
|
||||
fi
|
||||
@ -41,8 +44,6 @@ export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
|
||||
#
|
||||
#####
|
||||
|
||||
script=`basename $create_script`
|
||||
|
||||
# load the modules configuration
|
||||
. /etc/profile.d/modules.sh
|
||||
module use ~/modules
|
||||
@ -53,27 +54,42 @@ cd $build_root
|
||||
|
||||
# Loop making the tarballs
|
||||
module unload autotools
|
||||
for dir in $dirs; do
|
||||
ver=`basename $dir`
|
||||
|
||||
module load "autotools/ompi-$ver"
|
||||
|
||||
script_uri="$code_uri$dir/$create_script"
|
||||
script_exec="$build_root/$ver/$script"
|
||||
echo "=== Getting script from: $script_uri"
|
||||
wget --quiet --no-check-certificate --tries=10 $script_uri -O "$script_exec"
|
||||
if test ! $? -eq 0 ; then
|
||||
echo "wget of Open MPI nightly $ver tarball create script failed."
|
||||
exit 1
|
||||
for branch in $branches; do
|
||||
if test "$branch" = "master"; then
|
||||
code_uri=$master_code_uri
|
||||
raw_uri=$master_raw_uri
|
||||
else
|
||||
code_uri=$release_code_uri
|
||||
raw_uri=$release_raw_uri
|
||||
fi
|
||||
|
||||
# Form a URL-specific script name
|
||||
script=$branch-`basename $script_uri`
|
||||
|
||||
echo "=== Getting script from: $raw_uri"
|
||||
wget --quiet --no-check-certificate --tries=10 $raw_uri/$branch/$script_uri -O $script
|
||||
if test ! $? -eq 0 ; then
|
||||
echo "wget of OMPI nightly tarball create script failed."
|
||||
if test -f $script ; then
|
||||
echo "Using older version of $script for this run."
|
||||
else
|
||||
echo "No build script available. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
chmod +x $script
|
||||
|
||||
module load "autotools/ompi-$branch"
|
||||
|
||||
echo "=== Running script..."
|
||||
chmod +x "$script_exec"
|
||||
"$script_exec" \
|
||||
$build_root/$ver \
|
||||
$results_addr \
|
||||
$code_uri/$dir \
|
||||
$outputroot/$ver
|
||||
./$script \
|
||||
$build_root/$branch \
|
||||
$results_addr \
|
||||
$outputroot/$branch \
|
||||
$code_uri \
|
||||
$branch \
|
||||
>/dev/null 2>&1
|
||||
|
||||
module unload autotools
|
||||
echo "=== Done running script"
|
||||
|
||||
@ -81,5 +97,5 @@ for dir in $dirs; do
|
||||
# in here and clean up the old failed builds, we can accumulate
|
||||
# many over time. So remove any old failed bbuilds that are over
|
||||
# 4 weeks old.
|
||||
${script_dir}/remove-old.pl 28 $build_root/$ver
|
||||
${script_dir}/remove-old.pl 28 $build_root/$branch
|
||||
done
|
||||
|
||||
33
contrib/dist/build-server/openmpi-nightly-test-build.sh
поставляемый
33
contrib/dist/build-server/openmpi-nightly-test-build.sh
поставляемый
@ -1,33 +0,0 @@
|
||||
:
|
||||
buildroot=/home/jsquyres/openmpi
|
||||
installfile="$buildroot/nightly/`hostname`-installs.txt"
|
||||
addr=jsquyres@open-mpi.org
|
||||
#addr=testing@open-mpi.org
|
||||
|
||||
export PATH=$buildroot/local/bin:$PATH
|
||||
|
||||
versions="trunk v1.0"
|
||||
|
||||
for ver in $versions; do
|
||||
$buildroot/build_tarball.pl \
|
||||
--scratch $buildroot/nightly/$ver \
|
||||
--email $addr \
|
||||
--url http://www.open-mpi.org/nightly/$ver/ \
|
||||
--config $buildroot/$ver-config.txt \
|
||||
--leave-install $installfile \
|
||||
--make "" \
|
||||
--nocheck
|
||||
|
||||
# if test -s $installfile; then
|
||||
# dirs=`grep debug $installfile`
|
||||
# args=
|
||||
# for d in $dirs; do
|
||||
# args="--prefix $d $args"
|
||||
# done
|
||||
#
|
||||
# $buildroot/illegal_symbols_report.pl $args --email $addr --delete
|
||||
# rm -f $installfile
|
||||
# fi
|
||||
done
|
||||
exit 0
|
||||
|
||||
77
contrib/dist/build-server/openmpi-nightly-tmpbranch-tarball.sh
поставляемый
77
contrib/dist/build-server/openmpi-nightly-tmpbranch-tarball.sh
поставляемый
@ -1,77 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
#####
|
||||
#
|
||||
# Configuration options
|
||||
#
|
||||
#####
|
||||
|
||||
# e-mail address to send results to
|
||||
results_addr=testing@open-mpi.org
|
||||
#results_addr=timattox@open-mpi.org
|
||||
|
||||
# svn repository uri
|
||||
code_uri=http://svn.open-mpi.org/svn/ompi
|
||||
|
||||
# where to put built tarballs
|
||||
outputroot=/l/osl/www/www.open-mpi.org/nightly
|
||||
|
||||
# where to find the build script
|
||||
script_uri=${code_uri}/trunk/contrib/nightly/create_tarball.sh
|
||||
|
||||
# The tarballs to make
|
||||
#if [ $# -eq 0 ] ; then
|
||||
dirs="tmp/rhc-step2b"
|
||||
ver=trunk
|
||||
#else
|
||||
# dirs=$@
|
||||
# ver=trunk
|
||||
#fi
|
||||
|
||||
# Build root - scratch space
|
||||
build_root=/home/mpiteam/openmpi/nightly-tarball-build-root
|
||||
|
||||
export PATH=$HOME/local/bin:$PATH
|
||||
export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
|
||||
|
||||
#####
|
||||
#
|
||||
# Actually do stuff
|
||||
#
|
||||
#####
|
||||
|
||||
# load the modules configuration
|
||||
#JJH File No longer exists: . /etc/profile.d/00-modules.sh
|
||||
. /etc/profile.d/modules.sh
|
||||
module use ~/modules
|
||||
module load sowing
|
||||
|
||||
# get our nightly build script
|
||||
mkdir -p $build_root
|
||||
cd $build_root
|
||||
script=`basename $script_uri`
|
||||
wget --quiet --no-check-certificate --tries=10 $script_uri -O $script
|
||||
if test ! $? -eq 0 ; then
|
||||
echo "wget of Open MPI nightly tarball create script failed."
|
||||
if test -f $script ; then
|
||||
echo "Using older version of $script for this run."
|
||||
else
|
||||
echo "No build script available. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
chmod +x $script
|
||||
|
||||
# Loop making them
|
||||
for dir in $dirs; do
|
||||
# ver=`basename $dir`
|
||||
|
||||
module load "autotools/ompi-$ver"
|
||||
|
||||
./$script \
|
||||
$build_root/rhc-step2b \
|
||||
$results_addr \
|
||||
$code_uri/$dir \
|
||||
$outputroot/rhc-step2b >/dev/null 2>&1
|
||||
module unload "autotools"
|
||||
done
|
||||
8
contrib/dist/build-server/openmpi-release.sh
поставляемый
8
contrib/dist/build-server/openmpi-release.sh
поставляемый
@ -1,13 +1,10 @@
|
||||
#!/bin/sh -x
|
||||
|
||||
# The tarballs to make
|
||||
#dirs="/branches/v1.2 /branches/v1.1 /branches/v1.0 /trunk"
|
||||
|
||||
# The tarballs to make
|
||||
if [ $# -eq 0 ] ; then
|
||||
dirs="branches/v1.6"
|
||||
dirs="branches/v1.8"
|
||||
else
|
||||
dirs=$@
|
||||
dirs=$@
|
||||
fi
|
||||
|
||||
# Build root - scratch space
|
||||
@ -26,7 +23,6 @@ export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
|
||||
#####
|
||||
|
||||
# load the modules configuration
|
||||
#JJH File No longer exists: . /etc/profile.d/00-modules.sh
|
||||
. /etc/profile.d/modules.sh
|
||||
module use ~/modules
|
||||
module load sowing
|
||||
|
||||
75
contrib/dist/build-server/openmpi-test-tarball.sh
поставляемый
75
contrib/dist/build-server/openmpi-test-tarball.sh
поставляемый
@ -1,75 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
#####
|
||||
#
|
||||
# Configuration options
|
||||
#
|
||||
#####
|
||||
|
||||
# e-mail address to send results to
|
||||
results_addr=timattox@open-mpi.org
|
||||
|
||||
# svn repository uri
|
||||
code_uri=http://svn.open-mpi.org/svn/ompi
|
||||
|
||||
# where to put built tarballs
|
||||
#outputroot=/l/osl/www/www.open-mpi.org/nightly
|
||||
outputroot=/home/mpiteam/openmpi/testing
|
||||
|
||||
# where to find the build script
|
||||
script_uri=${code_uri}/trunk/contrib/nightly/create_tarball.sh
|
||||
|
||||
# The tarballs to make
|
||||
if [ $# -eq 0 ] ; then
|
||||
dirs="branches/v1.3"
|
||||
else
|
||||
dirs=$@
|
||||
fi
|
||||
|
||||
# Build root - scratch space
|
||||
build_root=/home/mpiteam/openmpi/testing-tarball-build-root
|
||||
|
||||
export PATH=$HOME/local/bin:$PATH
|
||||
export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
|
||||
|
||||
#####
|
||||
#
|
||||
# Actually do stuff
|
||||
#
|
||||
#####
|
||||
|
||||
# load the modules configuration
|
||||
#JJH File No longer exists: . /etc/profile.d/00-modules.sh
|
||||
. /etc/profile.d/modules.sh
|
||||
module use ~/modules
|
||||
module load sowing
|
||||
|
||||
# get our nightly build script
|
||||
mkdir -p $build_root
|
||||
cd $build_root
|
||||
script=`basename $script_uri`
|
||||
wget --quiet --no-check-certificate --tries=10 $script_uri -O $script
|
||||
if test ! $? -eq 0 ; then
|
||||
echo "wget of Open MPI nightly tarball create script failed."
|
||||
if test -f $script ; then
|
||||
echo "Using older version of $script for this run."
|
||||
else
|
||||
echo "No build script available. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
chmod +x $script
|
||||
|
||||
# Loop making them
|
||||
for dir in $dirs; do
|
||||
ver=`basename $dir`
|
||||
|
||||
module load "autotools/ompi-$ver"
|
||||
|
||||
./$script \
|
||||
$build_root/$ver \
|
||||
$results_addr \
|
||||
$code_uri/$dir \
|
||||
$outputroot/$ver >/dev/null 2>&1
|
||||
module unload "autotools"
|
||||
done
|
||||
223
contrib/nightly/create_tarball.sh
Исполняемый файл → Обычный файл
223
contrib/nightly/create_tarball.sh
Исполняемый файл → Обычный файл
@ -6,30 +6,31 @@
|
||||
# Copyright (c) 2004-2005 The University of Tennessee and The University
|
||||
# of Tennessee Research Foundation. All rights
|
||||
# reserved.
|
||||
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
||||
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
||||
# University of Stuttgart. All rights reserved.
|
||||
# Copyright (c) 2004-2005 The Regents of the University of California.
|
||||
# All rights reserved.
|
||||
# Copyright (c) 2006-2010 Cisco Systems, Inc. All rights reserved.
|
||||
# Copyright (c) 2006-2014 Cisco Systems, Inc. All rights reserved.
|
||||
# $COPYRIGHT$
|
||||
#
|
||||
#
|
||||
# Additional copyrights may follow
|
||||
#
|
||||
#
|
||||
# $HEADER$
|
||||
#
|
||||
# This script is used to create a nightly snapshot tarball of Open MPI.
|
||||
#
|
||||
# $1: scratch root
|
||||
# $2: e-mail address for destination
|
||||
# $3: SVN root
|
||||
# $4: dest dir
|
||||
# $5: version string for error e-mails, eg. trunk, v1.2, etc. (optional)
|
||||
# $3: git URL
|
||||
# $5: git branch
|
||||
#
|
||||
|
||||
scratch_root="$1"
|
||||
email="$2"
|
||||
svnroot="$3"
|
||||
destdir="$4"
|
||||
scratch_root=$1
|
||||
email=$2
|
||||
destdir=$3
|
||||
giturl=$4
|
||||
gitbranch=$5
|
||||
|
||||
# Set this to any value for additional output; typically only when
|
||||
# debugging
|
||||
@ -51,20 +52,16 @@ max_snapshots=5
|
||||
start_time="`date`"
|
||||
|
||||
# Sanity checks
|
||||
if test -z "$scratch_root" -o -z "$email" -o -z "$svnroot" \
|
||||
if test -z "$scratch_root" -o -z "$email" -o -z "$giturl" -o -z "$gitbranch" \
|
||||
-o -z "$destdir"; then
|
||||
echo "Must specify scratch root directory, e-mail address, SVN root, and destination directory"
|
||||
echo "$0 scratch_root email_addr dest_dir git_url git_branch"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get a version string to use if there is an error.
|
||||
# It will get replaced upon succesful "make distcheck" with the real version.
|
||||
# Extract (from the SVN root) a version string if one wasn't supplied.
|
||||
if test -n "$5"; then
|
||||
version="$5"
|
||||
else
|
||||
version=`basename $svnroot`
|
||||
fi
|
||||
# Use the branch name as the "version" string (for if there is an
|
||||
# error). This version string will be replaced upon successful "make
|
||||
# distcheck" with the real version.
|
||||
version=$gitbranch
|
||||
|
||||
# send a mail
|
||||
# should only be called after logdir is set
|
||||
@ -151,82 +148,6 @@ if test ! -f "$file"; then
|
||||
fi
|
||||
rm -f "$file"
|
||||
|
||||
# if there's a $destdir/latest_snapshot.txt, see if anything has
|
||||
# happened since that r number.
|
||||
desired_r=
|
||||
if test -f "$destdir/latest_snapshot.txt"; then
|
||||
# $r will be just an integer (not "r12345")
|
||||
r=`cat $destdir/latest_snapshot.txt | sed -e 's/.*r\([0-9]*\)/\1/'`
|
||||
if test -n "$debug"; then
|
||||
echo "** last snapshot r: $r"
|
||||
fi
|
||||
|
||||
# If the current HEAD is on this $svnroot, then we'll get a log
|
||||
# message. Otherwise, we'll get a single line of dashes.
|
||||
file=/tmp/svn-log.txt.$$
|
||||
svn log -r HEAD $svnroot > $file
|
||||
# if we got more than 1 line, then extract the r number from the
|
||||
# log message.
|
||||
need_build=0
|
||||
if test "`wc -l $file | awk '{ print $1}'`" != "1"; then
|
||||
# $head_r will be "rXXXXX"
|
||||
head_r=`head -n 2 $file | tail -n 1 | awk '{ print $1 }'`
|
||||
if test -n "$debug"; then
|
||||
echo "** found HEAD r: $head_r"
|
||||
fi
|
||||
|
||||
# If the head r is the same as the last_snapshot r, then exit
|
||||
# nicely
|
||||
rm -f /tmp/svn-log.txt.$$
|
||||
if test "r$r" = "$head_r"; then
|
||||
if test -n "$debug"; then
|
||||
echo "** svn HEAD r is same as last_snapshot -- not doing anything"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# If we get here, it means the head r is different than the
|
||||
# last_snapshot r, and therefore we need to build.
|
||||
need_build=1
|
||||
desired_r=$head_r
|
||||
fi
|
||||
|
||||
# If need_build still = 0, we know the r's are different. But has
|
||||
# anything happened on this branch since then?
|
||||
if test "$need_build" = "0"; then
|
||||
svn log -r HEAD:$r $svnroot > $file
|
||||
|
||||
# We'll definitely have at least one log message because we
|
||||
# included the last snapshot number in the svn log command
|
||||
# (i.e., we'll at least see the log message for that commit).
|
||||
# So there's no need to check for a single line of dashes
|
||||
# here.
|
||||
|
||||
# $last_commit_r will be "rXXXXX"
|
||||
last_commit_r=`head -n 2 $file | tail -n 1 | awk '{ print $1 }'`
|
||||
if test -n "$debug"; then
|
||||
echo "** found last commit r: $last_commit_r"
|
||||
fi
|
||||
|
||||
# If the head r is the same as the last_snapshot r, then exit
|
||||
# nicely
|
||||
rm -f $file
|
||||
if test "r$r" = "$last_commit_r"; then
|
||||
if test -n "$debug"; then
|
||||
echo "** Last commit is same r as last_snapshot -- not doing anything"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# If we get here, the r numbers didn't match, and we therefore
|
||||
# need a new snapshot.
|
||||
desired_r=$last_commit_r
|
||||
fi
|
||||
fi
|
||||
if test -n "$debug"; then
|
||||
echo "** we need a new snapshot"
|
||||
fi
|
||||
|
||||
# move into the scratch directory and ensure we have an absolute
|
||||
# pathname for it
|
||||
if test ! -d "$scratch_root"; then
|
||||
@ -238,52 +159,67 @@ fi
|
||||
cd "$scratch_root"
|
||||
scratch_root="`pwd`"
|
||||
|
||||
if test -n "$desired_r"; then
|
||||
# we got a desired r number from above, so use that
|
||||
# $svnr will be rXXXXX
|
||||
svnr=$desired_r
|
||||
else
|
||||
# we don't have a desired r number, so get the last r number of a
|
||||
# commit
|
||||
svn co -N "$svnroot" ompi > /dev/null 2>&1
|
||||
cd ompi
|
||||
# $svnr will be rXXXXX
|
||||
svnr="r`svn info . | egrep '^Last Changed Rev: [0-9]+' | awk '{ print $4 }'`"
|
||||
cd ..
|
||||
rm -rf ompi
|
||||
fi
|
||||
if test -n "$debug"; then
|
||||
echo "** making snapshot for r: $svnr"
|
||||
fi
|
||||
root="$scratch_root/create-$svnr"
|
||||
rm -rf "$root"
|
||||
mkdir "$root"
|
||||
cd "$root"
|
||||
# setup target directory where clone+logs will go
|
||||
clone_root="$scratch_root/hwloc-`date +%Y-%m-%d-%H%M%S`"
|
||||
rm -rf $clone_root
|
||||
mkdir -p $clone_root
|
||||
|
||||
# startup the logfile
|
||||
logdir="$root/logs"
|
||||
# startup the logfile (must be before do_command)
|
||||
logdir="$clone_root/logs"
|
||||
mkdir "$logdir"
|
||||
|
||||
# checkout a clean version
|
||||
r=`echo $svnr | cut -c2-`
|
||||
do_command "svn co $svnroot -r $r ompi"
|
||||
# Get a fresh git clone
|
||||
cd $clone_root
|
||||
do_command "git clone $giturl hwloc"
|
||||
cd hwloc
|
||||
do_command "git checkout $gitbranch"
|
||||
|
||||
# ensure that we append the SVN number on the official version number
|
||||
cd ompi
|
||||
svnversion="r`svnversion .`"
|
||||
version_files="`find . -name VERSION`"
|
||||
d=`date +'%b %d, %Y'`
|
||||
for file in $version_files; do
|
||||
sed -e 's/^want_repo_rev=.*/want_repo_rev=1/' \
|
||||
-e 's/^repo_rev=.*/repo_rev='$svnversion/ \
|
||||
-e 's@^date=.*@date="'"$d"' (nightly snapshot tarball)"@' \
|
||||
$file > $file.new
|
||||
cp -f $file.new $file
|
||||
rm -f $file.new
|
||||
done
|
||||
# Find the "git describe" string for this branch (remove a leading "hwloc-"
|
||||
# prefix, if there is one).
|
||||
describe=`git describe --tags --always | sed -e s/^hwloc-//`
|
||||
if test -n "$debug"; then
|
||||
echo "** found $gitbranch describe: $describe"
|
||||
fi
|
||||
version=$describe
|
||||
|
||||
# if there's a $destdir/latest_snapshot.txt, see if anything has
|
||||
# happened since the describe listed in that file
|
||||
if test -f "$destdir/latest_snapshot.txt"; then
|
||||
snapshot_describe=`cat $destdir/latest_snapshot.txt`
|
||||
if test -n "$debug"; then
|
||||
echo "** last snapshot describe: $snapshot_describe"
|
||||
fi
|
||||
|
||||
# Do we need a new snapshot?
|
||||
if test "$describe" = "$snapshot_describe"; then
|
||||
if test -n "$debug"; then
|
||||
echo "** git $gitbranch describe is same as latest_snapshot -- not doing anything"
|
||||
fi
|
||||
# Since we didn't do anything, there's no point in leaving the clone we
|
||||
# just created
|
||||
cd ..
|
||||
rm -rf $clone_root
|
||||
|
||||
# All done... nothing to see here...
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if test -n "$debug"; then
|
||||
echo "** making snapshot for describe: $describe"
|
||||
fi
|
||||
|
||||
# Ensure that VERSION is set to indicate that it wants a snapshot, and
|
||||
# insert the actual value that we want (so that hwloc_get_version.sh
|
||||
# will report exactly that version).
|
||||
sed -e 's/^snapshot=.*/snapshot=1/' \
|
||||
-e 's/^snapshot_version=.*/snapshot_version='$describe/ \
|
||||
VERSION > VERSION.new
|
||||
cp -f VERSION.new VERSION
|
||||
rm -f VERSION.new
|
||||
|
||||
# lie about our username in $USER so that autogen will skip all
|
||||
# .ompi_ignore'ed directories (i.e., so that we won't get
|
||||
# .ompi_ignore'ed directories (i.e., so that we won't get
|
||||
# .ompi_unignore'ed)
|
||||
USER="ompibuilder"
|
||||
export USER
|
||||
@ -294,17 +230,22 @@ do_command "./autogen.pl"
|
||||
# do config
|
||||
do_command "./configure --enable-dist"
|
||||
|
||||
# distcheck does many things; we need to ensure it doesn't pick up any
|
||||
# other OMPI installs via LD_LIBRARY_PATH. It may be a bit Draconian
|
||||
# to totally clean LD_LIBRARY_PATH (i.e., we may need something in there),
|
||||
# but at least in the current building setup, we don't. But be advised
|
||||
# that this may need to change in the future...
|
||||
# Do make distcheck (which will invoke config/distscript.csh to set
|
||||
# the right values in VERSION). distcheck does many things; we need
|
||||
# to ensure it doesn't pick up any other installs via LD_LIBRARY_PATH.
|
||||
# It may be a bit Draconian to totally clean LD_LIBRARY_PATH (i.e., we
|
||||
# may need something in there), but at least in the current building
|
||||
# setup, we don't. But be advised that this may need to change in the
|
||||
# future...
|
||||
save=$LD_LIBRARY_PATH
|
||||
LD_LIBRARY_PATH=
|
||||
do_command "make -j 8 distcheck"
|
||||
LD_LIBRARY_PATH=$save
|
||||
save=
|
||||
|
||||
# chmod the whole directory, so that core files are accessible by others
|
||||
chmod a+rX -R .
|
||||
|
||||
# move the resulting tarballs to the destdir
|
||||
gz="`/bin/ls openmpi*tar.gz`"
|
||||
bz2="`/bin/ls openmpi*tar.bz2`"
|
||||
|
||||
Загрузка…
Ссылка в новой задаче
Block a user