wiki:UABgridProjects
Last modified 6 years ago Last modified on 02/23/11 14:25:49

UABgrid Projects

This service is for hosting project resources for VOs and the dev site itself for uabgrid. It's built on a Debian base and will host trac/svn resources for VOs. The basic install was chosen with a web server and standard system configuration option.

Disable Default Services

This step is only required when the "standard system" option was chosen for the install. There are several services running that are not needed at all and some not during the initial roll out.

Turn off the ident (port 113) service:

cd /etc
patch -b inetd.conf << EOF
29c29
< ident         stream  tcp     wait    identd  /usr/sbin/identd        identd
---
> #ident                stream  tcp     wait    identd  /usr/sbin/identd        identd
EOF
/etc/init.d/openbsd-inetd force-reload

Turn off the services we don't need (smtp) and won't use for now (nfs):

/etc/init.d/portmap stop
/etc/init.d/nfs-common stop
/etc/init.d/exim4 stop
update-rc.d -f exim4 remove
update-rc.d -f portmap remove
update-rc.d -f nfs-common remove

These steps ensure a much more limited system service exposure to just port 80 and port 22.

Configure Network

Change the network interface config in /etc/network/interfaces

auto eth0
iface eth0 inet static
 address 138.26.125.65
 gateway 138.26.1.1
 netmask 255.255.0.0
 network 138.26.0.0
 broadcast 138.26.255.255

Additional Configurations

Follow the resolv.conf, ntp, and shibboleth config notes for the uabgrid-ca

Web site configuration

Creating projects.uabgrid and dev.uabgrid from the vhost template:

sed -e 's/_IPADDR_/138.26.125.65/' \
    -e 's/_FQDN_/projects.uabgrid.uab.edu/' \
    -e 's/_SHORTNAME_/projects.uabgrid/' \
    new-service-site > projects.uabgrid
sed -e 's/_IPADDR_/138.26.125.64/' \
    -e 's/_FQDN_/dev.uabgrid.uab.edu/' \
    -e 's/_SHORTNAME_/dev.uabgrid/' \
    new-service-site > dev.uabgrid

Make the vhost dirs

mkdir -p /srv/www/projects.uabgrid
mkdir -p /srv/www/dev.uabgrid
touch /srv/www/projects.uabgrid/index.html
touch /srv/www/dev.uabgrid/index.html

Enable the vhosts

a2ensite projects.uabgrid
a2ensite dev.uabgrid
a2dissite default

Shibboleth SP Configuration

shibboleth.xml customization:

sed -e 's/_FQDN_/projects.uabgrid.uab.edu/' -e 's/_EMAIL_/jpr@uab.edu/' \
  shibboleth.xml.template > shibboleth.xml

metadata update:

sed -e 's/_FQDN_/projects.uabgrid.uab.edu/' -e 's/_EMAIL_/jpr@uab.edu/' sp-metadata.xml.template

Test the service

https://projects.uabgrid.uab.edu/secure/phpinfo.php

Adding dev.uabgrid

This simply requires added ACS entries to the UABgrid metadata and making sure that the network interface alias is defined.

                        <AssertionConsumerService index="1" isDefault="false"
                                Binding="urn:oasis:names:tc:SAML:1.0:profiles:browser-post"
                                Location="https://dev.uabgrid.uab.edu/Shibboleth.sso/SAML/POST"/>
                        <AssertionConsumerService index="2" isDefault="true"
                                Binding="urn:oasis:names:tc:SAML:1.0:profiles:artifact-01"
                                Location="https://dev.uabgrid.uab.edu/Shibboleth.sso/SAML/Artifact"/>

/etc/network/interfaces

auto eth0:1
iface eth0:1 inet static
 address 138.26.125.64
 netmask 255.255.0.0

Start the interface

ifup eth0:1

https://dev.uabgrid.uab.edu/secure/phpinfo.php

Set up Trac

Debian4 includes a packaged 10.3.x release of Trac. The latest trac is 10.4, but this package is a good source for at least installing all the dependencies of the 10.x line. Since it's close to the lastest stable, we'll used it unless there is cause not to.

apt-get install trac

The /usr/share/doc/trac/README.Debain has some helpful information for how things are set up by default. Their aproache is the CGI route with the cgi naming exposed, which is not the way we want to go. The executable should be hidden. Most of this hiding is done via rewrite rules, so they just need to be duplicated appropriately.

Config for Trac vhosts

These are the final files. dev.uabgrid first then projects.uabgrid

<VirtualHost 138.26.125.64:80>
        ServerName dev.uabgrid.uab.edu
        ServerAdmin webmaster@dev.uabgrid.uab.edu
        ServerSignature On
        RewriteEngine On
        #RewriteLog /tmp/rw.log
        #RewriteLogLevel 3

        ErrorLog /var/log/apache2/error-dev.uabgrid.log
        LogLevel warn
        CustomLog /var/log/apache2/access-dev.uabgrid.log combined

        DocumentRoot /srv/www/dev.uabgrid

        #<Directory /srv/www/dev.uabgrid/>
        #        Options Indexes FollowSymLinks MultiViews
        #        AllowOverride None
        #        Order allow,deny
        #        allow from all
        #</Directory>

        Alias /secure /var/www/secure
        <Directory /var/www/secure>
                AllowOverride All
                Options All
                Order allow,deny
                Allow from all
        </Directory>

        #
        # Trac configuration
        #
        RewriteCond /srv/www/dev.uabgrid/$1             -d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /srv/www/dev.uabgrid/trac.cgi$2 [L,E=TRAC_ENV:/srv/www/dev.uabgrid/$1]
        RewriteCond $1$2                                !^/Shibboleth.sso.*
        RewriteRule ^(/[^/+])(/?.*)    /srv/www/dev.uabgrid/trac.cgi$1$2 [L,E=TRAC_ENV:/srv/www/dev.uabgrid/uabgrid]
        RewriteRule ^/$    /srv/www/dev.uabgrid/trac.cgi [L,E=TRAC_ENV:/srv/www/dev.uabgrid/uabgrid]



        Alias /trac/ "/usr/share/trac/htdocs/"
        <Directory "/usr/share/trac/htdocs/">
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                Allow from all
        </Directory>

        <Directory /srv/www/dev.uabgrid/>
                AllowOverride None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AddHandler cgi-script .cgi
                Order allow,deny
                Allow from all
        </Directory>

        <Location "/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </Location>

        <LocationMatch "/[[:alnum:]\-]+/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </LocationMatch>


</VirtualHost>

<VirtualHost 138.26.125.64:443>
        ServerName dev.uabgrid.uab.edu
        ServerAdmin webmaster@dev.uabgrid.uab.edu
        ServerSignature On
        RewriteEngine On

        ErrorLog /var/log/apache2/error-dev.uabgrid.log
        LogLevel warn
        CustomLog /var/log/apache2/access-dev.uabgrid.log combined

        DocumentRoot /srv/www/dev.uabgrid

        SSLEngine on
        SSLCertificateFile /etc/ssl/certs/dev.uabgrid.uab.edu.crt
        SSLCertificateKeyFile /etc/ssl/private/dev.uabgrid.uab.edu.key
        SSLCACertificatePath /etc/ssl/certs

        #<Directory /srv/www/dev.uabgrid/>
        #        Options Indexes FollowSymLinks MultiViews
        #        AllowOverride None
        #        Order allow,deny
        #        allow from all
        #</Directory>

        Alias /secure /var/www/secure
        <Directory /var/www/secure>
                AllowOverride All
                Options All
                Order allow,deny
                Allow from all
        </Directory>

        #
        # Trac configuration
        #
        RewriteCond /srv/www/dev.uabgrid/$1             -d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /srv/www/dev.uabgrid/trac.cgi$2 [L,E=TRAC_ENV:/srv/www/dev.uabgrid/$1]
        RewriteCond $1$2                                !^/Shibboleth.sso.*
        RewriteRule ^(/[^/+])(/?.*)    /srv/www/dev.uabgrid/trac.cgi$1$2 [L,E=TRAC_ENV:/srv/www/dev.uabgrid/uabgrid]
        RewriteRule ^/$    /srv/www/dev.uabgrid/trac.cgi [L,E=TRAC_ENV:/srv/www/dev.uabgrid/uabgrid]

        Alias /trac/ "/usr/share/trac/htdocs/"
        <Directory "/usr/share/trac/htdocs/">
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                Allow from all
        </Directory>


        <Directory /srv/www/dev.uabgrid/>
                AllowOverride None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AddHandler cgi-script .cgi
                Order allow,deny
                Allow from all
        </Directory>

        <Location "/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </Location>

        <LocationMatch "/[[:alnum:]\-]+/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </LocationMatch>

</VirtualHost>
<VirtualHost 138.26.125.65:80>
        ServerName projects.uabgrid.uab.edu
        ServerAdmin webmaster@projects.uabgrid.uab.edu
        ServerSignature On
        RewriteEngine On

        ErrorLog /var/log/apache2/error-projects.uabgrid.log
        LogLevel warn
        CustomLog /var/log/apache2/access-projects.uabgrid.log combined

        DocumentRoot /srv/www/projects.uabgrid

        # Project rename rules
        RewriteRule ^/sg-metasched(/?.*)    /sg-submit$1 [R=301,L]

        #<Directory /srv/www/projects.uabgrid/>
        #        Options Indexes FollowSymLinks MultiViews
        #        AllowOverride None
        #        Order allow,deny
        #        allow from all
        #</Directory>

        Alias /secure /var/www/secure
        <Directory /var/www/secure>
                AllowOverride All
                Options All
                Order allow,deny
                Allow from all
        </Directory>

        #
        # Trac configuration
        #
        RewriteCond /srv/www/projects.uabgrid/$1             !-d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /index.html [L]
        RewriteCond /srv/www/projects.uabgrid/$1             -d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /srv/www/projects.uabgrid/trac.cgi$2 [S=1,E=TRAC_ENV:/srv/www/projects.uabgrid/$1]


        Alias /trac/ "/usr/share/trac/htdocs/" 
        <Directory "/usr/share/trac/htdocs/">
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                Allow from all
        </Directory>


        <Directory /srv/www/projects.uabgrid/>
                AllowOverride None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AddHandler cgi-script .cgi
                Order allow,deny
                Allow from all
        </Directory>

        <LocationMatch "/[[:alnum:]\-]+/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </LocationMatch>


</VirtualHost>

<VirtualHost 138.26.125.65:443>
        ServerName projects.uabgrid.uab.edu
        ServerAdmin webmaster@projects.uabgrid.uab.edu
        ServerSignature On
        RewriteEngine On

        ErrorLog /var/log/apache2/error-projects.uabgrid.log
        LogLevel warn
        CustomLog /var/log/apache2/access-projects.uabgrid.log combined

        DocumentRoot /srv/www/projects.uabgrid

        SSLEngine on
        SSLCertificateFile /etc/ssl/certs/projects.uabgrid.uab.edu.crt
        SSLCertificateKeyFile /etc/ssl/private/projects.uabgrid.uab.edu.key
        SSLCACertificatePath /etc/ssl/certs

        # Project rename rules
        RewriteRule ^/sg-metasched(/?.*)    /sg-submit$1 [R=301,L]

        #<Directory /srv/www/projects.uabgrid/>
        #        Options Indexes FollowSymLinks MultiViews
        #        AllowOverride None
        #        Order allow,deny
        #        allow from all
        #</Directory>

        Alias /secure /var/www/secure
        <Directory /var/www/secure>
                AllowOverride All
                Options All
                Order allow,deny
                Allow from all
        </Directory>

        #
        # Trac configuration
        #
        RewriteCond /srv/www/projects.uabgrid/$1             !-d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /index.html [L]
        RewriteCond /srv/www/projects.uabgrid/$1             -d
        RewriteRule ^/([[:alnum:]\-]+)(/?.*)    /srv/www/projects.uabgrid/trac.cgi$2 [S=1,E=TRAC_ENV:/srv/www/projects.uabgrid/$1]


        Alias /trac/ "/usr/share/trac/htdocs/"
        <Directory "/usr/share/trac/htdocs/">
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                Allow from all
        </Directory>


        <Directory /srv/www/projects.uabgrid/>
                AllowOverride None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AddHandler cgi-script .cgi
                Order allow,deny
                Allow from all
        </Directory>

        <LocationMatch "/[[:alnum:]\-]+/login">
                AuthType shibboleth
                ShibRequireSession On
                Require user ~ ^.+$
        </LocationMatch>

</VirtualHost>

Trac References

MTA Configuration

In order to efficiently support the notification services of trac we need to configure messaging support for this host. The http://trac.edgewall.org/wiki/TracNotification docs are good and healthy to re-read to understand the flexibility of the SMTP configuration.

Because a base debian install comes with Exim4 MTA, the dpkg-reconfigure exim4-config command was used to just set up existing Exim4 rather than move to Sendmail. The debian exim4 instructions were sufficient to set this up. Chose to split configuration mainly to make it easer to learn about (and potentially modify) the config of exim4. The Internet site config option was chosen. Concern over opening up projects.uabgrid to accepting mail for anything or opening unintended services for exploit proved unfounded because the option allows choosing which interfaces to listen on with the default being just localhost, which is sufficient and secure. Also, if no hosts are specified in the domain config then local mail delivery is off. Left that setting blank. DNS is full service and the local mail format was standard mbox. This configuration can be reviewed in /etc/exim4/update-exim4.conf.conf which looks like this:

dc_eximconfig_configtype='internet'
dc_other_hostnames=''
dc_local_interfaces='127.0.0.1'
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='true'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'

The Debian exim4 wiki may also be helpful.

The email CC notification was tested and it comes through. Trac logging was used to verify MTA processing on projects.uabgrid.

Set Up Access to Subversion

For the time being, WebDav?-based access to the repositories on projects.uabgrid is not enabled. This requires traditional access via ssh. Note: This process needs to be replaced with webdav.

To configure access accounts need to be established on projects.uabgrid for the authorized users.

for user in ppreddy pavgi vladk
do
   useradd -m -c "$user@uab.edu" $user
done

Enable blazerid authn for this host. Create the file /etc/authn_ldap.conf:

ssl start_tls
ssl on

uri ldaps://ldap.uab.edu/

base ou=people,dc=uab,dc=edu

tls_cacertfile /etc/ssl/certs/ca-certificates.crt

Note: tls_cacertdir /etc/ssl/certs would seem more ideal because of the cleaner management of certificates but a bug seems to cause enumeration delays, see ticket:19.

Configure PAM to support ldap authn:

apt-get install libpam-ldap
cd /etc/pam.d
patch -l common-auth << EOF
10c10,11
< auth  required   pam_unix.so nullok_secure
---
> auth  sufficient      pam_unix.so nullok_secure
> auth    sufficient      pam_ldap.so use_first_pass config=/etc/auth_ldap.conf
EOF

Create a new group for the authorized users so they can have be given access to the repository files.

groupadd atlab
cd /srv/svn/projects.uabgrid
chgrp -R atlab gpir-sge/
find gpir-sge/ -perm /u+w -exec chmod g+w {} \;

Add the users to the new authorization group.

Provision Projects

Provisioning projects resources is a operating process that occurs whenever a collaboration group creates a project. The steps for setting up a project are being documented as part of the UABgrid provisioning project.

Configuring subversion access over HTTPS

  • Setup: Apache handles authentication using x509 certificates. A user needs to be part of a valid group which has access permissions for SVN repository. These permissions are specified in Location directive using AuthUserFile and AuthGroupFile. A username must be present in AuthUserFile (authentication) and it should also have an entry in AuthGroupFile group members.

Install prerequisites and backup

  • Install apache module for for DAV SVN
    # apt-get install libapache2-svn
    
  • Back up repositories
    # tar czf - svn/ www/ | ssh pavgi@meter.lab.ac.uab.edu -p1025 "dd of=projects-backup-20100519.tar.gz"  
    

  • SSL module is already configured on projects, so this step wasn't required.

Configuring HTTPS for Subversion repository

This configuration allows users to access SVN via the WebDAV interface via HTTPS. The SSL configuration of Apache forces the client to provide a user certificate (SSLVerifyClient require) and uses that identity to determine their rights in the repository.

The default client certificates available in the UABgrid environment are issued from the local UABgrid CA. This CA is not trusted by Apache by default and must be added to /etc/ssl/certs so the server will be trusted see ticket:135 for details

  • Change ownership:
    # cd svn/dev.uabgrid/
    # chown -R www-data *
    
  • Add Location directive for SVN repository in /etc/apache2/sites-enabled/dev.uabgrid
            <Location /svn/uabgrid>
                    DAV svn
                    SVNPath /srv/svn/dev.uabgrid/uabgrid
                    # http://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslusername
                    SSLRequireSSL
                    SSLVerifyClient require
                    SSLVerifyDepth 1
                    SSLOptions +FakeBasicAuth -StdEnvVars
                    # SSLUserName SSL_CLIENT_S_DN_Email
                    SVNPathAuthz on
                    AuthType Basic
                    AuthName "UABGrid Authorization Realm"
                    AuthUserFile /etc/svn-www/users
                    AuthGroupFile /etc/svn-www/groups
                    Require group atlab
                    # AuthzSVNAccessFile /etc/svn-authz
            </Location>
    
  • Create AuthUserFile and AuthGroupFile, following is an example entry:
    # cat /etc/svn-www/users 
    /C=US/ST=Alabama/L=Birmingham/O=University of Alabama at Birmingham/OU=UABgrid/CN=pavgi@uab.edu/emailAddress=pavgi@uab.edu:xx
    # cat /etc/svn-www/groups 
    atlab: "/C=US/ST=Alabama/L=Birmingham/O=University of Alabama at Birmingham/OU=UABgrid/CN=pavgi@uab.edu/emailAddress=pavgi@uab.edu"