Showing posts with label subversion. Show all posts
Showing posts with label subversion. Show all posts

2008-06-05

Nightly commits /etc into SVN

Simple script committing changes in /etc/postfix and /etc/network into SVN repository. Can be scheduled every night - it checks for changes and in doesn't try to commit if system is untouched.

/etc -> SVN



#!/bin/bash

export LC_ALL=C
export EDITOR=/usr/bin/vim
tmpconf=/var/tmp/conf2svn
root=/etc

for srv in serverA serverB serverC
do
  for dir in network postfix
  do
    rm -rf ${tmpconf}/${srv} > /dev/null 2>&1
    mkdir -p ${tmpconf}/${srv} > /dev/null 2>&1
    cd ${tmpconf}/${srv}
    svn checkout --quiet svn://svn.mydomain.com/repos/conf/${srv}/${dir} ./${dir}
    scp -q -r ${srv}:${root}/${dir} .
    cd ${tmpconf}/${srv}/${dir}
    svn diff | grep "" > /dev/null
    if [[ $? -eq 0 ]]
    then
      svn commit -m "Commiting last changes in live servers' configuration (/etc)."
    fi
  done
done

rm -rf $tmpconf > /dev/null 2>&1


If you import /etc/something into SVN, delete /etc/something (be careful) and check something out back into /etc you won't need to make temporary working copy (SVN workspace) into /var/tmp/conf2svn

Script can't add new files. It only commits changes in existing ones.

MySQL schemas -> SVN



#!/bin/bash

export LC_ALL=C
export EDITOR=/usr/bin/vim
tmpconf=/var/tmp/schema2svn

rm -rf $tmpconf > /dev/null 2>&1
mkdir $tmpconf > /dev/null 2>&1

cd $tmpconf
svn checkout --quiet svn://svn.mydomain.com/repos/schemas .

for srv in dbA dbB
do
  mysqldump --host=$srv --password=nkjn39c134 --no-data --all-databases > ${tmpconf}/${srv}_schema_dump.txt
done

for file in *txt
do
  grep -v -E -- '(^--|ENGINE=MyISAM AUTO_INCREMENT)' $file > ${file}.new
  mv ${file}.new $file
done

svn diff | grep "" > /dev/null
if [[ $? -eq 0 ]]
then
  svn commit --quiet -m "Committing schema changes on live servers."
fi

rm -rf $tmpconf > /dev/null 2>&1


___
If you implemented your own schemas of backup/archiving, please let me know - send me an e-mail, drop a comment or meet me on IRC.

2008-06-02

Subversion + Tracd

Update 2008-08-07: In case of more then one svnserve daemon (how-to), initializing script should contain more variables: project name, listening port and additional option in start-stop-daemon (PID file). The last is necessary because without it start-stop-daemon behaves like killall.

Subversion server starting script


There are 3 main ways to access SVN repository (book):

  1. file system (file:///)

  2. svnserve server (svn:// on port 3690)

  3. regular Web server e.g. Apache2 (http://)


The following script (based on this one) starts svnserve server (book) on default port and listening on IP 127.0.0.1 (to not expose code to external network). To show repository, removing --listen-host directive binds server to non-local IP. Additionally you can restrict access to the code on Trac level (regardless of svnserv's IP). If you bind svnserve to 127.0.0.1 the code will still be visible to everybody with access to Tracd (Trac standalone server). If you wish to limit access to the code on Tracd level, you can

  1. bind Tracd to 127.0.0.1 or

  2. remove access to source for 'anonymous' user from Tracd (using trac-admin or web interface after installing TracWebAdmin)



The second idea is much more flexible and gives you ability to grant more granular access.

The code below is written with some assumptions:

  1. Home directories for Subversion and Trac are accordingly in /home/subversion and /home/trac

  2. Repository is in /home/subversion/docs created by svnadmin create /home/subversion/docs

  3. Authentication is set: .../repo/conf/passwd and .../repo/conf/svnserve.conf are updated

  4. We bind svnserve to localhost (127.0.0.1), but we show the code through Trac, hence...

  5. the access rules are defined in Trac-admin



To make system completely private, add --hostname=$TRACD_HOST to line if start-stop-daemon --start --chuid $TRAC_USER:$TRAC_GROUP in script /etc/init.d/tracd. The variable TRACD_HOST is defined in /etc/default/tracd as 127.0.0.1.

/etc/init.d/svnserve



#!/bin/bash
#
# svnserve - brings up the svn server so anonymous users
# can access svn
#

# Get LSB functions
. /lib/lsb/init-functions
. /etc/default/rcS

SVNSERVE=/usr/bin/svnserve
SVN_USER=subversion
SVN_GROUP=users
PROJECT=docs2
SVN_REPO_PATH=/home/${SVN_USER}/${PROJECT}
SVNSERVE_PORT=3700 # standard port is 3690
# Check that the package is still installed
[ -x $SVNSERVE ] || exit 0;
[ -d $SVN_REPO_PATH ] || exit 0;

case $1 in
start)
log_begin_msg "Starting svnserve..."
if start-stop-daemon --start --chuid $SVN_USER:$SVN_GROUP --umask 002 --exec $SVNSERVE -- -d --listen-host 127.0.0.1 --listen-port=$SVNSERVE_PORT --root $SVN_REPO_PATH --pid-file=$SVN_REPO_PATH/svnserve.pid
then
log_end_msg 0
else
log_end_msg $?
fi
;;

stop)
log_begin_msg "Stopping svnserve..."
if start-stop-daemon --stop --pidfile=/var/run/svnserve_${PROJECT}.pid --exec $SVNSERVE --retry 2
then
log_end_msg 0
else
log_end_msg $?
fi
;;

restart|force-reload)
$0 stop && $0 start
;;

*)
echo "Usage: /etc/init.d/svnserve {start|stop|restart|force-reload}"
exit 1
;;

esac

exit 0


Trac standalone web server starting scripts


Trac authentication


htdigest -c /home/trac/docs/conf/users.htdigest \
Docs yourusername


WebAdmin plugin



export PYTHONPATH=/home/trac/docs/plugins
easy_install --install-dir=/home/trac/docs/plugins \
http://svn.edgewall.com/repos/trac/sandbox/webadmin


/etc/default/tracd



TRACD=/usr/bin/tracd
TRACD_HOST=127.0.0.1
TRACD_PORT=8000
TRAC_USER=trac
TRAC_GROUP=users
TRAC_INITENV=docs
TRAC_PROJECT=docs
PROJECT_REALM=Docs
TRAC_HOME=/home/$TRAC_USER
TRAC_ENV=${TRAC_HOME}/$TRAC_INITENV
TRAC_PID=${TRAC_ENV}/tracd.pid


/etc/init.d/tracd



#!/bin/bash
#
# tracd - brings up the trac daemon
#

# Get LSB functions
. /lib/lsb/init-functions
. /etc/default/rcS
. /etc/default/tracd
# Check that the package is still installed
[ -x $TRACD ] || exit 0;
[ -d $TRAC_ENV ] || exit 0;

case $1 in
start)
log_begin_msg "Starting tracd..."
if start-stop-daemon --start --chuid $TRAC_USER:$TRAC_GROUP --chdir $TRAC_HOME --umask 002 --exec $TRACD -- --daemonize --pidfile=$TRAC_PID -p $TRACD_PORT -a ${TRAC_INITENV},${TRAC_ENV}/conf/users.htdigest,${PROJECT_REALM} ${TRAC_PROJECT}
then
  log_end_msg 0
else
  log_end_msg $?
fi
;;

stop)
log_begin_msg "Stopping tracd"
if start-stop-daemon --stop --pidfile=$TRAC_PID
then
  log_end_msg 0
else
  log_end_msg $?
fi
;;

restart|force-reload)
$0 stop && $0 start
;;

*)
echo "Usage: /etc/init.d/tracd {start|stop|restart|force-reload}"
exit 1
;;
esac

exit 0


This procedure gives you Wiki ready to write all sorts of notes and documentation, but there is a disadvantage of Trac - site is flat. It means all pages are like sheets of paper laying on a table. Until you link them, they are "unstructured". As a Wiki for notes and documentation I can recommend TWiki.

2008-05-29

Developer on-line tools

When I started develop my small amateur apps, I was looking for a convenient on-line place for the code. Some kind of CVS. I found:

  1. DevjaVu

  2. Google Code

  3. Assembla

  4. Code Spaces

  5. Unfunddle

  6. Beanstalk



Some services promises many, but it wasn't exactly what I wanted. The big mistake I made was Launchpad. It took a while since I understood what is it and how to deal with it.

After small consideration I narrowed list to Google Code and Unfuddle and finally chose Google Code: I had an account, I knew their idea, I was familiar with the light blue interface.

Now I try to utilize the SVN as much, as I can. Tickets, milestones, tasks and so on. Great tool for it is Trac (my instance of Trac is here. Its a quite good tool and very similar to Google Code tools, but has different approach. Google is good for store Open Source projects, has much bigger capability to serve code to wide audience, while Trac is more flexible and customisable, and can be completely private.

Mail reminder

Small application: receives e-mail messages and writes it as a task to small database. Then - at triggered time - sends a reminder.

My first application in Ruby no Rails.

Hosted on Google Code.