Migrating from Perforce to Bazaar
I've a big fan of Perforce for ages.
Recently Perforce changed the licensing from 20 free users to 5. This broke my backups. Time to switch to something FOSS.
I decided to give Bazaar a crack, on paper it ticks all the boxes.
Below is how I exported from Perforce (with history) in to a new Bazaar repo and setup triggers/hooks to run builds after code is submitted.
Export repos from Perforce
Due to the nature of my repo layout, I have I need to export each of my projects individually. This gives me the chance to migrate one project at a time.
Took a while to decipher the docs for P4 exporting to git fast-import format, its a bit broken, but command below enabled and export and import including change history.
sudo aptitude install bzr
sudo aptitude install perforce-p4python python-fastimport
mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr branch lp:bzrp4
bzr branch lp:bzr-fastimport
# rename the repo
mv bzr-fastimport fastimport
Then you need to set your normal P4_xx environment and p4 login
N.B. the documented bzr fast-export-from-p4 does not seem to work.
With taht setup the following got me a local bzr repo (cairoruler is a C project I decided to test with)
~/.bazaar/plugins/bzrp4/git_p4.py export //depot/cairoruler@all > cairoruler.fi
bzr fast-import cairoruler.fi cairoruler.bzr
This generates a directory ./cairoruler.bzr/p4/master.remote . Careful the ./cairoruler.bzr looks like a repo (it has .bzr) but its ./cairoruler.bzr/p4/master.remote that has the newly imported workspace.
The p4 export code is based on the git export tool, hence the odd name git_p4.py.
So far this is only working on my local HD.
Setting up the bzr server
I created a full OS in a Linux container, any Linux VM or baremetal server will do.LXC gives me a playground like a VM but faster to build.
lxc-create -t ubuntu -n bzf -f lxc-config.conf
Then install in the container bzr with just
apt-get install bzr
In theory thats all you need to do to use bzr over SSH.
There are other server mechanisms but, be warned, bzr default protocols are not secure.
bzr serve --directory=/srv/bzr/repo
should not be exposed on the Internet. You need to use SSH or do your own SSH tunneling.
Since I'm running a LXC container with ssh used for other stuff I have to NAT a port (9922) on my LXC Host to port 22 on the LXC Guest.
iptables -t nat -A PREROUTING -p tcp -i eth0 -d $PUBLIC_IP_ADDRESS --dport 9922 -j DNAT --to $BZR_IP_ADDRESS:22
iptables -A FORWARD -p tcp -d $BZR_IP_ADDRESS --dport 9922 -j ACCEPT
Thus BZR ULRs look like this.
bzr co bzr+ssh://me@myhost:9922/home/bzr/myrepo myrepo
Uploading to the server
Bazaar uses normal Linux user auth so first thing you'll want to do is create users and groups for uploads and share SSH keys.This does not give you any fine grained read/write control to each project, tan pis.
There is no special command for setting a default remote location in bzr just push
bzr push bzr+ssh://me@myhost:9922/home/bzr/myrepo/cairoruler
Porting Triggers to Hooks
Perforce triggers are one-liners that run a command when you submit code.Bzr has no triggers but it has Hooks. Its all pretty hacky I'm afraid.
You have to write hooks as Python libraries and install them in you bzr deployment. N.B. they are not part of backup/restore.
Most hooks run on the client.
In order to write a hook that runs on the server to build code when its submitted you have to write a single global post_change_branch_tip(). This is the same hook for all branches and submits, so the first thing the code has to do is work out which vbranch has changed.
When the hook is called it is passed the variable ChangeBranchTipParams, which is very poorly documented.
To debug you can't user python's print function, if you do you mess up the bzr protocol, so you need to write debug to a file.
After a couple of hours of hacking I got this
/usr/lib/python2.7/dist-packages/bzrlib/plugins/linci_hook.py
"""A linci hook.
"""
from bzrlib import branch
import subprocess
import datetime
#
# params == bzrlib.branch.ChangeBranchTipParams
# params.branch The branch being changed. An object of type bzrlib.branch.BzrBranch7
# params.branch.base is a "URL" filtered-139959297573392:///var/bzr/root/cairoruler/ from which we can extract the url
# params.old_revno Revision number before the change.
# params.new_revno Revision number after the change.
# params.old_revid Tip revision id before the change.
# params.new_revid revision id after the change
#
def post_change_branch_tip(params):
#
# post_change_branch_tip is called under various circumstances, only fire if the rev no has changed.
#
if params.old_revno < params.new_revno:
file = open("/var/log/bzr-comitlog", "a")
file.write("%s %s\n" % (datetime.datetime.now(), params.branch.base))
path = params.branch.base.split('/')
branch_name = path[len(path) - 1]
if not branch_name:
branch_name = path[len(path) - 2]
if not branch_name:
return
if branch_name in ['cairoruler']:
file.write(datetime.datetime.now() + " executing: ci.sh " + branch_name + "\n")
file.close()
subprocess.call('/home/bzr/bin/ci.sh ' + branch_name, shell=True)
#
# register hook, 3rd parameter is what is shown by `bzr hooks`
#
branch.Branch.hooks.install_named_hook('post_change_branch_tip', post_change_branch_tip, 'linci post change hook')
Install this with
python /usr/lib/python2.7/dist-packages/bzrlib/plugins/linci_hook.py
Remember the script runs with the Linux permissions of the user that did the submit.
So now
bzr push
Sends latest changes and my linci server builds then and publishes a .deb as it did with Perforce.
Now I need a Sunday without a hangover to port the rest of my projects. It should be a scriptable process.
References:
https://launchpad.net/bzr-fastimport
http://doc.bazaar.canonical.com/bzr-0.12/server.htm
https://launchpad.net/bzrp4
http://people.canonical.com/~mwh/bzrlibapi/bzrlib.branch.html
Plus some help from folks on #bzr channel on irc.freenode.net
