应用Buildbot实施c++持续集成

参考:　

http://perl-qa.hexten.net/wiki/index.php/Buildbot

http://blog.chinaunix.net/u2/68938/showart_1076484.html

http://blog.csdn.net/yurenjimi/archive/2008/12/01/3413829.aspx

http://www.lothar.com/repos/buildbot-website/manual-0.7.5.html#Compile

buildbot在阿里大型c++研发项目中已经被证明是高效的持续集成运转的重要一环,正发挥check in build/daily build/daily test重要功效.

buildbot依赖Twisted和Python.

本次的环境(64位的操作系统不支持)

Python2.5.2

Linux b2b_plat_13672.6.9-42.ELsmp #1 SMP Wed Jul 12 23:27:17 EDT 2006 i686 i686 i386 GNU/Linux

为了邮件提醒,需要先安装sendmail并且配置成功

[root@b2b_plat_1367 ~]# rpm -qa |grep sendmail

sendmail-8.13.1-3.RHEL4.5

sendmail-cf-8.13.1-3.2.el4

1.     下载安装Twisted

wget -c http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/Twisted-8.1.0.tar.bz2
 bzip2 -d Twisted-8.1.0.tar.bz2

tar -xvf Twisted-8.1.0.tar
 cd Twisted-8.1.0

python setup.py install --prefix=$HOME

Twisted依赖zope.interface and Python

wget  -chttp://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz

tar -zxvf zope.interface-3.3.0.tar.gz

cd zope.interface-3.3.0
python setup.py build
python setup.py install --prefix=$HOME

2　安装　buildbot 0.7.7
wget -c http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.7.tar.gz#md5=4647a15a9d9b2db6a5b3493ac78e11d9
tar -zxvf buildbot-0.7.7.tar.gz

cd buildbot-0.7.7
python setup.py build
python setup.py install --home=~

设置环境变量:

export PYTHONPATH=$HOME/lib/python:/home/liangjz/lib/python2.5/site-packages/

3测试

 [liangjz@b2b_plat_1367 buildbot_master]$ buildbot --version

Buildbot version:0.7.7

Twisted version:8.1.0

4生成服务器端配置文件

mkdir ~/buildbot_master
buildbot create-master buildbot_master

请特别注意大型应用中svn扫描时间及缓冲时间应该设置长些.

同时确保Makefile目录结构正确,否则会导致make进程死循环,导致cpu%高达80%无法响应.

[liangjz@b2b_plat_1367 buildbot_master]$ cat master.cfg | grep -v '#'

c = BuildmasterConfig = {}

from buildbot.buildslave import BuildSlave

c['slaves'] = [BuildSlave("bot1name", "bot1passwd")]

c['slavePortnum'] = 9989

from buildbot.changes.svnpoller import SVNPoller  

c['change_source'] = SVNPoller("http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects",

                   svnuser='myusername', svnpasswd='mypassword', 

                   pollinterval=5)       

from buildbot.scheduler import Scheduler

c['schedulers'] = []

c['schedulers'].append(Scheduler(name="all", branch=None,

                                treeStableTimer=2*60,

                                builderNames=["buildbot-full"]))

from buildbot.process import factory

from buildbot.steps.source import SVN

from buildbot.steps.shell import Compile

from buildbot.steps.python_twisted import Trial

from buildbot.steps import source, shell

from buildbot.steps.shell import ShellCommand, WithProperties

f1 = factory.BuildFactory()

f1.addStep(SVN(mode='update', baseURL='http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects',   defaultBranch=''))

f1.addStep(Compile(command=["make"]))

f1.addStep(ShellCommand, command=["make", "test"])

b1 = {'name': "buildbot-full",

     'slavename': "bot1name",

     'builddir': "full",

     'factory': f1,

     }

c['builders'] = [b1]

c['status'] = []

from buildbot.status import html

c['status'].append(html.WebStatus(http_port=8010))

from buildbot.status import mail

c['status'].append(mail.MailNotifier(fromaddr="jianzhao.liangjz@alibaba-inc.com",                                   mode="all",                                   extraRecipients=["jianzhao.liangjz@alibaba-inc.com","teson.jinz@alibaba-inc.com"],                                   sendToInterestedUsers=False))

c['projectName'] = "Buildbot"

c['projectURL'] = "http://buildbot.sourceforge.net/"

c['buildbotURL'] =http://localhost:8010/

启动服务器:

　cd buildbot_master

buildbot start .

5配置客户端:

buildbot create-slave --umask=022 ~/buildslave b2b_plat_1367:9989    bot1name  bot1passwd

和

buildbot create-slave build_test/ localhost:8010   bot1name  bot1passwd

启动客户端:

cd   buildslave

buildbot start .

6造一个c++小程序

[liangjz@b2b_plat_1367 maven_proj]$ cat Makefile

PUBLIC_PATH=../../

LDPATH=\

       -L.

INCLUDE=\

       -I.

CFLAG = -c -g -Wall $(INCLUDE)

#CFLAG = -c -O3 $(INCLUDE)

GCC = gcc

.SUFFIXES: .o .cpp

OBJS=run.o

all: run

run: $(OBJS)

       $(GCC)  -o $@  $(OBJS)

.cpp.o:

       $(GCC) $(CFLAG) -o $@ $<

clean:

       rm run

       rm run.o

test:

       `pwd`/run

更改提交

Svn ci Makefile -m  “test”

代码变更被svnpoll检测到.

7查看结果:

http://10.20.136.7:8010/waterfall

8检查中间build过程:

[liangjz@b2b_plat_1367 build]$ pwd

/home/liangjz/buildslave/full/build

[liangjz@b2b_plat_1367 build]$ ll -al

total 40

drwxr-xr-x 4 liangjz liangjz 4096 Jul 11 17:07 .

drwxr-xr-x 3 liangjz liangjz 4096 Jul 11 17:07 ..

-rw-r--r-- 1 liangjz liangjz  69 Jul 11 17:07 .buildbot-sourcedata

-rw-r--r-- 1 liangjz liangjz 313 Jul 11 17:07 Makefile

-rwxr-xr-x 1 liangjz liangjz 4685 Jul 11 17:07 run

-rw-r--r-- 1 liangjz liangjz 100 Jul 11 17:07 run.c

-rw-r--r-- 1 liangjz liangjz 880 Jul 11 17:07 run.o

drwxr-xr-x 6 liangjz liangjz 4096 Jul 11 17:07 simple-webapp

drwxr-xr-x 6 liangjz liangjz 4096 Jul 11 17:07 .svn

9)收取邮件通知:

The Buildbot has finished a build of buildbot-full on Buildbot.

Full details are available at:

 http://10.20.136.7:8010/builders/buildbot-full/builds/8

Buildbot URL:http://10.20.136.7:8010/

Buildslave for this Build: bot1name

Build Reason:

Build Source Stamp: 16520

Blamelist: liangjz

Build succeeded!

sincerely,

 -The Buildbot

特别注意事项:

1)     buildslave需要单独测试,确保代码外部依赖库都ok

2)     makefile需要考虑是放到buildslave/full/build　目录,所以Makefile文件确保无路径依赖

另外,为方便整体项目编译，最好有一个Makefile做总控　类似

[liangjz@b2b_plat_1367 dragoon2.0]$ vi Makefile

all:

       cd depend/framework;make clean;make;cp libframework.a ../../lib/

       cd src/common;make clean;make

       cd src/master;make clean;make

       cd src/monitor;make clean;make

       cd src/agent;make clean;make

       cp /home/liangjz/dragoon2.0/bin/*   /home/liangjz/buildslave/full/build/bin/

3)     make test同样需要确认外部的文件依赖及目录结构正确

4)     为了统一发送自定义的邮件,需要c++使用统一的单元测试报告格式.