淘宝商城(天猫)高级技术专家.3年研发+3年性能测试调优/系统测试+4年团队管理与测试架构、研发系统实践. 新舞台新气象, 深化测试基础架构及研发架构,希望能在某个技术领域成为真正的技术大牛。欢迎荐才http://bbs.51testing.com/viewthread.php?tid=120496&extra=&page=1 .邮件: jianzhao.liangjz@alibaba-inc.com,MSN:liangjianzhao@163.com.微博:http://t.sina.com.cn/1674816524

应用Buildbot实施c++持续集成

上一篇 / 下一篇  2009-07-15 01:33:39 / 个人分类:自动化测试框架实现与优化

参考: 

http://perl-qa.hexten.net/wiki/index.php/Buildbot

http://blog.chinaunix.net/u2/68938/showart_1076484.html

http://blog.csdn.net/yurenjimi/archive/2008/12/01/3413829.aspx

http://www.lothar.com/repos/buildbot-website/manual-0.7.5.html#Compile

 

 

buildbot在阿里大型c++研发项目中已经被证明是高效的持续集成运转的重要一环,正发挥check in build/daily build/daily test重要功效.

 

buildbot依赖TwistedPython.

本次的环境(64位的操作系统不支持)

Python2.5.2

Linux b2b_plat_13672.6.9-42.ELsmp #1 SMP Wed Jul 12 23:27:17 EDT 2006 i686 i686 i386 GNU/Linux

 

 

为了邮件提醒,需要先安装sendmail并且配置成功

[root@b2b_plat_1367 ~]# rpm -qa |grep sendmail

sendmail-8.13.1-3.RHEL4.5

sendmail-cf-8.13.1-3.2.el4

 

 

1.     下载安装Twisted

wget -c http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/Twisted-8.1.0.tar.bz2
 bzip2 -d Twisted-8.1.0.tar.bz2

tar -xvf Twisted-8.1.0.tar
 cd Twisted-8.1.0

python setup.py install --prefix=$HOME

    

Twisted依赖zope.interface and Python


wget  -c
http://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz

tar -zxvf zope.interface-3.3.0.tar.gz

cd zope.interface-3.3.0
python setup.py build
python setup.py install --prefix=$HOME

2
 安装 buildbot 0.7.7
wget -c http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.7.tar.gz#md5=4647a15a9d9b2db6a5b3493ac78e11d9
tar -zxvf buildbot-0.7.7.tar.gz

cd buildbot-0.7.7
python setup.py build
python setup.py install --home=~

设置环境变量:

export PYTHONPATH=$HOME/lib/python:/home/liangjz/lib/python2.5/site-packages/

 

 

3测试

 [liangjz@b2b_plat_1367 buildbot_master]$ buildbot --version

Buildbot version:0.7.7

Twisted version:8.1.0

 

4生成服务器端配置文件


mkdir ~/buildbot_master
buildbot create-master buildbot_master

 

 

请特别注意大型应用中svn扫描时间及缓冲时间应该设置长些.

同时确保Makefile目录结构正确,否则会导致make进程死循环,导致cpu%高达80%无法响应.

 

[liangjz@b2b_plat_1367 buildbot_master]$ cat master.cfg | grep -v '#'

 

 

 

 

c = BuildmasterConfig = {}

 

 

from buildbot.buildslave import BuildSlave

 

c['slaves'] = [BuildSlave("bot1name", "bot1passwd")]

 

 

 

c['slavePortnum'] = 9989

 

 

 

 

from buildbot.changes.svnpoller import SVNPoller  

c['change_source'] = SVNPoller("http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects",

                   svnuser='myusername', svnpasswd='mypassword', 

                   pollinterval=5)       

 

 

 

 

 

 

 

from buildbot.scheduler import Scheduler

c['schedulers'] = []

c['schedulers'].append(Scheduler(name="all", branch=None,

                                treeStableTimer=2*60,

                                builderNames=["buildbot-full"]))

 

 

 

 

 

 

 

from buildbot.process import factory

from buildbot.steps.source import SVN

from buildbot.steps.shell import Compile

from buildbot.steps.python_twisted import Trial

from buildbot.steps import source, shell

from buildbot.steps.shell import ShellCommand, WithProperties

 

 

 

 

 

 

 

 

f1 = factory.BuildFactory()

f1.addStep(SVN(mode='update', baseURL='http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects',   defaultBranch=''))

f1.addStep(Compile(command=["make"]))

 

f1.addStep(ShellCommand, command=["make", "test"])

 

 

b1 = {'name': "buildbot-full",

     'slavename': "bot1name",

     'builddir': "full",

     'factory': f1,

     }

c['builders'] = [b1]

 

 

 

 

c['status'] = []

 

from buildbot.status import html

c['status'].append(html.WebStatus(http_port=8010))

 

 

from buildbot.status import mail

 

c['status'].append(mail.MailNotifier(fromaddr="jianzhao.liangjz@alibaba-inc.com",                                   mode="all",                                   extraRecipients=["jianzhao.liangjz@alibaba-inc.com","teson.jinz@alibaba-inc.com"],                                   sendToInterestedUsers=False))

 

 

 

 

 

 

 

 

c['projectName'] = "Buildbot"

c['projectURL'] = "http://buildbot.sourceforge.net/"

 

 

c['buildbotURL'] =http://localhost:8010/

 

 

启动服务器:

 cd buildbot_master

buildbot start .

 

 

5配置客户端:

 

buildbot create-slave --umask=022 ~/buildslave b2b_plat_1367:9989    bot1name  bot1passwd

 

 

buildbot create-slave build_test/ localhost:8010   bot1name  bot1passwd

 

 

启动客户端:

 

cd   buildslave

buildbot start .

 

 

6造一个c++小程序

 

 

[liangjz@b2b_plat_1367 maven_proj]$ cat Makefile

PUBLIC_PATH=../../

 

LDPATH=\

       -L.

 

INCLUDE=\

       -I.

 

 

CFLAG = -c -g -Wall $(INCLUDE)

#CFLAG = -c -O3 $(INCLUDE)

 

GCC = gcc

.SUFFIXES: .o .cpp

 

OBJS=run.o

      

 

 

all: run

 

 

run: $(OBJS)

       $(GCC)  -o $@  $(OBJS)

 

.cpp.o:

       $(GCC) $(CFLAG) -o $@ $<

 

clean:

       rm run

       rm run.o

 

test:

       `pwd`/run

 

 

 

更改提交

Svn ci Makefile -m  “test”

 

代码变更被svnpoll检测到.

 

7查看结果:

 

http://10.20.136.7:8010/waterfall

 

 

8检查中间build过程:

 

[liangjz@b2b_plat_1367 build]$ pwd

/home/liangjz/buildslave/full/build

[liangjz@b2b_plat_1367 build]$ ll -al

total 40

drwxr-xr-x 4 liangjz liangjz 4096 Jul 11 17:07 .

drwxr-xr-x 3 liangjz liangjz 4096 Jul 11 17:07 ..

-rw-r--r-- 1 liangjz liangjz  69 Jul 11 17:07 .buildbot-sourcedata

-rw-r--r-- 1 liangjz liangjz 313 Jul 11 17:07 Makefile

-rwxr-xr-x 1 liangjz liangjz 4685 Jul 11 17:07 run

-rw-r--r-- 1 liangjz liangjz 100 Jul 11 17:07 run.c

-rw-r--r-- 1 liangjz liangjz 880 Jul 11 17:07 run.o

drwxr-xr-x 6 liangjz liangjz 4096 Jul 11 17:07 simple-webapp

drwxr-xr-x 6 liangjz liangjz 4096 Jul 11 17:07 .svn

 

 

9)收取邮件通知:

The Buildbot has finished a build of buildbot-full on Buildbot.

Full details are available at:

 http://10.20.136.7:8010/builders/buildbot-full/builds/8

 

Buildbot URL:http://10.20.136.7:8010/

 

Buildslave for this Build: bot1name

 

Build Reason:

Build Source Stamp: 16520

Blamelist: liangjz

 

Build succeeded!

 

sincerely,

 -The Buildbot

 

 

 

特别注意事项:

 

1)     buildslave需要单独测试,确保代码外部依赖库都ok

2)     makefile需要考虑是放到buildslave/full/build 目录,所以Makefile文件确保无路径依赖

 

另外,为方便整体项目编译,最好有一个Makefile做总控 类似

 

[liangjz@b2b_plat_1367 dragoon2.0]$ vi Makefile

 

all:

       cd depend/framework;make clean;make;cp libframework.a ../../lib/

       cd src/common;make clean;make

       cd src/master;make clean;make

       cd src/monitor;make clean;make

       cd src/agent;make clean;make

       cp /home/liangjz/dragoon2.0/bin/*   /home/liangjz/buildslave/full/build/bin/

 

3)     make test同样需要确认外部的文件依赖及目录结构正确

4)     为了统一发送自定义的邮件,需要c++使用统一的单元测试报告格式.

 


TAG: buildbot

引用 删除 ygf616   /   2013-07-04 10:34:47
我最近在使用buildbot集成C#,构建过程有点问题,想请教你一下。可以留下你的联系方式吗?
我的电话:18612051472,qq:459308708
引用 删除 bugmenot   /   2010-10-08 16:29:49
我们用Hudson和Topo做持续集成,也不错
阿里巴巴一个测试架构师 引用 删除 liangjz   /   2009-11-11 21:17:25
64位linux 上启动后twist.log错误:
2009-11-11 20:28:22+0800 [-] _finished : [Failure instance: Traceback (failure with no frames): <type 'exceptions.IOError'>: got stderr: 'Upon execvpe svn [\'svn\', \'info\', \'--xml\', \'--non-interactive\', \'http://svn.alibaba-inc.com/repos/ali_QA/11_Test_Tools/liangjz\', \'--username=jianzhao.liangjz\', \'--password=5211314\'] in environment id 12030112\n:Traceback (most recent call last):\n  File "/root/lib/python2.5/site-packages/twisted/internet/process.py", line 394, in _fork\n    executable, args, environment)\n  File "/root/lib/python2.5/site-packages/twisted/internet/process.py", line 440, in _execChild\n    os.execvpe(executable, args, environment)\n  File "/usr/local/lib/python2.5/os.py", line 362, in execvpe\n    _execvpe(file, args, env)\n  File "/usr/local/lib/python2.5/os.py", line 389, in _execvpe\n    func(fullname, *argrest)\nOSError: [Errno 2] No such file or directory\n'
        ]
通过在master.cfg的条项:  c['change_source'] = SVNPoller 明确指定svn路径解决
c['change_source'] = SVNPoller("http://svn.alibaba-inc.com/repos/ali_QA/11_Test_Tools/liangjz/",
                    svnuser='jianzhao.liangjz', svnpasswd='5211314', svnbin="/usr/local/bin/svn",
                    pollinterval=30)
阿里巴巴一个测试架构师 引用 删除 liangjz   /   2009-07-18 22:28:42
http://code.google.com/p/chromium/wiki/BuildYourOwnBuildServer

可以看到样例
 

评分:0

我来说两句

显示全部

:loveliness: :handshake :victory: :funk: :time: :kiss: :call: :hug: :lol :'( :Q :L ;P :$ :P :o :@ :D :( :)

Open Toolbar