淘宝商城(天猫)高级技术专家.3年研发+3年性能测试调优/系统测试+4年团队管理与测试架构、研发系统实践. 新舞台新气象, 深化测试基础架构及研发架构,希望能在某个技术领域成为真正的技术大牛。欢迎荐才http://bbs.51testing.com/viewthread.php?tid=120496&extra=&page=1 .邮件: jianzhao.liangjz@alibaba-inc.com,MSN:liangjianzhao@163.com.微博:http://t.sina.com.cn/1674816524

发布新日志

  • 极具挑战性的测试领域

    2009-09-11 19:47:37

     测试工程师做到一定阶段,需要极强的创新能力及发散能力.

    可以考虑做的事情(部分为加分项):

     

    1) 架构及技术方案,设计评审

    如评审算法选择合理性,架构是否考虑可扩展性...

    2) bug 定位及提出fixed bug建议

    3) 性能诊断及瓶颈定位, 就是 profile 能力. 以及性能调优,容量建模及预测

    4) 大型复杂项目自动化框架建设

    5)复杂项目回归测试范围界定

          如从字节码角度出发追溯

    6)安全测试,fuzz测试及漏洞修复

    7)应对复杂多样的客户端os,浏览器,编程语言及服务器集群的大型分布式跨平台的应用测试. 如云计算平台

    8)根据bug 以及漏测缺陷, 反推残留的bug 地图

    9) 软件质量度量的维度及落地

    10)作为pm backup,从全局角度驱动项目质量提升

    11)研发测试代码 code review...

    12)...

     

     

      综合以上方向,测试做到高端,和研发高端是重合的.  需要具备研发的深厚功底以及测试的敏锐嗅觉,全局视觉, 极具挑战性

     

           事情做到极致,就能不断获得成就感….

     

  • 在测试效率和质量间寻找平衡支点

    2009-09-09 00:39:27

     

     测试当中,当系统隐Bug含的少于一定的数量时候,每发现一个bug付出的时间成本远高于系统刚提交QA测试的时刻.
    怎么判断这个拐点落在何处呢? 如果有合适工具,度量判断成本如何?适用范围是否足够? 误差有多大? 这些问题都是需要寻求答案的.
     
     那我们是否可从另外一些维度考虑?
      
     1) 降低产品发布标准. 比如允许最低级别的Bug< 一定数量时可以上线, 或者降低测试覆盖率.
     2) 提高线上故障的容忍度. 比如定义最低级别的1类故障容忍逃逸.

     上述都可能影响用户体验.

     从比较正面的角度出发考虑,
     
     1) 全方位的自动化测试, 包括web ui自动化/接口测试自动化/后台linux shell自动化/模块级自动化. 再结合cruisecontrol/continumm/ buildbot等持续集成工具
     实施持续集成,快速弹出bug ,大幅度提高测试效率.
     
     由于性能调优过程职责不够明晰,优先完成性能测试过程优化.
     
     2)回归测试范围有效界定, 比如研发出判断回归影响范围以及路径的工具, 回归测试不必整体软件全部回归
     3)针对sqa的质量月报,针对当月最影响测试效率的点重点突破
     4)分工精细化,不必每一个人都是全能型选手 ,降低学习成本. 初步划分为 linux  c++/java自动化/,持续集成,缺陷预防与故障分析, 性能测试,api用例设计等子领域
     5)知识库系统补充,充分应用wiki/svn ,bocked的时候可以第一时间搜索到答案
     6) 不断提高研发提交QA 测试门槛,比如代码覆盖率>50%并且过程持续保持水准

     7) 长期致力于测试开发方面的架构和前沿方法探索
     ..)

  • 提高研发提交测试的门槛的理想版

    2009-08-22 09:12:47

     

    1 每次check in svn之前在review board上完成Code Review(含正式代码及测试代码), commit代码时记录review boardreview id,SCM审计.

     

    SQL审核在提交QA测试需要完成

     

    2 单元测试代码覆盖率>60%[qa过程中参与测试代码review,确保单元测试代码符合规范],每次接收测试时研发单元测试代码用例通过率100%, QA执行验证.

     

    如果每退回一次,失败的测试用例需增加必要的测试代码覆盖

     

    3 完成持续集成且 build 成功

    Java continuum

    C++buildbot

     

    4 pm需经过流程培训, 并且来自中立方

     

    5 约定>10个冒烟点, 每次提交时冒烟测试通过率100%.

    对于失败的用例,研发需要增加测试代码覆盖

     

  • c/c++工程makefile及目录层次约束

    2009-08-22 08:33:53

     

       在尝试buildbot 做c++持续集成以及加入代码覆盖率度量所需的gcc 参数时,  makefile以及源代码目录需要做一些调整才能适应.  makefile 及目录组织需要规范化.

        makefile规范:
    1)不能硬编码目录路径,可以通过环境变量
    2)一个工程涉及多个平台( linux /aix/ solaris ) , 能够支持条件编译部分代码
    3)Gcc 需要条件支持 :    优化o   ,  调试 –g   ,  以及 代码覆盖   -fprofile-arcs -ftest-coverage

        gcov/lcov目录层次约束:  test目录的main函数所在.cpp目录层次不高于依赖的 .h 及.c/.cpp文件

  • 持续集成开展的必需条件

    2009-08-11 22:12:57

       根据当下在产品型测试实践梳理的几点想法:

    一 人才

    a 配管

    懂常用持续集成服务器,如cruisecontrol/continuum, buildbot 等原理及安装配置.
    另外,需要了解java 编译,c++ makefile,subversion等配置管理知识.

    b 研发/QA:  合格的. 最最重要的是有良好的质量意识及实践能力.

    c 主管/经理:
     
     他们是有力后盾,当涉及资源及机制时,是一锤定音者.
     
    二 技术

    最核心的是良好的单元测试编码, 集成测试编码,系统测试编码, web ui层自动化等不同level的自动化能力.

    为了确保单元测试代码有效性,可以考虑:

    1) 对单元测试代码做code review
    2) 研发交叉编写单元测试代码
    3) QA参与单元测试设计 或者编写单元测试代码

    研发编码水平,QA测试开发能力都应该在一个较高水位.

    三机制

     实践中,这个也是影响面最大的, 没有好的机制粘合,要做好持续集成基本不现实.

    在当下,我面临的问题:

    1)     源代码规范及makefile规范,  避免硬编码以满足buildbot需求

    2)     统一c++单元测试框架. 

    3)     统一单元测试输出结果

    4)     单元测试代码编写方面做更多摸索( 研发交叉编写单元测试?QA前期介入单元测试设计?...).    涉及到资源计划

    5)     集成测试方面的探索, 减少外部依赖

    6)      对buildbot /continuum 结果的快速响应以及相关的约定 ( 进入集成测试后,build 失败次数<n 以及每次build修复时间<n小时)

    7)     研发提交QA的标准, >50%代码覆盖率应在持续集成后一直保持并且经过有效review

    8)     项目退出标准细化 ( 需要保证连续n天0个高优先级bug 并且持续集成连续成功n天以上)

    9)...


    四 设备

    用钱能解决的问题最容易解决.^_^

    当同时做持续集成时,需要确保编译,运行测试验证,发送报告时间不要太长.

     

  • 应用Buildbot实施c++持续集成

    2009-07-15 01:33:39

    参考: 

    http://perl-qa.hexten.net/wiki/index.php/Buildbot

    http://blog.chinaunix.net/u2/68938/showart_1076484.html

    http://blog.csdn.net/yurenjimi/archive/2008/12/01/3413829.aspx

    http://www.lothar.com/repos/buildbot-website/manual-0.7.5.html#Compile

     

     

    buildbot 在阿里大型c++研发项目中已经被证明是高效的持续集成运转的重要一环,正发挥check in build/daily build/daily test重要功效.

     

    buildbot依赖TwistedPython.

    本次的环境(64位的操作系统不支持)

    Python 2.5.2

    Linux b2b_plat_1367 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:27:17 EDT 2006 i686 i686 i386 GNU/Linux

     

     

    为了邮件提醒,需要先安装sendmail 并且配置成功

    [root@b2b_plat_1367 ~]# rpm -qa |grep sendmail

    sendmail-8.13.1-3.RHEL4.5

    sendmail-cf-8.13.1-3.2.el4

     

     

    1.      下载安装 Twisted

    wget -c http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/Twisted-8.1.0.tar.bz2
     bzip2 -d Twisted-8.1.0.tar.bz2

    tar -xvf Twisted-8.1.0.tar
     cd Twisted-8.1.0

    python setup.py install --prefix=$HOME

        

    Twisted 依赖 zope.interface and Python


    wget   -c
    http://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz

    tar  -zxvf  zope.interface-3.3.0.tar.gz

    cd zope.interface-3.3.0
    python setup.py build
    python setup.py install --prefix=$HOME

    2
     安装 buildbot 0.7.7
    wget -c http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.7.tar.gz#md5=4647a15a9d9b2db6a5b3493ac78e11d9
    tar -zxvf  buildbot-0.7.7.tar.gz

    cd buildbot-0.7.7
    python setup.py build
    python setup.py install --home=~

    设置环境变量:

    export PYTHONPATH=$HOME/lib/python:/home/liangjz/lib/python2.5/site-packages/

     

     

    3 测试

     [liangjz@b2b_plat_1367 buildbot_master]$ buildbot --version

    Buildbot version: 0.7.7

    Twisted version: 8.1.0

     

    4 生成服务器端配置文件


    mkdir ~/buildbot_master
    buildbot create-master buildbot_master

     

     

    请特别注意大型应用中svn扫描时间及缓冲时间应该设置长些.

    同时确保Makefile 目录结构正确,否则会导致make 进程死循环,导致cpu%高达80%无法响应.

     

    [liangjz@b2b_plat_1367 buildbot_master]$ cat master.cfg | grep -v '#'

     

     

     

     

    c = BuildmasterConfig = {}

     

     

    from buildbot.buildslave import BuildSlave

     

    c['slaves'] = [BuildSlave("bot1name", "bot1passwd")]

     

     

     

    c['slavePortnum'] = 9989

     

     

     

     

    from buildbot.changes.svnpoller import SVNPoller  

    c['change_source'] = SVNPoller("http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects",

                        svnuser='myusername', svnpasswd='mypassword', 

                        pollinterval=5)       

     

     

     

     

     

     

     

    from buildbot.scheduler import Scheduler

    c['schedulers'] = []

    c['schedulers'].append(Scheduler(name="all", branch=None,

                                     treeStableTimer=2*60,

                                     builderNames=["buildbot-full"]))

     

     

     

     

     

     

     

    from buildbot.process import factory

    from buildbot.steps.source import SVN

    from buildbot.steps.shell import Compile

    from buildbot.steps.python_twisted import Trial

    from buildbot.steps import source, shell

    from buildbot.steps.shell import ShellCommand, WithProperties

     

     

     

     

     

     

     

     

    f1 = factory.BuildFactory()

    f1.addStep(SVN(mode='update', baseURL='http://svn.alibaba-inc.com/repos/ali_QA/20_Scripts/06_maven_projects',    defaultBranch=''))

    f1.addStep(Compile(command=["make"]))

     

    f1.addStep(ShellCommand, command=["make", "test"])

     

     

    b1 = {'name': "buildbot-full",

          'slavename': "bot1name",

          'builddir': "full",

          'factory': f1,

          }

    c['builders'] = [b1]

     

     

     

     

    c['status'] = []

     

    from buildbot.status import html

    c['status'].append(html.WebStatus(http_port=8010))

     

     

    from buildbot.status import mail

     

    c['status'].append(mail.MailNotifier(fromaddr="jianzhao.liangjz@alibaba-inc.com",                                    mode="all",                                    extraRecipients=["jianzhao.liangjz@alibaba-inc.com","teson.jinz@alibaba-inc.com"],                                    sendToInterestedUsers=False))

     

     

     

     

     

     

     

     

    c['projectName'] = "Buildbot"

    c['projectURL'] = "http://buildbot.sourceforge.net/"

     

     

    c['buildbotURL'] = http://localhost:8010/

     

     

    启动服务器:

      cd buildbot_master

    buildbot start .

     

     

    5 配置客户端:

     

    buildbot create-slave --umask=022 ~/buildslave b2b_plat_1367:9989     bot1name   bot1passwd

     

     

    buildbot create-slave build_test/ localhost:8010    bot1name   bot1passwd

     

     

    启动客户端:

     

    cd    buildslave

    buildbot start .

     

     

    6 造一个c++小程序

     

     

    [liangjz@b2b_plat_1367 maven_proj]$ cat Makefile

    PUBLIC_PATH=../../

     

    LDPATH=\

            -L.

     

    INCLUDE=\

            -I.

     

     

    CFLAG = -c -g -Wall $(INCLUDE)

    #CFLAG = -c -O3 $(INCLUDE)

     

    GCC = gcc

    .SUFFIXES: .o .cpp

     

    OBJS=run.o

          

     

     

    all: run

     

     

    run:  $(OBJS)

            $(GCC)   -o $@   $(OBJS)

     

    .cpp.o:

            $(GCC) $(CFLAG) -o $@ $<

     

    clean:

            rm  run

            rm  run.o

     

    test:

            `pwd`/run

     

     

     

    更改提交

    Svn ci  Makefile  -m   “test”

     

    代码变更被svnpoll检测到.

     

    7 查看结果:

     

    http://10.20.136.7:8010/waterfall

     

     

    8 检查中间build过程:

     

    [liangjz@b2b_plat_1367 build]$ pwd

    /home/liangjz/buildslave/full/build

    [liangjz@b2b_plat_1367 build]$ ll -al

    total 40

    drwxr-xr-x  4 liangjz liangjz 4096 Jul 11 17:07 .

    drwxr-xr-x  3 liangjz liangjz 4096 Jul 11 17:07 ..

    -rw-r--r--  1 liangjz liangjz   69 Jul 11 17:07 .buildbot-sourcedata

    -rw-r--r--  1 liangjz liangjz  313 Jul 11 17:07 Makefile

    -rwxr-xr-x  1 liangjz liangjz 4685 Jul 11 17:07 run

    -rw-r--r--  1 liangjz liangjz  100 Jul 11 17:07 run.c

    -rw-r--r--  1 liangjz liangjz  880 Jul 11 17:07 run.o

    drwxr-xr-x  6 liangjz liangjz 4096 Jul 11 17:07 simple-webapp

    drwxr-xr-x  6 liangjz liangjz 4096 Jul 11 17:07 .svn

     

     

    9)收取邮件通知:

    The Buildbot has finished a build of buildbot-full on Buildbot.

    Full details are available at:

     http://10.20.136.7:8010/builders/buildbot-full/builds/8

     

    Buildbot URL: http://10.20.136.7:8010/

     

    Buildslave for this Build: bot1name

     

    Build Reason:

    Build Source Stamp: 16520

    Blamelist: liangjz

     

    Build succeeded!

     

    sincerely,

     -The Buildbot

     

     

     

    特别注意事项:

     

    1)      buildslave需要单独测试,确保代码外部依赖库都ok

    2)      makefile需要考虑是放到buildslave/full/build 目录,所以Makefile文件确保无路径依赖

     

    另外,为方便整体项目编译,最好有一个Makefile做总控 类似

     

    [liangjz@b2b_plat_1367 dragoon2.0]$ vi Makefile

     

    all:

            cd depend/framework;make clean;make;cp libframework.a ../../lib/

            cd src/common;make clean;make

            cd src/master;make clean;make

            cd src/monitor;make clean;make

            cd src/agent;make clean;make

            cp /home/liangjz/dragoon2.0/bin/*    /home/liangjz/buildslave/full/build/bin/

     

    3)      make test 同样需要确认外部的文件依赖及目录结构正确

    4)      为了统一发送自定义的邮件,需要c++使用统一的单元测试报告格式.

     

  • linux sendmail应用

    2009-07-15 01:16:04

    需要安装sendmail sendmail-cf

     

    http://rpm.pbone.net/index.php3/stat/4/idpl/4203610/com/sendmail-cf-8.13.1-3.2.el4.i386.rpm.html

     

    http://cnaning.javaeye.com/blog/350143

    http://blog.chinaunix.net/u/12442/showart_1928452.html

     

    安装sendmail-cf

     

    Rpm –ihv   sendmail-cf-8.13.1-3.2.el4.i386.rpm.html

     

    启动sendmail(速度很慢)

    Service sendmail start

    [root@b2b_plat_1367 ~]# service sendmail start

    Starting sendmail:

     

     

     

     

     

     

    [  OK  ]

    Starting sm-client:

     

     

     

     

    [  OK  ]

     

     

    发邮件测试:

    mail -s "src test"  jianzhao.liangjz@alibaba-inc.com   < /var/log/maillog

     

     

    检查日志: tail –f /var/log/maillog

     

  • unitils数据库测试实例3

    2009-06-10 22:28:13

    UserDAOTest.java读取的test-common文件内容 :  <?xml version="1.0" encoding="GB2312"?><beans default-autowire="byName" xmlns="http://www.springframework.org/schema/beans"

             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

             xmlns:aop="http://www.springframework.org/schema/aop"

             xmlns:tx="http://www.springframework.org/schema/tx"

             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd

               http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd

               http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.0.xsd">

     

      <!-- for unitils test, read unitls.properties -->

      <bean id="dataSource" class="org.unitils.database.UnitilsDataSourceFactoryBean" /> 

        <bean id="transactionTemplate" class="org.springframework.transaction.support.TransactionTemplate"/>

     

      

       <bean id="sqlMapClient"

          class="org.springframework.orm.ibatis.SqlMapClientFactoryBean"  scope="singleton">

        <property name="configLocation" value="classpath:resources/sqlmap-config.xml" />

        <property name="dataSource" ref="dataSource" />

      </bean>

     

      <bean id="amoebaDao" class="com.ali.amoeba.brmmstest.ibatis.IBatisAmoebaDao">

        <property name="sqlMapClient" ref="sqlMapClient" />

      </bean>

     

      <bean id="transactionManager"

          class="org.springframework.jdbc.datasource.DataSourceTransactionManager"  scope="singleton">

        <property name="dataSource" ref="dataSource" />

      </bean>

     

        <bean id="baseSqlMapClientDAO" abstract="true">

            <property name="dataSource">

                <ref local="dataSource"/>

            </property>

            <property name="sqlMapClient">

                <ref local="sqlMapClient"/>

            </property>

        </bean> 

    </beans>  

    以上变化仅仅是    <bean id="dataSource" class="org.unitils.database.UnitilsDataSourceFactoryBean" />

     

    就是UnitilsDataSourceFactoryBean 去读取unitils.properities文件驱动程序及控制项目.其他的和spring testcontext读取的内容完全一样.   

    运行后amoeba数据库自动生成dbmaintain_scripts,可以清晰看出执行的脚本版本.

     

  • unitils数据库测试实例2

    2009-06-10 22:20:31

    UserDAOTest.java

     

    package com.ali.amoeba.brmmstest;

    import java.util.Arrays;

    import java.util.List;

    import org.junit.Ignore;

    import org.junit.Test;

    import org.springframework.context.ConfigurableApplicationContext;

    import org.springframework.context.support.ClassPathXmlApplicationContext;

    import org.unitils.UnitilsJUnit4;

    import org.unitils.database.annotations.Transactional;

    import org.unitils.database.util.TransactionMode;

    import org.unitils.dbunit.annotation.DataSet;

    import org.unitils.dbunit.annotation.ExpectedDataSet;

    import org.unitils.dbunit.datasetloadstrategy.impl.CleanInsertLoadStrategy;

    import org.unitils.dbunit.datasetloadstrategy.impl.InsertLoadStrategy;

    import org.unitils.reflectionassert.ReflectionAssert;

    import org.unitils.spring.annotation.SpringApplicationContext;

    import org.unitils.spring.annotation.SpringBeanByType;

    import com.ali.amoeba.brmmstest.dao.AmoebaDao;

    import com.ali.amoeba.brmmstest.dataobject.User;

    import org.junit.runner.RunWith;

    import org.unitils.UnitilsJUnit4TestClassRunner;

    @RunWith(UnitilsJUnit4TestClassRunner.class)

     

    @SpringApplicationContext({"classpath:resources/test-common.xml"})

     

    public class UserDAOTest { // extends UnitilsJUnit4 {

     

        /*用标注或者overwrite  createApplicationContext 方法都可以

        @SpringApplicationContext

        public ConfigurableApplicationContext createApplicationContext() {

           return new ClassPathXmlApplicationContext(new String[]{"classpath:resources/test-common.xml"});

        }

        */

     

        //从配置文件里面读取spring bean 的数据源

        protected AmoebaDao userDao;

        @SpringBeanByType

        public void setAmoebaDao(AmoebaDao userDao) {

           this.userDao = userDao;

        }

     

        @Test

        @DataSet(loadStrategy = CleanInsertLoadStrategy.class)

        @ExpectedDataSet

        public void testFindByName() {

           User user=new User();

           user.setFirst("doe");

           user.setLast("john");

           User result = userDao.findByName(user);

           System.out.println(result);

           ReflectionAssert.assertPropertyLenEquals("first", "doe", result);

        }

        @Transactional(TransactionMode.COMMIT)

        @Test

        public void testInsertUser() {

           User user=new User();

           user.setId(7777);

           user.setFirst("good");

           user.setLast("xxxx");

           user.setAge(31);

           userDao.insertUser(user);

        }

     

        //用一个xml数据源数据存储到db,过滤部分数据.然后与预定的另外一部分数据对比

        //达到数据可控

        //每次做之前该表数据都清空

        @Test

        @DataSet("UserDAOTest.testFindByMinimalAge.xml")

        @ExpectedDataSet("UserDAOTest.testFindByName-result.xml")

        public void testFindByMinimalAge() {

           List<User> result = userDao.findByMinimalAge(18);

           ReflectionAssert.assertPropertyLenEquals("first", Arrays.asList("methodDataset"), result);

        }

    }  

    UserDAOTest.xml内容都形如:

    <?xml version='1.0' encoding='UTF-8'?> 

    <dataset> 

          <user id="11"  first="doe"   last="john"   age="20" /> 

          <user id="1"  first="good"   last="bad"   age="10" /> 

    </dataset>

  • UNITILS数据库测试实例

    2009-06-10 20:47:22

    http://www.dbmaintain.org/overview.html

    http://www.slideshare.net/nevenfi/unit-testing-unitils-dbmaintain

    国外PPT : http://www.parleys.com/display/PARLEYS/Unitils

     

    http://www.slideshare.net/nevenfi/unit-testing-unitils-dbmaintain

     

    Unitils 2: unitils is dead, long live to spring-test

     

    http://203.208.37.132/search?q=cache:qwFKcsUwyLwJ:java-chimaera.blogspot.com/2008/12/unitils-2-unitils-is-dead-long-live-to.html+unitils&cd=43&hl=zh-CN&ct=clnk&gl=cn&st_usg=ALhdy2_5Qbp4RG814rODpTrovRqbSGjXHg

     

    http://blog.csdn.net/phphot/archive/2009/03/02/3949340.aspx

    http://www.blogjava.net/wangzhouyu/archive/2008/04/07/191137.html

    http://code.google.com/p/unitils-idea/

    http://rdc.taobao.com/blog/qa/?p=1895

     

    http://andyao.javaeye.com/blog/180231   unitils结合dbdeploy测试

     

    http://www.devx.com/Java/Article/35129

     

    样例代码 : http://www.devx.com/Java/Article/35129/1954

     

    Unitils 最核心的就是数据库DB xml 文件之间互相映射.

    Unitils 需要下载with-dependency的包 下面的例子用spring testcontext也能部分实现访问数据库.

    .

    有几个刚性要求 :

    1)      unitils.properities放在src同级目录

    2)      dataset xml文件默认放在和该类同一个目录

    3)      @ExpectedDataSet  对应的文件名命名规则及同目录存放

    public class UserDAOTest extends UnitilsJUnit4 {

     

        @Test @ExpectedDataSet

        public void testInactivateOldAccounts() {

            userDao.inactivateOldAccounts();

        }

    }

    Note that we have added the @ExpectedDataSet to the test method. This will instruct Unitils to look for a data set file named UserDAOTest.testInactivateOldAccounts-result.xml and compare the contents of the database with the contents of the data set:

     

    4)      Dbscripts目录下的自动化脚本命令规则

     

    The directory in which the scripts are located can be configured with following property:

    dbMaintainer.script.locations=myproject/dbscripts

    Multiple directories can be specified, separated by commas. Subdirectories are also scanned for script. files. All scripts are required to follow a particular naming pattern: they start with a version number, followed by an underscore and end with '.sql' (the supported extensions are configurable). The example below shows some typical scripts names. The leading zeroes in the example are not required, they are only added for convenience, to make sure they are shown in proper sequence in a file explorer window.

    dbscripts/ 001_initial.sql

               002_tracking_updates.sql

               003_auditing_updates.sql

    Suppose you add a new script, this time with version number 4: 004_create_user_admin_tables.sql. The next time you execute a database test, the database maintainer will notice that the database structure is no longer up to date. It will update the database schema incrementally by executing all of the new scripts, in this case only 004_create_user_admin_tables.sql.

     

    采用spring2.5+unitils 实现一个数据库的测试代码,最后生成的文件目录 :

         

     

    上述dboutput文件是第一次执行时由unitils自动导出的.

     

     

    工程的Java build path unitils.jar 及依赖的spring,dbunit,mysql引擎都加入.

     

    unitils.properities文件内容

     

     

     

    # comments documenting these unitils configuration properties removed for

    # brevity. look for commenting in unitils-default.properties in the root of the

    # unitils jar if needed.

     

    unitils.modules=database,dbunit,easymock,spring,inject

     

    unitils.module.hibernate.enabled=false

    # these placeholders are set in avaje.properties

    database.driverClassName=com.mysql.jdbc.Driver

    database.url=jdbc:mysql://localhost:3306/amoeba

    database.schemaNames=amoeba

    database.userName=root

    database.password=mysql

    database.dialect=mysql

     

    DatabaseModule.Transactional.value.default=commit

    #DbUnitModule.DataSet.loadStrategy.default=org.unitils.dbunit.datasetloadstrategy.InsertLoadStrategy

     

     

     

    # Indicates the database must be recreated from scratch when an already executed script. is updated. If false, the

    # DBMaintainer will give an error when an existing script. is updated.

    dbMaintainer.fromScratch.enabled=true

    # Indicates whether a from scratch update should be performed when the previous update failed, but

    # none of the scripts were modified since that last update. If false a new update will be tried only when

    # changes were made to the script. files.

    dbMaintainer.keepRetryingAfterError.enabled=false

     

    updateDataBaseSchema.enabled=true

    dbMaintainer.autoCreateExecutedScriptsTable=true

    dbMaintainer.useScriptFileLastModificationDates.enabled=true

    # list are recursively searched for files.

     

    #eclipse must be bin/scripts,or

    #Caused by: org.unitils.core.UnitilsException: File location scripts defined in property dbMaintainer.script.locations doesn't exist

     

    dbMaintainer.script.locations=bin/resources/dbscripts

    # Extension of the files containing the database update scripts

    dbMaintainer.script.fileExtensions=sql,ddl

     

    dbMaintainer.generateDataSetStructure.enabled=true

    dataSetStructureGenerator.xsd.dirName=dboutput/

    sequenceUpdater.sequencevalue.lowestacceptable=100

     

     

     

     

     

    # dbMaintainer.script.locations这个地方特别小心,在不同的运行ENV可能结果不同. 可以下载untils代码一起执行. 并且00n代表版本号, 建议在手工删除后tabledbmaintain_scripts; 可以从001开始,否则sql 语句版本从001开始.

     

     

    以上选项都是调试出来的, 特别注意:
    dbMaintainer.fromScratch.enabled=true

    updateDataBaseSchema.enabled=true

     

     

     

    log4j.properities内容:

    log4j.rootLogger=info,CONSOLE

    log4j.addivity.org.apache=true

    log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender

    log4j.appender.Threshold=debug

    org.unitils.appender.Threshold=debug

    log4j.appender.CONSOLE.Target=System.out

    log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout

    log4j.appender.CONSOLE.layout.ConversionPattern=[framework] %d - %c -%-4r [%t] %-5p %c %x - %m%n

     

     

     

  • 搭建CruiseControl2.8.2 +maven2.1+cobertura+checkstyle+findbug持续集成环境2

    2009-06-05 22:56:02

    第二步设置cruisecontrol  config.xml

     

    放置样例程序在: d:/cruisecontrol/maven_projects

     

    根据 http://bbs.scmroad.com/viewthread.php?tid=753&extra=page%3D1%20

    http://bbs.scmroad.com/viewthread.php?tid=758

    http://bbs.scmroad.com/viewthread.php?tid=836&extra=page%3D3

    再精简config.xml

     

     

    集成checkstyle.:

     

    http://confluence.public.thoughtworks.org/display/CC/CheckStyle

     

    集成corbertura:

    http://confluence.public.thoughtworks.org/display/CC/CruiseControlWithCobertura

     

    集成findbug:

    http://mojo.codehaus.org/findbugs-maven-plugin/2.1-SNAPSHOT/usage.html 配置maven pom.xml

    http://confluence.public.thoughtworks.org/display/CC/CruiseControlWithFindBugs

     

     

    集成其他工具:

    http://confluence.public.thoughtworks.org/display/CC/IntegratingWithOtherTools

     

     

    最后的配置文件cruisecontrol 配置文件config.xml内容:

     

    <cruisecontrol>

        <property name="cruisedir" value="D:/CruiseControl"/>  

        <property name="logdir" value="${cruisedir}/logs"/>  

        <property name="checkoutdir" value="${cruisedir}/maven_projects"/>  

        <property name="artifactdir" value="${cruisedir}/artifacts"/>  

        <property name="projectgoal" value="clean site package"/>  

        <property name="M2_EXE" value="D:/maven-2.1.0/bin/mvn.bat"/>  

       

        <plugin name="project">

                       <listeners>  

                <currentbuildstatuslistener file="${logdir}/${project.name}/status.txt"/>  

            </listeners>  

            <bootstrappers>  

                <svnbootstrapper localWorkingCopy="${checkoutdir}/${project.name}" />  

            </bootstrappers>  

            <modificationset quietperiod="30">  

                <svn localWorkingCopy="${checkoutdir}/${project.name}"/>  

            </modificationset>  

            <schedule interval="30">  

                <maven2 mvnscript="${M2_EXE}" 

                        pomfile="${checkoutdir}/${project.name}/pom.xml" 

                        goal="${projectgoal}"/>  

            </schedule>  

            <log dir="${logdir}/${project.name}">  

                <merge dir="${checkoutdir}/${project.name}/target/surefire-reports"/>  

            </log>  

            <publishers>  

                <onsuccess>  

                    <artifactspublisher dest="${artifactdir}/${project.name}"   

                                file="${checkoutdir}/${project.name}/target/{project.name}.jar"/>  

                </onsuccess>  

              

     

            </publishers>  

           

                  </plugin>

     

                  <!--

        <project name="ch03-simple"  />  

                  -->

          

                  <include.projects file="ch05-simple-web-proj.xml"/>

          

    </cruisecontrol>

     

     

    "ch05-simple-web-proj.xml" 内容是:

     

     

     

    <cruisecontrol>

     <property name="MY_PROJ_NAME" value="maven_projects/simple-webapp"/>

     <project name="simple-webapp">  

         

             <log>

                                         <merge file="${MY_PROJ_NAME}/target/checkstyle-result.xml"/>

                                          <merge file="${MY_PROJ_NAME}/target/pmd.xml"/>

                                          <merge dir="${MY_PROJ_NAME}/target/findbugsXml.xml"/>

     

                                </log>

            <schedule interval="40">  

                <maven2 mvnscript="${M2_EXE}" 

                        pomfile="${checkoutdir}/${project.name}/pom.xml" 

                        goal=" -U -Pexternal clean package site"/>  

            </schedule>  

            <publishers>  

                  

                <onsuccess>  

                             

                    <artifactspublisher dest="${artifactdir}/${project.name}"   

                                file="${checkoutdir}/${project.name}/target/simple-webapp.war"/>  

                              

                               

                    <artifactspublisher

                                                     dir="${MY_PROJ_NAME}/target/site" subdirectory="cobertura"

                                                     dest="artifacts/${project.name}" />

      

                </onsuccess>

               

                

            </publishers>  

        </project>

       

        </cruisecontrol>  

     

     

    最后界面展现:

  • 搭建CruiseControl2.8.2+maven2.1+cobertura+checkstyle+findbug持续集成环境

    2009-06-05 22:51:00

    参考:

    http://rdc.taobao.com/blog/qa/?p=1822

    http://rdc.taobao.com/blog/qa/?p=669

    http://emma.sourceforge.net/maven-emma-plugin/goals.html

    http://foxgem.javaeye.com/blog/24620

     

     

    下载: CruiseControl-2.8.2.exe, maven-2.1.0 并参考前述url设置M2_HOME,PATH变量. 

    第一步先配置pom.xml  

    依照maven权威指南  http://www.sonatype.com/books/maven-book/reference_zh/web.html  下载第五章样例simple-webapp

     

    参考http://mojo.codehaus.org/cobertura-maven-plugin/usage.html  配置coberturajetty, checkstyle, pmd, findbug 等相关的  pom.xml.配置完成需要单独测试pom.xml是否正确.

     

    Pom.xml内容:

     

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

      <modelVersion>4.0.0</modelVersion>

      <groupId>org.sonatype.mavenbook.ch05</groupId>

      <artifactId>simple-webapp</artifactId>

      <version>1-SNAPSHOT</version>

      <packaging>war</packaging>

      <name>Chapter 5 Simple Web Application Project</name>

      <url>http://sonatype.com/book</url>

      <dependencies>

        <dependency>

          <groupId>junit</groupId>

          <artifactId>junit</artifactId>

          <version>3.8.1</version>

          <scope>test</scope>

        </dependency>

        <dependency>

          <groupId>org.apache.geronimo.specs</groupId>

          <artifactId>geronimo-servlet_2.4_spec</artifactId>

          <version>1.1.1</version>

        </dependency>

      </dependencies>

     

      <!--

      <parent>

          <artifactId>parent</artifactId>

          <groupId>org.sonatype.mavenbook.ch05</groupId>

          <version>1-SNAPSHOT</version>

      </parent>

      -->

     

      <build>

     

         <finalName>simple-webapp</finalName>

        <plugins>

       

          <plugin>

            <artifactId>maven-assembly-plugin</artifactId>

            <configuration>

              <descriptorRefs>

                <descriptorRef>project</descriptorRef>

              </descriptorRefs>

            </configuration>

            <executions>

              <execution>

                <id>examples</id>

                <phase>package</phase>

                <goals>

                  <goal>attached</goal>

                </goals>

              </execution>

            </executions>

          </plugin>

         

         

        <plugin>

                                <groupId>org.mortbay.jetty</groupId>

                                <artifactId>maven-jetty-plugin</artifactId>

                               

                                <version>6.1.12</version>

                               

                                <configuration>

                                       <scanIntervalSeconds>10</scanIntervalSeconds>

                                       <stopKey>foo</stopKey>

                                       <stopPort>9999</stopPort>

                                </configuration>

                                       <!--

                                <executions>

                               

                                       <execution>

                                              <id>start-jetty</id>

                                              <phase>pre-integration-test</phase>

                                              <goals>

                                                     <goal>run</goal>

                                              </goals>

                                              <configuration>

                                                     <scanIntervalSeconds>10</scanIntervalSeconds>

                                                     <daemon>true</daemon>

                                              </configuration>

                                       </execution>

                               

                                              <execution> <id>stop-jetty</id>

                                              <phase>post-integration-test</phase> <goals> <goal>stop</goal>

                                              </goals> </execution>

                 

                                </executions>

                                -->

                         </plugin>

          <plugin>

            <artifactId>maven-compiler-plugin</artifactId>

            <configuration>

              <source>1.6</source>

              <target>1.6</target>

            </configuration>

          </plugin>  

         

                   

                          <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>cobertura-maven-plugin</artifactId>

            <configuration>

              <check>

                <branchRate>85</branchRate>

                <lineRate>85</lineRate>

                <haltOnFailure>true</haltOnFailure>

                <totalBranchRate>85</totalBranchRate>

                <totalLineRate>85</totalLineRate>

                <packageLineRate>85</packageLineRate>

                <packageBranchRate>85</packageBranchRate>

                <regexes>         

                  <regex>

                    <pattern>com.example.boringcode.*</pattern>

                    <branchRate>40</branchRate>

                    <lineRate>30</lineRate>

                  </regex>

                </regexes>

              </check>

            </configuration>

            <executions>

              <execution>

                <goals>

                  <goal>clean</goal>

                  <goal>check</goal>

                </goals>

              </execution>

            </executions>

          </plugin>

         

      

          <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-pmd-plugin</artifactId>

            <version>2.3</version>

          </plugin>

     

                <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-checkstyle-plugin</artifactId>

            <configuration>

              <enableRulesSummary>false</enableRulesSummary>

             </configuration>

          </plugin>

     

     

        </plugins>

      </build>

     

      <reporting>

         <plugins>

        

               <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-project-info-reports-plugin</artifactId>

            <version>2.0.1</version>

          </plugin>

         

        

          <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>cobertura-maven-plugin</artifactId>

          </plugin>

             

         <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-pmd-plugin</artifactId>

            <version>2.3</version>

            <configuration>

              <linkXref>true</linkXref>

              <sourceEncoding>utf-8</sourceEncoding>

              <minimumTokens>100</minimumTokens>

              <targetJdk>1.5</targetJdk>

              <excludes>

                <exclude>**/*Bean.java</exclude>

                <exclude>**/generated/*.java</exclude>

              </excludes>

              <excludeRoots>

                <excludeRoot>target/generated-sources/stubs</excludeRoot>

              </excludeRoots>

            </configuration>

          </plugin>

      

          <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-checkstyle-plugin</artifactId>

          </plugin>

     

                        <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>findbugs-maven-plugin</artifactId>

            <version>2.0</version>

            <configuration>

              <effort>Max</effort>

              <threshold>Low</threshold>

               <findbugsXmlOutput>true</findbugsXmlOutput>

              <!-- Optional directory to put findbugs xml report -->

              <findbugsXmlOutputDirectory>target</findbugsXmlOutputDirectory>

     

              <!--

              <includeFilterFile>lib-filter.xml</includeFilterFile>

              -->

            </configuration>

          </plugin> 

        </plugins>

      </reporting>

     

    </project>

      

    然后maven编译生成site报告: D:\project\mvn-examples-1.0\ch05-simple-web\simple-webapp>mvn  clean  -Dmaven.compile.debug  site    jetty:run         

    报告在: D:\project\mvn-examples-1.0\ch05-simple-web\simple-webapp\target\site\cobertura

     

    查看(1530) 评论(0) 收藏 分享 管理

  • 淘宝接口测试的几点收获

    2009-06-03 09:19:59

    淘宝主要的工具改进


    一 dbunit

     dbunit 通过poi技术支持主键

    二 unitils

    更改可以支持excel数据存储
     

    三 cruisecontrol

     
     1 增加贸易通通知

     2 增加clover插件支持

     3 项目相关人列表

     

    接口测试核心思路 

    1) qa前期介入单元测试,有3:2的研发:测试资源配比保证

    2) 数据格式抛弃excel

    3) cc 优先, 项目进度透明. 需要研发支持

    4) 有资深工程师对现有工具做二次开发


     

    当下可以参考的

    1) 工具技术培训

    2) 扩展的工具以及项目应用

    3) 流程改造
     

    远期应用

    1)       持续集成推广. 需要结合scm, 平台研发目前运作的apache continumm,可以做到local的单元测试集成

     

     

  • 2个非线程安全的例子.

    2009-05-15 21:34:44

    一个项目中积累的2个非线程安全的例子.

     

    1 java.util.regex.Matcher

     

    static SimpleDateFormat,

    是放在threadlocal来解决该问题的.

     

    见到:

     

    http://sharajava.javaeye.com/blog/81551

  • uname 的数据从哪里读取

    2009-05-15 21:16:47

    日常uname –a读取内核信息.如: 

    [root@b2b_plat_4212 ~]# uname  -a

    Linux b2b_plat_4212 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux

     

    比如说上述表明了是64 bit os, 以及用的kernal 2.6.18-128.el5

     

    这些信息从哪里来 ? 答案在/boot/grub/menu.lst

     

     

    [root@b2b_plat_4212 ~]# cat /boot/grub/menu.lst

    # grub.conf generated by anaconda

    #

    # Note that you do not have to rerun grub after making changes to this file

    # NOTICE:  You have a /boot partition.  This means that

    #          all kernel and initrd paths are relative to /boot/, eg.

    #          root (hd0,0)

    #          kernel /vmlinuz-version ro root=/dev/sda3

    #          initrd /initrd-version.img

    #boot=/dev/sda

    default=1

    timeout=5

    splashimage=(hd0,0)/grub/splash.xpm.gz

    hiddenmenu

    title Red Hat Enterprise Linux Server (2.6.18-128.el5xen)

            root (hd0,0)

            kernel /xen.gz-2.6.18-128.el5

            module /vmlinuz-2.6.18-128.el5xen ro root=LABEL=/ rhgb quiet

            module /initrd-2.6.18-128.el5xen.img

    title Red Hat Enterprise Linux Server-base (2.6.18-128.el5)

            root (hd0,0)

            kernel /vmlinuz-2.6.18-128.el5 ro root=LABEL=/ rhgb quiet

            initrd /initrd-2.6.18-128.el5.img

     

     

    实际测试中发现, 2.6.18-128.el5 性能优于2.6.18-128.el5xen  

     

     

  • java -XX:+HeapDumpOnOutOfMemoryError的应用

    2009-04-30 20:07:28

    web应用程序部署在jboss容器里,性能测试出现outofmemory异常后即jboss exit程序,研发想打印异常时调用栈信息。

     

    怎么办呢?

     

    run.conf 加入

    JAVA_OPTS="$JAVA_OPTS  -XX:+HeapDumpOnOutOfMemoryError "

     

    然后执行run.sh重启jboss.

     

    也有更土的办法,就是不断的kill  -3  pid  (仅仅限于linux 上) 或者 ctrl+break 中断(仅仅在windows) 打印thread dump

  • jetty 作为java自动化测试web server测试http应用

    2009-04-12 18:43:11

    http://www.javaeye.com/topic/154546

     

    import org.mortbay.jetty.Connector;

    import org.mortbay.jetty.Server;

    import org.mortbay.jetty.nio.SelectChannelConnector;

    import org.mortbay.jetty.webapp.WebAppContext;

    import org.mortbay.thread.BoundedThreadPool;

     

    public class JettyHello {

     

        public static void main(String[] args) throws Exception {

     

           Server server = new Server();

            BoundedThreadPool threadPool = new BoundedThreadPool();

           threadPool.setMaxThreads(100);

           server.setThreadPool(threadPool);

           Connector connector = new SelectChannelConnector();

           connector.setPort(8080);

           server.setConnectors(new Connector[] { connector });

           WebAppContext context = new WebAppContext(

                  "D:/jetty-6.1.0/webapps/test", "/test");

           server.addHandler(context);

           server.setStopAtShutdown(true);

           server.setSendServerVersion(true);

           System.out.println("start...");

           server.start();

           server.join();

           System.out.println("end...");

    }

     

    可以发起请求; 请注意没有修改jetty.xml

     

    http://localhost:8080/test/jsp/bean1.jsp

     

    需要的jar ,包括jsp2.1的安装包: D:\jetty-6.1.0\lib\jsp-2.1目录
    jsp2.1.jar
    jsp-api-2.1.jar
    core-3.1.1.jar

     

      剩下的测试相对简单,启动jetty后, 就通过httpunit 或者jwebunit或者httpclient或者URLConnection发起HTTP 请求.

     再检查http返回.

     

     

  • spring dm 退出时发生core dump以及QA避开陷阱

    2009-04-12 16:52:06

    症状 

    telnet localhost 2401

    >exit

    或者/home/aranda/springsource/bin/shutdown.sh

     spring dm退出时发生CORE DUMP:



    [aranda@dc_4 bin]$ *** glibc detected *** /home/aranda/software/jdk1.6.0_12/bin/java: corrupted double-linked list: 0x00000000509cb740 ***

    ======= Backtrace: =========

    /lib64/libc.so.6[0x3e11e7155c]

    /lib64/libc.so.6(cfree+0x8c)[0x3e11e74c5c]

    /home/aranda/aranda.home/bin/apr/lib64/libapr-1.so.0(apr_allocator_destroy+0x1b)[0x2aaaf61e559b]

    /home/aranda/aranda.home/bin/apr/lib64/libapr-1.so.0(apr_pool_terminate+0x2d)[0x2aaaf61e61fd]

    [0x2aaaab1467b0]

    ======= Memory map: ========

    40000000-40009000 r-xp 00000000 08:08 3442306                            /home/aranda/software/jdk1.6.0_12/bin/java

    40108000-4010a000 rwxp 00008000 08:08 3442306                            /home/aranda/software/jdk1.6.0_12/bin/java

    4010b000-4010e000 ---p 4010b000 00:00 0

    4010e000-4014c000 rwxp 4010e000 00:00 0

    42015000-42115[2009-04-12 14:40:57.227] Thread-1                 <SPOF0004I> Shutdown initiated.

    [2009-04-12 14:40:57.286] server-dm-1              <SPSC0002I> Shutting down ServletContainer.

    /home/aranda/springsource/bin/startup.sh: line 128:  4896 Aborted                 $JAVA_HOME/bin/java $JAVA_OPTS $DEBUG_OPTS $APP_OPTS $JMX_OPTS -Dcom.springsource.server.home=$SERVER_HOME -Dcom.springsource.server.configDir=$CONFIG_DIR -Djava.io.tmpdir=$SERVER_HOME/work/tmp/ -classpath $CLASSPATH com.springsource.server.kernel.bootstrap.Bootstrap

     

     

     

    [aranda@dc_4 ~]$ gdb   software/jdk1.6.0_12/bin/java  /home/core/core-java-1267-1239518181

    GNU gdb Red Hat Linux (6.5-37.el5rh)

     

    Core was generated by `/home/aranda/software/jdk1.6.0_12/bin/java -Djava.library.path=/home/aranda/ara'.

    Program terminated with signal 6, Aborted.

    #0  0x0000003e11e30155 in raise () from /lib64/libc.so.6

    (gdb) bt

    #0  0x0000003e11e30155 in raise () from /lib64/libc.so.6

    #1  0x0000003e11e31bf0 in abort () from /lib64/libc.so.6

    #2  0x0000003e11e6a38b in __libc_message () from /lib64/libc.so.6

    #3  0x0000003e11e7155c in _int_free () from /lib64/libc.so.6

    #4  0x0000003e11e74c5c in free () from /lib64/libc.so.6

    #5  0x00002aaaf61e559b in apr_allocator_destroy (allocator=0x5105b720) at memory/unix/apr_pools.c:134

    #6  0x00002aaaf61e61fd in apr_pool_terminate () at memory/unix/apr_pools.c:602

    #7  0x00002aaaab1467b0 in ?? ()

     

    解决方案:

     

    修改/home/aranda/springsource/config/servletContainer.config 下面的"enabled"false,

     

     

     

    "listeners": [

                            {

                                    /*

                                     * APR library loader.

                                     * Documentation at http://tomcat.apache.org/tomcat-6.0-doc/apr.html

                                     */

                                    "enabled":false,

                                    "className": "org.apache.catalina.core.AprLifecycleListener",

                                    "SSLEngine": "off"

                            },

     

     

    原因分析

     

    问题原因已经基本查明,无论64bit还是32bit JVM,只要使用了apr, spring-dm 都会crash.  已经把详细的原因分析发布到springsource官方论坛中,摘抄如下:

     When the shutdown sequence is initialized, the tomcat AprLifecycleListener will got "AFTER_STOP_EVENT", then Library.terminate() will be called and finally the c function apr_terminate() will be called. then all memories managed by APR library will be released.

    But unfortunately, inside dm-server, the AprLifecycleListner no-longer be the last one die.

    Even though apr library has already been terminated, but the shuttingdown sequence of dm-server is still in progress, and dm-server's own executor(an instance of DelegatingExecutor which is set into the AprEndPoint when starting up) still has the opportunity to handle the broken socket, which is the native Socket class associated with some apr data structure.
    Inside the AprEndpoint logic, it will invoke native apr c method to close/release those broken sockets, but those associated apr structures has already been released in previous apr_terminate() call!

     

     

    通过跟踪分析apr/tomcat/spring-dm的源代码, 我们认为修改dmserver/config/servletContainer.config, AprLifecycleListener关闭即可避免该crash.

    多说几句, 没有AprLifecycleListener, DM-Server还是安全的.因为其中的tomcat connector会自行初始化libtcnative;而在jvm退出的时候,libtcnative会收到jvm回调而释放apr管理的内存池.

     

    此外,这个问题即便不处理也没关系,因为运行时是不会发生这种crash. 这里的关键是: 我们是否把问题分析透彻了,并找到了故障根源.

     

     

    制定对策

     

    默认情况下linux  /etc/profile会关闭core dump输出

    # No core files by default

    ulimit -S -c 0 > /dev/null 2>&1

     

    对于c++应用以及java jni 应用, 应该修改/etc/profile文件为

    ulimit -S -c ulimited > /dev/null 2>&1

    或者在.bash_profile文件修改

    Ulimit –c ulimited

     

    默认在应用启动目录下产生core dump文件. 可以通过修改    /proc/sys/kernel/core_pattern

    /home/core/core-%e-%p-%t

    生成特定命名格式的文件 

    另外对于JAVA 应用,core dump会产生hs开头的文件

     

     Spring dm 1.0.1已经发现过多个BUG , 对于这类新应用大家特别小心, 应该全面检查各项输出,包括spring 日志/系统/var/log/message等日志.

        

     对于这些新应用,需要和研发/需求方非常明确支持的JVM, web server, application,OS 以及补丁版本. 一些微小差异也会导致BUG .

  • Hsqldb在单元测试中简单应用

    2009-04-06 12:25:26

    参考:

    http://www.blogjava.net/q5782129100/archive/2008/12/27/248600.html

    http://www.javaeye.com/topic/79802

    下载: http://hsqldb.org/

    Hsqldb 4种模式 可采用server模式退出时不用shutdown持久存储或者memory only模式作为单元测试数据库,可以保证测试原子性。

    ALIBABA B2B研发编写的单元测试代码中,已经有多年生命周期了J

     

    1)     server模式启动

    D:\hsqldb_1_8_0_10\bin>  java -cp ../lib/hsqldb.jar org.hsqldb.Server -database.

    0 file:mydb -dbname.0 xdb

     

    可以看到bin目录下有3个新文件: mydb.properties, mydb.log, mydb.script

       然后另外开一个控制台执行:

    D:\hsqldb_1_8_0_10\bin>   java -cp ../lib/hsqldb.jar org.hsqldb.util.DatabaseMan

    agerSwing

    Failed to load preferences.  Proceeding with defaults:

    选择: server模式, 以及xdb,密码为空.

     

     

    执行创建表以及记录:

     

    create table USER (

        USER_ID varchar(25) not null,

        PASSWORD varchar(25)  not null,

     

        constraint PK_USER primary key (USER_ID)

    );

    INSERT INTO USER VALUES('j2ee','j2ee');

     

     

           编写测试程序:

    import java.sql.Connection;

    import java.sql.DriverManager;

    import java.sql.ResultSet;

    import java.sql.SQLException;

    import java.sql.Statement;

     

    /**

     * TODO Comment of HSQLServerTest

     *

     * @author jianzhao.liangjz

     */

    public class HSQLServerTest {

         public static void main(String[] args){

            System.out.println("begin...");

            Connection connection = null;

            Statement stmt = null;

            ResultSet rs = null;

            try {

                Class.forName("org.hsqldb.jdbcDriver");

                connection = DriverManager.getConnection("jdbc:hsqldb:hsql://localhost/xdb", "sa", "");

                stmt = connection.createStatement();

                String sql = "select * from user";

                rs = stmt.executeQuery(sql);

                while (rs.next()) {

                    System.out.println("user_id=" + rs.getString("USER_ID"));

                    System.out.println("password=" + rs.getString("PASSWORD"));

                }

            } catch (Exception e) {

                e.printStackTrace();

            } finally {

                try {

                    if (rs != null)

                        rs.close();

                    if (stmt != null) {

                        stmt.close();

                    }

                    if (connection != null) {

                        connection.close();

                    }

                } catch (Exception e) {

                    e.printStackTrace();

                }

            }

        }

    }

     

    D:\hsqldb_1_8_0_10\bin>javac  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLServerTest.

    java

     

    D:\hsqldb_1_8_0_10\bin>

    D:\hsqldb_1_8_0_10\bin>java  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLServerTest

    begin...

    user_id=j2ee

    password=j2ee

     

    2)     memory only模式启动以及连接

    能创建表以及insert记录. 但不能开一个DB连接直接读取数据.

    所以MEM 的测试在一个进程内完成:

    import java.sql.Connection;

    import java.sql.DriverManager;

    import java.sql.ResultSet;

    import java.sql.SQLException;

    import java.sql.Statement;

     

    public class HSQLMemTest{

        public static void main(String[] args){

            try {

                 Class.forName("org.hsqldb.jdbcDriver" );           

               

                 Connection  c = DriverManager.getConnection("jdbc:hsqldb:mem:memdb", "sa", "");

                 if(c != null){

                     System.out.println("Connected db success!");

                     String sql = "create table USER (USER_ID varchar(25) not null, PASSWORD varchar(25)  not null,constraint PK_USER primary key (USER_ID) );";

                     Statement st = c.createStatement();

                     st.execute(sql);

                     sql = "INSERT INTO USER  VALUES('j2ee','j2ee');";

                     st.executeUpdate(sql);

                      

                      String sql_query = "select * from USER";

                    ResultSet rs = st.executeQuery(sql_query);

                               while (rs.next()) {

                                  System.out.println("user_id=" + rs.getString("USER_ID"));

                                  System.out.println("password=" + rs.getString("PASSWORD"));

                              }

                               if(st != null){

                         st.close();

                     }

                     c.close();

                 }

     

             } catch (Exception e) {

                 System.out.println("ERROR: failed to load HSQLDB JDBC driver.");

                 e.printStackTrace();

                return;

             }

     

         }

    }

    D:\hsqldb_1_8_0_10\bin>javac  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLMemTest.jav

    a

     

    D:\hsqldb_1_8_0_10\bin>java  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLMemTest

     

  • API测试排列组合的威力

    2009-03-14 11:41:11

     
    当下在做一个客户端编码方法调用为特定格式,通过http发送给服务器端. 服务器接收并翻译为内部方法调用,并执行方法,再响应给客户端的项目.
    即当下很红火的HTTP API测试.

    原始需求:
    支持JSON格式,XML格式,Hessian,http,param格式.

    JSON协议,xml格式,hessian 支持java原始类型,数组,日期以及List,Map类型
    http ,param协议仅仅支持原始类型.

    有心的朋友可能会发散:
    1) java 数组支持多少维? >=2维支持么?
    2) 日期支持字符串形式的格式? 具体的格式要包含时区么?
    3) 范型的List/Map 支持么?
    4) 内部嵌套的List/Map 支持么,比如 List类型内部嵌入Map?
    5) 支持null值否?
    6) ...

    随着一系列的问题抛出,你将会发现一个庞大的正交矩阵.

1613/9<123456789>
Open Toolbar