淘宝商城(天猫)高级技术专家.3年研发+3年性能测试调优/系统测试+4年团队管理与测试架构、研发系统实践. 新舞台新气象, 深化测试基础架构及研发架构,希望能在某个技术领域成为真正的技术大牛。欢迎荐才http://bbs.51testing.com/viewthread.php?tid=120496&extra=&page=1 .邮件: jianzhao.liangjz@alibaba-inc.com,MSN:liangjianzhao@163.com.微博:http://t.sina.com.cn/1674816524

发布新日志

  • JAVA应用监控javamelody

    2010-05-09 22:21:56

     

    http://code.google.com/p/javamelody/wiki/Screenshots

    http://wiki.hudson-ci.org/display/HUDSON/Monitoring

    http://code.google.com/p/javamelody/wiki/UserGuide

    http://code.google.com/p/javamelody/wiki/UserGuideAdvanced

     

    另外一款java 应用性能监控工具。

    呵呵,个人体会感觉界面、部署便利性不及jwebap

    本次试验在spring 程序上及hudson上成功。

     

    1.1.1    下载

    http://code.google.com/p/javamelody/downloads/list 下载javamelody-1.15.0.zip

    1.1.2    部署

    将解压后的2jar包放到被监控程序的 lib/目录下

    修改web.xml: 在最前面加入

     

           <filter>

                    <filter-name>monitoring</filter-name>

                    <filter-class>net.bull.javamelody.MonitoringFilter</filter-class>

            </filter>

            <filter-mapping>

                    <filter-name>monitoring</filter-name>

                    <url-pattern>/*</url-pattern>

            </filter-mapping>

            <listener>

                    <listener-class>net.bull.javamelody.SessionListener</listener-class>

            </listener>

    1.1.3    启动应用

    启动被监控应用。

    1.1.4    执行用户请求

    1.1.5    监控web应用

    http://host:port/context/monitoring 可以获取javamelody

      

     

    1.1.6    部署hudson plugin

    http://hudson-ci.org/download/plugins/ 下载。

    放在hudson/plugins目录下。

    http://host:port/hudson/monitoring 可以查看到性能数据。

     

  • java web应用性能监控工具jwebap

    2010-04-27 12:52:28

    性能测试中,我们经常希望一个平台能拦截、profile、分析性能数据,现实在一点点逼近. Jwebap

     

     效果图 http://www.jwebap.org/detect/console/?null

     官方文档:http://www.jwebap.org/index_zh.htm

     作者blog: http://leadyu.javaeye.com/

     

    Jwebap是无代码侵入、需要修改web.xml配置文件的java应用监控工具,轻易获取jdbc 调用/http调用、方法调用耗时及是否有jdbc泄露。 核心原理:应用asm 字节码操作框架实现动态monitor

       另外一个工具:http://jamonapi.sourceforge.net/

    1.1.1    下载

    http://sourceforge.net/projects/jwebap/files/

    1.1.2    部署

    解压后的jwebap_0.6.1.jar, tracer_0.6.1.jar jwebap_0.6.1_all\lib目录下jar复制到jboss-4.2.2.GA\server\default\lib 目录下。

     



    修改tracer_0.6.1.jar/META-INF/plugin.xml文件,红色为修改项:

    <?xml version="1.0" encoding="UTF-8"?>

    <plugin>

           <!--component-->

           <component name="HttpComponent" type="org.jwebap.plugin.tracer.http.HttpComponent">

                  <component-param>

                         <name>trace-filter-active-time</name>

                         <value>-1</value>

                         <description>(ms) timings filter's over time</description>

                  </component-param>

                  <component-param>

                         <name>trace-max-size</name>

                         <value>1000</value>

                         <description>max over-time trace size</description>

                  </component-param>

           </component>

           <component name="MethodComponent" type="org.jwebap.plugin.tracer.method.MethodComponent">

                  <component-param>

                         <name>trace-filter-active-time</name>

                         <value>-1</value>

                         <description>(ms) timings filter's over time</description>

                  </component-param>

                  <component-param>

                         <name>trace-max-size</name>

                         <value>1000</value>

                         <description>max over-time trace size</description>

                  </component-param>

                  <component-param  style="longtext">

                         <name>detect-clazzs</name>

                         <value>

                         com.ali.*;

                         </value>

                         <description>

                         package name and class name that monitored by MethodComponent,

                         e.g.: 'test.*;test.Test' , divided by ';'

                         </description>

                  </component-param>

           </component>

           <component name="JdbcComponent" type="org.jwebap.plugin.tracer.jdbc.JdbcComponent">

                  <component-param>

                         <name>trace-filter-active-time</name>

                         <value>-1</value>

                         <description>(ms) timings filter's over time</description>

                  </component-param>

                  <component-param>

                         <name>trace-max-size</name>

                         <value>1000</value>

                         <description>max over-time trace size</description>

                  </component-param>

                  <component-param  style="longtext">

                         <name>connection-listener</name>

                         <value>

                  org.jwebap.plugin.tracer.http.ServletOpenedConnectionListener;org.jwebap.plugin.tracer.method.MethodOpenedConnectionListener

                         </value>

                         <description>Connection Listener</description>

                  </component-param>

                  <component-param  style="longtext">

                         <name>driver-clazzs</name>

                         <value>org.apache.commons.dbcp.BasicDataSource</value>        

                         <!--

                         <value>com.mysql.jdbc.Driver</value>

                         -->

                         <description>

     

                         </description>

                  </component-param>

           </component>

          

    </plugin>

     

    修改好后重新替换入jar。如下是监控spring.war应用的操作过程。

     

    修改jwebap.xml放到jboss-4.2.2.GA\server\default\deploy\spring.war\WEB-INF

    <?xml version="1.0" encoding="UTF-8"?>

    <jwebap>

           <!--plugins-->

           <plugin name="Tracer" ref="${ABSOLUTE_PATH}/tracer_0.6.1.jar"/>

    </jwebap>

     

    ${ABSOLUTE_PATH} 是内置的变量,不必修改

     

    参考jwebap_0.6.1_all\bin\web-example.xml 修改jboss-4.2.2.GA\server\default\deploy\spring.war\WEB-INF\web.xml

     

    <!-- begin jwebap-->

           <context-param>

                  <param-name>jwebap-config</param-name>

                  <param-value>/WEB-INF/jwebap.xml</param-value>

           </context-param>

           <listener>    

                  <listener-class>org.jwebap.startup.JwebapListener</listener-class>     

           </listener>

           <filter>

                  <filter-name>PageDetectFilter</filter-name>

                  <filter-class>org.jwebap.plugin.tracer.http.DetectFilter</filter-class>

                  <init-param>

                         <param-name>excludeUrls</param-name>

                         <param-value>/detect;/detect/*;*.js;*.jpg;*.htm;*.html;*.gif;*.png;*.css;*.swf</param-value>

                  </init-param>

           </filter>

           <filter-mapping>

                  <filter-name>PageDetectFilter</filter-name>

                  <url-pattern>/*</url-pattern>

           </filter-mapping>

     

           <servlet>

                  <servlet-name>detect</servlet-name>

                  <servlet-class>org.jwebap.ui.controler.JwebapServlet</servlet-class>  

           </servlet>

     

           <servlet-mapping>

                  <servlet-name>detect</servlet-name>

                  <url-pattern>/detect/*</url-pattern>

           </servlet-mapping>

          

           <!-- end jwebap-->

     

    有一点特别注意的是,在被监控的程序如spring.war目录jboss-4.2.2.GA\server\default\deploy\spring.war\WEB-INF\lib加入数据库驱动程序如commons-dbcp.jar,否则无法获取jdbc  trace数据。

     

     

    1.1.3    启动应用及拦截

    启动jboss

    执行应用操作,ie 输入

    http://localhost:8080/spring/detect/console/?null 可以看到jwebap监控效果

    jdbc traces:

     

    http监控

     

     

    Method 监控

     

    1.1.4    已知限制及应用场合

      更多讨论可以在javaeye论坛搜索 jwebap

     

      对于普通的web程序,jwebap支持不错。可以考虑内部系统开发性能优化时应用。

     

    Webx工程只能监控jdbc调用,对于方法和http请求无法拦截.

  • 可与jprofiler/yourkit媲美的java诊断工具Visualvm

    2010-04-21 21:47:35

    http://zorufa876.javaeye.com/blog/625649

     

    一度我曾以为在java profile领域,很难有工具能超越jprofiler yourkit

    相比较jprofiler/ youkit 优势:

    1)      开源免费

    2)      性能开销比jprofiler小很多. kylin性能优化项目中,jprofiler性能损耗大,而visualvm基本不受影响。

    3)      被监控端无需额外安装agent

     劣势:

             没有jprofiler那样直观地树状展现方法调用栈。

     

     由于visualvm远程监控比本地监控少很多功能,所以visualvm建议安装在linux服务器端。

    Visualvm 要求在jdk1.6_07上。

    1.1.1    下载visualvm

           不用jdk 自带的,因为jdk自带的滞后。从https://visualvm.dev.java.net/download.html 下载。

    1.1.2    在被监控服务器上安装visualvm

           先部署好java 6以上环境。

           大部分应用都在linux上,所以visualvm安装在linux目录。

     

    1.1.3    Windows机器上安装xming,输出linux图形化界面

    主要是将linux display定向到windows xming server上。Xming使用请参考http://www.51testing.com/?uid-13997-action-viewspace-itemid-99183

     

     

     

     

    linux shell

     export DISPLAY=windows ip:0

    ./visualvm &

    如:

     

    1.1.4    安装插件

    Tools->plugin

     

     

    1.1.5    设置java 允许jmx 访问

     少这个步骤,则cpu sample会无数据。

      可以设置java jmx选项为无验证。

    -Dcom.sun.management.jmxremote.ssl=false     -Dcom.sun.management.jmxremote.authenticate=false

     

       也可以:

    [admin@QA-145-1 ~]$ ps -ef|grep java

    admin    32289 32281 99 14:06 pts/7    00:09:21 /usr/alibaba/java/bin/java -Dprogram.name=run.sh -server -Xms2g -Xmx2g -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=128m -XX:MaxPermSize=128m -XX:+UseCompressedOops -XX:+DoEscapeAnalysis -XX:+AggressiveOpts -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djboss.server.home.dir=/home/admin/kylin/deploy/jboss_server -Djboss.server.home.url=file:/home/admin/kylin/deploy/jboss_server -Djavax.xml.parsers.SAXParserFactory=org.apache.xerces.jaxp.SAXParserFactoryImpl -Dorg.apache.xerces.xni.parser.XMLParserConfiguration=org.apache.xerces.parsers.XML11Configuration -Dcom.sun.management.config.file=/home/admin/kylin/deploy/conf/jmx/jmx_monitor_management.properties -Dapplication.codeset=GBK -Ddatabase.codeset=ISO-8859-1 -Ddatabase.logging=false -Djava.endorsed.dirs=/usr/alibaba/jboss/lib/endorsed -classpath /usr/alibaba/jboss/bin/run.jar:/usr/alibaba/java/lib/tools.jar org.jboss.Main

     

    [admin@QA-145-1 ~]$ cat  /home/admin/kylin/deploy/conf/jmx/jmx_monitor_management.properties |grep -v '#'

    com.sun.management.jmxremote.port=2009

    com.sun.management.jmxremote.ssl=false

    com.sun.management.jmxremote.authenticate=true

    com.sun.management.jmxremote.password.file=/home/admin/kylin/deploy/conf/jmx/jmx_monitor_users.properties

    com.sun.management.jmxremote.access.file=/home/admin/kylin/deploy/conf/jmx/jmx_monitor_roles.properties

    [admin@QA-145-1 ~]$ cat  /home/admin/kylin/deploy/conf/jmx/jmx_monitor_users.properties |grep -v '#'

    monitor monitor

    control control1

     

    [admin@QA-145-1 ~]$ cat  /home/admin/kylin/deploy/conf/jmx/jmx_monitor_roles.properties |grep -v '#'

    monitor   readonly

    control   readwrite

     

    1.1.6    采样cpu 或者memory对象

    点击sample cpu sample检查哪些方法耗时多。

     

     

    Memory sample: 检查哪些类占内存多

     

    这里点击settings设置过滤规则。

     

    1.1.7    安装jconsole plugin

    点击'Add JAR/Folder'按钮,

           添加JDK_HOME/demo/management/JTop/JTop.jar

     

    参考:

    http://zzxanadu.javaeye.com/blog/600183

     

    1.1.8    安装eclipse 插件

     

    https://visualvm.dev.java.net/eclipse-launcher.html

     

    下载解压到eclipse安装目录。

     

     

    运行应用程序,弹出的launcher 选择visualvm,将激活visualvm.

  • 代码可测性度量工具Testability explorer

    2010-01-20 21:58:07

    可测性维度包括架构设计、代码2个层面。

       架构层面拟从2个角度出发积累 1) 将日常qa碰到可测性及架构建议沉淀下来 2)和架构师探讨sei  atam 架构评审方法

       代码层面可测性可单独用testability explorer度量,也可使用hudsontestability explorer插件度量。

     

    Google内部广泛使用的testability explorer静态分析java bytecode并按规则计算类的可测性分数,将类划分为优秀、可容忍、需要调整3个等级,默认分数分别为<=50,<=100; >100(见CommandLineConfig.java)。

           Testability explorer逐行递归分析判断是否不可注入、全局变量,如果是则分值加1,直到递归深度达到Depth。本质上,testability explorer驱动研发养成符合依赖注入风格的编码习惯。Google内部也组合testability explorer等多个工具一起提升代码质量。

    可测性判断用途:

    1)     加入到持续集成hudson/continuum内,成为进度度量指示器

    2)     研发根据可测性判断是否需要重构

     

    更多参考:http://misko.hevery.com/2008/07/30/top-10-things-which-make-your-code-hard-to-test/

    http://misko.hevery.com/category/testability/

     

        可测性和代码高效间需要做平衡,可测性不能做:

    1)      need work 的类就直接让研发重构代码

    2)      直接找bug

     

    很多的事实证明,测试领域没有银弹。测试需要借助工具在合适场合辅助提高测试效率,最核心的还是人的判断力。

    部署编译

     

     

    确认机器已经搭建好maven2的环境。

     

    设置环境变量(本机eclipse路径,更多看http://code.google.com/p/testability-explorer/wiki/HowToContribute):

     

    ECLIPSE_LAUNCHER_JAR=D:\eclipse-java-ganymede-SR1-win32\plugins\org.eclipse.osgi_3.4.2.R34x_v20080826-1230.jar 

     

    (否则提示错误:

    Caused by: D:\project\google_testability_explorer\eclipse-plugin\build.xml:13: Java returned: 1
            at org.apache.tools.ant.taskdefs.Java.execute(Java.java:108)
            at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)

     

     

    取出源代码:svn checkout http://testability-explorer.googlecode.com/svn/trunk/ testability-explorer

     

    cd testability-explorer

    执行安装部署:mvn install

    调整被测工程的pom.xml

    project节内加入:

     

    <reporting>

        <plugins>

            <plugin>

            <groupId>com.google.testability-explorer</groupId>

                        <artifactId>maven-testability-plugin</artifactId>       

            <version>1.3.4-SNAPSHOT</version>

          </plugin>

        </plugins>

    </reporting>

     

    具体的版本号根据:D:\project\google_testability_explorer\maven-testability-plugin\target生成的jar定。

    在 C:\Documents and Settings\jianzhao.liangjz\.m2\repository\com\google\testability-explorer\maven-testability-plugin\1.3.4-SNAPSHOTmaven-testability-plugin-1.3.4-SNAPSHOT.jar

     

    D:\project\google_testability_explorer\maven-testability-plugin\src\main\java\com\google\maven\TestabilityExplorerMojo.java  找到目标testability

     

     

    执行maven目标分析代码可测性

    D:\project\morgan\trunk\member.service.china> mvn clean install

     

    D:\project\morgan\trunk\member.service.china>mvn com.google.testability-explorer:maven-testability-plugin:1.3.4-SNAPSHOT:testability

     

    或者看: http://code.google.com/p/testability-explorer/wiki/HowToMakeARelease

     

    在顶级工程上执行上述命令也可,但是site目录会仅生成在各个子工程目录下。

     

    结果如下:

     

    Issues that cause the most untestable classes

    Cost: 6,376

    Class com.alibaba.china.biz.dal.EncryptPtp is hard to test because:

    It is expensive to construct an instance of the class, and every test will need to call a constructor.

    These static initializers will need to be run once, when the class is loaded. Why is it bad?

    On line 30,

    static void generateCipher(String) is declared

    contributing 20% of total class cost

    Instead of initializing this state statically, try initializing a fresh copy in each instance. Creating new objects in Java is cheap, and this static state can't be cleaned up between tests.

    Cost: 6,254

    Class com.alibaba.china.biz.common.security.BlowfishEncrypter is hard to test because:

    It is expensive to construct an instance of the class, and every test will need to call a constructor.

    These static initializers will need to be run once, when the class is loaded. Why is it bad?

    On line 29,

    static BlowfishEncrypter() is declared

    contributing 20% of total class cost

    Instead of initializing this state statically, try initializing a fresh copy in each instance. Creating new objects in Java is cheap, and this static state can't be cleaned up between tests.

    Cost: 4,982

    Class com.alibaba.china.member.internal.security.Authentication is hard to test because:

    It is expensive to construct an instance of the class, and every test will need to call a constructor

  • Findbugs和parasoft jtest8.4试验对比

    2009-11-21 00:11:09

        Findbugs是著名的开源java 静态代码分析工具,基于bytecode扫描,具备数据流分析能力.操作很简单,可以下载单独的swing图形化工具也可下载eclipse 插件运行. 原理请参考A Comparison of Bug Finding Tools for Java

         Parasoft jtest_win32_eclipse_plugin_8.4.20是商业工具.在某公司的支持下简单试用了bugdective模块. jtest发现的bug 多于findbugs.

     

    也发现jtest存在几个还可改良的点

    1)      可能为空值的对象传参入 jdk 原生的函数时,也提示Avoid  NullPointerException. hashset.add(obj)

     

    2)      对于一些非直接判断空值,存在误报. 如

     

    Static final <I,O>  List <O> map( List <? Extends O> iList)

    {

    Validate.notnull(iList);

    ListIterator <? Extends I> iter=iList.listIterator;

     

    }

     

    初步了解到要在扫描前定制如上2规则有难度.也期待有jtest使用经验的朋友说说jtest能带来的独到价值及其他商业工具的价值

     

  • java监控工具

    2009-10-23 23:40:40

     以下基于jdk5+版本,大部分工具都可同时支持linux/windows

    一sun 公司自带工具
    jstat
    jconsole
    jmap
    jstack
    jinfo

    另外有:
    The Java Heap Analysis Tool
    jvisulavm (http://java.sun.com/javase/6/docs/technotes/tools/share/jvisualvm.html)

    二 特定JVM厂家的产品包

    bea jRockit
    ibm公司有自主研发的diagnostics tools

    三 第三方公司profile工具

    jprofiler
    Borland Optimizeit Suite
    yourkit

    可用于profile性能热点, 甚至检测thread deadlock


    四 监控gc活动

    可用java参数  -Xloggc:<file>打印gc,  gcviewer查看
    也可用Visual Garbage Collection Monitoring Tool

    五 监控内存泄露

    eclipse memory analyzer

    六 thread dump

    windows上: ctrl+break
    linux : kill -3 java进程id

    甚至可简单用linux 命令strace,lstrace,pstack命令看进程情况

       工具是辅助人的. 最大的难点在如何判断数据是否有性能问题及如何解决问题. 这块需要靠JVM知识和java性能优化知识积累了.

  • unitils数据库测试实例3

    2009-06-10 22:28:13

    UserDAOTest.java读取的test-common文件内容 :  <?xml version="1.0" encoding="GB2312"?><beans default-autowire="byName" xmlns="http://www.springframework.org/schema/beans"

             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

             xmlns:aop="http://www.springframework.org/schema/aop"

             xmlns:tx="http://www.springframework.org/schema/tx"

             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd

               http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd

               http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.0.xsd">

     

      <!-- for unitils test, read unitls.properties -->

      <bean id="dataSource" class="org.unitils.database.UnitilsDataSourceFactoryBean" /> 

        <bean id="transactionTemplate" class="org.springframework.transaction.support.TransactionTemplate"/>

     

      

       <bean id="sqlMapClient"

          class="org.springframework.orm.ibatis.SqlMapClientFactoryBean"  scope="singleton">

        <property name="configLocation" value="classpath:resources/sqlmap-config.xml" />

        <property name="dataSource" ref="dataSource" />

      </bean>

     

      <bean id="amoebaDao" class="com.ali.amoeba.brmmstest.ibatis.IBatisAmoebaDao">

        <property name="sqlMapClient" ref="sqlMapClient" />

      </bean>

     

      <bean id="transactionManager"

          class="org.springframework.jdbc.datasource.DataSourceTransactionManager"  scope="singleton">

        <property name="dataSource" ref="dataSource" />

      </bean>

     

        <bean id="baseSqlMapClientDAO" abstract="true">

            <property name="dataSource">

                <ref local="dataSource"/>

            </property>

            <property name="sqlMapClient">

                <ref local="sqlMapClient"/>

            </property>

        </bean> 

    </beans>  

    以上变化仅仅是    <bean id="dataSource" class="org.unitils.database.UnitilsDataSourceFactoryBean" />

     

    就是UnitilsDataSourceFactoryBean 去读取unitils.properities文件驱动程序及控制项目.其他的和spring testcontext读取的内容完全一样.   

    运行后amoeba数据库自动生成dbmaintain_scripts,可以清晰看出执行的脚本版本.

     

  • unitils数据库测试实例2

    2009-06-10 22:20:31

    UserDAOTest.java

     

    package com.ali.amoeba.brmmstest;

    import java.util.Arrays;

    import java.util.List;

    import org.junit.Ignore;

    import org.junit.Test;

    import org.springframework.context.ConfigurableApplicationContext;

    import org.springframework.context.support.ClassPathXmlApplicationContext;

    import org.unitils.UnitilsJUnit4;

    import org.unitils.database.annotations.Transactional;

    import org.unitils.database.util.TransactionMode;

    import org.unitils.dbunit.annotation.DataSet;

    import org.unitils.dbunit.annotation.ExpectedDataSet;

    import org.unitils.dbunit.datasetloadstrategy.impl.CleanInsertLoadStrategy;

    import org.unitils.dbunit.datasetloadstrategy.impl.InsertLoadStrategy;

    import org.unitils.reflectionassert.ReflectionAssert;

    import org.unitils.spring.annotation.SpringApplicationContext;

    import org.unitils.spring.annotation.SpringBeanByType;

    import com.ali.amoeba.brmmstest.dao.AmoebaDao;

    import com.ali.amoeba.brmmstest.dataobject.User;

    import org.junit.runner.RunWith;

    import org.unitils.UnitilsJUnit4TestClassRunner;

    @RunWith(UnitilsJUnit4TestClassRunner.class)

     

    @SpringApplicationContext({"classpath:resources/test-common.xml"})

     

    public class UserDAOTest { // extends UnitilsJUnit4 {

     

        /*用标注或者overwrite  createApplicationContext 方法都可以

        @SpringApplicationContext

        public ConfigurableApplicationContext createApplicationContext() {

           return new ClassPathXmlApplicationContext(new String[]{"classpath:resources/test-common.xml"});

        }

        */

     

        //从配置文件里面读取spring bean 的数据源

        protected AmoebaDao userDao;

        @SpringBeanByType

        public void setAmoebaDao(AmoebaDao userDao) {

           this.userDao = userDao;

        }

     

        @Test

        @DataSet(loadStrategy = CleanInsertLoadStrategy.class)

        @ExpectedDataSet

        public void testFindByName() {

           User user=new User();

           user.setFirst("doe");

           user.setLast("john");

           User result = userDao.findByName(user);

           System.out.println(result);

           ReflectionAssert.assertPropertyLenEquals("first", "doe", result);

        }

        @Transactional(TransactionMode.COMMIT)

        @Test

        public void testInsertUser() {

           User user=new User();

           user.setId(7777);

           user.setFirst("good");

           user.setLast("xxxx");

           user.setAge(31);

           userDao.insertUser(user);

        }

     

        //用一个xml数据源数据存储到db,过滤部分数据.然后与预定的另外一部分数据对比

        //达到数据可控

        //每次做之前该表数据都清空

        @Test

        @DataSet("UserDAOTest.testFindByMinimalAge.xml")

        @ExpectedDataSet("UserDAOTest.testFindByName-result.xml")

        public void testFindByMinimalAge() {

           List<User> result = userDao.findByMinimalAge(18);

           ReflectionAssert.assertPropertyLenEquals("first", Arrays.asList("methodDataset"), result);

        }

    }  

    UserDAOTest.xml内容都形如:

    <?xml version='1.0' encoding='UTF-8'?> 

    <dataset> 

          <user id="11"  first="doe"   last="john"   age="20" /> 

          <user id="1"  first="good"   last="bad"   age="10" /> 

    </dataset>

  • UNITILS数据库测试实例

    2009-06-10 20:47:22

    http://www.dbmaintain.org/overview.html

    http://www.slideshare.net/nevenfi/unit-testing-unitils-dbmaintain

    国外PPT : http://www.parleys.com/display/PARLEYS/Unitils

     

    http://www.slideshare.net/nevenfi/unit-testing-unitils-dbmaintain

     

    Unitils 2: unitils is dead, long live to spring-test

     

    http://203.208.37.132/search?q=cache:qwFKcsUwyLwJ:java-chimaera.blogspot.com/2008/12/unitils-2-unitils-is-dead-long-live-to.html+unitils&cd=43&hl=zh-CN&ct=clnk&gl=cn&st_usg=ALhdy2_5Qbp4RG814rODpTrovRqbSGjXHg

     

    http://blog.csdn.net/phphot/archive/2009/03/02/3949340.aspx

    http://www.blogjava.net/wangzhouyu/archive/2008/04/07/191137.html

    http://code.google.com/p/unitils-idea/

    http://rdc.taobao.com/blog/qa/?p=1895

     

    http://andyao.javaeye.com/blog/180231   unitils结合dbdeploy测试

     

    http://www.devx.com/Java/Article/35129

     

    样例代码 : http://www.devx.com/Java/Article/35129/1954

     

    Unitils 最核心的就是数据库DB xml 文件之间互相映射.

    Unitils 需要下载with-dependency的包 下面的例子用spring testcontext也能部分实现访问数据库.

    .

    有几个刚性要求 :

    1)      unitils.properities放在src同级目录

    2)      dataset xml文件默认放在和该类同一个目录

    3)      @ExpectedDataSet  对应的文件名命名规则及同目录存放

    public class UserDAOTest extends UnitilsJUnit4 {

     

        @Test @ExpectedDataSet

        public void testInactivateOldAccounts() {

            userDao.inactivateOldAccounts();

        }

    }

    Note that we have added the @ExpectedDataSet to the test method. This will instruct Unitils to look for a data set file named UserDAOTest.testInactivateOldAccounts-result.xml and compare the contents of the database with the contents of the data set:

     

    4)      Dbscripts目录下的自动化脚本命令规则

     

    The directory in which the scripts are located can be configured with following property:

    dbMaintainer.script.locations=myproject/dbscripts

    Multiple directories can be specified, separated by commas. Subdirectories are also scanned for script. files. All scripts are required to follow a particular naming pattern: they start with a version number, followed by an underscore and end with '.sql' (the supported extensions are configurable). The example below shows some typical scripts names. The leading zeroes in the example are not required, they are only added for convenience, to make sure they are shown in proper sequence in a file explorer window.

    dbscripts/ 001_initial.sql

               002_tracking_updates.sql

               003_auditing_updates.sql

    Suppose you add a new script, this time with version number 4: 004_create_user_admin_tables.sql. The next time you execute a database test, the database maintainer will notice that the database structure is no longer up to date. It will update the database schema incrementally by executing all of the new scripts, in this case only 004_create_user_admin_tables.sql.

     

    采用spring2.5+unitils 实现一个数据库的测试代码,最后生成的文件目录 :

         

     

    上述dboutput文件是第一次执行时由unitils自动导出的.

     

     

    工程的Java build path unitils.jar 及依赖的spring,dbunit,mysql引擎都加入.

     

    unitils.properities文件内容

     

     

     

    # comments documenting these unitils configuration properties removed for

    # brevity. look for commenting in unitils-default.properties in the root of the

    # unitils jar if needed.

     

    unitils.modules=database,dbunit,easymock,spring,inject

     

    unitils.module.hibernate.enabled=false

    # these placeholders are set in avaje.properties

    database.driverClassName=com.mysql.jdbc.Driver

    database.url=jdbc:mysql://localhost:3306/amoeba

    database.schemaNames=amoeba

    database.userName=root

    database.password=mysql

    database.dialect=mysql

     

    DatabaseModule.Transactional.value.default=commit

    #DbUnitModule.DataSet.loadStrategy.default=org.unitils.dbunit.datasetloadstrategy.InsertLoadStrategy

     

     

     

    # Indicates the database must be recreated from scratch when an already executed script. is updated. If false, the

    # DBMaintainer will give an error when an existing script. is updated.

    dbMaintainer.fromScratch.enabled=true

    # Indicates whether a from scratch update should be performed when the previous update failed, but

    # none of the scripts were modified since that last update. If false a new update will be tried only when

    # changes were made to the script. files.

    dbMaintainer.keepRetryingAfterError.enabled=false

     

    updateDataBaseSchema.enabled=true

    dbMaintainer.autoCreateExecutedScriptsTable=true

    dbMaintainer.useScriptFileLastModificationDates.enabled=true

    # list are recursively searched for files.

     

    #eclipse must be bin/scripts,or

    #Caused by: org.unitils.core.UnitilsException: File location scripts defined in property dbMaintainer.script.locations doesn't exist

     

    dbMaintainer.script.locations=bin/resources/dbscripts

    # Extension of the files containing the database update scripts

    dbMaintainer.script.fileExtensions=sql,ddl

     

    dbMaintainer.generateDataSetStructure.enabled=true

    dataSetStructureGenerator.xsd.dirName=dboutput/

    sequenceUpdater.sequencevalue.lowestacceptable=100

     

     

     

     

     

    # dbMaintainer.script.locations这个地方特别小心,在不同的运行ENV可能结果不同. 可以下载untils代码一起执行. 并且00n代表版本号, 建议在手工删除后tabledbmaintain_scripts; 可以从001开始,否则sql 语句版本从001开始.

     

     

    以上选项都是调试出来的, 特别注意:
    dbMaintainer.fromScratch.enabled=true

    updateDataBaseSchema.enabled=true

     

     

     

    log4j.properities内容:

    log4j.rootLogger=info,CONSOLE

    log4j.addivity.org.apache=true

    log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender

    log4j.appender.Threshold=debug

    org.unitils.appender.Threshold=debug

    log4j.appender.CONSOLE.Target=System.out

    log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout

    log4j.appender.CONSOLE.layout.ConversionPattern=[framework] %d - %c -%-4r [%t] %-5p %c %x - %m%n

     

     

     

  • 搭建CruiseControl2.8.2 +maven2.1+cobertura+checkstyle+findbug持续集成环境2

    2009-06-05 22:56:02

    第二步设置cruisecontrol  config.xml

     

    放置样例程序在: d:/cruisecontrol/maven_projects

     

    根据 http://bbs.scmroad.com/viewthread.php?tid=753&extra=page%3D1%20

    http://bbs.scmroad.com/viewthread.php?tid=758

    http://bbs.scmroad.com/viewthread.php?tid=836&extra=page%3D3

    再精简config.xml

     

     

    集成checkstyle.:

     

    http://confluence.public.thoughtworks.org/display/CC/CheckStyle

     

    集成corbertura:

    http://confluence.public.thoughtworks.org/display/CC/CruiseControlWithCobertura

     

    集成findbug:

    http://mojo.codehaus.org/findbugs-maven-plugin/2.1-SNAPSHOT/usage.html 配置maven pom.xml

    http://confluence.public.thoughtworks.org/display/CC/CruiseControlWithFindBugs

     

     

    集成其他工具:

    http://confluence.public.thoughtworks.org/display/CC/IntegratingWithOtherTools

     

     

    最后的配置文件cruisecontrol 配置文件config.xml内容:

     

    <cruisecontrol>

        <property name="cruisedir" value="D:/CruiseControl"/>  

        <property name="logdir" value="${cruisedir}/logs"/>  

        <property name="checkoutdir" value="${cruisedir}/maven_projects"/>  

        <property name="artifactdir" value="${cruisedir}/artifacts"/>  

        <property name="projectgoal" value="clean site package"/>  

        <property name="M2_EXE" value="D:/maven-2.1.0/bin/mvn.bat"/>  

       

        <plugin name="project">

                       <listeners>  

                <currentbuildstatuslistener file="${logdir}/${project.name}/status.txt"/>  

            </listeners>  

            <bootstrappers>  

                <svnbootstrapper localWorkingCopy="${checkoutdir}/${project.name}" />  

            </bootstrappers>  

            <modificationset quietperiod="30">  

                <svn localWorkingCopy="${checkoutdir}/${project.name}"/>  

            </modificationset>  

            <schedule interval="30">  

                <maven2 mvnscript="${M2_EXE}" 

                        pomfile="${checkoutdir}/${project.name}/pom.xml" 

                        goal="${projectgoal}"/>  

            </schedule>  

            <log dir="${logdir}/${project.name}">  

                <merge dir="${checkoutdir}/${project.name}/target/surefire-reports"/>  

            </log>  

            <publishers>  

                <onsuccess>  

                    <artifactspublisher dest="${artifactdir}/${project.name}"   

                                file="${checkoutdir}/${project.name}/target/{project.name}.jar"/>  

                </onsuccess>  

              

     

            </publishers>  

           

                  </plugin>

     

                  <!--

        <project name="ch03-simple"  />  

                  -->

          

                  <include.projects file="ch05-simple-web-proj.xml"/>

          

    </cruisecontrol>

     

     

    "ch05-simple-web-proj.xml" 内容是:

     

     

     

    <cruisecontrol>

     <property name="MY_PROJ_NAME" value="maven_projects/simple-webapp"/>

     <project name="simple-webapp">  

         

             <log>

                                         <merge file="${MY_PROJ_NAME}/target/checkstyle-result.xml"/>

                                          <merge file="${MY_PROJ_NAME}/target/pmd.xml"/>

                                          <merge dir="${MY_PROJ_NAME}/target/findbugsXml.xml"/>

     

                                </log>

            <schedule interval="40">  

                <maven2 mvnscript="${M2_EXE}" 

                        pomfile="${checkoutdir}/${project.name}/pom.xml" 

                        goal=" -U -Pexternal clean package site"/>  

            </schedule>  

            <publishers>  

                  

                <onsuccess>  

                             

                    <artifactspublisher dest="${artifactdir}/${project.name}"   

                                file="${checkoutdir}/${project.name}/target/simple-webapp.war"/>  

                              

                               

                    <artifactspublisher

                                                     dir="${MY_PROJ_NAME}/target/site" subdirectory="cobertura"

                                                     dest="artifacts/${project.name}" />

      

                </onsuccess>

               

                

            </publishers>  

        </project>

       

        </cruisecontrol>  

     

     

    最后界面展现:

  • 搭建CruiseControl2.8.2+maven2.1+cobertura+checkstyle+findbug持续集成环境

    2009-06-05 22:51:00

    参考:

    http://rdc.taobao.com/blog/qa/?p=1822

    http://rdc.taobao.com/blog/qa/?p=669

    http://emma.sourceforge.net/maven-emma-plugin/goals.html

    http://foxgem.javaeye.com/blog/24620

     

     

    下载: CruiseControl-2.8.2.exe, maven-2.1.0 并参考前述url设置M2_HOME,PATH变量. 

    第一步先配置pom.xml  

    依照maven权威指南  http://www.sonatype.com/books/maven-book/reference_zh/web.html  下载第五章样例simple-webapp

     

    参考http://mojo.codehaus.org/cobertura-maven-plugin/usage.html  配置coberturajetty, checkstyle, pmd, findbug 等相关的  pom.xml.配置完成需要单独测试pom.xml是否正确.

     

    Pom.xml内容:

     

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

      <modelVersion>4.0.0</modelVersion>

      <groupId>org.sonatype.mavenbook.ch05</groupId>

      <artifactId>simple-webapp</artifactId>

      <version>1-SNAPSHOT</version>

      <packaging>war</packaging>

      <name>Chapter 5 Simple Web Application Project</name>

      <url>http://sonatype.com/book</url>

      <dependencies>

        <dependency>

          <groupId>junit</groupId>

          <artifactId>junit</artifactId>

          <version>3.8.1</version>

          <scope>test</scope>

        </dependency>

        <dependency>

          <groupId>org.apache.geronimo.specs</groupId>

          <artifactId>geronimo-servlet_2.4_spec</artifactId>

          <version>1.1.1</version>

        </dependency>

      </dependencies>

     

      <!--

      <parent>

          <artifactId>parent</artifactId>

          <groupId>org.sonatype.mavenbook.ch05</groupId>

          <version>1-SNAPSHOT</version>

      </parent>

      -->

     

      <build>

     

         <finalName>simple-webapp</finalName>

        <plugins>

       

          <plugin>

            <artifactId>maven-assembly-plugin</artifactId>

            <configuration>

              <descriptorRefs>

                <descriptorRef>project</descriptorRef>

              </descriptorRefs>

            </configuration>

            <executions>

              <execution>

                <id>examples</id>

                <phase>package</phase>

                <goals>

                  <goal>attached</goal>

                </goals>

              </execution>

            </executions>

          </plugin>

         

         

        <plugin>

                                <groupId>org.mortbay.jetty</groupId>

                                <artifactId>maven-jetty-plugin</artifactId>

                               

                                <version>6.1.12</version>

                               

                                <configuration>

                                       <scanIntervalSeconds>10</scanIntervalSeconds>

                                       <stopKey>foo</stopKey>

                                       <stopPort>9999</stopPort>

                                </configuration>

                                       <!--

                                <executions>

                               

                                       <execution>

                                              <id>start-jetty</id>

                                              <phase>pre-integration-test</phase>

                                              <goals>

                                                     <goal>run</goal>

                                              </goals>

                                              <configuration>

                                                     <scanIntervalSeconds>10</scanIntervalSeconds>

                                                     <daemon>true</daemon>

                                              </configuration>

                                       </execution>

                               

                                              <execution> <id>stop-jetty</id>

                                              <phase>post-integration-test</phase> <goals> <goal>stop</goal>

                                              </goals> </execution>

                 

                                </executions>

                                -->

                         </plugin>

          <plugin>

            <artifactId>maven-compiler-plugin</artifactId>

            <configuration>

              <source>1.6</source>

              <target>1.6</target>

            </configuration>

          </plugin>  

         

                   

                          <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>cobertura-maven-plugin</artifactId>

            <configuration>

              <check>

                <branchRate>85</branchRate>

                <lineRate>85</lineRate>

                <haltOnFailure>true</haltOnFailure>

                <totalBranchRate>85</totalBranchRate>

                <totalLineRate>85</totalLineRate>

                <packageLineRate>85</packageLineRate>

                <packageBranchRate>85</packageBranchRate>

                <regexes>         

                  <regex>

                    <pattern>com.example.boringcode.*</pattern>

                    <branchRate>40</branchRate>

                    <lineRate>30</lineRate>

                  </regex>

                </regexes>

              </check>

            </configuration>

            <executions>

              <execution>

                <goals>

                  <goal>clean</goal>

                  <goal>check</goal>

                </goals>

              </execution>

            </executions>

          </plugin>

         

      

          <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-pmd-plugin</artifactId>

            <version>2.3</version>

          </plugin>

     

                <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-checkstyle-plugin</artifactId>

            <configuration>

              <enableRulesSummary>false</enableRulesSummary>

             </configuration>

          </plugin>

     

     

        </plugins>

      </build>

     

      <reporting>

         <plugins>

        

               <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-project-info-reports-plugin</artifactId>

            <version>2.0.1</version>

          </plugin>

         

        

          <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>cobertura-maven-plugin</artifactId>

          </plugin>

             

         <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-pmd-plugin</artifactId>

            <version>2.3</version>

            <configuration>

              <linkXref>true</linkXref>

              <sourceEncoding>utf-8</sourceEncoding>

              <minimumTokens>100</minimumTokens>

              <targetJdk>1.5</targetJdk>

              <excludes>

                <exclude>**/*Bean.java</exclude>

                <exclude>**/generated/*.java</exclude>

              </excludes>

              <excludeRoots>

                <excludeRoot>target/generated-sources/stubs</excludeRoot>

              </excludeRoots>

            </configuration>

          </plugin>

      

          <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-checkstyle-plugin</artifactId>

          </plugin>

     

                        <plugin>

            <groupId>org.codehaus.mojo</groupId>

            <artifactId>findbugs-maven-plugin</artifactId>

            <version>2.0</version>

            <configuration>

              <effort>Max</effort>

              <threshold>Low</threshold>

               <findbugsXmlOutput>true</findbugsXmlOutput>

              <!-- Optional directory to put findbugs xml report -->

              <findbugsXmlOutputDirectory>target</findbugsXmlOutputDirectory>

     

              <!--

              <includeFilterFile>lib-filter.xml</includeFilterFile>

              -->

            </configuration>

          </plugin> 

        </plugins>

      </reporting>

     

    </project>

      

    然后maven编译生成site报告: D:\project\mvn-examples-1.0\ch05-simple-web\simple-webapp>mvn  clean  -Dmaven.compile.debug  site    jetty:run         

    报告在: D:\project\mvn-examples-1.0\ch05-simple-web\simple-webapp\target\site\cobertura

     

    查看(1530) 评论(0) 收藏 分享 管理

  • 淘宝接口测试的几点收获

    2009-06-03 09:19:59

    淘宝主要的工具改进


    一 dbunit

     dbunit 通过poi技术支持主键

    二 unitils

    更改可以支持excel数据存储
     

    三 cruisecontrol

     
     1 增加贸易通通知

     2 增加clover插件支持

     3 项目相关人列表

     

    接口测试核心思路 

    1) qa前期介入单元测试,有3:2的研发:测试资源配比保证

    2) 数据格式抛弃excel

    3) cc 优先, 项目进度透明. 需要研发支持

    4) 有资深工程师对现有工具做二次开发


     

    当下可以参考的

    1) 工具技术培训

    2) 扩展的工具以及项目应用

    3) 流程改造
     

    远期应用

    1)       持续集成推广. 需要结合scm, 平台研发目前运作的apache continumm,可以做到local的单元测试集成

     

     

  • 2个非线程安全的例子.

    2009-05-15 21:34:44

    一个项目中积累的2个非线程安全的例子.

     

    1 java.util.regex.Matcher

     

    static SimpleDateFormat,

    是放在threadlocal来解决该问题的.

     

    见到:

     

    http://sharajava.javaeye.com/blog/81551

  • java -XX:+HeapDumpOnOutOfMemoryError的应用

    2009-04-30 20:07:28

    web应用程序部署在jboss容器里,性能测试出现outofmemory异常后即jboss exit程序,研发想打印异常时调用栈信息。

     

    怎么办呢?

     

    run.conf 加入

    JAVA_OPTS="$JAVA_OPTS  -XX:+HeapDumpOnOutOfMemoryError "

     

    然后执行run.sh重启jboss.

     

    也有更土的办法,就是不断的kill  -3  pid  (仅仅限于linux 上) 或者 ctrl+break 中断(仅仅在windows) 打印thread dump

  • jetty 作为java自动化测试web server测试http应用

    2009-04-12 18:43:11

    http://www.javaeye.com/topic/154546

     

    import org.mortbay.jetty.Connector;

    import org.mortbay.jetty.Server;

    import org.mortbay.jetty.nio.SelectChannelConnector;

    import org.mortbay.jetty.webapp.WebAppContext;

    import org.mortbay.thread.BoundedThreadPool;

     

    public class JettyHello {

     

        public static void main(String[] args) throws Exception {

     

           Server server = new Server();

            BoundedThreadPool threadPool = new BoundedThreadPool();

           threadPool.setMaxThreads(100);

           server.setThreadPool(threadPool);

           Connector connector = new SelectChannelConnector();

           connector.setPort(8080);

           server.setConnectors(new Connector[] { connector });

           WebAppContext context = new WebAppContext(

                  "D:/jetty-6.1.0/webapps/test", "/test");

           server.addHandler(context);

           server.setStopAtShutdown(true);

           server.setSendServerVersion(true);

           System.out.println("start...");

           server.start();

           server.join();

           System.out.println("end...");

    }

     

    可以发起请求; 请注意没有修改jetty.xml

     

    http://localhost:8080/test/jsp/bean1.jsp

     

    需要的jar ,包括jsp2.1的安装包: D:\jetty-6.1.0\lib\jsp-2.1目录
    jsp2.1.jar
    jsp-api-2.1.jar
    core-3.1.1.jar

     

      剩下的测试相对简单,启动jetty后, 就通过httpunit 或者jwebunit或者httpclient或者URLConnection发起HTTP 请求.

     再检查http返回.

     

     

  • spring dm 退出时发生core dump以及QA避开陷阱

    2009-04-12 16:52:06

    症状 

    telnet localhost 2401

    >exit

    或者/home/aranda/springsource/bin/shutdown.sh

     spring dm退出时发生CORE DUMP:



    [aranda@dc_4 bin]$ *** glibc detected *** /home/aranda/software/jdk1.6.0_12/bin/java: corrupted double-linked list: 0x00000000509cb740 ***

    ======= Backtrace: =========

    /lib64/libc.so.6[0x3e11e7155c]

    /lib64/libc.so.6(cfree+0x8c)[0x3e11e74c5c]

    /home/aranda/aranda.home/bin/apr/lib64/libapr-1.so.0(apr_allocator_destroy+0x1b)[0x2aaaf61e559b]

    /home/aranda/aranda.home/bin/apr/lib64/libapr-1.so.0(apr_pool_terminate+0x2d)[0x2aaaf61e61fd]

    [0x2aaaab1467b0]

    ======= Memory map: ========

    40000000-40009000 r-xp 00000000 08:08 3442306                            /home/aranda/software/jdk1.6.0_12/bin/java

    40108000-4010a000 rwxp 00008000 08:08 3442306                            /home/aranda/software/jdk1.6.0_12/bin/java

    4010b000-4010e000 ---p 4010b000 00:00 0

    4010e000-4014c000 rwxp 4010e000 00:00 0

    42015000-42115[2009-04-12 14:40:57.227] Thread-1                 <SPOF0004I> Shutdown initiated.

    [2009-04-12 14:40:57.286] server-dm-1              <SPSC0002I> Shutting down ServletContainer.

    /home/aranda/springsource/bin/startup.sh: line 128:  4896 Aborted                 $JAVA_HOME/bin/java $JAVA_OPTS $DEBUG_OPTS $APP_OPTS $JMX_OPTS -Dcom.springsource.server.home=$SERVER_HOME -Dcom.springsource.server.configDir=$CONFIG_DIR -Djava.io.tmpdir=$SERVER_HOME/work/tmp/ -classpath $CLASSPATH com.springsource.server.kernel.bootstrap.Bootstrap

     

     

     

    [aranda@dc_4 ~]$ gdb   software/jdk1.6.0_12/bin/java  /home/core/core-java-1267-1239518181

    GNU gdb Red Hat Linux (6.5-37.el5rh)

     

    Core was generated by `/home/aranda/software/jdk1.6.0_12/bin/java -Djava.library.path=/home/aranda/ara'.

    Program terminated with signal 6, Aborted.

    #0  0x0000003e11e30155 in raise () from /lib64/libc.so.6

    (gdb) bt

    #0  0x0000003e11e30155 in raise () from /lib64/libc.so.6

    #1  0x0000003e11e31bf0 in abort () from /lib64/libc.so.6

    #2  0x0000003e11e6a38b in __libc_message () from /lib64/libc.so.6

    #3  0x0000003e11e7155c in _int_free () from /lib64/libc.so.6

    #4  0x0000003e11e74c5c in free () from /lib64/libc.so.6

    #5  0x00002aaaf61e559b in apr_allocator_destroy (allocator=0x5105b720) at memory/unix/apr_pools.c:134

    #6  0x00002aaaf61e61fd in apr_pool_terminate () at memory/unix/apr_pools.c:602

    #7  0x00002aaaab1467b0 in ?? ()

     

    解决方案:

     

    修改/home/aranda/springsource/config/servletContainer.config 下面的"enabled"false,

     

     

     

    "listeners": [

                            {

                                    /*

                                     * APR library loader.

                                     * Documentation at http://tomcat.apache.org/tomcat-6.0-doc/apr.html

                                     */

                                    "enabled":false,

                                    "className": "org.apache.catalina.core.AprLifecycleListener",

                                    "SSLEngine": "off"

                            },

     

     

    原因分析

     

    问题原因已经基本查明,无论64bit还是32bit JVM,只要使用了apr, spring-dm 都会crash.  已经把详细的原因分析发布到springsource官方论坛中,摘抄如下:

     When the shutdown sequence is initialized, the tomcat AprLifecycleListener will got "AFTER_STOP_EVENT", then Library.terminate() will be called and finally the c function apr_terminate() will be called. then all memories managed by APR library will be released.

    But unfortunately, inside dm-server, the AprLifecycleListner no-longer be the last one die.

    Even though apr library has already been terminated, but the shuttingdown sequence of dm-server is still in progress, and dm-server's own executor(an instance of DelegatingExecutor which is set into the AprEndPoint when starting up) still has the opportunity to handle the broken socket, which is the native Socket class associated with some apr data structure.
    Inside the AprEndpoint logic, it will invoke native apr c method to close/release those broken sockets, but those associated apr structures has already been released in previous apr_terminate() call!

     

     

    通过跟踪分析apr/tomcat/spring-dm的源代码, 我们认为修改dmserver/config/servletContainer.config, AprLifecycleListener关闭即可避免该crash.

    多说几句, 没有AprLifecycleListener, DM-Server还是安全的.因为其中的tomcat connector会自行初始化libtcnative;而在jvm退出的时候,libtcnative会收到jvm回调而释放apr管理的内存池.

     

    此外,这个问题即便不处理也没关系,因为运行时是不会发生这种crash. 这里的关键是: 我们是否把问题分析透彻了,并找到了故障根源.

     

     

    制定对策

     

    默认情况下linux  /etc/profile会关闭core dump输出

    # No core files by default

    ulimit -S -c 0 > /dev/null 2>&1

     

    对于c++应用以及java jni 应用, 应该修改/etc/profile文件为

    ulimit -S -c ulimited > /dev/null 2>&1

    或者在.bash_profile文件修改

    Ulimit –c ulimited

     

    默认在应用启动目录下产生core dump文件. 可以通过修改    /proc/sys/kernel/core_pattern

    /home/core/core-%e-%p-%t

    生成特定命名格式的文件 

    另外对于JAVA 应用,core dump会产生hs开头的文件

     

     Spring dm 1.0.1已经发现过多个BUG , 对于这类新应用大家特别小心, 应该全面检查各项输出,包括spring 日志/系统/var/log/message等日志.

        

     对于这些新应用,需要和研发/需求方非常明确支持的JVM, web server, application,OS 以及补丁版本. 一些微小差异也会导致BUG .

  • Hsqldb在单元测试中简单应用

    2009-04-06 12:25:26

    参考:

    http://www.blogjava.net/q5782129100/archive/2008/12/27/248600.html

    http://www.javaeye.com/topic/79802

    下载: http://hsqldb.org/

    Hsqldb 4种模式 可采用server模式退出时不用shutdown持久存储或者memory only模式作为单元测试数据库,可以保证测试原子性。

    ALIBABA B2B研发编写的单元测试代码中,已经有多年生命周期了J

     

    1)     server模式启动

    D:\hsqldb_1_8_0_10\bin>  java -cp ../lib/hsqldb.jar org.hsqldb.Server -database.

    0 file:mydb -dbname.0 xdb

     

    可以看到bin目录下有3个新文件: mydb.properties, mydb.log, mydb.script

       然后另外开一个控制台执行:

    D:\hsqldb_1_8_0_10\bin>   java -cp ../lib/hsqldb.jar org.hsqldb.util.DatabaseMan

    agerSwing

    Failed to load preferences.  Proceeding with defaults:

    选择: server模式, 以及xdb,密码为空.

     

     

    执行创建表以及记录:

     

    create table USER (

        USER_ID varchar(25) not null,

        PASSWORD varchar(25)  not null,

     

        constraint PK_USER primary key (USER_ID)

    );

    INSERT INTO USER VALUES('j2ee','j2ee');

     

     

           编写测试程序:

    import java.sql.Connection;

    import java.sql.DriverManager;

    import java.sql.ResultSet;

    import java.sql.SQLException;

    import java.sql.Statement;

     

    /**

     * TODO Comment of HSQLServerTest

     *

     * @author jianzhao.liangjz

     */

    public class HSQLServerTest {

         public static void main(String[] args){

            System.out.println("begin...");

            Connection connection = null;

            Statement stmt = null;

            ResultSet rs = null;

            try {

                Class.forName("org.hsqldb.jdbcDriver");

                connection = DriverManager.getConnection("jdbc:hsqldb:hsql://localhost/xdb", "sa", "");

                stmt = connection.createStatement();

                String sql = "select * from user";

                rs = stmt.executeQuery(sql);

                while (rs.next()) {

                    System.out.println("user_id=" + rs.getString("USER_ID"));

                    System.out.println("password=" + rs.getString("PASSWORD"));

                }

            } catch (Exception e) {

                e.printStackTrace();

            } finally {

                try {

                    if (rs != null)

                        rs.close();

                    if (stmt != null) {

                        stmt.close();

                    }

                    if (connection != null) {

                        connection.close();

                    }

                } catch (Exception e) {

                    e.printStackTrace();

                }

            }

        }

    }

     

    D:\hsqldb_1_8_0_10\bin>javac  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLServerTest.

    java

     

    D:\hsqldb_1_8_0_10\bin>

    D:\hsqldb_1_8_0_10\bin>java  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLServerTest

    begin...

    user_id=j2ee

    password=j2ee

     

    2)     memory only模式启动以及连接

    能创建表以及insert记录. 但不能开一个DB连接直接读取数据.

    所以MEM 的测试在一个进程内完成:

    import java.sql.Connection;

    import java.sql.DriverManager;

    import java.sql.ResultSet;

    import java.sql.SQLException;

    import java.sql.Statement;

     

    public class HSQLMemTest{

        public static void main(String[] args){

            try {

                 Class.forName("org.hsqldb.jdbcDriver" );           

               

                 Connection  c = DriverManager.getConnection("jdbc:hsqldb:mem:memdb", "sa", "");

                 if(c != null){

                     System.out.println("Connected db success!");

                     String sql = "create table USER (USER_ID varchar(25) not null, PASSWORD varchar(25)  not null,constraint PK_USER primary key (USER_ID) );";

                     Statement st = c.createStatement();

                     st.execute(sql);

                     sql = "INSERT INTO USER  VALUES('j2ee','j2ee');";

                     st.executeUpdate(sql);

                      

                      String sql_query = "select * from USER";

                    ResultSet rs = st.executeQuery(sql_query);

                               while (rs.next()) {

                                  System.out.println("user_id=" + rs.getString("USER_ID"));

                                  System.out.println("password=" + rs.getString("PASSWORD"));

                              }

                               if(st != null){

                         st.close();

                     }

                     c.close();

                 }

     

             } catch (Exception e) {

                 System.out.println("ERROR: failed to load HSQLDB JDBC driver.");

                 e.printStackTrace();

                return;

             }

     

         }

    }

    D:\hsqldb_1_8_0_10\bin>javac  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLMemTest.jav

    a

     

    D:\hsqldb_1_8_0_10\bin>java  -cp %CLASSPATH%;../lib/hsqldb.jar  HSQLMemTest

     

  • API测试排列组合的威力

    2009-03-14 11:41:11

     
    当下在做一个客户端编码方法调用为特定格式,通过http发送给服务器端. 服务器接收并翻译为内部方法调用,并执行方法,再响应给客户端的项目.
    即当下很红火的HTTP API测试.

    原始需求:
    支持JSON格式,XML格式,Hessian,http,param格式.

    JSON协议,xml格式,hessian 支持java原始类型,数组,日期以及List,Map类型
    http ,param协议仅仅支持原始类型.

    有心的朋友可能会发散:
    1) java 数组支持多少维? >=2维支持么?
    2) 日期支持字符串形式的格式? 具体的格式要包含时区么?
    3) 范型的List/Map 支持么?
    4) 内部嵌套的List/Map 支持么,比如 List类型内部嵌入Map?
    5) 支持null值否?
    6) ...

    随着一系列的问题抛出,你将会发现一个庞大的正交矩阵.

  • [论坛] jmeter 之短板以及建议解决方案

    2009-02-14 10:51:27

    随着JMeter的应用,发现JMeter的局限性越来越多,急需进一步扩展改进
           
    一 几百兆的sample 日志解析出现OutOfMemory

    最近的几个项目都是Java sample 日志,应用都是高达300 tps的,而响应时间都在百毫秒级别,所以在 <60分钟的运行过程中,生成JMeter 采样日志到达几百兆。
    用JMeter gui解析日志,多次出现OutOfMemory,不爽。
    规避但不治本方法:
    1) 放到>4G 内存的LINUX 机器上, 设置-Xmx2048m甚至更高启动JMeter.sh
    2) 放到64位的java 版本上
    3) 减少java sample运行时间或者次数,减少日志尺寸
    4) 对要求长时间的场景,采用shell 方式启动-关闭jmeter-重命名生成日志 的方式减少日志尺寸

    最根本方法:改写jmeter日志解析部分为NONE GUI,或者用C/c++效率更高的语言解析有规律日志

    二 分布式多台监控机器

    这个也不是jmeter 的长处。尤其是要求监控iowait%,netstat 连接数,NAS 上监控数据。

    解决办法:
    用loadrunner monitor+扩展DLL。

    三 被测程序为client API

    由于JMeter 运行消耗资源较大,无法清晰区分client api本身是否有短期对象、内存泄漏。

    在确认Client api自身没有并发问题、内存泄漏、短期对象问题后,
    可以client api内部加入一些度量数据输出到excel  +  结合jmeter获取更多的平均值、标准差等数据

    四  面向目标的场景控制

    比如要求控制服务器的资源在一定负载下。如要求linux 机器load 接近5时,求解TPS为多少?

    由于系统受到应用CACHE,OS CACHE,NAS CACHE 等影响,单纯采用JMeter 将耗费极大精力。

    解决方法:
    用java 多线程程序发起压力+另外线程检索/proc 目录数据视负载增加/减少压力线程数。同时将变化的线程数与资源负载输出到文件,将这些数据做趋势分析。

    再用线程数应用在jmeter上 反馈验证
  • jdk1.5.0_14+Eclipse SDK3.4.1+TPTP部署

    2009-01-20 13:15:13

    http://www.eclipse.org/tptp/home/downloads/

    下载tptp allinon包: tptp.runtime.allInOne.win32.win32.x86-TPTP-4.5.1.zip

     

    偶尝试了一下逐一安装TPTP,结果是失败。

     

    下载 agntctrl.win_ia32-TPTP-4.5.1.zip  (runtime版本)设置agntctrl

    D:\eclipse_tptp\agntctrl.win_ia32-TPTP-4.5\bin 放到PATH环境变量

    然后执行setconfig.bat设置环境变量

     

    启动代理 acserver

    D:\eclipse_tptp\agntctrl.win_ia32-TPTP-4.5.0\bin>acserver

     

    下载javaprofiler.win_ia32-TPTP-4.5.1.zip   (runtime版本)

    按照说明设置环境变量:

     

    a) Set TPTP_AC_HOME=<<Agent Controller Home>>

    b) Set JAVA_PROFILER_HOME=%TPTP_AC_HOME%\plugins\org.eclipse.tptp.javaprofiler

    c) Set PATH=%JAVA_PROFILER_HOME%;%PATH%;%TPTP_AC_HOME%\bin

    d) Set PATH=%PATH%;%JAVA_HOME%\bin

     

     

    启动JAVA 程序: java -XrunpiAgent:server=enabled CarModel

     

    eclipse 工程可以看到agent

     

     

     

    这里不支持probe insertion

     

     

    远程profile TOMCAT

    catalina.bat java 启动参数加入:-XrunpiAgent:server=enabled

     

    如:

    rem Execute Java with the applicable properties

    if not "%JPDA%" == "" goto doJpda

    if not "%SECURITY_POLICY_FILE%" == "" goto doSecurity

    echo "run OK"

    %_EXECJAVA% %JAVA_OPTS%  -XrunpiAgent:server=enabled

     

     

    然后eclipse ->run->profile configuration->attach to  agent

     

     

     

    实例:

    http://www.eclipse.org/tptp/home/documents/tutorials/profilingtool/profilingexample_32.html

421/3123>
Open Toolbar