服务端持续集成实战

发表于:2020-3-25 10:31

字体: | 上一篇 | 下一篇 | 我要投稿

 作者:cp    来源:搜狗测试

分享:
  前言
  基于Jenkins的服务端持续集成已在搜狗商业产品系统实现,实施流程如下图,今天介绍如何在服务端实施持续集成。
  Jenkins工程配置
  1.新建Jenkins Pipline工程
  New Item -> Pipline
  2.增加以下Params
  3.填入Git地址和JenkinsFile名称
  配置文件创建或修改
  1.build.xml
   <?xml version="1.0" encoding="UTF-8"?>
  <project xmlns:jacoco="antlib:org.jacoco.ant" default="jacoco" name="Jacoco">
  <!--Jacoco 的安装路径,需要放在ant程序所在的目录,否则会报错-->
  <property name="jacocoantPath"
  value=""/>
  <!--最终生成 .exec 文件的路径,Jacoco 就是根据这个文件生成最终的报告的-->
  <property name="integrationJacocoexecPath" value="./jacoco.exec"/>
  <!--合并后exec文件-->
  <property name="allJacocoexecPath" value=""/>
  <!--待合并的exec文件-->
  <property name="baseDir" value=""/>
  <!--生成覆盖率报告的路径-->
  <property name="reportfolderPath" value=""/>
  <!--源代码路径-->
  <property name="checkOrderSrcPath" value="src/main"/>
  <!--.class 文件路径-->
  <property name="checkOrderClasspath" value="build/classes"/>
  <!--让 ant 知道去哪儿找 Jacoco-->
  <target name="dump">
  <jacoco:dump address="" append="false"
  destfile="${integrationJacocoexecPath}"
  port="8044" reset="true"/>
  </target>
  <!--dump 任务:
  根据配置的 Ip 地址,和端口号,
  访问目标 Tomcat 服务,并生成 .exec 文件。-->
  <taskdef resource="org/jacoco/ant/antlib.xml" uri="antlib:org.jacoco.ant">
  <classpath path="${jacocoantPath}"/>
  </taskdef>
  </project>
   2.sonar-project.properties
   sonar.projectKey=rtbmanager:1.0
  # this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1.
  sonar.projectName=rtbmanager-1.0
  sonar.projectVersion=1.0
  # Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
  # This property is optional if sonar.modules is set.
  sonar.sources=src
  sonar.exclusions=**/test/**,**/target/**
  sonar.java.binaries=build/classes/
  sonar.java.source=1.8
  sonar.java.target=1.8
  # Encoding of the source code. Default is default system encoding
  sonar.sourceEncoding=UTF-8
  # Set jacoco Configuration
  # coverage
  sonar.core.codeCoveragePlugin=jacoco
  # exec binary files
  sonar.jacoco.reportPaths=build/jacoco-ut.exec
  # Path to the JaCoCo report file containing coverage data by integration tests(集成测试).
  # The path may be absolute or relative to the project base directory
  sonar.jacoco.itReportPath=cifiles/jacoco.exec
  # additional
  sonar.dynamicAnalysis=reuseReports
  #sonar.jacoco.reportMissing.force.zero=false
  sonar.coverage.exclusions=**/rtb/manager/config/**
  3.Java配置文件dev.conf
 JAVA_OPTS="-javaagent:/search/odin/jacocoagent.jar=includes=com.sogou.*,output=tcpserver,port=8044,address=127.0.0.1,append=true -Xverify:none"
  4.build.gradle/pom.xml
  build.gradle
   //Jacoco
  apply plugin: "jacoco"
  jacoco {
  toolVersion = "0.8.3"
  reportsDir = file("$buildDir/customJacocoReportDir")
  }
  jacocoTestReport {
  reports {
  xml.enabled false
  csv.enabled false
  html.destination file("${buildDir}/jacocoHtml")
  }
  }
  test {
  jacoco {
  destinationFile = file("$buildDir/jacoco-ut.exec")
  classDumpDir = file("$buildDir/classpathdumps")
  }
  }
  Jenkins Pipline文件修改
  1.Build Stage修改
  对于gradle工程来说,单元测试的执行在编译过程就会执行。
 ./gradlew build -Pprofile=${profile}
  该命令即可在编译过程执行单元测试,单元测试通过编译成功,反之失败。
  2.UnitTest Stage修改
  该stage用于单元测试代码覆盖率统计。
  修改classPattern参数,改为对应工程需要统计覆盖率类的目录。
  3.SonarQube Scan Stage
  该stage将编译后的程序提交至SonarQube,并根据SonarQube返回的结果判定该本次pipline的执行是否成功
  SonarQube Scanner的使用方式有两种,
  Jenkins插件模式
  已安装SonarQube Scanner插件
 def scannerHome = tool 'sonarqube_scanner';       sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties"
  Jenkins机器手动安装SonarQube Scanner程序(需要在Jenkins机器安装SonarQube Scanner)
 sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner  -D project.settings=cifiles/sonar-project.properties'
  4.ApiTest Stage
  该stage 用于执行接口自动化用例,同时统计其覆盖率,并与单元测试覆盖率合并,最终的覆盖率结果在SonarQube上展现
  1.修改build job: '{project}-apitest'
  2.修改Ant执行方式
  Ant的使用方式有两种,插件模式和手动安装模式,更推荐插件模式,以下是两种模式的代码信息
  插件模式
   withAnt(installation: 'ant'){
  sh 'ant dump -buildfile cifiles/build.xml'
  }
  直接安装
 sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile cifiles/build-dispatcher.xml'
  Jenkins Pipline样例
#!groovy
  node {
  def tomcat_health_suf = ':8080/actuator/health'
  def nginx_health_suf = ':80/actuator/health'
  def max_secs = 30
  stage('git clone') {
  checkout scm
  }
  stage('gradle build') {
  sh './gradlew build -Pprofile=${profile}'
  }
  stage('UnitTest'){
  if (unittest == 'true'){
  echo "deploy jacoco......"
  for (ip in serverIps.split(',')) {
  if (deployJacoco(ip, remote_user) == false) {
  echo '[FAILURE] Failed to deploy jacoco'
  currentBuild.result = 'FAILURE'
  return
  }
  }
  echo "starting unitTest jacocoCoverage......"
  /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/
  jacoco(
  buildOverBuild: false,
  changeBuildStatus: false,
  execPattern: 'build/jacoco-ut.exec',
  classPattern: 'build/classese',
  sourcePattern: 'src/main/java',
  exclusionPattern: 'src/test*',
  minimumMethodCoverage: '10',
  maximumMethodCoverage: '90',
  minimumClassCoverage: '40',
  maximumClassCoverage: '90',
  minimumLineCoverage: '5',
  maximumLineCoverage: '90'
  )
  }
  }
  stage('SonarQube Scan') {
  if (runSonar == 'true') {
  withSonarQubeEnv('sonarqube') {
  //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同
  echo "starting codeAnalyze with SonarQube......"
  echo "WORKSPACE: $WORKSPACE"
  //执行Sonar Scanner
  def scannerHome = tool 'sonarqube_scanner';
  sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties"
  // sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner  -D project.settings=cifiles/sonar-project.properties'
  }
  script {
  timeout(1) {
  //这里设置超时时间1分钟,如果Sonar Webhook失败,不会出现一直卡在检查状态
  //利用Sonar webhook功能通知pipeline代码检测结果,未通过质量阈,pipeline将会fail
  def qg = waitForQualityGate('sonarqube')
  //注意:这里waitForQualityGate()中的参数也要与之前SonarQube servers中Name的配置相同
  if (qg.status != 'OK') {
  error "未通过Sonarqube的代码质量阈检查,请及时修改!failure: ${qg.status}"
  }
  }
  }
  }
  }
  if (confirming == 'true') {
  stage('staging') {
  if (deploy(stagingIp, remote_user, "staging", tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) {
  echo '[FAILURE] Failed to build'
  currentBuild.result = 'FAILURE'
  return
  }
  }
  }
  stage('deploy') {
  if (confirming == 'true') {
  input 'make sure to publish?'
  }
  for (ip in serverIps.split(',')) {
  if (deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) {
  echo '[FAILURE] Failed to build'
  currentBuild.result = 'FAILURE'
  return
  }
  }
  }
  stage('ApiTest'){
  if (apiTest == 'true') {
  build job: 'rtbmanager-apitest'
  echo 'starting ant dump......'
  withAnt(installation: 'ant'){
  sh 'ant dump -buildfile cifiles/build.xml'
  }
  // sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile /usr/local/software/apache-ant-1.9.14/build-rtbmanager.xml'
  echo 'starting apiTest jacocoCoverage......'
  /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/
  jacoco(
  buildOverBuild: false,
  changeBuildStatus: false,
  execPattern: 'cifiles/jacoco.exec',
  classPattern: 'build/classes',
  sourcePattern: 'src/main/java',
  exclusionPattern: 'src/test*',
  minimumMethodCoverage: '70',
  maximumMethodCoverage: '90',
  minimumClassCoverage: '80',
  maximumClassCoverage: '90',
  minimumLineCoverage: '60',
  maximumLineCoverage: '90'
  )
  withSonarQubeEnv('sonarqube') {
  //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同
  echo "starting codeAnalyze with SonarQube......"
  //统计接口测试覆盖率,并同步至SonarQube
  def scannerHome = tool 'sonarqube_scanner';
  sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties"
  }
  }
  }
  }
  def deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) {
  if (nginx == 'true') {
  sh "ssh ${remote_user}@${ip} \"sudo systemctl stop openresty.service\""
  }
  sh "scp build/libs/app.jar ${remote_user}@${ip}:/search/odin/app/app.jar"
  sh "scp envfiles/${profile}.conf ${remote_user}@${ip}:/search/odin/app/app.conf"
  sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\""
  if (check(ip + tomcat_health_suf, max_secs, 8) == false) {
  return false
  }
  if (nginx == 'true') {
  sh "ssh ${remote_user}@${ip} \"sudo systemctl start openresty.service\""
  return check(ip + nginx_health_suf, max_secs, 1)
  }
  return true
  }
  def deployJacoco(ip, remote_user) {
  sh "scp -r cifiles/jacocoagent.jar ${remote_user}@${ip}:/search/odin/"
  sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\""
  return true
  }
  def check(url, max, initial_sleep_secs) {
  if (!url.startsWith("http")) {
  url = 'http://' + url
  }
  def cmd = $/eval "curl -s ${url} | sed 's/ //g' | grep '\"status\":\"UP\"' | wc -l"/$
  echo "${cmd}"
  def rc = "0";
  sleep(initial_sleep_secs); // seconds
  try {
  rc = sh(
  script: "${cmd}",
  returnStdout: true
  ).trim();
  echo rc
  } catch (Exception e) {
  }
  def i = 0;
  while (rc.equals("0") && i < max) {
  sleep(1); // seconds
  try {
  rc = sh(
  script: "${cmd}",
  returnStdout: true
  ).trim();
  echo rc
  } catch (Exception e) {
  }
  i++;
  }
  echo rc
  return rc.equals("0") ? false : true;
  }
  服务端持续集成效果展示
  Jenkins持续集成构建结果:

      本文内容不用于商业目的,如涉及知识产权问题,请权利人联系博为峰小编(021-64471599-8017),我们将立即处理
《2023软件测试行业现状调查报告》独家发布~

关注51Testing

联系我们

快捷面板 站点地图 联系我们 广告服务 关于我们 站长统计 发展历程

法律顾问:上海兰迪律师事务所 项棋律师
版权所有 上海博为峰软件技术股份有限公司 Copyright©51testing.com 2003-2024
投诉及意见反馈:webmaster@51testing.com; 业务联系:service@51testing.com 021-64471599-8017

沪ICP备05003035号

沪公网安备 31010102002173号