前言
基于Jenkins的服务端持续集成已在搜狗商业产品系统实现,实施流程如下图,今天介绍如何在服务端实施持续集成。
Jenkins工程配置
1.新建Jenkins Pipline工程
New Item -> Pipline
2.增加以下Params
3.填入Git地址和JenkinsFile名称
配置文件创建或修改
1.build.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns:jacoco="antlib:org.jacoco.ant" default="jacoco" name="Jacoco"> <!--Jacoco 的安装路径,需要放在ant程序所在的目录,否则会报错--> <property name="jacocoantPath" value=""/> <!--最终生成 .exec 文件的路径,Jacoco 就是根据这个文件生成最终的报告的--> <property name="integrationJacocoexecPath" value="./jacoco.exec"/> <!--合并后exec文件--> <property name="allJacocoexecPath" value=""/> <!--待合并的exec文件--> <property name="baseDir" value=""/> <!--生成覆盖率报告的路径--> <property name="reportfolderPath" value=""/> <!--源代码路径--> <property name="checkOrderSrcPath" value="src/main"/> <!--.class 文件路径--> <property name="checkOrderClasspath" value="build/classes"/> <!--让 ant 知道去哪儿找 Jacoco--> <target name="dump"> <jacoco:dump address="" append="false" destfile="${integrationJacocoexecPath}" port="8044" reset="true"/> </target> <!--dump 任务: 根据配置的 Ip 地址,和端口号, 访问目标 Tomcat 服务,并生成 .exec 文件。--> <taskdef resource="org/jacoco/ant/antlib.xml" uri="antlib:org.jacoco.ant"> <classpath path="${jacocoantPath}"/> </taskdef> </project> |
2.sonar-project.properties
sonar.projectKey=rtbmanager:1.0 # this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1. sonar.projectName=rtbmanager-1.0 sonar.projectVersion=1.0 # Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows. # This property is optional if sonar.modules is set. sonar.sources=src sonar.exclusions=**/test/**,**/target/** sonar.java.binaries=build/classes/ sonar.java.source=1.8 sonar.java.target=1.8 # Encoding of the source code. Default is default system encoding sonar.sourceEncoding=UTF-8 # Set jacoco Configuration # coverage sonar.core.codeCoveragePlugin=jacoco # exec binary files sonar.jacoco.reportPaths=build/jacoco-ut.exec # Path to the JaCoCo report file containing coverage data by integration tests(集成测试). # The path may be absolute or relative to the project base directory sonar.jacoco.itReportPath=cifiles/jacoco.exec # additional sonar.dynamicAnalysis=reuseReports #sonar.jacoco.reportMissing.force.zero=false sonar.coverage.exclusions=**/rtb/manager/config/** |
3.Java配置文件dev.conf
JAVA_OPTS="-javaagent:/search/odin/jacocoagent.jar=includes=com.sogou.*,output=tcpserver,port=8044,address=127.0.0.1,append=true -Xverify:none" |
4.build.gradle/pom.xml
build.gradle
//Jacoco apply plugin: "jacoco" jacoco { toolVersion = "0.8.3" reportsDir = file("$buildDir/customJacocoReportDir") } jacocoTestReport { reports { xml.enabled false csv.enabled false html.destination file("${buildDir}/jacocoHtml") } } test { jacoco { destinationFile = file("$buildDir/jacoco-ut.exec") classDumpDir = file("$buildDir/classpathdumps") } } |
Jenkins Pipline文件修改
1.Build Stage修改
对于gradle工程来说,单元测试的执行在编译过程就会执行。
./gradlew build -Pprofile=${profile} |
该命令即可在编译过程执行单元测试,单元测试通过编译成功,反之失败。
2.UnitTest Stage修改
该stage用于单元测试代码覆盖率统计。
修改classPattern参数,改为对应工程需要统计覆盖率类的目录。
3.SonarQube Scan Stage
该stage将编译后的程序提交至SonarQube,并根据SonarQube返回的结果判定该本次pipline的执行是否成功
SonarQube Scanner的使用方式有两种,
Jenkins插件模式
已安装SonarQube Scanner插件
def scannerHome = tool 'sonarqube_scanner'; sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties" |
Jenkins机器手动安装SonarQube Scanner程序(需要在Jenkins机器安装SonarQube Scanner)
sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties' |
4.ApiTest Stage
该stage 用于执行接口自动化用例,同时统计其覆盖率,并与单元测试覆盖率合并,最终的覆盖率结果在SonarQube上展现
1.修改build job: '{project}-apitest'
2.修改Ant执行方式
Ant的使用方式有两种,插件模式和手动安装模式,更推荐插件模式,以下是两种模式的代码信息
插件模式
withAnt(installation: 'ant'){ sh 'ant dump -buildfile cifiles/build.xml' } |
直接安装
sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile cifiles/build-dispatcher.xml' |
Jenkins Pipline样例
#!groovy node { def tomcat_health_suf = ':8080/actuator/health' def nginx_health_suf = ':80/actuator/health' def max_secs = 30 stage('git clone') { checkout scm } stage('gradle build') { sh './gradlew build -Pprofile=${profile}' } stage('UnitTest'){ if (unittest == 'true'){ echo "deploy jacoco......" for (ip in serverIps.split(',')) { if (deployJacoco(ip, remote_user) == false) { echo '[FAILURE] Failed to deploy jacoco' currentBuild.result = 'FAILURE' return } } echo "starting unitTest jacocoCoverage......" /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/ jacoco( buildOverBuild: false, changeBuildStatus: false, execPattern: 'build/jacoco-ut.exec', classPattern: 'build/classese', sourcePattern: 'src/main/java', exclusionPattern: 'src/test*', minimumMethodCoverage: '10', maximumMethodCoverage: '90', minimumClassCoverage: '40', maximumClassCoverage: '90', minimumLineCoverage: '5', maximumLineCoverage: '90' ) } } stage('SonarQube Scan') { if (runSonar == 'true') { withSonarQubeEnv('sonarqube') { //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同 echo "starting codeAnalyze with SonarQube......" echo "WORKSPACE: $WORKSPACE" //执行Sonar Scanner def scannerHome = tool 'sonarqube_scanner'; sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties" // sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties' } script { timeout(1) { //这里设置超时时间1分钟,如果Sonar Webhook失败,不会出现一直卡在检查状态 //利用Sonar webhook功能通知pipeline代码检测结果,未通过质量阈,pipeline将会fail def qg = waitForQualityGate('sonarqube') //注意:这里waitForQualityGate()中的参数也要与之前SonarQube servers中Name的配置相同 if (qg.status != 'OK') { error "未通过Sonarqube的代码质量阈检查,请及时修改!failure: ${qg.status}" } } } } } if (confirming == 'true') { stage('staging') { if (deploy(stagingIp, remote_user, "staging", tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) { echo '[FAILURE] Failed to build' currentBuild.result = 'FAILURE' return } } } stage('deploy') { if (confirming == 'true') { input 'make sure to publish?' } for (ip in serverIps.split(',')) { if (deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) { echo '[FAILURE] Failed to build' currentBuild.result = 'FAILURE' return } } } stage('ApiTest'){ if (apiTest == 'true') { build job: 'rtbmanager-apitest' echo 'starting ant dump......' withAnt(installation: 'ant'){ sh 'ant dump -buildfile cifiles/build.xml' } // sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile /usr/local/software/apache-ant-1.9.14/build-rtbmanager.xml' echo 'starting apiTest jacocoCoverage......' /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/ jacoco( buildOverBuild: false, changeBuildStatus: false, execPattern: 'cifiles/jacoco.exec', classPattern: 'build/classes', sourcePattern: 'src/main/java', exclusionPattern: 'src/test*', minimumMethodCoverage: '70', maximumMethodCoverage: '90', minimumClassCoverage: '80', maximumClassCoverage: '90', minimumLineCoverage: '60', maximumLineCoverage: '90' ) withSonarQubeEnv('sonarqube') { //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同 echo "starting codeAnalyze with SonarQube......" //统计接口测试覆盖率,并同步至SonarQube def scannerHome = tool 'sonarqube_scanner'; sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties" } } } } def deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) { if (nginx == 'true') { sh "ssh ${remote_user}@${ip} \"sudo systemctl stop openresty.service\"" } sh "scp build/libs/app.jar ${remote_user}@${ip}:/search/odin/app/app.jar" sh "scp envfiles/${profile}.conf ${remote_user}@${ip}:/search/odin/app/app.conf" sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\"" if (check(ip + tomcat_health_suf, max_secs, 8) == false) { return false } if (nginx == 'true') { sh "ssh ${remote_user}@${ip} \"sudo systemctl start openresty.service\"" return check(ip + nginx_health_suf, max_secs, 1) } return true } def deployJacoco(ip, remote_user) { sh "scp -r cifiles/jacocoagent.jar ${remote_user}@${ip}:/search/odin/" sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\"" return true } def check(url, max, initial_sleep_secs) { if (!url.startsWith("http")) { url = 'http://' + url } def cmd = $/eval "curl -s ${url} | sed 's/ //g' | grep '\"status\":\"UP\"' | wc -l"/$ echo "${cmd}" def rc = "0"; sleep(initial_sleep_secs); // seconds try { rc = sh( script: "${cmd}", returnStdout: true ).trim(); echo rc } catch (Exception e) { } def i = 0; while (rc.equals("0") && i < max) { sleep(1); // seconds try { rc = sh( script: "${cmd}", returnStdout: true ).trim(); echo rc } catch (Exception e) { } i++; } echo rc return rc.equals("0") ? false : true; } |
服务端持续集成效果展示
Jenkins持续集成构建结果:
本文内容不用于商业目的,如涉及知识产权问题,请权利人联系博为峰小编(021-64471599-8017),我们将立即处理