TP #2 : Pipelines gitlab
Setting up a GitLab Project
Difficulty: Easy- Create a
.gitlab-ci.ymlfile in the project’s root directory that contains:
stages:
- prepare
image: openjdk:17-slim
variables:
LIB_FOLDER: "lib"
SRC_FOLDER: "src"
TEST_FOLDER: "tests"
BUILD_FOLDER: "out"
JUNIT_REMOTE: "https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.9.3/junit-platform-console-standalone-1.9.3.jar"
JUNIT_LOCAL: "$LIB_FOLDER/"junit-platform-console-standalone.jar"
JACOCO_REMOTE: "https://search.maven.org/remotecontent?filepath=org/jacoco/jacoco/0.8.13/jacoco-0.8.13.zip"
JACOCO_LOCAL_ZIP: $LIB_FOLDER/"jacoco.zip"
before_script:
- apt-get update && apt-get install -y curl unzip
- mkdir -p lib
- curl $JUNIT_REMOTE -o $JUNIT_LOCAL
- curl -L $JACOCO_REMOTE -o $JACOCO_LOCAL_ZIP
- mkdir -p lib/jacoco
- unzip $JACOCO_LOCAL_ZIP -d $LIB_FOLDER/jacoco
print-info:
stage: prepare
script:
- echo "Using Java version:"
- java -version
This file means that there is a single stage in the pipeline (prepare). Since gitlab launches virtual machines to build your work, an image is needed. We choose to use a recent version of java (opendjdk:17-slim). Some variables are defined in order to prevent some copy/paste errors and the before_script
stage is used to install some dependencies (in our case junit and jacoco). Then a dummy job print-info is defined to print some information. And this job is triggered by the prepare stage. You can see that the script section of a job is a sequence of commands that will be executed.
- Add the
.gitlab-ci.ymlfile to your repository and commit and push your changes. - On the gitlab website, you should be able to see the pipeline being run and once finished a green checkmark is displayed.
- Add a second job to the
preparestage that displays the content of yourlibfolder (it might be useful for debugging purpose to be sure that you have all the neededjarfiles). - Commit and push your changes
Building your project
Difficulty: RxFrom the rest of this practical, you should reuse and ADAPT compilation commands that have been used in the previous practical (TP1)
- Add a
buildstage and acompilejob that will compile your source files in theout/classesfolder. - Commit your work and see if the pipeline is triggered and successful on
gitlab.
Testing your project
Difficulty: Rx- Add a
build-testsstage and acompile-testsjob that will compile you test files in theout/test-classesfolder - Commit your work and see if the pipeline is triggered and successful on
gitlab.
It should generate an error because the out/classes/*.class are not found.
- Create an artifact1 in your jobs (
compileandcompile-tests) for theoutfolder with:
artifacts:
paths:
- out/
expire_in: 1 week
Running the tests
Difficulty: Rx- Add a
teststage and arun-testsjob that will run the tests - Commit your work and see if the pipeline is triggered and successful on
gitlab. - Do not forget to add the necessary artifacts in your job.
Getting the code coverage
Difficulty: Rx- Add a
coveragestage and ajacocojob that will run the tests - Commit your work and see if the pipeline is triggered and successful on
gitlab. - Do not forget to add the necessary artifacts in your job.
Do the report
Difficulty: Hard- Add a
reportstage and abuild-reportjob that will usejacocoto generate the report based on thejacoco.execfile from thejacocojob.
xml or csv report are better options than html because they are easier to parse and to extract data.
(Bonus) Extract the code coverage
- In the
reportstage, add a job that will extract the code coverage from the report. You could use any tool you want (grep,xmllint…)
Wrapping up
- Create a
README.mdfile that includes the pipeline status and the code coverage badges (see https://docs.gitlab.com/ee/user/project/badges.html).
Checkstyle
- Download checkstyle…
- … and place it in your
libfolder - Create a
checkstyle.xmlto define the rules your source code should follow (you can use the file detailed at the end of this page) - Run
checkstyleon your source files with the command:
java -jar lib/checkstyle-11.1.0-all.jar -c checkstyle.xml src/
- Be sure that there are no errors
- Create a code smell in
Calculator.javaby modifying theaddmethod with
int c = a, d = b;
return c + d;
- Rerun
checkstyleand see that a code smell[MultipleVariableDeclarations]is issued. - Revert the file
Calculator.java
Adding Stages
- Define a
lintstage in the.gitlab-ci.ymlfile before thebuildstage. - Add a
checkstylejob that will usecheckstyleto check your source code.
Checking for duplicates
- At the root of your
$HOMEfolder, downloadpmdwith the command
curl -L https://github.com/pmd/pmd/releases/download/pmd_releases%2F7.17.0/pmd-dist-7.17.0-bin.zip -o pmd.zip
- Extract the zip file
- In you repository run the
cpdcommande with
~/pmd/bin/pmd cpd --minimum-tokens 80 --dir src/
- Ensure that there are no duplicates
- Create some duplicates in your code and ensure that they are detected with
cpd
Adding a cpd job
- Add a
check-duplicatesjob that will usecpdto check your source code.
Code quality with SonarQube
and create a new project and get the token.
- Download
sonar-scanneron your machine - In your root folder, create a file named
sonar-project.propertiescontaining:
# Project metadata
sonar.projectKey=Your Projet
sonar.projectName=Your Project Name
sonar.projectVersion=1.0
sonar.login=Your Token
sonar.host.url=http://cadmium.polytech-lille.org:9000
# Project base directory (if different from root)
sonar.projectBaseDir=.
# Source files
sonar.sources=src
# Binaries directory (build output)
sonar.java.binaries=out/**
# Test files (optional)
sonar.tests=tests
# Exclude specific folders
sonar.exclusions=**/lib/**
# Language and encoding
sonar.language=java
sonar.sourceEncoding=UTF-8
- In you project folder, launch
sonar-scanner. Once finished get the report at https://sonarqube.polytech-lille.fr/projects
Appendix
checkstyle.xml
<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
"-//Checkstyle//DTD Checkstyle Configuration 1.3//EN"
"https://checkstyle.org/dtds/configuration_1_3.dtd">
<!--
Checkstyle configuration that checks the Google coding conventions from Google Java Style
that can be found at https://google.github.io/styleguide/javaguide.html
Checkstyle is very configurable. Be sure to read the documentation at
http://checkstyle.org (or in your downloaded distribution).
To completely disable a check, just comment it out or delete it from the file.
To suppress certain violations please review suppression filters.
Authors: Max Vetrenko, Ruslan Diachenko, Roman Ivanov.
-->
<module name="Checker">
<module name="SuppressWarningsFilter"/>
<property name="charset" value="UTF-8"/>
<property name="severity" value="warning"/>
<property name="fileExtensions" value="java, properties, xml"/>
<!-- Excludes all 'module-info.java' files -->
<!-- See https://checkstyle.org/config_filefilters.html -->
<module name="BeforeExecutionExclusionFileFilter">
<property name="fileNamePattern" value="module\-info\.java$"/>
</module>
<!-- https://checkstyle.org/config_filters.html#SuppressionFilter -->
<module name="SuppressionFilter">
<property name="file" value="${org.checkstyle.google.suppressionfilter.config}"
default="checkstyle-suppressions.xml" />
<property name="optional" value="true"/>
</module>
<!-- Checks for whitespace -->
<!-- See http://checkstyle.org/config_whitespace.html -->
<module name="FileTabCharacter">
<property name="eachLine" value="true"/>
</module>
<module name="LineLength">
<property name="fileExtensions" value="java"/>
<property name="max" value="100"/>
<property name="ignorePattern" value="^package.*|^import.*|a href|href|http://|https://|ftp://"/>
</module>
<module name="TreeWalker">
<module name="OuterTypeFilename"/>
<module name="IllegalTokenText">
<property name="tokens" value="STRING_LITERAL, CHAR_LITERAL"/>
<property name="format"
value="\\u00(09|0(a|A)|0(c|C)|0(d|D)|22|27|5(C|c))|\\(0(10|11|12|14|15|42|47)|134)"/>
<property name="message"
value="Consider using special escape sequence instead of octal value or Unicode escaped value."/>
</module>
<module name="AvoidEscapedUnicodeCharacters">
<property name="allowEscapesForControlCharacters" value="true"/>
<property name="allowByTailComment" value="true"/>
<property name="allowNonPrintableEscapes" value="true"/>
</module>
<module name="AvoidStarImport"/>
<module name="OneTopLevelClass"/>
<module name="NoLineWrap">
<property name="tokens" value="PACKAGE_DEF, IMPORT, STATIC_IMPORT"/>
</module>
<module name="EmptyBlock">
<property name="option" value="TEXT"/>
<property name="tokens"
value="LITERAL_TRY, LITERAL_FINALLY, LITERAL_IF, LITERAL_ELSE, LITERAL_SWITCH"/>
</module>
<module name="NeedBraces">
<property name="tokens"
value="LITERAL_DO, LITERAL_ELSE, LITERAL_FOR, LITERAL_IF, LITERAL_WHILE"/>
</module>
<module name="LeftCurly">
<property name="tokens"
value="ANNOTATION_DEF, CLASS_DEF, CTOR_DEF, ENUM_CONSTANT_DEF, ENUM_DEF,
INTERFACE_DEF, LAMBDA, LITERAL_CASE, LITERAL_CATCH, LITERAL_DEFAULT,
LITERAL_DO, LITERAL_ELSE, LITERAL_FINALLY, LITERAL_FOR, LITERAL_IF,
LITERAL_SWITCH, LITERAL_SYNCHRONIZED, LITERAL_TRY, LITERAL_WHILE, METHOD_DEF,
OBJBLOCK, STATIC_INIT, RECORD_DEF, COMPACT_CTOR_DEF"/>
</module>
<module name="RightCurly">
<property name="id" value="RightCurlySame"/>
<property name="tokens"
value="LITERAL_TRY, LITERAL_CATCH, LITERAL_FINALLY, LITERAL_IF, LITERAL_ELSE,
LITERAL_DO"/>
</module>
<module name="RightCurly">
<property name="id" value="RightCurlyAlone"/>
<property name="option" value="alone"/>
<property name="tokens"
value="CLASS_DEF, METHOD_DEF, CTOR_DEF, LITERAL_FOR, LITERAL_WHILE, STATIC_INIT,
INSTANCE_INIT, ANNOTATION_DEF, ENUM_DEF, INTERFACE_DEF, RECORD_DEF,
COMPACT_CTOR_DEF"/>
</module>
<module name="SuppressionXpathSingleFilter">
<!-- suppresion is required till https://github.com/checkstyle/checkstyle/issues/7541 -->
<property name="id" value="RightCurlyAlone"/>
<property name="query" value="//RCURLY[parent::SLIST[count(./*)=1]
or preceding-sibling::*[last()][self::LCURLY]]"/>
</module>
<module name="WhitespaceAfter">
<property name="tokens"
value="COMMA, SEMI, TYPECAST, LITERAL_IF, LITERAL_ELSE, LITERAL_RETURN,
LITERAL_WHILE, LITERAL_DO, LITERAL_FOR, LITERAL_FINALLY, DO_WHILE, ELLIPSIS,
LITERAL_SWITCH, LITERAL_SYNCHRONIZED, LITERAL_TRY, LITERAL_CATCH, LAMBDA,
LITERAL_YIELD, LITERAL_CASE"/>
</module>
<module name="WhitespaceAround">
<property name="allowEmptyConstructors" value="true"/>
<property name="allowEmptyLambdas" value="true"/>
<property name="allowEmptyMethods" value="true"/>
<property name="allowEmptyTypes" value="true"/>
<property name="allowEmptyLoops" value="true"/>
<property name="ignoreEnhancedForColon" value="false"/>
<property name="tokens"
value="ASSIGN, BAND, BAND_ASSIGN, BOR, BOR_ASSIGN, BSR, BSR_ASSIGN, BXOR,
BXOR_ASSIGN, COLON, DIV, DIV_ASSIGN, DO_WHILE, EQUAL, GE, GT, LAMBDA, LAND,
LCURLY, LE, LITERAL_CATCH, LITERAL_DO, LITERAL_ELSE, LITERAL_FINALLY,
LITERAL_FOR, LITERAL_IF, LITERAL_RETURN, LITERAL_SWITCH, LITERAL_SYNCHRONIZED,
LITERAL_TRY, LITERAL_WHILE, LOR, LT, MINUS, MINUS_ASSIGN, MOD, MOD_ASSIGN,
NOT_EQUAL, PLUS, PLUS_ASSIGN, QUESTION, RCURLY, SL, SLIST, SL_ASSIGN, SR,
SR_ASSIGN, STAR, STAR_ASSIGN, LITERAL_ASSERT, TYPE_EXTENSION_AND"/>
<message key="ws.notFollowed"
value="WhitespaceAround: ''{0}'' is not followed by whitespace. Empty blocks
may only be represented as '{}' when not part of a multi-block statement (4.1.3)"/>
<message key="ws.notPreceded"
value="WhitespaceAround: ''{0}'' is not preceded with whitespace."/>
</module>
<module name="OneStatementPerLine"/>
<module name="MultipleVariableDeclarations"/>
<module name="ArrayTypeStyle"/>
<module name="MissingSwitchDefault"/>
<module name="FallThrough"/>
<module name="UpperEll"/>
<module name="ModifierOrder"/>
<module name="EmptyLineSeparator">
<property name="tokens"
value="PACKAGE_DEF, IMPORT, STATIC_IMPORT, CLASS_DEF, INTERFACE_DEF, ENUM_DEF,
STATIC_INIT, INSTANCE_INIT, METHOD_DEF, CTOR_DEF, VARIABLE_DEF, RECORD_DEF,
COMPACT_CTOR_DEF"/>
<property name="allowNoEmptyLineBetweenFields" value="true"/>
</module>
<module name="SeparatorWrap">
<property name="id" value="SeparatorWrapDot"/>
<property name="tokens" value="DOT"/>
<property name="option" value="nl"/>
</module>
<module name="SeparatorWrap">
<property name="id" value="SeparatorWrapComma"/>
<property name="tokens" value="COMMA"/>
<property name="option" value="EOL"/>
</module>
<module name="SeparatorWrap">
<!-- ELLIPSIS is EOL until https://github.com/google/styleguide/issues/259 -->
<property name="id" value="SeparatorWrapEllipsis"/>
<property name="tokens" value="ELLIPSIS"/>
<property name="option" value="EOL"/>
</module>
<module name="SeparatorWrap">
<!-- ARRAY_DECLARATOR is EOL until https://github.com/google/styleguide/issues/258 -->
<property name="id" value="SeparatorWrapArrayDeclarator"/>
<property name="tokens" value="ARRAY_DECLARATOR"/>
<property name="option" value="EOL"/>
</module>
<module name="SeparatorWrap">
<property name="id" value="SeparatorWrapMethodRef"/>
<property name="tokens" value="METHOD_REF"/>
<property name="option" value="nl"/>
</module>
<module name="PackageName">
<property name="format" value="^[a-z]+(\.[a-z][a-z0-9]*)*$"/>
<message key="name.invalidPattern"
value="Package name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="TypeName">
<property name="tokens" value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF,
ANNOTATION_DEF, RECORD_DEF"/>
<message key="name.invalidPattern"
value="Type name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="MemberName">
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9]*$"/>
<message key="name.invalidPattern"
value="Member name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="ParameterName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Parameter name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="LambdaParameterName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Lambda parameter name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="CatchParameterName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Catch parameter name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="LocalVariableName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Local variable name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="PatternVariableName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Pattern variable name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="ClassTypeParameterName">
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-z0-9]*[T]$)"/>
<message key="name.invalidPattern"
value="Class type name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="RecordComponentName">
<property name="format" value="^[a-z]([a-z0-9][a-zA-Z0-9]*)?$"/>
<message key="name.invalidPattern"
value="Record component name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="RecordTypeParameterName">
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-z0-9]*[T]$)"/>
<message key="name.invalidPattern"
value="Record type name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="MethodTypeParameterName">
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-z0-9]*[T]$)"/>
<message key="name.invalidPattern"
value="Method type name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="InterfaceTypeParameterName">
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-z0-9]*[T]$)"/>
<message key="name.invalidPattern"
value="Interface type name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="NoFinalizer"/>
<module name="GenericWhitespace">
<message key="ws.followed"
value="GenericWhitespace ''{0}'' is followed by whitespace."/>
<message key="ws.preceded"
value="GenericWhitespace ''{0}'' is preceded with whitespace."/>
<message key="ws.illegalFollow"
value="GenericWhitespace ''{0}'' should followed by whitespace."/>
<message key="ws.notPreceded"
value="GenericWhitespace ''{0}'' is not preceded with whitespace."/>
</module>
<module name="Indentation">
<property name="basicOffset" value="4"/>
<property name="braceAdjustment" value="4"/>
<property name="caseIndent" value="4"/>
<property name="throwsIndent" value="4"/>
<property name="lineWrappingIndentation" value="4"/>
<property name="arrayInitIndent" value="4"/>
</module>
<module name="AbbreviationAsWordInName">
<property name="ignoreFinal" value="false"/>
<property name="allowedAbbreviationLength" value="0"/>
<property name="tokens"
value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, ANNOTATION_DEF, ANNOTATION_FIELD_DEF,
PARAMETER_DEF, VARIABLE_DEF, METHOD_DEF, PATTERN_VARIABLE_DEF, RECORD_DEF,
RECORD_COMPONENT_DEF"/>
</module>
<module name="NoWhitespaceBeforeCaseDefaultColon"/>
<module name="OverloadMethodsDeclarationOrder"/>
<module name="VariableDeclarationUsageDistance"/>
<module name="CustomImportOrder">
<property name="sortImportsInGroupAlphabetically" value="true"/>
<property name="separateLineBetweenGroups" value="true"/>
<property name="customImportOrderRules" value="STATIC###THIRD_PARTY_PACKAGE"/>
<property name="tokens" value="IMPORT, STATIC_IMPORT, PACKAGE_DEF"/>
</module>
<module name="MethodParamPad">
<property name="tokens"
value="CTOR_DEF, LITERAL_NEW, METHOD_CALL, METHOD_DEF,
SUPER_CTOR_CALL, ENUM_CONSTANT_DEF, RECORD_DEF"/>
</module>
<module name="NoWhitespaceBefore">
<property name="tokens"
value="COMMA, SEMI, POST_INC, POST_DEC, DOT,
LABELED_STAT, METHOD_REF"/>
<property name="allowLineBreaks" value="true"/>
</module>
<module name="ParenPad">
<property name="tokens"
value="ANNOTATION, ANNOTATION_FIELD_DEF, CTOR_CALL, CTOR_DEF, DOT, ENUM_CONSTANT_DEF,
EXPR, LITERAL_CATCH, LITERAL_DO, LITERAL_FOR, LITERAL_IF, LITERAL_NEW,
LITERAL_SWITCH, LITERAL_SYNCHRONIZED, LITERAL_WHILE, METHOD_CALL,
METHOD_DEF, QUESTION, RESOURCE_SPECIFICATION, SUPER_CTOR_CALL, LAMBDA,
RECORD_DEF"/>
</module>
<module name="OperatorWrap">
<property name="option" value="NL"/>
<property name="tokens"
value="BAND, BOR, BSR, BXOR, DIV, EQUAL, GE, GT, LAND, LE, LITERAL_INSTANCEOF, LOR,
LT, MINUS, MOD, NOT_EQUAL, PLUS, QUESTION, SL, SR, STAR, METHOD_REF,
TYPE_EXTENSION_AND "/>
</module>
<module name="AnnotationLocation">
<property name="id" value="AnnotationLocationMostCases"/>
<property name="tokens"
value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF,
RECORD_DEF, COMPACT_CTOR_DEF"/>
</module>
<module name="AnnotationLocation">
<property name="id" value="AnnotationLocationVariables"/>
<property name="tokens" value="VARIABLE_DEF"/>
<property name="allowSamelineMultipleAnnotations" value="true"/>
</module>
<module name="NonEmptyAtclauseDescription"/>
<module name="RequireEmptyLineBeforeBlockTagGroup"/>
<module name="AtclauseOrder">
<property name="tagOrder" value="@param, @return, @throws, @deprecated"/>
<property name="target"
value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF, VARIABLE_DEF"/>
</module>
<module name="MethodName">
<property name="format" value="^[a-z][a-z0-9]\w*$"/>
<message key="name.invalidPattern"
value="Method name ''{0}'' must match pattern ''{1}''."/>
</module>
<module name="EmptyCatchBlock">
<property name="exceptionVariableName" value="expected"/>
</module>
<module name="CommentsIndentation">
<property name="tokens" value="SINGLE_LINE_COMMENT, BLOCK_COMMENT_BEGIN"/>
</module>
<!-- https://checkstyle.org/config_filters.html#SuppressionXpathFilter -->
<module name="SuppressionXpathFilter">
<property name="file" value="${org.checkstyle.google.suppressionxpathfilter.config}"
default="checkstyle-xpath-suppressions.xml" />
<property name="optional" value="true"/>
</module>
<module name="SuppressWarningsHolder" />
<module name="SuppressionCommentFilter">
<property name="offCommentFormat" value="CHECKSTYLE.OFF\: ([\w\|]+)" />
<property name="onCommentFormat" value="CHECKSTYLE.ON\: ([\w\|]+)" />
<property name="checkFormat" value="$1" />
</module>
<module name="SuppressWithNearbyCommentFilter">
<property name="commentFormat" value="CHECKSTYLE.SUPPRESS\: ([\w\|]+)"/>
<!-- $1 refers to the first match group in the regex defined in commentFormat -->
<property name="checkFormat" value="$1"/>
<!-- The check is suppressed in the next line of code after the comment -->
<property name="influenceFormat" value="1"/>
</module>
</module>
</module>
Footnotes
-
An artifact is a file that can be downloaded from the pipeline or be reused by other jobs. ↩