CONTINUOUS INTEGRATION SERVERS

TABLE OF CONTENTS

  • What is Continuous integration ?
  • Why we use CI servers?
  • TeamCity
  • TravisCI
  • Jenkins

What is Continuous integration?

What is Continuous integration

  • The practice, of merging all developer working copies to a shared mainline several times a day
  • Adopted as part of extreme programming (XP), which did advocate integrating more than once per day, perhaps as many as tens of times per day
  • Reduces the risks
  • Prevent integration problems. Improvement over frequent integration

What is NOT

  • Nigthly builds
  • Developer branches
  • Scheduled integrations points
  • Building via IDEs

Software Risks

  • Fixing bugs late is costly
  • Lack of team cohesion - "I thought you fixed that 2 months ago !?"
  • Poor quality code base - "Why can't I just include Foo and not require all of the other 10 libs?"

Software Risks (2)

  • Lack of deployable software
    • "It works on my machine!"
    • "The customer is coming, we need to demo ASAP!"
  • Lack of project visibility
    • "What do you mean the tests are failing?"
    • "What's our code coverage now?"

Main principles

  • Maintain a code repository
    • Use of a revision control system for the project's source code
  • Automate the build
    • A single command should have the capability of building the system
  • Make the build self-testing
  • Confirm that code behaves as expected
  • Through repeatable testing

Main principles (2)

  • Everyone commits to the baseline every day
    • Every committer should reduce the number of conflicting changes
  • Every commit (to baseline) should be built
    • System build commits to the current working version to verify that they integrate correctly
  • Keep the build fast
    • Quickly identify problems with integration

Main principles (2)

  • Test in a clone of the production environment
    • Strict test environment can lead to failures in tested systems when they deploy in the production environment
  • Make it easy to get the latest deliverables
    • Available to stakeholders and testers
  • Everyone can see the results of the latest build
    • Easy to find out whether the build breaks and, if so, why
  • Automate deployment
    • Running scripts after a build finishes to deploy to upper environment

Multi-stage CI

  • Technique intended to achieve highly integrated parallel development activity while reducing the scope of integration problems
  • Software moves in stages from a state of immaturity to a state of maturity, and the work is broken down into logical units performed by interdependent teams that integrate the different teams together over time

Recommended practices

  • Each developer works on their own task
  • Feature branches

Benefits

  • Rapid Feedback
  • Decreased time in the debugger
  • Identification of bugs early
  • Localization of bugs
  • Decreases the need for or length of code freezes
  • The product is always in a stable/releasable state (goes really well with Scrum's Potentially Shippable Product)
  • It is easy to get a new developer's system UD and running and keep it up to date

CI patterns

  • Active development line
    • Enforce policies that are effective in making your main development tine stable enough
  • Developer workspace
    • Define a structured workspace and allow for variation in the content and not the structure

CI patterns (2)

  • Integration build
    • All changes (and their dependencies) are built using a central integration build process
  • Private workspace
    • Where you control the versions of code and components

CI patterns (3)

  • Private build system
    • Before making a submission to source control, build the system using a Private System Build similar to the nightly build
  • Unit tests
    • Develop and run unit tests
  • Task level commit
    • Do one commit per small-grained, consistent task
  • Communication lines
    • Build such bridges between teams and stakeholders

CI patterns (4)

  • Repository
    • Single point of access, or a repository, for your code and related artifacts. At least Gitolite.
  • Third party codeline
    • Create workspaces and installation kits from this codeline

First steps

  • Commit early, commit often
  • NEVER commit broken code
  • Fix build failures immediately S
  • Start small, Fail fast
  • Act on metrics
  • Build in every target environment
  • Consider Build-Once-Deploy-Many
  • Create artifacts from every build (reports, binaries, etc.)

CI servers

Why we use CI servers?

  • Fosters team collaboration, with a UI interface that is friendly and usable across all teams
  • Supports manual and automated tasks
  • Easy to use Dev, QA, DBAs, release managers etc. Not just automation gurus
  • Views relevant to job function (release calendar, release pipeline, release tasks)

Why we use CI servers (2)

  • Integrates smoothly with ALM, change/release management and other process tools
  • Supports multiple pipelines, release trains, dependencies and other complex delivery patterns, not just single pipelines

CI servers

  • Define your end-to-end process in 30 min, not 30 hours
  • Tweak running pipelines to understand where "reality gets in the way of the plan"
  • nables data-driven improvement: provides metrics and analytics to speed up our pipeline
  • Comprehensive visual dashboards and reports

CI servers (2)

  • Scales to hundreds of users and pipelines in a highly available setup
  • Security model appropriate for enterprise CD: accessible to anyone in the enterprise, with controls
  • Audit trail for all changes to pipeline definitions and running releases

CI servers (3)

  • Integrations with all the types of tools encountered from Dev to Prod - not just build and test tools
  • Easy to create custom extensions without learning a complex plugin API
  • Simplified maintenance - no unpredictable upgrades, incompatible extensions etc.

Build agent

  • Introduce build servers, which automatically ran the unit tests periodically or even after every commit and report the results to the developers

Test automation

  • In addition to automated unit tests, teams using CI typically use a build server to implement continuous processes of applying quality control in general small pieces of effort, applied frequently
  • In addition to running the unit and integration tests, such processes run additional static and dynamic tests, measure and profile performance, extract and format documentation from the source code and facilitate manual QA processes

TeamCity

TeamCity

  • Continuous integration tool
    • Quality control
    • Team cooperation
    • IDE-independent
    • Java + .Net
  • Rich integrations with other tools - Ant, Maven, IDEs etc.
  • Easy administration
  • Multi-platform testing
  • Optimized task distribution
  • Easy Horizontal Scaling of agents

TeamCity (3)

  • Indicates explicitly who is fixing the build
  • Direct links to source code
  • Server checks for quality - Coverage and Duplications
  • Integrates local changes before pushing to VCS

TravisCI

TravisCI

  • Travis workers, VMs that run the builds, are distributed
  • Can use your GitHub account
    • Just enable repos, that you want Travis to build
  • Announce your build status
    • Emails, IRC, Slack etc.
  • To build just add .travis.yaml to your project

Jenkins

Jenkins

  • Written in Java and initially was supposed to be used as a CI tool
  • Over 600 plugins to customize Jenkins as per your need
  • Over 1000+ public repositories on Github, 500+ contributors, strong commit activity
  • Free open source and most widely used tool for maintaining continuous integration cycle
  • Wide OS platforms support

Jenkins (2)

  • Easy install, easy upgrade, easy configuration
  • Distributed builds. Arguably its most powerful feature.
  • Monitoring external jobs
  • No limit to the number of jobs, number of slave nodes
  • Plugin architecture
    • Support for various version control systems, authentication methods, notification, workflow building, and many more features can be added
  • Jenkins provides machine-consumable remote access API to its functionalities (can trigger jobs from Skype)

Jenkins (4)

  • General Phases
    • Unit Test
    • Code Quality Analysis
    • Deploy to Test Environment
    • Integration Test
    • Packaging and Archiving
    • Deploy to Pre-production Environment
    • Acceptance Test
    • Deploy to Production Environment

Jenkins (5)

  • With the help of Jenkins, we can create customized build pipeline to create a dashboard of our own and enable Continuous Delivery in easy steps
  • Advanced: we can split jobs with multiple code versions, test them simultaneously and based on results - deploy the most applicable one!
office@e-dojo.it

EXERCISE

Problem 1: Setup Jenkins

Download and setup local Jenkins server.

Problem 2: My First Jenkins Project

Create and configure simple Test project. Challenge: Run all your tests you have created so far, use your Github repo.