Unit testing

TABLE OF CONTENTS

  • What are unit tests and frameworks?
  • Unit testing Frameworks
  • Solving practical problems with NUnit
  • Good practices
  • Test-driven development
  • Mocking

What it is?

Unit Testing

“...software testing method by which individual units of source code...are tested to determine whether they are fit for use.”
~ Wikipedia

Unit Testing (2)

    Why we unit test?
  • Decrease the number of defects in the code
  • Improve design and in the same time the design is the major obstacle for unit testing (poor testability)
  • Unit tests are good documentation
  • Unit tests reduce the cost of change
  • Unit tests support fearless refactoring

Unit Testing (3)

  • Tests are specific pieces of code
  • Written by developers, not by QA engineers
  • Tests live into the code repository along with the code they test
  • Unit testing framework is needed

Unit Testing (4)

  • All classes / methods should be tested (private methods are debatable)
  • All unit tests should pass before committing in the source control repository

public void Withdraw(double amount)
{
    if(m_balance >= amount)
    {
        m_balance -= amount;
    }
    else
    {
        throw new ArgumentException(amount, "Withdrawal exceeds balance!")
    }
}
					

[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void Withdraw_AmountMoreThanBalance_Throws()
{
    // arrange
    var account = new CheckingAccount("John Doe", 10.0);
    // act
    account.Withdraw(20.0);
    // assert is handled by the ExpectedException
}
						

Frameworks

Frameworks

  • Unit testing frameworks help simplify the process of unit testing.
  • You can write unit tests without using any framework.
    • Slower
    • And more error-prone.

Frameworks

NUnit

What is NUnit?

  • It is xUnit based unit testing tool for all Microsoft .NET languages. The current production release, version 3.5, the first version of NUnit where the framework will be released separately from the console runner, engine and other extensions.
  • It is written entirely in C# and has been completely redesigned to take advantage of many .NET language features, for example custom attributes and other reflection related capabilities.

NUnit Features

    Categories
  • When categories are used, only the tests in the selected categories will be run.
  • 
    namespace NUnit.Tests
    {
     using System;
     using NUnit.Framework;
    
     [TestFixture]
     [Category("LongRunning")]
     public class LongRunningTests
     {
       // ...
     }
    }
     						 

NUnit Features (2)

    Categories
  • The Category attribute provides an alternative to suites for dealing with groups of tests.
  • 
    namespace NUnit.Tests
    {
      using System;
      using NUnit.Framework;
    
      [TestFixture]
      public class SuccessTests
      {
        [Test]
        [Category("Long")]
        public void VeryLongTest()
        { /* ... */ }
    }
     						 

NUnit Features (3)

    Custom Category Attributes
  • Beginning with NUnit 2.4, it is possible to define custom attributes that derive from CategoryAttribute and have them recognized by NUnit.
  • 
    [AttributeUsage(AttributeTargets.Method, AllowMultiple=false)]
    public class CriticalAttribute : CategoryAttribute { }
    
    ...
    
    [Test, Critical]
    public void MyTest()
    { /*...*/ }
    					 

NUnit Features (4)

    Attributes
  • NUnit uses custom attributes to identify tests.
    • ValueSource - used on individual parameters of a test method to identify a named source for the argument values to be supplied.
    • TestCaseSource - used on a parameterized test method to identify the source from which the required arguments will be provided.
    • Range - used to specify a range of values to be provided for an individual parameter of a parameterized test method.

NUnit Features (5)

  • Random - used to specify a set of random values to be provided for an individual parameter of a parameterized test method.
  • TestCase - serves the dual purpose of marking a method with parameters as a test method and providing inline data to be used when invoking that method.

NUnit Features (5)

    Assertions
  • Equality
    • AreEqual
    • AreNotEqual
  • Identity
    • AreSame
    • AreNotSame
    • Contains

NUnit Features (6)

    Assertions
  • Condition
    • IsTure
    • IsFalse
    • IsNull
    • IsNaN
    • IsEmpty
    • IsNotEmpty
  • Comparison
    • Greater
    • Less

Good practices

Good practices

    Naming Standards for Unit Tests
  • [TestMethod_StateUnderTest_ExpectedBehavior]()

public int Sum(params int[] values)
TestSumNumberIgnoredIfGreaterThan100()
						

Good practices (2)

    When Should a Test be Changed or Removed?
  • Passing test should never be removed
  • Passing test should only be changed to make it more readable
  • New features allow new functionality

Good practices (2)

    What Should Assert Messages Say?
  • Tells us what we expected to happen but didn't and what happened instead
  • Do not provide:
    • Empty or meaningless messages
    • Messages that repeat the name of the test case

Good practices (3)

Avoid Multiple Asserts in a Single Unit Test

void TestSum_AnyParamBiggerThan1000IsNotSummed()
{
	Assert.AreEqual(3, Sum(1001, 1, 2);
	Assert.AreEqual(3, Sum(1, 1001, 2);
	Assert.AreEqual(3, Sum(1, 2, 1001);
}
							
Writing good and effective unit tests is hard!

Good practices (4)

    The 20/70/0 Rule
  • First, spend 20% of development time writing automated unit tests.
  • Second, aim for 70% code coverage. This excludes third-party or generated code, so make sure code coverage tools can exclude this. Interestingly, technical people tend to think this is high, especially if no automated unit tests exist. Less technical people ask why the remaining 30% cannot be covered.
  • Third, ensure 0 failing tests. This is a critical part of CI. Fix failing tests immediately.

Good practices (4)

    F.I.R.S.T
  • Fast
  • Independent
  • Repeatable
  • Self-validating
  • Timely

Mocking

CODE DEPENDENCIES

  • Dependency Inversion Principle is your friend
  • Spaghetti Code is hard to test
  • Heavily coupled code is hard to isolate for unit and functional testing
  • Coupled code can lead to large unit tests and heavy SetUp methods
  • Depend on abstractions not implementation!

TESTING PRIORITIES

  • Output
  • Service Dependencies
  • State

The solution

  • Mocking assists in isolating the method/class/component under test.
  • Mocking allows you to simulate parts of the system during test
  • Isolation with unit test and functional test level problems

Mocking

Mocking allows unit testing code with dependencies

public class Crawler
{
  public IEnumerable<string> ExtractImageUrls(string pageUrl)
  {
    var client = new WebClient(); // dependency code
    var html = client.DownloadString(pageUrl); // dependency code
    return this.ParseImages(html);
  }
  private IEnumerable<string> ParseImages(string html)
	{...}
}
							

Mocking (2)

    How to Test a Unit with a Dependency?
  • Extract the dependency into interface
  • Class should receive the interface through Dependency Injection
  • Create a mock of the dependency with static behavior (i.e. always returning same data)
  • Pass mock object to the tested class
  • Assert if tested class behavior is correct

Mocking (3)

Create dependency interface

public interface IHtmlProvider
{
	string DownloadHtml(string pageUrl);
}
						
Create a class and implement the interface

public class HtmlProvider : IHtmlProvider
{
  public string DownloadHtml(string pageUrl)
  {
    var client = new WebClient(); // this is our extracted
    return client.DownloadString(pageUrl); // dependency code
  }
}
						

Mocking (4)

Set up a fake object in the Test project.

// Fake IHtmlProvider whose DownloadHtml()
// method always returns the same string
public class FakeHtmlProvider : IHtmlProvider
{
  public string DownloadHtml(string pageUrl)
  {
    string fakeHtml = "" +
    "" +
    "Mocks" +
    "" +
    "";
    return fakeHtml;
  }
}
					  

Mocking (5)

Pass the fake object in the tests to the test class constructor

[Test]
public void ExtractImageUrls_ShouldReturnCollectionOfPageImageUrls()
{
   var fakeHtmlProvider = new FakeHtmlProvider();
   var crawler = new Crawler(fakeHtmlProvider);
   var imageUrls = crawler.ExtractImageUrls(string.Empty)
   .OrderBy(url => url).ToList();

   var expectedResults = new [] {
   "mock.png",
   "pics/Unit_testing/slide.jpeg" };

   CollectionAssert.AreEqual(expectedResults,imageUrls);
}
						

Test doubles

  • Dummy - simplest of all. It’s a placeholder required to pass the unit test. Unit in the context (SUT) doesn’t exercise this placeholder.
  • Fake - used to simplify a dependency so that unit test can pass easily. There is very thin line between Fake and Stub!
  • Stub - used to provide indirect inputs to the SUT coming from its collaborators / dependencies. These inputs could be in form of objects, exceptions or primitive values. Unlike Fake, stubs are exercised by SUT.
  • Mock – Like Indirect Inputs that flow back to SUT from its collaborators, there are also Indirect Outputs. Indirect outputs are tricky to test as they don’t return to SUT and are encapsulated by collaborator.
  • Spy – a variation of behavior verification. Instead of setting up behavior expectations, Spy records calls made to the collaborator.

Moq

Moq

  • NET Mocking Framework that handles mocking boilerplate for you
  • Gives you powerful assertion tools
  • Can mock both abstraction and virtual concretions
  • Facilitates the mocking process by providing an API for creating fake objects (mocks)
  • Moq is intended to be simple to use, strongly typed (no magic strings!, and therefore full compiler-verified and refactoring-friendly) and minimalistic (while still fully functional!).

Moq (2)

    VERIFY
  • Used to assert that a moq method has been called under certain conditions
  • Allows you verify the amount of calls
  • Allows you to verify the parameters passed in

Moq (3)

    VERIFY - TIMES
  • The Times struct lets you specific invocation amount restrictions when verifying.
  • Exactly
  • At Most
  • At Least
  • Between
  • Once/Never as convenience

Moq (3)

    SETUP - RETURNS
  • Allows you to configure what a mocked out method will return
  • Chain .Returns(delegate) onto a setup method.
  • Return delegate
  • Type of Func<ParameterType1, ParameterType2,… ReturnType>
  • Input: All method arguments
  • Output: Method output
  • Great place to use lambda expressions

Moq (4)

    RETURN PITFALLS
  • Be careful of shortcuts.
  • Returns(SomeCollection.Count) will only be evaluated once, regardless of how many times the mocked method is invoked
  • Returns(() => SomeCollection.Count) will be evaluated every time.
  • This applies to just returning a variable too, if for some reason this would change in between invocations you need to use a delegate.

Moq (5)

    CALLBACKS
  • Arbitrary block of code to be executed every time a mocked out method is invoked.
  • Useful for functional testing when trying to simulate parts of the system.
  • Similar to returns it takes in a delegate
  • Action, matching the parameter type/order
  • No returns
  • You can chain a callback to a return

Moq (6)

    IT – VARIATIONS
  • It.Is
    Func<Ptype, bool> - Pass it a delegate that determines if it is a match
  • It.IsAny<Ptype> (Most commonly used)
    Passes if parameter is the supplied type
  • It.IsIn
    Passes if parameter is in the supplied collection
  • It.Regex
    Passes if string parameter passes the regex, fails if not string parameter

Moq (7)


[Test]
public void ExtractImageUrls_ShouldReturnCollectionOfPageImageUrls()
{
  var mock = new Mock<IHtmlProvider>();
  mock.Setup(p => p.DownloadHtml(It.IsAny<string>()))
  .Returns("...");
  var crawler = new Crawler(mock.HtmlProvider);
  var imageUrls = crawler.ExtractImageUrls(string.Empty)
  .OrderBy(url => url).ToList();
  var expectedResults = new [] {
  "mock.png",
  "pics/Unit_testing/slide.jpeg" };

  CollectionAssert.AreEqual(expectedResults, imageUrls);
}
						

Test Driven Development (TDD)

TDD

  • TDD is Mostly about Design
  • Gives Confidence
  • Enables Change
  • Automated Validation of your design
  • Documentation By Example, As Well As an Up To Date
  • “Live”, And Executable Specification
  • Provides Rapid Feedback
  • Quality of Implementation and Quality of Design

TDD (2)

  • Forces Constant Integration
  • Requires More Discipline
  • Brings Professionalism to Software Development
  • Isn’t The Only Testing You’ll Need To Do
  • Developers Write Tests too

TDD (3)

    Result
  • Higher Quality and Flexibility
  • Readability and Maintainability
  • Predicability and Simplicity
  • The Hard Stuff And Surprises Are Tackled Early
  • Affects Both Internal And External Quality
  • Maintained and Well-Designed Application

THE THREE LAWS OF TDD

  1. You May Not Write Production Code Until You Have Written A Failing Unit (or Acceptance) Test
  2. You May Not Write More Of A Unit (or Acceptance) Test Than Is Sufficient To Fail, And Compiling Is Failing
  3. You May Not Write More Production Code Than Is Sufficient To Pass The Currently Failing Test

Myths and Misconceptions

  • You need a 100% regression test suite
  • The unit tests form 100% of your design specification
  • You only need to unit test
  • TDD is sufficient for testing
  • TDD doesn't scale (partially true)
  • Your test suite takes too long to run
  • Not all developers know how to test
  • Everyone might not be taking a TDD approach
office@e-dojo.it

EXERCISE

Problem 1: Test your code

Implement unit tests for at least 3 of your homework's programs. Aim at 80% and above code coverage.

Problem 2: Moq

Implement unit tests using Moq for at least 1 of your homework's programs. Search for code dependencies.