Juha Riippi

By Juha Riippi

TwitterLinkedIn

CI for iOS and mobile development in general has been a tricky business until recently. During the last couple of years some excellent services and libraries have been launched to make it a lot easier. When I started investigating what is out there, I stumbled into some good looking SaaS-services for this:

  • Buddybuild: Appears to provide seamless testing, delivery and bug reporting tools for testers.
  • TravisCI: particularly good, if your projects are in Github
  • GreenhouseCI: Another option to consider. The basic setup is really simple in this one as well.

Our needs

While the SaaS options are all viable and working, I still felt, that I wanted a bit more control and features from our CI. We are using Gerrit for our code reviews, so I wanted the CI to:

  • Build and test each version submitted to Gerrit for review. It should give a negative review if build or tests fail
  • Build and test each version submitted to our development branch and notify about failures
  • Build, test and deploy each version from our master branch to HockeyApp and notify about failures.

We use Jenkins for CI in majority of our web projects, so that was already familiar to me. So it was the simplest choice. For a minimum setup, I installed Jenkins to an older Macbook along with iOS (and Android) development environment.

Gerrit Trigger for Jenkins

I wanted to get things going as easily as possible, so I installed the Gerrit Trigger for Jenkins. The setup was relatively painless and easy by following the instructions in the Gerrit Trigger plugin page.

My first attempt in building iOS in our CI machine was to use the Jenkins XCode Plugin. I managed to get it working, but it seemed to be kind of volatile and ready to explode at any given moment. A far better option is Felix Krause's excellent Fastlane automation toolchain. With Fastlane our Jenkins build and test configuration couldn't be simpler

Jenkins iOS build and test configuration with Fastlane

Jenkins iOS build and test configuration with Fastlane

Fastlane for building and testing

Fastlane itself requires some configuration, which is done in a Fastfile that defines the "lanes" and Appfile that contains common information about the environment such as Apple ID and Team IDs.

Our Appfile is just a few lines:

# The bundle identifier of your app
app_identifier "<the.app.identifier>" 
# Your Apple email address
apple_id "<the.developer@email.address>" 

# You can uncomment any of the lines below and add your own
# team selection in case you're in multiple teams
team_name "team_name"
team_id "team_id"

# you can even provide different app identifiers, 
# Apple IDs and team names per lane:
# https://github.com/KrauseFx/fastlane/blob/master/docs/Appfile.md

Our Fastfile is also very basic, we have basically only one lane, which builds the application, creates the .ipa package and tests the results:

platform :ios do
  before_all do
    # Update provisioning profiles and signing identities 
    # from Apple Developer
    sigh
  end

  desc "Runs all the tests"
  lane :test do
    gym(scheme: "TheProject")
    testonly
  end

  lane :testonly do
    xctest(
      workspace: './TheProject.xcworkspace',
      scheme: 'TheProject-simulator-tests',
      sdk: 'iphonesimulator',
      destination: "name=iPhone 6",
      reports: [
        {
          report: "junit",
          output: "test-reports/unit-tests.xml"
        }
      ]
    )
  end
end

 

As said, we use Fastlane in a relatively simple manner. It can be used to automate the whole process from taking App Store screenshots to actual App Store releases. More complex Fastlane examples can be found here.

Jenkins for HockeyApp

After Fastlane has done it's job for master branch builds, we want them deployed to HockeyApp for testers automatically. For this we use Jenkins HockeyApp plugin. Our configuration for this looks like this:

Screen Shot 2015-10-01 at 12.44.20

Jenkins HockeyApp plugin example configuration

Testing tips

I recently started experimenting with Apple's new UI Testing introduced in XCode 7. It still seems to have some glitches, but looks promising for acceptance testing apps. Apps developed in Vincit almost always have some sort of backend server for syncing and fetching data. In order to test applications with UI Testing, the backend needs to be either mocked to contain static test data, or there needs to be some way to recover the test server database to a certain "testing state".

I like the latter option as it also tests the backend code and APIs to some extent. For example, in one of our current projects we have a replicated production server that is used for testing. This testing server has an additional method in its API that resets certain users data to a hardcoded value. This makes it possible to test state changing methods and verify that they actually affect the backend data. Before each UI Test starts, we call the method that resets the user states, so we always know what kind of data we should receive.

For testing with location, XCode Schemes are useful. Schemes can be used to mock simulator location and they can even contain bundled data, for example login information so every test doesn't need to first log in to the application. For mocking location, a .gpx file is needed. These can be generated for example here. The mock location is set from XCode in Product->Scheme->Edit Scheme.

Screen Shot 2015-10-01 at 13.03.49

Setting mock location to a scheme in XCode 7

It's a good idea to have a separate testing scheme for mock locations and Bundle data. You can also define multiple test targets with different schemes, so you can run some test cases without location, some with login data etc.

Future improvements and Android

The solution is still very much work in progress, but serves multiple projects quite well. We do have occasional glitches and faults with Jenkins that kind of disrupt daily work, but I suppose thats the price of the added control that your own Jenkins instance brings compared to a SaaS solution. Future plans include a bigger test lab with actual devices attached to the CI machine. UI Automation tests with Appium has been prototyped for testing web projects in mobile browsers.

We also have a similar solution for Android projects. I haven't found a "Fastlane for Android" or similar, so there are a few additional steps for Android builds. But proper shell scripting is really helpful in Android world.

Did you enjoy this article?

Give us a clap!

Comments

  • Kalle Kyllönen

    Very good article! Can you write a new article when you have started to create test cases using appiumin and how those working with Iphone + jenkings + how test results are reported?

    • Juha Riippi Kalle Kyllönen

      Thanks! Sure, that sounds like a good topic.

  • Riku P

    Thanks for sharing this Juha. I just setup a new Yosemite Jenkins node using Fastlane for the builds.

  • Hi Juha, You might also be interested in http://www.buddybuild.com, which launched a little while after you wrote this.

  • Nice post! Readers will like to know that there are 2 fastlane tools for Android: Supply and Screengrab.

  • Recommend an automated quick application for iOS certificate tool, 3 seconds to create .P12 and .mobileprovision iOS certificate file

    Appuploader tool

Leave a Reply

Your email address will not be published. Required fields are marked *