How to move from BuddyBuild to GitLab CI in 4 hours

Background

A year and a half ago, the iOS FunCorp team moved to a new service for simple CI organization in iOS and Android projects.

Before that, we used CI on Bamboo, but there were a lot of problems with it, so we completely abandoned it and switched to BuddyBuild.

It worked so simply that you could not even know what CI is and how to upload the application to the AppStore, but calmly deal with code, tests and product development.
But times have changed, and BudduBuild is not the same, so we began to search for an alternative.
In this article, we will talk about the new solution that our team chose and give some scripts for organizing CI on our own.

Just means good

At BuddyBuild, we were attracted by simplicity. First you need to take just a few steps:

1. Log in via GitHub / GitLab / BitBucket.
2. Specify the repository with the project.
3. Give the service an account and distribution certificate.
4. Wait until the “magic happens” on the side of the service.

And right after that, you can get tests and artifacts across all branches, configure the assembly rules using a clear UI, quickly switch between Xcode versions and release them in the AppStore / TestFlight directly from the service.

When we started working with BuddyBuild, it could be used for free, but after a few months it ended. Now a starter pack costs $ 79 per month. For ourselves, we chose a plan with three competitive builds for $ 279.

BuddyBuild worked well, but it did not last long.

With the growing popularity of the service, as well as with the increase in the amount of code in iFunny, the build time increased from a stable 20 minutes for testing and building an artifact to 70 minutes.

We tried to find a solution for caching by means of the service, but nothing worthy was found in simple settings.

Meanwhile, Apple bought the service, and we made the final decision to abandon it.

CI / CD Requirements

Based on previous experience with CI, we had several requirements for the new system.

1. It should quickly deploy agents and a build environment with a minimal number of visits to agent UIs.
2. Update Xcode without entering the agent UI and be able to switch between multiple versions.
3. Integrate with pull requests.
4. Use a minimum of third-party dependencies for assembly and unloading in the AppStore.
5. Have the ability to customize and customize build steps for different branches.
6. Start the assembly of the artifact only by the button from the UI.

Gitlab ci

Along with the decision to abandon BuddyBuild, the idea came to migrate from GitHub to GitLab, and it already has a built-in CI / CD system, that is, we have the integration we need with merge requests.

Installing a working environment on an agent

First of all, you need to enable the ability to access the agent via SSH using Screen Sharing. This is done in the Sharing settings:



Now, to connect to the CI agent, we can use the terminal and the SSH client:

ssh user@local.ip

After successfully connecting via SSH, you need to disable password usage for the sudo command. It may seem that this is unsafe, but taking into account that all agents with us are available only inside the local network, for better automation we disable the password for sudo. For this:

sudo visudo

The standard Vim editor opens, in which you need to change the line:

%admin            ALL = (ALL) ALL

per line

%admin          ALL = (ALL) NOPASSWD: ALL

Many iOS developers do not like to delve into the console and rarely use Vim, so keep a step-by-step instruction on how to change these lines:

1. Press i, enter insert mode.
2. Arrows find the desired line and change it.
3. Next esc.
4. Enter: w to save.
5. Enter: q to exit visudo.

First of all, on each agent, we need the Homebrew package manager, which can be installed with the following command:

sudo echo | ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

The echo command is used here so that the installation confirmation message does not appear.
After that, you can set the required minimum of dependencies from Homebrew. Here is what we have chosen:

• XCPretty . It allows you to beautifully format the output of the standard command for the build of xcodebuild iOS projects;
  
  

  sudo gem install xcpretty

• fastlane . We use its capabilities to a minimum, since we do not want the project build to be heavily dependent on something outside;
  
  

  sudo gem install fastlane

• xcode-install . This tool will allow you to install Xcode without going into the AppStore and switch between multiple versions;
  
  

  sudo gem install xcode-install

• CocoaPods . This dependency manager is familiar to every iOS developer.
  
  

  sudo gem install cocoapods

  
  

Now you can install Xcode by running the following commands in sequence:

export FASTLANE_USER="your@account.todevapple"
export FASTLANE_PASSWORD="yourpasswordtoaccont"
xcversion install 9.2

You can not perform export, but then the Apple ID email and password will be requested during the installation process.

After these simple steps, the agent is ready to collect the majority of iOS-projects.

Agent Registration

In order for your agent to see the project on Gitlab CI, you need to install it and register it.

This can also be done using SSH and the command line on the agent:
First, download and install the agent:

curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64

We give the right to execute:

chmod +x /usr/local/bin/gitlab-runner

Install gitlab-runner as a service and run it:

gitlab-runner install
gitlab-runner start

Now you need to register the runner on the CI, this is done by the command:

gitlab-runner register -n --url CI_URL --registration-token TOKEN --tag-list fastlane,cocoapods,osx_10-13,xcode_9-2 --executor shell

This is a team that needs to be explained in detail.
CI_URL and TOKEN can be taken in the project settings on GitLab:
Settings -> CI / CD -> Runners -> Setup a specific Runner block manually
--tag-list : tags are specified that we will need later on when setting up the project itself.

They can also be changed, for example, depending on which versions of Xcode are installed on the agent or what type of tasks we plan to perform on it.
--executor shell : indicate that the agent must perform actions on the command line.

To avoid problems with CocoaPods, you also need to run the commands:

echo 'export LC_ALL="en_US.UTF-8"' >> ~/.bash_profile
echo 'export LANG=en_US.UTF-8' >> ~/.bash_profile

This is necessary so that there are no coding errors when installing the hearths.
After successful registration of the agent, it can be seen in the settings.
Settings -> CI / CD -> Runners

Project setup

It remains to configure the Xcode project to work with GitLab CI.

To do this is quite simple: you need to put the .gitlab-ci.yml file with the description in the root of the project.

After this file is added to the repository, the CI system will begin to execute the commands specified in it.

An example of our yml file:

stages:
  - test
  - archive
before_script:
    - git submodule init
    - git submodule update --recursive
    - pod install --repo-update
archive_project:
  stage: archive
  script:
    - fastlane match appstore
    - xcodebuild -workspace iFunny.xcworkspace -scheme iFunny archive -archivePath build/iFunny.xcarchive | xcpretty
  only:
    - master
    - triggers
    - web
  artifacts:
    paths:
    - build/iFunny.xcarchive
  tags:
    - xcode_9-2
    - osx_10-13
    - cocoapods
    - fastlane
test_project:
  stage: test
  script:
    - xcodebuild test  -workspace iFunny.xcworkspace -scheme iFunny -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.2' | xcpretty
  tags:
    - xcode_9-2
    - osx_10-13
    - cocoapods
    - fastlane

The yml format for GitLab CI is well described here .

So that you can easily use the sample file, I will describe the main points that we use:

• before_script - describe a set of instructions that must be completed before each work on the CI. For us, this is a standard set of updating submodules and installing hearths;
• archive_project - the name that is used to get the artifact with xcarchive.

In the script, specify all the instructions that must be completed:

fastlane match appstore

Fastlane has a good match command to synchronize certificates (for more details on using match, see the fastlane website ).

The main command to start the archive assembly:

xcodebuild -workspace Project.xcworkspace -scheme ProjectScheme archive -archivePath build/Project.xcarchive | xcpretty

Project.xcworkspace - file with workspace.
ProjectScheme - the scheme with the main target.
build / Project.xcarchive - the path along which the working artifact will be collected . Next, we use this path in artifacts: it tells CI where to get the archive from.

In the only block, indicate that this work should be performed only on the master branch or when starting from the web, that is, when starting from the Run Pipeline button in CI / CD.

tags are those tags that must be registered on the agent so that it can run this work. Now they completely repeat what we indicated when registering the agent.
Next in the file is a description of the operation of test_project .

Of the interesting things here is the test run line:

xcodebuild test  -workspace Project.xcworkspace -scheme ProjectScheme -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.2' | xcpretty

In the -destination setting , specify the simulator that is precisely available on the agents.

You can view the list of available devices on the agent through the SSH command:

instruments -s list

After the .gitlab-ci.yml file is configured , you can add it to the repository and tests will automatically be run in the branch containing the file.

Conclusion

To set up the whole bunch of GitLab and the embedded CI / CD system, we spent about half a working day, which is more than 20 minutes spent setting up BuddyBuild.
But what we got from the move:

• own agents for assembling projects and a system for working with them;
• It was possible to reduce from 70 to 6 minutes the time for assembly with tests or up to 30 minutes for a complete assembly with tests and archives;
• opportunities to optimize assembly time;
• any customization and integration with any service is available to us.

For our team, GitLab CI is a temporary solution, now we are preparing to move to Jenkins, in which we will add even more automation to the project.

Perhaps the experience described in the article will allow readers to switch to GitLab CI in less than 4 hours.

And to make it easier to do, here are a couple of scripts with the commands described in the article:

• installation of dependencies;
  install_dependencies.sh
  
• installation and registration of agents.
  install_agent.sh