Custom instruments: when signpost is not enough

    Instruments for Apple's Xcode is a tool for analyzing the performance of an iOS application. They are used to collect and display data that is needed in code debugging. Last year, Apple introduced Custom Instruments. This is an opportunity to expand the standard set of tools for profiling applications. When existing tools are not enough, you can create new ones yourself - they will collect, analyze and display the data as you need.

    A year has passed, and there are almost no new public tools and information on their creation on the network. So we decided to rectify the situation and share how we created our own Custom Instrument, which determines the reason for the weak isolation of unit tests. It is based on signpost technology (we wrote about itin the previous article ) and allows you to quickly and accurately determine the location of the blinking test.

    Theoretical minimum

    To create a new tool for Xcode, you need an understanding of two theoretical blocks. For those who want to figure it out on their own, we will immediately give the necessary links:

    For the rest - below is a brief summary of the necessary topics.

    First select File -> New -> Project -> macOS Category -> Instruments package. The created project includes a file with the extension .instrpkg, in which a new tool is declared declaratively in xml format. Let's get acquainted with the markup elements:

    Data schemas
    interval-schema, point-schema, etc.
    Describes the data structure as a table like sql schemes. Schemes are used in other markup elements to determine the type of data at the input and output of the model, for example, when describing a mapping (UI).
    Import data schemas
    Import ready-made schemes. It allows you to use data structures that are defined by Apple.
    Tool model
    Associates the tool with a .clp file, in which the logic of the tool is defined, and announces the expected data scheme at the input and output of the model.
    Tool description
    Describes the data model and determines how events will be displayed in the UI. The data model is described using the attributes create-table, create-parameter, etc. Tool charts are defined by graph attributes, and the parts table is defined by list, narrative, etc.

    If we want to supplement the logic of the new tool, then create a .clp file with CLIPS code. Basic language entities:

    • “Fact” is a certain event registered in the system using the assert command;
    • “Rule” is an if-block with specific syntax that contains a condition under which a set of actions is performed.

    Which rules and in what sequence will be activated are determined by the CLIPS itself based on incoming facts, the priorities of the rules and the conflict resolution mechanism.

    The language supports the creation of data types based on primitives, the use of arithmetic, logical operations and functions. As well as full-fledged object-oriented programming (OOP) with the definition of classes, sending messages, multiple inheritance.

    Consider the basic syntax of a language that allows you to create logic for custom tools.

    1. To create fact, we use the construction assert:

    CLIPS> (assert (duck))

    Thus, we get an entry duckin the fact table, which can be viewed using the command facts:

    CLIPS> (facts)

    To delete a fact, use the command retract: (retract duck)

    2. To create rule, use the construct defrule:

    CLIPS> (defrule duck) — создание правила с названием duck
    (animal-is duck) — если animal-is duck присутствует в таблице фактов
    (assert (sound-is quack))) — то создается новый факт sound-is quack

    3. To create and use variables, the following syntax is used (before the variable name there is an obligatory sign "?"):


    4. You can create new data types using:

    (deftemplate prospect
    (slot name (type STRING) (default ?DERIVE))
    (slot assets (type SYMBOL) (default rich))
    (slot age (type NUMBER) (default 80)))

    So, we defined a structure with the name prospect and three attributes name, assets and age of the corresponding type and a default value.

    5. Arithmetic and logical operations have prefix syntax. That is, to add 2 and 3, you must use the following construction:

    CLIPS> (+ 2 3)

    Or to compare two variables x and y:

    CLIPS> (> ?x ?y)

    Practical example

    In our project, we use the OCMock library to create stub objects. However, there are situations when a mok lives longer than the test for which it was created, and affects the isolation of other tests. As a result, this leads to the “blinking” (instability) of unit tests. In order to track the lifetime of tests and mocks, we will create our own tool. The following is an algorithm of actions.

    Step 1. Making markup for signpost events

    To detect problematic moxes, two categories of interval events are needed - the time of creation and destruction of the moxa, the start and end time of the test. To get these events, go to the library OCMockand mark them using signpostmethods initand the stopMockingclass OCClassMockObject.

    Next, go to the project under study, make markup in unit tests, methods setUpand tearDown:

    Step 2. Create a new tool from the Instrument Package template

    First, we determine the data type of the input. To do this, we .instrpkgimport the scheme in the file signpost. Now the events created signpostwill fall into the tool:

    Next, we determine the data type of the output. In this example, we will output simultaneous events. Each event will have a time and description. To do this, declare the scheme:

    Step 3. We describe the logic of the tool

    We create a separate file with the extension .clpin which we set the rules using the CLIPS language. To let the new tool know in which file the logic is defined, add a block modeler:

    In this block, using the attribute, production-systemspecify the relative path to the file with the logic. In the attributes outputand required-inputdetermine the data schema at the input and output, respectively.

    Step 4. We describe the specifics of the presentation of the tool (UI)

    It .instrpkgremains to describe the tool itself in the file , that is, the display of the results. Create a table for the data in the attribute create-tableusing the previously declared schema detected-mocks-narrativein the attribute schema-ref. And set up the type of information output - narrative (descriptive):

    Step 5. We write the logic code

    Let's move on to the file .clpin which the logic of the expert system is defined. The logic will be as follows: if the start time of the test intersects with the interval of the moka’s life, then we believe that this mok “came” from another test - which violates the isolation of the current unit-test. In order to eventually create an event with information of interest, you need to do the following steps:

    1. Define the mock and unitTest structures with fields - the time of the event, the event identifier, the name of the test and the class of the mok.

    2. Define the rules that will establish the facts of the types mockand unitTestbased on incoming events signpost:

    Read these rules are as follows: if the input we get a fact-type os- signpost with the desired subsystem, category, nameandevent-type, then create a new fact with the type that was defined above (unitTest or mock), and fill it with values. It is important to remember here - CLIPS is a case-sensitive language and the values ​​of subsystem, category, name and event-type must match what was used in the code of the project under study.

    The values ​​of the variables from the signpost events are transmitted as follows:

    3. We define the rules that release the completed events (they are redundant, since they do not affect the result).

    Step 6. Define the rule that will generate the results.

    You can read the rule like this.


    1) there is unitTest and mock;

    2) in this case, the beginning of the test occurs later than the existing moka;

    3) there is a table for storing results with the detected-mocks-narrative schema;


    4) create a new record;

    5) fill in with time;

    6) ... and a description.

    As a result, we see the following picture when using the new instrument:

    The source code for custom instrument and an example project for using the instrument can be viewed on GitHub .

    Tool debugging

    Debugger is used to debug custom tools.

    It allows you to

    : 1. See compiled code based on the description in instrpkg.
    2. See detailed information about what happens to the tool at runtime.

    3. Display a complete list and description of system data schemes that can be used as input in new tools.

    4. Execute arbitrary commands in the console. For example, display a list of rules with the list-defrules command or facts with the facts command

    Setup on CI server

    You can run tools from the command line - to profile the application during the execution of unit- or UI-tests on the CI-server. This will allow, for example, catching a memory leak as early as possible. To profile tests in the pipeline, we use the following commands:

    1. Launch tools with attributes:

    xcrun instruments -t  -l  -w 

    • where template_nameis the path to the template with tools or the name of the template. You can get the team xcrun instruments -s;
    • average_duration_ms - the recording time in milliseconds should be greater than or equal to the test execution time;
    • device_udid- simulator identifier. You can get the command xcrun instruments -s. Must match the identifier of the simulator on which the tests will be run.

    2. Running tests on the same simulator with the command:

    xcodebuild -workspace -scheme  -destination

    • where path_to_workspaceis the path to the Xcode workspace;
    • scheme_with_tests - scheme with tests;
    • device - simulator identifier.

    As a result, a report with the extension .trace will be created in the working directory, which can be opened by the Instruments application or by right-clicking on the file and selecting Show Package Contents.


    We examined an example of upgrading signpost to a full-fledged tool and told how to automatically apply it on the “runs” of the CI server and use it in solving the problem of “blinking” (unstable) tests.

    As you dive into the possibilities of custom instruments, you will better understand in what other cases you can use the instruments. For example, they also help us to understand the problems of multithreading - where and when to use thread-safe data access.
    Creating a new tool was quite simple. But the main thing is that after spending several days studying the mechanics and documentation to create it today, you will be able to avoid several sleepless nights in attempts to fix bugs.


    The article was written with @regno , Anton Vlasov, an iOS developer.

    Also popular now: