Siri Integration or “Here’s what I found in your application”

Tutorial

At WWDC 2016, Apple introduced SiriKit, a voice assistant API, to the world.

If you have not watched the WWDC session about SiriKit and expect that you can use Siri in any application, then you should know that only a few types of services are currently supported:

1) Audio and video calls,
2) Messages,
2) Payments,
3 ) Search for photos,
4) Training,
5) Travel (booking).

Also, as the documentation says, there is the possibility of interacting with the car using CarPlay (INSetClimateSettingsInCarIntent, INSetSeatTemperatureInCarIntent, etc.).

Thus, Siri can give the command "<call that, send a message, look for photos, etc.> through <your application name>".
Everything is arranged in such a way that you don’t have to interact directly with the neural network - the SDK provides simple protocols and a set of lightweight classes for transmitting information in methods. The developer can only implement these protocols.

For the lazy, at the end of the article there is a link to a demo application (we send a message to our friends from VK using Siri).

The cornerstone of SiriKit is Intent (Android developers, hello!). An object of the INIntent class (or its descendants) is the input that Siri generates. INIntent itself does not contain anything other than an identifier. The context role is delegated to its subclasses. The specific subclass depends on the type of application. For example, for an application with training, INStartWorkoutIntent contains information about the goal - how much time you need to spend on a particular exercise, where the training takes place, and so on. For the photo service, you can use the INSearchForPhotosIntent intent, which contains the geotag (CLPlacemark), the date the photo was created, the list of people in the photo, etc.

To process the incoming intent, the developer will need to create objects of the INIntentResolutionResult class (or its subclasses).

There are three stages of processing input information:

clarification (resolve),
confirmation
execution of an action (handle).

At each of these stages, we get the intent and must return the corresponding INIntentResolutionResult.

For example, if we are developing a messenger application, then at the refinement stage we uniquely determine the users to whom we send the message (resolveRecipientsForSendMessage), the contents of the message (resolveContentForSendMessage). At the confirmation stage, we verify (confirmSendMessage) that everything is ready for sending (for example, the user is authorized). Finally, at run time (handleSendMessage), we send a message to the selected recipients.

At any stage, there are two options for the development of the scenario: positive and negative. The choice of the path is delegated to the developer: Siri provides the processed data as an intent object, and the programmer decides what result to return to the system.

Intents extension

For your application to work with Siri, you need to add the NSSiriUsageDescription key to InfoPlist with text explaining to the end user why your application needs access to Siri (similar to NSLocationUsageDescription for geolocation).

And request permission:

[INPreferences requestSiriAuthorization:^(INSiriAuthorizationStatus status) {
}];

After that, you need to add a target with the IntentsExtension type to the project.

After adding a target, pay attention to its structure. The entry point for intents is an object of the INExtension class. Inside the class instance, in the method (handlerForIntent) it is selected which object will process the incoming intent. The name of the inheritor class INExtension and the supported types of intent are registered in the InfoPlist of the target. You can also specify which types of intents will not be available on the locked screen.

To process a specific type of intent, you need to implement a specialized protocol in your class (for example, INSendMessageIntentHandling). The protocol contains the methods necessary to go through the above steps (resolve, confirm, handle).

Consider the implementation of a handler class for the intent of sending a message.

As described above, at the first stage (clarification), it is required to uniquely identify users and the content of the message. If we talk about the content of the message, then in most cases it is enough to know that it is not empty. But with the recipients is not so simple. In the intent, we can get a list of users that Siri recognized. This is an array of objects of the INPerson class. For each recipient, you must find a match in the list of existing recipients.

There are three scenarios:

no match - negative scenario;
one correspondence - uniqueness is confirmed;
more than one match - you need to give the user the opportunity to select the correct contact from the list (INPersonResolutionResult -> disambiguationWithPeopleToDisambiguate).

Note - after selecting a user in case of disambiguation, the resolveRecipientsForSendMessage method will be repeated. When checking user matches, you need to consider that the method can be called several times.

At the confirmation stage, we check the necessary conditions for the possibility of sending a message. Finally, the handleSendMessage method (final stage) sends the message. At each stage there is a positive and negative work scenario.

Siri does not always understand what we want to say. To help her, you can use Vocabulary - a glossary of terms. They come in two types: static and dynamic. In the first case, we provide a special AppIntentVocabulary.plist, in which common terms are written. The dynamic dictionary is filled with specialized terms for the current user:

[[INVocabulary sharedVocabulary] setVocabularyStrings:[usersNames copy] ofType:INVocabularyStringTypeContactName];

It is not the Intent Extension that is responsible for the dictionary, but the main application. That is, you need to specify your additional terms before starting the Siri extension.

Intents UI Extension

In addition to data processing logic, SiriKit provides the ability to change the data display interface. There is an Intents UI Extension for this. By analogy with the Intents Extension, InfoPlist UI-target contains a list of supported intents, as well as the name of the storyboard-file.

Note - the same controller (entry point in the storyboard) will be created for any intent in the UI extension. To delegate display logic, it is recommended that you use child view controllers. The base controller must support the INUIHostedViewControlling protocol, which defines the interface configuration method, configureWithInteraction. Consider the parameters of this method.

interaction (INInteraction) contains the intent and its processing status. According to the class of transmitted intent, we can instantiate the corresponding controller;
context (INUIHostedViewContext) allows you to determine whether Siri is used directly or if Maps are involved;
completion (completion block) to which you want to transfer the size (CGSize) of the displayed controller (snippet). The size is limited by the minimum (hostedViewMinimumAllowedSize) and the maximum (hostedViewMaximumAllowedSize), the values of which are determined by the system in the NSExtensionContext (INUIHostedViewControlling) category.

In addition to the main protocol INUIHostedViewControlling, there is an additional protocol INUIHostedViewSiriProviding, which allows you to control the display of standard Siri interfaces for messages (displaysMessage) and maps (displaysMap). Apple’s requirements for creating interfaces for Siri include a ban on displaying ads in snippets. If you plan to use animation, it is recommended that you run it in viewDidAppear.

To transfer data between the main application and extensions, as before, you need to use Application groups (example in the demo).