Demystifying iOS 9 Spotlight Search for Developers

There are three search points built into iOS: Siri, Spotlight Search, and Safari search. Siri, one of the most iconic features of iOS, is something that most iPhone users are already familiar with. But many people are not aware of Spotlight search - something that has been an integral part of iOS long before Siri existed. Spotlight Search can be accessed when you slide the home screen downwards, or new to iOS 9, swipe right from your personal home screen. This reveals a search bar at the top of the screen.

The focus of this search tool in iOS 8 and earlier was searching the phone itself, so its results page lists apps that are on your phone, as well as emails, messages, and other private items within the Apple apps. It will also show a Wikipedia definition if it seems relevant. Finally, it offers an option to search the web via Safari as an additional step.

In contrast to Spotlight Search, Safari is all about the world outside your phone. In iOS 8, the user can choose which search engine to use for Safari search: Google, Yahoo, Bing, and DuckDuckGo. Naturally Google handles the majority of Safari search requests. Regardless of the search engine chosen, pre-filled search terms are presented in Safari, potentially with a Wikipedia match.

In iOS 9, Spotlight Search is much more prominent and more widely scoped than before. In fact, the first thing you see are “Siri Suggestions” based on your usage patterns. For example, if you use Safari regularly in the afternoon, Siri will suggest it for you around that time.

And, while it is still a place to find “things on your phone”, it is becoming a direct portal for “things not on your phone”. Already you see things nearby and news headlines sourced from the web via Apple’s own search engine, and other web-based results are just starting to appear now.

Yes, Apple has created a web search engine for iOS 9, which is a very big step. Say hello to Applebot.

In a similar vein to Spotlight, Apple’s stated aim for Safari is also to surface results and suggestions from Apple’s search index, before offering the ability to search the web with Google (or other chosen provider). As of iOS 9.1, Safari seems to be lagging behind Spotlight search by continuing to serve results from the user’s chosen search provider — perhaps to give Apple a slower roll-out of its search features and more time to fine-tune its algorithms.

What is particularly exciting about the launch of iOS 9 is that for the first time, it gives iOS developers the ability to hook into these search functions and include content from within apps. Furthermore, the promise is that search will ultimately include results not on the user’s phone. This is a huge step.

Let’s just put this into context. Google’s search engine has spawned an entire industry built around “SEO” — the process of optimizing your assets around Google’s search. When fully launched, Apple’s search engine will own roughly 50% of the mobile search market, which is 50% (and growing) of overall search traffic. Applebot is therefore big, which means “Apple Mobile Search Optimization” will also be big.

Apple Mobile Search Optimization efforts will have a much bigger impact on app discovery by mobile users than regular SEO.

Welcome to the newborn AMSO industry.

The bad news is that implementing all these features involves many new technologies from Apple as well as enhancement to old ones, and the combination of these elements can be confusing. The goal of this series is to walk through the various pieces and clarify how they might be implemented. Let’s start with something basic and build on it.

In support of this series, I have created a simple app for reference throughout. The project can be found on GitHub. It will also be updated as this series continues to cover other elements of the search toolkit.

The Foundation: CoreSpotlight Framework

This framework, completely new in iOS 9, allows you to contribute items to the iPhone’s local search index for discovery by the user. For example, before iOS 9, only content within the official Apple apps, such as Calendar, could be found via Spotlight search. Now, any app that publishes its calendar events through the CoreSpotlight framework can be found via Spotlight Search and even Siri.

Note that the focus here is on personal information; you don’t want your personal calendar events being found by users on another phone! Apple is very clear that the CoreSpotlight framework interacts exclusively with the private index on each person’s phone. It is not possible to use CoreSpotlight API to publish content outside the phone’s own index.

That does not mean that CoreSpotlight cannot be used to index already public information. It is certainly possible to use CoreSpotlight to pull information from the cloud and place it into the phone’s search index. It is simply a guarantee from Apple that if personal and confidential information is indexed with CoreSpotlight, it will remain confidential.

The framework comes in two parts: the CSSearchableItemAttributeSet object that allows the app to describe each item in detail, and a CSSearchableItem object that is used to give the item a unique ID. Each Spotlight entry is made up of a pair of these objects. The reason for using two objects instead of one is that CSSearchableItemAttributeSet objects are used for a different kind of search interaction (described later).

It’s pretty easy to do the basics.

For those referencing the sample code, setting up CoreSpotlight is done in AppDelegate.

…
@import CoreSpotlight;
@import MobileCoreServices;	
…

These two modules are needed for the new functionality.

For simplicity in the demo app, the application:didFinishLaunchingWithOptions: method is used to call a dedicated method for setting up CoreSpotlight:

if ([CSSearchableItemAttributeSet class])
    [self setUpCoreSpotlight];
//Check for iOS version that supports CoreSpotlight API

Moving down to the setUpCoreSpotlight method, we start the process by creating a CSSearchableItemAttributeSet object:

CSSearchableItemAttributeSet * attributeSet = 
    [[CSSearchableItemAttributeSet alloc]
        initWithItemContentType:(NSString *)kUTTypeItem];

The Content Type is important as it is supposed to govern the general way in which the item is treated by the search algorithms. “Supposed” is the operative word, since this is an area that is not well documented by Apple. There is a large menu of item types to pick from (image types, video types, audio types, a contact type), but trial and error is required to determine what impact (if any) it has on the user’s experience. Some types seem to display more text content than others when displayed in search results. It can also impact whether a thumbnail image is displayed along with the result. kUTTypeItem is a great place to start.

Now describe the object in more detail:

attributeSet.displayName = @"A Christmas Carol";

attributeSet.title = @"A Christmas Carol By Charles Dickens";
    //Sounds similar to displayName but is not displayed to user
    
attributeSet.contentDescription = @"Who would dare to say “Bah! Humbug” after reading A Christmas Carol? Charles Dickens wrote the novella in just six weeks before it was first published on December 19 1843 but his morality tale about a bitter old miser named Ebenezer Scrooge lives on to this day as a reminder of the importance of the Christmas spirit.";
    
attributeSet.keywords = @[@"A Christmas Carol", @"Charles Dickens", @"Victorian Literature"];
    
UIImage *image = [UIImage imageNamed:@"CC-Cover"];    
NSData *imageData = [NSData dataWithData:UIImagePNGRepresentation(image)];
attributeSet.thumbnailData = imageData;

There are lots of possible properties, specific to media types like images, videos and so on. More information is probably better, but trial and error is needed to determine what information is actually used and/or displayed in search results. At the time of writing, for example, star-ratings seem not to be displayed for a given index item, even if the attributeSet contains star rating data.

Most text properties are included in the search query, so even though the title property is not displayed, the text can be found via search. It is therefore not entirely clear how the keywords property is fundamentally different from the title property.

See Apple’s doc for more information: developer.apple.com

The final step is to package the CSSearchableItemAttributeSet with CSSearchableItem and register it with the index.

CSSearchableItem *item1 = [[CSSearchableItem alloc] initWithUniqueIdentifier:@”https://www.notestream.com/streams/564159e4e5c24” domainIdentifier:@"notestream.com" attributeSet:attributeSet]; 

Two new things here. The domainIdentifier property allows you to group items together for batch operations. For example, @“meetingItem” and @”reminderItem” in a calendar app would allow you to delete all Spotlight entries of one sort while leaving the other.

The uniqueIdentifier property has a more important role to play. First, it is passed back to your app when the user clicks on an item in the CoreSpotlight index (discussed later). Second, there are recommendations from Apple about what this should look like when using other elements of the search infrastructure. For now, we will use a string that is in fact a URL that uniquely represents this item. Technically this property could be any string, as long as it is unique to the item.

The last step: push the items you’ve created into the actual index.

[[CSSearchableIndex defaultSearchableIndex] indexSearchableItems:@[item1, item2, item3]
    completionHandler: ^(NSError * __nullable error) {
    if (!error)
        NSLog(@"Search item(s) journaled for indexing.");
}]; 

A couple of points to make here:

1. This method takes an array of objects (and there are other methods that can be used to batch the process if there is a lot of data to index).

2. It is important to realize that this method does not actually complete the indexing process. The completion handler is called only when your search items have been queued for indexing. Use the CSSearchableIndexDelegate to handle scenarios where the indexing process itself fails for some reason and your app needs to handle the situation.

Congratulations, you’ve added items to the phone’s CoreSpotlight search index!

How Search Works in iOS 9

Naturally, Apple keeps their search algorithms close to their chest, so it is worthwhile to install the sample app and play around with keywords. For example, it quickly becomes apparent that Apple is maximizing the results presented by giving very different results as the user typed each letter:

It looks like the principle is that the results thought more appropriate will be presented after just a couple of letters have been typed. As the user types more letters, those early results are dropped and other things (originally estimated to be less likely) are presented instead.

More generally, Apple has only a few things to say about how to improve ranking under their algorithms. This page contains Apple’s recommendations: Enhance Your Search Results

One of the key take-aways from Apple’s documentation is that ranking is improved when the app uses multiple search technologies to index content. We’ll dive into another of these technologies in the next article. Meanwhile, we still need to implement code to handle the scenario when a user clicks on a search result.

When the User Clicks

We’ve created the code to populate the index with potential search results, but what happens when the user clicks on a result? The answer is that CoreSpotlight borrows the application:continueUserActivity:restorationHandler: method from the UIApplicationDelegate protocol. This was originally introduced in iOS 8 for the handoff mechanism that allows user activities to be passed from device to device (e.g. a user sees a web URL on an iWatch, uses handoff to pick it up on the iPhone Safari browser, and ultimately view it on their Mac when they get to the office.)

The first challenge is to determine if the method was called due to a Handoff activity or from Spotlight Search. The activityType property of the activity parameter will tell you, but if the code might run on iOS 8, you’ll need to avoid a crash in that case.

Here’s how some simple code might look here:

- (BOOL)application:(UIApplication *)application continueUserActivity:(NSUserActivity *)activity restorationHandler:(void (^)(NSArray *))restorationHandler
{
    
    NSString * valueCSSearchableItemActionType;
    BOOL wasHandled = NO;
   
    if ([CSSearchableItemAttributeSet class]) //iOS 9
    {     
        valueCSSearchableItemActionType = CSSearchableItemActionType;
        
    } else { 
        // iOS 8 – This method was introduced in iOS 8, so iOS 7 is not a possible scenario
        valueCSSearchableItemActionType = @"not supported";
    }
        
    if ([activity.activityType isEqual: valueCSSearchableItemActionType])
    {	
        // Invoked via CoreSpotlight, we can assume iOS 9 from now on…

        NSString * activityIdentifier = [activity.userInfo valueForKey:CSSearchableItemActivityIdentifier];

        wasHandled = YES;
        NSLog(@"Continuing user activity %@", activityIdentifier);
        
    } else {
        
        //the app was launched via Handoff protocol
        //or with a Universal Link
    }
    
    return wasHandled;
}

Note the line of code:

NSString * activityIdentifier = [activity.userInfo valueForKey:CSSearchableItemActivityIdentifier];

This extracts the unique identifier that was placed into the spotlight index when the CSSearchableItem object was created. Naturally your app should use this unique identifier to present the user with the piece of content that they selected from the search index. For simplicity here, we just NSLog the unique identifier.

Recap

So far, we’ve examined just one part of iOS 9’s search functionality. We’ve run through the core functionality needed to load the phone’s Spotlight search index with things the user might find useful: documents, calendar items, contacts and so on. When the user searches for something, sees a result from your app and clicks on it, the app will be able to handle the request and display the appropriate item for the user.

What we have not yet discussed is the technologies that relate to the public index Apple is building. This index is intended to allow users to find content in your app even without it being installed on their phone. We’ll tackle that in the next post.