Sketch + Node.js: generate icons for many platforms and brands. Part 2

Transfer

This is the second part of the article about creating a tool that can export all the icons placed in a Sketch file: in different formats, for different platforms, with the possibility of A / B testing of each icon.

You can read the first part from the link .

Last time, we prepared Sketch files containing all the icons in the right styles and with the correct names. It's the turn of writing code.

Suffice it to say that we went through trial and error. After our team leader Nihil Verma, who laid the foundations of the script, developed the key source code, I started the process, which required at least three phases of refactoring and many modifications. For this reason, I will not go into the details of creating the script and focus on how it works today, in its final form.

Build script

The build script written on Node.js is fairly straightforward in its work: by importing dependencies, declaring a list of Sketch files for processing (it is a list of brands, each of which has a list of files related to it) and making sure that Sketch is installed on the client , the script processes the brands in turn, doing a series of actions with each of them.

Takes design tokens appropriate for brands (we need color values).
Clones brand-associated Sketch files, unzips them to extract internal JSON files, and processes some of their internal values (more on that later).
Reads the necessary metadata from these JSON files ( document.json , meta.json and pages / pageUniqueID.json ). We are interested in lists of common styles and resources / icons contained in files.
After a few more manipulations with JSON files, it recreates the archive and, using Sketch files (cloned and updated), exports and creates final output files for three platforms (iOS, Android, Mobile Web).

The relevant parts of the build script can be found here:

// ... modules imports here
const SKETCH_FILES = {
 badoo: ['icons_common'],
 blendr: ['icons_common', 'icons_blendr'],
 fiesta: ['icons_common', 'icons_fiesta'],
 hotornot: ['icons_common', 'icons_hotornot'],
};
const SKETCH_FOLDER_PATH = path.resolve(__dirname, '../src/');
const SKETCH_TEMP_PATH = path.resolve(SKETCH_FOLDER_PATH, 'tmp');
const DESTINATION_PATH = path.resolve(__dirname, '../dist');
console.log('Build started...');
if (sketchtool.check()) {
 console.log(`Processing Sketch file via ${sketchtool.version()}`);
 build();
} else {
 console.info('You need Sketch installed to run this script');
 process.exit(1);
}
// ----------------------------------------
function build() {
 // be sure to start with a blank slate
 del.sync([SKETCH_TEMP_PATH, DESTINATION_PATH]);
 // process all the brands declared in the list of Sketch files
 Object.keys(SKETCH_FILES).forEach(async (brand) => {
   // get the design tokens for the brand
   const brandTokens = getDesignTokens(brand);
  
   // prepare the Sketch files (unzipped) and get a list of them
   const sketchUnzipFolders = await prepareSketchFiles({
     brand,
     sketchFileNames: SKETCH_FILES[brand],
     sketchFolder: SKETCH_FOLDER_PATH,
     sketchTempFolder: SKETCH_TEMP_PATH
   });
   // get the Sketch metadata
   const sketchMetadata = getSketchMetadata(sketchUnzipFolders);
   const sketchDataSharedStyles = sketchMetadata.sharedStyles;
   const sketchDataAssets = sketchMetadata.assetsMetadata;
   generateAssetsPDF({
     platform: 'ios',
     brand,
     brandTokens,
     sketchDataSharedStyles,
     sketchDataAssets
   });
   generateAssetsSVGDynamicMobileWeb({
     platform: 'mw',
     brand,
     brandTokens,
     sketchDataSharedStyles,
     sketchDataAssets
   });
   generateAssetsVectorDrawableDynamicAndroid({
     platform: 'android',
     brand,
     brandTokens,
     sketchDataSharedStyles,
     sketchDataAssets
   });
 });
}

In fact, the pipeline code is much more complicated. The reason for this complexity lies in the functions prepareSketchFiles, getSketchMetadataand generateAssets[format][platform]. Below I will try to describe them in more detail.

Preparing Sketch Files

The first step in the assembly process is the preparation of sketch files, which will later be used to export resources for various platforms.

Files associated with a particular brand (for example, in the case of Blendr, these are icons_common.sketch and icons_blendr.sketch files ) are cloned into a temporary folder (more precisely, into a subfolder named after the brand being processed) and unzipped.

Then the JSON files are processed. A prefix is added to the name of resources subject to A / B testing - thus, when exporting, they will be saved in a subfolder with a predefined name (corresponding to the unique name of the experiment). You can understand whether a resource is subject to A / B testing by the name of the page on which it is stored: if it is, the name will contain the prefix " XP_ ".

In the example above, the exported resources will be stored in a subfolder of “ this__is_an_experiment ” with a file name of the form “ icon-name [variant-name] .ext ”.

Reading Sketch Metadata

The second important step is to extract all the necessary metadata from Sketch files, or rather, from internal JSON files. As we saw above, these are two main files ( document.json and meta.json ) and page files ( pages / pageUniqueId.json ).

The document.json file is used to get the list of common styles that appears under the property of the layerStyles object :

{
 "_class": "document",
 "do_objectID": "45D2DA82-B3F4-49D1-A886-9530678D71DC",
 "colorSpace": 1,
 ...
 "layerStyles": {
   "_class": "sharedStyleContainer",
   "objects": [
     {
       "_class": "sharedStyle",
       "do_objectID": "9BC39AAD-CDE6-4698-8EA5-689C3C942DB4",
       "name": "features/feature-like",
       "value": {
         "_class": "style",
         "fills": [
           {
             "_class": "fill",
             "isEnabled": true,
             "color": {
               "_class": "color",
               "alpha": 1,
               "blue": 0.10588235408067703,
               "green": 0.4000000059604645,
               "red": 1
             },
             "fillType": 0,
             "noiseIndex": 0,
             "noiseIntensity": 0,
             "patternFillType": 1,
             "patternTileScale": 1
           }
         ],
         "blur": {...},
         "startMarkerType": 0,
         "endMarkerType": 0,
         "miterLimit": 10,
         "windingRule": 1
       }
     },
     ...

We store basic information about each style in a key-value format object. It will be used later when we need to extract the name of the style based on a unique ID (property do_objectIDin Sketch):

const parsedSharedStyles = {};
parsedDocument.layerStyles.objects.forEach((object) => {
 parsedSharedStyles[object.do_objectID] = {
   name: object.name,
   isFill: _.get(object, 'value.fills[0].color') !== undefined,
   isBorder: _.get(object, 'value.borders[0].color') !== undefined,
 };
});

Now we go to the meta.json file and get a list of pages. We are interested in them unique-idand name:

{
 "commit": "623a23f2c4848acdbb1a38c2689e571eb73eb823",
 "pagesAndArtboards": {
   "EE6BE8D9-9FAD-4976-B0D8-AB33D2B5DBB7": {
     "name": "Icons",
     "artboards": {
       "3275987C-CE1B-4369-B789-06366EDA4C98": {
         "name": "badge-feature-like"
       },
       "C6992142-8439-45E7-A346-FC35FA01440F": {
         "name": "badge-feature-crush"
       },
       ...
       "7F58A1C4-D624-40E3-A8C6-6AF15FD0C32D": {
         "name": "tabbar-livestream"
       }
       ...
     }
   },
   "ACF82F4E-4B92-4BE1-A31C-DDEB2E54D761": {
     "name": "XP_this__is_an_experiment",
     "artboards": {
       "31A812E8-D960-499F-A10F-C2006DDAEB65": {
         "name": "this__is_an_experiment/tabbar-livestream[variant1]"
       },
       "20F03053-ED77-486B-9770-32E6BA73A0B8": {
         "name": "this__is_an_experiment/tabbar-livestream[variant2]"
       },
       "801E65A4-3CC6-411B-B097-B1DBD33EC6CC": {
         "name": "this__is_an_experiment/tabbar-livestream[control]"
       }
     }
   },

Then we read the JSON files corresponding to each page in the pages folder (I repeat that the file names are of the form [pageUniqueId] .json ) and study the resources stored on this page (they look like layers). Thus, we get the name , width / height of each icon, Sketch metadata for the icon of this layer, and if we are dealing with an experiment page , then the name of the A / B test and a variant of this icon.

Note: The page.json object has a very complex device, so I won’t stop there. If you are interested in what's inside, I advise you to create a new empty Sketch file, add some content to it and save it; then rename its extension to ZIP, unzip it and examine one of the files in the pages folder.

In the process of processing artboards, we will also create a list of experiments (and related resources). We will need it to determine which variations of the icon are used in which experiment — the names of the variations of the icon are tied to the “base” object.

For each processed Sketch file related to the brand, we create an object assetsMetadatathat looks like this:

{
 "navigation-bar-edit": {
   "do_objectID": "86321895-37CE-4B3B-9AA6-6838BEDB0977",
   ...sketch_artboard_properties,
   "name": "navigation-bar-edit",
   "assetname": "navigation-bar-edit",
   "source": "icons_common",
   "width": 48,
   "height": 48
   "layers": [
     {
       "do_objectID": "A15FA03C-DEA6-4732-9F85-CA0412A57DF4",
       "name": "Path",
       ...sketch_layer_properties,
       "sharedStyleID": "6A3C0FEE-C8A3-4629-AC48-4FC6005796F5",
       "style": {
         ...
         "fills": [
           {
             "_class": "fill",
             "isEnabled": true,
             "color": {
               "_class": "color",
               "alpha": 1,
               "blue": 0.8784313725490196,
               "green": 0.8784313725490196,
               "red": 0.8784313725490196
             },
           }
         ],
         "miterLimit": 10,
         "startMarkerType": 0,
         "windingRule": 1
       },
     },
   ],
   ...
 },
 "experiment-name/navigation-bar-edit[variant]": {
   "do_objectID": "00C0A829-D8ED-4E62-8346-E7EFBC04A7C7",
   ...sketch_artboard_properties,
   "name": "experiment-name/navigation-bar-edit[variant]",
   "assetname": "navigation-bar-edit",
   "source": "icons_common",
   "width": 48,
   "height": 48
   ...

As you can see, in the experiment, a single icon (in this case, navigation-bar-edit ) can correspond to many resources. At the same time, the same icon may appear under the same name in another Sketch file associated with the brand. This is very useful : we use this trick to compile a common set of icons, and then identify specific options according to the brand. That is why we declared the Sketch files associated with a particular brand as an array:

const SKETCH_FILES = {
 badoo: ['icons_common'],
 blendr: ['icons_common', 'icons_blendr'],
 fiesta: ['icons_common', 'icons_fiesta'],
 hotornot: ['icons_common', 'icons_hotornot'],
};

In this case, order is of fundamental importance. In fact, in the function called by the script, getSketchMetadatawe do not return objects assetsMetadataone by one per file in the form of a list. Instead, we carry out deep merging of objects - we combine them and return a single object assetsMetadata.

In general, this is nothing more than a “logical” merger of Sketch files and their resources into a single file. However, the logic is not as simple as it seems. Here's the diagram we created in an attempt to figure out what happens when icons with the same name (and possibly subject to A / B testing) in different files are associated with the same brand:

Creation of ready-made files in different formats for different platforms

The final stage of our process is directly creating icon files in different formats for different platforms (PDF for iOS, SVG / JSX for Web and VectorDrawable for Android).

As can be understood from the number of generateAssets[format][platform]parameters passed to the functions , this part of the pipeline is the most complex. This is where the process begins to break up and change depending on the platform. Below you will see the entire script’s logical progress and how the part related to resource generation is divided into three similar but different processes: To create ready-made resources with the correct colors that correspond to the brand being processed, we need to carry out a few more manipulations with JSON files . We go through all the layers to which the general style is applied, and replace the color values with the colors from the brand’s design token.

To generate files for Android, you need to perform an additional action (about it a little later): we change the property of fill-ruleeach layer from even-oddto non-zero(this is controlled by the property of the JSON object windingRule, in which 1 means “even / odd”, and 0 means “not equal to zero”) .

Having done these manipulations, we pack the JSON files back into a standard Sketch file to process and export resources with updated properties (cloned and updated files are ordinary Sketch files, they can be opened, viewed, edited, saved, etc. )

After that we use SketchTool (wrapped under Node) to automatically export all resources in formats suitable for platforms. For each of the files associated with the brand (or rather, their cloned and updated versions), we run the following command:

sketchtool.run(`export slices ${cloneSketchFile} --formats=svg --scales=1 --output=${destinationFolder} --overwriting`);

As you might guess, this command exports resources to the destination folder in a specific format, optionally using scaling (we retain the original scale for now). The key here is the option -overwriting: just as we conduct deep merging of objects assetsMetadata(corresponding to the “logical” Sketch files), when exporting, we merge many files into one directory (related to the brand / platform). This means that if the resource - identified by the name of the layer - already existed in the previous Sketch file, it will be overwritten during the next export. Again, this is nothing more than a normal merge operation.

However, in this example, some resources may turn out to be “ghosts”. This happens when the icon in the file is subjected to A / B testing, but is overwritten in the subsequent file. Then the variant files are exported to the destination folder, have a link in the object corresponding to the resource assetsMetadata(with their key and properties), but are not associated with any base resource (due to the deep merging of objects assetsMetadata). Such files will be deleted later, before completing the process.

As already mentioned, different platforms require different output formats. iOS files fit PDFs, and we can directly export them using the SketchTool command. JSX files are required for Mobile Web, and VectorDrawable for Android. For this reason, we export resources in the SVG format to a temporary folder and after that we process it.

PDFs for iOS

Oddly enough, PDF is the only (?) Format that Xcode and OS / iOS support for importing and rendering vector resources ( here is a short explanation of Apple’s choice).

Since we can directly export to PDF via SketchTool, no additional steps are required: just save the files directly to the destination folder, and that’s it.

React / JSX Web Files

In the case of the Web, we use the SVGR library Node, which allows you to convert SVG to React components. However, we want to do something cooler: “dynamically colorize” the icon during execution (colors are taken from tokens). To do this, before converting, we change the values fillfor vectors to which the general style was previously applied to values from tokens corresponding to this style.

So if the badge-feature-like.svg file exported from Sketch looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<svg width="128px" height="128px" viewBox="0 0 128 128" version="1.1" xmlns="<a href="http://www.w3.org/2000/svg">http://www.w3.org/2000/svg</a>" xmlns:xlink="<a href="http://www.w3.org/1999/xlink">http://www.w3.org/1999/xlink</a>">
 <!-- Generator: sketchtool 52.2 (67145) -<a href="http://www.bohemiancoding.com/sketch"> http://www.bohemiancoding.com/sketch</a> -->
 <title>badge-feature-like</title>
 <desc>Created with sketchtool.</desc>
 <g id="Icons" fill="none" fill-rule="evenodd">
   <g id="badge-feature-like">
     <circle id="circle" fill="#E71032" cx="64" cy="64" r="64">
     <path id="Shape" fill="#FFFFFF" d="M80.4061668,..."></path>
   </g>
 </g>
</svg>

then the final resource / badge-feature-like.js icon will look like this:

/* This file is generated automatically - DO NOT EDIT */
/* eslint-disable max-lines,max-len,camelcase */
const React = require('react');
module.exports = function badge_feature_like({ tokens }) {
 return (
   <svg data-origin="pipeline" viewBox="0 0 128 128">
     <g fill="none" fillRule="evenodd">
       <circle fill={tokens.TOKEN_COLOR_FEATURE_LIKED_YOU} cx={64} cy={64} r={64} />
       <path fill="#FFF" d="M80.4061668,..." />
     </g>
   </svg>
 );
};

As you can see, we replaced the static color value with a filldynamic one, taking values from tokens (they can be made available for the React component <Icon/>through the Context API, but this is a different story).

This replacement is possible thanks to Sketch metadata for the object’s resources assetsMetadata: recursively walking through the layers, you can create a DOM selector (for example, this is above #Icons#badge-feature-like #circle) and use it to search for a node in the SVG tree and replace its attribute value fill(for this we need the cheerio library ).

VectorDrawable Files for Android

Android supports vector graphics using the custom VectorDrawable vector format . Typically, converting from SVG to VectorDrawable is done directly in Android Studio . However, we wanted to fully automate the process, so we were looking for a way to convert using code.

Having studied various tools and libraries, we settled on svg2vectordrawable . It is not only actively supported (in any case, more active than everyone else), but also more functional than the rest.

The realities are that VectorDrawable and SVG are not the same in their functionality: some SVG functions (for example, radial gradients and complex selection) are not supported by VectorDrawable, while others started to be supported quite recently (starting with Android API 24). One of the problems arising from this is that older versions (up to 24) do not support the even-odd value of the fill-rule attribute . However, we at Badoo need support for Android 5 and above. That's why, at one of the earlier stages, we brought filleach vector in the Sketch files to a value non-zero.

In principle, designers can perform this action manually:

But this is easy to forget and make a mistake. Therefore, we decided to add an additional step to the process for Android, at which all vectors in JSON are automatically converted to non-zero. This is done so that when exporting icons to SVG, they are already in the required format, and each created VectorDrawable object is supported by Android devices 5. The

finished badge-feature-like.xml file looks like this:

<!-- This file is generated automatically - DO NOT EDIT -->
<vector xmlns:android="<a href="http://schemas.android.com/apk/res/android">http://schemas.android.com/apk/res/android</a>"
 android:width="128dp"
 android:height="128dp"
 android:viewportWidth="128"
 android:viewportHeight="128">
 <path
   android:fillColor="?color_feature_liked_you"
   android:pathData="M64 1a63 63 0 1 0 0 126A63 63 0 1 0 64 1z"
 />
 <path
   android:fillColor="#FFFFFF"
   android:pathData="M80.406 ..."
 />
</vector>

In VectorDrawable files, we insert variable names for the colors fillthat are associated with design tokens through common styles in Android applications. It is worth noting that Android Studio has strict requirements for organizing resources: no subfolders and capital letters in the names! So we had to come up with a new format for icons names: in the case they are subject to testing resources look like this: .

ic_icon-name__experiment-name__variant-name

JSON Dictionary as a Resource Library

After the resource files are saved in the final format, it remains only to collect all the meta-information obtained during the assembly and save it into a “dictionary” to use when the resources are imported and used by the code base of various platforms.

After extracting a flat list of icons from the object, assetsMetadata we go through it, checking each of them:

Is this an ordinary resource (for example tabbar-livestream); if so, then just leave it;
if this is an option for an A / B test (for example, experiment / tabbar-livestream [variant] ), we associate its name, path, names of the A / B test and variant with the property of the abtests
base resource (in our case, it is tabbar-livestream ) , after which we delete the record of the variant from the list / object (only the “base" element matters);
if it is a “ghost”, then delete the file and remove the entry from the list / object.

After completing this process, the dictionary will contain a list of all the basic icons (and their A / B tests, if any), and only them. Information about each of them includes the name, size, path and, if the icon is subject to A / B testing, information about its various options.

The dictionary is saved in JSON format in the destination folder for the brand and platform . Here, for example, is the assets.json file generated for the Blendr application for Mobile Web:

{
 "platform": "mw",
 "brand": "blendr",
 "assets": {
     "badge-feature-like": {
     "assetname": "badge-feature-like",
     "path": "assets/badge-feature-like.jsx",
     "width": 64,
     "height": 64,
     "source": "icons_common"
   },
   "navigation-bar-edit": {
     "assetname": "navigation-bar-edit",
     "path": "assets/navigation-bar-edit.jsx",
     "width": 48,
     "height": 48,
     "source": "icons_common"
   },
   "tabbar-livestream": {
     "assetname": "tabbar-livestream",
     "path": "assets/tabbar-livestream.jsx",
     "width": 128,
     "height": 128,
     "source": "icons_blendr",
     "abtest": {
       "this__is_an_experiment": {
         "control": "assets/this__is_an_experiment/tabbar-livestream__control.jsx",
         "variant1": "assets/this__is_an_experiment/tabbar-livestream__variant1.jsx",
         "variant2": "assets/this__is_an_experiment/tabbar-livestream__variant2.jsx"
       },
       "a_second-experiment": {
         "control": "assets/a_second-experiment/tabbar-livestream__control.jsx",
         "variantA": "assets/a_second-experiment/tabbar-livestream__variantA.jsx"
       }
     }
   },
   ...
 }
}

Now all that remains is to pack all the assets folders into ZIP archives for easier downloading.

Total

The process described in the article, from cloning and manipulating Sketch files to exporting and converting resources to formats supported by platforms and saving collected meta-information in the resource library, is repeated with every brand announced in the build script.

Here is a screenshot showing the appearance of the src and dist folders after the process is completed:

At this stage, with one simple command, you can upload all the resources (JSON, ZIP and resource files) to the remote storage and make them available for all platforms for downloading and use in code base.

How exactly the platforms receive and process resources (using custom scripts created specifically for this purpose) does not go beyond the scope of the article. And this question will probably be covered in one of the following posts by one of my colleagues.

Conclusion (and lessons learned)

I always loved Sketch. For many years, the program has been the default tool for developers and designers. Therefore, I was very curious to learn integration tools like html-sketchapp and other similar tools that we could use in the workflow and our pipelines.

I, like many others , have always striven for such an (ideal) process:

However, I must admit that I began to doubt that Sketch is a suitable tool, especially considering the design system. Therefore, I began to look at other services, such as Figma with its open APIs and Framer X with convenient integration with React, because I did not feel at Sketch movement towards integration with the code (whatever it was).

So, this project convinced me. Not completely, but in many ways.

Although Sketch does not open its APIs, the very device of the internal structure of its files serves as a kind of “unofficial” API. Creators could use encrypted names or hide keys in JSON objects, but instead they adhere to a clear, readable, and conceptual naming convention. I do not think this is an accident.

The fact that Sketch files can be managed in this way has opened the way for me to many future developments and improvements: from plugins to check the name, stylization and structure of layers for icons to integration with our Wiki and the documentation of our design system (mutual). Creating Node Applications on Electron or Carlo, we can make it easier for designers to complete many routine tasks.

One of the unexpected (at least for me) bonuses was that Sketch files with Cosmos icons became a “source of truth” - something similar happened with the Cosmos design system. If there is no icon, then it does not exist in the code base (or rather, it should not exist; but now we know that such cases are an exception). Now it seems obvious, but it was not always like that - again, at least for me.

When it came to us that Sketch files could be manipulated, what started as an MVP project turned into a deep immersion in their internal structure. We do not yet know where this trail will lead us, but so far we are lucky. Designers, developers, product managers, management - all agree that the innovation will simplify the work of each of us and prevent the occurrence of many errors. In addition, we have opened the door to new ways to use icons.

And the last: the conveyor described by me was built for our needs, and therefore it is adapted to our realities. Keep in mind that it may not meet the requirements of your company.

The main thing I wanted to share is that everything is possible. Maybe in a different way, with different approaches and other file formats, with less complexity (for example, you may not need support for several brands and A / B testing), but now you can automate the process of delivering icons with a custom Node.js script and Sketch

Look for your way! It is fun and quite simple.

Acknowledgments

This huge project was developed jointly with Nihil Verma (Mobile Web), who created the first version of the build script, Artyom Rudym (Android) and Igor Savelyev (iOS), who developed scripts for importing and receiving resources for their platforms.

Thanks guys! It was a real pleasure to work with you on the project and put it into practice.

Tags: