Intel Software Guard Extensions Tutorial Part 4, enclave design

Original author: John M.
  • Transfer
In the fourth part of a series of training materials on Intel Software Guard Extensions (Intel SGX), we will create an enclave and its interface. We will consider the boundaries of the enclave defined in the third part , and determine the necessary functions of the bridge, consider the influence of the functions of the bridge on the object model and create the project infrastructure necessary to integrate the enclave into our application. Instead of an ECALL enclave, we use stubs so far; we will move on to the full integration of the enclave in the fifth part of this series.

Along with this part of the series, the source code is provided: an enclave stub and interface functions; This code is available for download.

Application architecture

Before designing an enclave interface, you need to think about the overall architecture of the application. As we discussed in the first part , enclaves are implemented as dynamic link libraries (DLLs on Windows * and shared libraries on Linux *) and should only be linked with 100% native C code.

The Tutorial Password Manager graphical user interface is written in C # A mixed assembly written in C ++ / CLI is used to switch from managed to unmanaged code, but although this assembly contains native code, it does not consist of 100% native code, and therefore cannot directly interact with the Intel SGX enclave. Attempts to inject untrusted enclave bridge functions into C ++ / CLI assemblies will result in fatal errors:

Command line error D8045: cannot compile C file ’Enclave_u.c’; with the /clr option

This means that you need to place the functions of an untrusted bridge in a separate DLL-library, which consists entirely of native code. As a result, our application will have at least three DLLs: the C ++ / CLI core, the enclave bridge, and the enclave itself. This structure is shown in Fig. 1.

Figure 1. Components of a mixed enclave application.

Further refinement

Since the functions of the enclave bridge must be in a separate DLL-library, we will take the next step: we will put in this library all the functions that directly interact with the enclave. Such separation of application levels will simplify program management and debugging, as well as increase the convenience of integration by reducing the impact on other modules. If a class or module performs a specific task with a clearly defined boundary, changes to other modules are less likely to affect it.

In our case, the PasswordManagerCoreNative class should not be burdened with the additional task of creating enclave instances. This class only needs to know if the Intel SGX extension platform supports it in order to perform the corresponding function.

As an example, the following code fragment shows the unlock () method :

int PasswordManagerCoreNative::vault_unlock(const LPWSTR wpassphrase)
	int rv;
	UINT16 size;
	char *mbpassphrase = tombs(wpassphrase, -1, &size);
	if (mbpassphrase == NULL) return NL_STATUS_ALLOC;
	rv= vault.unlock(mbpassphrase);
	SecureZeroMemory(mbpassphrase, size);
	delete[] mbpassphrase;
	return rv;

This is a very simple method: it takes the user's passphrase in the form of wchar_t, converts it to a variable-length encoding (UTF-8), then calls the unlock () method in the storage object. Instead of cluttering this class and this method with enclave functions, it is better to add enclave support to this method by adding one line:

int PasswordManagerCoreNative::vault_unlock(const LPWSTR wpassphrase)
	int rv;
	UINT16 size;
	char *mbpassphrase = tombs(wpassphrase, -1, &size);
	if (mbpassphrase == NULL) return NL_STATUS_ALLOC;
	// Call the enclave bridge function if we support Intel SGX
	if (supports_sgx()) rv = ew_unlock(mbpassphrase);
	else rv= vault.unlock(mbpassphrase);
	SecureZeroMemory(mbpassphrase, size);
	delete[] mbpassphrase;
	return rv;

Our goal is to free this class from working with the enclave to the greatest extent possible. Other necessary additions for the PasswordManagerCoreNative class are : Intel SGX flag support and methods for setting and receiving this flag.

class PASSWORDMANAGERCORE_API PasswordManagerCoreNative
	int _supports_sgx;
	// Other class members ommitted for clarity
	void set_sgx_support(void) { _supports_sgx = 1; }
	int supports_sgx(void) { return _supports_sgx; }

Enclave Design

The general plan of the application is ready, so you can start designing the enclave and its interface. To do this, we return to the class diagram of the application kernel, which we first described in the third part - it is shown in Fig. 2. Enclosed objects are shaded in green, and untrusted components are shaded in blue.

Figure 2. Class diagram in Tutorial Password Manager with Intel Software Guard Extensions.

Only one connection crosses the enclave boundary: the connection between the PasswordManagerCoreNative object and the Vault object . This means that most of our ECALLs will simply be wrappers of class methods in Vault . Additional ECALLs must also be added to manage the enclave infrastructure. One of the difficulties in developing an enclave is that ECALL, OCALL, and bridge functions must be native C code, and we make extensive use of C ++ components. After starting the enclave, we will also need functions that fill the gap between C and C ++ (objects, constructors, overloads, and others).

The shells and functions of the bridge will be in their own DLL, which we will call EnclaveBridge.dll. For clarity, we will provide the shell functions with the prefix "ew_" (enclave wrapper - enclave shell), and the bridge functions that make up ECALL with the prefix "ve_" (vault enclave - enclave storage).

Calls from PasswordManagerCoreNative to the corresponding method in Vault will follow the path shown in Figure 3.

Figure 3. The path to performing the bridge and ECALL functions.

The method in PasswordManagerCoreNative calls the wrapper function in EnclaveBridge.dll. This shell, in turn, calls one or more ECALLs that enter the enclave and call the corresponding class method in the Vault object . After all ECALLs have completed, the wrapper function returns to the calling method in PasswordManagerCoreNative and provides it with the returned value.

Logistics Enclave

When creating an enclave, you first need to decide on a system to manage the enclave itself. The enclave must be running, and the resulting enclave identifier must be provided to ECALL functions. Ideally, all this should be transparent to the upper levels of the application.

The simplest solution for Tutorial Password Manager is to use global variables in the EnclaveBridge DLL to host enclave information. Such a decision is fraught with a restriction: in an enclave there can only be one active stream at a time. This is a reasonable solution, because the performance of the password manager will not increase anyway when using multiple threads to work with the repository. Most actions are controlled by the user interface; they do not form a significant load on the CPU.

To solve the transparency problem, each wrapper function must first call a function to check if the enclave is running, and run it if it is not already running. The logic is pretty simple:

#define ENCLAVE_FILE _T("Enclave.signed.dll")
static sgx_enclave_id_t enclaveId = 0;
static sgx_launch_token_t launch_token = { 0 };
static int updated= 0;
static int launched = 0;
static sgx_status_t sgx_status= SGX_SUCCESS;
// Ensure the enclave has been created/launched.
static int get_enclave(sgx_enclave_id_t *eid)
	if (launched) return 1;
	else return create_enclave(eid);
static int create_enclave(sgx_enclave_id_t *eid)
	sgx_status = sgx_create_enclave(ENCLAVE_FILE, SGX_DEBUG_FLAG, &launch_token, &updated, &enclaveId, NULL);
	if (sgx_status == SGX_SUCCESS) {
		if ( eid != NULL ) *eid = enclaveId;
		launched = 1;
		return 1;
	return 0;

First, each wrapper function calls the get_enclave () function , which checks if the enclave is running against a static variable. If yes, then this function (if necessary) places the enclave identifier in the eid pointer . This is an optional step because the enclave identifier is also stored in the global variable enclaveID , and you can use it directly.

What happens if the enclave is lost due to a power failure or an error that causes an abnormal termination? To do this, we check the return value of ECALL: it indicates the success or failure of the ECALL operation itself, and not the function called in the enclave.

sgx_status = ve_initialize(enclaveId, &vault_rv);

The return value of the function called in the enclave, if any, is passed through the pointer provided as the second argument to ECALL (these function prototypes are automatically created by Edger8r). Always check the return value of ECALL. Any result other than SGX_SUCCESS indicates that the program was unable to successfully enter the enclave, and the requested function was not launched. (Note that we also defined sgx_status as a global variable. This is another simplification due to the single-threaded architecture of our application).

We will add a function that analyzes the error returned by the ECALL function and checks the status of the enclave (lost, crash):

static int lost_enclave()
	if (sgx_status == SGX_ERROR_ENCLAVE_LOST || sgx_status == SGX_ERROR_ENCLAVE_CRASHED) {
		launched = 0;
		return 1;
	return 0;

These are fixable errors. In the upper levels, there is no logic capable of dealing with these conditions yet, but we provide it in the EnclaveBridge DLL to support further development of the program.

Also note the lack of function to destroy the enclave. While the password manager application is open in the user's system, an enclave exists in memory, even if the user has locked his vault. This is not a good way to work with enclaves. Enclaves consume resources from a far from unlimited pool, even with inaction. We will deal with this issue in a future installment of this series when we talk about sealing data.

Enclave Definition Language

Before moving on to enclave design, let's talk a little about the syntax of the Enclave Definition Language (EDL). The enclave bridge functions, both ECALL and OCALLs, have prototypes in the EDL file with the following general structure:

enclave {
	// Include files
	// Import other edl files
	// Data structure declarations to be used as parameters of the function prototypes in edl
	trusted {
	// Include file if any. It will be inserted in the trusted header file (enclave_t.h)
	// Trusted function prototypes (ECALLs)
	untrusted {
	// Include file if any. It will be inserted in the untrusted header file (enclave_u.h)
	// Untrusted function prototypes (OCALLs)

ECALL prototypes are in the trusted part, and OCALL are in the untrusted part. The syntax of the EDL language is similar to the syntax of C, and the prototypes of the EDL functions are very similar to the prototypes of the C functions, but not identical to them. In particular, the bridge function parameters and return values ​​are limited to some fundamental data types, and the EDL includes additional keywords and syntax to determine enclave behavior. The Intel Software Guard Extensions (Intel SGX) SDK User Guide describes the EDL syntax in detail and provides a tutorial on creating an example enclave. We will not repeat everything that is written there, but simply discuss the elements of this language related to our application.

When parameters are passed to enclave functions, they are placed in a protected memory space of the enclave. For parameters passed as values, no additional actions are required, since the values ​​are placed on the enclave's protected stack, as for calling any other functions. For pointers, the situation is completely different.

For parameters passed as pointers, the data referenced by the pointer must be passed to and from the enclave. The boundary procedures performing this data transfer must “know” two things:

  1. In which direction should the data be copied: to the bridge function, from the bridge function, or both ways?
  2. What is the size of the data buffer referenced by the pointer?

Pointer direction

When providing a pointer parameter function, you must specify the direction using the keywords in square brackets: Accordingly, [in], [out] or [in, out]. The meaning of these keywords is shown in table 1.
inThe buffer is copied from the application to the enclave. Changes will only affect the buffer inside the enclave.The buffer is copied from the enclave to the application. Changes will only affect the buffer outside the enclave.
outThe buffer will be allocated inside the enclave and initialized with zero values. It will be copied to the source buffer when ECALL exits.The buffer will be allocated outside the enclave and initialized with zero values. This untrusted buffer will be copied to the original buffer when OCALL exits.
in, out Data is copied back and forth. Same as in ECALL.
Table 1. Pointer direction parameters and their values ​​in ECALL and OCALL.

Note that the direction is relative to the called function of the bridge. For the ECALL function, [in] means “copy buffer to enclave”, but for OCALL the same parameter means “copy buffer to untrusted function”. (There is also a user_check parameter that can be used instead, but it does not relate to the subject of our discussion. For information on its purpose and use, see the SDK documentation.)

Buffer size

Boundary procedures compute the total size of the buffer in bytes as follows:

количество байт = element_size * element_count

By default, for border procedures, the value of element_count is 1, and element_size is calculated based on the element referenced by the pointer parameter, for example, for an integer pointer, element_size will be like this:


For a single element of a fixed data type, such as int or float, it is not necessary to provide any additional information in the prototype of the EDL function. The void pointer must be set to the size of the element; otherwise, an error will occur during compilation. For arrays, char and wchar_t strings, and other types where the data buffer is longer than one element, you must specify the number of elements in the buffer, otherwise only one element will be copied.

Add a count or size parameter (or both) to the keywords in square brackets. They can be set to a constant value or one of the parameters of the function. In most cases, the functionality of count and sizesame, but it is recommended to use them in the right context. Strictly speaking, size should only be specified when passing the void pointer. In other cases, use count.

When passing a string C and wstring (a char or wchar_t array with NULL termination), you can use the string or wstring parameter instead of count or size . In this case, the boundary procedures will determine the size of the buffer by obtaining the length of the string directly.

function([in, size=12] void *param);
function([in, count=len] char *buffer, uint32_t len);
function([in, string] char *cstr);

Please note that you can use string or wstring only if the direction [in] or [in, out] is specified. If only the [out] direction is specified, the line has not yet been created, so the boundary procedure cannot get the size of the buffer. If you specify [out, string], an error will occur during compilation.

Shell and bridge functions

Now you can define the shell and functions of the bridge. As stated above, most of our ECALLs will simply be wrappers of class methods in Vault . The class definition for public member functions is shown below:

	// Non-public methods and members ommitted for brevity
	int initialize();
	int initialize(const char *header, UINT16 size);
	int load_vault(const char *edata);
	int get_header(unsigned char *header, UINT16 *size);
	int get_vault(unsigned char *edate, UINT32 *size);
	UINT32 get_db_size();
	void lock();
	int unlock(const char *password);
	int set_master_password(const char *password);
	int change_master_password(const char *oldpass, const char *newpass);
	int accounts_get_count(UINT32 *count);
	int accounts_get_info(UINT32 idx, char *mbname, UINT16 *mbname_len, char *mblogin, UINT16 *mblogin_len, char *mburl, UINT16 *mburl_len);
	int accounts_get_password(UINT32 idx, char **mbpass, UINT16 *mbpass_len);
	int accounts_set_info(UINT32 idx, const char *mbname, UINT16 mbname_len, const char *mblogin, UINT16 mblogin_len, const char *mburl, UINT16 mburl_len);
	int accounts_set_password(UINT32 idx, const char *mbpass, UINT16 mbpass_len);
	int accounts_generate_password(UINT16 length, UINT16 pwflags, char *cpass);
	int is_valid() { return _VST_IS_VALID(state); }
	int is_locked() { return ((state&_VST_LOCKED) == _VST_LOCKED) ? 1 : 0; }

There are several problematic features in this class. Some of them are obvious: for example, the constructor, destructor, and overloads for initialize () . These are the C ++ components that we need to call using the C functions. Some problems are not so obvious, because they are inherent in the function device. Some of these problematic methods were improperly created on purpose, so that we could address certain problems in this tutorial, but other methods were incorrectly created without any far-reaching goals, it just happened. We will solve these problems sequentially by introducing both prototypes for shell functions and EDL prototypes for proxy / bridge procedures.

Constructor and Destructor

In a branch of code without using Intel SGX, the Vault class is a member of PasswordManagerCoreNative . This cannot be done in the Intel SGX code branch. However, an enclave can include C ++ code if the bridge functions themselves are C. functions.

Since we have limited the enclave to a single thread, we can make the Vault class a static global object in the enclave. This greatly simplifies the code and eliminates the need to create bridge and logic functions to create instances.

Overload of initialize () method

There are two prototypes of the initialize () method :

  1. A method with no arguments initializes a Vault object for a new password store with no content. This is a password store created by the user for the first time.
  2. The two-argument method initializes the Vault object from the repository file header. This is an existing password store that the user opens (and then tries to unlock).

This method is divided into two wrapper functions:

ENCLAVEBRIDGE_API int ew_initialize();
ENCLAVEBRIDGE_API int ew_initialize_from_header(const char *header, uint16_t hsize);

The corresponding ECALL functions are defined as follows:

public int ve_initialize ();
public int ve_initialize_from_header ([in, count=len] unsigned char *header, uint16_t len);

get_header ()

This method has a fundamental problem. Here is a prototype:

int get_header(unsigned char *header, uint16_t *size);

This function performs the following tasks:

  1. It receives the header block for the storage file and places it in the buffer pointed to by the header. The caller must allocate enough memory to store this data.
  2. If you pass a NULL pointer in the header parameter, then the uint16_t that the pointer points to sets the size of the header block, so the calling method knows how much memory to allocate.

This is a fairly common compression technique in some programming communities, but for enclaves a problem arises here: when transferring a pointer to ECALL or OCALL, the boundary functions copy the data referenced by the pointer to or from the enclave (or both). These boundary functions require a data buffer size to know how many bytes to copy. In the first case, a valid pointer with a variable size is used, which is not difficult, but in the second case we have a NULL pointer and a size equal to zero.

One could come up with such a prototype EDL for the ECALL function, in which it all works, but usually clarity is more important than brevity. Therefore, it is better to split the code into two ECALL functions:

public int ve_get_header_size ([out] uint16_t *sz);
public int ve_get_header ([out, count=len] unsigned char *header, uint16_t len);

The enclave shell function will provide the necessary logic so that we do not need to modify other classes:

ENCLAVEBRIDGE_API int ew_get_header(unsigned char *header, uint16_t *size)
	int vault_rv;
	if (!get_enclave(NULL)) return NL_STATUS_SGXERROR;
	if ( header == NULL ) sgx_status = ve_get_header_size(enclaveId, &vault_rv, size);
	else sgx_status = ve_get_header(enclaveId, &vault_rv, header, *size);

accounts_get_info ()

This method works similarly to get_header () : it passes a NULL pointer and returns the size of the object in the corresponding parameter. However, this method is not distinguished by elegance and convenience due to the many arguments of the parameters. It is better to divide it into two shell functions:

ENCLAVEBRIDGE_API int ew_accounts_get_info_sizes(uint32_t idx, uint16_t *mbname_sz, uint16_t *mblogin_sz, uint16_t *mburl_sz);
ENCLAVEBRIDGE_API int ew_accounts_get_info(uint32_t idx, char *mbname, uint16_t mbname_sz, char *mblogin, uint16_t mblogin_sz, char *mburl, uint16_t mburl_sz);

And two corresponding ECALL functions:

public int ve_accounts_get_info_sizes (uint32_t idx, [out] uint16_t *mbname_sz, [out] uint16_t *mblogin_sz, [out] uint16_t *mburl_sz);
public int ve_accounts_get_info (uint32_t idx, 
	[out, count=mbname_sz] char *mbname, uint16_t mbname_sz, 
	[out, count=mblogin_sz] char *mblogin, uint16_t mblogin_sz,
	[out, count=mburl_sz] char *mburl, uint16_t mburl_sz

accounts_get_password ()

This is the most problematic code in the entire application. Here is a prototype:

int accounts_get_password(UINT32 idx, char **mbpass, UINT16 *mbpass_len);

The first thing that catches your eye: it passes a pointer to a pointer to mbpass. This method allocates memory.

Clearly this is not a good idea. No other method in the Vault class allocates memory, so it is internally inconsistent, and the API violates the convention because it does not provide a method to free memory on behalf of the caller. This also raises a problem that is peculiar only to enclaves: an enclave cannot allocate memory in untrusted space.

This can be handled in a wrapper function. It could allocate memory and then form ECALL, and all this would be transparent to the caller, but despite this we are forced to change the method in the Vault class , so it’s enough to just fix everything as needed and make the appropriate changes to the classPasswordManagerCoreNative . The calling method should receive two functions: one to get the length of the password, the other to get the password, as in the two previous examples. The PasswordManagerCoreNative class should be responsible for allocating memory, and not any of these functions (the code branch should be changed without using Intel SGX).

ENCLAVEBRIDGE_API int ew_accounts_get_password_size(uint32_t idx, uint16_t *len); 
ENCLAVEBRIDGE_API int ew_accounts_get_password(uint32_t idx, char *mbpass, uint16_t len);

The definition of EDL should now look quite familiar:

public int ve_accounts_get_password_size (uint32_t idx, [out] uint16_t *mbpass_sz); 
public int ve_accounts_get_password (uint32_t idx, [out, count=mbpass_sz] char *mbpass, uint16_t mbpass_sz);

load_vault ()

The problem with load_vault () is not entirely obvious. The prototype is quite simple and may seem completely harmless:

int load_vault(const char *edata);

This method loads the encrypted serialized password database into the Vault object . Since the Vault object has already read the header, it knows what size the incoming buffer will be.

And the problem here is that the boundary functions of the enclave do not have this information. It is necessary to explicitly pass the length to ECALL so that the boundary function knows how many bytes to copy from the input buffer to the enclave's internal buffer, but size information is stored inside the enclave. It is not available for the boundary function.

The prototype of the wrapper function can reproduce the prototype of the class method:

ENCLAVEBRIDGE_API int ew_load_vault(const unsigned char *edata);

In this case, the ECALL function must pass the size of the header as a parameter so that it can be used to determine the size of the input data buffer in the EDL file:

public int ve_load_vault ([in, count=len] unsigned char *edata, uint32_t len)

To make all this transparent to the calling method, additional logic will be added to the wrapper function. She will be responsible for retrieving the storage size from the enclave and for passing this size as a parameter to the ECALL function.

ENCLAVEBRIDGE_API int ew_load_vault(const unsigned char *edata)
	int vault_rv;
	uint32_t dbsize;
	if (!get_enclave(NULL)) return NL_STATUS_SGXERROR;
	// We need to get the size of the password database before entering the enclave
	// to send the encrypted blob.
	sgx_status = ve_get_db_size(enclaveId, &dbsize);
	if (sgx_status == SGX_SUCCESS) {
		// Now we can send the encrypted vault data across.
		sgx_status = ve_load_vault(enclaveId, &vault_rv, (unsigned char *) edata, dbsize);

A few words about Unicode

In the third part, we mentioned that the PasswordManagerCoreNative class also deals with converting strings between wchar_t and char formats. Why do this at all, because enclaves support the wchar_t data type?

This decision is aimed at reducing the resource consumption of our application. On Windows, the wchar_t data type is a native encoding for Win32 APIs; it stores UTF-16 encoded characters . In UTF-16 encoding, each character occupies 16 bits: this decision was made to support characters other than ASCII, in particular, for languages ​​that do not use the Latin alphabet or contain many characters. The problem with UTF-16 encoding is that any character always has a length of 16 bits, even if it is plain ASCII text.

It is hardly advisable to store twice as much data on disk and inside the enclave, since user account information is usually plain ASCII text. We also note a decrease in performance due to the need to copy and encrypt more data. Therefore, in Tutorial Password Manager, all strings coming from .NET are converted to UTF-8 encoding. UTF-8 is a variable-length encoding that uses one to four bytes of 8 bits each to represent each character. This encoding is backward compatible with ASCII and is more compact than UTF-16 for plain ASCII text. There are situations when the use of UTF-8 will lead to the formation of longer lines than when using UTF-16, but since we are not creating a commercial program, but a training password manager, we will come to terms with this.

In a commercial application, you should choose the best encoding depending on the language of the system and write this encoding to the repository (so that it knows what encoding was used to create it, in case the repository is opened in a system with a different language).

Code example

As mentioned above, this part provides sample code to download . The attached archive includes the source code for the Tutorial Password Manager bridge DLL and the enclave DLL. Enclave functions so far are just stubs, they will be filled in the fifth part.

In future releases

In the fifth part of this tutorial, we will complete the creation of an enclave by moving the Crypto, DRNG, and Vault classes to the enclave and connecting them with the ECALL functions. Follow the news!

Also popular now: