PHP Virtual Machine 7
- Transfer
Good day to all! My name is Konstantin, at Badoo I work in the Features Team. Most likely, you already know that our backend is written in PHP and serves more than three hundred million users. So I could not miss the chance to translate this article by PHP core developer Nikita Popov. I am sure it will be useful to developers of all levels, but for beginners it may seem complicated. Enjoy (and useful) read!
This article provides an overview of the Zend virtual machine for PHP 7. This is not an exhaustive description, but I will try to cover most of the important parts, as well as some details.
The description is based on PHP version 7.2 (currently under development), but almost everything is true for PHP 7.0 / 7.1. However, the differences from the virtual machines of the PHP 5.x series are significant, and as a rule I did not draw parallels with them.
Most of the article deals with things at the level of instruction listings, and only a few sections at the end relate to the level of actual implementation of the virtual machine in C. Nevertheless, I want to provide links to the main files that make up the virtual machine:
Initially, there was an opcode. Saying “opcode”, we refer to the entire virtual machine instruction (including operands), but it can also indicate only the “actual” operation code, which is a small integer that determines the type of instruction. The intended meaning should be clear from the context. In the source code, the entire instructions are usually called oplines.
A separate instruction corresponds to the following zend_op structure :
Thus, opcodes are essentially instructions in the format of a “three-address code”. There is an opcode defining the type of instruction, there are two input operands op1 and op2 and one output operand result .
Not all instructions use all operands. The ADD statement (representing the + operator ) will use all three. The BOOL_NOT statement (representing the operator ! ) Uses only op1 and result. The ECHO instruction uses only op1. Some instructions may or may not use the operand. For example, DO_FCALLmay or may not have a result operand (depending on whether the return value of the function call is used). Some instructions require more than two input operands, in which case they will simply use the second auxiliary instruction ( OP_DATA ) for additional operands .
Next to these three standard operands, there is an additional numeric field extended_value , which can be used to store additional instruction modifiers. For example, for CAST, it may contain the target type to cast to.
Each operand has a type stored in op1_type , op2_type and result_typerespectively. Possible types: IS_UNUSED , IS_CONST , IS_TMPVAR , IS_VAR and IS_CV .
The last three types are designed for variable operands (with three different types of virtual machine variables), IS_CONST stands for constant operand ( 5 , or “string” , or even [1, 2, 3] ), while IS_UNUSED stands for operand, which is either not actually used or used as a 32-bit numerical value (the so-called direct operand). For example, the jump instruction will store the jump address in the UNUSED operand .
In the future, I will often demonstrate the fragments of the opcode that PHP generates. There are currently three ways to obtain such opcode dumps:
Of these, opcache provides the best result. The listings used in this article are based on opcache dumps, with minor syntax adjustments. The magic number 0x10000 is an abbreviation "before optimization", so we see the opcodes as they were created by the PHP compiler. 0x200000 will give you optimized opcodes. Opcache can also generate a lot more information. For example, 0x40000 will generate CFG, and 0x200000 will produce SSA . But let's not get ahead of events: for our purposes, the usual old linearized opcode dumps are enough.
Probably one of the most important points to consider when working with the PHP virtual machine is the use of three different types of variables. In PHP 5, TMPVAR, VAR, and CV had very different views on the virtual machine stack, and the methods for accessing them were also very different. In PHP 7 they became very similar because they use the same storage mechanism. However, there are important differences in the meanings that they may contain and in their semantics.
CV is short for “compiled variable”. It refers to the "real" PHP variable. If the function uses the $ a variable , then there will be a corresponding CV for it.
CV variables can be of type UNDEFto denote undefined variables. If UNDEF CV is used in the instruction, in most cases this will produce the well-known notification “undefined variable” (undefined variable). At the function input, all CVs that are not arguments are initialized as UNDEF.
CV variables are not destroyed by instructions. For example, the ADD statement $ a, $ b will not destroy the values stored in the variables $ a and $ b . Instead, all CV variables are destroyed simultaneously when leaving the scope. This also implies that all CV variables contain valid values for the duration of the function.
The TMPVAR and VAR variables, in turn, are temporary virtual machine variables. They are usually introduced as the operand of the result of some operation. For example, the code $ a = $ b + $ c + $ d will result in an opcode similar to the following:
TMP / VAR variables are always defined before use and as such cannot contain a UNDEF value. Unlike CVs, these types of variables are destroyed by the instructions in which they are used. In the above example, the second ADD will destroy the value of the operand T0, and after this point T0 should no longer be used. Similarly, ASSIGN will destroy the value of T1, making the variable T1 invalid.
It follows that TMP / VAR variables are usually very short-lived. In most cases, they live only within one instruction. Outside of this short range, the values in them are garbage.
So what are the differences between TMP and VAR variables? There are few of them. The difference was inherited from PHP 5, where TMPs were located on the virtual machine stack, and VARs were on the heap. In PHP 7, all variables are pushed onto the stack. Thus, at present, the main difference between TMP and VAR is that only the latter are allowed to contain references (this allows us to exclude dereferencing (DEREF) of TMP variables). In addition, VARs can contain two types of special values, namely class entries and INDIRECT values. The latter are used to process nontrivial assignments.
This table shows the main differences between the variables:
All PHP functions are represented as structures having a common zend_function header . The concept of “function” is interpreted somewhat more broadly and includes everything from “real” functions and methods to standalone pseudo-main code and eval code.
Custom functions use the zend_op_array structure . She has more than 30 fields, so I'll start with her smaller version:
The most important part here is, of course, opcodes , which are an array of opcodes (instructions). last - the number of opcodes in this array. Note that the terminology is somewhat confusing, since last sounds like it should be the index of the last opcode, while in fact it is the number of opcodes (one more than the last index). The same applies to all other last_ * values in the op_array structure.
last_var is the number of CVs, and T is the number of TMP and VAR (in most cases, we do not distinguish between them). vars is an array of names for CV.
literalsIs an array of literals found in code, something that CONST operands reference . Depending on the ABI, each CONST operand will either contain a pointer to an element of this literal table, or store an offset relative to its beginning.
There is something else in this structure, but it can be postponed.
With the exception of executor globals (EG), all execution status is stored on the virtual machine stack. The VM stack is distributed on pages of 256KiB, and the individual pages are linked through a linked list.
Each time a function is called, a new stack frame is allocated in the virtual machine stack, which has the following scheme:
The frame begins with the zend_execute_data structure , followed by an array of variable slots. The slots are all the same (simple zval), but they are used for different purposes. The first last_var slots are CVs, of which the first num_args contains function arguments. CV slots are followed by T- slots for TMP / VAR. Finally, sometimes there may be additional arguments stored at the end of the frame. They are used for func_get_args () .
The CV and TMP / VAR operands in the instructions are encoded as offsets relative to the beginning of the stack frame, so fetching a specific variable is a simple read from the cell at execute_data plus the specified offset.
The data at the beginning of the frame is defined as follows:
Most importantly, this structure contains opline , which is the currently executing instruction, and func , which is the currently executing function. Moreover:
I missed one field in the execute_data structure, namely call , since it requires some additional explanation of how function calls work.
All calls use the same sequence of instructions. var_dump ($ a, $ b) in global scope compiles to:
There are eight different types of INIT statements (depending on which call it is). INIT_FCALL is used to call functions (not class methods) that we recognize at compile time. Similarly, there are ten different SEND opcodes (depending on the type of arguments and function). There is only a small number of the four DO_CALL opcodes, where ICALL is used to call internal functions.
Although specific instructions may vary, the structure is always the same: INIT, SEND, DO. The main problem that the sequence of calls should handle is the nested function calls that compile something like this:
I formatted a sequence of opcodes to visualize which instructions correspond to which call.
The INIT opcode pushes a call frame onto the stack that contains enough space for all the variables and arguments of the function that we know about (if unpacking the arguments is involved, we can end up with more arguments). This call frame is initialized by the function being called, $ this and called_scope (in this case, both of them will be NULL, since we are calling the function).
A pointer to a new frame is stored in execute_data-> call , where execute_data is the frame of the calling function. In the future, we will denote this as EX (call) . It is noteworthy thatThe prev_execute_data of the new frame is set to the old EX (call) value . For example, INIT_FCALL to call foo will write the var_dump stack frame to prev_execute_data . Thus, prev_execute_data in this case forms a linked list of “unfinished” calls, while usually it provides a backtrace chain .
Then the SEND opcodes proceed to pass arguments to the EX (call) variable slots . At this point, all arguments are consecutive and can flow from the argument section to other CVs or TMPs. This will be fixed later.
Finally, DO_FCALL makes the actual call. What was EX (call), becomes the current function, and prev_execute_data changes to the calling function. In addition, the calling procedure depends on what function it is. Internal functions only need to call the handler function, while user-defined functions must complete the initialization of the stack frame.
This initialization involves tidying up the argument stack. PHP allows you to pass functions more arguments than it expects (and func_get_args relies on this). However, only the actual arguments have the corresponding CV. Any other arguments will be written to memory reserved for other CVs and TMPs. Essentially, these arguments will be placed after TMP, as a result of which the arguments will be split into two separate fragments.
It must be clarified that calls to user-defined functions do not involve recursion at the virtual machine level. They only mean switching from one execute_data to another, but the VM continues to run in a linear loop. Recursive virtual machine calls only occur if internal functions call user callbacks (for example, via array_map). For this reason, infinite recursion in PHP is usually interrupted due to lack of memory or an OOM error, but you can cause the stack to overflow with recursion through callbacks or magic methods.
PHP uses a large number of opcodes to pass arguments, the differences between which can be confusing due to their bad name.
SEND_VAL and SEND_VAR are the simplest options that pass arguments by value when the value is known at compile time. SEND_VAL is used for CONST and TMP operands, and SEND_VAR is used for VAR and CV.
SEND_REF, by contrast, is used for arguments that are known at compile time as references. Since only pointers can be passed by reference, this opcode accepts only VAR and CV.
SEND_VAL_EX and SEND_VAR_EX are SEND_VAL / SEND_VAR options for cases where we cannot statically determine whether an argument is passed by value or by reference. These opcodes check the type of the argument based on arginfo and behave accordingly. In most cases, the actual structure is not the arginfo structure, but a rather compact bit vector directly in the function structure.
And there is also SEND_VAR_NO_REF_EX. Do not try to understand anything from his name - this is an outright lie. This opcode is used when passing something that is not really a variable, but returns a VAR as a statically unknown argument. Two specific examples in which it is used are passing the result of a function call as an argument and passing the assignment result.
This case requires a separate opcode for two reasons: firstly, it will create the familiar message “Only variables should be passed by reference” if you try to pass something like assignment by reference (if SEND_VAR_EX was used, it would silently allow). Secondly, this opcode deals with the case when you may need to pass the result of a function call by reference, without raising any exceptions. The variant of this opcode SEND_VAR_NO_REF (without _EX) is a specialized variant for the case when we statically know that the link is expected, but do not know if the argument is it.
The SEND_UNPACK and SEND_ARRAY opcodes deal with unpacking arguments and nested calls to call_user_func_arrayrespectively. They both extract elements from the array and put them on the argument stack and differ in various details (for example, unpacking supports Traversables, but call_user_func_array does not). If decompression is used, it may be necessary to increase the stack frame (since the actual number of function arguments is unknown during initialization). In most cases, this increase can occur simply by moving the pointer to the top of the stack. However, if the border of the stack page is crossed, the new page should be allocated, and the entire call frame (including arguments already placed on the stack) should be copied to the new page (we will not be able to process the call frame crossing the page border).
The last opcode - SEND_USER - is used for internal calls to call_user_funcand deals with some of its features.
Although we have not yet discussed the various modes of obtaining data from variables, it's time to introduce the FUNC_ARG mode. Consider a simple call like func ($ a [0] [1] [2]) , for which we do not know at compile time, the argument will be passed by value or by reference. In these cases, the behavior will be very different. If the transfer is by value, and $ a is empty, this can create a bunch of “undefined index” notifications. If the transfer is by reference, we must silently initialize the nested arrays.
The data acquisition mode FUNC_ARG dynamically selects one of two behaviors (R or W), checking the arginfo of the current EX (call) function. For example, func ($ a [0] [1] [2]) the opcode sequence might look something like this:
The PHP virtual machine has four classes of opcodes for retrieving data:
They do exactly what you would expect from them, with the remark that the main variant of FETCH_ * is used only to access variable variables ($$ var) and superglobal variables: normal accesses to variables instead occur through a faster CV mechanism .
These data acquisition opcodes are presented in six versions:
We already learned that _FUNC_ARG chooses between _R and _W depending on how the function argument is passed - by value or by reference. Let's try to create some situations when we expect the appearance of different variants of FETCH_ *:
Unfortunately, the actual retrieval by index occurs only in the case of FETCH_DIM_R. Everything else is processed using special opcodes. Note that ASSIGN_DIM and ASSIGN_ADD use the optional OP_DATA because they need more than two input operands. The reason for using special opcodes like ASSIGN_DIM instead of something like FETCH_DIM_W + ASSIGN is (other than performance) that these operations can be overloaded (for example, in the case of ASSIGN_DIM using an object that implements ArrayAccess :: offsetSet ()) . In order to actually generate different types of samples, we need to increase the level of nesting:
Here we see that while external access uses specialized opcodes, nested indexes will be processed using FETCH with the appropriate fetch mode. These modes differ significantly in whether they generate an “Undefined offset” notification if the index does not exist, and whether they get a value for writing:
The case with UNSET is a little strange, because it will only extract existing offsets for writing and leave undefined without processing. A regular write-fetch initializes undefined offsets instead.
Write fetches return VARs that can contain either a normal zval or an INDIRECT pointer to another zval. Of course, in the first case, any changes applied to zval will not be visible, since the value is available only through the temporary VM variable. Although PHP forbids expressions like [] [0] = 42 , we still need to handle them for cases like call () [0] = 42 . Depending on whether the call () value is returned or the link, this expression may or may not have the observed effect.
A more typical case is when fetch returns an INDIRECT that contains a pointer to a variable memory cell (for example, a specific location in a hash table data array). Unfortunately, such pointers are fragile things and easily become invalid: any simultaneous writing to the array can cause memory redistribution, leaving a “hanging” pointer. Therefore, it is important to prevent user code from running between the point where the INDIRECT value is generated and where it is used.
Consider the following example:
Which generates:
It is noteworthy that this sequence first performs all side effects from left to right and only then makes the necessary selection-record (we call FETCH_DIM_W “deferred opline” here). This ensures that write-fetch and the statement using the result of fetch are executed directly one after another.
Consider another example:
Here we have a small problem: both sides of the assignment must be selected for recording. However, if we select $ arr [0] for writing and then $ arr [1] for writing, the latter may invalidate the former. This problem is solved as follows:
Here $ arr [1] is extracted first for writing, and then turned into a link using MAKE_REF. The result of MAKE_REF is no longer INDIRECT and cannot be canceled, so fetching from $ arr [0] can be done safely.
Exceptions are the root of all evil.
An exception is thrown by writing it to EG (exception) , where EG refers to executor globals. Throwing exceptions from C code does not involve unwinding the stack; instead, interrupts will propagate upward through the return failure codes or through an EG (exception) check . An exception is only processed when control returns to the virtual machine code.
Almost all VM instructions can directly or indirectly result in an exception in some circumstances. For example, an “Undefined Variable” notification may throw an exception if a custom error handler is used. We want to avoid checking EG (exception)after each VM instruction. To do this, we use a small trick:
When an exception is thrown, the current opline of the current execute data is replaced with the dummy opline HANDLE_EXCEPTION (this does not change the op array, but only redirects the pointer). The Opline in which the exception occurred is reserved in EG (opline_before_exception) . This means that when control returns to the main loop of the virtual machine, the HANDLE_EXCEPTION opcode will be called. There is a small problem with this scheme: it requires that: a) the opline stored in execute data actually execute opline at the moment (otherwise, opline_before_exception would be wrong); and b) the virtual machine used the opline from execute data to continue execution (otherwise HANDLE_EXCEPTION will not be called).
Although these requirements may seem trivial, they are not. The reason is that the virtual machine can work with another opline variable that is not synchronized with the opline stored in execute data. Before PHP 7, this only happened in the rarely used GOTO and SWITCH, and in PHP 7 it is actually the default mode of operation: if the compiler supports this, opline is stored in the global register.
Thus, before performing any operation that may throw an exception, the local opline must be written back to execute data (operation SAVE_OPLINE). Similarly, after any potentially dangerous operation, the local opline should be populated from execute data (usually with the CHECK_EXCEPTION operation).
Thus, HANDLE_EXCEPTION is called after an exception has been thrown. But what is he doing? First of all, it determines whether an exception has been thrown inside the try block. To do this, the op array contains the try_catch_elements array, which tracks the opline offsets for the try, catch, and finally blocks:
For now, we will pretend that finally blocks do not exist, since they are a separate problem. Assuming that we are actually inside the try block, the virtual machine needs to clear all the incomplete operations that started before the exception was thrown, and not slip past the end of the try block.
This includes the release of stack frames and associated data for all calls, as well as the release of all live temporary variables. In most cases, temporary data is short-lived to the point that the instruction that uses it immediately follows the one that generates it. However, sometimes their lifetime covers several potentially throwing instructions:
In this case, the variable T0 is active during instructions L1 and L2 and as such should be destroyed if the function throws an exception.
One particular type of time data that tends to have a long lifetime is loop variables. For instance:
Here the loop variable V0 lives from L1 to L3 (usually always covering the whole body of the loop). Life ranges are stored in the op array using the following structure:
Here var is the variable to which the range applies, start is the initial offset of the opline (not including the generation instruction), and end is the end of the offset of the opline (including the usage instruction). Of course, life ranges are only saved if temporary data is not used immediately.
The least significant bits of var are used to store the type of the variable, which can be one of the following:
The difficult question to consider in this context is whether temporary data should be freed if an instruction that generates or uses them throws an exception. Consider a simple code:
If an exception throws an ADD, will T2 be automatically freed, or is this ADD instruction responsible for this? Similarly, if ASSIGN throws an exception, should T2 be released automatically, or should ASSIGN take care of this itself? In the latter case, the answer is obvious: the instruction is always responsible for freeing its operands, even if an exception is thrown.
The case of the result operand is more complicated, because the answer here has changed between PHP 7.1 and 7.2. In PHP 7.1, the instruction was responsible for freeing the result in the event of an exception, and in PHP 7.2 it is automatically released (and the instruction is responsible for ensuring that the result is always populated). The motivation for this change is the way many basic instructions (such as ADD) are implemented. Their usual structure is approximately as follows:
This is problematic because PHP is in a very bad position, supporting not only exceptions and destructors, but throwing exceptions at destructors (this is where compiler developers scream in horror). Thus, step 3 may throw an exception after the result is already populated. To avoid memory leaks in this case, the responsibility for releasing the result operand was transferred from the instruction to the exception handling mechanism.
Once we have completed these cleanup operations, we can continue to execute the catch block. If catch is not (and finally not), we unwind the stack, that is, destroy the current frame in the stack and provide the parent frame with the ability to handle the exception.
In order for you to get a complete picture of how ugly all exception handling is, I will talk about another point related to the throwing exception destructor. This is not applicable in practice, but in fairness we still need to deal with it.
Consider the following code:
Now imagine that Dtor is a Traversable class with an exception throwing destructor. This code will lead to the following sequence of opcodes (indented with the loop body for readability):
It is important to note that return is compiled into a FE_FREE loop variable and RETURN. What happens if FE_FREE throws an exception? Indeed, Dtor has an exception throwing destructor. Usually we say that this statement is inside a try block, so we should call catch. However, at this point the loop variable is already destroyed! Catch throws an exception - and we will try to continue the iteration of the already dead loop variable.
The reason for this problem is that although throwing an exception in FE_FREE is inside the try block, it is a copy of FE_FREE in L11. Logically, this is where the exception actually occurred. This is why the FE_FREE generated by the interrupt is annotated as FREE_ON_RETURN. This instructs the exception handling mechanism to move the source of the exception to the original release statement. Thus, the above code will not trigger a catch block - it will throw an uncaught exception.
The history of PHP with finally blocks is somewhat dysfunctional. They were first introduced in PHP 5.5, but it was a really buggy implementation. PHP 5.6, 7.0 and 7.1 came with significant rework in this area. Each version corrected a number of errors, but could not achieve a completely correct implementation. And now, it looks like PHP 7.1 has finally succeeded (hopefully).
While writing this section, I was surprised to find that from the point of view of the current implementation, finally processing is not really that complicated. Indeed, in many ways, implementation has become easier, not harder. This shows how a lack of understanding of the problem can lead to an overly complex and slow implementation (although, in fairness, part of the complexity of the PHP 5 implementation stems directly from the lack of AST).
The finally blocks are executed whenever the control exits the try block, either in the usual way (for example, using return) or abnormally (throwing exceptions). There are several interesting cases to consider, which I will illustrate before embarking on an implementation. Consider:
What's happening? Finally wins, and the function returns 42.
Consider:
Again, finally wins, and the function returns 42. Finally always wins.
PHP forbids transition from finally blocks. For example, the following is prohibited:
Continue will generate a compilation error here. It’s important to understand that this limitation is purely “cosmetic,” and can be easily circumvented using the well-known delegation management scheme for catch:
The only real limitation that exists is that it is not possible to go to the finally block. For example, gettingo from outside the finally block to a label inside the finally block is forbidden.
With preliminary digressions, we can see how finally works. The implementation uses two opcodes: FAST_CALL and FAST_RET. Roughly speaking, FAST_CALL is intended to go to the finally block, and FAST_RET is intended to exit it. Consider the simplest case:
This code compiles into the following opcode sequence:
FAST_CALL saves its own location in T0 and goes into the finally block in L3. When FAST_RET is reached, it returns to the place stored in T0. In this case, it will be L2, where there is a jump through the finally block. This is the basic case when there is no special control flow (returns or exceptions).
Now consider the case with the exception:
When handling an exception, we should consider the position of the thrown exception relative to the nearest surrounding try / catch / finally block:
In this example, we look at the first three steps: first, throw an exception in try, initiating a jump to catch. Catch also throws an exception, triggering a jump to the finally block with an exception backup in FAST_CALL. The finally block also throws an exception, so the finally exception will throw a catch exception, set as its previous exception.
This code is slightly different from the previous example:
Entrance to all internal finally blocks occurs due to throwing exceptions, and exit - as usual (via FAST_RET). In this case, the above exception handling procedure is resumed starting from the parent try / catch / finally block. This parent try / catch is stored in the FAST_RET opcode (here, “try-catch (0)”).
This essentially covers finally interaction and exceptions. But what about return in finally?
The corresponding opcode sequence is:
The additional DISCARD_EXCEPTION opcode is responsible for refusing to further handle the exception raised in the try block (remember: returning to finally wins). What about return in try?
The return value here is 42, not 43, since it is determined by the string return $ a , and any further modification of $ a should not matter. The result of this code will be:
Two of the opcodes are not available, as they occur immediately after return. They will be deleted during optimization, but I show unoptimized opcodes. There are two interesting points here. First, $ a is copied to T3 using QM_ASSIGN (which is basically the “Copy to temporary variable” instruction). This prevents the subsequent modification of $ a from affecting the return value. Secondly, T3 is also passed to FAST_CALL, which will backup the value to T1. If later return from the try block is discarded (for example, because throw or return appears in finally), this mechanism will be used to free the unused return value.
All of these mechanisms are simple in themselves, but care must be taken when sharing them. Consider the following example, where Dtor is again a Traversable class with an exception throwing destructor:
This code generates the following opcodes:
The sequence for the first return (from internal try) is FAST_CALL L10, FE_FREE V4, FAST_CALL L19, RETURN. This will first call the internal finally block, then release the foreach loop variable, then call the external finally block and finally return the value. The sequence for the second return (from internal finally) is DISCARD_EXCEPTION T5, FE_FREE V4, FAST_CALL L19. First, the exception will be canceled (or here, the return value) of the internal try block, then the foreach loop variable will be freed, and finally the external finally block will be called. Note that in both cases, the order of these instructions is the reverse of the order of the corresponding blocks in the source code.
Generator functions can be paused and resumed, and therefore require special management of the virtual machine stack. Here is a simple generator:
The following opcodes are obtained from this:
Until GENERATOR_CREATE is reached, everything is executed as a normal function on a regular virtual machine stack. GENERATOR_CREATE creates a Generator object , as well as a execute_data structure distributed on the heap (including slots for variables and arguments, as usual), into which execute_data data is copied from the VM stack.
When generator execution resumes, the executor will use execute_data on the heap, but will continue to use the main virtual machine stack to place the call frames. The obvious problem is that, as the previous example shows, the generator may be interrupted during a call. Here, YIELD is executed at the point where the call frame for foo () has already been pushed onto the VM stack.
This relatively unusual case is handled by copying the active call frames to the generator structure when performing yield and restoring them when the generator resumes.
This design has been used since PHP 7.1. Previously, each generator had its own 4 KB VM page, which was loaded into the executor when the generator was restored. This avoided the need to copy call frames, but increased memory consumption.
Very often, conditional transitions immediately follow the comparison instructions. For instance:
Since this pattern is very common, all comparison opcodes (such as IS_EQUAL) implement the smart branch mechanism: they check whether the next instruction is a JMPZ or JMPNZ instruction, and if so, they themselves perform the jump operation.
This mechanism only checks whether the next instruction is JMPZ / JMPNZ, but does not check whether the operand of this instruction is the result of a comparison. This requires special care in cases where the comparison and subsequent transition are not related. For example, the code ($ a == $ b) + ($ d? $ E: $ f) generates:
Note that a NOP is inserted between IS_EQUAL and JMPZ. If this NOP were not there, the result of the comparison IS_EQUAL would be used for the transition, rather than the JMPZ operand.
Since the opcode arrays are open for sharing to several processes (without locks), they are strictly unchanged. However, runtime values can be cached in a separate runtime cache, which is usually an array of pointers. Literals can have a linked runtime cache entry (or more than one).
There are two types of runtime cache entries. The first is regular cache entries (such as those used in INIT_FCALL). After INIT_FCALL once searches for the called function (by its name), the function pointer will be cached in the corresponding slot of the cache cache.
The second type is polymorphic cache entries, which are two consecutive cache slots, where the first stores the beginning of the class, and the second - the offset of the class member. They are used for operations of type FETCH_OBJ_R, where the property offset in the property table of a particular class is cached. If the next call happens to a property of the same class (which is very likely), the cached value will be used. Otherwise, a more expensive search operation is performed, and the result is cached for the new class.
Prior to PHP 7.0, execution timeouts were used that were processed by switching to the shutdown sequence directly from the signal handler. As you can imagine, this caused all kinds of trouble. Starting with PHP 7.0, timeout processing is delayed until control is returned to the virtual machine. If the control does not return within a certain (set) period, then the process ends. Starting with PHP 7.1, pcntl signal handlers use the same mechanism as runtime timeouts.
When the signal is in the standby state, an interrupt flag is set, which is checked by the virtual machine at certain points. The check is not performed with each new instruction, but only with transitions and calls. Thus, the interrupt will not be processed immediately upon returning control to the virtual machine, but at the end of the current section of the linear control flow.
If you look at the virtual machine definitions file , you will find that the opcode handlers are defined as follows:
Here 1 is the opcode number, ZEND_ADD is its name, and the other two arguments indicate which types of operands the instruction accepts. The generated virtual machine code (generated using zend_vm_gen.php) will contain specialized handlers for each of the possible combinations of operand types. They will have names such as ZEND_ADD_SPEC_CONST_CONST_HANDLER.
Specialized handlers are generated by replacing certain macros in the body of the handler. The obvious are OP1_TYPE and OP2_TYPE, but operations such as GET_OP1_ZVAL_PTR () and FREE_OP1 () are also specialized.
The handler for ADD indicated that it accepts operands CONST | TMPVAR | CV. Here TMPVAR means that the opcode accepts both TMP and VAR, but requires that they not specialize separately. Remember that for most purposes, the only difference between TMP and VAR is that the latter can contain links. For an operation code such as ADD (where links are on the slow path anyway), having a separate specialization does not make sense. Some other opcodes that make this distinction will use TMP | VAR in the operand list.
In addition to specializing in operands, handlers can also specialize in other factors, such as the use of the return value.
ASSIGN_DIM specializes in the operand type of the following OP_DATA opcode:
Based on this signature, 2 * 4 * 4 = 32 different ASSIGN_DIM variants will be generated .
The second operand specification also contains an entry for NEXT . This is not specialization-specific, instead, it defines what the value of the UNUSED operand means in this context: it means that it is an add operation ( $ arr [] ).
Another example:
Here, the first UNUSED operand implies access on $ this . This is a general convention for opcodes associated with objects (e.g. FETCH_OBJ_R UNUSED , ' prop ' matches $ this-> prop ). The second UNUSED operand again implies an add operation. The third argument here defines the value of the extended_value operand: it contains a flag that distinguishes between $ a + = 1 , $ a [$ b] + = 1 and $ a-> b + = 1 . And finally, SPEC (DIM_OBJ)indicates that a specialized handler should be created for each of them. In this case, the number of generated handlers is nontrivial, since the VM generator knows that some combinations are impossible. For example, UNUSED op1 is applicable only for the case of OBJ, etc.
Finally, the virtual machine generator supports an additional, more complex specialization mechanism. At the end of the definition file, you will find several handlers for this form:
These handlers specialize not only in the type of operand, but also based on the possible types that the operand can take at run time. The mechanism by which possible types of operands are determined is part of the cache optimization infrastructure and is completely beyond the scope of this article. However, assuming such information is available, it should be clear that this is a handler for adding the form int + int -> int . In addition, the SPEC annotation tells the specialist (the specializer component) that options for two constant operands should not be generated and that the operation is commutative, so if we already have the CONST + TMPVARCV specialization, we do not need to generate TMPVARCV + CONST.
Many opcode handlers are implemented using the fast path / slow path separation, where several common cases are handled first, and then return to the general implementation.
It's time to take a look at some real code, so I just insert the full SL implementation here (left shift):
Implementation begins by fetching operands using GET_OPn_ZVAL_PTR_UNDEF in BP_VAR_R mode. Here, the UNDEF part means that in the case of CV, data uncertainty is not checked, instead, you simply return the UNDEF value as is. Once we have the operands, we check that both of them are integers, and the shift width is in the range. In this case, the result can be directly calculated, and we move on to the next opcode. It is noteworthy that the type check here does not care if the operands are UNDEF, so using GET_OPn_ZVAL_PTR_UNDEF is justified.
If the operands do not satisfy the fast path, we return to the general implementation, which begins with SAVE_OPLINE (). This is our signal for "potentially throwing exception operations." Before moving on, the case of undefined variables is handled. GET_OPn_UNDEF_CV in this case will give a notification about an undefined variable and return a NULL value.
Then the standard shift_left_function is called, and its result is written to EX_VAR (opline-> result.var) . Finally, the input operands are freed up (if necessary), and we move on to the next opcode with an exception check (which means that the opline is reloaded before moving on).
Thus, the quick way here avoids two checks for undefined variables, calling a standard function, freeing operands, and saving and reloading opline to handle exceptions. Similarly, most performance-sensitive opcodes are computed.
As you can see from the previous code, the implementation of a virtual machine allows you to freely use macros. Some of them are regular C language macros, while others are processed during virtual machine generation. In particular, there are several macros for fetching and freeing operands of instructions:
As you can see, there are several variations. The BP_VAR_ * arguments define the data acquisition mode and support the same modes as the FETCH_ * instructions (except for FUNC_ARG).
GET_OPn_ZVAL_PTR () is the main instruction for obtaining an operand. It will issue a notification of an undefined CV and will not dereference the operand. GET_OPn_ZVAL_PTR_UNDEF () , as we already learned, is an option that does not check for undefined CVs. GET_OPn_ZVAL_PTR_DEREF () enables DEREF for zval. This is part of the specialized GET operation, since dereferencing is only necessary for CV and VAR, but not for CONST and TMP. Since this macro must distinguish between TMP and VAR, it can only be used with the TMP | VAR specialization (but not the TMPVAR ).
Variants GET_OPn_OBJ_ZVAL_PTR * () additionally handle the case with a UNUSED operand. As mentioned above, under the $ this convention, access uses a UNUSED operand, so the GET_OPn_OBJ_ZVAL_PTR * () macros will return an EX (This) reference for UNUSED.
Finally, there are several PTR_PTR options . The naming here is a relic of the days of PHP 5, where zval double INDIRECT pointers were actually used. These macros are used in write operations and as such only support CV and VAR types (everything else returns NULL). They differ from regular PTR samples in that they are “de-INDIRECT-cosiness” VAR operands.
Macros FREE_OP * ()then used to free selected operands. To work, they need the definition of the variable zend_free_op free_opN , into which the GET operation saves this value for release. The basic operation FREE_OPn () will release TMP and VAR, but not CV and CONST. FREE_OPn_IF_VAR () does exactly what it says: it releases the operand only if it is a VAR.
The FREE_OP * _VAR_PTR () option is used in conjunction with PTR_PTR selections . It will only release VAR operands and only if they are not INDIRECT.
Options FREE_UNFETCHED_OP * ()used when an operand needs to be freed before it is received using GET. This usually happens if an exception is thrown before receiving the operand.
In addition to these specialized macros, there are also many more standard macros. The VM defines three macros that control what happens after the opcode handler starts:
CONTINUE will continue to execute opcodes as usual, while ENTER and LEAVE are used to enter / exit a nested function call. The specifics of their work depends on how the virtual machine is compiled (for example, whether global registers are used, and if so, which ones). In a broad sense, they will synchronize some state from global variables before continuing. RETURN is used to actually exit the main VM loop.
ZEND_VM_CONTINUE () expects the opline to be updated in advance. Of course, there are more macros related to this:
The table shows whether the macro contains implicit ZEND_VM_CONTINUE (), whether it will check for VM exceptions and interrupts.
Next to them is also SAVE_OPLINE () , LOAD_OPLINE () and HANDLE_EXCEPTION (). As mentioned in the section on exception handling, SAVE_OPLINE () is used before the first potentially throwing operation in the opcode handler. If necessary, it writes back the opline used by the VM (may be in the global register) to execute data. LOAD_OPLINE () is a reverse operation, but it is currently underused because it was actually added to ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION () and ZEND_VM_JMP (). HANDLE_EXCEPTION () is used to return from the opcode handler after you already know that an exception has been thrown. It performs a combination of LOAD_OPLINE and CONTINUE.
Of course, there are more macros (there are always more macros ...), but this article should cover the most important ones.
This article provides an overview of the Zend virtual machine for PHP 7. This is not an exhaustive description, but I will try to cover most of the important parts, as well as some details.
The description is based on PHP version 7.2 (currently under development), but almost everything is true for PHP 7.0 / 7.1. However, the differences from the virtual machines of the PHP 5.x series are significant, and as a rule I did not draw parallels with them.
Most of the article deals with things at the level of instruction listings, and only a few sections at the end relate to the level of actual implementation of the virtual machine in C. Nevertheless, I want to provide links to the main files that make up the virtual machine:
- zend_vm_def.h : virtual machine definition file;
- zend_vm_execute.h : generated virtual machine;
- zend_vm_gen.php : generating script;
- zend_execute.c : most of the directly serving code.
Opcodes
Initially, there was an opcode. Saying “opcode”, we refer to the entire virtual machine instruction (including operands), but it can also indicate only the “actual” operation code, which is a small integer that determines the type of instruction. The intended meaning should be clear from the context. In the source code, the entire instructions are usually called oplines.
A separate instruction corresponds to the following zend_op structure :
struct _zend_op {
const void *handler;
znode_op op1;
znode_op op2;
znode_op result;
uint32_t extended_value;
uint32_t lineno;
zend_uchar opcode;
zend_uchar op1_type;
zend_uchar op2_type;
zend_uchar result_type;
};
Thus, opcodes are essentially instructions in the format of a “three-address code”. There is an opcode defining the type of instruction, there are two input operands op1 and op2 and one output operand result .
Not all instructions use all operands. The ADD statement (representing the + operator ) will use all three. The BOOL_NOT statement (representing the operator ! ) Uses only op1 and result. The ECHO instruction uses only op1. Some instructions may or may not use the operand. For example, DO_FCALLmay or may not have a result operand (depending on whether the return value of the function call is used). Some instructions require more than two input operands, in which case they will simply use the second auxiliary instruction ( OP_DATA ) for additional operands .
Next to these three standard operands, there is an additional numeric field extended_value , which can be used to store additional instruction modifiers. For example, for CAST, it may contain the target type to cast to.
Each operand has a type stored in op1_type , op2_type and result_typerespectively. Possible types: IS_UNUSED , IS_CONST , IS_TMPVAR , IS_VAR and IS_CV .
The last three types are designed for variable operands (with three different types of virtual machine variables), IS_CONST stands for constant operand ( 5 , or “string” , or even [1, 2, 3] ), while IS_UNUSED stands for operand, which is either not actually used or used as a 32-bit numerical value (the so-called direct operand). For example, the jump instruction will store the jump address in the UNUSED operand .
Getting opcode dumps
In the future, I will often demonstrate the fragments of the opcode that PHP generates. There are currently three ways to obtain such opcode dumps:
# Opcache, since PHP 7.1
php -d opcache.opt_debug_level=0x10000 test.php
# phpdbg, since PHP 5.6
phpdbg -p* test.php
# vld, third-party extension
php -d vld.active=1 test.phpOf these, opcache provides the best result. The listings used in this article are based on opcache dumps, with minor syntax adjustments. The magic number 0x10000 is an abbreviation "before optimization", so we see the opcodes as they were created by the PHP compiler. 0x200000 will give you optimized opcodes. Opcache can also generate a lot more information. For example, 0x40000 will generate CFG, and 0x200000 will produce SSA . But let's not get ahead of events: for our purposes, the usual old linearized opcode dumps are enough.
Types of Variables
Probably one of the most important points to consider when working with the PHP virtual machine is the use of three different types of variables. In PHP 5, TMPVAR, VAR, and CV had very different views on the virtual machine stack, and the methods for accessing them were also very different. In PHP 7 they became very similar because they use the same storage mechanism. However, there are important differences in the meanings that they may contain and in their semantics.
CV is short for “compiled variable”. It refers to the "real" PHP variable. If the function uses the $ a variable , then there will be a corresponding CV for it.
CV variables can be of type UNDEFto denote undefined variables. If UNDEF CV is used in the instruction, in most cases this will produce the well-known notification “undefined variable” (undefined variable). At the function input, all CVs that are not arguments are initialized as UNDEF.
CV variables are not destroyed by instructions. For example, the ADD statement $ a, $ b will not destroy the values stored in the variables $ a and $ b . Instead, all CV variables are destroyed simultaneously when leaving the scope. This also implies that all CV variables contain valid values for the duration of the function.
The TMPVAR and VAR variables, in turn, are temporary virtual machine variables. They are usually introduced as the operand of the result of some operation. For example, the code $ a = $ b + $ c + $ d will result in an opcode similar to the following:
T0 = ADD $b, $c
T1 = ADD T0, $d
ASSIGN $a, T1TMP / VAR variables are always defined before use and as such cannot contain a UNDEF value. Unlike CVs, these types of variables are destroyed by the instructions in which they are used. In the above example, the second ADD will destroy the value of the operand T0, and after this point T0 should no longer be used. Similarly, ASSIGN will destroy the value of T1, making the variable T1 invalid.
It follows that TMP / VAR variables are usually very short-lived. In most cases, they live only within one instruction. Outside of this short range, the values in them are garbage.
So what are the differences between TMP and VAR variables? There are few of them. The difference was inherited from PHP 5, where TMPs were located on the virtual machine stack, and VARs were on the heap. In PHP 7, all variables are pushed onto the stack. Thus, at present, the main difference between TMP and VAR is that only the latter are allowed to contain references (this allows us to exclude dereferencing (DEREF) of TMP variables). In addition, VARs can contain two types of special values, namely class entries and INDIRECT values. The latter are used to process nontrivial assignments.
This table shows the main differences between the variables:
| Undef | Ref | INDIRECT | Consumed? | Named? | |
|---|---|---|---|---|---|
| CV | yes | yes | no | no | yes |
| Tmpvar | no | no | no | yes | no |
| Var | no | yes | yes | yes | no |
Op arrays
All PHP functions are represented as structures having a common zend_function header . The concept of “function” is interpreted somewhat more broadly and includes everything from “real” functions and methods to standalone pseudo-main code and eval code.
Custom functions use the zend_op_array structure . She has more than 30 fields, so I'll start with her smaller version:
struct _zend_ {
/* Common zend_function header here */
/* ... */
uint32_t last;
zend_op *opcodes;
int last_var;
uint32_t T;
zend_string **vars;
/* ... */
int last_literal;
zval *literals;
/* ... */
};
The most important part here is, of course, opcodes , which are an array of opcodes (instructions). last - the number of opcodes in this array. Note that the terminology is somewhat confusing, since last sounds like it should be the index of the last opcode, while in fact it is the number of opcodes (one more than the last index). The same applies to all other last_ * values in the op_array structure.
last_var is the number of CVs, and T is the number of TMP and VAR (in most cases, we do not distinguish between them). vars is an array of names for CV.
literalsIs an array of literals found in code, something that CONST operands reference . Depending on the ABI, each CONST operand will either contain a pointer to an element of this literal table, or store an offset relative to its beginning.
There is something else in this structure, but it can be postponed.
Stack frame diagram
With the exception of executor globals (EG), all execution status is stored on the virtual machine stack. The VM stack is distributed on pages of 256KiB, and the individual pages are linked through a linked list.
Each time a function is called, a new stack frame is allocated in the virtual machine stack, which has the following scheme:
+----------------------------------------+
| zend_execute_data |
+----------------------------------------+
| VAR[0] = ARG[1] | arguments
| ... |
| VAR[num_args-1] = ARG[N] |
| VAR[num_args] = CV[num_args] | remaining CVs
| ... |
| VAR[last_var-1] = CV[last_var-1] |
| VAR[last_var] = TMP[0] | TMP/VARs
| ... |
| VAR[last_var+T-1] = TMP[T] |
| ARG[N+1] (extra_args) | extra arguments
| ... |
+----------------------------------------+
The frame begins with the zend_execute_data structure , followed by an array of variable slots. The slots are all the same (simple zval), but they are used for different purposes. The first last_var slots are CVs, of which the first num_args contains function arguments. CV slots are followed by T- slots for TMP / VAR. Finally, sometimes there may be additional arguments stored at the end of the frame. They are used for func_get_args () .
The CV and TMP / VAR operands in the instructions are encoded as offsets relative to the beginning of the stack frame, so fetching a specific variable is a simple read from the cell at execute_data plus the specified offset.
The data at the beginning of the frame is defined as follows:
struct _zend_execute_data {
const zend_op *opline;
zend_execute_data *call;
zval *return_value;
zend_function *func;
zval This; /* this + call_info + num_args */
zend_class_entry *called_scope;
zend_execute_data *prev_execute_data;
zend_array *symbol_table;
void **run_time_cache; /* cache op_array->run_time_cache */
zval *literals; /* cache op_array->literals */
};
Most importantly, this structure contains opline , which is the currently executing instruction, and func , which is the currently executing function. Moreover:
- return_value pointer to zval in which the return value will be stored;
- This is a $ this object, but also the number of function arguments and a pair of call metadata flags in some unused zval spaces;
- called_scope the scope that static :: refers to in PHP code ;
- prev_execute_data points to the previous frame of the stack, to which execution will return after the completion of this function;
- symbol_table is usually an unused symbol table, used if some crazy person actually uses variable variables or similar functions;
- run_time_cache caches op_array-> run_time_cache to avoid indirect addressing when accessing this structure (which will be discussed below);
- literals caches the op array literal table for the same reason.
Function calls
I missed one field in the execute_data structure, namely call , since it requires some additional explanation of how function calls work.
All calls use the same sequence of instructions. var_dump ($ a, $ b) in global scope compiles to:
INIT_FCALL (2 args) "var_dump"
SEND_VAR $a
SEND_VAR $b
V0 = DO_ICALL # или просто DO_ICALL если retval не используетсяThere are eight different types of INIT statements (depending on which call it is). INIT_FCALL is used to call functions (not class methods) that we recognize at compile time. Similarly, there are ten different SEND opcodes (depending on the type of arguments and function). There is only a small number of the four DO_CALL opcodes, where ICALL is used to call internal functions.
Although specific instructions may vary, the structure is always the same: INIT, SEND, DO. The main problem that the sequence of calls should handle is the nested function calls that compile something like this:
# var_dump(foo($a), bar($b))
INIT_FCALL (2 args) "var_dump"
INIT_FCALL (1 arg) "foo"
SEND_VAR $a
V0 = DO_UCALL
SEND_VAR V0
INIT_FCALL (1 arg) "bar"
SEND_VAR $b
V1 = DO_UCALL
SEND_VAR V1
V2 = DO_ICALL
I formatted a sequence of opcodes to visualize which instructions correspond to which call.
The INIT opcode pushes a call frame onto the stack that contains enough space for all the variables and arguments of the function that we know about (if unpacking the arguments is involved, we can end up with more arguments). This call frame is initialized by the function being called, $ this and called_scope (in this case, both of them will be NULL, since we are calling the function).
A pointer to a new frame is stored in execute_data-> call , where execute_data is the frame of the calling function. In the future, we will denote this as EX (call) . It is noteworthy thatThe prev_execute_data of the new frame is set to the old EX (call) value . For example, INIT_FCALL to call foo will write the var_dump stack frame to prev_execute_data . Thus, prev_execute_data in this case forms a linked list of “unfinished” calls, while usually it provides a backtrace chain .
Then the SEND opcodes proceed to pass arguments to the EX (call) variable slots . At this point, all arguments are consecutive and can flow from the argument section to other CVs or TMPs. This will be fixed later.
Finally, DO_FCALL makes the actual call. What was EX (call), becomes the current function, and prev_execute_data changes to the calling function. In addition, the calling procedure depends on what function it is. Internal functions only need to call the handler function, while user-defined functions must complete the initialization of the stack frame.
This initialization involves tidying up the argument stack. PHP allows you to pass functions more arguments than it expects (and func_get_args relies on this). However, only the actual arguments have the corresponding CV. Any other arguments will be written to memory reserved for other CVs and TMPs. Essentially, these arguments will be placed after TMP, as a result of which the arguments will be split into two separate fragments.
It must be clarified that calls to user-defined functions do not involve recursion at the virtual machine level. They only mean switching from one execute_data to another, but the VM continues to run in a linear loop. Recursive virtual machine calls only occur if internal functions call user callbacks (for example, via array_map). For this reason, infinite recursion in PHP is usually interrupted due to lack of memory or an OOM error, but you can cause the stack to overflow with recursion through callbacks or magic methods.
Passing arguments
PHP uses a large number of opcodes to pass arguments, the differences between which can be confusing due to their bad name.
SEND_VAL and SEND_VAR are the simplest options that pass arguments by value when the value is known at compile time. SEND_VAL is used for CONST and TMP operands, and SEND_VAR is used for VAR and CV.
SEND_REF, by contrast, is used for arguments that are known at compile time as references. Since only pointers can be passed by reference, this opcode accepts only VAR and CV.
SEND_VAL_EX and SEND_VAR_EX are SEND_VAL / SEND_VAR options for cases where we cannot statically determine whether an argument is passed by value or by reference. These opcodes check the type of the argument based on arginfo and behave accordingly. In most cases, the actual structure is not the arginfo structure, but a rather compact bit vector directly in the function structure.
And there is also SEND_VAR_NO_REF_EX. Do not try to understand anything from his name - this is an outright lie. This opcode is used when passing something that is not really a variable, but returns a VAR as a statically unknown argument. Two specific examples in which it is used are passing the result of a function call as an argument and passing the assignment result.
This case requires a separate opcode for two reasons: firstly, it will create the familiar message “Only variables should be passed by reference” if you try to pass something like assignment by reference (if SEND_VAR_EX was used, it would silently allow). Secondly, this opcode deals with the case when you may need to pass the result of a function call by reference, without raising any exceptions. The variant of this opcode SEND_VAR_NO_REF (without _EX) is a specialized variant for the case when we statically know that the link is expected, but do not know if the argument is it.
The SEND_UNPACK and SEND_ARRAY opcodes deal with unpacking arguments and nested calls to call_user_func_arrayrespectively. They both extract elements from the array and put them on the argument stack and differ in various details (for example, unpacking supports Traversables, but call_user_func_array does not). If decompression is used, it may be necessary to increase the stack frame (since the actual number of function arguments is unknown during initialization). In most cases, this increase can occur simply by moving the pointer to the top of the stack. However, if the border of the stack page is crossed, the new page should be allocated, and the entire call frame (including arguments already placed on the stack) should be copied to the new page (we will not be able to process the call frame crossing the page border).
The last opcode - SEND_USER - is used for internal calls to call_user_funcand deals with some of its features.
Although we have not yet discussed the various modes of obtaining data from variables, it's time to introduce the FUNC_ARG mode. Consider a simple call like func ($ a [0] [1] [2]) , for which we do not know at compile time, the argument will be passed by value or by reference. In these cases, the behavior will be very different. If the transfer is by value, and $ a is empty, this can create a bunch of “undefined index” notifications. If the transfer is by reference, we must silently initialize the nested arrays.
The data acquisition mode FUNC_ARG dynamically selects one of two behaviors (R or W), checking the arginfo of the current EX (call) function. For example, func ($ a [0] [1] [2]) the opcode sequence might look something like this:
INIT_FCALL_BY_NAME "func"
V0 = FETCH_DIM_FUNC_ARG (arg 1) $a, 0
V1 = FETCH_DIM_FUNC_ARG (arg 1) V0, 1
V2 = FETCH_DIM_FUNC_ARG (arg 1) V1, 2
SEND_VAR_EX V2
DO_FCALL
Fetch modes
The PHP virtual machine has four classes of opcodes for retrieving data:
FETCH_ * // $ _GET, $$ var FETCH_DIM_ * // $ arr [0] FETCH_OBJ_ * // $ obj-> prop FETCH_STATIC_PROP_ * // A :: $ prop
They do exactly what you would expect from them, with the remark that the main variant of FETCH_ * is used only to access variable variables ($$ var) and superglobal variables: normal accesses to variables instead occur through a faster CV mechanism .
These data acquisition opcodes are presented in six versions:
_R _RW _W _IS _UNSET _FUNC_ARG
We already learned that _FUNC_ARG chooses between _R and _W depending on how the function argument is passed - by value or by reference. Let's try to create some situations when we expect the appearance of different variants of FETCH_ *:
// $arr[0];
V2 = FETCH_DIM_R $arr int(0)
FREE V2
// $arr[0] = $val;
ASSIGN_DIM $arr int(0)
OP_DATA $val
// $arr[0] += 1;
ASSIGN_ADD (dim) $arr int(0)
OP_DATA int(1)
// isset($arr[0]);
T5 = ISSET_ISEMPTY_DIM_OBJ (isset) $arr int(0)
FREE T5
// unset($arr[0]);
UNSET_DIM $arr int(0)
Unfortunately, the actual retrieval by index occurs only in the case of FETCH_DIM_R. Everything else is processed using special opcodes. Note that ASSIGN_DIM and ASSIGN_ADD use the optional OP_DATA because they need more than two input operands. The reason for using special opcodes like ASSIGN_DIM instead of something like FETCH_DIM_W + ASSIGN is (other than performance) that these operations can be overloaded (for example, in the case of ASSIGN_DIM using an object that implements ArrayAccess :: offsetSet ()) . In order to actually generate different types of samples, we need to increase the level of nesting:
// $arr[0][1];
V2 = FETCH_DIM_R $arr int(0)
V3 = FETCH_DIM_R V2 int(1)
FREE V3
// $arr[0][1] = $val;
V4 = FETCH_DIM_W $arr int(0)
ASSIGN_DIM V4 int(1)
OP_DATA $val
// $arr[0][1] += 1;
V6 = FETCH_DIM_RW $arr int(0)
ASSIGN_ADD (dim) V6 int(1)
OP_DATA int(1)
// isset($arr[0][1]);
V8 = FETCH_DIM_IS $arr int(0)
T9 = ISSET_ISEMPTY_DIM_OBJ (isset) V8 int(1)
FREE T9
// unset($arr[0][1]);
V10 = FETCH_DIM_UNSET $arr int(0)
UNSET_DIM V10 int(1)
Here we see that while external access uses specialized opcodes, nested indexes will be processed using FETCH with the appropriate fetch mode. These modes differ significantly in whether they generate an “Undefined offset” notification if the index does not exist, and whether they get a value for writing:
| Notice? | Write? | |
|---|---|---|
| R | yes | no |
| W | no | yes |
| Rw | yes | yes |
| IS | no | no |
| UNSET | no | yes-ish |
The case with UNSET is a little strange, because it will only extract existing offsets for writing and leave undefined without processing. A regular write-fetch initializes undefined offsets instead.
Data Recording and Memory Security
Write fetches return VARs that can contain either a normal zval or an INDIRECT pointer to another zval. Of course, in the first case, any changes applied to zval will not be visible, since the value is available only through the temporary VM variable. Although PHP forbids expressions like [] [0] = 42 , we still need to handle them for cases like call () [0] = 42 . Depending on whether the call () value is returned or the link, this expression may or may not have the observed effect.
A more typical case is when fetch returns an INDIRECT that contains a pointer to a variable memory cell (for example, a specific location in a hash table data array). Unfortunately, such pointers are fragile things and easily become invalid: any simultaneous writing to the array can cause memory redistribution, leaving a “hanging” pointer. Therefore, it is important to prevent user code from running between the point where the INDIRECT value is generated and where it is used.
Consider the following example:
$arr[a()][b()] = c();
Which generates:
INIT_FCALL_BY_NAME (0 args) "a"
V1 = DO_FCALL_BY_NAME
INIT_FCALL_BY_NAME (0 args) "b"
V3 = DO_FCALL_BY_NAME
INIT_FCALL_BY_NAME (0 args) "c"
V5 = DO_FCALL_BY_NAME
V2 = FETCH_DIM_W $arr V1
ASSIGN_DIM V2 V3
OP_DATA V5
It is noteworthy that this sequence first performs all side effects from left to right and only then makes the necessary selection-record (we call FETCH_DIM_W “deferred opline” here). This ensures that write-fetch and the statement using the result of fetch are executed directly one after another.
Consider another example:
$arr[0] =& $arr[1];
Here we have a small problem: both sides of the assignment must be selected for recording. However, if we select $ arr [0] for writing and then $ arr [1] for writing, the latter may invalidate the former. This problem is solved as follows:
V2 = FETCH_DIM_W $arr 1
V3 = MAKE_REF V2
V1 = FETCH_DIM_W $arr 0
ASSIGN_REF V1 V3
Here $ arr [1] is extracted first for writing, and then turned into a link using MAKE_REF. The result of MAKE_REF is no longer INDIRECT and cannot be canceled, so fetching from $ arr [0] can be done safely.
Exception Handling
Exceptions are the root of all evil.
An exception is thrown by writing it to EG (exception) , where EG refers to executor globals. Throwing exceptions from C code does not involve unwinding the stack; instead, interrupts will propagate upward through the return failure codes or through an EG (exception) check . An exception is only processed when control returns to the virtual machine code.
Almost all VM instructions can directly or indirectly result in an exception in some circumstances. For example, an “Undefined Variable” notification may throw an exception if a custom error handler is used. We want to avoid checking EG (exception)after each VM instruction. To do this, we use a small trick:
When an exception is thrown, the current opline of the current execute data is replaced with the dummy opline HANDLE_EXCEPTION (this does not change the op array, but only redirects the pointer). The Opline in which the exception occurred is reserved in EG (opline_before_exception) . This means that when control returns to the main loop of the virtual machine, the HANDLE_EXCEPTION opcode will be called. There is a small problem with this scheme: it requires that: a) the opline stored in execute data actually execute opline at the moment (otherwise, opline_before_exception would be wrong); and b) the virtual machine used the opline from execute data to continue execution (otherwise HANDLE_EXCEPTION will not be called).
Although these requirements may seem trivial, they are not. The reason is that the virtual machine can work with another opline variable that is not synchronized with the opline stored in execute data. Before PHP 7, this only happened in the rarely used GOTO and SWITCH, and in PHP 7 it is actually the default mode of operation: if the compiler supports this, opline is stored in the global register.
Thus, before performing any operation that may throw an exception, the local opline must be written back to execute data (operation SAVE_OPLINE). Similarly, after any potentially dangerous operation, the local opline should be populated from execute data (usually with the CHECK_EXCEPTION operation).
Thus, HANDLE_EXCEPTION is called after an exception has been thrown. But what is he doing? First of all, it determines whether an exception has been thrown inside the try block. To do this, the op array contains the try_catch_elements array, which tracks the opline offsets for the try, catch, and finally blocks:
typedef struct _zend_try_catch_element {
uint32_t try_op;
uint32_t catch_op; /* ketchup! */
uint32_t finally_op;
uint32_t finally_end;
} zend_try_catch_element;
For now, we will pretend that finally blocks do not exist, since they are a separate problem. Assuming that we are actually inside the try block, the virtual machine needs to clear all the incomplete operations that started before the exception was thrown, and not slip past the end of the try block.
This includes the release of stack frames and associated data for all calls, as well as the release of all live temporary variables. In most cases, temporary data is short-lived to the point that the instruction that uses it immediately follows the one that generates it. However, sometimes their lifetime covers several potentially throwing instructions:
# (array)[] + throwing()
L0: T0 = CAST (array) []
L1: INIT_FCALL (0 args) "throwing"
L2: V1 = DO_FCALL
L3: T2 = ADD T0, V1
In this case, the variable T0 is active during instructions L1 and L2 and as such should be destroyed if the function throws an exception.
One particular type of time data that tends to have a long lifetime is loop variables. For instance:
# foreach ($array as $value) throw $ex;
L0: V0 = FE_RESET_R $array, ->L4
L1: FE_FETCH_R V0, $value, ->L4
L2: THROW $ex
L3: JMP ->L1
L4: FE_FREE V0
Here the loop variable V0 lives from L1 to L3 (usually always covering the whole body of the loop). Life ranges are stored in the op array using the following structure:
typedef struct _zend_live_range {
uint32_t var; /* low bits are used for variable type (ZEND_LIVE_* macros) */
uint32_t start;
uint32_t end;
} zend_live_range;
Here var is the variable to which the range applies, start is the initial offset of the opline (not including the generation instruction), and end is the end of the offset of the opline (including the usage instruction). Of course, life ranges are only saved if temporary data is not used immediately.
The least significant bits of var are used to store the type of the variable, which can be one of the following:
- ZEND_LIVE_TMPVAR: this is a “normal” variable. It contains the usual zval value. Releasing this variable behaves like the FREE opcode;
- ZEND_LIVE_LOOP: this is a foreach loop variable that contains more than a simple zval. It corresponds to the FE_FREE opcode;
- ZEND_LIVE_SILENCE: used to implement the error suppression operator. The old error notification level is retained and later restored. If an exception is thrown, we obviously also want to restore it. Conforms to END_SILENCE;
- ZEND_LIVE_ROPE: used to concatenate strings, in which case an array of fixed-size zend_string * pointers living on the stack is temporary . In this case, all lines that have already been filled should be freed. Corresponds to approximately END_ROPE.
The difficult question to consider in this context is whether temporary data should be freed if an instruction that generates or uses them throws an exception. Consider a simple code:
T2 = ADD T0, T1
ASSIGN $v, T2
If an exception throws an ADD, will T2 be automatically freed, or is this ADD instruction responsible for this? Similarly, if ASSIGN throws an exception, should T2 be released automatically, or should ASSIGN take care of this itself? In the latter case, the answer is obvious: the instruction is always responsible for freeing its operands, even if an exception is thrown.
The case of the result operand is more complicated, because the answer here has changed between PHP 7.1 and 7.2. In PHP 7.1, the instruction was responsible for freeing the result in the event of an exception, and in PHP 7.2 it is automatically released (and the instruction is responsible for ensuring that the result is always populated). The motivation for this change is the way many basic instructions (such as ADD) are implemented. Their usual structure is approximately as follows:
- Чтение входных операндов.
- Выполнение операции, запись результата.
- Освобождение входных операндов (при необходимости).
This is problematic because PHP is in a very bad position, supporting not only exceptions and destructors, but throwing exceptions at destructors (this is where compiler developers scream in horror). Thus, step 3 may throw an exception after the result is already populated. To avoid memory leaks in this case, the responsibility for releasing the result operand was transferred from the instruction to the exception handling mechanism.
Once we have completed these cleanup operations, we can continue to execute the catch block. If catch is not (and finally not), we unwind the stack, that is, destroy the current frame in the stack and provide the parent frame with the ability to handle the exception.
In order for you to get a complete picture of how ugly all exception handling is, I will talk about another point related to the throwing exception destructor. This is not applicable in practice, but in fairness we still need to deal with it.
Consider the following code:
foreach (new Dtor as $value) {
try {
echo "Return";
return;
} catch (Exception $e) {
echo "Catch";
}
}
Now imagine that Dtor is a Traversable class with an exception throwing destructor. This code will lead to the following sequence of opcodes (indented with the loop body for readability):
L0: V0 = NEW 'Dtor', ->L2
L1: DO_FCALL
L2: V2 = FE_RESET_R V0, ->L11
L3: FE_FETCH_R V2, $value
L4: ECHO 'Return'
L5: FE_FREE (free on return) V2 # <- return
L6: RETURN null # <- return
L7: JMP ->L10
L8: CATCH 'Exception' $e
L9: ECHO 'Catch'
L10: JMP ->L3
L11: FE_FREE V2 # <- the duplicated instr
It is important to note that return is compiled into a FE_FREE loop variable and RETURN. What happens if FE_FREE throws an exception? Indeed, Dtor has an exception throwing destructor. Usually we say that this statement is inside a try block, so we should call catch. However, at this point the loop variable is already destroyed! Catch throws an exception - and we will try to continue the iteration of the already dead loop variable.
The reason for this problem is that although throwing an exception in FE_FREE is inside the try block, it is a copy of FE_FREE in L11. Logically, this is where the exception actually occurred. This is why the FE_FREE generated by the interrupt is annotated as FREE_ON_RETURN. This instructs the exception handling mechanism to move the source of the exception to the original release statement. Thus, the above code will not trigger a catch block - it will throw an uncaught exception.
Finally processing
The history of PHP with finally blocks is somewhat dysfunctional. They were first introduced in PHP 5.5, but it was a really buggy implementation. PHP 5.6, 7.0 and 7.1 came with significant rework in this area. Each version corrected a number of errors, but could not achieve a completely correct implementation. And now, it looks like PHP 7.1 has finally succeeded (hopefully).
While writing this section, I was surprised to find that from the point of view of the current implementation, finally processing is not really that complicated. Indeed, in many ways, implementation has become easier, not harder. This shows how a lack of understanding of the problem can lead to an overly complex and slow implementation (although, in fairness, part of the complexity of the PHP 5 implementation stems directly from the lack of AST).
The finally blocks are executed whenever the control exits the try block, either in the usual way (for example, using return) or abnormally (throwing exceptions). There are several interesting cases to consider, which I will illustrate before embarking on an implementation. Consider:
try {
throw new Exception();
} finally {
return 42;
}
What's happening? Finally wins, and the function returns 42.
Consider:
try {
return 24;
} finally {
return 42;
}
Again, finally wins, and the function returns 42. Finally always wins.
PHP forbids transition from finally blocks. For example, the following is prohibited:
foreach ($array as $value) {
try {
return 42;
} finally {
continue;
}
}
Continue will generate a compilation error here. It’s important to understand that this limitation is purely “cosmetic,” and can be easily circumvented using the well-known delegation management scheme for catch:
foreach ($array as $value) {
try {
try {
return 42;
} finally {
throw new JumpException;
}
} catch (JumpException $e) {
continue;
}
}
The only real limitation that exists is that it is not possible to go to the finally block. For example, gettingo from outside the finally block to a label inside the finally block is forbidden.
With preliminary digressions, we can see how finally works. The implementation uses two opcodes: FAST_CALL and FAST_RET. Roughly speaking, FAST_CALL is intended to go to the finally block, and FAST_RET is intended to exit it. Consider the simplest case:
try {
echo "try";
} finally {
echo "finally";
}
echo "finished";
This code compiles into the following opcode sequence:
L0: ECHO string("try")
L1: T0 = FAST_CALL ->L3
L2: JMP ->L5
L3: ECHO string("finally")
L4: FAST_RET T0
L5: ECHO string("finished")
L6: RETURN int(1)
FAST_CALL saves its own location in T0 and goes into the finally block in L3. When FAST_RET is reached, it returns to the place stored in T0. In this case, it will be L2, where there is a jump through the finally block. This is the basic case when there is no special control flow (returns or exceptions).
Now consider the case with the exception:
try {
throw new Exception("try");
} catch (Exception $e) {
throw new Exception("catch");
} finally {
throw new Exception("finally");
}
When handling an exception, we should consider the position of the thrown exception relative to the nearest surrounding try / catch / finally block:
- Throwing out from try with the appropriate catch: populating $ e and going into catch.
- Throwing out from catch or try without corresponding catch if there is a finally block: go to the finally block and create an exception using FAST_CALL in an temporary variable (instead of storing the return address there).
- Throwing from finally: if there is an exception backup created earlier by FAST_CALL, bind it as the previous one to the just thrown. Continue throwing the exception to the next try / catch / finally.
- In other cases: continue to raise the exception to the next try / catch / finally.
In this example, we look at the first three steps: first, throw an exception in try, initiating a jump to catch. Catch also throws an exception, triggering a jump to the finally block with an exception backup in FAST_CALL. The finally block also throws an exception, so the finally exception will throw a catch exception, set as its previous exception.
This code is slightly different from the previous example:
try {
try {
throw new Exception("try");
} finally {}
} catch (Exception $e) {
try {
throw new Exception("catch");
} finally {}
} finally {
try {
throw new Exception("finally");
} finally {}
}
Entrance to all internal finally blocks occurs due to throwing exceptions, and exit - as usual (via FAST_RET). In this case, the above exception handling procedure is resumed starting from the parent try / catch / finally block. This parent try / catch is stored in the FAST_RET opcode (here, “try-catch (0)”).
This essentially covers finally interaction and exceptions. But what about return in finally?
try {
throw new Exception("try");
} finally {
return 42;
}
The corresponding opcode sequence is:
L4: T0 = FAST_CALL ->L6
L5: JMP ->L9
L6: DISCARD_EXCEPTION T0
L7: RETURN 42
L8: FAST_RET T0
The additional DISCARD_EXCEPTION opcode is responsible for refusing to further handle the exception raised in the try block (remember: returning to finally wins). What about return in try?
try {
$a = 42;
return $a;
} finally {
++$a;
}
The return value here is 42, not 43, since it is determined by the string return $ a , and any further modification of $ a should not matter. The result of this code will be:
L0: ASSIGN $a, 42
L1: T3 = QM_ASSIGN $a
L2: T1 = FAST_CALL ->L6, T3
L3: RETURN T3
L4: T1 = FAST_CALL ->L6 # недоступно
L5: JMP ->L8 # недоступно
L6: PRE_INC $a
L7: FAST_RET T1
L8: RETURN null
Two of the opcodes are not available, as they occur immediately after return. They will be deleted during optimization, but I show unoptimized opcodes. There are two interesting points here. First, $ a is copied to T3 using QM_ASSIGN (which is basically the “Copy to temporary variable” instruction). This prevents the subsequent modification of $ a from affecting the return value. Secondly, T3 is also passed to FAST_CALL, which will backup the value to T1. If later return from the try block is discarded (for example, because throw or return appears in finally), this mechanism will be used to free the unused return value.
All of these mechanisms are simple in themselves, but care must be taken when sharing them. Consider the following example, where Dtor is again a Traversable class with an exception throwing destructor:
try {
foreach (new Dtor as $v) {
try {
return 1;
} finally {
return 2;
}
}
} finally {
echo "finally";
}
This code generates the following opcodes:
L0: V2 = NEW (0 args) "Dtor"
L1: DO_FCALL
L2: V4 = FE_RESET_R V2 ->L16
L3: FE_FETCH_R V4 $v ->L16
L4: T5 = FAST_CALL ->L10 # внутрений try
L5: FE_FREE (free on return) V4
L6: T1 = FAST_CALL ->L19
L7: RETURN 1
L8: T5 = FAST_CALL ->L10 # недоступно
L9: JMP ->L15
L10: DISCARD_EXCEPTION T5 # внутрений finally
L11: FE_FREE (free on return) V4
L12: T1 = FAST_CALL ->L19
L13: RETURN 2
L14: FAST_RET T5 try-catch(0)
L15: JMP ->L3
L16: FE_FREE V4
L17: T1 = FAST_CALL ->L19
L18: JMP ->L21
L19: ECHO "finally" # внешний finally
L20: FAST_RET T1
The sequence for the first return (from internal try) is FAST_CALL L10, FE_FREE V4, FAST_CALL L19, RETURN. This will first call the internal finally block, then release the foreach loop variable, then call the external finally block and finally return the value. The sequence for the second return (from internal finally) is DISCARD_EXCEPTION T5, FE_FREE V4, FAST_CALL L19. First, the exception will be canceled (or here, the return value) of the internal try block, then the foreach loop variable will be freed, and finally the external finally block will be called. Note that in both cases, the order of these instructions is the reverse of the order of the corresponding blocks in the source code.
Generators
Generator functions can be paused and resumed, and therefore require special management of the virtual machine stack. Here is a simple generator:
function gen($x) {
foo(yield $x);
}
The following opcodes are obtained from this:
$x = RECV 1
GENERATOR_CREATE
INIT_FCALL_BY_NAME (1 args) string("foo")
V1 = YIELD $x
SEND_VAR_NO_REF_EX V1 1
DO_FCALL_BY_NAME
GENERATOR_RETURN null
Until GENERATOR_CREATE is reached, everything is executed as a normal function on a regular virtual machine stack. GENERATOR_CREATE creates a Generator object , as well as a execute_data structure distributed on the heap (including slots for variables and arguments, as usual), into which execute_data data is copied from the VM stack.
When generator execution resumes, the executor will use execute_data on the heap, but will continue to use the main virtual machine stack to place the call frames. The obvious problem is that, as the previous example shows, the generator may be interrupted during a call. Here, YIELD is executed at the point where the call frame for foo () has already been pushed onto the VM stack.
This relatively unusual case is handled by copying the active call frames to the generator structure when performing yield and restoring them when the generator resumes.
This design has been used since PHP 7.1. Previously, each generator had its own 4 KB VM page, which was loaded into the executor when the generator was restored. This avoided the need to copy call frames, but increased memory consumption.
Smart branches
Very often, conditional transitions immediately follow the comparison instructions. For instance:
L0: T2 = IS_EQUAL $a, $b
L1: JMPZ T2 ->L3
L2: ECHO "equal"
Since this pattern is very common, all comparison opcodes (such as IS_EQUAL) implement the smart branch mechanism: they check whether the next instruction is a JMPZ or JMPNZ instruction, and if so, they themselves perform the jump operation.
This mechanism only checks whether the next instruction is JMPZ / JMPNZ, but does not check whether the operand of this instruction is the result of a comparison. This requires special care in cases where the comparison and subsequent transition are not related. For example, the code ($ a == $ b) + ($ d? $ E: $ f) generates:
L0: T5 = IS_EQUAL $a, $b
L1: NOP
L2: JMPZ $d ->L5
L3: T6 = QM_ASSIGN $e
L4: JMP ->L6
L5: T6 = QM_ASSIGN $f
L6: T7 = ADD T5 T6
L7: FREE T7
Note that a NOP is inserted between IS_EQUAL and JMPZ. If this NOP were not there, the result of the comparison IS_EQUAL would be used for the transition, rather than the JMPZ operand.
Ranthime cache
Since the opcode arrays are open for sharing to several processes (without locks), they are strictly unchanged. However, runtime values can be cached in a separate runtime cache, which is usually an array of pointers. Literals can have a linked runtime cache entry (or more than one).
There are two types of runtime cache entries. The first is regular cache entries (such as those used in INIT_FCALL). After INIT_FCALL once searches for the called function (by its name), the function pointer will be cached in the corresponding slot of the cache cache.
The second type is polymorphic cache entries, which are two consecutive cache slots, where the first stores the beginning of the class, and the second - the offset of the class member. They are used for operations of type FETCH_OBJ_R, where the property offset in the property table of a particular class is cached. If the next call happens to a property of the same class (which is very likely), the cached value will be used. Otherwise, a more expensive search operation is performed, and the result is cached for the new class.
Virtual machine interrupts
Prior to PHP 7.0, execution timeouts were used that were processed by switching to the shutdown sequence directly from the signal handler. As you can imagine, this caused all kinds of trouble. Starting with PHP 7.0, timeout processing is delayed until control is returned to the virtual machine. If the control does not return within a certain (set) period, then the process ends. Starting with PHP 7.1, pcntl signal handlers use the same mechanism as runtime timeouts.
When the signal is in the standby state, an interrupt flag is set, which is checked by the virtual machine at certain points. The check is not performed with each new instruction, but only with transitions and calls. Thus, the interrupt will not be processed immediately upon returning control to the virtual machine, but at the end of the current section of the linear control flow.
Specialization
If you look at the virtual machine definitions file , you will find that the opcode handlers are defined as follows:
ZEND_VM_HANDLER(1, ZEND_ADD, CONST|TMPVAR|CV, CONST|TMPVAR|CV)
Here 1 is the opcode number, ZEND_ADD is its name, and the other two arguments indicate which types of operands the instruction accepts. The generated virtual machine code (generated using zend_vm_gen.php) will contain specialized handlers for each of the possible combinations of operand types. They will have names such as ZEND_ADD_SPEC_CONST_CONST_HANDLER.
Specialized handlers are generated by replacing certain macros in the body of the handler. The obvious are OP1_TYPE and OP2_TYPE, but operations such as GET_OP1_ZVAL_PTR () and FREE_OP1 () are also specialized.
The handler for ADD indicated that it accepts operands CONST | TMPVAR | CV. Here TMPVAR means that the opcode accepts both TMP and VAR, but requires that they not specialize separately. Remember that for most purposes, the only difference between TMP and VAR is that the latter can contain links. For an operation code such as ADD (where links are on the slow path anyway), having a separate specialization does not make sense. Some other opcodes that make this distinction will use TMP | VAR in the operand list.
In addition to specializing in operands, handlers can also specialize in other factors, such as the use of the return value.
ASSIGN_DIM specializes in the operand type of the following OP_DATA opcode:
ZEND_VM_HANDLER(147, ZEND_ASSIGN_DIM,
VAR|CV, CONST|TMPVAR|UNUSED|NEXT|CV, SPEC(OP_DATA=CONST|TMP|VAR|CV))Based on this signature, 2 * 4 * 4 = 32 different ASSIGN_DIM variants will be generated .
The second operand specification also contains an entry for NEXT . This is not specialization-specific, instead, it defines what the value of the UNUSED operand means in this context: it means that it is an add operation ( $ arr [] ).
Another example:
ZEND_VM_HANDLER(23, ZEND_ASSIGN_ADD,
VAR|UNUSED|THIS|CV, CONST|TMPVAR|UNUSED|NEXT|CV, DIM_OBJ, SPEC(DIM_OBJ))Here, the first UNUSED operand implies access on $ this . This is a general convention for opcodes associated with objects (e.g. FETCH_OBJ_R UNUSED , ' prop ' matches $ this-> prop ). The second UNUSED operand again implies an add operation. The third argument here defines the value of the extended_value operand: it contains a flag that distinguishes between $ a + = 1 , $ a [$ b] + = 1 and $ a-> b + = 1 . And finally, SPEC (DIM_OBJ)indicates that a specialized handler should be created for each of them. In this case, the number of generated handlers is nontrivial, since the VM generator knows that some combinations are impossible. For example, UNUSED op1 is applicable only for the case of OBJ, etc.
Finally, the virtual machine generator supports an additional, more complex specialization mechanism. At the end of the definition file, you will find several handlers for this form:
ZEND_VM_TYPE_SPEC_HANDLER(
ZEND_ADD,
(res_info == MAY_BE_LONG && op1_info == MAY_BE_LONG && op2_info == MAY_BE_LONG),
ZEND_ADD_LONG_NO_OVERFLOW,
CONST|TMPVARCV, CONST|TMPVARCV, SPEC(NO_CONST_CONST,COMMUTATIVE)
)These handlers specialize not only in the type of operand, but also based on the possible types that the operand can take at run time. The mechanism by which possible types of operands are determined is part of the cache optimization infrastructure and is completely beyond the scope of this article. However, assuming such information is available, it should be clear that this is a handler for adding the form int + int -> int . In addition, the SPEC annotation tells the specialist (the specializer component) that options for two constant operands should not be generated and that the operation is commutative, so if we already have the CONST + TMPVARCV specialization, we do not need to generate TMPVARCV + CONST.
Separation into fast and slow paths
Many opcode handlers are implemented using the fast path / slow path separation, where several common cases are handled first, and then return to the general implementation.
It's time to take a look at some real code, so I just insert the full SL implementation here (left shift):
ZEND_VM_HANDLER(6, ZEND_SL, CONST|TMPVAR|CV, CONST|TMPVAR|CV)
{
USE_OPLINE
zend_free_op free_op1, free_op2;
zval *op1, *op2;
op1 = GET_OP1_ZVAL_PTR_UNDEF(BP_VAR_R);
op2 = GET_OP2_ZVAL_PTR_UNDEF(BP_VAR_R);
if (EXPECTED(Z_TYPE_INFO_P(op1) == IS_LONG)
&& EXPECTED(Z_TYPE_INFO_P(op2) == IS_LONG)
&& EXPECTED((zend_ulong)Z_LVAL_P(op2) < SIZEOF_ZEND_LONG * 8)) {
ZVAL_LONG(EX_VAR(opline->result.var), Z_LVAL_P(op1) << Z_LVAL_P(op2));
ZEND_VM_NEXT_OPCODE();
}
SAVE_OPLINE();
if (OP1_TYPE == IS_CV && UNEXPECTED(Z_TYPE_INFO_P(op1) == IS_UNDEF)) {
op1 = GET_OP1_UNDEF_CV(op1, BP_VAR_R);
}
if (OP2_TYPE == IS_CV && UNEXPECTED(Z_TYPE_INFO_P(op2) == IS_UNDEF)) {
op2 = GET_OP2_UNDEF_CV(op2, BP_VAR_R);
}
shift_left_function(EX_VAR(opline->result.var), op1, op2);
FREE_OP1();
FREE_OP2();
ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION();
}Implementation begins by fetching operands using GET_OPn_ZVAL_PTR_UNDEF in BP_VAR_R mode. Here, the UNDEF part means that in the case of CV, data uncertainty is not checked, instead, you simply return the UNDEF value as is. Once we have the operands, we check that both of them are integers, and the shift width is in the range. In this case, the result can be directly calculated, and we move on to the next opcode. It is noteworthy that the type check here does not care if the operands are UNDEF, so using GET_OPn_ZVAL_PTR_UNDEF is justified.
If the operands do not satisfy the fast path, we return to the general implementation, which begins with SAVE_OPLINE (). This is our signal for "potentially throwing exception operations." Before moving on, the case of undefined variables is handled. GET_OPn_UNDEF_CV in this case will give a notification about an undefined variable and return a NULL value.
Then the standard shift_left_function is called, and its result is written to EX_VAR (opline-> result.var) . Finally, the input operands are freed up (if necessary), and we move on to the next opcode with an exception check (which means that the opline is reloaded before moving on).
Thus, the quick way here avoids two checks for undefined variables, calling a standard function, freeing operands, and saving and reloading opline to handle exceptions. Similarly, most performance-sensitive opcodes are computed.
Virtual machine macros
As you can see from the previous code, the implementation of a virtual machine allows you to freely use macros. Some of them are regular C language macros, while others are processed during virtual machine generation. In particular, there are several macros for fetching and freeing operands of instructions:
OPn_TYPE OP_DATA_TYPE GET_OPn_ZVAL_PTR (BP_VAR_ *) GET_OPn_ZVAL_PTR_DEREF (BP_VAR_ *) GET_OPn_ZVAL_PTR_UNDEF (BP_VAR_ *) GET_OPn_ZVAL_PTR_PTR (BP_VAR_ *) GET_OPn_ZVAL_PTR_PTR_UNDEF (BP_VAR_ *) GET_OPn_OBJ_ZVAL_PTR (BP_VAR_ *) GET_OPn_OBJ_ZVAL_PTR_UNDEF(BP_VAR_*) GET_OPn_OBJ_ZVAL_PTR_DEREF(BP_VAR_*) GET_OPn_OBJ_ZVAL_PTR_PTR(BP_VAR_*) GET_OPn_OBJ_ZVAL_PTR_PTR_UNDEF(BP_VAR_*) GET_OP_DATA_ZVAL_PTR() GET_OP_DATA_ZVAL_PTR_DEREF() FREE_OPn() FREE_OPn_IF_VAR() FREE_OPn_VAR_PTR() FREE_UNFETCHED_OPn() FREE_OP_DATA() FREE_UNFETCHED_OP_DATA()
As you can see, there are several variations. The BP_VAR_ * arguments define the data acquisition mode and support the same modes as the FETCH_ * instructions (except for FUNC_ARG).
GET_OPn_ZVAL_PTR () is the main instruction for obtaining an operand. It will issue a notification of an undefined CV and will not dereference the operand. GET_OPn_ZVAL_PTR_UNDEF () , as we already learned, is an option that does not check for undefined CVs. GET_OPn_ZVAL_PTR_DEREF () enables DEREF for zval. This is part of the specialized GET operation, since dereferencing is only necessary for CV and VAR, but not for CONST and TMP. Since this macro must distinguish between TMP and VAR, it can only be used with the TMP | VAR specialization (but not the TMPVAR ).
Variants GET_OPn_OBJ_ZVAL_PTR * () additionally handle the case with a UNUSED operand. As mentioned above, under the $ this convention, access uses a UNUSED operand, so the GET_OPn_OBJ_ZVAL_PTR * () macros will return an EX (This) reference for UNUSED.
Finally, there are several PTR_PTR options . The naming here is a relic of the days of PHP 5, where zval double INDIRECT pointers were actually used. These macros are used in write operations and as such only support CV and VAR types (everything else returns NULL). They differ from regular PTR samples in that they are “de-INDIRECT-cosiness” VAR operands.
Macros FREE_OP * ()then used to free selected operands. To work, they need the definition of the variable zend_free_op free_opN , into which the GET operation saves this value for release. The basic operation FREE_OPn () will release TMP and VAR, but not CV and CONST. FREE_OPn_IF_VAR () does exactly what it says: it releases the operand only if it is a VAR.
The FREE_OP * _VAR_PTR () option is used in conjunction with PTR_PTR selections . It will only release VAR operands and only if they are not INDIRECT.
Options FREE_UNFETCHED_OP * ()used when an operand needs to be freed before it is received using GET. This usually happens if an exception is thrown before receiving the operand.
In addition to these specialized macros, there are also many more standard macros. The VM defines three macros that control what happens after the opcode handler starts:
ZEND_VM_CONTINUE () ZEND_VM_ENTER () ZEND_VM_LEAVE () ZEND_VM_RETURN ()
CONTINUE will continue to execute opcodes as usual, while ENTER and LEAVE are used to enter / exit a nested function call. The specifics of their work depends on how the virtual machine is compiled (for example, whether global registers are used, and if so, which ones). In a broad sense, they will synchronize some state from global variables before continuing. RETURN is used to actually exit the main VM loop.
ZEND_VM_CONTINUE () expects the opline to be updated in advance. Of course, there are more macros related to this:
| Continue? | Check exception? | Check interrupt? | |
|---|---|---|---|
| ZEND_VM_NEXT_OPCODE () | yes | no | no |
| ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION () | yes | yes | no |
| ZEND_VM_SET_NEXT_OPCODE (op) | no | no | no |
| ZEND_VM_SET_OPCODE (op) | no | no | yes |
| ZEND_VM_SET_RELATIVE_OPCODE (op, offset) | no | no | yes |
| ZEND_VM_JMP (op) | yes | yes | yes |
The table shows whether the macro contains implicit ZEND_VM_CONTINUE (), whether it will check for VM exceptions and interrupts.
Next to them is also SAVE_OPLINE () , LOAD_OPLINE () and HANDLE_EXCEPTION (). As mentioned in the section on exception handling, SAVE_OPLINE () is used before the first potentially throwing operation in the opcode handler. If necessary, it writes back the opline used by the VM (may be in the global register) to execute data. LOAD_OPLINE () is a reverse operation, but it is currently underused because it was actually added to ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION () and ZEND_VM_JMP (). HANDLE_EXCEPTION () is used to return from the opcode handler after you already know that an exception has been thrown. It performs a combination of LOAD_OPLINE and CONTINUE.
Of course, there are more macros (there are always more macros ...), but this article should cover the most important ones.
