Writing a viewer for the MS Exchange mail database (part 2)

Hello readers Habrahabr!

This is the end of the post started here .

In principle, we have already done almost everything, there is a small ending. Let me remind you that we took an EDB database, opened it using ESE technology, listed the available tables and started listing the columns inside these tables. It remains for us to get the columns, get the values ​​of the columns and voila, the base is read.

In a previous post, I did not add a function responsible for moving to the next column in the table, as well as a description of the SColumnInfo structure, here they are:

typedef struct tagColumnInfo
{
        DWORD dwId;
        std :: wstring sName;
        DWORD dwType;
        DWORD dwMaxSize;
} SColumnInfo;

bool CJetDBReaderCore :: TableEnd (std :: wstring sTableName)
{
    std :: map :: const_iterator iter = m_tables.find (sTableName);
    if (iter! = m_tables.end ())
    {
        if (_JetMove (m_sesid, iter-> second, JET_MoveNext, 0))
        {
            return true;
        }
        else return false; 
    }
 
    return true;
}

Accordingly, when it is not possible to move to the next element, this is a sign that the table has ended .

We got a list of columns and entered them into our structures, it remains to get the values.

Get cell values ​​in columns

We write the following function:

JET_ERR CJetDBReaderCore :: EnumColumnsValues ​​( 
    SDBTableInfo & sTableInfo, 
    std :: list & sColumnsInfo)
{
    typedef std :: basic_string CByteArray;
    JET_ERR jRes = OpenTable (sTableInfo.sTableName);
    if (jRes == JET_errSuccess)
    {
        JET_COLUMNLIST sColumnInfo;
        jRes = GetTableColumnInfo (sTableInfo.sTableName, & sColumnInfo, FALSE);
        if (jRes! = JET_errSuccess) return jRes;
 
        if (! MoveToFirst (sTableInfo.sTableName))
        {
            std :: vector Holders (sColumnsInfo.size ());
            std :: vector:: iterator HolderIt = Holders.begin ();
 
            std :: vector Row (sTableInfo.sColumnInfo.size ());
            std :: vector:: iterator It = Row.begin ();
 
            std :: list:: const_iterator ColumnInfoIt 
                = sColumnsInfo.begin ();
 
            for (int nColNum = 0; ColumnInfoIt! = sColumnsInfo.end (); 
                ++ ColumnInfoIt, ++ It, ++ HolderIt, ++ nColNum)
            {
                DWORD dwMaxSize = ColumnInfoIt-> dwMaxSize? ColumnInfoIt-> dwMaxSize: 
MAX_BUFFER_SIZE;
                It-> columnid = ColumnInfoIt-> dwId;
                It-> cbData = dwMaxSize;
                HolderIt-> assign (dwMaxSize, '\ 0');
                It-> pvData = const_cast(HolderIt-> data ());
                It-> itagSequence = 1;
            }
 
            do
            {
                GetColumns ( 
                    sTableInfo.sTableName, & Row [0], 
                    static_cast  (Row.size ()));
 
                ColumnInfoIt = sColumnsInfo.begin ();
                int iSubItem = 0;
                for (It = Row.begin (); It! = Row.end (); ++ It, 
                    ++ ColumnInfoIt, ++ iSubItem)
                {
                    if ((* It) .cbActual)
                    {
                        std :: wstring s;
                        std :: string tmp;
                        wchar_t buff [16];
 
                        switch (ColumnInfoIt-> dwType)
                        {
                        case JET_coltypBit:
                            s = * reinterpret_cast((* It) .pvData)? L "True": L "False";
                            break;
                        case JET_coltypText:
                            tmp.assign (reinterpret_cast((* It) .pvData), (* It) .cbActual);
                            s.assign (tmp.begin (), tmp.end ());
                            break;
                        case JET_coltypLongText:
                            s.assign (reinterpret_cast((* It) .pvData), 
                                (* It) .cbActual / sizeof (wchar_t));
                            break;
                        case JET_coltypUnsignedByte:
                            s = _ltow (* static_cast((* It) .pvData), buff, 10);
                            break;
                        case JET_coltypShort:
                            s = _ltow (* static_cast((* It) .pvData), buff, 10);
                            break;
                        case JET_coltypLong:
                            s = _ltow (* static_cast((* It) .pvData), buff, 10);
                            break;
                        case JET_coltypGUID:
                            wchar_t wszGuid [64];
                            :: StringFromGUID2 (* reinterpret_cast((* It) .pvData), 
                                wszGuid, sizeof (wszGuid));
                            USES_CONVERSION
                            s = wszGuid;
                            break;
                        }
 
                        sTableInfo.sColumnInfo [iSubItem] .sColumnValues.push_back (s);
                    }
 
                    else sTableInfo.sColumnInfo [iSubItem] .sColumnValues.push_back (L "");
 
                }
            }
            while (! TableEnd (sTableInfo.sTableName));
        }
 
        jRes = CloseTable (sTableInfo.sTableName);
    }
 
    return jRes;
}

We will go through all the lines and for each we subtract the value of the cells. And let's start by moving to the first element by placing the cursor in the corresponding position (MoveToFirst function, see previous post ).

Everything is very similar to the way we listed table names from MSysObjects, with one exception: then we knew that the data is of string type, and here the data can be arbitrary. Therefore, we will create a special byte'vye containers Holders where and preserve the future information, as well as the normal vector, then we will not have to think about clearing memory, she is destroyed when we come out of sight of the function.

Those.:

• For brevity, we declare:

  typedef std :: basic_string CByteArray

• Create Holder'y for the data, they will get information from the database:

  std :: vector Holders (sColumnsInfo.size ());

• And then in the cycle we will connect pieces of memory where the data and our holders will return, and for this we first allocate enough memory for them to hold all the information (we got the maximum data size when listing the columns):

  HolderIt-> assign (dwMaxSize, '\ 0');
  It-> pvData = const_cast(HolderIt-> data ());

• And now in the loop we will return the information for each column to our vector JET_RETRIEVECOLUMN, which uses the memory allocated for the holders:

   GetColumns (sTableInfo.sTableName, & Row [0], static_cast  (Row.size ()));

As a result, we subtract one line from the table. But since data is an array of bytes, it needs to be converted to something more readable, so we will try to parse the cells depending on their types. In this example, the following types are analyzed (as the most obvious):

• JET_coltypBit
• JET_coltypText
• JET_coltypLongText
• JET_coltypUnsignedByte
• JET_coltypShort
• JET_coltypLong
• JET_coltypGUID

A description of the types available in ESE can be found here .

Conclusion

So we wrote a viewer of the mail database! If you start digging into these endless tables, you can find a lot of interesting things, for example, what has changed significantly in Exchange 2010 compared to all previous versions and much, much more.

What we wrote has a significant drawback: visibility. We filled a bunch of structures that did not display in any way. Those. you need to write a GUI program that will clearly show the contents of all these structures. Writing it should not be difficult, because it will be a set of simple grids; the benefit of the structure we have already prepared and completed, that is, roughly speaking, it remains only to print. I will give a couple of examples of such tables below.

What did it give us practically? On the one hand, a little, because most of the information is in binary formJET_coltypBinary and a description of this format you will not find anywhere. Exchange can’t do without reverse engineering . But on the other hand, we can better understand “And how does it really work?”, And this can turn out to be invaluable information. Plus, this is useful material for the "start" if you have to do something similar in the future.

Where can this come in handy? This can be useful only in cases where another API is not working or is not satisfied with some requirements, for example, performance requirements. Examples of such tasks can be: antiviruses, audit systems, migration and recovery.

Finally, two small examples of the data obtained.
A piece of the list of tables in the database:


List of folders and their types:


Thank you for reading and taking your time!

PS This code is an adapted and reduced version for the post. Therefore, there are some flaws in the code, or rather gags in places of truncated functionality. Please do not pay attention to them, this is not production, but I wanted to show working examples. The code is fully working and written so that it can be placed on the Internet and at the same time do not “eat” all the space on the page. Thank you for understanding.

PPS I understand that, due to the specifics, this information is unlikely to be useful for a wide range of people, but if it helps even one person, I will be glad and the time spent on this post will pay off.

Related Links

1. Extensible Storage Engine on MSDN
2. An article in Russian (almost the only article in RuNet about using the ESE API)
3. ESE Feature Examples