June 25, 2019 at 11:18

Creating a listening application to view mobile MMORPG traffic

    This is the second part of a series of articles on the analysis of network traffic of mobile MMORPG. Sample loop topics:

    1. Parse message format between server and client.
    2. Writing a listening application to view game traffic in a convenient way.
    3. Traffic interception and its modification using a non-HTTP proxy server.
    4. The first steps to your own ("pirated") server.

    In this part I will describe the creation of a listening application (sniffer), which will allow us to filter events by their type and source, display information about the message and selectively save them for analysis, and also get into the game’s executable file (“binary”) a bit supporting information and add Protocol Buffers support to the app. Interested, I ask for cat.

    Tools Required


    To be able to repeat the steps described below, you will need:

    • Wireshark for packet analysis;
    • .NET
    • PcapDotNet library for working with WinPcap;
    • protobuf-net library for working with Protocol Buffers.

    Writing a Listening Application


    As we recall from the previous article , the game communicates via the TCP protocol, and in the framework of the session it does this with only one server and on one port. To be able to analyze game traffic, we need to perform the following tasks:

    • intercept packets of a mobile device;
    • filter game packages;
    • add the data of the next packet to the buffer for further processing;
    • Get game events from buffers as they are populated.

    These actions are implemented in a class Snifferthat uses the PcapDotNet library to intercept packets. In the method, Sniffwe pass the IP address of the adapter (in fact, this is the address of the PC from which Wi-Fi is distributed for the mobile device within the same network), the IP address of the mobile device, and the IP address of the server. Due to the inconsistency of the last two (after months of monitoring different platforms and servers, it turned out that the server is selected from a pool of ~ 50 servers, each of which still have 5-7 possible ports), I transmit only the first three octets. The use of this filtering is visible in the method IsTargetPacket.

    public class Sniffer
{
    private byte[] _data = new byte[4096];
    public bool Active { get; set; } = true;
    private string _adapterIP;
    private string _target;
    private string _server;
    private List _serverBuffer;
    private List _clientBuffer;
    private LivePacketDevice _device = null;
    private PacketCommunicator _communicator = null;
    private Action _eventCallback = null;
    public void Sniff(string ip, string target, string server)
    {
        _adapterIP = ip;
        _target = target;
        _server = server;
        _serverBuffer = new List();
        _clientBuffer = new List();
        IList allDevices = LivePacketDevice.AllLocalMachine;
        for (int i = 0; i != allDevices.Count; ++i)
        {
            LivePacketDevice device = allDevices[i];
            var address = device.Addresses[1].Address + "";
            if (address == "Internet " + _adapterIP)
            {
                _device = device;
            }
        }
        _communicator = _device.Open(65536, PacketDeviceOpenAttributes.Promiscuous, 1000);
        _communicator.SetFilter(_communicator.CreateFilter("ip and tcp"));
        new Thread(() =>
        {
            Thread.CurrentThread.IsBackground = true;
            BeginReceive();
        }).Start();
    }
    private void BeginReceive()
    {
        _communicator.ReceivePackets(0, OnReceive);
        do
        {
            PacketCommunicatorReceiveResult result = _communicator.ReceivePacket(out Packet packet);
            switch (result)
            {
                case PacketCommunicatorReceiveResult.Timeout: continue;
                case PacketCommunicatorReceiveResult.Ok: OnReceive(packet); break;
            }
        } while (Active);
    }
    public void AddEventCallback(Action callback)
    {
        _eventCallback = callback;
    }
    private void OnReceive(Packet packet)
    {
        if (Active)
        {
            IpV4Datagram ip = packet.Ethernet.IpV4;
            if (IsTargetPacket(ip))
            {
                try
                {
                    ParseData(ip);
                }
                catch (ObjectDisposedException)
                {
                }
                catch (EndOfStreamException e)
                {
                    Console.WriteLine(e);
                }
                catch (Exception)
                {
                    throw;
                }
            }
        }
    }
    private bool IsTargetPacket(IpV4Datagram ip)
    {
        var sourceIp = ip.Source.ToString();
        var destIp = ip.Destination.ToString();
        return (sourceIp != _adapterIP && destIp != _adapterIP) && (
               (sourceIp.StartsWith(_target) && destIp.StartsWith(_server)) ||
               (sourceIp.StartsWith(_server) && destIp.StartsWith(_target))
            );
    }
    private void ParseData(IpV4Datagram ip)
    {
        TcpDatagram tcp = ip.Tcp;
        if (tcp.Payload != null && tcp.PayloadLength > 0)
        {
            var payload = ExtractPayload(tcp);
            AddToBuffer(ip, payload);
            ProcessBuffers();
        }
    }
    private byte[] ExtractPayload(TcpDatagram tcp)
    {
        int payloadLength = tcp.PayloadLength;
        MemoryStream ms = tcp.Payload.ToMemoryStream();
        byte[] payload = new byte[payloadLength];
        ms.Read(payload, 0, payloadLength);
        return payload;
    }
    private void AddToBuffer(IpV4Datagram ip, byte[] payload)
    {
        if (ip.Destination.ToString().StartsWith(_target))
        {
            foreach (var value in payload)
                _serverBuffer.Add(value);
        }
        else
        {
            foreach (var value in payload)
                _clientBuffer.Add(value);
        }
    }
    private void ProcessBuffers()
    {
        ProcessBuffer(ref _serverBuffer);
        ProcessBuffer(ref _clientBuffer);
    }
    private void ProcessBuffer(ref List buffer)
    {
        // TODO
    }
    public void Suspend()
    {
        Active = false;
    }
    public void Resume()
    {
        Active = true;
    }
}

    Great, now we have two buffers with packet data from the client and server. Recall the format of events between the game and the server:

    struct Event {
    uint payload_length ;
    ushort event_code ;
    byte payload[payload_length] ;
};

    Based on this, you can create an event class Event:

    public enum EventSource
{
    Client, Server
}
public enum EventTypes : ushort
{
    Movement = 11,
    Ping = 30,
    Pong = 31,
    Teleport = 63,
    EnterDungeon = 217
}
public class Event {
    public uint ID;
    public uint Length { get; protected set; }
    public ushort Type { get; protected set; }
    public uint DataLength { get; protected set; }
    public string EventType { get; protected set; }
    public EventSource Direction { get; protected set; }
    protected byte[] _data;
    protected BinaryReader _br = null;
    public Event(byte[] data, EventSource direction)
    {
        _data = data;
        _br = new BinaryReader(new MemoryStream(_data));
        Length = _br.ReadUInt32();
        Type = _br.ReadUInt16();
        DataLength = 0;
        EventType = $"Unknown ({Type})";
        if (IsKnown())
        {
            EventType = ((EventTypes)Type).ToString();
        }
        Direction = direction;
    }
    public virtual void ParseData()
    {
    }
    public bool IsKnown()
    {
        return Enum.IsDefined(typeof(EventTypes), Type);
    }
    public byte[] GetPayload(bool hasDatLength = true)
    {
        var payloadLength = _data.Length - (hasDatLength ? 10 : 6);
        return new List(_data).GetRange(hasDatLength ? 10 : 6, payloadLength).ToArray();
    }
    public virtual void Save()
    {
        var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Packets", EventType);
        Directory.CreateDirectory(path);
        File.WriteAllBytes(path + $"/{ID}.dump", _data);
    }
    public override string ToString()
    {
        return $"Type {Type}. Data length: {Length}.";
    }
    protected ulong ReadVLQ(bool readFlag = true)
    {
        if (readFlag)
        {
            var flag = _br.ReadByte();
        }
        ulong vlq = 0;
        var i = 0;
        for (i = 0; ; i += 7)
        {
            var x = _br.ReadByte();
            vlq |= (ulong)(x & 0x7F) << i;
            if ((x & 0x80) != 0x80)
            {
                break;
            }
        }
        return vlq;
    }
}

    The class Eventwill be used as the base class for all game events. Here is an example class for the event Ping:

    public class Ping : Event
{
    private ulong _pingTime;
    public Ping(byte[] data) : base(data, EventSource.Client)
    {
        EventType = "Ping";
        DataLength = 4;
        _pingTime = _br.ReadUInt32();
    }
    public override string ToString()
    {
        return $"Pinging server at {_pingTime}ms.";
    }
}

    Now that we have the event class, we can add methods to Sniffer:

    private void ProcessBuffer(ref List buffer)
{
    if (buffer.Count > 0)
    {
        while (Active)
        {
            if (buffer.Count > 4) // Первые 4 байта в событии содержат размер полезной нагрузки ...
            {
                var eventLength = BitConverter.ToInt32(buffer.Take(4).ToArray(), 0) + 6; // ... поэтому размер события - это размер П.Н. + первые 4 байта + 2 байта кода события
                if (eventLength >= 6 && buffer.Count >= eventLength)
                {
                    var eventData = buffer.Take(eventLength).ToArray();
                    var ev = CreateEvent(eventData, direction);
                    buffer.RemoveRange(0, eventLength);
                    continue;
                }
            }
            break;
        }
    }
}
private Event CreateEvent(byte[] data, EventSource direction)
{
    var ev = new Event(data, direction);
    var eventType = Enum.GetName(typeof(EventTypes), ev.Type);
    if (eventType != null)
    {
        try
        {
            // Создаем экземпляр класса события (например, Ping).
            var className = "Events." + eventType;
            Type t = Type.GetType(className);
            ev = (Event)Activator.CreateInstance(t, data);
        }
        catch (Exception)
        {
            // Если специального класса нет - создаем экземпляр базового.
            ev = new Event(data, direction);
        }
        finally
        {
        }
    }
    _eventCallback?.Invoke(ev);
    return ev;
}

    Create a form class that will trigger the wiretap:

    public partial class MainForm : Form
{
    private Sniffer _sniffer = null;
    private List _events = new List();
    private List _eventTypesFilter = new List();
    private bool _showClientEvents = true;
    private bool _showServerEvents = true;
    private bool _showUnknownEvents = false;
    private bool _clearLogsOnRestart = true;
    private uint _eventId = 1;
    private void InitializeSniffer()
    {
        _sniffer = new Sniffer();
        _sniffer.AddEventCallback(NewEventThreaded);
        _sniffer.Sniff("192.168.137.1", "192.168.137.", "123.45.67.");
    }
    private void NewEventThreaded(Event ev)
    {
        events_table.Invoke(new NewEventCallback(NewEvent), ev);
    }
    public delegate void NewEventCallback(Event ev);
    private void NewEvent(Event ev)
    {
        ev.ID = _eventId++;
        _events.Add(ev);
        LogEvent(ev);
    }
    private void LogEvent(Event ev)
    {
        if (FilterEvent(ev))
        {
            var type = ev.GetType();
            events_table.Rows.Add(1);
            events_table.Rows[events_table.RowCount - 1].Cells[0].Value = ev.ID;
            events_table.Rows[events_table.RowCount - 1].Cells[1].Value = ev.EventType;
            events_table.Rows[events_table.RowCount - 1].Cells[2].Value = Enum.GetName(typeof(EventSource), ev.Direction);
            events_table.Rows[events_table.RowCount - 1].Cells[3].Value = ev.ToString();
        }
    }
    private void ReloadEvents()
    {
        events_table.Rows.Clear();
        events_table.Refresh();
        foreach (var ev in _events)
        {
            LogEvent(ev);
        }
    }
    private bool FilterEvent(Event ev)
    {
        return (
                (ev.Direction == EventSource.Client && _showClientEvents) ||
                (ev.Direction == EventSource.Server && _showServerEvents)
               ) && (_eventTypesFilter.Contains(ev.Type) || (!ev.IsKnown() && _showUnknownEvents));
    }
}

    Done! Now you can add a couple of tables for managing the list of events (it is filled through it _eventTypesFilter) and viewing in real time (main table events_table). For example, I filtered by the following criteria (method FilterEvent):

    • show events from the client;
    • show events from the server;
    • display of unknown events;
    • Show selected known events.

    Learning the game executable


    Although it is now possible to analyze the events of the game without problems, there is a huge amount of manual work to determine not only the meaning of all event codes, but also the structure of the payload, which will be quite difficult, especially if it changes depending on some fields. I decided to look for some information in the executable file of the game. Since the game is cross-platform (available on Windows, iOS and Android), the following options are available for analysis:

    • .exe file (I have located on the path C: / Program Files / WindowsApps /% appname% /;
    • a binary iOS file that is encrypted by Apple, but with JailBreak you can get the decrypted version using Crackulous or the like;
    • Android shared library It is located on the path / data / data /% app-vendor-name% / lib /.

    Having no idea which architecture to choose for Android and iOS, I started with an .exe file. We load the binary into the IDA, we see the choice of architectures.



    The purpose of our search is some very useful lines, which means decompilation of assembler is not included in the plans, but just in case, select “executable 80386”, since the “Binary File” and “MS-DOS executable” options are clearly not suitable. Click "OK", wait until the file is loaded into the database, and it is advisable to wait until the analysis of the file is complete. The end of the analysis can be recognized by the fact that the status bar in the bottom left will have the following status:



    Go to the Strings tab (View / Open subviews / Strings or Shift + F12). The line generation process may take some time. In my case, ~ 47k lines were found. The line location addresses have a prefix of the form .data, .rdataandothers . In my case, all the “interesting” lines were in a section the .rdatasize of which was ~ 44.5k records. Looking through the table you can see:

    • error messages and query segments during the login phase;
    • error strings and initialization information for the game, game engine, interfaces;
    • a lot of garbage;
    • a list of game tables on the client side;
    • used values ​​of the game engine in the game;
    • list of effects;
    • huge list of interface localization keys;
    • etc.

    Finally, closer to the end of the table comes what we were looking for.



    This is a list of event codes between client and server. This can simplify our lives when parsing the network protocol of the game. But we will not stop there! It is necessary to check whether it is possible to somehow get the numerical value of the event code. We see “familiar” from the previous article codes CMSG_PINGand SMSG_PONGhaving codes 30 ( ) and 31 ( ), respectively. Double-click on the line to go to this place in the code. Indeed, immediately after the string values ​​of the codes is a sequence of and . Well, that means you can parse the entire table and get a list of events and their numerical value, which will further simplify the analysis of the protocol.1E161F16



    0x10 0x1E0x10 0x1F

    Unfortunately, the Windows version of the game lags behind the mobile versions by a lot of versions, and therefore the information from .exe is not relevant, and although it can help, you should not rely on it entirely. Next, I decided to study the dynamic library with Android, as I saw on one forum that there, unlike iOS binaries, contains a lot of meta-information about classes. But alas, a search in the values ​​file CMSG_PINGdid not return results.

    Without hope, I’m doing the same search in the iOS binary - unbelievable, but the same data turned out to be there as in .exe! Upload the file to the IDA.



    I choose the first proposed option, because I’m not sure which one is needed. Again, we are waiting for the end of the file analysis (the binary is almost 4 times larger in size .exe, the analysis time, of course, also increased). We open a window with lines, which this time turned out to be 51k. Through Ctrl + Flooking CMSG_PINGand ... we do not find. Entering the code character by character, you can notice this result:



    For some reason, the IDA arranged the entire object Opcode.protoin one line. Double-click on this place in the code and see that the structure is described in the same way as in the .exe file, so you can cut it and convert it to Enum.

    Here, finally, it is worth recalling that in the comments on the previous article, aml suggested that the game message structure is an implementation of Protocol Buffers. If you look closely at the code in a binary file, you can see that the description is Opcodealso in this format.



    We will write a parser template for 010Editor to get all the code values.

    Updated Packed * Type Code for 010Editor
    Minor type changes contain field label validation to skip missing ones.

    uint PeekTag() {
    if (FTell() == FileSize()) {
        return 0;
    }
    Varint tag;
    FSkip(-tag.size);
    return tag._ >> 3;
}
struct Packed (uint fieldNumber) {
    if (PeekTag() != fieldNumber) {
        break;
    }
    Varint key ;
    local uint wiredType = key._ & 0x7;
    local uint field = key._ >> 3;
    local uint size = key.size;
    switch (wiredType) {
        case 1: double value; size += 8; break;
        case 5: float value; size += 4; break;
        default: Varint value; size += value.size; break;
    }
};
struct PackedString(uint fieldNumber) {
    if (PeekTag() != fieldNumber) {
        break;
    }
    Packed length(fieldNumber);
    char str[length.value._];
};



    struct Code {
    Packed size(2) ;
    PackedString code_name(1) ;
    Packed code_value(2) ;
    Printf("%s = %d,\n", code_name.str, code_value.value._); // Вывод значения в консоль для вставки в Enum
};
struct Property {
    Packed size(5) ;
    PackedString prop_name(1) ;
    while (FTell() - 0x176526B - prop_name.length.value._ < size.value._) {
        Code codes ;
    }
};
struct {
    FSkip(0x176526B);
    PackedString object(1) ;
    PackedString format(2) ;
    Property prop;
} file;

    The result is something like this:



    more interesting! Noticed pbin the description of the object? It would be necessary to look for other lines, suddenly there are a lot of such objects?



    The results are extremely unexpected. Apparently, the game’s executable file describes many types of data, including enumerations and message formats between the server and the client. Here is an example of a description of a type that describes the position of an object in the world: A



    quick search revealed two large places with descriptions of types, although a more detailed study will certainly reveal other small places. Having cut them out, I wrote a small script in C # to separate the descriptions by file (in structure this is similar to the description of the list of event codes) - it is easier to analyze them in 010Editor.

    class Program
{
    static void Main(string[] args)
    {
        var br = new BinaryReader(new FileStream("./BinaryFile.partX", FileMode.Open));
        while (br.BaseStream.Position < br.BaseStream.Length)
        {
            var startOffset = br.BaseStream.Position;
            var length = ReadVLQ(br, out int size);
            var tag = br.ReadByte();
            var eventName = br.ReadString();
            br.BaseStream.Position = startOffset;
            File.WriteAllBytes($"./parsed/{eventName}", br.ReadBytes((int)length + size + 1));
        }
    }
    static ulong ReadVLQ(BinaryReader br, out int size)
    {
        var flag = br.ReadByte();
        ulong vlq = 0;
        size = 0;
        var i = 0;
        for (i = 0; ; i += 7)
        {
            var x = br.ReadByte();
            vlq |= (ulong)(x & 0x7F) << i;
            size++;
            if ((x & 0x80) != 0x80)
            {
                break;
            }
        }
        return vlq;
    }
}

    I will not analyze the format for describing structures in detail, because either it is specific to the game in question, or it is generally accepted in the Protocol Buffersformat (if anyone knows for sure, please indicate in the comments). From what I could detect:

    • the description also comes in a format Protocol Buffers;
    • the description of each field contains its name, number and data type, for which its own type table was used:
      string TypeToStr (uint type) {
    switch (type) {
        case 2: return "Float";
        case 4: return "UInt64";
        case 5: return "UInt32";
        case 8: return "Boolean";
        case 9: return "String";
        case 11: return "Struct";
        case 14: return "Enum";
        default: local string s; SPrintf(s, "%Lu", type); return s;
    }
};
    • if the data type is an enumeration or structure, then there was a link to the desired object.

    Well, the last thing that remains for us is to use the information received in our listening application: parse messages using the library protobuf-net. Connect the library via NuGet, add using ProtoBuf;and you can create classes to describe messages. Take one example from a previous article: character movement. The formatted description of the format when highlighting segments looks something like this:



    Debug output allows you to create a short description from this:

    Field 1 (Type 13): time 
Field 2 (Struct .pb.CxGS_Vec3): position 
Field 3 (UInt64): guid 
Field 4 (Struct .pb.CxGS_Vec3): direction 
Field 5 (Struct .pb.CxGS_Vec3): speed 
Field 6 (UInt32): state 
Field 10 (UInt32): flag 
Field 11 (Float): y_speed 
Field 12 (Boolean): is_flying 
Field 7 (UInt32): emote_id 
Field 9 (UInt32): emote_duration 
Field 8 (Boolean): emote_loop

    Now you can create the appropriate class using the library protobuf-net.

    [ProtoContract]
public class MoveInfo : ProtoBufEvent
{
    [ProtoMember(3)]
    public ulong GUID;
    [ProtoMember(1)]
    public ulong Time;
    [ProtoMember(2)]
    public Vec3 Position;
    [ProtoMember(4)]
    public Vec3 Direction;
    [ProtoMember(5)]
    public Vec3 Speed;
    [ProtoMember(6)]
    public ulong State;
    [ProtoMember(7, IsRequired = false)]
    public uint EmoteID;
    [ProtoMember(8, IsRequired = false)]
    public bool EmoteLoop;
    [ProtoMember(9, IsRequired = false)]
    public uint EmoteDuration;
    [ProtoMember(10, IsRequired = false)]
    public uint Flag;
    [ProtoMember(11, IsRequired = false)]
    public float SpeedY;
    [ProtoMember(12)]
    public bool IsFlying;
    public override string ToString()
    {
        return $"{GUID}: {Position}";
    }
}

    For comparison, here is a template for the same event from a previous article:

    struct MoveEvent {
    uint data_length ;
    Packed move_time ;
    PackedVector3 position ;
    PackedVector3 direction ;
    PackedVector3 speed ;
    Packed state ;
};

    When inheriting a class, Eventwe can override the method ParseDataby deserializing the package data:

    class CMSG_MOVE_INFO : Event
{
    private MoveInfo _message;
    [...]
    public override void ParseData()
    {
        _message = MoveInfo.Deserialize(GetPayload());
    }
    public override string ToString()
    {
        return _message.ToString();
    }
}

    That's all. The next step is to redirect the game traffic to our proxy server for the purpose of injection, spoofing and cutting packages.
    Tags:

    Also popular now: