Hello!

Today we would like to share our list of materials on reverse engineering (RE). This list is very extensive, because our research department is primarily engaged in the tasks of RE. In our opinion, a selection of materials on the topic is good for starting, while it can be relevant for a long time.

For five years now we have been sending this list of links, resources, books to people who would like to get into our research department, but do not yet go through the level of knowledge or just begin their journey in the field of information security. Naturally, this list, like most materials / selections, will need updating and updating after some time.

Fun fact: we were shown how some companies send out our list of materials on their own, but only in a very old edition. And after this publication, they will finally be able to use its updated version with a clear conscience;)

So, let's move on to the list of materials!

1. Topics
   a. Reverse
   b. Search for vulnerabilities (fuzzing)
   c. Exploitation of vulnerabilities
   d. Malware analysis
2. Tools
   a. IDA Pro
   b. Radare2
   c. WinDBG (Ollydbg / Immunity Debugger / x64dbg)
   d. Gdb
   e. DBI
   f. SMT
   g. Python for automation
   h. BAF (Binary Analysis Frameworks)
3. Architecture
   a. x86-x86_64
   b. ARM
4. OS
   a. Windows
   b. Linux
   c. Mac OS (OSX) / iOS
   d. Android
5. File Formats
   a. PE
   b. ELF
   c. Mach-o
6. Programming
   a. C / C ++
   b. Assembler
7. Practice
   a. War games

1. Themes

In this section, we will consider the main areas of RE application. Let's start directly with the reverse development process itself, move on to search for vulnerabilities and exploit development, and, of course, get to the analysis of malware.

1.a reverse engineering

• "The Art of Disassembling " by Chris Kaspersky is not a new, but very good and still relevant book from Chris with a good systematization of knowledge and excellent material;
• " Practical Reverse Engineering: x86, x64, ARM, Windows Kernel, Reversing Tools, and Obfuscation " - a "new" book from several well-known information security experts, covering some new points and topics that are not in Chris's book;
• " Reversing for Beginners " by Denis Yurievich is a completely free book, already translated into many languages ​​of the world. Here, probably, the most remarkable thing is the presence of interesting tasks after each chapter, and at the same time for several architectures at once;
• " Practical RE tips " - an excellent webinar in English from Gynvael Coldwind, containing many useful tips and scripts about RE;
• The resource " OPENSECURITYTRAINING.INFO " contains good educational lectures and videos on RE in English;
• " Digging Through the Firmware " - a good series of articles by Practical Reverse Engineering - useful articles for those who are just going to plunge into the world of reversing device firmware;
• " Training: Security of BIOS / UEFI System Firmware from Attacker and Defender Perspectives " - if you want to plunge into the world of firmware security, UEFI BIOS, then you definitely need to familiarize yourself with these slides, which were previously in paid training at leading security conferences;
• CRYPTO101 is a bit of an introduction to cryptography, which is indispensable.

1.b Vulnerability Scan

• " Fuzzing: Brute Force Vulnerability Discovery " - although not a new book, is just right for understanding the basics of fuzzing. There is a translation into Russian, but it contains rather funny mistakes;
• " Automatic vulnerability search in programs without source codes " - a good introductory material in Russian, presented at PHDays 2011;
• " The Evolving Art of Fuzzing " - an article about the development of fuzzing;
• " Modern Security Vulnerability Discovery " - a compilation of different vulnerability search techniques in one document;
• " (State of) The Art of War: Offensive Techniques in Binary Analysis " - an all-in-one document about all existing vulnerability search techniques;
• " The Art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities " is far from a new, but still up-to-date book about different approaches to finding vulnerabilities.

1.c Examples of exploitation of vulnerabilities found

• " Exploit Writing Tutorials by Corelan Team " ( translation ) - a well-known series of posts on writing exploits and shellcodes, starting with the basics;
• " Exploit Development Community " ( partial translation ) - a series of articles on writing a combat exploit for IE 10 and 11 versions;
• " Modern Binary Exploitation " - materials from the RPISEC team from a training course they conducted at the Rensselaer Polytechnic Institute;
• " Vupen Company Blog Web Archive " - a sunk blog with examples of exploiting complex vulnerabilities in VirualBox, XEN, Firefox, IE10, Windows Kernel, Adobe Flash, Adobe Reader;
• " Project Zero " is a blog from the Google research team, where their experts often share interesting stories on exploiting various cool vulnerabilities;
• " Browser mitigations against memory corruption vulnerabilities " - protection technologies used in popular browsers:
  
  • " Browsers and app specific security mitigation. Part 1 "
  • " Browsers and app specific security mitigation. Part 2. Internet Explorer and Edge "
  • " Browsers and app specific security mitigation. Part 3. Google Chrome "
• " SoK: Eternal War in Memory " is an excellent document that shows an attack model and describes various mechanisms to prevent exploitation at different stages for various types of memory corruption vulnerabilities;
• " Writing Exploits for Win32 Systems from Scratch " - a detailed article on writing an exploit from scratch for a vulnerability in the SLMAIL program;
• Phrack - the famous hacker magazine Phrack. We recommend reading, first of all, articles of the category "The Art of Exploitation";
• " The Shellcoder's Handbook: Discovering and Exploiting Security Holes " is a legendary shellcode book.

1.d Malware Analysis

• " Practical Malware Labs " - sources for the book " Practical Malware Analysis ";
• " Malware Analyst's Cookbook and DVD: Tools and Techniques for Fighting Malicious Code " - we recommend this and previous books as one set for those who are interested in this topic;
• " Malware Analysis Tutorials: a Reverse Engineering Approach " ( translation ) - a rather long series of articles devoted to setting up the environment with the subsequent analysis of malware in it;
• " Course materials for Malware Analysis by RPISEC " is another course from RPISEC, only now about malware;
• " Computer viruses and antiviruses. The look of a programmer " - although the book discusses malicious programs from the time of DOS, it will still be useful, since in addition to analyzing the code of such programs, the author shows examples of writing antiviruses for each specific case.

2. Required tools

Below are the popular tools used with RE.

2.a IDA Pro

• " The IDA Pro Book: The Unofficial Guide to the World's Most Popular Disassembler " - a book through which your acquaintance with IDA Pro will pass easily and naturally :)
• " TiGa's Video Tutorial Series on IDA Pro " - a selection of small HOW-TO videos on the use of IDA Pro;
• " Open Analysis Live " - unlike the previous collection on the use of IDA Pro, this newer and updated. Basically, malware analysis is considered.

2.b Radare2

• " The radare2 book " - the main book on the use of the Radare2 framework for reverse;
• " Radare2 Cheatsheet " - "cheat sheet" for the main commands;
• " Radare Today - the blog of radare2 " is a framework blog. There are not only news, but also practical examples.

2.c WinDBG (Ollydbg / Immunity Debugger / x64dbg)

Without knowledge of the principles of the debugger and the ability to use it, you can not do too. Below we will consider debuggers for Windows, and in the next paragraph we will pay attention to the famous GDB. So, let's go:

• Advanced Windows Debugging: Developing and Administering Reliable, Robust, and Secure Software - first of all, this book is useful for understanding and "catching" errors such as heap damage;
• " Inside Windows Debugging: A Practical Guide to Debugging and Tracing Strategies in Windows " - this edition is a good complement to the previous book;
• "Introduction to cracking from scratch using OllyDbg" - unfortunately, the oldest resource wasm.ru was closed, but such a selection is easy to find because it was duplicated on many resources. In addition, forks began to appear on the network, only they already use x64dbg or IDA.

2.d gdb

• " gdb Debugging Full Example (Tutorial): ncurses " - GDB Implementation Guide;
• " GEF - Multi-Architecture GDB Enhanced Features for Exploiters & Reverse-Engineers " - an add-on for GDB in Python, adds many new useful commands that are useful for developing exploits;
• " GEF Tutorials " - a series of screencasts on the use of GEF.

2.e DBI

Programmable debugging is today an integral approach in the arsenal of any reverser. And DBI is one of the tools. More details:

• " Dynamic Binary Instrumentation in Information Security " - this article has already collected some generalized information about DBI;
• " Light And Dark Side Of Code Instrumentation " - this presentation will help you navigate the varieties of different code instrumentations and what and when you can help with the analysis of programs.

2.f SMT

What is an SMT solver? In short, an SMT solver is a program that can solve logical formulas.

The main idea of ​​using SMT in the field of software security is to translate a program code or algorithm into a logical formula, and then use the SMT solver to check one or another property of this code.

In other words, SMT provides a mathematical apparatus for semantic code analysis.
SMT-solvers have long been used in our field. They have proven themselves well for the following tasks:

• search for bugs (static analysis / fuzzing);
• deobfuscation;
• home cryptanalysis;
• symbolic performance (as a "engine");
• there are also some successes in the field of automatic exploit generation (for example, ROP generation).

During this time, SMT has lost its halo of mystery, more or less working tools for “ordinary” people have appeared.

Below are the sources that will help you dive into the topic:

• " SMT Solvers for Software Security, Sean Heelan, Rolf Rolles " is perhaps the first scientific work in which the use of SMT for solving problems in the field of software security was proposed. Gives an idea of ​​where and how SMT can find its place in this area;
• Z3 is one of the most popular and effective SMT solvers;
  
  • Z3 wiki - project repository;
  • " Getting Started with Z3: A Guide " - an online tutorial, SMT-solver for experiments;
  • Z3Py - Python binding for Z3;
  • " Experimenting with Z3 - Dead code elimination ";
  • " Experimenting with Z3 - Proving opaque predicates ";
• " Theorem prover, symbolic execution and practical reverse-engineering " - a good overview presentation, with examples of solving real problems and applying Z3Py;
• " Quick introduction into SAT / SMT solvers and symbolic execution " ( Russian version ) is a good book with interesting practical examples.
• " An introduction to the use SMT solvers " - overview.

2.g Python for automation

Today without knowledge of the basics of the Python language it will be very difficult, because this programming language is considered the most popular tool for automating various tasks in the field of information security (and not only). In addition, it is used in various utilities (for example, all the above utilities allow you to supplement the functionality with the help of this PL):

• " Gray Hat Python " ( translation ) is an excellent book that will explain how Python is useful in reverse;
• " The Beginner's Guide to IDAPython " - a free book about IDAPython;
• " Python Arsenal for Reverse Engineering " is a resource dedicated to various reverse engineering utilities and libraries using Python.

2.h BAF (Binary Analysis Frameworks)

For a little more advanced, we recommend that you pay attention to whole frameworks, which in their composition use the previously mentioned mechanisms and analysis tools to solve more complex problems. So here they are:

• " Overview and Usage of Binary Analysis Frameworks " - a small overview of BAF;

Some interesting frameworks / tools:

• Triton
  
  • Developer Case Studies
  • " Dynamic Binary Analysis and Obfuscated Codes "
  • How Triton can help to reverse virtual machine based software protections
• Angr
  
  • Solving kao's toy project with symbolic execution and angr
• Ponce
• Binary Analysis Platform .

3. Architecture

We will consider only a few popular architectures. At the end of the article in the section with additional materials you will find information on many others (MIPS, PowerPC, etc.).

3.a x86-x86_64

• " Intel 64 and IA-32 Architectures Software Developer Manuals " - previously such manuals were sent to the post office, but due to the large amount of material in them, printing became an expensive pleasure. Recommended as a desktop reference.

3.b ARM

• Azeria Labs (ARM Assembly Basics & ARM Exploit Development) - a site with articles on the basics of ARM assembler and the development of exploits for this architecture;
• Course " Introduction to ARM " - a two-day video course on ARM development and operation;
• VisUAL - visualization of the work of ARM-teams.

4. OS

Knowledge of the operating principles of popular Operating Systems.

4.a Windows

• " Windows Internals " is the foundational book for understanding how Windows works.
  The following points, although mainly related to the exploitation of vulnerabilities in this OS, but allow you to better understand the insides of Windows:
  
  • " Windows exploits, mostly precompiled "
  • " Exploit Development Environment "
  • " Windows Breakout from Defcon24 "
  • " Part 10: Kernel Exploitation -> Stack Overflow "
  • " Kernel and Driver explotation ".

4.b Linux

• " Linux insides " - an analogue of the book Windows Internals, but only for OS such as Linux.
  As with Windows, the following topics are related to exploit development:
  
  • " Heap Exploitation into Linux "
  • " A series tutorial for linux exploit development to newbie "
  • " Linux Kernel Exploitation "
  • " Programming Linux Anti-Reversing Techniques "

4.c Mac OS (OSX) / iOS

• " Reverse Engineering Resources Mac and iOS " - a selection of materials on this topic.

4.d Android

• " Android Hacker's Handbook " is probably the most popular Android security book;
• " Android Internals :: Power User's View " is a book about the internal mechanisms of this OS. Due to recent leaks, the material appeared in the public domain, as the author writes on his site and provides the opportunity to download the previous version.

5. Executable file formats

This section provides links explaining the details of popular executable file formats.

5.a PE

• " PE sections ";
• " PE header ";
• " Windows executable file format. PE32 and PE64 ";
• " Computer viruses inside and out ."

5.b ELF

• " Linux x64 Infection for Lamers (by a Lamer) ."

5.c Mach-O

• " Parsing mach-o files "

The famous researcher corkami makes very useful and interesting "posters" with a diagram of various file formats, including those mentioned above. We recommend using them as a cheat sheet. And the Kaitai Sctruct utility will help with the analysis.

6. Programming

One of our acquaintances once said that a good reverse is 80% good programmer. The ability to program and an understanding of what is being done and why is simplifying the process of researching someone else's program. Therefore, without programming in reverse, nowhere. And of course, automation of a routine task, as you probably already understood, is a very useful thing;)

6.a C / C ++

• Modern Memory Safety: C / C ++ Vulnerability Discovery, Exploitation, Hardening is a great course with great examples. Just must have stuff for everyone.

6.b ASM

• " A Crash Course in x86 Assembly for Reverse Engineers " - "crash course" for immersion in x86 Assembler, positioned as special for RE;
• " Assembly Programming Tutorial " - a guide to programming in Assembler, with the ability to run examples online as you study;
• " Assembler. 2nd Edition " - recommended as a reference;
• " x86 Assembly Guide " is an online version.

7. Practice

This section provides links to virtual machines and online resources that allow you to practice.

7.a War Games

• SmashTheStack Wargaming Network - This network with several wargame is supported by volunteers and is available online. We recommend starting with it;
• BinTut - local wargame;
• Reversing Workshop - master class on solving tasks from the annual competition "The Flare On Challenge" for 2016;
• Exploit-Challenges - a selection of vulnerable ARM binary files;
• ARM Reverse Engineering Exercises - the original repository "disappeared", but one of the forks was found on the open spaces of github;
• CTF Time - here you can find out the schedule of future CTF events and read the decisions of the past.

And finally, a few links with a lot of materials on the above topics:

• Compilation dedicated to the field of information security
• About exploiting vulnerabilities
• About reverse engineering:
  
  • Awesome-reversing
  • REMath Resource Overview
• About exploiting vulnerabilities in Windows
• About phasing
• About malware analysis
• And many more different awesome collections.