Diving into Reverse Engineering

Over the past couple of weeks I have been reading Practical Reverse Engineering. Reverse Engineering is one of the major topics that CTFs cover, and I wanted to deepen my knowledge here before moving on to Math and Cryptography (I have Applied Cryptography waiting for me on my bookshelf). I won’t spend too much time going over the first book, as I haven’t finished it yet and will write a full review later, but I wanted to record my thoughts about learning reverse engineering so far.

Practical Reverse Engineering is split up so that the first several chapters go over different flavors of assembly, x86 and ARM. The book has challenges…but they are in an analog format (literally written into the book), and aren’t as comprehensive as I would have liked, in light of the fact that I enjoyed solving the challenges and was searching for more to work on. Because of that, I went searching for other x86 challenges online and found several resources.

One of the great resources I found was from Trail of Bits. While they provide links to explanations of the basics of x86, the main part of their guide that I found useful was a link to labs from CMU and RIT. These two labs seemed to be a better place to start than what CTFs provide, as challenges in the latter format have a different goal than a teaching lab would (for example, making problems trickier and assuming prior knowledge).

Before starting with the challenges, I needed to first pick a tool to use. The Trail of Bits resource suggested using two main tools: a debugger to step through binaries and a binary analysis tool to visualize the inner works of the assembly programs. I didn’t deviate from the debugging tool (GDB, short for GNU Debugger) that they suggest because almost everyone uses it to step through programs and set breakpoints.

In contrast to choices for debugging tools, there is definitely more optionality in binary analysis tooling - software that one uses to look at the branches in assembly code. IDA Pro has been the industry standard for a long time. Given that the guide is somewhat old and that I knew other tools had been released recently, I decided to spend some more time exploring what my options are - I’m sure that I would be able to learn a new binary analysis tool, but I would prefer not to switch mid-stream. After some initial exploration, I narrowed my choices to the following:

• IDA Pro - somewhat old, industry standard, unclear whether people are switching over to something else
• Ghidra - people seem to be excited about this tool. It is produced by the NSA. I didn’t try it out.
• Radare2 - Radare is an open source tool for binary analysis, it is primarily text-based, with no-GUI, although people are developing GUI interfaces for it, like Cutter. Updates relatively frequently, which could be a plus. It does seem to have a steep learning curve, and might be useful for automation - LiveOverflow uses it for that.
• Binary Ninja - I found this tool while watching a video from LiveOverflow, a prominent YouTuber that solves CTF challenges (As a side note, he has a great playlist for beginners that I highly recommend. Some content isn’t useful for everyone, but it advances relatively quickly). Interestingly, one of the creators of Binary Ninja did a DefCon talk about CTF’s a while ago. There also was a great talk about automation with Binary Ninja that I found to be pretty interesting. Lastly, Trail of Bits has made efforts to work with the product, as they mention in one of their recent blog posts.

The other main tooling change I made was adding more information to my GDB display - I think this is somewhat essential. Customizing the display allows you to see the source code, register values, and other debugging information all on the same screen. I like the gdbinit file here, and might try out Voltron in the future.

I’m quite enjoying solving reverse engineering challenges, and I am building up to the “pwn” challenges, like exploit.education and Microcorruption from Square. I’ll be writing up more about my learning in those areas soon!