Decrypting Rust’s Secrets

A Journey into Reverse Engineering Rust’s AES Crypto Function

# Introduction:

In the world of software and cryptography, secrets are often hidden behind layers of code, algorithms and encryption. However, the desire to unveil these secrets, to understand how they work and ensure their reliability, has driven many curious minds to embark on the path of reverse engineering. This journey takes us deep into the heart of Rust, a language renowned for its saftey, preformance, and robust cryptographic libraries. In this blog post, we will reverse engineer a Rust AES (Advanced Encryption Standard) program I wrote using Cutter.

Rust: Decompilation Hellscape For Reversing

Decompiling Rust executables is not the same as decompiling executables from other languages like C for several reasons. Rust’s design principles and its compilation process introduces unique challenges for those attempting to reverse engineer or decompile Rust programs. Here are some key differences:

  1. Ownership and Borrowing Model: Rust’s ownership and borrowing model, designed to eliminate common programming errors like null pointer dereferences and data races, adds complexity to decompilation. The owenership model can result in the Rust compiler generating more optimized and sometimes harder-to-understand machine code.
  2. No Undefined Behavior: Rust is designed to be memory-safe, and its compiler enforces this by preventing undefined behavior. In languages like C/C++, undefined behavior can lead to unpredictable machine code and easier decompilation, while Rust provides clearer guarantees about program behavior.
  3. Abstraction and Zero-Cost Abstraction: Rust encourages the use of high-level abstrations without sacrificing runtime preformance. This can result in more complex intermediate representations in the compiled code, which might be harder to decompile into a human-readable form.
  4. Inlining and Optimization: The Rust compiler aggressively optimizes code, including inlining functions. This optimization can make it challenging to trace code flow, as the resulting machine code may not align with the original source code structure.
  5. Modern Compiler Infrastructure: Rust uses the LLVM compiler infrastructure, which is highly sophisticated and focused on generating efficient machine code. The interaction between Rust and LLVM can make the process of decompiling Rust code different from traditional C/C++ executables.
  6. Lack of Debug Symbols: Rust executables typically don’t include debug symbols in release builds, making it harder to recover variable and function names. Debug symbols can be a valuable resource for reverse engineering C programs.

In the world of malware, these challenges make a Rust choice language for development. This is also due to convenient static linking and support for many operating systems, yielding a binary with little to no dependencies.

Disecting Advanced Encryption Standard (AES)

The Advanced Encryption Standard (AES) algorithm breaks up plaintext string into 16 byte “blocks” that undergo several rounds of encryption:Offical AES Docs

  1. Expand provided key (Key Expansion)

2)AddRoundKey function on the plaintext block using the expanded key

  1. Multiple encryption Rounds a.SubBytes Function on block b.ShiftRows function on block c.MixColumns function on block
    1. AddRoundKey function
  2. Final SubBytes call
  3. Final ShiftRows call
  4. Final AddRoundKey call

The number of rounds that are used and the size of the expanded key, \boxed{\text{EK}}$, are dependent on the size of the provided key. The following table shows the correlation between the key length, the number of rounds, and the length of the expanded key:

Key Length (Bits) Number of Rounds Expanded Key Length (Words)
----------------- ----------------- ----------------------------
128 10 44
192 12 72
256 14 112
use crypto::aes;

fn main() {
    // Replace these with your plaintext and key
    let plaintext: [u8; 16] = [
        0x32, 0x43, 0xf6, 0xa8, 0x88, 0x5a, 0x30, 0x8d, 0x31, 0x31, 0x98, 0xa2, 0xe0, 0x37, 0x07, 0x34,
    ];
    let key: [u8; 16] = [
        0x2b, 0x7e, 0x15, 0x16, 0x28, 0xae, 0xd2, 0xa6, 0xab, 0xf7, 0x97, 0x75, 0x46, 0x20, 0x63, 0xed,
    ];

    let expanded_key = expand_key(&key);
    let mut state: [[u8; 4]; 4] = [[0; 4]; 4];
    state.copy_from_slice(&plaintext);

    cipher(&mut state, &expanded_key);
    print_state(&state);
}

fn cipher(state: &mut [[u8; 4]; 4], ek: &[u8; 176]) {
    let rounds = 10;
    add_round_key(state, &ek[0..16]);

    for round in 1..rounds {
        sub_bytes(state);
        shift_rows(state);
        mix_columns(state);
        add_round_key(state, &ek[round * 16..(round + 1) * 16]);
    }

    sub_bytes(state);
    shift_rows(state);
    add_round_key(state, &ek[160..176]);
}

fn add_round_key(state: &mut [[u8; 4]; 4], round_key: &[u8; 16]) {
    for i in 0..4 {
        for j in 0..4 {
            state[i][j] ^= round_key[i * 4 + j];
        }
    }
}

fn sub_bytes(state: &mut [[u8; 4]; 4]) {
    for i in 0..4 {
        for j in 0..4 {
            state[i][j] = s_box(state[i][j]);
        }
    }
}

fn shift_rows(state: &mut [[u8; 4]; 4]) {
    for i in 1..4 {
        state[i].rotate_left(i);
    }
}

fn mix_columns(state: &mut [[u8; 4]; 4]) {
    for i in 0..4 {
        let a = state[0][i];
        let b = state[1][i];
        let c = state[2][i];
        let d = state[3][i];

        state[0][i] = gf_mul(0x02, a) ^ gf_mul(0x03, b) ^ c ^ d;
        state[1][i] = a ^ gf_mul(0x02, b) ^ gf_mul(0x03, c) ^ d;
        state[2][i] = a ^ b ^ gf_mul(0x02, c) ^ gf_mul(0x03, d);
        state[3][i] = gf_mul(0x03, a) ^ a ^ b ^ gf_mul(0x02, d);
    }
}

fn gf_mul(a: u8, b: u8) -> u8 {
    let mut result = 0;
    let mut a = a;
    let mut b = b;
    while b != 0 {
        if (b & 1) != 0 {
            result ^= a;
        }
        let high_bit_set = (a & 0x80) != 0;
        a <<= 1;
        if high_bit_set {
            a ^= 0x1b; // AES polynomial
        }
        b >>= 1;
    }
    result
}

fn s_box(byte: u8) -> u8 {
    // Replace this with the AES S-box lookup table
    // You can define the S-box as an array or implement it as a function
}

fn expand_key(key: &[u8; 16]) -> [u8; 176] {
    // Implement the key expansion to generate the round keys
    // You will need to return a 176-byte array with the round keys
    // The example key expansion logic has been omitted for brevity
    [0; 176]
}

fn print_state(state: &[[u8; 4]; 4]) {
    for i in 0..4 {
        for j in 0..4 {
            print!("{:02x} ", state[j][i]);
        }
        println!();
    }
}

The State

The state for the traditional AES algorithm is depicted as the 4x4 matrix to which the plaintext block is mapped. The block is copied to this matrix row by row; meaning that the first byte will be in column one, row one and the second byte will be in column one, row two. This can be seen in the following image where P denotes the plaintext array and \(\boxed{\text{Px}}\) denotes the byte at index \(\boxed{\text{x}}\) of the plaintext array:

16 Byte AES State:

\begin{bmatrix} \boxed{P1} & \boxed{P2} & \boxed{P3} & \boxed{P4} \\ \boxed{P5} & \boxed{P6} & \boxed{P7} & \boxed{P8} \\ \boxed{P9} & \boxed{P10} & \boxed{P11} & \boxed{P12} \\ \boxed{P13} & \boxed{P14} & \boxed{P15} & \boxed{P16} \end{bmatrix}

basic AES State

Key Expansion

The expanded key for AES is a list of four-byte words that are derived from the original key. The first words in the array are the same as the original key. For example, the first four bytes in the key array would be the first word in the expanded key, \(\boxed{\text{EK}}\), like so:

  let key: [u8; 16] = [
        0x2b, 0x7e, 0x15, 0x16, 0x28, 0xae, 0xd2, 0xa6, 0xab, 0xf7, 0x97, 0x75, 0x46, 0x20, 0x63, 0xed,
    ];

fn expand_key(key: &[u8; 16]) -> [u8; 176] {
    [0; 176]
}

Each additional word is the result of XORing the previous word by the word at index \boxed{\text(i - NK}}$ where \boxed{\text{i}}$ is the current index and \boxed{\text{NK}}$ is the number of four-byte words that are in the original cipher key (4, 6, or 8 depending on the number of bits in the key). When the current index is a multiple of \boxed{\text{NK}}$, the previous word is transformed before undergoing the XOR. This consists of the word being rotated to the left by one (i.e \boxed{\text{0xAABBCCDD}}$ becomes \boxed{\text{0xBBCCDDAA}}\(), then each byte being substituted by a value in AES's Substitution Box (S-Box)- a 256 byte array. This new four-byte word is then XORed by a "round constant", \boxed{\text{RCON}}\). The round constant, \boxed{\text{RCON}}$, can be computed from the algorithm: \(2i-1\) << 24 which, however, uses a Galois Field to preform multiplication instead of normal Base-10 multiplication. This type of multiplication is beyond the scope of this article though, read about it here. The round constants can also be hardcoded in with the following values for the standard AES-128 algorithm:

Index 0 1 2 3 4 5 6 7 8 9 10
Values None 0x01 0x02 0x04 0x08 0x10 0x20 0x40 0x80 0x1b 0x36

The entire AES-128 key exansion algorithm could look like the followoing:

const RCON: [u32; 11] = [0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1B, 0x36];

fn subword(word: u32, sbox: &[u8; 256]) -> u32 {
    let mut nw: u32 = 0;
    for i in 0..4 {
        let byte = ((word >> (i * 8)) & 0xFF) as u8;
        nw |= (u32::from(sbox[byte as usize])) << (i * 8);
    }
    nw
}

fn rotword(word: u32) -> u32 {
    let mut bytes: [u8; 4] = [0; 4];
    for i in 0..4 {
        bytes[i] = ((word >> (i * 8)) & 0xFF) as u8;
    }
    let first_byte = bytes[0];
    for i in 0..3 {
        bytes[i] = bytes[i + 1];
    }
    bytes[3] = first_byte;
    let mut nw: u32 = 0;
    for i in 0..4 {
        nw |= (u32::from(bytes[i])) << (i * 8);
    }
    nw
}

fn key_expansion(key: &[u8; 16], sbox: &[u8; 256]) -> Vec<u32> {
    let mut ek: Vec<u32> = Vec::with_capacity(44);
    for i in 0..4 {
        let b: [u8; 4] = [
            key[i * 4],
            key[i * 4 + 1],
            key[i * 4 + 2],
            key[i * 4 + 3],
        ];
        let w = u32::from_ne_bytes(b);
        ek.push(w);
    }
    for i in 4..44 {
        let tmp = ek[i - 1];
        let rcon_index = i / 4;
        if i % 4 == 0 {
            let rcon_val = (u32::from(RCON[rcon_index])) << 24;
            let sub_rot_word = subword(rotword(tmp), sbox);
            ek.push(sub_rot_word ^ rcon_val);
        } else {
            ek.push(tmp ^ ek[i - 4]);
        }
    }
    ek
}

fn main() {
    // Define your S-box here (you can replace this with your actual S-box).
    let sbox: [u8; 256] = [0; 256];
    let key: [u8; 16] = [0x2b, 0x7e, 0x15, 0x16, 0x28, 0xae, 0xd2, 0xa6, 0xab, 0xf7, 0x97, 0x75, 0x46, 0x20, 0x63, 0xed];
    let expanded_key = key_expansion(&key, &sbox);
    println!("{:x?}", expanded_key);
}

Substitution Box (S-Box)

The AES S-Box is a 256-byte hardcoded array. This array is used for the SubWord function in the Key Expansion algorithm, as well as SubByte functuin in the main cipher. The bytes in the S-Box are as follows:

1 2 3 4 5 6 7 8 9 A B C D E F
0x63 0x7C                          

Date: 2023-10-24 Tue 00:00