How to Solve a Crackme Step by Step: Crackme 001 in x64dbg

Author: Diego J D Arias · Platform: Windows x64 · Tools: x64dbg (IDA optional) · Level: beginner Download the CrackMe In this tutorial we solve the crackme without knowing the key beforehand, by trying things, making mistakes and verifying every assumption with the debugger. The idea is that you can follow along with the binary open and, by the end, walk away with a method that works for other crackmes.

What you need: the crackme, x64dbg and a calculator.

Important: Always work in an isolated virtual machine (host-only or disconnected network) with a snapshot that includes memory.

Looking at the program before opening the debugger

We run the crackme and a window appears with four text boxes and a Register button. Obviously, the key is split into four parts.

Let’s try values that will be easy to recognize later inside the code, for example 777777777 888888888 999999999 111111111

Why those? Because if we later see a 7777 or a 1111 in a register, we know which box it came from. It’s best to avoid values like 1234 repeated in every box, since they would be hard to tell apart.

If nothing visible happens when you press Register, or an error message appears, we have to find out what is being checked.

Tip: Before debugging, we should try the program and write down what we see. It gives you strings, field sizes and messages that we can later use to search for references.

How to get to the validation function

We open the crackme in x64dbg. There are two quick ways to reach the function that validates the key:

• From the strings: right click → Search for → Current Module → String references. We look for the text of the success message, follow the reference and go up to the code that decides the result.
• From the input API: we set a breakpoint on GetDlgItemTextW, the Windows function that reads the text of a box, and when it stops, we look at who called it.

In this case, we will search for the string Failed.

Here we see two interesting strings, Failed and Success. Let’s go to Success and see what’s there.

00007FF735B6183E | FF15 EC180000         | call qword ptr ds:[<DefWindowProcW>]   |
00007FF735B61844 | E9 8A000000           | jmp crackme.001.7FF735B618D3           |
00007FF735B61849 | E8 C2F9FFFF           | call crackme.001.7FF735B61210          |
00007FF735B6184E | 45:33C9               | xor r9d,r9d                            |
00007FF735B61851 | 4C:8D05 201C0000      | lea r8,qword ptr ds:[7FF735B63478]     | 00007FF735B63478:L"Registration Result"
00007FF735B61858 | 33C9                  | xor ecx,ecx                            |
00007FF735B6185A | 84C0                  | test al,al                             |
00007FF735B6185C | 74 0F                 | je crackme.001.7FF735B6186D            |
00007FF735B6185E | 48:8D15 3B1C0000      | lea rdx,qword ptr ds:[7FF735B634A0]    | 00007FF735B634A0:L"Success"
00007FF735B61865 | FF15 3D180000         | call qword ptr ds:[<MessageBoxW>]      |
00007FF735B6186B | EB 64                 | jmp crackme.001.7FF735B618D1           |
00007FF735B6186D | 48:8D15 3C1C0000      | lea rdx,qword ptr ds:[7FF735B634B0]    | 00007FF735B634B0:L"Failed"
00007FF735B61874 | FF15 2E180000         | call qword ptr ds:[<MessageBoxW>]      |

If we look a little further up, we see a je crackme.001.7FF735B6186D that decides between Success and Failed. A little above that there is a call crackme.001.7FF735B61210, which is the one we are going to analyze. From here on we will work with the addresses of this executable (yours may differ if the module is loaded at another base).

Tip: don’t read the program from the beginning. Start from the ends: the API that reads the input or the message that reports the result.

Inside the function, the first thing we see is the prologue.

00007FF735B61210 | 48:8BC4               | mov rax,rsp                            |
00007FF735B61213 | 48:8958 10            | mov qword ptr ds:[rax+10],rbx          |
00007FF735B61217 | 48:8970 18            | mov qword ptr ds:[rax+18],rsi          |
00007FF735B6121B | 48:8978 20            | mov qword ptr ds:[rax+20],rdi          |
00007FF735B6121F | 55                    | push rbp                               |
00007FF735B61220 | 41:54                 | push r12                               |
00007FF735B61222 | 41:55                 | push r13                               |
00007FF735B61224 | 41:56                 | push r14                               |
00007FF735B61226 | 41:57                 | push r15                               |
00007FF735B61228 | 48:8D68 A1            | lea rbp,qword ptr ds:[rax-5F]          |
00007FF735B6122C | 48:81EC A0000000      | sub rsp,A0                             |
00007FF735B61233 | 0F2970 C8             | movaps xmmword ptr ds:[rax-38],xmm6    |
00007FF735B61237 | 48:8B05 CA3D0000      | mov rax,qword ptr ds:[7FF735B65008]    |
00007FF735B6123E | 48:33C4               | xor rax,rsp                            |

All of that is compiler noise: it saves registers, builds the stack frame and sets the stack cookie (the /GS protection). There is only one useful clue: movaps xmmword ptr ds:[rax-38],xmm6 . xmm6 is saved because the function is going to use floating-point numbers later on. We take note of it and move on.

We also see a GetDlgItem and a GetWindowTextLength, but the result is overwritten right away with xor eax,eax: it isn’t used. Noise as well.

00007FF735B61245 | 48:8BD9               | mov rbx,rcx                            |
00007FF735B61248 | BA 79000000           | mov edx,79                             | 79:'y'
00007FF735B6124D | 48:8B0D A43E0000      | mov rcx,qword ptr ds:[7FF735B650F8]    |
00007FF735B61254 | FF15 961E0000         | call qword ptr ds:[<GetDlgItem>]       |
00007FF735B6125A | 48:8BC8               | mov rcx,rax                            |
00007FF735B6125D | FF15 551E0000         | call qword ptr ds:[<GetWindowTextLengt |
00007FF735B61263 | 33C0                  | xor eax,eax                            |

Tips: we should always recognize the noise and skip it. The prologue, the cookie and discarded results don’t count.

The interesting part: how the input is read

This is where it gets interesting. This pattern appears four times:

00007FF735B61265 | 48:8945 D7            | mov qword ptr ss:[rbp-29],rax          |
00007FF735B61269 | 66:8945 DF            | mov word ptr ss:[rbp-21],ax            |
00007FF735B6126D | 44:8D48 05            | lea r9d,qword ptr ds:[rax+5]           | cchMax = 5
00007FF735B61271 | 4C:8D45 D7            | lea r8,qword ptr ss:[rbp-29]           | buffer where the text is stored
00007FF735B61275 | 8D50 79               | lea edx,qword ptr ds:[rax+79]          | control ID (0x79)
00007FF735B61278 | 48:8BCB               | mov rcx,rbx                            | window handle
00007FF735B6127B | FF15 671E0000         | call qword ptr ds:[<GetDlgItemTextW>]  |
00007FF735B61281 | 83F8 04               | cmp eax,4                              | if it returned fewer than 4 characters -> fail
00007FF735B61284 | 0F82 0B050000         | jb crackme.001.7FF735B61795            |

We read the arguments of GetDlgItemTextW(hDlg, nIDDlgItem, lpString, cchMax), which on x64 are passed in rcx, rdx, r8 and r9. And we draw three conclusions:

1. cchMax = 5: at most 4 characters plus the null terminator are read. Even though we typed 9 digits, the program only sees the first 4. Our 777777777 actually comes in as 7777.
2. cmp eax,4 / jb: if the box returns fewer than 4 characters, it jumps to the failure path. Each box needs exactly 4 useful characters (more doesn’t matter).
3. The four controls have IDs 0x79, 0x7C, 0x7A and 0x7B, in that reading order:

Watch out for something: the reading order doesn’t have to match the visual order of the boxes. We leave that question pending and check it shortly.

Tip: read the arguments of the APIs. That’s where restrictions show up (maximum lengths, IDs, etc.).

The conversion to a number looks like a maze!

After the texts are read, the code becomes long and confusing. Loops appear, comparisons with 7FFFFFFFFFFFFFFE, a memcpy, calls to unknown functions and blocks that seem to free memory. Before getting scared, it’s worth asking ourselves a question:

00007FF735B61320 | 48:FFC3               | inc rbx                                |
00007FF735B61323 | 66:393C58             | cmp word ptr ds:[rax+rbx*2],di         |
00007FF735B61327 | 75 F7                 | jne crackme.001.7FF735B61320           |
00007FF735B61329 | 49:BE FEFFFFFFFFFFFF7 | mov r14,7FFFFFFFFFFFFFFE               |
00007FF735B61333 | 49:3BDE               | cmp rbx,r14                            |
00007FF735B61336 | 0F87 93040000         | ja crackme.001.7FF735B617CF            |
00007FF735B6133C | 41:BC 0A000000        | mov r12d,A                             | 0A:'\n'
00007FF735B61342 | 48:C745 CF 07000000   | mov qword ptr ss:[rbp-31],7            |
00007FF735B6134A | 48:83FB 08            | cmp rbx,8                              |
00007FF735B6134E | 73 1E                 | jae crackme.001.7FF735B6136E           |
00007FF735B61350 | 48:895D C7            | mov qword ptr ss:[rbp-39],rbx          |
00007FF735B61354 | 48:03DB               | add rbx,rbx                            |
00007FF735B61357 | 4C:8BC3               | mov r8,rbx                             |
00007FF735B6135A | 48:8D55 D7            | lea rdx,qword ptr ss:[rbp-29]          |
00007FF735B6135E | 48:8D4D B7            | lea rcx,qword ptr ss:[rbp-49]          |
00007FF735B61362 | E8 FB180000           | call <JMP.&memcpy>                     |
00007FF735B61367 | 66:897C1D B7          | mov word ptr ss:[rbp+rbx-49],di        |
00007FF735B6136C | EB 4E                 | jmp crackme.001.7FF735B613BC           |
00007FF735B6136E | 4C:8BFB               | mov r15,rbx                            |
00007FF735B61371 | 49:83CF 07            | or r15,7                               |
00007FF735B61375 | 4D:3BFE               | cmp r15,r14                            |
00007FF735B61378 | 76 05                 | jbe crackme.001.7FF735B6137F           |
00007FF735B6137A | 4D:8BFE               | mov r15,r14                            |
00007FF735B6137D | EB 07                 | jmp crackme.001.7FF735B61386           |
00007FF735B6137F | 4D:3BFC               | cmp r15,r12                            |
00007FF735B61382 | 4D:0F42FC             | cmovb r15,r12                          |
00007FF735B61386 | 49:8D57 01            | lea rdx,qword ptr ds:[r15+1]           |
00007FF735B6138A | 48:8D4D B7            | lea rcx,qword ptr ss:[rbp-49]          |
00007FF735B6138E | E8 9D050000           | call crackme.001.7FF735B61930          |
00007FF735B61393 | 48:8BF8               | mov rdi,rax                            |
00007FF735B61396 | 48:8945 B7            | mov qword ptr ss:[rbp-49],rax          |
00007FF735B6139A | 48:895D C7            | mov qword ptr ss:[rbp-39],rbx          |
00007FF735B6139E | 4C:897D CF            | mov qword ptr ss:[rbp-31],r15          |
00007FF735B613A2 | 48:03DB               | add rbx,rbx                            |
00007FF735B613A5 | 4C:8BC3               | mov r8,rbx                             |
00007FF735B613A8 | 48:8D55 D7            | lea rdx,qword ptr ss:[rbp-29]          |
00007FF735B613AC | 48:8BC8               | mov rcx,rax                            |
00007FF735B613AF | E8 AE180000           | call <JMP.&memcpy>                     |

What does this block do with the data that matters?

If we look only at the essentials, the pattern is this:

00007FF735B613BC | 45:8BC4               | mov r8d,r12d                           | r8d = 10 (the base)
00007FF735B613BF | 33D2                  | xor edx,edx                            | edx = 0
00007FF735B613C1 | 48:8D4D B7            | lea rcx,qword ptr ss:[rbp-49]          | rcx = a temporary string
00007FF735B613C5 | E8 D6050000           | call crackme.001.7FF735B619A0          | <-- this returns a number in eax
00007FF735B613CA | 44:8BE8               | mov r13d,eax                           | the result is stored in r13d

With arguments (string, 0, 10) and a number as the return value, this smells like std::stoi(string, nullptr, 10): converting text to an integer in base 10. This is an inference, not a confirmed fact, so we will verify it later.

Everything else in the block is bookkeeping for a C++ std::wstring:

• the inc rbx / cmp word [...],0 / jne loop is an inline wcslen,
• 7FFFFFFFFFFFFFFE is the string’s max_size(),
• the comparison with 8 decides between storing the text in the internal buffer (SSO) or allocating memory on the heap, and with 4 characters it always goes through the internal buffer,
• the block with cmp rdx,1000 / add rdx,27 / cmp rax,1F is the string’s destructor, which gets skipped with 4 characters,
• the call [_invalid_parameter_noinfo_noreturn] followed by an int3 is an error path that should never run.

And something very helpful shows up: the same block repeats four times, once per box. So we understand it once, and for the rest we only look at what changes: the source buffer and the register where the result lands.

These four registers don’t get overwritten afterwards: that’s the sign that they are going to be used. We call them A, B, C and D according to the control ID: A = 0x79, B = 0x7A, C = 0x7B, D = 0x7C.

Tip: when a block repeats, understand it only once. And the registers that survive after a call are the data that will matter later.

Verifying with the debugger.

This is where it’s best to stop deducing and start looking. We set a breakpoint on the instruction that follows the second call 19A0, at 7FF7499E14C2, and run. With the input from step 1, we expected to see the value of the second box in eax (8888, which in hexadecimal is 0x22B8).

We see something else: eax = 457.

Remember that x64dbg shows registers in hexadecimal: 0x457 = 1111. In other words, the second number the program reads doesn’t come from our second box, but from the fourth one (the one that had 111111111).

Two things are confirmed by this:

• The function is stoi in base 10, because the result of converting "1111" was 1111.
• The reading order is not the visual order: ID 0x7C corresponds to the fourth box.

This leaves us with a partial mapping. We are sure about ID 0x7C; for the other three we can only assume, since they are consecutive IDs (0x79, 0x7A, 0x7B), that they follow the visual order. We note it down as a pending assumption.

Tip: a deduction is not a fact until it is confirmed with a breakpoint. A single look at eax right after a call is worth more than ten minutes of interpreting instructions.

Let’s find the validation, which is what really matters

We skip the whole destructor and arrive at 7FF735B61700. And here the type of code changes: there is no more string handling, there is floating-point arithmetic. This is the part that decides.

This nine-instruction pattern appears three times:

00007FF735B61700 | 6641:0F6EF5           | movd xmm6,r13d                         | the user's number, as an int
00007FF735B61705 | F3:0FE6F6             | cvtdq2pd xmm6,xmm6                     | converts it to double
00007FF735B61709 | F2:0F100D C71D0000    | movsd xmm1,qword ptr ds:[7FF735B634D8] | xmm1 = exponent
00007FF735B61711 | F2:0F1005 C71D0000    | movsd xmm0,qword ptr ds:[7FF735B634E0] | xmm0 = base
00007FF735B61719 | E8 68150000           | call <JMP.&pow>                        | xmm0 = pow(base, exponent)
00007FF735B6171E | F2:0F5905 A21D0000    | mulsd xmm0,qword ptr ds:[7FF735B634C8] | xmm0 = xmm0 * constant
00007FF735B61726 | 66:0F2EF0             | ucomisd xmm6,xmm0                      | user == calculated?
00007FF735B6172A | 7A 69                 | jp crackme.001.7FF735B61795            |
00007FF735B6172C | 75 67                 | jne crackme.001.7FF735B61795           | if it doesn't match -> fail

How to read it:

• The user’s number goes into xmm6, which survives the call to pow (that’s why the prologue had saved it).
• On Windows x64, pow(double, double) receives the base in xmm0 and the exponent in xmm1.
• ucomisd compares two doubles and jp/jne jump to fail if they are not exactly equal.

To solve it we need the values of those constants.

Let’s read the constants and see what they tell us.

In x64dbg: Ctrl+G → 7FF735B634C0. In the dump, right click → Float → Double. The dump shows two values per row:

Now we have to do the math

We solve each condition with the calculator, paying close attention to which is the base and which is the exponent:

| Field          | Calculation                       | Result    |
|----------------|-----------------------------------|-----------|
| A (0x79, r13d) | pow(8, 4) × 0.25  = 4096 × 0.25   |    1024   |
| B (0x7A, r15d) | pow(8, 4) × 0.5   = 4096 × 0.5    |    2048   |
| D (0x7C, r12d) | pow(4, 8) × 0.125 = 65536 × 0.125 |    8192   |
| C (0x7B, ebx)  | cmp ebx,200 → 0x200               |    512    |

There is a detail that must not be overlooked: in condition D, the base and the exponent are swapped compared to A and B (34D8 and 34E0 are loaded in the opposite order). That’s why the result is 65536 and not 4096.

The last condition is the simplest: cmp ebx,200 followed by sete al. If everything matches, al = 1 (correct key); if something failed, the jump to 7FF735B61795 does xor al,al and returns 0.

Tip: the real logic is whatever touches the user’s data: comparisons, arithmetic and constants. When the code only checks against a constant expression, the key is obtained by solving the expression: there is nothing to “reverse”.

Now let’s build the key and test it

We have the four values. We just need to remember the restriction from step 4: each box needs at least 4 characters, and 512 has only 3. We write it with a leading zero: 0512. stoi("0512", 10) returns 512, so it works.

Under the assumption that the IDs follow the visual order:

We enter them and press Register:

“Registration Result: Success” appears. The assumption about the order was correct.

If it had failed, the fallback plan was simple: leave 8192 in the fourth box (already confirmed) and try the 6 permutations of 1024, 2048 and 0512 in the first three. That’s very few combinations.

Summary: the path in one line

Try the program → reach the function through the input API or the success message → skip the noise → read the input restrictions → understand the repeated conversion template → verify with a breakpoint → solve the final comparison with the constants → test.

Method for the next crackme

1. Try the program with recognizable values.
2. Start from the ends (input API or success string), not from the beginning.
3. Recognize and skip the compiler noise.
4. Read the API arguments: the restrictions are there.
5. Repeated blocks are understood once; after that you only look at what changes.
6. Follow the registers that survive: they are the important data.
7. Verify with breakpoints instead of trusting a deduction.
8. When you only have to satisfy a comparison, solve the expression.

Ideas to practice

• Write a keygen in Python (here it’s trivial because the key is fixed, but it builds the habit).
• Patch sete al (0F 94 C0) to mov al,1 so it always returns success.
• Try the success-string shortcut on the next crackme, without help, and compare how much road it saves you.