Kernel Level Protections: Supervisor Mode Execution Protection (SMEP) - Part I

Supervisor Mode Execution Protection, or SMEP is a kernel protection mechanism originally developed by Intel in 2011 for their x86 and amd64 architecture processors. It prevents code running in the context of the kernel from executing any code in userland memory.

Kernel Level Protections: Supervisor Mode Execution Protection (SMEP) - Part I
Photo by Gabriel Heinzer / Unsplash


Supervisor Mode Execution Protection, or SMEP is a kernel protection mechanism originally developed by Intel in 2011 for their x86 and amd64 architecture processors. It prevents code running in the context of the kernel from executing any code in userland memory. In other words, all userland memory is marked as non-executable.

Background and the Need for SMEP

In September 2003, AMD released the Athlon 64 based on the Clawhammer core (130nm, 2-2.6GHz). It was notable for a few reasons, chiefly being one the first 64-bit processor for consumer desktops. It was also the first processor with AMD’s *NX bit* in the page table. Processors could execute memory where the bit is set to 0, but otherwise could not. Intel quickly played catch up with their Pentium 4 based on the Prescott core in 2004. Though, they called their implementation eXecute Never (XD). I will continue calling the feature NX throughout the article in recognition of AMD’s efforts to bring this security feature to consumers.

Windows began supporting this hardware-enforced feature with their implementation, Data Execution Prevention (DEP), in 2005 with Windows XP SP2 and Windows Server 2003 SP1. Linux jumped on it a bit sooner in 2004 with support for both userland processes and the kernel.

The result is that it’s more difficult for attackers to send and rely on executing code. Previously (and maybe still the case if you look at IoT devices), attackers could send an exploit and a payload in one go, gain control of the instruction pointer, jump to a known address, and start executing off the stack. It doesn’t solve issues like return or jump-oriented programming where attackers can redirect the instruction pointer to existing locations of executable memory.

Enter SMEP

On the kernel front, NX still left a hole where the kernel could access and execute memory written to user space. That’s where SMEP comes in. SMEP uses a previously unused bit in the cr4 register. The cr4 register on the amd64 architecture is a 64-bit register where each bit corresponds to a flag relating to some control mode operation. Examples include debugging extensions, physical address extension (extends 32-bit addresses to 36-bit), and, yes, SMEP. SMEP occupies the 20th-bit field. When the SMEP bit is set to one, instructions cannot be fetched from user-mode addresses. If attempted, a page fault is thrown.

While not bulletproof, this mitigation would have likely prevented the privilege escalation piece of Stuxnet, CVE-2010-2743. The vulnerability was in a Windows keyboard driver, which operated in kernel mode. The driver used a function jump table as a handler for different events. The index into that jump table was user-controlled. As a result (and with a lack of bounds checking), an attacker could supply an index out of the bounds of the jump table and execute code in userland. This resulted in code execution with kernel privileges. Not great. With SMEP, the userland page would have been non-executable, and a page fault would have been thrown.

SMEP’s Rollout to Users

The idea for SMEP first appeared in a Black Hat talk from 2008 titled “Detecting and Preventing Xen Hypervisor Subversions” by Joanna Rutkowska from Invisible Things Labs. Since the idea is so similar to NX, the name was actually pitched as NX+/XD+ (again, with the name fragmentation). Intel’s Fenghua Yu implemented the patch in 2011, bringing it into the Linux kernel. Microsoft added support by default with Windows 8.0, launched in 2012. The first CPUs supporting the feature appeared the following year, in April 2012, with Intel’s launch of the Ivy Bridge core (22nm). AMD added support much later with the fourth generation Excavator line, released in 2015. Pretty big gap there.

Outside of the amd64 landscape, arm64 gained the SMEP-equivalent feature called Privileged eXecute Never (PXN) in 2012. At the same time, they took it as a great time to rename the eXecute Never (XN, the same as NX/XD on amd64) bit to User eXecute Never (UXN). While the names are fragmented, it’s at least nice to have a consistent naming scheme between user and privileged modes (UXN & PXN vs. NX & SMEP).

Bypassing SMEP

My first naive thought regarding bypassing SMEP went something like, “If we’re already running in kernel mode, can’t we inline some assembly to modify the cr4 register at run time?” Nope, it turns out you can’t change the register value because it’s pinned. Look at the following kernel module loaded with sudo insmod main.ko.

int __init my_init_module(void) {
	struct cpuinfo_x86 *cpu;
	/* Get the first CPU. */
	cpu = &cpu_data(0);

	printk(KERN_INFO "[+] CPU Model: %d\\n", cpu->x86_model);
	/* Detect SMEP. */
	if (cpu_has(cpu, X86_FEATURE_SMEP)) 
        printk(KERN_INFO "[+] SMEP detected.\\n");
        printk(KERN_INFO "[-] SMEP not detected.\\n");
	/* Check the CR4 flags. */
	printk(KERN_INFO "[+] CR4: %lx\\n", __read_cr4());
	/* Toggle SMEP. */
	__write_cr4(__read_cr4() ^ X86_CR4_SMEP);
	/* Check the flags after the alteration. */
	printk(KERN_INFO "[+] CR4: %lx\\n", __read_cr4());
	return 0;

After loading the kernel module, you can inspect the logs with dmesg.

[68581.597175] [+] CPU Model: 63
[68581.597180] [+] SMEP detected.
[68581.597181] [+] CR4: 1706e0
[68581.597183] ------------[ cut here ]------------
[68581.597184] pinned CR4 bits changed: 0x100000!?
[68581.597187] WARNING: CPU: 6 PID: 56618 at arch/x86/kernel/cpu/common.c:455 native_write_cr4+0x79/0x80

<<Long stack trace ommited>

---[ end trace 0000000000000000 ]---
[68581.597404] [+] CR4: 1706e0

Note the complaint that the CR4 bits have changed. It turns out that along with SMAP, UMIP, FSGSBASE, and CET, SMEP cannot be overwritten through just a simple function call. They are pinned such that they cannot be cleared after boot. If any of the pinned bits have changed, a warning is thrown. Womp. Instead, we’ll have to think a bit harder.

One thought is that since NX and SMEP share similar backgrounds, the same bypasses may work - ROP. This turns out to be exactly the case. We can use ROP to call the kernel functions prepare_kernel_cred(0) to obtain credentials for a different context, 0 for root. We can then ROP to commit_creds() to install the credentials for the currently running task. In effect, elevating privileges. It seems easy, but you have to remember about KASLR. Leaking these addresses is its own headache.

I’ll leave the actual exercise for bypassing SMEP using ROP in a future article, but in the meantime, check out these two great write-ups that probably do a better job than I ever could.

What do you think? Were you aware of some of the kernel-level exploit mitigations?

Concluding Thoughts

Given its userland predecessor, NX/XD/DEP, it’s a little clearer how SMEP emerged as a supervisory mode protection mechanism. It doesn’t mean that code execution, even code stored in a lower privileged space, is impossible. Rather, it’s another layer of protection over other security mechanisms like Supervisor Mode Access Protection (SMAP) and Kernel Address Space Layout Randomization (KASLR). I hope to explore these mechanisms with you in a future blog post.

This article, and others, are available for free at and on Medium. Questions or comments? Let’s chat on LinkedIn!

Subscribe to Sean Deaton

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.