building a system call..

     Here we’ll be building a new system call for a 64-bit Ubuntu OS, to which we pass 2 summands and have it return the result. We’ll add this system call to the Linux kernel source, then build and install it. In this example I’ll be using kernel version 3.17.1, but it is always preferable to be working with the latest stable kernel verison if possible. At the time of this writing, the latest stable version is v3.17.4.

     We’ll need to implement our system call code, modify some kernel source files, and then compile and install the kernel. Once we have rebooted into the new kernel, we’ll make use of a helper program that will invoke the newly added system call.

There is no single/simple method of doing so, but regardless of the procedure, we need to:

     Once we have that out of the way, we can proceed to modifying the kernel source code. For this example, we’ll be modifying the kernel version 3.17.1 source code. Out system call will not only add 2 summands, it will also pront out a message to the kernel ring buffer, which we can then consult using the dmesg command in a terminal.

Let’s start by setting-up our system call directory:

cd /usr/src/
sudo mkdir addwm && cd addwm

     Now we can use any editor, but for this example I’ll use vim. Let’s call the file addwm.c:

sudo vim addwm
Inside the vim editor, we’ll add the following code:

/* addwm.c systemcall code */
#include <linux/kernel.h>
/* asmlinkage indicates we use the kernel stack to pass parameters */
asmlinkage long sys_addwm(int i, int j)
        printk(KERN_INFO "Addwm is working! Now adding %d and %d", i, j); 
        return i+j;

     Once done, we save and quit vim. Now before we venture away from this directory, we can also create the Makefile while we’re at it:

sudo vim Makefile
and add the following content to it:
obj-y := addwm.o

     Essentially, we’re telling make that this is going to be built-in with obj-y. obj-m would have denoted a module, but that’s for another entry in the blog.

     Now that we’ve tackled that part, we need to make some modifications to the main Makefile so that it knows where to look for “things” to build during compilation. It’s quite a large file, so if we’d like to know whre it is more or less we can issue the following command:

grep -n "kernel/ mm/" /user/src/linux-3.17.1/Makefile

     This will give us the line number where we need to make the change. To do so, we edit the kernel’s Makefile as follows:

sudo vim /usr/src/linux-3.17.1/Makefile

and we can then go to the line number (Esc+:+xxx, where xxx is the line number) change the following line of code:

core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/
core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ addwm/

     We could also use sed to replace it directly, but this seems safer as we have the time to inspect the occurences first. Once the line is modified, we save and exit. We’re almost there, we now need to add our system call to the 64-bit system call table, and also add the prototype function to the systm call header file.

     Because our helper program will invoke the system call using the syscall() function, we need to get the system call number so that the syscall() function knows which function to execute from the system call table. The system calls in the table are numbered, and as we add ours to the end of the list, we need to increment that number, and retain for later use in our hepler program.

     Let’s start with the adding our system call to the 64-bit system call table:

sudo vim /usr/src/linux-3.17.1/arch/x86/syscalls/syscall_64.tbl

..we can hit Ctrl+End to go to the end of the file, and add the following line:

321 64 addwm sys_addwm

where the last line used to be numbered 320, so here we’ve increented the number to 321 for the new system call. Once the line is added, we take note of the number 321, save and exit.

     Now We’ll add the prototype function to the end of the system call header file, right before the last #endif statement:

sudo vim /usr/src/linux-3.17.1/include/linux/syscalls.h

..we can hit Ctrl+End to go to the end of the file, and add the following line:

asmlinkage long sys_addwm(int i, int j);

note: we prefixed our system call with sys_.

     We’re now done editing the kernel source code and are ready to configure, compile, and install it! Instructions for doing so can be found here.

note: We need to compile and install our new kernel, and then boot it for our system call to work. We can still implement our hepler program ahead of time, but it will not work until we have our new kernel. Keep following the instructions for the helper program, and come back to the kernel compilation section when done with the helper program.

     We will now implement the helper program that will call our system call. We do so in the user’s home directory for instance, as there is no reason why it should be in the kernel source directory. We can create a directoy called programs/ in the user’s directory and place our helper program code there:

mkdir ~/programs/ & cd ~/programs/

     We can now use vim to implement our helper program:

vim addwm-main.c

and add the following code to the file:

/* integer validation modified from code found at:
 1 -
#include <stdio.h>
#include <ctype.h>
#include <syscall.h>
int main(int argc, char *argv[]){
    int var1, var2 = 0;
    long res = 0;
    char term;
    printf("Please enter 2 space separated numbers and hit the enter key..n:");
    if(scanf("%d%d%c", &var1, &var2, &term) != 3 || term != 'n'){
      printf("Invalid input, please try again.n");
    else {
      /* remember, our syscall number is 321,
         so we invoke it using its number.  */
      res = syscall(321, var1, var2);
      printf("%s: You entered %d and %d. Result is: %ldn", argv[0], var1, var2, res);
    return 0;

then save and exit. Although we could create a Makefile, we could also simply compile the helper program directly with the following command:

gcc -o addwm-main add-main.c

     This will produce the file addwm-main. If the kernel hasn’t been configured and compiled yet at this point, do so following the instructions here, then comback and execute the helper program as follows:

user1@VM:~/programs$ ./addwm-main
Please enter 2 space separated numbers and hit the enter key..
:934 765
./addwm-main: You entered 934 and 765. Result is: 1699