Calling c functions in x86_64 assembly, nasm

[DarkLightning7]

So ive been teaching myself x86 over the last couple days now im trying to call c functions and am getting segmentation faults. if someone could explain how to call them properly that would be great.

here is my code:
print integer is just he control string %d as i am trying to read and wrint integers

        lea rax, print_int
        push rax
        push 10
        push 0
        call fgets
        pop rax
        pop rax
        pop rax

on a seperate note here is my attempt to use atoi:

        lea rax, x
        push rax
        call atoi

should i be adjusting the stack after calling functions?
what is the proper push order for fgets atoi and printf?
where do atoi and fgets place their results?
do pop and push automaticly adjust the stack?

Thanks

[Ancient Dragon]

Read this article. It was written for 32-bit programming but the way to call c functions should be the same. Also make sure your progrzm links with the 64-bit c runtime libraries instead of 32-bit libraries.

do pop and push automaticly adjust the stack?

Maybe -- after retuning from c library function you may or may not need to pop the registers that you pushed. It depends on the c libraries you link to, libraries produced by Microsoft compilers may not be the same as those produced by Borland or other compilers. When you use assembly languaage you have to check the documentation of the libraries you want to use to find out how to clean up the stack after calling one of their functions. Here and here are brief articles about calling conventions for Microsoft compilers. I don't know about gcc or Borland compilers.

[DarkLightning7]

Thanks for the reply. As far as I can tell after reading a few articles is you always need to clean the stack after calling a c function. Im still getting a segmentation fault when I try to call fgets or atoi and printf fails to do anything. I am using gcc for linking and am on 64bit ubuntu. I have the assembling down, and my program runs after linking as long as i dont call c functions. If i do i get segmentation faults i have tried with and without the stack cleanup with no change.

push 0          ;push standard in
push 100        ;push maximum length to read
push string     ;push the store location
call fgets      ;call c function fgets
add rsp, 24     ;clean the stack

push  string    ;push the string to convert
call atoi       ;call c function atoi
pop r8          ;save the number for later
add rsp, 8      ;clear the stack

push r8         ;push value to print
push print_int  ;push format string
call printf     ;call c function printf
add rsp, 16     ;clear the stack

Here is the beginging of the program:

extern printf
extern atoi
extern fgets

[section .data]
print_int:    db "%d"
print_int_l:  $-print_int

[section .bss]
string:       resb 100

[section .text]
global _start:
_start:

assembly command:
nasm -f elf64 test.asm
linking command
gcc -nostartfiles -o test test.o

Thanks for the help.

[Ancient Dragon]

One possible problem is that the instruction push 0 pushes a 32-bit value onto the stack, not a 64-bit value. To test that out, store the value to be pushed into a register then push the register.

mov rax,0
push rax
mov rax,100
push rax
mov rax,string
push rax
call fgets
add rsp, 24 

[DarkLightning7]

Thanks for the quick response, I still get a segmentation fault though.

As far as i know push in x86_64 only pushes 64-bit values. Ive attempted to push 32bit registers and the assembler said not allowed in 64-bit mode.

[DarkLightning7]

ok so i removed the call fgets line and the program ran fine so there is something wrong with how im calling it or gcc is not linking it properly.

[Assembly Guy]

Could you disassemble your elf64 with objdump -d test.o? Also, try installing gdb, the GNU Debugger. I think you said you're using Ubuntu, so sudo apt-get install gdb should work. GDB provides you with the ability to run your program in such a way that you can perform debugging techniques such as stack traces.

[DarkLightning7]

Here is the objdump im not too familiar with x86 yet so some of it is mandarin to me :)

Disassembly of section .text:

0000000000000000 <_start>:
   0:   48 b8 00 00 00 00 00    movabs $0x0,%rax
   7:   00 00 00 
   a:   50                      push   %rax
   b:   48 b8 64 00 00 00 00    movabs $0x64,%rax
  12:   00 00 00 
  15:   50                      push   %rax
  16:   48 b8 00 00 00 00 00    movabs $0x0,%rax
  1d:   00 00 00 
  20:   50                      push   %rax
  21:   e8 00 00 00 00          callq  26 <_start+0x26>
  26:   48 b8 01 00 00 00 00    movabs $0x1,%rax
  2d:   00 00 00 
  30:   48 bb 00 00 00 00 00    movabs $0x0,%rbx
  37:   00 00 00 
  3a:   cd 80                   int    $0x80

The code it came from:

extern printf
extern fgets
extern atoi

[section .data]
        print_int:      db "%d"
        print_int_l:    equ $-print_int

[section .bss]
        string:         resb 100

[section .text]
global _start:
_start:

mov rax, 0
push rax
mov rax, 100
push rax
mov rax, string
push rax
call fgets

mov rax, 1
mov rbx, 0
int 80h

I'm not sure how to use the gdb debugger, but I will read on it. Just thougt about it not sure why im using fgets because there is a system call for that however atoi has the same problem.

Edit: atoi actualy does not cause a segmentation fault but printf does when push just a string and does nothing when i push a format string and a number.

[Ancient Dragon]

atoi() does not write to a buffer that you created like fgets() so that's probably why atoi(( works and fgets() doesn't.

[Assembly Guy]

Hang on, your disassembly shows:

21: callq 26 <_start+0x26>
26: movabs $0x1,%rax
30: movabs $0x0,%rbx
3a: cd 80 int $0x80

So isn't the code at 0x21 just calling the next instruction, instead of fgets? Correct me if I'm wrong.

[DarkLightning7]

ok i did a little searching and on the line with callq the important part is the 00 00 00 00. This is where the address of the called procedure0 will be placed when it is determined since the memory address of the procedures might be different on each run. At least that is my understanding of it.