Using C and Assembly code: 64Bit Calling convention

[polemon]

Hello, I'm trying to merge Assembly code and C code in a program for self study.

This is the assembly file (square.s):

--- SOF ---
; square.s - return int value
section .text
type square function
global square

square:
mov [rsp + 8], rcx
push r15
push r14
push r13
sub rsp, 16
lea r13, [rsp + 128]

imul rcx, rcx
mov rax, rcx

lea rsp, [r13 - 128]
add rsp, 16
pop r13
pop r14
pop r15
ret
--- EOF ---

And this is the C-code (inttest.c):
--- SOF ---
/* inttest.c - return int value from ASM */

#include <stdio.h>

int main() {
printf("The square of 2 is %d\n", square(2));
printf("The square of 10 is %d\n", square(10));
printf("The square of 9 is %d\n", square(9));

return 0;
}
--- EOF ---

I use YASM to assemble square.s:

yasm -ax86 -felf64 square.asm

And I use gcc to compile and link the rest of the program:

gcc -o inittest inttest.c square.o

As you can see in the code above, the called function in assembly, should calculate the square of the passed number and return the result.
However, this is what I get:

$ ./inittest
The square of 2 is 0
The square of 10 is 1
The square of 9 is 1

According to 64Bit calling conventions, RCX should contain the first integer argument, and RAX the return value.
(See: http://www.osronline.com/DDKx/kmarch/64bitamd_6wmf.htm)

What am I doing wrong?

Also, please clarify: how do I get the function to return non-integer values?

Regards,
--polemon
--
echo "cby...@cbyrzba.bet" | tr '[a-z]' '[n-za-m]'

[Noob]

polemon wrote:

> I'm trying to merge Assembly code and C code in a program for self study.

You're in the wrong group. You want comp.lang.asm.x86

cf. also http://www.x86-64.org/documentation.html