diff options
author | Carlos Santos <unixmania@gmail.com> | 2019-10-19 17:49:42 -0300 |
---|---|---|
committer | Waldemar Brodkorb <wbx@openadk.org> | 2019-10-30 10:27:52 +0100 |
commit | 01e863c89fc772a406fe56c6dddb39f71a570c06 (patch) | |
tree | 197cdfb0f9abe789b04283fbaca010e1f5d4a502 /include/ulimit.h | |
parent | 3f1c2ffd2fd13c368b7652f4ee66d35f65d3f369 (diff) |
Make __syscall_error return long, as expected by syscall() callers
The return type of syscall() is long so __syscall_error, which is jumped
to by syscall handlers to stash an error number into errno, must return
long too otherwhise it returs 4294967295L instead of -1L. For example,
syscall for x86_64 is defined in libc/sysdeps/linux/x86_64/syscall.S as
syscall:
movq %rdi, %rax /* Syscall number -> rax. */
movq %rsi, %rdi /* shift arg1 - arg5. */
movq %rdx, %rsi
movq %rcx, %rdx
movq %r8, %r10
movq %r9, %r8
movq 8(%rsp),%r9 /* arg6 is on the stack. */
syscall /* Do the system call. */
cmpq $-4095, %rax /* Check %rax for error. */
jae __syscall_error /* Branch forward if it failed. */
ret /* Return to caller. */
In libc/sysdeps/linux/x86_64/__syscall_error.c, __syscall_error is
defined as
int __syscall_error(void) attribute_hidden;
int __syscall_error(void)
{
register int err_no __asm__ ("%rcx");
__asm__ ("mov %rax, %rcx\n\t"
"neg %rcx");
__set_errno(err_no);
return -1;
}
So __syscall_error returns -1 as a 32-bit int in a 64-bit register, %rax
(0x00000000ffffffff, whose decimal value is decimal 4294967295) and a
test like this always returns false:
if (syscall(number, ...) == -1)
foo();
Fix the error by making __syscall_error return a long, like syscall().
The problem can be circumvented by the caller by coercing the returned
value to int before comparing it to -1:
if ((int) syscall(number, ...) == -1)
foo();
The same problem probably occurs on other 64-bit systems but so far only
x86_64 was tested, so this change must be considered experimental.
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Diffstat (limited to 'include/ulimit.h')
0 files changed, 0 insertions, 0 deletions