calloc.c: Avoid multiplication overflow

[odiferousmint]

Hello. I just stumbled upon this project and this particular file[1] by accident, but I saw a comment about wanting to avoid multiplication overflow, so you could do either:

if (num && SIZE_MAX / num < size) {
  errno = ENOMEM;
  return NULL;
}

which you would put at line 13, above size *= nmemb, or you do what reallocarray does:

/*
 * This is sqrt(SIZE_MAX+1), as s1*s2 <= SIZE_MAX
 * if both s1 < MUL_NO_OVERFLOW and s2 < MUL_NO_OVERFLOW
 */
#define MUL_NO_OVERFLOW ((size_t)1 << (sizeof(size_t) * 4))

void *calloc(size_t nmemb, size_t size)
{
  [...]
  if ((nmemb >= MUL_NO_OVERFLOW || size >= MUL_NO_OVERFLOW) &&
     nmemb > 0 && SIZE_MAX / nmemb < size) {
    errno = ENOMEM;
    return NULL;
  }
  [...]
}

Alternatively, you could use bool __builtin_mul_overflow (type1 a, type2 b, type3 *res), but I believe that is a GNU extension.

[1] https://github.com/is1200-example-projects/mcb32libc/blob/master/libc/calloc.c