Skip to content

ITS#10253 Fix incompatible pointer type

Ryan Tandy requested to merge ryan/openldap:its10253 into master

The code passed a ber_len_t * in place of a size_t * argument. This errors with GCC 14 (and warns with earlier compilers) if ber_len_t and size_t are different primitive types.

ber_len_t is always unsigned long. size_t is implementation-dependent and may be unsigned long, unsigned int, or something else.

On LP64 platforms (i.e. 64-bit Linux), gcc typically defines size_t as unsigned long, so the code works.

On ILP32 platforms (i.e. 32-bit Linux), gcc typically defines size_t as unsigned int, so the pointer types are incompatible. (GCC manual: "In C, two different primitive types are never compatible.") I think it was probably ok at runtime though, since the types are actually the same size.

The code always emitted -Wincompatible-pointer-types on ILP32. GCC 14 escalates this warning to an error by default. (Porting to GCC 14)

My patch reuses the existing size_t len variable. It is not referenced below this point, so I think it's ok to reuse.

I have not considered or tested what would happen if size_t is larger than long (LLP64?). The assignment keyhash.bv_len = len could overflow. However the existing code would also be wrong (and I think more likely to misbehave at runtime) as it passes a pointer to a different size value. Anyway, I'm personally not interested in spending time on GnuTLS on Windows (the only LLP64 I'm aware of) at this point. (Although the definition using long does make me wonder if there could be other LLP64 porting issues lurking...)

Merge request reports