[Python] ctypes + cgo and ownership of strings

Background

I am currently using a DLL I generated with golang in python. As a sanity check for debugging I added a function called return_string(), which takes a pointer to a string in C (char*) converts to go, then returns the result. Mostly this is a debugging tool to look for encoding issues between the two languages.

Specific issue

In python I've wrapped the function, and everything works, except if I free after copying the bytes in. Here's the function:

# import library
if platform().lower().startswith("windows"):
lib = cdll.LoadLibrary(os.path.join(os.path.dirname(os.path.realpath(__file__)), "lib.dll"))
else:
lib = cdll.LoadLibrary(os.path.join(os.path.dirname(os.path.realpath(__file__)), "lib.so"))

lib.return_string.argtypes = [c_char_p]
lib.return_string.restype = c_char_p

lib.FreeCString.argtypes = [c_char_p]

def prepare_string(data: str | bytes) -> c_char_p:
"""Takes in a string and returns a C-compatible string"""
if isinstance(data, str):
return c_char_p(data.encode())
return c_char_p(bytes(data))

def return_string(text: str | bytes) -> str:
"""Debugging function that shows you the Go representation of a C string and returns the python string version"""
c_input = prepare_string(text)
result = lib.return_string(c_input) # This is allocated in go using essentially malloc

if not result:
return ""

copied_bytes = string_at(result) # This should be a COPY into new python-managed memory afaik
decoded = copied_bytes.decode(errors="replace") # Gets to here no issues

lib.FreeCString(result) # Program silently fails here
return copied_bytes


If I remove the call to lib.FreeCString(result), everything works, but my understanding is that because the result variable is allocated in go with a malloc() call, it needs to be freed in python. But, when I free it, it's either double-freeing or free-after-use and I'm not sure which, or why.

Do I need to free result? and if not, where does the pointer allocated in go get cleaned up?

Go code

I don't think it's a go-side issue, but just in case, here's the go code as well:

// All code returns `unsafe.Pointer` because this is a package, and CGo breaks your types if you don't

// Used to convert a C-compatible string back to itself, good for debugging encoding issues
//
// Parameters:
// - cString: Pointer to the C string (*C.char).
//
// Returns:
// - Pointer to a new C string with the same content (*C.char).
// Note: The caller is responsible for freeing the allocated memory using FreeCString.
//
//export return_string
func return_string(cString unsafe.Pointer) unsafe.Pointer {

internalRepresentation := C.GoString((*C.char)(cString))
result := StringToCString(internalRepresentation)
return result
}

// Convert a string to a c-compatible C-string (glorified alias for C.CString)
//
// Parameters:
// - input: The Go string to convert.
//
// Returns:
// - A pointer to the newly allocated C string (*C.char).
// Note: The caller is responsible for freeing the allocated memory using FreeCString.
func StringToCString(input string) unsafe.Pointer {
return unsafe.Pointer(C.CString(input))
}

// Free a previously allocated C string from Go.
//
// Parameters:
// - ptr: Pointer to the C string to be freed (*C.char).
//
//export FreeCString
func FreeCString(ptr unsafe.Pointer) {
fmt.Println("FreeCString(): freeing", ptr)
if ptr != nil {
C.free(ptr)
}
}

Continue reading...