[Python] Why does Python instance of protobuf message take so much memory?

I'm investigating a memory issue related to protobuf message in Python.

Here is the simple protobuf message:

message LiveAgentMessage {
string text = 1;
}


So it gets compiled to a class in python with protoc, I ran the following script

import pydantic
import dataclasses
from pympler import asizeof
from foo_pb2 import LiveAgentMessage

# For comparison, I also included a dataclass and pydantic implementation of the message

@dataclasses.dataclass
class LiveAgentMessageDataclass:
text: str

class LiveAgentMessagePydantic(pydantic.BaseModel):
text: str


def get_size(obj):
return asizeof.asizeof(obj)

print(f'{get_size(bot_pb2.LiveAgentMessage())=:}')
print(f"{get_size(bot_pb2.LiveAgentMessage(text='a'))=:}")
print(f"{get_size(LiveAgentMessageDataclass(text=''))=:}")
print(f"{get_size(LiveAgentMessageDataclass(text='a'))=:}")
print(f"{get_size(LiveAgentMessagePydantic(text=''))=:}")
print(f"{get_size(LiveAgentMessagePydantic(text='a'))=:}")


The output is

get_size(LiveAgentMessage())=816
get_size(LiveAgentMessage(text='a'))=895904
get_size(LiveAgentMessageDataclass(text=''))=424
get_size(LiveAgentMessageDataclass(text='a'))=416
get_size(LiveAgentMessagePydantic(text=''))=352
get_size(LiveAgentMessagePydantic(text='a'))=352


I wonder why the LiveAgentMessage(text='a') (the second line) takes up so much memory with just a one-letter string, it's 10^4x compared to the first line?

I'm using pympler for size calculation, but not sure if that's the right one for the python proto class.

Continue reading...