[Python] Subscriber only receives first 2/3 of messages from publisher when using QoS1 or...

I am testing a mosquitto server using two python programs. The mosquitto broker is a docker on a linux machine, the python programs are running on my windows machine. One is subscribed to the topic "Test" and constantly listening and will print all received messages to stdout + add them to a log file. The second sends a specified amount of messages to the topic "Test" as fast as it can.

This works great when publisher and subscriber use QoS 0. However, when using QoS 1 for the publisher, still all of the messages get transmitted, but only 60-90% of the messages are received when using ~ 50 messages or more. I have tested this also with 100, 1000 and 10,000 messages - it's always about 2/3 of the messages which are received, regardless of the amount of messages sent. Also, no messages inbetween are dropped, only the ones at the end.

This gets even worse when changing the publisher to QoS 2, then only 30-50 % of my messages are received by the subscriber.

The QoS level of my subscriber is set at 0, changing it to 1 or 2 doesn't change anything.

Both scripts are connected as the same user.

I'm thinking the system might not be able to catch up with all the messages as quickly - but why isn't it dropping messages inbetween then, but only cutting them off at the end, regardless of message count?

Here is the publisher program:


import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, reason_code, properties):
print(f"Connected with result code {reason_code}")


broker_hostname = "myhost"
port = 1883

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(username="user", password="password")
client.on_connect = on_connect

client.connect(broker_hostname, port)
client.loop_start()

topic = "Test"
msg_count = 0

try:
while msg_count < 1000:
msg_count += 1
result = client.publish(topic, msg_count, qos=1)
status = result[0]
if status == 0:
print("Message " + str(msg_count) + " is published to topic " + topic)
else:
print("Failed to send message to topic " + topic)
if not client.is_connected():
print("Client not connected, exiting...")
break

finally:
client.disconnect()
client.loop_stop()

and the subscriber program:


import paho.mqtt.client as mqtt
import logging

logging.basicConfig(filename="log.txt", level=logging.INFO)


def on_connect(client, userdata, flags, reason_code, properties):
print(f"Connected with result code {reason_code}")

client.subscribe("Test", qos=0) # also tested with qos=2, no difference in behaviour


def on_message(client, userdata, msg):
print(msg.topic + " " + str(msg.payload))
logging.info(msg.payload)


broker_hostname = "myhost"
port = 1883

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(username="user", password="password")
client.on_connect = on_connect
client.on_message = on_message

client.connect(broker_hostname, port)
client.loop_forever()

My mosquitto conf:


persistence true
persistence_location /mosquitto/data/
log_type subscribe
log_type unsubscribe
log_type websockets
log_type error
log_type warning
log_type notice
log_type information
log_dest file /mosquitto/log/mosquitto.log
log_dest stdout

password_file /mosquitto/passwd_file
allow_anonymous false

# MQTT Default listener
listener 1883 0.0.0.0

# MQTT over WebSockets
listener 9001 0.0.0.0
protocol websockets

Continue reading...