[Python] Pytest can't simulate stale API chunk in deduplication logic — fetch_recent_klines()...

I'm testing a function `fetch_recent_klines()` that fetches 1m kline data from the Bybit V5 API.

It's designed to **halt and return `[]`** if it receives the same chunk of data twice — to detect stale API responses.

### Deduplication Logic

chunk_key = hashlib.md5(str([row[0] for row in chunk]).encode()).hexdigest()
if chunk_key in seen_chunks:
return []


The Mocked Test Setup:

def build_chunk():
return [[str(1717382400000), "", "", "", "", "1"] for _ in range(100)]

mock_response = {"result": {"list": build_chunk()}}
responses = [mock_response, mock_response, mock_response]

def side_effect(*args, **kwargs):
if responses:
return MagicMock(status_code=200, json=lambda: responses.pop(0))
return MagicMock(status_code=200, json=lambda: {"result": {"list": []}})


The Test

with patch("scan.requests.get", side_effect=side_effect):
result = scan.fetch_recent_klines("BTCUSDT", total=315)
assert result == []


The Problem

The function never returns []. Deduplication doesn't trigger, even though:

• 
  The mocked chunks are byte-for-byte identical
  
  
• 
  The hash is based only on timestamps
  
  
• 
  The same code does work with real API responses
  

I confirmed this with a debug message:

[DEBUG] Duplicate chunk detected, breaking early.

Full code + tests: https://gist.github.com/alexx1202/46925d4a11c462dabc327d2a8fd102a8

What I Tried

• 
  Varying total= (101, 200, 315)
  
  
• 
  Hashing only timestamps
  
  
• 
  Verifying row content is identical
  
  
• 
  Swapping json=lambda: with direct dicts
  

Despite all that, the test still returns full klines instead of triggering early termination.

How can I write this test to actually prove that the stale data deduplication logic is working?

Could MagicMock be interfering with the hash or identity comparison in some subtle way?

Continue reading...