[Python] Why does the presence of the Global Interpreter Lock not prevent data corruption...

I am trying to understand exactly what the Global Interpreter Lock (GIL) in Python protects, and what it does not protect.

An example code which demonstrates an answer to this question would be useful. I can't create such a thing myself, because I do not fully understand what state the GIL protects and what it does not protect.

Python has threads and those threads can share state. In languages like C, any many others, in order to access the shared state we might use something like a mutex, a semaphore, a condition variable or some other construct which provides synchronization or atomicity. Typically such things would be implemented by communication with the Operating System via system calls, which provides a single point of synchronization and control. (The OS thread.)

If I understand correctly, the Python GIL prevents more than one thread executing Python code - which is interpreted by the Python Interpreter.

Further, if I understand correctly, if we spawn 8 Python threads, then we still only have 1 Python Interpreter.

Those threads then become useful under two conditions:

• a thread has to go away and do some IO operation, which releases the GIL allowing another thread to execute Python code
• a thread calls into some code which is implemented in a language like C. In some cases this code will explicitly release and then later re-acquire the GIL. Numpy is an example of such a library.

I can't quite imagine why the GIL is not sufficient to protect the shared state between 2 or more Python threads.

Can someone explain this to me? A code example demonstrating why synchronization primatives such as a mutex are required would be very helpful.

Continue reading...