Multithreading performance #328
Replies: 27 comments 100 replies
-
|
I'd like to see some thoughts from @pitrou on this, since IIRC he touched it last. :-) |
Beta Was this translation helpful? Give feedback.
-
|
Apart from the fact that I don't understand where the mentioned "priorities" come from, it seems to me that the only way to evaluate this proposal is for someone to write the code and gather metrics of interest. |
Beta Was this translation helpful? Give feedback.
-
|
Actually I may be partly wrong here - the current threading model does not provide functionality to set thread priorities, so all threads in a process run using the processes O/S priority, and so for this model the GIL would not need to have branches for each O/S priority. However, my real-life experience with trying to balance a mix of CPU-light and CPU-heavy background threads whilst keeping the UI responsive suggests that: a. Mostly you want to run background threads at lower priority than the main thread. But sometimes you want to run CPU-light background threads at a higher priority than the main thread. So I believe that thread priorities is something that should be considered, even if the initial implementation doesn't provide or handle thread priorities and they are provided as a subsequent enhancement. I also note that Windows threading has the ability to mark threads as background activities - and this might be nice if it could be added as e.g. an IDLE thread priority where the O/S supports it. Additionally (though it has no impact on a GIL scheduler, consideration should be given to supporting thread I/O priority/nice (if such a thing actually exists for threads) as well as CPU priority. |
Beta Was this translation helpful? Give feedback.
-
@pitrou, IIUC, you're the primary author of the current GIL implementation. If that impression is indeed correct, wouldn't you be the best person to do that? Or if you don't have the time / interest, is there some way you could express more support for resolving this long-standing problem? |
Beta Was this translation helpful? Give feedback.
-
|
@thinkwelltwd This is open source software, and code that I wrote more than 12 years ago. CPython is a collective project with many talented contributors, such as @gpshead and @Fidget-Spinner in this discussion. I fully trust that they don't need my explicit support to judge whether solving these issues is worth their time. |
Beta Was this translation helpful? Give feedback.
-
I share that confidence in them. From the perspective of one who's watched this GIL discussion play out over the years and who noticed with great interest this latest resumption, it seems that the nature / tone of your comments sounds more disinterested and aloof than the history of your involvement merits. If that comes across as too critical or accusatory, I don't mean it that way. I thank you for your involvement. I also share @Sophist-UK interest in seeing this resolved. |
Beta Was this translation helpful? Give feedback.
-
@pitrou / Antoine - I think you are being somewhat modest here. Yes, it is an open source project, and yes there are indeed many talented contributors, yourself included. But whilst it was indeed over a decade ago that you wrote the new-gil code, and a similar amount of time ago that @dabeaz / Dave undertook his instrumented analysis, nevertheless this does mean that each of the two of you probably have more knowledge about the current GIL and how it behaves than the rest of us put together - and therefore better able to make an informed guess about whether current efforts are likely going in the right direction or whether it is worth considering a design of a bespoke solution as an alternative to the BSP. |
Beta Was this translation helpful? Give feedback.
-
|
@Sophist-UK Could you run Beazley's original experiment with Python 3.10? It's a very small amount of pure Python code. In his talk he claims a 7x slowdown when the server thread is competing with one CPU-bound thread. That should be simple to do. You can forget about the ZeroMQ version -- in fact any of the versions (e.g. Python + blocking sockets) would be fine. |
Beta Was this translation helpful? Give feedback.
-
|
(I am referring to the experiment Dave showed in https://www.youtube.com/watch?v=fwzPF2JLoeU . I simply want to know if things have at all improved since Python 3.2, which he used for his experiments.) |
Beta Was this translation helpful? Give feedback.
-
|
@gvanrossum Guido I am really not sure I have the skills. I certainly have never done anything like this before and I have no idea where to start. If it needs python / windows skills, then I would probably be OK. If it needs C/C++ skills then I am instantly out of my depth with way to big a learning curve. I will try to find the time in the next few days to research what David ran and try to run it again. I have an aging 4th gen i7 laptop with 4 cores and 8 hyperthreads so I probably have the hardware. I have py3,9 64bit installed and can easily install py3.10 alongside. However what I am trying to solve is not necessarily the same as David was looking at - it is some years since I last worked on the multithreading app, and reached out to David to understand the GIL issues, so I am not quite sure what my issues were then and no idea whether they still exist several years later. But I do have an understanding of the sorts of issues that might arise with one or more CPU bound threads i.e. CPU threads not getting fair shares, I/O threads getting no or infrequent CPU. I probably have the Python skills to tweak existing test code to test for the things we are talking about (except hooking into functionality that would e.g. record the end timestamp of an I/o and the timestamp when the thread that called that I/O next gets scheduled). But if I need to start tweaking C-code and recompile Python, I am going to be lost. |
Beta Was this translation helpful? Give feedback.
-
|
It only requires Python skills. |
Beta Was this translation helpful? Give feedback.
-
|
Ok - I'll give it a go then. I will try to find the code that Dave Beazley used as soon as I have some free time. |
Beta Was this translation helpful? Give feedback.
-
|
Are there any specific changes in the 3.10 GIL that means I should install 3.10. Or a 3.11 alpha? Or will 3.9 do? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know. If the problem seems resolved in 3.9 we'll assume it is still resolved in later versions. If it still exists in 3.9, someone will ask to redo the experiment in 3.10 and/or 3.11. Most important is that you post some working code here (a few files in a Gist might suffice). |
Beta Was this translation helpful? Give feedback.
-
|
My starting point as a requirement is a GIL that allows developers to know how their application is going to behave (i.e. determinism) and for it to behave in a reasonable way if they just chuck some multithreading code in there without understanding the GIL. That is a real life requirement. I tried the spin code in his presentation and it no longer locks up the console. So there has been some improvement in the GIL since 3.2 but that is not the same as meeting the above requirement. I had a quick look for a repository of the other code from this presentation. But the bitbucket link to it on Dave's EmbraceGIL page at http://dabeaz.com/talks/EmbraceGIL/ no longer works and a google search doesn't bring it up either. My next step is to try to find some code from another of Beazley's presentations and tweak it to have a mix of CPU and I/O bound threads and then try to mix and match these to see just how well CPU-bound threads scale and the impact they have on I/O bound threads. But another thing he points out in his presentation(s) is that the current competition approach has a heavy overhead as all the threads put the effort into attempting to get the GIL, when only one can actually win. A scheduler would pick one thread and signal it, so avoiding this overhead. Not sure how to measure this - Dave instrumented a version of Python to capture a trace of all GIL related actions from all threads and recreating that is way beyond my ability. |
Beta Was this translation helpful? Give feedback.
-
|
Could you reconstruct it from first principles given David's slides? He gave some pseudo-code, maybe you could turn that into working real code. |
Beta Was this translation helpful? Give feedback.
-
|
Or just use udp-ioclient.py and udp-iotest.py from https://bugs.python.org/issue7946. Get those running and report your results. |
Beta Was this translation helpful? Give feedback.
-
|
I have downloaded the UDP code and had a quick look - they look good as a basis. I will need to tweak the code so that we can see the CPU-bound thread performance depending on the number of threads. I would also like to check that there is no performance with the UDP echo server running in foreground vs. background. And I probably want to allow the client to be throttled or not so as to see how the CPU intensive workload is impacted depending on zero, low or high volumes of echos and to capture and do a histogram graph on the response times as well as the total throughput. But none of this is going to be that difficult. I will post results here once I have them. |
Beta Was this translation helpful? Give feedback.
-
|
That said I am not sure that I have any results to compare these runs to. We will have to evaluate them on their own merits. But if I install various key versions of Py3 and see what the differences are that might give us some insights too. |
Beta Was this translation helpful? Give feedback.
-
|
Beazley reported the following in his talk.
I would certainly hope that result (1) is still reproducible. Once you have that result, what is the slowdown with a CPU-consuming thread added? |
Beta Was this translation helpful? Give feedback.
-
|
When working on the new GIL I wrote a small concurrency benchmark which is still there in the source tree in |
Beta Was this translation helpful? Give feedback.
-
It is a superset of the UDP test server so will use this as a base. |
Beta Was this translation helpful? Give feedback.
-
How do you determine a thread is I/O bound? Assuming that one that releases the GIL rather than hitting the end of their interpreter loop time-slice is doing so for I/O is unrealistic. C API users release the GIL all the time to enable CPU parallelism during pure non-Python code. Not just for potentially blocking I/O. The interpreters loop time-slices seem like an insufficient proxy for CPU vs IO bound. Example: A thread running If this requires all C API uses to update code to use new explicit purpose GIL releasing APIs stating the reason (Blocking vs non-Python-CPU) they are releasing it, adoption could be a challenge and you may find an "unknown" scheduling state falling between or acting as both IO and CPU bound becomes necessary. I'm not suggesting more informative GIL releasing APIs are necessarily a bad idea, just that they wouldn't magically win on existing code. https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock |
Beta Was this translation helpful? Give feedback.
-
|
I could also be misunderstanding what you had in mind for time-slice. |
Beta Was this translation helpful? Give feedback.
-
|
I guess I am using I/O bound & CPU-bound when I should be talking about threads that are CPU-light or CPU-heavy or threads that time-out vs. threads that voluntarily give up the GIL (explicitly or by doing a blocking I/O). The idea of Foreground/Background scheduling is that processes that need short intermittent bursts of CPU are prioritised over those that use lots of CPU. Of course, whilst a thread that did a short-burst last time is probably going to do another short-burst next time, but that is not guaranteed, but providing that the time-slice is short enough it shouldn't make a huge amount of difference. As stated in my original post, backward compatibility is a requirement (item 3.) I haven't seen any reason that C API would need to change. The GIL scheduler only decides who to allocate the GIL to. If a C extension releases the GIL, it will then carry on executing as scheduled by the O/S until it decides it needs to reacquire the GIL when it gets put on the GIL Scheduler queue and waits to be allocated it. |
Beta Was this translation helpful? Give feedback.
-
I will try to add this as an alternative CPU heavy thread in the tests. But my gut reaction is that the scheduling of this thread when it doesn't have the GIL should be left to the operating system. It is up to the O/S - and the programmer influencing it by setting an O/S thread priority - to ensure that competing threads/processes get sufficient CPU. |
Beta Was this translation helpful? Give feedback.
-
|
I am making good progress with adding some stuff to @pitrou 's ccbench code e.g. improved environmental reporting at the start of the run, ability to set process affinity, and automatically setting high O/S process priority so that the impact of other system activities are minimised. I think that this script now has the basic functionality that I think is needed for some benchmarking, and I now need to think about:
A couple of key things you might want to try if you decide to check out the new script:
Any / all feedback on this new script and the next steps are very welcome as I do not have a clear vision yet of what I need to achieve next. I will post an example output in a subsequent comment in a few mins. |
Beta Was this translation helpful? Give feedback.
-
|
P.S. I am not (yet) measuring CPU overall usage for these benchmarks - I can probably only measure CPU usage for the process, but it would be useful to see how that varies from the expected amount for the workload undertaken. |
Beta Was this translation helpful? Give feedback.
-
|
Example output: ccbench_updated.py -a 3
|
Beta Was this translation helpful? Give feedback.
-
|
I wouldn't be so sure that the pi calculation is Python. It looks as if it is Python, but once you have a few hundred digits it spends all its time in the long integer code, which is all C. |
Beta Was this translation helpful? Give feedback.
-
|
I don't believe longobject releases the GIL though. hashlib things all do for larger inputs. as should compression algorithms. the accurate info on that is found by looking at their C code. |
Beta Was this translation helpful? Give feedback.
-
|
Ok I have tweaked the code a little more (to give std dev on throughput and to run the latency/bandwidth clients at high priority with full affinity) and rerun the example, and I am enclosing the file and the example results here and will give my first-cut analysis in the next comment. ccbench_updated.py -a 3
|
Beta Was this translation helpful? Give feedback.
-
|
To be honest, I'm not sure what you're trying to do here. What you're observing is already well known (and morally already tested for by the original version of the benchmark). |
Beta Was this translation helpful? Give feedback.
-
|
You can't really know whether the changes made were worthwhile until you have made them and looked at the results. My first stage was to tweak your (already excellent) code to give a bit more information, and to give a bit more flexibility on the affinity. But the real purpose of this was that Guido wanted the previous analysis repeated so as to confirm that there is still an issue to be solved. |
Beta Was this translation helpful? Give feedback.
-
|
Here is my first cut analysis of the above example:
Conclusions (personal view)So what conclusions might potentially be drawn from this about whether there is a need for a GIL scheduler.
Obviously, these are my personal observations and I am as capable as the next person of drawing false conclusions - so please chip in with your own observations and / or suggestions on how to proceed from here. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know if it helps at all, but I ran the tests on my Ryzen 9 3900X (Cores: 12, Hyperthreads: 24) |
Beta Was this translation helpful? Give feedback.
-
|
@luzfcb Fabio - I will take a look at the details of this in a minute, but having a linux run is useful to iron out the linux bugs 😄 |
Beta Was this translation helpful? Give feedback.
-
|
@luzfcb I didn't get anything more out of this - the results seem consistent with my earlier 3 and 8 core runs. But I am about to post a new set of results which include CPU measurements and which I think tell us something useful - so you might want to run 1-core and all-core tests to see if your results are any different (using the new code). |
Beta Was this translation helpful? Give feedback.
-
|
The test with 1-core ❯ sudo /home/luzfcb/.pyenv/versions/3.9.11/bin/python3.9 ccbench_updated.py -a 1 == CPython 3.9.11 (main) 64-bit == == x86_64 Linux 64bit on 'x86_64' with 24 cores == == Threads: pthread, Lock: semaphore, Version: NPTL 2.31 == == Check interval: None, Switch interval: 0.005 == == Cores: 12, Hyperthreads: 24, Priority: 0, I/O priority: pionice(ioclass=, value=4), Affinity: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] == !! Process I/O priority set to REALTIME !! !! Process affinity set to [0] !! == Cores: 12, Hyperthreads: 24, Priority: -20, I/O priority: pionice(ioclass=, value=0), Affinity: [0] == !! Process I/O priority set to REALTIME !! |
Beta Was this translation helpful? Give feedback.
-
|
@luzfcb Fabio - I have a possible fix for the display of the iopriority if you need it. I suspect that the impact is small, and it can probably wait until the next time I upload the script. |
Beta Was this translation helpful? Give feedback.
-
|
I have now added reporting of CPU usage and it turns out that this is particularly insightful: Additional observationsThroughputFor the Pi/regex workloads, we don't really learn anything new with the CPU information, but I believe that the results from the compression/hash tests tell us something quite important: As the threads build up to the number of cores, we see the CPU scaling proportionately - we are maxing out the CPUs with these GIL-releasing CPU-heavy workloads, however the throughput is not increasing. I don't think that this is some sort of bottleneck constraining execution, because if it was then the total CPU would not be proportional to the number of cores. So it seems that something is wasting a lot of CPU cycles, and one possible cause could be the competition-based approach for gaining the GIL. If this is the case, then by massively increasing the Switch Interval from 0.005s to (say) 0.1s, and creating a corresponding reduction in the number of timeslice based GIL switches, I would expect this overhead to decrease and to see a significant improvement in throughput. I tried this and the results are shown below, and we definitely see an improvements for some GIL-releasing workloads, but not so much for others - and at this stage I do not have an explanation for this. But for e.g. the hashing workloads we can see that for 2 threads you actually get 100% more throughput rather than only 60% more. But this quickly tails off for threads >= 3, and I have no theory about this either. (I did try a run with a Switch Interval of 0.5 and that was no better - but with a 2s run time, a switch interval of 1/4 of that may introduce other issues.) But if we look at the base results then the numbers are really very bad indeed - for the bz workload (which is the worst) the incremental figures as you add threads look like this: 1st thread : 100% through put for 100% CPU (baseline) If we combine the 5th to 16th threads into a single group, we get c. 56% increase in throughput for 400% increase in CPU. I think we can all agree that these are really not good figures and represent a massive opportunity for improving the performance of GIL-releasing C-based modules. IMO a good starting point for this would be to move away from a GIL handover approach based on competing for the GIL to one where the GIL is handed off explicitly to the chosen thread. BandwidthThe bandwidth results are also a bit odd as we see a tail off in the additional CPU usage. Edit: I now have a theory that the CPU-soaker threads are not getting sufficient access to the GIL to keep the non-GIL CPU load in runnable state, and as per some later discussions which have suggested the same thing from another direction, I intend (tomorrow?) to add some sleep statements to the client code in order to reduce the intensity of the latency, and throughput traffic and see if that changes the measurements. Results with CPU information addedccbench_updated.py -a 8
Throughput Results with Switch Interval of 0.1secccbench_updated.py -a 8 -t -I 0.1
|
Beta Was this translation helpful? Give feedback.
-
|
D'oh - I looked at the results again and a Switch Interval of 0.1 did bring the CPU utilisation back up - so this supports the theory that all this CPU is being wasted on GIL switches. |
Beta Was this translation helpful? Give feedback.
-
|
I am still struggling to work out why the bandwidth tests with compression/sha workloads tail off on CPU usage whilst the latency tests don't. The code is pretty similar, just the amount of data being different. If anyone can review the code and see if they can spot any structural difference between latency and bandwidth code that would explain this, I would be very grateful. |
Beta Was this translation helpful? Give feedback.
-
|
Ok - a potential theory on the CPU tailoff on bandwidth vs latency (though I think it might be a bit lame TBH): Latency: very fast execution, GIL-released workloads do not reach a point where they need to acquire the GIL in this time-scale. Bandwidth: Much more data to process, takes more CPU, during which the a few/some/many/all of the GIL-released workloads reach a point where they need the GIL again, so they all queue up for the GIL during which time CPU utilisation drops. Then one by one they grab the GIL, do something and release it again and go back to using up CPU. The trouble is, I can't imagine the bandwidth echo code taking that much more CPU that it would cause all the CPU threads to reach the point of needing the GIL, but then again, these echo requests are coming fast & furious, perhaps sufficient to create this issue. I might try adding a sleep option for the bandwidth client to introduce a delay between echos in order to allow the CPU threads all to get going again, and if that supports this theory. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question regarding David Beazley's original tests. He said it specifically wasn't a throughput or latency test. His client was (in pseudo-code) something like def client(n=10000):
msg = b"xxxxxxxx" * 1024
for i in range(n):
rpc(msg)
time.sleep(0.001)The server was a trivial echo server, with 0 or 1 pure-python CPU-consuming threads. (I don't think he showed the CPU-consuming code, but we could assume it's as stupid as Which of the benchmarks in your (or Antoine's) code corresponds to this? |
Beta Was this translation helpful? Give feedback.
-
|
However, I am not trying to duplicate exactly the results obtained by David - I am simply trying to establish whether a GIL scheduler would be beneficial compared to the current GIL mechanism. This is broadly similar to some of the live demonstrations that David did as part of some of his talks 12+ years ago, and definitely prompted by it, but my technical skills certainly do not come remotely close to those that David used for his excellent and highly fine-grained instrumentation of Python to determine exactly how the GIL behaves. I was challenged by Guido to prove that this problem still exists, and that is the purpose of my tweaking of Antoine's already excellent ccbench code to give greater insight into what might be happening. But this can never come close to producing the insights that Dave's instrumenting of the GIL at a very low level was able to achieve. |
Beta Was this translation helpful? Give feedback.
-
|
Some of the later tests (with the new GIL), were nothing more than launching a |
Beta Was this translation helpful? Give feedback.
-
It would help me a lot if someone could explain how the GIL code works in 3.9 and 3.10, and in particular whether it distinguishes between CPU-light and CPU-heavy threads or simply releases the GIL and hopes that the CPU-light thread happens to get it in preference to one of the CPU-heavy threads. There is (or was at some point) a FORCED_SWITCHING which prevented the same thread from getting the GIL back immediately if there was a runnable alternative thread ready to go, but if you look at the Bandwidth measurements with 1 Pi CPU-soaker thread running, it is clear that the echo server thread is NOT getting anything remotely close to 1/2 of the time-slices. Super-intense ping/echo clients are really nothing like real life - I/O workloads spend a lot of time waiting for the data they requested to arrive back from disk or network, and GUI workloads (to process mouse moves and keystrokes) are normally much less intensitive (at the microsecond level) as well. So let's see what happens to the measurements when I reduce the intensity of the ping/echo clients and see if that changes the measurements.
|
Beta Was this translation helpful? Give feedback.
-
Ah - if only it was only a 5ms delay. I think that we are talking about 2 different things here. So far I have been talking only about continuing the current ways that the GIL is given up - either part way through a 5ms time-slice (e.g. by starting a blocking I/O, waiting on a resource or signal or voluntarily giving up the GIL to continue execution of C code without it) or at the end of the 5-ms time-slice when you still have Python instructions to execute. The big 3 issues highlighted in the analysis to date seem to me to be:
If you want an I/O bound thread to pre-empt the thread that currently holds the GIL mid-way through a time-slice (and I am not saying that this is a bad thing - it may be a very good thing) that is a different and complementary discussion to the one about light vs. heavy CPU threads, deterministic scheduling of threads and the GIL transfer overheads. You could do pre-emptive GIL transfer either with or without a GIL scheduler. One way that pre-emption could work is:
In other words, you are sort of implementing interrupts and an interrupt stack, and computer-science has already identified the gotchas when programming such things like having excessive numbers of I/O threads that continually pre-empt each forever delaying the original thread, and possibly creating an ever increasing stack of interrupts. You can e.g. limit the stack to a single interrupt (so that one I/O thread cannot interrupt another I/O thread, but it can still get quite complicated. The answer might be (instead) to tweak the GIL scheduling algorithm to use a time-slice of (say) 0.001ms instead of 0.005ms - defining CPU-light threads as those that voluntarily give up the GIL within this 0.001ms, and to allow CPU-light threads a single time-slice but give CPU-heavy threads (say) 5 or 10 or 20 consecutive time slices of 0.001ms before they are moved from the front to the back of the CPU-heavy round-robin queue. Personally I think this would be a better (and easier) solution than a fully pre-emptive model. |
Beta Was this translation helpful? Give feedback.
-
I checked the code today - the latency/ping test had a delay. The bandwidth test didn't, but I have added some options and set a default bandwidth delay and the results are in a post below. |
Beta Was this translation helpful? Give feedback.
-
|
I would like to make a few general comments about some of things in this discussion as they pertain to my past work:
|
Beta Was this translation helpful? Give feedback.
-
|
@pitrou Antoine - I am not sure I would be quite as sceptical as that, because as Dave Beazley showed more than a decade ago, it is possible to instrument Python and demonstrate precisely what is happening with the current GIL implementation and thus prove beyond any doubt the guesses we are making here. But yes, in the end, as with the development of every new enhancement, you have to take an act of faith that the solution you are developing will fix the problem, and that is why you analyse as much as possible beforehand in order to minimise the risks of trying it out and finding it doesn't work. But this is true of almost every new endeavour - there are elements of risk. So we need to weigh up the potential benefits (which these measurements suggest are large) against the risks that the effort fails (which with decent analysis should be small). If the argument was always "but there is a risk it won't work, so we shouldn't bother", then there would never be any innovation. I am not however suggesting that we follow Edison's approach and approach this on a pure trial-and-error basis, trying thousands of different GIL solutions until we find one that works. Rather, I am suggesting that we do sufficient research and analysis to understand the scale of the problem and its causes, and that (assuming that it fits the causes) we then use a tried and tested computer science algorithm to solve the problem. I believe that Antoine's script (as modified by me) demonstrates clearly that the problems first identified by Dave Beazley over 12 years ago still exist in Python today and they are sufficiently big to take things on to the next step which is to understand the cause in detail. The symptoms we are seeing today appear to me to be extremely similar to those identified by Dave 12+ years ago, and so it seems reasonable to me to assume that Dave's instrumented analysis 12 years ago is still a good description of what is happening. A review of the changes made to the GIL in intervening years - which cannot be that large because the symptoms are still very similar - would give insight into where Dave's analysis might no longer apply, and we might then conclude that we have sufficient information to move forward without repeating this analysis or undertaking different analysis. Equally, someone with sufficient skills could repeat Dave's analysis (I am sure that his code can be remade available) and confirm whether it is the same, slightly different or wildly different. We might then move onto undertaking a high-level design of a solution (I am proposing a GIL scheduler, but there might well be other solutions that would be equally good or better) and determine whether this is likely to solve the problem. But in the end, yes, you have to bite the bullet, develop a prototype containing the basic functionality and run benchmarks on it to see if it works. But I suggest we follow the advice of the Knight Templar in the Indiana Jones movie and "choose wisely". As an aside, if you look at https://bugs.python.org/issue7946 you can see that someone tried to shortcut the above process - not understanding the issues, not considering what algorithm would provide the best and most easily implemented solution, and with hindsight we can see that they chose a solution (BFS) that was really not a good fit to the problem, was way to complex for the need, and was not cross-platform, essentially following the Edison trial-and-error method without having Edison's stubborn stickability. With hindsight (which is a lot easier than foresight), I might suggest that they chose BFS "unwisely". However, I think there is a lesson we can learn from this failed attempt - that it is not that difficult to implement a scheduler prototype - the BFS scheduler worked, it just wasn't the right solution to fix the problem or to be cross-platform. It seems that a single individual was able to retrofit a complex piece of existing Linux kernel scheduling code into Python in a reasonably short order. If I was a C programmer and understood the Python internals already, I doubt it would take me that long to code this - but I just don't have the skills. I am almost certainly being simplistic here, but there is already code that runs when the GIL is released - which signals all the threads to compete to get the GIL - and it seems to me that "all we need to do" (what an example of understatement that is) is to get it to select the first available thread from the two lists of those runnable threads waiting for the GIL - the CPU-light and CPU-heavy lists) and send a signal to that thread to take the GIL and run. Of course it is not quite that simple because you need to maintain that list of runnable threads as threads give up the GIL and as they become ready to reacquire the GIL - but I do not believe that implementation of a prototype is that hard. Then, having chosen if the results look encouraging, as you say there is more hard work to make it production quality and eliminate or mitigate the issues and iron out all the idiosyncrasies and minor incompatibilities that will arise (i.e. make sure that classifying a thread as CPU-light or CPU-heavy is done correctly for every different situation in which the GIL is released) and perhaps do some performance tweaking on e.g. the best switch-interval and the best multiple of switch intervals to give a CPY-heavy thread). So, I accept that this is a non-trivial amount of work. (Before I retired I was an IT Programme Manager with 30 years experience managing large programmes and projects, and of applying formal methods to the managing of risks in these programmes and projects - so I speak from professional experience on the planning and risk management side.) But equally, I believe that the analysis is showing that this is a non-trivial issue currently existing. |
Beta Was this translation helpful? Give feedback.
-
|
I took a quick look at the BFS patch from 2010 and it seems to include a lot of platform-specific code looking at timers etc. that I would expect to be in the Python code already - perhaps because it is a replacement for the GIL code at the time rather than an enhancement. But even with all the DIFF overheads and the comments (which are substantial) etc. it still only comes in at c. 1700 lines which in the big scheme of things seems quite small. |
Beta Was this translation helpful? Give feedback.
-
|
@Sophist-UK, I appreciate the time and effort you are putting into this. But for the sake of those following along, perhaps you might consider condensing your posts a bit? It's really tough for me to track your high-level thoughts and observations when they are essentially walls of text posted every few hours. A blog would be a great place to put the more in-depth analysis for those who are interested. |
Beta Was this translation helpful? Give feedback.
-
|
@brandtbucher Brandt - apologies for the "walls of text" but there seems to be a lot of scepticism here, and sceptical comments do need some explanation as to why they shouldn't apply in this case. |
Beta Was this translation helpful? Give feedback.
-
I am assuming that "work" production code refers to internal-use software and I suspect that these issues are likely to come up more in community / publicly purchasable code than internal products. Internal products are IME more likely to run on standardised and reasonably modern and capable hardware and are IMO likely to have smaller feature sets than community / commercial software. My own interest in this came up at around the time that Dave was doing his work from my own attempts to improve the performance of a community app that needed to keep the UI responsive whilst doing several streams of I/O and a single heavy CPU task in the background. And I spent a lot of time trying to make it work before coming to the conclusion that it was impossible. Yet here we are more than a decade later, and it is still an issue for those needing this functionality. |
Beta Was this translation helpful? Give feedback.
-
|
For reference for anyone who it might interest, here's the discussion about fixing the GIL the first time. https://mail.python.org/archives/list/[email protected]/thread/B5MUWPLGDBWXTUSKZEJLPAIGYM32XMPB/ |
Beta Was this translation helpful? Give feedback.
-
|
It is heartening to see that the same names are still here discussing this again. I am guessing by the 2009 date that this is discussing the rewrite of the GIL for Python 3.4 (??)?? |
Beta Was this translation helpful? Give feedback.
-
|
Updated Bandwidth measurements with a delay of 0.01s. AnalysisWe can see that the CPU levels for GIL-release C code are almost where we would expect at +700% for 8+ threads (rather than at the c. 460% on previous runs where there was no delay between sending a previous echo response and receiving the next one). This presumably allows the echo thread to give up the GIL and other CPU-soak threads that are waiting for the GIL to acquire it, get started on the next calculation and release the GIL again. I don't think that this has much if any impact on the analysis of what the shortcomings of the existing GIL are - it just resolved the unknown reason that CPU-soakers were not soaking up all the available CPU as we expected them to. Resultsccbench_updated.py -a 8 -b
|
Beta Was this translation helpful? Give feedback.
-
|
@Sophist-UK Could you submit your updates to the benchmark script as a PR, so they don't get lost? I suppose Antoine could review them. |
Beta Was this translation helpful? Give feedback.
-
|
@gvanrossum Guido - I was planning to do that anyway once it has stopped evolving. Having a consistent way of measuring and evaluating the performance of various versions of Python as the GIL evolves is important, regardless of whether this specific proposal gains any traction or not. |
Beta Was this translation helpful? Give feedback.
-
|
One thing I would add is that having introduced a delay to the echo workload, I am not sure that it is really any different to the ping workload. With the single exception of the size of the payload (c. 1k bytes vs. c. 10 bytes) there is really no difference - it was the intensity and lack of any delay that really distinguished the bandwidth test from the ping test. I still believe that an intense bandwidth test with zero delay does not match any real-life scenario, the question is whether it adds any beneficial insight into what is happening now, or possibly give insight or highlight issues in a future version of Python. Hence I am currently in two minds whether to leave both set of tests with delays, to switch bandwidth back to zero delay (so that it is different) or to simplify the code by removing one of the sets of tests. Comments please. |
Beta Was this translation helpful? Give feedback.
-
|
@gvanrossum Guido - As you requested, I submitted a PR with my updated script about 2 years ago and it was never reviewed or merged. Today it was closed unmerged as part of a different PR to remove several performance measurement scripts (including this one) from the core cpython repo. So unfortunately, this update script is being lost, along with several others (and an entire suite of benchmark tools by @vstinner) are indeed being lost. |
Beta Was this translation helpful? Give feedback.
-
|
I sometimes wonder if half the problem is the whole terminology of "priorities." It's like a magnet for complexity. Threads already have a concept of "daemonic." Maybe you could introduce a new concept like "holiness." When you create a new thread, make it "holy" by default--kind of like the "holy hand grenade" from Monty Python. If desired, a thread can declare itself "unholy" by calling some function like If anyone complains about further strangeness, you could just say "well, what were you expecting from a bunch of unholy threads?" |
Beta Was this translation helpful? Give feedback.
-
|
Lol. Including concepts from Monty Python is always a good thing. Or Terry Pratchett. Or Inspector Clouseau of the Sûreté. Of course, when I started this discussion I was fully expecting the Spanish Inquisition!! Regarding Priorities and Pre-emption, in the solution I have in my head, I see this as being implementable in four stages (only two of which are IMO essential):
|
Beta Was this translation helpful? Give feedback.
-
|
I am not sure that there is much more analysis I can do. In which case the time has come to decide whether this is an issue which needs addressing and if so what the next steps are... |
Beta Was this translation helpful? Give feedback.
-
|
Chiming in as a non-expert: I doubt I can contribute anything to the already heavily-researched field of schedulers, but I'm game to implement a different thread scheduler in CPython. I will probably use an algorithm that is already well received instead of conceiving a new one. In a few weeks time I will have loads of free time to research and hack on the C parts of this. Maybe I can finally put my rudimentary knowledge of the dinosaur OS book to some use :). |
Beta Was this translation helpful? Give feedback.
-
|
More thoughts: I question the usefulness of improving our thread scheduling considering async/await is increasing in popularity. I don't mean to start a threads vs async war, but I really wonder if improving the asyncio event loop would be a better use of our time. Besides, my own observations in prod is that heavily threaded applications use some other library anyways (eg. greenlet), and don't really rely on CPython's scheduler. |
Beta Was this translation helpful? Give feedback.
-
|
A good way to crater the performance of asyncio is to block the event loop by performing any kind of blocking (non-async) operation or by carrying out CPU-bound intensive processing. A common workaround is to offload such processing to a thread-pool. Thus, having thread scheduling work better in the presence of CPU-bound processing would also likely improve asyncio. |
Beta Was this translation helpful? Give feedback.
-
|
@Fidget-Spinner / Ken I am not on the Python Steering Committee, do not represent the Python team, and frankly don't understand the Python decision making processes, but I would like to express my personal thanks for this offer - which I interpret as an offer to create a prototype alternative GIL implementation which would be presented as a Pull Request on the Python/CPython master repo. As with all PRs, the choice of what is presented in the PR is up to the author - ultimately it will be your choice (not mine or anyone else's) on how you approach this. Of course, if others wish to offer alternative PRs that is also their choice. This is part of the fundamentals of how Open Source software works. However, it is my hope as the instigator of this discussion that you will be willing to approach this in a cooperative way, willing to listen to the input of others, in the hope that the key design decisions can be reached by consensus. IMO this would maximise the benefits and minimise the amount of duplicated effort. (I can see parallels in this previous statement with the analysis and my own opinions on this discussion - that we need to move away from the GIL being based on threads competing with one another for resources and towards a cooperative approach. LoL) There is a huge amount of Python expertise represented by the others here - including Guido whose contribution to Python can never be overstated, and Antoine who wrote the current GIL, so I would hope that there will be a goodly amount of supportive advice made available to you if you want / need it. Of course, if consensus cannot be reached the option is always open for someone to put forward alternative implementations as a different PR. Once you are happy with the PR, you are of course free to put it forward immediately for consideration for merging through the normal process, however I think that it would help this formal consideration if the interested parties here were to review it first and give some feedback. I would therefore suggest that once a prototype has been developed to a sufficient level as to A) successfully run a majority of Python single and multi-threaded code bases including the script I have been using, and B) that you believes demonstrates significant improvements in multi-threading performance, then we reconvene this discussion to allow the participants to review your prototype against this script and a range of other performance benchmarks and real-life applications so that we can see how the changes will impact performance in a spectrum of applications and benchmarks. I would certainly be happy to contribute to this testing effort. (If as part of your efforts you are able to create a classification of the types of GIL hand-off that exist, that might help us to build additional code profiles that will test specific aspects of how the new-new-gil behaves in specific cases.) Hopefully this will lead to a more robust solution and help everyone be confident that the PR is a sound one. Whilst I have a theoretical opinion on a solution, an ounce of practical delivery is worth a ton of theory - personally I will be happy to see any solution delivered that results in an even better Python (though of course my ego hopes that my proposed solution is the right one). If there is anything I can do to assist you with the choice of algorithms/design, act as a sounding board etc., I am happy to help. |
Beta Was this translation helpful? Give feedback.
-
I have never quite fully got to grips with async/await (because the parallel apps I worked on predated this functionality), but to the level I do understand it this loop seems much like the main event loop that UI frameworks like PyQT and wxPython employ. What I am unclear about is whether aync/await is compatible with event loops that these frameworks require? And since both examples are Python versions built by providing an interface layer on top of C-based widgets, if they are not compatible now I am unclear whether they could ever be made compatible with async/await. But either way, surely we should be providing reasonably efficient and effective implementations for both styles of coding, partly so that legacy code doesn't need to be rewritten and partly so that the developer can have a choice of approaches to suit their personal coding style, Python knowledge and the needs of the application? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know about asyncio, but I know Trio supports this kind of thing with "guest mode", and the docs for it may help with understanding the concepts more generally: https://trio.readthedocs.io/en/stable/reference-lowlevel.html#using-guest-mode-to-run-trio-on-top-of-other-event-loops |
Beta Was this translation helpful? Give feedback.
-
|
Going back to David's original bpo https://bugs.python.org/issue7946 (which you are also on). The most straightforward way I see recommended by Gregory is to update the BFS implementation for
Some of the arguments against the BFS implementation no longer apply though. Like Reading through the bpo, I'm not sure why nobody accepted David's implementation. Was it just due to a lack of conclusive benchmarks? Maybe @dabeaz could shed some light here please. His implementation seems least likely to break anything in our codebase, despite it purportedly performing worse than the full BFS. Disclaimer: my efforts may be fruitless, or it may take months. I don't see this being an easy task. I'm not an expert at all in the areas of python runtime state (unfortunately, I maintain other parts of CPython). |
Beta Was this translation helpful? Give feedback.
-
|
@Fidget-Spinner / Ken: Thanks for letting us know. 😃 |
Beta Was this translation helpful? Give feedback.
-
|
Curious if there were any takeaways from the recent PyCon that are relevant to this topic (maybe from the language summit)? |
Beta Was this translation helpful? Give feedback.
-
|
@jab I would assume that if there were, then someone would post here. I am not aware of anything. |
Beta Was this translation helpful? Give feedback.
-
|
We didn't talk about thread scheduling at the language summit, though there was discussion of several topics that affect performance, including Sam Gross's nogil work. |
Beta Was this translation helpful? Give feedback.
-
|
Clearly "nogil" would mean that a GIL scheduler was not needed. |
Beta Was this translation helpful? Give feedback.
-
|
I implemented my "holy" thread idea. It wholly solves my original performance problem. https://github.com/dabeaz-fork/cpython/blob/main/HOLY.md. |
Beta Was this translation helpful? Give feedback.
-
|
This is a great quick fix alternative to a better scheduler. A few smallish comments if I may:
|
Beta Was this translation helpful? Give feedback.
-
|
(1) Meh. Sorry about that. emacs decided it wanted to insert some tabs in there. |
Beta Was this translation helpful? Give feedback.
-
|
@dabeaz
@everyone |
Beta Was this translation helpful? Give feedback.
-
|
I think we have pretty much reached the end of this discussion. I have demonstrated that the current GIL continues to have some VERY poor performance characteristics, and I cannot express just how disappointed I am about the apparent lack of interest from the community in implementing a solution, despite the involvement in this thread of both @gvanrossum (BDfL of Python) and @pitrou (the author of the current GIL implementation). So, for a final time, I will ask again whether there is anyone interested in attempting to address this issue? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Sophist-UK I see you have a lot of enthusiasm for this specific issue and that the performance characteristics here are demonstrable, but I think everyone who is already subscribed to this thread has heard your call to action multiple times and there is no need to keep repeating it. If you would like to reach a broader audience I suggest trying to reach out to programmers on social media platforms (Reddit, Hackernews, etc.). That said IMO it still remains a dubious proposition that this needs solving. Most programmers when faced with the prospect of a GUI that needs to be associated with heavy computation come up with a server-client architecture where the GUI and the computation are sitting in different processes, potentially on different machines, making all of this a non-issue. And I'm not aware of any Python performance projects have called this out as a motivating issue. Further in the future if Python upstreams the nogil project this issue should disappear, or if PEP 684 (A Per-Interpreter GIL) proceeds to develop allowing users to run multiple interpreters then the workaround is simple to run the heavy computation in a separate interpreter. I'm not saying this isn't a good idea to solve but someone needs to be motivated to solve it. Perhaps that could be you? Maybe look at David's implementation and also read up on how the inner workings of CPython are implemented, there is an excellent book called "CPython Internals" which I am making my way through now. |
Beta Was this translation helpful? Give feedback.
-
|
@notatallshaw / Damian Thanks for these ideas. If I had the C skills, I would certainly consider coding a GIL scheduler myself, but the reality is that I don't. I am unclear, however, why you think this is a problem that doesn't need solving. Firstly, not all problems can be easily solved by a client-server architecture using multi-processing, because I believe that you need to use a copy of the data that you want the sub-process to handle, and there may be performance and / or real-time update issues that cannot be solved by multi-processing. Similarly as I understand it the proposed PEP 684 Per-interpreter GIL also requires data separation. So I am really not convinced that these are an alternative to a GIL which gives threads an appropriate share of CPU. It is also potentially the case that a multi-processing or Per-interpreter solution still requires multi-threading and would still hit these same problems. And of course, both the Per-Interpreter GIL and No-Gil changes are only potential changes at this stage and are not guaranteed to happen, and if they have considerable side-effects that require significant re-coding then it is potentially possible that they might even need to be in a separate Python version 4. Even if one or both of these get incorporated, then the time-scales for these are not yet defined - which I assume means that they could still be quite some time away. I would agree, however, that a No-Gil version of cPython would (obviously) not need a GIL scheduler, and since it enables true multi-tasking performance will be much better than could ever be achieved with a GIL scheduler. So, assuming that there was agreement that there is currently a multi-threading issue, the question then becomes one of if/when No-Gil will be implemented, and whether it is worth the effort to implement an interim Gil-Scheduler between now and No-Gil - and weighing this up requires intimate knowledge of the No-Gil roadmap and risks, and needs to be done by whatever the cPython steering group is, and is way above my (non-existent) pay-grade. So, as you can see, I am quite realistic and pragmatic about how the current multi-threading problem is resolved by the cPython powers-that-be, but at present we seem to be at a point where we have code that clearly demonstrates a problem (subject to the 3.10 tests mentioned below) but yet this is not accepted as a genuine issue by anyone in a position capable of prioritising it or weighing up the need for a solution before No-Gil is delivered. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks to @terryjreedy who posted elsewhere links to a summary of the recent GIL-related discussions. No-GILhttps://pyfound.blogspot.com/2022/05/the-2022-python-language-summit-python_11.html Summary: Will require small changes in C-extension modules, and separate GIL/NoGIL versions of both Python and C-extensions. Judged not to be a show-stopper, but will need careful planning - scheduled no earlier than 3.12. IMO, this sounds likely to happen, but GIL version likely to remain the default version and the version pre-installed with O/ses for some years. Per-Interpreter GILhttps://pyfound.blogspot.com/2022/05/the-2022-python-language-summit-per.html Summary: this will require a lot of code changes, not only in Python but also "any C-extension module that wishes to be compatible with sub-interpreters will have to make changes to their design". This seems to me to be an even BIGGER hurdle to overcome than the changes for No-GIL. IMO it feels far less likely to happen despite a growing awareness of the benefits. SummaryWith widespread adoption of No-Gil being several years away (my guess is up to a decade), and Per-Interpreter having even larger hurdles, it seems to me that a fix for the current performance issues should probably not be disregarded just because No-GIL and Per-Interpreter are on the horizon as potential enhancements. AsideFrom a Python coding perspective, because of the same need to use copies of objects for the extra processes / interpreters, Per-Interpreter GIL seems to me to be more akin to, and an alternative to, multi-processing but in a single Python instance / process. Conversely, because in both current GIL and No-GIL Python takes care of preventing multiple simultaneous access to objects, multi-threading with No-GIL seems to me to be an alternative and better performing way to do multi-threading. |
Beta Was this translation helpful? Give feedback.
-
|
According to a comment by reddit user skeeto (who may or may not be @skeeto here) in a thread about |
Beta Was this translation helpful? Give feedback.
-
|
Yup, that's me. I'm still surprised this significant GIL change went
unannounced, and that 7 months after release it seems nobody has even
noticed. I just checked again now, and the GIL still works this way on
CPython main.
|
Beta Was this translation helpful? Give feedback.
-
FWIW, the topic of benchmarking each PR has come up in discussions a few times recently. It couldn't be a required check, in part because the benchmarks take so long to run. Also, AFAIK getting the necessary tooling and hardware in place would be a huge project. Do you know of anything existing that would fit the need? |
Beta Was this translation helpful? Give feedback.
-
|
The problem with benchmarking is that you need a dedicated environment and standard for meaningful performance testing - and Github's environments are all shared. As a technical solution, the Python project could create a dedicated server and make it a Github runner, and then create a specialised performance script to take measurements and analyse them. BUT this would take resources that would probably be better spent on other things. But I am not sure it needs a technical solution. We have one script - though perhaps some other standardised benchmarks could be added - and all it needs is for reviewers to recognise PRs which e.g. might affect the GIL or bytecode performance, and run the script on the old and new versions of Python and report back on whether there are improvements or impacts. |
Beta Was this translation helpful? Give feedback.
-
|
Keep in mind that speed.python.org is updated regularly (or at least that's the intent). That should be sufficient to catch regressions. |
Beta Was this translation helpful? Give feedback.
-
|
I didn't know about http://speed.python.org but there is nothing in the list of benchmarks run to indicate anything looking at multi-threading performance with and without CPU soaks. And without specific benchmarks to measure multi-threading performance, all this site does is give a false sense of security. |
Beta Was this translation helpful? Give feedback.
-
|
I took a slightly more in depth look at the site documentation etc. The good news is that it has been going for 10 years on the exact same hardware, and also that it is being actively maintained. The bad news is that the documentation on what benchmarks are running is a bit thin on the ground. There is some discussion from 2016 about someone writing a thread benchmark, but the summary suggests to me that this is looking at the performance of the threading primatives and not the performance of multi-threaded applications with and without CPU sinks. The fact that someone looked at a threading benchmark suggests that there isn't anything else, and certainly not something that would look at this specific issue. Also, I am unclear whether there are benchmarks for the major C-extensions like numpy or not. Certainly there is not a bm_numpy at https://github.com/python/pyperformance/tree/main/pyperformance/data-files/benchmarks and the todo list still has Numpy on it since October 2016. So, whilst I genuinely applaud this site and the past and continuing efforts to create and maintain it, it really does need to have more real-life benchmarks reflecting real-life issues such as this one. |
Beta Was this translation helpful? Give feedback.
-
FYI, the Faster CPython team (my team at Microsoft) has its own dedicated host(s) where we run benchmarks. We post generally useful results in this repo (https://github.com/faster-cpython/ideas/tree/main/benchmark-results), but no fancy UI. Also see https://github.com/faster-cpython/ideas/wiki/All-About-Python-Benchmarking (and the related "tables" pages), which I added just yesterday and plan on filling in further. In fact, anyone can pitch in.
It is running the pyperformance default suite. See https://pyperformance.readthedocs.io/usage.html and https://pyperformance.readthedocs.io/benchmarks.html. Also note that pyperformance supports running external benchmarks. The Pyston benchmark suite has benchmarks like that: https://github.com/pyston/python-macrobenchmarks.
There are definitely gaps in the benchmark suite. You can open issues and submit PRs to pyperformance if you have ideas on benchmarks we could add.
Agreed! 😄 We'd love help in adding good benchmarks, if you have some time. It seems like you have valuable insight. |
Beta Was this translation helpful? Give feedback.
-
|
@ericsnowcurrently said:
So, as a member of the Faster cPython team, what is your take on the performance issues highlighted in this discussion and the need for a GIL dispatcher (new name rather than a scheduler which makes it sound more complex than it needs to be)? (So far I have been underwhelmed by the interest shown by the interest shown in this being a real and significant issue by other core Python developers who are in this discussion.)
When I have time I will try to massage the script used here into something that can be used with http://speed.python.org and submit a PR. If I do this will the new benchmarks be run against all currently supported versions of python (and ideally older ones) so that we can see the historical impact over time for the newly benchmarked area? |
Beta Was this translation helpful? Give feedback.
-
I don't have same level of expertise in this area as others that have already chimed in, so I don't have a lot to add. However, I don't mind giving it a shot. Honestly, I haven't followed the thread closely enough to give you a good answer right now. When I get a chance, I'll take a little time to catch up and share my thoughts. In the meantime, keep in mind that the GIL is a critical feature, yet the current implementation is relatively simple. Changes to something in that situation have to pass a fairly high bar, particularly due to the mostly volunteer nature of CPython development.
I'm sorry to hear that. I know that, as a group, we core devs are genuinely committed to solving problems for the community. If it seems like there's a lack of enthusiasm for some topic then it usually means either no one has had time to dig into it (we're mostly volunteers), there aren't enough experts, there was a gap in communication about the problem or solution, or some combination of those. I'm not saying it's necessarily the case here, but it might also be that the problem isn't worth solving, or a proposed solution isn't the right approach (and no one has had the time/motivation to find the right one). Typically this means the problem has hard-to-see complexities that imply challenging constraints which invalidate the obvious solutions. While we can't bend reality to our will, sadly, we can certainly do better about communicating about the challenging complexities, as well as being more open to new perspectives that clear the way to a solution.
❤️ Awesome! ❤️
IIRC, the automation for speed.python.org is relatively limited, so this would require some manual work. @pablogsal would know more. |
Beta Was this translation helpful? Give feedback.
-
Speed.python.org runs automatically against the latest version of python and old versions are benchmarked on demand, as their performance profiles don't change after they are released. |
Beta Was this translation helpful? Give feedback.
-
If and when you get time to look at this, please reach out to me. I do not believe that the scheduler I have in mind will be any more complex than the current GIL code. The fundamental difference is that when the GIL is released, a choice is made and a single thread signalled to take the GIL, rather than at present a signal being sent to all threads that then compete for the GIL. Maintaining a few linked lists of runnable threads is not difficult in C. A thread which gives up the GIL because it has used up its time-slice is immediately runnable and is added at the back of the CPU-Heavy runnable list. A thread which gives up the GIL e.g. to do I/O is not put on the CPU-Light queue until the I/O completes and it becomes runnable. The scheduler chooses the next thread from the front of the CPU-Light queue, or if that is empty from the front of the CPU-Heavy queue. In C programming terms, this is really not that complex. (There is a slightly more sophisticated version of this that allows the python programmer to assign a thread "priority" (say high/medium/low), and you maintain 6 runnable lists instead of 2, but you don't need that to get the main benefits IMO.) (Note: This is not something I dreamt up myself - this is the sort of priority scheduler that operating systems used to have when they were single core, (say) 45 years ago. It has a pedigree.) |
Beta Was this translation helpful? Give feedback.
-
Some years ago a guy named Dave Beazley instrumented Python and undertook an analysis of multithreading and why multi-threaded apps had poor perceived performance as seen by users particularly when there were one or more CPU bound threads. In essence Python release the GIL, all waiting threads became runnable, and whichever thread got in first would grab the GIL. This should in theory have been random rather than deterministic, but in essence what Dave Beazley showed was that at the end of a time interval, the current executing thread was (obviously) already scheduled by the operating system and so it got in first and grabbed the GIL back - and this would happen multiple times. In 2009 a change was made in Python 2.9 and 3.2 which implemented "FORCED_SWITCHING" (commit) to try to avoid CPU being given back to the same thread again and again, but the alternative thread that got the GIL was still random/pseudo-random.
In 2010 David Beazley ran his analysis again against Python 3.2 and noted a different set of issues. He raised these in https://bugs.python.org/issue7946. Soon after it was opened, someone actually implemented a full Linux BFS scheduler as a test (which I personally think was way too complex) and it wasn't beneficial, and there were (understandable) concerns about a scheduler needing to be platform agnostic. The subject was then not mentioned for about 10 years despite the consensus being that a scheduler was probably what was needed essentially to provide a deterministic solution, and was mentioned in passing in the last year or so of the issue before the issue was closed.
I am opening this discussion to suggest a specific and (IMO) easily implemented algorithm for a GIL scheduler. Unfortunately I am a Python programmer (and computer science nerd) and not a C-programmer so, whilst I have had real life experiences of the difficulties of delivering a Python app which has a consistently snappy GUI whilst doing a whole load of stuff in background threads, creating this GIL enhancement or even undertaking the analysis to confirm that it is still needed is beyond my skills.
The difficulties I have experienced can be summarised as:
a. Inability to assign an operating system priority to a thread (even though a lot of O/S allow this).
b. An inability for I/O and CPU bound threads to be differentiated and handled diffferently.
c. The random nature of who gets the GIL next.
d. The fluctuating nature of who gets the GIL next depending on the GIL implementation in the version of Python the code is run against.
Here is my wish list for a GIL scheduler, some but not all of which were discussed in the above issue 7946:
So, to the detail of my proposal:
Note: This is an entire different approach to the current scheme where threads compete for the GIL - the GIL scheduler would normally execute in the thread that is currently giving up the GIL and it would choose a new thread to run and signal it to take the GIL. I anticipate that the CPU used for scheduling will in most cases be less than that saved by not having all threads competing.
I should add that this type of simple scheduler is not a new one - it is pretty much a Priority / Foreground-Background / Round-Robin / Cooperative / Pre-Emptive scheduler (see wikipedia) combining several well understood algorithms that I first came across many decades ago in my mainframe days.
In general terms this seems like a very simple thing to implement, but of course reality is often never quite that easy.
And of course, I might not be up to date with the current state of the GIL, or I might be off the mark here in other ways, but I want to put this forward and see if people think it is a good idea and whether there is any interest in implementing it.
Thanks.
Beta Was this translation helpful? Give feedback.
All reactions