std::thread::hardware_concurrency on Threadripper

[Serg Kryvonos (ohhmm)]

AMD Threadripper has 128 hardware threads.
However std:: thread::hardware_concurrency reports only 64 hardware threads available.
This doesn't seem an expected behavior.
So I took a look on the process affinity and it shown only 64 selected logical processors.
Does std:: thread::hardware_concurrency reports number of logical processors according to the current process affinity?

[Alex Guteniev (AlexGuteniev)]

It reports number of hardware threads that are within processor group, so it is capped by 64.
It still matches the number of hardware threads you get by using C++ standard facilities only, like std::thread, std::async, <execution>.

To utilize more hardware threads need to manage groups. See Processor Groups as a starting point.

The value does not take into account affinity.
I attempted a PR to take affinity into account, see #594 .
It turned out to be too complicated (especially maintaining the same for 32-bit code)

[Serg Kryvonos (ohhmm)]

Thanks Alex Guteniev (@AlexGuteniev),

Seem an explication of OS implementation details that caused new standard compliance issue.
I would consider incapsulating processor group related details because those seem not mentioned in C++ standard yet.
May be expected for STL to use all accessible logical processors seamlessly.

Thanks,
Serg

[Alex Guteniev (AlexGuteniev)]

Yeah, C++ standard is apparently not the right place to include processor groups, and currently it deliberately limits CPU management by only having std::hardware_concurrency. So CPU scalability would have been better achieved:

• without having processor groups in the first place
• with having standard threads using thread groups transparently

Not very seamless way is access to it as non-standard extension. It is available already for std::thread via native_handle access, but not for std::async or <execution>. Maybe when problem becomes more important this solution would be implemented fully.

Billy O'Neal (@BillyONeal) may have more thoughts on that

[Charles Milette (sylveon)]

A problem I see with making it process group aware is that if you use thread::hardware_concurrency as a reference for any kind of concurrency, without using the Windows API to configure processor groups you'll be running your 128 threads on 64 logical processors only.

Sure, making the STL aware of processor groups might be doable, but consider the case where you use thread::hardware_concurrency without using std::thread (arguably, this is bad but unfortunately fairly common, just searching for hardware_concurrency on GitHub gives you plenty of examples where this is done)

I don't think it's a standards violation, as the standard says the value should only be considered as a hint. In this case, you're being hinted at how much threads are supported by default without additional configuration.

[Alexander Neumann (Neumann-A)]

Fun fact:

It reports number of hardware threads that are within processor group, so it is capped by 64.

It is capped by whatever number of processor the process group has. So on your z core machine you can create x<=z groups with y_x cpus. If you start an application the normal way you cannot even control in which group the process will be start by windows. (So if you split a 40 core machine into, 25 10 5 cores the application might run randomly in the 5 core group or 25 core group)

Last time I tested looking at the cpu selection overview for a multiprocessgroup application the task-manager crashed. Don't know if this has been fixed in windows by now.

std::thread definitly needs some kind of affinity support in the future.

[Curtis J Bezault (cbezault)]

Like the above commentators have pointed out std::hardware_concurrency is only a hint.
The reason it makes sense to return the number of threads in the processor group is for the reason that Charles Milette (@sylveon) gives. Without using Windows API calls there isn't a way to use all the threads you expect to be able to use if we returned the true number of processors on the system.

[Billy O'Neal (BillyONeal)]

Put another way, hardware_concurrency tells you how much concurrency can be accessed with std::thread, std::thread models CreateThread, and CreateThread only places threads in the process' default CPU group.

[Alex Guteniev (AlexGuteniev)]

By the way, a post about getting processor count, if you want exactly this, and not really available processors to std::thread
How can I get the number of processors in the system, when there are more than 64?

[Stephan T. Lavavej (StephanTLavavej)]

We later merged #5459 to fix this, after the OS changed behavior.