Loïc Grenié on Mon, 04 Dec 2023 09:42:05 +0100

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: PARI/GP pthread questions

On Sun Dec 3, 2023, at 15:10, Karim Belabas wrote:
* Bill Allombert [2023-12-03 13:15]:
> > 2)
> > Under 3.4.22 nbthreads
> > ...
> > * pthread: number of threads (unlimited, default: number of cores)
> > ...
> >
> > On 16C/32T AMD 7950X CPU I see 3200% CPU in top when starting
> > GP script with parforeach.
> >
> > So should above doc be corrected to
> >
> > "default: #cores * #threads_per_cor"
> >
> > or better to be correct for multi-CPU systems:
> When we wrote that, hyperthreading was only used in mainframes.
> It would be much better if the default was the number of cores instead of the number
> of hyperthreads.
> Unfortunately the GNU C library only report the number of CPU threads
> see getconf "_NPROCESSORS_CONF".
> I recommends to set nbthreads to the total number of cores.

Is there a difference with the simpler 'nproc' (from coreutils) ?

For systems with hyper-threading, one can get the number of physical cores
as follows on my laptop:

# grep '^cpu cores' /proc/cpuinfo | uniq
cpu cores       : 4


# nproc

Maybe we could include the number of physical cores in the output of
Configure --mt=pthreads ? And add to INSTALL.tex that we advise to
override the default value of nbthreads in gprc.

Not sure whether it would be wise to change the default in

   Maybe a good default could be to use the same method as nproc:
  look at how many virtual cores are allowed for gp. Assuming a
  32 virtual-core computer (16 physical cores, 2 threads/core),

  would have default nbthreads set to 32
taskset -c0-15 gp
  would have default nbthreads set to 16 (not 32)
taskset 0x8A gp
  would have default nbthreads set to 3

     I think this is what the user expects. If one runs a task on two
  threads sharing the same core, then performance (may) suffer, but
  not by that much. In all the tests I've done, running on half the virtual
  threads has always been worse than running on all. Hence defaulting
  on the number of physical cores may not be the best, while defaulting
  on what the user made available leaves open the possibility that the
  user knows what s/he wants.
  There exists also the possibility that the computer has 16 physical cores
  but the user is allowed to use only 4 threads, and defaulting nbthreads
  to 16 (or 32) is definitely bad.

     All in all, I remain convinced that the "nproc way", which is counting
  the number of virtual cores in the result of sched_getaffinity, is the
  best default nbthreads.

     I'd appreciate the possibility to ask pari to pin the threads
  to a specific (virtual)core, but that's a different matter...