ed19993b5b
This is a new implementation for condition variables, required after http://austingroupbugs.net/view.php?id=609 to fix bug 13165. In essence, we need to be stricter in which waiters a signal or broadcast is required to wake up; this couldn't be solved using the old algorithm. ISO C++ made a similar clarification, so this also fixes a bug in current libstdc++, for example. We can't use the old algorithm anymore because futexes do not guarantee to wake in FIFO order. Thus, when we wake, we can't simply let any waiter grab a signal, but we need to ensure that one of the waiters happening before the signal is woken up. This is something the previous algorithm violated (see bug 13165). There's another issue specific to condvars: ABA issues on the underlying futexes. Unlike mutexes that have just three states, or semaphores that have no tokens or a limited number of them, the state of a condvar is the *order* of the waiters. A waiter on a semaphore can grab a token whenever one is available; a condvar waiter must only consume a signal if it is eligible to do so as determined by the relative order of the waiter and the signal. Therefore, this new algorithm maintains two groups of waiters: Those eligible to consume signals (G1), and those that have to wait until previous waiters have consumed signals (G2). Once G1 is empty, G2 becomes the new G1. 64b counters are used to avoid ABA issues. This condvar doesn't yet use a requeue optimization (ie, on a broadcast, waking just one thread and requeueing all others on the futex of the mutex supplied by the program). I don't think doing the requeue is necessarily the right approach (but I haven't done real measurements yet): * If a program expects to wake many threads at the same time and make that scalable, a condvar isn't great anyway because of how it requires waiters to operate mutually exclusive (due to the mutex usage). Thus, a thundering herd problem is a scalability problem with or without the optimization. Using something like a semaphore might be more appropriate in such a case. * The scalability problem is actually at the mutex side; the condvar could help (and it tries to with the requeue optimization), but it should be the mutex who decides how that is done, and whether it is done at all. * Forcing all but one waiter into the kernel-side wait queue of the mutex prevents/avoids the use of lock elision on the mutex. Thus, it prevents the only cure against the underlying scalability problem inherent to condvars. * If condvars use short critical sections (ie, hold the mutex just to check a binary flag or such), which they should do ideally, then forcing all those waiter to proceed serially with kernel-based hand-off (ie, futex ops in the mutex' contended state, via the futex wait queues) will be less efficient than just letting a scalable mutex implementation take care of it. Our current mutex impl doesn't employ spinning at all, but if critical sections are short, spinning can be much better. * Doing the requeue stuff requires all waiters to always drive the mutex into the contended state. This leads to each waiter having to call futex_wake after lock release, even if this wouldn't be necessary. [BZ #13165] * nptl/pthread_cond_broadcast.c (__pthread_cond_broadcast): Rewrite to use new algorithm. * nptl/pthread_cond_destroy.c (__pthread_cond_destroy): Likewise. * nptl/pthread_cond_init.c (__pthread_cond_init): Likewise. * nptl/pthread_cond_signal.c (__pthread_cond_signal): Likewise. * nptl/pthread_cond_wait.c (__pthread_cond_wait): Likewise. (__pthread_cond_timedwait): Move here from pthread_cond_timedwait.c. (__condvar_confirm_wakeup, __condvar_cancel_waiting, __condvar_cleanup_waiting, __condvar_dec_grefs, __pthread_cond_wait_common): New. (__condvar_cleanup): Remove. * npt/pthread_condattr_getclock.c (pthread_condattr_getclock): Adapt. * npt/pthread_condattr_setclock.c (pthread_condattr_setclock): Likewise. * npt/pthread_condattr_getpshared.c (pthread_condattr_getpshared): Likewise. * npt/pthread_condattr_init.c (pthread_condattr_init): Likewise. * nptl/tst-cond1.c: Add comment. * nptl/tst-cond20.c (do_test): Adapt. * nptl/tst-cond22.c (do_test): Likewise. * sysdeps/aarch64/nptl/bits/pthreadtypes.h (pthread_cond_t): Adapt structure. * sysdeps/arm/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/ia64/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/m68k/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/microblaze/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/mips/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/nios2/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/s390/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/sh/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/tile/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/unix/sysv/linux/alpha/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/unix/sysv/linux/powerpc/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/x86/bits/pthreadtypes.h (pthread_cond_t): Likewise. * sysdeps/nptl/internaltypes.h (COND_NWAITERS_SHIFT): Remove. (COND_CLOCK_BITS): Adapt. * sysdeps/nptl/pthread.h (PTHREAD_COND_INITIALIZER): Adapt. * nptl/pthreadP.h (__PTHREAD_COND_CLOCK_MONOTONIC_MASK, __PTHREAD_COND_SHARED_MASK): New. * nptl/nptl-printers.py (CLOCK_IDS): Remove. (ConditionVariablePrinter, ConditionVariableAttributesPrinter): Adapt. * nptl/nptl_lock_constants.pysym: Adapt. * nptl/test-cond-printers.py: Adapt. * sysdeps/unix/sysv/linux/hppa/internaltypes.h (cond_compat_clear, cond_compat_check_and_clear): Adapt. * sysdeps/unix/sysv/linux/hppa/pthread_cond_timedwait.c: Remove file ... * sysdeps/unix/sysv/linux/hppa/pthread_cond_wait.c (__pthread_cond_timedwait): ... and move here. * nptl/DESIGN-condvar.txt: Remove file. * nptl/lowlevelcond.sym: Likewise. * nptl/pthread_cond_timedwait.c: Likewise. * sysdeps/unix/sysv/linux/i386/i486/pthread_cond_broadcast.S: Likewise. * sysdeps/unix/sysv/linux/i386/i486/pthread_cond_signal.S: Likewise. * sysdeps/unix/sysv/linux/i386/i486/pthread_cond_timedwait.S: Likewise. * sysdeps/unix/sysv/linux/i386/i486/pthread_cond_wait.S: Likewise. * sysdeps/unix/sysv/linux/i386/i586/pthread_cond_broadcast.S: Likewise. * sysdeps/unix/sysv/linux/i386/i586/pthread_cond_signal.S: Likewise. * sysdeps/unix/sysv/linux/i386/i586/pthread_cond_timedwait.S: Likewise. * sysdeps/unix/sysv/linux/i386/i586/pthread_cond_wait.S: Likewise. * sysdeps/unix/sysv/linux/i386/i686/pthread_cond_broadcast.S: Likewise. * sysdeps/unix/sysv/linux/i386/i686/pthread_cond_signal.S: Likewise. * sysdeps/unix/sysv/linux/i386/i686/pthread_cond_timedwait.S: Likewise. * sysdeps/unix/sysv/linux/i386/i686/pthread_cond_wait.S: Likewise. * sysdeps/unix/sysv/linux/x86_64/pthread_cond_broadcast.S: Likewise. * sysdeps/unix/sysv/linux/x86_64/pthread_cond_signal.S: Likewise. * sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S: Likewise. * sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: Likewise.
180 lines
4.8 KiB
C
180 lines
4.8 KiB
C
/* Copyright (C) 2002-2016 Free Software Foundation, Inc.
|
|
This file is part of the GNU C Library.
|
|
Contributed by Ulrich Drepper <drepper@redhat.com>, 2002.
|
|
|
|
The GNU C Library is free software; you can redistribute it and/or
|
|
modify it under the terms of the GNU Lesser General Public
|
|
License as published by the Free Software Foundation; either
|
|
version 2.1 of the License, or (at your option) any later version.
|
|
|
|
The GNU C Library is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
Lesser General Public License for more details.
|
|
|
|
You should have received a copy of the GNU Lesser General Public
|
|
License along with the GNU C Library; if not, see
|
|
<http://www.gnu.org/licenses/>. */
|
|
|
|
#ifndef _INTERNALTYPES_H
|
|
#define _INTERNALTYPES_H 1
|
|
|
|
#include <stdint.h>
|
|
#include <atomic.h>
|
|
#include <endian.h>
|
|
|
|
|
|
struct pthread_attr
|
|
{
|
|
/* Scheduler parameters and priority. */
|
|
struct sched_param schedparam;
|
|
int schedpolicy;
|
|
/* Various flags like detachstate, scope, etc. */
|
|
int flags;
|
|
/* Size of guard area. */
|
|
size_t guardsize;
|
|
/* Stack handling. */
|
|
void *stackaddr;
|
|
size_t stacksize;
|
|
/* Affinity map. */
|
|
cpu_set_t *cpuset;
|
|
size_t cpusetsize;
|
|
};
|
|
|
|
#define ATTR_FLAG_DETACHSTATE 0x0001
|
|
#define ATTR_FLAG_NOTINHERITSCHED 0x0002
|
|
#define ATTR_FLAG_SCOPEPROCESS 0x0004
|
|
#define ATTR_FLAG_STACKADDR 0x0008
|
|
#define ATTR_FLAG_OLDATTR 0x0010
|
|
#define ATTR_FLAG_SCHED_SET 0x0020
|
|
#define ATTR_FLAG_POLICY_SET 0x0040
|
|
|
|
|
|
/* Mutex attribute data structure. */
|
|
struct pthread_mutexattr
|
|
{
|
|
/* Identifier for the kind of mutex.
|
|
|
|
Bit 31 is set if the mutex is to be shared between processes.
|
|
|
|
Bit 0 to 30 contain one of the PTHREAD_MUTEX_ values to identify
|
|
the type of the mutex. */
|
|
int mutexkind;
|
|
};
|
|
|
|
|
|
/* Conditional variable attribute data structure. */
|
|
struct pthread_condattr
|
|
{
|
|
/* Combination of values:
|
|
|
|
Bit 0 : flag whether conditional variable will be
|
|
sharable between processes.
|
|
Bit 1-COND_CLOCK_BITS: Clock ID. COND_CLOCK_BITS is the number of bits
|
|
needed to represent the ID of the clock. */
|
|
int value;
|
|
};
|
|
#define COND_CLOCK_BITS 1
|
|
|
|
|
|
/* Read-write lock variable attribute data structure. */
|
|
struct pthread_rwlockattr
|
|
{
|
|
int lockkind;
|
|
int pshared;
|
|
};
|
|
|
|
|
|
/* Barrier data structure. See pthread_barrier_wait for a description
|
|
of how these fields are used. */
|
|
struct pthread_barrier
|
|
{
|
|
unsigned int in;
|
|
unsigned int current_round;
|
|
unsigned int count;
|
|
int shared;
|
|
unsigned int out;
|
|
};
|
|
/* See pthread_barrier_wait for a description. */
|
|
#define BARRIER_IN_THRESHOLD (UINT_MAX/2)
|
|
|
|
|
|
/* Barrier variable attribute data structure. */
|
|
struct pthread_barrierattr
|
|
{
|
|
int pshared;
|
|
};
|
|
|
|
|
|
/* Thread-local data handling. */
|
|
struct pthread_key_struct
|
|
{
|
|
/* Sequence numbers. Even numbers indicated vacant entries. Note
|
|
that zero is even. We use uintptr_t to not require padding on
|
|
32- and 64-bit machines. On 64-bit machines it helps to avoid
|
|
wrapping, too. */
|
|
uintptr_t seq;
|
|
|
|
/* Destructor for the data. */
|
|
void (*destr) (void *);
|
|
};
|
|
|
|
/* Check whether an entry is unused. */
|
|
#define KEY_UNUSED(p) (((p) & 1) == 0)
|
|
/* Check whether a key is usable. We cannot reuse an allocated key if
|
|
the sequence counter would overflow after the next destroy call.
|
|
This would mean that we potentially free memory for a key with the
|
|
same sequence. This is *very* unlikely to happen, A program would
|
|
have to create and destroy a key 2^31 times (on 32-bit platforms,
|
|
on 64-bit platforms that would be 2^63). If it should happen we
|
|
simply don't use this specific key anymore. */
|
|
#define KEY_USABLE(p) (((uintptr_t) (p)) < ((uintptr_t) ((p) + 2)))
|
|
|
|
|
|
/* Handling of read-write lock data. */
|
|
// XXX For now there is only one flag. Maybe more in future.
|
|
#define RWLOCK_RECURSIVE(rwlock) ((rwlock)->__data.__flags != 0)
|
|
|
|
|
|
/* Semaphore variable structure. */
|
|
struct new_sem
|
|
{
|
|
#if __HAVE_64B_ATOMICS
|
|
/* The data field holds both value (in the least-significant 32 bytes) and
|
|
nwaiters. */
|
|
# if __BYTE_ORDER == __LITTLE_ENDIAN
|
|
# define SEM_VALUE_OFFSET 0
|
|
# elif __BYTE_ORDER == __BIG_ENDIAN
|
|
# define SEM_VALUE_OFFSET 1
|
|
# else
|
|
# error Unsupported byte order.
|
|
# endif
|
|
# define SEM_NWAITERS_SHIFT 32
|
|
# define SEM_VALUE_MASK (~(unsigned int)0)
|
|
uint64_t data;
|
|
int private;
|
|
int pad;
|
|
#else
|
|
# define SEM_VALUE_SHIFT 1
|
|
# define SEM_NWAITERS_MASK ((unsigned int)1)
|
|
unsigned int value;
|
|
int private;
|
|
int pad;
|
|
unsigned int nwaiters;
|
|
#endif
|
|
};
|
|
|
|
struct old_sem
|
|
{
|
|
unsigned int value;
|
|
};
|
|
|
|
|
|
/* Compatibility type for old conditional variable interfaces. */
|
|
typedef struct
|
|
{
|
|
pthread_cond_t *cond;
|
|
} pthread_cond_2_0_t;
|
|
|
|
#endif /* internaltypes.h */
|