Skip to content

Architecture pass fixes: threads, GC and cppia - #475

Open
dimensionscape wants to merge 7 commits into
mainfrom
fix/arch-pass-2
Open

dimensionscape wants to merge 7 commits into
mainfrom
fix/arch-pass-2

Conversation

@dimensionscape

Copy link
Copy Markdown
Member

The second set of fixes from the architecture pass in #154: threads, GC and cppia. Each commit has a regression test that fails without its fix and passes with it on Windows.

Threads

cppia

GC

Fixes #451
Fixes #453
Fixes #456
Fixes #457
Fixes #458
Fixes #461
Fixes #466

Sys.sleep, Sys.command and Sys.getch entered and left a zone with plain
EnterGCFreeZone and ExitGCFreeZone. Called in a zone the caller had
entered, as with cpp.vm.Gc.enterGCFreeZone around a blocking call, which
docs/ThreadsAndStacks.md suggests, their exit ended the caller's zone,
and the caller's own exit then stopped the process with "GCFree Zone
mismatch". Mutex, Lock and Semaphore were made safe for this in
603fd03 and c3a9ee4. These three allocate nothing between entering
and leaving, so they use AutoGCFreeZone, which keeps a zone the caller
entered.

CFFI's gc_safe_point was a copy of the old __hxcpp_gc_safe_point,
calling PauseForCollect even in a zone, so an extension calling it
between gc_enter_blocking and gc_exit_blocking ended the process or
waited for a release that never came. It calls __hxcpp_gc_safe_point
now.

Test: GcFreeZoneLeafNatives. Before, the process stopped at the first
Sys.sleep; now Sys.sleep and Sys.command work in a zone, and a thread
calling gc_safe_point in one lets 100 collections through, 3 of 3 runs
on Windows x64.

Fixes #457
ThreadImpl.win32.cpp called GetThreadDescription and
SetThreadDescription directly, and it is always linked, since
Thread_obj::current() runs in InitAlloc. So every Windows program
imported both from KERNEL32.dll at load time, and they only exist from
Windows 10 1607: on Windows 8.1, Server 2012 R2 and early Windows 10,
the program did not start, HXCPP_WINXP_COMPAT or not.

They are looked up once with GetProcAddress. Where they are missing,
a thread's name is null and setting it does nothing. UWP builds, which
need Windows 10 and cannot use GetModuleHandle, call them directly.

Test: WindowsThreadNameImports reads the program's own import table:
KERNEL32.dll's entries named both before and neither now, on Windows
x64.

Fixes #456
TLSData with FAST set, which the collector's tlsStackContext is, read
fs:[0xE10 + slot*4] for any slot on 32-bit Windows. Only the first 64
slots are inline in the thread environment block; the others are in
TlsExpansionSlots, so for a slot of 64 or more the read landed on other
fields of the block. In a process that had used 64 TLS slots before
the runtime started, as a host with many DLLs can, every thread looked
unregistered and the first allocation stopped the process.

Slots of 64 and up go through TlsGetValue, as on x64. Reading the
expansion array directly is not safe either, since a thread that never
set such a slot has no array yet.

Test: TlsHighSlot, where an MSVC CRT initializer takes TLS slots up to
70 before the runtime starts, and a second thread allocates too. The
32-bit build stopped with "Bad local allocator" before and runs now;
the 64-bit build runs either way.

Fixes #458
Since #420, CountingSemaphore::tryAcquire waited with no limit for any
negative timeout, so that Haxe 5.0.0-preview.1's Lock.wait(), which
passes -1 when given no timeout, would block. But a timeout computed
as tryAcquire(deadline - now) goes negative once the deadline has
passed, and then waited for ever, where Haxe documents tryAcquire as
giving up when the timeout runs out. The Haxe nightly's Lock.wait
calls acquire() when given no timeout, so only preview.1 needs the
special case.

Exactly -1 still waits with no limit, on every backend. Any other
negative timeout returns at once, as 0 and NaN do.

Test: ThreadWaitTimeouts checks -1 and -0.5 apart now. tryAcquire(-0.5)
on an empty semaphore waited until the watchdog stopped the test
before, and returns false at once now, on Windows x64.

Fixes #461
hx::Object::_hx_getInterface read a script class's
interfaceScriptTables with std::map::operator[], so Std.isOfType on a
script object, or an interface cast, inserted an entry for each host
interface the class did not implement, from whichever thread asked,
with no lock, while other threads searched the same map. Threads
asking about the same classes hung or crashed.

HaxeNativeClass::findClass, hxObject, link's super class lookup and
HaxeNativeInterface::findInterface inserted the names they missed into
the host's class maps the same way, while loading a module on another
thread walked them in link(), so loads failed with "No base defined for
non-interface" or hung.

The maps are filled when classes link or the host starts and only read
after, with find(). link(), which sets the shared super classes on
every load, takes a lock, and the module id counter is atomic.

Test: CppiaThreadedLookups, 8 threads asking Std.isOfType about the
same 2000 script objects, then 8 threads loading one module at once,
with a watchdog. Before, 3 of 4 runs hung; now 5 of 5 pass, on Windows
x64.

Fixes #466
e1f6ac0 made a cpp.Pointer field, and a pointer boxed in a Dynamic,
keep what it points into, through MarkConservativePointer. That went
through the stack scan, which accepts a pointer only within
sgCheckInternalOffset of an allocation's start: 64 bytes, or 32 on
32-bit. Right for a stack word, which is usually also held from the
start somewhere, but a field may be all that holds it, and a pointer
from cpp.Pointer.arrayElem(a, 100), 400 bytes in, kept nothing.

MarkConservative's body takes the reach as a parameter now, and passes
it to GetMemType, GetEnclosingAllocType and GetEnclosingNurseryType.
A stack word keeps sgCheckInternalOffset; MarkConservativePointer
reaches back to the start of the block, and a large allocation is
checked against its own size. The walk back in a block stops at the
block's start, and the large allocation bounds check no longer adds
the reach to the highest base, which a large reach overflowed.

Test: GcPointerFieldInterior, 2000 fields 40 and 400 bytes into Int
arrays and one 200 KB into a large array. Before, 1999 of the 400-byte
fields and the large one read freed memory, and on 32-bit all 4000 did;
now none do, in the default, HXCPP_GC_MOVING and 32-bit builds on
Windows.

Fixes #451
With HXCPP_GC_INCREMENTAL:

- A finished cycle that is not a full reclaim only runs countRows,
  whose bytesInUse is whole rows, and Collect stored that as
  mLiveBytes because the cycle had finished. MEM_INFO_LIVE, the bytes
  of the objects that survived the last full reclaim, then reported
  every used row as live, seven times the real figure in the test. It
  is set by full reclaims only.
- Gc.trace counts what its collection marks. With a cycle part done,
  the collection finished it, and the objects its earlier slices had
  marked were not counted. A census already fails the partial cycle
  for this reason; Gc.trace does too now.

Test: IncrementalGcCounts, which the incremental gc job builds in
debug, as Gc.trace needs. Before, MEM_INFO_LIVE changed after the
cycle, and Gc.trace counted 90668 and 108653 of 200000; now both are
right, on Windows x64.

Fixes #453
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment