Repository navigation
Architecture pass fixes: threads, GC and cppia - #475
Open
dimensionscape wants to merge 7 commits into
Open
dimensionscape wants to merge 7 commits into
dimensionscape wants to merge 7 commits into
Conversation
Sys.sleep, Sys.command and Sys.getch entered and left a zone with plain EnterGCFreeZone and ExitGCFreeZone. Called in a zone the caller had entered, as with cpp.vm.Gc.enterGCFreeZone around a blocking call, which docs/ThreadsAndStacks.md suggests, their exit ended the caller's zone, and the caller's own exit then stopped the process with "GCFree Zone mismatch". Mutex, Lock and Semaphore were made safe for this in 603fd03 and c3a9ee4. These three allocate nothing between entering and leaving, so they use AutoGCFreeZone, which keeps a zone the caller entered. CFFI's gc_safe_point was a copy of the old __hxcpp_gc_safe_point, calling PauseForCollect even in a zone, so an extension calling it between gc_enter_blocking and gc_exit_blocking ended the process or waited for a release that never came. It calls __hxcpp_gc_safe_point now. Test: GcFreeZoneLeafNatives. Before, the process stopped at the first Sys.sleep; now Sys.sleep and Sys.command work in a zone, and a thread calling gc_safe_point in one lets 100 collections through, 3 of 3 runs on Windows x64. Fixes #457
ThreadImpl.win32.cpp called GetThreadDescription and SetThreadDescription directly, and it is always linked, since Thread_obj::current() runs in InitAlloc. So every Windows program imported both from KERNEL32.dll at load time, and they only exist from Windows 10 1607: on Windows 8.1, Server 2012 R2 and early Windows 10, the program did not start, HXCPP_WINXP_COMPAT or not. They are looked up once with GetProcAddress. Where they are missing, a thread's name is null and setting it does nothing. UWP builds, which need Windows 10 and cannot use GetModuleHandle, call them directly. Test: WindowsThreadNameImports reads the program's own import table: KERNEL32.dll's entries named both before and neither now, on Windows x64. Fixes #456
TLSData with FAST set, which the collector's tlsStackContext is, read fs:[0xE10 + slot*4] for any slot on 32-bit Windows. Only the first 64 slots are inline in the thread environment block; the others are in TlsExpansionSlots, so for a slot of 64 or more the read landed on other fields of the block. In a process that had used 64 TLS slots before the runtime started, as a host with many DLLs can, every thread looked unregistered and the first allocation stopped the process. Slots of 64 and up go through TlsGetValue, as on x64. Reading the expansion array directly is not safe either, since a thread that never set such a slot has no array yet. Test: TlsHighSlot, where an MSVC CRT initializer takes TLS slots up to 70 before the runtime starts, and a second thread allocates too. The 32-bit build stopped with "Bad local allocator" before and runs now; the 64-bit build runs either way. Fixes #458
Since #420, CountingSemaphore::tryAcquire waited with no limit for any negative timeout, so that Haxe 5.0.0-preview.1's Lock.wait(), which passes -1 when given no timeout, would block. But a timeout computed as tryAcquire(deadline - now) goes negative once the deadline has passed, and then waited for ever, where Haxe documents tryAcquire as giving up when the timeout runs out. The Haxe nightly's Lock.wait calls acquire() when given no timeout, so only preview.1 needs the special case. Exactly -1 still waits with no limit, on every backend. Any other negative timeout returns at once, as 0 and NaN do. Test: ThreadWaitTimeouts checks -1 and -0.5 apart now. tryAcquire(-0.5) on an empty semaphore waited until the watchdog stopped the test before, and returns false at once now, on Windows x64. Fixes #461
hx::Object::_hx_getInterface read a script class's interfaceScriptTables with std::map::operator[], so Std.isOfType on a script object, or an interface cast, inserted an entry for each host interface the class did not implement, from whichever thread asked, with no lock, while other threads searched the same map. Threads asking about the same classes hung or crashed. HaxeNativeClass::findClass, hxObject, link's super class lookup and HaxeNativeInterface::findInterface inserted the names they missed into the host's class maps the same way, while loading a module on another thread walked them in link(), so loads failed with "No base defined for non-interface" or hung. The maps are filled when classes link or the host starts and only read after, with find(). link(), which sets the shared super classes on every load, takes a lock, and the module id counter is atomic. Test: CppiaThreadedLookups, 8 threads asking Std.isOfType about the same 2000 script objects, then 8 threads loading one module at once, with a watchdog. Before, 3 of 4 runs hung; now 5 of 5 pass, on Windows x64. Fixes #466
e1f6ac0 made a cpp.Pointer field, and a pointer boxed in a Dynamic, keep what it points into, through MarkConservativePointer. That went through the stack scan, which accepts a pointer only within sgCheckInternalOffset of an allocation's start: 64 bytes, or 32 on 32-bit. Right for a stack word, which is usually also held from the start somewhere, but a field may be all that holds it, and a pointer from cpp.Pointer.arrayElem(a, 100), 400 bytes in, kept nothing. MarkConservative's body takes the reach as a parameter now, and passes it to GetMemType, GetEnclosingAllocType and GetEnclosingNurseryType. A stack word keeps sgCheckInternalOffset; MarkConservativePointer reaches back to the start of the block, and a large allocation is checked against its own size. The walk back in a block stops at the block's start, and the large allocation bounds check no longer adds the reach to the highest base, which a large reach overflowed. Test: GcPointerFieldInterior, 2000 fields 40 and 400 bytes into Int arrays and one 200 KB into a large array. Before, 1999 of the 400-byte fields and the large one read freed memory, and on 32-bit all 4000 did; now none do, in the default, HXCPP_GC_MOVING and 32-bit builds on Windows. Fixes #451
With HXCPP_GC_INCREMENTAL: - A finished cycle that is not a full reclaim only runs countRows, whose bytesInUse is whole rows, and Collect stored that as mLiveBytes because the cycle had finished. MEM_INFO_LIVE, the bytes of the objects that survived the last full reclaim, then reported every used row as live, seven times the real figure in the test. It is set by full reclaims only. - Gc.trace counts what its collection marks. With a cycle part done, the collection finished it, and the objects its earlier slices had marked were not counted. A census already fails the partial cycle for this reason; Gc.trace does too now. Test: IncrementalGcCounts, which the incremental gc job builds in debug, as Gc.trace needs. Before, MEM_INFO_LIVE changed after the cycle, and Gc.trace counted 90668 and 108653 of 200000; now both are right, on Windows x64. Fixes #453
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The second set of fixes from the architecture pass in #154: threads, GC and cppia. Each commit has a regression test that fails without its fix and passes with it on Windows.
Threads
Sys.sleep,Sys.commandandSys.getchended a GC-free zone the caller had entered, so the caller's exit stopped the process with "GCFree Zone mismatch", and CFFI'sgc_safe_pointpaused inside a zone. Test: GcFreeZoneLeafNatives.GetThreadDescriptionandSetThreadDescription, so none started before Windows 10 1607. They are looked up when used. Test: WindowsThreadNameImports reads the program's own import table.Semaphore.tryAcquirewaited for ever for any negative timeout. Only -1, which Haxe 5.0.0-preview.1'sLock.wait()passes, does now. Test: ThreadWaitTimeouts.cppia
Std.isOfTypeon script objects, and loading modules, inserted into maps other threads were searching, which hung or failed loads. Test: CppiaThreadedLookups.GC
cpp.Pointerfield kept its target only within 64 bytes of its start. It keeps the whole allocation now, large ones included. Test: GcPointerFieldInterior.MEM_INFO_LIVEafter a cycle andGc.traceduring one were wrong. Test: IncrementalGcCounts, added to the incremental gc job, built in debug forGc.trace.Fixes #451
Fixes #453
Fixes #456
Fixes #457
Fixes #458
Fixes #461
Fixes #466