Skip to content

Architecture pass fixes: crashes, corruption and stalls - #474

Open
dimensionscape wants to merge 8 commits into
mainfrom
fix/arch-pass-1
Open

dimensionscape wants to merge 8 commits into
mainfrom
fix/arch-pass-1

Conversation

@dimensionscape

Copy link
Copy Markdown
Member

The first fixes from the architecture pass in #154: the crashes, memory corruption and stalls. Each commit has a regression test that fails without its fix and passes with it on Windows x64.

GC

Threads

cppia

Also run locally: gc stress under moving, always-move and generational (30 s each), the cppia suite interpreted and with the JIT, and the 59 Cppia regression tests.

Fixes #450
Fixes #452
Fixes #454
Fixes #455
Fixes #462
Fixes #463
Fixes #465

MoveBlocks walked each source block's start bits through a local copy,
so the block kept every bit. When the blocks to move into ran out part
way through one, that block was kept with the start bits of the objects
it had moved out, whose headers are IMMIX_OBJECT_HAS_MOVED. A stale
stack word holding one of those addresses then reached
GetAllocTypeChecked, which took the header's mark byte of 0 for the
previous mark one collection in 16, and MarkAllocUnchecked marked 0xfe
rows from there, past the 256 bytes of the row table and into the
block's first data rows. Once marked, the header looked live on every
later collection.

The block's own bit is cleared as each object moves, and
GetAllocTypeChecked refuses a moved header outright.

Test: GcMovedStartBits, under HXCPP_GC_MOVING, finds a block emptied in
part and puts 407 of its moved-out addresses on the stack: the
collector marked all 407 as objects before and none now, 3 of 3 runs
on Windows x64.

Fixes #450
Three array builtins wrote to the wrong memory on valid code:

- The JIT's a[i]++ with a Float result called runFloatCrement, which
  takes a double * for the result, with two arguments, so it stored 8
  bytes through whatever the third argument register held. It passes
  the result's address now.
- The JIT's typed push checked the capacity and expanded before it ran
  its argument, so an argument that grew the same array to its
  capacity left the store past the buffer. The argument runs first.
- The interpreter's ArraySetter took a reference to the element before
  running the right-hand side, so a right-hand side that grew or
  reallocated the array stored into the old buffer. It works on a copy
  of the element and finds the element after, which also stores
  a[a.length] = a.length as the JIT and the other targets do.

Test: CppiaArrayWrites, interpreted and under the JIT. Before, the JIT
corrupted the Int array and its neighbour, and the interpreter stored
1 for 42 and 43 and appended 4 for 3; both engines print the expected
output now, on Windows x64.

Fixes #463
#404 fixed three typed-array helpers. Many other native helpers of
jitted code had the same problem, in two forms:

- getFieldVariant, addVariantObject, addVariantString,
  setFieldVariant(V), createEnum and others caught into
  StackContext::exception, but the jitted code had no checkException
  after calling them, so it ran on past the throw. The exception was
  then held only by StackContext::exception, which the collector did
  not mark, so code that allocated before the next check could free
  it and crash the catch or the next call.
- dynamicCatDynamic, the Variant arithmetic helpers, the obj*Inc and
  obj*Dec helpers, getItem to setDynamicI and the DynIndex helpers,
  jitGet, jitSet and jitUpdate, the typed array setters and
  increments, array_expand(_size), the Array<Any> stores and the
  string iterator did not catch at all, so the exception unwound
  through jitted frames, which have no unwind information, past the
  script's catch.

Each of these helpers catches now, every call to one is followed by
checkException, and the collector marks a pending exception when it
scans a thread's script stack, pinned, as a stack word is.

The try blocks in the templated array helpers take ArrayBuiltin.cpp
past MSVC's default limit on sections in an object file, so MSVC
compiles with -bigobj always, as debug and sanitizer builds already
did.

Test: CppiaJitHelperExceptions. Before, the JIT let toString's
exception from d.s += t escape the script to the host's catch; now
each case is caught in its own function, interpreted and under the
JIT, and nothing runs after a throw, on Windows x64.

Fixes #465
Finalizers run on the collecting thread, inside the collection, with
gPauseForCollect still set. The uncontended paths of Mutex, Condition,
Semaphore, Deque and Lock call __hxcpp_gc_safe_point when a collection
is pending, which went to SafePoint and PauseForCollect, and that ends
the process when the collection is under way. Stock hxcpp took these
locks in a GC-free zone, which the collecting thread can enter.

sgIsCollecting is set only once every other thread is stopped or in a
zone, so a thread outside a zone that reaches SafePoint while it is set
is the collecting thread, and SafePoint returns for it.

Test: GcFinalizerTakesMutex, 1000 finalizers that take and try a
Mutex. Before, the process ended at the first; now all but those the
stack scan keeps run, 3 of 3 runs on Windows x64. A finalizer that
allocates, as Deque.add does, still ends the process, as it should.

Fixes #452
__trace, __hxcpp_print_string and __hxcpp_println_string wrote to
stdout outside a zone. When stdout blocks (a pipe nobody reads, a slow
reader, a Windows console with text selected), the writing thread
blocked outside the zone, so the next collection on any thread waited
for it, and every other thread for the collection. They make their
text first, since nothing may allocate in a zone, and write in one,
through OutputZone, which enters none on a thread the collector does
not know.

On Windows, _hx_std_process_stdin_close called CloseHandle outside a
zone, and CloseHandle waits for a WriteFile in progress on the pipe,
for as long as the child does not read. Writes to stdin are now counted
apart, as PipeCall counts every pipe call for destroy, and stdin_close
refuses new ones and cancels those in progress before it closes, in a
zone.

Test: GcStdoutBlocked, a child whose stdout fills while it collects:
println and trace never collected before, and now all three modes do.
ProcessStdinCloseCollects: the collection and the close finished only
when the child exited before, and while it runs now, on Windows x64.

Fixes #454
Haxe 4's haxe.atomic.AtomicInt is a cpp.Pointer<Int>, and its methods
call the _hx_atomic_* functions, none of which polled gPauseForCollect.
A thread spinning on load() waiting for a flag the main thread sets
after a collection kept that collection waiting for ever. Haxe 5's
hx::atomic::AtomicInt calls __hxcpp_gc_safe_point in each operation,
and #119 did the same for Mutex.tryAcquire.

Overloads of the _hx_atomic_* functions that take ::cpp::Pointer<int>,
which is what Haxe 4 passes, now stop for a pending collection and call
the volatile int * forms, which the runtime's own spinlocks keep using
unchanged. The __hxcpp_atomic_object_* operations stop too.

Test: AtomicSpinSafePoint, with a native watchdog. Before, the
collection never finished; now it does, and the spinning thread sees
the flag, on Windows x64.

Fixes #455
The module constructor gives the host's interface method names slots 2
to sgNativeNameSlotCount-1, which is sgNativeNameSlotCount-2 entries,
and getInterfaceSlot gave a new name interfaceSlots.size()+1, which
for the first is sgNativeNameSlotCount-1: the host's last slot. A class
implementing a script interface and a host interface whose method has
that slot (IMap's clear, in the test host) stored both methods in one
slot of its interface table.

New names now count from sgNativeNameSlotCount, or from 1 as before
when the host registers no names.

Test: CppiaInterfaceSlots, a class implementing two script interfaces
and haxe.Constraints.IMap. Before, both engines crashed; now each call
reaches its own method, interpreted and under the JIT, on Windows x64.

Fixes #462
The TSan job reported races that are the tests' own, not the runtime's:

- GcMovedStartBits reads the headers of moved-out objects on purpose,
  in blocks the collector's zeroing thread may be clearing. Its
  header read is not instrumented now.
- ProcessStdinCloseCollects closes a pipe while another thread writes
  to it. On Windows that is the case #454 fixed, and close waits for
  the write; elsewhere close does not wait, so the test only checks
  Windows and prints the same lines elsewhere.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment