- 26 Jun, 2024 1 commit
-
-
Kirill Smelkov authored
Generate g for today's state of Go 1.23 (go1.23rc1-0-g7dff7439dc). Compared to Go1.22 many new fields and several new types were introduced as shown by the diff below. Regenerated files stay without changes for Go1.22 and previous releases. Support for Go1.23 remains incomplete because currently there is no way to access runtime.stopTheWorld via go:linkname : https://github.com/golang/go/issues/68167#issuecomment-2192263282 ---- 8< ---- diff --git a/zruntime_g_go1.22.go b/zruntime_g_go1.23.go index 910382b..4aa6799 100644 --- a/zruntime_g_go1.22.go +++ b/zruntime_g_go1.23.go @@ -1,7 +1,7 @@ // Code generated by g_typedef; DO NOT EDIT. -//go:build go1.22 && !go1.23 -// +build go1.22,!go1.23 +//go:build go1.23 && !go1.24 +// +build go1.23,!go1.24 package xruntime @@ -26,6 +26,7 @@ type g struct { sched gobuf syscallsp uintptr // if status==Gsyscall, syscallsp = sched.sp to use during gc syscallpc uintptr // if status==Gsyscall, syscallpc = sched.pc to use during gc + syscallbp uintptr // if status==Gsyscall, syscallbp = sched.bp to use in fpTraceback stktopsp uintptr // expected sp at top of stack, to check in traceback // param is a generic pointer parameter field used to pass // values in particular contexts where other storage for the @@ -95,14 +96,15 @@ type g struct { cgoCtxt []uintptr // cgo traceback context labels unsafe.Pointer // profiler labels timer *timer // cached timer for time.Sleep + sleepWhen int64 // when to sleep until selectDone atomic.Uint32 // are we participating in a select and did someone win the race? - coroarg *coro // argument during coroutine transfers - // goroutineProfiled indicates the status of this goroutine's stack for the // current in-progress goroutine profile goroutineProfiled goroutineProfileStateHolder + coroarg *coro // argument during coroutine transfers + // Per-G tracer state. trace gTraceState @@ -182,27 +184,51 @@ type funcval struct { fn uintptr } type timer struct { - // If this timer is on a heap, which P's heap it is on. - // puintptr rather than *p to match uintptr in the versions - // of this struct defined in other packages. - pp puintptr + // mu protects reads and writes to all fields, with exceptions noted below. + mu mutex + + astate uint8 // atomic copy of state bits at last unlock + state uint8 // state bits + isChan bool // timer has a channel; immutable; can be read without lock + blocked uint32 // number of goroutines blocked on timer's channel // Timer wakes up at when, and then at when+period, ... (period > 0 only) - // each time calling f(arg, now) in the timer goroutine, so f must be + // each time calling f(arg, seq, delay) in the timer goroutine, so f must be // a well-behaved function and not block. // - // when must be positive on an active timer. + // The arg and seq are client-specified opaque arguments passed back to f. + // When used from netpoll, arg and seq have meanings defined by netpoll + // and are completely opaque to this code; in that context, seq is a sequence + // number to recognize and squech stale function invocations. + // When used from package time, arg is a channel (for After, NewTicker) + // or the function to call (for AfterFunc) and seq is unused (0). + // + // Package time does not know about seq, but if this is a channel timer (t.isChan == true), + // this file uses t.seq as a sequence number to recognize and squelch + // sends that correspond to an earlier (stale) timer configuration, + // similar to its use in netpoll. In this usage (that is, when t.isChan == true), + // writes to seq are protected by both t.mu and t.sendLock, + // so reads are allowed when holding either of the two mutexes. + // + // The delay argument is nanotime() - t.when, meaning the delay in ns between + // when the timer should have gone off and now. Normally that amount is + // small enough not to matter, but for channel timers that are fed lazily, + // the delay can be arbitrarily long; package time subtracts it out to make + // it look like the send happened earlier than it actually did. + // (No one looked at the channel since then, or the send would have + // not happened so late, so no one can tell the difference.) when int64 period int64 - f func(interface{}, uintptr) + f func(arg interface{}, seq uintptr, delay int64) arg interface{} seq uintptr - // What to set the when field to in timerModifiedXX status. - nextwhen int64 + // If non-nil, the timers containing t. + ts *timers - // The status field holds one of the values below. - status atomic.Uint32 + // sendLock protects sends on the timer's channel. + // Not used for async (pre-Go 1.23) behavior when debug.asynctimerchan.Load() != 0. + sendLock mutex } type guintptr uintptr type puintptr uintptr @@ -221,6 +247,11 @@ type traceTime uint64 type coro struct { gp guintptr f func(*coro) + + // State for validating thread-lock interactions. + mp *m + lockedExt uint32 // mp's external LockOSThread counter at coro creation time. + lockedInt uint32 // mp's internal lockOSThread counter at coro creation time. } type traceSchedResourceState struct { // statusTraced indicates whether a status event was traced for this resource @@ -240,7 +271,18 @@ type traceSchedResourceState struct { // GoStatus and GoCreate events to omit a sequence number (implicitly 0). seq [2]uint64 } +type mutex struct { + // Empty struct if lock ranking is disabled, otherwise includes the lock rank + lockRankStruct + // Futex-based impl treats it as uint32 key, + // while sema-based impl as M* waitm. + // Used to be a union, but unions break precise GC. + key uintptr +} +type lockRankStruct struct { +} type uintreg uint // FIXME wrong on amd64p32 type m struct{} // FIXME stub type sudog struct{} // FIXME stub type timersBucket struct{} // FIXME stub +type timers struct{} // FIXME stub
-
- 07 Feb, 2024 1 commit
-
-
Kirill Smelkov authored
Generate g for today's state of Go 1.22 (go1.22.0-0-ga10e42f219). Compared to Go1.21 many new fields and a couple of new types were introduced as shown by the diff below. Regenerated files stay without changes for Go1.21 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.21.go b/zruntime_g_go1.22.go index c55d865..910382b 100644 --- a/zruntime_g_go1.21.go +++ b/zruntime_g_go1.22.go @@ -1,7 +1,7 @@ // Code generated by g_typedef; DO NOT EDIT. -//go:build go1.21 && !go1.22 -// +build go1.21,!go1.22 +//go:build go1.22 && !go1.23 +// +build go1.22,!go1.23 package xruntime @@ -13,7 +13,7 @@ type g struct { // stack describes the actual stack memory: [stack.lo, stack.hi). // stackguard0 is the stack pointer compared in the Go stack growth prologue. // It is stack.lo+StackGuard normally, but can be StackPreempt to trigger a preemption. - // stackguard1 is the stack pointer compared in the C stack growth prologue. + // stackguard1 is the stack pointer compared in the //go:systemstack stack growth prologue. // It is stack.lo+StackGuard on g0 and gsignal stacks. // It is ~0 on other goroutine stacks, to trigger a call to morestackc (and crash). stack stack // offset known to runtime/cgo @@ -30,7 +30,7 @@ type g struct { // param is a generic pointer parameter field used to pass // values in particular contexts where other storage for the // parameter would be difficult to find. It is currently used - // in three ways: + // in four ways: // 1. When a channel operation wakes up a blocked goroutine, it sets param to // point to the sudog of the completed blocking operation. // 2. By gcAssistAlloc1 to signal back to its caller that the goroutine completed @@ -38,6 +38,8 @@ type g struct { // stack may have moved in the meantime. // 3. By debugCallWrap to pass parameters to a new goroutine because allocating a // closure in the runtime is forbidden. + // 4. When a panic is recovered and control returns to the respective frame, + // param may point to a savedOpenDeferState. param unsafe.Pointer atomicstatus atomic.Uint32 stackLock uint32 // sigprof/scang lock; TODO: fold in to atomicstatus @@ -67,8 +69,13 @@ type g struct { // park on a chansend or chanrecv. Used to signal an unsafe point // for stack shrinking. parkingOnChan atomic.Bool + // inMarkAssist indicates whether the goroutine is in mark assist. + // Used by the execution tracer. + inMarkAssist bool + coroexit bool // argument to coroswitch_m raceignore int8 // ignore race detection events + nocgocallback bool // whether disable callback from C tracking bool // whether we're tracking this G for sched latency statistics trackingSeq uint8 // used to decide whether to track this G trackingStamp int64 // timestamp of when the G last started being tracked @@ -90,6 +97,8 @@ type g struct { timer *timer // cached timer for time.Sleep selectDone atomic.Uint32 // are we participating in a select and did someone win the race? + coroarg *coro // argument during coroutine transfers + // goroutineProfiled indicates the status of this goroutine's stack for the // current in-progress goroutine profile goroutineProfiled goroutineProfileStateHolder @@ -114,36 +123,39 @@ type _panic struct { argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink arg interface{} // argument to panic link *_panic // link to earlier panic - pc uintptr // where to return to in runtime if this panic is bypassed - sp unsafe.Pointer // where to return to in runtime if this panic is bypassed - recovered bool // whether this panic is over - aborted bool // the panic was aborted + + // startPC and startSP track where _panic.start was called. + startPC uintptr + startSP unsafe.Pointer + + // The current stack frame that we're running deferred calls for. + sp unsafe.Pointer + lr uintptr + fp unsafe.Pointer + + // retpc stores the PC where the panic should jump back to, if the + // function last returned by _panic.next() recovers the panic. + retpc uintptr + + // Extra state for handling open-coded defers. + deferBitsPtr *uint8 + slotsPtr unsafe.Pointer + + recovered bool // whether this panic has been recovered goexit bool + deferreturn bool } type _defer struct { - started bool heap bool - // openDefer indicates that this _defer is for a frame with open-coded - // defers. We have only one defer record for the entire frame (which may - // currently have 0, 1, or more defers active). - openDefer bool + rangefunc bool // true for rangefunc list sp uintptr // sp at time of defer pc uintptr // pc at time of defer fn func() // can be nil for open-coded defers - _panic *_panic // panic that is running defer link *_defer // next defer on G; can point to either heap or stack! - // If openDefer is true, the fields below record values about the stack - // frame and associated function that has the open-coded defer(s). sp - // above will be the sp for the frame, and pc will be address of the - // deferreturn call in the function. - fd unsafe.Pointer // funcdata for the function associated with the frame - varp uintptr // value of varp for the stack frame - // framepc is the current pc associated with the stack frame. Together, - // with sp above (which is the sp associated with the stack frame), - // framepc/sp can be used as pc/sp pair to continue a stack trace via - // gentraceback(). - framepc uintptr + // If rangefunc is true, *head is the head of the atomic linked list + // during a range-over-func execution. + head *atomic.Pointer[_defer] } type gobuf struct { // The offsets of sp, pc, and g are known to (hard-coded in) libmach. @@ -203,12 +215,31 @@ type ancestorInfo struct { } type goroutineProfileStateHolder atomic.Uint32 type gTraceState struct { - sysExitTime traceTime // timestamp when syscall has returned - tracedSyscallEnter bool // syscall or cgo was entered while trace was enabled or StartTrace has emitted EvGoInSyscall about this goroutine - seq uint64 // trace event sequencer - lastP puintptr // last P emitted an event for this goroutine + traceSchedResourceState } type traceTime uint64 +type coro struct { + gp guintptr + f func(*coro) +} +type traceSchedResourceState struct { + // statusTraced indicates whether a status event was traced for this resource + // a particular generation. + // + // There are 3 of these because when transitioning across generations, traceAdvance + // needs to be able to reliably observe whether a status was traced for the previous + // generation, while we need to clear the value for the next generation. + statusTraced [3]atomic.Uint32 + + // seq is the sequence counter for this scheduling resource's events. + // The purpose of the sequence counter is to establish a partial order between + // events that don't obviously happen serially (same M) in the stream ofevents. + // + // There are two of these so that we can reset the counter on each generation. + // This saves space in the resulting trace by keeping the counter small and allows + // GoStatus and GoCreate events to omit a sequence number (implicitly 0). + seq [2]uint64 +} type uintreg uint // FIXME wrong on amd64p32 type m struct{} // FIXME stub type sudog struct{} // FIXME stub
-
- 22 Aug, 2023 2 commits
-
-
Kirill Smelkov authored
Fix StopTheWorld crash due to change in runtime.stopTheWorld signature: In Go1.21 stopTheWorld changed from accepting reason as string into accepting reason as uint8 code: https://github.com/golang/go/commit/b1aadd034c1f https://golang.org/cl/494495 which leads to the following crash if reason is still passed as string: fatal error: index out of range goroutine 6 [running]: runtime.throw({0x531aad?, 0x0?}) /home/kirr/src/tools/go/go/src/runtime/panic.go:1077 +0x5c fp=0xc000042608 sp=0xc0000425d8 pc=0x4365bc runtime.panicCheck1(0x4785af?, {0x531aad, 0x12}) /home/kirr/src/tools/go/go/src/runtime/panic.go:58 +0x94 fp=0xc000042628 sp=0xc000042608 pc=0x434034 runtime.goPanicIndex(0x8d, 0x11) /home/kirr/src/tools/go/go/src/runtime/panic.go:113 +0x2e fp=0xc000042668 sp=0xc000042628 pc=0x4340ee runtime.stwReason.String(...) /home/kirr/src/tools/go/go/src/runtime/proc.go:1217 runtime.stopTheWorld(0x8d) /home/kirr/src/tools/go/go/src/runtime/proc.go:1260 +0xe6 fp=0xc0000426c8 sp=0xc000042668 pc=0x43b3a6 lab.nexedi.com/kirr/go123/tracing/internal/xruntime.StopTheWorld(...) /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/tracing/internal/xruntime/runtime.go:40 lab.nexedi.com/kirr/go123/tracing/internal/xruntime.TestStartStopTheWorld(0xc000007860) /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/tracing/internal/xruntime/runtime_test.go:75 +0x139 fp=0xc000042770 sp=0xc0000426c8 pc=0x4fa0f9 testing.tRunner(0xc000007860, 0x53a8b8) /home/kirr/src/tools/go/go/src/testing/testing.go:1595 +0xff fp=0xc0000427c0 sp=0xc000042770 pc=0x4bbd9f testing.(*T).Run.func1() /home/kirr/src/tools/go/go/src/testing/testing.go:1648 +0x25 fp=0xc0000427e0 sp=0xc0000427c0 pc=0x4bcd25 runtime.goexit() /home/kirr/src/tools/go/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x468901 created by testing.(*T).Run in goroutine 1 /home/kirr/src/tools/go/go/src/testing/testing.go:1648 +0x3ad -> Fix it by accounting that STW reason is uint8 on go1.21 and passing it to runtime correspondingly. Sadly we cannot express arbitrary string reason into fixed list of reasons that runtime and stdlib tracing are aware of, so we always use "unknown" for custom stop-the-world requests.
-
Kirill Smelkov authored
Generate g for today's state of Go 1.21 (go1.21.0-15-g70aa116c4a). Compared to Go1.20 new field parentGoid was added and tracing-related fields were moved to gTraceState struct. Regenerated files stay without changes for Go1.20 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.20.go b/zruntime_g_go1.21.go index 99f483a..c55d865 100644 --- a/zruntime_g_go1.20.go +++ b/zruntime_g_go1.21.go @@ -1,7 +1,7 @@ // Code generated by g_typedef; DO NOT EDIT. -//go:build go1.20 && !go1.21 -// +build go1.20,!go1.21 +//go:build go1.21 && !go1.22 +// +build go1.21,!go1.22 package xruntime @@ -69,20 +69,17 @@ type g struct { parkingOnChan atomic.Bool raceignore int8 // ignore race detection events - sysblocktraced bool // StartTrace has emitted EvGoInSyscall about this goroutine tracking bool // whether we're tracking this G for sched latency statistics trackingSeq uint8 // used to decide whether to track this G trackingStamp int64 // timestamp of when the G last started being tracked runnableTime int64 // the amount of time spent runnable, cleared when running, only used when tracking - sysexitticks int64 // cputicks when syscall has returned (for tracing) - traceseq uint64 // trace event sequencer - tracelastp puintptr // last P emitted an event for this goroutine lockedm muintptr sig uint32 writebuf []byte sigcode0 uintptr sigcode1 uintptr sigpc uintptr + parentGoid uint64 // goid of goroutine that created this goroutine gopc uintptr // pc of go statement that created this goroutine ancestors *[]ancestorInfo // ancestor information goroutine(s) that created this goroutine (only used if debug.tracebackancestors) startpc uintptr // pc of goroutine function @@ -97,6 +94,9 @@ type g struct { // current in-progress goroutine profile goroutineProfiled goroutineProfileStateHolder + // Per-G tracer state. + trace gTraceState + // gcAssistBytes is this G's GC assist credit in terms of // bytes allocated. If this is positive, then the G has credit // to allocate gcAssistBytes bytes without assisting. If this @@ -202,6 +202,13 @@ type ancestorInfo struct { gopc uintptr // pc of go statement that created this goroutine } type goroutineProfileStateHolder atomic.Uint32 +type gTraceState struct { + sysExitTime traceTime // timestamp when syscall has returned + tracedSyscallEnter bool // syscall or cgo was entered while trace was enabled or StartTrace has emitted EvGoInSyscall about this goroutine + seq uint64 // trace event sequencer + lastP puintptr // last P emitted an event for this goroutine +} +type traceTime uint64 type uintreg uint // FIXME wrong on amd64p32 type m struct{} // FIXME stub type sudog struct{} // FIXME stub
-
- 14 Jul, 2023 1 commit
-
-
Kirill Smelkov authored
-
- 05 Feb, 2023 1 commit
-
-
Kirill Smelkov authored
Generate g for today's state of Go 1.20 (go1.20-0-gde4748c47c). Compared to Go1.19 goid is changed from int64 -> to uint64, and that more variables, that are used atomically, now use atomic.X types. Regenerated files stay without changes for Go1.19 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.19.go b/zruntime_g_go1.20.go index 707aadd..99f483a 100644 --- a/zruntime_g_go1.19.go +++ b/zruntime_g_go1.20.go @@ -1,7 +1,7 @@ // Code generated by g_typedef; DO NOT EDIT. -//go:build go1.19 && !go1.20 -// +build go1.19,!go1.20 +//go:build go1.20 && !go1.21 +// +build go1.20,!go1.21 package xruntime @@ -39,9 +39,9 @@ type g struct { // 3. By debugCallWrap to pass parameters to a new goroutine because allocating a // closure in the runtime is forbidden. param unsafe.Pointer - atomicstatus uint32 + atomicstatus atomic.Uint32 stackLock uint32 // sigprof/scang lock; TODO: fold in to atomicstatus - goid int64 + goid uint64 schedlink guintptr waitsince int64 // approx time when the g become blocked waitreason waitReason // if status==Gwaiting @@ -65,14 +65,14 @@ type g struct { activeStackChans bool // parkingOnChan indicates that the goroutine is about to // park on a chansend or chanrecv. Used to signal an unsafe point - // for stack shrinking. It's a boolean value, but is updated atomically. - parkingOnChan uint8 + // for stack shrinking. + parkingOnChan atomic.Bool raceignore int8 // ignore race detection events sysblocktraced bool // StartTrace has emitted EvGoInSyscall about this goroutine tracking bool // whether we're tracking this G for sched latency statistics trackingSeq uint8 // used to decide whether to track this G - runnableStamp int64 // timestamp of when the G last became runnable, only used when tracking + trackingStamp int64 // timestamp of when the G last started being tracked runnableTime int64 // the amount of time spent runnable, cleared when running, only used when tracking sysexitticks int64 // cputicks when syscall has returned (for tracing) traceseq uint64 // trace event sequencer @@ -91,7 +91,7 @@ type g struct { cgoCtxt []uintptr // cgo traceback context labels unsafe.Pointer // profiler labels timer *timer // cached timer for time.Sleep - selectDone uint32 // are we participating in a select and did someone win the race? + selectDone atomic.Uint32 // are we participating in a select and did someone win the race? // goroutineProfiled indicates the status of this goroutine's stack for the // current in-progress goroutine profile @@ -190,7 +190,7 @@ type timer struct { nextwhen int64 // The status field holds one of the values below. - status uint32 + status atomic.Uint32 } type guintptr uintptr type puintptr uintptr @@ -198,7 +198,7 @@ type muintptr uintptr type waitReason uint8 type ancestorInfo struct { pcs []uintptr // pcs from the stack of this goroutine - goid int64 // goroutine id of this goroutine; original goroutine possibly dead + goid uint64 // goroutine id of this goroutine; original goroutine possibly dead gopc uintptr // pc of go statement that created this goroutine } type goroutineProfileStateHolder atomic.Uint32
-
- 21 Dec, 2022 1 commit
-
-
Kirill Smelkov authored
Else if a project, that uses go123, is compiled by go1.18 or go1.19, then go complains that go123 uses `any`, but the language, that go123 requires in it's go.mod, is only 1.14: kirr@deca:~/src/wendelin/wendelin.core/wcfs$ go build # lab.nexedi.com/kirr/go123/tracing/internal/xruntime /home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20221005052354-179529a7/tracing/internal/xruntime/zruntime_g_go1.19.go:114:12: predeclared any requires go1.18 or later (-lang was set to go1.14; check go.mod) /home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20221005052354-179529a7/tracing/internal/xruntime/zruntime_g_go1.19.go:184:14: predeclared any requires go1.18 or later (-lang was set to go1.14; check go.mod) /home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20221005052354-179529a7/tracing/internal/xruntime/zruntime_g_go1.19.go:185:9: predeclared any requires go1.18 or later (-lang was set to go1.14; check go.mod) -> Fix that by expanding `any` to what it is aliased to: $ go doc builtin.any package builtin // import "builtin" type any = interface{} any is an alias for interface{} and is equivalent to interface{} in all ways. Go123 wants to support older systems, and so we do not want to bump go123's required language, at least for now. For the reference: they started to use `any` somewhere in go1.18 (see 070bfdbb "tracing/runtime: Refresh for Go1.18").
-
- 05 Oct, 2022 1 commit
-
-
Kirill Smelkov authored
Generate g for today's state of Go 1.19 (go1.19.2-0-g895664482c). Compared to Go1.18 the only change is that new field `goroutineProfiled` is added to g struct. Regenerated files stay without changes for Go1.18 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.18.go b/zruntime_g_go1.19.go index 76f905d..3b33cbe 100644 --- a/zruntime_g_go1.18.go +++ b/zruntime_g_go1.19.go @@ -1,10 +1,11 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.18,!go1.19 +// +build go1.19,!go1.20 package xruntime import "unsafe" +import "sync/atomic" type g struct { // Stack parameters. @@ -91,6 +92,10 @@ type g struct { timer *timer // cached timer for time.Sleep selectDone uint32 // are we participating in a select and did someone win the race? + // goroutineProfiled indicates the status of this goroutine's stack for the + // current in-progress goroutine profile + goroutineProfiled goroutineProfileStateHolder + // gcAssistBytes is this G's GC assist credit in terms of // bytes allocated. If this is positive, then the G has credit // to allocate gcAssistBytes bytes without assisting. If this @@ -195,6 +200,7 @@ type ancestorInfo struct { goid int64 // goroutine id of this goroutine; original goroutine possibly dead gopc uintptr // pc of go statement that created this goroutine } +type goroutineProfileStateHolder atomic.Uint32 type uintreg uint // FIXME wrong on amd64p32 type m struct {} // FIXME stub type sudog struct {} // FIXME stub
-
- 16 Mar, 2022 1 commit
-
-
Kirill Smelkov authored
Regenerate for Go 1.18 release (go1.18-0-g4aa1efed48) The only change is interface{} -> any.
-
- 24 Nov, 2021 7 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
A deadlock scenario was possible before this patch: - main thread finished checks, wants to return, goes to deferred wg.Wait(); - that wg.Wait waits for other spawned threads to complete; - in one of that thread there is extra event being sent to a stream; - that send is blocked waiting for main thread to receive it, but the main thread is waiting in wg.Wait() and only after that it would go to call T.closeStreamTab() which marks all the channels as down -> deadlock. -> Fix it by explicitly detecting deadlocks not only in Recv, but also on Send. NOTE Contrary to Recv - which is always called from main thread - Send is always called from non-main threads. T.unsentv and the logic prepared in the previous patch care to handle that correctly.
-
Kirill Smelkov authored
If we don't, it won't be automatically the case when Send's will be adjusted to detect and log deadlocks. And then, if there will be no ordering, it will be hard to do tests in example_test.go which expects logging output in particular "main -> everything else" order.
-
Kirill Smelkov authored
This is possible if the system is incorrectly decomposed into serial streams. We want to show all pending events in that case, not only the first random one.
-
Kirill Smelkov authored
-> Reuse T.mu for serialization. closeStreamTab changes locking so that acquired t.mu covers whole time when that function could log anything.
-
Kirill Smelkov authored
There is no guaranty that the main thread would call Fatal or Fail if a non-main thread calls fatalfInNonMain. This way it was possible for a test to succeed if only non-main thread(s) failed somehow. -> Fix it by marking the test as failed if any non-main thread fails.
-
Kirill Smelkov authored
Generate g for today state of Go 1.18 (go1.17beta1-2522-g47db3bb443) Compared to Go1.17 there are small number of changes in _defer struct. Regenerated files stay without changes for Go1.17 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.17.go b/zruntime_g_go1.18.go index 7a76db6..5e2fafb 100644 --- a/zruntime_g_go1.17.go +++ b/zruntime_g_go1.18.go @@ -1,6 +1,6 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.17,!go1.18 +// +build go1.18,!go1.19 package xruntime @@ -115,7 +115,6 @@ type _panic struct { goexit bool } type _defer struct { - siz int32 // includes both arguments and results started bool heap bool // openDefer indicates that this _defer is for a frame with open-coded @@ -124,9 +123,9 @@ type _defer struct { openDefer bool sp uintptr // sp at time of defer pc uintptr // pc at time of defer - fn *funcval // can be nil for open-coded defers + fn func() // can be nil for open-coded defers _panic *_panic // panic that is running defer - link *_defer + link *_defer // next defer on G; can point to either heap or stack! // If openDefer is true, the fields below record values about the stack // frame and associated function that has the open-coded defer(s). sp
-
- 06 Sep, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 01 Jul, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 11 Jun, 2021 1 commit
-
-
Kirill Smelkov authored
Generate g for ~ Go 1.17beta1 (go1.17beta1-2-g770f1de8c5) Compared to Go1.16 there is one non-cosmetic change to g related to tracking scheduling latencies: https://github.com/golang/go/commit/bedfeed54a7a Regenerated files stay without changes for Go1.16 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.16.go b/zruntime_g_go1.17.go index 9604d6b..7a76db6 100644 --- a/zruntime_g_go1.16.go +++ b/zruntime_g_go1.17.go @@ -1,6 +1,6 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.16,!go1.17 +// +build go1.17,!go1.18 package xruntime @@ -25,7 +25,18 @@ type g struct { syscallsp uintptr // if status==Gsyscall, syscallsp = sched.sp to use during gc syscallpc uintptr // if status==Gsyscall, syscallpc = sched.pc to use during gc stktopsp uintptr // expected sp at top of stack, to check in traceback - param unsafe.Pointer // passed parameter on wakeup + // param is a generic pointer parameter field used to pass + // values in particular contexts where other storage for the + // parameter would be difficult to find. It is currently used + // in three ways: + // 1. When a channel operation wakes up a blocked goroutine, it sets param to + // point to the sudog of the completed blocking operation. + // 2. By gcAssistAlloc1 to signal back to its caller that the goroutine completed + // the GC cycle. It is unsafe to do so in any other way, because the goroutine's + // stack may have moved in the meantime. + // 3. By debugCallWrap to pass parameters to a new goroutine because allocating a + // closure in the runtime is forbidden. + param unsafe.Pointer atomicstatus uint32 stackLock uint32 // sigprof/scang lock; TODO: fold in to atomicstatus goid int64 @@ -57,6 +68,10 @@ type g struct { raceignore int8 // ignore race detection events sysblocktraced bool // StartTrace has emitted EvGoInSyscall about this goroutine + tracking bool // whether we're tracking this G for sched latency statistics + trackingSeq uint8 // used to decide whether to track this G + runnableStamp int64 // timestamp of when the G last became runnable, only used when tracking + runnableTime int64 // the amount of time spent runnable, cleared when running, only used when tracking sysexitticks int64 // cputicks when syscall has returned (for tracing) traceseq uint64 // trace event sequencer tracelastp puintptr // last P emitted an event for this goroutine @@ -142,7 +157,7 @@ type gobuf struct { pc uintptr g guintptr ctxt unsafe.Pointer - ret uintreg + ret uintptr lr uintptr bp uintptr // for framepointer-enabled architectures }
-
- 02 Mar, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 28 Jan, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 10 Jan, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 03 Jan, 2021 4 commits
-
-
Kirill Smelkov authored
Tracetest is a package to verify concurrent systems based on synchronous tracing. It is used in NEO/go tests and originates from what was initially explained in https://navytux.spb.ru/~kirr/neo.html in """On top of that package tracetest provides infrastructure for testing concurrent systems...""" See top-level package overview + example_test.go for details.
-
Kirill Smelkov authored
Some NEO/go tests need to start a cluster under tracing and then to continue running test workload with tracing disabled.
-
Kirill Smelkov authored
NetTrace now returns Tracer instead of general Networker. The reason we need this is because in the next patch Tracer will implement methods, additional to just what is required by Networker, to control itself.
-
Kirill Smelkov authored
Rename the interface that specifies methods to be implemented by trace receivers. Reason is that in the next patch we'll need to expose "Tracer" as the tracing networker itself.
-
- 29 Dec, 2020 3 commits
-
-
Kirill Smelkov authored
Generate g for ~ Go 1.16beta1 (go1.16beta1-34-g4fd9455882) Compared to Go1.15 there are only cosmetic changes: --- a/zruntime_g_go1.15.go +++ b/zruntime_g_go1.16.go @@ -1,6 +1,6 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.15,!go1.16 +// +build go1.16,!go1.17 package xruntime @@ -144,7 +144,7 @@ type gobuf struct { ctxt unsafe.Pointer ret uintreg lr uintptr - bp uintptr // for GOEXPERIMENT=framepointer + bp uintptr // for framepointer-enabled architectures } type funcval struct { fn uintptr @@ -158,6 +158,8 @@ type timer struct { // Timer wakes up at when, and then at when+period, ... (period > 0 only) // each time calling f(arg, now) in the timer goroutine, so f must be // a well-behaved function and not block. + // + // when must be positive on an active timer. when int64 period int64 f func(interface{}, uintptr) Regenerated files stay without changes for Go1.15 and previous releases.
-
Kirill Smelkov authored
When a virtnet network - e.g. lonet - is explicitly used in tests, the open and close sequence is easy to implement: network = lonet.Join(...) defer network.Close() hostα = network.NewHost("α") defer hostα.Close() hostβ = network.NewHost("β") defer hostβ.Close() ... However in other scenarios - for example when the code wants to call xnet.Networker factory that creates a networker out of several potential choices according to given parameters, only interface for network access-point (e.g. Host in lonet/virtnet case) is returned. The network itself (virtnet.SubNetwork) is not returned to the user. As it is this makes it impossible to close the network and release its resources. Instead of adding complexity to such "create-networker" functions to e.g. change Close of returned networker to also close subnetwork, or return both networker and a separate close func, let's teach virtnet to close SubNetwork automatically when last host of this subnetwork is closed. Make it opt-in since this behaviour is not universally wanted. This way a factory that is asked to connect to network as e.g. a node on lonet, could do the following: func neonet.Join(...) xnet.Networker { ... // join lonet "<net>" as host "<node>" network = lonet.Join("<net>") host = network.NewHost("<node>") network.AutoClose() // host.Close will close network return host } and network will be closed on host.Close() call. -> Add SubNetwork.AutoClose to schedule Close to be called after last host on the subnetwork is closed. -> Mirror this change in virtnet part of lonet.py
-
Kirill Smelkov authored
Even though there is no need to manually release resources for NetPlain and NetTLS, for other networkers - for example lonet - releasing resources is needed. In general programs cannot always use lonet explicitly, as there might be a factory function that creates a Networker according to given parameters. -> Adjust Networker interface to include general Close. -> Adjust NetPlain to follow added "interrupt-on-close" semantic for uniformity.
-
- 17 Dec, 2020 3 commits
-
-
Kirill Smelkov authored
Similarly to what we do in package xio (see 7ad867a3 "xio: Add Reader/Writer/... interfaces - io analogs that add support for contexts"), add utilities to bind xnet.Listener to context (giving net.Listener), and, on the other hand, adapt uncontextified net.Listener to xnet.Listener that handles cancellation. This utilities are needed because a lot of third-party code wants to work with net.Listener interface only. Contrary to xio, we can make it working reliably in both ways (because Accept already buffers accepting connections starting at OS level).
-
Kirill Smelkov authored
In the previous patch xnet.Networker was changed: Listen now accepts ctx and returns xnet.Listener instead of net.Listener. -> Adapt the code all around to that.
-
Kirill Smelkov authored
We already handle cancellation in Dial, but Accepting was out of luck until now. This makes it more difficult for clients to implement and wrap acceptors where they need to handle cancellations. This also makes it possible for a test or program to get stuck in Accept loop if it is not careful enough to manually handle ctx cancel around Accept calls. -> Fix it in one place - here, in xnet - so that users are offloaded from all this and can just call Accept(ctx) and rely on underlying implementation to handle ctx cancel. This patch: - introduces xnet.Listener interface, which is like net.Listener, but Accept goes with ctx argument. - changes Networker.Listen signature to return xnet.Listener instead of net.Listener. While we are here - changing it - also add ctx argument to Listen call itself. - Adds listenerCtx - which, given net.Listener, provides xnet.Listener by wrapping some logic around original. - Adapts NetPlain, NetTLS and NetTrace to provide updated interface. We'll fix up everything in other packages to match/use updated interface in the next patch.
-
- 01 Dec, 2020 1 commit
-
-
Kirill Smelkov authored
See nexedi/pygolang@21756bd3.
-
- 18 Oct, 2020 1 commit
-
-
Kirill Smelkov authored
Go1.14.10 and Go1.15.3 changed internal structure of g: https://github.com/golang/go/commit/878da0bf881b (go1.14) https://github.com/golang/go/commit/bf79f91d3dc4 (go1.15)
-
- 16 Sep, 2020 1 commit
-
-
Kirill Smelkov authored
A shorthand for when exception context is just formatted string. Simplifies func doSomething(path string) { defer exc.Context(func() interface{} { return fmt.Sprintf("doing something %s", path) })() ... to func doSomething(path string) { defer exc.Contextf("doing something %s", path) ...
-
- 15 Sep, 2020 1 commit
-
-
Kirill Smelkov authored
`go mod init` + build/test that fetched in dependencies.
-
- 27 May, 2020 4 commits
-
-
Kirill Smelkov authored
It was address-of error (the type) instead of &err.
-
Kirill Smelkov authored
https://github.com/golang/go/issues/36503 https://github.com/golang/go/issues/36448 They are there for 6 months already without being considered for real and so the progress is very unlikely, but still...
-
Kirill Smelkov authored
Generate g for ~ Go 1.15beta1 (go1.14beta1-1729-gc0e8e405c0) Compared to Go1.14 there are no changes. Regenerated files stay without changes for Go1.14 and previous releases.
-
Kirill Smelkov authored
-