Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions doc/rfc/stovepipe/steps/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,13 @@ Strategy and baseline are persisted on the Request and are **immutable**: a rede

Validation is expensive and shares a baseline, so heads arriving while an earlier one runs `build → buildsignal → record` must not all start builds at once.

**Runtime state** lives on the Queue row; the **concurrency cap** does not — it is deployment configuration, resolved at gate-check time the same way SubmitQueue separates `storage` from `queueconfig` (see [submitqueue/extension/queueconfig/README.md](../../../../submitqueue/extension/queueconfig/README.md)): pipeline stages read mutable state from the store and read knobs like `max_concurrent` from a config `Store` (or a wiring default for MVP). Config is not written by ingest/process/record and does not need optimistic locking.
**Runtime state** lives on the Queue row; the **concurrency cap** does not — it is deployment configuration, resolved at gate-check time the same way SubmitQueue separates `storage` from `queueconfig` (see [submitqueue/extension/queueconfig/README.md](../../../../submitqueue/extension/queueconfig/README.md)): pipeline stages read mutable state from the store and read knobs like `max_concurrent` from a config `Store`. Config is not written by ingest/process/record and does not need optimistic locking.

| Source | Field | Meaning |
|---|---|---|
| Queue row | `last_green_uri` | Bookmark `buildsignal` advances on a successful build; empty until first green. |
| Queue row | `in_flight_count` | Requests past `process` and not yet terminal. `process` increments on admit; `record` (or DLQ reconciliation) decrements on terminal. |
| Queue config | `max_concurrent` | Cap on concurrent in-flight validations. **Default 1** (global wiring default for MVP; per-queue override when a Stovepipe `queueconfig` extension lands). |
| Queue config | `max_concurrent` | Cap on concurrent in-flight validations. **Default 1** in the reference service baseline; deployment-wide and per-queue overrides live under the process stage configuration. |

A slot is held from admit until the build goes terminal (`process → build → buildsignal`), not just while `process` runs. It is released when the Request reaches **any** terminal state and `in_flight_count` is decremented — `buildsignal` recording the build's outcome, success *or* failure, or the DLQ reconciler forcing a terminal `failed` (see [integrity](#in_flight_count-integrity)). A build *failure* frees the slot just like a success; only a Request that never terminates keeps its slot.

Expand Down
4 changes: 4 additions & 0 deletions service/stovepipe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ The Stovepipe controllers live under [`stovepipe/controller/`](../../stovepipe/c
| `PORT` | no | gRPC listen address | `:8083` |
| `HOSTNAME` | no | Subscriber name for the queue consumers | `stovepipe-<unix_ts>` |

## Queue and stage configuration

The server accepts an optional `-config` YAML file for deployment-wide queue and stage settings and per-queue overrides. See the [configuration guide](server/README.md) for settings, validation. The service always loads its embedded baseline; the optional file overrides it.

## Running

### Docker Compose (recommended)
Expand Down
5 changes: 4 additions & 1 deletion service/stovepipe/server/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ go_library(
"//platform/extension/messagequeue/mysql:go_default_library",
"//platform/hook:go_default_library",
"//service/messagequeue:go_default_library",
"//service/stovepipe/server/config:go_default_library",
"//service/stovepipe/server/handler:go_default_library",
"//stovepipe/controller:go_default_library",
"//stovepipe/controller/build:go_default_library",
Expand All @@ -44,7 +45,7 @@ go_library(
"//stovepipe/extension/fixer/noop:go_default_library",
"//stovepipe/extension/projectissues/noop:go_default_library",
"//stovepipe/extension/projectstatus/noop:go_default_library",
"//stovepipe/extension/queueconfig/default:go_default_library",
"//stovepipe/extension/queueconfig:go_default_library",
"//stovepipe/extension/sourcecontrol:go_default_library",
"//stovepipe/extension/sourcecontrol/fake:go_default_library",
"//stovepipe/extension/storage:go_default_library",
Expand Down Expand Up @@ -96,11 +97,13 @@ go_test(
"//platform/consumer/mock:go_default_library",
"//platform/extension/messagequeue:go_default_library",
"//platform/extension/messagequeue/mock:go_default_library",
"//service/stovepipe/server/config:go_default_library",
"//stovepipe/controller/dlq:go_default_library",
"//stovepipe/core/messagequeue:go_default_library",
"//stovepipe/core/requestlog:go_default_library",
"//stovepipe/entity:go_default_library",
"//stovepipe/extension/buildissue/noop:go_default_library",
"//stovepipe/extension/queueconfig:go_default_library",
"//stovepipe/extension/storage:go_default_library",
"//stovepipe/extension/storage/mock:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
Expand Down
19 changes: 19 additions & 0 deletions service/stovepipe/server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Stovepipe service configuration

The server embeds `config/base.yaml` as its operational baseline and accepts an optional `-config` YAML file layered over it. Omitting the optional file still loads the baseline; there are no hidden Go fallback values. Invalid configuration fails startup before consumers start. Effective settings are logged at startup; updates require a restart.

## Stage settings

The `queueconfig.stages` section contains typed stage blocks. `process` owns `maxConcurrent` and `gateWaitDelay`. `buildSignal` owns `acceptedPollDelay` and `runningPollDelay`; `buildAnalysis` owns `pollDelay`; `issueTrack` owns `refreshDelay` and `recoveryWaitDelay`. Timing and non-timing settings belong to the stage that owns their behavior.

Each stage has its own YAML adapter, shared override application, and validation. The shared library uses integer milliseconds; host adapters accept duration strings such as `2m`, `30s`, or `250ms`, restricted to whole milliseconds. Partial layers are merged before semantic validation. Omitted fields inherit; explicit zero is not omission. The current limits and intervals must be positive. Unknown fields, duplicate keys, and multiple YAML documents are rejected.

## Layering and queues

Deployment-wide stage settings from the optional file override the service baseline. Entries under `queueconfig.queues`, keyed by queue name, contain their own `stages` blocks and override deployment settings. When multiple files specify the same queue, its fields are merged rather than replacing its entire configuration. Queue layers retain file order and are applied after deployment-wide settings.

Every queue override must name a queue in `MQ_TENANTS`; queues without overrides inherit the deployment configuration. Queue registration is separate from stage settings. The static store only serves registered queues.

The baseline retains a concurrency limit of one, a five-second admission wait, five-second accepted-build polling, two-second running-build polling, two-minute build analysis polling, and thirty-second issue refresh and recovery waits. These values are defined only in the baseline YAML.

Process-stage admission consumes this configuration. Other stage controllers retain their existing intervals.
34 changes: 34 additions & 0 deletions service/stovepipe/server/config/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = [
"build_analysis.go",
"build_signal.go",
"config.go",
"duration.go",
"issue_track.go",
"process.go",
"stages.go",
],
embedsrcs = ["base.yaml"],
importpath = "github.com/uber/submitqueue/service/stovepipe/server/config",
visibility = ["//service/stovepipe/server:__pkg__"],
deps = [
"//stovepipe/entity:go_default_library",
"//stovepipe/extension/queueconfig:go_default_library",
"//stovepipe/extension/queueconfig/static:go_default_library",
"@in_gopkg_yaml_v3//:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = ["config_test.go"],
embed = [":go_default_library"],
deps = [
"//stovepipe/extension/queueconfig:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
],
)
14 changes: 14 additions & 0 deletions service/stovepipe/server/config/base.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
queueconfig:
stages:
process:
maxConcurrent: 1
gateWaitDelay: 5s
buildSignal:
acceptedPollDelay: 5s
runningPollDelay: 2s
buildAnalysis:
pollDelay: 2m
issueTrack:
refreshDelay: 30s
recoveryWaitDelay: 30s
queues: {}
37 changes: 37 additions & 0 deletions service/stovepipe/server/config/build_analysis.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package config

import (
"fmt"
"time"

"github.com/uber/submitqueue/stovepipe/extension/queueconfig"
)

type buildAnalysisSettings struct {
PollDelay *time.Duration `yaml:"pollDelay"`
}

func (s buildAnalysisSettings) overrides() (queueconfig.BuildAnalysisOverrides, error) {
pollDelay, err := durationMilliseconds(s.PollDelay)
if err != nil {
return queueconfig.BuildAnalysisOverrides{}, fmt.Errorf("pollDelay: %w", err)
}

return queueconfig.BuildAnalysisOverrides{
PollDelayMs: pollDelay,
}, nil
}
45 changes: 45 additions & 0 deletions service/stovepipe/server/config/build_signal.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package config

import (
"fmt"
"time"

"github.com/uber/submitqueue/stovepipe/extension/queueconfig"
)

type buildSignalSettings struct {
AcceptedPollDelay *time.Duration `yaml:"acceptedPollDelay"`

RunningPollDelay *time.Duration `yaml:"runningPollDelay"`
}

func (s buildSignalSettings) overrides() (queueconfig.BuildSignalOverrides, error) {
acceptedPollDelay, err := durationMilliseconds(s.AcceptedPollDelay)
if err != nil {
return queueconfig.BuildSignalOverrides{}, fmt.Errorf("acceptedPollDelay: %w", err)
}

runningPollDelay, err := durationMilliseconds(s.RunningPollDelay)
if err != nil {
return queueconfig.BuildSignalOverrides{}, fmt.Errorf("runningPollDelay: %w", err)
}

return queueconfig.BuildSignalOverrides{
AcceptedPollDelayMs: acceptedPollDelay,
RunningPollDelayMs: runningPollDelay,
}, nil
}
120 changes: 120 additions & 0 deletions service/stovepipe/server/config/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package config loads the reference service's layered startup configuration.
package config

import (
"bytes"
_ "embed"
"fmt"
"io"
"os"

"github.com/uber/submitqueue/stovepipe/entity"
"github.com/uber/submitqueue/stovepipe/extension/queueconfig"
"github.com/uber/submitqueue/stovepipe/extension/queueconfig/static"
"gopkg.in/yaml.v3"
)

//go:embed base.yaml
var baseYAML []byte

type document struct {
QueueConfig queueSection `yaml:"queueconfig"`
}

type queueSection struct {
queueSettings `yaml:",inline"`

Queues map[string]queueSettings `yaml:"queues"`
}

type queueSettings struct {
Stages stageSettings `yaml:"stages"`
}

// Load layers an optional file over the embedded service baseline.
// Queue-specific settings override deployment settings. Updates require a restart.
func Load(path string, names []string) (queueconfig.Store, error) {
base, err := decodeDocument(bytes.NewReader(baseYAML))
if err != nil {
return nil, fmt.Errorf("decode service baseline: %w", err)
}

layers := []document{base}

if path != "" {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open configuration %q: %w", path, err)
}
defer file.Close()

layer, err := decodeDocument(file)
if err != nil {
return nil, fmt.Errorf("decode configuration %q: %w", path, err)
}

layers = append(layers, layer)
}

return newQueueConfigStore(names, layers...)
}

func decodeDocument(reader io.Reader) (document, error) {
decoder := yaml.NewDecoder(reader)
decoder.KnownFields(true)

var cfg document
if err := decoder.Decode(&cfg); err != nil {
return document{}, err
}

var extra any
if err := decoder.Decode(&extra); err != io.EOF {
if err != nil {
return document{}, err
}

return document{}, fmt.Errorf("configuration must contain exactly one YAML document")
}

return cfg, nil
}

func newQueueConfigStore(names []string, layers ...document) (queueconfig.Store, error) {
var deployment entity.StageConfig
overrides := make(map[string][]queueconfig.Overrides)

for _, layer := range layers {
common, err := layer.QueueConfig.Stages.overrides()
if err != nil {
return nil, fmt.Errorf("deployment configuration: %w", err)
}

deployment = common.Apply(deployment)

for name, settings := range layer.QueueConfig.Queues {
override, err := settings.Stages.overrides()
if err != nil {
return nil, fmt.Errorf("queue %q configuration: %w", name, err)
}

overrides[name] = append(overrides[name], override)
}
}

return static.NewStore(names, deployment, overrides)
}
Loading
Loading