multipleof4/ntfy
1
0
Fork 0
mirror of https://github.com/multipleof4/ntfy.git synced 2026-10-08 21:05:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Template exec context, redone

Browse Source
This commit is contained in:
binwiederhier
2026-07-10 13:18:11 +02:00
parent 1e4e3b6e36
commit 3f56dae54a
8 changed files with 310 additions and 189 deletions
Download Patch File Download Diff File Expand all files Collapse all files
template/gotext/README.md
+30 -18
View File
@@ -1,8 +1,8 @@
# `template/gotext/` -- vendored `text/template` with an execution deadline
# `template/gotext/` -- vendored `text/template` with context cancellation
This directory is a **verbatim copy of Go's standard-library `text/template` package**, plus one
small patch that adds a wall-clock execution deadline. It exists for exactly one reason: to stop
**user-supplied** message templates (`Template: yes`, see the [templating docs](https://ntfy.sh/docs/publish/#message-templating))
small patch that adds context-aware execution (`ExecuteContext`). It exists for exactly one reason:
to stop **user-supplied** message templates (`Template: yes`, see the [templating docs](https://ntfy.sh/docs/publish/#message-templating))
from burning CPU.
- **Source:** Go stdlib `text/template` (+ `internal/fmtsort`), `$(go env GOROOT)/src`
@@ -14,7 +14,8 @@ from burning CPU.
ntfy lets users send a Go template that is rendered against a JSON body. Go's `text/template`
**cannot be interrupted mid-execution** -- there is no context, no deadline, no cancellation
([golang/go#31107](https://github.com/golang/go/issues/31107) was declined). So a crafted template
([golang/go#31107](https://github.com/golang/go/issues/31107) proposed `ExecuteContext` but was
declined, over a bundled context-*values* feature, not cancellation itself). So a crafted template
with a tight or nested `{{range}}` (e.g. ranging over a large JSON array with a big loop body that
writes no output) can run for tens of seconds on a single request. That is a CPU denial of service
(GHSA-rhwf-xgc9-m9fp).
@@ -22,13 +23,17 @@ writes no output) can run for tens of seconds on a single request. That is a CPU
There is no way to add an interrupt from the outside -- the executor's per-node `walk` loop is
unexported. The only robust fix is to patch the executor itself. Rather than reach for fragile
heuristics (guessing iteration counts, wrapping every function, etc.), we vendor the package and add
a **single check inside `walk`**: every ~256 nodes it checks a wall-clock deadline and aborts (via
the normal `ExecError` path) if it has passed. This bounds CPU for *any* template shape -- cheap
loops and expensive functions alike -- by construction.
the cancellation half of #31107 as a patch: `ExecuteContext(ctx, ...)` that aborts with `ctx.Err()`
when `ctx` is canceled or its deadline passes. The check is a **single poll inside `walk`** of an
atomic flag that a `context.AfterFunc` watcher flips -- so it bounds CPU for *any* template shape
(cheap loops and expensive functions alike), it is exact (observed within one node), and it adds no
measurable overhead. If #31107's cancellation half ever lands upstream, this fork can be deleted and
the call site keeps compiling unchanged.
The one user-facing execution site (`server/server_template.go` `renderTemplate`) sets the deadline
with `SetExecutionDeadline` and maps the resulting error to a `400`. Trusted templates (operator
config: Twilio, `cmd/serve.go`) keep using the standard library -- they are not user-supplied.
The one user-facing execution site (`server/server_template.go` `renderTemplate`) wraps execution in
`context.WithTimeout` and calls `ExecuteContext`, mapping `context.DeadlineExceeded` to a `400`.
Trusted templates (operator config: Twilio, `cmd/serve.go`) keep using the standard library -- they
are not user-supplied.
## What's here
@@ -36,7 +41,7 @@ config: Twilio, `cmd/serve.go`) keep using the standard library -- they are not
|------|--------|
| `*.go` (`exec.go`, `funcs.go`, `template.go`, `option.go`, `helper.go`, `doc.go`) | verbatim from `$(go env GOROOT)/src/text/template/`, enumerated with `go list` so files added/removed upstream are picked up automatically |
| `fmtsort/sort.go` | verbatim from `$(go env GOROOT)/src/internal/fmtsort/` -- `exec.go` needs it, and `internal/...` packages can't be imported from outside GOROOT, so it comes along |
| `patches/0001-exec-deadline.patch` | our only real change (see below) |
| `patches/0001-exec-context.patch` | our only real change (see below) |
| `GENERATED_FROM` | the exact Go version `make update-template` last regenerated this copy from; provenance, written by that target |
The Go toolchain version this copy is pinned to lives in the repo-root [`.go-version`](../../.go-version)
@@ -49,19 +54,26 @@ plain import.
## The patch
`patches/` is a quilt-style ordered series (apply `0001-*`, then `0002-*`, ...). Today there is just
`0001-exec-deadline.patch` -- small, purely additive, and touching only `exec.go`/`template.go`:
`0001-exec-context.patch` -- small, purely additive, and touching only `exec.go`:
- adds `deadline`/`steps` fields to the executor `state` and a `deadline` field + a
`SetExecutionDeadline(time.Time)` method on `Template`
- adds the amortized deadline check at the top of `state.walk`
- adds the exported sentinel `ErrExecutionInterrupted` (detect with `errors.Is`)
- adds `ctx context.Context` and a shared `cancelled *atomic.Bool` to the executor `state`
- adds `ExecuteContext` / `ExecuteTemplateContext`; `Execute` / `ExecuteTemplate` become
`context.Background()` wrappers, so their behavior and cost are unchanged
- when `ctx.Done() != nil`, arms one `context.AfterFunc` watcher that flips the flag; `walk` polls it
per node and aborts via a `cancelError` that `errRecover` strips to the bare `ctx.Err()`
(`errors.Is(err, context.DeadlineExceeded)`)
The flag is a `*atomic.Bool` (not a value) because `walkTemplate` copies `state` for nested
`{{template}}` invocations; a shared pointer keeps one flag across all copies and avoids `go vet`
copylocks. `template.go` is unchanged -- the context is per-call, not stored on the `Template`.
Two *mechanical* transforms are applied by `make update-template` with `sed`, **not** the patch --
renaming the package to `gotext`, and rewriting the `internal/fmtsort` import to
`heckel.io/ntfy/v2/template/gotext/fmtsort`. Keeping them out of the patch means they apply to
whatever files `go list` returns, so they survive upstream files being added or removed.
whatever files `go list` returns, so they survive upstream files being added or removed. (These two
transforms are also the only difference between our patch and the upstream `text/template` diff.)
Keeping the patch tiny (deadline logic only, on two stable files) is deliberate: it makes re-basing
Keeping the patch tiny (cancellation only, on one stable file) is deliberate: it makes re-basing
onto a new Go release cheap.
## Updating (when bumping the Go toolchain)
template/gotext/exec.go
+66 -23
View File
@@ -5,14 +5,15 @@
package gotext
import (
"context"
"errors"
"fmt"
"io"
"reflect"
"runtime"
"strings"
"sync/atomic"
"text/template/parse"
"time"
"heckel.io/ntfy/v2/template/gotext/fmtsort"
)
@@ -34,13 +35,13 @@ func initMaxExecDepth() int {
// template so that multiple executions of the same template
// can execute in parallel.
type state struct {
tmpl *Template
wr io.Writer
node parse.Node // current node, for errors
vars []variable // push-down stack of variable values.
depth int // the height of the stack of executing templates.
deadline time.Time // ntfy: wall-clock bail-out; zero means no limit
steps int64 // ntfy: node counter for amortized deadline checks
tmpl *Template
ctx context.Context // ctx-ex: execution context; Execute uses context.Background.
wr io.Writer
node parse.Node // current node, for errors
vars []variable // push-down stack of variable values.
depth int // the height of the stack of executing templates.
cancelled *atomic.Bool // ctx-ex: shared flag set by the context.AfterFunc watcher; nil if ctx cannot be canceled
}
// variable holds the dynamic value of a variable such as $, $x etc.
@@ -135,10 +136,6 @@ func (e ExecError) Unwrap() error {
return e.Err
}
// ErrExecutionInterrupted is wrapped into the error returned by Execute when a template exceeds the
// deadline set via Template.SetExecutionDeadline. Detect it with errors.Is. (ntfy addition)
var ErrExecutionInterrupted = errors.New("template execution interrupted")
// errorf records an ExecError and terminates processing.
func (s *state) errorf(format string, args ...any) {
name := doublePercent(s.tmpl.Name())
@@ -168,6 +165,14 @@ func (s *state) writeError(err error) {
})
}
// cancelError is the wrapper type used internally when execution is aborted
// because the context is done. Like writeError, it is stripped in errRecover
// so the caller receives the original ctx.Err(). It is not an implementation
// of error, so it cannot escape from the package as an error value.
type cancelError struct {
Err error // Original context error.
}
// errRecover is the handler that turns panics into returns from the top
// level of Parse.
func errRecover(errp *error) {
@@ -178,6 +183,8 @@ func errRecover(errp *error) {
panic(e)
case writeError:
*errp = err.Err // Strip the wrapper.
case cancelError:
*errp = err.Err // Strip the wrapper; return the context error.
case ExecError:
*errp = err // Keep the wrapper.
default:
@@ -194,11 +201,19 @@ func errRecover(errp *error) {
// A template may be executed safely in parallel, although if parallel
// executions share a Writer the output may be interleaved.
func (t *Template) ExecuteTemplate(wr io.Writer, name string, data any) error {
return t.ExecuteTemplateContext(context.Background(), wr, name, data)
}
// ExecuteTemplateContext is like [Template.ExecuteTemplate], but aborts and
// returns ctx.Err() if ctx is canceled or its deadline is exceeded before
// execution completes. See [Template.ExecuteContext] for the cancellation
// semantics.
func (t *Template) ExecuteTemplateContext(ctx context.Context, wr io.Writer, name string, data any) error {
tmpl := t.Lookup(name)
if tmpl == nil {
return fmt.Errorf("template: no template %q associated with template %q", name, t.name)
}
return tmpl.Execute(wr, data)
return tmpl.ExecuteContext(ctx, wr, data)
}
// Execute applies a parsed template to the specified data object,
@@ -212,20 +227,47 @@ func (t *Template) ExecuteTemplate(wr io.Writer, name string, data any) error {
// If data is a [reflect.Value], the template applies to the concrete
// value that the reflect.Value holds, as in [fmt.Print].
func (t *Template) Execute(wr io.Writer, data any) error {
return t.execute(wr, data)
return t.executeContext(context.Background(), wr, data)
}
func (t *Template) execute(wr io.Writer, data any) (err error) {
// ExecuteContext is like [Template.Execute], but aborts and returns ctx.Err()
// (either [context.Canceled] or [context.DeadlineExceeded], retrievable with
// [errors.Is]) if ctx is canceled or its deadline is exceeded before execution
// completes.
//
// Cancellation is observed between node evaluations as the template is walked,
// so long-running renders -- including tight or nested {{range}} loops that
// write no output -- are aborted promptly. A template blocked inside a single
// function call is not interrupted until that call returns. Partial results may
// already have been written to wr.
func (t *Template) ExecuteContext(ctx context.Context, wr io.Writer, data any) error {
if err := ctx.Err(); err != nil {
return err
}
return t.executeContext(ctx, wr, data)
}
func (t *Template) executeContext(ctx context.Context, wr io.Writer, data any) (err error) {
defer errRecover(&err)
value, ok := data.(reflect.Value)
if !ok {
value = reflect.ValueOf(data)
}
state := &state{
tmpl: t,
wr: wr,
vars: []variable{{"$", value}},
deadline: t.deadline, // ntfy: wall-clock execution bail-out
tmpl: t,
ctx: ctx,
wr: wr,
vars: []variable{{"$", value}},
}
// If the context can be canceled, watch it with a single context.AfterFunc
// callback that flips an atomic flag; walk polls that flag per node (a cheap
// monomorphic atomic load) instead of calling ctx.Err() every node.
// Contexts that can never be canceled (Background, TODO) have a nil Done
// channel, so the default Execute path installs nothing and pays nothing.
if ctx.Done() != nil {
state.cancelled = new(atomic.Bool)
stop := context.AfterFunc(ctx, func() { state.cancelled.Store(true) })
defer stop()
}
if t.Tree == nil || t.Root == nil {
state.errorf("%q is an incomplete or empty template", t.Name())
@@ -269,10 +311,11 @@ var (
// generating output as they go.
func (s *state) walk(dot reflect.Value, node parse.Node) {
s.at(node)
// ntfy: amortized wall-clock bail-out to prevent CPU DoS from user-supplied templates
// (tight/nested ranges that never write output). See GHSA-rhwf-xgc9-m9fp.
if s.steps++; s.steps&0xff == 0 && !s.deadline.IsZero() && time.Now().After(s.deadline) {
s.errorf("execution interrupted: %w", ErrExecutionInterrupted)
// Abort if the context has been canceled or its deadline has passed. The
// flag is set by the watcher installed in executeContext; observing it here
// interrupts any template shape, including loops that write no output.
if s.cancelled != nil && s.cancelled.Load() {
panic(cancelError{s.ctx.Err()})
}
switch node := node.(type) {
case *parse.ActionNode:
template/gotext/patches/0001-exec-context.patch
+149
View File
@@ -0,0 +1,149 @@
--- a/exec.go 2026-07-10 01:31:35.188129862 +0200
+++ b/exec.go 2026-07-10 01:31:35.189129894 +0200
@@ -5,14 +5,17 @@
package gotext
import (
+ "context"
"errors"
"fmt"
- "heckel.io/ntfy/v2/template/gotext/fmtsort"
"io"
"reflect"
"runtime"
"strings"
+ "sync/atomic"
"text/template/parse"
+
+ "heckel.io/ntfy/v2/template/gotext/fmtsort"
)
// maxExecDepth specifies the maximum stack depth of templates within
@@ -32,11 +35,13 @@
// template so that multiple executions of the same template
// can execute in parallel.
type state struct {
- tmpl *Template
- wr io.Writer
- node parse.Node // current node, for errors
- vars []variable // push-down stack of variable values.
- depth int // the height of the stack of executing templates.
+ tmpl *Template
+ ctx context.Context // ctx-ex: execution context; Execute uses context.Background.
+ wr io.Writer
+ node parse.Node // current node, for errors
+ vars []variable // push-down stack of variable values.
+ depth int // the height of the stack of executing templates.
+ cancelled *atomic.Bool // ctx-ex: shared flag set by the context.AfterFunc watcher; nil if ctx cannot be canceled
}
// variable holds the dynamic value of a variable such as $, $x etc.
@@ -160,6 +165,14 @@
})
}
+// cancelError is the wrapper type used internally when execution is aborted
+// because the context is done. Like writeError, it is stripped in errRecover
+// so the caller receives the original ctx.Err(). It is not an implementation
+// of error, so it cannot escape from the package as an error value.
+type cancelError struct {
+ Err error // Original context error.
+}
+
// errRecover is the handler that turns panics into returns from the top
// level of Parse.
func errRecover(errp *error) {
@@ -170,6 +183,8 @@
panic(e)
case writeError:
*errp = err.Err // Strip the wrapper.
+ case cancelError:
+ *errp = err.Err // Strip the wrapper; return the context error.
case ExecError:
*errp = err // Keep the wrapper.
default:
@@ -186,11 +201,19 @@
// A template may be executed safely in parallel, although if parallel
// executions share a Writer the output may be interleaved.
func (t *Template) ExecuteTemplate(wr io.Writer, name string, data any) error {
+ return t.ExecuteTemplateContext(context.Background(), wr, name, data)
+}
+
+// ExecuteTemplateContext is like [Template.ExecuteTemplate], but aborts and
+// returns ctx.Err() if ctx is canceled or its deadline is exceeded before
+// execution completes. See [Template.ExecuteContext] for the cancellation
+// semantics.
+func (t *Template) ExecuteTemplateContext(ctx context.Context, wr io.Writer, name string, data any) error {
tmpl := t.Lookup(name)
if tmpl == nil {
return fmt.Errorf("template: no template %q associated with template %q", name, t.name)
}
- return tmpl.Execute(wr, data)
+ return tmpl.ExecuteContext(ctx, wr, data)
}
// Execute applies a parsed template to the specified data object,
@@ -204,10 +227,27 @@
// If data is a [reflect.Value], the template applies to the concrete
// value that the reflect.Value holds, as in [fmt.Print].
func (t *Template) Execute(wr io.Writer, data any) error {
- return t.execute(wr, data)
+ return t.executeContext(context.Background(), wr, data)
}
-func (t *Template) execute(wr io.Writer, data any) (err error) {
+// ExecuteContext is like [Template.Execute], but aborts and returns ctx.Err()
+// (either [context.Canceled] or [context.DeadlineExceeded], retrievable with
+// [errors.Is]) if ctx is canceled or its deadline is exceeded before execution
+// completes.
+//
+// Cancellation is observed between node evaluations as the template is walked,
+// so long-running renders -- including tight or nested {{range}} loops that
+// write no output -- are aborted promptly. A template blocked inside a single
+// function call is not interrupted until that call returns. Partial results may
+// already have been written to wr.
+func (t *Template) ExecuteContext(ctx context.Context, wr io.Writer, data any) error {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ return t.executeContext(ctx, wr, data)
+}
+
+func (t *Template) executeContext(ctx context.Context, wr io.Writer, data any) (err error) {
defer errRecover(&err)
value, ok := data.(reflect.Value)
if !ok {
@@ -215,9 +255,20 @@
}
state := &state{
tmpl: t,
+ ctx: ctx,
wr: wr,
vars: []variable{{"$", value}},
}
+ // If the context can be canceled, watch it with a single context.AfterFunc
+ // callback that flips an atomic flag; walk polls that flag per node (a cheap
+ // monomorphic atomic load) instead of calling ctx.Err() every node.
+ // Contexts that can never be canceled (Background, TODO) have a nil Done
+ // channel, so the default Execute path installs nothing and pays nothing.
+ if ctx.Done() != nil {
+ state.cancelled = new(atomic.Bool)
+ stop := context.AfterFunc(ctx, func() { state.cancelled.Store(true) })
+ defer stop()
+ }
if t.Tree == nil || t.Root == nil {
state.errorf("%q is an incomplete or empty template", t.Name())
}
@@ -260,6 +311,12 @@
// generating output as they go.
func (s *state) walk(dot reflect.Value, node parse.Node) {
s.at(node)
+ // Abort if the context has been canceled or its deadline has passed. The
+ // flag is set by the watcher installed in executeContext; observing it here
+ // interrupts any template shape, including loops that write no output.
+ if s.cancelled != nil && s.cancelled.Load() {
+ panic(cancelError{s.ctx.Err()})
+ }
switch node := node.(type) {
case *parse.ActionNode:
// Do not pop variables so they persist until next end.
template/gotext/patches/0001-exec-deadline.patch
-113
View File
@@ -1,113 +0,0 @@
diff -ruN a/exec.go b/exec.go
--- a/exec.go 2026-07-08 21:46:30.952555712 +0200
+++ b/exec.go 2026-07-08 21:46:30.953912265 +0200
@@ -7,12 +7,14 @@
import (
"errors"
"fmt"
- "heckel.io/ntfy/v2/template/gotext/fmtsort"
"io"
"reflect"
"runtime"
"strings"
"text/template/parse"
+ "time"
+
+ "heckel.io/ntfy/v2/template/gotext/fmtsort"
)
// maxExecDepth specifies the maximum stack depth of templates within
@@ -32,11 +34,13 @@
// template so that multiple executions of the same template
// can execute in parallel.
type state struct {
- tmpl *Template
- wr io.Writer
- node parse.Node // current node, for errors
- vars []variable // push-down stack of variable values.
- depth int // the height of the stack of executing templates.
+ tmpl *Template
+ wr io.Writer
+ node parse.Node // current node, for errors
+ vars []variable // push-down stack of variable values.
+ depth int // the height of the stack of executing templates.
+ deadline time.Time // ntfy: wall-clock bail-out; zero means no limit
+ steps int64 // ntfy: node counter for amortized deadline checks
}
// variable holds the dynamic value of a variable such as $, $x etc.
@@ -131,6 +135,10 @@
return e.Err
}
+// ErrExecutionInterrupted is wrapped into the error returned by Execute when a template exceeds the
+// deadline set via Template.SetExecutionDeadline. Detect it with errors.Is. (ntfy addition)
+var ErrExecutionInterrupted = errors.New("template execution interrupted")
+
// errorf records an ExecError and terminates processing.
func (s *state) errorf(format string, args ...any) {
name := doublePercent(s.tmpl.Name())
@@ -214,9 +222,10 @@
value = reflect.ValueOf(data)
}
state := &state{
- tmpl: t,
- wr: wr,
- vars: []variable{{"$", value}},
+ tmpl: t,
+ wr: wr,
+ vars: []variable{{"$", value}},
+ deadline: t.deadline, // ntfy: wall-clock execution bail-out
}
if t.Tree == nil || t.Root == nil {
state.errorf("%q is an incomplete or empty template", t.Name())
@@ -260,6 +269,11 @@
// generating output as they go.
func (s *state) walk(dot reflect.Value, node parse.Node) {
s.at(node)
+ // ntfy: amortized wall-clock bail-out to prevent CPU DoS from user-supplied templates
+ // (tight/nested ranges that never write output). See GHSA-rhwf-xgc9-m9fp.
+ if s.steps++; s.steps&0xff == 0 && !s.deadline.IsZero() && time.Now().After(s.deadline) {
+ s.errorf("execution interrupted: %w", ErrExecutionInterrupted)
+ }
switch node := node.(type) {
case *parse.ActionNode:
// Do not pop variables so they persist until next end.
diff -ruN a/template.go b/template.go
--- a/template.go 2026-07-08 21:46:30.952848382 +0200
+++ b/template.go 2026-07-08 21:46:30.953952891 +0200
@@ -9,13 +9,15 @@
"reflect"
"sync"
"text/template/parse"
+ "time"
)
// common holds the information shared by related templates.
type common struct {
- tmpl map[string]*Template // Map from name to defined templates.
- muTmpl sync.RWMutex // protects tmpl
- option option
+ tmpl map[string]*Template // Map from name to defined templates.
+ muTmpl sync.RWMutex // protects tmpl
+ option option
+ deadline time.Time // ntfy: wall-clock execution deadline (zero = none)
// We use two maps, one for parsing and one for execution.
// This separation makes the API cleaner since it doesn't
// expose reflection to the client.
@@ -49,6 +51,15 @@
return t.name
}
+// SetExecutionDeadline sets a wall-clock deadline after which Execute aborts with an error wrapping
+// ErrExecutionInterrupted. A zero deadline disables the limit. It bounds CPU for untrusted templates
+// that text/template cannot otherwise interrupt. (ntfy addition, see GHSA-rhwf-xgc9-m9fp.)
+func (t *Template) SetExecutionDeadline(deadline time.Time) *Template {
+ t.init()
+ t.deadline = deadline
+ return t
+}
+
// New allocates a new, undefined template associated with the given one and with the same
// delimiters. The association, which is transitive, allows one template to
// invoke another with a {{template}} action.
template/gotext/template.go
+3 -14
View File
@@ -9,15 +9,13 @@ import (
"reflect"
"sync"
"text/template/parse"
"time"
)
// common holds the information shared by related templates.
type common struct {
tmpl map[string]*Template // Map from name to defined templates.
muTmpl sync.RWMutex // protects tmpl
option option
deadline time.Time // ntfy: wall-clock execution deadline (zero = none)
tmpl map[string]*Template // Map from name to defined templates.
muTmpl sync.RWMutex // protects tmpl
option option
// We use two maps, one for parsing and one for execution.
// This separation makes the API cleaner since it doesn't
// expose reflection to the client.
@@ -51,15 +49,6 @@ func (t *Template) Name() string {
return t.name
}
// SetExecutionDeadline sets a wall-clock deadline after which Execute aborts with an error wrapping
// ErrExecutionInterrupted. A zero deadline disables the limit. It bounds CPU for untrusted templates
// that text/template cannot otherwise interrupt. (ntfy addition, see GHSA-rhwf-xgc9-m9fp.)
func (t *Template) SetExecutionDeadline(deadline time.Time) *Template {
t.init()
t.deadline = deadline
return t
}
// New allocates a new, undefined template associated with the given one and with the same
// delimiters. The association, which is transitive, allows one template to
// invoke another with a {{template}} action.