Owner: Architect:

Francesco Ilario

Story (Required)

As a Primaza Developer,
I would like Application Agent not processing create/update events during shutdown
So that no new binding happens when termination is asked

Background (Required)

Agents may take some time terminating, as they are in charge of Primaza's resource deletion in their namespace.
A mechanisms to prevent agent from processing non-deletion events on watched resources is required.

See epic for arch document link.

Glossary

See glossary in architecture document

Out of scope

NA

In Scope

• Termination management

Approach(Required)

A termination context should be initialized at Agent startup and invalidated when Agent deletion is requested. Agent deletion is monitored in the controller that watches the agent's Deployment.

Controllers should handle deletion events even if termination context is invalidated. Controllers should check the termination context before processing non-deletion events.

In the following an example in pseudo-Go of the approach:

type MyReconciler struct{
    // ...
    TerminationContext context.Context
}

func (r *MyReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    s, err := r.getResource(ctx, req)
    if err != nil {
        if errors.IsNotFound(err) {
            return r.handleDeletion(ctx, req)
        }
        return ctrl.Result{}, err
    }

    if r.TerminationContext.Err() != nil {
        log.Info("Agent is asked for termination, won't reconcile", "request", req)
        return ctrl.Result{}, nil
    }

    // TODO: above `r.reconcile` should be invoked with a context that merges `ctx` and `r.TerminationContext`.
    // As of now, no easy and standard way of doing this seems to exists.
    //
    // In future something similar to the following should be implemented:
    //
    //  mctx := context.Merge(ctx, r.TerminationContext)
    //  r.reconcile(mctx, s)

    return r.reconcile(ctx, s)
}

Demo requirements(Required)

NA

Dependencies

NA

Edge Case

NA

BDD Tests

Unit tests may suite better here.

Acceptance Criteria

• Development
  Agents don't process new or updated primaza resources during termination

• QE
  There are test cases for agent shutdown

• Docs
  Deferred to an ad-hoc story
  Update architecture document with any changes while implementing

INVEST Checklist

Dependencies identified
Blockers noted and expected delivery timelines set
Design is implementable
Acceptance criteria agreed upon
Story estimated

Legend

Unknown
Verified
Unsatisfied