Experiment 1
Keep η = 0.2 and cap = 5. Change only λ from 1 to 0.
Check the prediction
The objective now only penalizes distance from 3, so the optimum becomes 3 instead of 1.5.
Optimization and tradeoffs
The best answer depends on the objective.
Derive a constrained quadratic optimum, explain the penalty tradeoff, and distinguish algorithm failure from a bad objective.
Start with: Functions and slopes. A derivative is the local slope; a constraint limits allowed choices.
01 · Commit to a prediction
What do you expect?
No. Completing the square gives 2(x−1.5)² + 4.5, minimized at x = 1.5 if the constraint allows it.
Responses stay in this page only. Reloading or closing may discard them. Nothing is transmitted, saved or synchronized.
02 · Change an assumption
Predict. Change. Explain.
Before changing a control, say what should move and why. Start with the experiments below. Reset restores the starting model; it preserves your written responses.
Experiment 2
Reset. Lower only the cap to 1.
Check the prediction
The feasible optimum is 1. Its derivative is not zero: moving toward 1.5 would improve the objective but violate the constraint.
0 to 3 · step 0.25
0 to 1 · step 0.05
0 to 5 · step 0.25
Calculated model output. The table gives the same values. Displayed values are rounded; calculations keep full precision.
Read the plotted data as a table
03 · Connect the mechanism
From the picture to the quantities.
An optimizer improves the objective you supplied. The penalty defines what a compromise costs; it is not a fact discovered by the algorithm. Gradient descent steps against the slope. Projection clips the next value to the feasible interval. A large step can oscillate even when the objective has one clear minimum.
f(x) = (x−3)² + λx², 0 ≤ x ≤ c
f′(x) = 2[(1+λ)x−3]
xₜ₊₁ = clip(xₜ − ηf′(xₜ), 0, c)
x* = min[c, 3/(1+λ)]
f′(x) = 2[(1+λ)x−3]
xₜ₊₁ = clip(xₜ − ηf′(xₜ), 0, c)
x* = min[c, 3/(1+λ)]
x, cap c, λ and η are dimensionless in this illustrative cost model. λ is 0 to 3, η is 0 to 1, c is 0 to 5. clip replaces values outside [0,c] by the nearest boundary. Initial x₀ = 0; t counts 20 updates. Objective values are arbitrary cost units, not currency or measured welfare.
Worked example
For λ = 1, f(x) = x²−6x+9+x² = 2(x−1.5)²+4.5. That independently establishes the optimum. From x₀ = 0 with η = 0.2 and cap = 5, the first gradient is −6 and x₁ = 1.2. Next gradient is −1.2, giving x₂ = 1.44. If cap = 1, clipping makes the feasible optimum 1.
Assumptions and limits
This is a convex quadratic, so there are no competing local minima. Without clipping, error multiplies by 1−2η(1+λ); convergence requires 0 < η < 1/(1+λ). That bound also suffices for the projected iteration here; clipping can change behavior outside it. No uncertainty, multiple stakeholders or unmeasured harms are encoded. Improving a proxy need not improve the outcome you actually value.
The annotated sources distinguish established results from this lesson’s original examples.
04 · Follow the structure
Where else does this apply?
Regularized estimation
A fit penalty trades data agreement against parameter magnitude.
Boundary: Penalty strength does not prove improved predictions; held-out evidence is needed.
Engineering design
A setting balances deviation from a target against resource use.
Boundary: A scalar penalty assumes a chosen exchange rate between goals and may omit hard safety constraints.
05 · Retrieve without hints
Close the explanation. Try a new case.
Write an answer before opening its feedback. Later, return directly here without rereading above. Recognition, explanation and transfer are separate outcomes. No page action or answer reveal measures mastery.
Recognition and explanation
Reveal reasoning and rubric
Yes, at a constraint boundary where the improving direction is infeasible. For λ = 1 and cap = 1, the derivative is −2 but x cannot increase.
Self-check: Identify the improving direction and the blocked feasible move.
Calculation
Reveal reasoning and rubric
f(x) = 3x²−6x+9 = 3(x−1)²+6. The feasible minimum is x = 1.
Self-check: Derive the result from the objective and check the constraint.
Novel transfer
Reveal reasoning and rubric
Not by itself. Convergence concerns the stated objective. A proxy that omits quality needs revision and outcome validation. Smaller steps address numerical behavior, not the missing goal.
Self-check: Separate objective validity from optimization accuracy.
On a later day, try again and record actual evidence in the curriculum. A later unaided explanation and a fresh transfer problem give stronger evidence than immediate familiarity. No reminder is scheduled.
Sources & scope
What supports the lesson?
- Boyd and Vandenberghe, Convex Optimization, §§4.7 and 9.3. Tradeoffs and gradient descent. The scalar constrained example here is derived separately.
Original teaching examples. Reference links need a connection; the lesson itself does not. Built 2026-10-11. Learner understanding is not assessed.