What Are Convex Functions?

Convex functions are a class of real-valued functions defined on convex domains with the property that the line segment connecting any two points on the graph of the function lies entirely at or above the graph. Formally, a function f from a convex subset of a vector space to the real numbers is convex if, for any two points x and y in the domain and any scalar t between zero and one, the function value at the weighted combination tx + (1-t)y does not exceed the corresponding weighted combination of f(x) and f(y). This inequality, called Jensen's inequality in its general form, captures the idea that the function curves upward and has no local valleys that are not also global minima. Convex functions are central to optimization theory, mathematical analysis, probability, and many branches of engineering.

The study of convexity draws on real analysis, linear algebra, and geometry. The concept extends to higher dimensions through multivariable calculus, where the positive semi-definiteness of the Hessian matrix of second derivatives provides the standard second-order characterization for twice-differentiable functions.

Definitions and Key Properties

A function f on a convex domain satisfies the convexity inequality for all pairs of domain points and all values of the interpolation parameter t in the closed unit interval. Strict convexity strengthens this to a strict inequality for t in the open interval, ruling out flat segments. Strong convexity, used in convergence analysis of optimization algorithms, requires that the function curves upward at least as fast as a fixed quadratic. For a twice continuously differentiable function, convexity is equivalent to a positive semi-definite Hessian; for a scalar function of one variable, this reduces to a non-negative second derivative everywhere on the domain. The Stanford convex optimization course notes by Boyd and Vandenberghe give a systematic development of these conditions along with examples and counterexamples drawn from engineering problems.

Convex Optimization

The most consequential application of convex functions is in optimization: any local minimum of a convex function over a convex feasible set is also a global minimum. This property means that gradient-based and interior-point algorithms guaranteed to converge to local optima automatically find the globally best solution, eliminating the need for exhaustive search or restarts from multiple initial conditions. Linear programs, quadratic programs with positive semi-definite objectives, second-order cone programs, and semidefinite programs are all special cases of convex optimization. The Princeton lecture notes on the theory of convex functions outline the hierarchy of problem classes and their computational complexity. Disciplined convex programming frameworks such as CVX and CVXPY allow practitioners to specify convex optimization problems in a natural mathematical syntax and have them solved automatically by interior-point solvers.

Convexity in Machine Learning and Signal Processing

The objective functions of many statistical estimation and machine learning problems are convex. Least-squares regression, logistic regression, and support vector machine training all minimize convex loss functions over convex constraint sets, guaranteeing that numerical solvers find the globally optimal parameters given enough computation. Regularization terms such as the L1 (lasso) and L2 (ridge) penalties are convex, enabling efficient computation of sparse or small-norm solutions. In signal processing, convexity arises in problems of basis pursuit, compressed sensing recovery, and filter design. Stanford's CS229 course on convex optimization for machine learning details how gradient descent, stochastic gradient descent, and Newton's method behave differently on strongly versus weakly convex objectives.

Applications

Convex functions have applications across engineering, statistics, and applied mathematics, including:

• Network routing and resource allocation, where convex cost functions enable globally optimal traffic flow and bandwidth assignment
• Portfolio optimization in financial engineering, where mean-variance objectives are quadratic and convex
• Control synthesis, where linear matrix inequality formulations express controller design as semidefinite programs
• Antenna array beamforming, where convex constraints shape radiation patterns subject to side-lobe and gain requirements