Mathematical Optimization for Data Visualization
In general, you could say data visualization is about maximizing information acquisition and minimizing confusion and effort. However, I do not know of any simple mathematical definition of the insights/ink ratio, so this page is limited to more concrete applications.
Graph layouts
The often used force-directed layouts combine an implicit objective function and a first order method to minimise it. Layered graph layouts use another objective function and other heuristics. Not only node positioning but also edge routing etc. can be subject to optimization.
Label positioning
Good label positions should be close to the labeled elements, while avoiding overlaps. This can be formulated and optimized mathematically. See, for instance Mathematically optimize label positions in scatter plots and (Making Line Plots Delightful with Optimized Direct Labeling. Going further, the choice of instances to label may also be an interesting problem.
Reordering/seriation
In categorical heatmaps, for instance, the rows and/or columns of may represent categorical variables whose order is not predetermined. In this case, applying a reordering or “seriation” algorithm can be very advantageous and reveal otherwise hidden structures.
Dimensionality reduction
Dimensionality reduction, projection pursuit etc.
Color palette optimization
Euler and bubble plots
In some cases, you need optimization to make a diagram useful at all, or even to understand if it is feasible.