Quality Indicator CLI¶

The Quality Indicator CLI is a command-line interface for computing quality indicators between two fronts (solution front and reference front). This tool provides an easy way to evaluate the performance of multi-objective optimization algorithms.

Features¶

This CLI tool supports the following quality indicators:

Additive Epsilon (epsilon): Measures the minimum additive factor needed to weakly dominate the reference front
Inverted Generational Distance (igd): Measures the average distance from reference points to the nearest solution
Inverted Generational Distance Plus (igdplus): IGD variant using dominance-based distance calculation
Hypervolume (hv): Volume of objective space dominated by the front
Normalized Hypervolume (nhv): Hypervolume normalized by the reference front’s hypervolume
All indicators: Compute all indicators at once

Installation¶

The CLI is included with jMetalPy. No additional installation is required.

Usage¶

Basic Usage¶

python -m jmetal.util.quality_indicator_cli <front.csv> <reference.csv> <indicator> [options]

Examples¶

Compute IGD between two fronts:

python -m jmetal.util.quality_indicator_cli front.csv reference.csv igd

Compute all indicators with custom reference point:

python -m jmetal.util.quality_indicator_cli front.csv reference.csv all --ref-point 2.0,2.0

Normalize fronts and output as JSON:

python -m jmetal.util.quality_indicator_cli front.csv reference.csv all --normalize --format json

Compute epsilon indicator only:

python -m jmetal.util.quality_indicator_cli front.csv reference.csv epsilon

Options¶

--normalize: Normalize both fronts using reference_only strategy
--ref-point V1,V2,...: Custom reference point for HV/NHV (overrides auto-generation)
--format {text,json}: Output format (default: text)
--margin M: Margin added when auto-building reference point (default: 0.1)
-h, --help: Show help message

File Format¶

CSV files should contain numeric data with one solution per row and one objective per column.

Example front.csv:

0,1.0
2,0.8
4,0.6
6,0.4
8,0.2
0,0.0

Example reference.csv:

1,0.9
3,0.7
5,0.5
7,0.3
9,0.1

Output¶

Text Format (default)¶

Result (epsilon): 0.1
Result (igd): 0.1414213562373095
Result (igdplus): 0.1
Result (hv): 0.84
Result (nhv): -0.037037037037037

JSON Format¶

{
  "epsilon": 0.1,
  "igd": 0.1414213562373095,
  "igdplus": 0.1,
  "hv": 0.84,
  "nhv": -0.037037037037037
}

Important Notes¶

Reference Points¶

HV and NHV require a reference point that is dominated by all solutions in the front
If no reference point is provided, one is automatically generated using the maximum values of the reference front plus a margin
For normalized data, the default reference point is [1.1, 1.1, ...]

Normalization¶

Uses “reference_only” strategy: normalizes both fronts based on the bounds of the reference front
Useful when fronts have different scales or when you want to focus on relative performance

Normalized Hypervolume (NHV)¶

Calculated as: NHV = 1 - HV(front) / HV(reference)
Can be negative if the solution front dominates the reference front
Values closer to 0 indicate better performance

Error Handling¶

The CLI provides informative error messages for common issues:

File not found
Invalid CSV format
Dimension mismatches between fronts
Invalid reference point format
Missing reference points for HV/NHV indicators

Integration with jMetalPy¶

This CLI tool is built on top of jMetalPy’s quality indicator implementations and can be used:

As a standalone tool for evaluating algorithm results
In experimental pipelines and scripts
For comparing different optimization runs
In continuous integration systems for performance monitoring

Practical Example¶

Let’s walk through a complete example using the CLI:

Generate sample data files:

import numpy as np

# Create a sample solution front
front = np.array([[0.1, 0.9], [0.3, 0.7], [0.5, 0.5], [0.7, 0.3], [0.9, 0.1]])
np.savetxt('my_front.csv', front, delimiter=',')

# Create a reference front (e.g., true Pareto front)
reference = np.array([[0.0, 1.0], [0.2, 0.8], [0.4, 0.6], [0.6, 0.4], [0.8, 0.2], [1.0, 0.0]])
np.savetxt('reference_front.csv', reference, delimiter=',')

Compute all quality indicators:

python -m jmetal.util.quality_indicator_cli my_front.csv reference_front.csv all --format json

Expected output:

{
  "epsilon": 0.1,
  "igd": 0.1414213562373095,
  "igdplus": 0.1,
  "hv": 0.84,
  "nhv": 0.15
}

This provides a comprehensive evaluation of your algorithm’s performance compared to the reference front.

Technical Implementation¶

The CLI tool implements the exact same algorithms as jMetal for consistency and compatibility:

IGD formula: IGD = (Σ(d^pow))^(1/pow) / N where d is the minimum distance from each reference point to the solution front
Hypervolume: Uses the WFG algorithm for efficient computation
Epsilon indicator: Implements the additive epsilon metric for convergence assessment

This ensures that results are directly comparable with other tools in the multi-objective optimization community.

Table Of Contents

Contents