API — ppbcc.performance_portability

Module Overview

Application-efficiency and performance-portability analysis.

Metrics

Calculate application efficiency and performance portability.

ppbcc.performance_portability.metrics.calculate_metrics(df, description_is_workload, non_zero_pp=False, hardware_universe=None, application_universe=None)

Calculate application efficiency and performance portability.

Duplicate measurements are reduced to their median runtime. Application efficiencies are computed per hardware/workload and then arithmetically averaged over workloads. PP is the harmonic mean across all selected hardware. By default, an absent result contributes zero and makes PP zero. In non-zero PP mode, platforms with zero efficiency are omitted from the harmonic mean; the application-efficiency output still retains those zeros.

Parameters:

• df (DataFrame) – Selected benchmark rows.

• description_is_workload (bool) – Whether Description belongs to the workload key rather than the application label.

• non_zero_pp (bool) – Whether to exclude unsupported platforms from PP.

• hardware_universe (Iterable[str] | None) – Optional complete platform set. This is used for per-size scaling so a platform with no row at one size contributes zero rather than disappearing from that size’s PP calculation.

• application_universe (Iterable[str] | None) – Optional complete application set. This is used for per-size calculations so an implementation with no row at one size receives zero rather than disappearing from that size.

Returns:

A pair (efficiency, portability). The first DataFrame has one row per application/hardware; the second has one row per application.

Raises:

ValueError – If there are no positive numeric wall-clock runtimes.

Return type:

tuple[DataFrame, DataFrame]

ppbcc.performance_portability.metrics.calculate_metrics_by_size(df, description_is_workload, non_zero_pp=False)

Calculate both metrics independently for each problem size.

Parameters:

• df (DataFrame) – Selected rows for one benchmark problem.

• description_is_workload (bool) – Whether Description belongs to the workload key.

• non_zero_pp (bool) – Whether unsupported platforms are excluded from PP.

Returns:

Efficiency and portability DataFrames that retain problem size.

Return type:

tuple[DataFrame, DataFrame]

ppbcc.performance_portability.metrics.calculate_scaling_metrics(df, description_is_workload, non_zero_pp=False)

Calculate performance portability independently for each problem size.

Parameters:

• df (DataFrame)

• description_is_workload (bool)

• non_zero_pp (bool)

Return type:

DataFrame

ppbcc.performance_portability.metrics.calculate_average_size_metrics(df, description_is_workload, non_zero_pp=False, average_over='pp')

Average application efficiency and PP over sizes.

Application efficiency is calculated independently at every size and then arithmetically averaged, so every problem size has equal weight. PP is reduced in one of two ways:

• pp: PP is calculated at every size and the scores are averaged arithmetically.

• efficiency: the size sweep is treated as one benchmark. PP is the harmonic mean over platforms of the size-averaged efficiencies, i.e. exactly the PP of the plotted application-efficiency panel.

The two differ whenever an application’s platform ranking changes with the size: the arithmetic mean of harmonic means is not a harmonic mean.

Parameters:

• df (DataFrame) – Selected rows for one benchmark problem.

• description_is_workload (bool) – Whether Description belongs to the workload key.

• non_zero_pp (bool) – Whether unsupported platforms are excluded from PP.

• average_over (str) – pp or efficiency, see above.

Returns:

Size-averaged application efficiency and performance portability.

Raises:

ValueError – If average_over is unsupported.

Return type:

tuple[DataFrame, DataFrame]

ppbcc.performance_portability.metrics.calculate_extreme_size_metrics(df, description_is_workload, mode, non_zero_pp=False)

Select each application’s best or worst metrics over problem sizes.

Application efficiency is reduced independently for every application and hardware pair. Performance portability is reduced independently for every application, so both Cascade panels represent the requested size-axis extreme of the metric they display.

Parameters:

• df (DataFrame) – Selected rows for one benchmark problem.

• description_is_workload (bool) – Whether Description belongs to the workload key.

• mode (str) – best for maxima or worst for minima.

• non_zero_pp (bool) – Whether unsupported platforms are excluded from PP.

Returns:

Application-efficiency and performance-portability extrema.

Raises:

ValueError – If mode is not best or worst.

Return type:

tuple[DataFrame, DataFrame]

ppbcc.performance_portability.metrics.calculate_export_metrics(df, description_is_workload, non_zero_pp=False, average_over='pp')

Calculate export metrics without combining sizes or precisions.

Returns one efficiency row per application, size, precision, and hardware, and one portability row per application, size, and precision. Additional rows with Problem Size average contain per-precision arithmetic means over the independently calculated size efficiencies. Their PP is the mean of the per-size PP scores for average_over="pp", or the harmonic mean of the averaged efficiencies for average_over="efficiency" (see calculate_average_size_metrics()).

Parameters:

• df (DataFrame)

• description_is_workload (bool)

• non_zero_pp (bool)

• average_over (str)

Return type:

tuple[DataFrame, DataFrame]

ppbcc.performance_portability.metrics.calculate_runtimes(df, description_is_workload, time_column, remove_description=False)

Reduce benchmark rows to one runtime per application and platform.

Duplicate measurements are reduced to their median. Rows whose runtime is missing or not positive are dropped with a warning, because a time bar on a logarithmic axis cannot show them.

Parameters:

• df (DataFrame) – Selected benchmark rows of one problem size and precision.

• description_is_workload (bool) – Whether Description belongs to the workload key rather than the application label.

• time_column (str) – Runtime column to reduce, e.g. Kernel Time.

• remove_description (bool) – Whether implementation variants of one paradigm are combined; the fastest variant then stands for the paradigm.

Returns:

One row per application, hardware and precision with the runtime in nanoseconds in Runtime (ns).

Raises:

ValueError – If no positive runtime remains.

Return type:

DataFrame

Data Selection

Load and select benchmark CSV data for portability analysis.

ppbcc.performance_portability.selection.parse_problem_size(value)

Parse a numeric problem size or a size-summary sentinel.

Parameters:

value (str) – Numeric size or all, average, best, or worst.

Returns:

Numeric size or a normalized summary sentinel.

Raises:

argparse.ArgumentTypeError – If the value is unsupported.

Return type:

float | str

ppbcc.performance_portability.selection.configure_logging(verbosity)

Configure Loguru consistently with benchmark.py.

Parameters:

verbosity (int) – Number of -v flags supplied by the user.

Return type:

None

ppbcc.performance_portability.selection.load_benchmark_csvs(paths)

Read and concatenate benchmark result CSV files.

Parameters:

paths (list[Path]) – CSV files produced by benchmark.py.

Returns:

Concatenated benchmark data with a source-file column.

Raises:

• FileNotFoundError – If an input path does not exist.

• ValueError – If a file does not have the benchmark output schema.

Return type:

DataFrame

ppbcc.performance_portability.selection.select_problem_rows(df, name, description_include, description_exclude, problem_size, precision)

Select a problem and optionally filter its descriptions.

Parameters:

• df (DataFrame) – Combined benchmark results.

• name (str) – Requested benchmark problem.

• description_include (str | None) – Optional regular expression descriptions must match.

• description_exclude (str | None) – Optional regular expression descriptions must not match.

• problem_size (float | None) – Optional exact problem size.

• precision (int | None) – Optional floating-point precision in bits.

Returns:

A tuple of filtered rows, resolved problem name, and a flag indicating whether Description is a workload dimension.

Raises:

ValueError – If selection leaves no rows or required values are invalid.

Return type:

tuple[DataFrame, str, bool]

Complexity Join

Load and join code-complexity data with portability results.

ppbcc.performance_portability.complexity.CPP_TOKENS = frozenset({'cplusplus', 'cpp', 'cpu'})

Framework labels that name the sequential C++ reference implementation.

ppbcc.performance_portability.complexity.resolve_complexity_metric(df, requested, source)

Resolve or calculate a requested code-complexity metric.

Parameters:

• df (DataFrame) – Raw complexity data.

• requested (str) – Metric name or alias supplied on the command line.

• source (Path) – Complexity CSV path.

Returns:

A pair containing enriched complexity data and the canonical metric column name.

Raises:

ValueError – If the metric is unknown or cannot be calculated.

Return type:

tuple[DataFrame, str]

ppbcc.performance_portability.complexity.load_complexity_baseline(path, problem_query, metric_request)

Load the absolute sequential C++ value of one complexity metric.

Normalized data says only that CPP is 100 %; a plot that wants to state what 100 % stands for has to read the unscaled value back out.

Parameters:

• path (Path) – Complexity CSV path.

• problem_query (str) – User-supplied benchmark problem query.

• metric_request (str) – Requested metric or alias.

Returns:

The CPP value of the resolved metric, unscaled.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the input is invalid or no CPP baseline is available.

Return type:

float

ppbcc.performance_portability.complexity.load_complexity_data(path, problem_query, metric_request, normalize=True)

Load, select, calculate, and optionally normalize complexity data.

Parameters:

• path (Path) – Complexity CSV path.

• problem_query (str) – User-supplied benchmark problem query.

• metric_request (str) – Requested metric or alias.

• normalize (bool) – Whether to express values as a percentage of CPP.

Returns:

A pair of selected complexity rows and the plotted metric label.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the input is invalid or no CPP baseline is available.

Return type:

tuple[DataFrame, str]

ppbcc.performance_portability.complexity.load_complexity_export_data(path, problem_query)

Load every numeric complexity metric and derive Halstead metrics.

Source metric columns such as SLOC and the primitive Halstead counts are retained. When all four primitive counts are available, all supported derived Halstead metrics are calculated even if they were not requested for the plot.

Parameters:

• path (Path) – Complexity CSV path.

• problem_query (str) – User-supplied benchmark problem query.

Returns:

Median numeric complexity metrics grouped by Framework.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the input schema or Halstead counts are invalid, or no numeric metric columns are available.

Return type:

DataFrame

ppbcc.performance_portability.complexity.merge_portability_complexity(portability, complexity, metric)

Match application PP values with code-complexity rows.

Exact application labels are preferred. For labels containing a description, the description alone and then the base paradigm are tried as fallbacks; this maps Cuda[Cublas] to Cublas and polyhedral model labels such as Cuda[Eros] to Cuda.

Parameters:

• portability (DataFrame) – Performance portability per application.

• complexity (DataFrame) – Complexity per framework.

• metric (str) – Complexity metric column.

Returns:

Matched application, PP, and complexity values.

Raises:

ValueError – If none of the applications can be matched.

Return type:

DataFrame

ppbcc.performance_portability.complexity.add_complexity_to_export(data, complexity)

Add all matching complexity values to an export DataFrame.

Export rows are preserved when no complexity framework matches an application; the metrics are left empty for those rows.

Parameters:

• data (DataFrame) – Application-efficiency or performance-portability data.

• complexity (DataFrame) – Complexity values keyed by Framework.

Returns:

A copy of data with every complexity metric after Application.

Return type:

DataFrame

ppbcc.performance_portability.complexity.append_cpp_complexity_row(data, complexity, problem)

Append the complexity-only CPP baseline to an export DataFrame.

CPP is the complexity reference implementation but has no benchmark measurements. Its size, precision, hardware, and performance metric cells therefore remain empty in both exported datasets.

Parameters:

• data (DataFrame)

• complexity (DataFrame)

• problem (str)

Return type:

DataFrame

ppbcc.performance_portability.complexity.export_metrics_to_csv(efficiency, portability, output)

Export metric DataFrames next to the plot using its filename prefix.

Parameters:

• efficiency (DataFrame) – Application-efficiency data to export.

• portability (DataFrame) – Performance-portability data to export.

• output (Path) – Resolved plot path whose stem supplies the filename prefix.

Returns:

The application-efficiency and performance-portability CSV paths.

Return type:

tuple[Path, Path]

Rank Correlation

Rank-correlate paradigm orderings between benchmark problems.

A single benchmark problem orders the paradigms it was run with. Whether that order carries over to another problem is a question about ranks, not values: Spearman’s \(\rho\) between two problems is high when both put the same paradigms on top, and near zero when one problem says nothing about the other.

Every variable is reduced to one number per paradigm, because a paradigm is what the problems have in common — implementation variants such as Cuda[Naive] and Cuda[SharedMemory] exist in one problem and not the next. The sequential C++ reference is left out: it is the complexity baseline, not one of the compared paradigms, and it has no benchmark results to rank.

ppbcc.performance_portability.correlation.CORRELATION_PP = 'pp'

--correlation value that ranks paradigms by performance portability.

ppbcc.performance_portability.correlation.PP_ALIASES = frozenset({'p3', 'performance-portability', 'phi', 'portability', 'pp'})

Accepted spellings of CORRELATION_PP.

ppbcc.performance_portability.correlation.MINIMUM_PAIR_SIZE = 3

Smallest number of shared paradigms for which a rho is worth reporting.

ppbcc.performance_portability.correlation.RANK = 'Rank (best first)'

Column of the exported per-paradigm table holding the rank of a value.

ppbcc.performance_portability.correlation.CORRELATION_METHOD = 'spearman'

Correlation method; the table is about orderings, not about values.

ppbcc.performance_portability.correlation.resolve_correlation_variable(request)

Resolve a --correlation request to a rankable variable.

Parameters:

request (str) – User-supplied variable name or alias.

Returns:

CORRELATION_PP, or the canonical complexity metric column.

Raises:

ValueError – If the request names neither PP nor a complexity metric.

Return type:

str

ppbcc.performance_portability.correlation.base_paradigm(label)

Strip a bracketed implementation variant from a label.

Parameters:

label (str) – Application or Framework label, e.g. Cuda[SharedMemory].

Returns:

The bare paradigm name, e.g. Cuda.

Return type:

str

ppbcc.performance_portability.correlation.portability_by_paradigm(portability, paradigms)

Reduce performance portability to one value per paradigm.

Where a paradigm was benchmarked in several variants, its best-performing variant stands for it, the same convention the time bar plot uses for runtimes.

Parameters:

• portability (DataFrame) – One PP value per application, as produced by ppbcc.performance_portability.metrics.

• paradigms (Iterable[str]) – Paradigm universe of this problem; labels outside it are dropped.

Returns:

PP indexed by paradigm.

Return type:

Series

ppbcc.performance_portability.correlation.complexity_by_paradigm(path, problem_query, metric_request, paradigms)

Reduce one complexity metric to one value per benchmarked paradigm.

Variants of a paradigm are reduced to their median, which is how ppbcc.performance_portability.complexity.load_complexity_data() already reduces several rows carrying one Framework label. Values are read unscaled: dividing every paradigm of a problem by that problem’s C++ baseline is a positive rescaling and cannot change the problem’s order.

Parameters:

• path (Path) – Complexity CSV path.

• problem_query (str) – User-supplied benchmark problem query.

• metric_request (str) – Requested metric or alias.

• paradigms (Iterable[str]) – Paradigm universe of this problem, from the benchmark CSVs. Frameworks outside it — the C++ baseline, or a paradigm that was measured for complexity but never benchmarked — are dropped, so every variable ranks the same paradigms.

Returns:

The metric indexed by paradigm, and the resolved metric column name.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If no benchmarked paradigm has a complexity value.

Return type:

tuple[Series, str]

ppbcc.performance_portability.correlation.build_value_table(values)

Align the per-problem paradigm values into one wide table.

Parameters:

values (Mapping[str, Series]) – One value series per problem, indexed by paradigm.

Returns:

Paradigms as rows and problems as columns, with a missing value where a paradigm has no value for a problem.

Raises:

ValueError – If fewer than two problems are given.

Return type:

DataFrame

ppbcc.performance_portability.correlation.rank_correlation_matrix(table)

Calculate Spearman’s rho between every pair of problems.

Each pair is correlated over the paradigms both problems have a value for, so a paradigm missing from one problem costs only the pairs that problem takes part in.

Parameters:

table (DataFrame) – Wide value table from build_value_table().

Returns:

A square matrix of rank correlations, problems on both axes.

Raises:

ValueError – If no pair of problems shares enough paradigms to rank.

Return type:

DataFrame

ppbcc.performance_portability.correlation.shared_paradigm_counts(table)

Count the paradigms each pair of problems has a value for.

Parameters:

table (DataFrame) – Wide value table from build_value_table().

Returns:

A square matrix of counts, problems on both axes.

Return type:

DataFrame

ppbcc.performance_portability.correlation.rank_table(table, variable, value_column)

Rank the paradigms of every problem, best first.

“Best” is the largest value for performance portability and the smallest one for a complexity metric. The convention only labels the export: the correlations are calculated from the values themselves, and reversing every problem’s direction at once leaves them unchanged.

Parameters:

• table (DataFrame) – Wide value table from build_value_table().

• variable (str) – CORRELATION_PP or a complexity metric column, which decides which end of the scale rank one sits at.

• value_column (str) – Name the values are written under.

Returns:

One row per problem and paradigm with the value and its rank.

Return type:

DataFrame

ppbcc.performance_portability.correlation.write_rank_correlation_csv(matrix, output)

Write the rank-correlation matrix next to the other analysis outputs.

Parameters:

• matrix (DataFrame) – Square matrix from rank_correlation_matrix().

• output (Path) – Resolved output path.

Returns:

The written path.

Return type:

Path

Command Line Interface

Argument definitions for the P2- and P3-analysis CLIs.

ppbcc.performance_portability.options.P2_CHARTS = ('cascade', 'heatmap', 'double-heatmap', 'boxplot', 'time-barplot')

Charts that need benchmark results only.

ppbcc.performance_portability.options.P3_CHARTS = ('navchart', 'combined', 'complexity-comparison', 'rank-correlation')

Charts that combine benchmark results with code complexity.

ppbcc.performance_portability.options.RANK_CORRELATION = 'rank-correlation'

The P3 output that is a CSV table rather than a figure.

ppbcc.performance_portability.options.build_p2_parser()

Build the parser for charts that need benchmark results only.

Returns:

The configured argument parser.

Return type:

ArgumentParser

ppbcc.performance_portability.options.build_p3_parser()

Build the parser for charts that combine benchmarks and complexity.

Returns:

The configured argument parser.

Return type:

ArgumentParser

Command-line orchestration for P2 and P3 analysis.

ppbcc.performance_portability.cli.p2analysis_main(argv=None)

Run the P2 analysis command-line program (benchmark results only).

Parameters:

argv (list[str] | None) – Command-line arguments, or None to read sys.argv.

Returns:

zero on success, one on failure.

Return type:

Process exit status

ppbcc.performance_portability.cli.p3analysis_main(argv=None)

Run the P3 analysis command-line program (benchmarks and complexity).

Parameters:

argv (list[str] | None) – Command-line arguments, or None to read sys.argv.

Returns:

zero on success, one on failure.

Return type:

Process exit status

Shared Constants

Shared column names and runtime-unit conversions.

ppbcc.constants.AGGREGATED_KERNEL = 'all kernels of the region'

Kernel name given to a row that stands for every launch of one region.

ppbcc.constants.TIME_COLUMNS = {'force-update': 'Force Update Time', 'kernel': 'Kernel Time', 'neighbor-search': 'Neighbor Search Time', 'wall-clock': 'Wall Clock Time'}

Runtime columns selectable for the time bar plot, keyed by their CLI name.

Peak Performance of the Platforms

Published peak floating-point performance of the benchmarked platforms.

Every value is the datasheet peak in FLOP/s: execution units times the FMA operations per unit times the (boost) core clock, as the sources below tabulate it. Where a source gives a base and a boost clock value, the boost value is used, since it is the peak a card reaches under sustained load. Sources were accessed on 2026-09-16.

These are not measured ceilings. ppbcc profile scales Nsight Compute’s peak_sustained counters with the clock a card actually ran at, which for the RTX 5080 of the study gives 57.4 TFLOP/s against the tabulated 56.3.

Caveats worth knowing before comparing platforms:

• The GeForce FP64 figures of the sources are one to two orders of magnitude below FP32. They are reproduced as tabulated, without explanation there.

• The source labels the Instinct MI210 column Vector TFLOPS, which is not necessarily the same unit of work as the NVIDIA per-core FMA figure, so its value may not be directly comparable with the NVIDIA ones.

• The Intel Max 1550 has two Xe-HPC stacks, and SYCL/Level Zero exposes each as its own device. The benchmark runs on one stack, so the published whole-card figure is halved.

ppbcc.hardware.PEAK_PERFORMANCE: dict[str, dict[int, float]] = {'AMD MI210': {32: 181000000000000.0, 64: 22630000000000.0}, 'Intel Max 1550': {32: 26000000000000.0, 64: 26000000000000.0}, 'NVIDIA GH200': {32: 66900000000000.0, 64: 33500000000000.0}, 'NVIDIA RTX3080': {32: 29770000000000.0, 64: 465000000000.0}, 'NVIDIA RTX4060': {32: 15110000000000.0, 64: 236000000000.0}, 'NVIDIA RTX5080': {32: 56300000000000.0, 64: 880000000000.0}}

Peak performance in FLOP/s, keyed by the Hardware label of the benchmark CSVs and then by floating-point precision in bits.

ppbcc.hardware.peak_performance(hardware, precision)

Return the published peak performance of one platform.

Parameters:

• hardware (str) – Hardware label as it appears in the benchmark CSVs. Matching ignores case, spaces and punctuation.

• precision (int) – Floating-point precision in bits (32 or 64).

Returns:

Peak performance in FLOP/s.

Raises:

ValueError – If the platform or the precision is not tabulated.

Return type:

float