SKSurrogate FAQ

What is SKSurrogate?

SKSurrogate is an open-source Python package built on scikit-learn for surrogate-based optimization of expensive objectives — hyperparameter and pipeline search, statistical evaluation, sensitivity analysis and experiment design — with a control plane that runs and logs searches and a research workspace that records evidence. It is MIT licensed and installs with pip.

Is SKSurrogate a replacement for scikit-learn?

No. It is a search and evaluation layer on top of scikit-learn. SurrogateRandomCV is a BaseSearchCV subclass with the same call shape as RandomizedSearchCV, and your estimators, splitters, scorers and pipelines are used as they are.

Does it work with scikit-learn pipelines and parameter grids?

Yes. You pass an estimator or a pipeline plus a param_distributions mapping, with numeric ranges as tuples and categorical choices as lists. Because the class is a BaseSearchCV subclass, best_params_, best_score_, cv_results_ and refitting behave as they do in scikit-learn.

Is it faster than Optuna?

Not at small budgets. On six public datasets at 30 evaluations Optuna-TPE is top on five of six and the whole table spans about half a point of accuracy; at 60 evaluations the surrogate arm leads all six. The surrogate's advantage is evaluation efficiency rather than wall-clock speed: in the pipeline-space study it reached within 1 per cent of its final best in a median of 6 evaluations against 11 to 17 for the other arms, while the Gaussian-process fit grows as O(n^3) and search overhead consumed about half of a 120-second budget.

How many evaluations does the search need?

Budgets are counted in candidate-model evaluations, not seconds, so they are hardware-independent. The published equal-budget suite used 30 and 60 evaluations with seven seeds per cell; the pipeline-space study used equal wall-clock, equal 40 evaluations, and a wall-budget cap.

What surrogate model does it use by default?

Since v0.2 the default response surface is a stock, un-tuned RBF GaussianProcessRegressor. The previous cubic Hilbert polynomial remains available as HilbertRegressor, and any scikit-learn regressor can be supplied. The change was made on the evidence of the equal-budget benchmark, where the Gaussian process beat the polynomial surface at both budgets.

Does it handle classification and regression?

Yes. AML(preset="classification") and AML(preset="regression") need no search configuration. For pandas DataFrames the preset fits numeric, categorical, datetime and text preprocessing inside each cross-validation fold when cat_cols is omitted.

How does it handle categorical features?

Through the preset's fold-local preprocessing, or explicitly with cat_cols. This is worth checking in your own pipeline: the benchmark study found ordinal encoding of unordered categories to be a defect in the package itself. Replacing it with a mixed categorical surface lifted best cross-validated score by at least 0.3 points on four of six datasets and moved the search above TPE on three of them.

What guarantee do the prediction intervals have?

The split-conformal intervals assume that calibration and future examples are exchangeable and that the model was chosen independently of the calibration set. Under those assumptions they provide finite-sample marginal coverage — not per-subgroup or conditional coverage, and not coverage under distribution shift. Coverage is meant to be measured on held-out labels with prediction_interval_diagnostics.

Can I run it without a server or cloud account?

Yes. It is a Python package. The control plane with its API and UI is optional and runs from a Docker Compose profile on a single port, 8013, with an optional API key; nothing in the loop calls an external service.

How do I cite SKSurrogate?

Cite the archived release rather than the repository URL. The software archive recorded in CITATION.cff carries a DOI, and the benchmark records and the D-track study have their own DOIs. Ready-to-paste BibTeX and BibLaTeX are on the citation page.

What is SKSurrogate not?

Not a managed cloud service, not a fixed model zoo, and not a chat assistant. The shipped tooling is deterministic, the search space is your pipeline plus the ranges you declare, and every evaluation is recorded. A bounded, opt-in cited-note synthesis exists; a conversational research assistant is roadmap, not this release.

Where to go next

The quickstart installs and runs the tuner; the benchmark page has the tables and the caveats; the concepts explain surrogate optimization, conformal intervals and nested cross-validation in the terms the documentation uses.