Version 1.0

For a short description of the main highlights of the release, please refer to Release Highlights for scikit-learn 1.0.

Legend for changelogs

• Major Feature something big that you couldn’t do before.

• Feature something that you couldn’t do before.

• Efficiency an existing feature now may not require as much computation or memory.

• Enhancement a miscellaneous minor improvement.

• Fix something that previously didn’t work as documented – or according to reasonable expectations – should now work.

• API Change you will need to change your code to have the same effect in the future; or a feature will be removed in the future.

Version 1.0.2

December 2021

• Fix cluster.Birch, feature_selection.RFECV, ensemble.RandomForestRegressor, ensemble.RandomForestClassifier, ensemble.GradientBoostingRegressor, and ensemble.GradientBoostingClassifier do not raise warning when fitted on a pandas DataFrame anymore. #21578 by Thomas Fan.

Changelog

sklearn.cluster

• Fix Fixed an infinite loop in cluster.SpectralClustering by moving an iteration counter from try to except. #21271 by Tyler Martin.

sklearn.datasets

• Fix datasets.fetch_openml is now thread safe. Data is first downloaded to a temporary subfolder and then renamed. #21833 by Siavash Rezazadeh.

sklearn.decomposition

• Fix Fixed the constraint on the objective function of decomposition.DictionaryLearning, decomposition.MiniBatchDictionaryLearning, decomposition.SparsePCA and decomposition.MiniBatchSparsePCA to be convex and match the referenced article. #19210 by Jérémie du Boisberranger.

sklearn.ensemble

• Fix ensemble.RandomForestClassifier, ensemble.RandomForestRegressor, ensemble.ExtraTreesClassifier, ensemble.ExtraTreesRegressor, and ensemble.RandomTreesEmbedding now raise a ValueError when bootstrap=False and max_samples is not None. #21295 Haoyin Xu.

• Fix Solve a bug in ensemble.GradientBoostingClassifier where the exponential loss was computing the positive gradient instead of the negative one. #22050 by Guillaume Lemaitre.

sklearn.feature_selection

• Fix Fixed feature_selection.SelectFromModel by improving support for base estimators that do not set feature_names_in_. #21991 by Thomas Fan.

sklearn.impute

• Fix Fix a bug in linear_model.RidgeClassifierCV where the method predict was performing an argmax on the scores obtained from decision_function instead of returning the multilabel indicator matrix. #19869 by Guillaume Lemaitre.

sklearn.linear_model

• Fix linear_model.LassoLarsIC now correctly computes AIC and BIC. An error is now raised when n_features > n_samples and when the noise variance is not provided. #21481 by Guillaume Lemaitre and Andrés Babino.

sklearn.manifold

• Fix Fixed an unnecessary error when fitting manifold.Isomap with a precomputed dense distance matrix where the neighbors graph has multiple disconnected components. #21915 by Tom Dupre la Tour.

sklearn.metrics

• Fix All sklearn.metrics.DistanceMetric subclasses now correctly support read-only buffer attributes. This fixes a regression introduced in 1.0.0 with respect to 0.24.2. #21694 by Julien Jerphanion.

• Fix All sklearn.metrics.MinkowskiDistance now accepts a weight parameter that makes it possible to write code that behaves consistently both with scipy 1.8 and earlier versions. In turn this means that all neighbors-based estimators (except those that use algorithm="kd_tree") now accept a weight parameter with metric="minkowski" to yield results that are always consistent with scipy.spatial.distance.cdist. #21741 by Olivier Grisel.

sklearn.multiclass

• Fix multiclass.OneVsRestClassifier.predict_proba does not error when fitted on constant integer targets. #21871 by Thomas Fan.

sklearn.neighbors

• Fix neighbors.KDTree and neighbors.BallTree correctly support read-only buffer attributes. #21845 by Thomas Fan.

sklearn.preprocessing

• Fix Fixes compatibility bug with NumPy 1.22 in preprocessing.OneHotEncoder. #21517 by Thomas Fan.

sklearn.tree

• Fix Prevents tree.plot_tree from drawing out of the boundary of the figure. #21917 by Thomas Fan.

• Fix Support loading pickles of decision tree models when the pickle has been generated on a platform with a different bitness. A typical example is to train and pickle the model on 64 bit machine and load the model on a 32 bit machine for prediction. #21552 by Loïc Estève.

sklearn.utils

• Fix utils.estimator_html_repr now escapes all the estimator descriptions in the generated HTML. #21493 by Aurélien Geron.

Version 1.0.1

October 2021

Fixed models

• Fix Non-fit methods in the following classes do not raise a UserWarning when fitted on DataFrames with valid feature names: covariance.EllipticEnvelope, ensemble.IsolationForest, ensemble.AdaBoostClassifier, neighbors.KNeighborsClassifier, neighbors.KNeighborsRegressor, neighbors.RadiusNeighborsClassifier, neighbors.RadiusNeighborsRegressor. #21199 by Thomas Fan.

sklearn.calibration

• Fix Fixed calibration.CalibratedClassifierCV to take into account sample_weight when computing the base estimator prediction when ensemble=False. #20638 by Julien Bohné.

• Fix Fixed a bug in calibration.CalibratedClassifierCV with method="sigmoid" that was ignoring the sample_weight when computing the Bayesian priors. #21179 by Guillaume Lemaitre.

sklearn.cluster

• Fix Fixed a bug in cluster.KMeans, ensuring reproducibility and equivalence between sparse and dense input. #21195 by Jérémie du Boisberranger.

sklearn.ensemble

• Fix Fixed a bug that could produce a segfault in rare cases for ensemble.HistGradientBoostingClassifier and ensemble.HistGradientBoostingRegressor. #21130 Christian Lorentzen.

sklearn.gaussian_process

• Fix Compute y_std properly with multi-target in sklearn.gaussian_process.GaussianProcessRegressor allowing proper normalization in multi-target scene. #20761 by Patrick de C. T. R. Ferreira.

sklearn.feature_extraction

• Efficiency Fixed an efficiency regression introduced in version 1.0.0 in the transform method of feature_extraction.text.CountVectorizer which no longer checks for uppercase characters in the provided vocabulary. #21251 by Jérémie du Boisberranger.

• Fix Fixed a bug in feature_extraction.text.CountVectorizer and feature_extraction.text.TfidfVectorizer by raising an error when ‘min_idf’ or ‘max_idf’ are floating-point numbers greater than 1. #20752 by Alek Lefebvre.

sklearn.linear_model

• Fix Improves stability of linear_model.LassoLars for different versions of openblas. #21340 by Thomas Fan.

• Fix linear_model.LogisticRegression now raises a better error message when the solver does not support sparse matrices with int64 indices. #21093 by Tom Dupre la Tour.

sklearn.neighbors

• Fix neighbors.KNeighborsClassifier, neighbors.KNeighborsRegressor, neighbors.RadiusNeighborsClassifier, neighbors.RadiusNeighborsRegressor with metric="precomputed" raises an error for bsr and dok sparse matrices in methods: fit, kneighbors and radius_neighbors, due to handling of explicit zeros in bsr and dok sparse graph formats. #21199 by Thomas Fan.

sklearn.pipeline

• Fix pipeline.Pipeline.get_feature_names_out correctly passes feature names out from one step of a pipeline to the next. #21351 by Thomas Fan.

sklearn.svm

• Fix svm.SVC and svm.SVR check for an inconsistency in its internal representation and raise an error instead of segfaulting. This fix also resolves CVE-2020-28975. #21336 by Thomas Fan.

sklearn.utils

• Enhancement utils.validation._check_sample_weight can perform a non-negativity check on the sample weights. It can be turned on using the only_non_negative bool parameter. Estimators that check for non-negative weights are updated: linear_model.LinearRegression (here the previous error message was misleading), ensemble.AdaBoostClassifier, ensemble.AdaBoostRegressor, neighbors.KernelDensity. #20880 by Guillaume Lemaitre and András Simon.

• Fix Solve a bug in sklearn.utils.metaestimators.if_delegate_has_method where the underlying check for an attribute did not work with NumPy arrays. #21145 by Zahlii.

Miscellaneous

• Fix Fitting an estimator on a dataset that has no feature names, that was previously fitted on a dataset with feature names no longer keeps the old feature names stored in the feature_names_in_ attribute. #21389 by Jérémie du Boisberranger.

Version 1.0.0

September 2021

Minimal dependencies

Version 1.0.0 of scikit-learn requires python 3.7+, numpy 1.14.6+ and scipy 1.1.0+. Optional minimal dependency is matplotlib 2.2.2+.

Enforcing keyword-only arguments

In an effort to promote clear and non-ambiguous use of the library, most constructor and function parameters must now be passed as keyword arguments (i.e. using the param=value syntax) instead of positional. If a keyword-only parameter is used as positional, a TypeError is now raised. #15005 #20002 by Joel Nothman, Adrin Jalali, Thomas Fan, Nicolas Hug, and Tom Dupre la Tour. See SLEP009 for more details.

Changed models

The following estimators and functions, when fit with the same data and parameters, may produce different models from the previous version. This often occurs due to changes in the modelling logic (bug fixes or enhancements), or in random sampling procedures.

• Fix manifold.TSNE now avoids numerical underflow issues during affinity matrix computation.

• Fix manifold.Isomap now connects disconnected components of the neighbors graph along some minimum distance pairs, instead of changing every infinite distances to zero.

• Fix The splitting criterion of tree.DecisionTreeClassifier and tree.DecisionTreeRegressor can be impacted by a fix in the handling of rounding errors. Previously some extra spurious splits could occur.

• Fix model_selection.train_test_split with a stratify parameter and model_selection.StratifiedShuffleSplit may lead to slightly different results.

Details are listed in the changelog below.

(While we are trying to better inform users by providing this information, we cannot assure that this list is complete.)

Changelog

• API Change The option for using the squared error via loss and criterion parameters was made more consistent. The preferred way is by setting the value to "squared_error". Old option names are still valid, produce the same models, but are deprecated and will be removed in version 1.2. #19310 by Christian Lorentzen.

  • For ensemble.ExtraTreesRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.GradientBoostingRegressor, loss="ls" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.RandomForestRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.HistGradientBoostingRegressor, loss="least_squares" is deprecated, use "squared_error" instead which is now the default.

  • For linear_model.RANSACRegressor, loss="squared_loss" is deprecated, use "squared_error" instead.

  • For linear_model.SGDRegressor, loss="squared_loss" is deprecated, use "squared_error" instead which is now the default.

  • For tree.DecisionTreeRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For tree.ExtraTreeRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

• API Change The option for using the absolute error via loss and criterion parameters was made more consistent. The preferred way is by setting the value to "absolute_error". Old option names are still valid, produce the same models, but are deprecated and will be removed in version 1.2. #19733 by Christian Lorentzen.

  • For ensemble.ExtraTreesRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For ensemble.GradientBoostingRegressor, loss="lad" is deprecated, use "absolute_error" instead.

  • For ensemble.RandomForestRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For ensemble.HistGradientBoostingRegressor, loss="least_absolute_deviation" is deprecated, use "absolute_error" instead.

  • For linear_model.RANSACRegressor, loss="absolute_loss" is deprecated, use "absolute_error" instead which is now the default.

  • For tree.DecisionTreeRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For tree.ExtraTreeRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

• API Change np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. #20165 by Thomas Fan.

• API Change get_feature_names_out has been added to the transformer API to get the names of the output features. get_feature_names has in turn been deprecated. #18444 by Thomas Fan.

• API Change All estimators store feature_names_in_ when fitted on pandas Dataframes. These feature names are compared to names seen in non-fit methods, e.g. transform and will raise a FutureWarning if they are not consistent, see also Release Highlights for scikit-learn 1.0. These FutureWarning s will become ValueError s in 1.2. #18010 by Thomas Fan.

sklearn.base

• Fix config_context is now threadsafe. #18736 by Thomas Fan.

sklearn.calibration

• Feature calibration.CalibrationDisplay added to plot calibration curves. #17443 by Lucy Liu.

• Fix The predict and predict_proba methods of calibration.CalibratedClassifierCV can now properly be used on prefitted pipelines. #19641 by Alek Lefebvre.

• Fix Fixed an error when using a ensemble.VotingClassifier as base_estimator in calibration.CalibratedClassifierCV. #20087 by Clément Fauchereau.

sklearn.cluster

• Efficiency The "k-means++" initialization of cluster.KMeans and cluster.MiniBatchKMeans is now faster, especially in multicore settings. #19002 by Jon Crall and Jérémie du Boisberranger.

• Efficiency cluster.KMeans with algorithm='elkan' is now faster in multicore settings. #19052 by Yusuke Nagasaka.

• Efficiency cluster.MiniBatchKMeans is now faster in multicore settings. #17622 by Jérémie du Boisberranger.

• Efficiency cluster.OPTICS can now cache the output of the computation of the tree, using the memory parameter. #19024 by Frankie Robertson.

• Enhancement The predict and fit_predict methods of cluster.AffinityPropagation now accept sparse data type for input data. #20117 by Venkatachalam Natchiappan

• Fix Fixed a bug in cluster.MiniBatchKMeans where the sample weights were partially ignored when the input is sparse. #17622 by Jérémie du Boisberranger.

• Fix Improved convergence detection based on center change in cluster.MiniBatchKMeans which was almost never achievable. #17622 by Jérémie du Boisberranger.

• Fix cluster.AgglomerativeClustering now supports readonly memory-mapped datasets. #19883 by Julien Jerphanion.

• Fix cluster.AgglomerativeClustering correctly connects components when connectivity and affinity are both precomputed and the number of connected components is greater than 1. #20597 by Thomas Fan.

• Fix cluster.FeatureAgglomeration does not accept a **params kwarg in the fit function anymore, resulting in a more concise error message. #20899 by Adam Li.

• Fix Fixed a bug in cluster.KMeans, ensuring reproducibility and equivalence between sparse and dense input. #20200 by Jérémie du Boisberranger.

• API Change cluster.Birch attributes, fit_ and partial_fit_, are deprecated and will be removed in 1.2. #19297 by Thomas Fan.

• API Change the default value for the batch_size parameter of cluster.MiniBatchKMeans was changed from 100 to 1024 due to efficiency reasons. The n_iter_ attribute of cluster.MiniBatchKMeans now reports the number of started epochs and the n_steps_ attribute reports the number of mini batches processed. #17622 by Jérémie du Boisberranger.

• API Change cluster.spectral_clustering raises an improved error when passed a np.matrix. #20560 by Thomas Fan.

sklearn.compose

• Enhancement compose.ColumnTransformer now records the output of each transformer in output_indices_. #18393 by Luca Bittarello.

• Enhancement compose.ColumnTransformer now allows DataFrame input to have its columns appear in a changed order in transform. Further, columns that are dropped will not be required in transform, and additional columns will be ignored if remainder='drop'. #19263 by Thomas Fan.

• Enhancement Adds **predict_params keyword argument to compose.TransformedTargetRegressor.predict that passes keyword argument to the regressor. #19244 by Ricardo.

• Fix compose.ColumnTransformer.get_feature_names supports non-string feature names returned by any of its transformers. However, note that get_feature_names is deprecated, use get_feature_names_out instead. #18459 by Albert Villanova del Moral and Alonso Silva Allende.

• Fix compose.TransformedTargetRegressor now takes nD targets with an adequate transformer. #18898 by Oras Phongpanagnam.

• API Change Adds verbose_feature_names_out to compose.ColumnTransformer. This flag controls the prefixing of feature names out in get_feature_names_out. #18444 and #21080 by Thomas Fan.

sklearn.covariance

• Fix Adds arrays check to covariance.ledoit_wolf and covariance.ledoit_wolf_shrinkage. #20416 by Hugo Defois.

• API Change Deprecates the following keys in cv_results_: 'mean_score', 'std_score', and 'split(k)_score' in favor of 'mean_test_score' 'std_test_score', and 'split(k)_test_score'. #20583 by Thomas Fan.

sklearn.datasets

• Enhancement datasets.fetch_openml now supports categories with missing values when returning a pandas dataframe. #19365 by Thomas Fan and Amanda Dsouza and EL-ATEIF Sara.

• Enhancement datasets.fetch_kddcup99 raises a better message when the cached file is invalid. #19669 Thomas Fan.

• Enhancement Replace usages of __file__ related to resource file I/O with importlib.resources to avoid the assumption that these resource files (e.g. iris.csv) already exist on a filesystem, and by extension to enable compatibility with tools such as PyOxidizer. #20297 by Jack Liu.

• Fix Shorten data file names in the openml tests to better support installing on Windows and its default 260 character limit on file names. #20209 by Thomas Fan.

• Fix datasets.fetch_kddcup99 returns dataframes when return_X_y=True and as_frame=True. #19011 by Thomas Fan.

• API Change Deprecates datasets.load_boston in 1.0 and it will be removed in 1.2. Alternative code snippets to load similar datasets are provided. Please report to the docstring of the function for details. #20729 by Guillaume Lemaitre.

sklearn.decomposition

• Enhancement added a new approximate solver (randomized SVD, available with eigen_solver='randomized') to decomposition.KernelPCA. This significantly accelerates computation when the number of samples is much larger than the desired number of components. #12069 by Sylvain Marié.

• Fix Fixes incorrect multiple data-conversion warnings when clustering boolean data. #19046 by Surya Prakash.

• Fix Fixed decomposition.dict_learning, used by decomposition.DictionaryLearning, to ensure determinism of the output. Achieved by flipping signs of the SVD output which is used to initialize the code. #18433 by Bruno Charron.

• Fix Fixed a bug in decomposition.MiniBatchDictionaryLearning, decomposition.MiniBatchSparsePCA and decomposition.dict_learning_online where the update of the dictionary was incorrect. #19198 by Jérémie du Boisberranger.

• Fix Fixed a bug in decomposition.DictionaryLearning, decomposition.SparsePCA, decomposition.MiniBatchDictionaryLearning, decomposition.MiniBatchSparsePCA, decomposition.dict_learning and decomposition.dict_learning_online where the restart of unused atoms during the dictionary update was not working as expected. #19198 by Jérémie du Boisberranger.

• API Change In decomposition.DictionaryLearning, decomposition.MiniBatchDictionaryLearning, decomposition.dict_learning and decomposition.dict_learning_online, transform_alpha will be equal to alpha instead of 1.0 by default starting from version 1.2 #19159 by Benoît Malézieux.

• API Change Rename variable names in decomposition.KernelPCA to improve readability. lambdas_ and alphas_ are renamed to eigenvalues_ and eigenvectors_, respectively. lambdas_ and alphas_ are deprecated and will be removed in 1.2. #19908 by Kei Ishikawa.

• API Change The alpha and regularization parameters of decomposition.NMF and decomposition.non_negative_factorization are deprecated and will be removed in 1.2. Use the new parameters alpha_W and alpha_H instead. #20512 by Jérémie du Boisberranger.

sklearn.dummy

• API Change Attribute n_features_in_ in dummy.DummyRegressor and dummy.DummyRegressor is deprecated and will be removed in 1.2. #20960 by Thomas Fan.

sklearn.ensemble

• Enhancement HistGradientBoostingClassifier and HistGradientBoostingRegressor take cgroups quotas into account when deciding the number of threads used by OpenMP. This avoids performance problems caused by over-subscription when using those classes in a docker container for instance. #20477 by Thomas Fan.

• Enhancement HistGradientBoostingClassifier and HistGradientBoostingRegressor are no longer experimental. They are now considered stable and are subject to the same deprecation cycles as all other estimators. #19799 by Nicolas Hug.

• Enhancement Improve the HTML rendering of the ensemble.StackingClassifier and ensemble.StackingRegressor. #19564 by Thomas Fan.

• Enhancement Added Poisson criterion to ensemble.RandomForestRegressor. #19836 by Brian Sun.

• Fix Do not allow to compute out-of-bag (OOB) score in ensemble.RandomForestClassifier and ensemble.ExtraTreesClassifier with multiclass-multioutput target since scikit-learn does not provide any metric supporting this type of target. Additional private refactoring was performed. #19162 by Guillaume Lemaitre.

• Fix Improve numerical precision for weights boosting in ensemble.AdaBoostClassifier and ensemble.AdaBoostRegressor to avoid underflows. #10096 by Fenil Suchak.

• Fix Fixed the range of the argument max_samples to be (0.0, 1.0] in ensemble.RandomForestClassifier, ensemble.RandomForestRegressor, where max_samples=1.0 is interpreted as using all n_samples for bootstrapping. #20159 by @murata-yu.

• Fix Fixed a bug in ensemble.AdaBoostClassifier and ensemble.AdaBoostRegressor where the sample_weight parameter got overwritten during fit. #20534 by Guillaume Lemaitre.

• API Change Removes tol=None option in ensemble.HistGradientBoostingClassifier and ensemble.HistGradientBoostingRegressor. Please use tol=0 for the same behavior. #19296 by Thomas Fan.

sklearn.feature_extraction

• Fix Fixed a bug in feature_extraction.text.HashingVectorizer where some input strings would result in negative indices in the transformed data. #19035 by Liu Yu.

• Fix Fixed a bug in feature_extraction.DictVectorizer by raising an error with unsupported value type. #19520 by Jeff Zhao.

• Fix Fixed a bug in feature_extraction.image.img_to_graph and feature_extraction.image.grid_to_graph where singleton connected components were not handled properly, resulting in a wrong vertex indexing. #18964 by Bertrand Thirion.

• Fix Raise a warning in feature_extraction.text.CountVectorizer with lowercase=True when there are vocabulary entries with uppercase characters to avoid silent misses in the resulting feature vectors. #19401 by Zito Relova

sklearn.feature_selection

• Feature feature_selection.r_regression computes Pearson’s R correlation coefficients between the features and the target. #17169 by Dmytro Lituiev and Julien Jerphanion.

• Enhancement feature_selection.RFE.fit accepts additional estimator parameters that are passed directly to the estimator’s fit method. #20380 by Iván Pulido, Felipe Bidu, Gil Rutter, and Adrin Jalali.

• Fix Fix a bug in isotonic.isotonic_regression where the sample_weight passed by a user were overwritten during fit. #20515 by Carsten Allefeld.

• Fix Change feature_selection.SequentialFeatureSelector to allow for unsupervised modelling so that the fit signature need not do any y validation and allow for y=None. #19568 by Shyam Desai.

• API Change Raises an error in feature_selection.VarianceThreshold when the variance threshold is negative. #20207 by Tomohiro Endo

• API Change Deprecates grid_scores_ in favor of split scores in cv_results_ in feature_selection.RFECV. grid_scores_ will be removed in version 1.2. #20161 by Shuhei Kayawari and @arka204.

sklearn.inspection

• Enhancement Add max_samples parameter in inspection.permutation_importance. It enables to draw a subset of the samples to compute the permutation importance. This is useful to keep the method tractable when evaluating feature importance on large datasets. #20431 by Oliver Pfaffel.

• Enhancement Add kwargs to format ICE and PD lines separately in partial dependence plots inspection.plot_partial_dependence and inspection.PartialDependenceDisplay.plot. #19428 by Mehdi Hamoumi.

• Fix Allow multiple scorers input to inspection.permutation_importance. #19411 by Simona Maggio.

• API Change inspection.PartialDependenceDisplay exposes a class method: from_estimator. inspection.plot_partial_dependence is deprecated in favor of the class method and will be removed in 1.2. #20959 by Thomas Fan.

sklearn.kernel_approximation

• Fix Fix a bug in kernel_approximation.Nystroem where the attribute component_indices_ did not correspond to the subset of sample indices used to generate the approximated kernel. #20554 by Xiangyin Kong.

sklearn.linear_model

• Major Feature Added linear_model.QuantileRegressor which implements linear quantile regression with L1 penalty. #9978 by David Dale and Christian Lorentzen.

• Feature The new linear_model.SGDOneClassSVM provides an SGD implementation of the linear One-Class SVM. Combined with kernel approximation techniques, this implementation approximates the solution of a kernelized One Class SVM while benefiting from a linear complexity in the number of samples. #10027 by Albert Thomas.

• Feature Added sample_weight parameter to linear_model.LassoCV and linear_model.ElasticNetCV. #16449 by Christian Lorentzen.

• Feature Added new solver lbfgs (available with solver="lbfgs") and positive argument to linear_model.Ridge. When positive is set to True, forces the coefficients to be positive (only supported by lbfgs). #20231 by Toshihiro Nakae.

• Efficiency The implementation of linear_model.LogisticRegression has been optimised for dense matrices when using solver='newton-cg' and multi_class!='multinomial'. #19571 by Julien Jerphanion.

• Enhancement fit method preserves dtype for numpy.float32 in linear_model.Lars, linear_model.LassoLars, linear_model.LassoLars, linear_model.LarsCV and linear_model.LassoLarsCV. #20155 by Takeshi Oura.

• Enhancement Validate user-supplied gram matrix passed to linear models via the precompute argument. #19004 by Adam Midvidy.

• Fix linear_model.ElasticNet.fit no longer modifies sample_weight in place. #19055 by Thomas Fan.

• Fix linear_model.Lasso and linear_model.ElasticNet no longer have a dual_gap_ not corresponding to their objective. #19172 by Mathurin Massias

• Fix sample_weight are now fully taken into account in linear models when normalize=True for both feature centering and feature scaling. #19426 by Alexandre Gramfort and Maria Telenczuk.

• Fix Points with residuals equal to residual_threshold are now considered as inliers for linear_model.RANSACRegressor. This allows fitting a model perfectly on some datasets when residual_threshold=0. #19499 by Gregory Strubel.

• Fix Sample weight invariance for linear_model.Ridge was fixed in #19616 by Oliver Grisel and Christian Lorentzen.

• Fix The dictionary params in linear_model.enet_path and linear_model.lasso_path should only contain parameter of the coordinate descent solver. Otherwise, an error will be raised. #19391 by Shao Yang Hong.

• API Change Raise a warning in linear_model.RANSACRegressor that from version 1.2, min_samples need to be set explicitly for models other than linear_model.LinearRegression. #19390 by Shao Yang Hong.

• API Change : The parameter normalize of linear_model.LinearRegression is deprecated and will be removed in 1.2. Motivation for this deprecation: normalize parameter did not take any effect if fit_intercept was set to False and therefore was deemed confusing. The behavior of the deprecated LinearModel(normalize=True) can be reproduced with a Pipeline with LinearModel (where LinearModel is LinearRegression, Ridge, RidgeClassifier, RidgeCV or RidgeClassifierCV) as follows: make_pipeline(StandardScaler(with_mean=False), LinearModel()). The normalize parameter in LinearRegression was deprecated in #17743 by Maria Telenczuk and Alexandre Gramfort. Same for Ridge, RidgeClassifier, RidgeCV, and RidgeClassifierCV, in: #17772 by Maria Telenczuk and Alexandre Gramfort. Same for BayesianRidge, ARDRegression in: #17746 by Maria Telenczuk. Same for Lasso, LassoCV, ElasticNet, ElasticNetCV, MultiTaskLasso, MultiTaskLassoCV, MultiTaskElasticNet, MultiTaskElasticNetCV, in: #17785 by Maria Telenczuk and Alexandre Gramfort.

• API Change The normalize parameter of OrthogonalMatchingPursuit and OrthogonalMatchingPursuitCV will default to False in 1.2 and will be removed in 1.4. #17750 by Maria Telenczuk and Alexandre Gramfort. Same for Lars LarsCV LassoLars LassoLarsCV LassoLarsIC, in #17769 by Maria Telenczuk and Alexandre Gramfort.

• API Change Keyword validation has moved from __init__ and set_params to fit for the following estimators conforming to scikit-learn’s conventions: SGDClassifier, SGDRegressor, SGDOneClassSVM, PassiveAggressiveClassifier, and PassiveAggressiveRegressor. #20683 by Guillaume Lemaitre.

sklearn.manifold

• Enhancement Implement 'auto' heuristic for the learning_rate in manifold.TSNE. It will become default in 1.2. The default initialization will change to pca in 1.2. PCA initialization will be scaled to have standard deviation 1e-4 in 1.2. #19491 by Dmitry Kobak.

• Fix Change numerical precision to prevent underflow issues during affinity matrix computation for manifold.TSNE. #19472 by Dmitry Kobak.

• Fix manifold.Isomap now uses scipy.sparse.csgraph.shortest_path to compute the graph shortest path. It also connects disconnected components of the neighbors graph along some minimum distance pairs, instead of changing every infinite distances to zero. #20531 by Roman Yurchak and Tom Dupre la Tour.

• Fix Decrease the numerical default tolerance in the lobpcg call in manifold.spectral_embedding to prevent numerical instability. #21194 by Andrew Knyazev.

sklearn.metrics

• Feature metrics.mean_pinball_loss exposes the pinball loss for quantile regression. #19415 by Xavier Dupré and Oliver Grisel.

• Feature metrics.d2_tweedie_score calculates the D^2 regression score for Tweedie deviances with power parameter power. This is a generalization of the r2_score and can be interpreted as percentage of Tweedie deviance explained. #17036 by Christian Lorentzen.

• Feature metrics.mean_squared_log_error now supports squared=False. #20326 by Uttam kumar.

• Efficiency Improved speed of metrics.confusion_matrix when labels are integral. #9843 by Jon Crall.

• Enhancement A fix to raise an error in metrics.hinge_loss when pred_decision is 1d whereas it is a multiclass classification or when pred_decision parameter is not consistent with the labels parameter. #19643 by Pierre Attard.

• Fix metrics.ConfusionMatrixDisplay.plot uses the correct max for colormap. #19784 by Thomas Fan.

• Fix Samples with zero sample_weight values do not affect the results from metrics.det_curve, metrics.precision_recall_curve and metrics.roc_curve. #18328 by Albert Villanova del Moral and Alonso Silva Allende.

• Fix avoid overflow in metrics.adjusted_rand_score with large amount of data. #20312 by Divyanshu Deoli.

• API Change metrics.ConfusionMatrixDisplay exposes two class methods from_estimator and from_predictions allowing to create a confusion matrix plot using an estimator or the predictions. metrics.plot_confusion_matrix is deprecated in favor of these two class methods and will be removed in 1.2. #18543 by Guillaume Lemaitre.

• API Change metrics.PrecisionRecallDisplay exposes two class methods from_estimator and from_predictions allowing to create a precision-recall curve using an estimator or the predictions. metrics.plot_precision_recall_curve is deprecated in favor of these two class methods and will be removed in 1.2. #20552 by Guillaume Lemaitre.

• API Change metrics.DetCurveDisplay exposes two class methods from_estimator and from_predictions allowing to create a confusion matrix plot using an estimator or the predictions. metrics.plot_det_curve is deprecated in favor of these two class methods and will be removed in 1.2. #19278 by Guillaume Lemaitre.

sklearn.mixture

• Fix Ensure that the best parameters are set appropriately in the case of divergency for mixture.GaussianMixture and mixture.BayesianGaussianMixture. #20030 by Tingshan Liu and Benjamin Pedigo.

sklearn.model_selection

• Feature added model_selection.StratifiedGroupKFold, that combines model_selection.StratifiedKFold and model_selection.GroupKFold, providing an ability to split data preserving the distribution of classes in each split while keeping each group within a single split. #18649 by Leandro Hermida and Rodion Martynov.

• Enhancement warn only once in the main process for per-split fit failures in cross-validation. #20619 by Loïc Estève

• Enhancement The model_selection.BaseShuffleSplit base class is now public. #20056 by @pabloduque0.

• Fix Avoid premature overflow in model_selection.train_test_split. #20904 by Tomasz Jakubek.

sklearn.naive_bayes

• Fix The fit and partial_fit methods of the discrete naive Bayes classifiers (naive_bayes.BernoulliNB, naive_bayes.CategoricalNB, naive_bayes.ComplementNB, and naive_bayes.MultinomialNB) now correctly handle the degenerate case of a single class in the training set. #18925 by David Poznik.

• API Change The attribute sigma_ is now deprecated in naive_bayes.GaussianNB and will be removed in 1.2. Use var_ instead. #18842 by Hong Shao Yang.

sklearn.neighbors

• Enhancement The creation of neighbors.KDTree and neighbors.BallTree has been improved for their worst-cases time complexity from \(\mathcal{O}(n^2)\) to \(\mathcal{O}(n)\). #19473 by jiefangxuanyan and Julien Jerphanion.

• Fix neighbors.DistanceMetric subclasses now support readonly memory-mapped datasets. #19883 by Julien Jerphanion.

• Fix neighbors.NearestNeighbors, neighbors.KNeighborsClassifier, neighbors.RadiusNeighborsClassifier, neighbors.KNeighborsRegressor and neighbors.RadiusNeighborsRegressor do not validate weights in __init__ and validate weights in fit instead. #20072 by Juan Carlos Alfaro Jiménez.

• API Change The parameter kwargs of neighbors.RadiusNeighborsClassifier is deprecated and will be removed in 1.2. #20842 by Juan Martín Loyola.

sklearn.neural_network

• Fix neural_network.MLPClassifier and neural_network.MLPRegressor now correctly support continued training when loading from a pickled file. #19631 by Thomas Fan.

sklearn.pipeline

• API Change The predict_proba and predict_log_proba methods of the pipeline.Pipeline now support passing prediction kwargs to the final estimator. #19790 by Christopher Flynn.

sklearn.preprocessing

• Feature The new preprocessing.SplineTransformer is a feature preprocessing tool for the generation of B-splines, parametrized by the polynomial degree of the splines, number of knots n_knots and knot positioning strategy knots. #18368 by Christian Lorentzen. preprocessing.SplineTransformer also supports periodic splines via the extrapolation argument. #19483 by Malte Londschien. preprocessing.SplineTransformer supports sample weights for knot position strategy "quantile". #20526 by Malte Londschien.

• Feature preprocessing.OrdinalEncoder supports passing through missing values by default. #19069 by Thomas Fan.

• Feature preprocessing.OneHotEncoder now supports handle_unknown='ignore' and dropping categories. #19041 by Thomas Fan.

• Feature preprocessing.PolynomialFeatures now supports passing a tuple to degree, i.e. degree=(min_degree, max_degree). #20250 by Christian Lorentzen.

• Efficiency preprocessing.StandardScaler is faster and more memory efficient. #20652 by Thomas Fan.

• Efficiency Changed algorithm argument for cluster.KMeans in preprocessing.KBinsDiscretizer from auto to full. #19934 by Hleb Levitski.

• Efficiency The implementation of fit for preprocessing.PolynomialFeatures transformer is now faster. This is especially noticeable on large sparse input. #19734 by Fred Robinson.

• Fix The preprocessing.StandardScaler.inverse_transform method now raises error when the input data is 1D. #19752 by Zhehao Liu.

• Fix preprocessing.scale, preprocessing.StandardScaler and similar scalers detect near-constant features to avoid scaling them to very large values. This problem happens in particular when using a scaler on sparse data with a constant column with sample weights, in which case centering is typically disabled. #19527 by Oliver Grisel and Maria Telenczuk and #19788 by Jérémie du Boisberranger.

• Fix preprocessing.StandardScaler.inverse_transform now correctly handles integer dtypes. #19356 by @makoeppel.

• Fix preprocessing.OrdinalEncoder.inverse_transform is not supporting sparse matrix and raises the appropriate error message. #19879 by Guillaume Lemaitre.

• Fix The fit method of preprocessing.OrdinalEncoder will not raise error when handle_unknown='ignore' and unknown categories are given to fit. #19906 by Zhehao Liu.

• Fix Fix a regression in preprocessing.OrdinalEncoder where large Python numeric would raise an error due to overflow when casted to C type (np.float64 or np.int64). #20727 by Guillaume Lemaitre.

• Fix preprocessing.FunctionTransformer does not set n_features_in_ based on the input to inverse_transform. #20961 by Thomas Fan.

• API Change The n_input_features_ attribute of preprocessing.PolynomialFeatures is deprecated in favor of n_features_in_ and will be removed in 1.2. #20240 by Jérémie du Boisberranger.

sklearn.svm

• API Change The parameter **params of svm.OneClassSVM.fit is deprecated and will be removed in 1.2. #20843 by Juan Martín Loyola.

sklearn.tree

• Enhancement Add fontname argument in tree.export_graphviz for non-English characters. #18959 by Zero and wstates.

• Fix Improves compatibility of tree.plot_tree with high DPI screens. #20023 by Thomas Fan.

• Fix Fixed a bug in tree.DecisionTreeClassifier, tree.DecisionTreeRegressor where a node could be split whereas it should not have been due to incorrect handling of rounding errors. #19336 by Jérémie du Boisberranger.

• API Change The n_features_ attribute of tree.DecisionTreeClassifier, tree.DecisionTreeRegressor, tree.ExtraTreeClassifier and tree.ExtraTreeRegressor is deprecated in favor of n_features_in_ and will be removed in 1.2. #20272 by Jérémie du Boisberranger.

sklearn.utils

• Enhancement Deprecated the default value of the random_state=0 in randomized_svd. Starting in 1.2, the default value of random_state will be set to None. #19459 by Cindy Bezuidenhout and Clifford Akai-Nettey.

• Enhancement Added helper decorator utils.metaestimators.available_if to provide flexibility in metaestimators making methods available or unavailable on the basis of state, in a more readable way. #19948 by Joel Nothman.

• Enhancement utils.validation.check_is_fitted now uses __sklearn_is_fitted__ if available, instead of checking for attributes ending with an underscore. This also makes pipeline.Pipeline and preprocessing.FunctionTransformer pass check_is_fitted(estimator). #20657 by Adrin Jalali.

• Fix Fixed a bug in utils.sparsefuncs.mean_variance_axis where the precision of the computed variance was very poor when the real variance is exactly zero. #19766 by Jérémie du Boisberranger.

• Fix The docstrings of properties that are decorated with utils.deprecated are now properly wrapped. #20385 by Thomas Fan.

• Fix utils.stats._weighted_percentile now correctly ignores zero-weighted observations smaller than the smallest observation with positive weight for percentile=0. Affected classes are dummy.DummyRegressor for quantile=0 and ensemble.HuberLossFunction and ensemble.HuberLossFunction for alpha=0. #20528 by Malte Londschien.

• Fix utils._safe_indexing explicitly takes a dataframe copy when integer indices are provided avoiding to raise a warning from Pandas. This warning was previously raised in resampling utilities and functions using those utilities (e.g. model_selection.train_test_split, model_selection.cross_validate, model_selection.cross_val_score, model_selection.cross_val_predict). #20673 by Joris Van den Bossche.

• Fix Fix a regression in utils.is_scalar_nan where large Python numbers would raise an error due to overflow in C types (np.float64 or np.int64). #20727 by Guillaume Lemaitre.

• Fix Support for np.matrix is deprecated in check_array in 1.0 and will raise a TypeError in 1.2. #20165 by Thomas Fan.

• API Change utils._testing.assert_warns and utils._testing.assert_warns_message are deprecated in 1.0 and will be removed in 1.2. Used pytest.warns context manager instead. Note that these functions were not documented and part from the public API. #20521 by Olivier Grisel.

• API Change Fixed several bugs in utils.graph.graph_shortest_path, which is now deprecated. Use scipy.sparse.csgraph.shortest_path instead. #20531 by Tom Dupre la Tour.

Code and documentation contributors

Thanks to everyone who has contributed to the maintenance and improvement of the project since version 0.24, including:

Abdulelah S. Al Mesfer, Abhinav Gupta, Adam J. Stewart, Adam Li, Adam Midvidy, Adrian Garcia Badaracco, Adrian Sadłocha, Adrin Jalali, Agamemnon Krasoulis, Alberto Rubiales, Albert Thomas, Albert Villanova del Moral, Alek Lefebvre, Alessia Marcolini, Alexandr Fonari, Alihan Zihna, Aline Ribeiro de Almeida, Amanda, Amanda Dsouza, Amol Deshmukh, Ana Pessoa, Anavelyz, Andreas Mueller, Andrew Delong, Ashish, Ashvith Shetty, Atsushi Nukariya, Aurélien Geron, Avi Gupta, Ayush Singh, baam, BaptBillard, Benjamin Pedigo, Bertrand Thirion, Bharat Raghunathan, bmalezieux, Brian Rice, Brian Sun, Bruno Charron, Bryan Chen, bumblebee, caherrera-meli, Carsten Allefeld, CeeThinwa, Chiara Marmo, chrissobel, Christian Lorentzen, Christopher Yeh, Chuliang Xiao, Clément Fauchereau, cliffordEmmanuel, Conner Shen, Connor Tann, David Dale, David Katz, David Poznik, Dimitri Papadopoulos Orfanos, Divyanshu Deoli, dmallia17, Dmitry Kobak, DS_anas, Eduardo Jardim, EdwinWenink, EL-ATEIF Sara, Eleni Markou, EricEllwanger, Eric Fiegel, Erich Schubert, Ezri-Mudde, Fatos Morina, Felipe Rodrigues, Felix Hafner, Fenil Suchak, flyingdutchman23, Flynn, Fortune Uwha, Francois Berenger, Frankie Robertson, Frans Larsson, Frederick Robinson, frellwan, Gabriel S Vicente, Gael Varoquaux, genvalen, Geoffrey Thomas, geroldcsendes, Hleb Levitski, Glen, Glòria Macià Muñoz, gregorystrubel, groceryheist, Guillaume Lemaitre, guiweber, Haidar Almubarak, Hans Moritz Günther, Haoyin Xu, Harris Mirza, Harry Wei, Harutaka Kawamura, Hassan Alsawadi, Helder Geovane Gomes de Lima, Hugo DEFOIS, Igor Ilic, Ikko Ashimine, Isaack Mungui, Ishaan Bhat, Ishan Mishra, Iván Pulido, iwhalvic, J Alexander, Jack Liu, James Alan Preiss, James Budarz, James Lamb, Jannik, Jeff Zhao, Jennifer Maldonado, Jérémie du Boisberranger, Jesse Lima, Jianzhu Guo, jnboehm, Joel Nothman, JohanWork, John Paton, Jonathan Schneider, Jon Crall, Jon Haitz Legarreta Gorroño, Joris Van den Bossche, José Manuel Nápoles Duarte, Juan Carlos Alfaro Jiménez, Juan Martin Loyola, Julien Jerphanion, Julio Batista Silva, julyrashchenko, JVM, Kadatatlu Kishore, Karen Palacio, Kei Ishikawa, kmatt10, kobaski, Kot271828, Kunj, KurumeYuta, kxytim, lacrosse91, LalliAcqua, Laveen Bagai, Leonardo Rocco, Leonardo Uieda, Leopoldo Corona, Loic Esteve, LSturtew, Luca Bittarello, Luccas Quadros, Lucy Jiménez, Lucy Liu, ly648499246, Mabu Manaileng, Manimaran, makoeppel, Marco Gorelli, Maren Westermann, Mariangela, Maria Telenczuk, marielaraj, Martin Hirzel, Mateo Noreña, Mathieu Blondel, Mathis Batoul, mathurinm, Matthew Calcote, Maxime Prieur, Maxwell, Mehdi Hamoumi, Mehmet Ali Özer, Miao Cai, Michal Karbownik, michalkrawczyk, Mitzi, mlondschien, Mohamed Haseeb, Mohamed Khoualed, Muhammad Jarir Kanji, murata-yu, Nadim Kawwa, Nanshan Li, naozin555, Nate Parsons, Neal Fultz, Nic Annau, Nicolas Hug, Nicolas Miller, Nico Stefani, Nigel Bosch, Nikita Titov, Nodar Okroshiashvili, Norbert Preining, novaya, Ogbonna Chibuike Stephen, OGordon100, Oliver Pfaffel, Olivier Grisel, Oras Phongpanangam, Pablo Duque, Pablo Ibieta-Jimenez, Patric Lacouth, Paulo S. Costa, Paweł Olszewski, Peter Dye, PierreAttard, Pierre-Yves Le Borgne, PranayAnchuri, Prince Canuma, putschblos, qdeffense, RamyaNP, ranjanikrishnan, Ray Bell, Rene Jean Corneille, Reshama Shaikh, ricardojnf, RichardScottOZ, Rodion Martynov, Rohan Paul, Roman Lutz, Roman Yurchak, Samuel Brice, Sandy Khosasi, Sean Benhur J, Sebastian Flores, Sebastian Pölsterl, Shao Yang Hong, shinehide, shinnar, shivamgargsya, Shooter23, Shuhei Kayawari, Shyam Desai, simonamaggio, Sina Tootoonian, solosilence, Steven Kolawole, Steve Stagg, Surya Prakash, swpease, Sylvain Marié, Takeshi Oura, Terence Honles, TFiFiE, Thomas A Caswell, Thomas J. Fan, Tim Gates, TimotheeMathieu, Timothy Wolodzko, Tim Vink, t-jakubek, t-kusanagi, tliu68, Tobias Uhmann, tom1092, Tomás Moreyra, Tomás Ronald Hughes, Tom Dupré la Tour, Tommaso Di Noto, Tomohiro Endo, TONY GEORGE, Toshihiro NAKAE, tsuga, Uttam kumar, vadim-ushtanit, Vangelis Gkiastas, Venkatachalam N, Vilém Zouhar, Vinicius Rios Fuck, Vlasovets, waijean, Whidou, xavier dupré, xiaoyuchai, Yasmeen Alsaedy, yoch, Yosuke KOBAYASHI, Yu Feng, YusukeNagasaka, yzhenman, Zero, ZeyuSun, ZhaoweiWang, Zito, Zito Relova