7.1. Toy datasets#
scikit-learn comes with a few small standard datasets that do not require to download any file from some external website.
They can be loaded using the following functions:
|
Load and return the iris dataset (classification). |
|
Load and return the diabetes dataset (regression). |
|
Load and return the digits dataset (classification). |
|
Load and return the physical exercise Linnerud dataset. |
|
Load and return the wine dataset (classification). |
|
Load and return the breast cancer wisconsin dataset (classification). |
These datasets are useful to quickly illustrate the behavior of the various algorithms implemented in scikit-learn. They are however often too small to be representative of real world machine learning tasks.
7.1.1. Iris plants dataset#
Data Set Characteristics:
- Number of Instances:
150 (50 in each of three classes)
- Number of Attributes:
4 numeric, predictive attributes and the class
- Attribute Information:
sepal length in cm
sepal width in cm
petal length in cm
petal width in cm
- class:
Iris-Setosa
Iris-Versicolour
Iris-Virginica
- Summary Statistics:
sepal length: |
4.3 |
7.9 |
5.84 |
0.83 |
0.7826 |
sepal width: |
2.0 |
4.4 |
3.05 |
0.43 |
-0.4194 |
petal length: |
1.0 |
6.9 |
3.76 |
1.76 |
0.9490 (high!) |
petal width: |
0.1 |
2.5 |
1.20 |
0.76 |
0.9565 (high!) |
- Missing Attribute Values:
None
- Class Distribution:
33.3% for each of 3 classes.
- Creator:
R.A. Fisher
- Donor:
Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
- Date:
July, 1988
The famous Iris database, first used by Sir R.A. Fisher. The dataset is taken from Fisher’s paper. Note that it’s the same as in R, but not as in the UCI Machine Learning Repository, which has two wrong data points.
This is perhaps the best known database to be found in the pattern recognition literature. Fisher’s paper is a classic in the field and is referenced frequently to this day. (See Duda & Hart, for example.) The data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant. One class is linearly separable from the other 2; the latter are NOT linearly separable from each other.
References#
Fisher, R.A. “The use of multiple measurements in taxonomic problems” Annual Eugenics, 7, Part II, 179-188 (1936); also in “Contributions to Mathematical Statistics” (John Wiley, NY, 1950).
Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis. (Q327.D83) John Wiley & Sons. ISBN 0-471-22361-1. See page 218.
Dasarathy, B.V. (1980) “Nosing Around the Neighborhood: A New System Structure and Classification Rule for Recognition in Partially Exposed Environments”. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, No. 1, 67-71.
Gates, G.W. (1972) “The Reduced Nearest Neighbor Rule”. IEEE Transactions on Information Theory, May 1972, 431-433.
See also: 1988 MLC Proceedings, 54-64. Cheeseman et al”s AUTOCLASS II conceptual clustering system finds 3 classes in the data.
Many, many more …
7.1.2. Diabetes dataset#
Ten baseline variables, age, sex, body mass index, average blood pressure, and six blood serum measurements were obtained for each of n = 442 diabetes patients, as well as the response of interest, a quantitative measure of disease progression one year after baseline.
Data Set Characteristics:
- Number of Instances:
442
- Number of Attributes:
First 10 columns are numeric predictive values
- Target:
Column 11 is a quantitative measure of disease progression one year after baseline
- Attribute Information:
age age in years
sex
bmi body mass index
bp average blood pressure
s1 tc, total serum cholesterol
s2 ldl, low-density lipoproteins
s3 hdl, high-density lipoproteins
s4 tch, total cholesterol / HDL
s5 ltg, possibly log of serum triglycerides level
s6 glu, blood sugar level
Note: Each of these 10 feature variables have been mean centered and scaled by the standard deviation times the square root of n_samples
(i.e. the sum of squares of each column totals 1).
Source URL: https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html
For more information see: Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani (2004) “Least Angle Regression,” Annals of Statistics (with discussion), 407-499. (https://web.stanford.edu/~hastie/Papers/LARS/LeastAngle_2002.pdf)
7.1.3. Optical recognition of handwritten digits dataset#
Data Set Characteristics:
- Number of Instances:
1797
- Number of Attributes:
64
- Attribute Information:
8x8 image of integer pixels in the range 0..16.
- Missing Attribute Values:
None
- Creator:
Alpaydin (alpaydin ‘@’ boun.edu.tr)
- Date:
July; 1998
This is a copy of the test set of the UCI ML hand-written digits datasets https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits
The data set contains images of hand-written digits: 10 classes where each class refers to a digit.
Preprocessing programs made available by NIST were used to extract normalized bitmaps of handwritten digits from a preprinted form. From a total of 43 people, 30 contributed to the training set and different 13 to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of 4x4 and the number of on pixels are counted in each block. This generates an input matrix of 8x8 where each element is an integer in the range 0..16. This reduces dimensionality and gives invariance to small distortions.
For info on NIST preprocessing routines, see M. D. Garris, J. L. Blue, G. T. Candela, D. L. Dimmick, J. Geist, P. J. Grother, S. A. Janet, and C. L. Wilson, NIST Form-Based Handprint Recognition System, NISTIR 5469, 1994.
References#
C. Kaynak (1995) Methods of Combining Multiple Classifiers and Their Applications to Handwritten Digit Recognition, MSc Thesis, Institute of Graduate Studies in Science and Engineering, Bogazici University.
Alpaydin, C. Kaynak (1998) Cascading Classifiers, Kybernetika.
Ken Tang and Ponnuthurai N. Suganthan and Xi Yao and A. Kai Qin. Linear dimensionalityreduction using relevance weighted LDA. School of Electrical and Electronic Engineering Nanyang Technological University. 2005.
Claudio Gentile. A New Approximate Maximal Margin Classification Algorithm. NIPS. 2000.
7.1.4. Linnerrud dataset#
Data Set Characteristics:
- Number of Instances:
20
- Number of Attributes:
3
- Missing Attribute Values:
None
The Linnerud dataset is a multi-output regression dataset. It consists of three exercise (data) and three physiological (target) variables collected from twenty middle-aged men in a fitness club:
- physiological - CSV containing 20 observations on 3 physiological variables:
Weight, Waist and Pulse.
- exercise - CSV containing 20 observations on 3 exercise variables:
Chins, Situps and Jumps.
References#
Tenenhaus, M. (1998). La regression PLS: theorie et pratique. Paris: Editions Technic.
7.1.5. Wine recognition dataset#
Data Set Characteristics:
- Number of Instances:
178
- Number of Attributes:
13 numeric, predictive attributes and the class
- Attribute Information:
Alcohol
Malic acid
Ash
Alcalinity of ash
Magnesium
Total phenols
Flavanoids
Nonflavanoid phenols
Proanthocyanins
Color intensity
Hue
OD280/OD315 of diluted wines
Proline
- class:
class_0
class_1
class_2
- Summary Statistics:
Alcohol: |
11.0 |
14.8 |
13.0 |
0.8 |
Malic Acid: |
0.74 |
5.80 |
2.34 |
1.12 |
Ash: |
1.36 |
3.23 |
2.36 |
0.27 |
Alcalinity of Ash: |
10.6 |
30.0 |
19.5 |
3.3 |
Magnesium: |
70.0 |
162.0 |
99.7 |
14.3 |
Total Phenols: |
0.98 |
3.88 |
2.29 |
0.63 |
Flavanoids: |
0.34 |
5.08 |
2.03 |
1.00 |
Nonflavanoid Phenols: |
0.13 |
0.66 |
0.36 |
0.12 |
Proanthocyanins: |
0.41 |
3.58 |
1.59 |
0.57 |
Colour Intensity: |
1.3 |
13.0 |
5.1 |
2.3 |
Hue: |
0.48 |
1.71 |
0.96 |
0.23 |
OD280/OD315 of diluted wines: |
1.27 |
4.00 |
2.61 |
0.71 |
Proline: |
278 |
1680 |
746 |
315 |
- Missing Attribute Values:
None
- Class Distribution:
class_0 (59), class_1 (71), class_2 (48)
- Creator:
R.A. Fisher
- Donor:
Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
- Date:
July, 1988
This is a copy of UCI ML Wine recognition datasets. https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data
The data is the results of a chemical analysis of wines grown in the same region in Italy by three different cultivators. There are thirteen different measurements taken for different constituents found in the three types of wine.
Original Owners:
Forina, M. et al, PARVUS - An Extendible Package for Data Exploration, Classification and Correlation. Institute of Pharmaceutical and Food Analysis and Technologies, Via Brigata Salerno, 16147 Genoa, Italy.
Citation:
Lichman, M. (2013). UCI Machine Learning Repository [https://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
References#
(1) S. Aeberhard, D. Coomans and O. de Vel, Comparison of Classifiers in High Dimensional Settings, Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of Mathematics and Statistics, James Cook University of North Queensland. (Also submitted to Technometrics).
The data was used with many others for comparing various classifiers. The classes are separable, though only RDA has achieved 100% correct classification. (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) (All results using the leave-one-out technique)
(2) S. Aeberhard, D. Coomans and O. de Vel, “THE CLASSIFICATION PERFORMANCE OF RDA” Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of Mathematics and Statistics, James Cook University of North Queensland. (Also submitted to Journal of Chemometrics).
7.1.6. Breast cancer wisconsin (diagnostic) dataset#
Data Set Characteristics:
- Number of Instances:
569
- Number of Attributes:
30 numeric, predictive attributes and the class
- Attribute Information:
radius (mean of distances from center to points on the perimeter)
texture (standard deviation of gray-scale values)
perimeter
area
smoothness (local variation in radius lengths)
compactness (perimeter^2 / area - 1.0)
concavity (severity of concave portions of the contour)
concave points (number of concave portions of the contour)
symmetry
fractal dimension (“coastline approximation” - 1)
The mean, standard error, and “worst” or largest (mean of the three worst/largest values) of these features were computed for each image, resulting in 30 features. For instance, field 0 is Mean Radius, field 10 is Radius SE, field 20 is Worst Radius.
- class:
WDBC-Malignant
WDBC-Benign
- Summary Statistics:
radius (mean): |
6.981 |
28.11 |
texture (mean): |
9.71 |
39.28 |
perimeter (mean): |
43.79 |
188.5 |
area (mean): |
143.5 |
2501.0 |
smoothness (mean): |
0.053 |
0.163 |
compactness (mean): |
0.019 |
0.345 |
concavity (mean): |
0.0 |
0.427 |
concave points (mean): |
0.0 |
0.201 |
symmetry (mean): |
0.106 |
0.304 |
fractal dimension (mean): |
0.05 |
0.097 |
radius (standard error): |
0.112 |
2.873 |
texture (standard error): |
0.36 |
4.885 |
perimeter (standard error): |
0.757 |
21.98 |
area (standard error): |
6.802 |
542.2 |
smoothness (standard error): |
0.002 |
0.031 |
compactness (standard error): |
0.002 |
0.135 |
concavity (standard error): |
0.0 |
0.396 |
concave points (standard error): |
0.0 |
0.053 |
symmetry (standard error): |
0.008 |
0.079 |
fractal dimension (standard error): |
0.001 |
0.03 |
radius (worst): |
7.93 |
36.04 |
texture (worst): |
12.02 |
49.54 |
perimeter (worst): |
50.41 |
251.2 |
area (worst): |
185.2 |
4254.0 |
smoothness (worst): |
0.071 |
0.223 |
compactness (worst): |
0.027 |
1.058 |
concavity (worst): |
0.0 |
1.252 |
concave points (worst): |
0.0 |
0.291 |
symmetry (worst): |
0.156 |
0.664 |
fractal dimension (worst): |
0.055 |
0.208 |
- Missing Attribute Values:
None
- Class Distribution:
212 - Malignant, 357 - Benign
- Creator:
Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian
- Donor:
Nick Street
- Date:
November, 1995
This is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets. https://goo.gl/U2Uwz2
Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. They describe characteristics of the cell nuclei present in the image.
Separating plane described above was obtained using Multisurface Method-Tree (MSM-T) [K. P. Bennett, “Decision Tree Construction Via Linear Programming.” Proceedings of the 4th Midwest Artificial Intelligence and Cognitive Science Society, pp. 97-101, 1992], a classification method which uses linear programming to construct a decision tree. Relevant features were selected using an exhaustive search in the space of 1-4 features and 1-3 separating planes.
The actual linear program used to obtain the separating plane in the 3-dimensional space is that described in: [K. P. Bennett and O. L. Mangasarian: “Robust Linear Programming Discrimination of Two Linearly Inseparable Sets”, Optimization Methods and Software 1, 1992, 23-34].
This database is also available through the UW CS ftp server:
ftp ftp.cs.wisc.edu cd math-prog/cpo-dataset/machine-learn/WDBC/
References#
W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on Electronic Imaging: Science and Technology, volume 1905, pages 861-870, San Jose, CA, 1993.
O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and prognosis via linear programming. Operations Research, 43(4), pages 570-577, July-August 1995.
W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994) 163-171.