work/smartcore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge potential next release v0.4 (#187) Breaking Changes

Browse Source 
* First draft of the new n-dimensional arrays + NB use case
* Improves default implementation of multiple Array methods
* Refactors tree methods
* Adds matrix decomposition routines
* Adds matrix decomposition methods to ndarray and nalgebra bindings
* Refactoring + linear regression now uses array2
* Ridge & Linear regression
* LBFGS optimizer & logistic regression
* LBFGS optimizer & logistic regression
* Changes linear methods, metrics and model selection methods to new n-dimensional arrays
* Switches KNN and clustering algorithms to new n-d array layer
* Refactors distance metrics
* Optimizes knn and clustering methods
* Refactors metrics module
* Switches decomposition methods to n-dimensional arrays
* Linalg refactoring - cleanup rng merge (#172)
* Remove legacy DenseMatrix and BaseMatrix implementation. Port the new Number, FloatNumber and Array implementation into module structure.
* Exclude AUC metrics. Needs reimplementation
* Improve developers walkthrough

New traits system in place at `src/numbers` and `src/linalg`
Co-authored-by: Lorenzo <tunedconsulting@gmail.com>

* Provide SupervisedEstimator with a constructor to avoid explicit dynamical box allocation in 'cross_validate' and 'cross_validate_predict' as required by the use of 'dyn' as per Rust 2021
* Implement getters to use as_ref() in src/neighbors
* Implement getters to use as_ref() in src/naive_bayes
* Implement getters to use as_ref() in src/linear
* Add Clone to src/naive_bayes
* Change signature for cross_validate and other model_selection functions to abide to use of dyn in Rust 2021
* Implement ndarray-bindings. Remove FloatNumber from implementations
* Drop nalgebra-bindings support (as decided in conf-call to go for ndarray)
* Remove benches. Benches will have their own repo at smartcore-benches
* Implement SVC
* Implement SVC serialization. Move search parameters in dedicated module
* Implement SVR. Definitely too slow
* Fix compilation issues for wasm (#202)

Co-authored-by: Luis Moreno <morenol@users.noreply.github.com>
* Fix tests (#203)

* Port linalg/traits/stats.rs
* Improve methods naming
* Improve Display for DenseMatrix

Co-authored-by: Montana Low <montanalow@users.noreply.github.com>
Co-authored-by: VolodymyrOrlov <volodymyr.orlov@gmail.com>
This commit is contained in:
Lorenzo
2022-10-31 10:44:57 +00:00
committed by GitHub
parent bb71656137
commit 52eb6ce023
110 changed files with 10327 additions and 9107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/preprocessing/numerical.rs
+70 -58
View File
@@ -4,7 +4,7 @@
//! ### Usage Example
//! ```
//! use smartcore::api::{Transformer, UnsupervisedEstimator};
//! use smartcore::linalg::naive::dense_matrix::DenseMatrix;
//! use smartcore::linalg::basic::matrix::DenseMatrix;
//! use smartcore::preprocessing::numerical;
//! let data = DenseMatrix::from_2d_vec(&vec![
//! vec![0.0, 0.0],
@@ -27,10 +27,13 @@
//! ])
//! );
//! ```
use std::marker::PhantomData;
use crate::api::{Transformer, UnsupervisedEstimator};
use crate::error::{Failed, FailedError};
use crate::linalg::Matrix;
use crate::math::num::RealNumber;
use crate::linalg::basic::arrays::Array2;
use crate::numbers::basenum::Number;
use crate::numbers::realnum::RealNumber;
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};
@@ -59,29 +62,46 @@ impl Default for StandardScalerParameters {
/// scaling sensitive models like neural network or nearest
/// neighbors based models.
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct StandardScaler<T: RealNumber> {
means: Vec<T>,
stds: Vec<T>,
#[derive(Clone, Debug, Default, PartialEq)]
pub struct StandardScaler<T: Number + RealNumber> {
means: Vec<f64>,
stds: Vec<f64>,
parameters: StandardScalerParameters,
_phantom: PhantomData<T>,
}
impl<T: RealNumber> StandardScaler<T> {
#[allow(dead_code)]
impl<T: Number + RealNumber> StandardScaler<T> {
fn new(parameters: StandardScalerParameters) -> Self
where
T: Number + RealNumber,
{
Self {
means: vec![],
stds: vec![],
parameters: StandardScalerParameters {
with_mean: parameters.with_mean,
with_std: parameters.with_std,
},
_phantom: PhantomData,
}
}
/// When the mean should be adjusted, the column mean
/// should be kept. Otherwise, replace it by zero.
fn adjust_column_mean(&self, mean: T) -> T {
fn adjust_column_mean(&self, mean: f64) -> f64 {
if self.parameters.with_mean {
mean
} else {
T::zero()
0f64
}
}
/// When the standard-deviation should be adjusted, the column
/// standard-deviation should be kept. Otherwise, replace it by one.
fn adjust_column_std(&self, std: T) -> T {
fn adjust_column_std(&self, std: f64) -> f64 {
if self.parameters.with_std {
ensure_std_valid(std)
} else {
T::one()
1f64
}
}
}
@@ -90,19 +110,24 @@ impl<T: RealNumber> StandardScaler<T> {
/// negative or zero, it should replaced by the smallest
/// positive value the type can have. That way we can savely
/// divide the columns with the resulting scalar.
fn ensure_std_valid<T: RealNumber>(value: T) -> T {
fn ensure_std_valid<T: Number + RealNumber>(value: T) -> T {
value.max(T::min_positive_value())
}
/// During `fit` the `StandardScaler` computes the column means and standard deviation.
impl<T: RealNumber, M: Matrix<T>> UnsupervisedEstimator<M, StandardScalerParameters>
impl<T: Number + RealNumber, M: Array2<T>> UnsupervisedEstimator<M, StandardScalerParameters>
for StandardScaler<T>
{
fn fit(x: &M, parameters: StandardScalerParameters) -> Result<Self, Failed> {
fn fit(x: &M, parameters: StandardScalerParameters) -> Result<Self, Failed>
where
T: Number + RealNumber,
M: Array2<T>,
{
Ok(Self {
means: x.column_mean(),
stds: x.std(0),
stds: x.std_dev(0),
parameters,
_phantom: Default::default(),
})
}
}
@@ -110,7 +135,7 @@ impl<T: RealNumber, M: Matrix<T>> UnsupervisedEstimator<M, StandardScalerParamet
/// During `transform` the `StandardScaler` applies the summary statistics
/// computed during `fit` to set the mean of each column to zero and the
/// standard deviation to one.
impl<T: RealNumber, M: Matrix<T>> Transformer<M> for StandardScaler<T> {
impl<T: Number + RealNumber, M: Array2<T>> Transformer<M> for StandardScaler<T> {
fn transform(&self, x: &M) -> Result<M, Failed> {
let (_, n_cols) = x.shape();
if n_cols != self.means.len() {
@@ -131,8 +156,8 @@ impl<T: RealNumber, M: Matrix<T>> Transformer<M> for StandardScaler<T> {
.enumerate()
.map(|(column_index, (column_mean, column_std))| {
x.take_column(column_index)
.sub_scalar(self.adjust_column_mean(*column_mean))
.div_scalar(self.adjust_column_std(*column_std))
.sub_scalar(T::from(self.adjust_column_mean(*column_mean)).unwrap())
.div_scalar(T::from(self.adjust_column_std(*column_std)).unwrap())
})
.collect(),
)
@@ -144,8 +169,8 @@ impl<T: RealNumber, M: Matrix<T>> Transformer<M> for StandardScaler<T> {
/// a matrix by stacking the columns horizontally.
fn build_matrix_from_columns<T, M>(columns: Vec<M>) -> Option<M>
where
T: RealNumber,
M: Matrix<T>,
T: Number + RealNumber,
M: Array2<T>,
{
if let Some(output_matrix) = columns.first().cloned() {
return Some(
@@ -166,7 +191,7 @@ mod tests {
mod helper_functionality {
use super::super::{build_matrix_from_columns, ensure_std_valid};
use crate::linalg::naive::dense_matrix::DenseMatrix;
use crate::linalg::basic::matrix::DenseMatrix;
#[test]
fn combine_three_columns() {
@@ -197,20 +222,16 @@ mod tests {
mod standard_scaler {
use super::super::{StandardScaler, StandardScalerParameters};
use crate::api::{Transformer, UnsupervisedEstimator};
use crate::linalg::naive::dense_matrix::DenseMatrix;
use crate::linalg::BaseMatrix;
use crate::linalg::basic::arrays::Array2;
use crate::linalg::basic::matrix::DenseMatrix;
#[test]
fn dont_adjust_mean_if_used() {
assert_eq!(
(StandardScaler {
means: vec![],
stds: vec![],
parameters: StandardScalerParameters {
with_mean: true,
with_std: true
}
})
(StandardScaler::<f64>::new(StandardScalerParameters {
with_mean: true,
with_std: true
}))
.adjust_column_mean(1.0),
1.0
)
@@ -218,14 +239,10 @@ mod tests {
#[test]
fn replace_mean_with_zero_if_not_used() {
assert_eq!(
(StandardScaler {
means: vec![],
stds: vec![],
parameters: StandardScalerParameters {
with_mean: false,
with_std: true
}
})
(StandardScaler::<f64>::new(StandardScalerParameters {
with_mean: false,
with_std: true
}))
.adjust_column_mean(1.0),
0.0
)
@@ -233,14 +250,10 @@ mod tests {
#[test]
fn dont_adjust_std_if_used() {
assert_eq!(
(StandardScaler {
means: vec![],
stds: vec![],
parameters: StandardScalerParameters {
with_mean: true,
with_std: true
}
})
(StandardScaler::<f64>::new(StandardScalerParameters {
with_mean: true,
with_std: true
}))
.adjust_column_std(10.0),
10.0
)
@@ -248,14 +261,10 @@ mod tests {
#[test]
fn replace_std_with_one_if_not_used() {
assert_eq!(
(StandardScaler {
means: vec![],
stds: vec![],
parameters: StandardScalerParameters {
with_mean: true,
with_std: false
}
})
(StandardScaler::<f64>::new(StandardScalerParameters {
with_mean: true,
with_std: false
}))
.adjust_column_std(10.0),
1.0
)
@@ -331,7 +340,8 @@ mod tests {
parameters: StandardScalerParameters {
with_mean: true,
with_std: true
}
},
_phantom: Default::default(),
})
)
}
@@ -355,7 +365,7 @@ mod tests {
);
assert!(
&DenseMatrix::from_2d_vec(&vec![fitted_scaler.stds]).approximate_eq(
&DenseMatrix::<f64>::from_2d_vec(&vec![fitted_scaler.stds]).approximate_eq(
&DenseMatrix::from_2d_array(&[&[
0.29426447500954,
0.16758497615485,
@@ -378,6 +388,7 @@ mod tests {
with_mean: true,
with_std: false,
},
_phantom: Default::default(),
};
assert_eq!(
@@ -397,6 +408,7 @@ mod tests {
with_mean: false,
with_std: true,
},
_phantom: Default::default(),
};
assert_eq!(