work/smartcore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge potential next release v0.4 (#187) Breaking Changes

Browse Source 
* First draft of the new n-dimensional arrays + NB use case
* Improves default implementation of multiple Array methods
* Refactors tree methods
* Adds matrix decomposition routines
* Adds matrix decomposition methods to ndarray and nalgebra bindings
* Refactoring + linear regression now uses array2
* Ridge & Linear regression
* LBFGS optimizer & logistic regression
* LBFGS optimizer & logistic regression
* Changes linear methods, metrics and model selection methods to new n-dimensional arrays
* Switches KNN and clustering algorithms to new n-d array layer
* Refactors distance metrics
* Optimizes knn and clustering methods
* Refactors metrics module
* Switches decomposition methods to n-dimensional arrays
* Linalg refactoring - cleanup rng merge (#172)
* Remove legacy DenseMatrix and BaseMatrix implementation. Port the new Number, FloatNumber and Array implementation into module structure.
* Exclude AUC metrics. Needs reimplementation
* Improve developers walkthrough

New traits system in place at `src/numbers` and `src/linalg`
Co-authored-by: Lorenzo <tunedconsulting@gmail.com>

* Provide SupervisedEstimator with a constructor to avoid explicit dynamical box allocation in 'cross_validate' and 'cross_validate_predict' as required by the use of 'dyn' as per Rust 2021
* Implement getters to use as_ref() in src/neighbors
* Implement getters to use as_ref() in src/naive_bayes
* Implement getters to use as_ref() in src/linear
* Add Clone to src/naive_bayes
* Change signature for cross_validate and other model_selection functions to abide to use of dyn in Rust 2021
* Implement ndarray-bindings. Remove FloatNumber from implementations
* Drop nalgebra-bindings support (as decided in conf-call to go for ndarray)
* Remove benches. Benches will have their own repo at smartcore-benches
* Implement SVC
* Implement SVC serialization. Move search parameters in dedicated module
* Implement SVR. Definitely too slow
* Fix compilation issues for wasm (#202)

Co-authored-by: Luis Moreno <morenol@users.noreply.github.com>
* Fix tests (#203)

* Port linalg/traits/stats.rs
* Improve methods naming
* Improve Display for DenseMatrix

Co-authored-by: Montana Low <montanalow@users.noreply.github.com>
Co-authored-by: VolodymyrOrlov <volodymyr.orlov@gmail.com>
This commit is contained in:
Lorenzo
2022-10-31 10:44:57 +00:00
committed by GitHub
parent bb71656137
commit 52eb6ce023
110 changed files with 10327 additions and 9107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/linear/bg_solver.rs
+78 -48
View File
@@ -1,13 +1,42 @@
//! This is a generic solver for Ax = b type of equation
//!
//! Example:
//! ```
//! use smartcore::linalg::basic::arrays::Array1;
//! use smartcore::linalg::basic::arrays::Array2;
//! use smartcore::linalg::basic::matrix::DenseMatrix;
//! use smartcore::linear::bg_solver::*;
//! use smartcore::numbers::floatnum::FloatNumber;
//! use smartcore::linear::bg_solver::BiconjugateGradientSolver;
//!
//! pub struct BGSolver {}
//! impl<'a, T: FloatNumber, X: Array2<T>> BiconjugateGradientSolver<'a, T, X> for BGSolver {}
//!
//! let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
//! let b = vec![40., 51., 28.];
//! let expected = vec![1.0, 2.0, 3.0];
//! let mut x = Vec::zeros(3);
//! let solver = BGSolver {};
//! let err: f64 = solver.solve_mut(&a, &b, &mut x, 1e-6, 6).unwrap();
//! ```
//!
//! for more information take a look at [this Wikipedia article](https://en.wikipedia.org/wiki/Biconjugate_gradient_method)
//! and [this paper](https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf)
use crate::error::Failed;
use crate::linalg::Matrix;
use crate::math::num::RealNumber;
use crate::linalg::basic::arrays::{Array, Array1, Array2, ArrayView1, MutArrayView1};
use crate::numbers::floatnum::FloatNumber;
pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
fn solve_mut(&self, a: &M, b: &M, x: &mut M, tol: T, max_iter: usize) -> Result<T, Failed> {
///
pub trait BiconjugateGradientSolver<'a, T: FloatNumber, X: Array2<T>> {
///
fn solve_mut(
&self,
a: &'a X,
b: &Vec<T>,
x: &mut Vec<T>,
tol: T,
max_iter: usize,
) -> Result<T, Failed> {
if tol <= T::zero() {
return Err(Failed::fit("tolerance shoud be > 0"));
}
@@ -16,25 +45,25 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
return Err(Failed::fit("maximum number of iterations should be > 0"));
}
let (n, _) = b.shape();
let n = b.shape();
let mut r = M::zeros(n, 1);
let mut rr = M::zeros(n, 1);
let mut z = M::zeros(n, 1);
let mut zz = M::zeros(n, 1);
let mut r = Vec::zeros(n);
let mut rr = Vec::zeros(n);
let mut z = Vec::zeros(n);
let mut zz = Vec::zeros(n);
self.mat_vec_mul(a, x, &mut r);
for j in 0..n {
r.set(j, 0, b.get(j, 0) - r.get(j, 0));
rr.set(j, 0, r.get(j, 0));
r[j] = b[j] - r[j];
rr[j] = r[j];
}
let bnrm = b.norm(T::two());
self.solve_preconditioner(a, &r, &mut z);
let bnrm = b.norm(2f64);
self.solve_preconditioner(a, &r[..], &mut z[..]);
let mut p = M::zeros(n, 1);
let mut pp = M::zeros(n, 1);
let mut p = Vec::zeros(n);
let mut pp = Vec::zeros(n);
let mut bkden = T::zero();
let mut err = T::zero();
@@ -43,35 +72,33 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
self.solve_preconditioner(a, &rr, &mut zz);
for j in 0..n {
bknum += z.get(j, 0) * rr.get(j, 0);
bknum += z[j] * rr[j];
}
if iter == 1 {
for j in 0..n {
p.set(j, 0, z.get(j, 0));
pp.set(j, 0, zz.get(j, 0));
}
p[..n].copy_from_slice(&z[..n]);
pp[..n].copy_from_slice(&zz[..n]);
} else {
let bk = bknum / bkden;
for j in 0..n {
p.set(j, 0, bk * p.get(j, 0) + z.get(j, 0));
pp.set(j, 0, bk * pp.get(j, 0) + zz.get(j, 0));
p[j] = bk * pp[j] + z[j];
pp[j] = bk * pp[j] + zz[j];
}
}
bkden = bknum;
self.mat_vec_mul(a, &p, &mut z);
let mut akden = T::zero();
for j in 0..n {
akden += z.get(j, 0) * pp.get(j, 0);
akden += z[j] * pp[j];
}
let ak = bknum / akden;
self.mat_t_vec_mul(a, &pp, &mut zz);
for j in 0..n {
x.set(j, 0, x.get(j, 0) + ak * p.get(j, 0));
r.set(j, 0, r.get(j, 0) - ak * z.get(j, 0));
rr.set(j, 0, rr.get(j, 0) - ak * zz.get(j, 0));
x[j] += ak * p[j];
r[j] -= ak * z[j];
rr[j] -= ak * zz[j];
}
self.solve_preconditioner(a, &r, &mut z);
err = r.norm(T::two()) / bnrm;
err = T::from_f64(r.norm(2f64) / bnrm).unwrap();
if err <= tol {
break;
@@ -81,36 +108,38 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
Ok(err)
}
fn solve_preconditioner(&self, a: &M, b: &M, x: &mut M) {
///
fn solve_preconditioner(&self, a: &'a X, b: &[T], x: &mut [T]) {
let diag = Self::diag(a);
let n = diag.len();
for (i, diag_i) in diag.iter().enumerate().take(n) {
if *diag_i != T::zero() {
x.set(i, 0, b.get(i, 0) / *diag_i);
x[i] = b[i] / *diag_i;
} else {
x.set(i, 0, b.get(i, 0));
x[i] = b[i];
}
}
}
// y = Ax
fn mat_vec_mul(&self, a: &M, x: &M, y: &mut M) {
y.copy_from(&a.matmul(x));
/// y = Ax
fn mat_vec_mul(&self, a: &X, x: &Vec<T>, y: &mut Vec<T>) {
y.copy_from(&x.xa(false, a));
}
// y = Atx
fn mat_t_vec_mul(&self, a: &M, x: &M, y: &mut M) {
y.copy_from(&a.ab(true, x, false));
/// y = Atx
fn mat_t_vec_mul(&self, a: &X, x: &Vec<T>, y: &mut Vec<T>) {
y.copy_from(&x.xa(true, a));
}
fn diag(a: &M) -> Vec<T> {
///
fn diag(a: &X) -> Vec<T> {
let (nrows, ncols) = a.shape();
let n = nrows.min(ncols);
let mut d = Vec::with_capacity(n);
for i in 0..n {
d.push(a.get(i, i));
d.push(*a.get((i, i)));
}
d
@@ -120,28 +149,29 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
#[cfg(test)]
mod tests {
use super::*;
use crate::linalg::naive::dense_matrix::*;
use crate::linalg::basic::arrays::Array2;
use crate::linalg::basic::matrix::DenseMatrix;
pub struct BGSolver {}
impl<T: RealNumber, M: Matrix<T>> BiconjugateGradientSolver<T, M> for BGSolver {}
impl<T: FloatNumber, X: Array2<T>> BiconjugateGradientSolver<'_, T, X> for BGSolver {}
#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
#[test]
fn bg_solver() {
let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
let b = DenseMatrix::from_2d_array(&[&[40., 51., 28.]]);
let expected = DenseMatrix::from_2d_array(&[&[1.0, 2.0, 3.0]]);
let b = vec![40., 51., 28.];
let expected = vec![1.0, 2.0, 3.0];
let mut x = DenseMatrix::zeros(3, 1);
let mut x = Vec::zeros(3);
let solver = BGSolver {};
let err: f64 = solver
.solve_mut(&a, &b.transpose(), &mut x, 1e-6, 6)
.unwrap();
let err: f64 = solver.solve_mut(&a, &b, &mut x, 1e-6, 6).unwrap();
assert!(x.transpose().approximate_eq(&expected, 1e-4));
assert!(x
.iter()
.zip(expected.iter())
.all(|(&a, &b)| (a - b).abs() < 1e-4));
assert!((err - 0.0).abs() < 1e-4);
}
}