Handle kernel serialization (#232)

* Handle kernel serialization
* Do not use typetag in WASM
* enable tests for serialization
* Update serde feature deps

Co-authored-by: Luis Moreno <morenol@users.noreply.github.com>
Co-authored-by: Lorenzo <tunedconsulting@gmail.com>

.github/CODEOWNERS

@@ -0,0 +1,7 @@
+# These owners will be the default owners for everything in
+# the repo. Unless a later match takes precedence,
+# Developers in this list will be requested for
+# review when someone opens a pull request.
+*       @VolodymyrOrlov
+*       @morenol
+*       @Mec-iS

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,22 @@
+# Code of Conduct
+
+As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality.
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery
+* Personal attacks
+* Trolling or insulting/derogatory comments
+* Public or private harassment
+* Publishing other's private information, such as physical or electronic addresses, without explicit permission
+* Other unethical or unprofessional conduct.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. By adopting this Code of Conduct, project maintainers commit themselves to fairly and consistently applying these principles to every aspect of managing this project. Project maintainers who do not follow or enforce the Code of Conduct may be permanently removed from the project team.
+
+This code of conduct applies both within project spaces and in public spaces when an individual is representing the project or its community.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
+
+This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.2.0, available at [http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/)

.github/CONTRIBUTING.md

@@ -0,0 +1,70 @@
+# **Contributing**
+
+When contributing to this repository, please first discuss the change you wish to make via issue,
+email, or any other method with the owners of this repository before making a change.
+
+Please note we have a [code of conduct](CODE_OF_CONDUCT.md), please follow it in all your interactions with the project.
+
+## Background
+
+We try to follow these principles:
+* follow as much as possible the sklearn API to give a frictionless user experience for practitioners already familiar with it
+* use only pure-Rust implementations for safety and future-proofing (with some low-level limited exceptions)
+* do not use macros in the library code to allow readability and transparent behavior
+* priority is not on "big data" dataset, try to be fast for small/average dataset with limited memory footprint.
+
+## Pull Request Process
+
+1. Open a PR following the template (erase the part of the template you don't need).
+2. Update the CHANGELOG.md with details of changes to the interface if they are breaking changes, this includes new environment variables, exposed ports useful file locations and container parameters.
+3. Pull Request can be merged in once you have the sign-off of one other developer, or if you do not have permission to do that you may request the reviewer to merge it for you.
+
+### generic guidelines
+Take a look to the conventions established by existing code:
+* Every module should come with some reference to scientific literature that allows relating the code to research. Use the `//!` comments at the top of the module to tell readers about the basics of the procedure you are implementing.
+* Every module should provide a Rust doctest, a brief test embedded with the documentation that explains how to use the procedure implemented.
+* Every module should provide comprehensive tests at the end, in its `mod tests {}` sub-module. These tests can be flagged or not with configuration flags to allow WebAssembly target.
+* Run `cargo doc --no-deps --open` and read the generated documentation in the browser to be sure that your changes reflects in the documentation and new code is documented.
+
+#### digging deeper
+* a nice overview of the codebase is given by [static analyzer](https://mozilla.github.io/rust-code-analysis/metrics.html):
+```
+$ cargo install rust-code-analysis-cli
+// print metrics for every module
+$ rust-code-analysis-cli -m -O json -o . -p src/ --pr
+// print full AST for a module
+$ rust-code-analysis-cli -p src/algorithm/neighbour/fastpair.rs --ls 22 --le 213 -d > ast.txt
+```
+* find more information about what happens in your binary with [`twiggy`](https://rustwasm.github.io/twiggy/install.html). This need a compiled binary so create a brief `main {}` function using `smartcore` and then point `twiggy` to that file.
+
+## Issue Report Process
+
+1. Go to the project's issues.
+2. Select the template that better fits your issue.
+3. Read carefully the instructions and write within the template guidelines.
+4. Submit it and wait for support.
+
+## Reviewing process
+
+1. After a PR is opened maintainers are notified
+2. Probably changes will be required to comply with the workflow, these commands are run automatically and all tests shall pass:
+    * **Coverage** (optional): `tarpaulin` is used with command `cargo tarpaulin --out Lcov --all-features -- --test-threads 1`
+    * **Formatting**: run `rustfmt src/*.rs` to apply automatic formatting
+    * **Linting**: `clippy` is used with command `cargo clippy --all-features -- -Drust-2018-idioms -Dwarnings`
+    * **Testing**: multiple test pipelines are run for different targets
+3. When everything is OK, code is merged.
+
+
+## Contribution Best Practices
+
+* Read this [how-to about Github workflow here](https://guides.github.com/introduction/flow/) if you are not familiar with.
+
+* Read all the texts related to [contributing for an OS community](https://github.com/HTTP-APIs/hydrus/tree/master/.github).
+
+* Read this [how-to about writing a PR](https://github.com/blog/1943-how-to-write-the-perfect-pull-request) and this [other how-to about writing a issue](https://wiredcraft.com/blog/how-we-write-our-github-issues/)
+  
+* **read history**: search past open or closed issues for your problem before opening a new issue.
+
+* **PRs on develop**: any change should be PRed first in `development`
+
+* **testing**:  everything should work and be tested as defined in the workflow. If any is failing for non-related reasons, annotate the test failure in the PR comment.

.github/DEVELOPERS.md

@@ -0,0 +1,43 @@
+# smartcore: Introduction to modules
+
+Important source of information:
+* [Rust API guidelines](https://rust-lang.github.io/api-guidelines/about.html)
+
+## Walkthrough: traits system and basic structures
+
+#### numbers
+The library is founded on basic traits provided by `num-traits`. Basic traits are in `src/numbers`. These traits are used to define all the procedures in the library to make everything safer and provide constraints to what implementations can handle.
+
+#### linalg
+`numbers` are made at use in linear algebra structures in the **`src/linalg/basic`** module. These sub-modules define the traits used all over the code base. 
+
+* *arrays*: In particular data structures like `Array`, `Array1` (1-dimensional), `Array2` (matrix, 2-D); plus their "views" traits. Views are used to provide no-footprint access to data, they have composed traits to allow writing (mutable traits: `MutArray`, `ArrayViewMut`, ...).
+* *matrix*: This provides the main entrypoint to matrices operations and currently the only structure provided in the shape of `struct DenseMatrix`. A matrix can be instantiated and automatically make available all the traits in "arrays" (sparse matrices implementation will be provided).
+* *vector*: Convenience traits are implemented for `std::Vec` to allow extensive reuse.
+
+These are all traits and by definition they do not allow instantiation. For instantiable structures see implementation like `DenseMatrix` with relative constructor.
+
+#### linalg/traits
+The traits in `src/linalg/traits` are closely linked to Linear Algebra's theoretical framework. These traits are used to specify characteristics and constraints for types accepted by various algorithms. For example these allow to define if a matrix is `QRDecomposable` and/or `SVDDecomposable`. See docstring for referencese to theoretical framework.
+
+As above these are all traits and by definition they do not allow instantiation. They are mostly used to provide constraints for implementations. For example, the implementation for Linear Regression requires the input data `X` to be in `smartcore`'s trait system `Array2<FloatNumber> + QRDecomposable<TX> + SVDDecomposable<TX>`, a 2-D matrix that is both QR and SVD decomposable; that is what the provided strucure `linalg::arrays::matrix::DenseMatrix` happens to be: `impl<T: FloatNumber> QRDecomposable<T> for DenseMatrix<T> {};impl<T: FloatNumber> SVDDecomposable<T> for DenseMatrix<T> {}`. 
+
+#### metrics
+Implementations for metrics (classification, regression, cluster, ...) and distance measure (Euclidean, Hamming, Manhattan, ...). For example: `Accuracy`, `F1`, `AUC`, `Precision`, `R2`. As everything else in the code base, these implementations reuse `numbers` and `linalg` traits and structures.
+
+These are collected in structures like `pub struct ClassificationMetrics<T> {}` that implements `metrics::Metrics`, these are groups of functions (classification, regression, cluster, ...) that provide instantiation for the structures. Each of those instantiation can be passed around using the relative function, like `pub fn accuracy<T: Number + RealNumber + FloatNumber, V: ArrayView1<T>>(y_true: &V, y_pred: &V) -> T`. This provides a mechanism for metrics to be passed to higher interfaces like the `cross_validate`:
+```rust
+let results =
+  cross_validate(
+      BiasedEstimator::new(),  // custom estimator
+      &x, &y,                  // input data
+      NoParameters {},         // extra parameters
+      cv,                      // type of cross validator
+      &accuracy                // **metrics function** <--------
+  ).unwrap();
+```
+
+TODO: complete for all modules
+
+## Notebooks
+Proceed to the [**notebooks**](https://github.com/smartcorelib/smartcore-jupyter/) to see these modules in action.

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,25 @@
+### I'm submitting a 
+- [ ] bug report.
+- [ ] improvement.
+- [ ] feature request.
+
+### Current Behaviour:
+<!-- Describe about the bug -->
+
+### Expected Behaviour:
+<!-- Describe what will happen if bug is removed -->
+
+### Steps to reproduce:
+<!-- If you can then please provide the steps to reproduce the bug -->
+
+### Snapshot:
+<!-- If you can then please provide the screenshot of the issue you are facing -->
+
+### Environment:
+<!-- Please provide the following environment details if relevant --> 
+* rustc version
+* cargo version
+* OS details
+
+### Do you want to work on this issue?
+<!-- yes/no -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,29 @@
+<!-- Please create (if there is not one yet) a issue before sending a PR -->
+<!-- Add issue number (Eg: fixes #123) -->
+<!-- Always provide changes in existing tests or new tests -->
+
+Fixes #
+
+### Checklist
+- [ ] My branch is up-to-date with development branch.
+- [ ] Everything works and tested on latest stable Rust.
+- [ ] Coverage and Linting have been applied 
+
+### Current behaviour
+<!-- Describe the code you are going to change and its behaviour -->
+
+### New expected behaviour
+<!-- Describe the new code and its expected behaviour -->
+
+### Change logs
+
+<!-- #### Added -->
+<!-- Edit these points below to describe the new features added with this PR -->
+<!-- - Feature 1 -->
+<!-- - Feature 2 -->
+
+
+<!-- #### Changed -->
+<!-- Edit these points below to describe the changes made in existing functionality with this PR -->
+<!-- - Change 1 -->
+<!-- - Change 1 -->

.github/workflows/ci.yml

@@ -2,35 +2,37 @@ name: CI
 
 on:
   push:
-    branches: [ main, development ]
+    branches: [main, development]
   pull_request:
-    branches: [ development ]
+    branches: [development]
 
 jobs:
   tests:
     runs-on: "${{ matrix.platform.os }}-latest"
     strategy:
       matrix:
-        platform: [
-          { os: "windows", target: "x86_64-pc-windows-msvc" },
-          { os: "windows", target: "i686-pc-windows-msvc" },
-          { os: "ubuntu", target: "x86_64-unknown-linux-gnu" },
-          { os: "ubuntu", target: "i686-unknown-linux-gnu" },
-          { os: "ubuntu", target: "wasm32-unknown-unknown" },
-          { os: "macos", target: "aarch64-apple-darwin" },
-        ]    
+        platform:
+          [
+            { os: "windows", target: "x86_64-pc-windows-msvc" },
+            { os: "windows", target: "i686-pc-windows-msvc" },
+            { os: "ubuntu", target: "x86_64-unknown-linux-gnu" },
+            { os: "ubuntu", target: "i686-unknown-linux-gnu" },
+            { os: "ubuntu", target: "wasm32-unknown-unknown" },
+            { os: "macos", target: "aarch64-apple-darwin" },
+            { os: "ubuntu", target: "wasm32-wasi" },
+          ]
     env:
       TZ: "/usr/share/zoneinfo/your/location"
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Cache .cargo and target
         uses: actions/cache@v2
         with:
           path: |
             ~/.cargo
             ./target
-          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml') }}
-          restore-keys: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml') }}
+          key: ${{ runner.os }}-cargo-${{ matrix.platform.target }}-${{ hashFiles('**/Cargo.toml') }}
+          restore-keys: ${{ runner.os }}-cargo-${{ matrix.platform.target }}-${{ hashFiles('**/Cargo.toml') }}
       - name: Install Rust toolchain
         uses: actions-rs/toolchain@v1
         with:
@@ -40,12 +42,20 @@ jobs:
           default: true
       - name: Install test runner for wasm
         if: matrix.platform.target == 'wasm32-unknown-unknown'
-        run:  curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
-      - name: Stable Build
+        run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+      - name: Install test runner for wasi
+        if: matrix.platform.target == 'wasm32-wasi'
+        run: curl https://wasmtime.dev/install.sh -sSf | bash
+      - name: Stable Build with all features
         uses: actions-rs/cargo@v1
         with:
           command: build
           args: --all-features --target ${{ matrix.platform.target }}
+      - name: Stable Build without features
+        uses: actions-rs/cargo@v1
+        with:
+          command: build
+          args: --target ${{ matrix.platform.target }}
       - name: Tests
         if: matrix.platform.target == 'x86_64-unknown-linux-gnu' || matrix.platform.target == 'x86_64-pc-windows-msvc' || matrix.platform.target == 'aarch64-apple-darwin'
         uses: actions-rs/cargo@v1
@@ -55,3 +65,40 @@ jobs:
       - name: Tests in WASM
         if: matrix.platform.target == 'wasm32-unknown-unknown'
         run: wasm-pack test --node -- --all-features
+      - name: Tests in WASI
+        if: matrix.platform.target == 'wasm32-wasi'
+        run: |
+          export WASMTIME_HOME="$HOME/.wasmtime"
+          export PATH="$WASMTIME_HOME/bin:$PATH"
+          cargo install cargo-wasi && cargo wasi test
+
+  check_features:
+    runs-on: "${{ matrix.platform.os }}-latest"
+    strategy:
+      matrix:
+        platform: [{ os: "ubuntu" }]
+        features: ["--features serde", "--features datasets", ""]
+    env:
+      TZ: "/usr/share/zoneinfo/your/location"
+    steps:
+      - uses: actions/checkout@v3
+      - name: Cache .cargo and target
+        uses: actions/cache@v2
+        with:
+          path: |
+            ~/.cargo
+            ./target
+          key: ${{ runner.os }}-cargo-features-${{ hashFiles('**/Cargo.toml') }}
+          restore-keys: ${{ runner.os }}-cargo-features-${{ hashFiles('**/Cargo.toml') }}
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+          target: ${{ matrix.platform.target }}
+          profile: minimal
+          default: true
+      - name: Stable Build
+        uses: actions-rs/cargo@v1
+        with:
+          command: build
+          args: --no-default-features ${{ matrix.features }}

.github/workflows/coverage.yml

@@ -39,6 +39,6 @@ jobs:
           command: tarpaulin
           args: --out Lcov --all-features -- --test-threads 1
       - name: Upload to codecov.io
-        uses: codecov/codecov-action@v1
+        uses: codecov/codecov-action@v2
         with:
           fail_ci_if_error:     true

.gitignore

@@ -17,3 +17,15 @@ smartcore.code-workspace
 
 # OS
 .DS_Store
+
+
+flamegraph.svg
+perf.data
+perf.data.old
+src.dot
+out.svg
+
+FlameGraph/
+out.stacks
+*.json
+*.txt

CHANGELOG.md

@@ -4,7 +4,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.3.0] - 2022-11-09 
+
+## Added
+- WARNING: Breaking changes!
+- Complete refactoring with **extensive API changes** that includes:
+    * moving to a new traits system, less structs more traits
+    * adapting all the modules to the new traits system
+    * moving to Rust 2021, use of object-safe traits and `as_ref`
+    * reorganization of the code base, eliminate duplicates
+- implements `readers` (needs "serde" feature) for read/write CSV file, extendible to other formats
+- default feature is now Wasm-/Wasi-first
+
+## Changed
+- WARNING: Breaking changes!
+- Seeds to multiple algorithims that depend on random number generation
+- Added a new parameter to `train_test_split` to define the seed
+- changed use of "serde" feature
+
+## Dropped
+- WARNING: Breaking changes!
+- Drop `nalgebra-bindings` feature, only `ndarray` as supported library
+
+## [0.2.1] - 2021-05-10
 
 ## Added
 - L2 regularization penalty to the Logistic Regression

Cargo.toml

@@ -1,55 +1,65 @@
 [package]
 name = "smartcore"
-description = "The most advanced machine learning library in rust."
+description = "Machine Learning in Rust."
 homepage = "https://smartcorelib.org"
-version = "0.2.1"
-authors = ["SmartCore Developers"]
-edition = "2018"
+version = "0.3.0"
+authors = ["smartcore Developers"]
+edition = "2021"
 license = "Apache-2.0"
 documentation = "https://docs.rs/smartcore"
 repository = "https://github.com/smartcorelib/smartcore"
 readme = "README.md"
 keywords = ["machine-learning", "statistical", "ai", "optimization", "linear-algebra"]
 categories = ["science"]
-
-[features]
-default = ["datasets"]
-ndarray-bindings = ["ndarray"]
-nalgebra-bindings = ["nalgebra"]
-datasets = []
-fp_bench = []
+exclude = [
+    ".github",
+    ".gitignore",
+    "smartcore.iml",
+    "smartcore.svg",
+    "tests/"
+]
 
 [dependencies]
+approx = "0.5.1"
+cfg-if = "1.0.0"
 ndarray = { version = "0.15", optional = true }
-nalgebra = { version = "0.31", optional = true }
-num-traits = "0.2"
+num-traits = "0.2.12"
 num = "0.4"
-rand = "0.8"
-rand_distr = "0.4"
+rand = { version = "0.8.5", default-features = false, features = ["small_rng"] }
+rand_distr = { version = "0.4", optional = true }
 serde = { version = "1", features = ["derive"], optional = true }
-itertools = "0.10.3"
+
+[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
+typetag = { version = "0.2", optional = true }
+
+[features]
+default = []
+serde = ["dep:serde", "dep:typetag"]
+ndarray-bindings = ["dep:ndarray"]
+datasets = ["dep:rand_distr", "std_rand", "serde"]
+std_rand = ["rand/std_rng", "rand/std"]
+# used by wasm32-unknown-unknown for in-browser usage
+js = ["getrandom/js"]
 
 [target.'cfg(target_arch = "wasm32")'.dependencies]
-getrandom = { version = "0.2", features = ["js"] }
+getrandom = { version = "*", optional = true }
+
+[target.'cfg(all(target_arch = "wasm32", not(target_os = "wasi")))'.dev-dependencies]
+wasm-bindgen-test = "0.3"
 
 [dev-dependencies]
-criterion = "0.3"
+itertools = "*"
 serde_json = "1.0"
 bincode = "1.3.1"
 
-[target.'cfg(target_arch = "wasm32")'.dev-dependencies]
-wasm-bindgen-test = "0.3"
+[workspace]
 
-[[bench]]
-name = "distance"
-harness = false
+[profile.test]
+debug = 1
+opt-level = 3
 
-[[bench]]
-name = "naive_bayes"
-harness = false
-required-features = ["ndarray-bindings", "nalgebra-bindings"]
-
-[[bench]]
-name = "fastpair"
-harness = false
-required-features = ["fp_bench"]
+[profile.release]
+strip = true 
+lto = true
+codegen-units = 1
+overflow-checks = true

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2019-present at smartcore developers (smartcorelib.org) 
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,18 +1,21 @@
 <p align="center">
   <a href="https://smartcorelib.org">
-    <img src="smartcore.svg" width="450" alt="SmartCore">    
+    <img src="smartcore.svg" width="450" alt="smartcore">    
   </a>  
 </p>
 <p align = "center">
     <strong>
-        <a href="https://smartcorelib.org">User guide</a> | <a href="https://docs.rs/smartcore/">API</a> | <a href="https://github.com/smartcorelib/smartcore-examples">Examples</a>
+        <a href="https://smartcorelib.org">User guide</a> | <a href="https://docs.rs/smartcore/">API</a> | <a href="https://github.com/smartcorelib/smartcore-jupyter">Notebooks</a>
     </strong>
 </p>
 
 -----
 
 <p align = "center">
-<b>The Most Advanced Machine Learning Library In Rust.</b>
+<b>Machine Learning in Rust</b>
 </p>
 
------
+-----
+[![CI](https://github.com/smartcorelib/smartcore/actions/workflows/ci.yml/badge.svg)](https://github.com/smartcorelib/smartcore/actions/workflows/ci.yml)
+
+To start getting familiar with the new smartcore v0.5 API, there is now available a [**Jupyter Notebook environment repository**](https://github.com/smartcorelib/smartcore-jupyter). Please see instructions there, contributions welcome see [CONTRIBUTING](.github/CONTRIBUTING.md).

benches/distance.rs

@@ -1,18 +0,0 @@
-#[macro_use]
-extern crate criterion;
-extern crate smartcore;
-
-use criterion::black_box;
-use criterion::Criterion;
-use smartcore::math::distance::*;
-
-fn criterion_benchmark(c: &mut Criterion) {
-    let a = vec![1., 2., 3.];
-
-    c.bench_function("Euclidean Distance", move |b| {
-        b.iter(|| Distances::euclidian().distance(black_box(&a), black_box(&a)))
-    });
-}
-
-criterion_group!(benches, criterion_benchmark);
-criterion_main!(benches);

benches/fastpair.rs

@@ -1,56 +0,0 @@
-use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};
-
-// to run this bench you have to change the declaraion in mod.rs ---> pub mod fastpair;
-use smartcore::algorithm::neighbour::fastpair::FastPair;
-use smartcore::linalg::naive::dense_matrix::*;
-use std::time::Duration;
-
-fn closest_pair_bench(n: usize, m: usize) -> () {
-    let x = DenseMatrix::<f64>::rand(n, m);
-    let fastpair = FastPair::new(&x);
-    let result = fastpair.unwrap();
-
-    result.closest_pair();
-}
-
-fn closest_pair_brute_bench(n: usize, m: usize) -> () {
-    let x = DenseMatrix::<f64>::rand(n, m);
-    let fastpair = FastPair::new(&x);
-    let result = fastpair.unwrap();
-
-    result.closest_pair_brute();
-}
-
-fn bench_fastpair(c: &mut Criterion) {
-    let mut group = c.benchmark_group("FastPair");
-
-    // with full samples size (100) the test will take too long
-    group.significance_level(0.1).sample_size(30);
-    // increase from default 5.0 secs
-    group.measurement_time(Duration::from_secs(60));
-
-    for n_samples in [100_usize, 1000_usize].iter() {
-        for n_features in [10_usize, 100_usize, 1000_usize].iter() {
-            group.bench_with_input(
-                BenchmarkId::from_parameter(format!(
-                    "fastpair --- n_samples: {}, n_features: {}",
-                    n_samples, n_features
-                )),
-                n_samples,
-                |b, _| b.iter(|| closest_pair_bench(*n_samples, *n_features)),
-            );
-            group.bench_with_input(
-                BenchmarkId::from_parameter(format!(
-                    "brute --- n_samples: {}, n_features: {}",
-                    n_samples, n_features
-                )),
-                n_samples,
-                |b, _| b.iter(|| closest_pair_brute_bench(*n_samples, *n_features)),
-            );
-        }
-    }
-    group.finish();
-}
-
-criterion_group!(benches, bench_fastpair);
-criterion_main!(benches);

benches/naive_bayes.rs

@@ -1,73 +0,0 @@
-use criterion::BenchmarkId;
-use criterion::{black_box, criterion_group, criterion_main, Criterion};
-
-use nalgebra::DMatrix;
-use ndarray::Array2;
-use smartcore::linalg::naive::dense_matrix::DenseMatrix;
-use smartcore::linalg::BaseMatrix;
-use smartcore::linalg::BaseVector;
-use smartcore::naive_bayes::gaussian::GaussianNB;
-
-pub fn gaussian_naive_bayes_fit_benchmark(c: &mut Criterion) {
-    let mut group = c.benchmark_group("GaussianNB::fit");
-
-    for n_samples in [100_usize, 1000_usize, 10000_usize].iter() {
-        for n_features in [10_usize, 100_usize, 1000_usize].iter() {
-            let x = DenseMatrix::<f64>::rand(*n_samples, *n_features);
-            let y: Vec<f64> = (0..*n_samples)
-                .map(|i| (i % *n_samples / 5_usize) as f64)
-                .collect::<Vec<f64>>();
-            group.bench_with_input(
-                BenchmarkId::from_parameter(format!(
-                    "n_samples: {}, n_features: {}",
-                    n_samples, n_features
-                )),
-                n_samples,
-                |b, _| {
-                    b.iter(|| {
-                        GaussianNB::fit(black_box(&x), black_box(&y), Default::default()).unwrap();
-                    })
-                },
-            );
-        }
-    }
-    group.finish();
-}
-
-pub fn gaussian_naive_matrix_datastructure(c: &mut Criterion) {
-    let mut group = c.benchmark_group("GaussianNB");
-    let classes = (0..10000).map(|i| (i % 25) as f64).collect::<Vec<f64>>();
-
-    group.bench_function("DenseMatrix", |b| {
-        let x = DenseMatrix::<f64>::rand(10000, 500);
-        let y = <DenseMatrix<f64> as BaseMatrix<f64>>::RowVector::from_array(&classes);
-
-        b.iter(|| {
-            GaussianNB::fit(black_box(&x), black_box(&y), Default::default()).unwrap();
-        })
-    });
-
-    group.bench_function("ndarray", |b| {
-        let x = Array2::<f64>::rand(10000, 500);
-        let y = <Array2<f64> as BaseMatrix<f64>>::RowVector::from_array(&classes);
-
-        b.iter(|| {
-            GaussianNB::fit(black_box(&x), black_box(&y), Default::default()).unwrap();
-        })
-    });
-
-    group.bench_function("ndalgebra", |b| {
-        let x = DMatrix::<f64>::rand(10000, 500);
-        let y = <DMatrix<f64> as BaseMatrix<f64>>::RowVector::from_array(&classes);
-
-        b.iter(|| {
-            GaussianNB::fit(black_box(&x), black_box(&y), Default::default()).unwrap();
-        })
-    });
-}
-criterion_group!(
-    benches,
-    gaussian_naive_bayes_fit_benchmark,
-    gaussian_naive_matrix_datastructure
-);
-criterion_main!(benches);

smartcore.svg

@@ -76,5 +76,5 @@
        y="81.876823"
        x="91.861809"
        id="tspan842"
-       sodipodi:role="line">SmartCore</tspan></text>
+       sodipodi:role="line">smartcore</tspan></text>
 </svg>

src/algorithm/neighbour/bbd_tree.rs

@@ -1,45 +1,45 @@
 use std::fmt::Debug;
 
-use crate::linalg::Matrix;
-use crate::math::distance::euclidian::*;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::metrics::distance::euclidian::*;
+use crate::numbers::basenum::Number;
 
 #[derive(Debug)]
-pub struct BBDTree<T: RealNumber> {
-    nodes: Vec<BBDTreeNode<T>>,
+pub struct BBDTree {
+    nodes: Vec<BBDTreeNode>,
     index: Vec<usize>,
     root: usize,
 }
 
 #[derive(Debug)]
-struct BBDTreeNode<T: RealNumber> {
+struct BBDTreeNode {
     count: usize,
     index: usize,
-    center: Vec<T>,
-    radius: Vec<T>,
-    sum: Vec<T>,
-    cost: T,
+    center: Vec<f64>,
+    radius: Vec<f64>,
+    sum: Vec<f64>,
+    cost: f64,
     lower: Option<usize>,
     upper: Option<usize>,
 }
 
-impl<T: RealNumber> BBDTreeNode<T> {
-    fn new(d: usize) -> BBDTreeNode<T> {
+impl BBDTreeNode {
+    fn new(d: usize) -> BBDTreeNode {
         BBDTreeNode {
             count: 0,
             index: 0,
-            center: vec![T::zero(); d],
-            radius: vec![T::zero(); d],
-            sum: vec![T::zero(); d],
-            cost: T::zero(),
+            center: vec![0f64; d],
+            radius: vec![0f64; d],
+            sum: vec![0f64; d],
+            cost: 0f64,
             lower: Option::None,
             upper: Option::None,
         }
     }
 }
 
-impl<T: RealNumber> BBDTree<T> {
-    pub fn new<M: Matrix<T>>(data: &M) -> BBDTree<T> {
+impl BBDTree {
+    pub fn new<T: Number, M: Array2<T>>(data: &M) -> BBDTree {
         let nodes = Vec::new();
 
         let (n, _) = data.shape();
@@ -61,18 +61,18 @@ impl<T: RealNumber> BBDTree<T> {
 
     pub(crate) fn clustering(
         &self,
-        centroids: &[Vec<T>],
-        sums: &mut Vec<Vec<T>>,
+        centroids: &[Vec<f64>],
+        sums: &mut Vec<Vec<f64>>,
         counts: &mut Vec<usize>,
         membership: &mut Vec<usize>,
-    ) -> T {
+    ) -> f64 {
         let k = centroids.len();
 
         counts.iter_mut().for_each(|v| *v = 0);
         let mut candidates = vec![0; k];
         for i in 0..k {
             candidates[i] = i;
-            sums[i].iter_mut().for_each(|v| *v = T::zero());
+            sums[i].iter_mut().for_each(|v| *v = 0f64);
         }
 
         self.filter(
@@ -89,13 +89,13 @@ impl<T: RealNumber> BBDTree<T> {
     fn filter(
         &self,
         node: usize,
-        centroids: &[Vec<T>],
+        centroids: &[Vec<f64>],
         candidates: &[usize],
         k: usize,
-        sums: &mut Vec<Vec<T>>,
+        sums: &mut Vec<Vec<f64>>,
         counts: &mut Vec<usize>,
         membership: &mut Vec<usize>,
-    ) -> T {
+    ) -> f64 {
         let d = centroids[0].len();
 
         let mut min_dist =
@@ -163,9 +163,9 @@ impl<T: RealNumber> BBDTree<T> {
     }
 
     fn prune(
-        center: &[T],
-        radius: &[T],
-        centroids: &[Vec<T>],
+        center: &[f64],
+        radius: &[f64],
+        centroids: &[Vec<f64>],
         best_index: usize,
         test_index: usize,
     ) -> bool {
@@ -177,22 +177,22 @@ impl<T: RealNumber> BBDTree<T> {
 
         let best = &centroids[best_index];
         let test = &centroids[test_index];
-        let mut lhs = T::zero();
-        let mut rhs = T::zero();
+        let mut lhs = 0f64;
+        let mut rhs = 0f64;
         for i in 0..d {
             let diff = test[i] - best[i];
             lhs += diff * diff;
-            if diff > T::zero() {
+            if diff > 0f64 {
                 rhs += (center[i] + radius[i] - best[i]) * diff;
             } else {
                 rhs += (center[i] - radius[i] - best[i]) * diff;
             }
         }
 
-        lhs >= T::two() * rhs
+        lhs >= 2f64 * rhs
     }
 
-    fn build_node<M: Matrix<T>>(&mut self, data: &M, begin: usize, end: usize) -> usize {
+    fn build_node<T: Number, M: Array2<T>>(&mut self, data: &M, begin: usize, end: usize) -> usize {
         let (_, d) = data.shape();
 
         let mut node = BBDTreeNode::new(d);
@@ -200,17 +200,17 @@ impl<T: RealNumber> BBDTree<T> {
         node.count = end - begin;
         node.index = begin;
 
-        let mut lower_bound = vec![T::zero(); d];
-        let mut upper_bound = vec![T::zero(); d];
+        let mut lower_bound = vec![0f64; d];
+        let mut upper_bound = vec![0f64; d];
 
         for i in 0..d {
-            lower_bound[i] = data.get(self.index[begin], i);
-            upper_bound[i] = data.get(self.index[begin], i);
+            lower_bound[i] = data.get((self.index[begin], i)).to_f64().unwrap();
+            upper_bound[i] = data.get((self.index[begin], i)).to_f64().unwrap();
         }
 
         for i in begin..end {
             for j in 0..d {
-                let c = data.get(self.index[i], j);
+                let c = data.get((self.index[i], j)).to_f64().unwrap();
                 if lower_bound[j] > c {
                     lower_bound[j] = c;
                 }
@@ -220,32 +220,32 @@ impl<T: RealNumber> BBDTree<T> {
             }
         }
 
-        let mut max_radius = T::from(-1.).unwrap();
+        let mut max_radius = -1f64;
         let mut split_index = 0;
         for i in 0..d {
-            node.center[i] = (lower_bound[i] + upper_bound[i]) / T::two();
-            node.radius[i] = (upper_bound[i] - lower_bound[i]) / T::two();
+            node.center[i] = (lower_bound[i] + upper_bound[i]) / 2f64;
+            node.radius[i] = (upper_bound[i] - lower_bound[i]) / 2f64;
             if node.radius[i] > max_radius {
                 max_radius = node.radius[i];
                 split_index = i;
             }
         }
 
-        if max_radius < T::from(1E-10).unwrap() {
+        if max_radius < 1E-10 {
             node.lower = Option::None;
             node.upper = Option::None;
             for i in 0..d {
-                node.sum[i] = data.get(self.index[begin], i);
+                node.sum[i] = data.get((self.index[begin], i)).to_f64().unwrap();
             }
 
             if end > begin + 1 {
                 let len = end - begin;
                 for i in 0..d {
-                    node.sum[i] *= T::from(len).unwrap();
+                    node.sum[i] *= len as f64;
                 }
             }
 
-            node.cost = T::zero();
+            node.cost = 0f64;
             return self.add_node(node);
         }
 
@@ -254,8 +254,10 @@ impl<T: RealNumber> BBDTree<T> {
         let mut i2 = end - 1;
         let mut size = 0;
         while i1 <= i2 {
-            let mut i1_good = data.get(self.index[i1], split_index) < split_cutoff;
-            let mut i2_good = data.get(self.index[i2], split_index) >= split_cutoff;
+            let mut i1_good =
+                data.get((self.index[i1], split_index)).to_f64().unwrap() < split_cutoff;
+            let mut i2_good =
+                data.get((self.index[i2], split_index)).to_f64().unwrap() >= split_cutoff;
 
             if !i1_good && !i2_good {
                 self.index.swap(i1, i2);
@@ -281,9 +283,9 @@ impl<T: RealNumber> BBDTree<T> {
                 self.nodes[node.lower.unwrap()].sum[i] + self.nodes[node.upper.unwrap()].sum[i];
         }
 
-        let mut mean = vec![T::zero(); d];
+        let mut mean = vec![0f64; d];
         for (i, mean_i) in mean.iter_mut().enumerate().take(d) {
-            *mean_i = node.sum[i] / T::from(node.count).unwrap();
+            *mean_i = node.sum[i] / node.count as f64;
         }
 
         node.cost = BBDTree::node_cost(&self.nodes[node.lower.unwrap()], &mean)
@@ -292,17 +294,17 @@ impl<T: RealNumber> BBDTree<T> {
         self.add_node(node)
     }
 
-    fn node_cost(node: &BBDTreeNode<T>, center: &[T]) -> T {
+    fn node_cost(node: &BBDTreeNode, center: &[f64]) -> f64 {
         let d = center.len();
-        let mut scatter = T::zero();
+        let mut scatter = 0f64;
         for (i, center_i) in center.iter().enumerate().take(d) {
-            let x = (node.sum[i] / T::from(node.count).unwrap()) - *center_i;
+            let x = (node.sum[i] / node.count as f64) - *center_i;
             scatter += x * x;
         }
-        node.cost + T::from(node.count).unwrap() * scatter
+        node.cost + node.count as f64 * scatter
     }
 
-    fn add_node(&mut self, new_node: BBDTreeNode<T>) -> usize {
+    fn add_node(&mut self, new_node: BBDTreeNode) -> usize {
         let idx = self.nodes.len();
         self.nodes.push(new_node);
         idx
@@ -312,9 +314,12 @@ impl<T: RealNumber> BBDTree<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn bbdtree_iris() {
         let data = DenseMatrix::from_2d_array(&[

src/algorithm/neighbour/cover_tree.rs

@@ -4,12 +4,12 @@
 //!
 //! ```
 //! use smartcore::algorithm::neighbour::cover_tree::*;
-//! use smartcore::math::distance::Distance;
+//! use smartcore::metrics::distance::Distance;
 //!
 //! #[derive(Clone)]
 //! struct SimpleDistance {} // Our distance function
 //!
-//! impl Distance<i32, f64> for SimpleDistance {
+//! impl Distance<i32> for SimpleDistance {
 //!   fn distance(&self, a: &i32, b: &i32) -> f64 { // simple simmetrical scalar distance
 //!     (a - b).abs() as f64
 //!   }
@@ -29,28 +29,27 @@ use serde::{Deserialize, Serialize};
 
 use crate::algorithm::sort::heap_select::HeapSelection;
 use crate::error::{Failed, FailedError};
-use crate::math::distance::Distance;
-use crate::math::num::RealNumber;
+use crate::metrics::distance::Distance;
 
 /// Implements Cover Tree algorithm
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct CoverTree<T, F: RealNumber, D: Distance<T, F>> {
-    base: F,
-    inv_log_base: F,
+pub struct CoverTree<T, D: Distance<T>> {
+    base: f64,
+    inv_log_base: f64,
     distance: D,
-    root: Node<F>,
+    root: Node,
     data: Vec<T>,
     identical_excluded: bool,
 }
 
-impl<T, F: RealNumber, D: Distance<T, F>> PartialEq for CoverTree<T, F, D> {
+impl<T, D: Distance<T>> PartialEq for CoverTree<T, D> {
     fn eq(&self, other: &Self) -> bool {
         if self.data.len() != other.data.len() {
             return false;
         }
         for i in 0..self.data.len() {
-            if self.distance.distance(&self.data[i], &other.data[i]) != F::zero() {
+            if self.distance.distance(&self.data[i], &other.data[i]) != 0f64 {
                 return false;
             }
         }
@@ -60,36 +59,36 @@ impl<T, F: RealNumber, D: Distance<T, F>> PartialEq for CoverTree<T, F, D> {
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-struct Node<F: RealNumber> {
+struct Node {
     idx: usize,
-    max_dist: F,
-    parent_dist: F,
-    children: Vec<Node<F>>,
+    max_dist: f64,
+    parent_dist: f64,
+    children: Vec<Node>,
     _scale: i64,
 }
 
 #[derive(Debug)]
-struct DistanceSet<F: RealNumber> {
+struct DistanceSet {
     idx: usize,
-    dist: Vec<F>,
+    dist: Vec<f64>,
 }
 
-impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D> {
+impl<T: Debug + PartialEq, D: Distance<T>> CoverTree<T, D> {
     /// Construct a cover tree.
     /// * `data` - vector of data points to search for.
     /// * `distance` - distance metric to use for searching. This function should extend [`Distance`](../../../math/distance/index.html) interface.
-    pub fn new(data: Vec<T>, distance: D) -> Result<CoverTree<T, F, D>, Failed> {
-        let base = F::from_f64(1.3).unwrap();
+    pub fn new(data: Vec<T>, distance: D) -> Result<CoverTree<T, D>, Failed> {
+        let base = 1.3f64;
         let root = Node {
             idx: 0,
-            max_dist: F::zero(),
-            parent_dist: F::zero(),
+            max_dist: 0f64,
+            parent_dist: 0f64,
             children: Vec::new(),
             _scale: 0,
         };
         let mut tree = CoverTree {
             base,
-            inv_log_base: F::one() / base.ln(),
+            inv_log_base: 1f64 / base.ln(),
             distance,
             root,
             data,
@@ -104,7 +103,7 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
     /// Find k nearest neighbors of `p`
     /// * `p` - look for k nearest points to `p`
     /// * `k` - the number of nearest neighbors to return
-    pub fn find(&self, p: &T, k: usize) -> Result<Vec<(usize, F, &T)>, Failed> {
+    pub fn find(&self, p: &T, k: usize) -> Result<Vec<(usize, f64, &T)>, Failed> {
         if k == 0 {
             return Err(Failed::because(FailedError::FindFailed, "k should be > 0"));
         }
@@ -119,13 +118,13 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
         let e = self.get_data_value(self.root.idx);
         let mut d = self.distance.distance(e, p);
 
-        let mut current_cover_set: Vec<(F, &Node<F>)> = Vec::new();
-        let mut zero_set: Vec<(F, &Node<F>)> = Vec::new();
+        let mut current_cover_set: Vec<(f64, &Node)> = Vec::new();
+        let mut zero_set: Vec<(f64, &Node)> = Vec::new();
 
         current_cover_set.push((d, &self.root));
 
         let mut heap = HeapSelection::with_capacity(k);
-        heap.add(F::max_value());
+        heap.add(std::f64::MAX);
 
         let mut empty_heap = true;
         if !self.identical_excluded || self.get_data_value(self.root.idx) != p {
@@ -134,7 +133,7 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
         }
 
         while !current_cover_set.is_empty() {
-            let mut next_cover_set: Vec<(F, &Node<F>)> = Vec::new();
+            let mut next_cover_set: Vec<(f64, &Node)> = Vec::new();
             for par in current_cover_set {
                 let parent = par.1;
                 for c in 0..parent.children.len() {
@@ -146,7 +145,7 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
                     }
 
                     let upper_bound = if empty_heap {
-                        F::infinity()
+                        std::f64::INFINITY
                     } else {
                         *heap.peek()
                     };
@@ -169,7 +168,7 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
             current_cover_set = next_cover_set;
         }
 
-        let mut neighbors: Vec<(usize, F, &T)> = Vec::new();
+        let mut neighbors: Vec<(usize, f64, &T)> = Vec::new();
         let upper_bound = *heap.peek();
         for ds in zero_set {
             if ds.0 <= upper_bound {
@@ -189,25 +188,25 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
     /// Find all nearest neighbors within radius `radius` from `p`
     /// * `p` - look for k nearest points to `p`
     /// * `radius` - radius of the search
-    pub fn find_radius(&self, p: &T, radius: F) -> Result<Vec<(usize, F, &T)>, Failed> {
-        if radius <= F::zero() {
+    pub fn find_radius(&self, p: &T, radius: f64) -> Result<Vec<(usize, f64, &T)>, Failed> {
+        if radius <= 0f64 {
             return Err(Failed::because(
                 FailedError::FindFailed,
                 "radius should be > 0",
             ));
         }
 
-        let mut neighbors: Vec<(usize, F, &T)> = Vec::new();
+        let mut neighbors: Vec<(usize, f64, &T)> = Vec::new();
 
-        let mut current_cover_set: Vec<(F, &Node<F>)> = Vec::new();
-        let mut zero_set: Vec<(F, &Node<F>)> = Vec::new();
+        let mut current_cover_set: Vec<(f64, &Node)> = Vec::new();
+        let mut zero_set: Vec<(f64, &Node)> = Vec::new();
 
         let e = self.get_data_value(self.root.idx);
         let mut d = self.distance.distance(e, p);
         current_cover_set.push((d, &self.root));
 
         while !current_cover_set.is_empty() {
-            let mut next_cover_set: Vec<(F, &Node<F>)> = Vec::new();
+            let mut next_cover_set: Vec<(f64, &Node)> = Vec::new();
             for par in current_cover_set {
                 let parent = par.1;
                 for c in 0..parent.children.len() {
@@ -240,23 +239,23 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
         Ok(neighbors)
     }
 
-    fn new_leaf(&self, idx: usize) -> Node<F> {
+    fn new_leaf(&self, idx: usize) -> Node {
         Node {
             idx,
-            max_dist: F::zero(),
-            parent_dist: F::zero(),
+            max_dist: 0f64,
+            parent_dist: 0f64,
             children: Vec::new(),
             _scale: 100,
         }
     }
 
     fn build_cover_tree(&mut self) {
-        let mut point_set: Vec<DistanceSet<F>> = Vec::new();
-        let mut consumed_set: Vec<DistanceSet<F>> = Vec::new();
+        let mut point_set: Vec<DistanceSet> = Vec::new();
+        let mut consumed_set: Vec<DistanceSet> = Vec::new();
 
         let point = &self.data[0];
         let idx = 0;
-        let mut max_dist = -F::one();
+        let mut max_dist = -1f64;
 
         for i in 1..self.data.len() {
             let dist = self.distance.distance(point, &self.data[i]);
@@ -284,16 +283,16 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
         p: usize,
         max_scale: i64,
         top_scale: i64,
-        point_set: &mut Vec<DistanceSet<F>>,
-        consumed_set: &mut Vec<DistanceSet<F>>,
-    ) -> Node<F> {
+        point_set: &mut Vec<DistanceSet>,
+        consumed_set: &mut Vec<DistanceSet>,
+    ) -> Node {
         if point_set.is_empty() {
             self.new_leaf(p)
         } else {
             let max_dist = self.max(point_set);
             let next_scale = (max_scale - 1).min(self.get_scale(max_dist));
             if next_scale == std::i64::MIN {
-                let mut children: Vec<Node<F>> = Vec::new();
+                let mut children: Vec<Node> = Vec::new();
                 let mut leaf = self.new_leaf(p);
                 children.push(leaf);
                 while !point_set.is_empty() {
@@ -304,13 +303,13 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
                 }
                 Node {
                     idx: p,
-                    max_dist: F::zero(),
-                    parent_dist: F::zero(),
+                    max_dist: 0f64,
+                    parent_dist: 0f64,
                     children,
                     _scale: 100,
                 }
             } else {
-                let mut far: Vec<DistanceSet<F>> = Vec::new();
+                let mut far: Vec<DistanceSet> = Vec::new();
                 self.split(point_set, &mut far, max_scale);
 
                 let child = self.batch_insert(p, next_scale, top_scale, point_set, consumed_set);
@@ -319,14 +318,14 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
                     point_set.append(&mut far);
                     child
                 } else {
-                    let mut children: Vec<Node<F>> = vec![child];
-                    let mut new_point_set: Vec<DistanceSet<F>> = Vec::new();
-                    let mut new_consumed_set: Vec<DistanceSet<F>> = Vec::new();
+                    let mut children: Vec<Node> = vec![child];
+                    let mut new_point_set: Vec<DistanceSet> = Vec::new();
+                    let mut new_consumed_set: Vec<DistanceSet> = Vec::new();
 
                     while !point_set.is_empty() {
-                        let set: DistanceSet<F> = point_set.remove(point_set.len() - 1);
+                        let set: DistanceSet = point_set.remove(point_set.len() - 1);
 
-                        let new_dist: F = set.dist[set.dist.len() - 1];
+                        let new_dist = set.dist[set.dist.len() - 1];
 
                         self.dist_split(
                             point_set,
@@ -374,7 +373,7 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
                     Node {
                         idx: p,
                         max_dist: self.max(consumed_set),
-                        parent_dist: F::zero(),
+                        parent_dist: 0f64,
                         children,
                         _scale: (top_scale - max_scale),
                     }
@@ -385,12 +384,12 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
 
     fn split(
         &self,
-        point_set: &mut Vec<DistanceSet<F>>,
-        far_set: &mut Vec<DistanceSet<F>>,
+        point_set: &mut Vec<DistanceSet>,
+        far_set: &mut Vec<DistanceSet>,
         max_scale: i64,
     ) {
         let fmax = self.get_cover_radius(max_scale);
-        let mut new_set: Vec<DistanceSet<F>> = Vec::new();
+        let mut new_set: Vec<DistanceSet> = Vec::new();
         for n in point_set.drain(0..) {
             if n.dist[n.dist.len() - 1] <= fmax {
                 new_set.push(n);
@@ -404,13 +403,13 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
 
     fn dist_split(
         &self,
-        point_set: &mut Vec<DistanceSet<F>>,
-        new_point_set: &mut Vec<DistanceSet<F>>,
+        point_set: &mut Vec<DistanceSet>,
+        new_point_set: &mut Vec<DistanceSet>,
         new_point: &T,
         max_scale: i64,
     ) {
         let fmax = self.get_cover_radius(max_scale);
-        let mut new_set: Vec<DistanceSet<F>> = Vec::new();
+        let mut new_set: Vec<DistanceSet> = Vec::new();
         for mut n in point_set.drain(0..) {
             let new_dist = self
                 .distance
@@ -426,24 +425,24 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
         point_set.append(&mut new_set);
     }
 
-    fn get_cover_radius(&self, s: i64) -> F {
-        self.base.powf(F::from_i64(s).unwrap())
+    fn get_cover_radius(&self, s: i64) -> f64 {
+        self.base.powf(s as f64)
     }
 
     fn get_data_value(&self, idx: usize) -> &T {
         &self.data[idx]
     }
 
-    fn get_scale(&self, d: F) -> i64 {
-        if d == F::zero() {
+    fn get_scale(&self, d: f64) -> i64 {
+        if d == 0f64 {
             std::i64::MIN
         } else {
-            (self.inv_log_base * d.ln()).ceil().to_i64().unwrap()
+            (self.inv_log_base * d.ln()).ceil() as i64
         }
     }
 
-    fn max(&self, distance_set: &[DistanceSet<F>]) -> F {
-        let mut max = F::zero();
+    fn max(&self, distance_set: &[DistanceSet]) -> f64 {
+        let mut max = 0f64;
         for n in distance_set {
             if max < n.dist[n.dist.len() - 1] {
                 max = n.dist[n.dist.len() - 1];
@@ -457,19 +456,22 @@ impl<T: Debug + PartialEq, F: RealNumber, D: Distance<T, F>> CoverTree<T, F, D>
 mod tests {
 
     use super::*;
-    use crate::math::distance::Distances;
+    use crate::metrics::distance::Distances;
 
     #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
     #[derive(Debug, Clone)]
     struct SimpleDistance {}
 
-    impl Distance<i32, f64> for SimpleDistance {
+    impl Distance<i32> for SimpleDistance {
         fn distance(&self, a: &i32, b: &i32) -> f64 {
             (a - b).abs() as f64
         }
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn cover_tree_test() {
         let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9];
@@ -486,7 +488,10 @@ mod tests {
         let knn: Vec<i32> = knn.iter().map(|v| *v.2).collect();
         assert_eq!(vec!(3, 4, 5, 6, 7), knn);
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn cover_tree_test1() {
         let data = vec![
@@ -505,7 +510,10 @@ mod tests {
 
         assert_eq!(vec!(0, 1, 2), knn);
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
@@ -513,7 +521,7 @@ mod tests {
 
         let tree = CoverTree::new(data, SimpleDistance {}).unwrap();
 
-        let deserialized_tree: CoverTree<i32, f64, SimpleDistance> =
+        let deserialized_tree: CoverTree<i32, SimpleDistance> =
             serde_json::from_str(&serde_json::to_string(&tree).unwrap()).unwrap();
 
         assert_eq!(tree, deserialized_tree);

src/algorithm/neighbour/distances.rs

@@ -1,48 +0,0 @@
-//!
-//! Dissimilarities for vector-vector distance
-//!
-//! Representing distances as pairwise dissimilarities, so to build a
-//! graph of closest neighbours. This representation can be reused for
-//! different implementations (initially used in this library for FastPair).
-use std::cmp::{Eq, Ordering, PartialOrd};
-
-#[cfg(feature = "serde")]
-use serde::{Deserialize, Serialize};
-
-use crate::math::num::RealNumber;
-
-///
-/// The edge of the subgraph is defined by `PairwiseDistance`.
-/// The calling algorithm can store a list of distsances as
-/// a list of these structures.
-///
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug, Clone, Copy)]
-pub struct PairwiseDistance<T: RealNumber> {
-    /// index of the vector in the original `Matrix` or list
-    pub node: usize,
-
-    /// index of the closest neighbor in the original `Matrix` or same list
-    pub neighbour: Option<usize>,
-
-    /// measure of distance, according to the algorithm distance function
-    /// if the distance is None, the edge has value "infinite" or max distance
-    /// each algorithm has to match
-    pub distance: Option<T>,
-}
-
-impl<T: RealNumber> Eq for PairwiseDistance<T> {}
-
-impl<T: RealNumber> PartialEq for PairwiseDistance<T> {
-    fn eq(&self, other: &Self) -> bool {
-        self.node == other.node
-            && self.neighbour == other.neighbour
-            && self.distance == other.distance
-    }
-}
-
-impl<T: RealNumber> PartialOrd for PairwiseDistance<T> {
-    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
-        self.distance.partial_cmp(&other.distance)
-    }
-}

src/algorithm/neighbour/fastpair.rs

@@ -1,7 +1,5 @@
-#![allow(non_snake_case)]
-use itertools::Itertools;
 ///
-/// # FastPair: Data-structure for the dynamic closest-pair problem.
+/// ### FastPair: Data-structure for the dynamic closest-pair problem.
 ///
 /// Reference:
 ///  Eppstein, David: Fast hierarchical clustering and other applications of
@@ -9,8 +7,8 @@ use itertools::Itertools;
 ///
 /// Example:
 /// ```
-/// use smartcore::algorithm::neighbour::distances::PairwiseDistance;
-/// use smartcore::linalg::naive::dense_matrix::DenseMatrix;
+/// use smartcore::metrics::distance::PairwiseDistance;
+/// use smartcore::linalg::basic::matrix::DenseMatrix;
 /// use smartcore::algorithm::neighbour::fastpair::FastPair;
 /// let x = DenseMatrix::<f64>::from_2d_array(&[
 ///     &[5.1, 3.5, 1.4, 0.2],
@@ -27,11 +25,14 @@ use itertools::Itertools;
 /// <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 use std::collections::HashMap;
 
-use crate::algorithm::neighbour::distances::PairwiseDistance;
+use num::Bounded;
+
 use crate::error::{Failed, FailedError};
-use crate::linalg::Matrix;
-use crate::math::distance::euclidian::Euclidian;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::metrics::distance::euclidian::Euclidian;
+use crate::metrics::distance::PairwiseDistance;
+use crate::numbers::floatnum::FloatNumber;
+use crate::numbers::realnum::RealNumber;
 
 ///
 /// Inspired by Python implementation:
@@ -41,7 +42,7 @@ use crate::math::num::RealNumber;
 /// affinity used is Euclidean so to allow linkage with single, ward, complete and average
 ///
 #[derive(Debug, Clone)]
-pub struct FastPair<'a, T: RealNumber, M: Matrix<T>> {
+pub struct FastPair<'a, T: RealNumber + FloatNumber, M: Array2<T>> {
     /// initial matrix
     samples: &'a M,
     /// closest pair hashmap (connectivity matrix for closest pairs)
@@ -50,7 +51,7 @@ pub struct FastPair<'a, T: RealNumber, M: Matrix<T>> {
     pub neighbours: Vec<usize>,
 }
 
-impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
+impl<'a, T: RealNumber + FloatNumber, M: Array2<T>> FastPair<'a, T, M> {
     ///
     /// Constructor
     /// Instantiate and inizialise the algorithm
@@ -74,7 +75,7 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
     }
 
     ///
-    /// Initialise `FastPair` by passing a `Matrix`.
+    /// Initialise `FastPair` by passing a `Array2`.
     /// Build a FastPairs data-structure from a set of (new) points.
     ///
     fn init(&mut self) {
@@ -98,8 +99,8 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
                 index_row_i,
                 PairwiseDistance {
                     node: index_row_i,
-                    neighbour: None,
-                    distance: Some(T::max_value()),
+                    neighbour: Option::None,
+                    distance: Some(<T as Bounded>::max_value()),
                 },
             );
         }
@@ -120,13 +121,19 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
                 );
 
                 let d = Euclidian::squared_distance(
-                    &(self.samples.get_row_as_vec(index_row_i)),
-                    &(self.samples.get_row_as_vec(index_row_j)),
+                    &Vec::from_iterator(
+                        self.samples.get_row(index_row_i).iterator(0).copied(),
+                        self.samples.shape().1,
+                    ),
+                    &Vec::from_iterator(
+                        self.samples.get_row(index_row_j).iterator(0).copied(),
+                        self.samples.shape().1,
+                    ),
                 );
-                if d < nbd.unwrap() {
+                if d < nbd.unwrap().to_f64().unwrap() {
                     // set this j-value to be the closest neighbour
                     index_closest = index_row_j;
-                    nbd = Some(d);
+                    nbd = Some(T::from(d).unwrap());
                 }
             }
 
@@ -139,12 +146,12 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
         // No more neighbors, terminate conga line.
         // Last person on the line has no neigbors
         distances.get_mut(&max_index).unwrap().neighbour = Some(max_index);
-        distances.get_mut(&(len - 1)).unwrap().distance = Some(T::max_value());
+        distances.get_mut(&(len - 1)).unwrap().distance = Some(<T as Bounded>::max_value());
 
         // compute sparse matrix (connectivity matrix)
         let mut sparse_matrix = M::zeros(len, len);
         for (_, p) in distances.iter() {
-            sparse_matrix.set(p.node, p.neighbour.unwrap(), p.distance.unwrap());
+            sparse_matrix.set((p.node, p.neighbour.unwrap()), p.distance.unwrap());
         }
 
         self.distances = distances;
@@ -172,32 +179,6 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
         }
     }
 
-    ///
-    /// Brute force algorithm, used only for comparison and testing
-    ///
-    #[cfg(feature = "fp_bench")]
-    pub fn closest_pair_brute(&self) -> PairwiseDistance<T> {
-        let m = self.samples.shape().0;
-
-        let mut closest_pair = PairwiseDistance {
-            node: 0,
-            neighbour: None,
-            distance: Some(T::max_value()),
-        };
-        for pair in (0..m).combinations(2) {
-            let d = Euclidian::squared_distance(
-                &(self.samples.get_row_as_vec(pair[0])),
-                &(self.samples.get_row_as_vec(pair[1])),
-            );
-            if d < closest_pair.distance.unwrap() {
-                closest_pair.node = pair[0];
-                closest_pair.neighbour = Some(pair[1]);
-                closest_pair.distance = Some(d);
-            }
-        }
-        closest_pair
-    }
-
     //
     // Compute distances from input to all other points in data-structure.
     // input is the row index of the sample matrix
@@ -210,10 +191,19 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
                 distances.push(PairwiseDistance {
                     node: index_row,
                     neighbour: Some(*other),
-                    distance: Some(Euclidian::squared_distance(
-                        &(self.samples.get_row_as_vec(index_row)),
-                        &(self.samples.get_row_as_vec(*other)),
-                    )),
+                    distance: Some(
+                        T::from(Euclidian::squared_distance(
+                            &Vec::from_iterator(
+                                self.samples.get_row(index_row).iterator(0).copied(),
+                                self.samples.shape().1,
+                            ),
+                            &Vec::from_iterator(
+                                self.samples.get_row(*other).iterator(0).copied(),
+                                self.samples.shape().1,
+                            ),
+                        ))
+                        .unwrap(),
+                    ),
                 })
             }
         }
@@ -225,7 +215,39 @@ impl<'a, T: RealNumber, M: Matrix<T>> FastPair<'a, T, M> {
 mod tests_fastpair {
 
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::{arrays::Array, matrix::DenseMatrix};
+
+    ///
+    /// Brute force algorithm, used only for comparison and testing
+    ///
+    pub fn closest_pair_brute(fastpair: &FastPair<f64, DenseMatrix<f64>>) -> PairwiseDistance<f64> {
+        use itertools::Itertools;
+        let m = fastpair.samples.shape().0;
+
+        let mut closest_pair = PairwiseDistance {
+            node: 0,
+            neighbour: Option::None,
+            distance: Some(f64::max_value()),
+        };
+        for pair in (0..m).combinations(2) {
+            let d = Euclidian::squared_distance(
+                &Vec::from_iterator(
+                    fastpair.samples.get_row(pair[0]).iterator(0).copied(),
+                    fastpair.samples.shape().1,
+                ),
+                &Vec::from_iterator(
+                    fastpair.samples.get_row(pair[1]).iterator(0).copied(),
+                    fastpair.samples.shape().1,
+                ),
+            );
+            if d < closest_pair.distance.unwrap() {
+                closest_pair.node = pair[0];
+                closest_pair.neighbour = Some(pair[1]);
+                closest_pair.distance = Some(d);
+            }
+        }
+        closest_pair
+    }
 
     #[test]
     fn fastpair_init() {
@@ -284,7 +306,7 @@ mod tests_fastpair {
         };
         assert_eq!(closest_pair, expected_closest_pair);
 
-        let closest_pair_brute = fastpair.closest_pair_brute();
+        let closest_pair_brute = closest_pair_brute(&fastpair);
         assert_eq!(closest_pair_brute, expected_closest_pair);
     }
 
@@ -302,7 +324,7 @@ mod tests_fastpair {
             neighbour: Some(3),
             distance: Some(4.0),
         };
-        assert_eq!(closest_pair, fastpair.closest_pair_brute());
+        assert_eq!(closest_pair, closest_pair_brute(&fastpair));
         assert_eq!(closest_pair, expected_closest_pair);
     }
 
@@ -459,11 +481,16 @@ mod tests_fastpair {
         let expected: HashMap<_, _> = dissimilarities.into_iter().collect();
 
         for i in 0..(x.shape().0 - 1) {
-            let input_node = result.samples.get_row_as_vec(i);
             let input_neighbour: usize = expected.get(&i).unwrap().neighbour.unwrap();
             let distance = Euclidian::squared_distance(
-                &input_node,
-                &result.samples.get_row_as_vec(input_neighbour),
+                &Vec::from_iterator(
+                    result.samples.get_row(i).iterator(0).copied(),
+                    result.samples.shape().1,
+                ),
+                &Vec::from_iterator(
+                    result.samples.get_row(input_neighbour).iterator(0).copied(),
+                    result.samples.shape().1,
+                ),
             );
 
             assert_eq!(i, expected.get(&i).unwrap().node);
@@ -518,7 +545,7 @@ mod tests_fastpair {
         let result = fastpair.unwrap();
 
         let dissimilarity1 = result.closest_pair();
-        let dissimilarity2 = result.closest_pair_brute();
+        let dissimilarity2 = closest_pair_brute(&result);
 
         assert_eq!(dissimilarity1, dissimilarity2);
     }
@@ -550,7 +577,7 @@ mod tests_fastpair {
 
         let mut min_dissimilarity = PairwiseDistance {
             node: 0,
-            neighbour: None,
+            neighbour: Option::None,
             distance: Some(f64::MAX),
         };
         for p in dissimilarities.iter() {

src/algorithm/neighbour/linear_search.rs

@@ -3,12 +3,12 @@
 //! see [KNN algorithms](../index.html)
 //! ```
 //! use smartcore::algorithm::neighbour::linear_search::*;
-//! use smartcore::math::distance::Distance;
+//! use smartcore::metrics::distance::Distance;
 //!
 //! #[derive(Clone)]
 //! struct SimpleDistance {} // Our distance function
 //!
-//! impl Distance<i32, f64> for SimpleDistance {
+//! impl Distance<i32> for SimpleDistance {
 //!   fn distance(&self, a: &i32, b: &i32) -> f64 { // simple simmetrical scalar distance
 //!     (a - b).abs() as f64
 //!   }
@@ -25,38 +25,31 @@
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 use std::cmp::{Ordering, PartialOrd};
-use std::marker::PhantomData;
 
 use crate::algorithm::sort::heap_select::HeapSelection;
 use crate::error::{Failed, FailedError};
-use crate::math::distance::Distance;
-use crate::math::num::RealNumber;
+use crate::metrics::distance::Distance;
 
 /// Implements Linear Search algorithm, see [KNN algorithms](../index.html)
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct LinearKNNSearch<T, F: RealNumber, D: Distance<T, F>> {
+pub struct LinearKNNSearch<T, D: Distance<T>> {
     distance: D,
     data: Vec<T>,
-    f: PhantomData<F>,
 }
 
-impl<T, F: RealNumber, D: Distance<T, F>> LinearKNNSearch<T, F, D> {
+impl<T, D: Distance<T>> LinearKNNSearch<T, D> {
     /// Initializes algorithm.
     /// * `data` - vector of data points to search for.
     /// * `distance` - distance metric to use for searching. This function should extend [`Distance`](../../../math/distance/index.html) interface.
-    pub fn new(data: Vec<T>, distance: D) -> Result<LinearKNNSearch<T, F, D>, Failed> {
-        Ok(LinearKNNSearch {
-            data,
-            distance,
-            f: PhantomData,
-        })
+    pub fn new(data: Vec<T>, distance: D) -> Result<LinearKNNSearch<T, D>, Failed> {
+        Ok(LinearKNNSearch { data, distance })
     }
 
     /// Find k nearest neighbors
     /// * `from` - look for k nearest points to `from`
     /// * `k` - the number of nearest neighbors to return
-    pub fn find(&self, from: &T, k: usize) -> Result<Vec<(usize, F, &T)>, Failed> {
+    pub fn find(&self, from: &T, k: usize) -> Result<Vec<(usize, f64, &T)>, Failed> {
         if k < 1 || k > self.data.len() {
             return Err(Failed::because(
                 FailedError::FindFailed,
@@ -64,11 +57,11 @@ impl<T, F: RealNumber, D: Distance<T, F>> LinearKNNSearch<T, F, D> {
             ));
         }
 
-        let mut heap = HeapSelection::<KNNPoint<F>>::with_capacity(k);
+        let mut heap = HeapSelection::<KNNPoint>::with_capacity(k);
 
         for _ in 0..k {
             heap.add(KNNPoint {
-                distance: F::infinity(),
+                distance: std::f64::INFINITY,
                 index: None,
             });
         }
@@ -93,15 +86,15 @@ impl<T, F: RealNumber, D: Distance<T, F>> LinearKNNSearch<T, F, D> {
     /// Find all nearest neighbors within radius `radius` from `p`
     /// * `p` - look for k nearest points to `p`
     /// * `radius` - radius of the search
-    pub fn find_radius(&self, from: &T, radius: F) -> Result<Vec<(usize, F, &T)>, Failed> {
-        if radius <= F::zero() {
+    pub fn find_radius(&self, from: &T, radius: f64) -> Result<Vec<(usize, f64, &T)>, Failed> {
+        if radius <= 0f64 {
             return Err(Failed::because(
                 FailedError::FindFailed,
                 "radius should be > 0",
             ));
         }
 
-        let mut neighbors: Vec<(usize, F, &T)> = Vec::new();
+        let mut neighbors: Vec<(usize, f64, &T)> = Vec::new();
 
         for i in 0..self.data.len() {
             let d = self.distance.distance(from, &self.data[i]);
@@ -116,41 +109,44 @@ impl<T, F: RealNumber, D: Distance<T, F>> LinearKNNSearch<T, F, D> {
 }
 
 #[derive(Debug)]
-struct KNNPoint<F: RealNumber> {
-    distance: F,
+struct KNNPoint {
+    distance: f64,
     index: Option<usize>,
 }
 
-impl<F: RealNumber> PartialOrd for KNNPoint<F> {
+impl PartialOrd for KNNPoint {
     fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
         self.distance.partial_cmp(&other.distance)
     }
 }
 
-impl<F: RealNumber> PartialEq for KNNPoint<F> {
+impl PartialEq for KNNPoint {
     fn eq(&self, other: &Self) -> bool {
         self.distance == other.distance
     }
 }
 
-impl<F: RealNumber> Eq for KNNPoint<F> {}
+impl Eq for KNNPoint {}
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::math::distance::Distances;
+    use crate::metrics::distance::Distances;
 
     #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
     #[derive(Debug, Clone)]
     struct SimpleDistance {}
 
-    impl Distance<i32, f64> for SimpleDistance {
+    impl Distance<i32> for SimpleDistance {
         fn distance(&self, a: &i32, b: &i32) -> f64 {
             (a - b).abs() as f64
         }
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn knn_find() {
         let data1 = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
@@ -197,7 +193,10 @@ mod tests {
 
         assert_eq!(vec!(1, 2, 3), found_idxs2);
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn knn_point_eq() {
         let point1 = KNNPoint {

src/algorithm/neighbour/mod.rs

@@ -33,16 +33,14 @@
 use crate::algorithm::neighbour::cover_tree::CoverTree;
 use crate::algorithm::neighbour::linear_search::LinearKNNSearch;
 use crate::error::Failed;
-use crate::math::distance::Distance;
-use crate::math::num::RealNumber;
+use crate::metrics::distance::Distance;
+use crate::numbers::basenum::Number;
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 pub(crate) mod bbd_tree;
 /// tree data structure for fast nearest neighbor search
 pub mod cover_tree;
-/// dissimilarities for vector-vector distance. Linkage algorithms used in fastpair
-pub mod distances;
 /// fastpair closest neighbour algorithm
 pub mod fastpair;
 /// very simple algorithm that sequentially checks each element of the list until a match is found or the whole list has been searched.
@@ -59,15 +57,22 @@ pub enum KNNAlgorithmName {
     CoverTree,
 }
 
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug)]
-pub(crate) enum KNNAlgorithm<T: RealNumber, D: Distance<Vec<T>, T>> {
-    LinearSearch(LinearKNNSearch<Vec<T>, T, D>),
-    CoverTree(CoverTree<Vec<T>, T, D>),
+impl Default for KNNAlgorithmName {
+    fn default() -> Self {
+        KNNAlgorithmName::CoverTree
+    }
 }
 
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug)]
+pub(crate) enum KNNAlgorithm<T: Number, D: Distance<Vec<T>>> {
+    LinearSearch(LinearKNNSearch<Vec<T>, D>),
+    CoverTree(CoverTree<Vec<T>, D>),
+}
+
+// TODO: missing documentation
 impl KNNAlgorithmName {
-    pub(crate) fn fit<T: RealNumber, D: Distance<Vec<T>, T>>(
+    pub(crate) fn fit<T: Number, D: Distance<Vec<T>>>(
         &self,
         data: Vec<Vec<T>>,
         distance: D,
@@ -83,8 +88,8 @@ impl KNNAlgorithmName {
     }
 }
 
-impl<T: RealNumber, D: Distance<Vec<T>, T>> KNNAlgorithm<T, D> {
-    pub fn find(&self, from: &Vec<T>, k: usize) -> Result<Vec<(usize, T, &Vec<T>)>, Failed> {
+impl<T: Number, D: Distance<Vec<T>>> KNNAlgorithm<T, D> {
+    pub fn find(&self, from: &Vec<T>, k: usize) -> Result<Vec<(usize, f64, &Vec<T>)>, Failed> {
         match *self {
             KNNAlgorithm::LinearSearch(ref linear) => linear.find(from, k),
             KNNAlgorithm::CoverTree(ref cover) => cover.find(from, k),
@@ -94,8 +99,8 @@ impl<T: RealNumber, D: Distance<Vec<T>, T>> KNNAlgorithm<T, D> {
     pub fn find_radius(
         &self,
         from: &Vec<T>,
-        radius: T,
-    ) -> Result<Vec<(usize, T, &Vec<T>)>, Failed> {
+        radius: f64,
+    ) -> Result<Vec<(usize, f64, &Vec<T>)>, Failed> {
         match *self {
             KNNAlgorithm::LinearSearch(ref linear) => linear.find_radius(from, radius),
             KNNAlgorithm::CoverTree(ref cover) => cover.find_radius(from, radius),

src/algorithm/sort/heap_select.rs

@@ -95,14 +95,20 @@ impl<T: PartialOrd + Debug> HeapSelection<T> {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn with_capacity() {
         let heap = HeapSelection::<i32>::with_capacity(3);
         assert_eq!(3, heap.k);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_add() {
         let mut heap = HeapSelection::with_capacity(3);
@@ -120,7 +126,10 @@ mod tests {
         assert_eq!(vec![2, 0, -5], heap.get());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_add1() {
         let mut heap = HeapSelection::with_capacity(3);
@@ -135,7 +144,10 @@ mod tests {
         assert_eq!(vec![0f64, -1f64, -5f64], heap.get());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_add2() {
         let mut heap = HeapSelection::with_capacity(3);
@@ -148,7 +160,10 @@ mod tests {
         assert_eq!(vec![5.6568, 2.8284, 0.0], heap.get());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_add_ordered() {
         let mut heap = HeapSelection::with_capacity(3);

src/algorithm/sort/quick_sort.rs

@@ -1,4 +1,4 @@
-use num_traits::Float;
+use num_traits::Num;
 
 pub trait QuickArgSort {
     fn quick_argsort_mut(&mut self) -> Vec<usize>;
@@ -6,7 +6,7 @@ pub trait QuickArgSort {
     fn quick_argsort(&self) -> Vec<usize>;
 }
 
-impl<T: Float> QuickArgSort for Vec<T> {
+impl<T: Num + PartialOrd + Copy> QuickArgSort for Vec<T> {
     fn quick_argsort(&self) -> Vec<usize> {
         let mut v = self.clone();
         v.quick_argsort_mut()
@@ -113,7 +113,10 @@ impl<T: Float> QuickArgSort for Vec<T> {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn with_capacity() {
         let arr1 = vec![0.3, 0.1, 0.2, 0.4, 0.9, 0.5, 0.7, 0.6, 0.8];

src/api.rs

@@ -16,8 +16,12 @@ pub trait UnsupervisedEstimator<X, P> {
         P: Clone;
 }
 
-/// An estimator for supervised learning, , that provides method `fit` to learn from data and training values
-pub trait SupervisedEstimator<X, Y, P> {
+/// An estimator for supervised learning, that provides method `fit` to learn from data and training values
+pub trait SupervisedEstimator<X, Y, P>: Predictor<X, Y> {
+    /// Empty constructor, instantiate an empty estimator. Object is dropped as soon as `fit()` is called.
+    /// used to pass around the correct `fit()` implementation.
+    /// by calling `::fit()`. mostly used to be used with `model_selection::cross_validate(...)`
+    fn new() -> Self;
     /// Fit a model to a training dataset, estimate model's parameters.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - target training values of size _N_.
@@ -28,6 +32,24 @@ pub trait SupervisedEstimator<X, Y, P> {
         P: Clone;
 }
 
+/// An estimator for supervised learning.
+/// In this one parameters are borrowed instead of moved, this is useful for parameters that carry
+/// references. Also to be used when there is no predictor attached to the estimator.
+pub trait SupervisedEstimatorBorrow<'a, X, Y, P> {
+    /// Empty constructor, instantiate an empty estimator. Object is dropped as soon as `fit()` is called.
+    /// used to pass around the correct `fit()` implementation.
+    /// by calling `::fit()`. mostly used to be used with `model_selection::cross_validate(...)`
+    fn new() -> Self;
+    /// Fit a model to a training dataset, estimate model's parameters.
+    /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
+    /// * `y` - target training values of size _N_.
+    /// * `&parameters` - hyperparameters of an algorithm
+    fn fit(x: &'a X, y: &'a Y, parameters: &'a P) -> Result<Self, Failed>
+    where
+        Self: Sized,
+        P: Clone;
+}
+
 /// Implements method predict that estimates target value from new data
 pub trait Predictor<X, Y> {
     /// Estimate target values from new data.
@@ -35,9 +57,19 @@ pub trait Predictor<X, Y> {
     fn predict(&self, x: &X) -> Result<Y, Failed>;
 }
 
+/// Implements method predict that estimates target value from new data, with borrowing
+pub trait PredictorBorrow<'a, X, T> {
+    /// Estimate target values from new data.
+    /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
+    fn predict(&self, x: &'a X) -> Result<Vec<T>, Failed>;
+}
+
 /// Implements method transform that filters or modifies input data
 pub trait Transformer<X> {
     /// Transform data by modifying or filtering it
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     fn transform(&self, x: &X) -> Result<X, Failed>;
 }
+
+/// empty parameters for an estimator, see `BiasedEstimator`
+pub trait NoParameters {}

src/cluster/dbscan.rs

@@ -19,18 +19,19 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::basic::arrays::Array2;
 //! use smartcore::cluster::dbscan::*;
-//! use smartcore::math::distance::Distances;
+//! use smartcore::metrics::distance::Distances;
 //! use smartcore::neighbors::KNNAlgorithmName;
 //! use smartcore::dataset::generator;
 //!
 //! // Generate three blobs
 //! let blobs = generator::make_blobs(100, 2, 3);
-//! let x = DenseMatrix::from_vec(blobs.num_samples, blobs.num_features, &blobs.data);
+//! let x: DenseMatrix<f32> = DenseMatrix::from_iterator(blobs.data.into_iter(), 100, 2, 0);
 //! // Fit the algorithm and predict cluster labels
-//! let labels = DBSCAN::fit(&x, DBSCANParameters::default().with_eps(3.0)).
-//!     and_then(|dbscan| dbscan.predict(&x));
+//! let labels: Vec<u32> = DBSCAN::fit(&x, DBSCANParameters::default().with_eps(3.0)).
+//!     and_then(|dbscan| dbscan.predict(&x)).unwrap();
 //!
 //! println!("{:?}", labels);
 //! ```
@@ -41,7 +42,7 @@
 //! * ["Density-Based Clustering in Spatial Databases: The Algorithm GDBSCAN and its Applications", Sander J., Ester M., Kriegel HP., Xu X.](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.63.1629&rep=rep1&type=pdf)
 
 use std::fmt::Debug;
-use std::iter::Sum;
+use std::marker::PhantomData;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
@@ -49,47 +50,58 @@ use serde::{Deserialize, Serialize};
 use crate::algorithm::neighbour::{KNNAlgorithm, KNNAlgorithmName};
 use crate::api::{Predictor, UnsupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::{row_iter, Matrix};
-use crate::math::distance::euclidian::Euclidian;
-use crate::math::distance::{Distance, Distances};
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::metrics::distance::euclidian::Euclidian;
+use crate::metrics::distance::{Distance, Distances};
+use crate::numbers::basenum::Number;
 use crate::tree::decision_tree_classifier::which_max;
 
 /// DBSCAN clustering algorithm
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct DBSCAN<T: RealNumber, D: Distance<Vec<T>, T>> {
+pub struct DBSCAN<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>, D: Distance<Vec<TX>>> {
     cluster_labels: Vec<i16>,
     num_classes: usize,
-    knn_algorithm: KNNAlgorithm<T, D>,
-    eps: T,
+    knn_algorithm: KNNAlgorithm<TX, D>,
+    eps: f64,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_x: PhantomData<X>,
+    _phantom_y: PhantomData<Y>,
 }
 
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 /// DBSCAN clustering algorithm parameters
-pub struct DBSCANParameters<T: RealNumber, D: Distance<Vec<T>, T>> {
+pub struct DBSCANParameters<T: Number, D: Distance<Vec<T>>> {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// a function that defines a distance between each pair of point in training data.
     /// This function should extend [`Distance`](../../math/distance/trait.Distance.html) trait.
     /// See [`Distances`](../../math/distance/struct.Distances.html) for a list of available functions.
     pub distance: D,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The number of samples (or total weight) in a neighborhood for a point to be considered as a core point.
     pub min_samples: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The maximum distance between two samples for one to be considered as in the neighborhood of the other.
-    pub eps: T,
+    pub eps: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// KNN algorithm to use.
     pub algorithm: KNNAlgorithmName,
+    #[cfg_attr(feature = "serde", serde(default))]
+    _phantom_t: PhantomData<T>,
 }
 
-impl<T: RealNumber, D: Distance<Vec<T>, T>> DBSCANParameters<T, D> {
+impl<T: Number, D: Distance<Vec<T>>> DBSCANParameters<T, D> {
     /// a function that defines a distance between each pair of point in training data.
     /// This function should extend [`Distance`](../../math/distance/trait.Distance.html) trait.
     /// See [`Distances`](../../math/distance/struct.Distances.html) for a list of available functions.
-    pub fn with_distance<DD: Distance<Vec<T>, T>>(self, distance: DD) -> DBSCANParameters<T, DD> {
+    pub fn with_distance<DD: Distance<Vec<T>>>(self, distance: DD) -> DBSCANParameters<T, DD> {
         DBSCANParameters {
             distance,
             min_samples: self.min_samples,
             eps: self.eps,
             algorithm: self.algorithm,
+            _phantom_t: PhantomData,
         }
     }
     /// The number of samples (or total weight) in a neighborhood for a point to be considered as a core point.
@@ -98,7 +110,7 @@ impl<T: RealNumber, D: Distance<Vec<T>, T>> DBSCANParameters<T, D> {
         self
     }
     /// The maximum distance between two samples for one to be considered as in the neighborhood of the other.
-    pub fn with_eps(mut self, eps: T) -> Self {
+    pub fn with_eps(mut self, eps: f64) -> Self {
         self.eps = eps;
         self
     }
@@ -109,7 +121,113 @@ impl<T: RealNumber, D: Distance<Vec<T>, T>> DBSCANParameters<T, D> {
     }
 }
 
-impl<T: RealNumber, D: Distance<Vec<T>, T>> PartialEq for DBSCAN<T, D> {
+/// DBSCAN grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct DBSCANSearchParameters<T: Number, D: Distance<Vec<T>>> {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// a function that defines a distance between each pair of point in training data.
+    /// This function should extend [`Distance`](../../math/distance/trait.Distance.html) trait.
+    /// See [`Distances`](../../math/distance/struct.Distances.html) for a list of available functions.
+    pub distance: Vec<D>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The number of samples (or total weight) in a neighborhood for a point to be considered as a core point.
+    pub min_samples: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The maximum distance between two samples for one to be considered as in the neighborhood of the other.
+    pub eps: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// KNN algorithm to use.
+    pub algorithm: Vec<KNNAlgorithmName>,
+    _phantom_t: PhantomData<T>,
+}
+
+/// DBSCAN grid search iterator
+pub struct DBSCANSearchParametersIterator<T: Number, D: Distance<Vec<T>>> {
+    dbscan_search_parameters: DBSCANSearchParameters<T, D>,
+    current_distance: usize,
+    current_min_samples: usize,
+    current_eps: usize,
+    current_algorithm: usize,
+}
+
+impl<T: Number, D: Distance<Vec<T>>> IntoIterator for DBSCANSearchParameters<T, D> {
+    type Item = DBSCANParameters<T, D>;
+    type IntoIter = DBSCANSearchParametersIterator<T, D>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        DBSCANSearchParametersIterator {
+            dbscan_search_parameters: self,
+            current_distance: 0,
+            current_min_samples: 0,
+            current_eps: 0,
+            current_algorithm: 0,
+        }
+    }
+}
+
+impl<T: Number, D: Distance<Vec<T>>> Iterator for DBSCANSearchParametersIterator<T, D> {
+    type Item = DBSCANParameters<T, D>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_distance == self.dbscan_search_parameters.distance.len()
+            && self.current_min_samples == self.dbscan_search_parameters.min_samples.len()
+            && self.current_eps == self.dbscan_search_parameters.eps.len()
+            && self.current_algorithm == self.dbscan_search_parameters.algorithm.len()
+        {
+            return None;
+        }
+
+        let next = DBSCANParameters {
+            distance: self.dbscan_search_parameters.distance[self.current_distance].clone(),
+            min_samples: self.dbscan_search_parameters.min_samples[self.current_min_samples],
+            eps: self.dbscan_search_parameters.eps[self.current_eps],
+            algorithm: self.dbscan_search_parameters.algorithm[self.current_algorithm].clone(),
+            _phantom_t: PhantomData,
+        };
+
+        if self.current_distance + 1 < self.dbscan_search_parameters.distance.len() {
+            self.current_distance += 1;
+        } else if self.current_min_samples + 1 < self.dbscan_search_parameters.min_samples.len() {
+            self.current_distance = 0;
+            self.current_min_samples += 1;
+        } else if self.current_eps + 1 < self.dbscan_search_parameters.eps.len() {
+            self.current_distance = 0;
+            self.current_min_samples = 0;
+            self.current_eps += 1;
+        } else if self.current_algorithm + 1 < self.dbscan_search_parameters.algorithm.len() {
+            self.current_distance = 0;
+            self.current_min_samples = 0;
+            self.current_eps = 0;
+            self.current_algorithm += 1;
+        } else {
+            self.current_distance += 1;
+            self.current_min_samples += 1;
+            self.current_eps += 1;
+            self.current_algorithm += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl<T: Number> Default for DBSCANSearchParameters<T, Euclidian<T>> {
+    fn default() -> Self {
+        let default_params = DBSCANParameters::default();
+
+        DBSCANSearchParameters {
+            distance: vec![default_params.distance],
+            min_samples: vec![default_params.min_samples],
+            eps: vec![default_params.eps],
+            algorithm: vec![default_params.algorithm],
+            _phantom_t: PhantomData,
+        }
+    }
+}
+
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>, D: Distance<Vec<TX>>> PartialEq
+    for DBSCAN<TX, TY, X, Y, D>
+{
     fn eq(&self, other: &Self) -> bool {
         self.cluster_labels.len() == other.cluster_labels.len()
             && self.num_classes == other.num_classes
@@ -118,47 +236,50 @@ impl<T: RealNumber, D: Distance<Vec<T>, T>> PartialEq for DBSCAN<T, D> {
     }
 }
 
-impl<T: RealNumber> Default for DBSCANParameters<T, Euclidian> {
+impl<T: Number> Default for DBSCANParameters<T, Euclidian<T>> {
     fn default() -> Self {
         DBSCANParameters {
             distance: Distances::euclidian(),
             min_samples: 5,
-            eps: T::half(),
-            algorithm: KNNAlgorithmName::CoverTree,
+            eps: 0.5f64,
+            algorithm: KNNAlgorithmName::default(),
+            _phantom_t: PhantomData,
         }
     }
 }
 
-impl<T: RealNumber + Sum, M: Matrix<T>, D: Distance<Vec<T>, T>>
-    UnsupervisedEstimator<M, DBSCANParameters<T, D>> for DBSCAN<T, D>
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>, D: Distance<Vec<TX>>>
+    UnsupervisedEstimator<X, DBSCANParameters<TX, D>> for DBSCAN<TX, TY, X, Y, D>
 {
-    fn fit(x: &M, parameters: DBSCANParameters<T, D>) -> Result<Self, Failed> {
+    fn fit(x: &X, parameters: DBSCANParameters<TX, D>) -> Result<Self, Failed> {
         DBSCAN::fit(x, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>, D: Distance<Vec<T>, T>> Predictor<M, M::RowVector>
-    for DBSCAN<T, D>
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>, D: Distance<Vec<TX>>> Predictor<X, Y>
+    for DBSCAN<TX, TY, X, Y, D>
 {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber + Sum, D: Distance<Vec<T>, T>> DBSCAN<T, D> {
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>, D: Distance<Vec<TX>>>
+    DBSCAN<TX, TY, X, Y, D>
+{
     /// Fit algorithm to _NxM_ matrix where _N_ is number of samples and _M_ is number of features.
     /// * `data` - training instances to cluster
     /// * `k` - number of clusters
     /// * `parameters` - cluster parameters
-    pub fn fit<M: Matrix<T>>(
-        x: &M,
-        parameters: DBSCANParameters<T, D>,
-    ) -> Result<DBSCAN<T, D>, Failed> {
+    pub fn fit(
+        x: &X,
+        parameters: DBSCANParameters<TX, D>,
+    ) -> Result<DBSCAN<TX, TY, X, Y, D>, Failed> {
         if parameters.min_samples < 1 {
             return Err(Failed::fit("Invalid minPts"));
         }
 
-        if parameters.eps <= T::zero() {
+        if parameters.eps <= 0f64 {
             return Err(Failed::fit("Invalid radius: "));
         }
 
@@ -170,13 +291,19 @@ impl<T: RealNumber + Sum, D: Distance<Vec<T>, T>> DBSCAN<T, D> {
         let n = x.shape().0;
         let mut y = vec![undefined; n];
 
-        let algo = parameters
-            .algorithm
-            .fit(row_iter(x).collect(), parameters.distance)?;
+        let algo = parameters.algorithm.fit(
+            x.row_iter()
+                .map(|row| row.iterator(0).cloned().collect())
+                .collect(),
+            parameters.distance,
+        )?;
 
-        for (i, e) in row_iter(x).enumerate() {
+        let mut row = vec![TX::zero(); x.shape().1];
+
+        for (i, e) in x.row_iter().enumerate() {
             if y[i] == undefined {
-                let mut neighbors = algo.find_radius(&e, parameters.eps)?;
+                e.iterator(0).zip(row.iter_mut()).for_each(|(&x, r)| *r = x);
+                let mut neighbors = algo.find_radius(&row, parameters.eps)?;
                 if neighbors.len() < parameters.min_samples {
                     y[i] = outlier;
                 } else {
@@ -227,18 +354,25 @@ impl<T: RealNumber + Sum, D: Distance<Vec<T>, T>> DBSCAN<T, D> {
             num_classes: k as usize,
             knn_algorithm: algo,
             eps: parameters.eps,
+            _phantom_ty: PhantomData,
+            _phantom_x: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
     /// Predict clusters for `x`
     /// * `x` - matrix with new data to transform of size _KxM_ , where _K_ is number of new samples and _M_ is number of features.
-    pub fn predict<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
-        let (n, m) = x.shape();
-        let mut result = M::zeros(1, n);
-        let mut row = vec![T::zero(); m];
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
+        let (n, _) = x.shape();
+        let mut result = Y::zeros(n);
+
+        let mut row = vec![TX::zero(); x.shape().1];
 
         for i in 0..n {
-            x.copy_row_as_vec(i, &mut row);
+            x.get_row(i)
+                .iterator(0)
+                .zip(row.iter_mut())
+                .for_each(|(&x, r)| *r = x);
             let neighbors = self.knn_algorithm.find_radius(&row, self.eps)?;
             let mut label = vec![0usize; self.num_classes + 1];
             for neighbor in neighbors {
@@ -251,24 +385,50 @@ impl<T: RealNumber + Sum, D: Distance<Vec<T>, T>> DBSCAN<T, D> {
             }
             let class = which_max(&label);
             if class != self.num_classes {
-                result.set(0, i, T::from(class).unwrap());
+                result.set(i, TY::from(class + 1).unwrap());
             } else {
-                result.set(0, i, -T::one());
+                result.set(i, TY::zero());
             }
         }
 
-        Ok(result.to_row_vector())
+        Ok(result)
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
     #[cfg(feature = "serde")]
-    use crate::math::distance::euclidian::Euclidian;
+    use crate::metrics::distance::euclidian::Euclidian;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters: DBSCANSearchParameters<f64, Euclidian<f64>> = DBSCANSearchParameters {
+            min_samples: vec![10, 100],
+            eps: vec![1., 2.],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.min_samples, 10);
+        assert_eq!(next.eps, 1.);
+        let next = iter.next().unwrap();
+        assert_eq!(next.min_samples, 100);
+        assert_eq!(next.eps, 1.);
+        let next = iter.next().unwrap();
+        assert_eq!(next.min_samples, 10);
+        assert_eq!(next.eps, 2.);
+        let next = iter.next().unwrap();
+        assert_eq!(next.min_samples, 100);
+        assert_eq!(next.eps, 2.);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn fit_predict_dbscan() {
         let x = DenseMatrix::from_2d_array(&[
@@ -285,7 +445,7 @@ mod tests {
             &[3.0, 5.0],
         ]);
 
-        let expected_labels = vec![0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, -1.0];
+        let expected_labels = vec![1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 0];
 
         let dbscan = DBSCAN::fit(
             &x,
@@ -295,12 +455,15 @@ mod tests {
         )
         .unwrap();
 
-        let predicted_labels = dbscan.predict(&x).unwrap();
+        let predicted_labels: Vec<i32> = dbscan.predict(&x).unwrap();
 
         assert_eq!(expected_labels, predicted_labels);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
@@ -329,9 +492,25 @@ mod tests {
 
         let dbscan = DBSCAN::fit(&x, Default::default()).unwrap();
 
-        let deserialized_dbscan: DBSCAN<f64, Euclidian> =
+        let deserialized_dbscan: DBSCAN<f32, f32, DenseMatrix<f32>, Vec<f32>, Euclidian<f32>> =
             serde_json::from_str(&serde_json::to_string(&dbscan).unwrap()).unwrap();
 
         assert_eq!(dbscan, deserialized_dbscan);
     }
+
+    #[cfg(feature = "datasets")]
+    #[test]
+    fn from_vec() {
+        use crate::dataset::generator;
+
+        // Generate three blobs
+        let blobs = generator::make_blobs(100, 2, 3);
+        let x: DenseMatrix<f32> = DenseMatrix::from_iterator(blobs.data.into_iter(), 100, 2, 0);
+        // Fit the algorithm and predict cluster labels
+        let labels: Vec<i32> = DBSCAN::fit(&x, DBSCANParameters::default().with_eps(3.0))
+            .and_then(|dbscan| dbscan.predict(&x))
+            .unwrap();
+
+        println!("{:?}", labels);
+    }
 }

src/cluster/hierarchical.rs

@@ -1,63 +0,0 @@
-/// # Hierarchical clustering
-/// 
-/// Implement hierarchical clustering methods:
-/// * Agglomerative clustering (current)
-/// * Bisecting K-Means (future)
-/// * Fastcluster (future)
-/// 
-
-/* 
-class AgglomerativeClustering():
-    """
-    Parameters
-    ----------
-    n_clusters : int or None, default=2
-        The number of clusters to find. It must be ``None`` if
-        ``distance_threshold`` is not ``None``.
-    affinity : str or callable, default='euclidean'
-        If linkage is "ward", only "euclidean" is accepted.
-    linkage : {'ward',}, default='ward'
-        Which linkage criterion to use. The linkage criterion determines which
-        distance to use between sets of observation. The algorithm will merge
-        the pairs of cluster that minimize this criterion.
-        - 'ward' minimizes the variance of the clusters being merged.
-    compute_distances : bool, default=False
-        Computes distances between clusters even if `distance_threshold` is not
-        used. This can be used to make dendrogram visualization, but introduces
-        a computational and memory overhead.
-    """
-
-    def fit(X):
-        # compute tree
-        # <https://github.com/scikit-learn/scikit-learn/blob/02ebf9e68fe1fc7687d9e1047b9e465ae0fd945e/sklearn/cluster/_agglomerative.py#L172>
-        parents, childern = ward_tree(X, ....)
-        # compute clusters
-        # <https://github.com/scikit-learn/scikit-learn/blob/70c495250fea7fa3c8c1a4631e6ddcddc9f22451/sklearn/cluster/_hierarchical_fast.pyx#L98>
-        labels = _hierarchical.hc_get_heads(parents)
-        # assign cluster numbers
-        self.labels_ = np.searchsorted(np.unique(labels), labels)
-
-*/
-
-// implement ward tree
-// use scipy.cluster.hierarchy.ward
-// <https://github.com/scipy/scipy/blob/main/scipy/cluster/hierarchy.py#L738>
-// use linkage
-// <https://github.com/scipy/scipy/blob/main/scipy/cluster/hierarchy.py#L837>
-// use nn_chain
-// <https://github.com/scipy/scipy/blob/main/scipy/cluster/_hierarchy.pyx#L906>
-
-// implement hc_get_heads
-
-
-mod tests {
-    // >>> from sklearn.cluster import AgglomerativeClustering
-    // >>> import numpy as np
-    // >>> X = np.array([[1, 2], [1, 4], [1, 0],
-    // ...               [4, 2], [4, 4], [4, 0]])
-    // >>> clustering = AgglomerativeClustering().fit(X)
-    // >>> clustering
-    // AgglomerativeClustering()
-    // >>> clustering.labels_
-    // array([1, 1, 1, 0, 0, 0])
-}

src/cluster/kmeans.rs

@@ -11,12 +11,12 @@
 //! these re-calculated centroids becoming the new centers of their respective clusters. Next all instances of the training set are re-assigned to their closest cluster again.
 //! This iterative process continues until convergence is achieved and the clusters are considered settled.
 //!
-//! Initial choice of K data points is very important and has big effect on performance of the algorithm. SmartCore uses k-means++ algorithm to initialize cluster centers.
+//! Initial choice of K data points is very important and has big effect on performance of the algorithm. `smartcore` uses k-means++ algorithm to initialize cluster centers.
 //!
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::cluster::kmeans::*;
 //!
 //! // Iris data
@@ -44,7 +44,7 @@
 //!            ]);
 //!
 //! let kmeans = KMeans::fit(&x, KMeansParameters::default().with_k(2)).unwrap(); // Fit to data, 2 clusters
-//! let y_hat = kmeans.predict(&x).unwrap(); // use the same points for prediction
+//! let y_hat: Vec<u8> = kmeans.predict(&x).unwrap(); // use the same points for prediction
 //! ```
 //!
 //! ## References:
@@ -52,32 +52,37 @@
 //! * ["An Introduction to Statistical Learning", James G., Witten D., Hastie T., Tibshirani R., 10.3.1 K-Means Clustering](http://faculty.marshall.usc.edu/gareth-james/ISL/)
 //! * ["k-means++: The Advantages of Careful Seeding", Arthur D., Vassilvitskii S.](http://ilpubs.stanford.edu:8090/778/1/2006-13.pdf)
 
-use rand::Rng;
 use std::fmt::Debug;
-use std::iter::Sum;
+use std::marker::PhantomData;
 
+use rand::Rng;
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 use crate::algorithm::neighbour::bbd_tree::BBDTree;
 use crate::api::{Predictor, UnsupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::Matrix;
-use crate::math::distance::euclidian::*;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::metrics::distance::euclidian::*;
+use crate::numbers::basenum::Number;
+use crate::rand_custom::get_rng_impl;
 
 /// K-Means clustering algorithm
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct KMeans<T: RealNumber> {
+pub struct KMeans<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>> {
     k: usize,
     _y: Vec<usize>,
     size: Vec<usize>,
-    _distortion: T,
-    centroids: Vec<Vec<T>>,
+    _distortion: f64,
+    centroids: Vec<Vec<f64>>,
+    _phantom_tx: PhantomData<TX>,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_x: PhantomData<X>,
+    _phantom_y: PhantomData<Y>,
 }
 
-impl<T: RealNumber> PartialEq for KMeans<T> {
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>> PartialEq for KMeans<TX, TY, X, Y> {
     fn eq(&self, other: &Self) -> bool {
         if self.k != other.k
             || self.size != other.size
@@ -91,7 +96,7 @@ impl<T: RealNumber> PartialEq for KMeans<T> {
                     return false;
                 }
                 for j in 0..self.centroids[i].len() {
-                    if (self.centroids[i][j] - other.centroids[i][j]).abs() > T::epsilon() {
+                    if (self.centroids[i][j] - other.centroids[i][j]).abs() > std::f64::EPSILON {
                         return false;
                     }
                 }
@@ -101,13 +106,20 @@ impl<T: RealNumber> PartialEq for KMeans<T> {
     }
 }
 
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 /// K-Means clustering algorithm parameters
 pub struct KMeansParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Number of clusters.
     pub k: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Maximum number of iterations of the k-means algorithm for a single run.
     pub max_iter: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Determines random number generation for centroid initialization.
+    /// Use an int to make the randomness deterministic
+    pub seed: Option<u64>,
 }
 
 impl KMeansParameters {
@@ -128,27 +140,118 @@ impl Default for KMeansParameters {
         KMeansParameters {
             k: 2,
             max_iter: 100,
+            seed: Option::None,
         }
     }
 }
 
-impl<T: RealNumber + Sum, M: Matrix<T>> UnsupervisedEstimator<M, KMeansParameters> for KMeans<T> {
-    fn fit(x: &M, parameters: KMeansParameters) -> Result<Self, Failed> {
+/// KMeans grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct KMeansSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Number of clusters.
+    pub k: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Maximum number of iterations of the k-means algorithm for a single run.
+    pub max_iter: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Determines random number generation for centroid initialization.
+    /// Use an int to make the randomness deterministic
+    pub seed: Vec<Option<u64>>,
+}
+
+/// KMeans grid search iterator
+pub struct KMeansSearchParametersIterator {
+    kmeans_search_parameters: KMeansSearchParameters,
+    current_k: usize,
+    current_max_iter: usize,
+    current_seed: usize,
+}
+
+impl IntoIterator for KMeansSearchParameters {
+    type Item = KMeansParameters;
+    type IntoIter = KMeansSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        KMeansSearchParametersIterator {
+            kmeans_search_parameters: self,
+            current_k: 0,
+            current_max_iter: 0,
+            current_seed: 0,
+        }
+    }
+}
+
+impl Iterator for KMeansSearchParametersIterator {
+    type Item = KMeansParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_k == self.kmeans_search_parameters.k.len()
+            && self.current_max_iter == self.kmeans_search_parameters.max_iter.len()
+            && self.current_seed == self.kmeans_search_parameters.seed.len()
+        {
+            return None;
+        }
+
+        let next = KMeansParameters {
+            k: self.kmeans_search_parameters.k[self.current_k],
+            max_iter: self.kmeans_search_parameters.max_iter[self.current_max_iter],
+            seed: self.kmeans_search_parameters.seed[self.current_seed],
+        };
+
+        if self.current_k + 1 < self.kmeans_search_parameters.k.len() {
+            self.current_k += 1;
+        } else if self.current_max_iter + 1 < self.kmeans_search_parameters.max_iter.len() {
+            self.current_k = 0;
+            self.current_max_iter += 1;
+        } else if self.current_seed + 1 < self.kmeans_search_parameters.seed.len() {
+            self.current_k = 0;
+            self.current_max_iter = 0;
+            self.current_seed += 1;
+        } else {
+            self.current_k += 1;
+            self.current_max_iter += 1;
+            self.current_seed += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for KMeansSearchParameters {
+    fn default() -> Self {
+        let default_params = KMeansParameters::default();
+
+        KMeansSearchParameters {
+            k: vec![default_params.k],
+            max_iter: vec![default_params.max_iter],
+            seed: vec![default_params.seed],
+        }
+    }
+}
+
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    UnsupervisedEstimator<X, KMeansParameters> for KMeans<TX, TY, X, Y>
+{
+    fn fit(x: &X, parameters: KMeansParameters) -> Result<Self, Failed> {
         KMeans::fit(x, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for KMeans<T> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>> Predictor<X, Y>
+    for KMeans<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber + Sum> KMeans<T> {
+impl<TX: Number, TY: Number, X: Array2<TX>, Y: Array1<TY>> KMeans<TX, TY, X, Y> {
     /// Fit algorithm to _NxM_ matrix where _N_ is number of samples and _M_ is number of features.
     /// * `data` - training instances to cluster    
     /// * `parameters` - cluster parameters
-    pub fn fit<M: Matrix<T>>(data: &M, parameters: KMeansParameters) -> Result<KMeans<T>, Failed> {
+    pub fn fit(data: &X, parameters: KMeansParameters) -> Result<KMeans<TX, TY, X, Y>, Failed> {
         let bbd = BBDTree::new(data);
 
         if parameters.k < 2 {
@@ -167,10 +270,10 @@ impl<T: RealNumber + Sum> KMeans<T> {
 
         let (n, d) = data.shape();
 
-        let mut distortion = T::max_value();
-        let mut y = KMeans::kmeans_plus_plus(data, parameters.k);
+        let mut distortion = std::f64::MAX;
+        let mut y = KMeans::<TX, TY, X, Y>::kmeans_plus_plus(data, parameters.k, parameters.seed);
         let mut size = vec![0; parameters.k];
-        let mut centroids = vec![vec![T::zero(); d]; parameters.k];
+        let mut centroids = vec![vec![0f64; d]; parameters.k];
 
         for i in 0..n {
             size[y[i]] += 1;
@@ -178,23 +281,23 @@ impl<T: RealNumber + Sum> KMeans<T> {
 
         for i in 0..n {
             for j in 0..d {
-                centroids[y[i]][j] += data.get(i, j);
+                centroids[y[i]][j] += data.get((i, j)).to_f64().unwrap();
             }
         }
 
         for i in 0..parameters.k {
             for j in 0..d {
-                centroids[i][j] /= T::from(size[i]).unwrap();
+                centroids[i][j] /= size[i] as f64;
             }
         }
 
-        let mut sums = vec![vec![T::zero(); d]; parameters.k];
+        let mut sums = vec![vec![0f64; d]; parameters.k];
         for _ in 1..=parameters.max_iter {
             let dist = bbd.clustering(&centroids, &mut sums, &mut size, &mut y);
             for i in 0..parameters.k {
                 if size[i] > 0 {
                     for j in 0..d {
-                        centroids[i][j] = T::from(sums[i][j]).unwrap() / T::from(size[i]).unwrap();
+                        centroids[i][j] = sums[i][j] / size[i] as f64;
                     }
                 }
             }
@@ -212,48 +315,61 @@ impl<T: RealNumber + Sum> KMeans<T> {
             size,
             _distortion: distortion,
             centroids,
+            _phantom_tx: PhantomData,
+            _phantom_ty: PhantomData,
+            _phantom_x: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
     /// Predict clusters for `x`
     /// * `x` - matrix with new data to transform of size _KxM_ , where _K_ is number of new samples and _M_ is number of features.
-    pub fn predict<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
-        let (n, m) = x.shape();
-        let mut result = M::zeros(1, n);
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
+        let (n, _) = x.shape();
+        let mut result = Y::zeros(n);
 
-        let mut row = vec![T::zero(); m];
+        let mut row = vec![0f64; x.shape().1];
 
         for i in 0..n {
-            let mut min_dist = T::max_value();
+            let mut min_dist = std::f64::MAX;
             let mut best_cluster = 0;
 
             for j in 0..self.k {
-                x.copy_row_as_vec(i, &mut row);
+                x.get_row(i)
+                    .iterator(0)
+                    .zip(row.iter_mut())
+                    .for_each(|(&x, r)| *r = x.to_f64().unwrap());
                 let dist = Euclidian::squared_distance(&row, &self.centroids[j]);
                 if dist < min_dist {
                     min_dist = dist;
                     best_cluster = j;
                 }
             }
-            result.set(0, i, T::from(best_cluster).unwrap());
+            result.set(i, TY::from_usize(best_cluster).unwrap());
         }
 
-        Ok(result.to_row_vector())
+        Ok(result)
     }
 
-    fn kmeans_plus_plus<M: Matrix<T>>(data: &M, k: usize) -> Vec<usize> {
-        let mut rng = rand::thread_rng();
-        let (n, m) = data.shape();
+    fn kmeans_plus_plus(data: &X, k: usize, seed: Option<u64>) -> Vec<usize> {
+        let mut rng = get_rng_impl(seed);
+        let (n, _) = data.shape();
         let mut y = vec![0; n];
-        let mut centroid = data.get_row_as_vec(rng.gen_range(0..n));
+        let mut centroid: Vec<TX> = data
+            .get_row(rng.gen_range(0..n))
+            .iterator(0)
+            .cloned()
+            .collect();
 
-        let mut d = vec![T::max_value(); n];
-
-        let mut row = vec![T::zero(); m];
+        let mut d = vec![std::f64::MAX; n];
+        let mut row = vec![TX::zero(); data.shape().1];
 
         for j in 1..k {
             for i in 0..n {
-                data.copy_row_as_vec(i, &mut row);
+                data.get_row(i)
+                    .iterator(0)
+                    .zip(row.iter_mut())
+                    .for_each(|(&x, r)| *r = x);
                 let dist = Euclidian::squared_distance(&row, &centroid);
 
                 if dist < d[i] {
@@ -262,12 +378,12 @@ impl<T: RealNumber + Sum> KMeans<T> {
                 }
             }
 
-            let mut sum: T = T::zero();
+            let mut sum = 0f64;
             for i in d.iter() {
                 sum += *i;
             }
-            let cutoff = T::from(rng.gen::<f64>()).unwrap() * sum;
-            let mut cost = T::zero();
+            let cutoff = rng.gen::<f64>() * sum;
+            let mut cost = 0f64;
             let mut index = 0;
             while index < n {
                 cost += d[index];
@@ -277,11 +393,14 @@ impl<T: RealNumber + Sum> KMeans<T> {
                 index += 1;
             }
 
-            data.copy_row_as_vec(index, &mut centroid);
+            centroid = data.get_row(index).iterator(0).cloned().collect();
         }
 
         for i in 0..n {
-            data.copy_row_as_vec(i, &mut row);
+            data.get_row(i)
+                .iterator(0)
+                .zip(row.iter_mut())
+                .for_each(|(&x, r)| *r = x);
             let dist = Euclidian::squared_distance(&row, &centroid);
 
             if dist < d[i] {
@@ -297,25 +416,61 @@ impl<T: RealNumber + Sum> KMeans<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn invalid_k() {
-        let x = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.]]);
+        let x = DenseMatrix::from_2d_array(&[&[1, 2, 3], &[4, 5, 6]]);
 
-        assert!(KMeans::fit(&x, KMeansParameters::default().with_k(0)).is_err());
+        assert!(KMeans::<i32, i32, DenseMatrix<i32>, Vec<i32>>::fit(
+            &x,
+            KMeansParameters::default().with_k(0)
+        )
+        .is_err());
         assert_eq!(
             "Fit failed: invalid number of clusters: 1",
-            KMeans::fit(&x, KMeansParameters::default().with_k(1))
-                .unwrap_err()
-                .to_string()
+            KMeans::<i32, i32, DenseMatrix<i32>, Vec<i32>>::fit(
+                &x,
+                KMeansParameters::default().with_k(1)
+            )
+            .unwrap_err()
+            .to_string()
         );
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
     #[test]
-    fn fit_predict_iris() {
+    fn search_parameters() {
+        let parameters = KMeansSearchParameters {
+            k: vec![2, 4],
+            max_iter: vec![10, 100],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.k, 2);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.k, 4);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.k, 2);
+        assert_eq!(next.max_iter, 100);
+        let next = iter.next().unwrap();
+        assert_eq!(next.k, 4);
+        assert_eq!(next.max_iter, 100);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn fit_predict() {
         let x = DenseMatrix::from_2d_array(&[
             &[5.1, 3.5, 1.4, 0.2],
             &[4.9, 3.0, 1.4, 0.2],
@@ -341,14 +496,17 @@ mod tests {
 
         let kmeans = KMeans::fit(&x, Default::default()).unwrap();
 
-        let y = kmeans.predict(&x).unwrap();
+        let y: Vec<usize> = kmeans.predict(&x).unwrap();
 
         for i in 0..y.len() {
             assert_eq!(y[i] as usize, kmeans._y[i]);
         }
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
@@ -375,9 +533,10 @@ mod tests {
             &[5.2, 2.7, 3.9, 1.4],
         ]);
 
-        let kmeans = KMeans::fit(&x, Default::default()).unwrap();
+        let kmeans: KMeans<f32, f32, DenseMatrix<f32>, Vec<f32>> =
+            KMeans::fit(&x, Default::default()).unwrap();
 
-        let deserialized_kmeans: KMeans<f64> =
+        let deserialized_kmeans: KMeans<f32, f32, DenseMatrix<f32>, Vec<f32>> =
             serde_json::from_str(&serde_json::to_string(&kmeans).unwrap()).unwrap();
 
         assert_eq!(kmeans, deserialized_kmeans);

src/dataset/boston.rs

@@ -69,7 +69,10 @@ mod tests {
         assert!(serialize_data(&dataset, "boston.xy").is_ok());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn boston_dataset() {
         let dataset = load_dataset();

src/dataset/breast_cancer.rs

@@ -30,11 +30,16 @@ use crate::dataset::deserialize_data;
 use crate::dataset::Dataset;
 
 /// Get dataset
-pub fn load_dataset() -> Dataset<f32, f32> {
+pub fn load_dataset() -> Dataset<f32, u32> {
     let (x, y, num_samples, num_features) =
         match deserialize_data(std::include_bytes!("breast_cancer.xy")) {
             Err(why) => panic!("Can't deserialize breast_cancer.xy. {}", why),
-            Ok((x, y, num_samples, num_features)) => (x, y, num_samples, num_features),
+            Ok((x, y, num_samples, num_features)) => (
+                x,
+                y.into_iter().map(|x| x as u32).collect(),
+                num_samples,
+                num_features,
+            ),
         };
 
     Dataset {
@@ -66,20 +71,22 @@ pub fn load_dataset() -> Dataset<f32, f32> {
 #[cfg(test)]
 mod tests {
 
-    #[cfg(not(target_arch = "wasm32"))]
-    use super::super::*;
     use super::*;
 
-    #[test]
-    #[ignore]
-    #[cfg(not(target_arch = "wasm32"))]
-    fn refresh_cancer_dataset() {
-        // run this test to generate breast_cancer.xy file.
-        let dataset = load_dataset();
-        assert!(serialize_data(&dataset, "breast_cancer.xy").is_ok());
-    }
+    // TODO: implement serialization
+    // #[test]
+    // #[ignore]
+    // #[cfg(not(target_arch = "wasm32"))]
+    // fn refresh_cancer_dataset() {
+    //     // run this test to generate breast_cancer.xy file.
+    //     let dataset = load_dataset();
+    //     assert!(serialize_data(&dataset, "breast_cancer.xy").is_ok());
+    // }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn cancer_dataset() {
         let dataset = load_dataset();

src/dataset/diabetes.rs

@@ -23,11 +23,16 @@ use crate::dataset::deserialize_data;
 use crate::dataset::Dataset;
 
 /// Get dataset
-pub fn load_dataset() -> Dataset<f32, f32> {
+pub fn load_dataset() -> Dataset<f32, u32> {
     let (x, y, num_samples, num_features) =
         match deserialize_data(std::include_bytes!("diabetes.xy")) {
             Err(why) => panic!("Can't deserialize diabetes.xy. {}", why),
-            Ok((x, y, num_samples, num_features)) => (x, y, num_samples, num_features),
+            Ok((x, y, num_samples, num_features)) => (
+                x,
+                y.into_iter().map(|x| x as u32).collect(),
+                num_samples,
+                num_features,
+            ),
         };
 
     Dataset {
@@ -50,20 +55,22 @@ pub fn load_dataset() -> Dataset<f32, f32> {
 #[cfg(test)]
 mod tests {
 
-    #[cfg(not(target_arch = "wasm32"))]
-    use super::super::*;
     use super::*;
 
-    #[cfg(not(target_arch = "wasm32"))]
-    #[test]
-    #[ignore]
-    fn refresh_diabetes_dataset() {
-        // run this test to generate diabetes.xy file.
-        let dataset = load_dataset();
-        assert!(serialize_data(&dataset, "diabetes.xy").is_ok());
-    }
+    // TODO: fix serialization
+    // #[cfg(not(target_arch = "wasm32"))]
+    // #[test]
+    // #[ignore]
+    // fn refresh_diabetes_dataset() {
+    //     // run this test to generate diabetes.xy file.
+    //     let dataset = load_dataset();
+    //     assert!(serialize_data(&dataset, "diabetes.xy").is_ok());
+    // }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn boston_dataset() {
         let dataset = load_dataset();

src/dataset/digits.rs

@@ -1,4 +1,4 @@
-//! # Optical Recognition of Handwritten Digits Data Set
+//! # Optical Recognition of Handwritten Digits Dataset
 //!
 //! | Number of Instances | Number of Attributes | Missing Values? | Associated Tasks: |
 //! |-|-|-|-|
@@ -57,7 +57,10 @@ mod tests {
         let dataset = load_dataset();
         assert!(serialize_data(&dataset, "digits.xy").is_ok());
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn digits_dataset() {
         let dataset = load_dataset();

src/dataset/generator.rs

@@ -48,7 +48,7 @@ pub fn make_blobs(
 }
 
 /// Make a large circle containing a smaller circle in 2d.
-pub fn make_circles(num_samples: usize, factor: f32, noise: f32) -> Dataset<f32, f32> {
+pub fn make_circles(num_samples: usize, factor: f32, noise: f32) -> Dataset<f32, u32> {
     if !(0.0..1.0).contains(&factor) {
         panic!("'factor' has to be between 0 and 1.");
     }
@@ -79,7 +79,7 @@ pub fn make_circles(num_samples: usize, factor: f32, noise: f32) -> Dataset<f32,
 
     Dataset {
         data: x,
-        target: y,
+        target: y.into_iter().map(|x| x as u32).collect(),
         num_samples,
         num_features: 2,
         feature_names: (0..2).map(|n| n.to_string()).collect(),
@@ -89,7 +89,7 @@ pub fn make_circles(num_samples: usize, factor: f32, noise: f32) -> Dataset<f32,
 }
 
 /// Make two interleaving half circles in 2d
-pub fn make_moons(num_samples: usize, noise: f32) -> Dataset<f32, f32> {
+pub fn make_moons(num_samples: usize, noise: f32) -> Dataset<f32, u32> {
     let num_samples_out = num_samples / 2;
     let num_samples_in = num_samples - num_samples_out;
 
@@ -116,7 +116,7 @@ pub fn make_moons(num_samples: usize, noise: f32) -> Dataset<f32, f32> {
 
     Dataset {
         data: x,
-        target: y,
+        target: y.into_iter().map(|x| x as u32).collect(),
         num_samples,
         num_features: 2,
         feature_names: (0..2).map(|n| n.to_string()).collect(),
@@ -137,7 +137,10 @@ mod tests {
 
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_make_blobs() {
         let dataset = make_blobs(10, 2, 3);
@@ -150,7 +153,10 @@ mod tests {
         assert_eq!(dataset.num_samples, 10);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_make_circles() {
         let dataset = make_circles(10, 0.5, 0.05);
@@ -163,7 +169,10 @@ mod tests {
         assert_eq!(dataset.num_samples, 10);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_make_moons() {
         let dataset = make_moons(10, 0.05);

src/dataset/iris.rs

@@ -1,4 +1,4 @@
-//! # The Iris Dataset flower
+//! # The Iris flower dataset
 //!
 //! | Number of Instances | Number of Attributes | Missing Values? | Associated Tasks: |
 //! |-|-|-|-|
@@ -19,11 +19,17 @@ use crate::dataset::deserialize_data;
 use crate::dataset::Dataset;
 
 /// Get dataset
-pub fn load_dataset() -> Dataset<f32, f32> {
-    let (x, y, num_samples, num_features) = match deserialize_data(std::include_bytes!("iris.xy")) {
-        Err(why) => panic!("Can't deserialize iris.xy. {}", why),
-        Ok((x, y, num_samples, num_features)) => (x, y, num_samples, num_features),
-    };
+pub fn load_dataset() -> Dataset<f32, u32> {
+    let (x, y, num_samples, num_features): (Vec<f32>, Vec<u32>, usize, usize) =
+        match deserialize_data(std::include_bytes!("iris.xy")) {
+            Err(why) => panic!("Can't deserialize iris.xy. {}", why),
+            Ok((x, y, num_samples, num_features)) => (
+                x,
+                y.into_iter().map(|x| x as u32).collect(),
+                num_samples,
+                num_features,
+            ),
+        };
 
     Dataset {
         data: x,
@@ -50,20 +56,24 @@ pub fn load_dataset() -> Dataset<f32, f32> {
 #[cfg(test)]
 mod tests {
 
-    #[cfg(not(target_arch = "wasm32"))]
-    use super::super::*;
+    // #[cfg(not(target_arch = "wasm32"))]
+    // use super::super::*;
     use super::*;
 
-    #[cfg(not(target_arch = "wasm32"))]
-    #[test]
-    #[ignore]
-    fn refresh_iris_dataset() {
-        // run this test to generate iris.xy file.
-        let dataset = load_dataset();
-        assert!(serialize_data(&dataset, "iris.xy").is_ok());
-    }
+    // TODO: fix serialization
+    // #[cfg(not(target_arch = "wasm32"))]
+    // #[test]
+    // #[ignore]
+    // fn refresh_iris_dataset() {
+    //     // run this test to generate iris.xy file.
+    //     let dataset = load_dataset();
+    //     assert!(serialize_data(&dataset, "iris.xy").is_ok());
+    // }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn iris_dataset() {
         let dataset = load_dataset();

src/dataset/mod.rs

@@ -1,6 +1,6 @@
 //! Datasets
 //!
-//! In this module you will find small datasets that are used in SmartCore for demonstration purpose mostly.
+//! In this module you will find small datasets that are used in `smartcore` mostly for demonstration purposes.
 pub mod boston;
 pub mod breast_cancer;
 pub mod diabetes;
@@ -9,7 +9,7 @@ pub mod generator;
 pub mod iris;
 
 #[cfg(not(target_arch = "wasm32"))]
-use crate::math::num::RealNumber;
+use crate::numbers::{basenum::Number, realnum::RealNumber};
 #[cfg(not(target_arch = "wasm32"))]
 use std::fs::File;
 use std::io;
@@ -55,7 +55,7 @@ impl<X, Y> Dataset<X, Y> {
 // Running this in wasm throws: operation not supported on this platform.
 #[cfg(not(target_arch = "wasm32"))]
 #[allow(dead_code)]
-pub(crate) fn serialize_data<X: RealNumber, Y: RealNumber>(
+pub(crate) fn serialize_data<X: Number + RealNumber, Y: RealNumber>(
     dataset: &Dataset<X, Y>,
     filename: &str,
 ) -> Result<(), io::Error> {
@@ -121,7 +121,10 @@ pub(crate) fn deserialize_data(
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn as_matrix() {
         let dataset = Dataset {

src/decomposition/pca.rs

@@ -10,7 +10,7 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::decomposition::pca::*;
 //!
 //! // Iris data
@@ -52,24 +52,33 @@ use serde::{Deserialize, Serialize};
 
 use crate::api::{Transformer, UnsupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::linalg::traits::evd::EVDDecomposable;
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 
 /// Principal components analysis algorithm
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct PCA<T: RealNumber, M: Matrix<T>> {
-    eigenvectors: M,
+pub struct PCA<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> {
+    eigenvectors: X,
     eigenvalues: Vec<T>,
-    projection: M,
+    projection: X,
     mu: Vec<T>,
     pmu: Vec<T>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for PCA<T, M> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> PartialEq
+    for PCA<T, X>
+{
     fn eq(&self, other: &Self) -> bool {
-        if self.eigenvectors != other.eigenvectors
-            || self.eigenvalues.len() != other.eigenvalues.len()
+        if self.eigenvalues.len() != other.eigenvalues.len()
+            || self
+                .eigenvectors
+                .iterator(0)
+                .zip(other.eigenvectors.iterator(0))
+                .any(|(&a, &b)| (a - b).abs() > T::epsilon())
         {
             false
         } else {
@@ -83,11 +92,14 @@ impl<T: RealNumber, M: Matrix<T>> PartialEq for PCA<T, M> {
     }
 }
 
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 /// PCA parameters
 pub struct PCAParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Number of components to keep.
     pub n_components: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// By default, covariance matrix is used to compute principal components.
     /// Enable this flag if you want to use correlation matrix instead.
     pub use_correlation_matrix: bool,
@@ -116,24 +128,105 @@ impl Default for PCAParameters {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> UnsupervisedEstimator<M, PCAParameters> for PCA<T, M> {
-    fn fit(x: &M, parameters: PCAParameters) -> Result<Self, Failed> {
+/// PCA grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct PCASearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Number of components to keep.
+    pub n_components: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// By default, covariance matrix is used to compute principal components.
+    /// Enable this flag if you want to use correlation matrix instead.
+    pub use_correlation_matrix: Vec<bool>,
+}
+
+/// PCA grid search iterator
+pub struct PCASearchParametersIterator {
+    pca_search_parameters: PCASearchParameters,
+    current_k: usize,
+    current_use_correlation_matrix: usize,
+}
+
+impl IntoIterator for PCASearchParameters {
+    type Item = PCAParameters;
+    type IntoIter = PCASearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        PCASearchParametersIterator {
+            pca_search_parameters: self,
+            current_k: 0,
+            current_use_correlation_matrix: 0,
+        }
+    }
+}
+
+impl Iterator for PCASearchParametersIterator {
+    type Item = PCAParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_k == self.pca_search_parameters.n_components.len()
+            && self.current_use_correlation_matrix
+                == self.pca_search_parameters.use_correlation_matrix.len()
+        {
+            return None;
+        }
+
+        let next = PCAParameters {
+            n_components: self.pca_search_parameters.n_components[self.current_k],
+            use_correlation_matrix: self.pca_search_parameters.use_correlation_matrix
+                [self.current_use_correlation_matrix],
+        };
+
+        if self.current_k + 1 < self.pca_search_parameters.n_components.len() {
+            self.current_k += 1;
+        } else if self.current_use_correlation_matrix + 1
+            < self.pca_search_parameters.use_correlation_matrix.len()
+        {
+            self.current_k = 0;
+            self.current_use_correlation_matrix += 1;
+        } else {
+            self.current_k += 1;
+            self.current_use_correlation_matrix += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for PCASearchParameters {
+    fn default() -> Self {
+        let default_params = PCAParameters::default();
+
+        PCASearchParameters {
+            n_components: vec![default_params.n_components],
+            use_correlation_matrix: vec![default_params.use_correlation_matrix],
+        }
+    }
+}
+
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>>
+    UnsupervisedEstimator<X, PCAParameters> for PCA<T, X>
+{
+    fn fit(x: &X, parameters: PCAParameters) -> Result<Self, Failed> {
         PCA::fit(x, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Transformer<M> for PCA<T, M> {
-    fn transform(&self, x: &M) -> Result<M, Failed> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> Transformer<X>
+    for PCA<T, X>
+{
+    fn transform(&self, x: &X) -> Result<X, Failed> {
         self.transform(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> PCA<T, X> {
     /// Fits PCA to your data.
     /// * `data` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `n_components` - number of components to keep.
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
-    pub fn fit(data: &M, parameters: PCAParameters) -> Result<PCA<T, M>, Failed> {
+    pub fn fit(data: &X, parameters: PCAParameters) -> Result<PCA<T, X>, Failed> {
         let (m, n) = data.shape();
 
         if parameters.n_components > n {
@@ -143,13 +236,17 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
             )));
         }
 
-        let mu = data.column_mean();
+        let mu: Vec<T> = data
+            .mean_by(0)
+            .iter()
+            .map(|&v| T::from_f64(v).unwrap())
+            .collect();
 
         let mut x = data.clone();
 
-        for (c, mu_c) in mu.iter().enumerate().take(n) {
+        for (c, &mu_c) in mu.iter().enumerate().take(n) {
             for r in 0..m {
-                x.sub_element_mut(r, c, *mu_c);
+                x.sub_element_mut((r, c), mu_c);
             }
         }
 
@@ -165,33 +262,33 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
 
             eigenvectors = svd.V;
         } else {
-            let mut cov = M::zeros(n, n);
+            let mut cov = X::zeros(n, n);
 
             for k in 0..m {
                 for i in 0..n {
                     for j in 0..=i {
-                        cov.add_element_mut(i, j, x.get(k, i) * x.get(k, j));
+                        cov.add_element_mut((i, j), *x.get((k, i)) * *x.get((k, j)));
                     }
                 }
             }
 
             for i in 0..n {
                 for j in 0..=i {
-                    cov.div_element_mut(i, j, T::from(m).unwrap());
-                    cov.set(j, i, cov.get(i, j));
+                    cov.div_element_mut((i, j), T::from(m).unwrap());
+                    cov.set((j, i), *cov.get((i, j)));
                 }
             }
 
             if parameters.use_correlation_matrix {
                 let mut sd = vec![T::zero(); n];
                 for (i, sd_i) in sd.iter_mut().enumerate().take(n) {
-                    *sd_i = cov.get(i, i).sqrt();
+                    *sd_i = cov.get((i, i)).sqrt();
                 }
 
                 for i in 0..n {
                     for j in 0..=i {
-                        cov.div_element_mut(i, j, sd[i] * sd[j]);
-                        cov.set(j, i, cov.get(i, j));
+                        cov.div_element_mut((i, j), sd[i] * sd[j]);
+                        cov.set((j, i), *cov.get((i, j)));
                     }
                 }
 
@@ -203,7 +300,7 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
 
                 for (i, sd_i) in sd.iter().enumerate().take(n) {
                     for j in 0..n {
-                        eigenvectors.div_element_mut(i, j, *sd_i);
+                        eigenvectors.div_element_mut((i, j), *sd_i);
                     }
                 }
             } else {
@@ -215,17 +312,17 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
             }
         }
 
-        let mut projection = M::zeros(parameters.n_components, n);
+        let mut projection = X::zeros(parameters.n_components, n);
         for i in 0..n {
             for j in 0..parameters.n_components {
-                projection.set(j, i, eigenvectors.get(i, j));
+                projection.set((j, i), *eigenvectors.get((i, j)));
             }
         }
 
         let mut pmu = vec![T::zero(); parameters.n_components];
         for (k, mu_k) in mu.iter().enumerate().take(n) {
             for (i, pmu_i) in pmu.iter_mut().enumerate().take(parameters.n_components) {
-                *pmu_i += projection.get(i, k) * (*mu_k);
+                *pmu_i += *projection.get((i, k)) * (*mu_k);
             }
         }
 
@@ -240,7 +337,7 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
 
     /// Run dimensionality reduction for `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn transform(&self, x: &M) -> Result<M, Failed> {
+    pub fn transform(&self, x: &X) -> Result<X, Failed> {
         let (nrows, ncols) = x.shape();
         let (_, n_components) = self.projection.shape();
         if ncols != self.mu.len() {
@@ -254,14 +351,14 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
         let mut x_transformed = x.matmul(&self.projection);
         for r in 0..nrows {
             for c in 0..n_components {
-                x_transformed.sub_element_mut(r, c, self.pmu[c]);
+                x_transformed.sub_element_mut((r, c), self.pmu[c]);
             }
         }
         Ok(x_transformed)
     }
 
     /// Get a projection matrix
-    pub fn components(&self) -> &M {
+    pub fn components(&self) -> &X {
         &self.projection
     }
 }
@@ -269,7 +366,31 @@ impl<T: RealNumber, M: Matrix<T>> PCA<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
+
+    #[test]
+    fn search_parameters() {
+        let parameters = PCASearchParameters {
+            n_components: vec![2, 4],
+            use_correlation_matrix: vec![true, false],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 2);
+        assert_eq!(next.use_correlation_matrix, true);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 4);
+        assert_eq!(next.use_correlation_matrix, true);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 2);
+        assert_eq!(next.use_correlation_matrix, false);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 4);
+        assert_eq!(next.use_correlation_matrix, false);
+        assert!(iter.next().is_none());
+    }
 
     fn us_arrests_data() -> DenseMatrix<f64> {
         DenseMatrix::from_2d_array(&[
@@ -325,7 +446,10 @@ mod tests {
             &[6.8, 161.0, 60.0, 15.6],
         ])
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn pca_components() {
         let us_arrests = us_arrests_data();
@@ -339,9 +463,16 @@ mod tests {
 
         let pca = PCA::fit(&us_arrests, Default::default()).unwrap();
 
-        assert!(expected.approximate_eq(&pca.components().abs(), 0.4));
+        assert!(relative_eq!(
+            expected,
+            pca.components().abs(),
+            epsilon = 1e-3
+        ));
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_covariance() {
         let us_arrests = us_arrests_data();
@@ -435,10 +566,11 @@ mod tests {
 
         let pca = PCA::fit(&us_arrests, PCAParameters::default().with_n_components(4)).unwrap();
 
-        assert!(pca
-            .eigenvectors
-            .abs()
-            .approximate_eq(&expected_eigenvectors.abs(), 1e-4));
+        assert!(relative_eq!(
+            pca.eigenvectors.abs(),
+            &expected_eigenvectors.abs(),
+            epsilon = 1e-4
+        ));
 
         for i in 0..pca.eigenvalues.len() {
             assert!((pca.eigenvalues[i].abs() - expected_eigenvalues[i].abs()).abs() < 1e-8);
@@ -446,12 +578,17 @@ mod tests {
 
         let us_arrests_t = pca.transform(&us_arrests).unwrap();
 
-        assert!(us_arrests_t
-            .abs()
-            .approximate_eq(&expected_projection.abs(), 1e-4));
+        assert!(relative_eq!(
+            us_arrests_t.abs(),
+            &expected_projection.abs(),
+            epsilon = 1e-4
+        ));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_correlation() {
         let us_arrests = us_arrests_data();
@@ -551,10 +688,11 @@ mod tests {
         )
         .unwrap();
 
-        assert!(pca
-            .eigenvectors
-            .abs()
-            .approximate_eq(&expected_eigenvectors.abs(), 1e-4));
+        assert!(relative_eq!(
+            pca.eigenvectors.abs(),
+            &expected_eigenvectors.abs(),
+            epsilon = 1e-4
+        ));
 
         for i in 0..pca.eigenvalues.len() {
             assert!((pca.eigenvalues[i].abs() - expected_eigenvalues[i].abs()).abs() < 1e-8);
@@ -562,43 +700,47 @@ mod tests {
 
         let us_arrests_t = pca.transform(&us_arrests).unwrap();
 
-        assert!(us_arrests_t
-            .abs()
-            .approximate_eq(&expected_projection.abs(), 1e-4));
+        assert!(relative_eq!(
+            us_arrests_t.abs(),
+            &expected_projection.abs(),
+            epsilon = 1e-4
+        ));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let iris = DenseMatrix::from_2d_array(&[
-            &[5.1, 3.5, 1.4, 0.2],
-            &[4.9, 3.0, 1.4, 0.2],
-            &[4.7, 3.2, 1.3, 0.2],
-            &[4.6, 3.1, 1.5, 0.2],
-            &[5.0, 3.6, 1.4, 0.2],
-            &[5.4, 3.9, 1.7, 0.4],
-            &[4.6, 3.4, 1.4, 0.3],
-            &[5.0, 3.4, 1.5, 0.2],
-            &[4.4, 2.9, 1.4, 0.2],
-            &[4.9, 3.1, 1.5, 0.1],
-            &[7.0, 3.2, 4.7, 1.4],
-            &[6.4, 3.2, 4.5, 1.5],
-            &[6.9, 3.1, 4.9, 1.5],
-            &[5.5, 2.3, 4.0, 1.3],
-            &[6.5, 2.8, 4.6, 1.5],
-            &[5.7, 2.8, 4.5, 1.3],
-            &[6.3, 3.3, 4.7, 1.6],
-            &[4.9, 2.4, 3.3, 1.0],
-            &[6.6, 2.9, 4.6, 1.3],
-            &[5.2, 2.7, 3.9, 1.4],
-        ]);
+    // Disable this test for now
+    // TODO: implement deserialization for new DenseMatrix
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn pca_serde() {
+    //     let iris = DenseMatrix::from_2d_array(&[
+    //         &[5.1, 3.5, 1.4, 0.2],
+    //         &[4.9, 3.0, 1.4, 0.2],
+    //         &[4.7, 3.2, 1.3, 0.2],
+    //         &[4.6, 3.1, 1.5, 0.2],
+    //         &[5.0, 3.6, 1.4, 0.2],
+    //         &[5.4, 3.9, 1.7, 0.4],
+    //         &[4.6, 3.4, 1.4, 0.3],
+    //         &[5.0, 3.4, 1.5, 0.2],
+    //         &[4.4, 2.9, 1.4, 0.2],
+    //         &[4.9, 3.1, 1.5, 0.1],
+    //         &[7.0, 3.2, 4.7, 1.4],
+    //         &[6.4, 3.2, 4.5, 1.5],
+    //         &[6.9, 3.1, 4.9, 1.5],
+    //         &[5.5, 2.3, 4.0, 1.3],
+    //         &[6.5, 2.8, 4.6, 1.5],
+    //         &[5.7, 2.8, 4.5, 1.3],
+    //         &[6.3, 3.3, 4.7, 1.6],
+    //         &[4.9, 2.4, 3.3, 1.0],
+    //         &[6.6, 2.9, 4.6, 1.3],
+    //         &[5.2, 2.7, 3.9, 1.4],
+    //     ]);
 
-        let pca = PCA::fit(&iris, Default::default()).unwrap();
+    //     let pca = PCA::fit(&iris, Default::default()).unwrap();
 
-        let deserialized_pca: PCA<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&pca).unwrap()).unwrap();
+    //     let deserialized_pca: PCA<f64, DenseMatrix<f64>> =
+    //         serde_json::from_str(&serde_json::to_string(&pca).unwrap()).unwrap();
 
-        assert_eq!(pca, deserialized_pca);
-    }
+    //     assert_eq!(pca, deserialized_pca);
+    // }
 }

src/decomposition/svd.rs

@@ -7,7 +7,7 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::decomposition::svd::*;
 //!
 //! // Iris data
@@ -51,27 +51,36 @@ use serde::{Deserialize, Serialize};
 
 use crate::api::{Transformer, UnsupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::linalg::traits::evd::EVDDecomposable;
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 
 /// SVD
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct SVD<T: RealNumber, M: Matrix<T>> {
-    components: M,
+pub struct SVD<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> {
+    components: X,
     phantom: PhantomData<T>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for SVD<T, M> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> PartialEq
+    for SVD<T, X>
+{
     fn eq(&self, other: &Self) -> bool {
         self.components
-            .approximate_eq(&other.components, T::from_f64(1e-8).unwrap())
+            .iterator(0)
+            .zip(other.components.iterator(0))
+            .all(|(&a, &b)| (a - b).abs() <= T::epsilon())
     }
 }
 
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 /// SVD parameters
 pub struct SVDParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Number of components to keep.
     pub n_components: usize,
 }
@@ -90,24 +99,83 @@ impl SVDParameters {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> UnsupervisedEstimator<M, SVDParameters> for SVD<T, M> {
-    fn fit(x: &M, parameters: SVDParameters) -> Result<Self, Failed> {
+/// SVD grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct SVDSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Maximum number of iterations of the k-means algorithm for a single run.
+    pub n_components: Vec<usize>,
+}
+
+/// SVD grid search iterator
+pub struct SVDSearchParametersIterator {
+    svd_search_parameters: SVDSearchParameters,
+    current_n_components: usize,
+}
+
+impl IntoIterator for SVDSearchParameters {
+    type Item = SVDParameters;
+    type IntoIter = SVDSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        SVDSearchParametersIterator {
+            svd_search_parameters: self,
+            current_n_components: 0,
+        }
+    }
+}
+
+impl Iterator for SVDSearchParametersIterator {
+    type Item = SVDParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_n_components == self.svd_search_parameters.n_components.len() {
+            return None;
+        }
+
+        let next = SVDParameters {
+            n_components: self.svd_search_parameters.n_components[self.current_n_components],
+        };
+
+        self.current_n_components += 1;
+
+        Some(next)
+    }
+}
+
+impl Default for SVDSearchParameters {
+    fn default() -> Self {
+        let default_params = SVDParameters::default();
+
+        SVDSearchParameters {
+            n_components: vec![default_params.n_components],
+        }
+    }
+}
+
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>>
+    UnsupervisedEstimator<X, SVDParameters> for SVD<T, X>
+{
+    fn fit(x: &X, parameters: SVDParameters) -> Result<Self, Failed> {
         SVD::fit(x, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Transformer<M> for SVD<T, M> {
-    fn transform(&self, x: &M) -> Result<M, Failed> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> Transformer<X>
+    for SVD<T, X>
+{
+    fn transform(&self, x: &X) -> Result<X, Failed> {
         self.transform(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> SVD<T, M> {
+impl<T: Number + RealNumber, X: Array2<T> + SVDDecomposable<T> + EVDDecomposable<T>> SVD<T, X> {
     /// Fits SVD to your data.
     /// * `data` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `n_components` - number of components to keep.
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
-    pub fn fit(x: &M, parameters: SVDParameters) -> Result<SVD<T, M>, Failed> {
+    pub fn fit(x: &X, parameters: SVDParameters) -> Result<SVD<T, X>, Failed> {
         let (_, p) = x.shape();
 
         if parameters.n_components >= p {
@@ -119,7 +187,7 @@ impl<T: RealNumber, M: Matrix<T>> SVD<T, M> {
 
         let svd = x.svd()?;
 
-        let components = svd.V.slice(0..p, 0..parameters.n_components);
+        let components = X::from_slice(svd.V.slice(0..p, 0..parameters.n_components).as_ref());
 
         Ok(SVD {
             components,
@@ -129,7 +197,7 @@ impl<T: RealNumber, M: Matrix<T>> SVD<T, M> {
 
     /// Run dimensionality reduction for `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn transform(&self, x: &M) -> Result<M, Failed> {
+    pub fn transform(&self, x: &X) -> Result<X, Failed> {
         let (n, p) = x.shape();
         let (p_c, k) = self.components.shape();
         if p_c != p {
@@ -143,7 +211,7 @@ impl<T: RealNumber, M: Matrix<T>> SVD<T, M> {
     }
 
     /// Get a projection matrix
-    pub fn components(&self) -> &M {
+    pub fn components(&self) -> &X {
         &self.components
     }
 }
@@ -151,9 +219,28 @@ impl<T: RealNumber, M: Matrix<T>> SVD<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::arrays::Array;
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = SVDSearchParameters {
+            n_components: vec![10, 100],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_components, 100);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn svd_decompose() {
         // https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/USArrests.html
@@ -223,43 +310,47 @@ mod tests {
 
         assert_eq!(svd.components.shape(), (x.shape().1, 2));
 
-        assert!(x_transformed
-            .slice(0..5, 0..2)
-            .approximate_eq(&expected, 1e-4));
+        assert!(relative_eq!(
+            DenseMatrix::from_slice(x_transformed.slice(0..5, 0..2).as_ref()),
+            &expected,
+            epsilon = 1e-4
+        ));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let iris = DenseMatrix::from_2d_array(&[
-            &[5.1, 3.5, 1.4, 0.2],
-            &[4.9, 3.0, 1.4, 0.2],
-            &[4.7, 3.2, 1.3, 0.2],
-            &[4.6, 3.1, 1.5, 0.2],
-            &[5.0, 3.6, 1.4, 0.2],
-            &[5.4, 3.9, 1.7, 0.4],
-            &[4.6, 3.4, 1.4, 0.3],
-            &[5.0, 3.4, 1.5, 0.2],
-            &[4.4, 2.9, 1.4, 0.2],
-            &[4.9, 3.1, 1.5, 0.1],
-            &[7.0, 3.2, 4.7, 1.4],
-            &[6.4, 3.2, 4.5, 1.5],
-            &[6.9, 3.1, 4.9, 1.5],
-            &[5.5, 2.3, 4.0, 1.3],
-            &[6.5, 2.8, 4.6, 1.5],
-            &[5.7, 2.8, 4.5, 1.3],
-            &[6.3, 3.3, 4.7, 1.6],
-            &[4.9, 2.4, 3.3, 1.0],
-            &[6.6, 2.9, 4.6, 1.3],
-            &[5.2, 2.7, 3.9, 1.4],
-        ]);
+    // Disable this test for now
+    // TODO: implement deserialization for new DenseMatrix
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn serde() {
+    //     let iris = DenseMatrix::from_2d_array(&[
+    //         &[5.1, 3.5, 1.4, 0.2],
+    //         &[4.9, 3.0, 1.4, 0.2],
+    //         &[4.7, 3.2, 1.3, 0.2],
+    //         &[4.6, 3.1, 1.5, 0.2],
+    //         &[5.0, 3.6, 1.4, 0.2],
+    //         &[5.4, 3.9, 1.7, 0.4],
+    //         &[4.6, 3.4, 1.4, 0.3],
+    //         &[5.0, 3.4, 1.5, 0.2],
+    //         &[4.4, 2.9, 1.4, 0.2],
+    //         &[4.9, 3.1, 1.5, 0.1],
+    //         &[7.0, 3.2, 4.7, 1.4],
+    //         &[6.4, 3.2, 4.5, 1.5],
+    //         &[6.9, 3.1, 4.9, 1.5],
+    //         &[5.5, 2.3, 4.0, 1.3],
+    //         &[6.5, 2.8, 4.6, 1.5],
+    //         &[5.7, 2.8, 4.5, 1.3],
+    //         &[6.3, 3.3, 4.7, 1.6],
+    //         &[4.9, 2.4, 3.3, 1.0],
+    //         &[6.6, 2.9, 4.6, 1.3],
+    //         &[5.2, 2.7, 3.9, 1.4],
+    //     ]);
 
-        let svd = SVD::fit(&iris, Default::default()).unwrap();
+    //     let svd = SVD::fit(&iris, Default::default()).unwrap();
 
-        let deserialized_svd: SVD<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&svd).unwrap()).unwrap();
+    //     let deserialized_svd: SVD<f32, DenseMatrix<f32>> =
+    //         serde_json::from_str(&serde_json::to_string(&svd).unwrap()).unwrap();
 
-        assert_eq!(svd, deserialized_svd);
-    }
+    //     assert_eq!(svd, deserialized_svd);
+    // }
 }

src/ensemble/mod.rs

@@ -7,7 +7,7 @@
 //! set and then aggregate their individual predictions to form a final prediction. In classification setting the overall prediction is the most commonly
 //! occurring majority class among the individual predictions.
 //!
-//! In SmartCore you will find implementation of RandomForest - a popular averaging algorithms based on randomized [decision trees](../tree/index.html).
+//! In `smartcore` you will find implementation of RandomForest - a popular averaging algorithms based on randomized [decision trees](../tree/index.html).
 //! Random forests provide an improvement over bagged trees by way of a small tweak that decorrelates the trees. As in bagging, we build a number of
 //! decision trees on bootstrapped training samples. But when building these decision trees, each time a split in a tree is considered,
 //! a random sample of _m_ predictors is chosen as split candidates from the full set of _p_ predictors.

src/ensemble/random_forest_classifier.rs

@@ -8,7 +8,7 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::ensemble::random_forest_classifier::RandomForestClassifier;
 //!
 //! // Iris dataset
@@ -35,8 +35,8 @@
 //!              &[5.2, 2.7, 3.9, 1.4],
 //!         ]);
 //! let y = vec![
-//!              0., 0., 0., 0., 0., 0., 0., 0.,
-//!              1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
+//!              0, 0, 0, 0, 0, 0, 0, 0,
+//!              1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 //!         ];
 //!
 //! let classifier = RandomForestClassifier::fit(&x, &y, Default::default()).unwrap();
@@ -45,8 +45,8 @@
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
-use rand::rngs::StdRng;
-use rand::{Rng, SeedableRng};
+use rand::Rng;
+
 use std::default::Default;
 use std::fmt::Debug;
 
@@ -55,8 +55,11 @@ use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::{Failed, FailedError};
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+
+use crate::rand_custom::get_rng_impl;
 use crate::tree::decision_tree_classifier::{
     which_max, DecisionTreeClassifier, DecisionTreeClassifierParameters, SplitCriterion,
 };
@@ -66,20 +69,28 @@ use crate::tree::decision_tree_classifier::{
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 pub struct RandomForestClassifierParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Split criteria to use when building a tree. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
     pub criterion: SplitCriterion,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Tree max depth. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
     pub max_depth: Option<u16>,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The minimum number of samples required to be at a leaf node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
     pub min_samples_leaf: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The minimum number of samples required to split an internal node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
     pub min_samples_split: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The number of trees in the forest.
     pub n_trees: u16,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Number of random sample of predictors to use as split candidates.
     pub m: Option<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Whether to keep samples used for tree generation. This is required for OOB prediction.
     pub keep_samples: bool,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Seed used for bootstrap sampling and feature selection for each tree.
     pub seed: u64,
 }
@@ -87,10 +98,14 @@ pub struct RandomForestClassifierParameters {
 /// Random Forest Classifier
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct RandomForestClassifier<T: RealNumber> {
-    _parameters: RandomForestClassifierParameters,
-    trees: Vec<DecisionTreeClassifier<T>>,
-    classes: Vec<T>,
+pub struct RandomForestClassifier<
+    TX: Number + FloatNumber + PartialOrd,
+    TY: Number + Ord,
+    X: Array2<TX>,
+    Y: Array1<TY>,
+> {
+    trees: Option<Vec<DecisionTreeClassifier<TX, TY, X, Y>>>,
+    classes: Option<Vec<TY>>,
     samples: Option<Vec<Vec<bool>>>,
 }
 
@@ -139,22 +154,24 @@ impl RandomForestClassifierParameters {
     }
 }
 
-impl<T: RealNumber> PartialEq for RandomForestClassifier<T> {
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number + Ord, X: Array2<TX>, Y: Array1<TY>>
+    PartialEq for RandomForestClassifier<TX, TY, X, Y>
+{
     fn eq(&self, other: &Self) -> bool {
-        if self.classes.len() != other.classes.len() || self.trees.len() != other.trees.len() {
+        if self.classes.as_ref().unwrap().len() != other.classes.as_ref().unwrap().len()
+            || self.trees.as_ref().unwrap().len() != other.trees.as_ref().unwrap().len()
+        {
             false
         } else {
-            for i in 0..self.classes.len() {
-                if (self.classes[i] - other.classes[i]).abs() > T::epsilon() {
-                    return false;
-                }
-            }
-            for i in 0..self.trees.len() {
-                if self.trees[i] != other.trees[i] {
-                    return false;
-                }
-            }
-            true
+            self.classes
+                .iter()
+                .zip(other.classes.iter())
+                .all(|(a, b)| a == b)
+                && self
+                    .trees
+                    .iter()
+                    .zip(other.trees.iter())
+                    .all(|(a, b)| a == b)
         }
     }
 }
@@ -163,7 +180,7 @@ impl Default for RandomForestClassifierParameters {
     fn default() -> Self {
         RandomForestClassifierParameters {
             criterion: SplitCriterion::Gini,
-            max_depth: None,
+            max_depth: Option::None,
             min_samples_leaf: 1,
             min_samples_split: 2,
             n_trees: 100,
@@ -174,65 +191,298 @@ impl Default for RandomForestClassifierParameters {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>>
-    SupervisedEstimator<M, M::RowVector, RandomForestClassifierParameters>
-    for RandomForestClassifier<T>
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number + Ord, X: Array2<TX>, Y: Array1<TY>>
+    SupervisedEstimator<X, Y, RandomForestClassifierParameters>
+    for RandomForestClassifier<TX, TY, X, Y>
 {
-    fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: RandomForestClassifierParameters,
-    ) -> Result<Self, Failed> {
+    fn new() -> Self {
+        Self {
+            trees: Option::None,
+            classes: Option::None,
+            samples: Option::None,
+        }
+    }
+    fn fit(x: &X, y: &Y, parameters: RandomForestClassifierParameters) -> Result<Self, Failed> {
         RandomForestClassifier::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for RandomForestClassifier<T> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number + Ord, X: Array2<TX>, Y: Array1<TY>>
+    Predictor<X, Y> for RandomForestClassifier<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber> RandomForestClassifier<T> {
+/// RandomForestClassifier grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct RandomForestClassifierSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Split criteria to use when building a tree. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub criterion: Vec<SplitCriterion>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Tree max depth. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub max_depth: Vec<Option<u16>>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The minimum number of samples required to be at a leaf node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub min_samples_leaf: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The minimum number of samples required to split an internal node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub min_samples_split: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The number of trees in the forest.
+    pub n_trees: Vec<u16>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Number of random sample of predictors to use as split candidates.
+    pub m: Vec<Option<usize>>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Whether to keep samples used for tree generation. This is required for OOB prediction.
+    pub keep_samples: Vec<bool>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Seed used for bootstrap sampling and feature selection for each tree.
+    pub seed: Vec<u64>,
+}
+
+/// RandomForestClassifier grid search iterator
+pub struct RandomForestClassifierSearchParametersIterator {
+    random_forest_classifier_search_parameters: RandomForestClassifierSearchParameters,
+    current_criterion: usize,
+    current_max_depth: usize,
+    current_min_samples_leaf: usize,
+    current_min_samples_split: usize,
+    current_n_trees: usize,
+    current_m: usize,
+    current_keep_samples: usize,
+    current_seed: usize,
+}
+
+impl IntoIterator for RandomForestClassifierSearchParameters {
+    type Item = RandomForestClassifierParameters;
+    type IntoIter = RandomForestClassifierSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        RandomForestClassifierSearchParametersIterator {
+            random_forest_classifier_search_parameters: self,
+            current_criterion: 0,
+            current_max_depth: 0,
+            current_min_samples_leaf: 0,
+            current_min_samples_split: 0,
+            current_n_trees: 0,
+            current_m: 0,
+            current_keep_samples: 0,
+            current_seed: 0,
+        }
+    }
+}
+
+impl Iterator for RandomForestClassifierSearchParametersIterator {
+    type Item = RandomForestClassifierParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_criterion
+            == self
+                .random_forest_classifier_search_parameters
+                .criterion
+                .len()
+            && self.current_max_depth
+                == self
+                    .random_forest_classifier_search_parameters
+                    .max_depth
+                    .len()
+            && self.current_min_samples_leaf
+                == self
+                    .random_forest_classifier_search_parameters
+                    .min_samples_leaf
+                    .len()
+            && self.current_min_samples_split
+                == self
+                    .random_forest_classifier_search_parameters
+                    .min_samples_split
+                    .len()
+            && self.current_n_trees
+                == self
+                    .random_forest_classifier_search_parameters
+                    .n_trees
+                    .len()
+            && self.current_m == self.random_forest_classifier_search_parameters.m.len()
+            && self.current_keep_samples
+                == self
+                    .random_forest_classifier_search_parameters
+                    .keep_samples
+                    .len()
+            && self.current_seed == self.random_forest_classifier_search_parameters.seed.len()
+        {
+            return None;
+        }
+
+        let next = RandomForestClassifierParameters {
+            criterion: self.random_forest_classifier_search_parameters.criterion
+                [self.current_criterion]
+                .clone(),
+            max_depth: self.random_forest_classifier_search_parameters.max_depth
+                [self.current_max_depth],
+            min_samples_leaf: self
+                .random_forest_classifier_search_parameters
+                .min_samples_leaf[self.current_min_samples_leaf],
+            min_samples_split: self
+                .random_forest_classifier_search_parameters
+                .min_samples_split[self.current_min_samples_split],
+            n_trees: self.random_forest_classifier_search_parameters.n_trees[self.current_n_trees],
+            m: self.random_forest_classifier_search_parameters.m[self.current_m],
+            keep_samples: self.random_forest_classifier_search_parameters.keep_samples
+                [self.current_keep_samples],
+            seed: self.random_forest_classifier_search_parameters.seed[self.current_seed],
+        };
+
+        if self.current_criterion + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .criterion
+                .len()
+        {
+            self.current_criterion += 1;
+        } else if self.current_max_depth + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .max_depth
+                .len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth += 1;
+        } else if self.current_min_samples_leaf + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .min_samples_leaf
+                .len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf += 1;
+        } else if self.current_min_samples_split + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .min_samples_split
+                .len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split += 1;
+        } else if self.current_n_trees + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .n_trees
+                .len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees += 1;
+        } else if self.current_m + 1 < self.random_forest_classifier_search_parameters.m.len() {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m += 1;
+        } else if self.current_keep_samples + 1
+            < self
+                .random_forest_classifier_search_parameters
+                .keep_samples
+                .len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m = 0;
+            self.current_keep_samples += 1;
+        } else if self.current_seed + 1 < self.random_forest_classifier_search_parameters.seed.len()
+        {
+            self.current_criterion = 0;
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m = 0;
+            self.current_keep_samples = 0;
+            self.current_seed += 1;
+        } else {
+            self.current_criterion += 1;
+            self.current_max_depth += 1;
+            self.current_min_samples_leaf += 1;
+            self.current_min_samples_split += 1;
+            self.current_n_trees += 1;
+            self.current_m += 1;
+            self.current_keep_samples += 1;
+            self.current_seed += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for RandomForestClassifierSearchParameters {
+    fn default() -> Self {
+        let default_params = RandomForestClassifierParameters::default();
+
+        RandomForestClassifierSearchParameters {
+            criterion: vec![default_params.criterion],
+            max_depth: vec![default_params.max_depth],
+            min_samples_leaf: vec![default_params.min_samples_leaf],
+            min_samples_split: vec![default_params.min_samples_split],
+            n_trees: vec![default_params.n_trees],
+            m: vec![default_params.m],
+            keep_samples: vec![default_params.keep_samples],
+            seed: vec![default_params.seed],
+        }
+    }
+}
+
+impl<TX: FloatNumber + PartialOrd, TY: Number + Ord, X: Array2<TX>, Y: Array1<TY>>
+    RandomForestClassifier<TX, TY, X, Y>
+{
     /// Build a forest of trees from the training set.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - the target class values
-    pub fn fit<M: Matrix<T>>(
-        x: &M,
-        y: &M::RowVector,
+    pub fn fit(
+        x: &X,
+        y: &Y,
         parameters: RandomForestClassifierParameters,
-    ) -> Result<RandomForestClassifier<T>, Failed> {
+    ) -> Result<RandomForestClassifier<TX, TY, X, Y>, Failed> {
         let (_, num_attributes) = x.shape();
-        let y_m = M::from_row_vector(y.clone());
-        let (_, y_ncols) = y_m.shape();
+        let y_ncols = y.shape();
         let mut yi: Vec<usize> = vec![0; y_ncols];
-        let classes = y_m.unique();
+        let classes = y.unique();
 
         for (i, yi_i) in yi.iter_mut().enumerate().take(y_ncols) {
-            let yc = y_m.get(0, i);
-            *yi_i = classes.iter().position(|c| yc == *c).unwrap();
+            let yc = y.get(i);
+            *yi_i = classes.iter().position(|c| yc == c).unwrap();
         }
 
-        let mtry = parameters.m.unwrap_or_else(|| {
-            (T::from(num_attributes).unwrap())
-                .sqrt()
-                .floor()
-                .to_usize()
-                .unwrap()
-        });
+        let mtry = parameters
+            .m
+            .unwrap_or_else(|| ((num_attributes as f64).sqrt().floor()) as usize);
 
-        let mut rng = StdRng::seed_from_u64(parameters.seed);
-        let classes = y_m.unique();
+        let mut rng = get_rng_impl(Some(parameters.seed));
+        let classes = y.unique();
         let k = classes.len();
-        let mut trees: Vec<DecisionTreeClassifier<T>> = Vec::new();
+        // TODO: use with_capacity here
+        let mut trees: Vec<DecisionTreeClassifier<TX, TY, X, Y>> = Vec::new();
 
         let mut maybe_all_samples: Option<Vec<Vec<bool>>> = Option::None;
         if parameters.keep_samples {
+            // TODO: use with_capacity here
             maybe_all_samples = Some(Vec::new());
         }
 
         for _ in 0..parameters.n_trees {
-            let samples = RandomForestClassifier::<T>::sample_with_replacement(&yi, k, &mut rng);
+            let samples: Vec<usize> =
+                RandomForestClassifier::<TX, TY, X, Y>::sample_with_replacement(&yi, k, &mut rng);
             if let Some(ref mut all_samples) = maybe_all_samples {
                 all_samples.push(samples.iter().map(|x| *x != 0).collect())
             }
@@ -242,38 +492,40 @@ impl<T: RealNumber> RandomForestClassifier<T> {
                 max_depth: parameters.max_depth,
                 min_samples_leaf: parameters.min_samples_leaf,
                 min_samples_split: parameters.min_samples_split,
+                seed: Some(parameters.seed),
             };
-            let tree =
-                DecisionTreeClassifier::fit_weak_learner(x, y, samples, mtry, params, &mut rng)?;
+            let tree = DecisionTreeClassifier::fit_weak_learner(x, y, samples, mtry, params)?;
             trees.push(tree);
         }
 
         Ok(RandomForestClassifier {
-            _parameters: parameters,
-            trees,
-            classes,
+            trees: Some(trees),
+            classes: Some(classes),
             samples: maybe_all_samples,
         })
     }
 
     /// Predict class for `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
-        let mut result = M::zeros(1, x.shape().0);
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
+        let mut result = Y::zeros(x.shape().0);
 
         let (n, _) = x.shape();
 
         for i in 0..n {
-            result.set(0, i, self.classes[self.predict_for_row(x, i)]);
+            result.set(
+                i,
+                self.classes.as_ref().unwrap()[self.predict_for_row(x, i)],
+            );
         }
 
-        Ok(result.to_row_vector())
+        Ok(result)
     }
 
-    fn predict_for_row<M: Matrix<T>>(&self, x: &M, row: usize) -> usize {
-        let mut result = vec![0; self.classes.len()];
+    fn predict_for_row(&self, x: &X, row: usize) -> usize {
+        let mut result = vec![0; self.classes.as_ref().unwrap().len()];
 
-        for tree in self.trees.iter() {
+        for tree in self.trees.as_ref().unwrap().iter() {
             result[tree.predict_for_row(x, row)] += 1;
         }
 
@@ -281,7 +533,7 @@ impl<T: RealNumber> RandomForestClassifier<T> {
     }
 
     /// Predict OOB classes for `x`. `x` is expected to be equal to the dataset used in training.
-    pub fn predict_oob<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict_oob(&self, x: &X) -> Result<Y, Failed> {
         let (n, _) = x.shape();
         if self.samples.is_none() {
             Err(Failed::because(
@@ -294,20 +546,28 @@ impl<T: RealNumber> RandomForestClassifier<T> {
                 "Prediction matrix must match matrix used in training for OOB predictions.",
             ))
         } else {
-            let mut result = M::zeros(1, n);
+            let mut result = Y::zeros(n);
 
             for i in 0..n {
-                result.set(0, i, self.classes[self.predict_for_row_oob(x, i)]);
+                result.set(
+                    i,
+                    self.classes.as_ref().unwrap()[self.predict_for_row_oob(x, i)],
+                );
             }
-
-            Ok(result.to_row_vector())
+            Ok(result)
         }
     }
 
-    fn predict_for_row_oob<M: Matrix<T>>(&self, x: &M, row: usize) -> usize {
-        let mut result = vec![0; self.classes.len()];
+    fn predict_for_row_oob(&self, x: &X, row: usize) -> usize {
+        let mut result = vec![0; self.classes.as_ref().unwrap().len()];
 
-        for (tree, samples) in self.trees.iter().zip(self.samples.as_ref().unwrap()) {
+        for (tree, samples) in self
+            .trees
+            .as_ref()
+            .unwrap()
+            .iter()
+            .zip(self.samples.as_ref().unwrap())
+        {
             if !samples[row] {
                 result[tree.predict_for_row(x, row)] += 1;
             }
@@ -343,12 +603,38 @@ impl<T: RealNumber> RandomForestClassifier<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
     use crate::metrics::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
     #[test]
-    fn fit_predict_iris() {
+    fn search_parameters() {
+        let parameters = RandomForestClassifierSearchParameters {
+            n_trees: vec![10, 100],
+            m: vec![None, Some(1)],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 10);
+        assert_eq!(next.m, None);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 100);
+        assert_eq!(next.m, None);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 10);
+        assert_eq!(next.m, Some(1));
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 100);
+        assert_eq!(next.m, Some(1));
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn fit_predict() {
         let x = DenseMatrix::from_2d_array(&[
             &[5.1, 3.5, 1.4, 0.2],
             &[4.9, 3.0, 1.4, 0.2],
@@ -371,16 +657,14 @@ mod tests {
             &[6.6, 2.9, 4.6, 1.3],
             &[5.2, 2.7, 3.9, 1.4],
         ]);
-        let y = vec![
-            0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
-        ];
+        let y = vec![0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1];
 
         let classifier = RandomForestClassifier::fit(
             &x,
             &y,
             RandomForestClassifierParameters {
                 criterion: SplitCriterion::Gini,
-                max_depth: None,
+                max_depth: Option::None,
                 min_samples_leaf: 1,
                 min_samples_split: 2,
                 n_trees: 100,
@@ -394,7 +678,10 @@ mod tests {
         assert!(accuracy(&y, &classifier.predict(&x).unwrap()) >= 0.95);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn fit_predict_iris_oob() {
         let x = DenseMatrix::from_2d_array(&[
@@ -419,16 +706,14 @@ mod tests {
             &[6.6, 2.9, 4.6, 1.3],
             &[5.2, 2.7, 3.9, 1.4],
         ]);
-        let y = vec![
-            0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
-        ];
+        let y = vec![0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1];
 
         let classifier = RandomForestClassifier::fit(
             &x,
             &y,
             RandomForestClassifierParameters {
                 criterion: SplitCriterion::Gini,
-                max_depth: None,
+                max_depth: Option::None,
                 min_samples_leaf: 1,
                 min_samples_split: 2,
                 n_trees: 100,
@@ -445,7 +730,10 @@ mod tests {
         );
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
@@ -471,13 +759,11 @@ mod tests {
             &[6.6, 2.9, 4.6, 1.3],
             &[5.2, 2.7, 3.9, 1.4],
         ]);
-        let y = vec![
-            0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
-        ];
+        let y = vec![0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1];
 
         let forest = RandomForestClassifier::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_forest: RandomForestClassifier<f64> =
+        let deserialized_forest: RandomForestClassifier<f64, i64, DenseMatrix<f64>, Vec<i64>> =
             bincode::deserialize(&bincode::serialize(&forest).unwrap()).unwrap();
 
         assert_eq!(forest, deserialized_forest);

src/ensemble/random_forest_regressor.rs

@@ -8,7 +8,7 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::ensemble::random_forest_regressor::*;
 //!
 //! // Longley dataset (https://www.statsmodels.org/stable/datasets/generated/longley.html)
@@ -43,8 +43,7 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 
-use rand::rngs::StdRng;
-use rand::{Rng, SeedableRng};
+use rand::Rng;
 use std::default::Default;
 use std::fmt::Debug;
 
@@ -53,8 +52,11 @@ use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::{Failed, FailedError};
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+
+use crate::rand_custom::get_rng_impl;
 use crate::tree::decision_tree_regressor::{
     DecisionTreeRegressor, DecisionTreeRegressorParameters,
 };
@@ -64,18 +66,25 @@ use crate::tree::decision_tree_regressor::{
 /// Parameters of the Random Forest Regressor
 /// Some parameters here are passed directly into base estimator.
 pub struct RandomForestRegressorParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Tree max depth. See [Decision Tree Regressor](../../tree/decision_tree_regressor/index.html)
     pub max_depth: Option<u16>,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The minimum number of samples required to be at a leaf node. See [Decision Tree Regressor](../../tree/decision_tree_regressor/index.html)
     pub min_samples_leaf: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The minimum number of samples required to split an internal node. See [Decision Tree Regressor](../../tree/decision_tree_regressor/index.html)
     pub min_samples_split: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The number of trees in the forest.
     pub n_trees: usize,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Number of random sample of predictors to use as split candidates.
     pub m: Option<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Whether to keep samples used for tree generation. This is required for OOB prediction.
     pub keep_samples: bool,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Seed used for bootstrap sampling and feature selection for each tree.
     pub seed: u64,
 }
@@ -83,9 +92,13 @@ pub struct RandomForestRegressorParameters {
 /// Random Forest Regressor
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct RandomForestRegressor<T: RealNumber> {
-    _parameters: RandomForestRegressorParameters,
-    trees: Vec<DecisionTreeRegressor<T>>,
+pub struct RandomForestRegressor<
+    TX: Number + FloatNumber + PartialOrd,
+    TY: Number,
+    X: Array2<TX>,
+    Y: Array1<TY>,
+> {
+    trees: Option<Vec<DecisionTreeRegressor<TX, TY, X, Y>>>,
     samples: Option<Vec<Vec<bool>>>,
 }
 
@@ -131,7 +144,7 @@ impl RandomForestRegressorParameters {
 impl Default for RandomForestRegressorParameters {
     fn default() -> Self {
         RandomForestRegressorParameters {
-            max_depth: None,
+            max_depth: Option::None,
             min_samples_leaf: 1,
             min_samples_split: 2,
             n_trees: 10,
@@ -142,113 +155,316 @@ impl Default for RandomForestRegressorParameters {
     }
 }
 
-impl<T: RealNumber> PartialEq for RandomForestRegressor<T> {
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number, X: Array2<TX>, Y: Array1<TY>> PartialEq
+    for RandomForestRegressor<TX, TY, X, Y>
+{
     fn eq(&self, other: &Self) -> bool {
-        if self.trees.len() != other.trees.len() {
+        if self.trees.as_ref().unwrap().len() != other.trees.as_ref().unwrap().len() {
             false
         } else {
-            for i in 0..self.trees.len() {
-                if self.trees[i] != other.trees[i] {
-                    return false;
-                }
-            }
-            true
+            self.trees
+                .iter()
+                .zip(other.trees.iter())
+                .all(|(a, b)| a == b)
         }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>>
-    SupervisedEstimator<M, M::RowVector, RandomForestRegressorParameters>
-    for RandomForestRegressor<T>
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    SupervisedEstimator<X, Y, RandomForestRegressorParameters>
+    for RandomForestRegressor<TX, TY, X, Y>
 {
-    fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: RandomForestRegressorParameters,
-    ) -> Result<Self, Failed> {
+    fn new() -> Self {
+        Self {
+            trees: Option::None,
+            samples: Option::None,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: RandomForestRegressorParameters) -> Result<Self, Failed> {
         RandomForestRegressor::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for RandomForestRegressor<T> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    Predictor<X, Y> for RandomForestRegressor<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber> RandomForestRegressor<T> {
+/// RandomForestRegressor grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct RandomForestRegressorSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Tree max depth. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub max_depth: Vec<Option<u16>>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The minimum number of samples required to be at a leaf node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub min_samples_leaf: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The minimum number of samples required to split an internal node. See [Decision Tree Classifier](../../tree/decision_tree_classifier/index.html)
+    pub min_samples_split: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The number of trees in the forest.
+    pub n_trees: Vec<usize>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Number of random sample of predictors to use as split candidates.
+    pub m: Vec<Option<usize>>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Whether to keep samples used for tree generation. This is required for OOB prediction.
+    pub keep_samples: Vec<bool>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Seed used for bootstrap sampling and feature selection for each tree.
+    pub seed: Vec<u64>,
+}
+
+/// RandomForestRegressor grid search iterator
+pub struct RandomForestRegressorSearchParametersIterator {
+    random_forest_regressor_search_parameters: RandomForestRegressorSearchParameters,
+    current_max_depth: usize,
+    current_min_samples_leaf: usize,
+    current_min_samples_split: usize,
+    current_n_trees: usize,
+    current_m: usize,
+    current_keep_samples: usize,
+    current_seed: usize,
+}
+
+impl IntoIterator for RandomForestRegressorSearchParameters {
+    type Item = RandomForestRegressorParameters;
+    type IntoIter = RandomForestRegressorSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        RandomForestRegressorSearchParametersIterator {
+            random_forest_regressor_search_parameters: self,
+            current_max_depth: 0,
+            current_min_samples_leaf: 0,
+            current_min_samples_split: 0,
+            current_n_trees: 0,
+            current_m: 0,
+            current_keep_samples: 0,
+            current_seed: 0,
+        }
+    }
+}
+
+impl Iterator for RandomForestRegressorSearchParametersIterator {
+    type Item = RandomForestRegressorParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_max_depth
+            == self
+                .random_forest_regressor_search_parameters
+                .max_depth
+                .len()
+            && self.current_min_samples_leaf
+                == self
+                    .random_forest_regressor_search_parameters
+                    .min_samples_leaf
+                    .len()
+            && self.current_min_samples_split
+                == self
+                    .random_forest_regressor_search_parameters
+                    .min_samples_split
+                    .len()
+            && self.current_n_trees == self.random_forest_regressor_search_parameters.n_trees.len()
+            && self.current_m == self.random_forest_regressor_search_parameters.m.len()
+            && self.current_keep_samples
+                == self
+                    .random_forest_regressor_search_parameters
+                    .keep_samples
+                    .len()
+            && self.current_seed == self.random_forest_regressor_search_parameters.seed.len()
+        {
+            return None;
+        }
+
+        let next = RandomForestRegressorParameters {
+            max_depth: self.random_forest_regressor_search_parameters.max_depth
+                [self.current_max_depth],
+            min_samples_leaf: self
+                .random_forest_regressor_search_parameters
+                .min_samples_leaf[self.current_min_samples_leaf],
+            min_samples_split: self
+                .random_forest_regressor_search_parameters
+                .min_samples_split[self.current_min_samples_split],
+            n_trees: self.random_forest_regressor_search_parameters.n_trees[self.current_n_trees],
+            m: self.random_forest_regressor_search_parameters.m[self.current_m],
+            keep_samples: self.random_forest_regressor_search_parameters.keep_samples
+                [self.current_keep_samples],
+            seed: self.random_forest_regressor_search_parameters.seed[self.current_seed],
+        };
+
+        if self.current_max_depth + 1
+            < self
+                .random_forest_regressor_search_parameters
+                .max_depth
+                .len()
+        {
+            self.current_max_depth += 1;
+        } else if self.current_min_samples_leaf + 1
+            < self
+                .random_forest_regressor_search_parameters
+                .min_samples_leaf
+                .len()
+        {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf += 1;
+        } else if self.current_min_samples_split + 1
+            < self
+                .random_forest_regressor_search_parameters
+                .min_samples_split
+                .len()
+        {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split += 1;
+        } else if self.current_n_trees + 1
+            < self.random_forest_regressor_search_parameters.n_trees.len()
+        {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees += 1;
+        } else if self.current_m + 1 < self.random_forest_regressor_search_parameters.m.len() {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m += 1;
+        } else if self.current_keep_samples + 1
+            < self
+                .random_forest_regressor_search_parameters
+                .keep_samples
+                .len()
+        {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m = 0;
+            self.current_keep_samples += 1;
+        } else if self.current_seed + 1 < self.random_forest_regressor_search_parameters.seed.len()
+        {
+            self.current_max_depth = 0;
+            self.current_min_samples_leaf = 0;
+            self.current_min_samples_split = 0;
+            self.current_n_trees = 0;
+            self.current_m = 0;
+            self.current_keep_samples = 0;
+            self.current_seed += 1;
+        } else {
+            self.current_max_depth += 1;
+            self.current_min_samples_leaf += 1;
+            self.current_min_samples_split += 1;
+            self.current_n_trees += 1;
+            self.current_m += 1;
+            self.current_keep_samples += 1;
+            self.current_seed += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for RandomForestRegressorSearchParameters {
+    fn default() -> Self {
+        let default_params = RandomForestRegressorParameters::default();
+
+        RandomForestRegressorSearchParameters {
+            max_depth: vec![default_params.max_depth],
+            min_samples_leaf: vec![default_params.min_samples_leaf],
+            min_samples_split: vec![default_params.min_samples_split],
+            n_trees: vec![default_params.n_trees],
+            m: vec![default_params.m],
+            keep_samples: vec![default_params.keep_samples],
+            seed: vec![default_params.seed],
+        }
+    }
+}
+
+impl<TX: Number + FloatNumber + PartialOrd, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    RandomForestRegressor<TX, TY, X, Y>
+{
     /// Build a forest of trees from the training set.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - the target class values
-    pub fn fit<M: Matrix<T>>(
-        x: &M,
-        y: &M::RowVector,
+    pub fn fit(
+        x: &X,
+        y: &Y,
         parameters: RandomForestRegressorParameters,
-    ) -> Result<RandomForestRegressor<T>, Failed> {
+    ) -> Result<RandomForestRegressor<TX, TY, X, Y>, Failed> {
         let (n_rows, num_attributes) = x.shape();
 
         let mtry = parameters
             .m
             .unwrap_or((num_attributes as f64).sqrt().floor() as usize);
 
-        let mut rng = StdRng::seed_from_u64(parameters.seed);
-        let mut trees: Vec<DecisionTreeRegressor<T>> = Vec::new();
+        let mut rng = get_rng_impl(Some(parameters.seed));
+        let mut trees: Vec<DecisionTreeRegressor<TX, TY, X, Y>> = Vec::new();
 
         let mut maybe_all_samples: Option<Vec<Vec<bool>>> = Option::None;
         if parameters.keep_samples {
+            // TODO: use with_capacity here
             maybe_all_samples = Some(Vec::new());
         }
 
         for _ in 0..parameters.n_trees {
-            let samples = RandomForestRegressor::<T>::sample_with_replacement(n_rows, &mut rng);
+            let samples: Vec<usize> =
+                RandomForestRegressor::<TX, TY, X, Y>::sample_with_replacement(n_rows, &mut rng);
+
+            // keep samples is flag is on
             if let Some(ref mut all_samples) = maybe_all_samples {
                 all_samples.push(samples.iter().map(|x| *x != 0).collect())
             }
+
             let params = DecisionTreeRegressorParameters {
                 max_depth: parameters.max_depth,
                 min_samples_leaf: parameters.min_samples_leaf,
                 min_samples_split: parameters.min_samples_split,
+                seed: Some(parameters.seed),
             };
-            let tree =
-                DecisionTreeRegressor::fit_weak_learner(x, y, samples, mtry, params, &mut rng)?;
+            let tree = DecisionTreeRegressor::fit_weak_learner(x, y, samples, mtry, params)?;
             trees.push(tree);
         }
 
         Ok(RandomForestRegressor {
-            _parameters: parameters,
-            trees,
+            trees: Some(trees),
             samples: maybe_all_samples,
         })
     }
 
     /// Predict class for `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
-        let mut result = M::zeros(1, x.shape().0);
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
+        let mut result = Y::zeros(x.shape().0);
 
         let (n, _) = x.shape();
 
         for i in 0..n {
-            result.set(0, i, self.predict_for_row(x, i));
+            result.set(i, self.predict_for_row(x, i));
         }
 
-        Ok(result.to_row_vector())
+        Ok(result)
     }
 
-    fn predict_for_row<M: Matrix<T>>(&self, x: &M, row: usize) -> T {
-        let n_trees = self.trees.len();
+    fn predict_for_row(&self, x: &X, row: usize) -> TY {
+        let n_trees = self.trees.as_ref().unwrap().len();
 
-        let mut result = T::zero();
+        let mut result = TY::zero();
 
-        for tree in self.trees.iter() {
+        for tree in self.trees.as_ref().unwrap().iter() {
             result += tree.predict_for_row(x, row);
         }
 
-        result / T::from(n_trees).unwrap()
+        result / TY::from_usize(n_trees).unwrap()
     }
 
     /// Predict OOB classes for `x`. `x` is expected to be equal to the dataset used in training.
-    pub fn predict_oob<M: Matrix<T>>(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict_oob(&self, x: &X) -> Result<Y, Failed> {
         let (n, _) = x.shape();
         if self.samples.is_none() {
             Err(Failed::because(
@@ -261,21 +477,27 @@ impl<T: RealNumber> RandomForestRegressor<T> {
                 "Prediction matrix must match matrix used in training for OOB predictions.",
             ))
         } else {
-            let mut result = M::zeros(1, n);
+            let mut result = Y::zeros(n);
 
             for i in 0..n {
-                result.set(0, i, self.predict_for_row_oob(x, i));
+                result.set(i, self.predict_for_row_oob(x, i));
             }
 
-            Ok(result.to_row_vector())
+            Ok(result)
         }
     }
 
-    fn predict_for_row_oob<M: Matrix<T>>(&self, x: &M, row: usize) -> T {
+    fn predict_for_row_oob(&self, x: &X, row: usize) -> TY {
         let mut n_trees = 0;
-        let mut result = T::zero();
+        let mut result = TY::zero();
 
-        for (tree, samples) in self.trees.iter().zip(self.samples.as_ref().unwrap()) {
+        for (tree, samples) in self
+            .trees
+            .as_ref()
+            .unwrap()
+            .iter()
+            .zip(self.samples.as_ref().unwrap())
+        {
             if !samples[row] {
                 result += tree.predict_for_row(x, row);
                 n_trees += 1;
@@ -283,7 +505,7 @@ impl<T: RealNumber> RandomForestRegressor<T> {
         }
 
         // TODO: What to do if there are no oob trees?
-        result / T::from(n_trees).unwrap()
+        result / TY::from(n_trees).unwrap()
     }
 
     fn sample_with_replacement(nrows: usize, rng: &mut impl Rng) -> Vec<usize> {
@@ -299,10 +521,36 @@ impl<T: RealNumber> RandomForestRegressor<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
     use crate::metrics::mean_absolute_error;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = RandomForestRegressorSearchParameters {
+            n_trees: vec![10, 100],
+            m: vec![None, Some(1)],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 10);
+        assert_eq!(next.m, None);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 100);
+        assert_eq!(next.m, None);
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 10);
+        assert_eq!(next.m, Some(1));
+        let next = iter.next().unwrap();
+        assert_eq!(next.n_trees, 100);
+        assert_eq!(next.m, Some(1));
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn fit_longley() {
         let x = DenseMatrix::from_2d_array(&[
@@ -332,7 +580,7 @@ mod tests {
             &x,
             &y,
             RandomForestRegressorParameters {
-                max_depth: None,
+                max_depth: Option::None,
                 min_samples_leaf: 1,
                 min_samples_split: 2,
                 n_trees: 1000,
@@ -347,7 +595,10 @@ mod tests {
         assert!(mean_absolute_error(&y, &y_hat) < 1.0);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn fit_predict_longley_oob() {
         let x = DenseMatrix::from_2d_array(&[
@@ -377,7 +628,7 @@ mod tests {
             &x,
             &y,
             RandomForestRegressorParameters {
-                max_depth: None,
+                max_depth: Option::None,
                 min_samples_leaf: 1,
                 min_samples_split: 2,
                 n_trees: 1000,
@@ -391,10 +642,16 @@ mod tests {
         let y_hat = regressor.predict(&x).unwrap();
         let y_hat_oob = regressor.predict_oob(&x).unwrap();
 
+        println!("{:?}", mean_absolute_error(&y, &y_hat));
+        println!("{:?}", mean_absolute_error(&y, &y_hat_oob));
+
         assert!(mean_absolute_error(&y, &y_hat) < mean_absolute_error(&y, &y_hat_oob));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
@@ -423,7 +680,7 @@ mod tests {
 
         let forest = RandomForestRegressor::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_forest: RandomForestRegressor<f64> =
+        let deserialized_forest: RandomForestRegressor<f64, f64, DenseMatrix<f64>, Vec<f64>> =
             bincode::deserialize(&bincode::serialize(&forest).unwrap()).unwrap();
 
         assert_eq!(forest, deserialized_forest);

src/error/mod.rs

@@ -30,6 +30,8 @@ pub enum FailedError {
     DecompositionFailed,
     /// Can't solve for x
     SolutionFailed,
+    /// Erro in input
+    ParametersError,
 }
 
 impl Failed {
@@ -94,6 +96,7 @@ impl fmt::Display for FailedError {
             FailedError::FindFailed => "Find failed",
             FailedError::DecompositionFailed => "Decomposition failed",
             FailedError::SolutionFailed => "Can't find solution",
+            FailedError::ParametersError => "Error in input, check parameters",
         };
         write!(f, "{}", failed_err_str)
     }

src/lib.rs

@@ -8,27 +8,76 @@
 #![warn(missing_docs)]
 #![warn(rustdoc::missing_doc_code_examples)]
 
-//! # SmartCore
+//! # smartcore
 //!
-//! Welcome to SmartCore, the most advanced machine learning library in Rust!
+//! Welcome to `smartcore`, machine learning in Rust!
 //!
-//! SmartCore features various classification, regression and clustering algorithms including support vector machines, random forests, k-means and DBSCAN,
+//! `smartcore` features various classification, regression and clustering algorithms including support vector machines, random forests, k-means and DBSCAN,
 //! as well as tools for model selection and model evaluation.
 //!
-//! SmartCore is well integrated with a with wide variaty of libraries that provide support for large, multi-dimensional arrays and matrices. At this moment,
-//! all Smartcore's algorithms work with ordinary Rust vectors, as well as matrices and vectors defined in these packages:
-//! * [ndarray](https://docs.rs/ndarray)
-//! * [nalgebra](https://docs.rs/nalgebra/)
+//! `smartcore` provides its own traits system that extends Rust standard library, to deal with linear algebra and common
+//! computational models. Its API is designed using well recognizable patterns. Extra features (like support for [ndarray](https://docs.rs/ndarray)
+//! structures) is available via optional features.
 //!
 //! ## Getting Started
 //!
-//! To start using SmartCore simply add the following to your Cargo.toml file:
+//! To start using `smartcore` latest stable version simply add the following to your `Cargo.toml` file:
 //! ```ignore
 //! [dependencies]
-//! smartcore = "0.2.0"
+//! smartcore = "*"
 //! ```
 //!
-//! All machine learning algorithms in SmartCore are grouped into these broad categories:
+//! To start using smartcore development version with latest unstable additions:
+//! ```ignore
+//! [dependencies]
+//! smartcore = { git = "https://github.com/smartcorelib/smartcore", branch = "development" }
+//! ```
+//!
+//! There are different features that can be added to the base library, for example to add sample datasets:
+//! ```ignore
+//! [dependencies]
+//! smartcore = { git = "https://github.com/smartcorelib/smartcore", features = ["datasets"] }
+//! ```
+//! Check `smartcore`'s `Cargo.toml` for available features.
+//!
+//! ## Using Jupyter
+//! For quick introduction, Jupyter Notebooks are available [here](https://github.com/smartcorelib/smartcore-jupyter/tree/main/notebooks).
+//! You can set up a local environment to run Rust notebooks using [EVCXR](https://github.com/google/evcxr)
+//! following [these instructions](https://depth-first.com/articles/2020/09/21/interactive-rust-in-a-repl-and-jupyter-notebook-with-evcxr/).
+//!
+//!
+//! ## First Example
+//! For example, you can use this code to fit a [K Nearest Neighbors classifier](neighbors/knn_classifier/index.html) to a dataset that is defined as standard Rust vector:
+//!
+//! ```
+//! // DenseMatrix definition
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! // KNNClassifier
+//! use smartcore::neighbors::knn_classifier::*;
+//! // Various distance metrics
+//! use smartcore::metrics::distance::*;
+//!
+//! // Turn Rust vector-slices with samples into a matrix
+//! let x = DenseMatrix::from_2d_array(&[
+//!    &[1., 2.],
+//!    &[3., 4.],
+//!    &[5., 6.],
+//!    &[7., 8.],
+//!    &[9., 10.]]);
+//! // Our classes are defined as a vector
+//! let y = vec![2, 2, 2, 3, 3];
+//!
+//! // Train classifier
+//! let knn = KNNClassifier::fit(&x, &y, Default::default()).unwrap();
+//!
+//! // Predict classes
+//! let y_hat = knn.predict(&x).unwrap();
+//! ```
+//!
+//! ## Overview
+//!
+//! ### Supported algorithms
+//! All machine learning algorithms are grouped into these broad categories:
 //! * [Clustering](cluster/index.html), unsupervised clustering of unlabeled data.
 //! * [Matrix Decomposition](decomposition/index.html), various methods for matrix decomposition.
 //! * [Linear Models](linear/index.html), regression and classification methods where output is assumed to have linear relation to explanatory variables
@@ -38,37 +87,16 @@
 //! * [Naive Bayes](naive_bayes/index.html), statistical classification technique based on Bayes Theorem
 //! * [SVM](svm/index.html), support vector machines
 //!
-//!
-//! For example, you can use this code to fit a [K Nearest Neighbors classifier](neighbors/knn_classifier/index.html) to a dataset that is defined as standard Rust vector:
-//!
-//! ```
-//! // DenseMatrix defenition
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! // KNNClassifier
-//! use smartcore::neighbors::knn_classifier::*;
-//! // Various distance metrics
-//! use smartcore::math::distance::*;
-//!
-//! // Turn Rust vectors with samples into a matrix
-//! let x = DenseMatrix::from_2d_array(&[
-//!    &[1., 2.],
-//!    &[3., 4.],
-//!    &[5., 6.],
-//!    &[7., 8.],
-//!    &[9., 10.]]);
-//! // Our classes are defined as a Vector
-//! let y = vec![2., 2., 2., 3., 3.];
-//!
-//! // Train classifier
-//! let knn = KNNClassifier::fit(&x, &y, Default::default()).unwrap();
-//!
-//! // Predict classes
-//! let y_hat = knn.predict(&x).unwrap();
-//! ```
+//! ### Linear Algebra traits system
+//! For an introduction to `smartcore`'s traits system see [this notebook](https://github.com/smartcorelib/smartcore-jupyter/blob/5523993c53c6ec1fd72eea130ef4e7883121c1ea/notebooks/01-A-little-bit-about-numbers.ipynb)
 
-/// Various algorithms and helper methods that are used elsewhere in SmartCore
+/// Foundamental numbers traits
+pub mod numbers;
+
+/// Various algorithms and helper methods that are used elsewhere in smartcore
 pub mod algorithm;
 pub mod api;
+
 /// Algorithms for clustering of unlabeled data
 pub mod cluster;
 /// Various datasets
@@ -79,23 +107,28 @@ pub mod decomposition;
 /// Ensemble methods, including Random Forest classifier and regressor
 pub mod ensemble;
 pub mod error;
-/// Diverse collection of linear algebra abstractions and methods that power SmartCore algorithms
+/// Diverse collection of linear algebra abstractions and methods that power smartcore algorithms
 pub mod linalg;
 /// Supervised classification and regression models that assume linear relationship between dependent and explanatory variables.
 pub mod linear;
-/// Helper methods and classes, including definitions of distance metrics
-pub mod math;
 /// Functions for assessing prediction error.
 pub mod metrics;
+/// TODO: add docstring for model_selection
 pub mod model_selection;
 ///  Supervised learning algorithms based on applying the Bayes theorem with the independence assumptions between predictors
 pub mod naive_bayes;
 /// Supervised neighbors-based learning methods
 pub mod neighbors;
-pub(crate) mod optimization;
+/// Optimization procedures
+pub mod optimization;
 /// Preprocessing utilities
 pub mod preprocessing;
+/// Reading in data from serialized formats
+#[cfg(feature = "serde")]
+pub mod readers;
 /// Support Vector Machines
 pub mod svm;
 /// Supervised tree-based learning methods
 pub mod tree;
+
+pub(crate) mod rand_custom;

src/linalg/basic/matrix.rs

@@ -0,0 +1,716 @@
+use std::fmt;
+use std::fmt::{Debug, Display};
+use std::ops::Range;
+use std::slice::Iter;
+
+use approx::{AbsDiffEq, RelativeEq};
+#[cfg(feature = "serde")]
+use serde::{Deserialize, Serialize};
+
+use crate::linalg::basic::arrays::{
+    Array, Array2, ArrayView1, ArrayView2, MutArray, MutArrayView2,
+};
+use crate::linalg::traits::cholesky::CholeskyDecomposable;
+use crate::linalg::traits::evd::EVDDecomposable;
+use crate::linalg::traits::lu::LUDecomposable;
+use crate::linalg::traits::qr::QRDecomposable;
+use crate::linalg::traits::stats::{MatrixPreprocessing, MatrixStats};
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
+
+/// Dense matrix
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct DenseMatrix<T> {
+    ncols: usize,
+    nrows: usize,
+    values: Vec<T>,
+    column_major: bool,
+}
+
+/// View on dense matrix
+#[derive(Debug, Clone)]
+pub struct DenseMatrixView<'a, T: Debug + Display + Copy + Sized> {
+    values: &'a [T],
+    stride: usize,
+    nrows: usize,
+    ncols: usize,
+    column_major: bool,
+}
+
+/// Mutable view on dense matrix
+#[derive(Debug)]
+pub struct DenseMatrixMutView<'a, T: Debug + Display + Copy + Sized> {
+    values: &'a mut [T],
+    stride: usize,
+    nrows: usize,
+    ncols: usize,
+    column_major: bool,
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> DenseMatrixView<'a, T> {
+    fn new(m: &'a DenseMatrix<T>, rows: Range<usize>, cols: Range<usize>) -> Self {
+        let (start, end, stride) = if m.column_major {
+            (
+                rows.start + cols.start * m.nrows,
+                rows.end + (cols.end - 1) * m.nrows,
+                m.nrows,
+            )
+        } else {
+            (
+                rows.start * m.ncols + cols.start,
+                (rows.end - 1) * m.ncols + cols.end,
+                m.ncols,
+            )
+        };
+        DenseMatrixView {
+            values: &m.values[start..end],
+            stride,
+            nrows: rows.end - rows.start,
+            ncols: cols.end - cols.start,
+            column_major: m.column_major,
+        }
+    }
+
+    fn iter<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(
+                (0..self.nrows).flat_map(move |r| (0..self.ncols).map(move |c| self.get((r, c)))),
+            ),
+            _ => Box::new(
+                (0..self.ncols).flat_map(move |c| (0..self.nrows).map(move |r| self.get((r, c)))),
+            ),
+        }
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> fmt::Display for DenseMatrixView<'a, T> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "DenseMatrix: nrows: {:?}, ncols: {:?}",
+            self.nrows, self.ncols
+        )?;
+        writeln!(f, "column_major: {:?}", self.column_major)?;
+        self.display(f)
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> DenseMatrixMutView<'a, T> {
+    fn new(m: &'a mut DenseMatrix<T>, rows: Range<usize>, cols: Range<usize>) -> Self {
+        let (start, end, stride) = if m.column_major {
+            (
+                rows.start + cols.start * m.nrows,
+                rows.end + (cols.end - 1) * m.nrows,
+                m.nrows,
+            )
+        } else {
+            (
+                rows.start * m.ncols + cols.start,
+                (rows.end - 1) * m.ncols + cols.end,
+                m.ncols,
+            )
+        };
+        DenseMatrixMutView {
+            values: &mut m.values[start..end],
+            stride,
+            nrows: rows.end - rows.start,
+            ncols: cols.end - cols.start,
+            column_major: m.column_major,
+        }
+    }
+
+    fn iter<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(
+                (0..self.nrows).flat_map(move |r| (0..self.ncols).map(move |c| self.get((r, c)))),
+            ),
+            _ => Box::new(
+                (0..self.ncols).flat_map(move |c| (0..self.nrows).map(move |r| self.get((r, c)))),
+            ),
+        }
+    }
+
+    fn iter_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &mut T> + 'b> {
+        let column_major = self.column_major;
+        let stride = self.stride;
+        let ptr = self.values.as_mut_ptr();
+        match axis {
+            0 => Box::new((0..self.nrows).flat_map(move |r| {
+                (0..self.ncols).map(move |c| unsafe {
+                    &mut *ptr.add(if column_major {
+                        r + c * stride
+                    } else {
+                        r * stride + c
+                    })
+                })
+            })),
+            _ => Box::new((0..self.ncols).flat_map(move |c| {
+                (0..self.nrows).map(move |r| unsafe {
+                    &mut *ptr.add(if column_major {
+                        r + c * stride
+                    } else {
+                        r * stride + c
+                    })
+                })
+            })),
+        }
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> fmt::Display for DenseMatrixMutView<'a, T> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "DenseMatrix: nrows: {:?}, ncols: {:?}",
+            self.nrows, self.ncols
+        )?;
+        writeln!(f, "column_major: {:?}", self.column_major)?;
+        self.display(f)
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> DenseMatrix<T> {
+    /// Create new instance of `DenseMatrix` without copying data.
+    /// `values` should be in column-major order.
+    pub fn new(nrows: usize, ncols: usize, values: Vec<T>, column_major: bool) -> Self {
+        DenseMatrix {
+            ncols,
+            nrows,
+            values,
+            column_major,
+        }
+    }
+
+    /// New instance of `DenseMatrix` from 2d array.
+    pub fn from_2d_array(values: &[&[T]]) -> Self {
+        DenseMatrix::from_2d_vec(&values.iter().map(|row| Vec::from(*row)).collect())
+    }
+
+    /// New instance of `DenseMatrix` from 2d vector.
+    pub fn from_2d_vec(values: &Vec<Vec<T>>) -> Self {
+        let nrows = values.len();
+        let ncols = values
+            .first()
+            .unwrap_or_else(|| panic!("Cannot create 2d matrix from an empty vector"))
+            .len();
+        let mut m_values = Vec::with_capacity(nrows * ncols);
+
+        for c in 0..ncols {
+            for r in values.iter().take(nrows) {
+                m_values.push(r[c])
+            }
+        }
+
+        DenseMatrix::new(nrows, ncols, m_values, true)
+    }
+
+    /// Iterate over values of matrix
+    pub fn iter(&self) -> Iter<'_, T> {
+        self.values.iter()
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> fmt::Display for DenseMatrix<T> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "DenseMatrix: nrows: {:?}, ncols: {:?}",
+            self.nrows, self.ncols
+        )?;
+        writeln!(f, "column_major: {:?}", self.column_major)?;
+        self.display(f)
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized + PartialEq> PartialEq for DenseMatrix<T> {
+    fn eq(&self, other: &Self) -> bool {
+        if self.ncols != other.ncols || self.nrows != other.nrows {
+            return false;
+        }
+
+        let len = self.values.len();
+        let other_len = other.values.len();
+
+        if len != other_len {
+            return false;
+        }
+
+        match self.column_major == other.column_major {
+            true => self
+                .values
+                .iter()
+                .zip(other.values.iter())
+                .all(|(&v1, v2)| v1.eq(v2)),
+            false => self
+                .iterator(0)
+                .zip(other.iterator(0))
+                .all(|(&v1, v2)| v1.eq(v2)),
+        }
+    }
+}
+
+impl<T: Number + RealNumber + AbsDiffEq> AbsDiffEq for DenseMatrix<T>
+where
+    T::Epsilon: Copy,
+{
+    type Epsilon = T::Epsilon;
+
+    fn default_epsilon() -> T::Epsilon {
+        T::default_epsilon()
+    }
+
+    // equality in differences in absolute values, according to an epsilon
+    fn abs_diff_eq(&self, other: &Self, epsilon: T::Epsilon) -> bool {
+        if self.ncols != other.ncols || self.nrows != other.nrows {
+            false
+        } else {
+            self.values
+                .iter()
+                .zip(other.values.iter())
+                .all(|(v1, v2)| T::abs_diff_eq(v1, v2, epsilon))
+        }
+    }
+}
+
+impl<T: Number + RealNumber + RelativeEq> RelativeEq for DenseMatrix<T>
+where
+    T::Epsilon: Copy,
+{
+    fn default_max_relative() -> T::Epsilon {
+        T::default_max_relative()
+    }
+
+    fn relative_eq(&self, other: &Self, epsilon: T::Epsilon, max_relative: T::Epsilon) -> bool {
+        if self.ncols != other.ncols || self.nrows != other.nrows {
+            false
+        } else {
+            self.iterator(0)
+                .zip(other.iterator(0))
+                .all(|(v1, v2)| T::relative_eq(v1, v2, epsilon, max_relative))
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> Array<T, (usize, usize)> for DenseMatrix<T> {
+    fn get(&self, pos: (usize, usize)) -> &T {
+        let (row, col) = pos;
+        if row >= self.nrows || col >= self.ncols {
+            panic!(
+                "Invalid index ({},{}) for {}x{} matrix",
+                row, col, self.nrows, self.ncols
+            );
+        }
+        if self.column_major {
+            &self.values[col * self.nrows + row]
+        } else {
+            &self.values[col + self.ncols * row]
+        }
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows, self.ncols)
+    }
+
+    fn is_empty(&self) -> bool {
+        self.ncols > 0 && self.nrows > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(
+                (0..self.nrows).flat_map(move |r| (0..self.ncols).map(move |c| self.get((r, c)))),
+            ),
+            _ => Box::new(
+                (0..self.ncols).flat_map(move |c| (0..self.nrows).map(move |r| self.get((r, c)))),
+            ),
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> MutArray<T, (usize, usize)> for DenseMatrix<T> {
+    fn set(&mut self, pos: (usize, usize), x: T) {
+        if self.column_major {
+            self.values[pos.1 * self.nrows + pos.0] = x;
+        } else {
+            self.values[pos.1 + pos.0 * self.ncols] = x;
+        }
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        let ptr = self.values.as_mut_ptr();
+        let column_major = self.column_major;
+        let (nrows, ncols) = self.shape();
+        match axis {
+            0 => Box::new((0..self.nrows).flat_map(move |r| {
+                (0..self.ncols).map(move |c| unsafe {
+                    &mut *ptr.add(if column_major {
+                        r + c * nrows
+                    } else {
+                        r * ncols + c
+                    })
+                })
+            })),
+            _ => Box::new((0..self.ncols).flat_map(move |c| {
+                (0..self.nrows).map(move |r| unsafe {
+                    &mut *ptr.add(if column_major {
+                        r + c * nrows
+                    } else {
+                        r * ncols + c
+                    })
+                })
+            })),
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> ArrayView2<T> for DenseMatrix<T> {}
+
+impl<T: Debug + Display + Copy + Sized> MutArrayView2<T> for DenseMatrix<T> {}
+
+impl<T: Debug + Display + Copy + Sized> Array2<T> for DenseMatrix<T> {
+    fn get_row<'a>(&'a self, row: usize) -> Box<dyn ArrayView1<T> + 'a> {
+        Box::new(DenseMatrixView::new(self, row..row + 1, 0..self.ncols))
+    }
+
+    fn get_col<'a>(&'a self, col: usize) -> Box<dyn ArrayView1<T> + 'a> {
+        Box::new(DenseMatrixView::new(self, 0..self.nrows, col..col + 1))
+    }
+
+    fn slice<'a>(&'a self, rows: Range<usize>, cols: Range<usize>) -> Box<dyn ArrayView2<T> + 'a> {
+        Box::new(DenseMatrixView::new(self, rows, cols))
+    }
+
+    fn slice_mut<'a>(
+        &'a mut self,
+        rows: Range<usize>,
+        cols: Range<usize>,
+    ) -> Box<dyn MutArrayView2<T> + 'a>
+    where
+        Self: Sized,
+    {
+        Box::new(DenseMatrixMutView::new(self, rows, cols))
+    }
+
+    fn fill(nrows: usize, ncols: usize, value: T) -> Self {
+        DenseMatrix::new(nrows, ncols, vec![value; nrows * ncols], true)
+    }
+
+    fn from_iterator<I: Iterator<Item = T>>(iter: I, nrows: usize, ncols: usize, axis: u8) -> Self {
+        DenseMatrix::new(nrows, ncols, iter.collect(), axis != 0)
+    }
+
+    fn transpose(&self) -> Self {
+        let mut m = self.clone();
+        m.ncols = self.nrows;
+        m.nrows = self.ncols;
+        m.column_major = !self.column_major;
+        m
+    }
+}
+
+impl<T: Number + RealNumber> QRDecomposable<T> for DenseMatrix<T> {}
+impl<T: Number + RealNumber> CholeskyDecomposable<T> for DenseMatrix<T> {}
+impl<T: Number + RealNumber> EVDDecomposable<T> for DenseMatrix<T> {}
+impl<T: Number + RealNumber> LUDecomposable<T> for DenseMatrix<T> {}
+impl<T: Number + RealNumber> SVDDecomposable<T> for DenseMatrix<T> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> Array<T, (usize, usize)> for DenseMatrixView<'a, T> {
+    fn get(&self, pos: (usize, usize)) -> &T {
+        if self.column_major {
+            &self.values[(pos.0 + pos.1 * self.stride)]
+        } else {
+            &self.values[(pos.0 * self.stride + pos.1)]
+        }
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows, self.ncols)
+    }
+
+    fn is_empty(&self) -> bool {
+        self.nrows * self.ncols > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        self.iter(axis)
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> Array<T, usize> for DenseMatrixView<'a, T> {
+    fn get(&self, i: usize) -> &T {
+        if self.nrows == 1 {
+            if self.column_major {
+                &self.values[i * self.stride]
+            } else {
+                &self.values[i]
+            }
+        } else if self.ncols == 1 || (!self.column_major && self.nrows == 1) {
+            if self.column_major {
+                &self.values[i]
+            } else {
+                &self.values[i * self.stride]
+            }
+        } else {
+            panic!("This is neither a column nor a row");
+        }
+    }
+
+    fn shape(&self) -> usize {
+        if self.nrows == 1 {
+            self.ncols
+        } else if self.ncols == 1 {
+            self.nrows
+        } else {
+            panic!("This is neither a column nor a row");
+        }
+    }
+
+    fn is_empty(&self) -> bool {
+        self.nrows * self.ncols > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        self.iter(axis)
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView2<T> for DenseMatrixView<'a, T> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView1<T> for DenseMatrixView<'a, T> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> Array<T, (usize, usize)> for DenseMatrixMutView<'a, T> {
+    fn get(&self, pos: (usize, usize)) -> &T {
+        if self.column_major {
+            &self.values[(pos.0 + pos.1 * self.stride)]
+        } else {
+            &self.values[(pos.0 * self.stride + pos.1)]
+        }
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows, self.ncols)
+    }
+
+    fn is_empty(&self) -> bool {
+        self.nrows * self.ncols > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        self.iter(axis)
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArray<T, (usize, usize)>
+    for DenseMatrixMutView<'a, T>
+{
+    fn set(&mut self, pos: (usize, usize), x: T) {
+        if self.column_major {
+            self.values[(pos.0 + pos.1 * self.stride)] = x;
+        } else {
+            self.values[(pos.0 * self.stride + pos.1)] = x;
+        }
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        self.iter_mut(axis)
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArrayView2<T> for DenseMatrixMutView<'a, T> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView2<T> for DenseMatrixMutView<'a, T> {}
+
+impl<T: RealNumber> MatrixStats<T> for DenseMatrix<T> {}
+
+impl<T: RealNumber> MatrixPreprocessing<T> for DenseMatrix<T> {}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use approx::relative_eq;
+
+    #[test]
+    fn test_display() {
+        let x = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.], &[7., 8., 9.]]);
+
+        println!("{}", &x);
+    }
+
+    #[test]
+    fn test_get_row_col() {
+        let x = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.], &[7., 8., 9.]]);
+
+        assert_eq!(15.0, x.get_col(1).sum());
+        assert_eq!(15.0, x.get_row(1).sum());
+        assert_eq!(81.0, x.get_col(1).dot(&(*x.get_row(1))));
+    }
+
+    #[test]
+    fn test_row_major() {
+        let mut x = DenseMatrix::new(2, 3, vec![1, 2, 3, 4, 5, 6], false);
+
+        assert_eq!(5, *x.get_col(1).get(1));
+        assert_eq!(7, x.get_col(1).sum());
+        assert_eq!(5, *x.get_row(1).get(1));
+        assert_eq!(15, x.get_row(1).sum());
+        x.slice_mut(0..2, 1..2)
+            .iterator_mut(0)
+            .for_each(|v| *v += 2);
+        assert_eq!(vec![1, 4, 3, 4, 7, 6], *x.values);
+    }
+
+    #[test]
+    fn test_get_slice() {
+        let x = DenseMatrix::from_2d_array(&[&[1, 2, 3], &[4, 5, 6], &[7, 8, 9], &[10, 11, 12]]);
+
+        assert_eq!(
+            vec![4, 5, 6],
+            DenseMatrix::from_slice(&(*x.slice(1..2, 0..3))).values
+        );
+        let second_row: Vec<i32> = x.slice(1..2, 0..3).iterator(0).map(|x| *x).collect();
+        assert_eq!(vec![4, 5, 6], second_row);
+        let second_col: Vec<i32> = x.slice(0..3, 1..2).iterator(0).map(|x| *x).collect();
+        assert_eq!(vec![2, 5, 8], second_col);
+    }
+
+    #[test]
+    fn test_iter_mut() {
+        let mut x = DenseMatrix::from_2d_array(&[&[1, 2, 3], &[4, 5, 6], &[7, 8, 9]]);
+
+        assert_eq!(vec![1, 4, 7, 2, 5, 8, 3, 6, 9], x.values);
+        // add +2 to some elements
+        x.slice_mut(1..2, 0..3)
+            .iterator_mut(0)
+            .for_each(|v| *v += 2);
+        assert_eq!(vec![1, 6, 7, 2, 7, 8, 3, 8, 9], x.values);
+        // add +1 to some others
+        x.slice_mut(0..3, 1..2)
+            .iterator_mut(0)
+            .for_each(|v| *v += 1);
+        assert_eq!(vec![1, 6, 7, 3, 8, 9, 3, 8, 9], x.values);
+
+        // rewrite matrix as indices of values per axis 1 (row-wise)
+        x.iterator_mut(1).enumerate().for_each(|(a, b)| *b = a);
+        assert_eq!(vec![0, 1, 2, 3, 4, 5, 6, 7, 8], x.values);
+        // rewrite matrix as indices of values per axis 0 (column-wise)
+        x.iterator_mut(0).enumerate().for_each(|(a, b)| *b = a);
+        assert_eq!(vec![0, 3, 6, 1, 4, 7, 2, 5, 8], x.values);
+        // rewrite some by slice
+        x.slice_mut(0..3, 0..2)
+            .iterator_mut(0)
+            .enumerate()
+            .for_each(|(a, b)| *b = a);
+        assert_eq!(vec![0, 2, 4, 1, 3, 5, 2, 5, 8], x.values);
+        x.slice_mut(0..2, 0..3)
+            .iterator_mut(1)
+            .enumerate()
+            .for_each(|(a, b)| *b = a);
+        assert_eq!(vec![0, 1, 4, 2, 3, 5, 4, 5, 8], x.values);
+    }
+
+    #[test]
+    fn test_str_array() {
+        let mut x =
+            DenseMatrix::from_2d_array(&[&["1", "2", "3"], &["4", "5", "6"], &["7", "8", "9"]]);
+
+        assert_eq!(vec!["1", "4", "7", "2", "5", "8", "3", "6", "9"], x.values);
+        x.iterator_mut(0).for_each(|v| *v = "str");
+        assert_eq!(
+            vec!["str", "str", "str", "str", "str", "str", "str", "str", "str"],
+            x.values
+        );
+    }
+
+    #[test]
+    fn test_transpose() {
+        let x = DenseMatrix::<&str>::from_2d_array(&[&["1", "2", "3"], &["4", "5", "6"]]);
+
+        assert_eq!(vec!["1", "4", "2", "5", "3", "6"], x.values);
+        assert!(x.column_major == true);
+
+        // transpose
+        let x = x.transpose();
+        assert_eq!(vec!["1", "4", "2", "5", "3", "6"], x.values);
+        assert!(x.column_major == false); // should change column_major
+    }
+
+    #[test]
+    fn test_from_iterator() {
+        let data = vec![1, 2, 3, 4, 5, 6];
+
+        let m = DenseMatrix::from_iterator(data.iter(), 2, 3, 0);
+
+        // make a vector into a 2x3 matrix.
+        assert_eq!(
+            vec![1, 2, 3, 4, 5, 6],
+            m.values.iter().map(|e| **e).collect::<Vec<i32>>()
+        );
+        assert!(m.column_major == false);
+    }
+
+    #[test]
+    fn test_take() {
+        let a = DenseMatrix::from_2d_array(&[&[1, 2, 3], &[4, 5, 6]]);
+        let b = DenseMatrix::from_2d_array(&[&[1, 2], &[3, 4], &[5, 6]]);
+
+        println!("{}", a);
+        // take column 0 and 2
+        assert_eq!(vec![1, 3, 4, 6], a.take(&[0, 2], 1).values);
+        println!("{}", b);
+        // take rows 0 and 2
+        assert_eq!(vec![1, 2, 5, 6], b.take(&[0, 2], 0).values);
+    }
+
+    #[test]
+    fn test_mut() {
+        let a = DenseMatrix::from_2d_array(&[&[1.3, -2.1, 3.4], &[-4., -5.3, 6.1]]);
+
+        let a = a.abs();
+        assert_eq!(vec![1.3, 4.0, 2.1, 5.3, 3.4, 6.1], a.values);
+
+        let a = a.neg();
+        assert_eq!(vec![-1.3, -4.0, -2.1, -5.3, -3.4, -6.1], a.values);
+    }
+
+    #[test]
+    fn test_reshape() {
+        let a = DenseMatrix::from_2d_array(&[&[1, 2, 3], &[4, 5, 6], &[7, 8, 9], &[10, 11, 12]]);
+
+        let a = a.reshape(2, 6, 0);
+        assert_eq!(vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], a.values);
+        assert!(a.ncols == 6 && a.nrows == 2 && a.column_major == false);
+
+        let a = a.reshape(3, 4, 1);
+        assert_eq!(vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], a.values);
+        assert!(a.ncols == 4 && a.nrows == 3 && a.column_major == true);
+    }
+
+    #[test]
+    fn test_eq() {
+        let a = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.]]);
+        let b = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.], &[7., 8., 9.]]);
+        let c = DenseMatrix::from_2d_array(&[
+            &[1. + f32::EPSILON, 2., 3.],
+            &[4., 5., 6. + f32::EPSILON],
+        ]);
+        let d = DenseMatrix::from_2d_array(&[&[1. + 0.5, 2., 3.], &[4., 5., 6. + f32::EPSILON]]);
+
+        assert!(!relative_eq!(a, b));
+        assert!(!relative_eq!(a, d));
+        assert!(relative_eq!(a, c));
+    }
+}

src/linalg/basic/mod.rs

@@ -0,0 +1,8 @@
+/// `Array`, `ArrayView` and related multidimensional
+pub mod arrays;
+
+/// foundamental implementation for a `DenseMatrix` construct
+pub mod matrix;
+
+/// foundamental implementation for 1D constructs
+pub mod vector;

src/linalg/basic/vector.rs

@@ -0,0 +1,327 @@
+use std::fmt::{Debug, Display};
+use std::ops::Range;
+
+use crate::linalg::basic::arrays::{Array, Array1, ArrayView1, MutArray, MutArrayView1};
+
+/// Provide mutable window on array
+#[derive(Debug)]
+pub struct VecMutView<'a, T: Debug + Display + Copy + Sized> {
+    ptr: &'a mut [T],
+}
+
+/// Provide window on array
+#[derive(Debug, Clone)]
+pub struct VecView<'a, T: Debug + Display + Copy + Sized> {
+    ptr: &'a [T],
+}
+
+impl<T: Debug + Display + Copy + Sized> Array<T, usize> for Vec<T> {
+    fn get(&self, i: usize) -> &T {
+        &self[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter())
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> MutArray<T, usize> for Vec<T> {
+    fn set(&mut self, i: usize, x: T) {
+        self[i] = x
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter_mut())
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> ArrayView1<T> for Vec<T> {}
+
+impl<T: Debug + Display + Copy + Sized> MutArrayView1<T> for Vec<T> {}
+
+impl<T: Debug + Display + Copy + Sized> Array1<T> for Vec<T> {
+    fn slice<'a>(&'a self, range: Range<usize>) -> Box<dyn ArrayView1<T> + 'a> {
+        assert!(
+            range.end <= self.len(),
+            "`range` should be <= {}",
+            self.len()
+        );
+        let view = VecView { ptr: &self[range] };
+        Box::new(view)
+    }
+
+    fn slice_mut<'b>(&'b mut self, range: Range<usize>) -> Box<dyn MutArrayView1<T> + 'b> {
+        assert!(
+            range.end <= self.len(),
+            "`range` should be <= {}",
+            self.len()
+        );
+        let view = VecMutView {
+            ptr: &mut self[range],
+        };
+        Box::new(view)
+    }
+
+    fn fill(len: usize, value: T) -> Self {
+        vec![value; len]
+    }
+
+    fn from_iterator<I: Iterator<Item = T>>(iter: I, len: usize) -> Self
+    where
+        Self: Sized,
+    {
+        let mut v: Vec<T> = Vec::with_capacity(len);
+        iter.take(len).for_each(|i| v.push(i));
+        v
+    }
+
+    fn from_vec_slice(slice: &[T]) -> Self {
+        let mut v: Vec<T> = Vec::with_capacity(slice.len());
+        slice.iter().for_each(|i| v.push(*i));
+        v
+    }
+
+    fn from_slice(slice: &dyn ArrayView1<T>) -> Self {
+        let mut v: Vec<T> = Vec::with_capacity(slice.shape());
+        slice.iterator(0).for_each(|i| v.push(*i));
+        v
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> Array<T, usize> for VecMutView<'a, T> {
+    fn get(&self, i: usize) -> &T {
+        &self.ptr[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.ptr.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.ptr.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.ptr.iter())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArray<T, usize> for VecMutView<'a, T> {
+    fn set(&mut self, i: usize, x: T) {
+        self.ptr[i] = x;
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.ptr.iter_mut())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView1<T> for VecMutView<'a, T> {}
+impl<'a, T: Debug + Display + Copy + Sized> MutArrayView1<T> for VecMutView<'a, T> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> Array<T, usize> for VecView<'a, T> {
+    fn get(&self, i: usize) -> &T {
+        &self.ptr[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.ptr.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.ptr.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.ptr.iter())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView1<T> for VecView<'a, T> {}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::numbers::basenum::Number;
+
+    fn dot_product<T: Number, V: Array1<T>>(v: &V) -> T {
+        let vv = V::zeros(10);
+        let v_s = vv.slice(0..3);
+        let dot = v_s.dot(v);
+        dot
+    }
+
+    fn vector_ops<T: Number + PartialOrd, V: Array1<T>>(_: &V) -> T {
+        let v = V::zeros(10);
+        v.max()
+    }
+
+    #[test]
+    fn test_get_set() {
+        let mut x = vec![1, 2, 3];
+        assert_eq!(3, *x.get(2));
+        x.set(1, 1);
+        assert_eq!(1, *x.get(1));
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_set() {
+        vec![1, 2, 3].set(3, 1);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_get() {
+        vec![1, 2, 3].get(3);
+    }
+
+    #[test]
+    fn test_len() {
+        let x = vec![1, 2, 3];
+        assert_eq!(3, x.len());
+    }
+
+    #[test]
+    fn test_is_empty() {
+        assert!(vec![1; 0].is_empty());
+        assert!(!vec![1, 2, 3].is_empty());
+    }
+
+    #[test]
+    fn test_iterator() {
+        let v: Vec<i32> = vec![1, 2, 3].iterator(0).map(|&v| v * 2).collect();
+        assert_eq!(vec![2, 4, 6], v);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_iterator() {
+        let _ = vec![1, 2, 3].iterator(1);
+    }
+
+    #[test]
+    fn test_mut_iterator() {
+        let mut x = vec![1, 2, 3];
+        x.iterator_mut(0).for_each(|v| *v = *v * 2);
+        assert_eq!(vec![2, 4, 6], x);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_mut_iterator() {
+        let _ = vec![1, 2, 3].iterator_mut(1);
+    }
+
+    #[test]
+    fn test_slice() {
+        let x = vec![1, 2, 3, 4, 5];
+        let x_slice = x.slice(2..3);
+        assert_eq!(1, x_slice.shape());
+        assert_eq!(3, *x_slice.get(0));
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_slice() {
+        vec![1, 2, 3].slice(0..4);
+    }
+
+    #[test]
+    fn test_mut_slice() {
+        let mut x = vec![1, 2, 3, 4, 5];
+        let mut x_slice = x.slice_mut(2..4);
+        x_slice.set(0, 9);
+        assert_eq!(2, x_slice.shape());
+        assert_eq!(9, *x_slice.get(0));
+        assert_eq!(4, *x_slice.get(1));
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_failed_mut_slice() {
+        vec![1, 2, 3].slice_mut(0..4);
+    }
+
+    #[test]
+    fn test_init() {
+        assert_eq!(Vec::fill(3, 0), vec![0, 0, 0]);
+        assert_eq!(
+            Vec::from_iterator([0, 1, 2, 3].iter().cloned(), 3),
+            vec![0, 1, 2]
+        );
+        assert_eq!(Vec::from_vec_slice(&[0, 1, 2]), vec![0, 1, 2]);
+        assert_eq!(Vec::from_vec_slice(&[0, 1, 2, 3, 4][2..]), vec![2, 3, 4]);
+        assert_eq!(Vec::from_slice(&vec![1, 2, 3, 4, 5]), vec![1, 2, 3, 4, 5]);
+        assert_eq!(
+            Vec::from_slice(vec![1, 2, 3, 4, 5].slice(0..3).as_ref()),
+            vec![1, 2, 3]
+        );
+    }
+
+    #[test]
+    fn test_mul_scalar() {
+        let mut x = vec![1., 2., 3.];
+
+        let mut y = Vec::<f32>::zeros(10);
+
+        y.slice_mut(0..2).add_scalar_mut(1.0);
+        y.sub_scalar(1.0);
+        x.slice_mut(0..2).sub_scalar_mut(2.);
+
+        assert_eq!(vec![-1.0, 0.0, 3.0], x);
+    }
+
+    #[test]
+    fn test_dot() {
+        let y_i = vec![1, 2, 3];
+        let y = vec![1.0, 2.0, 3.0];
+
+        println!("Regular dot1: {:?}", dot_product(&y));
+
+        let x = vec![4.0, 5.0, 6.0];
+        assert_eq!(32.0, y.slice(0..3).dot(&(*x.slice(0..3))));
+        assert_eq!(32.0, y.slice(0..3).dot(&x));
+        assert_eq!(32.0, y.dot(&x));
+        assert_eq!(14, y_i.dot(&y_i));
+    }
+
+    #[test]
+    fn test_operators() {
+        let mut x: Vec<f32> = Vec::zeros(10);
+
+        x.add_scalar(15.0);
+        {
+            let mut x_s = x.slice_mut(0..5);
+            x_s.add_scalar_mut(1.0);
+            assert_eq!(
+                vec![1.0, 1.0, 1.0, 1.0, 1.0],
+                x_s.iterator(0).copied().collect::<Vec<f32>>()
+            );
+        }
+
+        assert_eq!(1.0, x.slice(2..3).min());
+
+        assert_eq!(vec![1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], x);
+    }
+
+    #[test]
+    fn test_vector_ops() {
+        let x = vec![1., 2., 3.];
+
+        vector_ops(&x);
+    }
+}

src/linalg/mod.rs

@@ -1,785 +1,9 @@
-#![allow(clippy::wrong_self_convention)]
-//! # Linear Algebra and Matrix Decomposition
-//!
-//! Most machine learning algorithms in SmartCore depend on linear algebra and matrix decomposition methods from this module.
-//!
-//! Traits [`BaseMatrix`](trait.BaseMatrix.html), [`Matrix`](trait.Matrix.html) and [`BaseVector`](trait.BaseVector.html) define
-//! abstract methods that can be implemented for any two-dimensional and one-dimentional arrays (matrix and vector).
-//! Functions from these traits are designed for SmartCore machine learning algorithms and should not be used directly in your code.
-//! If you still want to use functions from `BaseMatrix`, `Matrix` and `BaseVector` please be aware that methods defined in these
-//! traits might change in the future.
-//!
-//! One reason why linear algebra traits are public is to allow for different types of matrices and vectors to be plugged into SmartCore.
-//! Once all methods defined in `BaseMatrix`, `Matrix` and `BaseVector` are implemented for your favourite type of matrix and vector you
-//! should be able to run SmartCore algorithms on it. Please see `nalgebra_bindings` and `ndarray_bindings` modules for an example of how
-//! it is done for other libraries.
-//!
-//! You will also find verious matrix decomposition methods that work for any matrix that extends [`Matrix`](trait.Matrix.html).
-//! For example, to decompose matrix defined as [Vec](https://doc.rust-lang.org/std/vec/struct.Vec.html):
-//!
-//! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::svd::*;
-//!
-//! let A = DenseMatrix::from_2d_array(&[
-//!            &[0.9000, 0.4000, 0.7000],
-//!            &[0.4000, 0.5000, 0.3000],
-//!            &[0.7000, 0.3000, 0.8000],
-//!         ]);
-//!
-//! let svd = A.svd().unwrap();
-//!
-//! let s: Vec<f64> = svd.s;
-//! let v: DenseMatrix<f64> = svd.V;
-//! let u: DenseMatrix<f64> = svd.U;
-//! ```
+/// basic data structures for linear algebra constructs: arrays and views
+pub mod basic;
+
+/// traits associated to algebraic constructs
+pub mod traits;
 
-pub mod cholesky;
-/// The matrix is represented in terms of its eigenvalues and eigenvectors.
-pub mod evd;
-pub mod high_order;
-/// Factors a matrix as the product of a lower triangular matrix and an upper triangular matrix.
-pub mod lu;
-/// Dense matrix with column-major order that wraps [Vec](https://doc.rust-lang.org/std/vec/struct.Vec.html).
-pub mod naive;
-/// [nalgebra](https://docs.rs/nalgebra/) bindings.
-#[cfg(feature = "nalgebra-bindings")]
-pub mod nalgebra_bindings;
-/// [ndarray](https://docs.rs/ndarray) bindings.
 #[cfg(feature = "ndarray-bindings")]
-pub mod ndarray_bindings;
-/// QR factorization that factors a matrix into a product of an orthogonal matrix and an upper triangular matrix.
-pub mod qr;
-pub mod stats;
-/// Singular value decomposition.
-pub mod svd;
-
-use std::fmt::{Debug, Display};
-use std::marker::PhantomData;
-use std::ops::Range;
-
-use crate::math::num::RealNumber;
-use cholesky::CholeskyDecomposableMatrix;
-use evd::EVDDecomposableMatrix;
-use high_order::HighOrderOperations;
-use lu::LUDecomposableMatrix;
-use qr::QRDecomposableMatrix;
-use stats::{MatrixPreprocessing, MatrixStats};
-use svd::SVDDecomposableMatrix;
-
-/// Column or row vector
-pub trait BaseVector<T: RealNumber>: Clone + Debug {
-    /// Get an element of a vector
-    /// * `i` - index of an element
-    fn get(&self, i: usize) -> T;
-
-    /// Set an element at `i` to `x`
-    /// * `i` - index of an element
-    /// * `x` - new value
-    fn set(&mut self, i: usize, x: T);
-
-    /// Get number of elevemnt in the vector
-    fn len(&self) -> usize;
-
-    /// Returns true if the vector is empty.
-    fn is_empty(&self) -> bool {
-        self.len() == 0
-    }
-
-    /// Create a new vector from a &[T]
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// let a: [f64; 5] = [0., 0.5, 2., 3., 4.];
-    /// let v: Vec<f64> = BaseVector::from_array(&a);
-    /// assert_eq!(v, vec![0., 0.5, 2., 3., 4.]);
-    /// ```
-    fn from_array(f: &[T]) -> Self {
-        let mut v = Self::zeros(f.len());
-        for (i, elem) in f.iter().enumerate() {
-            v.set(i, *elem);
-        }
-        v
-    }
-
-    /// Return a vector with the elements of the one-dimensional array.
-    fn to_vec(&self) -> Vec<T>;
-
-    /// Create new vector with zeros of size `len`.
-    fn zeros(len: usize) -> Self;
-
-    /// Create new vector with ones of size `len`.
-    fn ones(len: usize) -> Self;
-
-    /// Create new vector of size `len` where each element is set to `value`.
-    fn fill(len: usize, value: T) -> Self;
-
-    /// Vector dot product
-    fn dot(&self, other: &Self) -> T;
-
-    /// Returns True if matrices are element-wise equal within a tolerance `error`.
-    fn approximate_eq(&self, other: &Self, error: T) -> bool;
-
-    /// Returns [L2 norm] of the vector(https://en.wikipedia.org/wiki/Matrix_norm).
-    fn norm2(&self) -> T;
-
-    /// Returns [vectors norm](https://en.wikipedia.org/wiki/Matrix_norm) of order `p`.
-    fn norm(&self, p: T) -> T;
-
-    /// Divide single element of the vector by `x`, write result to original vector.
-    fn div_element_mut(&mut self, pos: usize, x: T);
-
-    /// Multiply single element of the vector by `x`, write result to original vector.
-    fn mul_element_mut(&mut self, pos: usize, x: T);
-
-    /// Add single element of the vector to `x`, write result to original vector.
-    fn add_element_mut(&mut self, pos: usize, x: T);
-
-    /// Subtract `x` from single element of the vector, write result to original vector.
-    fn sub_element_mut(&mut self, pos: usize, x: T);
-
-    /// Subtract scalar
-    fn sub_scalar_mut(&mut self, x: T) -> &Self {
-        for i in 0..self.len() {
-            self.set(i, self.get(i) - x);
-        }
-        self
-    }
-
-    /// Subtract scalar
-    fn add_scalar_mut(&mut self, x: T) -> &Self {
-        for i in 0..self.len() {
-            self.set(i, self.get(i) + x);
-        }
-        self
-    }
-
-    /// Subtract scalar
-    fn mul_scalar_mut(&mut self, x: T) -> &Self {
-        for i in 0..self.len() {
-            self.set(i, self.get(i) * x);
-        }
-        self
-    }
-
-    /// Subtract scalar
-    fn div_scalar_mut(&mut self, x: T) -> &Self {
-        for i in 0..self.len() {
-            self.set(i, self.get(i) / x);
-        }
-        self
-    }
-
-    /// Add vectors, element-wise
-    fn add_scalar(&self, x: T) -> Self {
-        let mut r = self.clone();
-        r.add_scalar_mut(x);
-        r
-    }
-
-    /// Subtract vectors, element-wise
-    fn sub_scalar(&self, x: T) -> Self {
-        let mut r = self.clone();
-        r.sub_scalar_mut(x);
-        r
-    }
-
-    /// Multiply vectors, element-wise
-    fn mul_scalar(&self, x: T) -> Self {
-        let mut r = self.clone();
-        r.mul_scalar_mut(x);
-        r
-    }
-
-    /// Divide vectors, element-wise
-    fn div_scalar(&self, x: T) -> Self {
-        let mut r = self.clone();
-        r.div_scalar_mut(x);
-        r
-    }
-
-    /// Add vectors, element-wise, overriding original vector with result.
-    fn add_mut(&mut self, other: &Self) -> &Self;
-
-    /// Subtract vectors, element-wise, overriding original vector with result.
-    fn sub_mut(&mut self, other: &Self) -> &Self;
-
-    /// Multiply vectors, element-wise, overriding original vector with result.
-    fn mul_mut(&mut self, other: &Self) -> &Self;
-
-    /// Divide vectors, element-wise, overriding original vector with result.
-    fn div_mut(&mut self, other: &Self) -> &Self;
-
-    /// Add vectors, element-wise
-    fn add(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.add_mut(other);
-        r
-    }
-
-    /// Subtract vectors, element-wise
-    fn sub(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.sub_mut(other);
-        r
-    }
-
-    /// Multiply vectors, element-wise
-    fn mul(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.mul_mut(other);
-        r
-    }
-
-    /// Divide vectors, element-wise
-    fn div(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.div_mut(other);
-        r
-    }
-
-    /// Calculates sum of all elements of the vector.
-    fn sum(&self) -> T;
-
-    /// Returns unique values from the vector.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// let a = vec!(1., 2., 2., -2., -6., -7., 2., 3., 4.);
-    ///
-    ///assert_eq!(a.unique(), vec![-7., -6., -2., 1., 2., 3., 4.]);
-    /// ```
-    fn unique(&self) -> Vec<T>;
-
-    /// Computes the arithmetic mean.
-    fn mean(&self) -> T {
-        self.sum() / T::from_usize(self.len()).unwrap()
-    }
-    /// Computes variance.
-    fn var(&self) -> T {
-        let n = self.len();
-
-        let mut mu = T::zero();
-        let mut sum = T::zero();
-        let div = T::from_usize(n).unwrap();
-        for i in 0..n {
-            let xi = self.get(i);
-            mu += xi;
-            sum += xi * xi;
-        }
-        mu /= div;
-        sum / div - mu.powi(2)
-    }
-    /// Computes the standard deviation.
-    fn std(&self) -> T {
-        self.var().sqrt()
-    }
-
-    /// Copies content of `other` vector.
-    fn copy_from(&mut self, other: &Self);
-
-    /// Take elements from an array.
-    fn take(&self, index: &[usize]) -> Self {
-        let n = index.len();
-
-        let mut result = Self::zeros(n);
-
-        for (i, idx) in index.iter().enumerate() {
-            result.set(i, self.get(*idx));
-        }
-
-        result
-    }
-}
-
-/// Generic matrix type.
-pub trait BaseMatrix<T: RealNumber>: Clone + Debug {
-    /// Row vector that is associated with this matrix type,
-    /// e.g. if we have an implementation of sparce matrix
-    /// we should have an associated sparce vector type that
-    /// represents a row in this matrix.
-    type RowVector: BaseVector<T> + Clone + Debug;
-
-    /// Transforms row vector `vec` into a 1xM matrix.
-    fn from_row_vector(vec: Self::RowVector) -> Self;
-
-    /// Transforms 1-d matrix of 1xM into a row vector.
-    fn to_row_vector(self) -> Self::RowVector;
-
-    /// Get an element of the matrix.
-    /// * `row` - row number
-    /// * `col` - column number
-    fn get(&self, row: usize, col: usize) -> T;
-
-    /// Get a vector with elements of the `row`'th row
-    /// * `row` - row number
-    fn get_row_as_vec(&self, row: usize) -> Vec<T>;
-
-    /// Get the `row`'th row
-    /// * `row` - row number
-    fn get_row(&self, row: usize) -> Self::RowVector;
-
-    /// Copies a vector with elements of the `row`'th row into `result`
-    /// * `row` - row number
-    /// * `result` - receiver for the row
-    fn copy_row_as_vec(&self, row: usize, result: &mut Vec<T>);
-
-    /// Get a vector with elements of the `col`'th column
-    /// * `col` - column number
-    fn get_col_as_vec(&self, col: usize) -> Vec<T>;
-
-    /// Copies a vector with elements of the `col`'th column into `result`
-    /// * `col` - column number
-    /// * `result` - receiver for the col
-    fn copy_col_as_vec(&self, col: usize, result: &mut Vec<T>);
-
-    /// Set an element at `col`, `row` to `x`
-    fn set(&mut self, row: usize, col: usize, x: T);
-
-    /// Create an identity matrix of size `size`
-    fn eye(size: usize) -> Self;
-
-    /// Create new matrix with zeros of size `nrows` by `ncols`.
-    fn zeros(nrows: usize, ncols: usize) -> Self;
-
-    /// Create new matrix with ones of size `nrows` by `ncols`.
-    fn ones(nrows: usize, ncols: usize) -> Self;
-
-    /// Create new matrix of size `nrows` by `ncols` where each element is set to `value`.
-    fn fill(nrows: usize, ncols: usize, value: T) -> Self;
-
-    /// Return the shape of an array.
-    fn shape(&self) -> (usize, usize);
-
-    /// Stack arrays in sequence vertically (row wise).
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.]]);
-    /// let b = DenseMatrix::from_2d_array(&[&[1., 2.], &[3., 4.]]);
-    /// let expected = DenseMatrix::from_2d_array(&[
-    ///     &[1., 2., 3., 1., 2.],
-    ///     &[4., 5., 6., 3., 4.]
-    /// ]);
-    ///
-    /// assert_eq!(a.h_stack(&b), expected);
-    /// ```
-    fn h_stack(&self, other: &Self) -> Self;
-
-    /// Stack arrays in sequence horizontally (column wise).
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_array(1, 3, &[1., 2., 3.]);
-    /// let b = DenseMatrix::from_array(1, 3, &[4., 5., 6.]);
-    /// let expected = DenseMatrix::from_2d_array(&[
-    ///     &[1., 2., 3.],
-    ///     &[4., 5., 6.]
-    /// ]);
-    ///
-    /// assert_eq!(a.v_stack(&b), expected);
-    /// ```
-    fn v_stack(&self, other: &Self) -> Self;
-
-    /// Matrix product.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_2d_array(&[&[1., 2.], &[3., 4.]]);
-    /// let expected = DenseMatrix::from_2d_array(&[
-    ///     &[7., 10.],
-    ///     &[15., 22.]
-    /// ]);
-    ///
-    /// assert_eq!(a.matmul(&a), expected);
-    /// ```
-    fn matmul(&self, other: &Self) -> Self;
-
-    /// Vector dot product
-    /// Both matrices should be of size _1xM_
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_array(1, 3, &[1., 2., 3.]);
-    /// let b = DenseMatrix::from_array(1, 3, &[4., 5., 6.]);
-    ///
-    /// assert_eq!(a.dot(&b), 32.);
-    /// ```
-    fn dot(&self, other: &Self) -> T;
-
-    /// Return a slice of the matrix.
-    /// * `rows` - range of rows to return
-    /// * `cols` - range of columns to return
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let m = DenseMatrix::from_2d_array(&[
-    ///             &[1., 2., 3., 1.],
-    ///             &[4., 5., 6., 3.],
-    ///             &[7., 8., 9., 5.]
-    ///         ]);
-    /// let expected = DenseMatrix::from_2d_array(&[&[2., 3.], &[5., 6.]]);
-    /// let result = m.slice(0..2, 1..3);
-    /// assert_eq!(result, expected);
-    /// ```
-    fn slice(&self, rows: Range<usize>, cols: Range<usize>) -> Self;
-
-    /// Returns True if matrices are element-wise equal within a tolerance `error`.
-    fn approximate_eq(&self, other: &Self, error: T) -> bool;
-
-    /// Add matrices, element-wise, overriding original matrix with result.
-    fn add_mut(&mut self, other: &Self) -> &Self;
-
-    /// Subtract matrices, element-wise, overriding original matrix with result.
-    fn sub_mut(&mut self, other: &Self) -> &Self;
-
-    /// Multiply matrices, element-wise, overriding original matrix with result.
-    fn mul_mut(&mut self, other: &Self) -> &Self;
-
-    /// Divide matrices, element-wise, overriding original matrix with result.
-    fn div_mut(&mut self, other: &Self) -> &Self;
-
-    /// Divide single element of the matrix by `x`, write result to original matrix.
-    fn div_element_mut(&mut self, row: usize, col: usize, x: T);
-
-    /// Multiply single element of the matrix by `x`, write result to original matrix.
-    fn mul_element_mut(&mut self, row: usize, col: usize, x: T);
-
-    /// Add single element of the matrix to `x`, write result to original matrix.
-    fn add_element_mut(&mut self, row: usize, col: usize, x: T);
-
-    /// Subtract `x` from single element of the matrix, write result to original matrix.
-    fn sub_element_mut(&mut self, row: usize, col: usize, x: T);
-
-    /// Add matrices, element-wise
-    fn add(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.add_mut(other);
-        r
-    }
-
-    /// Subtract matrices, element-wise
-    fn sub(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.sub_mut(other);
-        r
-    }
-
-    /// Multiply matrices, element-wise
-    fn mul(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.mul_mut(other);
-        r
-    }
-
-    /// Divide matrices, element-wise
-    fn div(&self, other: &Self) -> Self {
-        let mut r = self.clone();
-        r.div_mut(other);
-        r
-    }
-
-    /// Add `scalar` to the matrix, override original matrix with result.
-    fn add_scalar_mut(&mut self, scalar: T) -> &Self;
-
-    /// Subtract `scalar` from the elements of matrix, override original matrix with result.
-    fn sub_scalar_mut(&mut self, scalar: T) -> &Self;
-
-    /// Multiply `scalar` by the elements of matrix, override original matrix with result.
-    fn mul_scalar_mut(&mut self, scalar: T) -> &Self;
-
-    /// Divide elements of the matrix by `scalar`, override original matrix with result.
-    fn div_scalar_mut(&mut self, scalar: T) -> &Self;
-
-    /// Add `scalar` to the matrix.
-    fn add_scalar(&self, scalar: T) -> Self {
-        let mut r = self.clone();
-        r.add_scalar_mut(scalar);
-        r
-    }
-
-    /// Subtract `scalar` from the elements of matrix.
-    fn sub_scalar(&self, scalar: T) -> Self {
-        let mut r = self.clone();
-        r.sub_scalar_mut(scalar);
-        r
-    }
-
-    /// Multiply `scalar` by the elements of matrix.
-    fn mul_scalar(&self, scalar: T) -> Self {
-        let mut r = self.clone();
-        r.mul_scalar_mut(scalar);
-        r
-    }
-
-    /// Divide elements of the matrix by `scalar`.
-    fn div_scalar(&self, scalar: T) -> Self {
-        let mut r = self.clone();
-        r.div_scalar_mut(scalar);
-        r
-    }
-
-    /// Reverse or permute the axes of the matrix, return new matrix.
-    fn transpose(&self) -> Self;
-
-    /// Create new `nrows` by `ncols` matrix and populate it with random samples from a uniform distribution over [0, 1).
-    fn rand(nrows: usize, ncols: usize) -> Self;
-
-    /// Returns [L2 norm](https://en.wikipedia.org/wiki/Matrix_norm).
-    fn norm2(&self) -> T;
-
-    /// Returns [matrix norm](https://en.wikipedia.org/wiki/Matrix_norm) of order `p`.
-    fn norm(&self, p: T) -> T;
-
-    /// Returns the average of the matrix columns.
-    fn column_mean(&self) -> Vec<T>;
-
-    /// Numerical negative, element-wise. Overrides original matrix.
-    fn negative_mut(&mut self);
-
-    /// Numerical negative, element-wise.
-    fn negative(&self) -> Self {
-        let mut result = self.clone();
-        result.negative_mut();
-        result
-    }
-
-    /// Returns new matrix of shape `nrows` by `ncols` with data copied from original matrix.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_array(1, 6, &[1., 2., 3., 4., 5., 6.]);
-    /// let expected = DenseMatrix::from_2d_array(&[
-    ///             &[1., 2., 3.],
-    ///             &[4., 5., 6.]
-    ///         ]);
-    ///
-    /// assert_eq!(a.reshape(2, 3), expected);
-    /// ```
-    fn reshape(&self, nrows: usize, ncols: usize) -> Self;
-
-    /// Copies content of `other` matrix.
-    fn copy_from(&mut self, other: &Self);
-
-    /// Calculate the absolute value element-wise. Overrides original matrix.
-    fn abs_mut(&mut self) -> &Self;
-
-    /// Calculate the absolute value element-wise.
-    fn abs(&self) -> Self {
-        let mut result = self.clone();
-        result.abs_mut();
-        result
-    }
-
-    /// Calculates sum of all elements of the matrix.
-    fn sum(&self) -> T;
-
-    /// Calculates max of all elements of the matrix.
-    fn max(&self) -> T;
-
-    /// Calculates min of all elements of the matrix.
-    fn min(&self) -> T;
-
-    /// Calculates max(|a - b|) of two matrices
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    ///
-    /// let a = DenseMatrix::from_array(2, 3, &[1., 2., 3., 4., -5., 6.]);
-    /// let b = DenseMatrix::from_array(2, 3, &[2., 3., 4., 1., 0., -12.]);
-    ///
-    /// assert_eq!(a.max_diff(&b), 18.);
-    /// assert_eq!(b.max_diff(&b), 0.);
-    /// ```
-    fn max_diff(&self, other: &Self) -> T {
-        self.sub(other).abs().max()
-    }
-
-    /// Calculates [Softmax function](https://en.wikipedia.org/wiki/Softmax_function). Overrides the matrix with result.
-    fn softmax_mut(&mut self);
-
-    /// Raises elements of the matrix to the power of `p`
-    fn pow_mut(&mut self, p: T) -> &Self;
-
-    /// Returns new matrix with elements raised to the power of `p`
-    fn pow(&mut self, p: T) -> Self {
-        let mut result = self.clone();
-        result.pow_mut(p);
-        result
-    }
-
-    /// Returns the indices of the maximum values in each row.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// let a = DenseMatrix::from_array(2, 3, &[1., 2., 3., -5., -6., -7.]);
-    ///
-    /// assert_eq!(a.argmax(), vec![2, 0]);
-    /// ```
-    fn argmax(&self) -> Vec<usize>;
-
-    /// Returns vector with unique values from the matrix.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// let a = DenseMatrix::from_array(3, 3, &[1., 2., 2., -2., -6., -7., 2., 3., 4.]);
-    ///
-    ///assert_eq!(a.unique(), vec![-7., -6., -2., 1., 2., 3., 4.]);
-    /// ```
-    fn unique(&self) -> Vec<T>;
-
-    /// Calculates the covariance matrix
-    fn cov(&self) -> Self;
-
-    /// Take elements from an array along an axis.
-    fn take(&self, index: &[usize], axis: u8) -> Self {
-        let (n, p) = self.shape();
-
-        let k = match axis {
-            0 => p,
-            _ => n,
-        };
-
-        let mut result = match axis {
-            0 => Self::zeros(index.len(), p),
-            _ => Self::zeros(n, index.len()),
-        };
-
-        for (i, idx) in index.iter().enumerate() {
-            for j in 0..k {
-                match axis {
-                    0 => result.set(i, j, self.get(*idx, j)),
-                    _ => result.set(j, i, self.get(j, *idx)),
-                };
-            }
-        }
-
-        result
-    }
-    /// Take an individual column from the matrix.
-    fn take_column(&self, column_index: usize) -> Self {
-        self.take(&[column_index], 1)
-    }
-}
-
-/// Generic matrix with additional mixins like various factorization methods.
-pub trait Matrix<T: RealNumber>:
-    BaseMatrix<T>
-    + SVDDecomposableMatrix<T>
-    + EVDDecomposableMatrix<T>
-    + QRDecomposableMatrix<T>
-    + LUDecomposableMatrix<T>
-    + CholeskyDecomposableMatrix<T>
-    + MatrixStats<T>
-    + MatrixPreprocessing<T>
-    + HighOrderOperations<T>
-    + PartialEq
-    + Display
-{
-}
-
-pub(crate) fn row_iter<F: RealNumber, M: BaseMatrix<F>>(m: &M) -> RowIter<'_, F, M> {
-    RowIter {
-        m,
-        pos: 0,
-        max_pos: m.shape().0,
-        phantom: PhantomData,
-    }
-}
-
-pub(crate) struct RowIter<'a, T: RealNumber, M: BaseMatrix<T>> {
-    m: &'a M,
-    pos: usize,
-    max_pos: usize,
-    phantom: PhantomData<&'a T>,
-}
-
-impl<'a, T: RealNumber, M: BaseMatrix<T>> Iterator for RowIter<'a, T, M> {
-    type Item = Vec<T>;
-
-    fn next(&mut self) -> Option<Vec<T>> {
-        let res = if self.pos < self.max_pos {
-            Some(self.m.get_row_as_vec(self.pos))
-        } else {
-            None
-        };
-        self.pos += 1;
-        res
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
-    use crate::linalg::BaseMatrix;
-    use crate::linalg::BaseVector;
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn mean() {
-        let m = vec![1., 2., 3.];
-
-        assert_eq!(m.mean(), 2.0);
-    }
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn std() {
-        let m = vec![1., 2., 3.];
-
-        assert!((m.std() - 0.81f64).abs() < 1e-2);
-    }
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn var() {
-        let m = vec![1., 2., 3., 4.];
-
-        assert!((m.var() - 1.25f64).abs() < std::f64::EPSILON);
-    }
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn vec_take() {
-        let m = vec![1., 2., 3., 4., 5.];
-
-        assert_eq!(m.take(&vec!(0, 0, 4, 4)), vec![1., 1., 5., 5.]);
-    }
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn take() {
-        let m = DenseMatrix::from_2d_array(&[
-            &[1.0, 2.0],
-            &[3.0, 4.0],
-            &[5.0, 6.0],
-            &[7.0, 8.0],
-            &[9.0, 10.0],
-        ]);
-
-        let expected_0 = DenseMatrix::from_2d_array(&[&[3.0, 4.0], &[3.0, 4.0], &[7.0, 8.0]]);
-
-        let expected_1 = DenseMatrix::from_2d_array(&[
-            &[2.0, 1.0],
-            &[4.0, 3.0],
-            &[6.0, 5.0],
-            &[8.0, 7.0],
-            &[10.0, 9.0],
-        ]);
-
-        assert_eq!(m.take(&vec!(1, 1, 3), 0), expected_0);
-        assert_eq!(m.take(&vec!(1, 0), 1), expected_1);
-    }
-
-    #[test]
-    fn take_second_column_from_matrix() {
-        let four_columns: DenseMatrix<f64> = DenseMatrix::from_2d_array(&[
-            &[0.0, 1.0, 2.0, 3.0],
-            &[0.0, 1.0, 2.0, 3.0],
-            &[0.0, 1.0, 2.0, 3.0],
-            &[0.0, 1.0, 2.0, 3.0],
-        ]);
-
-        let second_column = four_columns.take_column(1);
-        assert_eq!(
-            second_column,
-            DenseMatrix::from_2d_array(&[&[1.0], &[1.0], &[1.0], &[1.0]]),
-            "The second column was not extracted correctly"
-        );
-    }
-}
+/// ndarray bindings
+pub mod ndarray;

src/linalg/naive/mod.rs

@@ -1,26 +0,0 @@
-//! # Simple Dense Matrix
-//!
-//! Implements [`BaseMatrix`](../../trait.BaseMatrix.html) and [`BaseVector`](../../trait.BaseVector.html) for [Vec](https://doc.rust-lang.org/std/vec/struct.Vec.html).
-//! Data is stored in dense format with [column-major order](https://en.wikipedia.org/wiki/Row-_and_column-major_order).
-//!
-//! Example:
-//!
-//! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//!
-//! // 3x3 matrix
-//! let A = DenseMatrix::from_2d_array(&[
-//!            &[0.9000, 0.4000, 0.7000],
-//!            &[0.4000, 0.5000, 0.3000],
-//!            &[0.7000, 0.3000, 0.8000],
-//!         ]);
-//!
-//! // row vector
-//! let B = DenseMatrix::from_array(1, 3, &[0.9, 0.4, 0.7]);
-//!
-//! // column vector
-//! let C = DenseMatrix::from_vec(3, 1, &vec!(0.9, 0.4, 0.7));
-//! ```
-
-/// Add this module to use Dense Matrix
-pub mod dense_matrix;

src/linalg/ndarray/matrix.rs

@@ -0,0 +1,286 @@
+use std::fmt::{Debug, Display};
+use std::ops::Range;
+
+use crate::linalg::basic::arrays::{
+    Array as BaseArray, Array2, ArrayView1, ArrayView2, MutArray, MutArrayView2,
+};
+
+use crate::linalg::traits::cholesky::CholeskyDecomposable;
+use crate::linalg::traits::evd::EVDDecomposable;
+use crate::linalg::traits::lu::LUDecomposable;
+use crate::linalg::traits::qr::QRDecomposable;
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
+
+use ndarray::{s, Array, ArrayBase, ArrayView, ArrayViewMut, Ix2, OwnedRepr};
+
+impl<T: Debug + Display + Copy + Sized> BaseArray<T, (usize, usize)>
+    for ArrayBase<OwnedRepr<T>, Ix2>
+{
+    fn get(&self, pos: (usize, usize)) -> &T {
+        &self[[pos.0, pos.1]]
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows(), self.ncols())
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(self.iter()),
+            _ => Box::new(
+                (0..self.ncols()).flat_map(move |c| (0..self.nrows()).map(move |r| &self[[r, c]])),
+            ),
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> MutArray<T, (usize, usize)>
+    for ArrayBase<OwnedRepr<T>, Ix2>
+{
+    fn set(&mut self, pos: (usize, usize), x: T) {
+        self[[pos.0, pos.1]] = x
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        let ptr = self.as_mut_ptr();
+        let stride = self.strides();
+        let (rstride, cstride) = (stride[0] as usize, stride[1] as usize);
+        match axis {
+            0 => Box::new(self.iter_mut()),
+            _ => Box::new((0..self.ncols()).flat_map(move |c| {
+                (0..self.nrows()).map(move |r| unsafe { &mut *ptr.add(r * rstride + c * cstride) })
+            })),
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> ArrayView2<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+
+impl<T: Debug + Display + Copy + Sized> MutArrayView2<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> BaseArray<T, (usize, usize)> for ArrayView<'a, T, Ix2> {
+    fn get(&self, pos: (usize, usize)) -> &T {
+        &self[[pos.0, pos.1]]
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows(), self.ncols())
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(self.iter()),
+            _ => Box::new(
+                (0..self.ncols()).flat_map(move |c| (0..self.nrows()).map(move |r| &self[[r, c]])),
+            ),
+        }
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> Array2<T> for ArrayBase<OwnedRepr<T>, Ix2> {
+    fn get_row<'a>(&'a self, row: usize) -> Box<dyn ArrayView1<T> + 'a> {
+        Box::new(self.row(row))
+    }
+
+    fn get_col<'a>(&'a self, col: usize) -> Box<dyn ArrayView1<T> + 'a> {
+        Box::new(self.column(col))
+    }
+
+    fn slice<'a>(&'a self, rows: Range<usize>, cols: Range<usize>) -> Box<dyn ArrayView2<T> + 'a> {
+        Box::new(self.slice(s![rows, cols]))
+    }
+
+    fn slice_mut<'a>(
+        &'a mut self,
+        rows: Range<usize>,
+        cols: Range<usize>,
+    ) -> Box<dyn MutArrayView2<T> + 'a>
+    where
+        Self: Sized,
+    {
+        Box::new(self.slice_mut(s![rows, cols]))
+    }
+
+    fn fill(nrows: usize, ncols: usize, value: T) -> Self {
+        Array::from_elem([nrows, ncols], value)
+    }
+
+    fn from_iterator<I: Iterator<Item = T>>(iter: I, nrows: usize, ncols: usize, axis: u8) -> Self {
+        let a = Array::from_iter(iter.take(nrows * ncols))
+            .into_shape((nrows, ncols))
+            .unwrap();
+        match axis {
+            0 => a,
+            _ => a.reversed_axes().into_shape((nrows, ncols)).unwrap(),
+        }
+    }
+
+    fn transpose(&self) -> Self {
+        self.t().to_owned()
+    }
+}
+
+impl<T: Number + RealNumber> QRDecomposable<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+impl<T: Number + RealNumber> CholeskyDecomposable<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+impl<T: Number + RealNumber> EVDDecomposable<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+impl<T: Number + RealNumber> LUDecomposable<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+impl<T: Number + RealNumber> SVDDecomposable<T> for ArrayBase<OwnedRepr<T>, Ix2> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView2<T> for ArrayView<'a, T, Ix2> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> BaseArray<T, (usize, usize)>
+    for ArrayViewMut<'a, T, Ix2>
+{
+    fn get(&self, pos: (usize, usize)) -> &T {
+        &self[[pos.0, pos.1]]
+    }
+
+    fn shape(&self) -> (usize, usize) {
+        (self.nrows(), self.ncols())
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(
+            axis == 1 || axis == 0,
+            "For two dimensional array `axis` should be either 0 or 1"
+        );
+        match axis {
+            0 => Box::new(self.iter()),
+            _ => Box::new(
+                (0..self.ncols()).flat_map(move |c| (0..self.nrows()).map(move |r| &self[[r, c]])),
+            ),
+        }
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArray<T, (usize, usize)>
+    for ArrayViewMut<'a, T, Ix2>
+{
+    fn set(&mut self, pos: (usize, usize), x: T) {
+        self[[pos.0, pos.1]] = x
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        let ptr = self.as_mut_ptr();
+        let stride = self.strides();
+        let (rstride, cstride) = (stride[0] as usize, stride[1] as usize);
+        match axis {
+            0 => Box::new(self.iter_mut()),
+            _ => Box::new((0..self.ncols()).flat_map(move |c| {
+                (0..self.nrows()).map(move |r| unsafe { &mut *ptr.add(r * rstride + c * cstride) })
+            })),
+        }
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArrayView2<T> for ArrayViewMut<'a, T, Ix2> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView2<T> for ArrayViewMut<'a, T, Ix2> {}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use ndarray::{arr2, Array2 as NDArray2};
+
+    #[test]
+    fn test_get_set() {
+        let mut a = arr2(&[[1, 2, 3], [4, 5, 6]]);
+
+        assert_eq!(*BaseArray::get(&a, (1, 1)), 5);
+        a.set((1, 1), 9);
+        assert_eq!(a, arr2(&[[1, 2, 3], [4, 9, 6]]));
+    }
+
+    #[test]
+    fn test_iterator() {
+        let a = arr2(&[[1, 2, 3], [4, 5, 6]]);
+
+        let v: Vec<i32> = a.iterator(0).map(|&v| v).collect();
+        assert_eq!(v, vec!(1, 2, 3, 4, 5, 6));
+    }
+
+    #[test]
+    fn test_mut_iterator() {
+        let mut a = arr2(&[[1, 2, 3], [4, 5, 6]]);
+
+        a.iterator_mut(0).enumerate().for_each(|(i, v)| *v = i);
+        assert_eq!(a, arr2(&[[0, 1, 2], [3, 4, 5]]));
+        a.iterator_mut(1).enumerate().for_each(|(i, v)| *v = i);
+        assert_eq!(a, arr2(&[[0, 2, 4], [1, 3, 5]]));
+    }
+
+    #[test]
+    fn test_slice() {
+        let x = arr2(&[[1, 2, 3], [4, 5, 6]]);
+        let x_slice = Array2::slice(&x, 0..2, 1..2);
+        assert_eq!((2, 1), x_slice.shape());
+        let v: Vec<i32> = x_slice.iterator(0).map(|&v| v).collect();
+        assert_eq!(v, [2, 5]);
+    }
+
+    #[test]
+    fn test_slice_iter() {
+        let x = arr2(&[[1, 2, 3], [4, 5, 6]]);
+        let x_slice = Array2::slice(&x, 0..2, 0..3);
+        assert_eq!(
+            x_slice.iterator(0).map(|&v| v).collect::<Vec<i32>>(),
+            vec![1, 2, 3, 4, 5, 6]
+        );
+        assert_eq!(
+            x_slice.iterator(1).map(|&v| v).collect::<Vec<i32>>(),
+            vec![1, 4, 2, 5, 3, 6]
+        );
+    }
+
+    #[test]
+    fn test_slice_mut_iter() {
+        let mut x = arr2(&[[1, 2, 3], [4, 5, 6]]);
+        {
+            let mut x_slice = Array2::slice_mut(&mut x, 0..2, 0..3);
+            x_slice
+                .iterator_mut(0)
+                .enumerate()
+                .for_each(|(i, v)| *v = i);
+        }
+        assert_eq!(x, arr2(&[[0, 1, 2], [3, 4, 5]]));
+        {
+            let mut x_slice = Array2::slice_mut(&mut x, 0..2, 0..3);
+            x_slice
+                .iterator_mut(1)
+                .enumerate()
+                .for_each(|(i, v)| *v = i);
+        }
+        assert_eq!(x, arr2(&[[0, 2, 4], [1, 3, 5]]));
+    }
+
+    #[test]
+    fn test_c_from_iterator() {
+        let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12];
+        let a: NDArray2<i32> = Array2::from_iterator(data.clone().into_iter(), 4, 3, 0);
+        println!("{}", a);
+        let a: NDArray2<i32> = Array2::from_iterator(data.into_iter(), 4, 3, 1);
+        println!("{}", a);
+    }
+}

src/linalg/ndarray/mod.rs

@@ -0,0 +1,4 @@
+/// matrix bindings
+pub mod matrix;
+/// vector bindings
+pub mod vector;

src/linalg/ndarray/vector.rs

@@ -0,0 +1,184 @@
+use std::fmt::{Debug, Display};
+use std::ops::Range;
+
+use crate::linalg::basic::arrays::{
+    Array as BaseArray, Array1, ArrayView1, MutArray, MutArrayView1,
+};
+
+use ndarray::{s, Array, ArrayBase, ArrayView, ArrayViewMut, Ix1, OwnedRepr};
+
+impl<T: Debug + Display + Copy + Sized> BaseArray<T, usize> for ArrayBase<OwnedRepr<T>, Ix1> {
+    fn get(&self, i: usize) -> &T {
+        &self[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter())
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> MutArray<T, usize> for ArrayBase<OwnedRepr<T>, Ix1> {
+    fn set(&mut self, i: usize, x: T) {
+        self[i] = x
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter_mut())
+    }
+}
+
+impl<T: Debug + Display + Copy + Sized> ArrayView1<T> for ArrayBase<OwnedRepr<T>, Ix1> {}
+
+impl<T: Debug + Display + Copy + Sized> MutArrayView1<T> for ArrayBase<OwnedRepr<T>, Ix1> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> BaseArray<T, usize> for ArrayView<'a, T, Ix1> {
+    fn get(&self, i: usize) -> &T {
+        &self[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView1<T> for ArrayView<'a, T, Ix1> {}
+
+impl<'a, T: Debug + Display + Copy + Sized> BaseArray<T, usize> for ArrayViewMut<'a, T, Ix1> {
+    fn get(&self, i: usize) -> &T {
+        &self[i]
+    }
+
+    fn shape(&self) -> usize {
+        self.len()
+    }
+
+    fn is_empty(&self) -> bool {
+        self.len() > 0
+    }
+
+    fn iterator<'b>(&'b self, axis: u8) -> Box<dyn Iterator<Item = &'b T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> MutArray<T, usize> for ArrayViewMut<'a, T, Ix1> {
+    fn set(&mut self, i: usize, x: T) {
+        self[i] = x;
+    }
+
+    fn iterator_mut<'b>(&'b mut self, axis: u8) -> Box<dyn Iterator<Item = &'b mut T> + 'b> {
+        assert!(axis == 0, "For one dimensional array `axis` should == 0");
+        Box::new(self.iter_mut())
+    }
+}
+
+impl<'a, T: Debug + Display + Copy + Sized> ArrayView1<T> for ArrayViewMut<'a, T, Ix1> {}
+impl<'a, T: Debug + Display + Copy + Sized> MutArrayView1<T> for ArrayViewMut<'a, T, Ix1> {}
+
+impl<T: Debug + Display + Copy + Sized> Array1<T> for ArrayBase<OwnedRepr<T>, Ix1> {
+    fn slice<'a>(&'a self, range: Range<usize>) -> Box<dyn ArrayView1<T> + 'a> {
+        assert!(
+            range.end <= self.len(),
+            "`range` should be <= {}",
+            self.len()
+        );
+        Box::new(self.slice(s![range]))
+    }
+
+    fn slice_mut<'b>(&'b mut self, range: Range<usize>) -> Box<dyn MutArrayView1<T> + 'b> {
+        assert!(
+            range.end <= self.len(),
+            "`range` should be <= {}",
+            self.len()
+        );
+        Box::new(self.slice_mut(s![range]))
+    }
+
+    fn fill(len: usize, value: T) -> Self {
+        Array::from_elem(len, value)
+    }
+
+    fn from_iterator<I: Iterator<Item = T>>(iter: I, len: usize) -> Self
+    where
+        Self: Sized,
+    {
+        Array::from_iter(iter.take(len))
+    }
+
+    fn from_vec_slice(slice: &[T]) -> Self {
+        Array::from_iter(slice.iter().copied())
+    }
+
+    fn from_slice(slice: &dyn ArrayView1<T>) -> Self {
+        Array::from_iter(slice.iterator(0).copied())
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use ndarray::arr1;
+
+    #[test]
+    fn test_get_set() {
+        let mut a = arr1(&[1, 2, 3]);
+
+        assert_eq!(*BaseArray::get(&a, 1), 2);
+        a.set(1, 9);
+        assert_eq!(a, arr1(&[1, 9, 3]));
+    }
+
+    #[test]
+    fn test_iterator() {
+        let a = arr1(&[1, 2, 3]);
+
+        let v: Vec<i32> = a.iterator(0).map(|&v| v).collect();
+        assert_eq!(v, vec!(1, 2, 3));
+    }
+
+    #[test]
+    fn test_mut_iterator() {
+        let mut a = arr1(&[1, 2, 3]);
+
+        a.iterator_mut(0).for_each(|v| *v = 1);
+        assert_eq!(a, arr1(&[1, 1, 1]));
+    }
+
+    #[test]
+    fn test_slice() {
+        let x = arr1(&[1, 2, 3, 4, 5]);
+        let x_slice = Array1::slice(&x, 2..3);
+        assert_eq!(1, x_slice.shape());
+        assert_eq!(3, *x_slice.get(0));
+    }
+
+    #[test]
+    fn test_mut_slice() {
+        let mut x = arr1(&[1, 2, 3, 4, 5]);
+        let mut x_slice = Array1::slice_mut(&mut x, 2..4);
+        x_slice.set(0, 9);
+        assert_eq!(2, x_slice.shape());
+        assert_eq!(9, *x_slice.get(0));
+        assert_eq!(4, *x_slice.get(1));
+    }
+}

src/linalg/stats.rs

@@ -1,207 +0,0 @@
-//! # Various Statistical Methods
-//!
-//! This module provides reference implementations for  various statistical functions.
-//! Concrete implementations of the `BaseMatrix` trait are free to override these methods for better performance.
-
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
-
-/// Defines baseline implementations for various statistical functions
-pub trait MatrixStats<T: RealNumber>: BaseMatrix<T> {
-    /// Computes the arithmetic mean along the specified axis.
-    fn mean(&self, axis: u8) -> Vec<T> {
-        let (n, m) = match axis {
-            0 => {
-                let (n, m) = self.shape();
-                (m, n)
-            }
-            _ => self.shape(),
-        };
-
-        let mut x: Vec<T> = vec![T::zero(); n];
-
-        let div = T::from_usize(m).unwrap();
-
-        for (i, x_i) in x.iter_mut().enumerate().take(n) {
-            for j in 0..m {
-                *x_i += match axis {
-                    0 => self.get(j, i),
-                    _ => self.get(i, j),
-                };
-            }
-            *x_i /= div;
-        }
-
-        x
-    }
-
-    /// Computes variance along the specified axis.
-    fn var(&self, axis: u8) -> Vec<T> {
-        let (n, m) = match axis {
-            0 => {
-                let (n, m) = self.shape();
-                (m, n)
-            }
-            _ => self.shape(),
-        };
-
-        let mut x: Vec<T> = vec![T::zero(); n];
-
-        let div = T::from_usize(m).unwrap();
-
-        for (i, x_i) in x.iter_mut().enumerate().take(n) {
-            let mut mu = T::zero();
-            let mut sum = T::zero();
-            for j in 0..m {
-                let a = match axis {
-                    0 => self.get(j, i),
-                    _ => self.get(i, j),
-                };
-                mu += a;
-                sum += a * a;
-            }
-            mu /= div;
-            *x_i = sum / div - mu.powi(2);
-        }
-
-        x
-    }
-
-    /// Computes the standard deviation along the specified axis.
-    fn std(&self, axis: u8) -> Vec<T> {
-        let mut x = self.var(axis);
-
-        let n = match axis {
-            0 => self.shape().1,
-            _ => self.shape().0,
-        };
-
-        for x_i in x.iter_mut().take(n) {
-            *x_i = x_i.sqrt();
-        }
-
-        x
-    }
-
-    /// standardize values by removing the mean and scaling to unit variance
-    fn scale_mut(&mut self, mean: &[T], std: &[T], axis: u8) {
-        let (n, m) = match axis {
-            0 => {
-                let (n, m) = self.shape();
-                (m, n)
-            }
-            _ => self.shape(),
-        };
-
-        for i in 0..n {
-            for j in 0..m {
-                match axis {
-                    0 => self.set(j, i, (self.get(j, i) - mean[i]) / std[i]),
-                    _ => self.set(i, j, (self.get(i, j) - mean[i]) / std[i]),
-                }
-            }
-        }
-    }
-}
-
-/// Defines baseline implementations for various matrix processing functions
-pub trait MatrixPreprocessing<T: RealNumber>: BaseMatrix<T> {
-    /// Each element of the matrix greater than the threshold becomes 1, while values less than or equal to the threshold become 0
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// use crate::smartcore::linalg::stats::MatrixPreprocessing;
-    /// let mut a = DenseMatrix::from_array(2, 3, &[0., 2., 3., -5., -6., -7.]);
-    /// let expected = DenseMatrix::from_array(2, 3, &[0., 1., 1., 0., 0., 0.]);
-    /// a.binarize_mut(0.);
-    ///
-    /// assert_eq!(a, expected);
-    /// ```
-
-    fn binarize_mut(&mut self, threshold: T) {
-        let (nrows, ncols) = self.shape();
-        for row in 0..nrows {
-            for col in 0..ncols {
-                if self.get(row, col) > threshold {
-                    self.set(row, col, T::one());
-                } else {
-                    self.set(row, col, T::zero());
-                }
-            }
-        }
-    }
-    /// Returns new matrix where elements are binarized according to a given threshold.
-    /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// use crate::smartcore::linalg::stats::MatrixPreprocessing;
-    /// let a = DenseMatrix::from_array(2, 3, &[0., 2., 3., -5., -6., -7.]);
-    /// let expected = DenseMatrix::from_array(2, 3, &[0., 1., 1., 0., 0., 0.]);
-    ///
-    /// assert_eq!(a.binarize(0.), expected);
-    /// ```
-    fn binarize(&self, threshold: T) -> Self {
-        let mut m = self.clone();
-        m.binarize_mut(threshold);
-        m
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
-    use crate::linalg::BaseVector;
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn mean() {
-        let m = DenseMatrix::from_2d_array(&[
-            &[1., 2., 3., 1., 2.],
-            &[4., 5., 6., 3., 4.],
-            &[7., 8., 9., 5., 6.],
-        ]);
-        let expected_0 = vec![4., 5., 6., 3., 4.];
-        let expected_1 = vec![1.8, 4.4, 7.];
-
-        assert_eq!(m.mean(0), expected_0);
-        assert_eq!(m.mean(1), expected_1);
-    }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn std() {
-        let m = DenseMatrix::from_2d_array(&[
-            &[1., 2., 3., 1., 2.],
-            &[4., 5., 6., 3., 4.],
-            &[7., 8., 9., 5., 6.],
-        ]);
-        let expected_0 = vec![2.44, 2.44, 2.44, 1.63, 1.63];
-        let expected_1 = vec![0.74, 1.01, 1.41];
-
-        assert!(m.std(0).approximate_eq(&expected_0, 1e-2));
-        assert!(m.std(1).approximate_eq(&expected_1, 1e-2));
-    }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn var() {
-        let m = DenseMatrix::from_2d_array(&[&[1., 2., 3., 4.], &[5., 6., 7., 8.]]);
-        let expected_0 = vec![4., 4., 4., 4.];
-        let expected_1 = vec![1.25, 1.25];
-
-        assert!(m.var(0).approximate_eq(&expected_0, std::f64::EPSILON));
-        assert!(m.var(1).approximate_eq(&expected_1, std::f64::EPSILON));
-    }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn scale() {
-        let mut m = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[4., 5., 6.]]);
-        let expected_0 = DenseMatrix::from_2d_array(&[&[-1., -1., -1.], &[1., 1., 1.]]);
-        let expected_1 = DenseMatrix::from_2d_array(&[&[-1.22, 0.0, 1.22], &[-1.22, 0.0, 1.22]]);
-
-        {
-            let mut m = m.clone();
-            m.scale_mut(&m.mean(0), &m.std(0), 0);
-            assert!(m.approximate_eq(&expected_0, std::f32::EPSILON));
-        }
-
-        m.scale_mut(&m.mean(1), &m.std(1), 1);
-        assert!(m.approximate_eq(&expected_1, 1e-2));
-    }
-}

src/linalg/traits/cholesky.rs

@@ -8,8 +8,8 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use crate::smartcore::linalg::cholesky::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::traits::cholesky::*;
 //!
 //! let A = DenseMatrix::from_2d_array(&[
 //!                 &[25., 15., -5.],
@@ -34,17 +34,18 @@ use std::fmt::Debug;
 use std::marker::PhantomData;
 
 use crate::error::{Failed, FailedError};
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 
 #[derive(Debug, Clone)]
 /// Results of Cholesky decomposition.
-pub struct Cholesky<T: RealNumber, M: BaseMatrix<T>> {
+pub struct Cholesky<T: Number + RealNumber, M: Array2<T>> {
     R: M,
     t: PhantomData<T>,
 }
 
-impl<T: RealNumber, M: BaseMatrix<T>> Cholesky<T, M> {
+impl<T: Number + RealNumber, M: Array2<T>> Cholesky<T, M> {
     pub(crate) fn new(R: M) -> Cholesky<T, M> {
         Cholesky { R, t: PhantomData }
     }
@@ -57,7 +58,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> Cholesky<T, M> {
         for i in 0..n {
             for j in 0..n {
                 if j <= i {
-                    R.set(i, j, self.R.get(i, j));
+                    R.set((i, j), *self.R.get((i, j)));
                 }
             }
         }
@@ -72,7 +73,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> Cholesky<T, M> {
         for i in 0..n {
             for j in 0..n {
                 if j <= i {
-                    R.set(j, i, self.R.get(i, j));
+                    R.set((j, i), *self.R.get((i, j)));
                 }
             }
         }
@@ -87,25 +88,25 @@ impl<T: RealNumber, M: BaseMatrix<T>> Cholesky<T, M> {
         if bn != rn {
             return Err(Failed::because(
                 FailedError::SolutionFailed,
-                "Can\'t solve Ax = b for x. Number of rows in b != number of rows in R.",
+                "Can\'t solve Ax = b for x. FloatNumber of rows in b != number of rows in R.",
             ));
         }
 
         for k in 0..bn {
             for j in 0..m {
                 for i in 0..k {
-                    b.sub_element_mut(k, j, b.get(i, j) * self.R.get(k, i));
+                    b.sub_element_mut((k, j), *b.get((i, j)) * *self.R.get((k, i)));
                 }
-                b.div_element_mut(k, j, self.R.get(k, k));
+                b.div_element_mut((k, j), *self.R.get((k, k)));
             }
         }
 
         for k in (0..bn).rev() {
             for j in 0..m {
                 for i in k + 1..bn {
-                    b.sub_element_mut(k, j, b.get(i, j) * self.R.get(i, k));
+                    b.sub_element_mut((k, j), *b.get((i, j)) * *self.R.get((i, k)));
                 }
-                b.div_element_mut(k, j, self.R.get(k, k));
+                b.div_element_mut((k, j), *self.R.get((k, k)));
             }
         }
         Ok(b)
@@ -113,7 +114,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> Cholesky<T, M> {
 }
 
 /// Trait that implements Cholesky decomposition routine for any matrix.
-pub trait CholeskyDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
+pub trait CholeskyDecomposable<T: Number + RealNumber>: Array2<T> {
     /// Compute the Cholesky decomposition of a matrix.
     fn cholesky(&self) -> Result<Cholesky<T, Self>, Failed> {
         self.clone().cholesky_mut()
@@ -136,13 +137,13 @@ pub trait CholeskyDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             for k in 0..j {
                 let mut s = T::zero();
                 for i in 0..k {
-                    s += self.get(k, i) * self.get(j, i);
+                    s += *self.get((k, i)) * *self.get((j, i));
                 }
-                s = (self.get(j, k) - s) / self.get(k, k);
-                self.set(j, k, s);
+                s = (*self.get((j, k)) - s) / *self.get((k, k));
+                self.set((j, k), s);
                 d += s * s;
             }
-            d = self.get(j, j) - d;
+            d = *self.get((j, j)) - d;
 
             if d < T::zero() {
                 return Err(Failed::because(
@@ -151,7 +152,7 @@ pub trait CholeskyDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                 ));
             }
 
-            self.set(j, j, d.sqrt());
+            self.set((j, j), d.sqrt());
         }
 
         Ok(Cholesky::new(self))
@@ -166,8 +167,12 @@ pub trait CholeskyDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn cholesky_decompose() {
         let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
@@ -177,16 +182,19 @@ mod tests {
             DenseMatrix::from_2d_array(&[&[5.0, 3.0, -1.0], &[0.0, 3.0, 1.0], &[0.0, 0.0, 3.0]]);
         let cholesky = a.cholesky().unwrap();
 
-        assert!(cholesky.L().abs().approximate_eq(&l.abs(), 1e-4));
-        assert!(cholesky.U().abs().approximate_eq(&u.abs(), 1e-4));
-        assert!(cholesky
-            .L()
-            .matmul(&cholesky.U())
-            .abs()
-            .approximate_eq(&a.abs(), 1e-4));
+        assert!(relative_eq!(cholesky.L().abs(), l.abs(), epsilon = 1e-4));
+        assert!(relative_eq!(cholesky.U().abs(), u.abs(), epsilon = 1e-4));
+        assert!(relative_eq!(
+            cholesky.L().matmul(&cholesky.U()).abs(),
+            a.abs(),
+            epsilon = 1e-4
+        ));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn cholesky_solve_mut() {
         let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
@@ -195,10 +203,10 @@ mod tests {
 
         let cholesky = a.cholesky().unwrap();
 
-        assert!(cholesky
-            .solve(b.transpose())
-            .unwrap()
-            .transpose()
-            .approximate_eq(&expected, 1e-4));
+        assert!(relative_eq!(
+            cholesky.solve(b.transpose()).unwrap().transpose(),
+            expected,
+            epsilon = 1e-4
+        ));
     }
 }

src/linalg/traits/evd.rs

@@ -12,8 +12,8 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::evd::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::traits::evd::*;
 //!
 //! let A = DenseMatrix::from_2d_array(&[
 //!                  &[0.9000, 0.4000, 0.7000],
@@ -25,19 +25,6 @@
 //! let eigenvectors: DenseMatrix<f64> = evd.V;
 //! let eigenvalues: Vec<f64> = evd.d;
 //! ```
-//! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::evd::*;
-//!
-//! let A = DenseMatrix::from_2d_array(&[
-//!     &[-5.0, 2.0],
-//!     &[-7.0, 4.0],
-//! ]);
-//!
-//! let evd = A.evd(false).unwrap();
-//! let eigenvectors: DenseMatrix<f64> = evd.V;
-//! let eigenvalues: Vec<f64> = evd.d;
-//! ```
 //!
 //! ## References:
 //! * ["Numerical Recipes: The Art of Scientific Computing",  Press W.H., Teukolsky S.A., Vetterling W.T, Flannery B.P, 3rd ed., Section 11 Eigensystems](http://numerical.recipes/)
@@ -48,14 +35,15 @@
 #![allow(non_snake_case)]
 
 use crate::error::Failed;
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 use num::complex::Complex;
 use std::fmt::Debug;
 
 #[derive(Debug, Clone)]
 /// Results of eigen decomposition
-pub struct EVD<T: RealNumber, M: BaseMatrix<T>> {
+pub struct EVD<T: Number + RealNumber, M: Array2<T>> {
     /// Real part of eigenvalues.
     pub d: Vec<T>,
     /// Imaginary part of eigenvalues.
@@ -65,7 +53,7 @@ pub struct EVD<T: RealNumber, M: BaseMatrix<T>> {
 }
 
 /// Trait that implements EVD decomposition routine for any matrix.
-pub trait EVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
+pub trait EVDDecomposable<T: Number + RealNumber>: Array2<T> {
     /// Compute the eigen decomposition of a square matrix.
     /// * `symmetric` - whether the matrix is symmetric
     fn evd(&self, symmetric: bool) -> Result<EVD<T, Self>, Failed> {
@@ -106,14 +94,14 @@ pub trait EVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             sort(&mut d, &mut e, &mut V);
         }
 
-        Ok(EVD { d, e, V })
+        Ok(EVD { V, d, e })
     }
 }
 
-fn tred2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
+fn tred2<T: Number + RealNumber, M: Array2<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
     let (n, _) = V.shape();
     for (i, d_i) in d.iter_mut().enumerate().take(n) {
-        *d_i = V.get(n - 1, i);
+        *d_i = *V.get((n - 1, i));
     }
 
     for i in (1..n).rev() {
@@ -125,9 +113,9 @@ fn tred2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
         if scale == T::zero() {
             e[i] = d[i - 1];
             for (j, d_j) in d.iter_mut().enumerate().take(i) {
-                *d_j = V.get(i - 1, j);
-                V.set(i, j, T::zero());
-                V.set(j, i, T::zero());
+                *d_j = *V.get((i - 1, j));
+                V.set((i, j), T::zero());
+                V.set((j, i), T::zero());
             }
         } else {
             for d_k in d.iter_mut().take(i) {
@@ -148,11 +136,11 @@ fn tred2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
 
             for j in 0..i {
                 f = d[j];
-                V.set(j, i, f);
-                g = e[j] + V.get(j, j) * f;
+                V.set((j, i), f);
+                g = e[j] + *V.get((j, j)) * f;
                 for k in j + 1..=i - 1 {
-                    g += V.get(k, j) * d[k];
-                    e[k] += V.get(k, j) * f;
+                    g += *V.get((k, j)) * d[k];
+                    e[k] += *V.get((k, j)) * f;
                 }
                 e[j] = g;
             }
@@ -169,46 +157,46 @@ fn tred2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
                 f = d[j];
                 g = e[j];
                 for k in j..=i - 1 {
-                    V.sub_element_mut(k, j, f * e[k] + g * d[k]);
+                    V.sub_element_mut((k, j), f * e[k] + g * d[k]);
                 }
-                d[j] = V.get(i - 1, j);
-                V.set(i, j, T::zero());
+                d[j] = *V.get((i - 1, j));
+                V.set((i, j), T::zero());
             }
         }
         d[i] = h;
     }
 
     for i in 0..n - 1 {
-        V.set(n - 1, i, V.get(i, i));
-        V.set(i, i, T::one());
+        V.set((n - 1, i), *V.get((i, i)));
+        V.set((i, i), T::one());
         let h = d[i + 1];
         if h != T::zero() {
             for (k, d_k) in d.iter_mut().enumerate().take(i + 1) {
-                *d_k = V.get(k, i + 1) / h;
+                *d_k = *V.get((k, i + 1)) / h;
             }
             for j in 0..=i {
                 let mut g = T::zero();
                 for k in 0..=i {
-                    g += V.get(k, i + 1) * V.get(k, j);
+                    g += *V.get((k, i + 1)) * *V.get((k, j));
                 }
                 for (k, d_k) in d.iter().enumerate().take(i + 1) {
-                    V.sub_element_mut(k, j, g * (*d_k));
+                    V.sub_element_mut((k, j), g * (*d_k));
                 }
             }
         }
         for k in 0..=i {
-            V.set(k, i + 1, T::zero());
+            V.set((k, i + 1), T::zero());
         }
     }
     for (j, d_j) in d.iter_mut().enumerate().take(n) {
-        *d_j = V.get(n - 1, j);
-        V.set(n - 1, j, T::zero());
+        *d_j = *V.get((n - 1, j));
+        V.set((n - 1, j), T::zero());
     }
-    V.set(n - 1, n - 1, T::one());
+    V.set((n - 1, n - 1), T::one());
     e[0] = T::zero();
 }
 
-fn tql2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
+fn tql2<T: Number + RealNumber, M: Array2<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
     let (n, _) = V.shape();
     for i in 1..n {
         e[i - 1] = e[i];
@@ -277,9 +265,9 @@ fn tql2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
                     d[i + 1] = h + s * (c * g + s * d[i]);
 
                     for k in 0..n {
-                        h = V.get(k, i + 1);
-                        V.set(k, i + 1, s * V.get(k, i) + c * h);
-                        V.set(k, i, c * V.get(k, i) - s * h);
+                        h = *V.get((k, i + 1));
+                        V.set((k, i + 1), s * *V.get((k, i)) + c * h);
+                        V.set((k, i), c * *V.get((k, i)) - s * h);
                     }
                 }
                 p = -s * s2 * c3 * el1 * e[l] / dl1;
@@ -308,15 +296,15 @@ fn tql2<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, d: &mut [T], e: &mut [T]) {
             d[k] = d[i];
             d[i] = p;
             for j in 0..n {
-                p = V.get(j, i);
-                V.set(j, i, V.get(j, k));
-                V.set(j, k, p);
+                p = *V.get((j, i));
+                V.set((j, i), *V.get((j, k)));
+                V.set((j, k), p);
             }
         }
     }
 }
 
-fn balance<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<T> {
+fn balance<T: Number + RealNumber, M: Array2<T>>(A: &mut M) -> Vec<T> {
     let radix = T::two();
     let sqrdx = radix * radix;
 
@@ -334,8 +322,8 @@ fn balance<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<T> {
             let mut c = T::zero();
             for j in 0..n {
                 if j != i {
-                    c += A.get(j, i).abs();
-                    r += A.get(i, j).abs();
+                    c += A.get((j, i)).abs();
+                    r += A.get((i, j)).abs();
                 }
             }
             if c != T::zero() && r != T::zero() {
@@ -356,10 +344,10 @@ fn balance<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<T> {
                     g = T::one() / f;
                     *scale_i *= f;
                     for j in 0..n {
-                        A.mul_element_mut(i, j, g);
+                        A.mul_element_mut((i, j), g);
                     }
                     for j in 0..n {
-                        A.mul_element_mut(j, i, f);
+                        A.mul_element_mut((j, i), f);
                     }
                 }
             }
@@ -369,7 +357,7 @@ fn balance<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<T> {
     scale
 }
 
-fn elmhes<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<usize> {
+fn elmhes<T: Number + RealNumber, M: Array2<T>>(A: &mut M) -> Vec<usize> {
     let (n, _) = A.shape();
     let mut perm = vec![0; n];
 
@@ -377,35 +365,31 @@ fn elmhes<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<usize> {
         let mut x = T::zero();
         let mut i = m;
         for j in m..n {
-            if A.get(j, m - 1).abs() > x.abs() {
-                x = A.get(j, m - 1);
+            if A.get((j, m - 1)).abs() > x.abs() {
+                x = *A.get((j, m - 1));
                 i = j;
             }
         }
         *perm_m = i;
         if i != m {
             for j in (m - 1)..n {
-                let swap = A.get(i, j);
-                A.set(i, j, A.get(m, j));
-                A.set(m, j, swap);
+                A.swap((i, j), (m, j));
             }
             for j in 0..n {
-                let swap = A.get(j, i);
-                A.set(j, i, A.get(j, m));
-                A.set(j, m, swap);
+                A.swap((j, i), (j, m));
             }
         }
         if x != T::zero() {
             for i in (m + 1)..n {
-                let mut y = A.get(i, m - 1);
+                let mut y = *A.get((i, m - 1));
                 if y != T::zero() {
                     y /= x;
-                    A.set(i, m - 1, y);
+                    A.set((i, m - 1), y);
                     for j in m..n {
-                        A.sub_element_mut(i, j, y * A.get(m, j));
+                        A.sub_element_mut((i, j), y * *A.get((m, j)));
                     }
                     for j in 0..n {
-                        A.add_element_mut(j, m, y * A.get(j, i));
+                        A.add_element_mut((j, m), y * *A.get((j, i)));
                     }
                 }
             }
@@ -415,24 +399,24 @@ fn elmhes<T: RealNumber, M: BaseMatrix<T>>(A: &mut M) -> Vec<usize> {
     perm
 }
 
-fn eltran<T: RealNumber, M: BaseMatrix<T>>(A: &M, V: &mut M, perm: &[usize]) {
+fn eltran<T: Number + RealNumber, M: Array2<T>>(A: &M, V: &mut M, perm: &[usize]) {
     let (n, _) = A.shape();
     for mp in (1..n - 1).rev() {
         for k in mp + 1..n {
-            V.set(k, mp, A.get(k, mp - 1));
+            V.set((k, mp), *A.get((k, mp - 1)));
         }
         let i = perm[mp];
         if i != mp {
             for j in mp..n {
-                V.set(mp, j, V.get(i, j));
-                V.set(i, j, T::zero());
+                V.set((mp, j), *V.get((i, j)));
+                V.set((i, j), T::zero());
             }
-            V.set(i, mp, T::one());
+            V.set((i, mp), T::one());
         }
     }
 }
 
-fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &mut [T]) {
+fn hqr2<T: Number + RealNumber, M: Array2<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &mut [T]) {
     let (n, _) = A.shape();
     let mut z = T::zero();
     let mut s = T::zero();
@@ -443,7 +427,7 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
 
     for i in 0..n {
         for j in i32::max(i as i32 - 1, 0)..n as i32 {
-            anorm += A.get(i, j as usize).abs();
+            anorm += A.get((i, j as usize)).abs();
         }
     }
 
@@ -454,43 +438,43 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
         loop {
             let mut l = nn;
             while l > 0 {
-                s = A.get(l - 1, l - 1).abs() + A.get(l, l).abs();
+                s = A.get((l - 1, l - 1)).abs() + A.get((l, l)).abs();
                 if s == T::zero() {
                     s = anorm;
                 }
-                if A.get(l, l - 1).abs() <= T::epsilon() * s {
-                    A.set(l, l - 1, T::zero());
+                if A.get((l, l - 1)).abs() <= T::epsilon() * s {
+                    A.set((l, l - 1), T::zero());
                     break;
                 }
                 l -= 1;
             }
-            let mut x = A.get(nn, nn);
+            let mut x = *A.get((nn, nn));
             if l == nn {
                 d[nn] = x + t;
-                A.set(nn, nn, x + t);
+                A.set((nn, nn), x + t);
                 if nn == 0 {
                     break 'outer;
                 } else {
                     nn -= 1;
                 }
             } else {
-                let mut y = A.get(nn - 1, nn - 1);
-                let mut w = A.get(nn, nn - 1) * A.get(nn - 1, nn);
+                let mut y = *A.get((nn - 1, nn - 1));
+                let mut w = *A.get((nn, nn - 1)) * *A.get((nn - 1, nn));
                 if l == nn - 1 {
                     p = T::half() * (y - x);
                     q = p * p + w;
                     z = q.abs().sqrt();
                     x += t;
-                    A.set(nn, nn, x);
-                    A.set(nn - 1, nn - 1, y + t);
+                    A.set((nn, nn), x);
+                    A.set((nn - 1, nn - 1), y + t);
                     if q >= T::zero() {
-                        z = p + RealNumber::copysign(z, p);
+                        z = p + <T as RealNumber>::copysign(z, p);
                         d[nn - 1] = x + z;
                         d[nn] = x + z;
                         if z != T::zero() {
                             d[nn] = x - w / z;
                         }
-                        x = A.get(nn, nn - 1);
+                        x = *A.get((nn, nn - 1));
                         s = x.abs() + z.abs();
                         p = x / s;
                         q = z / s;
@@ -498,19 +482,19 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                         p /= r;
                         q /= r;
                         for j in nn - 1..n {
-                            z = A.get(nn - 1, j);
-                            A.set(nn - 1, j, q * z + p * A.get(nn, j));
-                            A.set(nn, j, q * A.get(nn, j) - p * z);
+                            z = *A.get((nn - 1, j));
+                            A.set((nn - 1, j), q * z + p * *A.get((nn, j)));
+                            A.set((nn, j), q * *A.get((nn, j)) - p * z);
                         }
                         for i in 0..=nn {
-                            z = A.get(i, nn - 1);
-                            A.set(i, nn - 1, q * z + p * A.get(i, nn));
-                            A.set(i, nn, q * A.get(i, nn) - p * z);
+                            z = *A.get((i, nn - 1));
+                            A.set((i, nn - 1), q * z + p * *A.get((i, nn)));
+                            A.set((i, nn), q * *A.get((i, nn)) - p * z);
                         }
                         for i in 0..n {
-                            z = V.get(i, nn - 1);
-                            V.set(i, nn - 1, q * z + p * V.get(i, nn));
-                            V.set(i, nn, q * V.get(i, nn) - p * z);
+                            z = *V.get((i, nn - 1));
+                            V.set((i, nn - 1), q * z + p * *V.get((i, nn)));
+                            V.set((i, nn), q * *V.get((i, nn)) - p * z);
                         }
                     } else {
                         d[nn] = x + p;
@@ -531,22 +515,22 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                     if its == 10 || its == 20 {
                         t += x;
                         for i in 0..nn + 1 {
-                            A.sub_element_mut(i, i, x);
+                            A.sub_element_mut((i, i), x);
                         }
-                        s = A.get(nn, nn - 1).abs() + A.get(nn - 1, nn - 2).abs();
-                        y = T::from(0.75).unwrap() * s;
-                        x = T::from(0.75).unwrap() * s;
-                        w = T::from(-0.4375).unwrap() * s * s;
+                        s = A.get((nn, nn - 1)).abs() + A.get((nn - 1, nn - 2)).abs();
+                        y = T::from_f64(0.75).unwrap() * s;
+                        x = T::from_f64(0.75).unwrap() * s;
+                        w = T::from_f64(-0.4375).unwrap() * s * s;
                     }
                     its += 1;
                     let mut m = nn - 2;
                     while m >= l {
-                        z = A.get(m, m);
+                        z = *A.get((m, m));
                         r = x - z;
                         s = y - z;
-                        p = (r * s - w) / A.get(m + 1, m) + A.get(m, m + 1);
-                        q = A.get(m + 1, m + 1) - z - r - s;
-                        r = A.get(m + 2, m + 1);
+                        p = (r * s - w) / *A.get((m + 1, m)) + *A.get((m, m + 1));
+                        q = *A.get((m + 1, m + 1)) - z - r - s;
+                        r = *A.get((m + 2, m + 1));
                         s = p.abs() + q.abs() + r.abs();
                         p /= s;
                         q /= s;
@@ -554,27 +538,27 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                         if m == l {
                             break;
                         }
-                        let u = A.get(m, m - 1).abs() * (q.abs() + r.abs());
+                        let u = A.get((m, m - 1)).abs() * (q.abs() + r.abs());
                         let v = p.abs()
-                            * (A.get(m - 1, m - 1).abs() + z.abs() + A.get(m + 1, m + 1).abs());
+                            * (A.get((m - 1, m - 1)).abs() + z.abs() + A.get((m + 1, m + 1)).abs());
                         if u <= T::epsilon() * v {
                             break;
                         }
                         m -= 1;
                     }
                     for i in m..nn - 1 {
-                        A.set(i + 2, i, T::zero());
+                        A.set((i + 2, i), T::zero());
                         if i != m {
-                            A.set(i + 2, i - 1, T::zero());
+                            A.set((i + 2, i - 1), T::zero());
                         }
                     }
                     for k in m..nn {
                         if k != m {
-                            p = A.get(k, k - 1);
-                            q = A.get(k + 1, k - 1);
+                            p = *A.get((k, k - 1));
+                            q = *A.get((k + 1, k - 1));
                             r = T::zero();
                             if k + 1 != nn {
-                                r = A.get(k + 2, k - 1);
+                                r = *A.get((k + 2, k - 1));
                             }
                             x = p.abs() + q.abs() + r.abs();
                             if x != T::zero() {
@@ -583,14 +567,14 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                                 r /= x;
                             }
                         }
-                        let s = RealNumber::copysign((p * p + q * q + r * r).sqrt(), p);
+                        let s = <T as RealNumber>::copysign((p * p + q * q + r * r).sqrt(), p);
                         if s != T::zero() {
                             if k == m {
                                 if l != m {
-                                    A.set(k, k - 1, -A.get(k, k - 1));
+                                    A.set((k, k - 1), -*A.get((k, k - 1)));
                                 }
                             } else {
-                                A.set(k, k - 1, -s * x);
+                                A.set((k, k - 1), -s * x);
                             }
                             p += s;
                             x = p / s;
@@ -599,32 +583,33 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                             q /= p;
                             r /= p;
                             for j in k..n {
-                                p = A.get(k, j) + q * A.get(k + 1, j);
+                                p = *A.get((k, j)) + q * *A.get((k + 1, j));
                                 if k + 1 != nn {
-                                    p += r * A.get(k + 2, j);
-                                    A.sub_element_mut(k + 2, j, p * z);
+                                    p += r * *A.get((k + 2, j));
+                                    A.sub_element_mut((k + 2, j), p * z);
                                 }
-                                A.sub_element_mut(k + 1, j, p * y);
-                                A.sub_element_mut(k, j, p * x);
+                                A.sub_element_mut((k + 1, j), p * y);
+                                A.sub_element_mut((k, j), p * x);
                             }
+
                             let mmin = if nn < k + 3 { nn } else { k + 3 };
-                            for i in 0..mmin + 1 {
-                                p = x * A.get(i, k) + y * A.get(i, k + 1);
+                            for i in 0..(mmin + 1) {
+                                p = x * *A.get((i, k)) + y * *A.get((i, k + 1));
                                 if k + 1 != nn {
-                                    p += z * A.get(i, k + 2);
-                                    A.sub_element_mut(i, k + 2, p * r);
+                                    p += z * *A.get((i, k + 2));
+                                    A.sub_element_mut((i, k + 2), p * r);
                                 }
-                                A.sub_element_mut(i, k + 1, p * q);
-                                A.sub_element_mut(i, k, p);
+                                A.sub_element_mut((i, k + 1), p * q);
+                                A.sub_element_mut((i, k), p);
                             }
                             for i in 0..n {
-                                p = x * V.get(i, k) + y * V.get(i, k + 1);
+                                p = x * *V.get((i, k)) + y * *V.get((i, k + 1));
                                 if k + 1 != nn {
-                                    p += z * V.get(i, k + 2);
-                                    V.sub_element_mut(i, k + 2, p * r);
+                                    p += z * *V.get((i, k + 2));
+                                    V.sub_element_mut((i, k + 2), p * r);
                                 }
-                                V.sub_element_mut(i, k + 1, p * q);
-                                V.sub_element_mut(i, k, p);
+                                V.sub_element_mut((i, k + 1), p * q);
+                                V.sub_element_mut((i, k), p);
                             }
                         }
                     }
@@ -643,14 +628,14 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
             let na = nn.wrapping_sub(1);
             if q == T::zero() {
                 let mut m = nn;
-                A.set(nn, nn, T::one());
+                A.set((nn, nn), T::one());
                 if nn > 0 {
                     let mut i = nn - 1;
                     loop {
-                        let w = A.get(i, i) - p;
+                        let w = *A.get((i, i)) - p;
                         r = T::zero();
                         for j in m..=nn {
-                            r += A.get(i, j) * A.get(j, nn);
+                            r += *A.get((i, j)) * *A.get((j, nn));
                         }
                         if e[i] < T::zero() {
                             z = w;
@@ -663,23 +648,23 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                                 if t == T::zero() {
                                     t = T::epsilon() * anorm;
                                 }
-                                A.set(i, nn, -r / t);
+                                A.set((i, nn), -r / t);
                             } else {
-                                let x = A.get(i, i + 1);
-                                let y = A.get(i + 1, i);
+                                let x = *A.get((i, i + 1));
+                                let y = *A.get((i + 1, i));
                                 q = (d[i] - p).powf(T::two()) + e[i].powf(T::two());
                                 t = (x * s - z * r) / q;
-                                A.set(i, nn, t);
+                                A.set((i, nn), t);
                                 if x.abs() > z.abs() {
-                                    A.set(i + 1, nn, (-r - w * t) / x);
+                                    A.set((i + 1, nn), (-r - w * t) / x);
                                 } else {
-                                    A.set(i + 1, nn, (-s - y * t) / z);
+                                    A.set((i + 1, nn), (-s - y * t) / z);
                                 }
                             }
-                            t = A.get(i, nn).abs();
+                            t = A.get((i, nn)).abs();
                             if T::epsilon() * t * t > T::one() {
                                 for j in i..=nn {
-                                    A.div_element_mut(j, nn, t);
+                                    A.div_element_mut((j, nn), t);
                                 }
                             }
                         }
@@ -692,25 +677,25 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                 }
             } else if q < T::zero() {
                 let mut m = na;
-                if A.get(nn, na).abs() > A.get(na, nn).abs() {
-                    A.set(na, na, q / A.get(nn, na));
-                    A.set(na, nn, -(A.get(nn, nn) - p) / A.get(nn, na));
+                if A.get((nn, na)).abs() > A.get((na, nn)).abs() {
+                    A.set((na, na), q / *A.get((nn, na)));
+                    A.set((na, nn), -(*A.get((nn, nn)) - p) / *A.get((nn, na)));
                 } else {
-                    let temp = Complex::new(T::zero(), -A.get(na, nn))
-                        / Complex::new(A.get(na, na) - p, q);
-                    A.set(na, na, temp.re);
-                    A.set(na, nn, temp.im);
+                    let temp = Complex::new(T::zero(), -*A.get((na, nn)))
+                        / Complex::new(*A.get((na, na)) - p, q);
+                    A.set((na, na), temp.re);
+                    A.set((na, nn), temp.im);
                 }
-                A.set(nn, na, T::zero());
-                A.set(nn, nn, T::one());
+                A.set((nn, na), T::zero());
+                A.set((nn, nn), T::one());
                 if nn >= 2 {
                     for i in (0..nn - 1).rev() {
-                        let w = A.get(i, i) - p;
+                        let w = *A.get((i, i)) - p;
                         let mut ra = T::zero();
                         let mut sa = T::zero();
                         for j in m..=nn {
-                            ra += A.get(i, j) * A.get(j, na);
-                            sa += A.get(i, j) * A.get(j, nn);
+                            ra += *A.get((i, j)) * *A.get((j, na));
+                            sa += *A.get((i, j)) * *A.get((j, nn));
                         }
                         if e[i] < T::zero() {
                             z = w;
@@ -720,11 +705,11 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                             m = i;
                             if e[i] == T::zero() {
                                 let temp = Complex::new(-ra, -sa) / Complex::new(w, q);
-                                A.set(i, na, temp.re);
-                                A.set(i, nn, temp.im);
+                                A.set((i, na), temp.re);
+                                A.set((i, nn), temp.im);
                             } else {
-                                let x = A.get(i, i + 1);
-                                let y = A.get(i + 1, i);
+                                let x = *A.get((i, i + 1));
+                                let y = *A.get((i + 1, i));
                                 let mut vr =
                                     (d[i] - p).powf(T::two()) + (e[i]).powf(T::two()) - q * q;
                                 let vi = T::two() * q * (d[i] - p);
@@ -736,33 +721,32 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
                                 let temp =
                                     Complex::new(x * r - z * ra + q * sa, x * s - z * sa - q * ra)
                                         / Complex::new(vr, vi);
-                                A.set(i, na, temp.re);
-                                A.set(i, nn, temp.im);
+                                A.set((i, na), temp.re);
+                                A.set((i, nn), temp.im);
                                 if x.abs() > z.abs() + q.abs() {
                                     A.set(
-                                        i + 1,
-                                        na,
-                                        (-ra - w * A.get(i, na) + q * A.get(i, nn)) / x,
+                                        (i + 1, na),
+                                        (-ra - w * *A.get((i, na)) + q * *A.get((i, nn))) / x,
                                     );
                                     A.set(
-                                        i + 1,
-                                        nn,
-                                        (-sa - w * A.get(i, nn) - q * A.get(i, na)) / x,
+                                        (i + 1, nn),
+                                        (-sa - w * *A.get((i, nn)) - q * *A.get((i, na))) / x,
                                     );
                                 } else {
-                                    let temp =
-                                        Complex::new(-r - y * A.get(i, na), -s - y * A.get(i, nn))
-                                            / Complex::new(z, q);
-                                    A.set(i + 1, na, temp.re);
-                                    A.set(i + 1, nn, temp.im);
+                                    let temp = Complex::new(
+                                        -r - y * *A.get((i, na)),
+                                        -s - y * *A.get((i, nn)),
+                                    ) / Complex::new(z, q);
+                                    A.set((i + 1, na), temp.re);
+                                    A.set((i + 1, nn), temp.im);
                                 }
                             }
                         }
-                        t = T::max(A.get(i, na).abs(), A.get(i, nn).abs());
+                        t = T::max(A.get((i, na)).abs(), A.get((i, nn)).abs());
                         if T::epsilon() * t * t > T::one() {
                             for j in i..=nn {
-                                A.div_element_mut(j, na, t);
-                                A.div_element_mut(j, nn, t);
+                                A.div_element_mut((j, na), t);
+                                A.div_element_mut((j, nn), t);
                             }
                         }
                     }
@@ -774,31 +758,31 @@ fn hqr2<T: RealNumber, M: BaseMatrix<T>>(A: &mut M, V: &mut M, d: &mut [T], e: &
             for i in 0..n {
                 z = T::zero();
                 for k in 0..=j {
-                    z += V.get(i, k) * A.get(k, j);
+                    z += *V.get((i, k)) * *A.get((k, j));
                 }
-                V.set(i, j, z);
+                V.set((i, j), z);
             }
         }
     }
 }
 
-fn balbak<T: RealNumber, M: BaseMatrix<T>>(V: &mut M, scale: &[T]) {
+fn balbak<T: Number + RealNumber, M: Array2<T>>(V: &mut M, scale: &[T]) {
     let (n, _) = V.shape();
     for (i, scale_i) in scale.iter().enumerate().take(n) {
         for j in 0..n {
-            V.mul_element_mut(i, j, *scale_i);
+            V.mul_element_mut((i, j), *scale_i);
         }
     }
 }
 
-fn sort<T: RealNumber, M: BaseMatrix<T>>(d: &mut [T], e: &mut [T], V: &mut M) {
+fn sort<T: Number + RealNumber, M: Array2<T>>(d: &mut [T], e: &mut [T], V: &mut M) {
     let n = d.len();
     let mut temp = vec![T::zero(); n];
     for j in 1..n {
         let real = d[j];
         let img = e[j];
         for (k, temp_k) in temp.iter_mut().enumerate().take(n) {
-            *temp_k = V.get(k, j);
+            *temp_k = *V.get((k, j));
         }
         let mut i = j as i32 - 1;
         while i >= 0 {
@@ -808,14 +792,14 @@ fn sort<T: RealNumber, M: BaseMatrix<T>>(d: &mut [T], e: &mut [T], V: &mut M) {
             d[i as usize + 1] = d[i as usize];
             e[i as usize + 1] = e[i as usize];
             for k in 0..n {
-                V.set(k, i as usize + 1, V.get(k, i as usize));
+                V.set((k, i as usize + 1), *V.get((k, i as usize)));
             }
             i -= 1;
         }
-        d[(i + 1) as usize] = real;
-        e[(i + 1) as usize] = img;
+        d[i as usize + 1] = real;
+        e[i as usize + 1] = img;
         for (k, temp_k) in temp.iter().enumerate().take(n) {
-            V.set(k, (i + 1) as usize, *temp_k);
+            V.set((k, i as usize + 1), *temp_k);
         }
     }
 }
@@ -823,8 +807,13 @@ fn sort<T: RealNumber, M: BaseMatrix<T>>(d: &mut [T], e: &mut [T], V: &mut M) {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_symmetric() {
         let A = DenseMatrix::from_2d_array(&[
@@ -843,7 +832,11 @@ mod tests {
 
         let evd = A.evd(true).unwrap();
 
-        assert!(eigen_vectors.abs().approximate_eq(&evd.V.abs(), 1e-4));
+        assert!(relative_eq!(
+            eigen_vectors.abs(),
+            evd.V.abs(),
+            epsilon = 1e-4
+        ));
         for i in 0..eigen_values.len() {
             assert!((eigen_values[i] - evd.d[i]).abs() < 1e-4);
         }
@@ -851,7 +844,10 @@ mod tests {
             assert!((0f64 - evd.e[i]).abs() < std::f64::EPSILON);
         }
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_asymmetric() {
         let A = DenseMatrix::from_2d_array(&[
@@ -870,7 +866,11 @@ mod tests {
 
         let evd = A.evd(false).unwrap();
 
-        assert!(eigen_vectors.abs().approximate_eq(&evd.V.abs(), 1e-4));
+        assert!(relative_eq!(
+            eigen_vectors.abs(),
+            evd.V.abs(),
+            epsilon = 1e-4
+        ));
         for i in 0..eigen_values.len() {
             assert!((eigen_values[i] - evd.d[i]).abs() < 1e-4);
         }
@@ -878,7 +878,10 @@ mod tests {
             assert!((0f64 - evd.e[i]).abs() < std::f64::EPSILON);
         }
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_complex() {
         let A = DenseMatrix::from_2d_array(&[
@@ -900,7 +903,11 @@ mod tests {
 
         let evd = A.evd(false).unwrap();
 
-        assert!(eigen_vectors.abs().approximate_eq(&evd.V.abs(), 1e-4));
+        assert!(relative_eq!(
+            eigen_vectors.abs(),
+            evd.V.abs(),
+            epsilon = 1e-4
+        ));
         for i in 0..eigen_values_d.len() {
             assert!((eigen_values_d[i] - evd.d[i]).abs() < 1e-4);
         }

src/linalg/traits/high_order.rs

@@ -1,15 +1,16 @@
 //! In this module you will find composite of matrix operations that are used elsewhere
 //! for improved efficiency.
 
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
 
 /// High order matrix operations.
-pub trait HighOrderOperations<T: RealNumber>: BaseMatrix<T> {
+pub trait HighOrderOperations<T: Number>: Array2<T> {
     /// Y = AB
     /// ```
-    /// use smartcore::linalg::naive::dense_matrix::*;
-    /// use smartcore::linalg::high_order::HighOrderOperations;
+    /// use smartcore::linalg::basic::matrix::*;
+    /// use smartcore::linalg::traits::high_order::HighOrderOperations;
+    /// use smartcore::linalg::basic::arrays::Array2;
     ///
     /// let a = DenseMatrix::from_2d_array(&[&[1., 2.], &[3., 4.], &[5., 6.]]);
     /// let b = DenseMatrix::from_2d_array(&[&[5., 6.], &[7., 8.], &[9., 10.]]);
@@ -26,3 +27,7 @@ pub trait HighOrderOperations<T: RealNumber>: BaseMatrix<T> {
         }
     }
 }
+
+mod tests {
+    /* TODO: Add tests  */
+}

src/linalg/traits/lu.rs

@@ -11,8 +11,8 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::lu::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::traits::lu::*;
 //!
 //! let A = DenseMatrix::from_2d_array(&[
 //!                  &[1., 2., 3.],
@@ -38,26 +38,27 @@ use std::fmt::Debug;
 use std::marker::PhantomData;
 
 use crate::error::Failed;
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
-
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 #[derive(Debug, Clone)]
 /// Result of LU decomposition.
-pub struct LU<T: RealNumber, M: BaseMatrix<T>> {
+pub struct LU<T: Number + RealNumber, M: Array2<T>> {
     LU: M,
     pivot: Vec<usize>,
-    _pivot_sign: i8,
+    #[allow(dead_code)]
+    pivot_sign: i8,
     singular: bool,
     phantom: PhantomData<T>,
 }
 
-impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
-    pub(crate) fn new(LU: M, pivot: Vec<usize>, _pivot_sign: i8) -> LU<T, M> {
+impl<T: Number + RealNumber, M: Array2<T>> LU<T, M> {
+    pub(crate) fn new(LU: M, pivot: Vec<usize>, pivot_sign: i8) -> LU<T, M> {
         let (_, n) = LU.shape();
 
         let mut singular = false;
         for j in 0..n {
-            if LU.get(j, j) == T::zero() {
+            if LU.get((j, j)) == &T::zero() {
                 singular = true;
                 break;
             }
@@ -66,7 +67,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
         LU {
             LU,
             pivot,
-            _pivot_sign,
+            pivot_sign,
             singular,
             phantom: PhantomData,
         }
@@ -80,9 +81,9 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
         for i in 0..n_rows {
             for j in 0..n_cols {
                 match i.cmp(&j) {
-                    Ordering::Greater => L.set(i, j, self.LU.get(i, j)),
-                    Ordering::Equal => L.set(i, j, T::one()),
-                    Ordering::Less => L.set(i, j, T::zero()),
+                    Ordering::Greater => L.set((i, j), *self.LU.get((i, j))),
+                    Ordering::Equal => L.set((i, j), T::one()),
+                    Ordering::Less => L.set((i, j), T::zero()),
                 }
             }
         }
@@ -98,9 +99,9 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
         for i in 0..n_rows {
             for j in 0..n_cols {
                 if i <= j {
-                    U.set(i, j, self.LU.get(i, j));
+                    U.set((i, j), *self.LU.get((i, j)));
                 } else {
-                    U.set(i, j, T::zero());
+                    U.set((i, j), T::zero());
                 }
             }
         }
@@ -114,7 +115,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
         let mut piv = M::zeros(n, n);
 
         for i in 0..n {
-            piv.set(i, self.pivot[i], T::one());
+            piv.set((i, self.pivot[i]), T::one());
         }
 
         piv
@@ -131,7 +132,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
         let mut inv = M::zeros(n, n);
 
         for i in 0..n {
-            inv.set(i, i, T::one());
+            inv.set((i, i), T::one());
         }
 
         self.solve(inv)
@@ -156,33 +157,33 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
 
         for j in 0..b_n {
             for i in 0..m {
-                X.set(i, j, b.get(self.pivot[i], j));
+                X.set((i, j), *b.get((self.pivot[i], j)));
             }
         }
 
         for k in 0..n {
             for i in k + 1..n {
                 for j in 0..b_n {
-                    X.sub_element_mut(i, j, X.get(k, j) * self.LU.get(i, k));
+                    X.sub_element_mut((i, j), *X.get((k, j)) * *self.LU.get((i, k)));
                 }
             }
         }
 
         for k in (0..n).rev() {
             for j in 0..b_n {
-                X.div_element_mut(k, j, self.LU.get(k, k));
+                X.div_element_mut((k, j), *self.LU.get((k, k)));
             }
 
             for i in 0..k {
                 for j in 0..b_n {
-                    X.sub_element_mut(i, j, X.get(k, j) * self.LU.get(i, k));
+                    X.sub_element_mut((i, j), *X.get((k, j)) * *self.LU.get((i, k)));
                 }
             }
         }
 
         for j in 0..b_n {
             for i in 0..m {
-                b.set(i, j, X.get(i, j));
+                b.set((i, j), *X.get((i, j)));
             }
         }
 
@@ -191,7 +192,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> LU<T, M> {
 }
 
 /// Trait that implements LU decomposition routine for any matrix.
-pub trait LUDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
+pub trait LUDecomposable<T: Number + RealNumber>: Array2<T> {
     /// Compute the LU decomposition of a square matrix.
     fn lu(&self) -> Result<LU<T, Self>, Failed> {
         self.clone().lu_mut()
@@ -209,18 +210,18 @@ pub trait LUDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 
         for j in 0..n {
             for (i, LUcolj_i) in LUcolj.iter_mut().enumerate().take(m) {
-                *LUcolj_i = self.get(i, j);
+                *LUcolj_i = *self.get((i, j));
             }
 
             for i in 0..m {
                 let kmax = usize::min(i, j);
                 let mut s = T::zero();
                 for (k, LUcolj_k) in LUcolj.iter().enumerate().take(kmax) {
-                    s += self.get(i, k) * (*LUcolj_k);
+                    s += *self.get((i, k)) * (*LUcolj_k);
                 }
 
                 LUcolj[i] -= s;
-                self.set(i, j, LUcolj[i]);
+                self.set((i, j), LUcolj[i]);
             }
 
             let mut p = j;
@@ -231,17 +232,15 @@ pub trait LUDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             }
             if p != j {
                 for k in 0..n {
-                    let t = self.get(p, k);
-                    self.set(p, k, self.get(j, k));
-                    self.set(j, k, t);
+                    self.swap((p, k), (j, k));
                 }
                 piv.swap(p, j);
                 pivsign = -pivsign;
             }
 
-            if j < m && self.get(j, j) != T::zero() {
+            if j < m && self.get((j, j)) != &T::zero() {
                 for i in j + 1..m {
-                    self.div_element_mut(i, j, self.get(j, j));
+                    self.div_element_mut((i, j), *self.get((j, j)));
                 }
             }
         }
@@ -258,9 +257,13 @@ pub trait LUDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose() {
         let a = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[0., 1., 5.], &[5., 6., 0.]]);
@@ -271,17 +274,20 @@ mod tests {
         let expected_pivot =
             DenseMatrix::from_2d_array(&[&[0., 0., 1.], &[0., 1., 0.], &[1., 0., 0.]]);
         let lu = a.lu().unwrap();
-        assert!(lu.L().approximate_eq(&expected_L, 1e-4));
-        assert!(lu.U().approximate_eq(&expected_U, 1e-4));
-        assert!(lu.pivot().approximate_eq(&expected_pivot, 1e-4));
+        assert!(relative_eq!(lu.L(), expected_L, epsilon = 1e-4));
+        assert!(relative_eq!(lu.U(), expected_U, epsilon = 1e-4));
+        assert!(relative_eq!(lu.pivot(), expected_pivot, epsilon = 1e-4));
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn inverse() {
         let a = DenseMatrix::from_2d_array(&[&[1., 2., 3.], &[0., 1., 5.], &[5., 6., 0.]]);
         let expected =
             DenseMatrix::from_2d_array(&[&[-6.0, 3.6, 1.4], &[5.0, -3.0, -1.0], &[-1.0, 0.8, 0.2]]);
         let a_inv = a.lu().and_then(|lu| lu.inverse()).unwrap();
-        assert!(a_inv.approximate_eq(&expected, 1e-4));
+        assert!(relative_eq!(a_inv, expected, epsilon = 1e-4));
     }
 }

src/linalg/traits/mod.rs

@@ -0,0 +1,15 @@
+#![allow(clippy::wrong_self_convention)]
+
+pub mod cholesky;
+/// The matrix is represented in terms of its eigenvalues and eigenvectors.
+pub mod evd;
+pub mod high_order;
+/// Factors a matrix as the product of a lower triangular matrix and an upper triangular matrix.
+pub mod lu;
+
+/// QR factorization that factors a matrix into a product of an orthogonal matrix and an upper triangular matrix.
+pub mod qr;
+/// statistacal tools for DenseMatrix
+pub mod stats;
+/// Singular value decomposition.
+pub mod svd;

src/linalg/traits/qr.rs

@@ -6,8 +6,8 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::qr::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::traits::qr::*;
 //!
 //! let A = DenseMatrix::from_2d_array(&[
 //!                 &[0.9, 0.4, 0.7],
@@ -28,20 +28,22 @@
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 #![allow(non_snake_case)]
 
-use crate::error::Failed;
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
 use std::fmt::Debug;
 
+use crate::error::Failed;
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
+
 #[derive(Debug, Clone)]
 /// Results of QR decomposition.
-pub struct QR<T: RealNumber, M: BaseMatrix<T>> {
+pub struct QR<T: Number + RealNumber, M: Array2<T>> {
     QR: M,
     tau: Vec<T>,
     singular: bool,
 }
 
-impl<T: RealNumber, M: BaseMatrix<T>> QR<T, M> {
+impl<T: Number + RealNumber, M: Array2<T>> QR<T, M> {
     pub(crate) fn new(QR: M, tau: Vec<T>) -> QR<T, M> {
         let mut singular = false;
         for tau_elem in tau.iter() {
@@ -59,9 +61,9 @@ impl<T: RealNumber, M: BaseMatrix<T>> QR<T, M> {
         let (_, n) = self.QR.shape();
         let mut R = M::zeros(n, n);
         for i in 0..n {
-            R.set(i, i, self.tau[i]);
+            R.set((i, i), self.tau[i]);
             for j in i + 1..n {
-                R.set(i, j, self.QR.get(i, j));
+                R.set((i, j), *self.QR.get((i, j)));
             }
         }
         R
@@ -73,16 +75,16 @@ impl<T: RealNumber, M: BaseMatrix<T>> QR<T, M> {
         let mut Q = M::zeros(m, n);
         let mut k = n - 1;
         loop {
-            Q.set(k, k, T::one());
+            Q.set((k, k), T::one());
             for j in k..n {
-                if self.QR.get(k, k) != T::zero() {
+                if self.QR.get((k, k)) != &T::zero() {
                     let mut s = T::zero();
                     for i in k..m {
-                        s += self.QR.get(i, k) * Q.get(i, j);
+                        s += *self.QR.get((i, k)) * *Q.get((i, j));
                     }
-                    s = -s / self.QR.get(k, k);
+                    s = -s / *self.QR.get((k, k));
                     for i in k..m {
-                        Q.add_element_mut(i, j, s * self.QR.get(i, k));
+                        Q.add_element_mut((i, j), s * *self.QR.get((i, k)));
                     }
                 }
             }
@@ -114,23 +116,23 @@ impl<T: RealNumber, M: BaseMatrix<T>> QR<T, M> {
             for j in 0..b_ncols {
                 let mut s = T::zero();
                 for i in k..m {
-                    s += self.QR.get(i, k) * b.get(i, j);
+                    s += *self.QR.get((i, k)) * *b.get((i, j));
                 }
-                s = -s / self.QR.get(k, k);
+                s = -s / *self.QR.get((k, k));
                 for i in k..m {
-                    b.add_element_mut(i, j, s * self.QR.get(i, k));
+                    b.add_element_mut((i, j), s * *self.QR.get((i, k)));
                 }
             }
         }
 
         for k in (0..n).rev() {
             for j in 0..b_ncols {
-                b.set(k, j, b.get(k, j) / self.tau[k]);
+                b.set((k, j), *b.get((k, j)) / self.tau[k]);
             }
 
             for i in 0..k {
                 for j in 0..b_ncols {
-                    b.sub_element_mut(i, j, b.get(k, j) * self.QR.get(i, k));
+                    b.sub_element_mut((i, j), *b.get((k, j)) * *self.QR.get((i, k)));
                 }
             }
         }
@@ -140,7 +142,7 @@ impl<T: RealNumber, M: BaseMatrix<T>> QR<T, M> {
 }
 
 /// Trait that implements QR decomposition routine for any matrix.
-pub trait QRDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
+pub trait QRDecomposable<T: Number + RealNumber>: Array2<T> {
     /// Compute the QR decomposition of a matrix.
     fn qr(&self) -> Result<QR<T, Self>, Failed> {
         self.clone().qr_mut()
@@ -156,26 +158,26 @@ pub trait QRDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
         for (k, r_diagonal_k) in r_diagonal.iter_mut().enumerate().take(n) {
             let mut nrm = T::zero();
             for i in k..m {
-                nrm = nrm.hypot(self.get(i, k));
+                nrm = nrm.hypot(*self.get((i, k)));
             }
 
             if nrm.abs() > T::epsilon() {
-                if self.get(k, k) < T::zero() {
+                if self.get((k, k)) < &T::zero() {
                     nrm = -nrm;
                 }
                 for i in k..m {
-                    self.div_element_mut(i, k, nrm);
+                    self.div_element_mut((i, k), nrm);
                 }
-                self.add_element_mut(k, k, T::one());
+                self.add_element_mut((k, k), T::one());
 
                 for j in k + 1..n {
                     let mut s = T::zero();
                     for i in k..m {
-                        s += self.get(i, k) * self.get(i, j);
+                        s += *self.get((i, k)) * *self.get((i, j));
                     }
-                    s = -s / self.get(k, k);
+                    s = -s / *self.get((k, k));
                     for i in k..m {
-                        self.add_element_mut(i, j, s * self.get(i, k));
+                        self.add_element_mut((i, j), s * *self.get((i, k)));
                     }
                 }
             }
@@ -194,8 +196,12 @@ pub trait QRDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose() {
         let a = DenseMatrix::from_2d_array(&[&[0.9, 0.4, 0.7], &[0.4, 0.5, 0.3], &[0.7, 0.3, 0.8]]);
@@ -210,11 +216,14 @@ mod tests {
             &[0.0, 0.0, -0.1999],
         ]);
         let qr = a.qr().unwrap();
-        assert!(qr.Q().abs().approximate_eq(&q.abs(), 1e-4));
-        assert!(qr.R().abs().approximate_eq(&r.abs(), 1e-4));
+        assert!(relative_eq!(qr.Q().abs(), q.abs(), epsilon = 1e-4));
+        assert!(relative_eq!(qr.R().abs(), r.abs(), epsilon = 1e-4));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn qr_solve_mut() {
         let a = DenseMatrix::from_2d_array(&[&[0.9, 0.4, 0.7], &[0.4, 0.5, 0.3], &[0.7, 0.3, 0.8]]);
@@ -225,6 +234,6 @@ mod tests {
             &[0.4729730, 0.6621622],
         ]);
         let w = a.qr_solve_mut(b).unwrap();
-        assert!(w.approximate_eq(&expected_w, 1e-2));
+        assert!(relative_eq!(w, expected_w, epsilon = 1e-2));
     }
 }

src/linalg/traits/stats.rs

@@ -0,0 +1,294 @@
+//! # Various Statistical Methods
+//!
+//! This module provides reference implementations for  various statistical functions.
+//! Concrete implementations of the `BaseMatrix` trait are free to override these methods for better performance.
+
+//! This methods shall be used when dealing with `DenseMatrix`. Use the ones in `linalg::arrays` for `Array` types.
+
+use crate::linalg::basic::arrays::{Array2, ArrayView2, MutArrayView2};
+use crate::numbers::realnum::RealNumber;
+
+/// Defines baseline implementations for various statistical functions
+pub trait MatrixStats<T: RealNumber>: ArrayView2<T> + Array2<T> {
+    /// Computes the arithmetic mean along the specified axis.
+    fn mean(&self, axis: u8) -> Vec<T> {
+        let (n, _m) = match axis {
+            0 => {
+                let (n, m) = self.shape();
+                (m, n)
+            }
+            _ => self.shape(),
+        };
+
+        let mut x: Vec<T> = vec![T::zero(); n];
+
+        for (i, x_i) in x.iter_mut().enumerate().take(n) {
+            let vec = match axis {
+                0 => self.get_col(i).iterator(0).copied().collect::<Vec<T>>(),
+                _ => self.get_row(i).iterator(0).copied().collect::<Vec<T>>(),
+            };
+            *x_i = Self::_mean_of_vector(&vec[..]);
+        }
+        x
+    }
+
+    /// Computes variance along the specified axis.
+    fn var(&self, axis: u8) -> Vec<T> {
+        let (n, _m) = match axis {
+            0 => {
+                let (n, m) = self.shape();
+                (m, n)
+            }
+            _ => self.shape(),
+        };
+
+        let mut x: Vec<T> = vec![T::zero(); n];
+
+        for (i, x_i) in x.iter_mut().enumerate().take(n) {
+            let vec = match axis {
+                0 => self.get_col(i).iterator(0).copied().collect::<Vec<T>>(),
+                _ => self.get_row(i).iterator(0).copied().collect::<Vec<T>>(),
+            };
+            *x_i = Self::_var_of_vec(&vec[..], Option::None);
+        }
+
+        x
+    }
+
+    /// Computes the standard deviation along the specified axis.
+    fn std(&self, axis: u8) -> Vec<T> {
+        let mut x = Self::var(self, axis);
+
+        let n = match axis {
+            0 => self.shape().1,
+            _ => self.shape().0,
+        };
+
+        for x_i in x.iter_mut().take(n) {
+            *x_i = x_i.sqrt();
+        }
+
+        x
+    }
+
+    /// <http://en.wikipedia.org/wiki/Arithmetic_mean>
+    /// Taken from `statistical`
+    /// The MIT License (MIT)
+    /// Copyright (c) 2015 Jeff Belgum
+    fn _mean_of_vector(v: &[T]) -> T {
+        let len = num::cast(v.len()).unwrap();
+        v.iter().fold(T::zero(), |acc: T, elem| acc + *elem) / len
+    }
+
+    /// Taken from statistical
+    /// The MIT License (MIT)
+    /// Copyright (c) 2015 Jeff Belgum
+    fn _sum_square_deviations_vec(v: &[T], c: Option<T>) -> T {
+        let c = match c {
+            Some(c) => c,
+            None => Self::_mean_of_vector(v),
+        };
+
+        let sum = v
+            .iter()
+            .map(|x| (*x - c) * (*x - c))
+            .fold(T::zero(), |acc, elem| acc + elem);
+        assert!(sum >= T::zero(), "negative sum of square root deviations");
+        sum
+    }
+
+    /// <http://en.wikipedia.org/wiki/Variance#Sample_variance>
+    /// Taken from statistical
+    /// The MIT License (MIT)
+    /// Copyright (c) 2015 Jeff Belgum
+    fn _var_of_vec(v: &[T], xbar: Option<T>) -> T {
+        assert!(v.len() > 1, "variance requires at least two data points");
+        let len: T = num::cast(v.len()).unwrap();
+        let sum = Self::_sum_square_deviations_vec(v, xbar);
+        sum / len
+    }
+
+    /// standardize values by removing the mean and scaling to unit variance
+    fn standard_scale_mut(&mut self, mean: &[T], std: &[T], axis: u8) {
+        let (n, m) = match axis {
+            0 => {
+                let (n, m) = self.shape();
+                (m, n)
+            }
+            _ => self.shape(),
+        };
+
+        for i in 0..n {
+            for j in 0..m {
+                match axis {
+                    0 => self.set((j, i), (*self.get((j, i)) - mean[i]) / std[i]),
+                    _ => self.set((i, j), (*self.get((i, j)) - mean[i]) / std[i]),
+                }
+            }
+        }
+    }
+}
+
+//TODO: this is processing. Should have its own "processing.rs" module
+/// Defines baseline implementations for various matrix processing functions
+pub trait MatrixPreprocessing<T: RealNumber>: MutArrayView2<T> + Clone {
+    /// Each element of the matrix greater than the threshold becomes 1, while values less than or equal to the threshold become 0
+    /// ```rust
+    /// use smartcore::linalg::basic::matrix::DenseMatrix;
+    /// use smartcore::linalg::traits::stats::MatrixPreprocessing;
+    /// let mut a = DenseMatrix::from_2d_array(&[&[0., 2., 3.], &[-5., -6., -7.]]);
+    /// let expected = DenseMatrix::from_2d_array(&[&[0., 1., 1.],&[0., 0., 0.]]);
+    /// a.binarize_mut(0.);
+    ///
+    /// assert_eq!(a, expected);
+    /// ```
+
+    fn binarize_mut(&mut self, threshold: T) {
+        let (nrows, ncols) = self.shape();
+        for row in 0..nrows {
+            for col in 0..ncols {
+                if *self.get((row, col)) > threshold {
+                    self.set((row, col), T::one());
+                } else {
+                    self.set((row, col), T::zero());
+                }
+            }
+        }
+    }
+    /// Returns new matrix where elements are binarized according to a given threshold.
+    /// ```rust
+    /// use smartcore::linalg::basic::matrix::DenseMatrix;
+    /// use smartcore::linalg::traits::stats::MatrixPreprocessing;
+    /// let a = DenseMatrix::from_2d_array(&[&[0., 2., 3.], &[-5., -6., -7.]]);
+    /// let expected = DenseMatrix::from_2d_array(&[&[0., 1., 1.],&[0., 0., 0.]]);
+    ///
+    /// assert_eq!(a.binarize(0.), expected);
+    /// ```
+    fn binarize(self, threshold: T) -> Self
+    where
+        Self: Sized,
+    {
+        let mut m = self;
+        m.binarize_mut(threshold);
+        m
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use crate::linalg::basic::arrays::Array1;
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use crate::linalg::traits::stats::MatrixStats;
+
+    #[test]
+    fn test_mean() {
+        let m = DenseMatrix::from_2d_array(&[
+            &[1., 2., 3., 1., 2.],
+            &[4., 5., 6., 3., 4.],
+            &[7., 8., 9., 5., 6.],
+        ]);
+        let expected_0 = vec![4., 5., 6., 3., 4.];
+        let expected_1 = vec![1.8, 4.4, 7.];
+
+        assert_eq!(m.mean(0), expected_0);
+        assert_eq!(m.mean(1), expected_1);
+    }
+
+    #[test]
+    fn test_var() {
+        let m = DenseMatrix::from_2d_array(&[&[1., 2., 3., 4.], &[5., 6., 7., 8.]]);
+        let expected_0 = vec![4., 4., 4., 4.];
+        let expected_1 = vec![1.25, 1.25];
+
+        assert!(m.var(0).approximate_eq(&expected_0, 1e-6));
+        assert!(m.var(1).approximate_eq(&expected_1, 1e-6));
+        assert_eq!(m.mean(0), vec![3.0, 4.0, 5.0, 6.0]);
+        assert_eq!(m.mean(1), vec![2.5, 6.5]);
+    }
+
+    #[test]
+    fn test_var_other() {
+        let m = DenseMatrix::from_2d_array(&[
+            &[0.0, 0.25, 0.25, 1.25, 1.5, 1.75, 2.75, 3.25],
+            &[0.0, 0.25, 0.25, 1.25, 1.5, 1.75, 2.75, 3.25],
+        ]);
+        let expected_0 = vec![0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0];
+        let expected_1 = vec![1.25, 1.25];
+
+        assert!(m.var(0).approximate_eq(&expected_0, std::f64::EPSILON));
+        assert!(m.var(1).approximate_eq(&expected_1, std::f64::EPSILON));
+        assert_eq!(
+            m.mean(0),
+            vec![0.0, 0.25, 0.25, 1.25, 1.5, 1.75, 2.75, 3.25]
+        );
+        assert_eq!(m.mean(1), vec![1.375, 1.375]);
+    }
+
+    #[test]
+    fn test_std() {
+        let m = DenseMatrix::from_2d_array(&[
+            &[1., 2., 3., 1., 2.],
+            &[4., 5., 6., 3., 4.],
+            &[7., 8., 9., 5., 6.],
+        ]);
+        let expected_0 = vec![
+            2.449489742783178,
+            2.449489742783178,
+            2.449489742783178,
+            1.632993161855452,
+            1.632993161855452,
+        ];
+        let expected_1 = vec![0.7483314773547883, 1.019803902718557, 1.4142135623730951];
+
+        println!("{:?}", m.var(0));
+
+        assert!(m.std(0).approximate_eq(&expected_0, f64::EPSILON));
+        assert!(m.std(1).approximate_eq(&expected_1, f64::EPSILON));
+        assert_eq!(m.mean(0), vec![4.0, 5.0, 6.0, 3.0, 4.0]);
+        assert_eq!(m.mean(1), vec![1.8, 4.4, 7.0]);
+    }
+
+    #[test]
+    fn test_scale() {
+        let m: DenseMatrix<f64> =
+            DenseMatrix::from_2d_array(&[&[1., 2., 3., 4.], &[5., 6., 7., 8.]]);
+
+        let expected_0: DenseMatrix<f64> =
+            DenseMatrix::from_2d_array(&[&[-1., -1., -1., -1.], &[1., 1., 1., 1.]]);
+        let expected_1: DenseMatrix<f64> = DenseMatrix::from_2d_array(&[
+            &[
+                -1.3416407864998738,
+                -0.4472135954999579,
+                0.4472135954999579,
+                1.3416407864998738,
+            ],
+            &[
+                -1.3416407864998738,
+                -0.4472135954999579,
+                0.4472135954999579,
+                1.3416407864998738,
+            ],
+        ]);
+
+        assert_eq!(m.mean(0), vec![3.0, 4.0, 5.0, 6.0]);
+        assert_eq!(m.mean(1), vec![2.5, 6.5]);
+
+        assert_eq!(m.var(0), vec![4., 4., 4., 4.]);
+        assert_eq!(m.var(1), vec![1.25, 1.25]);
+
+        assert_eq!(m.std(0), vec![2., 2., 2., 2.]);
+        assert_eq!(m.std(1), vec![1.118033988749895, 1.118033988749895]);
+
+        {
+            let mut m = m.clone();
+            m.standard_scale_mut(&m.mean(0), &m.std(0), 0);
+            assert_eq!(&m, &expected_0);
+        }
+
+        {
+            let mut m = m.clone();
+            m.standard_scale_mut(&m.mean(1), &m.std(1), 1);
+            assert_eq!(&m, &expected_1);
+        }
+    }
+}

src/linalg/traits/svd.rs

@@ -10,8 +10,8 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::linalg::svd::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::traits::svd::*;
 //!
 //! let A = DenseMatrix::from_2d_array(&[
 //!                 &[0.9, 0.4, 0.7],
@@ -34,32 +34,35 @@
 #![allow(non_snake_case)]
 
 use crate::error::Failed;
-use crate::linalg::BaseMatrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 use std::fmt::Debug;
 
 /// Results of SVD decomposition
 #[derive(Debug, Clone)]
-pub struct SVD<T: RealNumber, M: SVDDecomposableMatrix<T>> {
+pub struct SVD<T: Number + RealNumber, M: SVDDecomposable<T>> {
     /// Left-singular vectors of _A_
     pub U: M,
     /// Right-singular vectors of _A_
     pub V: M,
     /// Singular values of the original matrix
     pub s: Vec<T>,
-    _full: bool,
+    ///
     m: usize,
+    ///
     n: usize,
+    ///
     tol: T,
 }
 
-impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
+impl<T: Number + RealNumber, M: SVDDecomposable<T>> SVD<T, M> {
     /// Diagonal matrix with singular values
     pub fn S(&self) -> M {
         let mut s = M::zeros(self.U.shape().1, self.V.shape().0);
 
         for i in 0..self.s.len() {
-            s.set(i, i, self.s[i]);
+            s.set((i, i), self.s[i]);
         }
 
         s
@@ -67,7 +70,7 @@ impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
 }
 
 /// Trait that implements SVD decomposition routine for any matrix.
-pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
+pub trait SVDDecomposable<T: Number + RealNumber>: Array2<T> {
     /// Solves Ax = b. Overrides original matrix in the process.
     fn svd_solve_mut(self, b: Self) -> Result<Self, Failed> {
         self.svd_mut().and_then(|svd| svd.solve(b))
@@ -106,31 +109,31 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 
             if i < m {
                 for k in i..m {
-                    scale += U.get(k, i).abs();
+                    scale += U.get((k, i)).abs();
                 }
 
                 if scale.abs() > T::epsilon() {
                     for k in i..m {
-                        U.div_element_mut(k, i, scale);
-                        s += U.get(k, i) * U.get(k, i);
+                        U.div_element_mut((k, i), scale);
+                        s += *U.get((k, i)) * *U.get((k, i));
                     }
 
-                    let mut f = U.get(i, i);
-                    g = -RealNumber::copysign(s.sqrt(), f);
+                    let mut f = *U.get((i, i));
+                    g = -<T as RealNumber>::copysign(s.sqrt(), f);
                     let h = f * g - s;
-                    U.set(i, i, f - g);
+                    U.set((i, i), f - g);
                     for j in l - 1..n {
                         s = T::zero();
                         for k in i..m {
-                            s += U.get(k, i) * U.get(k, j);
+                            s += *U.get((k, i)) * *U.get((k, j));
                         }
                         f = s / h;
                         for k in i..m {
-                            U.add_element_mut(k, j, f * U.get(k, i));
+                            U.add_element_mut((k, j), f * *U.get((k, i)));
                         }
                     }
                     for k in i..m {
-                        U.mul_element_mut(k, i, scale);
+                        U.mul_element_mut((k, i), scale);
                     }
                 }
             }
@@ -142,37 +145,37 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
 
             if i < m && i + 1 != n {
                 for k in l - 1..n {
-                    scale += U.get(i, k).abs();
+                    scale += U.get((i, k)).abs();
                 }
 
                 if scale.abs() > T::epsilon() {
                     for k in l - 1..n {
-                        U.div_element_mut(i, k, scale);
-                        s += U.get(i, k) * U.get(i, k);
+                        U.div_element_mut((i, k), scale);
+                        s += *U.get((i, k)) * *U.get((i, k));
                     }
 
-                    let f = U.get(i, l - 1);
-                    g = -RealNumber::copysign(s.sqrt(), f);
+                    let f = *U.get((i, l - 1));
+                    g = -<T as RealNumber>::copysign(s.sqrt(), f);
                     let h = f * g - s;
-                    U.set(i, l - 1, f - g);
+                    U.set((i, l - 1), f - g);
 
                     for (k, rv1_k) in rv1.iter_mut().enumerate().take(n).skip(l - 1) {
-                        *rv1_k = U.get(i, k) / h;
+                        *rv1_k = *U.get((i, k)) / h;
                     }
 
                     for j in l - 1..m {
                         s = T::zero();
                         for k in l - 1..n {
-                            s += U.get(j, k) * U.get(i, k);
+                            s += *U.get((j, k)) * *U.get((i, k));
                         }
 
                         for (k, rv1_k) in rv1.iter().enumerate().take(n).skip(l - 1) {
-                            U.add_element_mut(j, k, s * (*rv1_k));
+                            U.add_element_mut((j, k), s * (*rv1_k));
                         }
                     }
 
                     for k in l - 1..n {
-                        U.mul_element_mut(i, k, scale);
+                        U.mul_element_mut((i, k), scale);
                     }
                 }
             }
@@ -184,24 +187,24 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             if i < n - 1 {
                 if g != T::zero() {
                     for j in l..n {
-                        v.set(j, i, (U.get(i, j) / U.get(i, l)) / g);
+                        v.set((j, i), (*U.get((i, j)) / *U.get((i, l))) / g);
                     }
                     for j in l..n {
                         let mut s = T::zero();
                         for k in l..n {
-                            s += U.get(i, k) * v.get(k, j);
+                            s += *U.get((i, k)) * *v.get((k, j));
                         }
                         for k in l..n {
-                            v.add_element_mut(k, j, s * v.get(k, i));
+                            v.add_element_mut((k, j), s * *v.get((k, i)));
                         }
                     }
                 }
                 for j in l..n {
-                    v.set(i, j, T::zero());
-                    v.set(j, i, T::zero());
+                    v.set((i, j), T::zero());
+                    v.set((j, i), T::zero());
                 }
             }
-            v.set(i, i, T::one());
+            v.set((i, i), T::one());
             g = rv1[i];
             l = i;
         }
@@ -210,7 +213,7 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             l = i + 1;
             g = w[i];
             for j in l..n {
-                U.set(i, j, T::zero());
+                U.set((i, j), T::zero());
             }
 
             if g.abs() > T::epsilon() {
@@ -218,23 +221,23 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                 for j in l..n {
                     let mut s = T::zero();
                     for k in l..m {
-                        s += U.get(k, i) * U.get(k, j);
+                        s += *U.get((k, i)) * *U.get((k, j));
                     }
-                    let f = (s / U.get(i, i)) * g;
+                    let f = (s / *U.get((i, i))) * g;
                     for k in i..m {
-                        U.add_element_mut(k, j, f * U.get(k, i));
+                        U.add_element_mut((k, j), f * *U.get((k, i)));
                     }
                 }
                 for j in i..m {
-                    U.mul_element_mut(j, i, g);
+                    U.mul_element_mut((j, i), g);
                 }
             } else {
                 for j in i..m {
-                    U.set(j, i, T::zero());
+                    U.set((j, i), T::zero());
                 }
             }
 
-            U.add_element_mut(i, i, T::one());
+            U.add_element_mut((i, i), T::one());
         }
 
         for k in (0..n).rev() {
@@ -269,10 +272,10 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                         c = g * h;
                         s = -f * h;
                         for j in 0..m {
-                            let y = U.get(j, nm);
-                            let z = U.get(j, i);
-                            U.set(j, nm, y * c + z * s);
-                            U.set(j, i, z * c - y * s);
+                            let y = *U.get((j, nm));
+                            let z = *U.get((j, i));
+                            U.set((j, nm), y * c + z * s);
+                            U.set((j, i), z * c - y * s);
                         }
                     }
                 }
@@ -282,7 +285,7 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                     if z < T::zero() {
                         w[k] = -z;
                         for j in 0..n {
-                            v.set(j, k, -v.get(j, k));
+                            v.set((j, k), -*v.get((j, k)));
                         }
                     }
                     break;
@@ -299,7 +302,8 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                 let mut h = rv1[k];
                 let mut f = ((y - z) * (y + z) + (g - h) * (g + h)) / (T::two() * h * y);
                 g = f.hypot(T::one());
-                f = ((x - z) * (x + z) + h * ((y / (f + RealNumber::copysign(g, f))) - h)) / x;
+                f = ((x - z) * (x + z) + h * ((y / (f + <T as RealNumber>::copysign(g, f))) - h))
+                    / x;
                 let mut c = T::one();
                 let mut s = T::one();
 
@@ -319,10 +323,10 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                     y *= c;
 
                     for jj in 0..n {
-                        x = v.get(jj, j);
-                        z = v.get(jj, i);
-                        v.set(jj, j, x * c + z * s);
-                        v.set(jj, i, z * c - x * s);
+                        x = *v.get((jj, j));
+                        z = *v.get((jj, i));
+                        v.set((jj, j), x * c + z * s);
+                        v.set((jj, i), z * c - x * s);
                     }
 
                     z = f.hypot(h);
@@ -336,10 +340,10 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                     f = c * g + s * y;
                     x = c * y - s * g;
                     for jj in 0..m {
-                        y = U.get(jj, j);
-                        z = U.get(jj, i);
-                        U.set(jj, j, y * c + z * s);
-                        U.set(jj, i, z * c - y * s);
+                        y = *U.get((jj, j));
+                        z = *U.get((jj, i));
+                        U.set((jj, j), y * c + z * s);
+                        U.set((jj, i), z * c - y * s);
                     }
                 }
 
@@ -366,19 +370,19 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
             for i in inc..n {
                 let sw = w[i];
                 for (k, su_k) in su.iter_mut().enumerate().take(m) {
-                    *su_k = U.get(k, i);
+                    *su_k = *U.get((k, i));
                 }
                 for (k, sv_k) in sv.iter_mut().enumerate().take(n) {
-                    *sv_k = v.get(k, i);
+                    *sv_k = *v.get((k, i));
                 }
                 let mut j = i;
                 while w[j - inc] < sw {
                     w[j] = w[j - inc];
                     for k in 0..m {
-                        U.set(k, j, U.get(k, j - inc));
+                        U.set((k, j), *U.get((k, j - inc)));
                     }
                     for k in 0..n {
-                        v.set(k, j, v.get(k, j - inc));
+                        v.set((k, j), *v.get((k, j - inc)));
                     }
                     j -= inc;
                     if j < inc {
@@ -387,10 +391,10 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
                 }
                 w[j] = sw;
                 for (k, su_k) in su.iter().enumerate().take(m) {
-                    U.set(k, j, *su_k);
+                    U.set((k, j), *su_k);
                 }
                 for (k, sv_k) in sv.iter().enumerate().take(n) {
-                    v.set(k, j, *sv_k);
+                    v.set((k, j), *sv_k);
                 }
             }
             if inc <= 1 {
@@ -401,21 +405,21 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
         for k in 0..n {
             let mut s = 0.;
             for i in 0..m {
-                if U.get(i, k) < T::zero() {
+                if U.get((i, k)) < &T::zero() {
                     s += 1.;
                 }
             }
             for j in 0..n {
-                if v.get(j, k) < T::zero() {
+                if v.get((j, k)) < &T::zero() {
                     s += 1.;
                 }
             }
             if s > (m + n) as f64 / 2. {
                 for i in 0..m {
-                    U.set(i, k, -U.get(i, k));
+                    U.set((i, k), -*U.get((i, k)));
                 }
                 for j in 0..n {
-                    v.set(j, k, -v.get(j, k));
+                    v.set((j, k), -*v.get((j, k)));
                 }
             }
         }
@@ -424,21 +428,12 @@ pub trait SVDDecomposableMatrix<T: RealNumber>: BaseMatrix<T> {
     }
 }
 
-impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
+impl<T: Number + RealNumber, M: SVDDecomposable<T>> SVD<T, M> {
     pub(crate) fn new(U: M, V: M, s: Vec<T>) -> SVD<T, M> {
         let m = U.shape().0;
         let n = V.shape().0;
-        let _full = s.len() == m.min(n);
         let tol = T::half() * (T::from(m + n).unwrap() + T::one()).sqrt() * s[0] * T::epsilon();
-        SVD {
-            U,
-            V,
-            s,
-            _full,
-            m,
-            n,
-            tol,
-        }
+        SVD { U, V, s, m, n, tol }
     }
 
     pub(crate) fn solve(&self, mut b: M) -> Result<M, Failed> {
@@ -458,7 +453,7 @@ impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
                 let mut r = T::zero();
                 if self.s[j] > self.tol {
                     for i in 0..self.m {
-                        r += self.U.get(i, j) * b.get(i, k);
+                        r += *self.U.get((i, j)) * *b.get((i, k));
                     }
                     r /= self.s[j];
                 }
@@ -468,9 +463,9 @@ impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
             for j in 0..self.n {
                 let mut r = T::zero();
                 for (jj, tmp_jj) in tmp.iter().enumerate().take(self.n) {
-                    r += self.V.get(j, jj) * (*tmp_jj);
+                    r += *self.V.get((j, jj)) * (*tmp_jj);
                 }
-                b.set(j, k, r);
+                b.set((j, k), r);
             }
         }
 
@@ -481,8 +476,13 @@ impl<T: RealNumber, M: SVDDecomposableMatrix<T>> SVD<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use approx::relative_eq;
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_symmetric() {
         let A = DenseMatrix::from_2d_array(&[
@@ -507,13 +507,16 @@ mod tests {
 
         let svd = A.svd().unwrap();
 
-        assert!(V.abs().approximate_eq(&svd.V.abs(), 1e-4));
-        assert!(U.abs().approximate_eq(&svd.U.abs(), 1e-4));
+        assert!(relative_eq!(V.abs(), svd.V.abs(), epsilon = 1e-4));
+        assert!(relative_eq!(U.abs(), svd.U.abs(), epsilon = 1e-4));
         for i in 0..s.len() {
             assert!((s[i] - svd.s[i]).abs() < 1e-4);
         }
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_asymmetric() {
         let A = DenseMatrix::from_2d_array(&[
@@ -708,13 +711,16 @@ mod tests {
 
         let svd = A.svd().unwrap();
 
-        assert!(V.abs().approximate_eq(&svd.V.abs(), 1e-4));
-        assert!(U.abs().approximate_eq(&svd.U.abs(), 1e-4));
+        assert!(relative_eq!(V.abs(), svd.V.abs(), epsilon = 1e-4));
+        assert!(relative_eq!(U.abs(), svd.U.abs(), epsilon = 1e-4));
         for i in 0..s.len() {
             assert!((s[i] - svd.s[i]).abs() < 1e-4);
         }
     }
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn solve() {
         let a = DenseMatrix::from_2d_array(&[&[0.9, 0.4, 0.7], &[0.4, 0.5, 0.3], &[0.7, 0.3, 0.8]]);
@@ -722,10 +728,13 @@ mod tests {
         let expected_w =
             DenseMatrix::from_2d_array(&[&[-0.20, -1.28], &[0.87, 2.22], &[0.47, 0.66]]);
         let w = a.svd_solve_mut(b).unwrap();
-        assert!(w.approximate_eq(&expected_w, 1e-2));
+        assert!(relative_eq!(w, expected_w, epsilon = 1e-2));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn decompose_restore() {
         let a = DenseMatrix::from_2d_array(&[&[1.0, 2.0, 3.0, 4.0], &[5.0, 6.0, 7.0, 8.0]]);
@@ -736,8 +745,6 @@ mod tests {
 
         let a_hat = u.matmul(s).matmul(&v.transpose());
 
-        for (a, a_hat) in a.iter().zip(a_hat.iter()) {
-            assert!((a - a_hat).abs() < 1e-3)
-        }
+        assert!(relative_eq!(a, a_hat, epsilon = 1e-3));
     }
 }

src/linear/bg_solver.rs

@@ -1,13 +1,42 @@
 //! This is a generic solver for Ax = b type of equation
 //!
+//! Example:
+//! ```
+//! use smartcore::linalg::basic::arrays::Array1;
+//! use smartcore::linalg::basic::arrays::Array2;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linear::bg_solver::*;
+//! use smartcore::numbers::floatnum::FloatNumber;
+//! use smartcore::linear::bg_solver::BiconjugateGradientSolver;
+//!
+//! pub struct BGSolver {}
+//! impl<'a, T: FloatNumber, X: Array2<T>> BiconjugateGradientSolver<'a, T, X> for BGSolver {}
+//!
+//! let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
+//! let b = vec![40., 51., 28.];
+//! let expected = vec![1.0, 2.0, 3.0];
+//! let mut x = Vec::zeros(3);
+//! let solver = BGSolver {};
+//! let err: f64 = solver.solve_mut(&a, &b, &mut x, 1e-6, 6).unwrap();
+//! ```
+//!
 //! for more information take a look at [this Wikipedia article](https://en.wikipedia.org/wiki/Biconjugate_gradient_method)
 //! and [this paper](https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf)
 use crate::error::Failed;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array, Array1, Array2, ArrayView1, MutArrayView1};
+use crate::numbers::floatnum::FloatNumber;
 
-pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
-    fn solve_mut(&self, a: &M, b: &M, x: &mut M, tol: T, max_iter: usize) -> Result<T, Failed> {
+///
+pub trait BiconjugateGradientSolver<'a, T: FloatNumber, X: Array2<T>> {
+    ///
+    fn solve_mut(
+        &self,
+        a: &'a X,
+        b: &Vec<T>,
+        x: &mut Vec<T>,
+        tol: T,
+        max_iter: usize,
+    ) -> Result<T, Failed> {
         if tol <= T::zero() {
             return Err(Failed::fit("tolerance shoud be > 0"));
         }
@@ -16,25 +45,25 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
             return Err(Failed::fit("maximum number of iterations should be > 0"));
         }
 
-        let (n, _) = b.shape();
+        let n = b.shape();
 
-        let mut r = M::zeros(n, 1);
-        let mut rr = M::zeros(n, 1);
-        let mut z = M::zeros(n, 1);
-        let mut zz = M::zeros(n, 1);
+        let mut r = Vec::zeros(n);
+        let mut rr = Vec::zeros(n);
+        let mut z = Vec::zeros(n);
+        let mut zz = Vec::zeros(n);
 
         self.mat_vec_mul(a, x, &mut r);
 
         for j in 0..n {
-            r.set(j, 0, b.get(j, 0) - r.get(j, 0));
-            rr.set(j, 0, r.get(j, 0));
+            r[j] = b[j] - r[j];
+            rr[j] = r[j];
         }
 
-        let bnrm = b.norm(T::two());
-        self.solve_preconditioner(a, &r, &mut z);
+        let bnrm = b.norm(2f64);
+        self.solve_preconditioner(a, &r[..], &mut z[..]);
 
-        let mut p = M::zeros(n, 1);
-        let mut pp = M::zeros(n, 1);
+        let mut p = Vec::zeros(n);
+        let mut pp = Vec::zeros(n);
         let mut bkden = T::zero();
         let mut err = T::zero();
 
@@ -43,35 +72,33 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
 
             self.solve_preconditioner(a, &rr, &mut zz);
             for j in 0..n {
-                bknum += z.get(j, 0) * rr.get(j, 0);
+                bknum += z[j] * rr[j];
             }
             if iter == 1 {
-                for j in 0..n {
-                    p.set(j, 0, z.get(j, 0));
-                    pp.set(j, 0, zz.get(j, 0));
-                }
+                p[..n].copy_from_slice(&z[..n]);
+                pp[..n].copy_from_slice(&zz[..n]);
             } else {
                 let bk = bknum / bkden;
                 for j in 0..n {
-                    p.set(j, 0, bk * p.get(j, 0) + z.get(j, 0));
-                    pp.set(j, 0, bk * pp.get(j, 0) + zz.get(j, 0));
+                    p[j] = bk * pp[j] + z[j];
+                    pp[j] = bk * pp[j] + zz[j];
                 }
             }
             bkden = bknum;
             self.mat_vec_mul(a, &p, &mut z);
             let mut akden = T::zero();
             for j in 0..n {
-                akden += z.get(j, 0) * pp.get(j, 0);
+                akden += z[j] * pp[j];
             }
             let ak = bknum / akden;
             self.mat_t_vec_mul(a, &pp, &mut zz);
             for j in 0..n {
-                x.set(j, 0, x.get(j, 0) + ak * p.get(j, 0));
-                r.set(j, 0, r.get(j, 0) - ak * z.get(j, 0));
-                rr.set(j, 0, rr.get(j, 0) - ak * zz.get(j, 0));
+                x[j] += ak * p[j];
+                r[j] -= ak * z[j];
+                rr[j] -= ak * zz[j];
             }
             self.solve_preconditioner(a, &r, &mut z);
-            err = r.norm(T::two()) / bnrm;
+            err = T::from_f64(r.norm(2f64) / bnrm).unwrap();
 
             if err <= tol {
                 break;
@@ -81,36 +108,38 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
         Ok(err)
     }
 
-    fn solve_preconditioner(&self, a: &M, b: &M, x: &mut M) {
+    ///
+    fn solve_preconditioner(&self, a: &'a X, b: &[T], x: &mut [T]) {
         let diag = Self::diag(a);
         let n = diag.len();
 
         for (i, diag_i) in diag.iter().enumerate().take(n) {
             if *diag_i != T::zero() {
-                x.set(i, 0, b.get(i, 0) / *diag_i);
+                x[i] = b[i] / *diag_i;
             } else {
-                x.set(i, 0, b.get(i, 0));
+                x[i] = b[i];
             }
         }
     }
 
-    // y = Ax
-    fn mat_vec_mul(&self, a: &M, x: &M, y: &mut M) {
-        y.copy_from(&a.matmul(x));
+    /// y = Ax
+    fn mat_vec_mul(&self, a: &X, x: &Vec<T>, y: &mut Vec<T>) {
+        y.copy_from(&x.xa(false, a));
     }
 
-    // y = Atx
-    fn mat_t_vec_mul(&self, a: &M, x: &M, y: &mut M) {
-        y.copy_from(&a.ab(true, x, false));
+    /// y = Atx
+    fn mat_t_vec_mul(&self, a: &X, x: &Vec<T>, y: &mut Vec<T>) {
+        y.copy_from(&x.xa(true, a));
     }
 
-    fn diag(a: &M) -> Vec<T> {
+    ///
+    fn diag(a: &X) -> Vec<T> {
         let (nrows, ncols) = a.shape();
         let n = nrows.min(ncols);
 
         let mut d = Vec::with_capacity(n);
         for i in 0..n {
-            d.push(a.get(i, i));
+            d.push(*a.get((i, i)));
         }
 
         d
@@ -120,28 +149,29 @@ pub trait BiconjugateGradientSolver<T: RealNumber, M: Matrix<T>> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::arrays::Array2;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
     pub struct BGSolver {}
 
-    impl<T: RealNumber, M: Matrix<T>> BiconjugateGradientSolver<T, M> for BGSolver {}
+    impl<T: FloatNumber, X: Array2<T>> BiconjugateGradientSolver<'_, T, X> for BGSolver {}
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
     #[test]
     fn bg_solver() {
         let a = DenseMatrix::from_2d_array(&[&[25., 15., -5.], &[15., 18., 0.], &[-5., 0., 11.]]);
-        let b = DenseMatrix::from_2d_array(&[&[40., 51., 28.]]);
-        let expected = DenseMatrix::from_2d_array(&[&[1.0, 2.0, 3.0]]);
+        let b = vec![40., 51., 28.];
+        let expected = vec![1.0, 2.0, 3.0];
 
-        let mut x = DenseMatrix::zeros(3, 1);
+        let mut x = Vec::zeros(3);
 
         let solver = BGSolver {};
 
-        let err: f64 = solver
-            .solve_mut(&a, &b.transpose(), &mut x, 1e-6, 6)
-            .unwrap();
+        let err: f64 = solver.solve_mut(&a, &b, &mut x, 1e-6, 6).unwrap();
 
-        assert!(x.transpose().approximate_eq(&expected, 1e-4));
+        assert!(x
+            .iter()
+            .zip(expected.iter())
+            .all(|(&a, &b)| (a - b).abs() < 1e-4));
         assert!((err - 0.0).abs() < 1e-4);
     }
 }

src/linear/elastic_net.rs

@@ -17,7 +17,7 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::linear::elastic_net::*;
 //!
 //! // Longley dataset (https://www.statsmodels.org/stable/datasets/generated/longley.html)
@@ -55,32 +55,39 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 use std::fmt::Debug;
+use std::marker::PhantomData;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array, Array1, Array2, MutArray};
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+use crate::numbers::realnum::RealNumber;
 
 use crate::linear::lasso_optimizer::InteriorPointOptimizer;
 
 /// Elastic net parameters
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct ElasticNetParameters<T: RealNumber> {
+pub struct ElasticNetParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Regularization parameter.
-    pub alpha: T,
+    pub alpha: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The elastic net mixing parameter, with 0 <= l1_ratio <= 1.
     /// For l1_ratio = 0 the penalty is an L2 penalty.
     /// For l1_ratio = 1 it is an L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
-    pub l1_ratio: T,
+    pub l1_ratio: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the standard deviation.
     pub normalize: bool,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The tolerance for the optimization
-    pub tol: T,
+    pub tol: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The maximum number of iterations
     pub max_iter: usize,
 }
@@ -88,21 +95,23 @@ pub struct ElasticNetParameters<T: RealNumber> {
 /// Elastic net
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct ElasticNet<T: RealNumber, M: Matrix<T>> {
-    coefficients: M,
-    intercept: T,
+pub struct ElasticNet<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> {
+    coefficients: Option<X>,
+    intercept: Option<TX>,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_y: PhantomData<Y>,
 }
 
-impl<T: RealNumber> ElasticNetParameters<T> {
+impl ElasticNetParameters {
     /// Regularization parameter.
-    pub fn with_alpha(mut self, alpha: T) -> Self {
+    pub fn with_alpha(mut self, alpha: f64) -> Self {
         self.alpha = alpha;
         self
     }
     /// The elastic net mixing parameter, with 0 <= l1_ratio <= 1.
     /// For l1_ratio = 0 the penalty is an L2 penalty.
     /// For l1_ratio = 1 it is an L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
-    pub fn with_l1_ratio(mut self, l1_ratio: T) -> Self {
+    pub fn with_l1_ratio(mut self, l1_ratio: f64) -> Self {
         self.l1_ratio = l1_ratio;
         self
     }
@@ -112,7 +121,7 @@ impl<T: RealNumber> ElasticNetParameters<T> {
         self
     }
     /// The tolerance for the optimization
-    pub fn with_tol(mut self, tol: T) -> Self {
+    pub fn with_tol(mut self, tol: f64) -> Self {
         self.tol = tol;
         self
     }
@@ -123,61 +132,205 @@ impl<T: RealNumber> ElasticNetParameters<T> {
     }
 }
 
-impl<T: RealNumber> Default for ElasticNetParameters<T> {
+impl Default for ElasticNetParameters {
     fn default() -> Self {
         ElasticNetParameters {
-            alpha: T::one(),
-            l1_ratio: T::half(),
+            alpha: 1.0,
+            l1_ratio: 0.5,
             normalize: true,
-            tol: T::from_f64(1e-4).unwrap(),
+            tol: 1e-4,
             max_iter: 1000,
         }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for ElasticNet<T, M> {
-    fn eq(&self, other: &Self) -> bool {
-        self.coefficients == other.coefficients
-            && (self.intercept - other.intercept).abs() <= T::epsilon()
+/// ElasticNet grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct ElasticNetSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Regularization parameter.
+    pub alpha: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The elastic net mixing parameter, with 0 <= l1_ratio <= 1.
+    /// For l1_ratio = 0 the penalty is an L2 penalty.
+    /// For l1_ratio = 1 it is an L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
+    pub l1_ratio: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the standard deviation.
+    pub normalize: Vec<bool>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The tolerance for the optimization
+    pub tol: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The maximum number of iterations
+    pub max_iter: Vec<usize>,
+}
+
+/// ElasticNet grid search iterator
+pub struct ElasticNetSearchParametersIterator {
+    lasso_regression_search_parameters: ElasticNetSearchParameters,
+    current_alpha: usize,
+    current_l1_ratio: usize,
+    current_normalize: usize,
+    current_tol: usize,
+    current_max_iter: usize,
+}
+
+impl IntoIterator for ElasticNetSearchParameters {
+    type Item = ElasticNetParameters;
+    type IntoIter = ElasticNetSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        ElasticNetSearchParametersIterator {
+            lasso_regression_search_parameters: self,
+            current_alpha: 0,
+            current_l1_ratio: 0,
+            current_normalize: 0,
+            current_tol: 0,
+            current_max_iter: 0,
+        }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, ElasticNetParameters<T>>
-    for ElasticNet<T, M>
+impl Iterator for ElasticNetSearchParametersIterator {
+    type Item = ElasticNetParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_alpha == self.lasso_regression_search_parameters.alpha.len()
+            && self.current_l1_ratio == self.lasso_regression_search_parameters.l1_ratio.len()
+            && self.current_normalize == self.lasso_regression_search_parameters.normalize.len()
+            && self.current_tol == self.lasso_regression_search_parameters.tol.len()
+            && self.current_max_iter == self.lasso_regression_search_parameters.max_iter.len()
+        {
+            return None;
+        }
+
+        let next = ElasticNetParameters {
+            alpha: self.lasso_regression_search_parameters.alpha[self.current_alpha],
+            l1_ratio: self.lasso_regression_search_parameters.alpha[self.current_l1_ratio],
+            normalize: self.lasso_regression_search_parameters.normalize[self.current_normalize],
+            tol: self.lasso_regression_search_parameters.tol[self.current_tol],
+            max_iter: self.lasso_regression_search_parameters.max_iter[self.current_max_iter],
+        };
+
+        if self.current_alpha + 1 < self.lasso_regression_search_parameters.alpha.len() {
+            self.current_alpha += 1;
+        } else if self.current_l1_ratio + 1 < self.lasso_regression_search_parameters.l1_ratio.len()
+        {
+            self.current_alpha = 0;
+            self.current_l1_ratio += 1;
+        } else if self.current_normalize + 1
+            < self.lasso_regression_search_parameters.normalize.len()
+        {
+            self.current_alpha = 0;
+            self.current_l1_ratio = 0;
+            self.current_normalize += 1;
+        } else if self.current_tol + 1 < self.lasso_regression_search_parameters.tol.len() {
+            self.current_alpha = 0;
+            self.current_l1_ratio = 0;
+            self.current_normalize = 0;
+            self.current_tol += 1;
+        } else if self.current_max_iter + 1 < self.lasso_regression_search_parameters.max_iter.len()
+        {
+            self.current_alpha = 0;
+            self.current_l1_ratio = 0;
+            self.current_normalize = 0;
+            self.current_tol = 0;
+            self.current_max_iter += 1;
+        } else {
+            self.current_alpha += 1;
+            self.current_l1_ratio += 1;
+            self.current_normalize += 1;
+            self.current_tol += 1;
+            self.current_max_iter += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for ElasticNetSearchParameters {
+    fn default() -> Self {
+        let default_params = ElasticNetParameters::default();
+
+        ElasticNetSearchParameters {
+            alpha: vec![default_params.alpha],
+            l1_ratio: vec![default_params.l1_ratio],
+            normalize: vec![default_params.normalize],
+            tol: vec![default_params.tol],
+            max_iter: vec![default_params.max_iter],
+        }
+    }
+}
+
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> PartialEq
+    for ElasticNet<TX, TY, X, Y>
 {
-    fn fit(x: &M, y: &M::RowVector, parameters: ElasticNetParameters<T>) -> Result<Self, Failed> {
+    fn eq(&self, other: &Self) -> bool {
+        if self.intercept() != other.intercept() {
+            return false;
+        }
+        if self.coefficients().shape() != other.coefficients().shape() {
+            return false;
+        }
+        self.coefficients()
+            .iterator(0)
+            .zip(other.coefficients().iterator(0))
+            .all(|(&a, &b)| (a - b).abs() <= TX::epsilon())
+    }
+}
+
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    SupervisedEstimator<X, Y, ElasticNetParameters> for ElasticNet<TX, TY, X, Y>
+{
+    fn new() -> Self {
+        Self {
+            coefficients: Option::None,
+            intercept: Option::None,
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: ElasticNetParameters) -> Result<Self, Failed> {
         ElasticNet::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for ElasticNet<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> Predictor<X, Y>
+    for ElasticNet<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> ElasticNet<T, M> {
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    ElasticNet<TX, TY, X, Y>
+{
     /// Fits elastic net regression to your data.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - target values
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
     pub fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: ElasticNetParameters<T>,
-    ) -> Result<ElasticNet<T, M>, Failed> {
+        x: &X,
+        y: &Y,
+        parameters: ElasticNetParameters,
+    ) -> Result<ElasticNet<TX, TY, X, Y>, Failed> {
         let (n, p) = x.shape();
 
-        if y.len() != n {
+        if y.shape() != n {
             return Err(Failed::fit("Number of rows in X should = len(y)"));
         }
 
-        let n_float = T::from_usize(n).unwrap();
+        let n_float = n as f64;
 
-        let l1_reg = parameters.alpha * parameters.l1_ratio * n_float;
-        let l2_reg = parameters.alpha * (T::one() - parameters.l1_ratio) * n_float;
+        let l1_reg = TX::from_f64(parameters.alpha * parameters.l1_ratio * n_float).unwrap();
+        let l2_reg =
+            TX::from_f64(parameters.alpha * (1.0 - parameters.l1_ratio) * n_float).unwrap();
 
-        let y_mean = y.mean();
+        let y_mean = TX::from_f64(y.mean_by()).unwrap();
 
         let (w, b) = if parameters.normalize {
             let (scaled_x, col_mean, col_std) = Self::rescale_x(x)?;
@@ -186,68 +339,92 @@ impl<T: RealNumber, M: Matrix<T>> ElasticNet<T, M> {
 
             let mut optimizer = InteriorPointOptimizer::new(&x, p);
 
-            let mut w =
-                optimizer.optimize(&x, &y, l1_reg * gamma, parameters.max_iter, parameters.tol)?;
+            let mut w = optimizer.optimize(
+                &x,
+                &y,
+                l1_reg * gamma,
+                parameters.max_iter,
+                TX::from_f64(parameters.tol).unwrap(),
+            )?;
 
             for i in 0..p {
-                w.set(i, 0, gamma * w.get(i, 0) / col_std[i]);
+                w.set(i, gamma * *w.get(i) / col_std[i]);
             }
 
-            let mut b = T::zero();
+            let mut b = TX::zero();
 
             for i in 0..p {
-                b += w.get(i, 0) * col_mean[i];
+                b += *w.get(i) * col_mean[i];
             }
 
             b = y_mean - b;
 
-            (w, b)
+            (X::from_column(&w), b)
         } else {
             let (x, y, gamma) = Self::augment_x_and_y(x, y, l2_reg);
 
             let mut optimizer = InteriorPointOptimizer::new(&x, p);
 
-            let mut w =
-                optimizer.optimize(&x, &y, l1_reg * gamma, parameters.max_iter, parameters.tol)?;
+            let mut w = optimizer.optimize(
+                &x,
+                &y,
+                l1_reg * gamma,
+                parameters.max_iter,
+                TX::from_f64(parameters.tol).unwrap(),
+            )?;
 
             for i in 0..p {
-                w.set(i, 0, gamma * w.get(i, 0));
+                w.set(i, gamma * *w.get(i));
             }
 
-            (w, y_mean)
+            (X::from_column(&w), y_mean)
         };
 
         Ok(ElasticNet {
-            intercept: b,
-            coefficients: w,
+            intercept: Some(b),
+            coefficients: Some(w),
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
     /// Predict target values from `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
         let (nrows, _) = x.shape();
-        let mut y_hat = x.matmul(&self.coefficients);
-        y_hat.add_mut(&M::fill(nrows, 1, self.intercept));
-        Ok(y_hat.transpose().to_row_vector())
+        let mut y_hat = x.matmul(self.coefficients.as_ref().unwrap());
+        let bias = X::fill(nrows, 1, self.intercept.unwrap());
+        y_hat.add_mut(&bias);
+        Ok(Y::from_iterator(
+            y_hat.iterator(0).map(|&v| TY::from(v).unwrap()),
+            nrows,
+        ))
     }
 
     /// Get estimates regression coefficients
-    pub fn coefficients(&self) -> &M {
-        &self.coefficients
+    pub fn coefficients(&self) -> &X {
+        self.coefficients.as_ref().unwrap()
     }
 
     /// Get estimate of intercept
-    pub fn intercept(&self) -> T {
-        self.intercept
+    pub fn intercept(&self) -> &TX {
+        self.intercept.as_ref().unwrap()
     }
 
-    fn rescale_x(x: &M) -> Result<(M, Vec<T>, Vec<T>), Failed> {
-        let col_mean = x.mean(0);
-        let col_std = x.std(0);
+    fn rescale_x(x: &X) -> Result<(X, Vec<TX>, Vec<TX>), Failed> {
+        let col_mean: Vec<TX> = x
+            .mean_by(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
+        let col_std: Vec<TX> = x
+            .std_dev(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
 
-        for i in 0..col_std.len() {
-            if (col_std[i] - T::zero()).abs() < T::epsilon() {
+        for (i, col_std_i) in col_std.iter().enumerate() {
+            if (*col_std_i - TX::zero()).abs() < TX::epsilon() {
                 return Err(Failed::fit(&format!(
                     "Cannot rescale constant column {}",
                     i
@@ -260,25 +437,25 @@ impl<T: RealNumber, M: Matrix<T>> ElasticNet<T, M> {
         Ok((scaled_x, col_mean, col_std))
     }
 
-    fn augment_x_and_y(x: &M, y: &M::RowVector, l2_reg: T) -> (M, M::RowVector, T) {
+    fn augment_x_and_y(x: &X, y: &Y, l2_reg: TX) -> (X, Vec<TX>, TX) {
         let (n, p) = x.shape();
 
-        let gamma = T::one() / (T::one() + l2_reg).sqrt();
+        let gamma = TX::one() / (TX::one() + l2_reg).sqrt();
         let padding = gamma * l2_reg.sqrt();
 
-        let mut y2 = M::RowVector::zeros(n + p);
-        for i in 0..y.len() {
-            y2.set(i, y.get(i));
+        let mut y2 = Vec::<TX>::zeros(n + p);
+        for i in 0..y.shape() {
+            y2.set(i, TX::from(*y.get(i)).unwrap());
         }
 
-        let mut x2 = M::zeros(n + p, p);
+        let mut x2 = X::zeros(n + p, p);
 
         for j in 0..p {
             for i in 0..n {
-                x2.set(i, j, gamma * x.get(i, j));
+                x2.set((i, j), gamma * *x.get((i, j)));
             }
 
-            x2.set(j + n, j, padding);
+            x2.set((j + n, j), padding);
         }
 
         (x2, y2, gamma)
@@ -288,10 +465,36 @@ impl<T: RealNumber, M: Matrix<T>> ElasticNet<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
     use crate::metrics::mean_absolute_error;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = ElasticNetSearchParameters {
+            alpha: vec![0., 1.],
+            max_iter: vec![10, 100],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 0.);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 0.);
+        assert_eq!(next.max_iter, 100);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        assert_eq!(next.max_iter, 100);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn elasticnet_longley() {
         let x = DenseMatrix::from_2d_array(&[
@@ -335,7 +538,10 @@ mod tests {
         assert!(mean_absolute_error(&y_hat, &y) < 30.0);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn elasticnet_fit_predict1() {
         let x = DenseMatrix::from_2d_array(&[
@@ -398,43 +604,44 @@ mod tests {
         assert!(mae_l1 < 2.0);
         assert!(mae_l2 < 2.0);
 
-        assert!(l1_model.coefficients().get(0, 0) > l1_model.coefficients().get(1, 0));
-        assert!(l1_model.coefficients().get(0, 0) > l1_model.coefficients().get(2, 0));
+        assert!(l1_model.coefficients().get((0, 0)) > l1_model.coefficients().get((1, 0)));
+        assert!(l1_model.coefficients().get((0, 0)) > l1_model.coefficients().get((2, 0)));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
-            &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
-            &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
-            &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
-            &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
-            &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
-            &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
-            &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
-            &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
-            &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
-            &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
-            &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
-            &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
-            &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
-            &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
-            &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
-        ]);
+    // TODO: serialization for the new DenseMatrix needs to be implemented
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn serde() {
+    //     let x = DenseMatrix::from_2d_array(&[
+    //         &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
+    //         &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
+    //         &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
+    //         &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
+    //         &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
+    //         &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
+    //         &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
+    //         &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
+    //         &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
+    //         &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
+    //         &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
+    //         &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
+    //         &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
+    //         &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
+    //         &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
+    //         &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
+    //     ]);
 
-        let y = vec![
-            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
-            114.2, 115.7, 116.9,
-        ];
+    //     let y = vec![
+    //         83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
+    //         114.2, 115.7, 116.9,
+    //     ];
 
-        let lr = ElasticNet::fit(&x, &y, Default::default()).unwrap();
+    //     let lr = ElasticNet::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_lr: ElasticNet<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
+    //     let deserialized_lr: ElasticNet<f64, f64, DenseMatrix<f64>, Vec<f64>> =
+    //         serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
 
-        assert_eq!(lr, deserialized_lr);
-    }
+    //     assert_eq!(lr, deserialized_lr);
+    // }
 }

src/linear/lasso.rs

@@ -23,28 +23,34 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 use std::fmt::Debug;
+use std::marker::PhantomData;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
+use crate::linalg::basic::arrays::{Array1, Array2, ArrayView1};
 use crate::linear::lasso_optimizer::InteriorPointOptimizer;
-use crate::math::num::RealNumber;
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+use crate::numbers::realnum::RealNumber;
 
 /// Lasso regression parameters
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct LassoParameters<T: RealNumber> {
+pub struct LassoParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Controls the strength of the penalty to the loss function.
-    pub alpha: T,
+    pub alpha: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// If true the regressors X will be normalized before regression
     /// by subtracting the mean and dividing by the standard deviation.
     pub normalize: bool,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The tolerance for the optimization
-    pub tol: T,
+    pub tol: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// The maximum number of iterations
     pub max_iter: usize,
 }
@@ -52,14 +58,16 @@ pub struct LassoParameters<T: RealNumber> {
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
 /// Lasso regressor
-pub struct Lasso<T: RealNumber, M: Matrix<T>> {
-    coefficients: M,
-    intercept: T,
+pub struct Lasso<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> {
+    coefficients: Option<X>,
+    intercept: Option<TX>,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_y: PhantomData<Y>,
 }
 
-impl<T: RealNumber> LassoParameters<T> {
+impl LassoParameters {
     /// Regularization parameter.
-    pub fn with_alpha(mut self, alpha: T) -> Self {
+    pub fn with_alpha(mut self, alpha: f64) -> Self {
         self.alpha = alpha;
         self
     }
@@ -69,7 +77,7 @@ impl<T: RealNumber> LassoParameters<T> {
         self
     }
     /// The tolerance for the optimization
-    pub fn with_tol(mut self, tol: T) -> Self {
+    pub fn with_tol(mut self, tol: f64) -> Self {
         self.tol = tol;
         self
     }
@@ -80,48 +88,162 @@ impl<T: RealNumber> LassoParameters<T> {
     }
 }
 
-impl<T: RealNumber> Default for LassoParameters<T> {
+impl Default for LassoParameters {
     fn default() -> Self {
         LassoParameters {
-            alpha: T::one(),
+            alpha: 1f64,
             normalize: true,
-            tol: T::from_f64(1e-4).unwrap(),
+            tol: 1e-4,
             max_iter: 1000,
         }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for Lasso<T, M> {
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> PartialEq
+    for Lasso<TX, TY, X, Y>
+{
     fn eq(&self, other: &Self) -> bool {
-        self.coefficients == other.coefficients
-            && (self.intercept - other.intercept).abs() <= T::epsilon()
+        self.intercept == other.intercept
+            && self.coefficients().shape() == other.coefficients().shape()
+            && self
+                .coefficients()
+                .iterator(0)
+                .zip(other.coefficients().iterator(0))
+                .all(|(&a, &b)| (a - b).abs() <= TX::epsilon())
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, LassoParameters<T>>
-    for Lasso<T, M>
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>>
+    SupervisedEstimator<X, Y, LassoParameters> for Lasso<TX, TY, X, Y>
 {
-    fn fit(x: &M, y: &M::RowVector, parameters: LassoParameters<T>) -> Result<Self, Failed> {
+    fn new() -> Self {
+        Self {
+            coefficients: Option::None,
+            intercept: Option::None,
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: LassoParameters) -> Result<Self, Failed> {
         Lasso::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for Lasso<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> Predictor<X, Y>
+    for Lasso<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Lasso<T, M> {
+/// Lasso grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct LassoSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Controls the strength of the penalty to the loss function.
+    pub alpha: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// If true the regressors X will be normalized before regression
+    /// by subtracting the mean and dividing by the standard deviation.
+    pub normalize: Vec<bool>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The tolerance for the optimization
+    pub tol: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// The maximum number of iterations
+    pub max_iter: Vec<usize>,
+}
+
+/// Lasso grid search iterator
+pub struct LassoSearchParametersIterator {
+    lasso_search_parameters: LassoSearchParameters,
+    current_alpha: usize,
+    current_normalize: usize,
+    current_tol: usize,
+    current_max_iter: usize,
+}
+
+impl IntoIterator for LassoSearchParameters {
+    type Item = LassoParameters;
+    type IntoIter = LassoSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        LassoSearchParametersIterator {
+            lasso_search_parameters: self,
+            current_alpha: 0,
+            current_normalize: 0,
+            current_tol: 0,
+            current_max_iter: 0,
+        }
+    }
+}
+
+impl Iterator for LassoSearchParametersIterator {
+    type Item = LassoParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_alpha == self.lasso_search_parameters.alpha.len()
+            && self.current_normalize == self.lasso_search_parameters.normalize.len()
+            && self.current_tol == self.lasso_search_parameters.tol.len()
+            && self.current_max_iter == self.lasso_search_parameters.max_iter.len()
+        {
+            return None;
+        }
+
+        let next = LassoParameters {
+            alpha: self.lasso_search_parameters.alpha[self.current_alpha],
+            normalize: self.lasso_search_parameters.normalize[self.current_normalize],
+            tol: self.lasso_search_parameters.tol[self.current_tol],
+            max_iter: self.lasso_search_parameters.max_iter[self.current_max_iter],
+        };
+
+        if self.current_alpha + 1 < self.lasso_search_parameters.alpha.len() {
+            self.current_alpha += 1;
+        } else if self.current_normalize + 1 < self.lasso_search_parameters.normalize.len() {
+            self.current_alpha = 0;
+            self.current_normalize += 1;
+        } else if self.current_tol + 1 < self.lasso_search_parameters.tol.len() {
+            self.current_alpha = 0;
+            self.current_normalize = 0;
+            self.current_tol += 1;
+        } else if self.current_max_iter + 1 < self.lasso_search_parameters.max_iter.len() {
+            self.current_alpha = 0;
+            self.current_normalize = 0;
+            self.current_tol = 0;
+            self.current_max_iter += 1;
+        } else {
+            self.current_alpha += 1;
+            self.current_normalize += 1;
+            self.current_tol += 1;
+            self.current_max_iter += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl Default for LassoSearchParameters {
+    fn default() -> Self {
+        let default_params = LassoParameters::default();
+
+        LassoSearchParameters {
+            alpha: vec![default_params.alpha],
+            normalize: vec![default_params.normalize],
+            tol: vec![default_params.tol],
+            max_iter: vec![default_params.max_iter],
+        }
+    }
+}
+
+impl<TX: FloatNumber + RealNumber, TY: Number, X: Array2<TX>, Y: Array1<TY>> Lasso<TX, TY, X, Y> {
     /// Fits Lasso regression to your data.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - target values
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
-    pub fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: LassoParameters<T>,
-    ) -> Result<Lasso<T, M>, Failed> {
+    pub fn fit(x: &X, y: &Y, parameters: LassoParameters) -> Result<Lasso<TX, TY, X, Y>, Failed> {
         let (n, p) = x.shape();
 
         if n <= p {
@@ -130,11 +252,11 @@ impl<T: RealNumber, M: Matrix<T>> Lasso<T, M> {
             ));
         }
 
-        if parameters.alpha < T::zero() {
+        if parameters.alpha < 0f64 {
             return Err(Failed::fit("alpha should be >= 0"));
         }
 
-        if parameters.tol <= T::zero() {
+        if parameters.tol <= 0f64 {
             return Err(Failed::fit("tol should be > 0"));
         }
 
@@ -142,71 +264,98 @@ impl<T: RealNumber, M: Matrix<T>> Lasso<T, M> {
             return Err(Failed::fit("max_iter should be > 0"));
         }
 
-        if y.len() != n {
+        if y.shape() != n {
             return Err(Failed::fit("Number of rows in X should = len(y)"));
         }
 
-        let l1_reg = parameters.alpha * T::from_usize(n).unwrap();
+        let y: Vec<TX> = y.iterator(0).map(|&v| TX::from(v).unwrap()).collect();
+
+        let l1_reg = TX::from_f64(parameters.alpha * n as f64).unwrap();
 
         let (w, b) = if parameters.normalize {
             let (scaled_x, col_mean, col_std) = Self::rescale_x(x)?;
 
             let mut optimizer = InteriorPointOptimizer::new(&scaled_x, p);
 
-            let mut w =
-                optimizer.optimize(&scaled_x, y, l1_reg, parameters.max_iter, parameters.tol)?;
+            let mut w = optimizer.optimize(
+                &scaled_x,
+                &y,
+                l1_reg,
+                parameters.max_iter,
+                TX::from_f64(parameters.tol).unwrap(),
+            )?;
 
             for (j, col_std_j) in col_std.iter().enumerate().take(p) {
-                w.set(j, 0, w.get(j, 0) / *col_std_j);
+                w[j] /= *col_std_j;
             }
 
-            let mut b = T::zero();
+            let mut b = TX::zero();
 
             for (i, col_mean_i) in col_mean.iter().enumerate().take(p) {
-                b += w.get(i, 0) * *col_mean_i;
+                b += w[i] * *col_mean_i;
             }
 
-            b = y.mean() - b;
-            (w, b)
+            b = TX::from_f64(y.mean_by()).unwrap() - b;
+            (X::from_column(&w), b)
         } else {
             let mut optimizer = InteriorPointOptimizer::new(x, p);
 
-            let w = optimizer.optimize(x, y, l1_reg, parameters.max_iter, parameters.tol)?;
+            let w = optimizer.optimize(
+                x,
+                &y,
+                l1_reg,
+                parameters.max_iter,
+                TX::from_f64(parameters.tol).unwrap(),
+            )?;
 
-            (w, y.mean())
+            (X::from_column(&w), TX::from_f64(y.mean_by()).unwrap())
         };
 
         Ok(Lasso {
-            intercept: b,
-            coefficients: w,
+            intercept: Some(b),
+            coefficients: Some(w),
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
     /// Predict target values from `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
         let (nrows, _) = x.shape();
-        let mut y_hat = x.matmul(&self.coefficients);
-        y_hat.add_mut(&M::fill(nrows, 1, self.intercept));
-        Ok(y_hat.transpose().to_row_vector())
+        let mut y_hat = x.matmul(self.coefficients());
+        let bias = X::fill(nrows, 1, self.intercept.unwrap());
+        y_hat.add_mut(&bias);
+        Ok(Y::from_iterator(
+            y_hat.iterator(0).map(|&v| TY::from(v).unwrap()),
+            nrows,
+        ))
     }
 
     /// Get estimates regression coefficients
-    pub fn coefficients(&self) -> &M {
-        &self.coefficients
+    pub fn coefficients(&self) -> &X {
+        self.coefficients.as_ref().unwrap()
     }
 
     /// Get estimate of intercept
-    pub fn intercept(&self) -> T {
-        self.intercept
+    pub fn intercept(&self) -> &TX {
+        self.intercept.as_ref().unwrap()
     }
 
-    fn rescale_x(x: &M) -> Result<(M, Vec<T>, Vec<T>), Failed> {
-        let col_mean = x.mean(0);
-        let col_std = x.std(0);
+    fn rescale_x(x: &X) -> Result<(X, Vec<TX>, Vec<TX>), Failed> {
+        let col_mean: Vec<TX> = x
+            .mean_by(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
+        let col_std: Vec<TX> = x
+            .std_dev(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
 
         for (i, col_std_i) in col_std.iter().enumerate() {
-            if (*col_std_i - T::zero()).abs() < T::epsilon() {
+            if (*col_std_i - TX::zero()).abs() < TX::epsilon() {
                 return Err(Failed::fit(&format!(
                     "Cannot rescale constant column {}",
                     i
@@ -223,10 +372,36 @@ impl<T: RealNumber, M: Matrix<T>> Lasso<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
     use crate::metrics::mean_absolute_error;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = LassoSearchParameters {
+            alpha: vec![0., 1.],
+            max_iter: vec![10, 100],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 0.);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        assert_eq!(next.max_iter, 10);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 0.);
+        assert_eq!(next.max_iter, 100);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        assert_eq!(next.max_iter, 100);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn lasso_fit_predict() {
         let x = DenseMatrix::from_2d_array(&[
@@ -275,39 +450,40 @@ mod tests {
         assert!(mean_absolute_error(&y_hat, &y) < 2.0);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
-            &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
-            &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
-            &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
-            &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
-            &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
-            &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
-            &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
-            &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
-            &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
-            &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
-            &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
-            &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
-            &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
-            &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
-            &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
-        ]);
+    // TODO: serialization for the new DenseMatrix needs to be implemented
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn serde() {
+    //     let x = DenseMatrix::from_2d_array(&[
+    //         &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
+    //         &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
+    //         &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
+    //         &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
+    //         &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
+    //         &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
+    //         &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
+    //         &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
+    //         &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
+    //         &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
+    //         &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
+    //         &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
+    //         &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
+    //         &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
+    //         &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
+    //         &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
+    //     ]);
 
-        let y = vec![
-            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
-            114.2, 115.7, 116.9,
-        ];
+    //     let y = vec![
+    //         83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
+    //         114.2, 115.7, 116.9,
+    //     ];
 
-        let lr = Lasso::fit(&x, &y, Default::default()).unwrap();
+    //     let lr = Lasso::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_lr: Lasso<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
+    //     let deserialized_lr: Lasso<f64, f64, DenseMatrix<f64>, Vec<f64>> =
+    //         serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
 
-        assert_eq!(lr, deserialized_lr);
-    }
+    //     assert_eq!(lr, deserialized_lr);
+    // }
 }

src/linear/lasso_optimizer.rs

@@ -12,21 +12,23 @@
 //!
 
 use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
+use crate::linalg::basic::arrays::{Array1, Array2, ArrayView1, MutArray, MutArrayView1};
 use crate::linear::bg_solver::BiconjugateGradientSolver;
-use crate::math::num::RealNumber;
+use crate::numbers::floatnum::FloatNumber;
 
-pub struct InteriorPointOptimizer<T: RealNumber, M: Matrix<T>> {
-    ata: M,
+///
+pub struct InteriorPointOptimizer<T: FloatNumber, X: Array2<T>> {
+    ata: X,
     d1: Vec<T>,
     d2: Vec<T>,
     prb: Vec<T>,
     prs: Vec<T>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
-    pub fn new(a: &M, n: usize) -> InteriorPointOptimizer<T, M> {
+///
+impl<T: FloatNumber, X: Array2<T>> InteriorPointOptimizer<T, X> {
+    ///
+    pub fn new(a: &X, n: usize) -> InteriorPointOptimizer<T, X> {
         InteriorPointOptimizer {
             ata: a.ab(true, a, false),
             d1: vec![T::zero(); n],
@@ -36,14 +38,15 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
         }
     }
 
+    ///
     pub fn optimize(
         &mut self,
-        x: &M,
-        y: &M::RowVector,
+        x: &X,
+        y: &Vec<T>,
         lambda: T,
         max_iter: usize,
         tol: T,
-    ) -> Result<M, Failed> {
+    ) -> Result<Vec<T>, Failed> {
         let (n, p) = x.shape();
         let p_f64 = T::from_usize(p).unwrap();
 
@@ -58,50 +61,53 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
         let gamma = T::from_f64(-0.25).unwrap();
         let mu = T::two();
 
-        let y = M::from_row_vector(y.sub_scalar(y.mean())).transpose();
+        // let y = M::from_row_vector(y.sub_scalar(y.mean_by())).transpose();
+        let y = y.sub_scalar(T::from_f64(y.mean_by()).unwrap());
 
         let mut max_ls_iter = 100;
         let mut pitr = 0;
-        let mut w = M::zeros(p, 1);
+        let mut w = Vec::zeros(p);
         let mut neww = w.clone();
-        let mut u = M::ones(p, 1);
+        let mut u = Vec::ones(p);
         let mut newu = u.clone();
 
-        let mut f = M::fill(p, 2, -T::one());
+        let mut f = X::fill(p, 2, -T::one());
         let mut newf = f.clone();
 
         let mut q1 = vec![T::zero(); p];
         let mut q2 = vec![T::zero(); p];
 
-        let mut dx = M::zeros(p, 1);
-        let mut du = M::zeros(p, 1);
-        let mut dxu = M::zeros(2 * p, 1);
-        let mut grad = M::zeros(2 * p, 1);
+        let mut dx = Vec::zeros(p);
+        let mut du = Vec::zeros(p);
+        let mut dxu = Vec::zeros(2 * p);
+        let mut grad = Vec::zeros(2 * p);
 
-        let mut nu = M::zeros(n, 1);
+        let mut nu = Vec::zeros(n);
         let mut dobj = T::zero();
         let mut s = T::infinity();
         let mut t = T::one()
             .max(T::one() / lambda)
             .min(T::two() * p_f64 / T::from(1e-3).unwrap());
 
+        let lambda_f64 = lambda.to_f64().unwrap();
+
         for ntiter in 0..max_iter {
-            let mut z = x.matmul(&w);
+            let mut z = w.xa(true, x);
 
             for i in 0..n {
-                z.set(i, 0, z.get(i, 0) - y.get(i, 0));
-                nu.set(i, 0, T::two() * z.get(i, 0));
+                z[i] -= y[i];
+                nu[i] = T::two() * z[i];
             }
 
             // CALCULATE DUALITY GAP
-            let xnu = x.ab(true, &nu, false);
-            let max_xnu = xnu.norm(T::infinity());
-            if max_xnu > lambda {
-                let lnu = lambda / max_xnu;
+            let xnu = nu.xa(false, x);
+            let max_xnu = xnu.norm(std::f64::INFINITY);
+            if max_xnu > lambda_f64 {
+                let lnu = T::from_f64(lambda_f64 / max_xnu).unwrap();
                 nu.mul_scalar_mut(lnu);
             }
 
-            let pobj = z.dot(&z) + lambda * w.norm(T::one());
+            let pobj = z.dot(&z) + lambda * T::from_f64(w.norm(1f64)).unwrap();
             dobj = dobj.max(gamma * nu.dot(&nu) - nu.dot(&y));
 
             let gap = pobj - dobj;
@@ -118,22 +124,22 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
 
             // CALCULATE NEWTON STEP
             for i in 0..p {
-                let q1i = T::one() / (u.get(i, 0) + w.get(i, 0));
-                let q2i = T::one() / (u.get(i, 0) - w.get(i, 0));
+                let q1i = T::one() / (u[i] + w[i]);
+                let q2i = T::one() / (u[i] - w[i]);
                 q1[i] = q1i;
                 q2[i] = q2i;
                 self.d1[i] = (q1i * q1i + q2i * q2i) / t;
                 self.d2[i] = (q1i * q1i - q2i * q2i) / t;
             }
 
-            let mut gradphi = x.ab(true, &z, false);
+            let mut gradphi = z.xa(false, x);
 
             for i in 0..p {
-                let g1 = T::two() * gradphi.get(i, 0) - (q1[i] - q2[i]) / t;
+                let g1 = T::two() * gradphi[i] - (q1[i] - q2[i]) / t;
                 let g2 = lambda - (q1[i] + q2[i]) / t;
-                gradphi.set(i, 0, g1);
-                grad.set(i, 0, -g1);
-                grad.set(i + p, 0, -g2);
+                gradphi[i] = g1;
+                grad[i] = -g1;
+                grad[i + p] = -g2;
             }
 
             for i in 0..p {
@@ -141,7 +147,7 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
                 self.prs[i] = self.prb[i] * self.d1[i] - self.d2[i].powi(2);
             }
 
-            let normg = grad.norm2();
+            let normg = T::from_f64(grad.norm2()).unwrap();
             let mut pcgtol = min_pcgtol.min(eta * gap / T::one().min(normg));
             if ntiter != 0 && pitr == 0 {
                 pcgtol *= min_pcgtol;
@@ -152,10 +158,8 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
                 pitr = pcgmaxi;
             }
 
-            for i in 0..p {
-                dx.set(i, 0, dxu.get(i, 0));
-                du.set(i, 0, dxu.get(i + p, 0));
-            }
+            dx[..p].copy_from_slice(&dxu[..p]);
+            du[..p].copy_from_slice(&dxu[p..(p + p)]);
 
             // BACKTRACKING LINE SEARCH
             let phi = z.dot(&z) + lambda * u.sum() - Self::sumlogneg(&f) / t;
@@ -165,16 +169,20 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
             let lsiter = 0;
             while lsiter < max_ls_iter {
                 for i in 0..p {
-                    neww.set(i, 0, w.get(i, 0) + s * dx.get(i, 0));
-                    newu.set(i, 0, u.get(i, 0) + s * du.get(i, 0));
-                    newf.set(i, 0, neww.get(i, 0) - newu.get(i, 0));
-                    newf.set(i, 1, -neww.get(i, 0) - newu.get(i, 0));
+                    neww[i] = w[i] + s * dx[i];
+                    newu[i] = u[i] + s * du[i];
+                    newf.set((i, 0), neww[i] - newu[i]);
+                    newf.set((i, 1), -neww[i] - newu[i]);
                 }
 
-                if newf.max() < T::zero() {
-                    let mut newz = x.matmul(&neww);
+                if newf
+                    .iterator(0)
+                    .fold(T::neg_infinity(), |max, v| v.max(max))
+                    < T::zero()
+                {
+                    let mut newz = neww.xa(true, x);
                     for i in 0..n {
-                        newz.set(i, 0, newz.get(i, 0) - y.get(i, 0));
+                        newz[i] -= y[i];
                     }
 
                     let newphi = newz.dot(&newz) + lambda * newu.sum() - Self::sumlogneg(&newf) / t;
@@ -200,54 +208,46 @@ impl<T: RealNumber, M: Matrix<T>> InteriorPointOptimizer<T, M> {
         Ok(w)
     }
 
-    fn sumlogneg(f: &M) -> T {
+    ///
+    fn sumlogneg(f: &X) -> T {
         let (n, _) = f.shape();
         let mut sum = T::zero();
         for i in 0..n {
-            sum += (-f.get(i, 0)).ln();
-            sum += (-f.get(i, 1)).ln();
+            sum += (-*f.get((i, 0))).ln();
+            sum += (-*f.get((i, 1))).ln();
         }
         sum
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> BiconjugateGradientSolver<T, M> for InteriorPointOptimizer<T, M> {
-    fn solve_preconditioner(&self, a: &M, b: &M, x: &mut M) {
+///
+impl<'a, T: FloatNumber, X: Array2<T>> BiconjugateGradientSolver<'a, T, X>
+    for InteriorPointOptimizer<T, X>
+{
+    ///
+    fn solve_preconditioner(&self, a: &'a X, b: &[T], x: &mut [T]) {
         let (_, p) = a.shape();
 
         for i in 0..p {
-            x.set(
-                i,
-                0,
-                (self.d1[i] * b.get(i, 0) - self.d2[i] * b.get(i + p, 0)) / self.prs[i],
-            );
-            x.set(
-                i + p,
-                0,
-                (-self.d2[i] * b.get(i, 0) + self.prb[i] * b.get(i + p, 0)) / self.prs[i],
-            );
+            x[i] = (self.d1[i] * b[i] - self.d2[i] * b[i + p]) / self.prs[i];
+            x[i + p] = (-self.d2[i] * b[i] + self.prb[i] * b[i + p]) / self.prs[i];
         }
     }
 
-    fn mat_vec_mul(&self, _: &M, x: &M, y: &mut M) {
+    ///
+    fn mat_vec_mul(&self, _: &X, x: &Vec<T>, y: &mut Vec<T>) {
         let (_, p) = self.ata.shape();
-        let atax = self.ata.matmul(&x.slice(0..p, 0..1));
+        let x_slice = Vec::from_slice(x.slice(0..p).as_ref());
+        let atax = x_slice.xa(true, &self.ata);
 
         for i in 0..p {
-            y.set(
-                i,
-                0,
-                T::two() * atax.get(i, 0) + self.d1[i] * x.get(i, 0) + self.d2[i] * x.get(i + p, 0),
-            );
-            y.set(
-                i + p,
-                0,
-                self.d2[i] * x.get(i, 0) + self.d1[i] * x.get(i + p, 0),
-            );
+            y[i] = T::two() * atax[i] + self.d1[i] * x[i] + self.d2[i] * x[i + p];
+            y[i + p] = self.d2[i] * x[i] + self.d1[i] * x[i + p];
         }
     }
 
-    fn mat_t_vec_mul(&self, a: &M, x: &M, y: &mut M) {
+    ///
+    fn mat_t_vec_mul(&self, a: &X, x: &Vec<T>, y: &mut Vec<T>) {
         self.mat_vec_mul(a, x, y);
     }
 }

src/linear/linear_regression.rs

@@ -12,14 +12,14 @@
 //! \\[\hat{\beta} = (X^TX)^{-1}X^Ty \\]
 //!
 //! the \\((X^TX)^{-1}\\) term is both computationally expensive and numerically unstable. An alternative approach is to use a matrix decomposition to avoid this operation.
-//! SmartCore uses [SVD](../../linalg/svd/index.html) and [QR](../../linalg/qr/index.html) matrix decomposition to find estimates of \\(\hat{\beta}\\).
+//! `smartcore` uses [SVD](../../linalg/svd/index.html) and [QR](../../linalg/qr/index.html) matrix decomposition to find estimates of \\(\hat{\beta}\\).
 //! The QR decomposition is more computationally efficient and more numerically stable than calculating the normal equation directly,
 //! but does not work for all data matrices. Unlike the QR decomposition, all matrices have an SVD decomposition.
 //!
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::linear::linear_regression::*;
 //!
 //! // Longley dataset (https://www.statsmodels.org/stable/datasets/generated/longley.html)
@@ -61,21 +61,26 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 use std::fmt::Debug;
+use std::marker::PhantomData;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::linalg::traits::qr::QRDecomposable;
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug, Clone)]
+#[derive(Debug, Default, Clone, Eq, PartialEq)]
 /// Approach to use for estimation of regression coefficients. QR is more efficient but SVD is more stable.
 pub enum LinearRegressionSolverName {
     /// QR decomposition, see [QR](../../linalg/qr/index.html)
     QR,
+    #[default]
     /// SVD decomposition, see [SVD](../../linalg/svd/index.html)
     SVD,
 }
@@ -84,27 +89,11 @@ pub enum LinearRegressionSolverName {
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
 pub struct LinearRegressionParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Solver to use for estimation of regression coefficients.
     pub solver: LinearRegressionSolverName,
 }
 
-/// Linear Regression
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug)]
-pub struct LinearRegression<T: RealNumber, M: Matrix<T>> {
-    coefficients: M,
-    intercept: T,
-    _solver: LinearRegressionSolverName,
-}
-
-impl LinearRegressionParameters {
-    /// Solver to use for estimation of regression coefficients.
-    pub fn with_solver(mut self, solver: LinearRegressionSolverName) -> Self {
-        self.solver = solver;
-        self
-    }
-}
-
 impl Default for LinearRegressionParameters {
     fn default() -> Self {
         LinearRegressionParameters {
@@ -113,43 +102,157 @@ impl Default for LinearRegressionParameters {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for LinearRegression<T, M> {
-    fn eq(&self, other: &Self) -> bool {
-        self.coefficients == other.coefficients
-            && (self.intercept - other.intercept).abs() <= T::epsilon()
+/// Linear Regression
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug)]
+pub struct LinearRegression<
+    TX: Number + RealNumber,
+    TY: Number,
+    X: Array2<TX> + QRDecomposable<TX> + SVDDecomposable<TX>,
+    Y: Array1<TY>,
+> {
+    coefficients: Option<X>,
+    intercept: Option<TX>,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_y: PhantomData<Y>,
+}
+
+impl LinearRegressionParameters {
+    /// Solver to use for estimation of regression coefficients.
+    pub fn with_solver(mut self, solver: LinearRegressionSolverName) -> Self {
+        self.solver = solver;
+        self
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, LinearRegressionParameters>
-    for LinearRegression<T, M>
+/// Linear Regression grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct LinearRegressionSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Solver to use for estimation of regression coefficients.
+    pub solver: Vec<LinearRegressionSolverName>,
+}
+
+/// Linear Regression grid search iterator
+pub struct LinearRegressionSearchParametersIterator {
+    linear_regression_search_parameters: LinearRegressionSearchParameters,
+    current_solver: usize,
+}
+
+impl IntoIterator for LinearRegressionSearchParameters {
+    type Item = LinearRegressionParameters;
+    type IntoIter = LinearRegressionSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        LinearRegressionSearchParametersIterator {
+            linear_regression_search_parameters: self,
+            current_solver: 0,
+        }
+    }
+}
+
+impl Iterator for LinearRegressionSearchParametersIterator {
+    type Item = LinearRegressionParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_solver == self.linear_regression_search_parameters.solver.len() {
+            return None;
+        }
+
+        let next = LinearRegressionParameters {
+            solver: self.linear_regression_search_parameters.solver[self.current_solver].clone(),
+        };
+
+        self.current_solver += 1;
+
+        Some(next)
+    }
+}
+
+impl Default for LinearRegressionSearchParameters {
+    fn default() -> Self {
+        let default_params = LinearRegressionParameters::default();
+
+        LinearRegressionSearchParameters {
+            solver: vec![default_params.solver],
+        }
+    }
+}
+
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + QRDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > PartialEq for LinearRegression<TX, TY, X, Y>
 {
-    fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: LinearRegressionParameters,
-    ) -> Result<Self, Failed> {
+    fn eq(&self, other: &Self) -> bool {
+        self.intercept == other.intercept
+            && self.coefficients().shape() == other.coefficients().shape()
+            && self
+                .coefficients()
+                .iterator(0)
+                .zip(other.coefficients().iterator(0))
+                .all(|(&a, &b)| (a - b).abs() <= TX::epsilon())
+    }
+}
+
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + QRDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > SupervisedEstimator<X, Y, LinearRegressionParameters> for LinearRegression<TX, TY, X, Y>
+{
+    fn new() -> Self {
+        Self {
+            coefficients: Option::None,
+            intercept: Option::None,
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: LinearRegressionParameters) -> Result<Self, Failed> {
         LinearRegression::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for LinearRegression<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + QRDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > Predictor<X, Y> for LinearRegression<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> LinearRegression<T, M> {
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + QRDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > LinearRegression<TX, TY, X, Y>
+{
     /// Fits Linear Regression to your data.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - target values
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
     pub fn fit(
-        x: &M,
-        y: &M::RowVector,
+        x: &X,
+        y: &Y,
         parameters: LinearRegressionParameters,
-    ) -> Result<LinearRegression<T, M>, Failed> {
-        let y_m = M::from_row_vector(y.clone());
-        let b = y_m.transpose();
+    ) -> Result<LinearRegression<TX, TY, X, Y>, Failed> {
+        let b = X::from_iterator(
+            y.iterator(0).map(|&v| TX::from(v).unwrap()),
+            y.shape(),
+            1,
+            0,
+        );
         let (x_nrows, num_attributes) = x.shape();
         let (y_nrows, _) = b.shape();
 
@@ -159,59 +262,77 @@ impl<T: RealNumber, M: Matrix<T>> LinearRegression<T, M> {
             ));
         }
 
-        let a = x.h_stack(&M::ones(x_nrows, 1));
+        let a = x.h_stack(&X::ones(x_nrows, 1));
 
         let w = match parameters.solver {
             LinearRegressionSolverName::QR => a.qr_solve_mut(b)?,
             LinearRegressionSolverName::SVD => a.svd_solve_mut(b)?,
         };
 
-        let wights = w.slice(0..num_attributes, 0..1);
+        let weights = X::from_slice(w.slice(0..num_attributes, 0..1).as_ref());
 
         Ok(LinearRegression {
-            intercept: w.get(num_attributes, 0),
-            coefficients: wights,
-            _solver: parameters.solver,
+            intercept: Some(*w.get((num_attributes, 0))),
+            coefficients: Some(weights),
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
     /// Predict target values from `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
         let (nrows, _) = x.shape();
-        let mut y_hat = x.matmul(&self.coefficients);
-        y_hat.add_mut(&M::fill(nrows, 1, self.intercept));
-        Ok(y_hat.transpose().to_row_vector())
+        let bias = X::fill(nrows, 1, *self.intercept());
+        let mut y_hat = x.matmul(self.coefficients());
+        y_hat.add_mut(&bias);
+        Ok(Y::from_iterator(
+            y_hat.iterator(0).map(|&v| TY::from(v).unwrap()),
+            nrows,
+        ))
     }
 
     /// Get estimates regression coefficients
-    pub fn coefficients(&self) -> &M {
-        &self.coefficients
+    pub fn coefficients(&self) -> &X {
+        self.coefficients.as_ref().unwrap()
     }
 
     /// Get estimate of intercept
-    pub fn intercept(&self) -> T {
-        self.intercept
+    pub fn intercept(&self) -> &TX {
+        self.intercept.as_ref().unwrap()
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = LinearRegressionSearchParameters {
+            solver: vec![
+                LinearRegressionSolverName::QR,
+                LinearRegressionSolverName::SVD,
+            ],
+        };
+        let mut iter = parameters.into_iter();
+        assert_eq!(iter.next().unwrap().solver, LinearRegressionSolverName::QR);
+        assert_eq!(iter.next().unwrap().solver, LinearRegressionSolverName::SVD);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn ols_fit_predict() {
         let x = DenseMatrix::from_2d_array(&[
             &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
-            &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
             &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
             &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
             &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
-            &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
-            &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
-            &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
             &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
             &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
             &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
@@ -223,8 +344,7 @@ mod tests {
         ]);
 
         let y: Vec<f64> = vec![
-            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
-            114.2, 115.7, 116.9,
+            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8,
         ];
 
         let y_hat_qr = LinearRegression::fit(
@@ -251,39 +371,44 @@ mod tests {
             .all(|(&a, &b)| (a - b).abs() <= 5.0));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
-            &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
-            &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
-            &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
-            &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
-            &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
-            &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
-            &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
-            &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
-            &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
-            &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
-            &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
-            &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
-            &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
-            &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
-            &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
-        ]);
+    // TODO: serialization for the new DenseMatrix needs to be implemented
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn serde() {
+    //     let x = DenseMatrix::from_2d_array(&[
+    //         &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
+    //         &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
+    //         &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
+    //         &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
+    //         &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
+    //         &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
+    //         &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
+    //         &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
+    //         &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
+    //         &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
+    //         &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
+    //         &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
+    //         &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
+    //         &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
+    //         &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
+    //         &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
+    //     ]);
 
-        let y = vec![
-            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
-            114.2, 115.7, 116.9,
-        ];
+    //     let y = vec![
+    //         83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
+    //         114.2, 115.7, 116.9,
+    //     ];
 
-        let lr = LinearRegression::fit(&x, &y, Default::default()).unwrap();
+    //     let lr = LinearRegression::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_lr: LinearRegression<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
+    //     let deserialized_lr: LinearRegression<f64, f64, DenseMatrix<f64>, Vec<f64>> =
+    //         serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
 
-        assert_eq!(lr, deserialized_lr);
-    }
+    //     assert_eq!(lr, deserialized_lr);
+
+    //     let default = LinearRegressionParameters::default();
+    //     let parameters: LinearRegressionParameters = serde_json::from_str("{}").unwrap();
+    //     assert_eq!(parameters.solver, default.solver);
+    // }
 }

src/linear/mod.rs

@@ -20,10 +20,10 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 
-pub(crate) mod bg_solver;
+pub mod bg_solver;
 pub mod elastic_net;
 pub mod lasso;
-pub(crate) mod lasso_optimizer;
+pub mod lasso_optimizer;
 pub mod linear_regression;
 pub mod logistic_regression;
 pub mod ridge_regression;

src/linear/ridge_regression.rs

@@ -12,14 +12,14 @@
 //! where \\(\alpha \geq 0\\) is a tuning parameter that controls strength of regularization. When \\(\alpha = 0\\) the penalty term has no effect, and ridge regression will produce the least squares estimates.
 //! However, as \\(\alpha \rightarrow \infty\\), the impact of the shrinkage penalty grows, and the ridge regression coefficient estimates will approach zero.
 //!
-//! SmartCore uses [SVD](../../linalg/svd/index.html) and [Cholesky](../../linalg/cholesky/index.html) matrix decomposition to find estimates of \\(\hat{\beta}\\).
+//! `smartcore` uses [SVD](../../linalg/svd/index.html) and [Cholesky](../../linalg/cholesky/index.html) matrix decomposition to find estimates of \\(\hat{\beta}\\).
 //! The Cholesky decomposition is more computationally efficient and more numerically stable than calculating the normal equation directly,
 //! but does not work for all data matrices. Unlike the Cholesky decomposition, all matrices have an SVD decomposition.
 //!
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::linear::ridge_regression::*;
 //!
 //! // Longley dataset (https://www.statsmodels.org/stable/datasets/generated/longley.html)
@@ -57,18 +57,21 @@
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 use std::fmt::Debug;
+use std::marker::PhantomData;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::linalg::traits::cholesky::CholeskyDecomposable;
+use crate::linalg::traits::svd::SVDDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug, Clone)]
+#[derive(Debug, Clone, Eq, PartialEq)]
 /// Approach to use for estimation of regression coefficients. Cholesky is more efficient but SVD is more stable.
 pub enum RidgeRegressionSolverName {
     /// Cholesky decomposition, see [Cholesky](../../linalg/cholesky/index.html)
@@ -77,10 +80,16 @@ pub enum RidgeRegressionSolverName {
     SVD,
 }
 
+impl Default for RidgeRegressionSolverName {
+    fn default() -> Self {
+        RidgeRegressionSolverName::Cholesky
+    }
+}
+
 /// Ridge Regression parameters
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct RidgeRegressionParameters<T: RealNumber> {
+pub struct RidgeRegressionParameters<T: Number + RealNumber> {
     /// Solver to use for estimation of regression coefficients.
     pub solver: RidgeRegressionSolverName,
     /// Controls the strength of the penalty to the loss function.
@@ -90,16 +99,109 @@ pub struct RidgeRegressionParameters<T: RealNumber> {
     pub normalize: bool,
 }
 
+/// Ridge Regression grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct RidgeRegressionSearchParameters<T: Number + RealNumber> {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Solver to use for estimation of regression coefficients.
+    pub solver: Vec<RidgeRegressionSolverName>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Regularization parameter.
+    pub alpha: Vec<T>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// If true the regressors X will be normalized before regression
+    /// by subtracting the mean and dividing by the standard deviation.
+    pub normalize: Vec<bool>,
+}
+
+/// Ridge Regression grid search iterator
+pub struct RidgeRegressionSearchParametersIterator<T: Number + RealNumber> {
+    ridge_regression_search_parameters: RidgeRegressionSearchParameters<T>,
+    current_solver: usize,
+    current_alpha: usize,
+    current_normalize: usize,
+}
+
+impl<T: Number + RealNumber> IntoIterator for RidgeRegressionSearchParameters<T> {
+    type Item = RidgeRegressionParameters<T>;
+    type IntoIter = RidgeRegressionSearchParametersIterator<T>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        RidgeRegressionSearchParametersIterator {
+            ridge_regression_search_parameters: self,
+            current_solver: 0,
+            current_alpha: 0,
+            current_normalize: 0,
+        }
+    }
+}
+
+impl<T: Number + RealNumber> Iterator for RidgeRegressionSearchParametersIterator<T> {
+    type Item = RidgeRegressionParameters<T>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_alpha == self.ridge_regression_search_parameters.alpha.len()
+            && self.current_solver == self.ridge_regression_search_parameters.solver.len()
+        {
+            return None;
+        }
+
+        let next = RidgeRegressionParameters {
+            solver: self.ridge_regression_search_parameters.solver[self.current_solver].clone(),
+            alpha: self.ridge_regression_search_parameters.alpha[self.current_alpha],
+            normalize: self.ridge_regression_search_parameters.normalize[self.current_normalize],
+        };
+
+        if self.current_alpha + 1 < self.ridge_regression_search_parameters.alpha.len() {
+            self.current_alpha += 1;
+        } else if self.current_solver + 1 < self.ridge_regression_search_parameters.solver.len() {
+            self.current_alpha = 0;
+            self.current_solver += 1;
+        } else if self.current_normalize + 1
+            < self.ridge_regression_search_parameters.normalize.len()
+        {
+            self.current_alpha = 0;
+            self.current_solver = 0;
+            self.current_normalize += 1;
+        } else {
+            self.current_alpha += 1;
+            self.current_solver += 1;
+            self.current_normalize += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl<T: Number + RealNumber> Default for RidgeRegressionSearchParameters<T> {
+    fn default() -> Self {
+        let default_params = RidgeRegressionParameters::default();
+
+        RidgeRegressionSearchParameters {
+            solver: vec![default_params.solver],
+            alpha: vec![default_params.alpha],
+            normalize: vec![default_params.normalize],
+        }
+    }
+}
+
 /// Ridge regression
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct RidgeRegression<T: RealNumber, M: Matrix<T>> {
-    coefficients: M,
-    intercept: T,
-    _solver: RidgeRegressionSolverName,
+pub struct RidgeRegression<
+    TX: Number + RealNumber,
+    TY: Number,
+    X: Array2<TX> + CholeskyDecomposable<TX> + SVDDecomposable<TX>,
+    Y: Array1<TY>,
+> {
+    coefficients: Option<X>,
+    intercept: Option<TX>,
+    _phantom_ty: PhantomData<TY>,
+    _phantom_y: PhantomData<Y>,
 }
 
-impl<T: RealNumber> RidgeRegressionParameters<T> {
+impl<T: Number + RealNumber> RidgeRegressionParameters<T> {
     /// Regularization parameter.
     pub fn with_alpha(mut self, alpha: T) -> Self {
         self.alpha = alpha;
@@ -117,51 +219,83 @@ impl<T: RealNumber> RidgeRegressionParameters<T> {
     }
 }
 
-impl<T: RealNumber> Default for RidgeRegressionParameters<T> {
+impl<T: Number + RealNumber> Default for RidgeRegressionParameters<T> {
     fn default() -> Self {
         RidgeRegressionParameters {
-            solver: RidgeRegressionSolverName::Cholesky,
-            alpha: T::one(),
+            solver: RidgeRegressionSolverName::default(),
+            alpha: T::from_f64(1.0).unwrap(),
             normalize: true,
         }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> PartialEq for RidgeRegression<T, M> {
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + CholeskyDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > PartialEq for RidgeRegression<TX, TY, X, Y>
+{
     fn eq(&self, other: &Self) -> bool {
-        self.coefficients == other.coefficients
-            && (self.intercept - other.intercept).abs() <= T::epsilon()
+        self.intercept() == other.intercept()
+            && self.coefficients().shape() == other.coefficients().shape()
+            && self
+                .coefficients()
+                .iterator(0)
+                .zip(other.coefficients().iterator(0))
+                .all(|(&a, &b)| (a - b).abs() <= TX::epsilon())
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, RidgeRegressionParameters<T>>
-    for RidgeRegression<T, M>
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + CholeskyDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > SupervisedEstimator<X, Y, RidgeRegressionParameters<TX>> for RidgeRegression<TX, TY, X, Y>
 {
-    fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: RidgeRegressionParameters<T>,
-    ) -> Result<Self, Failed> {
+    fn new() -> Self {
+        Self {
+            coefficients: Option::None,
+            intercept: Option::None,
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: RidgeRegressionParameters<TX>) -> Result<Self, Failed> {
         RidgeRegression::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for RidgeRegression<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + CholeskyDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > Predictor<X, Y> for RidgeRegression<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
+impl<
+        TX: Number + RealNumber,
+        TY: Number,
+        X: Array2<TX> + CholeskyDecomposable<TX> + SVDDecomposable<TX>,
+        Y: Array1<TY>,
+    > RidgeRegression<TX, TY, X, Y>
+{
     /// Fits ridge regression to your data.
     /// * `x` - _NxM_ matrix with _N_ observations and _M_ features in each observation.
     /// * `y` - target values
     /// * `parameters` - other parameters, use `Default::default()` to set parameters to default values.
     pub fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: RidgeRegressionParameters<T>,
-    ) -> Result<RidgeRegression<T, M>, Failed> {
+        x: &X,
+        y: &Y,
+        parameters: RidgeRegressionParameters<TX>,
+    ) -> Result<RidgeRegression<TX, TY, X, Y>, Failed> {
         //w = inv(X^t X + alpha*Id) * X.T y
 
         let (n, p) = x.shape();
@@ -172,11 +306,16 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
             ));
         }
 
-        if y.len() != n {
+        if y.shape() != n {
             return Err(Failed::fit("Number of rows in X should = len(y)"));
         }
 
-        let y_column = M::from_row_vector(y.clone()).transpose();
+        let y_column = X::from_iterator(
+            y.iterator(0).map(|&v| TX::from(v).unwrap()),
+            y.shape(),
+            1,
+            0,
+        );
 
         let (w, b) = if parameters.normalize {
             let (scaled_x, col_mean, col_std) = Self::rescale_x(x)?;
@@ -185,7 +324,7 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
             let mut x_t_x = x_t.matmul(&scaled_x);
 
             for i in 0..p {
-                x_t_x.add_element_mut(i, i, parameters.alpha);
+                x_t_x.add_element_mut((i, i), parameters.alpha);
             }
 
             let mut w = match parameters.solver {
@@ -194,16 +333,16 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
             };
 
             for (i, col_std_i) in col_std.iter().enumerate().take(p) {
-                w.set(i, 0, w.get(i, 0) / *col_std_i);
+                w.set((i, 0), *w.get((i, 0)) / *col_std_i);
             }
 
-            let mut b = T::zero();
+            let mut b = TX::zero();
 
             for (i, col_mean_i) in col_mean.iter().enumerate().take(p) {
-                b += w.get(i, 0) * *col_mean_i;
+                b += *w.get((i, 0)) * *col_mean_i;
             }
 
-            let b = y.mean() - b;
+            let b = TX::from_f64(y.mean_by()).unwrap() - b;
 
             (w, b)
         } else {
@@ -212,7 +351,7 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
             let mut x_t_x = x_t.matmul(x);
 
             for i in 0..p {
-                x_t_x.add_element_mut(i, i, parameters.alpha);
+                x_t_x.add_element_mut((i, i), parameters.alpha);
             }
 
             let w = match parameters.solver {
@@ -220,22 +359,31 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
                 RidgeRegressionSolverName::SVD => x_t_x.svd_solve_mut(x_t_y)?,
             };
 
-            (w, T::zero())
+            (w, TX::zero())
         };
 
         Ok(RidgeRegression {
-            intercept: b,
-            coefficients: w,
-            _solver: parameters.solver,
+            intercept: Some(b),
+            coefficients: Some(w),
+            _phantom_ty: PhantomData,
+            _phantom_y: PhantomData,
         })
     }
 
-    fn rescale_x(x: &M) -> Result<(M, Vec<T>, Vec<T>), Failed> {
-        let col_mean = x.mean(0);
-        let col_std = x.std(0);
+    fn rescale_x(x: &X) -> Result<(X, Vec<TX>, Vec<TX>), Failed> {
+        let col_mean: Vec<TX> = x
+            .mean_by(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
+        let col_std: Vec<TX> = x
+            .std_dev(0)
+            .iter()
+            .map(|&v| TX::from_f64(v).unwrap())
+            .collect();
 
         for (i, col_std_i) in col_std.iter().enumerate() {
-            if (*col_std_i - T::zero()).abs() < T::epsilon() {
+            if (*col_std_i - TX::zero()).abs() < TX::epsilon() {
                 return Err(Failed::fit(&format!(
                     "Cannot rescale constant column {}",
                     i
@@ -250,31 +398,52 @@ impl<T: RealNumber, M: Matrix<T>> RidgeRegression<T, M> {
 
     /// Predict target values from `x`
     /// * `x` - _KxM_ data where _K_ is number of observations and _M_ is number of features.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
         let (nrows, _) = x.shape();
-        let mut y_hat = x.matmul(&self.coefficients);
-        y_hat.add_mut(&M::fill(nrows, 1, self.intercept));
-        Ok(y_hat.transpose().to_row_vector())
+        let mut y_hat = x.matmul(self.coefficients());
+        y_hat.add_mut(&X::fill(nrows, 1, self.intercept.unwrap()));
+        Ok(Y::from_iterator(
+            y_hat.iterator(0).map(|&v| TY::from(v).unwrap()),
+            nrows,
+        ))
     }
 
     /// Get estimates regression coefficients
-    pub fn coefficients(&self) -> &M {
-        &self.coefficients
+    pub fn coefficients(&self) -> &X {
+        self.coefficients.as_ref().unwrap()
     }
 
     /// Get estimate of intercept
-    pub fn intercept(&self) -> T {
-        self.intercept
+    pub fn intercept(&self) -> &TX {
+        self.intercept.as_ref().unwrap()
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
     use crate::metrics::mean_absolute_error;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = RidgeRegressionSearchParameters {
+            alpha: vec![0., 1.],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        assert_eq!(iter.next().unwrap().alpha, 0.);
+        assert_eq!(
+            iter.next().unwrap().solver,
+            RidgeRegressionSolverName::Cholesky
+        );
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn ridge_fit_predict() {
         let x = DenseMatrix::from_2d_array(&[
@@ -330,39 +499,40 @@ mod tests {
         assert!(mean_absolute_error(&y_hat_svd, &y) < 2.0);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    #[cfg(feature = "serde")]
-    fn serde() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
-            &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
-            &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
-            &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
-            &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
-            &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
-            &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
-            &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
-            &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
-            &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
-            &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
-            &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
-            &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
-            &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
-            &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
-            &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
-        ]);
+    // TODO: implement serialization for new DenseMatrix
+    // #[cfg_attr(all(target_arch = "wasm32", not(target_os = "wasi")), wasm_bindgen_test::wasm_bindgen_test)]
+    // #[test]
+    // #[cfg(feature = "serde")]
+    // fn serde() {
+    //     let x = DenseMatrix::from_2d_array(&[
+    //         &[234.289, 235.6, 159.0, 107.608, 1947., 60.323],
+    //         &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
+    //         &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
+    //         &[284.599, 335.1, 165.0, 110.929, 1950., 61.187],
+    //         &[328.975, 209.9, 309.9, 112.075, 1951., 63.221],
+    //         &[346.999, 193.2, 359.4, 113.270, 1952., 63.639],
+    //         &[365.385, 187.0, 354.7, 115.094, 1953., 64.989],
+    //         &[363.112, 357.8, 335.0, 116.219, 1954., 63.761],
+    //         &[397.469, 290.4, 304.8, 117.388, 1955., 66.019],
+    //         &[419.180, 282.2, 285.7, 118.734, 1956., 67.857],
+    //         &[442.769, 293.6, 279.8, 120.445, 1957., 68.169],
+    //         &[444.546, 468.1, 263.7, 121.950, 1958., 66.513],
+    //         &[482.704, 381.3, 255.2, 123.366, 1959., 68.655],
+    //         &[502.601, 393.1, 251.4, 125.368, 1960., 69.564],
+    //         &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
+    //         &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
+    //     ]);
 
-        let y = vec![
-            83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
-            114.2, 115.7, 116.9,
-        ];
+    //     let y = vec![
+    //         83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
+    //         114.2, 115.7, 116.9,
+    //     ];
 
-        let lr = RidgeRegression::fit(&x, &y, Default::default()).unwrap();
+    //     let lr = RidgeRegression::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_lr: RidgeRegression<f64, DenseMatrix<f64>> =
-            serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
+    //     let deserialized_lr: RidgeRegression<f64, f64, DenseMatrix<f64>, Vec<f64>> =
+    //         serde_json::from_str(&serde_json::to_string(&lr).unwrap()).unwrap();
 
-        assert_eq!(lr, deserialized_lr);
-    }
+    //     assert_eq!(lr, deserialized_lr);
+    // }
 }

src/math/distance/euclidian.rs

@@ -1,70 +0,0 @@
-//! # Euclidian Metric Distance
-//!
-//! The Euclidean distance (L2) between two points \\( x \\) and \\( y \\) in n-space is defined as
-//!
-//! \\[ d(x, y) = \sqrt{\sum_{i=1}^n (x-y)^2} \\]
-//!
-//! Example:
-//!
-//! ```
-//! use smartcore::math::distance::Distance;
-//! use smartcore::math::distance::euclidian::Euclidian;
-//!
-//! let x = vec![1., 1.];
-//! let y = vec![2., 2.];
-//!
-//! let l2: f64 = Euclidian{}.distance(&x, &y);
-//! ```
-//!
-//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
-//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
-#[cfg(feature = "serde")]
-use serde::{Deserialize, Serialize};
-
-use crate::math::num::RealNumber;
-
-use super::Distance;
-
-/// Euclidean distance is a measure of the true straight line distance between two points in Euclidean n-space.
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug, Clone)]
-pub struct Euclidian {}
-
-impl Euclidian {
-    #[inline]
-    pub(crate) fn squared_distance<T: RealNumber>(x: &[T], y: &[T]) -> T {
-        if x.len() != y.len() {
-            panic!("Input vector sizes are different.");
-        }
-
-        let mut sum = T::zero();
-        for i in 0..x.len() {
-            let d = x[i] - y[i];
-            sum += d * d;
-        }
-
-        sum
-    }
-}
-
-impl<T: RealNumber> Distance<Vec<T>, T> for Euclidian {
-    fn distance(&self, x: &Vec<T>, y: &Vec<T>) -> T {
-        Euclidian::squared_distance(x, y).sqrt()
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn squared_distance() {
-        let a = vec![1., 2., 3.];
-        let b = vec![4., 5., 6.];
-
-        let l2: f64 = Euclidian {}.distance(&a, &b);
-
-        assert!((l2 - 5.19615242).abs() < 1e-8);
-    }
-}

src/math/distance/manhattan.rs

@@ -1,61 +0,0 @@
-//! # Manhattan Distance
-//!
-//! The Manhattan distance between two points \\(x \in ℝ^n \\) and \\( y \in ℝ^n \\) in n-dimensional space is the sum of the distances in each dimension.
-//!
-//! \\[ d(x, y) = \sum_{i=0}^n \lvert x_i - y_i \rvert \\]
-//!
-//! Example:
-//!
-//! ```
-//! use smartcore::math::distance::Distance;
-//! use smartcore::math::distance::manhattan::Manhattan;
-//!
-//! let x = vec![1., 1.];
-//! let y = vec![2., 2.];
-//!
-//! let l1: f64 = Manhattan {}.distance(&x, &y);
-//! ```
-//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
-//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
-#[cfg(feature = "serde")]
-use serde::{Deserialize, Serialize};
-
-use crate::math::num::RealNumber;
-
-use super::Distance;
-
-/// Manhattan distance
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug, Clone)]
-pub struct Manhattan {}
-
-impl<T: RealNumber> Distance<Vec<T>, T> for Manhattan {
-    fn distance(&self, x: &Vec<T>, y: &Vec<T>) -> T {
-        if x.len() != y.len() {
-            panic!("Input vector sizes are different");
-        }
-
-        let mut dist = T::zero();
-        for i in 0..x.len() {
-            dist += (x[i] - y[i]).abs();
-        }
-
-        dist
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn manhattan_distance() {
-        let a = vec![1., 2., 3.];
-        let b = vec![4., 5., 6.];
-
-        let l1: f64 = Manhattan {}.distance(&a, &b);
-
-        assert!((l1 - 9.0).abs() < 1e-8);
-    }
-}

src/math/distance/mod.rs

@@ -1,65 +0,0 @@
-//! # Collection of Distance Functions
-//!
-//! Many algorithms in machine learning require a measure of distance between data points. Distance metric (or metric) is a function that defines a distance between a pair of point elements of a set.
-//! Formally, the distance can be any metric measure that is defined as \\( d(x, y) \geq 0\\) and follows three conditions:
-//! 1. \\( d(x, y) = 0 \\) if and only \\( x = y \\), positive definiteness
-//! 1. \\( d(x, y) = d(y, x) \\), symmetry
-//! 1. \\( d(x, y) \leq d(x, z) + d(z, y) \\), subadditivity or triangle inequality
-//!
-//! for all \\(x, y, z \in Z \\)
-//!
-//! A good distance metric helps to improve the performance of classification, clustering and information retrieval algorithms significantly.
-//!
-//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
-//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
-
-/// Euclidean Distance is the straight-line distance between two points in Euclidean spacere that presents the shortest distance between these points.
-pub mod euclidian;
-/// Hamming Distance between two strings is the number of positions at which the corresponding symbols are different.
-pub mod hamming;
-/// The Mahalanobis distance is the distance between two points in multivariate space.
-pub mod mahalanobis;
-/// Also known as rectilinear distance, city block distance, taxicab metric.
-pub mod manhattan;
-/// A generalization of both the Euclidean distance and the Manhattan distance.
-pub mod minkowski;
-
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
-
-/// Distance metric, a function that calculates distance between two points
-pub trait Distance<T, F: RealNumber>: Clone {
-    /// Calculates distance between _a_ and _b_
-    fn distance(&self, a: &T, b: &T) -> F;
-}
-
-/// Multitude of distance metric functions
-pub struct Distances {}
-
-impl Distances {
-    /// Euclidian distance, see [`Euclidian`](euclidian/index.html)
-    pub fn euclidian() -> euclidian::Euclidian {
-        euclidian::Euclidian {}
-    }
-
-    /// Minkowski distance, see [`Minkowski`](minkowski/index.html)
-    /// * `p` - function order. Should be >= 1
-    pub fn minkowski(p: u16) -> minkowski::Minkowski {
-        minkowski::Minkowski { p }
-    }
-
-    /// Manhattan distance, see [`Manhattan`](manhattan/index.html)
-    pub fn manhattan() -> manhattan::Manhattan {
-        manhattan::Manhattan {}
-    }
-
-    /// Hamming distance, see [`Hamming`](hamming/index.html)
-    pub fn hamming() -> hamming::Hamming {
-        hamming::Hamming {}
-    }
-
-    /// Mahalanobis distance, see [`Mahalanobis`](mahalanobis/index.html)
-    pub fn mahalanobis<T: RealNumber, M: Matrix<T>>(data: &M) -> mahalanobis::Mahalanobis<T, M> {
-        mahalanobis::Mahalanobis::new(data)
-    }
-}

src/math/mod.rs

@@ -1,4 +0,0 @@
-/// Multitude of distance metrics are defined here
-pub mod distance;
-pub mod num;
-pub(crate) mod vector;

src/math/vector.rs

@@ -1,42 +0,0 @@
-use crate::math::num::RealNumber;
-use std::collections::HashMap;
-
-use crate::linalg::BaseVector;
-pub trait RealNumberVector<T: RealNumber> {
-    fn unique_with_indices(&self) -> (Vec<T>, Vec<usize>);
-}
-
-impl<T: RealNumber, V: BaseVector<T>> RealNumberVector<T> for V {
-    fn unique_with_indices(&self) -> (Vec<T>, Vec<usize>) {
-        let mut unique = self.to_vec();
-        unique.sort_by(|a, b| a.partial_cmp(b).unwrap());
-        unique.dedup();
-
-        let mut index = HashMap::with_capacity(unique.len());
-        for (i, u) in unique.iter().enumerate() {
-            index.insert(u.to_i64().unwrap(), i);
-        }
-
-        let mut unique_index = Vec::with_capacity(self.len());
-        for idx in 0..self.len() {
-            unique_index.push(index[&self.get(idx).to_i64().unwrap()]);
-        }
-
-        (unique, unique_index)
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-    #[test]
-    fn unique_with_indices() {
-        let v1 = vec![0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 4.0];
-        assert_eq!(
-            (vec!(0.0, 1.0, 2.0, 4.0), vec!(0, 0, 1, 1, 2, 0, 3)),
-            v1.unique_with_indices()
-        );
-    }
-}

src/metrics/accuracy.rs

@@ -8,10 +8,20 @@
 //!
 //! ```
 //! use smartcore::metrics::accuracy::Accuracy;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![0., 2., 1., 3.];
 //! let y_true: Vec<f64> = vec![0., 1., 2., 3.];
 //!
-//! let score: f64 = Accuracy {}.get_score(&y_pred, &y_true);
+//! let score: f64 = Accuracy::new().get_score( &y_true, &y_pred);
+//! ```
+//! With integers:
+//! ```
+//! use smartcore::metrics::accuracy::Accuracy;
+//! use smartcore::metrics::Metrics;
+//! let y_pred: Vec<i64> = vec![0, 2, 1, 3];
+//! let y_true: Vec<i64> = vec![0, 1, 2, 3];
+//!
+//! let score: f64 = Accuracy::new().get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
@@ -19,37 +29,53 @@
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+use std::marker::PhantomData;
+
+use crate::metrics::Metrics;
 
 /// Accuracy metric.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct Accuracy {}
+pub struct Accuracy<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl Accuracy {
+impl<T: Number> Metrics<T> for Accuracy<T> {
+    /// create a typed object to call Accuracy functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Function that calculated accuracy score.
     /// * `y_true` - cround truth (correct) labels
     /// * `y_pred` - predicted labels, as returned by a classifier.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
-        let n = y_true.len();
+        let n = y_true.shape();
 
-        let mut positive = 0;
+        let mut positive: i32 = 0;
         for i in 0..n {
-            if y_true.get(i) == y_pred.get(i) {
+            if *y_true.get(i) == *y_pred.get(i) {
                 positive += 1;
             }
         }
 
-        T::from_i64(positive).unwrap() / T::from_usize(n).unwrap()
+        positive as f64 / n as f64
     }
 }
 
@@ -57,16 +83,35 @@ impl Accuracy {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
-    fn accuracy() {
+    fn accuracy_float() {
         let y_pred: Vec<f64> = vec![0., 2., 1., 3.];
         let y_true: Vec<f64> = vec![0., 1., 2., 3.];
 
-        let score1: f64 = Accuracy {}.get_score(&y_pred, &y_true);
-        let score2: f64 = Accuracy {}.get_score(&y_true, &y_true);
+        let score1: f64 = Accuracy::<f64>::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Accuracy::<f64>::new().get_score(&y_true, &y_true);
 
         assert!((score1 - 0.5).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);
     }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn accuracy_int() {
+        let y_pred: Vec<i32> = vec![0, 2, 1, 3];
+        let y_true: Vec<i32> = vec![0, 1, 2, 3];
+
+        let score1: f64 = Accuracy::<i32>::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Accuracy::<i32>::new().get_score(&y_true, &y_true);
+
+        assert_eq!(score1, 0.5);
+        assert_eq!(score2, 1.0);
+    }
 }

src/metrics/auc.rs

@@ -2,16 +2,17 @@
 //! Computes the area under the receiver operating characteristic (ROC) curve that is equal to the probability that a classifier will rank a
 //! randomly chosen positive instance higher than a randomly chosen negative one.
 //!
-//! SmartCore calculates ROC AUC from Wilcoxon or Mann-Whitney U test.
+//! `smartcore` calculates ROC AUC from Wilcoxon or Mann-Whitney U test.
 //!
 //! Example:
 //! ```
 //! use smartcore::metrics::auc::AUC;
+//! use smartcore::metrics::Metrics;
 //!
 //! let y_true: Vec<f64> = vec![0., 0., 1., 1.];
 //! let y_pred: Vec<f64> = vec![0.1, 0.4, 0.35, 0.8];
 //!
-//! let score1: f64 = AUC {}.get_score(&y_true, &y_pred);
+//! let score1: f64 = AUC::new().get_score(&y_true, &y_pred);
 //! ```
 //!
 //! ## References:
@@ -20,32 +21,48 @@
 //! * ["The ROC-AUC and the Mann-Whitney U-test", Haupt, J.](https://johaupt.github.io/roc-auc/model%20evaluation/Area_under_ROC_curve.html)
 #![allow(non_snake_case)]
 
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::algorithm::sort::quick_sort::QuickArgSort;
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, ArrayView1};
+use crate::numbers::floatnum::FloatNumber;
+
+use crate::metrics::Metrics;
 
 /// Area Under the Receiver Operating Characteristic Curve (ROC AUC)
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct AUC {}
+pub struct AUC<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl AUC {
+impl<T: FloatNumber + PartialOrd> Metrics<T> for AUC<T> {
+    /// create a typed object to call AUC functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// AUC score.
-    /// * `y_true` - cround truth (correct) labels.
-    /// * `y_pred_probabilities` - probability estimates, as returned by a classifier.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred_prob: &V) -> T {
+    /// * `y_true` - ground truth (correct) labels.
+    /// * `y_pred_prob` - probability estimates, as returned by a classifier.
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred_prob: &dyn ArrayView1<T>) -> f64 {
         let mut pos = T::zero();
         let mut neg = T::zero();
 
-        let n = y_true.len();
+        let n = y_true.shape();
 
         for i in 0..n {
-            if y_true.get(i) == T::zero() {
+            if y_true.get(i) == &T::zero() {
                 neg += T::one();
-            } else if y_true.get(i) == T::one() {
+            } else if y_true.get(i) == &T::one() {
                 pos += T::one();
             } else {
                 panic!(
@@ -55,21 +72,22 @@ impl AUC {
             }
         }
 
-        let mut y_pred = y_pred_prob.to_vec();
+        let y_pred: Vec<T> =
+            Array1::<T>::from_iterator(y_pred_prob.iterator(0).copied(), y_pred_prob.shape());
+        // TODO: try to use `crate::algorithm::sort::quick_sort` here
+        let label_idx: Vec<usize> = y_pred.argsort();
 
-        let label_idx = y_pred.quick_argsort_mut();
-
-        let mut rank = vec![T::zero(); n];
+        let mut rank = vec![0f64; n];
         let mut i = 0;
         while i < n {
-            if i == n - 1 || y_pred[i] != y_pred[i + 1] {
-                rank[i] = T::from_usize(i + 1).unwrap();
+            if i == n - 1 || y_pred.get(i) != y_pred.get(i + 1) {
+                rank[i] = (i + 1) as f64;
             } else {
                 let mut j = i + 1;
-                while j < n && y_pred[j] == y_pred[i] {
+                while j < n && y_pred.get(j) == y_pred.get(i) {
                     j += 1;
                 }
-                let r = T::from_usize(i + 1 + j).unwrap() / T::two();
+                let r = (i + 1 + j) as f64 / 2f64;
                 for rank_k in rank.iter_mut().take(j).skip(i) {
                     *rank_k = r;
                 }
@@ -78,14 +96,16 @@ impl AUC {
             i += 1;
         }
 
-        let mut auc = T::zero();
+        let mut auc = 0f64;
         for i in 0..n {
-            if y_true.get(label_idx[i]) == T::one() {
+            if y_true.get(label_idx[i]) == &T::one() {
                 auc += rank[i];
             }
         }
+        let pos = pos.to_f64().unwrap();
+        let neg = neg.to_f64().unwrap();
 
-        (auc - (pos * (pos + T::one()) / T::two())) / (pos * neg)
+        (auc - (pos * (pos + 1f64) / 2f64)) / (pos * neg)
     }
 }
 
@@ -93,14 +113,17 @@ impl AUC {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn auc() {
         let y_true: Vec<f64> = vec![0., 0., 1., 1.];
         let y_pred: Vec<f64> = vec![0.1, 0.4, 0.35, 0.8];
 
-        let score1: f64 = AUC {}.get_score(&y_true, &y_pred);
-        let score2: f64 = AUC {}.get_score(&y_true, &y_true);
+        let score1: f64 = AUC::new().get_score(&y_true, &y_pred);
+        let score2: f64 = AUC::new().get_score(&y_true, &y_true);
 
         assert!((score1 - 0.75).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);

src/metrics/cluster_hcv.rs

@@ -1,41 +1,85 @@
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
 use crate::metrics::cluster_helpers::*;
+use crate::numbers::basenum::Number;
+
+use crate::metrics::Metrics;
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
 /// Homogeneity, completeness and V-Measure scores.
-pub struct HCVScore {}
+pub struct HCVScore<T> {
+    _phantom: PhantomData<T>,
+    homogeneity: Option<f64>,
+    completeness: Option<f64>,
+    v_measure: Option<f64>,
+}
 
-impl HCVScore {
-    /// Computes Homogeneity, completeness and V-Measure scores at once.
-    /// * `labels_true` - ground truth class labels to be used as a reference.
-    /// * `labels_pred` - cluster labels to evaluate.    
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(
-        &self,
-        labels_true: &V,
-        labels_pred: &V,
-    ) -> (T, T, T) {
-        let labels_true = labels_true.to_vec();
-        let labels_pred = labels_pred.to_vec();
-        let entropy_c = entropy(&labels_true);
-        let entropy_k = entropy(&labels_pred);
-        let contingency = contingency_matrix(&labels_true, &labels_pred);
-        let mi: T = mutual_info_score(&contingency);
+impl<T: Number + Ord> HCVScore<T> {
+    /// return homogenity score
+    pub fn homogeneity(&self) -> Option<f64> {
+        self.homogeneity
+    }
+    /// return completeness score
+    pub fn completeness(&self) -> Option<f64> {
+        self.completeness
+    }
+    /// return v_measure score
+    pub fn v_measure(&self) -> Option<f64> {
+        self.v_measure
+    }
+    /// run computation for measures
+    pub fn compute(&mut self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) {
+        let entropy_c: Option<f64> = entropy(y_true);
+        let entropy_k: Option<f64> = entropy(y_pred);
+        let contingency = contingency_matrix(y_true, y_pred);
+        let mi = mutual_info_score(&contingency);
 
-        let homogeneity = entropy_c.map(|e| mi / e).unwrap_or_else(T::one);
-        let completeness = entropy_k.map(|e| mi / e).unwrap_or_else(T::one);
+        let homogeneity = entropy_c.map(|e| mi / e).unwrap_or(0f64);
+        let completeness = entropy_k.map(|e| mi / e).unwrap_or(0f64);
 
-        let v_measure_score = if homogeneity + completeness == T::zero() {
-            T::zero()
+        let v_measure_score = if homogeneity + completeness == 0f64 {
+            0f64
         } else {
-            T::two() * homogeneity * completeness / (T::one() * homogeneity + completeness)
+            2.0f64 * homogeneity * completeness / (1.0f64 * homogeneity + completeness)
         };
 
-        (homogeneity, completeness, v_measure_score)
+        self.homogeneity = Some(homogeneity);
+        self.completeness = Some(completeness);
+        self.v_measure = Some(v_measure_score);
+    }
+}
+
+impl<T: Number + Ord> Metrics<T> for HCVScore<T> {
+    /// create a typed object to call HCVScore functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+            homogeneity: Option::None,
+            completeness: Option::None,
+            v_measure: Option::None,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+            homogeneity: Option::None,
+            completeness: Option::None,
+            v_measure: Option::None,
+        }
+    }
+    /// Computes Homogeneity, completeness and V-Measure scores at once.
+    /// * `y_true` - ground truth class labels to be used as a reference.
+    /// * `y_pred` - cluster labels to evaluate.    
+    fn get_score(&self, _y_true: &dyn ArrayView1<T>, _y_pred: &dyn ArrayView1<T>) -> f64 {
+        // this functions should not be used for this struct
+        // use homogeneity(), completeness(), v_measure()
+        // TODO: implement Metrics -> Result<T, Failed>
+        0f64
     }
 }
 
@@ -43,15 +87,19 @@ impl HCVScore {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn homogeneity_score() {
-        let v1 = vec![0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 4.0];
-        let v2 = vec![1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0];
-        let scores = HCVScore {}.get_score(&v1, &v2);
+        let v1 = vec![0, 0, 1, 1, 2, 0, 4];
+        let v2 = vec![1, 0, 0, 0, 0, 1, 0];
+        let mut scores = HCVScore::new();
+        scores.compute(&v1, &v2);
 
-        assert!((0.2548f32 - scores.0).abs() < 1e-4);
-        assert!((0.5440f32 - scores.1).abs() < 1e-4);
-        assert!((0.3471f32 - scores.2).abs() < 1e-4);
+        assert!((0.2548 - scores.homogeneity.unwrap() as f64).abs() < 1e-4);
+        assert!((0.5440 - scores.completeness.unwrap() as f64).abs() < 1e-4);
+        assert!((0.3471 - scores.v_measure.unwrap() as f64).abs() < 1e-4);
     }
 }

src/metrics/cluster_helpers.rs

@@ -1,12 +1,12 @@
 #![allow(clippy::ptr_arg)]
 use std::collections::HashMap;
 
-use crate::math::num::RealNumber;
-use crate::math::vector::RealNumberVector;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
 
-pub fn contingency_matrix<T: RealNumber>(
-    labels_true: &Vec<T>,
-    labels_pred: &Vec<T>,
+pub fn contingency_matrix<T: Number + Ord, V: ArrayView1<T> + ?Sized>(
+    labels_true: &V,
+    labels_pred: &V,
 ) -> Vec<Vec<usize>> {
     let (classes, class_idx) = labels_true.unique_with_indices();
     let (clusters, cluster_idx) = labels_pred.unique_with_indices();
@@ -24,28 +24,30 @@ pub fn contingency_matrix<T: RealNumber>(
     contingency_matrix
 }
 
-pub fn entropy<T: RealNumber>(data: &[T]) -> Option<T> {
-    let mut bincounts = HashMap::with_capacity(data.len());
+pub fn entropy<T: Number + Ord, V: ArrayView1<T> + ?Sized>(data: &V) -> Option<f64> {
+    let mut bincounts = HashMap::with_capacity(data.shape());
 
-    for e in data.iter() {
+    for e in data.iterator(0) {
         let k = e.to_i64().unwrap();
         bincounts.insert(k, bincounts.get(&k).unwrap_or(&0) + 1);
     }
 
-    let mut entropy = T::zero();
-    let sum = T::from_usize(bincounts.values().sum()).unwrap();
+    let mut entropy = 0f64;
+    let sum: i64 = bincounts.values().sum();
 
     for &c in bincounts.values() {
         if c > 0 {
-            let pi = T::from_usize(c).unwrap();
-            entropy -= (pi / sum) * (pi.ln() - sum.ln());
+            let pi = c as f64;
+            let pi_ln = pi.ln();
+            let sum_ln = (sum as f64).ln();
+            entropy -= (pi / sum as f64) * (pi_ln - sum_ln);
         }
     }
 
     Some(entropy)
 }
 
-pub fn mutual_info_score<T: RealNumber>(contingency: &[Vec<usize>]) -> T {
+pub fn mutual_info_score(contingency: &[Vec<usize>]) -> f64 {
     let mut contingency_sum = 0;
     let mut pi = vec![0; contingency.len()];
     let mut pj = vec![0; contingency[0].len()];
@@ -64,48 +66,50 @@ pub fn mutual_info_score<T: RealNumber>(contingency: &[Vec<usize>]) -> T {
         }
     }
 
-    let contingency_sum = T::from_usize(contingency_sum).unwrap();
+    let contingency_sum = contingency_sum as f64;
     let contingency_sum_ln = contingency_sum.ln();
-    let pi_sum_l = T::from_usize(pi.iter().sum()).unwrap().ln();
-    let pj_sum_l = T::from_usize(pj.iter().sum()).unwrap().ln();
+    let pi_sum: usize = pi.iter().sum();
+    let pj_sum: usize = pj.iter().sum();
+    let pi_sum_l = (pi_sum as f64).ln();
+    let pj_sum_l = (pj_sum as f64).ln();
 
-    let log_contingency_nm: Vec<T> = nz_val
+    let log_contingency_nm: Vec<f64> = nz_val.iter().map(|v| (*v as f64).ln()).collect();
+    let contingency_nm: Vec<f64> = nz_val
         .iter()
-        .map(|v| T::from_usize(*v).unwrap().ln())
-        .collect();
-    let contingency_nm: Vec<T> = nz_val
-        .iter()
-        .map(|v| T::from_usize(*v).unwrap() / contingency_sum)
+        .map(|v| (*v as f64) / contingency_sum)
         .collect();
     let outer: Vec<usize> = nzx
         .iter()
         .zip(nzy.iter())
         .map(|(&x, &y)| pi[x] * pj[y])
         .collect();
-    let log_outer: Vec<T> = outer
+    let log_outer: Vec<f64> = outer
         .iter()
-        .map(|&o| -T::from_usize(o).unwrap().ln() + pi_sum_l + pj_sum_l)
+        .map(|&o| -(o as f64).ln() + pi_sum_l + pj_sum_l)
         .collect();
 
-    let mut result = T::zero();
+    let mut result = 0f64;
 
     for i in 0..log_outer.len() {
         result += (contingency_nm[i] * (log_contingency_nm[i] - contingency_sum_ln))
             + contingency_nm[i] * log_outer[i]
     }
 
-    result.max(T::zero())
+    result.max(0f64)
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn contingency_matrix_test() {
-        let v1 = vec![0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 4.0];
-        let v2 = vec![1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0];
+        let v1 = vec![0, 0, 1, 1, 2, 0, 4];
+        let v2 = vec![1, 0, 0, 0, 0, 1, 0];
 
         assert_eq!(
             vec!(vec!(1, 2), vec!(2, 0), vec!(1, 0), vec!(1, 0)),
@@ -113,20 +117,26 @@ mod tests {
         );
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn entropy_test() {
-        let v1 = vec![0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 4.0];
+        let v1 = vec![0, 0, 1, 1, 2, 0, 4];
 
-        assert!((1.2770f32 - entropy(&v1).unwrap()).abs() < 1e-4);
+        assert!((1.2770 - entropy(&v1).unwrap() as f64).abs() < 1e-4);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn mutual_info_score_test() {
-        let v1 = vec![0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 4.0];
-        let v2 = vec![1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0];
-        let s: f32 = mutual_info_score(&contingency_matrix(&v1, &v2));
+        let v1 = vec![0, 0, 1, 1, 2, 0, 4];
+        let v2 = vec![1, 0, 0, 0, 0, 1, 0];
+        let s = mutual_info_score(&contingency_matrix(&v1, &v2));
 
         assert!((0.3254 - s).abs() < 1e-4);
     }

src/metrics/distance/euclidian.rs

@@ -0,0 +1,92 @@
+//! # Euclidian Metric Distance
+//!
+//! The Euclidean distance (L2) between two points \\( x \\) and \\( y \\) in n-space is defined as
+//!
+//! \\[ d(x, y) = \sqrt{\sum_{i=1}^n (x-y)^2} \\]
+//!
+//! Example:
+//!
+//! ```
+//! use smartcore::metrics::distance::Distance;
+//! use smartcore::metrics::distance::euclidian::Euclidian;
+//!
+//! let x = vec![1., 1.];
+//! let y = vec![2., 2.];
+//!
+//! let l2: f64 = Euclidian::new().distance(&x, &y);
+//! ```
+//!
+//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
+//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+#[cfg(feature = "serde")]
+use serde::{Deserialize, Serialize};
+use std::marker::PhantomData;
+
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+
+use super::Distance;
+
+/// Euclidean distance is a measure of the true straight line distance between two points in Euclidean n-space.
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct Euclidian<T> {
+    _t: PhantomData<T>,
+}
+
+impl<T: Number> Default for Euclidian<T> {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: Number> Euclidian<T> {
+    /// instatiate the initial structure
+    pub fn new() -> Euclidian<T> {
+        Euclidian { _t: PhantomData }
+    }
+
+    /// return sum of squared distances
+    #[inline]
+    pub(crate) fn squared_distance<A: ArrayView1<T>>(x: &A, y: &A) -> f64 {
+        if x.shape() != y.shape() {
+            panic!("Input vector sizes are different.");
+        }
+
+        let sum: f64 = x
+            .iterator(0)
+            .zip(y.iterator(0))
+            .map(|(&a, &b)| {
+                let r = a - b;
+                (r * r).to_f64().unwrap()
+            })
+            .sum();
+
+        sum
+    }
+}
+
+impl<T: Number, A: ArrayView1<T>> Distance<A> for Euclidian<T> {
+    fn distance(&self, x: &A, y: &A) -> f64 {
+        Euclidian::squared_distance(x, y).sqrt()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn squared_distance() {
+        let a = vec![1, 2, 3];
+        let b = vec![4, 5, 6];
+
+        let l2: f64 = Euclidian::new().distance(&a, &b);
+
+        assert!((l2 - 5.19615242).abs() < 1e-8);
+    }
+}

src/metrics/distance/hamming.rs

@@ -6,13 +6,13 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::math::distance::Distance;
-//! use smartcore::math::distance::hamming::Hamming;
+//! use smartcore::metrics::distance::Distance;
+//! use smartcore::metrics::distance::hamming::Hamming;
 //!
 //! let a = vec![1, 0, 0, 1, 0, 0, 1];
 //! let b = vec![1, 1, 0, 0, 1, 0, 1];
 //!
-//! let h: f64 = Hamming {}.distance(&a, &b);
+//! let h: f64 = Hamming::new().distance(&a, &b);
 //!
 //! ```
 //!
@@ -21,30 +21,48 @@
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
-
-use crate::math::num::RealNumber;
+use std::marker::PhantomData;
 
 use super::Distance;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
 
 /// While comparing two integer-valued vectors of equal length, Hamming distance is the number of bit positions in which the two bits are different
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct Hamming {}
+pub struct Hamming<T: Number> {
+    _t: PhantomData<T>,
+}
 
-impl<T: PartialEq, F: RealNumber> Distance<Vec<T>, F> for Hamming {
-    fn distance(&self, x: &Vec<T>, y: &Vec<T>) -> F {
-        if x.len() != y.len() {
+impl<T: Number> Hamming<T> {
+    /// instatiate the initial structure
+    pub fn new() -> Hamming<T> {
+        Hamming { _t: PhantomData }
+    }
+}
+
+impl<T: Number> Default for Hamming<T> {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: Number, A: ArrayView1<T>> Distance<A> for Hamming<T> {
+    fn distance(&self, x: &A, y: &A) -> f64 {
+        if x.shape() != y.shape() {
             panic!("Input vector sizes are different");
         }
 
-        let mut dist = 0;
-        for i in 0..x.len() {
-            if x[i] != y[i] {
-                dist += 1;
-            }
-        }
+        let dist: usize = x
+            .iterator(0)
+            .zip(y.iterator(0))
+            .map(|(a, b)| match a != b {
+                true => 1,
+                false => 0,
+            })
+            .sum();
 
-        F::from_i64(dist).unwrap() / F::from_usize(x.len()).unwrap()
+        dist as f64 / x.shape() as f64
     }
 }
 
@@ -52,13 +70,16 @@ impl<T: PartialEq, F: RealNumber> Distance<Vec<T>, F> for Hamming {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn hamming_distance() {
         let a = vec![1, 0, 0, 1, 0, 0, 1];
         let b = vec![1, 1, 0, 0, 1, 0, 1];
 
-        let h: f64 = Hamming {}.distance(&a, &b);
+        let h: f64 = Hamming::new().distance(&a, &b);
 
         assert!((h - 0.42857142).abs() < 1e-8);
     }

src/metrics/distance/mahalanobis.rs

@@ -14,9 +14,10 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
-//! use smartcore::math::distance::Distance;
-//! use smartcore::math::distance::mahalanobis::Mahalanobis;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
+//! use smartcore::linalg::basic::arrays::ArrayView2;
+//! use smartcore::metrics::distance::Distance;
+//! use smartcore::metrics::distance::mahalanobis::Mahalanobis;
 //!
 //! let data = DenseMatrix::from_2d_array(&[
 //!                   &[64., 580., 29.],
@@ -26,7 +27,7 @@
 //!                   &[73., 600., 55.],
 //! ]);
 //!
-//! let a = data.column_mean();
+//! let a = data.mean_by(0);
 //! let b = vec![66., 640., 44.];
 //!
 //! let mahalanobis = Mahalanobis::new(&data);
@@ -42,85 +43,89 @@
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
 #![allow(non_snake_case)]
 
-use std::marker::PhantomData;
-
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
-
-use crate::math::num::RealNumber;
+use std::marker::PhantomData;
 
 use super::Distance;
-use crate::linalg::Matrix;
+use crate::linalg::basic::arrays::{Array, Array2, ArrayView1};
+use crate::linalg::basic::matrix::DenseMatrix;
+use crate::linalg::traits::lu::LUDecomposable;
+use crate::numbers::basenum::Number;
 
 /// Mahalanobis distance.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct Mahalanobis<T: RealNumber, M: Matrix<T>> {
+pub struct Mahalanobis<T: Number, M: Array2<f64>> {
     /// covariance matrix of the dataset
     pub sigma: M,
     /// inverse of the covariance matrix
     pub sigmaInv: M,
-    t: PhantomData<T>,
+    _t: PhantomData<T>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> Mahalanobis<T, M> {
+impl<T: Number, M: Array2<f64> + LUDecomposable<f64>> Mahalanobis<T, M> {
     /// Constructs new instance of `Mahalanobis` from given dataset
     /// * `data` - a matrix of _NxM_ where _N_ is number of observations and _M_ is number of attributes
-    pub fn new(data: &M) -> Mahalanobis<T, M> {
-        let sigma = data.cov();
+    pub fn new<X: Array2<T>>(data: &X) -> Mahalanobis<T, M> {
+        let (_, m) = data.shape();
+        let mut sigma = M::zeros(m, m);
+        data.cov(&mut sigma);
         let sigmaInv = sigma.lu().and_then(|lu| lu.inverse()).unwrap();
         Mahalanobis {
             sigma,
             sigmaInv,
-            t: PhantomData,
+            _t: PhantomData,
         }
     }
 
     /// Constructs new instance of `Mahalanobis` from given covariance matrix
     /// * `cov` - a covariance matrix
-    pub fn new_from_covariance(cov: &M) -> Mahalanobis<T, M> {
+    pub fn new_from_covariance<X: Array2<f64> + LUDecomposable<f64>>(cov: &X) -> Mahalanobis<T, X> {
         let sigma = cov.clone();
         let sigmaInv = sigma.lu().and_then(|lu| lu.inverse()).unwrap();
         Mahalanobis {
             sigma,
             sigmaInv,
-            t: PhantomData,
+            _t: PhantomData,
         }
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Distance<Vec<T>, T> for Mahalanobis<T, M> {
-    fn distance(&self, x: &Vec<T>, y: &Vec<T>) -> T {
+impl<T: Number, A: ArrayView1<T>> Distance<A> for Mahalanobis<T, DenseMatrix<f64>> {
+    fn distance(&self, x: &A, y: &A) -> f64 {
         let (nrows, ncols) = self.sigma.shape();
-        if x.len() != nrows {
+        if x.shape() != nrows {
             panic!(
                 "Array x[{}] has different dimension with Sigma[{}][{}].",
-                x.len(),
+                x.shape(),
                 nrows,
                 ncols
             );
         }
 
-        if y.len() != nrows {
+        if y.shape() != nrows {
             panic!(
                 "Array y[{}] has different dimension with Sigma[{}][{}].",
-                y.len(),
+                y.shape(),
                 nrows,
                 ncols
             );
         }
 
-        let n = x.len();
-        let mut z = vec![T::zero(); n];
-        for i in 0..n {
-            z[i] = x[i] - y[i];
-        }
+        let n = x.shape();
+
+        let z: Vec<f64> = x
+            .iterator(0)
+            .zip(y.iterator(0))
+            .map(|(&a, &b)| (a - b).to_f64().unwrap())
+            .collect();
 
         // np.dot(np.dot((a-b),VI),(a-b).T)
-        let mut s = T::zero();
+        let mut s = 0f64;
         for j in 0..n {
             for i in 0..n {
-                s += self.sigmaInv.get(i, j) * z[i] * z[j];
+                s += *self.sigmaInv.get((i, j)) * z[i] * z[j];
             }
         }
 
@@ -131,9 +136,13 @@ impl<T: RealNumber, M: Matrix<T>> Distance<Vec<T>, T> for Mahalanobis<T, M> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::arrays::ArrayView2;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn mahalanobis_distance() {
         let data = DenseMatrix::from_2d_array(&[
@@ -144,7 +153,7 @@ mod tests {
             &[73., 600., 55.],
         ]);
 
-        let a = data.column_mean();
+        let a = data.mean_by(0);
         let b = vec![66., 640., 44.];
 
         let mahalanobis = Mahalanobis::new(&data);

src/metrics/distance/manhattan.rs

@@ -0,0 +1,82 @@
+//! # Manhattan Distance
+//!
+//! The Manhattan distance between two points \\(x \in ℝ^n \\) and \\( y \in ℝ^n \\) in n-dimensional space is the sum of the distances in each dimension.
+//!
+//! \\[ d(x, y) = \sum_{i=0}^n \lvert x_i - y_i \rvert \\]
+//!
+//! Example:
+//!
+//! ```
+//! use smartcore::metrics::distance::Distance;
+//! use smartcore::metrics::distance::manhattan::Manhattan;
+//!
+//! let x = vec![1., 1.];
+//! let y = vec![2., 2.];
+//!
+//! let l1: f64 = Manhattan::new().distance(&x, &y);
+//! ```
+//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
+//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+#[cfg(feature = "serde")]
+use serde::{Deserialize, Serialize};
+use std::marker::PhantomData;
+
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+
+use super::Distance;
+
+/// Manhattan distance
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct Manhattan<T: Number> {
+    _t: PhantomData<T>,
+}
+
+impl<T: Number> Manhattan<T> {
+    /// instatiate the initial structure
+    pub fn new() -> Manhattan<T> {
+        Manhattan { _t: PhantomData }
+    }
+}
+
+impl<T: Number> Default for Manhattan<T> {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: Number, A: ArrayView1<T>> Distance<A> for Manhattan<T> {
+    fn distance(&self, x: &A, y: &A) -> f64 {
+        if x.shape() != y.shape() {
+            panic!("Input vector sizes are different");
+        }
+
+        let dist: f64 = x
+            .iterator(0)
+            .zip(y.iterator(0))
+            .map(|(&a, &b)| (a - b).to_f64().unwrap().abs())
+            .sum();
+
+        dist
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn manhattan_distance() {
+        let a = vec![1., 2., 3.];
+        let b = vec![4., 5., 6.];
+
+        let l1: f64 = Manhattan::new().distance(&a, &b);
+
+        assert!((l1 - 9.0).abs() < 1e-8);
+    }
+}

src/metrics/distance/minkowski.rs

@@ -8,14 +8,14 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::math::distance::Distance;
-//! use smartcore::math::distance::minkowski::Minkowski;
+//! use smartcore::metrics::distance::Distance;
+//! use smartcore::metrics::distance::minkowski::Minkowski;
 //!
 //! let x = vec![1., 1.];
 //! let y = vec![2., 2.];
 //!
-//! let l1: f64 = Minkowski { p: 1 }.distance(&x, &y);
-//! let l2: f64 = Minkowski { p: 2 }.distance(&x, &y);
+//! let l1: f64 = Minkowski::new(1).distance(&x, &y);
+//! let l2: f64 = Minkowski::new(2).distance(&x, &y);
 //!
 //! ```
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
@@ -23,37 +23,47 @@
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
+use std::marker::PhantomData;
 
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
 
 use super::Distance;
 
 /// Defines the Minkowski distance of order `p`
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct Minkowski {
+pub struct Minkowski<T: Number> {
     /// order, integer
     pub p: u16,
+    _t: PhantomData<T>,
 }
 
-impl<T: RealNumber> Distance<Vec<T>, T> for Minkowski {
-    fn distance(&self, x: &Vec<T>, y: &Vec<T>) -> T {
-        if x.len() != y.len() {
+impl<T: Number> Minkowski<T> {
+    /// instatiate the initial structure
+    pub fn new(p: u16) -> Minkowski<T> {
+        Minkowski { p, _t: PhantomData }
+    }
+}
+
+impl<T: Number, A: ArrayView1<T>> Distance<A> for Minkowski<T> {
+    fn distance(&self, x: &A, y: &A) -> f64 {
+        if x.shape() != y.shape() {
             panic!("Input vector sizes are different");
         }
         if self.p < 1 {
             panic!("p must be at least 1");
         }
 
-        let mut dist = T::zero();
-        let p_t = T::from_u16(self.p).unwrap();
+        let p_t = self.p as f64;
 
-        for i in 0..x.len() {
-            let d = (x[i] - y[i]).abs();
-            dist += d.powf(p_t);
-        }
+        let dist: f64 = x
+            .iterator(0)
+            .zip(y.iterator(0))
+            .map(|(&a, &b)| (a - b).to_f64().unwrap().abs().powf(p_t))
+            .sum();
 
-        dist.powf(T::one() / p_t)
+        dist.powf(1f64 / p_t)
     }
 }
 
@@ -61,15 +71,18 @@ impl<T: RealNumber> Distance<Vec<T>, T> for Minkowski {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn minkowski_distance() {
         let a = vec![1., 2., 3.];
         let b = vec![4., 5., 6.];
 
-        let l1: f64 = Minkowski { p: 1 }.distance(&a, &b);
-        let l2: f64 = Minkowski { p: 2 }.distance(&a, &b);
-        let l3: f64 = Minkowski { p: 3 }.distance(&a, &b);
+        let l1: f64 = Minkowski::new(1).distance(&a, &b);
+        let l2: f64 = Minkowski::new(2).distance(&a, &b);
+        let l3: f64 = Minkowski::new(3).distance(&a, &b);
 
         assert!((l1 - 9.0).abs() < 1e-8);
         assert!((l2 - 5.19615242).abs() < 1e-8);
@@ -82,6 +95,6 @@ mod tests {
         let a = vec![1., 2., 3.];
         let b = vec![4., 5., 6.];
 
-        let _: f64 = Minkowski { p: 0 }.distance(&a, &b);
+        let _: f64 = Minkowski::new(0).distance(&a, &b);
     }
 }

src/metrics/distance/mod.rs

@@ -0,0 +1,116 @@
+//! # Collection of Distance Functions
+//!
+//! Many algorithms in machine learning require a measure of distance between data points. Distance metric (or metric) is a function that defines a distance between a pair of point elements of a set.
+//! Formally, the distance can be any metric measure that is defined as \\( d(x, y) \geq 0\\) and follows three conditions:
+//! 1. \\( d(x, y) = 0 \\) if and only \\( x = y \\), positive definiteness
+//! 1. \\( d(x, y) = d(y, x) \\), symmetry
+//! 1. \\( d(x, y) \leq d(x, z) + d(z, y) \\), subadditivity or triangle inequality
+//!
+//! for all \\(x, y, z \in Z \\)
+//!
+//! A good distance metric helps to improve the performance of classification, clustering and information retrieval algorithms significantly.
+//!
+//! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
+//! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+
+/// Euclidean Distance is the straight-line distance between two points in Euclidean spacere that presents the shortest distance between these points.
+pub mod euclidian;
+/// Hamming Distance between two strings is the number of positions at which the corresponding symbols are different.
+pub mod hamming;
+/// The Mahalanobis distance is the distance between two points in multivariate space.
+pub mod mahalanobis;
+/// Also known as rectilinear distance, city block distance, taxicab metric.
+pub mod manhattan;
+/// A generalization of both the Euclidean distance and the Manhattan distance.
+pub mod minkowski;
+
+use std::cmp::{Eq, Ordering, PartialOrd};
+
+use crate::linalg::basic::arrays::Array2;
+use crate::linalg::traits::lu::LUDecomposable;
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
+
+#[cfg(feature = "serde")]
+use serde::{Deserialize, Serialize};
+
+/// Distance metric, a function that calculates distance between two points
+pub trait Distance<T>: Clone {
+    /// Calculates distance between _a_ and _b_
+    fn distance(&self, a: &T, b: &T) -> f64;
+}
+
+/// Multitude of distance metric functions
+pub struct Distances {}
+
+impl Distances {
+    /// Euclidian distance, see [`Euclidian`](euclidian/index.html)
+    pub fn euclidian<T: Number>() -> euclidian::Euclidian<T> {
+        euclidian::Euclidian::new()
+    }
+
+    /// Minkowski distance, see [`Minkowski`](minkowski/index.html)
+    /// * `p` - function order. Should be >= 1
+    pub fn minkowski<T: Number>(p: u16) -> minkowski::Minkowski<T> {
+        minkowski::Minkowski::new(p)
+    }
+
+    /// Manhattan distance, see [`Manhattan`](manhattan/index.html)
+    pub fn manhattan<T: Number>() -> manhattan::Manhattan<T> {
+        manhattan::Manhattan::new()
+    }
+
+    /// Hamming distance, see [`Hamming`](hamming/index.html)
+    pub fn hamming<T: Number>() -> hamming::Hamming<T> {
+        hamming::Hamming::new()
+    }
+
+    /// Mahalanobis distance, see [`Mahalanobis`](mahalanobis/index.html)
+    pub fn mahalanobis<T: Number, M: Array2<T>, C: Array2<f64> + LUDecomposable<f64>>(
+        data: &M,
+    ) -> mahalanobis::Mahalanobis<T, C> {
+        mahalanobis::Mahalanobis::new(data)
+    }
+}
+
+///
+/// ### Pairwise dissimilarities.
+///
+/// Representing distances as pairwise dissimilarities, so to build a
+/// graph of closest neighbours. This representation can be reused for
+/// different implementations
+/// (initially used in this library for [FastPair](algorithm/neighbour/fastpair)).
+/// The edge of the subgraph is defined by `PairwiseDistance`.
+/// The calling algorithm can store a list of distances as
+/// a list of these structures.
+///
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone, Copy)]
+pub struct PairwiseDistance<T: RealNumber> {
+    /// index of the vector in the original `Matrix` or list
+    pub node: usize,
+
+    /// index of the closest neighbor in the original `Matrix` or same list
+    pub neighbour: Option<usize>,
+
+    /// measure of distance, according to the algorithm distance function
+    /// if the distance is None, the edge has value "infinite" or max distance
+    /// each algorithm has to match
+    pub distance: Option<T>,
+}
+
+impl<T: RealNumber> Eq for PairwiseDistance<T> {}
+
+impl<T: RealNumber> PartialEq for PairwiseDistance<T> {
+    fn eq(&self, other: &Self) -> bool {
+        self.node == other.node
+            && self.neighbour == other.neighbour
+            && self.distance == other.distance
+    }
+}
+
+impl<T: RealNumber> PartialOrd for PairwiseDistance<T> {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        self.distance.partial_cmp(&other.distance)
+    }
+}

src/metrics/f1.rs

@@ -10,48 +10,71 @@
 //!
 //! ```
 //! use smartcore::metrics::f1::F1;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![0., 0., 1., 1., 1., 1.];
 //! let y_true: Vec<f64> = vec![0., 1., 1., 0., 1., 0.];
 //!
-//! let score: f64 = F1 {beta: 1.0}.get_score(&y_pred, &y_true);
+//! let beta = 1.0; // beta default is equal 1.0 anyway
+//! let score: f64 = F1::new_with(beta).get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
 use crate::metrics::precision::Precision;
 use crate::metrics::recall::Recall;
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+use crate::numbers::realnum::RealNumber;
+
+use crate::metrics::Metrics;
 
 /// F-measure
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct F1<T: RealNumber> {
+pub struct F1<T> {
     /// a positive real factor
-    pub beta: T,
+    pub beta: f64,
+    _phantom: PhantomData<T>,
 }
 
-impl<T: RealNumber> F1<T> {
+impl<T: Number + RealNumber + FloatNumber> Metrics<T> for F1<T> {
+    fn new() -> Self {
+        let beta: f64 = 1f64;
+        Self {
+            beta,
+            _phantom: PhantomData,
+        }
+    }
+    /// create a typed object to call Recall functions
+    fn new_with(beta: f64) -> Self {
+        Self {
+            beta,
+            _phantom: PhantomData,
+        }
+    }
     /// Computes F1 score
     /// * `y_true` - cround truth (correct) labels.
     /// * `y_pred` - predicted labels, as returned by a classifier.
-    pub fn get_score<V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
         let beta2 = self.beta * self.beta;
 
-        let p = Precision {}.get_score(y_true, y_pred);
-        let r = Recall {}.get_score(y_true, y_pred);
+        let p = Precision::new().get_score(y_true, y_pred);
+        let r = Recall::new().get_score(y_true, y_pred);
 
-        (T::one() + beta2) * (p * r) / (beta2 * p + r)
+        (1f64 + beta2) * (p * r) / ((beta2 * p) + r)
     }
 }
 
@@ -59,14 +82,21 @@ impl<T: RealNumber> F1<T> {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn f1() {
         let y_pred: Vec<f64> = vec![0., 0., 1., 1., 1., 1.];
         let y_true: Vec<f64> = vec![0., 1., 1., 0., 1., 0.];
 
-        let score1: f64 = F1 { beta: 1.0 }.get_score(&y_pred, &y_true);
-        let score2: f64 = F1 { beta: 1.0 }.get_score(&y_true, &y_true);
+        let beta = 1.0;
+        let score1: f64 = F1::new_with(beta).get_score(&y_true, &y_pred);
+        let score2: f64 = F1::new_with(beta).get_score(&y_true, &y_true);
+
+        println!("{:?}", score1);
+        println!("{:?}", score2);
 
         assert!((score1 - 0.57142857).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);

src/metrics/mean_absolute_error.rs

@@ -10,45 +10,65 @@
 //!
 //! ```
 //! use smartcore::metrics::mean_absolute_error::MeanAbsoluteError;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![3., -0.5, 2., 7.];
 //! let y_true: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 //!
-//! let mse: f64 = MeanAbsoluteError {}.get_score(&y_pred, &y_true);
+//! let mse: f64 = MeanAbsoluteError::new().get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+
+use crate::metrics::Metrics;
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
 /// Mean Absolute Error
-pub struct MeanAbsoluteError {}
+pub struct MeanAbsoluteError<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl MeanAbsoluteError {
+impl<T: Number + FloatNumber> Metrics<T> for MeanAbsoluteError<T> {
+    /// create a typed object to call MeanAbsoluteError functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Computes mean absolute error
     /// * `y_true` - Ground truth (correct) target values.
     /// * `y_pred` - Estimated target values.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
-        let n = y_true.len();
-        let mut ras = T::zero();
+        let n = y_true.shape();
+        let mut ras: T = T::zero();
         for i in 0..n {
-            ras += (y_true.get(i) - y_pred.get(i)).abs();
+            let res: T = *y_true.get(i) - *y_pred.get(i);
+            ras += res.abs();
         }
 
-        ras / T::from_usize(n).unwrap()
+        ras.to_f64().unwrap() / n as f64
     }
 }
 
@@ -56,14 +76,17 @@ impl MeanAbsoluteError {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn mean_absolute_error() {
         let y_true: Vec<f64> = vec![3., -0.5, 2., 7.];
         let y_pred: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 
-        let score1: f64 = MeanAbsoluteError {}.get_score(&y_pred, &y_true);
-        let score2: f64 = MeanAbsoluteError {}.get_score(&y_true, &y_true);
+        let score1: f64 = MeanAbsoluteError::new().get_score(&y_true, &y_pred);
+        let score2: f64 = MeanAbsoluteError::new().get_score(&y_true, &y_true);
 
         assert!((score1 - 0.5).abs() < 1e-8);
         assert!((score2 - 0.0).abs() < 1e-8);

src/metrics/mean_squared_error.rs

@@ -10,45 +10,65 @@
 //!
 //! ```
 //! use smartcore::metrics::mean_squared_error::MeanSquareError;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![3., -0.5, 2., 7.];
 //! let y_true: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 //!
-//! let mse: f64 = MeanSquareError {}.get_score(&y_pred, &y_true);
+//! let mse: f64 = MeanSquareError::new().get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+
+use crate::metrics::Metrics;
 
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
 /// Mean Squared Error
-pub struct MeanSquareError {}
+pub struct MeanSquareError<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl MeanSquareError {
+impl<T: Number + FloatNumber> Metrics<T> for MeanSquareError<T> {
+    /// create a typed object to call MeanSquareError functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Computes mean squared error
     /// * `y_true` - Ground truth (correct) target values.
     /// * `y_pred` - Estimated target values.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
-        let n = y_true.len();
+        let n = y_true.shape();
         let mut rss = T::zero();
         for i in 0..n {
-            rss += (y_true.get(i) - y_pred.get(i)).square();
+            let res = *y_true.get(i) - *y_pred.get(i);
+            rss += res * res;
         }
 
-        rss / T::from_usize(n).unwrap()
+        rss.to_f64().unwrap() / n as f64
     }
 }
 
@@ -56,14 +76,17 @@ impl MeanSquareError {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn mean_squared_error() {
         let y_true: Vec<f64> = vec![3., -0.5, 2., 7.];
         let y_pred: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 
-        let score1: f64 = MeanSquareError {}.get_score(&y_pred, &y_true);
-        let score2: f64 = MeanSquareError {}.get_score(&y_true, &y_true);
+        let score1: f64 = MeanSquareError::new().get_score(&y_true, &y_pred);
+        let score2: f64 = MeanSquareError::new().get_score(&y_true, &y_true);
 
         assert!((score1 - 0.375).abs() < 1e-8);
         assert!((score2 - 0.0).abs() < 1e-8);

src/metrics/mod.rs

@@ -4,7 +4,7 @@
 //! In a feedback loop you build your model first, then you get feedback from metrics, improve it and repeat until your model achieve desirable performance.
 //! Evaluation metrics helps to explain the performance of a model and compare models based on an objective criterion.
 //!
-//! Choosing the right metric is crucial while evaluating machine learning models. In SmartCore you will find metrics for these classes of ML models:
+//! Choosing the right metric is crucial while evaluating machine learning models. In `smartcore` you will find metrics for these classes of ML models:
 //!
 //! * [Classification metrics](struct.ClassificationMetrics.html)
 //! * [Regression metrics](struct.RegressionMetrics.html)
@@ -12,7 +12,7 @@
 //!
 //! Example:
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::linear::logistic_regression::LogisticRegression;
 //! use smartcore::metrics::*;
 //!
@@ -38,26 +38,29 @@
 //!             &[6.6, 2.9, 4.6, 1.3],
 //!             &[5.2, 2.7, 3.9, 1.4],
 //!   ]);
-//! let y: Vec<f64> = vec![
-//!             0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
+//! let y: Vec<i8> = vec![
+//!             0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 //!   ];
 //!
 //! let lr = LogisticRegression::fit(&x, &y, Default::default()).unwrap();
 //!
 //! let y_hat = lr.predict(&x).unwrap();
 //!
-//! let acc = ClassificationMetrics::accuracy().get_score(&y, &y_hat);
+//! let acc = ClassificationMetricsOrd::accuracy().get_score(&y, &y_hat);
 //! // or
 //! let acc = accuracy(&y, &y_hat);
 //! ```
 
 /// Accuracy score.
 pub mod accuracy;
-/// Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.
+// TODO: reimplement AUC
+// /// Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.
 pub mod auc;
 /// Compute the homogeneity, completeness and V-Measure scores.
 pub mod cluster_hcv;
 pub(crate) mod cluster_helpers;
+/// Multitude of distance metrics are defined here
+pub mod distance;
 /// F1 score, also known as balanced F-score or F-measure.
 pub mod f1;
 /// Mean absolute error regression loss.
@@ -71,150 +74,225 @@ pub mod r2;
 /// Computes the recall.
 pub mod recall;
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, ArrayView1};
+use crate::numbers::basenum::Number;
+use crate::numbers::floatnum::FloatNumber;
+use crate::numbers::realnum::RealNumber;
+
+use std::marker::PhantomData;
+
+/// A trait to be implemented by all metrics
+pub trait Metrics<T> {
+    /// instantiate a new Metrics trait-object
+    /// <https://doc.rust-lang.org/error-index.html#E0038>
+    fn new() -> Self
+    where
+        Self: Sized;
+    /// used to instantiate metric with a paramenter
+    fn new_with(_parameter: f64) -> Self
+    where
+        Self: Sized;
+    /// compute score realated to this metric
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64;
+}
 
 /// Use these metrics to compare classification models.
-pub struct ClassificationMetrics {}
+pub struct ClassificationMetrics<T> {
+    phantom: PhantomData<T>,
+}
+
+/// Use these metrics to compare classification models for
+/// numbers that require `Ord`.
+pub struct ClassificationMetricsOrd<T> {
+    phantom: PhantomData<T>,
+}
 
 /// Metrics for regression models.
-pub struct RegressionMetrics {}
+pub struct RegressionMetrics<T> {
+    phantom: PhantomData<T>,
+}
 
 /// Cluster metrics.
-pub struct ClusterMetrics {}
-
-impl ClassificationMetrics {
-    /// Accuracy score, see [accuracy](accuracy/index.html).
-    pub fn accuracy() -> accuracy::Accuracy {
-        accuracy::Accuracy {}
-    }
+pub struct ClusterMetrics<T> {
+    phantom: PhantomData<T>,
+}
 
+impl<T: Number + RealNumber + FloatNumber> ClassificationMetrics<T> {
     /// Recall, see [recall](recall/index.html).
-    pub fn recall() -> recall::Recall {
-        recall::Recall {}
+    pub fn recall() -> recall::Recall<T> {
+        recall::Recall::new()
     }
 
     /// Precision, see [precision](precision/index.html).
-    pub fn precision() -> precision::Precision {
-        precision::Precision {}
+    pub fn precision() -> precision::Precision<T> {
+        precision::Precision::new()
     }
 
     /// F1 score, also known as balanced F-score or F-measure, see [F1](f1/index.html).
-    pub fn f1<T: RealNumber>(beta: T) -> f1::F1<T> {
-        f1::F1 { beta }
+    pub fn f1(beta: f64) -> f1::F1<T> {
+        f1::F1::new_with(beta)
     }
 
     /// Area Under the Receiver Operating Characteristic Curve (ROC AUC), see [AUC](auc/index.html).
-    pub fn roc_auc_score() -> auc::AUC {
-        auc::AUC {}
+    pub fn roc_auc_score() -> auc::AUC<T> {
+        auc::AUC::<T>::new()
     }
 }
 
-impl RegressionMetrics {
+impl<T: Number + Ord> ClassificationMetricsOrd<T> {
+    /// Accuracy score, see [accuracy](accuracy/index.html).
+    pub fn accuracy() -> accuracy::Accuracy<T> {
+        accuracy::Accuracy::new()
+    }
+}
+
+impl<T: Number + FloatNumber> RegressionMetrics<T> {
     /// Mean squared error, see [mean squared error](mean_squared_error/index.html).
-    pub fn mean_squared_error() -> mean_squared_error::MeanSquareError {
-        mean_squared_error::MeanSquareError {}
+    pub fn mean_squared_error() -> mean_squared_error::MeanSquareError<T> {
+        mean_squared_error::MeanSquareError::new()
     }
 
     /// Mean absolute error, see [mean absolute error](mean_absolute_error/index.html).
-    pub fn mean_absolute_error() -> mean_absolute_error::MeanAbsoluteError {
-        mean_absolute_error::MeanAbsoluteError {}
+    pub fn mean_absolute_error() -> mean_absolute_error::MeanAbsoluteError<T> {
+        mean_absolute_error::MeanAbsoluteError::new()
     }
 
     /// Coefficient of determination (R2), see [R2](r2/index.html).
-    pub fn r2() -> r2::R2 {
-        r2::R2 {}
+    pub fn r2() -> r2::R2<T> {
+        r2::R2::<T>::new()
     }
 }
 
-impl ClusterMetrics {
+impl<T: Number + Ord> ClusterMetrics<T> {
     /// Homogeneity and completeness and V-Measure scores at once.
-    pub fn hcv_score() -> cluster_hcv::HCVScore {
-        cluster_hcv::HCVScore {}
+    pub fn hcv_score() -> cluster_hcv::HCVScore<T> {
+        cluster_hcv::HCVScore::<T>::new()
     }
 }
 
 /// Function that calculated accuracy score, see [accuracy](accuracy/index.html).
 /// * `y_true` - cround truth (correct) labels
 /// * `y_pred` - predicted labels, as returned by a classifier.
-pub fn accuracy<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    ClassificationMetrics::accuracy().get_score(y_true, y_pred)
+pub fn accuracy<T: Number + Ord, V: ArrayView1<T>>(y_true: &V, y_pred: &V) -> f64 {
+    let obj = ClassificationMetricsOrd::<T>::accuracy();
+    obj.get_score(y_true, y_pred)
 }
 
 /// Calculated recall score, see [recall](recall/index.html)
 /// * `y_true` - cround truth (correct) labels.
 /// * `y_pred` - predicted labels, as returned by a classifier.
-pub fn recall<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    ClassificationMetrics::recall().get_score(y_true, y_pred)
+pub fn recall<T: Number + RealNumber + FloatNumber, V: ArrayView1<T>>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    let obj = ClassificationMetrics::<T>::recall();
+    obj.get_score(y_true, y_pred)
 }
 
 /// Calculated precision score, see [precision](precision/index.html).
 /// * `y_true` - cround truth (correct) labels.
 /// * `y_pred` - predicted labels, as returned by a classifier.
-pub fn precision<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    ClassificationMetrics::precision().get_score(y_true, y_pred)
+pub fn precision<T: Number + RealNumber + FloatNumber, V: ArrayView1<T>>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    let obj = ClassificationMetrics::<T>::precision();
+    obj.get_score(y_true, y_pred)
 }
 
 /// Computes F1 score, see [F1](f1/index.html).
 /// * `y_true` - cround truth (correct) labels.
 /// * `y_pred` - predicted labels, as returned by a classifier.
-pub fn f1<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V, beta: T) -> T {
-    ClassificationMetrics::f1(beta).get_score(y_true, y_pred)
+pub fn f1<T: Number + RealNumber + FloatNumber, V: ArrayView1<T>>(
+    y_true: &V,
+    y_pred: &V,
+    beta: f64,
+) -> f64 {
+    let obj = ClassificationMetrics::<T>::f1(beta);
+    obj.get_score(y_true, y_pred)
 }
 
 /// AUC score, see [AUC](auc/index.html).
 /// * `y_true` - cround truth (correct) labels.
 /// * `y_pred_probabilities` - probability estimates, as returned by a classifier.
-pub fn roc_auc_score<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred_probabilities: &V) -> T {
-    ClassificationMetrics::roc_auc_score().get_score(y_true, y_pred_probabilities)
+pub fn roc_auc_score<
+    T: Number + RealNumber + FloatNumber + PartialOrd,
+    V: ArrayView1<T> + Array1<T> + Array1<T>,
+>(
+    y_true: &V,
+    y_pred_probabilities: &V,
+) -> f64 {
+    let obj = ClassificationMetrics::<T>::roc_auc_score();
+    obj.get_score(y_true, y_pred_probabilities)
 }
 
 /// Computes mean squared error, see [mean squared error](mean_squared_error/index.html).
 /// * `y_true` - Ground truth (correct) target values.
 /// * `y_pred` - Estimated target values.
-pub fn mean_squared_error<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    RegressionMetrics::mean_squared_error().get_score(y_true, y_pred)
+pub fn mean_squared_error<T: Number + FloatNumber, V: ArrayView1<T>>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    RegressionMetrics::<T>::mean_squared_error().get_score(y_true, y_pred)
 }
 
 /// Computes mean absolute error, see [mean absolute error](mean_absolute_error/index.html).
 /// * `y_true` - Ground truth (correct) target values.
 /// * `y_pred` - Estimated target values.
-pub fn mean_absolute_error<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    RegressionMetrics::mean_absolute_error().get_score(y_true, y_pred)
+pub fn mean_absolute_error<T: Number + FloatNumber, V: ArrayView1<T>>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    RegressionMetrics::<T>::mean_absolute_error().get_score(y_true, y_pred)
 }
 
 /// Computes R2 score, see [R2](r2/index.html).
 /// * `y_true` - Ground truth (correct) target values.
 /// * `y_pred` - Estimated target values.
-pub fn r2<T: RealNumber, V: BaseVector<T>>(y_true: &V, y_pred: &V) -> T {
-    RegressionMetrics::r2().get_score(y_true, y_pred)
+pub fn r2<T: Number + FloatNumber, V: ArrayView1<T>>(y_true: &V, y_pred: &V) -> f64 {
+    RegressionMetrics::<T>::r2().get_score(y_true, y_pred)
 }
 
 /// Homogeneity metric of a cluster labeling given a ground truth (range is between 0.0 and 1.0).
 /// A cluster result satisfies homogeneity if all of its clusters contain only data points which are members of a single class.
 /// * `labels_true` - ground truth class labels to be used as a reference.
 /// * `labels_pred` - cluster labels to evaluate.
-pub fn homogeneity_score<T: RealNumber, V: BaseVector<T>>(labels_true: &V, labels_pred: &V) -> T {
-    ClusterMetrics::hcv_score()
-        .get_score(labels_true, labels_pred)
-        .0
+pub fn homogeneity_score<
+    T: Number + FloatNumber + RealNumber + Ord,
+    V: ArrayView1<T> + Array1<T>,
+>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    let mut obj = ClusterMetrics::<T>::hcv_score();
+    obj.compute(y_true, y_pred);
+    obj.homogeneity().unwrap()
 }
 
 ///
 /// Completeness metric of a cluster labeling given a ground truth (range is between 0.0 and 1.0).
 /// * `labels_true` - ground truth class labels to be used as a reference.
 /// * `labels_pred` - cluster labels to evaluate.
-pub fn completeness_score<T: RealNumber, V: BaseVector<T>>(labels_true: &V, labels_pred: &V) -> T {
-    ClusterMetrics::hcv_score()
-        .get_score(labels_true, labels_pred)
-        .1
+pub fn completeness_score<
+    T: Number + FloatNumber + RealNumber + Ord,
+    V: ArrayView1<T> + Array1<T>,
+>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    let mut obj = ClusterMetrics::<T>::hcv_score();
+    obj.compute(y_true, y_pred);
+    obj.completeness().unwrap()
 }
 
 /// The harmonic mean between homogeneity and completeness.
 /// * `labels_true` - ground truth class labels to be used as a reference.
 /// * `labels_pred` - cluster labels to evaluate.
-pub fn v_measure_score<T: RealNumber, V: BaseVector<T>>(labels_true: &V, labels_pred: &V) -> T {
-    ClusterMetrics::hcv_score()
-        .get_score(labels_true, labels_pred)
-        .2
+pub fn v_measure_score<T: Number + FloatNumber + RealNumber + Ord, V: ArrayView1<T> + Array1<T>>(
+    y_true: &V,
+    y_pred: &V,
+) -> f64 {
+    let mut obj = ClusterMetrics::<T>::hcv_score();
+    obj.compute(y_true, y_pred);
+    obj.v_measure().unwrap()
 }

src/metrics/precision.rs

@@ -10,66 +10,84 @@
 //!
 //! ```
 //! use smartcore::metrics::precision::Precision;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![0., 1., 1., 0.];
 //! let y_true: Vec<f64> = vec![0., 0., 1., 1.];
 //!
-//! let score: f64 = Precision {}.get_score(&y_pred, &y_true);
+//! let score: f64 = Precision::new().get_score(&y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+use std::collections::HashSet;
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::realnum::RealNumber;
+
+use crate::metrics::Metrics;
 
 /// Precision metric.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct Precision {}
+pub struct Precision<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl Precision {
+impl<T: RealNumber> Metrics<T> for Precision<T> {
+    /// create a typed object to call Precision functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Calculated precision score
-    /// * `y_true` - cround truth (correct) labels.
+    /// * `y_true` - ground truth (correct) labels.
     /// * `y_pred` - predicted labels, as returned by a classifier.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
+        let mut classes = HashSet::new();
+        for i in 0..y_true.shape() {
+            classes.insert(y_true.get(i).to_f64_bits());
+        }
+        let classes = classes.len();
+
         let mut tp = 0;
-        let mut p = 0;
-        let n = y_true.len();
-        for i in 0..n {
-            if y_true.get(i) != T::zero() && y_true.get(i) != T::one() {
-                panic!(
-                    "Precision can only be applied to binary classification: {}",
-                    y_true.get(i)
-                );
-            }
-
-            if y_pred.get(i) != T::zero() && y_pred.get(i) != T::one() {
-                panic!(
-                    "Precision can only be applied to binary classification: {}",
-                    y_pred.get(i)
-                );
-            }
-
-            if y_pred.get(i) == T::one() {
-                p += 1;
-
-                if y_true.get(i) == T::one() {
+        let mut fp = 0;
+        for i in 0..y_true.shape() {
+            if y_pred.get(i) == y_true.get(i) {
+                if classes == 2 {
+                    if *y_true.get(i) == T::one() {
+                        tp += 1;
+                    }
+                } else {
                     tp += 1;
                 }
+            } else if classes == 2 {
+                if *y_true.get(i) == T::one() {
+                    fp += 1;
+                }
+            } else {
+                fp += 1;
             }
         }
 
-        T::from_i64(tp).unwrap() / T::from_i64(p).unwrap()
+        tp as f64 / (tp as f64 + fp as f64)
     }
 }
 
@@ -77,16 +95,41 @@ impl Precision {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn precision() {
         let y_true: Vec<f64> = vec![0., 1., 1., 0.];
         let y_pred: Vec<f64> = vec![0., 0., 1., 1.];
 
-        let score1: f64 = Precision {}.get_score(&y_pred, &y_true);
-        let score2: f64 = Precision {}.get_score(&y_pred, &y_pred);
+        let score1: f64 = Precision::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Precision::new().get_score(&y_pred, &y_pred);
 
         assert!((score1 - 0.5).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);
+
+        let y_true: Vec<f64> = vec![0., 1., 1., 0., 1., 0.];
+        let y_pred: Vec<f64> = vec![0., 0., 1., 1., 1., 1.];
+
+        let score3: f64 = Precision::new().get_score(&y_true, &y_pred);
+        assert!((score3 - 0.6666666666).abs() < 1e-8);
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn precision_multiclass() {
+        let y_true: Vec<f64> = vec![0., 0., 0., 1., 1., 1., 2., 2., 2.];
+        let y_pred: Vec<f64> = vec![0., 1., 2., 0., 1., 2., 0., 1., 2.];
+
+        let score1: f64 = Precision::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Precision::new().get_score(&y_pred, &y_pred);
+
+        assert!((score1 - 0.333333333).abs() < 1e-8);
+        assert!((score2 - 1.0).abs() < 1e-8);
     }
 }

src/metrics/r2.rs

@@ -10,59 +10,70 @@
 //!
 //! ```
 //! use smartcore::metrics::mean_absolute_error::MeanAbsoluteError;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![3., -0.5, 2., 7.];
 //! let y_true: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 //!
-//! let mse: f64 = MeanAbsoluteError {}.get_score(&y_pred, &y_true);
+//! let mse: f64 = MeanAbsoluteError::new().get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::basenum::Number;
+
+use crate::metrics::Metrics;
 
 /// Coefficient of Determination (R2)
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct R2 {}
+pub struct R2<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl R2 {
+impl<T: Number> Metrics<T> for R2<T> {
+    /// create a typed object to call R2 functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Computes R2 score
     /// * `y_true` - Ground truth (correct) target values.
     /// * `y_pred` - Estimated target values.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
-        let n = y_true.len();
-
-        let mut mean = T::zero();
-
-        for i in 0..n {
-            mean += y_true.get(i);
-        }
-
-        mean /= T::from_usize(n).unwrap();
+        let n = y_true.shape();
 
+        let mean: f64 = y_true.mean_by();
         let mut ss_tot = T::zero();
         let mut ss_res = T::zero();
 
         for i in 0..n {
-            let y_i = y_true.get(i);
-            let f_i = y_pred.get(i);
-            ss_tot += (y_i - mean).square();
-            ss_res += (y_i - f_i).square();
+            let y_i = *y_true.get(i);
+            let f_i = *y_pred.get(i);
+            ss_tot += (y_i - T::from(mean).unwrap()) * (y_i - T::from(mean).unwrap());
+            ss_res += (y_i - f_i) * (y_i - f_i);
         }
 
-        T::one() - (ss_res / ss_tot)
+        (T::one() - ss_res / ss_tot).to_f64().unwrap()
     }
 }
 
@@ -70,14 +81,17 @@ impl R2 {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn r2() {
         let y_true: Vec<f64> = vec![3., -0.5, 2., 7.];
         let y_pred: Vec<f64> = vec![2.5, 0.0, 2., 8.];
 
-        let score1: f64 = R2 {}.get_score(&y_true, &y_pred);
-        let score2: f64 = R2 {}.get_score(&y_true, &y_true);
+        let score1: f64 = R2::new().get_score(&y_true, &y_pred);
+        let score2: f64 = R2::new().get_score(&y_true, &y_true);
 
         assert!((score1 - 0.948608137).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);

src/metrics/recall.rs

@@ -10,66 +10,85 @@
 //!
 //! ```
 //! use smartcore::metrics::recall::Recall;
+//! use smartcore::metrics::Metrics;
 //! let y_pred: Vec<f64> = vec![0., 1., 1., 0.];
 //! let y_true: Vec<f64> = vec![0., 0., 1., 1.];
 //!
-//! let score: f64 = Recall {}.get_score(&y_pred, &y_true);
+//! let score: f64 = Recall::new().get_score( &y_true, &y_pred);
 //! ```
 //!
 //! <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
 //! <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+
+use std::collections::HashSet;
+use std::convert::TryInto;
+use std::marker::PhantomData;
+
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
-use crate::linalg::BaseVector;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::ArrayView1;
+use crate::numbers::realnum::RealNumber;
+
+use crate::metrics::Metrics;
 
 /// Recall metric.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug)]
-pub struct Recall {}
+pub struct Recall<T> {
+    _phantom: PhantomData<T>,
+}
 
-impl Recall {
+impl<T: RealNumber> Metrics<T> for Recall<T> {
+    /// create a typed object to call Recall functions
+    fn new() -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
+    fn new_with(_parameter: f64) -> Self {
+        Self {
+            _phantom: PhantomData,
+        }
+    }
     /// Calculated recall score
     /// * `y_true` - cround truth (correct) labels.
     /// * `y_pred` - predicted labels, as returned by a classifier.
-    pub fn get_score<T: RealNumber, V: BaseVector<T>>(&self, y_true: &V, y_pred: &V) -> T {
-        if y_true.len() != y_pred.len() {
+    fn get_score(&self, y_true: &dyn ArrayView1<T>, y_pred: &dyn ArrayView1<T>) -> f64 {
+        if y_true.shape() != y_pred.shape() {
             panic!(
                 "The vector sizes don't match: {} != {}",
-                y_true.len(),
-                y_pred.len()
+                y_true.shape(),
+                y_pred.shape()
             );
         }
 
+        let mut classes = HashSet::new();
+        for i in 0..y_true.shape() {
+            classes.insert(y_true.get(i).to_f64_bits());
+        }
+        let classes: i64 = classes.len().try_into().unwrap();
+
         let mut tp = 0;
-        let mut p = 0;
-        let n = y_true.len();
-        for i in 0..n {
-            if y_true.get(i) != T::zero() && y_true.get(i) != T::one() {
-                panic!(
-                    "Recall can only be applied to binary classification: {}",
-                    y_true.get(i)
-                );
-            }
-
-            if y_pred.get(i) != T::zero() && y_pred.get(i) != T::one() {
-                panic!(
-                    "Recall can only be applied to binary classification: {}",
-                    y_pred.get(i)
-                );
-            }
-
-            if y_true.get(i) == T::one() {
-                p += 1;
-
-                if y_pred.get(i) == T::one() {
+        let mut fne = 0;
+        for i in 0..y_true.shape() {
+            if y_pred.get(i) == y_true.get(i) {
+                if classes == 2 {
+                    if *y_true.get(i) == T::one() {
+                        tp += 1;
+                    }
+                } else {
                     tp += 1;
                 }
+            } else if classes == 2 {
+                if *y_true.get(i) != T::one() {
+                    fne += 1;
+                }
+            } else {
+                fne += 1;
             }
         }
-
-        T::from_i64(tp).unwrap() / T::from_i64(p).unwrap()
+        tp as f64 / (tp as f64 + fne as f64)
     }
 }
 
@@ -77,16 +96,41 @@ impl Recall {
 mod tests {
     use super::*;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn recall() {
         let y_true: Vec<f64> = vec![0., 1., 1., 0.];
         let y_pred: Vec<f64> = vec![0., 0., 1., 1.];
 
-        let score1: f64 = Recall {}.get_score(&y_pred, &y_true);
-        let score2: f64 = Recall {}.get_score(&y_pred, &y_pred);
+        let score1: f64 = Recall::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Recall::new().get_score(&y_pred, &y_pred);
 
         assert!((score1 - 0.5).abs() < 1e-8);
         assert!((score2 - 1.0).abs() < 1e-8);
+
+        let y_true: Vec<f64> = vec![0., 1., 1., 0., 1., 0.];
+        let y_pred: Vec<f64> = vec![0., 0., 1., 1., 1., 1.];
+
+        let score3: f64 = Recall::new().get_score(&y_true, &y_pred);
+        assert!((score3 - 0.5).abs() < 1e-8);
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
+    #[test]
+    fn recall_multiclass() {
+        let y_true: Vec<f64> = vec![0., 0., 0., 1., 1., 1., 2., 2., 2.];
+        let y_pred: Vec<f64> = vec![0., 1., 2., 0., 1., 2., 0., 1., 2.];
+
+        let score1: f64 = Recall::new().get_score(&y_true, &y_pred);
+        let score2: f64 = Recall::new().get_score(&y_pred, &y_pred);
+
+        assert!((score1 - 0.333333333).abs() < 1e-8);
+        assert!((score2 - 1.0).abs() < 1e-8);
     }
 }

src/model_selection/hyper_tuning/grid_search.rs

@@ -0,0 +1,239 @@
+// TODO: missing documentation
+
+use crate::{
+    api::{Predictor, SupervisedEstimator},
+    error::{Failed, FailedError},
+    linalg::basic::arrays::{Array2, Array1},
+    numbers::realnum::RealNumber,
+    numbers::basenum::Number,
+};
+
+use crate::model_selection::{cross_validate, BaseKFold, CrossValidationResult};
+
+/// Parameters for GridSearchCV
+#[derive(Debug)]
+pub struct GridSearchCVParameters<
+    T: Number,
+    M: Array2<T>,
+    C: Clone,
+    I: Iterator<Item = C>,
+    E: Predictor<M, M::RowVector>,
+    F: Fn(&M, &M::RowVector, C) -> Result<E, Failed>,
+    K: BaseKFold,
+    S: Fn(&M::RowVector, &M::RowVector) -> T,
+> {
+    _phantom: std::marker::PhantomData<(T, M)>,
+
+    parameters_search: I,
+    estimator: F,
+    score: S,
+    cv: K,
+}
+
+impl<
+        T: RealNumber,
+        M: Array2<T>,
+        C: Clone,
+        I: Iterator<Item = C>,
+        E: Predictor<M, M::RowVector>,
+        F: Fn(&M, &M::RowVector, C) -> Result<E, Failed>,
+        K: BaseKFold,
+        S: Fn(&M::RowVector, &M::RowVector) -> T,
+    > GridSearchCVParameters<T, M, C, I, E, F, K, S>
+{
+    /// Create new GridSearchCVParameters
+    pub fn new(parameters_search: I, estimator: F, score: S, cv: K) -> Self {
+        GridSearchCVParameters {
+            _phantom: std::marker::PhantomData,
+            parameters_search,
+            estimator,
+            score,
+            cv,
+        }
+    }
+}
+/// Exhaustive search over specified parameter values for an estimator.
+#[derive(Debug)]
+pub struct GridSearchCV<T: RealNumber, M: Array2<T>, C: Clone, E: Predictor<M, M::RowVector>> {
+    _phantom: std::marker::PhantomData<(T, M)>,
+    predictor: E,
+    /// Cross validation results.
+    pub cross_validation_result: CrossValidationResult<T>,
+    /// best parameter
+    pub best_parameter: C,
+}
+
+impl<T: RealNumber, M: Array2<T>, E: Predictor<M, M::RowVector>, C: Clone>
+    GridSearchCV<T, M, C, E>
+{
+    ///  Search for the best estimator by testing all possible combinations with cross-validation using given metric.
+    /// * `x` - features, matrix of size _NxM_ where _N_ is number of samples and _M_ is number of attributes.
+    /// * `y` - target values, should be of size _N_
+    /// * `gs_parameters` - GridSearchCVParameters struct
+    pub fn fit<
+        I: Iterator<Item = C>,
+        K: BaseKFold,
+        F: Fn(&M, &M::RowVector, C) -> Result<E, Failed>,
+        S: Fn(&M::RowVector, &M::RowVector) -> T,
+    >(
+        x: &M,
+        y: &M::RowVector,
+        gs_parameters: GridSearchCVParameters<T, M, C, I, E, F, K, S>,
+    ) -> Result<Self, Failed> {
+        let mut best_result: Option<CrossValidationResult<T>> = None;
+        let mut best_parameters = None;
+        let parameters_search = gs_parameters.parameters_search;
+        let estimator = gs_parameters.estimator;
+        let cv = gs_parameters.cv;
+        let score = gs_parameters.score;
+
+        for parameters in parameters_search {
+            let result = cross_validate(&estimator, x, y, &parameters, &cv, &score)?;
+            if best_result.is_none()
+                || result.mean_test_score() > best_result.as_ref().unwrap().mean_test_score()
+            {
+                best_parameters = Some(parameters);
+                best_result = Some(result);
+            }
+        }
+
+        if let (Some(best_parameter), Some(cross_validation_result)) =
+            (best_parameters, best_result)
+        {
+            let predictor = estimator(x, y, best_parameter.clone())?;
+            Ok(Self {
+                _phantom: gs_parameters._phantom,
+                predictor,
+                cross_validation_result,
+                best_parameter,
+            })
+        } else {
+            Err(Failed::because(
+                FailedError::FindFailed,
+                "there were no parameter sets found",
+            ))
+        }
+    }
+
+    /// Return grid search cross validation results
+    pub fn cv_results(&self) -> &CrossValidationResult<T> {
+        &self.cross_validation_result
+    }
+
+    /// Return best parameters found
+    pub fn best_parameters(&self) -> &C {
+        &self.best_parameter
+    }
+
+    /// Call predict on the estimator with the best found parameters
+    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+        self.predictor.predict(x)
+    }
+}
+
+impl<
+        T: RealNumber,
+        M: Array2<T>,
+        C: Clone,
+        I: Iterator<Item = C>,
+        E: Predictor<M, M::RowVector>,
+        F: Fn(&M, &M::RowVector, C) -> Result<E, Failed>,
+        K: BaseKFold,
+        S: Fn(&M::RowVector, &M::RowVector) -> T,
+    > SupervisedEstimator<M, M::RowVector, GridSearchCVParameters<T, M, C, I, E, F, K, S>>
+    for GridSearchCV<T, M, C, E>
+{
+    fn fit(
+        x: &M,
+        y: &M::RowVector,
+        parameters: GridSearchCVParameters<T, M, C, I, E, F, K, S>,
+    ) -> Result<Self, Failed> {
+        GridSearchCV::fit(x, y, parameters)
+    }
+}
+
+impl<T: RealNumber, M: Array2<T>, C: Clone, E: Predictor<M, M::RowVector>>
+    Predictor<M, M::RowVector> for GridSearchCV<T, M, C, E>
+{
+    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+        self.predict(x)
+    }
+}
+
+#[cfg(test)]
+mod tests {
+
+    use crate::{
+        linalg::naive::dense_matrix::DenseMatrix,
+        linear::logistic_regression::{LogisticRegression, LogisticRegressionSearchParameters},
+        metrics::accuracy,
+        model_selection::{
+            hyper_tuning::grid_search::{self, GridSearchCVParameters},
+            KFold,
+        },
+    };
+    use grid_search::GridSearchCV;
+
+    #[test]
+    fn test_grid_search() {
+        let x = DenseMatrix::from_2d_array(&[
+            &[5.1, 3.5, 1.4, 0.2],
+            &[4.9, 3.0, 1.4, 0.2],
+            &[4.7, 3.2, 1.3, 0.2],
+            &[4.6, 3.1, 1.5, 0.2],
+            &[5.0, 3.6, 1.4, 0.2],
+            &[5.4, 3.9, 1.7, 0.4],
+            &[4.6, 3.4, 1.4, 0.3],
+            &[5.0, 3.4, 1.5, 0.2],
+            &[4.4, 2.9, 1.4, 0.2],
+            &[4.9, 3.1, 1.5, 0.1],
+            &[7.0, 3.2, 4.7, 1.4],
+            &[6.4, 3.2, 4.5, 1.5],
+            &[6.9, 3.1, 4.9, 1.5],
+            &[5.5, 2.3, 4.0, 1.3],
+            &[6.5, 2.8, 4.6, 1.5],
+            &[5.7, 2.8, 4.5, 1.3],
+            &[6.3, 3.3, 4.7, 1.6],
+            &[4.9, 2.4, 3.3, 1.0],
+            &[6.6, 2.9, 4.6, 1.3],
+            &[5.2, 2.7, 3.9, 1.4],
+        ]);
+        let y = vec![
+            0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
+        ];
+
+        let cv = KFold {
+            n_splits: 5,
+            ..KFold::default()
+        };
+
+        let parameters = LogisticRegressionSearchParameters {
+            alpha: vec![0., 1.],
+            ..Default::default()
+        };
+
+        let grid_search = GridSearchCV::fit(
+            &x,
+            &y,
+            GridSearchCVParameters {
+                estimator: LogisticRegression::fit,
+                score: accuracy,
+                cv,
+                parameters_search: parameters.into_iter(),
+                _phantom: Default::default(),
+            },
+        )
+        .unwrap();
+        let best_parameters = grid_search.best_parameters();
+
+        assert!([1.].contains(&best_parameters.alpha));
+
+        let cv_results = grid_search.cv_results();
+
+        assert_eq!(cv_results.mean_test_score(), 0.9);
+
+        let x = DenseMatrix::from_2d_array(&[&[5., 3., 1., 0.]]);
+        let result = grid_search.predict(&x).unwrap();
+        assert_eq!(result, vec![0.]);
+    }
+}

src/model_selection/hyper_tuning/mod.rs

@@ -0,0 +1,2 @@
+mod grid_search;
+pub use grid_search::{GridSearchCV, GridSearchCVParameters};

src/model_selection/kfold.rs

@@ -1,12 +1,12 @@
 //! # KFold
 //!
 //! Defines k-fold cross validator.
+use std::fmt::{Debug, Display};
 
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::Array2;
 use crate::model_selection::BaseKFold;
+use crate::rand_custom::get_rng_impl;
 use rand::seq::SliceRandom;
-use rand::thread_rng;
 
 /// K-Folds cross-validator
 pub struct KFold {
@@ -14,17 +14,25 @@ pub struct KFold {
     pub n_splits: usize, // cannot exceed std::usize::MAX
     /// Whether to shuffle the data before splitting into batches
     pub shuffle: bool,
+    /// When shuffle is True, seed affects the ordering of the indices.
+    /// Which controls the randomness of each fold
+    pub seed: Option<u64>,
 }
 
 impl KFold {
-    fn test_indices<T: RealNumber, M: Matrix<T>>(&self, x: &M) -> Vec<Vec<usize>> {
+    fn test_indices<T: Debug + Display + Copy + Sized, M: Array2<T>>(
+        &self,
+        x: &M,
+    ) -> Vec<Vec<usize>> {
         // number of samples (rows) in the matrix
         let n_samples: usize = x.shape().0;
 
         // initialise indices
         let mut indices: Vec<usize> = (0..n_samples).collect();
+        let mut rng = get_rng_impl(self.seed);
+
         if self.shuffle {
-            indices.shuffle(&mut thread_rng());
+            indices.shuffle(&mut rng);
         }
         //  return a new array of given shape n_split, filled with each element of n_samples divided by n_splits.
         let mut fold_sizes = vec![n_samples / self.n_splits; self.n_splits];
@@ -46,7 +54,7 @@ impl KFold {
         return_values
     }
 
-    fn test_masks<T: RealNumber, M: Matrix<T>>(&self, x: &M) -> Vec<Vec<bool>> {
+    fn test_masks<T: Debug + Display + Copy + Sized, M: Array2<T>>(&self, x: &M) -> Vec<Vec<bool>> {
         let mut return_values: Vec<Vec<bool>> = Vec::with_capacity(self.n_splits);
         for test_index in self.test_indices(x).drain(..) {
             // init mask
@@ -66,6 +74,7 @@ impl Default for KFold {
         KFold {
             n_splits: 3,
             shuffle: true,
+            seed: Option::None,
         }
     }
 }
@@ -81,6 +90,12 @@ impl KFold {
         self.shuffle = shuffle;
         self
     }
+
+    /// When shuffle is True, random_state affects the ordering of the indices.
+    pub fn with_seed(mut self, seed: Option<u64>) -> Self {
+        self.seed = seed;
+        self
+    }
 }
 
 /// An iterator over indices that split data into training and test set.
@@ -122,7 +137,7 @@ impl BaseKFold for KFold {
         self.n_splits
     }
 
-    fn split<T: RealNumber, M: Matrix<T>>(&self, x: &M) -> Self::Output {
+    fn split<T: Debug + Display + Copy + Sized, M: Array2<T>>(&self, x: &M) -> Self::Output {
         if self.n_splits < 2 {
             panic!("Number of splits is too small: {}", self.n_splits);
         }
@@ -142,14 +157,18 @@ impl BaseKFold for KFold {
 mod tests {
 
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_kfold_return_test_indices_simple() {
         let k = KFold {
             n_splits: 3,
             shuffle: false,
+            seed: Option::None,
         };
         let x: DenseMatrix<f64> = DenseMatrix::rand(33, 100);
         let test_indices = k.test_indices(&x);
@@ -159,12 +178,16 @@ mod tests {
         assert_eq!(test_indices[2], (22..33).collect::<Vec<usize>>());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_kfold_return_test_indices_odd() {
         let k = KFold {
             n_splits: 3,
             shuffle: false,
+            seed: Option::None,
         };
         let x: DenseMatrix<f64> = DenseMatrix::rand(34, 100);
         let test_indices = k.test_indices(&x);
@@ -174,12 +197,16 @@ mod tests {
         assert_eq!(test_indices[2], (23..34).collect::<Vec<usize>>());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_kfold_return_test_mask_simple() {
         let k = KFold {
             n_splits: 2,
             shuffle: false,
+            seed: Option::None,
         };
         let x: DenseMatrix<f64> = DenseMatrix::rand(22, 100);
         let test_masks = k.test_masks(&x);
@@ -200,12 +227,16 @@ mod tests {
         }
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_kfold_return_split_simple() {
         let k = KFold {
             n_splits: 2,
             shuffle: false,
+            seed: Option::None,
         };
         let x: DenseMatrix<f64> = DenseMatrix::rand(22, 100);
         let train_test_splits: Vec<(Vec<usize>, Vec<usize>)> = k.split(&x).collect();
@@ -216,7 +247,10 @@ mod tests {
         assert_eq!(train_test_splits[1].1, (11..22).collect::<Vec<usize>>());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_kfold_return_split_simple_shuffle() {
         let k = KFold {
@@ -232,12 +266,16 @@ mod tests {
         assert_eq!(train_test_splits[1].1.len(), 11_usize);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn numpy_parity_test() {
         let k = KFold {
             n_splits: 3,
             shuffle: false,
+            seed: Option::None,
         };
         let x: DenseMatrix<f64> = DenseMatrix::rand(10, 4);
         let expected: Vec<(Vec<usize>, Vec<usize>)> = vec![
@@ -253,7 +291,10 @@ mod tests {
         }
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn numpy_parity_test_shuffle() {
         let k = KFold {

src/model_selection/mod.rs

@@ -7,12 +7,12 @@
 //! Splitting data into multiple subsets helps us to find the right combination of hyperparameters, estimate model performance and choose the right model for
 //! the data.
 //!
-//! In SmartCore a random split into training and test sets can be quickly computed with the [train_test_split](./fn.train_test_split.html) helper function.
+//! In `smartcore` a random split into training and test sets can be quickly computed with the [train_test_split](./fn.train_test_split.html) helper function.
 //!
 //! ```
-//! use crate::smartcore::linalg::BaseMatrix;
-//! use smartcore::linalg::naive::dense_matrix::DenseMatrix;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::model_selection::train_test_split;
+//! use smartcore::linalg::basic::arrays::Array;
 //!
 //! //Iris data
 //! let x = DenseMatrix::from_2d_array(&[
@@ -41,7 +41,7 @@
 //!           0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
 //! ];
 //!
-//! let (x_train, x_test, y_train, y_test) = train_test_split(&x, &y, 0.2, true);
+//! let (x_train, x_test, y_train, y_test) = train_test_split(&x, &y, 0.2, true, None);
 //!
 //! println!("X train: {:?}, y train: {}, X test: {:?}, y test: {}",
 //!             x_train.shape(), y_train.len(), x_test.shape(), y_test.len());
@@ -55,10 +55,12 @@
 //! The simplest way to run cross-validation is to use the [cross_val_score](./fn.cross_validate.html) helper function on your estimator and the dataset.
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::DenseMatrix;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::model_selection::{KFold, cross_validate};
 //! use smartcore::metrics::accuracy;
 //! use smartcore::linear::logistic_regression::LogisticRegression;
+//! use smartcore::api::SupervisedEstimator;
+//! use smartcore::linalg::basic::arrays::Array;
 //!
 //! //Iris data
 //! let x = DenseMatrix::from_2d_array(&[
@@ -83,17 +85,18 @@
 //!           &[6.6, 2.9, 4.6, 1.3],
 //!           &[5.2, 2.7, 3.9, 1.4],
 //!           ]);
-//! let y: Vec<f64> = vec![
-//!           0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
+//! let y: Vec<i32> = vec![
+//!           0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 //! ];
 //!
 //! let cv = KFold::default().with_n_splits(3);
 //!
-//! let results = cross_validate(LogisticRegression::fit,   //estimator
-//!                                 &x, &y,                 //data
-//!                                 Default::default(),     //hyperparameters
-//!                                 cv,                     //cross validation split
-//!                                 &accuracy).unwrap();    //metric
+//! let results = cross_validate(
+//!     LogisticRegression::new(),   //estimator
+//!     &x, &y,                 //data
+//!     Default::default(),     //hyperparameters
+//!     &cv,                     //cross validation split
+//!     &accuracy).unwrap();    //metric
 //!
 //! println!("Training accuracy: {}, test accuracy: {}",
 //!     results.mean_test_score(), results.mean_train_score());
@@ -102,16 +105,22 @@
 //! The function [cross_val_predict](./fn.cross_val_predict.html) has a similar interface to `cross_val_score`,
 //! but instead of test error it calculates predictions for all samples in the test set.
 
-use crate::api::Predictor;
-use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
 use rand::seq::SliceRandom;
-use rand::thread_rng;
+use std::fmt::{Debug, Display};
 
+#[allow(unused_imports)]
+use crate::api::{Predictor, SupervisedEstimator};
+use crate::error::Failed;
+use crate::linalg::basic::arrays::{Array1, Array2};
+use crate::numbers::basenum::Number;
+use crate::numbers::realnum::RealNumber;
+use crate::rand_custom::get_rng_impl;
+
+// TODO: fix this module
+// pub(crate) mod hyper_tuning;
 pub(crate) mod kfold;
 
+// pub use hyper_tuning::{GridSearchCV, GridSearchCVParameters};
 pub use kfold::{KFold, KFoldIter};
 
 /// An interface for the K-Folds cross-validator
@@ -120,7 +129,7 @@ pub trait BaseKFold {
     type Output: Iterator<Item = (Vec<usize>, Vec<usize>)>;
     /// Return a tuple containing the the training set indices for that split and
     /// the testing set indices for that split.
-    fn split<T: RealNumber, M: Matrix<T>>(&self, x: &M) -> Self::Output;
+    fn split<T: Number, X: Array2<T>>(&self, x: &X) -> Self::Output;
     /// Returns the number of splits
     fn n_splits(&self) -> usize;
 }
@@ -130,25 +139,32 @@ pub trait BaseKFold {
 /// * `y` - target values, should be of size _N_
 /// * `test_size`, (0, 1] - the proportion of the dataset to include in the test split.
 /// * `shuffle`, - whether or not to shuffle the data before splitting
-pub fn train_test_split<T: RealNumber, M: Matrix<T>>(
-    x: &M,
-    y: &M::RowVector,
+pub fn train_test_split<
+    TX: Debug + Display + Copy + Sized,
+    TY: Debug + Display + Copy + Sized,
+    X: Array2<TX>,
+    Y: Array1<TY>,
+>(
+    x: &X,
+    y: &Y,
     test_size: f32,
     shuffle: bool,
-) -> (M, M, M::RowVector, M::RowVector) {
-    if x.shape().0 != y.len() {
+    seed: Option<u64>,
+) -> (X, X, Y, Y) {
+    if x.shape().0 != y.shape() {
         panic!(
             "x and y should have the same number of samples. |x|: {}, |y|: {}",
             x.shape().0,
-            y.len()
+            y.shape()
         );
     }
+    let mut rng = get_rng_impl(seed);
 
     if test_size <= 0. || test_size > 1.0 {
         panic!("test_size should be between 0 and 1");
     }
 
-    let n = y.len();
+    let n = y.shape();
 
     let n_test = ((n as f32) * test_size) as usize;
 
@@ -159,7 +175,7 @@ pub fn train_test_split<T: RealNumber, M: Matrix<T>>(
     let mut indices: Vec<usize> = (0..n).collect();
 
     if shuffle {
-        indices.shuffle(&mut thread_rng());
+        indices.shuffle(&mut rng);
     }
 
     let x_train = x.take(&indices[n_test..n], 0);
@@ -172,21 +188,29 @@ pub fn train_test_split<T: RealNumber, M: Matrix<T>>(
 
 /// Cross validation results.
 #[derive(Clone, Debug)]
-pub struct CrossValidationResult<T: RealNumber> {
+pub struct CrossValidationResult {
     /// Vector with test scores on each cv split
-    pub test_score: Vec<T>,
+    pub test_score: Vec<f64>,
     /// Vector with training scores on each cv split
-    pub train_score: Vec<T>,
+    pub train_score: Vec<f64>,
 }
 
-impl<T: RealNumber> CrossValidationResult<T> {
+impl CrossValidationResult {
     /// Average test score
-    pub fn mean_test_score(&self) -> T {
-        self.test_score.sum() / T::from_usize(self.test_score.len()).unwrap()
+    pub fn mean_test_score(&self) -> f64 {
+        let mut sum = 0f64;
+        for s in self.test_score.iter() {
+            sum += *s;
+        }
+        sum / self.test_score.len() as f64
     }
     /// Average training score
-    pub fn mean_train_score(&self) -> T {
-        self.train_score.sum() / T::from_usize(self.train_score.len()).unwrap()
+    pub fn mean_train_score(&self) -> f64 {
+        let mut sum = 0f64;
+        for s in self.train_score.iter() {
+            sum += *s;
+        }
+        sum / self.train_score.len() as f64
     }
 }
 
@@ -197,26 +221,27 @@ impl<T: RealNumber> CrossValidationResult<T> {
 /// * `parameters` - parameters of selected estimator. Use `Default::default()` for default parameters.
 /// * `cv` - the cross-validation splitting strategy, should be an instance of [`BaseKFold`](./trait.BaseKFold.html)
 /// * `score` - a metric to use for evaluation, see [metrics](../metrics/index.html)
-pub fn cross_validate<T, M, H, E, K, F, S>(
-    fit_estimator: F,
-    x: &M,
-    y: &M::RowVector,
+pub fn cross_validate<TX, TY, X, Y, H, E, K, S>(
+    _estimator: E, // just an empty placeholder to allow passing `fit()`
+    x: &X,
+    y: &Y,
     parameters: H,
-    cv: K,
-    score: S,
-) -> Result<CrossValidationResult<T>, Failed>
+    cv: &K,
+    score: &S,
+) -> Result<CrossValidationResult, Failed>
 where
-    T: RealNumber,
-    M: Matrix<T>,
+    TX: Number + RealNumber,
+    TY: Number,
+    X: Array2<TX>,
+    Y: Array1<TY>,
     H: Clone,
-    E: Predictor<M, M::RowVector>,
     K: BaseKFold,
-    F: Fn(&M, &M::RowVector, H) -> Result<E, Failed>,
-    S: Fn(&M::RowVector, &M::RowVector) -> T,
+    E: SupervisedEstimator<X, Y, H>,
+    S: Fn(&Y, &Y) -> f64,
 {
     let k = cv.n_splits();
-    let mut test_score = Vec::with_capacity(k);
-    let mut train_score = Vec::with_capacity(k);
+    let mut test_score: Vec<f64> = Vec::with_capacity(k);
+    let mut train_score: Vec<f64> = Vec::with_capacity(k);
 
     for (train_idx, test_idx) in cv.split(x) {
         let train_x = x.take(&train_idx, 0);
@@ -224,10 +249,12 @@ where
         let test_x = x.take(&test_idx, 0);
         let test_y = y.take(&test_idx);
 
-        let estimator = fit_estimator(&train_x, &train_y, parameters.clone())?;
+        // NOTE: we use here only the estimator "class", the  actual struct get dropped
+        let computed =
+            <E as SupervisedEstimator<X, Y, H>>::fit(&train_x, &train_y, parameters.clone())?;
 
-        train_score.push(score(&train_y, &estimator.predict(&train_x)?));
-        test_score.push(score(&test_y, &estimator.predict(&test_x)?));
+        train_score.push(score(&train_y, &computed.predict(&train_x)?));
+        test_score.push(score(&test_y, &computed.predict(&test_x)?));
     }
 
     Ok(CrossValidationResult {
@@ -243,33 +270,35 @@ where
 /// * `y` - target values, should be of size _N_
 /// * `parameters` - parameters of selected estimator. Use `Default::default()` for default parameters.
 /// * `cv` - the cross-validation splitting strategy, should be an instance of [`BaseKFold`](./trait.BaseKFold.html)
-pub fn cross_val_predict<T, M, H, E, K, F>(
-    fit_estimator: F,
-    x: &M,
-    y: &M::RowVector,
+pub fn cross_val_predict<TX, TY, X, Y, H, E, K>(
+    _estimator: E, // just an empty placeholder to allow passing `fit()`
+    x: &X,
+    y: &Y,
     parameters: H,
-    cv: K,
-) -> Result<M::RowVector, Failed>
+    cv: &K,
+) -> Result<Y, Failed>
 where
-    T: RealNumber,
-    M: Matrix<T>,
+    TX: Number,
+    TY: Number,
+    X: Array2<TX>,
+    Y: Array1<TY>,
     H: Clone,
-    E: Predictor<M, M::RowVector>,
     K: BaseKFold,
-    F: Fn(&M, &M::RowVector, H) -> Result<E, Failed>,
+    E: SupervisedEstimator<X, Y, H>,
 {
-    let mut y_hat = M::RowVector::zeros(y.len());
+    let mut y_hat = Y::zeros(y.shape());
 
     for (train_idx, test_idx) in cv.split(x) {
         let train_x = x.take(&train_idx, 0);
         let train_y = y.take(&train_idx);
         let test_x = x.take(&test_idx, 0);
 
-        let estimator = fit_estimator(&train_x, &train_y, parameters.clone())?;
+        let computed =
+            <E as SupervisedEstimator<X, Y, H>>::fit(&train_x, &train_y, parameters.clone())?;
 
-        let y_test_hat = estimator.predict(&test_x)?;
+        let y_test_hat = computed.predict(&test_x)?;
         for (i, &idx) in test_idx.iter().enumerate() {
-            y_hat.set(idx, y_test_hat.get(i));
+            y_hat.set(idx, *y_test_hat.get(i));
         }
     }
 
@@ -280,19 +309,29 @@ where
 mod tests {
 
     use super::*;
-    use crate::linalg::naive::dense_matrix::*;
+    use crate::algorithm::neighbour::KNNAlgorithmName;
+    use crate::api::NoParameters;
+    use crate::linalg::basic::arrays::Array;
+    use crate::linalg::basic::matrix::DenseMatrix;
+    use crate::linear::logistic_regression::LogisticRegression;
+    use crate::metrics::distance::Distances;
     use crate::metrics::{accuracy, mean_absolute_error};
+    use crate::model_selection::cross_validate;
     use crate::model_selection::kfold::KFold;
-    use crate::neighbors::knn_regressor::KNNRegressor;
+    use crate::neighbors::knn_regressor::{KNNRegressor, KNNRegressorParameters};
+    use crate::neighbors::KNNWeightFunction;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_train_test_split() {
         let n = 123;
         let x: DenseMatrix<f64> = DenseMatrix::rand(n, 3);
         let y = vec![0f64; n];
 
-        let (x_train, x_test, y_train, y_test) = train_test_split(&x, &y, 0.2, true);
+        let (x_train, x_test, y_train, y_test) = train_test_split(&x, &y, 0.2, true, None);
 
         assert!(
             x_train.shape().0 > (n as f64 * 0.65) as usize
@@ -307,31 +346,36 @@ mod tests {
     }
 
     #[derive(Clone)]
-    struct NoParameters {}
+    struct BiasedParameters {}
+    impl NoParameters for BiasedParameters {}
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_cross_validate_biased() {
         struct BiasedEstimator {}
 
-        impl BiasedEstimator {
-            fn fit<M: Matrix<f32>>(
-                _: &M,
-                _: &M::RowVector,
-                _: NoParameters,
-            ) -> Result<BiasedEstimator, Failed> {
+        impl<X: Array2<f32>, Y: Array1<u32>, P: NoParameters> SupervisedEstimator<X, Y, P>
+            for BiasedEstimator
+        {
+            fn new() -> Self {
+                Self {}
+            }
+            fn fit(_: &X, _: &Y, _: P) -> Result<BiasedEstimator, Failed> {
                 Ok(BiasedEstimator {})
             }
         }
 
-        impl<M: Matrix<f32>> Predictor<M, M::RowVector> for BiasedEstimator {
-            fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+        impl<X: Array2<f32>, Y: Array1<u32>> Predictor<X, Y> for BiasedEstimator {
+            fn predict(&self, x: &X) -> Result<Y, Failed> {
                 let (n, _) = x.shape();
-                Ok(M::RowVector::zeros(n))
+                Ok(Y::zeros(n))
             }
         }
 
-        let x = DenseMatrix::from_2d_array(&[
+        let x: DenseMatrix<f32> = DenseMatrix::from_2d_array(&[
             &[5.1, 3.5, 1.4, 0.2],
             &[4.9, 3.0, 1.4, 0.2],
             &[4.7, 3.2, 1.3, 0.2],
@@ -353,23 +397,31 @@ mod tests {
             &[6.6, 2.9, 4.6, 1.3],
             &[5.2, 2.7, 3.9, 1.4],
         ]);
-        let y = vec![
-            0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
-        ];
+        let y: Vec<u32> = vec![0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1];
 
         let cv = KFold {
             n_splits: 5,
             ..KFold::default()
         };
 
-        let results =
-            cross_validate(BiasedEstimator::fit, &x, &y, NoParameters {}, cv, &accuracy).unwrap();
+        let results = cross_validate(
+            BiasedEstimator {},
+            &x,
+            &y,
+            BiasedParameters {},
+            &cv,
+            &accuracy,
+        )
+        .unwrap();
 
         assert_eq!(0.4, results.mean_test_score());
         assert_eq!(0.4, results.mean_train_score());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_cross_validate_knn() {
         let x = DenseMatrix::from_2d_array(&[
@@ -401,11 +453,11 @@ mod tests {
         };
 
         let results = cross_validate(
-            KNNRegressor::fit,
+            KNNRegressor::new(),
             &x,
             &y,
             Default::default(),
-            cv,
+            &cv,
             &mean_absolute_error,
         )
         .unwrap();
@@ -414,10 +466,13 @@ mod tests {
         assert!(results.mean_train_score() < results.mean_test_score());
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn test_cross_val_predict_knn() {
-        let x = DenseMatrix::from_2d_array(&[
+        let x: DenseMatrix<f64> = DenseMatrix::from_2d_array(&[
             &[234.289, 235.6, 159., 107.608, 1947., 60.323],
             &[259.426, 232.5, 145.6, 108.632, 1948., 61.122],
             &[258.054, 368.2, 161.6, 109.773, 1949., 60.171],
@@ -435,18 +490,69 @@ mod tests {
             &[518.173, 480.6, 257.2, 127.852, 1961., 69.331],
             &[554.894, 400.7, 282.7, 130.081, 1962., 70.551],
         ]);
-        let y = vec![
+        let y: Vec<f64> = vec![
             83.0, 88.5, 88.2, 89.5, 96.2, 98.1, 99.0, 100.0, 101.2, 104.6, 108.4, 110.8, 112.6,
             114.2, 115.7, 116.9,
         ];
 
-        let cv = KFold {
+        let cv: KFold = KFold {
             n_splits: 2,
             ..KFold::default()
         };
 
-        let y_hat = cross_val_predict(KNNRegressor::fit, &x, &y, Default::default(), cv).unwrap();
+        let y_hat: Vec<f64> = cross_val_predict(
+            KNNRegressor::new(),
+            &x,
+            &y,
+            KNNRegressorParameters::default()
+                .with_k(3)
+                .with_distance(Distances::euclidian())
+                .with_algorithm(KNNAlgorithmName::LinearSearch)
+                .with_weight(KNNWeightFunction::Distance),
+            &cv,
+        )
+        .unwrap();
 
         assert!(mean_absolute_error(&y, &y_hat) < 10.0);
     }
+
+    #[test]
+    fn test_cross_validation_accuracy() {
+        let x = DenseMatrix::from_2d_array(&[
+            &[5.1, 3.5, 1.4, 0.2],
+            &[4.9, 3.0, 1.4, 0.2],
+            &[4.7, 3.2, 1.3, 0.2],
+            &[4.6, 3.1, 1.5, 0.2],
+            &[5.0, 3.6, 1.4, 0.2],
+            &[5.4, 3.9, 1.7, 0.4],
+            &[4.6, 3.4, 1.4, 0.3],
+            &[5.0, 3.4, 1.5, 0.2],
+            &[4.4, 2.9, 1.4, 0.2],
+            &[4.9, 3.1, 1.5, 0.1],
+            &[7.0, 3.2, 4.7, 1.4],
+            &[6.4, 3.2, 4.5, 1.5],
+            &[6.9, 3.1, 4.9, 1.5],
+            &[5.5, 2.3, 4.0, 1.3],
+            &[6.5, 2.8, 4.6, 1.5],
+            &[5.7, 2.8, 4.5, 1.3],
+            &[6.3, 3.3, 4.7, 1.6],
+            &[4.9, 2.4, 3.3, 1.0],
+            &[6.6, 2.9, 4.6, 1.3],
+            &[5.2, 2.7, 3.9, 1.4],
+        ]);
+        let y: Vec<i32> = vec![0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1];
+
+        let cv = KFold::default().with_n_splits(3);
+
+        let results = cross_validate(
+            LogisticRegression::new(),
+            &x,
+            &y,
+            Default::default(),
+            &cv,
+            &accuracy,
+        )
+        .unwrap();
+        println!("{:?}", results);
+    }
 }

src/naive_bayes/bernoulli.rs

@@ -6,7 +6,7 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::naive_bayes::bernoulli::BernoulliNB;
 //!
 //! // Training data points are:
@@ -14,56 +14,69 @@
 //! // Chinese Chinese Shanghai (class: China)
 //! // Chinese Macao (class: China)
 //! // Tokyo Japan Chinese (class: Japan)
-//! let x = DenseMatrix::<f64>::from_2d_array(&[
-//!           &[1., 1., 0., 0., 0., 0.],
-//!           &[0., 1., 0., 0., 1., 0.],
-//!           &[0., 1., 0., 1., 0., 0.],
-//!           &[0., 1., 1., 0., 0., 1.],
+//! let x = DenseMatrix::from_2d_array(&[
+//!           &[1, 1, 0, 0, 0, 0],
+//!           &[0, 1, 0, 0, 1, 0],
+//!           &[0, 1, 0, 1, 0, 0],
+//!           &[0, 1, 1, 0, 0, 1],
 //! ]);
-//! let y = vec![0., 0., 0., 1.];
+//! let y: Vec<u32> = vec![0, 0, 0, 1];
 //!
 //! let nb = BernoulliNB::fit(&x, &y, Default::default()).unwrap();
 //!
 //! // Testing data point is:
 //! // Chinese Chinese Chinese Tokyo Japan
-//! let x_test = DenseMatrix::<f64>::from_2d_array(&[&[0., 1., 1., 0., 0., 1.]]);
+//! let x_test = DenseMatrix::from_2d_array(&[&[0, 1, 1, 0, 0, 1]]);
 //! let y_hat = nb.predict(&x_test).unwrap();
 //! ```
 //!
 //! ## References:
 //!
 //! * ["Introduction to Information Retrieval", Manning C. D., Raghavan P., Schutze H., 2009, Chapter 13 ](https://nlp.stanford.edu/IR-book/information-retrieval-book.html)
+use std::fmt;
+
+use num_traits::Unsigned;
+
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::row_iter;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
-use crate::math::vector::RealNumberVector;
+use crate::linalg::basic::arrays::{Array1, Array2, ArrayView1};
 use crate::naive_bayes::{BaseNaiveBayes, NBDistribution};
+use crate::numbers::basenum::Number;
 
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 /// Naive Bayes classifier for Bearnoulli features
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug)]
-struct BernoulliNBDistribution<T: RealNumber> {
+#[derive(Debug, Clone)]
+struct BernoulliNBDistribution<T: Number + Ord + Unsigned> {
     /// class labels known to the classifier
     class_labels: Vec<T>,
     /// number of training samples observed in each class
     class_count: Vec<usize>,
     /// probability of each class
-    class_priors: Vec<T>,
+    class_priors: Vec<f64>,
     /// Number of samples encountered for each (class, feature)
     feature_count: Vec<Vec<usize>>,
     /// probability of features per class
-    feature_log_prob: Vec<Vec<T>>,
+    feature_log_prob: Vec<Vec<f64>>,
     /// Number of features of each sample
     n_features: usize,
 }
 
-impl<T: RealNumber> PartialEq for BernoulliNBDistribution<T> {
+impl<T: Number + Ord + Unsigned> fmt::Display for BernoulliNBDistribution<T> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "BernoulliNBDistribution: n_features: {:?}",
+            self.n_features
+        )?;
+        writeln!(f, "class_labels: {:?}", self.class_labels)?;
+        Ok(())
+    }
+}
+
+impl<T: Number + Ord + Unsigned> PartialEq for BernoulliNBDistribution<T> {
     fn eq(&self, other: &Self) -> bool {
         if self.class_labels == other.class_labels
             && self.class_count == other.class_count
@@ -76,7 +89,7 @@ impl<T: RealNumber> PartialEq for BernoulliNBDistribution<T> {
                 .iter()
                 .zip(other.feature_log_prob.iter())
             {
-                if !a.approximate_eq(b, T::epsilon()) {
+                if !a.iter().zip(b.iter()).all(|(a, b)| (a - b).abs() < 1e-4) {
                     return false;
                 }
             }
@@ -87,25 +100,27 @@ impl<T: RealNumber> PartialEq for BernoulliNBDistribution<T> {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> NBDistribution<T, M> for BernoulliNBDistribution<T> {
-    fn prior(&self, class_index: usize) -> T {
+impl<X: Number + PartialOrd, Y: Number + Ord + Unsigned> NBDistribution<X, Y>
+    for BernoulliNBDistribution<Y>
+{
+    fn prior(&self, class_index: usize) -> f64 {
         self.class_priors[class_index]
     }
 
-    fn log_likelihood(&self, class_index: usize, j: &M::RowVector) -> T {
-        let mut likelihood = T::zero();
-        for feature in 0..j.len() {
-            let value = j.get(feature);
-            if value == T::one() {
+    fn log_likelihood<'a>(&'a self, class_index: usize, j: &'a Box<dyn ArrayView1<X> + 'a>) -> f64 {
+        let mut likelihood = 0f64;
+        for feature in 0..j.shape() {
+            let value = *j.get(feature);
+            if value == X::one() {
                 likelihood += self.feature_log_prob[class_index][feature];
             } else {
-                likelihood += (T::one() - self.feature_log_prob[class_index][feature].exp()).ln();
+                likelihood += (1f64 - self.feature_log_prob[class_index][feature].exp()).ln();
             }
         }
         likelihood
     }
 
-    fn classes(&self) -> &Vec<T> {
+    fn classes(&self) -> &Vec<Y> {
         &self.class_labels
     }
 }
@@ -113,23 +128,26 @@ impl<T: RealNumber, M: Matrix<T>> NBDistribution<T, M> for BernoulliNBDistributi
 /// `BernoulliNB` parameters. Use `Default::default()` for default values.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct BernoulliNBParameters<T: RealNumber> {
+pub struct BernoulliNBParameters<T: Number> {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
-    pub alpha: T,
+    pub alpha: f64,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Prior probabilities of the classes. If specified the priors are not adjusted according to the data
-    pub priors: Option<Vec<T>>,
+    pub priors: Option<Vec<f64>>,
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Threshold for binarizing (mapping to booleans) of sample features. If None, input is presumed to already consist of binary vectors.
     pub binarize: Option<T>,
 }
 
-impl<T: RealNumber> BernoulliNBParameters<T> {
+impl<T: Number + PartialOrd> BernoulliNBParameters<T> {
     /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
-    pub fn with_alpha(mut self, alpha: T) -> Self {
+    pub fn with_alpha(mut self, alpha: f64) -> Self {
         self.alpha = alpha;
         self
     }
     /// Prior probabilities of the classes. If specified the priors are not adjusted according to the data
-    pub fn with_priors(mut self, priors: Vec<T>) -> Self {
+    pub fn with_priors(mut self, priors: Vec<f64>) -> Self {
         self.priors = Some(priors);
         self
     }
@@ -140,17 +158,102 @@ impl<T: RealNumber> BernoulliNBParameters<T> {
     }
 }
 
-impl<T: RealNumber> Default for BernoulliNBParameters<T> {
+impl<T: Number + PartialOrd> Default for BernoulliNBParameters<T> {
     fn default() -> Self {
         Self {
-            alpha: T::one(),
-            priors: None,
+            alpha: 1f64,
+            priors: Option::None,
             binarize: Some(T::zero()),
         }
     }
 }
 
-impl<T: RealNumber> BernoulliNBDistribution<T> {
+/// BernoulliNB grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct BernoulliNBSearchParameters<T: Number> {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
+    pub alpha: Vec<f64>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Prior probabilities of the classes. If specified the priors are not adjusted according to the data
+    pub priors: Vec<Option<Vec<f64>>>,
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Threshold for binarizing (mapping to booleans) of sample features. If None, input is presumed to already consist of binary vectors.
+    pub binarize: Vec<Option<T>>,
+}
+
+/// BernoulliNB grid search iterator
+pub struct BernoulliNBSearchParametersIterator<T: Number> {
+    bernoulli_nb_search_parameters: BernoulliNBSearchParameters<T>,
+    current_alpha: usize,
+    current_priors: usize,
+    current_binarize: usize,
+}
+
+impl<T: Number> IntoIterator for BernoulliNBSearchParameters<T> {
+    type Item = BernoulliNBParameters<T>;
+    type IntoIter = BernoulliNBSearchParametersIterator<T>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        BernoulliNBSearchParametersIterator {
+            bernoulli_nb_search_parameters: self,
+            current_alpha: 0,
+            current_priors: 0,
+            current_binarize: 0,
+        }
+    }
+}
+
+impl<T: Number> Iterator for BernoulliNBSearchParametersIterator<T> {
+    type Item = BernoulliNBParameters<T>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_alpha == self.bernoulli_nb_search_parameters.alpha.len()
+            && self.current_priors == self.bernoulli_nb_search_parameters.priors.len()
+            && self.current_binarize == self.bernoulli_nb_search_parameters.binarize.len()
+        {
+            return None;
+        }
+
+        let next = BernoulliNBParameters {
+            alpha: self.bernoulli_nb_search_parameters.alpha[self.current_alpha],
+            priors: self.bernoulli_nb_search_parameters.priors[self.current_priors].clone(),
+            binarize: self.bernoulli_nb_search_parameters.binarize[self.current_binarize],
+        };
+
+        if self.current_alpha + 1 < self.bernoulli_nb_search_parameters.alpha.len() {
+            self.current_alpha += 1;
+        } else if self.current_priors + 1 < self.bernoulli_nb_search_parameters.priors.len() {
+            self.current_alpha = 0;
+            self.current_priors += 1;
+        } else if self.current_binarize + 1 < self.bernoulli_nb_search_parameters.binarize.len() {
+            self.current_alpha = 0;
+            self.current_priors = 0;
+            self.current_binarize += 1;
+        } else {
+            self.current_alpha += 1;
+            self.current_priors += 1;
+            self.current_binarize += 1;
+        }
+
+        Some(next)
+    }
+}
+
+impl<T: Number + std::cmp::PartialOrd> Default for BernoulliNBSearchParameters<T> {
+    fn default() -> Self {
+        let default_params = BernoulliNBParameters::<T>::default();
+
+        BernoulliNBSearchParameters {
+            alpha: vec![default_params.alpha],
+            priors: vec![default_params.priors],
+            binarize: vec![default_params.binarize],
+        }
+    }
+}
+
+impl<TY: Number + Ord + Unsigned> BernoulliNBDistribution<TY> {
     /// Fits the distribution to a NxM matrix where N is number of samples and M is number of features.
     /// * `x` - training data.
     /// * `y` - vector with target values (classes) of length N.
@@ -158,14 +261,14 @@ impl<T: RealNumber> BernoulliNBDistribution<T> {
     /// priors are adjusted according to the data.
     /// * `alpha` - Additive (Laplace/Lidstone) smoothing parameter.
     /// * `binarize` - Threshold for binarizing.
-    pub fn fit<M: Matrix<T>>(
-        x: &M,
-        y: &M::RowVector,
-        alpha: T,
-        priors: Option<Vec<T>>,
+    fn fit<TX: Number + PartialOrd, X: Array2<TX>, Y: Array1<TY>>(
+        x: &X,
+        y: &Y,
+        alpha: f64,
+        priors: Option<Vec<f64>>,
     ) -> Result<Self, Failed> {
         let (n_samples, n_features) = x.shape();
-        let y_samples = y.len();
+        let y_samples = y.shape();
         if y_samples != n_samples {
             return Err(Failed::fit(&format!(
                 "Size of x should equal size of y; |x|=[{}], |y|=[{}]",
@@ -179,16 +282,15 @@ impl<T: RealNumber> BernoulliNBDistribution<T> {
                 n_samples
             )));
         }
-        if alpha < T::zero() {
+        if alpha < 0f64 {
             return Err(Failed::fit(&format!(
                 "Alpha should be greater than 0; |alpha|=[{}]",
                 alpha
             )));
         }
 
-        let y = y.to_vec();
+        let (class_labels, indices) = y.unique_with_indices();
 
-        let (class_labels, indices) = <Vec<T> as RealNumberVector<T>>::unique_with_indices(&y);
         let mut class_count = vec![0_usize; class_labels.len()];
 
         for class_index in indices.iter() {
@@ -205,14 +307,14 @@ impl<T: RealNumber> BernoulliNBDistribution<T> {
         } else {
             class_count
                 .iter()
-                .map(|&c| T::from(c).unwrap() / T::from(n_samples).unwrap())
+                .map(|&c| c as f64 / (n_samples as f64))
                 .collect()
         };
 
         let mut feature_in_class_counter = vec![vec![0_usize; n_features]; class_labels.len()];
 
-        for (row, class_index) in row_iter(x).zip(indices) {
-            for (idx, row_i) in row.iter().enumerate().take(n_features) {
+        for (row, class_index) in x.row_iter().zip(indices) {
+            for (idx, row_i) in row.iterator(0).enumerate().take(n_features) {
                 feature_in_class_counter[class_index][idx] +=
                     row_i.to_usize().ok_or_else(|| {
                         Failed::fit(&format!(
@@ -230,9 +332,8 @@ impl<T: RealNumber> BernoulliNBDistribution<T> {
                 feature_count
                     .iter()
                     .map(|&count| {
-                        ((T::from(count).unwrap() + alpha)
-                            / (T::from(class_count[class_index]).unwrap() + alpha * T::two()))
-                        .ln()
+                        ((count as f64 + alpha) / (class_count[class_index] as f64 + alpha * 2f64))
+                            .ln()
                     })
                     .collect()
             })
@@ -253,40 +354,66 @@ impl<T: RealNumber> BernoulliNBDistribution<T> {
 /// distribution.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, PartialEq)]
-pub struct BernoulliNB<T: RealNumber, M: Matrix<T>> {
-    inner: BaseNaiveBayes<T, M, BernoulliNBDistribution<T>>,
-    binarize: Option<T>,
+pub struct BernoulliNB<
+    TX: Number + PartialOrd,
+    TY: Number + Ord + Unsigned,
+    X: Array2<TX>,
+    Y: Array1<TY>,
+> {
+    inner: Option<BaseNaiveBayes<TX, TY, X, Y, BernoulliNBDistribution<TY>>>,
+    binarize: Option<TX>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, BernoulliNBParameters<T>>
-    for BernoulliNB<T, M>
+impl<TX: Number + PartialOrd, TY: Number + Ord + Unsigned, X: Array2<TX>, Y: Array1<TY>>
+    fmt::Display for BernoulliNB<TX, TY, X, Y>
 {
-    fn fit(x: &M, y: &M::RowVector, parameters: BernoulliNBParameters<T>) -> Result<Self, Failed> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "BernoulliNB:\ninner: {:?}\nbinarize: {:?}",
+            self.inner.as_ref().unwrap(),
+            self.binarize.as_ref().unwrap()
+        )?;
+        Ok(())
+    }
+}
+
+impl<TX: Number + PartialOrd, TY: Number + Ord + Unsigned, X: Array2<TX>, Y: Array1<TY>>
+    SupervisedEstimator<X, Y, BernoulliNBParameters<TX>> for BernoulliNB<TX, TY, X, Y>
+{
+    fn new() -> Self {
+        Self {
+            inner: Option::None,
+            binarize: Option::None,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: BernoulliNBParameters<TX>) -> Result<Self, Failed> {
         BernoulliNB::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for BernoulliNB<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<TX: Number + PartialOrd, TY: Number + Ord + Unsigned, X: Array2<TX>, Y: Array1<TY>>
+    Predictor<X, Y> for BernoulliNB<TX, TY, X, Y>
+{
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> BernoulliNB<T, M> {
+impl<TX: Number + PartialOrd, TY: Number + Ord + Unsigned, X: Array2<TX>, Y: Array1<TY>>
+    BernoulliNB<TX, TY, X, Y>
+{
     /// Fits BernoulliNB with given data
     /// * `x` - training data of size NxM where N is the number of samples and M is the number of
     /// features.
     /// * `y` - vector with target values (classes) of length N.
     /// * `parameters` - additional parameters like class priors, alpha for smoothing and
     /// binarizing threshold.
-    pub fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: BernoulliNBParameters<T>,
-    ) -> Result<Self, Failed> {
+    pub fn fit(x: &X, y: &Y, parameters: BernoulliNBParameters<TX>) -> Result<Self, Failed> {
         let distribution = if let Some(threshold) = parameters.binarize {
             BernoulliNBDistribution::fit(
-                &(x.binarize(threshold)),
+                &Self::binarize(x, threshold),
                 y,
                 parameters.alpha,
                 parameters.priors,
@@ -297,7 +424,7 @@ impl<T: RealNumber, M: Matrix<T>> BernoulliNB<T, M> {
 
         let inner = BaseNaiveBayes::fit(distribution)?;
         Ok(Self {
-            inner,
+            inner: Some(inner),
             binarize: parameters.binarize,
         })
     }
@@ -305,49 +432,88 @@ impl<T: RealNumber, M: Matrix<T>> BernoulliNB<T, M> {
     /// Estimates the class labels for the provided data.
     /// * `x` - data of shape NxM where N is number of data points to estimate and M is number of features.
     /// Returns a vector of size N with class estimates.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
         if let Some(threshold) = self.binarize {
-            self.inner.predict(&(x.binarize(threshold)))
+            self.inner
+                .as_ref()
+                .unwrap()
+                .predict(&Self::binarize(x, threshold))
         } else {
-            self.inner.predict(x)
+            self.inner.as_ref().unwrap().predict(x)
         }
     }
 
     /// Class labels known to the classifier.
     /// Returns a vector of size n_classes.
-    pub fn classes(&self) -> &Vec<T> {
-        &self.inner.distribution.class_labels
+    pub fn classes(&self) -> &Vec<TY> {
+        &self.inner.as_ref().unwrap().distribution.class_labels
     }
 
     /// Number of training samples observed in each class.
     /// Returns a vector of size n_classes.
     pub fn class_count(&self) -> &Vec<usize> {
-        &self.inner.distribution.class_count
+        &self.inner.as_ref().unwrap().distribution.class_count
     }
 
     /// Number of features of each sample
     pub fn n_features(&self) -> usize {
-        self.inner.distribution.n_features
+        self.inner.as_ref().unwrap().distribution.n_features
     }
 
     /// Number of samples encountered for each (class, feature)
     /// Returns a 2d vector of shape (n_classes, n_features)
     pub fn feature_count(&self) -> &Vec<Vec<usize>> {
-        &self.inner.distribution.feature_count
+        &self.inner.as_ref().unwrap().distribution.feature_count
     }
 
     /// Empirical log probability of features given a class
-    pub fn feature_log_prob(&self) -> &Vec<Vec<T>> {
-        &self.inner.distribution.feature_log_prob
+    pub fn feature_log_prob(&self) -> &Vec<Vec<f64>> {
+        &self.inner.as_ref().unwrap().distribution.feature_log_prob
+    }
+
+    fn binarize_mut(x: &mut X, threshold: TX) {
+        let (nrows, ncols) = x.shape();
+        for row in 0..nrows {
+            for col in 0..ncols {
+                if *x.get((row, col)) > threshold {
+                    x.set((row, col), TX::one());
+                } else {
+                    x.set((row, col), TX::zero());
+                }
+            }
+        }
+    }
+
+    fn binarize(x: &X, threshold: TX) -> X {
+        let mut new_x = x.clone();
+        Self::binarize_mut(&mut new_x, threshold);
+        new_x
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters: BernoulliNBSearchParameters<f64> = BernoulliNBSearchParameters {
+            alpha: vec![1., 2.],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 2.);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_bernoulli_naive_bayes() {
         // Tests that BernoulliNB when alpha=1.0 gives the same values as
@@ -360,16 +526,18 @@ mod tests {
         // Chinese Chinese Shanghai (class: China)
         // Chinese Macao (class: China)
         // Tokyo Japan Chinese (class: Japan)
-        let x = DenseMatrix::<f64>::from_2d_array(&[
-            &[1., 1., 0., 0., 0., 0.],
-            &[0., 1., 0., 0., 1., 0.],
-            &[0., 1., 0., 1., 0., 0.],
-            &[0., 1., 1., 0., 0., 1.],
+        let x = DenseMatrix::from_2d_array(&[
+            &[1.0, 1.0, 0.0, 0.0, 0.0, 0.0],
+            &[0.0, 1.0, 0.0, 0.0, 1.0, 0.0],
+            &[0.0, 1.0, 0.0, 1.0, 0.0, 0.0],
+            &[0.0, 1.0, 1.0, 0.0, 0.0, 1.0],
         ]);
-        let y = vec![0., 0., 0., 1.];
+        let y: Vec<u32> = vec![0, 0, 0, 1];
         let bnb = BernoulliNB::fit(&x, &y, Default::default()).unwrap();
 
-        assert_eq!(bnb.inner.distribution.class_priors, &[0.75, 0.25]);
+        let distribution = bnb.inner.clone().unwrap().distribution;
+
+        assert_eq!(&distribution.class_priors, &[0.75, 0.25]);
         assert_eq!(
             bnb.feature_log_prob(),
             &[
@@ -394,38 +562,41 @@ mod tests {
 
         // Testing data point is:
         //  Chinese Chinese Chinese Tokyo Japan
-        let x_test = DenseMatrix::<f64>::from_2d_array(&[&[0., 1., 1., 0., 0., 1.]]);
+        let x_test = DenseMatrix::from_2d_array(&[&[0.0, 1.0, 1.0, 0.0, 0.0, 1.0]]);
         let y_hat = bnb.predict(&x_test).unwrap();
 
-        assert_eq!(y_hat, &[1.]);
+        assert_eq!(y_hat, &[1]);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn bernoulli_nb_scikit_parity() {
-        let x = DenseMatrix::<f64>::from_2d_array(&[
-            &[2., 4., 0., 0., 2., 1., 2., 4., 2., 0.],
-            &[3., 4., 0., 2., 1., 0., 1., 4., 0., 3.],
-            &[1., 4., 2., 4., 1., 0., 1., 2., 3., 2.],
-            &[0., 3., 3., 4., 1., 0., 3., 1., 1., 1.],
-            &[0., 2., 1., 4., 3., 4., 1., 2., 3., 1.],
-            &[3., 2., 4., 1., 3., 0., 2., 4., 0., 2.],
-            &[3., 1., 3., 0., 2., 0., 4., 4., 3., 4.],
-            &[2., 2., 2., 0., 1., 1., 2., 1., 0., 1.],
-            &[3., 3., 2., 2., 0., 2., 3., 2., 2., 3.],
-            &[4., 3., 4., 4., 4., 2., 2., 0., 1., 4.],
-            &[3., 4., 2., 2., 1., 4., 4., 4., 1., 3.],
-            &[3., 0., 1., 4., 4., 0., 0., 3., 2., 4.],
-            &[2., 0., 3., 3., 1., 2., 0., 2., 4., 1.],
-            &[2., 4., 0., 4., 2., 4., 1., 3., 1., 4.],
-            &[0., 2., 2., 3., 4., 0., 4., 4., 4., 4.],
+        let x = DenseMatrix::from_2d_array(&[
+            &[2, 4, 0, 0, 2, 1, 2, 4, 2, 0],
+            &[3, 4, 0, 2, 1, 0, 1, 4, 0, 3],
+            &[1, 4, 2, 4, 1, 0, 1, 2, 3, 2],
+            &[0, 3, 3, 4, 1, 0, 3, 1, 1, 1],
+            &[0, 2, 1, 4, 3, 4, 1, 2, 3, 1],
+            &[3, 2, 4, 1, 3, 0, 2, 4, 0, 2],
+            &[3, 1, 3, 0, 2, 0, 4, 4, 3, 4],
+            &[2, 2, 2, 0, 1, 1, 2, 1, 0, 1],
+            &[3, 3, 2, 2, 0, 2, 3, 2, 2, 3],
+            &[4, 3, 4, 4, 4, 2, 2, 0, 1, 4],
+            &[3, 4, 2, 2, 1, 4, 4, 4, 1, 3],
+            &[3, 0, 1, 4, 4, 0, 0, 3, 2, 4],
+            &[2, 0, 3, 3, 1, 2, 0, 2, 4, 1],
+            &[2, 4, 0, 4, 2, 4, 1, 3, 1, 4],
+            &[0, 2, 2, 3, 4, 0, 4, 4, 4, 4],
         ]);
-        let y = vec![2., 2., 0., 0., 0., 2., 1., 1., 0., 1., 0., 0., 2., 0., 2.];
+        let y: Vec<u32> = vec![2, 2, 0, 0, 0, 2, 1, 1, 0, 1, 0, 0, 2, 0, 2];
         let bnb = BernoulliNB::fit(&x, &y, Default::default()).unwrap();
 
         let y_hat = bnb.predict(&x).unwrap();
 
-        assert_eq!(bnb.classes(), &[0., 1., 2.]);
+        assert_eq!(bnb.classes(), &[0, 1, 2]);
         assert_eq!(bnb.class_count(), &[7, 3, 5]);
         assert_eq!(bnb.n_features(), 10);
         assert_eq!(
@@ -437,46 +608,50 @@ mod tests {
             ]
         );
 
-        assert!(bnb
-            .inner
-            .distribution
-            .class_priors
-            .approximate_eq(&vec!(0.46, 0.2, 0.33), 1e-2));
-        assert!(bnb.feature_log_prob()[1].approximate_eq(
+        // test Display
+        println!("{}", &bnb);
+
+        let distribution = bnb.inner.clone().unwrap().distribution;
+
+        assert_eq!(
+            &distribution.class_priors,
+            &vec!(0.4666666666666667, 0.2, 0.3333333333333333)
+        );
+        assert_eq!(
+            &bnb.feature_log_prob()[1],
             &vec![
-                -0.22314355,
-                -0.22314355,
-                -0.22314355,
-                -0.91629073,
-                -0.22314355,
-                -0.51082562,
-                -0.22314355,
-                -0.51082562,
-                -0.51082562,
-                -0.22314355
-            ],
-            1e-1
-        ));
-        assert!(y_hat.approximate_eq(
-            &vec!(2.0, 2.0, 0.0, 0.0, 0.0, 2.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0),
-            1e-5
-        ));
+                -0.2231435513142097,
+                -0.2231435513142097,
+                -0.2231435513142097,
+                -0.916290731874155,
+                -0.2231435513142097,
+                -0.5108256237659907,
+                -0.2231435513142097,
+                -0.5108256237659907,
+                -0.5108256237659907,
+                -0.2231435513142097
+            ]
+        );
+        assert_eq!(y_hat, vec!(2, 2, 0, 0, 0, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0));
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
-        let x = DenseMatrix::<f64>::from_2d_array(&[
-            &[1., 1., 0., 0., 0., 0.],
-            &[0., 1., 0., 0., 1., 0.],
-            &[0., 1., 0., 1., 0., 0.],
-            &[0., 1., 1., 0., 0., 1.],
+        let x = DenseMatrix::from_2d_array(&[
+            &[1, 1, 0, 0, 0, 0],
+            &[0, 1, 0, 0, 1, 0],
+            &[0, 1, 0, 1, 0, 0],
+            &[0, 1, 1, 0, 0, 1],
         ]);
-        let y = vec![0., 0., 0., 1.];
+        let y: Vec<u32> = vec![0, 0, 0, 1];
 
         let bnb = BernoulliNB::fit(&x, &y, Default::default()).unwrap();
-        let deserialized_bnb: BernoulliNB<f64, DenseMatrix<f64>> =
+        let deserialized_bnb: BernoulliNB<i32, u32, DenseMatrix<i32>, Vec<u32>> =
             serde_json::from_str(&serde_json::to_string(&bnb).unwrap()).unwrap();
 
         assert_eq!(bnb, deserialized_bnb);

src/naive_bayes/categorical.rs

@@ -6,50 +6,53 @@
 //! Example:
 //!
 //! ```
-//! use smartcore::linalg::naive::dense_matrix::*;
+//! use smartcore::linalg::basic::matrix::DenseMatrix;
 //! use smartcore::naive_bayes::categorical::CategoricalNB;
 //!
 //! let x = DenseMatrix::from_2d_array(&[
-//!              &[3., 4., 0., 1.],
-//!              &[3., 0., 0., 1.],
-//!              &[4., 4., 1., 2.],
-//!              &[4., 2., 4., 3.],
-//!              &[4., 2., 4., 2.],
-//!              &[4., 1., 1., 0.],
-//!              &[1., 1., 1., 1.],
-//!              &[0., 4., 1., 0.],
-//!              &[0., 3., 2., 1.],
-//!              &[0., 3., 1., 1.],
-//!              &[3., 4., 0., 1.],
-//!              &[3., 4., 2., 4.],
-//!              &[0., 3., 1., 2.],
-//!              &[0., 4., 1., 2.],
+//!              &[3, 4, 0, 1],
+//!              &[3, 0, 0, 1],
+//!              &[4, 4, 1, 2],
+//!              &[4, 2, 4, 3],
+//!              &[4, 2, 4, 2],
+//!              &[4, 1, 1, 0],
+//!              &[1, 1, 1, 1],
+//!              &[0, 4, 1, 0],
+//!              &[0, 3, 2, 1],
+//!              &[0, 3, 1, 1],
+//!              &[3, 4, 0, 1],
+//!              &[3, 4, 2, 4],
+//!              &[0, 3, 1, 2],
+//!              &[0, 4, 1, 2],
 //!          ]);
-//! let y = vec![0., 0., 1., 1., 1., 0., 1., 0., 1., 1., 1., 1., 1., 0.];
+//! let y: Vec<u32> = vec![0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0];
 //!
 //! let nb = CategoricalNB::fit(&x, &y, Default::default()).unwrap();
 //! let y_hat = nb.predict(&x).unwrap();
 //! ```
+use std::fmt;
+
+use num_traits::Unsigned;
+
 use crate::api::{Predictor, SupervisedEstimator};
 use crate::error::Failed;
-use crate::linalg::BaseVector;
-use crate::linalg::Matrix;
-use crate::math::num::RealNumber;
+use crate::linalg::basic::arrays::{Array1, Array2, ArrayView1};
 use crate::naive_bayes::{BaseNaiveBayes, NBDistribution};
+use crate::numbers::basenum::Number;
 #[cfg(feature = "serde")]
 use serde::{Deserialize, Serialize};
 
 /// Naive Bayes classifier for categorical features
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[derive(Debug)]
-struct CategoricalNBDistribution<T: RealNumber> {
+#[derive(Debug, Clone)]
+struct CategoricalNBDistribution<T: Number + Unsigned> {
     /// number of training samples observed in each class
     class_count: Vec<usize>,
     /// class labels known to the classifier
     class_labels: Vec<T>,
     /// probability of each class
-    class_priors: Vec<T>,
-    coefficients: Vec<Vec<Vec<T>>>,
+    class_priors: Vec<f64>,
+    coefficients: Vec<Vec<Vec<f64>>>,
     /// Number of features of each sample
     n_features: usize,
     /// Number of categories for each feature
@@ -60,7 +63,19 @@ struct CategoricalNBDistribution<T: RealNumber> {
     category_count: Vec<Vec<Vec<usize>>>,
 }
 
-impl<T: RealNumber> PartialEq for CategoricalNBDistribution<T> {
+impl<T: Number + Ord + Unsigned> fmt::Display for CategoricalNBDistribution<T> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "CategoricalNBDistribution: n_features: {:?}",
+            self.n_features
+        )?;
+        writeln!(f, "class_labels: {:?}", self.class_labels)?;
+        Ok(())
+    }
+}
+
+impl<T: Number + Unsigned> PartialEq for CategoricalNBDistribution<T> {
     fn eq(&self, other: &Self) -> bool {
         if self.class_labels == other.class_labels
             && self.class_priors == other.class_priors
@@ -80,7 +95,7 @@ impl<T: RealNumber> PartialEq for CategoricalNBDistribution<T> {
                         return false;
                     }
                     for (a_i_j, b_i_j) in a_i.iter().zip(b_i.iter()) {
-                        if (*a_i_j - *b_i_j).abs() > T::epsilon() {
+                        if (*a_i_j - *b_i_j).abs() > std::f64::EPSILON {
                             return false;
                         }
                     }
@@ -93,29 +108,29 @@ impl<T: RealNumber> PartialEq for CategoricalNBDistribution<T> {
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> NBDistribution<T, M> for CategoricalNBDistribution<T> {
-    fn prior(&self, class_index: usize) -> T {
+impl<T: Number + Unsigned> NBDistribution<T, T> for CategoricalNBDistribution<T> {
+    fn prior(&self, class_index: usize) -> f64 {
         if class_index >= self.class_labels.len() {
-            T::zero()
+            0f64
         } else {
             self.class_priors[class_index]
         }
     }
 
-    fn log_likelihood(&self, class_index: usize, j: &M::RowVector) -> T {
+    fn log_likelihood<'a>(&'a self, class_index: usize, j: &'a Box<dyn ArrayView1<T> + 'a>) -> f64 {
         if class_index < self.class_labels.len() {
-            let mut likelihood = T::zero();
-            for feature in 0..j.len() {
-                let value = j.get(feature).floor().to_usize().unwrap();
+            let mut likelihood = 0f64;
+            for feature in 0..j.shape() {
+                let value = j.get(feature).to_usize().unwrap();
                 if self.coefficients[feature][class_index].len() > value {
                     likelihood += self.coefficients[feature][class_index][value];
                 } else {
-                    return T::zero();
+                    return 0f64;
                 }
             }
             likelihood
         } else {
-            T::zero()
+            0f64
         }
     }
 
@@ -124,13 +139,24 @@ impl<T: RealNumber, M: Matrix<T>> NBDistribution<T, M> for CategoricalNBDistribu
     }
 }
 
-impl<T: RealNumber> CategoricalNBDistribution<T> {
+impl<T: Number + Unsigned, X: Array2<T>, Y: Array1<T>> fmt::Display for CategoricalNB<T, X, Y> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        writeln!(
+            f,
+            "CategoricalNB:\ninner: {:?}",
+            self.inner.as_ref().unwrap()
+        )?;
+        Ok(())
+    }
+}
+
+impl<T: Number + Unsigned> CategoricalNBDistribution<T> {
     /// Fits the distribution to a NxM matrix where N is number of samples and M is number of features.
     /// * `x` - training data.
     /// * `y` - vector with target values (classes) of length N.
     /// * `alpha` - Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
-    pub fn fit<M: Matrix<T>>(x: &M, y: &M::RowVector, alpha: T) -> Result<Self, Failed> {
-        if alpha < T::zero() {
+    pub fn fit<X: Array2<T>, Y: Array1<T>>(x: &X, y: &Y, alpha: f64) -> Result<Self, Failed> {
+        if alpha < 0f64 {
             return Err(Failed::fit(&format!(
                 "alpha should be >= 0, alpha=[{}]",
                 alpha
@@ -138,7 +164,7 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
         }
 
         let (n_samples, n_features) = x.shape();
-        let y_samples = y.len();
+        let y_samples = y.shape();
         if y_samples != n_samples {
             return Err(Failed::fit(&format!(
                 "Size of x should equal size of y; |x|=[{}], |y|=[{}]",
@@ -152,11 +178,7 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
                 n_samples
             )));
         }
-        let y: Vec<usize> = y
-            .to_vec()
-            .iter()
-            .map(|y_i| y_i.floor().to_usize().unwrap())
-            .collect();
+        let y: Vec<usize> = y.iterator(0).map(|y_i| y_i.to_usize().unwrap()).collect();
 
         let y_max = y
             .iter()
@@ -164,7 +186,7 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
             .ok_or_else(|| Failed::fit("Failed to get the labels of y."))?;
 
         let class_labels: Vec<T> = (0..*y_max + 1)
-            .map(|label| T::from(label).unwrap())
+            .map(|label| T::from_usize(label).unwrap())
             .collect();
         let mut class_count = vec![0_usize; class_labels.len()];
         for elem in y.iter() {
@@ -174,9 +196,9 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
         let mut n_categories: Vec<usize> = Vec::with_capacity(n_features);
         for feature in 0..n_features {
             let feature_max = x
-                .get_col_as_vec(feature)
-                .iter()
-                .map(|f_i| f_i.floor().to_usize().unwrap())
+                .get_col(feature)
+                .iterator(0)
+                .map(|f_i| f_i.to_usize().unwrap())
                 .max()
                 .ok_or_else(|| {
                     Failed::fit(&format!(
@@ -187,34 +209,32 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
             n_categories.push(feature_max + 1);
         }
 
-        let mut coefficients: Vec<Vec<Vec<T>>> = Vec::with_capacity(class_labels.len());
+        let mut coefficients: Vec<Vec<Vec<f64>>> = Vec::with_capacity(class_labels.len());
         let mut category_count: Vec<Vec<Vec<usize>>> = Vec::with_capacity(class_labels.len());
         for (feature_index, &n_categories_i) in n_categories.iter().enumerate().take(n_features) {
-            let mut coef_i: Vec<Vec<T>> = Vec::with_capacity(n_features);
+            let mut coef_i: Vec<Vec<f64>> = Vec::with_capacity(n_features);
             let mut category_count_i: Vec<Vec<usize>> = Vec::with_capacity(n_features);
             for (label, &label_count) in class_labels.iter().zip(class_count.iter()) {
                 let col = x
-                    .get_col_as_vec(feature_index)
-                    .iter()
+                    .get_col(feature_index)
+                    .iterator(0)
                     .enumerate()
-                    .filter(|(i, _j)| T::from(y[*i]).unwrap() == *label)
+                    .filter(|(i, _j)| T::from_usize(y[*i]).unwrap() == *label)
                     .map(|(_, j)| *j)
                     .collect::<Vec<T>>();
                 let mut feat_count: Vec<usize> = vec![0_usize; n_categories_i];
                 for row in col.iter() {
-                    let index = row.floor().to_usize().unwrap();
+                    let index = row.to_usize().unwrap();
                     feat_count[index] += 1;
                 }
 
                 let coef_i_j = feat_count
                     .iter()
-                    .map(|c| {
-                        ((T::from(*c).unwrap() + alpha)
-                            / (T::from(label_count).unwrap()
-                                + T::from(n_categories_i).unwrap() * alpha))
+                    .map(|&c| {
+                        ((c as f64 + alpha) / (label_count as f64 + n_categories_i as f64 * alpha))
                             .ln()
                     })
-                    .collect::<Vec<T>>();
+                    .collect::<Vec<f64>>();
                 category_count_i.push(feat_count);
                 coef_i.push(coef_i_j);
             }
@@ -224,8 +244,8 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
 
         let class_priors = class_count
             .iter()
-            .map(|&count| T::from(count).unwrap() / T::from(n_samples).unwrap())
-            .collect::<Vec<T>>();
+            .map(|&count| count as f64 / n_samples as f64)
+            .collect::<Vec<f64>>();
 
         Ok(Self {
             class_count,
@@ -242,140 +262,211 @@ impl<T: RealNumber> CategoricalNBDistribution<T> {
 /// `CategoricalNB` parameters. Use `Default::default()` for default values.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, Clone)]
-pub struct CategoricalNBParameters<T: RealNumber> {
+pub struct CategoricalNBParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
     /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
-    pub alpha: T,
+    pub alpha: f64,
 }
 
-impl<T: RealNumber> CategoricalNBParameters<T> {
+impl CategoricalNBParameters {
     /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
-    pub fn with_alpha(mut self, alpha: T) -> Self {
+    pub fn with_alpha(mut self, alpha: f64) -> Self {
         self.alpha = alpha;
         self
     }
 }
 
-impl<T: RealNumber> Default for CategoricalNBParameters<T> {
+impl Default for CategoricalNBParameters {
     fn default() -> Self {
-        Self { alpha: T::one() }
+        Self { alpha: 1f64 }
+    }
+}
+
+/// CategoricalNB grid search parameters
+#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone)]
+pub struct CategoricalNBSearchParameters {
+    #[cfg_attr(feature = "serde", serde(default))]
+    /// Additive (Laplace/Lidstone) smoothing parameter (0 for no smoothing).
+    pub alpha: Vec<f64>,
+}
+
+/// CategoricalNB grid search iterator
+pub struct CategoricalNBSearchParametersIterator {
+    categorical_nb_search_parameters: CategoricalNBSearchParameters,
+    current_alpha: usize,
+}
+
+impl IntoIterator for CategoricalNBSearchParameters {
+    type Item = CategoricalNBParameters;
+    type IntoIter = CategoricalNBSearchParametersIterator;
+
+    fn into_iter(self) -> Self::IntoIter {
+        CategoricalNBSearchParametersIterator {
+            categorical_nb_search_parameters: self,
+            current_alpha: 0,
+        }
+    }
+}
+
+impl Iterator for CategoricalNBSearchParametersIterator {
+    type Item = CategoricalNBParameters;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if self.current_alpha == self.categorical_nb_search_parameters.alpha.len() {
+            return None;
+        }
+
+        let next = CategoricalNBParameters {
+            alpha: self.categorical_nb_search_parameters.alpha[self.current_alpha],
+        };
+
+        self.current_alpha += 1;
+
+        Some(next)
+    }
+}
+
+impl Default for CategoricalNBSearchParameters {
+    fn default() -> Self {
+        let default_params = CategoricalNBParameters::default();
+
+        CategoricalNBSearchParameters {
+            alpha: vec![default_params.alpha],
+        }
     }
 }
 
 /// CategoricalNB implements the categorical naive Bayes algorithm for categorically distributed data.
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[derive(Debug, PartialEq)]
-pub struct CategoricalNB<T: RealNumber, M: Matrix<T>> {
-    inner: BaseNaiveBayes<T, M, CategoricalNBDistribution<T>>,
+pub struct CategoricalNB<T: Number + Unsigned, X: Array2<T>, Y: Array1<T>> {
+    inner: Option<BaseNaiveBayes<T, T, X, Y, CategoricalNBDistribution<T>>>,
 }
 
-impl<T: RealNumber, M: Matrix<T>> SupervisedEstimator<M, M::RowVector, CategoricalNBParameters<T>>
-    for CategoricalNB<T, M>
+impl<T: Number + Unsigned, X: Array2<T>, Y: Array1<T>>
+    SupervisedEstimator<X, Y, CategoricalNBParameters> for CategoricalNB<T, X, Y>
 {
-    fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: CategoricalNBParameters<T>,
-    ) -> Result<Self, Failed> {
+    fn new() -> Self {
+        Self {
+            inner: Option::None,
+        }
+    }
+
+    fn fit(x: &X, y: &Y, parameters: CategoricalNBParameters) -> Result<Self, Failed> {
         CategoricalNB::fit(x, y, parameters)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> Predictor<M, M::RowVector> for CategoricalNB<T, M> {
-    fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
+impl<T: Number + Unsigned, X: Array2<T>, Y: Array1<T>> Predictor<X, Y> for CategoricalNB<T, X, Y> {
+    fn predict(&self, x: &X) -> Result<Y, Failed> {
         self.predict(x)
     }
 }
 
-impl<T: RealNumber, M: Matrix<T>> CategoricalNB<T, M> {
+impl<T: Number + Unsigned, X: Array2<T>, Y: Array1<T>> CategoricalNB<T, X, Y> {
     /// Fits CategoricalNB with given data
     /// * `x` - training data of size NxM where N is the number of samples and M is the number of
     /// features.
     /// * `y` - vector with target values (classes) of length N.
     /// * `parameters` - additional parameters like alpha for smoothing
-    pub fn fit(
-        x: &M,
-        y: &M::RowVector,
-        parameters: CategoricalNBParameters<T>,
-    ) -> Result<Self, Failed> {
+    pub fn fit(x: &X, y: &Y, parameters: CategoricalNBParameters) -> Result<Self, Failed> {
         let alpha = parameters.alpha;
         let distribution = CategoricalNBDistribution::fit(x, y, alpha)?;
         let inner = BaseNaiveBayes::fit(distribution)?;
-        Ok(Self { inner })
+        Ok(Self { inner: Some(inner) })
     }
 
     /// Estimates the class labels for the provided data.
     /// * `x` - data of shape NxM where N is number of data points to estimate and M is number of features.
     /// Returns a vector of size N with class estimates.
-    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> {
-        self.inner.predict(x)
+    pub fn predict(&self, x: &X) -> Result<Y, Failed> {
+        self.inner.as_ref().unwrap().predict(x)
     }
 
     /// Class labels known to the classifier.
     /// Returns a vector of size n_classes.
     pub fn classes(&self) -> &Vec<T> {
-        &self.inner.distribution.class_labels
+        &self.inner.as_ref().unwrap().distribution.class_labels
     }
 
     /// Number of training samples observed in each class.
     /// Returns a vector of size n_classes.
     pub fn class_count(&self) -> &Vec<usize> {
-        &self.inner.distribution.class_count
+        &self.inner.as_ref().unwrap().distribution.class_count
     }
 
     /// Number of features of each sample
     pub fn n_features(&self) -> usize {
-        self.inner.distribution.n_features
+        self.inner.as_ref().unwrap().distribution.n_features
     }
 
     /// Number of features of each sample
     pub fn n_categories(&self) -> &Vec<usize> {
-        &self.inner.distribution.n_categories
+        &self.inner.as_ref().unwrap().distribution.n_categories
     }
 
     /// Holds arrays of shape (n_classes, n_categories of respective feature)
     /// for each feature. Each array provides the number of samples
     /// encountered for each class and category of the specific feature.
     pub fn category_count(&self) -> &Vec<Vec<Vec<usize>>> {
-        &self.inner.distribution.category_count
+        &self.inner.as_ref().unwrap().distribution.category_count
     }
     /// Holds arrays of shape (n_classes, n_categories of respective feature)
     /// for each feature. Each array provides the empirical log probability
     /// of categories given the respective feature and class, ``P(x_i|y)``.
-    pub fn feature_log_prob(&self) -> &Vec<Vec<Vec<T>>> {
-        &self.inner.distribution.coefficients
+    pub fn feature_log_prob(&self) -> &Vec<Vec<Vec<f64>>> {
+        &self.inner.as_ref().unwrap().distribution.coefficients
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::linalg::naive::dense_matrix::DenseMatrix;
+    use crate::linalg::basic::matrix::DenseMatrix;
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[test]
+    fn search_parameters() {
+        let parameters = CategoricalNBSearchParameters {
+            alpha: vec![1., 2.],
+            ..Default::default()
+        };
+        let mut iter = parameters.into_iter();
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 1.);
+        let next = iter.next().unwrap();
+        assert_eq!(next.alpha, 2.);
+        assert!(iter.next().is_none());
+    }
+
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_categorical_naive_bayes() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[0., 2., 1., 0.],
-            &[0., 2., 1., 1.],
-            &[1., 2., 1., 0.],
-            &[2., 1., 1., 0.],
-            &[2., 0., 0., 0.],
-            &[2., 0., 0., 1.],
-            &[1., 0., 0., 1.],
-            &[0., 1., 1., 0.],
-            &[0., 0., 0., 0.],
-            &[2., 1., 0., 0.],
-            &[0., 1., 0., 1.],
-            &[1., 1., 1., 1.],
-            &[1., 2., 0., 0.],
-            &[2., 1., 1., 1.],
+        let x = DenseMatrix::<u32>::from_2d_array(&[
+            &[0, 2, 1, 0],
+            &[0, 2, 1, 1],
+            &[1, 2, 1, 0],
+            &[2, 1, 1, 0],
+            &[2, 0, 0, 0],
+            &[2, 0, 0, 1],
+            &[1, 0, 0, 1],
+            &[0, 1, 1, 0],
+            &[0, 0, 0, 0],
+            &[2, 1, 0, 0],
+            &[0, 1, 0, 1],
+            &[1, 1, 1, 1],
+            &[1, 2, 0, 0],
+            &[2, 1, 1, 1],
         ]);
-        let y = vec![0., 0., 1., 1., 1., 0., 1., 0., 1., 1., 1., 1., 1., 0.];
+        let y: Vec<u32> = vec![0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0];
 
         let cnb = CategoricalNB::fit(&x, &y, Default::default()).unwrap();
 
         // checking parity with scikit
-        assert_eq!(cnb.classes(), &[0., 1.]);
+        assert_eq!(cnb.classes(), &[0, 1]);
         assert_eq!(cnb.class_count(), &[5, 9]);
         assert_eq!(cnb.n_features(), 4);
         assert_eq!(cnb.n_categories(), &[3, 3, 2, 2]);
@@ -427,65 +518,70 @@ mod tests {
             ]
         );
 
-        let x_test = DenseMatrix::from_2d_array(&[&[0., 2., 1., 0.], &[2., 2., 0., 0.]]);
+        let x_test = DenseMatrix::from_2d_array(&[&[0, 2, 1, 0], &[2, 2, 0, 0]]);
         let y_hat = cnb.predict(&x_test).unwrap();
-        assert_eq!(y_hat, vec![0., 1.]);
+        assert_eq!(y_hat, vec![0, 1]);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     fn run_categorical_naive_bayes2() {
-        let x = DenseMatrix::from_2d_array(&[
-            &[3., 4., 0., 1.],
-            &[3., 0., 0., 1.],
-            &[4., 4., 1., 2.],
-            &[4., 2., 4., 3.],
-            &[4., 2., 4., 2.],
-            &[4., 1., 1., 0.],
-            &[1., 1., 1., 1.],
-            &[0., 4., 1., 0.],
-            &[0., 3., 2., 1.],
-            &[0., 3., 1., 1.],
-            &[3., 4., 0., 1.],
-            &[3., 4., 2., 4.],
-            &[0., 3., 1., 2.],
-            &[0., 4., 1., 2.],
+        let x = DenseMatrix::<u32>::from_2d_array(&[
+            &[3, 4, 0, 1],
+            &[3, 0, 0, 1],
+            &[4, 4, 1, 2],
+            &[4, 2, 4, 3],
+            &[4, 2, 4, 2],
+            &[4, 1, 1, 0],
+            &[1, 1, 1, 1],
+            &[0, 4, 1, 0],
+            &[0, 3, 2, 1],
+            &[0, 3, 1, 1],
+            &[3, 4, 0, 1],
+            &[3, 4, 2, 4],
+            &[0, 3, 1, 2],
+            &[0, 4, 1, 2],
         ]);
-        let y = vec![0., 0., 1., 1., 1., 0., 1., 0., 1., 1., 1., 1., 1., 0.];
+        let y: Vec<u32> = vec![0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0];
 
         let cnb = CategoricalNB::fit(&x, &y, Default::default()).unwrap();
         let y_hat = cnb.predict(&x).unwrap();
-        assert_eq!(
-            y_hat,
-            vec![0., 0., 1., 1., 1., 0., 1., 0., 1., 1., 0., 1., 1., 1.]
-        );
+        assert_eq!(y_hat, vec![0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1]);
+
+        println!("{}", &cnb);
     }
 
-    #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
+    #[cfg_attr(
+        all(target_arch = "wasm32", not(target_os = "wasi")),
+        wasm_bindgen_test::wasm_bindgen_test
+    )]
     #[test]
     #[cfg(feature = "serde")]
     fn serde() {
-        let x = DenseMatrix::<f64>::from_2d_array(&[
-            &[3., 4., 0., 1.],
-            &[3., 0., 0., 1.],
-            &[4., 4., 1., 2.],
-            &[4., 2., 4., 3.],
-            &[4., 2., 4., 2.],
-            &[4., 1., 1., 0.],
-            &[1., 1., 1., 1.],
-            &[0., 4., 1., 0.],
-            &[0., 3., 2., 1.],
-            &[0., 3., 1., 1.],
-            &[3., 4., 0., 1.],
-            &[3., 4., 2., 4.],
-            &[0., 3., 1., 2.],
-            &[0., 4., 1., 2.],
+        let x = DenseMatrix::from_2d_array(&[
+            &[3, 4, 0, 1],
+            &[3, 0, 0, 1],
+            &[4, 4, 1, 2],
+            &[4, 2, 4, 3],
+            &[4, 2, 4, 2],
+            &[4, 1, 1, 0],
+            &[1, 1, 1, 1],
+            &[0, 4, 1, 0],
+            &[0, 3, 2, 1],
+            &[0, 3, 1, 1],
+            &[3, 4, 0, 1],
+            &[3, 4, 2, 4],
+            &[0, 3, 1, 2],
+            &[0, 4, 1, 2],
         ]);
 
-        let y = vec![0., 0., 1., 1., 1., 0., 1., 0., 1., 1., 1., 1., 1., 0.];
+        let y: Vec<u32> = vec![0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0];
         let cnb = CategoricalNB::fit(&x, &y, Default::default()).unwrap();
 
-        let deserialized_cnb: CategoricalNB<f64, DenseMatrix<f64>> =
+        let deserialized_cnb: CategoricalNB<u32, DenseMatrix<u32>, Vec<u32>> =
             serde_json::from_str(&serde_json::to_string(&cnb).unwrap()).unwrap();
 
         assert_eq!(cnb, deserialized_cnb);