Search
  • Panaton

Types of ML Algorithms - grouped and explained

To better understand the Machine Learning algorithms – you need to know how they group around their similarities. This is why in this article we wanted to present to you the different types of ML Algorithms. By understanding their close relationship and also their differences you will be able to implement the right one in every single case.


  1. Supervised Learning Algorithms – ML model consists of a target (outcome variable/label) by a given set of observations or a dependent variable predicted by a certain set of predictors (independent variables). By using this set of variables, we can generate a working function that maps all inputs to the desired outputs to predict new observations. This kind of ML training process continues until the model achieves a desired level of accuracy on the training data. Supervised learning algorithms are usually grouped by the characteristics of the target variable which can be classification (discrete target variable), regression (continuous target variable), or forecasting (feature predictions based on past and present data used for analyzing trends).

  2. Unsupervised Learning Algorithms – unlike the supervised ML algorithm, this one does not have any desired target or outcome beforehand. The machine is given large data input and it is left unsupervised to determine relationships/correlations between the available data. The main goal here is to recognize patterns by clustering data in different datasets and grouping them in a way that makes sense. By organizing the data, ML is arranging it into clusters: the first type of unsupervised ML algorithm. On the segmented data that is been given – it performs separate analysis on each data set to find patterns. The other type of unsupervised learning algorithm is called dimension reduction. Its name comes from the function that it performs – it decreases the number of variables used for finding the desired information.

  3. Reinforcement Learning Algorithms – here ML algorithms are trained to make a specific decision based on the trial and error approach. It learns from experience and tries to avoid wrong patterns so it can make accurate business decisions. Reinforcement machine learning algorithms constantly evaluate each result, explore different possibilities/options and monitor current data for optimal outcomes. It makes adapted responses based on the situation to reach the best-desired result.

Algorithms Grouped by Similarity


Often ML algorithms are also grouped by the way they work. It is a useful approach because you can easily know that they process data similarly. It is not the best way to distinguish them but it serves the purpose of intuitively separating them. Here are the 10 most commonly used ML algorithms:

  1. Regression Algorithms – it is a process that models the relationship between variables that is iteratively refined using a measure of the error in the predictions made by the model. This ML method is mostly used for statistics. - Linear Regression - Logistic Regression - Stepwise Regression - Multivariate Adaptive Regression Splines (MARS) - Locally Estimated Scatterplot Smoothing (LOESS)

  2. Instance-based Algorithms – models’ decision-based problem algorithms with instances of training data that are important / required to the ML model. It is used to make predictions, to find the best match possible, and find similarities. It is also called a memory-based machine learning algorithm. - k-Nearest Neighbor (kNN) - Learning Vector Quantization (LVQ) - Self-Organizing Map (SOM) - Locally Weighted Learning (LWL) - Support Vector Machines (SVM)

  3. Regularization Algorithms – it simplifies another method (usually regression methods). It penalizes models based on their complexity. - Ridge Regression - Least Absolute Shrinkage and Selection Operator (LASSO) - Elastic Net - Least-Angle Regression (LARS)

  4. Decision Tree Algorithms – ML model based on decisions that come from values of attributes in the data. - Classification and Regression Tree (CART) - Iterative Dichotomiser 3 (ID3) - C4.5 and C5.0 (different versions of a powerful approach) - Chi-squared Automatic Interaction Detection (CHAID) - Decision Stump - M5 - Conditional Decision Trees

  5. Clustering Algorithms – centroid-based and hierarchal ML method that describes the class of problem/methods. - k-Means - k-Medians - Expectation Maximisation (EM) - Hierarchical Clustering

  6. Association Rule Learning Algorithms – finds rules that best explain the relationships between different variables in data. - Apriori algorithm - Eclat algorithm

  7. Artificial Neural Network Algorithms (ANNA) – following the model of biological neural networks. - Perceptron - Multilayer Perceptrons (MLP) - Back-Propagation - Stochastic Gradient Descent - Hopfield Network - Radial Basis Function Network (RBF

  8. Deep Learning Algorithms – based on ANNA with many other computations. - Convolutional Neural Network (CNN) - Recurrent Neural Networks (RNNs) - Long Short-Term Memory Networks (LSTMs) - Stacked Auto-Encoders - Deep Boltzmann Machine (DBM) - Deep Belief Networks (DBN)

  9. Dimensionality Reduction Algorithms – ML method that summarizes data while using less information. - Principal Component Analysis (PCA) - Principal Component Regression (PCR) - Partial Least Squares Regression (PLSR) - Sammon Mapping - Multidimensional Scaling (MDS) - Projection Pursuit - Linear Discriminant Analysis (LDA) - Mixture Discriminant Analysis (MDA) - Quadratic Discriminant Analysis (QDA) - Flexible Discriminant Analysis (FDA)

  10. Ensemble Algorithms – created by many weak models that are trained separately and their predictions are combined to make an overall prediction. - Boosting - Bootstrapped Aggregation (Bagging) - AdaBoost - Weighted Average (Blending) - Stacked Generalization (Stacking) - Gradient Boosting Machines (GBM) - Gradient Boosted Regression Trees (GBRT)Sammon Mapping - Random Forest

There are of course some ML algorithms that are more popular or more used than others, but the best one for you might not be one of these two. The most valuable ML algorithm for you is the one that will give you the result you desire. If you would like to know what will fit your business model and have a high-level understanding of ML algorithms – be sure to hire a data scientist that will guide you along the way.