MAXIMUM MACHINE LEARNING COVERAGE WITH LIMITED DATA
Both global and local robustness play essential roles in designing AI systems with superior performance and adaptability. The key is striking a balance between the two, depending on the specific application requirements.
For achieving local robustness, the amount of data required can be considerably smaller compared to global robustness. The training data needs to be concentrated on the targeted region of interest, providing sufficient coverage of the inputs relevant to the specific task. We call this the specification set which consists of an infinite amount of possible training samples.
Infinite? So, how much coverage, i.e. guarantee of success, can be achieved?
Machine Learning Coverage (MLC) in a specification set
No matter how many training samples you collect, the result will always be:
0 % of MLC, since the mathematically required number of training samples in an infinite continuous space is infinite!
This proves our case that for robust neural network training, you never have enough data to achieve 100 % MLC.
Unless… you apply the Spiki approach, which limits the amount of samples needed to a finite amount and, at the same time, clearly specifies which samples need to be measured in order to ensure robustness.
Excited? Get in touch and find out how Spiki’s AI can help boost your application in robotics, transportation, smart home, speech command recognition and other domains!
We have already learnt that a neural network architecture is based on specifications and measured datapoints for training. The more parameters and dimensional axes, the more complex and harder to robustly train the network. Need an example?
Local Robustness
Such a Lego block is rather straightforward to describe and to specify: it has a specific length, width and height, a specific colour, and it has a specific number of knobs. These are five main characteristics which can be clearly defined and measured.
A Lego block is a prime example of local robustness. It has a limited number of dimensional axes and parameters. It can be clearly specified.
Now, with each Lego block added, you square the number of combinable parameters: length, width, height, colour and knobs might differ with each block. Imagine how many combinable parameters you get only in a very simple lego house, consisting of less than 500 blocks: The number rises expontentially!
.
… vs. Global Robustness!
Global Robustness
Now take this Lego bulldozer. Are you able to discern single blocks with their particular characteristics? Can you pinpoint how they are built together? There may be several ways in which single bolts, bits and pieces are put together. Looking at the whole vehicle, it is not possible to clearly distinguish and specify each and every item and its parameters.
Add movement to this list set of parameters – the bulldozer moves through time and space! You may guess…
Global robustness encompasses an innummerable amount of variables and parameters. It cannot be clearly specified.
An infinite amount of data would be needed in order to process such a vast system as a whole. Local robustness on the other hand limits this amount to a finite number which can actually be handled very well.
This is the Spiki approach to neural network training. It ensures maximum machine learning coverage (MLC), i.e. accuracy within a specified range of operation, the specification space. Find out more in our next article.
Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust.
Several factors contribute to the robustness of AI models during the training phase. These include the quality and diversity of the training data, the choice of architecture and hyperparameters, the regularization techniques used, and the optimization algorithms employed. The careful design of these elements can significantly impact the model’s ability to handle uncertainties and generalizability to new data.
Global vs. Local Robustness explained
In the pursuit of robustness, two essential concepts emerge: global robustness and local robustness. Understanding the distinction between these two aspects can help us design AI models that strike the right balance between generalization and adaptability.
Global Robustness in AI Systems: aiming for Reliability
Global robustness refers to an AI model’s ability to maintain its performance across a broad range of inputs, even when those inputs deviate significantly from the training data. A globally robust model can handle various perturbations and distribution shifts without compromising its reliability. In essence, global robustness focuses on the model’s ability to generalize well to unseen data and diverse conditions.
Characteristics of globally robust models include:
Generalization to Unseen Data: Globally robust models demonstrate strong generalization capabilities, making them reliable when exposed to new, previously unseen data.
Stable Performance: These models exhibit consistent performance under various environmental conditions and input variations, making them more dependable in real-world applications.
Fewer Adversarial Vulnerabilities: Globally robust models are less susceptible to adversarial attacks, making them more secure against attempts to deceive or manipulate the AI system.
Global robustness allows AI models to perform well in different environments, making them applicable across diverse use cases. However, pursuing global robustness may lead to a performance trade-off, where the model’s accuracy on specific tasks may not be as high as specialized models.
Achieving global robustness requires more complex architectures and training procedures, leading to higher computational costs, since all parameters and all their variations need to be taken into account. This in turn means that you need infinite training data to realize global robustness.
Higher accuracy: Local Robustness in AI Systems
Local robustness, on the other hand, emphasizes an AI model’s ability to perform well in a specific region of the input space. Instead of focusing on broad generalization, locally robust models are optimized to excel within a limited range of inputs, potentially allowing for higher accuracy on those specific inputs.
Characteristics of locally robust models include:
High Performance in Targeted Areas: Locally robust models may outperform globally robust models in specific regions of the input space relevant to the task at hand.
Potential for Specialization: These models can be fine-tuned to excel in specific niche applications, making them valuable in domain-specific scenarios.
Local robustness allows for higher accuracy on specific tasks, making them ideal for targeted applications. Focusing on local robustness can result in simpler models and faster training times, since you need fewer samples to train the model at hand.
In the next article, let us examine what this means in practical terms for neural network training and how the two approaches can match.
Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust.
Artificial Intelligence (AI) has rapidly become an integral part of our lives. It is revolutionizing industries such as healthcare, aviation, transportation, smart homes and more. From personalized recommendations to communication or autonomous vehicles, AI’s transformative capabilities are reshaping the way we live and work.
Importance of AI Robustness in Ensuring Reliable and Safe Systems
As AI continues to expand its reach, ensuring the reliability and safety of AI systems becomes paramount.
Robustness in AI training plays a critical role in guaranteeing that these systems perform consistently, accurately, and predictably under different circumstances. A robust AI model can withstand various challenges, including noisy data, changes in the environment, and even deliberate attempts to deceive the system.
In training, robustness is the process of equipping AI models to handle different scenarios effectively and maintain their accuracy and reliability in real-world applications. This can be achieved with two approaches: training for global or for local robustness, which are discussed in our next article.
Regulatory attempts and guidelines will shape AI development
Many nations, supranational bodies and institutions are currently working on regulations for AI development and deployment. Their main goal is to enhance trust and ensure the accountability, transparency and reliability of AI, which is of particular importance in high stake safey critical use cases. Independent auditing companies like TÜV SÜD and EASA are providing guidelines on this matter, too (see for example European Union Aviation Safety Agency 2021, EASA Concept Paper First usable guidance for Level 1 machine learning applications, Issue 01, Link).
Uncertainties are inevitable in real-world data, arising from noisy sensors, varying environmental conditions, and incomplete information. Robust AI models should be able to account for and cope with such uncertainties to provide reliable results. Additionally, adversarial inputs are crafted with the intent of deceiving AI systems, making robustness against these attacks crucial for maintaining security and trust in AI applications.
Find out how we approach this problem and how global and local robustness come into play in our next articles!
Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust.
We have already learnt that neural network training requires a vast amount of data to effectively capture complex patterns and generalize well. This need for extensive data is expensive for companies due to the costs associated with data collection, storage, labeling, and the computational resources required for training large-scale models, making it a resource-intensive endeavor.
Machine learning and neural network training can work with various types of data, but the choice often depends on the specific problem and the physical nature of the data available. Here are some common types of data used:
Image and Video Data: For example object recognition in images, where you identify and classify objects within photographs.
Audio Data: For example speech recognition, where you transcribe spoken words into text.
Multi-modal Data: For example autonomous vehicle perception, where data from cameras, LIDAR, radar, and other sensors are combined to make driving decisions.
Sensor Data: For example predictive maintenance in manufacturing, where data from sensors on machinery is used to predict when maintenance is needed to avoid breakdowns.
The choice of data type and representation depends on the problem’s requirements and the information available.
Discrete vs. Continuous Data in NN Training
Two different types of data can be used in neural network training according to the task at hand:
Discrete Data: Discrete data consists of distinct, separate, and countable values. These values often represent categories, counts, or labels with clear boundaries. Examples include categorical variables (e.g., types of animals, colors), ordinal variables (e.g., levels of satisfaction), or count data (e.g., the number of cars in a parking lot).
Continuous Data: Continuous data, on the other hand, represents a continuum of values with no clear separation between them. This type of data can take on any value within a given range. Examples include numerical variables (e.g., temperature, height, weight) and real-valued measurements (e.g., time, distance).
The handling, representation, and preprocessing of these data types in neural network training differ based on their fundamental nature.
Continuous Data Complexity: Managing the Infinite Possibilities
Handling continuous data can be more challenging compared to discrete data due to several reasons:
Infinite Possible Values: Continuous data can take on an infinite number of values within a given range. This makes it computationally intensive to work with, as you can’t store or process every possible value individually. In contrast, discrete data has a finite set of possible values, making it easier to manage.
Precision and Noise: Continuous data often involves measurements and observations that come with varying degrees of precision and noise. This introduces uncertainty into the data and requires careful handling to account for measurement errors and variations.
Data Representation: Discrete data can be easily represented using integers or categorical labels, while continuous data requires more complex representations, usually involving floating-point numbers. This adds complexity to processing and storage.
Granularity: Continuous data can be extremely granular, requiring sophisticated techniques to capture meaningful patterns. Discrete data might already come in a more structured and understandable format.
Dimensionality: Continuous data often leads to high-dimensional feature spaces, especially when dealing with multiple continuous variables. This can result in the “curse of dimensionality,” where distance-based methods struggle due to increased sparsity of data points.
Algorithm Sensitivity: Many algorithms are designed for discrete data or work better with it. Adapting these algorithms to continuous data requires careful consideration and often additional mathematical techniques.
Measuring the world?
In summary, handling continuous data requires a deeper understanding of the underlying mathematical properties, domain-specific considerations, and often the use of specialized algorithms and techniques to effectively process and extract meaningful insights from the data.
The goal of machine learning is to create models that generalize well to unseen data, which is termed robustness. Achieving good generalization is partly dependent on having an infinite amount of data but also on having enough diverse and representative data to capture the underlying patterns in the data distribution.
High-dimensional continuous data tends to result in a larger number of parameters, especially if you have many continuous features such as movement in time and space. Since the number of parameters rises exponentially, it gets harder to capture all the necessary measurements and input data needed for robust training. This is where a tradeoff between local and global robustness comes into play when trying to solve the “never enough data problem”.
Find out how we approach this problem in our next articles!
Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust.
