Tradeoff: Global  vs. Local Robustness?  

Tradeoff: Global vs. Local Robustness?  

Striking the Balance in AI Training

Several factors contribute to the robustness of AI models during the training phase. These include the quality and diversity of the training data, the choice of architecture and hyperparameters, the regularization techniques used, and the optimization algorithms employed. The careful design of these elements can significantly impact the model’s ability to handle uncertainties and generalizability to new data.  

Global vs. Local Robustness explained 

In the pursuit of robustness, two essential concepts emerge: global robustness and local robustness. Understanding the distinction between these two aspects can help us design AI models that strike the right balance between generalization and adaptability. 

Global Robustness in AI Systems: aiming for Reliability

Global robustness refers to an AI model’s ability to maintain its performance across a broad range of inputs, even when those inputs deviate significantly from the training data. A globally robust model can handle various perturbations and distribution shifts without compromising its reliability. In essence, global robustness focuses on the model’s ability to generalize well to unseen data and diverse conditions. 

Characteristics of globally robust models include: 

Generalization to Unseen Data: Globally robust models demonstrate strong generalization capabilities, making them reliable when exposed to new, previously unseen data. 

Stable Performance: These models exhibit consistent performance under various environmental conditions and input variations, making them more dependable in real-world applications. 

Fewer Adversarial Vulnerabilities: Globally robust models are less susceptible to adversarial attacks, making them more secure against attempts to deceive or manipulate the AI system. 

Global robustness allows AI models to perform well in different environments, making them applicable across diverse use cases. However, pursuing global robustness may lead to a performance trade-off, where the model’s accuracy on specific tasks may not be as high as specialized models. 

Achieving global robustness requires more complex architectures and training procedures, leading to higher computational costs, since all parameters and all their variations need to be taken into account. This in turn means that you need infinite training data to realize global robustness. 

Higher accuracy: Local Robustness in AI Systems 

Local robustness, on the other hand, emphasizes an AI model’s ability to perform well in a specific region of the input space. Instead of focusing on broad generalization, locally robust models are optimized to excel within a limited range of inputs, potentially allowing for higher accuracy on those specific inputs. 

Characteristics of locally robust models include: 

High Performance in Targeted Areas: Locally robust models may outperform globally robust models in specific regions of the input space relevant to the task at hand. 

Potential for Specialization: These models can be fine-tuned to excel in specific niche applications, making them valuable in domain-specific scenarios. 

Local robustness allows for higher accuracy on specific tasks, making them ideal for targeted applications. Focusing on local robustness can result in simpler models and faster training times, since you need fewer samples to train the model at hand.

In the next article, let us examine what this means in practical terms for neural network training and how the two approaches can match. 

Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust. 

Global vs. Local Robustness in AI Training   

Global vs. Local Robustness in AI Training   

Games of Thought

Artificial Intelligence (AI) has rapidly become an integral part of our lives. It is revolutionizing industries such as healthcare, aviation, transportation, smart homes and more. From personalized recommendations to communication or autonomous vehicles, AI’s transformative capabilities are reshaping the way we live and work. 

Importance of AI Robustness in Ensuring Reliable and Safe Systems

As AI continues to expand its reach, ensuring the reliability and safety of AI systems becomes paramount.

Robustness in AI training plays a critical role in guaranteeing that these systems perform consistently, accurately, and predictably under different circumstances. A robust AI model can withstand various challenges, including noisy data, changes in the environment, and even deliberate attempts to deceive the system. 

In training, robustness is the process of equipping AI models to handle different scenarios effectively and maintain their accuracy and reliability in real-world applications. This can be achieved with two approaches: training for global or for local robustness, which are discussed in our next article. 

Regulatory attempts and guidelines will shape AI development

Many nations, supranational bodies and institutions are currently working on regulations for AI development and deployment. Their main goal is to enhance trust and ensure the accountability, transparency and reliability of AI, which is of particular importance in high stake safey critical use cases. Independent auditing companies like TÜV SÜD and EASA are providing guidelines on this matter, too (see for example European Union Aviation Safety Agency 2021, EASA Concept Paper First usable guidance for Level 1 machine learning applications, Issue 01, Link). 

Uncertainties are inevitable in real-world data, arising from noisy sensors, varying environmental conditions, and incomplete information. Robust AI models should be able to account for and cope with such uncertainties to provide reliable results. Additionally, adversarial inputs are crafted with the intent of deceiving AI systems, making robustness against these attacks crucial for maintaining security and trust in AI applications. 

Find out how we approach this problem and how global and local robustness come into play in our next articles! 

Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust. 

The Data Bottleneck and how we approach it  

The Data Bottleneck and how we approach it  

Diverse, complex, and never enough

We have already learnt that neural network training requires a vast amount of data to effectively capture complex patterns and generalize well. This need for extensive data is expensive for companies due to the costs associated with data collection, storage, labeling, and the computational resources required for training large-scale models, making it a resource-intensive endeavor. 

Machine learning and neural network training can work with various types of data, but the choice often depends on the specific problem and the physical nature of the data available. Here are some common types of data used:  

  • Image and Video Data: For example object recognition in images, where you identify and classify objects within photographs. 
  • Audio Data: For example speech recognition, where you transcribe spoken words into text. 
  • Multi-modal Data: For example autonomous vehicle perception, where data from cameras, LIDAR, radar, and other sensors are combined to make driving decisions. 
  • Sensor Data: For example predictive maintenance in manufacturing, where data from sensors on machinery is used to predict when maintenance is needed to avoid breakdowns. 

The choice of data type and representation depends on the problem’s requirements and the information available.  

Discrete vs. Continuous Data in NN Training 

Two different types of data can be used in neural network training according to the task at hand: 

  • Discrete Data: Discrete data consists of distinct, separate, and countable values. These values often represent categories, counts, or labels with clear boundaries. Examples include categorical variables (e.g., types of animals, colors), ordinal variables (e.g., levels of satisfaction), or count data (e.g., the number of cars in a parking lot). 
  • Continuous Data: Continuous data, on the other hand, represents a continuum of values with no clear separation between them. This type of data can take on any value within a given range. Examples include numerical variables (e.g., temperature, height, weight) and real-valued measurements (e.g., time, distance). 

The handling, representation, and preprocessing of these data types in neural network training differ based on their fundamental nature.  

Continuous Data Complexity: Managing the Infinite Possibilities

Handling continuous data can be more challenging compared to discrete data due to several reasons: 

  • Infinite Possible Values: Continuous data can take on an infinite number of values within a given range. This makes it computationally intensive to work with, as you can’t store or process every possible value individually. In contrast, discrete data has a finite set of possible values, making it easier to manage. 
  • Precision and Noise: Continuous data often involves measurements and observations that come with varying degrees of precision and noise. This introduces uncertainty into the data and requires careful handling to account for measurement errors and variations. 
  • Data Representation: Discrete data can be easily represented using integers or categorical labels, while continuous data requires more complex representations, usually involving floating-point numbers. This adds complexity to processing and storage. 
  • Granularity: Continuous data can be extremely granular, requiring sophisticated techniques to capture meaningful patterns. Discrete data might already come in a more structured and understandable format. 
  • Dimensionality: Continuous data often leads to high-dimensional feature spaces, especially when dealing with multiple continuous variables. This can result in the “curse of dimensionality,” where distance-based methods struggle due to increased sparsity of data points. 
  • Algorithm Sensitivity: Many algorithms are designed for discrete data or work better with it. Adapting these algorithms to continuous data requires careful consideration and often additional mathematical techniques.

Measuring the world? 

In summary, handling continuous data requires a deeper understanding of the underlying mathematical properties, domain-specific considerations, and often the use of specialized algorithms and techniques to effectively process and extract meaningful insights from the data. 

The goal of machine learning is to create models that generalize well to unseen data, which is termed robustness. Achieving good generalization is partly dependent on having an infinite amount of data but also on having enough diverse and representative data to capture the underlying patterns in the data distribution. 

High-dimensional continuous data tends to result in a larger number of parameters, especially if you have many continuous features such as movement in time and space. Since the number of parameters rises exponentially, it gets harder to capture all the necessary measurements and input data needed for robust training. This is where a tradeoff between local and global robustness comes into play when trying to solve the “never enough data problem”. 

Find out how we approach this problem in our next articles! 

Excited? Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust. 

Is Robust AI an Asset for YOUR Business?   

Is Robust AI an Asset for YOUR Business?   

Where robustness is needed in real life applications

After some well-founded theoretical observations about robustness in our latest articles, let us put them to the test. There are many real-world use cases where robustness is crucial for AI systems, and where implementing robustness techniques can lead to significant cost savings. Here are a few examples: 

Autonomous vehicles: Self-driving cars rely heavily on computer vision algorithms to navigate roads, avoid obstacles, and make split-second decisions. However, these algorithms are only as good as the data they’ve been trained on, and they can be vulnerable to unexpected inputs like extreme weather conditions or unusual road configurations. By building robustness techniques into the training process, developers can help ensure that autonomous vehicles are able to handle a wide range of inputs and scenarios, which can improve safety and reduce the risk of accidents. Additionally, a robust system may be less likely to require retraining or modifications down the line, which can save significant time and money in the long run. 

Aviation: Robustness techniques are highly relevant in the aviation industry because they help ensure the safety, reliability, and efficiency of aircraft operations. By employing these techniques, aviation systems can handle unforeseen circumstances, adversarial attacks, and data anomalies, which are crucial in such a safety-critical domain. AI models can, for example, be trained to detect and diagnose faults in aircraft systems. Robustness techniques enable these models to handle noisy sensor data, intermittent failures, and partial system malfunctions. By accurately identifying faults in real-time, maintenance crews can take immediate actions to prevent potential safety risks. Secondly, machine learning models in aviation often need to make decisions based on limited or uncertain data. Robustness techniques enable the models to quantify and propagate uncertainties, providing decision-makers with more reliable predictions. This is particularly crucial for tasks like weather forecasting, route planning, or aircraft trajectory optimization, where uncertainties can significantly impact safety and operational efficiency. 

Medical diagnosis: AI systems are increasingly being used to assist with medical diagnosis, but the accuracy and reliability of these systems is critical. In medical settings, a misdiagnosis can have serious consequences for patients, and it’s essential that AI systems be able to handle unexpected inputs like rare diseases or unusual symptoms. By implementing robustness techniques during training, developers can help ensure that these systems are better able to generalize to new inputs and make more accurate predictions. This can lead to better patient outcomes, reduced healthcare costs, and potentially significant cost savings for hospitals and healthcare providers. 

Fraud detection: Fraud is a major problem in many industries, and AI systems can be used to help detect and prevent fraudulent activity. However, fraudsters are often highly skilled at exploiting weaknesses in AI systems, and it’s essential that these systems be able to resist adversarial attacks. By building robustness techniques into the training process, developers can help ensure that fraud detection systems are more resilient to these attacks and better able to detect and prevent fraud. This can lead to significant cost savings for businesses and financial institutions, as fraudulent activity can be extremely costly and damaging. 

Natural language processing: Natural language processing (NLP) is a rapidly growing field that has many applications, from chatbots to virtual assistants to language translation. However, NLP models are often highly sensitive to variations in language use, and they can struggle to handle unexpected inputs like misspellings or regional dialects. By implementing robustness techniques like data augmentation and regularization during training, developers can help ensure that NLP models are better able to handle these variations and make more accurate predictions. This can lead to more effective communication with customers and significant cost savings for businesses that rely on NLP-based systems. 

Overall, there are many real-world applications where robustness is crucial for AI systems, and where implementing robustness techniques during training can lead to significant cost savings. By building robustness into AI systems from the start, developers can help ensure that these systems are better able to handle unexpected inputs, resist adversarial attacks, and make more accurate predictions. This can lead to improved safety, better patient outcomes, reduced fraud, and more effective communication with customers, all while potentially saving time and money. 

Conclusion: leave the work to Spiki 

In conclusion, robustness is a crucial component of neural network training that can improve performance and save time and money in the long run. By implementing robustness techniques, AI developers can create systems that are better able to handle unexpected inputs and resist adversarial attacks. The benefits can be substantial, especially in high-stakes applications like healthcare and finance. As AI continues to play an increasingly important role in our lives, robustness will become even more important. 

Creating a robust neural network is both costly and time-consuming. Collecting and processing the input data needed, and training, testing and retraining the model are huge challenges for companies not specialised in this field. So why not leave those tasks to Spiki?   

We have developed a unique approach to limit the amount of data needed for our neural network training. Our clients get a fully trained neural network model, which can be deployed via microcontroller, FPGA, as a cloud service or even ASIC as a future step in our product development. 


Get in touch and learn how to unlock the full potential of your business with Spiki’s AI you can trust. 

Spiki’s revolutionary voice command recognition

Spiki’s revolutionary voice command recognition

Frustrated with hard-to-use smart devices?


You know the feeling? Just bought a new, shiny smart phone but you hardly find your way through it. Endlessly clicking on icons and checkboxes makes you dizzy. You try to hit all the right buttons but it will not do what you want, and without Internet connection, it is not so smart any more.  

We feel you.  

This is why Spiki makes smart phones – or any smart device, for that matter – understand you. Our mission: make navigating your smart device easy for you. 

Spiki develops and implements easy, intuitive voice command recognition for customer-specific digital navigation needs. Why is this even necessary? 

Get out of the digital maze – with Spiki. 

Did you ever end up in a digital navigation menu without knowing how you got there – or how to get out of it again? For many elderly people, navigating a phone, let alone a smart phone, is a digital maze. You have to know exactly which functions to activate and how and where to press the right buttons. Doing this with bad eyesight or clumsy fingers can be a challenge.  

A recent study underlines the various barriers to smartphone and tablet use by senior citizens. Users of different versions of dedicated “senior mobile phones”, which have bigger buttons and a bigger display, frequently report that just reducing the buttons as compared to a conventional smart phone and arranging them all over the device does not make handling any easier. They still have to know where they want to navigate to and which buttons to press.

A smart phone should do exactly what you want it to. Always.  

We imagined it would be much easier if phones or tablets could process spoken commands to do exactly what is required. And they should do so 100 % reliably, in any circumstances. Handling any smart device with Spiki’s digital navigation is as easy as talking to yourself. Simple commands which are intuitive and do not require Internet connection help seniors, and all other smart device owners, reclaim their confidence and feel at ease with their smart device. 

Obvious commands like “call my doctor”, “switch on torchlight”, “look up weather forecast”, “open WhatsApp” or “write SMS to my daughter” are comlemented with life-simplifying functions like ringing and vibrating when asked “where’s my phone” or calling help if you call out for “help”. It goes without saying that the Spiki voice command recognition operates even in a noisy environment, or if the users suffer from speech slur.  

Without needing the Internet or any extra hardware, users have full control over their mobile phone and tablet, with their hands free. 

Spiki’s voice command recognition excels in many application areas. 

  • It works without Internet connectivity  
  • It ensures privacy of customer data  
  • It is usable in noisy environments  
  • It requires no additional hardware (e.g., Alexa) 
  • The hotword detection – shown as a red dot in the following comparative chart – is four times more accurate (99.48 % accuracy) than existing methods, even restricted to a similar model size (measured in KiloBytes): 
Spiki's model outperforms similar speech recognition models by a factor of 4.

Spiki’s model clearly outperforms similar detection models such as Google Speech Commands. Spiki’s command recognition can improve ease of use and user experience over various devices and applications, such as smart watches, home applications, switches, industrial control systems, UAVs and other vehicles.

Excited? Contact us!