Search
Close this search box.
Categories
Bridge project

Low-Code Programming of Spatial Contexts for Logistic Tasks in Mobile Robotics

DIREC project

Low-code programming of spatial contexts for logistic tasks in mobile robotics

Summary

Low-volume production represents a large share of the Danish manufacturing industry. An unmet need in this industry is flexibility and adaptability of manufacturing processes. Existing solutions for automating industrial logistics tasks include combinations of automated storage, conveyor belts, and mobile robots with special loading and unloading docks. 

However, these solutions require major investments and are not cost efficient for low-volume production, and today, low-volume production is often labor intensive.

Together with industrial partners, this project will investigate production scenarios where a machine can be operated by untrained personnel by using low-code development for adaptive and re-configurable robot programming of logistic tasks.

Project period: 2022-2025
Budget: DKK 7,15 million

An unmet need in industry is flexibility and adaptability of manufacturing processes in low-volume production. Low-volume production represents a large share of the Danish manufacturing industry. Existing solutions for automating industrial logistics tasks include combinations of automated storage, conveyor belts, and mobile robots with special loading and unloading docks. However, these solutions require major investments and are not cost efficient for low-volume production.

Therefore, low-volume production is today labor intensive, as automation technology and software are not yet cost effective for such production scenarios where a machine can be operated by untrained personnel. The need for flexibility, ease of programming, and fast adaptability of manufacturing processes is recognized in both Europe and USA. EuRobotics highlights the need for systems that can be easily re-programmed without the use of skilled system configuration personnel. Furthermore, the American roadmap for robotics  highlights adaptable and reconfigurable assembly and manipulation as an important capability for manufacturing.

The company Enabled Robotics (ER) aims to provide easy programming as an integral part of their products. Their mobile manipulator ER-FLEX consists of a robot arm and a mobile platform. The ER-FLEX mobile collaborative robot provides an opportunity to automate logistic tasks in low-volume production. This includes manipulation of objects in production in a less invasive and more cost-efficient way, reusing existing machinery and traditional storage racks. However, this setting also challenges the robots due to the variability in rack locations, shelf locations, box types, object types, and drop off points.

Today the ER-FLEX can be programmed by means of block-based features, which can be configured to high-level robot behaviors. While this approach offers an easier programming experience, the operator must still have a good knowledge of robotics and programming to define the desired behavior. In order to enable the product to be accessible to a wider audience of users in low-volume production companies, robot behavior programming has to be defined in a simpler and intuitive manner. In addition, a solution is needed that address the variability in a time-efficient and adaptive way to program the 3D spatial context.

Low-code software development is an emerging research topic in software engineering. Research in this area has investigated the development of software platforms that allow non-technical people to develop fully functional application software without having to make use of a general-purpose programming language. The scope of most low-code development platforms, however, has been limited to create software-only solutions for business processes automation of low-to-moderate complexity.

Programming of robot tasks still relies on dedicated personnel with special training. In recent years, the emergence of digital twins, block-based programming languages, and collaborative robots that can be programmed by demonstration, has made a breakthrough in this field. However, existing solutions still lack the ability to address variability for programming logistics and manipulation tasks in an everchanging environment.

Current low-code development platforms do not support robotic systems. The extensive use of hardware components and sensorial data in robotics makes it challenging to translate low-level manipulations into a high-level language that is understandable for non-programmers. In this project we will tackle this by constraining the problem focusing on the spatial dimension and by using machine learning for adaptability. Therefore, the first research question we want to investigate in this project is whether and how the low-code development paradigm can support robot programming of spatial logistic task in indoor environments. The second research question will address how to apply ML-based methods for remapping between high-level instructions and the physical world to derive and execute new task-specific robot manipulation and logistic actions.

Therefore, the overall aim of this project is to investigate the use of low-code development for adaptive and re-configurable robot programming of logistic tasks. Through a case study proposed by ER, the project builds on SDU’s previous work on domain-specific languages (DSLs) to propose a solution for high-level programming of the 3D spatial context in natural language and work on using machine learning for adaptable programming of robotic skills. RUC will participate in the project with interaction competences to optimize the usability of the approach.

Our research methodology to solve this problem is oriented towards design science, which provides a concrete framework for dynamic validation in an industrial setting. For the problem investigation, we are planning a systematic literature review around existing solutions to address the issues of 3D space mapping and variability of logistic tasks. For the design and implementation, we will first address the requirement of building a spatial representation of the task conditions and the environment using external sensors, which will give us a map for deploying the ER platform. Furthermore, to minimizing the input that the users need to provide to link the programming parameters to the physical world we will investigate and apply sensor-based user interface technologies and machine learning. The designed solutions will be combined into the low-code development platform that will allow for the high-level robot programming.

Finally, for validation the resultant low-code development platform will be tested for logistics-manipulation tasks with the industry partner Enabled Robotics, both at a mockup test setup which will be established in the SDU I4.0 lab and at a customer site with increasing difficulty in terms of variability.

Value creation

Making it easier to program robotic solutions enables both new users of the technology and new use cases. This contributes to the DIREC’s long-term goal of building up research capacity as this project focuses on building the competences necessary to address challenges within software engineering, cyber-physical systems (robotics), interaction design, and machine learning.

Scientific value
The project’s scientific value is to develop new methods and techniques for low-code programming of robotic systems with novel user interface technologies and machine learning approaches to address variability. This addresses the lack of approaches for low-code development of robotic skills for logistic tasks. We expect to publish at least four high-quality research articles and to demonstrate the potential of the developed technologies in concrete real-world applications.

Capacity building
The project will build and strengthen the research capacity in Denmark directly through the education of one PhD candidate, and through the collaboration between researchers, domain experts, and end-users that will lead to R&D growth in the industrial sector. In particular, research competences in the intersection of software engineering and robotics to support the digital foundation for this sector.

Societal and business value
The project will create societal and business value by providing new solutions for programming robotic systems. A 2020 market report predicts that the market for autonomous mobile robots will grow from 310M DKK in 2021 to 3,327M DKK in 2024 with inquiries from segments such as the semiconductor manufacturers, automotive, automotive suppliers, pharma, and manufacturing in general. ER wants to tap into these market opportunities by providing an efficient and flexible solution for internal logistics. ER would like to position its solution with benefits such as making logistics smoother and programmable by a wide customer base while alleviating problems with shortage of labor. This project enables ER to improve their product in regard to key parameters. The project will provide significant societal value and directly contribute to SDGs 9 (Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation).

Impact

The project will provide a strong contribution to the digital foundation for robotics based on software competences and support Denmark being a digital frontrunner in this area.

Participants

Project Manager

Thiago Rocha Silva

Associate Professor

University of Southern Denmark
Maersk Mc-Kinney Moller Institute

E: trsi@mmmi.sdu.dk

Aljaz Kramberger

Associate Professor

University of Southern Denmark
Maersk Mc-Kinney Moller Institute

Mikkel Baun Kjærgaard

Professor

University of Southern Denmark
Maersk Mc-Kinney Moller Institute

Mads Hobye

Associate Professor

Roskilde University
Department of People and Technology

Lars Peter Ellekilde

Chief Executive Officer

Enabled Robotics ApS

Anahide Silahli

PhD

University of Southern Denmark
Maersk Mc-Kinney Moller Institute

Partners

Categories
Bridge project

Embedded AI

DIREC project

Embedded AI

Summary

AI currently relies on large data centers and centralized systems, necessitating data movement to algorithms. To address this limitation, AI is evolving towards a decentralized network of devices, bringing algorithms directly to the data. This shift, enabled by algorithmic agility and autonomous data discovery, will reduce the need for high-bandwidth connectivity and enhance data security and privacy, facilitating real-time edge learning. This transformation is driven by the integration of AI and IoT, forming the “Artificial Intelligence of Things” (AIoT), and the rise of Embedded AI (eAI), which processes data on edge devices rather than in the cloud. 

Embedded AI offers increased responsiveness, functionality, security, and privacy. However, it requires engineers to develop new skills in embedded systems. Companies are hiring data scientists to leverage AI for optimizing products and services in various industries. This project aims to develop tools and methods to transition AI from cloud to edge devices, demonstrated through industrial use cases.

 

Project period: 2022-2024
Budget: DKK 16,2 million

AI is currently limited by the need for massive data centres and centralized architectures, as well as the need to move this data to algorithms. To overcome this key limitation, AI will evolve from today’s highly structured, controlled, and centralized architecture to a more flexible, adaptive, and distributed network of devices. This transformation will bring algorithms to the data, made possible by algorithmic agility and autonomous data discovery, and it will drastically reduce the need for high-bandwidth connectivity, which is required to transport massive data sets and eliminate any potential sacrifice of the data’s security and privacy. Furthermore, it will eventually allow true real-time learning at the edge.

This transformation is enabled by the merging of AI and IoT into “Artificial Intelligence of Things” (AIoT), and has created an emerging sector of Embedded AI (eAI), where all or parts of the AI processing are done on the sensor devices at the edge, rather than sent to the cloud. The major drivers for Embedded AI are increased responsiveness and functionality, reduced data transfer, and increased resilience, security, and privacy. To deliver these benefits, development engineers need to acquire new skills in embedded development and systems design.

To enter and compete in the AI era, companies are hiring data scientists to build expertise in AI and create value from data. This is true for many companies developing embedded systems, for instance, to control water, heat, and air flow in large facilities, large ship engines, or industrial robots, all with the aim of optimizing their products and services. However, there is a challenging gap between programming AI in the cloud using tools like Tensorflow, and programming at the edge, where resources are extremely constrained. This project will develop methods and tools to migrate AI algorithms from the cloud to a distributed network of AI-enabled edge-devices. The methods will be demonstrated on several use cases from the industrial partners.

Research problems and aims
In a traditional, centralized AI architecture, all the technology blocks would be combined in the cloud or at a single cluster (Edge computing) to enable AI. Data collected by IoT, i.e., individual edge-devices, will be sent towards the cloud. To limit the amount of data needed to be sent, data aggregation may be performed along the way to the cloud. The AI stack, the training, and the later inference, will be performed in the cloud, and results for actions will be transferred back to the relevant edge-devices. While the cloud provides complex AI algorithms which can analyse huge datasets fast and efficiently, it cannot deliver true real-time response and data security and privacy may be challenged.

When it comes to Embedded AI, where AI algorithms are moved to the edge, there is a need to transform the foundation of the AI Stack by enabling transformational advances, algorithmic agility and distributed processing will enable AI to perceive and learn in real-time by mirroring critical AI functions across multiple disparate systems, platforms, sensors, and devices operating at the edge. We propose to address these challenges in the following steps, starting with single edge-devices.

  1. Tiny inference engines – Algorithmic agility of the inference engines will require new AI algorithms and new processing architectures and connectivity. We will explore suitable microcontroller architectures and reconfigurable platform technologies, such as Microchip’s low power FPGA’s, for implementing optimized inference engines. Focus will be on achieving real-time performance and robustness. This will be tested on cases from the industry partners.
  2. µBrains – Extending the edge-devices from pure inference engines to also provide local learning. This will allow local devices to provide continuous improvements. We will explore suitable reconfigurable platform technologies with ultra-low power consumption, such as Renesas’ DRP’s using 1/10 of the power budget of current solutions, and Microchip’s low power FPGA’s for optimizing neural networks. Focus will be on ensuring the performance, scheduling, and resource allocation of the new AI algorithms running on very resource constrained edge-devices.
  3. Collective intelligence – The full potential of Embedded AI will require distributed algorithmic processing of the AI algorithms. This will be based on federated learning and computing (microelectronics) optimized for neural networks, but new models of distributed systems and stochastic analysis, is necessary to ensure the performance, prioritization, scheduling, resource allocation, and security of the new AI algorithms—especially with the very dynamic and opportunistic communications associated with IoT.

     

The expected outcome is an AI framework which supports autonomous discovery and processing of disparate data from a distributed collection of AI-enabled edge-devices. All three presented steps will be tested on cases from the industry partners.

Value Creation
Deep neural networks have changed the capabilities of machine learning reaching higher accuracy than hitherto. They are in all cases learning from unstructured data now the de facto standard. These networks often include millions of parameters and may take months to train on dedicated hardware in terms of GPUs in the cloud. This has resulted in high demand of data scientists with AI skills and hence, an increased demand for educating such profiles. However, an increased use of IoT to collect data at the edge has created a wish for training and executing deep neural networks at the edge rather than transferring all data to the cloud for processing. As IoT end- or edge devices are characterized by low memory, low processing power, and low energy (powered by battery or energy harvesting), training or executing deep neural networks is considered infeasible. However, developing dedicated accelerators, novel hardware circuits, and architectures, or executing smaller discretized networks may provide feasible solutions for the edge.

The academic partners DTU, KU, AU, and CBS, will not only create scientific value from the results disseminated through the four PhDs, but will also create important knowledge, experience, and real-life cases to be included in the education, and hence, create capacity building in this important merging field of embedded AI or AIoT.

The industry partners Indesmatech, Grundfos, MAN ES, and VELUX are all strong examples of companies who will benefit from mastering embedded AI, i.e., being able to select the right tools and execution platforms for implementing and deploying embedded AI in their products.

  • Indesmatech expect to gain leading edge knowledge about how AI can be implemented on various chip processing platforms, with a focus on finding the best and most efficient path to build cost and performance effective industrial solutions across industries as their customers are represented from most industries.
  • Grundfos will create value in applications like condition monitoring of pump and pump systems, predictive maintenance, heat energy optimization in buildings and waste-water treatment where very complex tasks can be optimized significant by AI. The possibility to deploy embedded AI directly on low cost and low power End and Edge devices instead of large cloud platforms, will give Grundfos a significant competitive advantage by reducing total energy consumption, data traffic, product cost, while at the same time increase real time application performance and secure customers data privacy.
  • MAN ES will create value from using embedded AI to predict problems faster than today. Features such as condition monitoring and dynamic engine optimization will give MAN ES competitive advantages, and the exploitation of embedded AI together with their large amount of data collected in the cloud will in the long run create marked advantages for MAN ES.

VELUX will increase their competitive edge by attaining a better understanding of the ability to implement the right level of embedded AI into their products. The design of new digital smart products with embedded intelligence, will create value from driving the digital product transformation of VELUX.

The four companies represent a general trend where several industries depend on their ability to develop, design and engineer high tech products with software, sensors and electronic solutions as embedded systems to their core products. Notably firms in the machine sub-industry of manufacturers of pumps, windmills and motors, and companies in the electronics industry, which are manufacturing computer and communication equipment and other electronic equipment. These industries have very high export with an 80 percent export share of total sales.

Digital and electronics solutions compose a very high share of the value added. In total, the machine sub-industry’s more than 250 companies and the electronics industry’s more than 500 companies in total exported equipment worth 100 billion DKK in 2020 and had more than 38.000 employees.[1] The majority of electronics educated have a master’s or bachelor’s degree in engineering and the share of engineers has risen since 2008.[2] 

Digitalization, IoT and AI are data driven and a large volume of data will have economic and environmental impact. AI will increase the demand for computing, which today depends on major providers of cloud services and transfer of data. The operating costs of energy related to this will increase, and according to EU’s Joint Research Center (JRC), it will account for 3-4 percent of Europe’s total energy consumption[3]. Thus, less energy consuming and less costly solutions, are needed. The EU-Commission find that fundamental new data processing technologies encompassing the edge are required. Embedded AI will make this possible by moving computing to sensors where data is generated, instead of moving data to computing. [4]All in all, the rising demand and need for these new high-tech solutions calls for development of Embedded AI capabilities and will have a positive impact on Danish industries, in terms of growth and job-creation.  

[1] Calculations on data from Statistics Denmark, Statistikbanken, tables: FIKS33; GF2 
[2] ”Elektronik-giver-beskaftigelse-i-mange-brancher” DI Digital, 2021 
[3] Artificial Intelligence, A European Perspective”, JRC, EUR 29425, 2018 
[4] “2030 Digital Compass, The European way for the Digital Decade”, EU Commission, 2021  

Impact

The project creates value not only scientifically from the results disseminated through the four PhDs, but will also create important knowledge, experience, and real-life cases to be included in education, and hence, create capacity building in this important merging field of embedded AI or AIoT.

News / coverage

Participants

Project Manager

Xenofon Fafoutis

Associate Professor

Technical University of Denmark
DTU Compute

E: xefa@dtu.dk

Peter Gorm Larsen

Professor

Aarhus University
Dept. of Electrical and Computer Engineering

Jalil Boudjadar

Associate Professor

Aarhus University
Dept. of Electrical and Computer Engineering

Jan Damsgaard

Professor

Copenhagen Business School
Department of Digitalization

Ben Eaton

Associate Professor

Copenhagen Business School
Department of Digitalization

Thorkild Kvisgaard

Head of Electronics

Grundfos

Thomas S. Toftegaard

Director, Smart Product Technology

Velux

Rune Domsten

Co-founder & CEO

Indesmatech

Jan Madsen

Professor

Technical University of Denmark
DTU Compute

Henrik R. Olesen

Senior Manager

MAN Energy Solutions

Reza Toorajipour

PhD Student

Copenhagen Business School
Department of Digitalization

Iman Sharifirad

PhD Student

Aarhus University
Dept. of Electrical and Computer Engineering

Amin Hasanpour

PhD Student

Technical University of Denmark
DTU Compute

Partners

Categories
Bridge project

Mobility Analytics using Sparse Mobility Data and Open Spatial Data

DIREC project

Mobility Analytics using Sparse Mobility Data and Open Spatial Data

Summary

Both society and industry have a substantial interest in well-functioning outdoor and indoor mobility infrastructures that are efficient, predictable, environmentally friendly, and safe. For outdoor mobility, reduction of congestion is high on the political agenda as is the reduction of CO2 emissions, as the transportation sector is the second largest in terms of greenhouse gas emissions. For indoor mobility, corridors and elevators represent bottlenecks for mobility in large building complexes.  

The amount of mobility-related data has increased massively which enables an increasingly wide range of analyses. When combined with digital representations of road networks and building interiors, this data holds the potential for enabling a more fine-grained understanding of mobility and for enabling more efficient, predictable, and environmentally friendly mobility.   

Project period: 2021-2024
Budget: DKK 9,41 million

The mobility of people and things is an important societal process that facilitates and affects the lives of most people. Thus, society, including industry, has a substantial interest in well-functioning outdoor and indoor mobility infrastructures that are efficient, predictable, environmentally friendly, and safe. For outdoor mobility, reduction of congestion is high on the political agenda – it is estimated that congestion costs Denmark 30 billion DKK per year. Similarly, the reduction of CO2 emissions from transportation is on the political agenda, as the transportation sector is the second largest in terms of greenhouse gas emissions. Danish municipalities are interested in understanding the potentials for integrating various types of e-bikes in transportation planning. Increased use of such bicycles may contribute substantially to the greening of transportation and may also ease congestion and thus improve travel times. For indoor mobility, corridors and elevators represent bottlenecks for mobility in large building complexes (e.g. hospitals, factories and university campuses). With the addition of mobile robots, humans and robots will also be fighting to use the same space when moving indoors. Heavy use of corridors is also a source of noise that negatively impacts building occupants.

The ongoing, sweeping digitalisation has also reached outdoor and indoor mobility. Thus, increasingly massive volumes of mobility-related data, e.g. from sensors embedded in the road and building infrastructures, networked positioning (e.g. GPS or UWB) devices (e.g. smartphones and in-vehicle navigation devices) or indoor mobile robots, are becoming available. This enables an increasingly wide range of analyses related to mobility. When combined with digital representations of road networks and building interiors, this data holds the potential for enabling a more fine-grained understanding of mobility and for enabling more efficient, predictable, and environmentally friendly mobility. Long movement times equate with congestion and bad overall experiences.

The above data foundation offers a basis for understanding how well a road network or building performs across different days and across the duration of a day, and it offers the potential for decreased movement times by means of improved mobility flows and routing. However, there is an unmet need for low-cost tools that can be used by municipalities and building providers (e.g. mobile robot manufactures) that are capable of enabling a wide range of analytics on top of mobility data.

  1. Build extract-transform-load (ETL) prototypes that are able to ingest high and low frequency spatial data (e.g. GPS and indoor positioning data). These prototypes must enable map-matching of spatial data to open road network and building representations and must enable privacy protection.
  2. Design effective data warehouse schemas that can be populated with ingested spatial data.
  3. Build mobility analytics warehouse systems that are able to support a broad range of analyses in interactive time.
  4. Build software systems that enable users to formulate analyses and visualise results in maps-based interfaces for both indoor and outdoor use. This includes infrastructure for the mapping of user input into database queries and the maps-based display of results returned by the data warehouse system.
  5. Develop a range of advanced analyses that address user needs. Possible analyses include congestion maps, isochrones, aggregate travel-path analyses, origin-destination travel time matrices, and what-if analyses where the effects of reconstruction are estimated (e.g. adding an additional lane to a stretch of road or changing corridors). For outdoors settings, CO2-emissions analyses based on vehicular environmental impact models and GPS data are also considered.
  6. Develop transfer learning techniques that make it possible to leverage spatial data from dense spatio-temporal “regions” for enabling analyses in sparse spatio-temporal regions.

Value creation
The envisioned prototype software infrastructure characterised above aims to be able to replace commercial road network maps with the crowd sourced OpenStreetMap (OSM) map and for indoors enable new data sources about the indoor geography. The open data might not be curated, which means that new quality control tools are required to ensure that computed travel times are correct. This will reduce cost.

Next, the project will provide means of leveraging available spatial data as efficiently and effectively as possible. In particular, while more and more data becomes available, the available data will remain sparse in relation to important analyses. This is due to the cost of data that can be purchased and due to the lack of desired data. Thus, it is important to be able to exploit available data as well as possible. We will examine how to transfer data from locations and times with ample data to locations and times with insufficient data. For example, we will study transfer learning techniques for this purpose; and as part of this, we will study feature learning. This will reduce cost and will enable new analyses that where not possible previously due to a lack of data.

Rambøll will be able to in-source the software infrastructure and host analytics for municipalities. Mobile Industrial Robotics (MiR) will be able to in-source the software infrastructure and host analytics for building owners. Additional value will be created because the above studies will be conducted for multiple transportation modes, with a focus on cars and different kinds of e-bikes. We have access to a unique data foundation that will enable these studies.

Impact

The project will provide a prototype software infrastructure that aims to be able to replace commercial road network maps with the crowd sourced OpenStreetMap (OSM) and for indoors enable new data sources about the indoor geography.

The open data might not be curated, which means that new quality control tools are required to ensure that computed travel times are correct. This will reduce cost.

News / coverage

Participants

Project Manager

Christian S. Jensen

Professor

Aalborg University
Department of Computer Science

E: csj@cs.aau.dk

Ira Assent

Professor

Aarhus University
Department of Computer Science

Kristian Torp

Professor

Aalborg University
Department of Computer Science

Bin Yang

Professor

Aalborg University
Department of Computer Science

Mads Darø Kristensen

Principal Application Architect

The Alexandra Institute

Søren Krogh Sørensen

Senior Software Engineer

The Alexandra Institute

Frederik Palludan Madsen

Software Engineer

The Alexandra Institute

Mikkel Baun Kjærgaard

Professor

University of Southern Denmark
The Maersk Mc-Kinney Moller Institute

Norbert Krüger

Professor

University of Southern Denmark
The Maersk Mc-Kinney Moller Institute

Leon Bodenhagen

Associate Professor

University of Southern Denmark The Maersk Mc-Kinney Moller Institute

Brian Rosenkilde Jeppesen

Project Manager Roads and Traffic

Rambøll

Stig Grønning Søbjærg

Engineer

Rambøll

Mads Graungaard

Mobility and Traffic Engineer

Rambøll

Johan Poulsgaard

Engineer

Rambøll

Christoffer Bø

Traffic and Mobility Planner

Rambøll

Morten Steen Nørby

Software Manager

Mobile Industrial Robots

Kasper Fromm Pedersen

Research Assistant

Aalborg University
Dept. of Computer Science

Helene Hauschultz

PhD Student

Aarhus University
Department of Mathematical Science

Avgi Kollakidou

PHD student

University of Southern Denmark
The Maersk Mc-Kinney Moller Institute

Hao Miao

PHD STUDENT

Aalborg University
Department of Computer Science

partners

Categories
Bridge project

Deep Learning and Automation of Image-Based Quality of Seeds and Grains

DIREC project

Deep Learning and Automation of Image-based Quality of Seeds and Grains

Summary

Today, manual visual inspection of grain is still one of the most important quality assurance procedures throughout the value chain of bringing cereals from the field to the table.

Together with FOSS, this project aims to develop and validate a method of automated image-based solutions that can replace subjective manual inspection and improve performance, robustness and consistency of the inspection. The method has the potential of providing the grain industry with a disruptive new tool for ensuring quality and optimising the value of agricultural commodities.

Project period: 2020-2024
Budget: DKK 3,91 million

To derive maximum value from the data there is a need to develop methods of training data algorithms to automatically be able to provide industry with the best possible feedback on the quality of incoming materials. The purpose is to develop a framework which replaces the current feature-based models with deep learning methods. By using these methods, the potential is significantly to reduce the labor needed to expand the application of EyeFoss™ into new applications; e.g. maize, coffee, while at the same time increase the performance of the algorithms in accurately and reliably describing the quality of cereals.

This project aims at developing and validating, with industrial partners, a method of using deep learning neural networks to monitor quality of seeds and grains using multispectral image data. The method has the potential of providing the grain industry with a disruptive new tool for ensuring quality and optimising the value of agricultural commodities. The ambition of the project is to end up with an operationally implemented deep learning framework for deploying EyeFoss™ to new applications in the industry. In order to the achieve this, the project will team up with DTU Compute as a strong competence centre on deep learning as well as a major player within the European grain industry (to be selected).

The research aim of the project is the development of AI methods and tools that enable industry to develop new solutions for automated image-based quality assessment. End-to-end learning of features and representations for object classification by deep neural networks can lead to significant performance improvements. Several recent mechanisms have been developed for further improving performance and reducing the need for manual annotation work (labelling) including semi-supervised learning strategies and data augmentation.

Semi-supervised learning  combines generative models that are trained without labels (unsupervised learning), application of pre-trained networks (transfer learning) with supervised learning on small sets of labelled data. Data augmentation employs both knowledge based transformations, such as translations and rotations and more general learned transformations like parameterised “warps” to increase variability in the training data and increase robustness to natural variation.

Scientific value
The scientific value of the project will be new methods and open source tools and associated knowledge of their performance and properties in an industrial setup.

Capacity building
The aim of the project is to educate one PhD student in close collaboration with FOSS – the aim is that the student will be present at FOSS at least 40% of the time to secure a close integration and knowledge exchange with the development team at FOSS working on introducing EyeFossTM to the market. Specific focus will be on exchange at the faculty level as well; the aim is to have faculty from DTU Compute present at FOSS and vice-versa for senior FOSS specialists that supervise the PhD student. This will secure better networking, anchoring and capacity building also at the senior level. The PhD project will additionally be supported by a master-level program already established between the universities and FOSS.

Societal impact
Specifically, the project aims to provide FOSS with new tools to assist in scaling the market potential of the EyeFossTM from its current potential of 20m EUR/year. Adding, in a cost-efficient way, applications for visual inspection of products like maize, rice or coffee has the potential to at least double the market potential. In addition, the contributions will be of generic relevance to companies developing image-based solutions for food quality/integrity assessment and will provide excellent application and AI integration knowledge of commercial solutions already on-the-market to other Danish companies.

Impact

The project has the potential of providing the grain industry with a disruptive new tool for ensuring quality and optimising the value of agricultural commodities.

News / coverage

Participants

Project Manager

Lars Kai Hansen

Professor

Technical University of Denmark
DTU Compute

E: lkai@dtu.dk

Kim Steenstrup Pedersen

Professor

University of Copenhagen
Department of Computer Science

Lenka Hýlová

PHD Fellow

Technical University of Denmark
DTU Compute

Thomas Nikolajsen

Head of Front-end Innovation

FOSS

Toke Lund-Hansen

Head of Spectroscopy team

FOSS

Erik Schou Dreier

Senior Scientist

FOSS

Partners

Categories
Bridge project

Edge-based AI Systems for Predictive Maintenance

DIREC project

Edge -based AI systems for predictive maintenance

Summary

Downtime of equipment is costly and a source of safety, security and legal issues. Today, organisations adopt a conservative schedule of preventive maintenance independent of the condition of equipment. This results in unnecessary service costs and occasional interruptions of production due to unexpected failures. 

Therefore, it is imperative in many domains to transition from regular maintenance to predictive maintenance. The report ‘An AI Nation: Harnessing the Opportunity of Artificial Intelligence in Denmark’ estimates that enabling predictive maintenance via AI has a 14-19 billion potential for the private sector in Denmark. 

Together with industrial partners, this project aims to uncover how AI and data-driven methods can solve the problem, so that companies have the least possible wasted time when production is at a standstill. 

Project period: 2020-2024
Budget: DKK 12,24 million

The research aim of the project is methods and tools that enable industry to develop new solutions with accurate AI-based maintenance predictions on edge-based software platforms.

The resulting applications must be deployed to collect and process large amounts of data locally. This data feeds high accuracy predictive models deployed at the edge, adapted to changing local conditions, and maintained with minimum intervention from operators.

These models should cover abstractions allowing the understanding of relevant dependencies in the data. The key research problem is to devise architectures and solutions that scale to the entire fleets of equipment with accurate AI predictions.

This also requires that resources in terms of processing power, storage and communication, are optimised in order to obtain low-power and real-time performance, leading to a resource efficiency vs prediction accuracy trade-off. The project will establish a bridge to enable Danish companies to develop and use AI-based predictive maintenance within several domains.

Scientific value

The scientific value of the project is new methods and tools and associated knowledge of their performance and properties in field tests. These are important contributions to provide excellent knowledge to Danish companies and education programs within AI and IoT.

Capacity building

For capacity building the value of the project is to educate 3 PhD students (including 1 Industrial PhD) and 1 Post Doc in close collaboration with industry. The open source availability of general project outcomes and industry collaboration enable several exploitation paths. In addition, for the master-level the projects will offer an industry program to 15 students at 3 universities.

Business and societal value

The business and societal value is on a national level estimated to a 14-19 billion potential for the Danish private sector. In the project we target both the medical, robotic, industrial and energy sectors. These are Danish frontrunners in adopting the technology and creating inspiration for wider adoption by the Danish private sector. For the public sector enabling equipment with higher operation efficiency will positively impact the efficiency of the sector.

Impact

It is estimated that enabling predictive maintenance via AI has a 14-19 billion potential for the private sector in Denmark. 

News / coverage

Participants

Project Manager

Mikkel Baun Kjærgaard

Professor

University of Southern Denmark
The Maersk Mc-Kinney Moller Institute

E: mbkj@mmmi.sdu.dk

Philippe Bonnet

Professor

IT University of Copenhagen
Department of Computer Science

Xenofon Fafoutis

Associate Professor

Technical University of Denmark
Dept. of Applied Mathematics and Computer Science

Jan Madsen

Professor

Technical University of Denmark
DTU Compute

Rasmus Larsen

AI Specialist

The Alexandra Institute

Alexandre Alapetite

Software Solutions Architect

The Alexandra Institute

Felix Blaga

Founder

Octavic

Kasper Hjort Bertelsen

PhD Student

IT University of Copenhagen
Department of Computer Science

Emil Stubbe Kolvig-Raun

PhD Student

Universal Robots

Ahmad Rzgar Hamid

PhD Student

University of Southern Denmark
The Maersk-McKinney Moller Institute

Emil Njor

PhD Student

Technical University of Denmark

Morten Boris Højgaard

Head of Incubation and Partnerships

Universal Robots

Miguel Enrique Campusano Araya

Assistant Professor

University of Southern Denmark

Partners