Cleaning a factory with robots – RoboClean

In food and drink manufacturing, a significant amount of employee time is dedicated to cleaning, which bears a major impact on employee productivity and manufacturing efficiency. The process of cleaning factory equipment typically unfolds as part of a process known as Clean-in-Place and is beginning to take advantage of novel technologies such as in-line sensors, the IoT, and machine learning. However, the work of cleaning the factory floor is still primarily completed by human workers following strict industry standards specified by the British Retail Consortium (BRC).

RoboClean seeks to understand and address the industry need for cleaning support technologies and is developing systems for deploying robots to assist in the cleaning of factories. Furthermore, the robots will be designed to detect and report the unwanted presence of allergens to prevent food safety events using smart sensor data analytics (e.g. for wheat gluten proteinpeanuts in cereals, and peanuts in wheat flour). Additionally, the project aims to tackle one of the biggest challenges facing manufacturers, which is the cross contamination of allergens within the manufacturing environment. Regular cleaning is a critical step to preventing this, but this challenge is exacerbated as manufacturers strive to provide more variety and alternative formulations (e.g. gluten free) and are required to verify the effectiveness of cleaning procedures for removing allergens from equipment as per the BRC industry standards. The Food Standards Agency states that the number of food and safety events relating to all allergens has roughly doubled between 2014/15 and 2017/18 highlighting the pressing need to integrate smart sensors into the manufacturing and cleaning processes.

Furthermore, a key focus for the project is to develop an understanding of human-robot collaboration in complex environments such as factories (building upon studies of robots in-the-wild), and how to coordinate multiple cleaning robots as co-bot teams (i.e. multi-agent collaboration). These foci will help to deliver novel solutions for monitoring and delivering cleaning to the required standards in an efficient and safe manner, alongside–and with– human workers on a factory floor. The outcomes of this project will include the design, implementation, and evaluation of an interactive connected system enabling novel human-robot collaboration and sensor data collection in a factory by engaging with partners in industry (British Pepper and Spice) and the third sector (the Food and Drink Forum).

RoboClean is led by Joel E. Fischer and Nik Watson, and formed of members from across four departments at the University.

Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Brian Logan
School of Computer Science, University of Nottingham

Carolina Fuentes
Horizon Digital Economy Research, University of Nottingham

Roberto Santos
Digital Research Service, Information Services, University of Nottingham

Ahmed Rady and Nik Watson
Faculty of Engineering, University of Nottingham


This project is funded by the University of Nottingham Smart Products Beacon of Excellence and Horizon Digital Economy Research.

This post’s content is based upon work by all members of the project, and previous project summaries.

Cleaning a factory with robots


Smart Products Beacon – “Sensors support machine learning”

Nicholas Watson, Assistant Professor, Faculty of Engineering discusses whether online sensors and machine learning can deliver industry 4.0 to the food and drink manufacturing sector in the Journal of the Institute of Food Science and Technology, vol 33 issue 4 December 2019.

“Manufacturing is experiencing the 4th industrial revolution, which is the use of Industrial Digital Technologies (IDTs) to produce new and existing products. Industrial digital technologies include sensors, robotics, the industrial internet of things (IoT), additive manufacturing, artificial intelligence, virtual and augmented reality, digital twins and cloud computing. At the heart of Industry 4.0 is the enhanced collection and use of data. Industry 4.0 is predicted to have a positive impact of over £450bn to UK manufacturing over the next ten years[1], with benefits such as increased productivity and reduced costs and environmental impacts. But what does this mean for the UK’s largest manufacturing sector, food and drink?”

Link to article (page 20)

University of Nottingham Smart Products Beacon – job opportunity

Research Associat/Fellow (fixed term)

Reference: SC1494719

Closing date: Tuesday 4th February 2020

Job Type: Research

Department: Smart Products Beacon Computer Science

Salary:  £27511 – £40322 per annum (pro rate if applicable) depending on skills and experience (minimum £300943 with relevant PhD). Salary progression beyond this scale is subject to performance

Applications are invited for a Computer Science and/or Engineering based Research Associate/Fellow within The Smart Products Beacon.

The Smart Products Beacon explores how leading edge technologies emerging from Computer Science and Engineering can fundamentally disrupt the nature of products and how they are made. This University led initiative tackles how the combination of physical and digital technologies, from robotically-enabled and additive manufacturing to artificial intelligence and mixed reality, can produce smarter and better products. We also work to ensure that products are produced in responsible ways to embody the fair and transparent use of personal data, operate safely, and respect human values.

The purpose of this role will be to support the Smart Products Beacon in establishing its research agenda by contributing to the creation of an independent research program, linking across a number of disciplines to develop, deploy and study Beacon related projects. A preference will be given to applicants in the following areas, but other skills will be considered if clear evidence of their link to the Beacon can be provided.

  • Software platform development
  • Artificial intelligence
  • Security
  • Development and integration of sensors and interfaces
  • Advanced manufacturing techniques (robotics, additive manufacturing, etc.)
  • User studies

The post holder will be expected to:

  • Create and lead an independent research program
  • Work as part of a multi-disciplinary team to enhance impact
  • Have the flexibility to work on several ongoing projects while developing their own work
  • Contribute to, and lead, high quality publications and proposals

The role holder will have the opportunity to use their initiative and creativity to identify areas for research, develop research methods and extend their research portfolio.

This is a full time, fixed term post for 3 years. Job share arrangements may be considered.

Informal enquiries may be addressed to Professor Steve Benford.  Applications must be submitted online; please note that applications sent by email will not be accepted.

Our University has always been a supportive, inclusive, caring and positive community. We warmly welcome those of different cultures, ethnicities and beliefs – indeed this very diversity is vital to our success, it is fundamental to our values and enriches life on campus. We welcome applications from UK, Europe and from across the globe. For more information on the support we offer our international colleagues, visit;

Professor Steve Benford explains the Smart Products Beacon

The Smart products beacon is tackling two big questions. What are smart products? And how are they made?

A smart product is one that uses digital technologies and especially personal data to become more adaptive, personalised and valuable. It captures data throughout its lifetime – through both manufacture and use – and uses this to adapt itself to consumers. In so doing it blends aspects of goods, services and experiences, the three dominant product logics from economics and business into new forms. Sounds a bit abstract? Let’s take an example .…

There was a time when a car was made of rubber and iron. A car is also something you bought and owned. But those days are passing. A modern car is part software, containing an engine management system that can adapt its driving behaviour, and also hosts a variety of other services for navigation and entertainment. Some might say the modern car is really a mobile phone on wheels. For many consumers, a car is now also now a service that they lease rather than a good that they own.

But the transformation doesn’t end there. In a possible future world of autonomous cars, mobility itself may be the service, with consumers summoning vehicles on demand that adapt themselves on the fly to their preferences and history of previous travel. In this world, the physical units become interchangeable and it is the data that matters. You step into a car and it becomes yours by loading your personal profile and adapting itself to you. In those case the car is the data. As Neo learns when he visits the Oracle: “There is no spoon” (only data).

If smart products are made from new materials – personal data – then they are also made in new ways. Digitally native products such as social media are inherently co-created. Consumers either explicitly provide content in the form of the videos and photos they upload directly or implicitly provide it through their records of searches, views and likes. Smart products, even future cars, will be similarly co-created as both manufacturers and consumers engage with digital platforms and data-driven product life-cycles.

This raises a further important question – how can consumers trust future products with their personal data? How can they be sure that products are safe and secure and that they can retain control of their own data?

This vision of co-creating trusted smart products lies at the heart of our beacon. We think that it applies to all manner of products, from high value goods to consumer goods to digital media experiences. We’re looking forward to exploring the possibilities further over the coming years.

keynote talk by Sarah Brin, Strategic Partnerships Manager, Meow Wolf

The Smart Products Beacon is delighted to be supporting a keynote talk by Sarah Brin, Strategic Partnerships Manager, Meow Wolf at the Broadway on Monday 9th December, 6pm.

Sarah will speak about the creative challenges and questions surrounding the development of immersive experiences supported by emerging technologies.

An art historian and creative producer, Sarah specialises in previously unanticipated situations involving technology, the public, and organisational change/infrastructure. She’s created programs, exhibitions, and publications for organisations like Autodesk, SFMOMA, British Council, MOCA Los Angeles, the European Union and elsewhere. She cares about building just, sustainable and inviting things.

Sarah will cover key aspects of Meow Wolf’s creative process, recommendations for creatives working at the intersection of art and technology, and address questions regarding the responsibilities of cultural producers in times of dire political crisis.

Meow Wolf are a New Mexico-based arts and entertainment group creating immersive and interactive experiences that transport audiences of all ages into fantastic realms of story and exploration. This includes art installations, video and music production, and extended reality content.

Meow Wolf’s radical practice champions otherness, weirdness, radical inclusion and the power of creativity to change the world.

Book your tickets here.

Connected Everything II: Launch of Feasibility Studies Call

Connected Everything is the EPSRC funded network focussed on addressing the question “how do we support the future of manufacturing in the UK?”. In our first three years of funding, we supported the Manufacturing Made Smarter proposal development, including directly inputting into the definition of its key research challenges. We have now been awarded a further three years funding to deliver a network of networks which will accelerate multi-disciplinary collaboration, foster new collaborations between industry and academia and tackle emerging challenges which will underpin the UK academic community’s research in support of people, technologies, products and systems for digital manufacturing. Through a range of activities, including feasibility studies, networking, and thematic research, Connected Everything II (CEII) will bring together new teams within a multidisciplinary community to explore new ideas, demonstrate novel technologies in the context of digital manufacturing, and accelerate impact of research into industry.

As one of our initial activities, we are launching our first funding call for feasibility studies at this event in London on the morning of 28 November. Places are limited so please register early.



My internship on the RoboClean project – Jane Slinger

My internship with the RoboClean team involved developing a custom Alexa skill to control Neato vacuum cleaners by voice. This will enable further development to link with the voice interface if required, as the other aspects of the project involve web systems and multi-agent systems. I also helped run a study to find out how users would interact with the potential system in a lab environment.

I enjoyed the work as it was in an area that interested me and had some challenges in the code to overcome, leading me to learn more about how the systems worked to explore different solutions. It was nice to be able to build on skills about Alexa development learnt in my 3rd year project and include linking to the neato API through HTTP requests and a 3rd party library. This included setting up the Account Linking on the Alexa skill and then adapting some of the code from libraries to work with node.js on the backend instead of front-end JS-based methods that were already in place.

Designing the interactions with the robot and the user was also very interesting as I wanted to make sure that the system would prompt for the necessary information about the robot, and location to clean, without becoming annoying for the user.

The internship will help with my studies and future work as it has given me experience of working with a research team, building on areas I had some experience in as well as expanding to other technical skills that I hadn’t used before, and will be useful in the future.

Written by Jane Slinger

I-CUBE call for Participants

We are looking for participants for the I-CUBE project’s first study, taking place at the School of Computer Science, this November on Jubilee Campus.

This initial call is for employees of the University and members of the public, more generally. We will make a separate call for student participants. All participants need to be 18 years old or over.

If you are interested in taking part please use this Doodle link: to select your appointment and participate in our study.

The study’s task is to instruct a trainee ‘robot’ to sort a pile of clothes into separate washing loads according to a detailed list of tasks. This is to examine human interactions in a prescribed situation. There is a short questionnaire-interview to complete after the task.

You will be both video and audio recorded while instructing and responding to the trainee ‘robot’ as well as audio-recorded for the interview.

The experiment is expected to take approximately 45 minutes of your time and you will be reimbursed with £10 worth of shopping vouchers.

Charlotte Gray shares her experiences of working on RoboClean

I was introduced to the RoboClean project at Horizon whilst interning with the Advanced Data Analysis Centre. The project investigates the ways in which end-users interact with a robot vacuum cleaner and how a robot responds to user utterances; the aim being to inform its effective design and use within food factories.

I was invited to continue my internship for 5 more weeks within Horizon to help with the analysis of data collected through an elicitation study. Overall, this has been a really valuable and rewarding experience. Coming from an academic background in Sociology, I found working closely with researchers specialising in Computer Science exposed me to different research aims and challenges than I had previously encountered. This has been insightful for me as it has not only helped develop new skills in research analysis and interview techniques, but also applied the principles of a range of research methods gained during my academic studies over the past 2-years to cutting edge technological developments.

I have been responsible for transcribing participants’ audio data, analysing visual data, and creating a summary written report of participants’ interview responses. The focus of the report was on the benefits, limitations, and disadvantages experienced by users from the user-robot interactions. The attendance at a range of team meetings has also been beneficial in understanding interactions within a work environment, especially where individuals are working together from across a range of disciplines. Combined with the skills I have learned at workload prioritisation and management, this has made me confident to face future work situations and dilemmas. Additionally, I have written literature reviews on the topic of human-robot interaction. Being able to explore these new topics has also helped me see how issues explored in Sociology are becoming increasingly influenced by the world of technology, for example, how individuals’ day-to-day lives are mediated by the introduction of robots to the workplace. The multidisciplinary projects throughout Horizon have therefore also been interesting to work alongside, clearly showing the benefit of collaborative projects in producing innovative findings.

Contributing to a research project which is aiming for publication in a research journal has been hugely rewarding and exciting, and has made the idea of working in a similar environment after graduating a lot more persuasive.

Written by Charlotte Gray

Smart Products Beacon – Soonchild and Creative Captioning – Tailoring theatre productions for D/deaf audiences

For theatre audiences on a spectrum from D/deaf to hard of hearing, it is often difficult to keep up with performances. Even in cases where the performance is signed, or has captions, these accessibility additions often feel ‘tacked on’ and are typically located out of the action on stage, requiring audiences to share attention between the performance and the support. Working with Red Earth Theatre, a production company with a long history of “Total communication” in which actors sign on stage, we have been developing ways to deliver accessibility right into the heart of a performance.

Red Earth’s new show Soonchild, is touring the UK now, supported by funding from the University of Nottingham Smart Products Beacon, as well as the AHRC and the Arts Council. The show is captioned right across the set with beautiful, designed in, images, video and text delivered using new software developed at the Mixed Reality Laboratory.

The project team developed a software called ‘captionomatic’ which uses the principles of projection mapping to turn whole theatre sets into projection surfaces. While projection mapping itself is by no means a new concept, our approach has been to both simplify the process and to fit it into the wider theatre-tech ecology. Our innovation is to take a 3D model of the set – easily produced from the scale-model of the set pieces typically built for any performance, and project this onto the real set, using a simple system of point-matching to correctly align the physical set with its digital twin. Once that 3D model is in place, we are then able to project images, video, text and whatever else onto those set pieces respecting occlusion and creating an immersive canvas on which to display content.

We provide tools to read in the script from a word document, produce a compete set of captions, then generate the necessary cues which can be fired by QLab (or similar theatrical management software) to drive our system. Theatre designers need only edit the target locations and the look and feel of the text to create beautiful captions around their sets. Different sets of captions can be delivered for different audiences as necessary – so some shows may be fully captioned while others may only have key points highlighted. We know from our research that different audiences have different preferences for how captions are delivered, and our system allows theatre companies to quickly and confidently make adjustments – even between performances of the same show. Setup of the system in a new location takes only a few minutes, something that is absolutely necessary for touring productions.

More broadly, this new approach to projection mapping allows substantial creativity with digital media in theatre that extends beyond accessibility. Critically, it substantially reduces the technical barrier to entry of including projection-mapped media in a show.  Soonchild demonstrates this with some beautiful interactions between live actors and pre-recorded media – in this case shadow puppetry, projected on set as if live.

The software was demonstrated at an accessible theatre technology day at Wolverhampton Arena theatre, and plans for additional workshops and training are in the works. Despite being developed for Soonchild, the software has been designed to be easily applicable to many different types of shows and thus is open source and free, requiring only off the shelf hardware (a PC and projector). We will also be making the hardware used in the show – a projector powerful enough to compete with theatrical lighting – available for other production companies to borrow and experiment with once Soonchild’s tour is complete.

This work was developed in partnership between Red Earth Theatre, The Mixed Reality Laboratory, The School of English and Department of Modern Languages and Cultures.

The project website is available here.

Soonchild will be performed in Nottingham at the Lakeside Arts Theatre in Nottingham on Sunday 24th November – more information can be found here.