A team from Socionext and Osaka University have developed a new method of deep learning, which enables image recognition and object detection in extremely low-light conditions. By merging multiple models, the new method enables the detection of objects without generating huge datasets, a task previously thought to be essential.

Socionext plans to incorporate this new method into the company’s image signal processors to develop new SoCs, as well as new camera systems around those SoCs, for automotive, security, industrial and other applications that require high-performance image recognition.
A major challenge throughout the evolution of computer vision technology has been to improve the image-recognition performance under poor lighting conditions for applications such as in-vehicle cameras and surveillance systems. Previously, a deep learning method using RAW image data from sensors has been developed, called “Learning to See in the Dark”, Chen et al. This method, however, requires a dataset in excess of 200,000 images and over 1.5 million annotations for end-to-end learning. Preparing large datasets with RAW images is both too costly and time-prohibitive. 

The joint research team, led by Professor Hajime Nagahara, proposed the new domain adaptation method, which builds a required model using existing datasets by utilising machine learning techniques such as Transfer Learning and Knowledge Distillation. The new method resolves the challenge through the following steps: (1) building an inference model with existing datasets, (2) extracting knowledge from the aforementioned inference model, (3) merging the models by glue layers, and (4) building generative model by knowledge distillation. It enables the learning of a desired image recognition model using the existing datasets.

Using this domain adaptation method, the team has built an object detection model “YOLO in the Dark” using RAW images taken in extreme dark conditions, with the YOLO model. Learning of the object detection model with RAW images can be achieved with the existing dataset, without generating additional datasets. In contrast to the case where the object cannot be detected by brightness enhancement of images with existing YOLO model, the proposed new model made it possible to recognise RAW images and detect objects. The amount of computing resources needed in this new model is about the half of the baseline model, which uses the combination of previous models.

This “direct recognition of RAW images” by the method is expected to be used for object detection in extremely dark conditions, along with many other applications. Socionext plans to incorporate this new method into the company’s image signal processors to develop new SoCs, as well as new camera systems around such SoCs, and offer leading edge solutions for applications including automotive, security, industrial and others that require high performance image recognition.