"When our company started business in 2000, companies (including DENSO) were briskly working on (development of equipment for) the car navigation system. Accordingly, people in this industry were talking about what should come next (as the technological trend concerning automobiles). Founded as a group company of DENSO that provides technologies and products related to automobiles, DENSO IT LABORATORY, INC. has a mission of research and development specializing in advanced information technology (IT) to meet the needs of the times. IoT (Internet of Things) is being talked about more and more widely in public today. Automobiles are also more connected with networks than before. Environment surrounding vehicles is significantly changing now. For example, vehicles are not just considered as a mere means of moving; increased attention is paid to the new value they bring about, such as smartness of today's vehicles or the automated driving function, says Mr. Akira Yoshizawa, Senior Researcher, Manager, Research & Development Group of the Company. Our user to feature in this issue is DENSO IT LABORATORY, INC., which conducts research and development on IT for use in several years later to support various scenes in life including vehicles. In particular, we focus on "HMI (Human Machine Interface) Team", dealing mainly with the fields of cognitive science and user interface (UI) within Research & Development Group.

Office of DENSO IT LABORATORY, INC.

The Company introduced "UC-win/ Road", FORUM8's three-dimensional (3D) real-time VR, about 5 years ago when they started research on what the vehicle cockpit should be like in terms of HMI. Since then, they have been utilizing it to the fullest not only as a simulator for research & development but also as a tool for effective demonstration. In addition, they are trying to find out new innovations while drawing out potential capability of UC-win/Road.

DENSO IT LABORATORY, INC. was founded in August 2000 as a group company to operate business specializing in research & development while utilizing the market and technical strength DENSO had built up. DENSO provides products and services in a wide range of fields including automobiles, living-related equipment, and industrial equipment. In contrast, DENSO IT LABORATORY is responsible for performing research & development in advance in consideration of commercialization in several years.

Researches in DENSO IT LABORATORY are composed of 4 areas in large as follows: 1) "Computer vision", a technology indispensable for realizing "secure driving" and serving as a vehicle's eye, being expected to be applied in a variety of ways to industrial robots or entertainment. 2) "Natural language processing" for understanding the contents of utterance obtained by voice recognition in the information interface to allow access to the target information through voice interaction. 3) "Signal processing, control theory, and time series processing" associated with three elements of correcting observed values obtained from a sensor, detecting signals, and converting them into control input, all of which are necessary for driving a real machine. 4) "Cognitive science and UI" (including HMI) for connecting humans with information equipment ("things") in consideration of safety as well as usability.

Research areas of DENSO IT LABORATORY, INC.

Main outcomes developed so far range widely over diverse technologies and products. In the area of "computer vision", they include: (i) "SPADE", a technology that allows plural objects to be recognized in real time; (ii) detection technology such as pedestrian detection using deep learning and so on for the purpose of reducing traffic accidents, and its learning method of a node distributed type; and (iii) technology of automatic generation of high-precision maps for automated driving. In the area of "natural language processing", they include the software for determining types of words from the text written in various languages. In the area of "cognitive science and UI", they include (i) HMI for providing the driver with information without diverting the driver's attention by allowing a vehicle to grasp the driver's condition and predict the driver's behavior; (ii) technology for estimating driving load and careless state of the driver; and (iii) "NaviCon", a smartphone application that links with other applications and/or Web service contents and transmits location information they provide to the car navigation system.

The Company has about 30 employees at present. Reflecting the philosophy put in its foundation, all the employees except the top management and a few members in the general affairs section belong to Research & Development Group. Putting IT at the top level in the world to full use, they aim at "creating abundant car life" and "providing attractive new values".

Though they are trying to commercialize their idea based on the knowhow of DENSO, their research and development themes are not limited to vehicles or DENSO's existing product lineup. In other words, the name of "DENSO" prefixed to the company name makes the Company's image sound like a specialist in business for automobiles; actually, however, in the trend that not only vehicles but also every single thing is trying to connect each other through a network, researchers having different technologies or backbones are proceeding researches with free ideas, in consideration of wide commercial lineups in the future.

As part of this, Mr. Yoshizawa clearly says that not a few employees of the Company do not necessarily have background of automobiles; so to speak, "are not infected with vehicle culture". Work environment is built up with care based on the stand point of "Feel happy if I had something like this" so that unique proposals can be made voluntarily. Among them, technologies available not only in vehicle-related scenes but also in a variety of life scenes have gradually been commercialized as products, such as "DAYS7", a lifelog viewer application that allows smart arrangement by linking with various lifelogs. In particular, he pays attention to the increased weight on software including the change in environment that surrounds automobiles. He says that the Company is going to reach the very important phase in which outcomes of their research and development go out in the world on a greater scale than before from now on.
		Mr. Akira Yoshizawa, Senior Researcher, Manager, Research & Development Group, DENSO IT LABORATORY, INC.

Organized with 3 dedicated researchers as core staff, "HMI Team" targets research & development towards realization of safe and secure vehicles within the areas of cognitive science and UI. They work in cooperation with other researchers as required depending on the project. There Mr. Yoshizawa also takes charge of management of the whole team as well as his own researches.

It was a project on HMI in the vehicle cockpit he was working on that made them decide introducing UC-win/Road. They thought, "There are various parts within a vehicle including a display of a navigation system. They should not sound the alarm or communicate individually but be designed as a whole cockpit under an integrated concept." To find out what kind of cockpit it should be, they collected parts etc. stored in the Company to make up a vehicle shape with frames.

In the beginning, though putting a steering wheel on it, they started with displaying a movie on the front screen, to provide environment where one could feel as if they were driving a vehicle. They even didn't have a driving simulator (DS); therefore, they did not assume a function of simulation.

When they created one as a part of feasibility study, a staff member of DENSO headquarters advised them, saying "there is a software program I recommend". It was UC-win/Road that he mentioned.

Utilizing DS for research use based on UC-win/Road

"The first impression I had when I touched it ("UC-win/Road") was that (having various functions as a simulator) it could allow us to try a lot of things. Besides, I thought using its CG function wisely could allow creation of a wellarranged town with reality." In addition, he was also attracted to its high scalability such as its supporting controller as well as its portability allowing the users to take it with them for demonstration outside of the Company by putting the data in PC.

Adoption of UC-win/Road allowed them to prepare simulation environment, and research for finding out desirable HMI got started on a full scale.

"If you really seek for realty, it is better to have real-car environment." However, if they try to perform what is associated with safe driving such as looking away or careless driving, risk of accidents is inevitable. Moreover, when driving a vehicle outside various accompanying events happen. As this causes a problem in terms of reproducibility, a simulator was adopted as an essential tool in this research, says he.

However, functions mounted in the thenversion of UC-win/Road were not enough to cover the research theme. Therefore, the product was partially customized. For example, the HMI system being examined and the simulator were linked with each other to allow data communication. In this conversion, when a command is transmitted from the system on the user's side to the simulator through the network, the vehicle in the simulator reflects it and changes its movement. Mr. Yoshizawa mentions FORUM8's flexible support for such customization as a great advantage.

The Team has built up a DS based on UC-win/Road now. Setting up diverse risk factors, they use it for evaluation and research on necessary functions for driving of the test subjects and HMI to compensate it. At the same time, they use it effectively in their demonstration of research outcomes to the visitors to the Company or at various exhibitions.

When they introduced UC-win/ Road, they aimed at integrating the car navigation system and meters etc. to increase their added values from the standpoint of supporting the driver's driving. In his opinion, however, "cooperation with the driver ranging to the control systems is getting important" in recent years in consideration of ADAS (Advanced Driver Assistance System) and automated driving. This evaluation makes the role of simulators become larger and larger, explains Mr. Yoshizawa.

He points out that, for example, such functions will be required as to represent more complicated environments like driver-to-pedestrian or vehicle-to-vehicle cooperation, in different situations where a vehicle is travelling alone as well as where plural vehicles and people are passing. He mentions his hope to clear these problems using UC-win/Road. On the other hand, to make software

On the other hand, to make software grow, a cycle is important in which data are accumulated through their use and the software itself is refined based on them. This is also true about the simulator as well as the "connected car", a car having a communication function. He says that the software gets smarter as the data on how to use it are collected by allowing various people to have experiences.

"In this sense, it is also required to view the vehicle not only as a thing for just travelling but also as a moving terminal, for which data should be utilized effectively."

HMI team members