Augmented reality

The augmented reality is one of the lines of research of the Computer Graphics Group of the UPV. In this sense, the researchers have participated in different projects that combine reality and virtuality. With this, it is intended to create a system such that users can not distinguish between the real scene and the virtual elements included in it.

In this area, you can see the augmented reality projects in which the research team of the Polythecnic University of Valencia has collaborated.

AR3 Senses

Researchers from the Universitat Politècnica de València and the University of Zaragoza have developed an augmented reality app to evaluate spatial memory. Among its applications, it could be used in patients with Alzheimer’s, as a tool for the analysis and training of skills related to this memory – as long as the patient is not at an advanced stage of the disease. It could also be of great help in the infant stage, for the early identification of alterations related to that spatial memory. The work of researchers from the UPV and Unizar has been published in the IEEE Access journal. In addition, you can also find more information about this project in the following link.

Video of the app to evaluate the spatial memory

CHILDMNEMOS Project

This project was founded by the Spanish Goverment (National funds) from 2013-2015. Coordinated project. Participants: ai2 of the UPV (coordinator), University of Zaragoza, UAM of Madrid and Escola d’Estiu (UPV).

In CHILDMNEMOS has been developed and validated innovative Virtual Reality (VR) and Augmented Reality (AR) systems for the assessment of spatial short-term memory. It has been shown that both AR and VR are suitable for the assessment of spatial short term memory in both children and adults. It has also been found that these technologies can be used as an alternative to traditional methods with the mentioned objective. Specifically, we have developed the following systems:

ARSM, AR on mobile devices showing virtual objects into real boxes in a real environment. The participant must remember the place where these virtual objects were.

MnemoCity is to locate places associated with visual stimuli presented in a virtual environment (a park). The visualization is stereoscopic and used a large display. For the interaction have been used the Wii Balance Board and Wii Remote vs. a gamepad.

 

MnemoMaze is a virtual environment that recreates a maze with hedges of 3 meters of high, with intersections in which the participants, after a phase of habituation, they must remember the way to reach the end. The display devices and for interaction are the same as the used in MnemoCity.

SAAM-2K is to locate places associated with auditory stimuli. Karotz, environmental devices, with the form of a rabbit are used. The participant hears a sound and should remember the Karotz that emitted the sound.

VRS-OBJ recreates a maze. The display is stereoscopic and the Oculus Rift is used.

All tasks performed are compared with traditional tests or other developed systems. In our studies participated 494 children and 262 adults. From these studies, the following additional conclusions were obtained:

  • Significant improvement in the performance of the group of older children was found.
  • The visuospatial short-term memory is not fully developed at the age of 9 years.
  • No statistically significant differences were found between gender.
  • The participants liked the tasks, which seemed easy to use.

APRENDRA Project

Aprendra Project was founded by the Spanish Goverment (National funds) from 2010-2012. The participants of this project were: ai2 of the UPV (coordinator), AIJU and Escola d’Estiu (UPV).

In the APRENDRA project, we have developed innovative Augmented Reality systems for edutainment in which new devices have been used, tangible interfaces and different visualization and interaction technologies. Specifically, the following systems have been developed using:

  1. Mobile Devices: iPhone, Tablet PCs (HP Slate 500).
  2. Tangible Interfaces, e.g., rotatory support.
  3. Visualization systems:
    • Video see-through (e.g.iPhone).
    • Autostereoscopy (autostereoscopic display of 46″ and LG Optimus 3D (autostereoscopic mobile)).
    • Frontal Projection.
  4. Natural interaction (using gestures).

All developed systems were compared with traditional learning or other developed systems. In our studies have participated 782 children. From these studies have obtained the following conclusions:

  • No statistically significant differences were found with regard to the acquired knowledge between the developed systems (autonomous) and traditional learning (guided). These results indicate that these systems could provide versatility in the learning process, and that learning could take place at any place and any time without supervision. Therefore, it could be a useful tool to complement the learning process and assist in training needs.
  • In mobile devices, the characteristics of the device (screen size and weight) were not a factor that had a significant influence on the lessons learned. However, the management of smaller devices, lower weight is a factor appreciated by younger children (7-8 years).
  • Children learn significantly more with collaborative learning.
  • Children have the depth sense (3D) with the autostereoscopic devices used, although it is a factor that had a significant influence on learning. However, it is an aspect that can be exploited in education when the sense of depth is relevant to transmit knowledge.
  • Interaction using gestures is a natural interaction which facilitates system management. The children can learn in a similar way as they are used to play.

Two of the systems developed are for learning multiculturalism or the water cycle.

  • Learning multiculturalism. The first game developed in APRENDRA. The iPhone was used. Children learn about food, animals, and monuments that can be found in the poorest continents. The game was included in the activity program of the Summer School of the Technical University of Valencia (july 2010). Eighty-four children played the AR game and equivalent traditional games.

  • Learning the water cycle. The second game developed in APRENDRA. An iPhone and a Tablet PC were used. Children learn about the water cycle. The game was included in the activity program of the Summer School of the Technical University of Valencia (july 2011). Seventy-nine children played with the two devices.

A third system developed are for learning about five historical ages (Prehistory, Ancient Times, the Middle Ages, the Early Modern Period, and the Contemporary Period). The system has two different versions. One of the versions is an autostereoscopic system that lets children see themselves as a background in the game and renders the elements in 3D without the need for special glasses; the second version is a frontal projection system that projects the image on a table in 2D and works similarly to a touch table. The Microsoft Kinect© is used in both systems for the interaction. A comparative study to check different aspects between these two versions was carried out. A total of 128 children from 7 to 11 years old participated in this study.

Interview to Mari Carmen Juan about the Aprendra project

AR Systems for the treatment of Phobia to Small Animals

AR System with visible markers and HMD video see-through

For this project, the system was used for the treatment of 10 patients. Six with phobia to cockroaches and 4 with phobia to spiders. Six with phobia to cockroaches and 4 with phobia to spiders.

AR System with invisible markers and HMD video see-through

A presence and anxiety studies were carried out comparing the visible marker system and this system with 25 participants.

Phobia to small animals. Invisible Markers - 1

AR System with visible markers and HMD optical see-through

A presence and anxiety studies were carried out comparing the visible marker system and this system with 23 participants.

Phobia to small animals. Optical See-through - 1

AR Systems for Learning/Edutainment

ARCoins

AR for learning about numismatics

ARCoins

 

ARBreakfast

AR for Therapeutic Education in Diabetes

ARBreakfast ARBreakfast
HCLearn HCLearn

ARDental

An Augmented Reality system for learning dental morphology developed in collaboration with the training center Folguera-Vicent.

ARDental ARDental

 

ARBrain

AR for for learning brain estructures

ARBrain ARBrain

Storytelling

Two AR interactive storytelling systems that used tangible cubes. The first showed the story in only one face of the cube and the second presented the story in all visible faces of the cube. The game was included in the activity program of the Summer School of the Polytechnic University of Valencia (july 07). Forty-four children took part in the activity. The children played the game using different visualization systems.

Interactive Storytelling - 1 Interactive Storytelling - 2

Human Body

Children learnt the interior of the human body. The game was included in the activity program of the Polythecnic University of Valencia (july 07). Forty children took part in the activity. The children played the game using different visualization systems.

Human Body - 1 Human Body - 2

Endangered Animals – Magnetic Cubes

The system used magnet and tangible cubes as the user interface. The game was included in the activity program of the Summer School of the Polythecnic University of Valencia (july 08). Forty-six children played the AR game and the equivalent real game.

Endangered animals. Magnet Cubes - 1 Endangered animals. Magnet Cubes - 2

Endangered Animals – Finding Matching Pairs

The game was included in the activity program of the Polythecnic University of Valencia (july 08). Thirty-two children played the AR game and the equivalent real game.

Endangered animals. Finding Matching Pairs - 1 Endangered animals. Finding Matching Pairs - 2

Learning Words

The game includes several options such as complete a word, spell a word, look for the intruder, etc. This game was tested by thirty-two children of a primary school.

Learning Words - 1 Learning Words - 2

Recycling

AR in mobile phones. The Nokia N95 with 8 Gb was used. The game was included in the activity program of the Polythecnic University of Valencia (july 08). Eighty-three children played the AR game and the equivalent basic game run on the Nokia N95 with 8 Gb.

Learning Words - 1 Learning Words - 2

Other Projects

2018

Augmented Reality Through Various Sensorial Channels. Application to Orientation and Spatial Location Processes

Responsible Researcher: M. Carmen Juan Lizandra

Code: TIN2017-87044-R

Start Date: 01/01/2018

Description: This project has consisted in the development of an Augmented Reality application using new techniques of localization in interiors, taking advantage of technological advances and validating it by means of a study of spatial orientation in users.

2013

  • Development and validation of virtual and augmented reality systems to assess short-term spatial memory in children

Responsible Researcher: Mari Carmen Juan Lizandra, Javier Lluch Crespo, José Antonio Gil-Gómez

Code: TIN2012-37381-C02-01

Project duration: 01/01/2013-31-12-2013

  • Spatial orientation in real dimensions during development: support in a technological system

Responsible Researcher: M.Carmen Juan Lizandra

Code: (2013/B001).

Project Duration: 01/01/2013 – 31/12/2013).

  • Use of New Technologies for the Interactive Presentation of Clinical Cases

Responsible Researchers: Roberto Vivó, M. Carmen Juan and Montserrat Robles Viejo.

Code: SP20120393

Project Duration: 31/12/12 – 31/12/13

  • Development and Validation of Augmented Reality Solutions Embedded in Fpga Type Programmable Devices

Responsible Researchers: M. Carmen Juan Lizandra, Emilio Camahort Gurrea, Roberto Vivó.

Code: 2716

Project Duration: 01/12/11 – 01/07/13

  • Development and Validation of Augmented Reality Systems for Learning-Entertainment

Responsible Researchers: M. Carmen Juan Lizandra, Juan Martínez-Morató

Code: TIN2009-14319-C02-01

Project Duration:01/01/2010-01/01/2013

2012

Uso de Nuevas Tecnologías para la Presentación Interactiva de Casos Clínicos / Use of New Technologies for the Interactive Presentation of Clinical Cases

Responsible Researchers: M. Carmen Juan, Roberto Vivó.

Code: SP20120393

Project Duration: 31/12/12 – 31/12/13

2011

  • Desarrollo y Validación de Soluciones de Realidad Aumentada Embebidas en Dispositivos Programables de Tipo Fpga / Development and Validation of Augmented Reality Solutions Embedded in Fpga Type Programmable Devices

Responsible Researchers: M. Carmen Juan, Emilio Camahort, Roberto Vivó.

Code: 2716

Project Duration: 01/12/11 – 01/07/13

2010

  • Development and Validation of an Augmented Reality Library with Recognition of User’s Physical Parts for Their Interaction

Responsible Researchers: M. Carmen Juan Lizandra, Ramón Mollá, María José Vicent, Francisco José Abad, Roberto Vivó, Emilio Camahort.

Code: PAID-05-09-4221

Project Duration: December 2009- December 2010

 

  • Desarrollo y Validación de Sistemas de Realidad Aumentada para Aprendizaje-Entretenimiento / Development and Validation of Augmented Reality Systems for Learning-Entertainment

Responsible researchers: M. Carmen Juan, Juan Martínez-Morató, Fernando Juan.

Code: TIN2009-14319-C02-01

Project Duration: 01/01/10 – 01/01/13

 

2009

  • Greenhunt: Mobile Phone Game to Promote Energy Saving Behaviors. Pn in Services Technologies for the Information Society. Engineering and Technology of the Environment.

Responsible Researchers: M. Carmen Juan Lizandra, David Furió Ferri and Irene Zaragozá Álvarez.

Code: PCI2006-A7-0676

Project Duration: 2008-2009

Description: The objective of this project is to learn to recycle

2007

  • Advanced Tools For Developing Highly Realistic Computer Games

Responsible Researchers: Roberto Vivó, Ramón Mollá, Javier Lluch Crespo, Francisco José Abad, Emilio Camahort

Code: TIN2004-20303-E

Project Duration: 2004-2007

Description: The GameTools project is a European Union project from the 6th Framework Programme that brings together leading European computer graphic experts from universities in Austria, France, Hungary and Spain with European industrial partners from the fields of computer game development and virtual reality to create next generation realtime 3D libraries for Geometry, Visibility and Global Ilumination for the PC platform with future extensions to consoles PS2, XBox, PS3, and XBox.

2006

  • Psychologically Augmented Social Interaction Over Networks

Responsible Researchers: M. Carmen Juan, José Antonio Gil-Gómez. José Antonio Lozano Quilis.

Code: 27654

Project Duration: 2006-2010

2004

  • Desarrollo de Sistemas Avanzados de Ultrasonografia Diagnostica e Intervencionista-Mediclab / Development of Advanced Systems of Diagnostic and Interventional Ultrasonography-Mediclab

Responsible Researchers: M. Carmen Juan, José Antonio Lozano Quilis, José Antonio Gil-Gómez.

Code: TEC2004-06647-C03-03

Project Duration: 13/12/04 – 13/12/07

 

1997

  • Sistema de Cirugía Guiada por Ordenador Mediante Realidad Aumentada. Aplicación a Neurocirugía y Radiocirugía Cerebral/Development of Advanced Systems of Diagnostic and Interventional Ultrasonography-Mediclab

Responsible researchers: M. Carmen Juan, Carlos Montserrat Aranda

Code: 971901003646

Project Duration: 01/06/97 – 30/06/99

 

*Consult all the research lines of the GIG.