• Publikationen
Show publication details

Matthies, Denys J.C.; Roumen, Thijs; Kuijper, Arjan; Urban, Bodo

CapSoles: Who Is Walking on What Kind of Floor?

2017

Association for Computing Machinery (ACM): MobileHCI 2017 : Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York: ACM, 2017, 14 p.

International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) <19, 2017, Vienna, Austria>

Foot interfaces, such as pressure-sensitive insoles, still yield unused potential such as for implicit interaction. In this paper, we introduce CapSoles, enabling smart insoles to implicitly identify who is walking on what kind of floor. Our insole prototype relies on capacitive sensing and is able to sense plantar pressure distribution underneath the foot, plus a capacitive ground coupling effect. By using machine-learning algorithms, we evaluated the identification of 13 users, while walking, with a confidence of ~95% after a recognition delay of ~1s. Once the user's gait is known, again we can discover irregularities in gait plus a varying ground coupling. While both effects in combination are usually unique for several ground surfaces, we demonstrate to distinguish six kinds of floors, which are sand, lawn, paving stone, carpet, linoleum, and tartan with an average accuracy of ~82%. Moreover, we demonstrate the unique effects of wet and electrostatically charged surfaces.

Show publication details

Matthies, Denys J.C.; Strecker, Bernhard A.; Urban, Bodo

EarFieldSensing: A Novel In-Ear Electric Field Sensing to Enrich Wearable Gesture Input through Facial Expressions

2017

Mark, Gloria (Ed.) et al.: CHI '17. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York: ACM Press, 2017, pp. 1911-1922

Conference on Human Factors in Computing Systems (CHI) <35, 2017, Denver, CO, USA>

EarFieldSensing (EarFS) is a novel input method for mobile and wearable computing using facial expressions. Facial muscle movements induce both electric field changes and physical deformations, which are detectable with electrodes placed inside the ear canal. The chosen ear-plug form factor is rather unobtrusive and allows for facial gesture recognition while utilizing the close proximity to the face. We collected 25 facial-related gestures and used them to compare the performance levels of several electric sensing technologies (EMG, CS, EFS, EarFS) with varying electrode setups. Our developed wearable fine-tuned electric field sensing employs differential amplification to effectively cancel out environmental noise while still being sensitive towards small facial-movement-related electric field changes and artifacts from ear canal deformations. By comparing a mobile with a stationary scenario, we found that EarFS continues to perform better in a mobile scenario. Quantitative results show EarFS to be capable of detecting a set of 5 facial gestures with a precision of 90% while sitting and 85.2% while walking. We provide detailed instructions to enable replication of our low-cost sensing device. Applying it to different positions of our body will also allow to sense a variety of other gestures and activities.

Show publication details

Strecker, Bernhard Arthur; Matthies, Denys J.C. (Betreuer); Thaller, Manfred (Betreuer)

A Wearable Facial Expression Control Interface to Accomplish Microinteractions in Mobile Scenarios

2016

Köln, Univ., Master Thesis, 2016

In our modern society, computing technology has become ubiquitous. Starting with the advent of smartphones, wearable computing has become ever more commonplace and pervasive in recent years. Interaction does no longer only happen in confined environments. Having access to computer interface anytime and anywhere necessitates new interaction styles to fully benefit from the manifold technological and interactional possibilities computers offer. Since hands and eyes of users on the move are likely to be busy with other activities, alternative interaction strategies are worthwhile to consider. The human face is very communicative and consistently used for human-to-human interaction, but close to never used for human-computer-interaction. In this thesis, facial expression recognition with unobtrusive sensing technology is investigated. An overview over the concepts of wearable computing, periphal computer interaction and microinteractions is given. Since bulky hardware such as cameras cannot be used in mobile contexts without attracting attention and distracting the user, alternative technologies have been tested. Focus was laid on thoroughly testing seven different earplugs with mounted electrodes. A newly developed gesture set of 25 facial gestures was tested in conjunction with capacitive sensing, electric field sensing, and electromyography sensing technology. Facial muscle movements lead to electric and mechanical changes, which can be sensed inside the ear-canal. Classification accuraries above 90% for 5 gestures were achieved with both industrial grade capacitive sensing technology and a self-developed electric field sensing board. The theory behind each of the sensing technologies is explained depth throughout this thesis.

Show publication details

Matthies, Denys J.C.; Bieber, Gerald; Kaulbars, Uwe

AGIS: Automated Tool Detection & Hand-Arm Vibration Estimation using an Unmodified Smartwatch

2016

Matthies, Denys J.C. (Ed.) et al.: iWOAR 2016 : 3rd international Workshop on Sensor-based Activity Recognition and Interaction. New York: ACM Press, 2016. (ACM International Conference Proceedings Series 1183), Art. No. 8, 4 p.

International Workshop on Sensor-based Activity Recognition (iWOAR) <3, 2016, Rostock, Germany>

Over the past three decades, it has been known that longlasting and intense hand-arm vibrations (HAV) can cause serious diseases, such as the Raynaud- / White Finger- Syndrome. In order to protect workers nowadays, the longterm use of tools such as a drill, grinder, rotary hammer etc. underlie strict legal regulations. However, users rarely comply with these regulations because it is quite hard to manually estimate vibration intensity throughout the day. Therefore, we propose a wearable system that automatically counts the daily HAV exposure doses due to the fact that we are able to determine the currently used tool. With the implementation of AGIS, we demonstrate the technical feasibility of using the integrated microphone and accelerometer from a commercial smartwatch. In contrast to prior works, our approach does not require a technical modification of the smartwatch nor an instrumentation of the environment or the tool. A pilot study shows our proofof- concept to be applicable in real workshop environments.

Show publication details

Matthies, Denys J.C.; Haescher, Marian; Bieber, Gerald; Urban, Bodo

iWOAR 2016: 3rd international Workshop on Sensor-based Activity Recognition and Interaction

2016

New York : ACM Press, 2016

International Workshop on Sensor-based Activity Recognition (iWOAR) <3, 2016, Rostock, Germany>

ACM International Conference Proceedings Series 1183

Wearable sensors potentially enable for a better and unobtrusive recognition of human activity and the state of rest, sleep, stress and drive the ongoing trend of the quantified self-movement. As an enabling technology, powerful, while yet inexpensive MEMS-Chips (micro-electro-mechanical system) push the penetration of a broad variety of mobile devices. Thereby, these devices gain high interest, not only in terms of general customer products, but also as integrated systems in an industrial context, either way to enable continuous monitoring of complex life processes and workplace situations. Another challenge that research is facing concerns the limited human abilities of interaction in context of mobility and in situations, in which high attention is being demanded. New and alternative ways are needed to be found in order to take advantage of all human capabilities to enable safe and unobtrusive interaction.

Show publication details

Chua, Soon Hau; Perrault, Simon T.; Matthies, Denys J.C.; Zhao, Shengdong

Positioning Glass: Investigating Display Positions of Monocular Optical See-Through Head-Mounted Display

2016

Proceedings of the Fourth International Symposium on Chinese CHI : ChineseCHI 2016. New York: ACM, 2016. (ACM International Conference Proceedings Series 01219), Article 1, 6 p.

International Symposium of Chinese CHI (ChineseCHI) <4, 2016, San Jose, CA, USA>

The physical position of the display on a monocular OSTHMD in relative to our eye is an important factor of the performance and usability in dual-task scenarios. We investigated 9 different display positions in a modern dual-task scenario with 27 participants. The experiment-involved participants responding to 3 different types of notifications displayed on the HMD while performing a visually intensive primary task. We found that although the notifications at the middle and bottom center positions were noticed quicker, the top and the peripheral positions were more comfortable, unobtrusive, and preferred. In particular, middle-right strikes the best balance between performance and usability in the dual-task scenario we studied. Our findings and discussions demonstrated the need for further work and a more rigorous investigation in dual-task scenarios with characteristics dissimilar to ours.

Show publication details

Matthies, Denys J.C.; Haescher, Marian; Bieber, Gerald; Salomon, Ralf; Urban, Bodo

SeismoPen: Pulse Recognition via a Smart Pen

2016

Association for Computing Machinery (ACM): Proceedings of the 9th International Conference on PErvasive Technologies Related to Assistive Environments : PETRA 2016. New York: ACM, 2016. (ACM International Conference Proceedings Series (ICPS) 01198), Article No. 36, 4 p.

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <9, 2016, Corfu, Greece>

We propose SeismoPen, an enhanced ballpoint pen, which is capable of calculating the patient's heart rate. This is enabled when being pressed the pen towards the patient's throat so it can sense and analyze the seismographic micro-eruption caused by the pulsing blood. We developed a suitable algorithm and tested three sensor setups in which we attached (1) a force-sensing resistor (FSR), (2) an accelerometer, and (3) a piezoelectric transducer to the pen's head. We also conducted a user study, which resulted in suggesting SeismoPen to be potentially more accepted by users, since it is less obtrusive than alternative measurement methods. In contrast to medical devices, this simple pen looks less perilous and potentially reduces the risk of triggering symptoms of a white coat hypertension.

Show publication details

Haescher, Marian; Trimpop, John; Matthies, Denys J.C.; Urban, Bodo

SeismoTracker: Upgrade Any Smart Wearable to Enable a Sensing of Heart Rate, Respiration Rate, and Microvibrations

2016

Kaye, Jofish (Conference Chair) et al.: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing : CHI EA 2016. New York: ACM Press, 2016, pp. 2209-2216

Conference on Human Factors in Computing Systems (CHI) <34, 2016, San Jose, CA, USA>

In this paper we present a method to enable any smart Wearable to sense vital data in resting states. These resting states (e.g. sleeping, sitting calmly, etc.) imply the presence of low-amplitude body-motions. Our approach relies on seismocardiography (SCG), which only requires a built-in accelerometer. Compared to commonly applied technologies, such as photoplethysmography (PPG), our approach is not only tracking heart rate (HR), but also respiration rate (RR), and microvibrations (MV) of the muscles, while being also computational inexpensive. In addition, we can calculate several other parameters, such as HR variability and RR variability. Our extracted vital parameters match with the vital data gathered from clinical state-of-the art technology. These data allow us to gain an impression on the user's activity, quality of sleep, arousal and stress level over the whole day, week, month, or year. Moreover, we can detect whether a device is actually worn or doffed, which is crucial when connecting such data with health services. We implemented our method on two current smartwatches: a Simvalley AW420 RX as well as on a LG G Watch R and recorded user data for several months. A web platform enables to keep track of one's data.

Show publication details

Klamet, Josephin; Matthies, Denys J.C.; Minge, Michael

WeaRelaxAble: A Wearable System to Enhance Stress Resistance using Various Kinds of Feedback Stimuli

2016

Matthies, Denys J.C. (Ed.) et al.: iWOAR 2016 : 3rd international Workshop on Sensor-based Activity Recognition and Interaction. New York: ACM Press, 2016. (ACM International Conference Proceedings Series 1183), Art. No. 2, 6 p.

International Workshop on Sensor-based Activity Recognition (iWOAR) <3, 2016, Rostock, Germany>

This paper introduces a wearable feedback device that aims at relaxing the user in stressful situations. The system, which is called WeaRelaxAble, provides various feedback modalities, such as vibration, ambient light, acoustic stimuli and heat in order to reduce the user's stress level. The development of WeaRelaxAble is based on two studies: At first, all five kinds of feedback and appropriate body positions for stimulation were evaluated with 15 participants. Based on the findings of this initial study, we built a wearable Arduino prototype to prove the feasibility of our concept. The experience while using the system was tested with 26 test subjects under laboratory conditions. We conclude with a concept design of a wrist-worn device that provides acoustic and visual feedback. As tactile stimulation, a shirt would provide vibration at the positions of the shoulders as well as heat at the loins. Users can explicitly activate the system at any time and in any combination of feedback modalities.

Show publication details

Meier, Anita; Matthies, Denys J.C.; Heidmann, Frank

A Brief Survey on Understanding the Interaction between Human and Technology at the Task of Pedestrian Navigation

2015

Schulz, Hans-Jörg (Ed.) et al.: Proceedings of the International Summer School on Visual Computing 2015. Stuttgart: Fraunhofer Verlag, 2015, pp. 135-140

International Summer School on Visual Computing <1, 2015, Rostock, Germany>

In this paper we present a brief summary of an online survey we conducted in 2014. 135 participants successfully completed this survey, whereby 46% of the subjects were females and 54% males. We found out, that nowadays many users fall back on using smartphones in order to orientate themselves in unknown environments. In terms of analog navigation methods, people still rely on street name signs and landmarks such as characteristic buildings. However, relying on current smartphone habits for navigation tasks, visual attention is usually heavily drawn, which can cause a reduced perception and potentially makes smartphone map navigation more dangerous.

Show publication details

Haescher, Marian; Matthies, Denys J.C.; Trimpop, John; Urban, Bodo

A Study on Measuring Heart- and Respiration-Rate via Wrist-Worn Accelerometer-based Seismocardiography (SCG) in Comparison to Commonly Applied Technologies

2015

Matthies, Denys J.C. (Ed.) et al.: iWOAR 2015 : 2nd international Workshop on Sensor-based Activity Recognition and Interaction. New York: ACM Press, 2015, 6 p.

International Workshop on Sensor-based Activity Recognition (iWOAR) <2, 2015, Rostock, Germany>

Since the human body is a living organism, it emits various life signs which can be traced with an action potential sensitive electromyography, but also with motion sensitive sensors such as typical inertial sensors. In this paper, we present a possibility to recognize the heart rate (HR), respiration rate (RR), and the muscular microvibrations (MV) by an accelerometer worn on the wrist. We compare our seismocardiography (SCG) / ballistocardiography (BCG) approach to commonly used measuring methods. In conclusion, our study confirmed that SCG/BCD with a wrist-worn accelerometer also provides accurate vital parameters. While the recognized RR deviated slightly from the ground truth (SD=16.61%), the detection of HR is nonsignificantly different (SD=1.63%) to the gold standard.

Show publication details

Haescher, Marian; Trimpop, John; Matthies, Denys J.C.; Bieber, Gerald; Urban, Bodo; Kirste, Thomas

aHead: Considering the Head Position in a Multi-sensory Setup of Wearables to Recognize Everyday Activities with Intelligent Sensor Fusions

2015

Kurosu, Masaaki (Ed.): Human-Computer Interaction. Proceedings Part II : Interaction Technologies. Springer International Publishing, 2015. (Lecture Notes in Computer Science (LNCS) 9170), pp. 741-752

International Conference on Human-Computer Interaction (HCII) <17, 2015, Los Angeles, CA, USA>

In this paper we examine the feasibility of Human Activity Recognition (HAR) based on head mounted sensors, both as stand-alone sensors and as part of a wearable multi-sensory network. To prove the feasibility of such setting, an interactive online HAR-system has been implemented to enable for multi-sensory activity recognition while making use of a hierarchical sensor fusion. Our system incorporates 3 sensor positions distributed over the body, which are head (smart glasses), wrist (smartwatch), and hip (smartphone). We are able to reliably distinguish 7 daily activities, which are: resting, being active, walking, running, jumping, cycling and office work. The results of our field study with 14 participants clearly indicate that the head position is applicable for HAR. Moreover, we demonstrate an intelligent multi-sensory fusion concept that increases the recognition performance up to 86.13 % (recall). Furthermore, we found the head to possess very distinctive movement patterns regarding activities of daily living.

Show publication details

Haescher, Marian; Matthies, Denys J.C.; Urban, Bodo

Anomaly Detection with Smartwatches as an Opportunity for Implicit Interaction

2015

Association for Computing Machinery (ACM): MobileHCI 2015 : Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York: ACM, 2015, pp. 955-958

International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) <17, 2015, Copenhagen, Denmark>

In this paper we introduce application scenarios for implicit interaction with Smartwatches for the purpose of user assistance, to create awareness, and to enhance as well as simplify the interaction with Wearables. We envision three scenarios (1) the detection of sleep apnea, (2) the detection of epileptic seizures, and (3) a detection of accidents such as falling, car crashes etc., which are presented and discussed. Therefore, the recognition of all incidents described will be discussed under the meta-topic of anomaly detection.

Show publication details

Matthies, Denys J.C.; Perrault, Simon T.; Urban, Bodo; Zhao, Shengdong

Botential: Localizing On-Body Gestures by Measuring Electrical Signatures on the Human Skin

2015

Association for Computing Machinery (ACM): MobileHCI 2015 : Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York: ACM, 2015, pp. 207-216

International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) <17, 2015, Copenhagen, Denmark>

We present Botential, an on-body interaction method for a wearable input device that can identify the location of onbody tapping gestures, using the entire human body as an interactive surface to expand the usually limited interaction space in the context of mobility. When the sensor is being touched, Botential identifies a body part's unique electric signature, which depends on its physiological and anatomical compositions. This input method exhibits a number of advantages over previous approaches, which include: 1) utilizing the existing signal the human body already emits, to accomplish input with various body parts, 2) the ability to also sense soft and long touches, 3) an increased sensing range that covers the whole body, and 4) the ability to detect taps and hovering through clothes.

Show publication details

Haescher, Marian; Matthies, Denys J.C.; Bieber, Gerald; Urban, Bodo

CapWalk: A Capacitive Recognition of Walking-Based Activities as a Wearable Assistive Technology

2015

Association for Computing Machinery (ACM): Proceedings of the 8th International Conference on PErvasive Technologies Related to Assistive Environments : PETRA 2015. New York: ACM, 2015. (ACM International Conference Proceedings Series (ICPS) 1032)

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <8, 2015, Corfu, Greece>

In this research project, we present an alternative approach to recognize various walking-based activities based on the technology of capacitive sensing. While accelerometry-based walking detections suffer from reduced accuracy at low speeds, the technology of capacitive sensing uses physical distance parameters, which makes it invariant to the duration of step performance. Determining accurate levels of walking activity is a crucial factor for people who perform walking with tiny step lengths such as elderlies or patients with pathologic conditions. In contrast to other gait analysis solutions, CapWalk is mobile and less affected by external influences such as bad lighting conditions, while it is also invariant to external acceleration artifacts. Our approach enables a reliable recognition of very slow walking speeds, in which accelerometer-based implementations can fail or provide high deviations. In CapWalk we present three different capacitive sensing prototypes (Leg Band, Chest Band, Insole) in the setup of loading mode to demonstrate recognition of sneaking, normal walking, fast walking, jogging, and walking while carrying weight. Our designs are wearable and could easily be integrated into wearable objects, such as shoes, pants or jackets. We envision such gathered information to be used to assist certain user groups such as diabetics, whose optimal insulin dose is depending on bread units and physical activity or elderlies whose personalized dosage of medication can be better determined based on their physical activity.

Show publication details

Meier, Anita; Matthies, Denys J.C.; Urban, Bodo; Wettach, Reto

Exploring Vibrotactile Feedback on the Body and Foot for the Purpose of Pedestrian Navigation

2015

Matthies, Denys J.C. (Ed.) et al.: iWOAR 2015 : 2nd international Workshop on Sensor-based Activity Recognition and Interaction. New York: ACM Press, 2015, 11 p.

International Workshop on Sensor-based Activity Recognition (iWOAR) <2, 2015, Rostock, Germany>

In this paper, we present an evaluation of vibrotactile onbody feedback for the purpose of pedestrian navigation. For this specific task, many researchers already provide different approaches such as vibrating belts, wristbands or shoes. Still, there are issues left that have to be considered, such as which body position is most suitable, what kind of vibration patterns are easy to interpret, and how applicable are vibrotactile feedback systems in real scenarios. To find answers, we reconstructed prototypes commonly found in literature and continued to further evaluate different foot-related designs. On the one hand, we learned that vibrotactile feedback at the foot reduces visual attention and thus also potentially reduces stress. However, on the other hand, we found that urban space can be very diverse, and ambiguous and therefore a vibrotactile system cannot completely replace common path finding systems for pedestrians. Rather, we envision such a system to be applied complementary as an assistive technology.

Show publication details

Matthies, Denys J.C.; Haescher, Marian; Aehnelt, Mario; Bieber, Gerald; Urban, Bodo

iWOAR 2015: 2nd international Workshop on Sensor-based Activity Recognition and Interaction

2015

New York : ACM Press, 2015

International Workshop on Sensor-based Activity Recognition (iWOAR) <2, 2015, Rostock, Germany>

Wearable sensors potentially enable for a better and unobtrusive recognition of human activity and the state of rest, sleep, stress and drive the ongoing trend of the quantified self-movement. As an enabling technology, powerful, while yet inexpensive MEMS-Chips (micro-electro-mechanical system) push the penetration of a broad variety of mobile devices. Thereby, these devices gain high interest, not only in terms of general customer products, but also as integrated systems in an industrial context, either way to enable continuous monitoring of complex life processes and workplace situations. Another challenge that research is facing concerns the limited human abilities of interaction in context of mobility and in situations, in which high attention is being demanded. New and alternative ways are needed to be found in order to take advantage of all human capabilities to enable safe and unobtrusive interaction. This conference-like workshop is initiated and organized by the Fraunhofer IGD in Rostock. While we established the iWOAR workshop the 2nd time - in Rostock, North Germany - we aim to establish an annual tradition with the approach to bring together science and industry. It offers scientists, interested parties, and users in the area of sensor-based activity recognition and interaction the possibility to an exchange of experiences and a presentation of best-practice examples, as well as technical and scientific results. The workshop focuses on technologies for human activity recognition and interaction via inertial sensors (accelerometers, gyroscopes etc.) and their scientific applications.

Show publication details

Matthies, Denys J.C.; Haescher, Marian; Alm, Rebekka; Urban, Bodo

Properties of a Peripheral Head-Mounted Display (PHMD)

2015

Stephanidis, Constantine (Ed.): HCI International 2015 - Posters' Extended Abstracts. Proceedings Part I : HCI International 2015. Springer International Publishing, 2015. (Communications in Computer and Information Science (CCIS) 528), pp. 208-213

International Conference on Human-Computer Interaction (HCII) <17, 2015, Los Angeles, CA, USA>

Show publication details

Trimpop, John; Haescher, Marian; Bieber, Gerald; Matthies, Denys J.C.; Lämmel, Friedrich; Burggraf, Paul

The Digital Health Companion: Personalized Health Support on Smartwatches via Recognition of Activity- and Vital-Data

2015

Schulz, Hans-Jörg (Ed.) et al.: Proceedings of the International Summer School on Visual Computing 2015. Stuttgart: Fraunhofer Verlag, 2015, pp. 125-133

International Summer School on Visual Computing <1, 2015, Rostock, Germany>

It has been shown that in various fields of social life, people tend to seek opportunities to measure their daily activities, bodily behaviors, and health related parameters. These kinds of activity tracking should be accomplished comfortably, unobtrusively and implicitly. Tracking behavior can be important for certain user groups, such as the growing population of elderlies. These people have a substantially higher risk of falling down, as they often live alone and thus have a greater need for other supporting services, as emergencies quickly occur. We would like to support these people, while providing a comfortable emergency detection and a monitoring of physical activities. Moreover, we believe such tracking applications to be beneficial for any user group, since we can perceive the trend of quantified self: knowing about one's own body characteristics, which is expressed in body movement. Simultaneously, we also perceive that a strong desire for a comprehensive monitoring of vital and health data is emerging. In this paper we describe the concept and implementation of the Digital Health Companion, a smart health support system that combines research developments of activity, vital data, and anomaly recognition with the functionality of contemporary smartwatches. The system's health monitoring includes an emergency detection and allows for the prevention of health risks in the short and long term through the recognition of body movement patterns.