An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones
October 5th, 2020
The presented protocol integrates various evaluation methods and demonstrates a method to evaluate the keyboard design on smartphones. Pairs matched by English characters are proposed as the input material, and the transition time between two keys is used as the dependent variable.
This open source protocol and toolkit could inspire the novice to start their studies in keyboard designs, optimize the user experience, and improve the revenue of input method at the companies. This protocol provides a systematic method to evaluate the keyboard design more precisely on smartphones. Various highly accepted indicators, and the new special indicators are involved in this protocol.
A novice researcher may be confused about choosing appropriate evaluating indicators, and experimental programming. We provide subjective indicators, objective indicators, and Python script to reference. Demonstrating the procedure will be Yincheng Wang and Ke Wang, students from my laboratory.
Begin by reading the consent form to the participant regarding the experimental procedure, and their discretion for participation, including the experiment procedure task, and whether they will encounter any mental or physical injuries. If the participant agrees to participate, ask them to sign the informed consent form. Measure the hand size of every participant to eliminate the effect of hand size difference, and to provide repeatable data for future research.
Collect demographic data, such as age, gender, precise input experience, and occupation of the participants. Disinfect all devices and clean the body parts of the participant that will touch the devices. Ask participants to wash their hands and clean their smartphone screens.
Ask the participants to wear portable wireless physiological detectors, or a motion capture system. Then ask them to put the portable wireless physiological detection wristband on their non-dominant hand to record a galvanic skin response, and heart rate, without noise interference. To collect precise body and finger movement place passive markers of the motion capture system on the fingernails, the proximal phalanx of the finger, cervical vertebrae from C3 to C5, and arm.
Stick wireless electrodes to the skin of the arms and forearms to detect the electromyographic activity. Then calibrate all devices. Allow the participant to complete the training task, which is used to improve their familiarity with the input tasks and keyboards.
Only allow participants whose accuracy reaches 80%or more in 150 seconds to enter the formal trials. Let participants complete formal trials under all experimental conditions described in the text manuscript, having them complete the input task in a random or a balanced order. Each pair, word, or sentence represents a trial, and different experimental designs produce different experimental conditions.
After each condition, ask the participants to complete all the questionnaires at random to obtain subjective feedback. And allow them to rest for one minute. Express appreciation to participants with monetary or material rewards.
This study was used to evaluate two keyboard layouts, namely the curved QWERTY and traditional QWERTY, with different sizes of buttons. The statistical analysis of the input performance, subjective feedback, and fitted ellipse of each button is shown here. The transition time between two keys with a small button size was significantly longer than that between keys with a large button size in the curved QWERTY.
The word error rate of the traditional QWERTY was significantly lower than that of the curved QWERTY. The main effect of keyboard layout is significant in intent to use"and subjective workload, showing that participants perceive less subjective workload and are more likely to use the curved QWERTY when compared to the traditional QWERTY. The six dimensions of the subjective workload are shown here.
In the area of the fitted ellipse, the interaction between keyboard layout and button size is significant, showing that for both small and large button size, the area of the traditional QWERTY is larger than that of the curved QWERTY. For both keyboard layouts, the area of the small button is smaller than that of the large button. In the offset of the fitted ellipse, the interaction between keyboard layout and button size is only significant in the offset in the Y direction.
In the curved QWERTY, the offset in the Y direction of the small button is significantly shorter than that of the big button. Remember to keep the screen and participant's hand clean before experiment. Training tasks and balanced experimental conditions help to avoid the practice effect and other confounding factors.
This protocol can be adopted in the evaluation of keyboard design in the touch screen of other smart devices, such as a wearable smart watches, tablet PC, and virtual reality devices.
View the full transcript and gain access to thousands of scientific videos
Overview
This protocol provides a systematic method to evaluate keyboard design on smartphones, integrating various evaluation methods. It aims to optimize user experience and enhance input method revenue for companies.
Key Study Components
Area of Science
- Human-Computer Interaction
- Usability Testing
- Design Optimization
Background
- Smartphone keyboard design significantly impacts user experience.
- Choosing appropriate evaluation indicators can be challenging for novice researchers.
- This protocol includes both subjective and objective indicators.
- Python scripts are provided for reference to aid in experimental programming.
Purpose of Study
- To evaluate keyboard design more precisely on smartphones.
- To inspire novice researchers in the field of keyboard design.
- To improve the overall user experience with input methods.
Methods Used
- Integration of various evaluation methods.
- Use of matched pairs based on English characters as input material.
- Measurement of transition time between keys as a dependent variable.
- Involvement of subjective and objective indicators.
Main Results
- The protocol offers a comprehensive approach to keyboard design evaluation.
- It provides clarity on selecting appropriate evaluation indicators.
- Demonstration of the procedure by students from the laboratory.
- Potential to enhance user experience and company revenue.
Conclusions
- This protocol is a valuable resource for novice researchers.
- It systematically addresses the complexities of keyboard design evaluation.
- Encourages further exploration and optimization in keyboard design.
Frequently Asked Questions
Executive Industry Relevance
This protocol provides a systematic method to evaluate keyboard design on smartphones, integrating subjective and objective indicators to optimize user experience and inform design decisions. It supports human-computer interaction research by offering standardized evaluation tools that reduce ambiguity in assessing input efficiency and comfort. The approach enables reproducible data collection and analysis, facilitating cross-study comparisons and informing iterative design improvements for touchscreen interfaces.
Strategic Applications in Biopharma R&D
Early Discovery & Target Validation
- Scientific Value: Enables hypothesis testing of keyboard layout effects on user performance and subjective workload through controlled experimental conditions.
- Operational Value: Provides repeatable procedures for measuring transition time, error rates, and physiological responses to support consistent evaluation across design iterations.
Screening & Assay Development
- Scientific Value: Generates quantitative behavioral data and subjective feedback to assess design variables such as button size and layout curvature.
- Operational Value: Includes Python scripts for data analysis and visualization, enabling scalable processing of touchpoint coordinates and performance metrics.
Translational & Preclinical Research
- Scientific Value: Supports translational continuity by linking laboratory-based evaluation to real-world usability outcomes in smart device interaction.
- Operational Value: Adaptable to other touchscreen platforms like tablets, wearables, and VR devices, extending utility beyond smartphones.
Pipeline & Workflow Integration
The method fits within early discovery workflows by providing standardized usability testing that informs design optimization prior to large-scale deployment.
- Discovery Biology: Enables systematic interrogation of design hypotheses using objective metrics like transition time and word error rate, and subjective dimensions like workload and intent to use.
- Screening: Delivers assay-ready outputs through coordinated collection of touchpoint data, physiological signals, and questionnaire responses under balanced experimental conditions.
- Analytics: Produces measurable readouts including fitted ellipse area and offset, transition times, and error rates that allow quantitative comparison of keyboard layouts.
- Translational Research: Connects controlled lab evaluation to practical usability in smart devices, supporting iterative design refinement.
- Enterprise Reuse: Offers open-source tools and protocols that can be reused across device form factors and design iterations, reducing redundant development effort.
Operational & Enterprise Impact
- Scientific Value: Reduces mechanistic ambiguity in design evaluation by integrating multiple indicators of performance, comfort, and user preference.
- Operational Value: Ensures standardization through consent procedures, hand size measurement, device calibration, and controlled task ordering to minimize confounding.
- Strategic Value: Informs go/no-go design decisions by identifying trade-offs between input efficiency and subjective comfort, reducing risk of poor user adoption.
- Portfolio Impact: Supports risk-adjusted advancement by highlighting design trade-offs early, enabling prioritization of layouts with better user experience profiles.
Implementation Considerations
- Requires expertise in human-computer interaction study design, data collection, and statistical analysis of behavioral and physiological data.
- Needs instrumentation for motion capture, physiological monitoring (galvanic skin response, heart rate, EMG), and touchscreen data logging.
- Demands cross-team standardization in participant preparation, device hygiene, and task execution to ensure data reproducibility.
- Involves adaptation considerations when applying to different screen sizes, input modalities, or user populations beyond one-handed thumb typing.
- Limited to evaluating design variables within the scope of collected metrics; does not assess long-term ergonomic effects or muscle fatigue beyond session duration.
