Tracking Progress — Wearables in Review: Key Learnings & Observations

INITIAL UNBOXING

The first thing participants were asked to do after opening their devices and starting them up was to record their initial impressions.

They were asked to consider the tone of the packaging, the process of starting their device, connecting it to required software, how any associated applications handled account set up, and initial goal setting.

Based on this feedback, a number of trends emerged:

Startup is a chance to use personal, human touches to establish an immediate connection.

Given that these devices are designed to be with users all day, everyday, it’s not surprising that people responded positively to devices that expressed some sense of humanness from the outset. While this is most obviously expressed through small touches in language and visual design, it runs throughout the product experience.

In nearly every case, the expectation was that the devices should serve as a sort of digital personal trainer, introducing the user to both a new device and a new way of living. The Shine was the most standoffish of all the devices, and whether because of this, or simply a case of poor design, Shine users started off disconnected from their devices and never fully recovered.

  • “After (battery installation) there were ZERO instructions on what to do next. No tips on how to use the device, what to do with the device, or instruction to go get the app.”
  • “Once it syncs, there’s no great walkthrough that gets you excited. You just kind of see a white table and some activities.”
  • “As soon as I plugged it in, the face of the device showed icons making it clear that it wanted to talk to my phone. I went to Bluetooth settings, chose the device, and it walked me through app setup.”
  • “I liked the interconnectedness of the touch points – package / website / app / band”
  • “Eesh. Initial setup not the best. I feel like I’m a capable person generally but in the onboarding process … I felt like a monkey trying to set it up.”
  • “Setting up the app/device was fun, it felt like a game.”
  • “The site thanks me by name and I like that.”
  • “I hit power, and really liked the friendly lower-case ‘hello!’ that greeted me.”
Get going quickly – reveal deeper features later.

In every case, participants were excited to get started with their device and the way in which each device guided them through this process greatly affected their initial emotional response to it.

The ability to start new users off quickly with a minimal set of information, and then release new information as needed may be a good way to capitalize on initial excitement. In this way, simple game mechanics could be used to effectively “level up” users as they master basic behaviors.

For example, rather than having users understand every possible aspect of data collection a device might be capable of, or asking a new user to set a long- term goal as a first step, devices might be better served by simply asking the user to begin paying attention to how active they are each day, monitoring their daily step counts, and then propose a goal based on this data. Another approach might be to allow a user to establish a history within something they’re already good at — step counting for someone who is already a runner, for example — and then “leveling” them up into a more challenging activity like food tracking later.

  • “Skimming through the directions, there were times … I did feel overwhelmed with the volume of new information”
  • “I liked that the setup was an online walkthrough, and I didn’t have to refer back and forth between my computer and a paper manual.”
  • “Withings earned brownie points here for not making the instructions feel like an assigned textbook.”
  • “Super easy instructional booklet. Almost no words. Just good pictures.”
Be clear about why you’re asking for information and how it helps the user. Don’t ask for more than you’ve earned.

While goal setting is an important element to any sort of behavior change, how each app handles this is important to establishing a trusted relationship with the user. As mentioned in the observations around initial setup, users seem to view their device as something akin to a personal trainer. Perhaps because of this, many of the social norms we apply to people seem to apply in this context as well. Asking for personal information too soon, or without proper context, was off-putting and left people suspicious of intent.

Establishing a relationship first, clearly explaining why the device needs this information, and how it helps the user could go a long way to not only establishing trust, but also furthering a bond between the user and device.

  • “I felt pressured to make something up for the goal, not sure if I’m skewing my data?”
  • “In setting up my stats I had to admit that I’m a few years older and a few pounds heavier than they expected me to be.”
  • “The first thing the app had me do was add my height and weight. Then, it immediately asked me if I wanted to set a weight goal. A little presumptuous, no? I just want to be fit!”
  • “Didn’t give me shit for not wanting to login with Facebook.”

WEEKS 1 AND 2

For the first two weeks of research, participants were free to go about their days normally but asked to record any thoughts they had about their device — things they liked, frustrations, how the devices were helping or not helping them. We met as a group once each week to discuss how things were going, and compare experiences.

Based on their written thoughts and recordings of the group meetings, a number of trends began to emerge. I’ve grouped these into three main areas.

The first two are based on how participants felt the device helped or hurt their motivation, as well as how the device handled data tracking. The third area is focused on the design and user experience of the device and its associated software.

KEY OBSERVATIONS AROUND MOTIVATION

People come motivated and ready for change.

Each volunteer came into this project excited by the devices and ready to like them.

The immediate sense people conveyed over the first few days of our research was one of enthusiasm, curiosity, and a genuine interest in using the devices to modify their behavior. The biggest challenge — as we’ll see — is understanding how to maintain this enthusiasm over time

  • “I’m already paying more attention to how much my body is moving. I went on a run last night to see how close I could get to my goal on a short day.”
  • “I’ve been trying to come up with a way to examine my life with this device in a repeatable way that I can track over time.”
  • “I think the biggest impression the FuelBand has made on me thus far is keeping the question of ‘Why aren’t you being active right now?’ in the front of my mind.”
  • “But just as often I’m made conscious of unnecessary inactivity and am motivated to get moving. I’ve gone on two runs in the past four days. I walked 20 blocks each way to brunch on Saturday instead of driving. I didn’t drive the 4 blocks to my boyfriend’s house last night even though it was chilly and I was carrying things.”
Celebrate my victories

A key element to maintaining motivation is recognizing when users achieve their goals, both large and small.

Several devices did this, and it was universally appreciated. For devices that didn’t, it was an emotional let-down for users to have put forth additional effort towards these goals, only to have them go by unnoticed.

At the same time, there is a risk around being seen as being too forthcoming with applause. As much as users wanted their accomplishments recognized, they were also instinctually aware when they had done something worthy of recognition and when they hadn’t.

In instances where the device response matched the emotion of the user, rewards served to amplify the experience. In cases where the response seemed disproportionate to the accomplishment, users were left feeling talked down to and annoyed.

  • “The fun animations I see on my wrist when I meet goals and online when I unlock trophies or achievements are still neat. The LED display is still charming.”
  • “I finally took more than 10,000 steps in one day. I got nothing. No badge, No ‘way to go!’ Just the same ol’ charts I’ve been getting. It’s pretty anticlimactic.”
  • “I WISH I could have screen-capped the epic animation that happened on my Nike Fuel app the first time I met my daily goal”
  • “I went for my first run with the Pulse and passed my step goal for the first time today. Sort of anticlimactic. It didn’t do anything special for me. It felt like a good friend forgetting my birthday or something.”
  • “My activity trumps whatever badge they give me.”
  • “...encouraging words like ‘Champ’ — are more statements (for me) of how little Fitbit knows me. Certainly, I don’t expect anything else of the device. But I don't want to be a ‘Champ’.”
Grow with users

As important as goal setting is, the ability for devices to adapt these goals and grow with a user is equally important.

Although there was some confusion about how to go about setting an effective goal (discussed in more detail later), devices that bypassed this with a preset and fixed goal lost their appeal within the first two weeks as users either hit or missed this target and were left with nothing else to accomplish.

The scope of goals was also an issue. All devices included daily activity goals, but almost none of them allowed for more long-term goal setting. The exception to this was Fitbit and Withings, which allow users to set a weight loss goal and a date by which they’d like to reach their target weight. Of these two, only Fitbit actively adapted to the progress a user made toward their goal by adjusting a daily calorie allotment based on user activity.

The thread running through both initial goal setting and long-term goal tracking is effectively conveying to users that they have a history of progress, and more to be gained if they continue.

Reflecting this notion of both past and future is a core concept in game mechanics: not only have you have worked hard to achieve your goals, but if you keep playing you can get to a new level. What’s around the next corner? What’s the next thing I can earn? What’s the next mission? These same mechanics could be used in this space to help maintain motivation.

  • “With the Pulse, you can’t change your goal, and nothing happens when you meet it.”
  • “The inability to do anything with those goals has definitely made me feel more disconnected.”
  • “It’s been motivating in a small sense...I want to see what happens...”
  • “I was also somewhat disappointed to find that when I reach my goal for the day and tap my device it does a nice little all around light up as if to say ‘you did it!’ but what happens then? Do I set my goal higher to keep tracking how I’m doing today or do I just keep seeing that I reached my goal? Once I reach my goal I no longer need to tap the device that day which makes it way less useful or fun.”
  • “I’m bummed there’s no way to change my step goal. I feel a little less challenged because of this. 10,000 seems like a good goal but being someone who can easily be pushed further by a competitive element, it lumps me into the same category as every other person in the world. I want challenge!”
  • “So I think my goal is not something I will hit every day, and I think I will keep tweaking it until I find something that feels right.”

KEY OBSERVATIONS AROUND DATA TRACKING:

Don’t just report — tell a story

A major promise of health and wellness wearables is the ability to modify one’s lifestyle based on insights provided by previously unavailable data. Almost by definition, many of these users won’t have the inclination to try and piece together the underlying meaning of all this data. That’s why they’re using this product in the first place

So, while the most visible part of any health and wellness wearable is the hardware, the most critical point of a long-term connection to the user will be the point at which they access the story of their data — in most cases, the app. Articulating why certain data is being collected in the first place, and then adding a context to that data — not just what the data is, but but what a user should do with it — are all critical elements in establishing and maintaining that connection.

It’s unfortunate, then, that nearly all of the devices we tested were so inconsistent at doing just this. Data was presented in aesthetically pleasing graphs and charts, but the underlying narrative of the data proved nearly impossible for users to understand. This was most evident with sleep tracking. Users were frustrated by seemingly arbitrary targets for “hours slept,” with no information other than whether or not they hit that target.

  • “I would also feel better about the experience if they did a better job at instructing me at how I should interpret the data.”
  • “It’s really interesting to look at and has a couple cute animations - but I'm not sure what my take away from this is. I already know that I sleep every night and walk every day. I know that I eat at semi-regular intervals.”
  • “Throughout the app the data I’m presented with has no notion of relationship — and really, that’s why you put data into a graphic in the first place: to tease out relationships that aren’t visible otherwise.”
  • “Seeing data one day at a time prevents me from being able to see these patterns emerge, which feels like possibly the greatest possible failure for any device that’s designed to collect data over long periods of time.”
  • “I’m noticing that the accompanying app doesn’t do much to motivate me. It just takes the raw data and spits it out in simple, albeit pretty, charts and graphs.”
  • “It also doesn’t provide much in depth sleep info, just how many hours of sleep and how many of deep sleep. There isn’t much to investigate.”
  • “...it really doesn’t track much that is really useful. Walking is cool, sleep is cool, but outside of ‘you walked this many miles today’ and ‘you slept this many hours’ it kind of falls flat.”
  • “This thing is obsessed with me getting more than 8 hours of sleep. ‘That's perfect, let's try to sleep a little bit more next time.’ So.. is it perfect or do I need to sleep more?”
BE THE EXPERT

On the other side of the coin, instances where the devices took on the role of authority rather than simply reporter were appreciated by participants.

In these cases, helping the user understand exactly what they’d need to do to reach a specific goal moves the device from simply a data reporting tool to a partner helping the user meet an important goal for them on a human level. Devices shouldn’t expect users to know already how to achieve their goals, only that they’ll have them.

  • “I told Fitbit what I wanted to weigh by February and, based on my daily activity, it tells me how many calories I’m allowed to eat to get there. I love this feature and like I said, just this single thing will likely keep me with the Fitbit for a while.”
  • “I don’t like how the weight goal allows me to set both a target date and desired amount of weight change per week. It should give me a formula for how much weight should be gained/lost each week to meet my goal.”
  • “Should I intentionally set my goal lower for tomorrow if I know I won’t have time to exercise beyond my work commute? Or should I keep my goal more or less consistent across each day?”
Food logging is terrible

Issues of data reporting and narrative aside, the process of accessing activity data reasonably well executed.

The same cannot be said for nutrition tracking.

Of the devices we tested, only two companies — Fitbit and Jawbone — tracked nutrition. Between the two of them, only Fitbit seeks to do anything other than simply record data.

Although both attempt to speed up and simplify meal logging ( for example, the Jawbone UP app contains a barcode scanner) input screens remain generally cumbersome and time-consuming to use. Outside of pre-saved meals from major chain restaurants, and some large national food brands, users are expected to enter detailed information about the nutritional information of their meal as well as specific, often confusing insights into quantities (i.e. how many grams of waffle a user ate). In both devices, entering nutritional data in a quick, non-disruptive way was nearly impossible.

Both Fitbit and Jawbone get some credit for including food tracking at all. However, in the case of Jawbone, the result is not much more than a calorie-counting mechanism that isn’t tied to any sort of goal setting and it doesn't provide much insight as to what, if anything, the user should be doing differently. In the case of Fitbit, while they do include some meaningful integration of nutritional data to help users achieve their goals, the interface for maintaining an accurate food log is so poor that it’s hard to imagine many people maintaining it for more than a week or two.

  • “I was too lazy to search through the sh*tty food database that didn’t even have a grilled chicken sandwich in their library.”
  • “Things like logging food are just terrible, with an interface from like 1999 and no sense of things like appropriate units for food. Example: why are you asking me how many ounces of egg I had instead of how many eggs?”
  • “And food logging. I’m just going to leave it as: awful.”
  • “In general, they don’t have the foods I’m eating. They have things LIKE it, but not IT. Once I do find something in general ballpark (e.g. "Wine"), their measurements are strange. Who drinks a gram of wine?”

KEY OBSERVATIONS AROUND DESIGN AND UX

Simple and clear is better than complete and complex

Most participants preferred a simplified but easily understood presentation of data to more complete — but more complex — data reporting.

  • “I keep getting frustrated trying to read the charts. They are nicely laid out but for some reason difficult to interpret.”
  • “Telling my Mom I climbed 1,000 floors since I got my device is not as fun or interesting as telling her I climbed my way to skydiving elevation.”
  • “When we first started I thought the whole points concept was stupid...but the more I’ve done this I wish I had points so that I could just see overall how I’m doing that day. Like right now I have all these data points, but I’m not sure what I should be looking at. It’s like ‘okay so I slept eight hours, and I took 10,000 steps and I’ve done all these different things’ but how am I doing?”
  • “I like that a Fuel point is a Fuel point.”
The best interface is no interface. Let the device disappear in my life.

The devices that people enjoyed the most over the course of our research were those that required minimal interaction.

People were most satisfied with devices that synced wirelessly, didn’t draw attention to themselves, and were able to adjust themselves to different contexts without specific input from the user.

  • “I dislike how I have to physically plug the UP into the phone every time I want to sync the band.”
  • “After using the Pulse for a week or more, I am glad this thing sits on my belt. It’s something I truly forget about during the day and often times don’t remember I’m wearing until I change and get ready for bed.”
  • “The way to set it up to track my sleeping was really confusing at first. Then I figured out that I need to tap it three times to track my sleep. Well, nobody ever told me I need to tap it again in the morning to stop sleep tracking.”
  • “After reading some others’ experiences with their devices … I am excited that I got this band. I don’t have to log anything, I don't have to take it off to sync it... all I have to do is wear it. It’s totally unobtrusive”
  • “At first I felt mad at myself for forgetting. However, I quickly turned that frustration to the device. Why do I have to tell it when I am going to sleep? If it can tell when I wake up in the middle of the night, when I’m in deep sleep, and when I actually get up to go on with my day, why do I’have to tell it when I’m right about to hit the sack? I’m already in a sleepy state — I’m surprised it took me until now to forget to turn it on!”
Flourishes can get old in a hurry

Most products are faced with the challenge of adding unexpected and delightful touches to otherwise routine elements. The tipping point from design to decoration comes when these flourishes begin to mute the functionality they contain.

In the devices we tested, the line in the sand came between making required functionality special versus adding layers of affect on top something a user does many times over the course of a day. In this second case, the charm of unique interactions or animations wore off quickly. For devices that maintained their footing firmly in the former space, these bits of design flourish were a welcome bit of humanizing fun.

  • “I’m happy to find that it will sync automatically if I just wait a minute but their initial instruction is to physically tap the phone with the device. This makes me look super dumb in public.”
  • “The interface is a little primitive but there’s something charming about that. One button, a bunch of LEDs, keep tapping and you’ll get what you’re looking for.”
  • “After I got over the fun factor of tapping it everyone once in a while, I started to feel a little silly doing it.”

  • “The Pulse’s touchscreen is starting to get pretty annoying. It seems like it’s less responsive than when I first started wearing the device...The touchscreen is a pain point that makes me think why do you need the screen in the first place? Not sure I see any value in the screen after 1-2 wks. worth of wear”
  • “The LED display is still charming. I feel like I look sorta cool when I’m on a run waiting at an intersection and check my FuelPoints.”
  • “At one point I was kind of digging it. It gives me one more thing to do while I’m waiting for things or to make me seem like I do cool sh*t. So tapping it twice every once in a while was fun for the first few days.”

WEEK 3: INDIVIDUAL COMPETITION

After the enthusiasm of the first week had settled quickly into an increasingly bored (and occasionally frustrated) week two, we decided to shake things up.

For our third week, the team was asked to vote on a group activity: either a common shared goal the group could work towards as a team, or a group-wide individual competition.

They unanimously selected the competition.

Because there was little functional parity between our test devices, “steps” were settled on as the competition metric we’d use as it was the one area all devices could consistently track and report back. The competition consisted of four events and team members could participate in as many or as few as they wanted to.

The four events were:

  • Most steps from running
  • Most steps in a single day
  • Increase steps by 20% each day
  • The biggest improvement in steps from the second week

The Sunday before the event started, we sent all participants an email describing each of the challenges and informing them that the winner of each challenge would be getting a prize at our weekly meeting – trophies for the first three events, and $50 REI gift card to the winner of the Most Improved event.

INSIGHTS

A number of insights immediately emerged as the competition began.

It can be hard to get specific data in specific ways.

Faced with reporting data based on achieving specific goals, many team members quickly found shortcomings in their devices’ ability to show data formatted in useful ways.

  • “...the Shine gives you zero ability to slice up the day and know how many steps you took in a given time period.”
  • “The FuelBand doesn’t know if I’m running or not, so I’ve just been checking my steps right before and right after I run. I used the advanced technology of writing my steps on my arm”
  • “Fitbit doesn’t do a great job segmenting running data.”
Competition design can be used to bring people into the fold.

Competition can be used as a lever, turning previously low performance into an advantage. While competitions like “Most steps from running” clearly target people already predisposed to a specific sort of activity, other competitions like “Increase steps by 20% each day” turned out to dramatically favor those who had otherwise not been in the habit of logging many steps. Trent, who ended up winning that competition, began the week logging 7,400 steps on Monday, compared to “Most Steps From Running” winner Jenny who logged 17,000. Projecting these titles forward, Jenny would have needed 35,000 steps on Friday, while Trent needed 15,000. He ended up logging over 17,000. There was a similar effect with the “Biggest Improvement From Week Two” competition. In both cases, careful competition design allowed us to pull in a wider group of people into the fold than we might have with more traditional targets.

Having a specific target increased everyone’s activity.
Weeks two and three week over week growth in team step totals
Weeks two and three week over week growth in team step totals.

WEEK 4: GROUP COMPETITION

The fourth week featured a team-based competition. The main group was divided into 3 teams of four (two participants abstained), balancing the highest performers up to this point across each team.

The team competition featured a different format as well. Instead of specific events, challenges focused on the cumulative effort of all team members. Some of these required the team to do the activity together, others were based on the individual efforts of each team member. Every challenge was given a point value based on it’s relative difficulty, and the team with the highest point total at the end of the week was awarded a group lunch.

The Challenges were:

  • Have a different person lead your team in steps each day = 15 points
  • Highest team average = 15 point
  • First team to 60,000 steps = 15 points
  • First team to 120,000 steps = 15 points
  • Everyone on your team walks 500 steps together = 10 points (Can be won multiple times)
  • Biggest single day: 10 points
  • Have each person on your team increase their step total each day = 25 points
  • Have each person on your team have at least 7,000 steps each day = 30 points

INSIGHTS

The group competition increased activity levels overall, though not as much as the individual competition. Whether this is due to the nature of group vs. individual competition or simply the format of the challenges would require, among other things, a more consistent format. That said, the additional responsibility of being on a team correlated to a greater rate of participation than the individual competition.

Step performance of all participants week-over-week.
Step performance of all participants week-over-week.

“It was interesting … between the individual and the team challenges. The individual challenges were about like ‘...what do I want to do for myself?’ and the team thing it was sort of a guilt motivation and I am completely driven by guilt.”


Next Section