Home

Imagine you’re sitting down to eat, but before you take a bite, you whip out your smartphone, fire up a special app, and take a picture of your food. The app identifies the food types by appearance, then calculates the size of your portions, and estimates your intake of calories, carbs, protein, vitamin, mineral, sugar, salt, and so on. Later you can review your data in a variety of ways. You can see your calorie intake for the day, or compare yourself to other people who are your same age, size, activity level, and so on.

At the end of a meal, if you have some food left, you can snap another picture so the app can calculate the net of what you actually ate.

If it seems impossible that an app could recognize food types, consider that software can already recognize faces, voices, specific songs, and fingerprints. Recognizing broccoli can’t be that much harder. And anything that has a label or a wrapper, such as Diet Coke or a Snickers candy bar, would be relatively easy for the app to identify.

Soups and casseroles would be harder to identify and analyze. The app might ask you to supply some information on the main components of the dish.  If you said it was a casserole with potato, chicken, and garlic, the app would know that garlic is a minor ingredient and potato is the main ingredient. It might even look at similar recipes in its database and take an average.

The app would not be perfect at estimating, even with your frequent tweaks. But it would be far better than your own guessing.  And it would be great at telling you where your diet is lacking. You might think you have a good diet, only to discover that you aren’t getting enough variety of fruits and veggies.

Now imagine that an accessory for this app is a small waterproof motion detector that you can clip to your footwear. It comes with a watch that also has motion detection. When your smartphone is nearby, the two motion detectors wirelessly download how much movement your arms and legs have experienced that day.  That would be a rough proxy for exercise. You would have to add any data for weight training because that doesn’t require much movement.

Now your app has your total nutrition and exercise profile. You could round out its knowledge by telling it your age, weight, gender, whether you smoke, and other relevant health questions. From that point on your app could predict your life expectancy and even your odds of dying from specific types of preventable diseases. Perhaps your watch could display both the current time and how many days you have left if you keep living the way you are.

Two factors that most influence human behavior are the ability to measure progress and the framework used to rank performance. This app solves both problems. Allow me to expand on this.

I’ve noticed that losers compare themselves to the average of other people, whereas winners compare themselves to their own natural potential. The loser can find comfort in knowing there are plenty of other slackers, and he is average (good enough) among them. The winner compares his progress to his personal potential and doesn’t stop until he achieves it.

Researchers have found that simply being near overweight people has a large influence on your own weight. This is probably a result of looking around and deciding that eating a little extra is normal, and good enough. The app I described would change your point of reference by continually reinforcing your own potential.  In time, your frame of reference would be less about your chubby friends and more about how you are doing compared to your own best, as measured by your app.

In your opinion, this app is…

1.       Inevitable?

2.       Already available?

3.       Impossible?

4.       Impractical?

 

Update: Someone is already working on food identification.

 

 

 

 
Rank Up Rank Down Votes:  +83
  • Print
  • Share
  • Share:

Comments

Sort By:
Nov 18, 2010
While someone else was working on it, it seems the Japanese did it.

http://www.stuff.co.nz/technology/digital-living/4347472/Japan-mobile-phone-app-takes-photos-counts-calories
 
 
Oct 2, 2010
My mom has an App on her smart-phone that is similar to the one you described. You can take pictures of items, and it can recognize a wide variety of objects fairly quickly. It also recognizes labels, and anything with a bar code. Not very much of a stretch.
 
 
Sep 30, 2010
That food recognition software by CMU is pretty cool. But I'm going to start a restaurant that serves junk food disguised as healthy vegetables. Actually, a reverse form of this is already available in some Buddhist vegetarian restaurants, where for example tofo is made to look like chicken.
But on the food and exercise app, I prefer (at least I prefer to think) that people learn about the health benefits of different foods and make these calculations without using a device. Call me old fashioned.
 
 
Sep 29, 2010
apparently cannot post website--anyway, many different websites have calorie consumption/exercise tracking to keep up with your diet and exercise habits and allow you to make adjustments according to your goals (lose fat, gain muscle, maintain current weight) and activity levels.
 
 
Sep 29, 2010
Not a current app for a smart phone, but check out this website--has many of the options but without the convenience of automation: !$%*!$%*!$%*!$%*!$%*!$%*!$%*!$%*!$%*!$%*!$%*!$%*!$% many websites like this already available, I see your idea as probable--although the app may just require one to enter their exercise information separately instead of having an automation feature. Perhaps it could be hooked up or synced to a GPS tracker like the Garmin ForeRunner (sp?) to download the pertinent exercise data instead?
 
 
+3 Rank Up Rank Down
Sep 29, 2010
Measuring heart-rate may be a more effective measure of exercise taken, and will work equally well for high-resistance excercise (weights) as low (running) without needing to enter any compensating information by hand.

This will also automatically allow for weight and fitness (fat and/or unfit people will get higher heart rates for the same exercise, but will burn off more calories doing so), once basic information like height and gender have been entered.

The heart-rate monitor could be built into a watch, which would download information to your smartphone by bluetooth, either continually if they are close by, or if not the watch could store the info and download it to the smartphone when they next meet.

This is all todays technology - in fact I'm surprised someone isn't already selling an HRM to work with the iPhone. Maybe they are. In fact I think I'll spend the next half-hour going to look for one...
 
 
Sep 28, 2010
Identifying the food when not in discrete lumps will be hard. Sure identifying broccoli may be easy, but soup?

Size of portions is trickier. For large chunk food you have to estimate the size of a lumpy pile, and the size of the air spaces between. For a bowl of soup, you have to estimate the volume of the bowl. Mind you when doing this at home, you can calibrate the bowls, and it only has to recognize that this is the Corelle bowl. How thick is that pizza? What fraction bread, veggies, meat, cheese?

Calories are even trickier. Is the white sauce on pasta just a simple flour and butter and spices sauce, or a rich alfredo sauce? How much oil in the pesto? Is the chicken al-la-King made with lean breast meat, or greasy chicken thighs? And what is the chicken to noodle ratio? How much butter is on the cauliflower? Is that cheese cake made with cream cheese whipping cream, or is it cottage cheese and non-fat sour cream?

The exercise sensor is easier. If you are willing to attach sensors, then I bet a throat mike could integrate breathing sounds to figure out how much exercise you got during the day. The same mike could transcribe all your conversations for the day too.
 
 
Sep 28, 2010
I think the idea is intriguing, but ultimately wouldn't be very effective. Here's why:

If you're taking a picture of the food in front of you, then you've already made the food. If you're in a restaurant, you've already ordered the food and paid for it. Either way, a commitment bias would persuade you to justify eating the food. It seems much more efficient to plug these sort of indicators in to a program before committing yourself to the food; no picture is required for this, and many apps out there already give you the option of doing this.
 
 
Sep 28, 2010
I think that this sort of product is feasible, but I don't see it having much of an impact. The only people who would use it regularly are the control-freak types who probably don't need it for weight-loss purposes anyway because they've already got that controlled. Modifying human behavior is so much more complicated than simply providing them with information. People overeat for a plethora of reasons - stress being one of them. Overeating in response to stress is a mild addiction actually. Eating releases endorphins which soothe the individual, making their daily stress easier to cope with. Morbidly obese folks fall into this same cycle only on a larger scale.

If you don't treat the underlying issues that are causing the overeating and lack of exercise, it doesn't matter how much, or the quality of information an app could give you to turn the tide. People would just ignore their app like they already ignore their doctor's advice, their diets, and whatever well-meaning resources they've already tried to tap into.

So I suppose I'd rate it #4 as impractical because it is the fancy schmancy equivalent of a diet - which we all know come and go with the breeze. In that sense, one could probably sell a gajillion of them, because people will buy anything that promises to make them skinny, but would it change our lives? - absolutely not.
 
 
Sep 28, 2010
<sarcasm>How about a mobile phone app that snaps pictures of birds, uses advanced organic 3D object recognition and image de-mapping algorithms (that don't currently exist) to match the picture against the million species of birds (a library of which in digital format doesn't exist), and then displays hotkeys that play birdsong mp3s correlating to food, mating, child-rearing, and other communication for the particular species (a library of which also doesn't exist). Such an amazing app would appeal to a several ocd bird-fans worldwide, would only cost a half billion dollars to produce, and would help spur more powerful mobile devices (which would need to exist to house libraries of data and perform special-function 3D processing algorithms). I can come up with a thousand brilliant ideas just like this in the span of a few minutes.</sarcasm>
 
 
+1 Rank Up Rank Down
Sep 28, 2010
Speaking as someone who has had compulsive tendencies, this is a terrible idea.

Jeez, just live life. This obsession with counting everything is not healthy, and in my opinion outweighs any gain from caloric reductions.
 
 
Sep 28, 2010
There's an iPhone app called Nutrition Menu that already does much of what you're asking for (not the picture stuff - that's 20 years out - sorry to be the bearer of bad news), and combined with the Nike widget/app, you can get most of what you want from an iPhone today.
 
 
Sep 28, 2010
5. Awesome!
 
 
Sep 28, 2010
I gotta go with impossible, as described. The problem is with the photo. If you could add something to actually figure out what's in the food, it moves to impractical (as in, "available as an expensive diet aid but not used by the general public"). You can't tell the difference from a photo if real sugar or a sugar substitute was used and the caloric difference that makes is huge. That's just a single example of similar looking foods having large differences in caloric impact.
 
 
Sep 28, 2010
Impractical.... the people who would need this the most would be too lazy to use it.
 
 
Sep 28, 2010
Steve: "The new app is great, it is such a time and money saver, watch this."

App: "Let's see, Steve, you ate a Big Mac with large fries and a coke for lunch, and you just exercised 4.2 minutes. You know, Steve, if you continued to have sex for another 3.5 hours, you could burn off the calories for your entire lunch.

Steve's Wife: "If he could last 3.5 hours we could save money by firing the pool boy?"

Steve: "Huh?"
 
 
Sep 28, 2010
why does everyone seem to expect that people only ever eat out or by pre packaged/ tinned foods? some of us take pride in and enjoy eating home prepared/cooked food.
 
 
Sep 27, 2010
Of course we could just switch to those futuristic food pills that they used to predict or Soylent (Red, Green, Orange, Yellow) where everything is standard and there will be no guessing. Make all that food legislation that has been passed be pointless, we can't eat anything "bad"
 
 
-1 Rank Up Rank Down
Sep 27, 2010
Inevitable
 
 
+2 Rank Up Rank Down
Sep 27, 2010
Inevitable, but really only effective when we get cheap chemical 'sniffers' attached to our e-devices. Inevitable, but ultimately pointless, really. There is no great secret to not being overweight, for those educated enough to realise that much of what is sold as food isn't.
 
 
 
Get the new Dilbert app!
Old Dilbert Blog