Facebook machine learning aims to modify faces, hands and... outfits




 

The latest research out of Facebook sets machine learning models to tasks that, to us, seem rather ordinary - but for a computer are still monstrously difficult. These projects aim to anonymize faces, improvise hand movements and - perhaps hardest of all - give credible fashion advice.

The research here was presented recently at the International Conference on Computer Vision, among a few dozen other papers from the company, which has invested heavily in AI research, computer vision in particular.

Modifying faces in motion is something we've all come to associate with "deepfakes" and other nefarious applications. But the Facebook team felt there was actually a potentially humanitarian application of the technology.

Deepfakes use a carefully cultivated understanding of the face's features and landmarks to map one person's expressions and movements onto a completely different face. The Facebook team used the same features and landmarks, but instead uses them to tweak the face just enough that it's no longer recognizable to facial recognition engines.

This could allow someone who, for whatever reason, wants to appear on video but not be recognized publicly to do so without something as clunky as a mask or completely fabricated face. Instead, they'd look a bit like themselves, but with slightly wider-set eyes, a thinner mouth, higher forehead and so on.

The system they created appears to work well, but would of course require some optimization before it can be deployed as a product. But one can imagine how useful such a thing might be, either for those at risk of retribution from political oppressors or more garden variety privacy preferences.

In virtual spaces it can be difficult to recognize someone at all - partly because of the lack of nonverbal cues we perceive constantly in real life. This next piece of research attempts to capture, catalog and reproduce these movements, or at least the ones we make with our hands.

It's a little funny to think about, but really there's not a lot of data on how exactly people move their hands when they talk. So the researchers recorded 50 full hours of pairs of people having ordinary conversations - or as ordinary as they could while suited up in high-end motion capture gear.

These (relatively) natural conversations, and the body and hand motions that went with them, were then ingested by the machine learning model; it learned to associate, for example, that when people said "back then" they'd point behind them, or when they said "all over the place," they'd make a sweeping gesture.

What might this be used for? More natural-seeming conversations in virtual environments, perhaps, but maybe also by animators who'd like to base the motions of their characters in real life without doing motion capture of their own. It turns out that the database Facebook put together is really like nothing else out there in scale or detail, which is valuable in and of itself.

Similarly unique, but arguably more frivolous, is this system meant to help you improve your outfit. If we're going to have smart mirrors, they ought to be able to make suggestions, right?


Fashion++ is a system that, having ingested a large library of images labeled with both the pieces worn (e.g. hat, scarf, skirt) and overall fashionability (obviously a subjective measure), can then look at a given outfit and suggest changes. Nothing major - it isn't that sophisticated - but rather small things like removing a layer or tucking in a shirt.

It's far from a digital fashion assistant, but the paper documents early success in making suggestions for outfits that real people found credible and perhaps even a good idea. That's pretty impressive, given how complex this problem proves to be when you really consider it, and how ill-defined "fashionable" really is.

Facebook's ICCV research shows that the company and its researchers are looking fairly broadly at the question of what computer vision can accomplish. It's always nice to detect faces in a photo faster or more accurately, or infer location from the objects in a room, but clearly there are many more obscure or surprising aspects of digital life that could be improved with a little visual intelligence. You can check out the rest of the papers here.

COMMENTS

More Related News

NOT REAL NEWS: A look at what didn
NOT REAL NEWS: A look at what didn't happen this week
  • US
  • 2019-12-06 19:23:45Z

CLAIM: Democratic presidential candidate Sen. Bernie Sanders has been quietly closing offices in South Carolina as a sign his campaign is ending. THE FACTS: Sanders is not closing his campaign offices in South Carolina. In fact, Sanders has nine offices in the state and the campaign is expanding locations, Sarah Ford, a campaign spokeswoman for the Vermont senator, told The Associated Press in an email.

'Not a dry eye': 5-year-old Michigan boy's kindergarten classmates turn out to witness his adoption

Michael Clark Jr., 5, was adopted by David Eaton and Andrea Melvin in front of an audience of his kindergarten classmates in a Michigan courtroom.

Unfounded rumors about suspicious white vans are going viral on Facebook and putting drivers
Unfounded rumors about suspicious white vans are going viral on Facebook and putting drivers' lives at risk
  • US
  • 2019-12-05 19:27:45Z

Multiple people who drive white vans have spoken out to say they've been harassed because of the posts, and one driver was even killed.

Video captures the moment an Amazon delivery driver cries out with joy and breaks into dance after discovering free snacks on a doorstep
Video captures the moment an Amazon delivery driver cries out with joy and breaks into dance after discovering free snacks on a doorstep

Mail carriers spend long hours delivering anywhere from 150 to 400 packages daily during the peak holiday season.

EU agrees tough line on digital currencies like Facebook
EU agrees tough line on digital currencies like Facebook's Libra

Private digital currencies like Facebook's Libra should not be allowed in the European Union until the risks they could pose are clearly addressed, EU finance ministers agreed on Thursday. The move confirms the bloc's tough line on Libra, which has attracted criticism from global regulators over its possible impact on the financial system since it was announced last June. "No global stablecoin arrangement should begin operation in the European Union until the legal, regulatory and oversight challenges and risks have been adequately identified and addressed," the ministers said in a joint statement.

Leave a Comment

Your email address will not be published. Required fields are marked with *

Cancel reply

Comments

Top News: Economy