PaymentsJournal
  • Analysts Coverage
  • Truth In Data
  • Podcasts
  • Videos
  • Industry Opinions
  • COVID-19
  • News
  • Events
No Result
View All Result
PaymentsJournal
  • Analysts Coverage
  • Truth In Data
  • Podcasts
  • Videos
  • Industry Opinions
  • COVID-19
  • News
  • Events
No Result
View All Result
PaymentsJournal
No Result
View All Result

Machines Can Now Read Your Emotions

George Pliev by George Pliev
December 18, 2018
in Artificial Intelligence, Industry Opinions
0
machine learning emotions

machine learning emotions

2
SHARES
0
VIEWS
Share on FacebookShare on TwitterShare on LinkedIn

Machine learning can be used for a multitude of tasks in the payments industry. Things like fraud detection and customer retention are great examples. Recently had the opportunity to talk with George Pliev the CEO of Neurodata Lab about how he and his team are using machine learning to interpret the emotional state of people.

Does the machine recognize mood with voice, facial features, text? 

We use multimodal approach to emotion recognition. We developed a cloud-based technology that can recognize emotions by analyzing human data from several channels, including voice, facial expressions, body movements, and psychophysiological parameters. Thus we work

To train a machine learning algorithm, for instance, a neural network, to recognize emotions, a dataset is needed – these are photos, videos or audio data in which people express different emotions. For instance, talk shows or specific research datasets like our RAMAS and Emotion Miner Data Corpus. To train algorithms we use data that has been previously analysed, ‘marked up’, by a large number of people. Based on what kind of emotion most people saw in a particular video fragment, the algorithm will be trained to detect this emotion in new videos that it has not seen before. Neural networks thus learn how to associate a particular set of emotion cues coming from different channels (face, voice, body) with a particular set of emotions and even cognitive states.

How accurate it is?

When analyzing audiovisual content, the algorithm can be some percent sure about an emotional expression. For instance, algorithm can predict with some probability that this person expresses 96% happiness, 3.6% surprise and 0.4% mix of other emotions. The more data the system analyzes, the better it gets at predictions.
In some cases the algorithm can give false predictions, but the potential number of those is tiny. At the same time, multimodal emotion recognition systems are more accurate than unimodal, meaning affective data coming from the face will be confirmed by that received from the voice.

Who is using this type of tech?

We collaborate with both big corporations (Rosbank, Société General Group, Microsoft, Samsung) and start-ups (Promobot, a robotics company with whom we are going to CES). They represent the industries where emotional analysis is of big interest and can be used for a number of solutions:

  • Natural interaction for human robotics, virtual assistants, chatbots
  • Predictive analytics for HR/recruitment
  • Customer Experience Management
  • In-cabin analytics of driver’s state and solutions for self-driving cars

At some point companies ran into the idea that the decision about a purchase is massively influenced not only by what consumers think about the product, but what they actually feel about it. Objective emotional analytics can be an invaluable tool. Neuromarketing instruments used for tiny focus groups were able to provide some cues about how customers feel about products. Emotion analytics open a new era for companies to recognize the emotions of each customer, right at the time of purchase.

Same with video recruitment – HR people want to have a full picture about a candidate, and emotion-related analytics can say a lot about a person. For instance, the style of speaking reflects valuable information about one’s personality. Studies have shown that nonverbal behavior is as important as the verbal response in job interviews.

Emotions have also been playing an increasingly important role in human-machine interactions. You might have heard about Vector the cute robot by Anki. It is a direct proof human robots are moving towards the Emotion AI. The same happens in service robotics – Emotion AI-enhanced robots are more attractive for people. Empathy is an important element of communication, even in human-to-human communication. If the interlocutors pay attention to the feelings and emotions of each other, the effectiveness of their communication increases by many times.

The second direction is about wellness and healthcare. Emotion recognition can help people with autism and impaired social skills understand true moods of the people around. At the same time, people can use emotion cues based for instance on the analysis of their physiology to manage stress and prevent the episodes of depression.

Emotion AI can also be applied in social and political contexts. Last week we finished a project with the Swiss biggest media holding — we analyzed the emotions in 800 videos of speeches of the Swiss parliament members. For each politician we then created an emotional profile. This may change the rules of the game in certain circumstances.

Where do you see this going in the next 5 years?

In business, Emotion AI-based systems will be used for constant, objective and correct evaluations of customer service. AI systems for different sort of analysis of employee work and customer experience will be spreading across the stores, customer service offices and call-centres. These systems will allow not only to analyze how many customers there were in the store at a certain day or what was their journey through the store, but automatically analyze their attitude towards the product/service, while simultaneously estimating how polite is the employee towards the customer. This will set a new standard in customer experience and employee performance management.

This is especially relevant for retail banking, where high quality customer service means success and revenue. No wonder that in professional CX communities and associations members are mostly represented by CX people from financial services.

Second obvious trend are human-computer interfaces. These are Emotion AI-enhanced robots, personal digital assistants, chatbots. If we delve further into possible applications of Emotion AI, we will come to medicine. It’s one thing when a device simply picks up, “understands” your mood and according to it switches on music, adjusts lights or makes coffee. The other is when it evaluates the degree of fatigue or determines any deviations from the norm by a respected type. Or a disease. In short, emotional technologies today and further will be in demand in biometrics and security systems, in robotics and the gaming industry, AR/VR, intelligent transport systems (unmanned vehicles).

Are there issues with different speech patterns or facial features in different parts of the world? 

Several decades ago Paul Ekman’s universal theory of emotions was very wide-spread. He thought that as far as facial expression is concerned, people display and recognize some emotions, which Ekman called ‘basic’, in universal ways. No matter where we are and whom we are talking to, we will always recognize when our interlocutor is expressing five emotions: anger, fear, disgust, happiness, sadness.

Today this theory is broadly criticized, for example by James Russell, Beatrice de Gelder, or most famously Lisa Barrett. They claimed that emotional expression differs from culture to culture, from person to person. To train AI-algorithms to correctly recognize emotions it is crucial to use specific affective data and take into account the culture, language, gender and even age when trying to determine what emotions are expressed.

Tags: AIMachine Learning
2
SHARES
0
VIEWS
Share on FacebookShare on TwitterShare on LinkedIn

    Analyst Coverage, Payments Data, and News Delivered Daily
    Sign up for the PaymentsJournal Newsletter to get exclusive insight and data from Mercator Advisory Group analysts and industry professionals.

    Must Reads

    eBay’s Upgraded Approach to Payment Processing Meets the Demands of Modern Consumers

    eBay’s Upgraded Approach to Payment Processing Meets the Demands of Modern Consumers

    January 22, 2021
    It’s Time for Retailers to Offer the Best Gift of All In-Store: Digital Gift Cards

    It’s Time for Retailers to Offer the Best Gift of All In-Store: Digital Gift Cards

    January 21, 2021
    Building C-Store Customer Loyalty Programs With Relevant Rewards

    Building C-Store Customer Loyalty Programs With Relevant Rewards

    January 20, 2021
    How PayPal Achieves High Authorization Rates

    How PayPal Achieves High Authorization Rates

    January 19, 2021
    Explaining the Bill Payment Ecosystem

    Explaining the Bill Payment Ecosystem

    January 15, 2021
    QSRs Can Address Loyalty Program Shortcomings by Serving Up Better Offers

    QSRs Can Address Loyalty Program Shortcomings by Serving Up Better Offers

    January 14, 2021
    How Merchants Can Prevent Account Takeovers—and Why Failing to Do So Amplifies Operational Expenses

    How Merchants Can Prevent Account Takeovers—and Why Failing to Do So Amplifies Operational Expenses

    January 13, 2021
    How Banks Can Leverage Tech Partnerships to Enable Innovation for Commercial Clients

    How Banks Can Leverage Tech Partnerships to Enable Innovation for Commercial Clients

    January 11, 2021

    Connect With Us

    • Advertise With Us
    • About Us
    • Terms of Use
    • Analysts Coverage
    • Truth In Data
    • Podcasts
    • Videos
    • Industry Opinions
    • COVID-19
    • News
    • Events

    © 2021 PaymentsJournal.com

    • Analysts Coverage
    • Truth In Data
    • Podcasts
    • Industry Opinions
    • Faster Payments
    • News
    • Jobs
    • Events
    No Result
    View All Result
    ×

    Login

    Forgotten Password?

    Lost your password?
    | Back to Login

      Subscribe!

      Thank you for visiting PaymentsJournal! Please subscribe to our newsletter to receive consumer data insights and daily analysis from Mercator analysts and industry experts.

      ×

      How will COVID-19 Effect the Payments Industry?

      Check out our latest:

      – Consumer Data – Complimentary Reports
      – Podcasts – Mercator Analyst Commentary
      – Industry Opinions

      ×

      WEBINAR:
      How Digital Acceleration Will Affect Payment Industry

      Please join us for this panel discussion on addressing the challenges to pave the way to payments innovation and profitability and gain insights on the key trends and challenges impacting the payments landscape in North America.

      REGISTER