Application Development for Virtual Voice Assistant

By 5 June 2020 June 8th, 2020 No Comments

Application Development for Virtual Voice Assistant

Sharing is caring!

Application Development for Virtual Voice Assistant

Gone are the days when we used only UI based applications, this is the era of talking to electronic devices. The ridiculously improved technology and research in the field of artificial intelligence (AI) gave birth to a new breed of intelligent systems called smart virtual assistants; the machines which can understand as you speak (Amazon Alexa can understand languages other than English), and follow your commands without you instructing it with a mouse, keyboard or other device. This machine not only remembers your preferences but also learns based on your behavior. A few popular examples of smart virtual assistants are Amazon Alexa, Google Home, Apple Siri, Microsoft Cortana, and Samsung Bixby.

Our Experience

More than 80% of users find it friendly and more convenient to speak with a virtual agent. There is no doubt virtual assistants are helping organizations of all sizes to provide better solutions to their clients and employees alike to quickly achieve their goals.

We have summarized our experience into three best practices to consider when developing a skill for Alexa. This applies to application development for any virtual assistant, not just Alexa.

  1. Secure Interaction: Security is the highest priority, doesn’t matter how large or small a system or organization is. Especially if your client is from banking and finance industry, you need to consider the best security in every aspect of a system design, because of its substantial operational risk factors. As per a study, virtual assistants are an integral part of every system which belongs to FSIs. An application designed for voice assistants have unique security requirements due to the exposed nature of the interactions. The first security measure for a voice based system is to use an account linking mechanism which enables users to pre-configure an account associated the device. There are certain chances of having a devices associated with multiple accounts, which are then identified by a vocal pattern. These configurations can be done online or through an accompanying mobile app which some degree of authentication to verify the identity of the user. Another option is to implement a multifactor authentication, usually using a mobile device within user’s reach. Meanwhile skill on virtual assistant will be paused and waiting for response from the authentication process. It will resume interaction once received a notification of a successful authentication. Alternatively, users can be sent an OTP which they can speak into virtual assistant device but this approach should be used with situation where risk is small, or your are sure that user is in an isolated environment such as a closed door office. Additionally, a session can be maintained for a limited time to eliminate the need to speak the password into device for every request.
  1. Add a Personal Touch: What is a smart virtual assistant if it cannot emulate a human like interaction using natural language. A key factor in these interactions to include an element of personalization. Don’t underestimate a personalized experience, it can help validate the context and support a more efficient experience. Just like the personalized salutations used in emails to convey to the clients that the communication is authentic, a voice assistant can respond to its user by their name. Our solution for the finance company involving the Alexa skill development made usage of the account linkage option to identify its user and welcome him / her.
  1. Let it be Concise: Users like a voice assistant and more likely to use it when their interaction with it is quick and simple. If you are planning to a solution with a voice assistant, avoid the usage of verbose response and complex use cases. If any use case is complex, it should be broken down in to smaller pieces. Even though a single data element is delivered at a time to avoid verbose and confusing responses, users shouldn’t be bothered with too many interactions back and forth. Note that to achieve the speed and conciseness of the response, its accuracy should never be compromised. When the result set is limited to no more than three items, those items are listed for the user to choose from. This model not only presents the best opportunity to fulfill the request, but also makes the interaction simple and facilitates the retrieval of additional information.

Common Challenges Faced

  • Providing Flexibility: Using a traditional input method like a mouse, keyboard or touch doesn’t seems much challenging to us anymore as we have developed hundreds of application using these input methods. But using voice as a input comes with new challenges of its own. As an experiment, we designed the skill for our finance client in such a flexible way that their employees can ask Alexa for the priorities while still busy with their other priorities. A part of this flexible behavior is in-built in a voice assistant as it uses Natural Language Processing (NLP), it means a voice assistant is capable enough to understand and accept requests spoken in the way human instinctively speak. Some people like to give a voice assistant all information at once, while some prefers to let it prompt them for what is further required. An example for BFSI clients would be, “Alexa, fetch me the status of Mr. Lembart’s loan application”. In this case, all the information required to fulfill the request is provided as part of the skill invocation. But it could have been divided into multiple steps. So, providing this level of flexibility without causing any confusion to the user was quite a challenge.
  • Keeping the conversation flowing: Having a human way of dialogue with a machine is quite an enjoying experience, but correct way of its implementation is one of the biggest challenge we faced during the skill development for our research project for banking app. We learned that you need to ensure that the details given by user are retained and used throughout the entirety of the dialogue as necessary. Users shouldn’t be asked to repeat the information they have already provided. Our team come up with a solution to let the skill prompt the user for their PIN only if they haven’t provided it, otherwise they can access all the secure services without being prompted again and again for the PIN for a certain time period.


Having a smart assistant which can understand you speak is obviously an asset to your team and company. It’s going to only improve your productivity by letting it do things quickly for you instead of you fighting with computer using traditional ways. Cloud based and AI enabled processing power, helps you perform tasks that would otherwise be assigned to basic software which might not be automated. Voice assistants offers you seamless day-to-day business operations which were constantly supervised, be it access to reports, system updates, or data analysis.

The Road Ahead:

There’s no doubt the increasing popularity of voice assistants not opens new doors of opportunities for organizations to interact with their customers and employees but also give new business challenges for the solution providers like us. The excitement around this technology helps showcase the companies that have voice assistant skill as innovators and pioneers in their field. The ubiquitous nature of these devices provides us the opportunities to take advantage of the unique nature of particular environment (home, office, or even a car).

Get started with Bloom