Rasa dialog trees without the trees

Cover Image

Introduction

Rasa is an open-source chatbot framework to create and automate conversational applications. It provides several features to help you efficiently develop your bot. You can generate your training data, train your model, run tests, and connect your bot to APIs and different messaging channels.

How does it work?

The Rasa framework contains two libraries, Rasa Core and Rasa NLU.

Rasa NLU performs the natural language understanding of the framework. It can be used to interpret and classify intents (user statements of intent). It can also extract “named entities” based on machine learning techniques to make the bot able to understand the user inputs.

Rasa Core manages the conversation state using the following parameters to execute the action needed and send a message to the user:

  • Tracker is used to keep track of the conversation history and save it in memory.
  • Policy is the decision-maker of Rasa, it decides on what action the bot should take at every step in the conversation.
  • Featurizer with the help of tracker, it generates the vector representation of the current conversation state so the action can pass later a tracker instance after every interaction with the user.

Even though Rasa can participate in a dialog, it just doesn’t use dialog trees. The chart below shows an example of a real chatbot project in progress. This bot focuses on helping people tackle their depression thoughts and boost their mental health.

Rasa depends on one of the most interesting concepts in computer science called a FSM (Finite State Machine) which you can use to define how your software works. Here is are the basics of how a FSM works:

  • It is a computational model that consists of one or more states.
  • There can be only in a single active state at any given time.
  • An input to the machine causes a transition from the current state to the next state.
  • Each state defines the rules that determine which state to switch to based on the inputs to the machine.

diagram tree example

The diagram above shows an example of an FSM. Here, shapes represent the machine (bot) states, this machine starts with an initiator or start state called OPEN and ends with a terminator or exit state called END.

The Get to know some depression statistics is an example of inputs a user can enter to change the current state. The user input defines the state to switch to and the machine switches to the next state before processing the next user input. However, it is also possible to create a state transition back to the same state. In this case the user input will keep the machine running in the same state.

light switching diagram

Let’s take an example of a light bulb. As seen above, this object has two states, it’s either turned on or turned off at any given time. If we turn on the bulb, this can cause a transition from turned off to turned on and vice versa. ON defines the state to switch to which is OFF, and OFF defines the state to switch to which is ON. In other words, if the current state is ON, clicking on ON changes nothing, but pushing OFF changes the state to OFF. If we are in state OFF, pressing OFF keeps the machine in the same state, but pushing ON makes a transition to state ON.

Conversation flow

In Rasa, stories represent and manage the conversation flow between the user and the bot. Here, the user inputs are defined as intents which are just the identifiers you defined in the training data. The bot responses are defined as actions. In other words, stories define the dialog tree. The dialog tree is actually a graph – connections between bot states. A FSM and a dialog tree are just a special kind of graph. Each state is something the bot will do or say. The edges or transitions between states are the actions or statements the user makes. So, the stories create the design of the conversation that determines the steps of the entire interaction.

NLU and Intent classification

NLU can define the training data and the example utterances to categorize what the user wants to achieve during an interaction with a bot. Another of using NLU module is to identify and extract the structured information from user inputs. The overall meaning of the user input is called its intent. The entities are small pieces of information, usually a word or two, that the bot extracts from the user statement.

Rasa gives you the freedom to define the NLU training data by classifying the user utterances into categories that represent intents. Those components are the skeleton of NLU to describe what users might say in each category, you can give them any name you want but it is preferable for each intent to have a name that matches the goal users like to accomplish with that intent.

Entity recognition

The term ENTITIES belongs to a big area of research in NLP named NER which stands for Named Entity Recognition. Researchers and professors use the academic term “entities” to refer to a particular object in the world (person, animal, place, thing, and even concept).

In Rasa, entities are structured pieces of information that can be extracted from user messages, it can hold important detail about the user like numbers, dates, names so that the bot could use them later in the conversation. Let’s take an example for a flight booking, it would be very useful for your bot to know which detail in user input refers to a destination.

Also, there are other ways entities can be extracted in Rasa:

  • Synonyms can be used in case users may refer to the information the bot wants to extract in multiple ways, so the bot can map the extracted entities to a single value.

  • Lookup tables used to extract words users may say that refer to the entities you set in your training examples.

  • Regular expressions help you to assign a pattern to an entity name to improve the extraction.

Entities can be extracted with regular expressions or other NLP algorithms like neural networks and word vectors. Regular expressions are much faster but more brittle than more sophisticated NLP algorithms that use neural networks to extract entities and recognize intent.

Note that before you decide whether you should use entities or not, it’s preferable to think about what information the bot needs for the user goals.

Rasa policies

Policies in Rasa are the decision-makers in every step of a conversation, each policy predicts the action with a particular confidence level. A policy with a high level of confidence is the one that decides on the action needed for the next step of a conversation, and the policy priority will be considered in case of the same confidence levels.

There are different machine learning policies and each policy has its own way to make a decision:

  • TED policy is a transform embedding architecture that performs next action prediction and entity recognition tasks based on transformer encoders.

  • UnexpecTED intent policy is an auxiliary policy that enables the assistant to review the conversation so that can handle properly the unexpected user intents.

  • Memoization policy compares the current conversation to the available stories in the training data. If there is a matching, the next action will be predicted with a confidence level of 1.

  • Augmented memoization policy learns from stories in the training as Memoization policy does. The only difference is the forgetting mechanism that helps Augmented memoization find a match by reducing the conversation history.

On the other hand, rule-based policies are fixed behavior policies that tackle conversations and make predictions based on the scenarios you trained them. Also, they can be used in tandem with machine learning policies.

Useful features

Rasa has some other useful features that can help to manage the conversation state and improve the bot performance. For example, Tracker Store keeps track of your conversation history as you interact with your bot, the conversation will be stored in the memory or in the SQL database.

Another feature that can be important for evaluating your bot is called Markers. This feature is used to express the important events in dialogs and mark them when they occurred. It helps to evaluate the bot performance based on how those specific events play out in a conversation. In other words, markers are kind of conditions that allow you to mark the points of interest in your dialog to evaluate your bot. As some code examples highlighted in this blog, you can create an independent story with Markers and you can set up a Tracker Store along with to keep track of the conversation and export the Markers.

Common problems

However, some features in Rasa can cause some redundancy and confusion while improving the bot performance. For example, the entity extraction feature is useful for filling forms or getting information from the user, but the feature is not designed to make if and else decisions to create forks in the dialog tree.

Also, the lookup table doesn’t work properly when the user puts in a truly “unseen” item, an item that you didn’t put into the lookup table. On the other hand, intents in Rasa are callable at all times and the checkpoints that are used to reduce the repetitivity can make the conversation flow hard to understand and make the bot miss everything. In addition, the unfriendly documentation can be very difficult for newbies to go along with and even hard sometimes for developers to understand.

Conclusion

To conclude, this blog post introduced Rasa which provides a platform to create and maintain conversational chatbots. We discussed how Rasa operates in the background, and then we demonstrated how this framework designs the dialog without any need for dialog trees. The blog highlighted some strengths and weaknesses Rasa has in terms of intent classification and entity extraction with a particular focus on some features that might have some future customization to improve the framework performance.

Hopefully, this blog post meets your expectation. If you have any questions, feel free to connect and hit me up at any time on LinkedIn.

See you in the next blog and keep moving forward!

References

Rasa open source documentation
Finite State Machines – Massachusetts Institute of Technology
Named Entity Recognition – Natural Language Processing of Semitic Languages
Markers in Rasa Open Source

Risks and Rewards of Open-Source

Maybe you’re a startup seeking funding from investors that value “IP”. Maybe you have an idea for a cool new killer app to save the world. Should open source software be a part of your business plan?

You’re worried about competitors using our software to outcompete you in the marketplace. You’re worried about investors asking you to “build a moat” around your business.

I hear you. I’ve been there … literally.

I’ve been a data scientist, CTO, and cofounder at more than 7 startups in the past 15 years. Most started out with significant funding. Some hired a great CTO (me) to help steer the ship. Some were spinoffs backed by the likes of Intel and Sharp Labs. All but one of those have since dissolved or stagnated. None have gone public or receive 10s of millions in funding. It’s not that interesting that I’ve been unlucky with the imbalanced dice that determines a startup’s fate. 1

What is interesting is the post mortem.In fact, I can almost do a premortem on any company I talk to now. I really only need to ask one question of the founders.

Are you 100% focused on creating a product that maximizes value for your customers?

Of course, almost all founders will say “Yes! Absolutely!”So the real question is, can they prove it to you. And there’s a good way to find out. So the real question to ask them (or yourself) is

If you discovered something free, something that would improve the customer experience, your operations efficiency, adoption rate, security, and trustworthiness of your product, would you do it?

Again, most founders will still say yes.No founder will ever leave money on the table. But if they don’t ask you for an example, then they’ve failed your test. If they do ask for an example, that’s your chance to dig deeper into their product and see if maybe open source could give their product and their business a boost.

Do you think open source would improve your customer value?

Most people agree that open source definitely improves the efficiency of an operation.No more NDAs, no more lawyers, no more patents and patent trolls harassing your business. And no more private repos without vital productivity-enhancing features like CI-CD pipelines or issue trackers or project management tools. You may not be aware, though, that it will also improve the efficiency and efficacy of your recruiting. Your most valuable employees will likely come from a vetted pool of contributors to your open source project. And developers will flock to you even when all you can offer them is some equity and a brilliant idea. A battle-hardened developer knows that if he works for a closed-source startup that goes bust, he (and the early customers) walk away empty-handed.

In all but one of the startups in my past, I failed to convince the founders to open source their core technology. The only one that is still growing exponentially is the one fanatically dedicated to open source — Tangible AI. It’s still early yet (19 months in), but I like our chances.

Here are some other recent success stories for open source along with their current age and funding.

More interested in the Unicorns? Though most of these serve developers, so open source is clearly the right choice, they are still worth mentioning (it’s no wonder your developers love to work for open source companies). And as a data scientist doing linear regression in my head on this data, I can’t help but notice the correlation between openness and growth rate. 2 Gitlab even open sources their business process documentation and product development meeting minutes.

Execution, execution, execution

For retail, it’s location. For software, it’s execution. Nobody cares about your idea, not your investors, not your customers. All they care about is can you make their lives better. And if you can’t execute quickly and efficiently on your idea, it’s doesn’t matter to anyone (except your competitors who will run with it and succeed where you failed).

And as any manager knows communication is key when it comes to building a performant team that builds killer apps. Users become contributors to your feature selection and quality control processes. Their feedback matters much more than what your UX designer’s user interviews and focus groups say. Your users are using your product in the real world. You can’t get better feedback than organic, grassroots feedback. Your main struggle with your developers is just getting them to ignore their own personal use cases, so they will build a product for your users instead of for themselves.

For a closed source company, any time or energy your team spends protecting your idea is wasted the moment someone outside of your organization comes up with a better idea or approach or feature. Your relationship with your users is upside down. Your users’ ideas are a threat to your success in a closed source company. They are more likely to share it with their developer friends than to share it with the company that is charging them for a product that they just thought up a way to improve.

If you decide you want to have a monopoly, a closed-source foundation for your company, you will have to be the very best at serving your customers. That’s hard when you and your teammates are all alone in a vast sea of ideas and tech swirling around you. It’s much easier to be the best when you harvest the wisdom and ideas from your users with an open-source foundation. Later, once you are a roaring success, you can start thinking of ways to upsell or monetize your users. There’s very little risk to starting out as open source. You can always button things up on those features you think are going to be most profitable.

But my hope is, you will never want to turn back. Once you’ve experienced the power of an open-source community behind you, you will never see business the same way again.

Footnotes


  1. https://review42.com/resources/what-percentage-of-startups-fail/ “90% of startups fail” ↩︎
  2. https://www.crunchbase.com/discover/organization.companies/80aa15a6fd10845452fff1c9419a07c8 “open source on Crunchbase” ↩︎

Pro AI

Teaching prosocial instincts to AI systems and the engineers that build them.