SizerBot is here to assist the Sizer team is answering questions that are of repetitive nature. This guide breaks down our approach for Phase 1 and our plan to continuously build on feedback generated.
Types of questions on the Sizer Channel
We classify questions on Sizer into two specific buckets:
- The first set of questions typically hold answers based on unique configurations and combinations of sizing
- The second set of questions are more generic such that its answers can be found through the wiki
What questions does the bot address?
Extensive research within the fields of ‘conversational AI’ are underway to mine through the first set of questions. This requires larger sets of data as well as the ability to interpret the context of questions, which we are currently exploring for the next few phases of the bot.
For the first phase we are targeting a small subset of repetitive questions that we are building on using content from this channel as well as from the Sizer product managers.
How does the bot work?
Based on a list of questions stored, we use a matching algorithm to identify if a similar question, as asked on the slack channel, exists in the database. While we can get the bot to provide a response to every question asked on the slack channel, we are avoiding this by increasing the threshold to 90%. This means, unless the probability of the match is greater than 90%, no answer will be provided. This is often referred to as the trade-off between precision and recall which helps address the question of do we want the bot to provide more answers, or do we want the bot to provide more answers accurately?
Improving the bots performance
We currently have two ways of improving the bots performance:
- When a question is answered by the bot – The feedback provided is used to upvote or downvote the bots response
- When a question is not answered by the bot – We first identify if the question is repetitive in nature, if so the questions are automatically logged back to a database through a separate slack channel
Suggestions
We are always open to more suggestions and recommendations on how to improve this process. You can reach out to revathi.anilkumar@nutanix.com with your feedback.