Do you develop bots for Cisco Webex Teams (formerly Cisco Spark)? Do you ever wish that there was a way to create a set of regression tests to ensure that your bot behaves consistently given the same input? Or maybe you wish that there was a way to speed up your iterative development/debug process when you are working on a bot response to a complex set of input? Or perhaps, you just wished you could do some bot development work when you were in a place that didn’t have good internet connectivity?
If so, I’m happy to let you know that the Webex API Emulator, originally created by Stève Sfartz, API Evangelist at Cisco DevNet, can help make your wishes come true. The emulator is a simple node.js project which you can download from GitHub and run locally. It loads a tokens.json configuration file that tells it about the users that can interact with it. This file, which is editable by you, allows you to create “authorization tokens” and “personIds” for your bot, your tester and any other “people” that might be added to spaces as part of your tests. Think of it as your own, local, common identity server.
Let’s start making those wishes come true by dealing with the last wish first, the idea of doing bot development when you are offline. Once the emulator is up and running you can use a tool like Postman to send requests to the Webex API Emulator to do things like create spaces, add a user (or bot) to a space, or post a message to the space. The emulator also supports the webhooks API so if you configure your bot to talk to your locally running instance of the emulator, your bot will get notified when it is added to a space or when a message is sent to it. Once you can can send a message to your bot via the emulator, you can open up your favorite editor or debugger, and you can start stepping through your bot’s responses to these commands, building and debugging responses to these inputs, all without an internet connection.
We’ve glossed over a few things here, but we’ll get back to it. Assume you are now able to send commands to your bot via Postman and see the Webex API requests that your bot makes in response, all offline. Handy for working offline, right? But once you get used to this, you may start to realize that running in this mode simplifies your development motion even when you DO have an internet connection. Imagine your bot needs to have a particular response in certain specific, potentially complex, circumstances. For example if you send your bot a message like “who from my company is in this room”, you might want it to list people from your company but also be sure that it skips people from other companies. You’d want to make sure that the bot behaves properly in circumstances where there are no people from your company, some people from your company, no people from other companies and some people from other companies. To test this you’d ideally create a several spaces and add several combinations of people to each of those space, and then type in your requests in all those spaces. Doing this manually in the Webex Teams app can be time consuming and it might not make your test subjects very happy to keep getting added to different spaces as you test different variations!
With Postman and the emulator, you can make the API calls to create the spaces, others to add each of the users, and another set to send in the commands. These can then be saved in a postman collection and run with a couple of keystrokes. Your bot didn’t respond as you expected? Simply stop the emulator and your bot and restart them (in that order!). This brings you back to a pristine environment where no spaces have been created yet. Alternately, have another set of commands to delete the spaces and run your test again as often as you like, until it works perfectly for all the variations that you care to test for!
Hmm, as we start building up these sets of API calls to emulate user input that we want our bot to respond to, we are starting to get close to what some might call a regression test. There is still one problem though, a true regression test should also validate the responses from your bot. Luckily the Webex API Emulator also supports “bot test mode”. In this mode we can instruct the emulator to look for bot responses to certain input and provide that back in its responses to us.
In order to better understand how bot test mode works we’ve created a bot-test-framework-example to demonstrate a set of tests in action. It includes a link to the Webex API Emulator project, a sample bot, a full collection of Postman tests and a detailed set of instructions to help you get this going. Since the Webex API emulator is written in javascript we chose a node.js sparkbotstarter project originally written by Victor Algaze, based on the node-flint framework created by Nick Marus. We hope to add some bots built with python in the future. The emulator doesn’t care about your framework, but you DO need to be able to configure the Webex API endpoint that your bot sends requests to.
Now, let’s get a flavor for how bot test mode works.
Before we could run any tests we needed to start our Webex API Emulator in “bot test mode” and start our bot using an API token specified in the emulator’s tokens.json file. When we’ve got the the emulator and our bot running properly we’ll have some terminal windows that look something like this:
Once those services are running we can start sending in our test input. The first couple of test cases don’t generate any bot reaction, they simply set our state.
- We create a new space via a POST to the /rooms API and store the roomId in a postman environment variable called “_room”.
- We then add our bot to the space via a POST the /memberships API.
Now we are ready to send a message to the bot and check if it responds as we expect. Here’s what that request looks like in postman:
This is a fairly standard POST to the /messages endpoint with a body which specifies the roomId of the previously created space (stored in a postman environment variable), and includes some markdown which mentions our bot and sends the command “/leave”. (Note that all of the environment variables used in these test cases are explained in more detail in the bot-test-framework-example readme). Now let’s look at what makes this request a true “regression test”.
Here are the headers for this request:
The Authorization and Content-Type headers should look familiar to anyone who has used postman to send requests to the Webex API, but there is a new header “X-Bot-Responses” as well. When the emulator sees a request with this header, it understands that we expect the bot under test to make two subsequent requests in response to this command. It will “intercept” the original response to this request and “hold” it until it gets the two bot requests and then build out a consolidated response that looks something like this:
This response contains two objects:
- A testFrameworkResponse which is the emulator’s original response to our POST to the /messages endpoint
- An array of botResponses, which show us the requests that the bot made in response to our test input. In this case, our bot made a POST to the /messages endpoint with the text “OK. I know when I’m not wanted…”. It then made a DELETE call to the memberships API and deleted its membership in the space, effectively leaving the room.
Finally, notice the “Test Results” circled in the image above. For each step in our test suite, we created Postman test cases to validate that the API response was as expected, and when appropriate, that the bot responded as expected as well. We won’t delve too deeply into how these tests work, but a full set of example postman tests, including the test cases are provided in the bot-test-framework-example.
While there are a lot of moving parts here to make this work, we think it can ultimately speed up your bot development process. I’ve been using this for a few months now and the regression tests have helped me discover bugs in my bots that I would have depended on users finding for me without it. I also love being able to work on a bot while on an airplane with sketchy internet connectivity!
If you decide to give the Webex API Emulator and bot-test-framework-example a try, we’d love to hear how it goes. git Join the Ask Webex API Emulator space on Webex Teams and give us feedback. Right now this is just another Webex API developer community supported project, and we hope this post makes it clear that tools like this only happen with an active and engaged developer community who make contributions like Stève, Victor and Nick have done. Who knows though, if usage of the emulator really takes off, we may someday be able to turn it into a full fledged service provided by Cisco.
Let us know what you think!
-- JP Shipherd, Sr. Product Manager
