The Test Log: Xamarin Test Cloud’s Indispensable Debugging Tool

It’s a Log Way to the Top

(if you want a 5 star app)

It may not seem like much, but this feature may add quite a bit of insight to your UI Tests if you haven’t used it before.

Since launch, Xamarin Test Cloud has provided three types of very useful logs:

  • The Console Log, which is syslog relay on iOS or logcat on Android;
  • Test Failures, which appear when assertions in the test raise exceptions;
  • Stack Trace, to help pinpoint the source of errors and crashes in the actual codebase of the app.

All of that is useful and critical information to a tester. Until recently, there was no easy way to write your own custom information to a log unless you were using the app.Invoke “backdoor logging” hack.

Now, all you have to do is simply write whatever you want to stdout in your test script. In Xamarin.UITest, you would use Console.WriteLine(string). This appears in a new area called the Test Log, which is shown here in a single device view.

The Test Log button.
The Test Log button in a Xamarin Test Cloud report’s single device view.

Upon investigating the Test Log, we see this warm, fuzzy, familiar output (for NUnit users):

Our friendly console log. It's big, it's better, it's good!
Our friendly console log. It’s big, it’s better, it’s good!

This means that you can very easily write anything you like to the console, in any way you choose (NLog; whatever that crazy enterprise logging block from .NET 2 was; you name it!) The most basic approach is to call Console.WriteLine("message") from your test. As tests run across the devices, each device generates its own test log for that session.

The Test Log is free and easy to understand, so you should use it often.

How to use the Test Log

When you’re working with any remotely hosted resource, having as much log data available to you is absolutely critical. Your app running on the device can generate its own logs, which appear in Console Log tab, but what about logging from the execution of the test?  Why would we do that? Here are some common applications that you can use today.

  1. Better understand what’s happening on each device individually. If your test fails on a 3″ screen because the keyboard blocks the password field, you can know for sure with some nice logging. Just create a nice extension method called ToLogFormattedString for the Xamarin.UITest.AppResult class, and then use Console.WriteLine(app.Query().ToLogFormattedString()) in the test. This will dump out a pretty-printed control hierarchy for you to use. Since app.Query() returns only fully visible elements by default, you can use this technique to find out what elements can and cannot be “seen” by the automation framework (and by extension, your users) on each device.
  2. Generate your own diagnostics and metrics. If you’re trying to optimize tests, you can output execution timing to that log.
  3. Validate response data from service endpoints. Some E2E tests go all the way back to the database. An example use case would be creating a user: you’d use the UI to create a new user with a randomly generated, unique user ID created by the test code. You can automate the creation workflow, and at the end of the process, hit your REST API and try to pull a valid user with that same ID – you can then assert that the fields in the database match the ones entered by your test. Outputting JSON to the console log can help you understand what exactly is going over the wire.
  4. Sanity check your work. Because sometimes you need to make sure your code “got to there.”

The test log is just another way Xamarin Test Cloud gives you the information you need, when you need it, to get faster feedback and make better apps for the world to enjoy.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s