Automating Smoke Tests in Sitecore

After editing a page or updating a few lines of code do you ever wish you could do more than rely on a few clicks through your site to ensure that every page is in working order? Consider making an automated smoke test.

A smoke test comes from the electronics world, where the fastest test of a prototype is to plug it in and watch for smoke. If it smokes, something is wrong; there is no need for further testing. The smoke test does not mean testing is done but it quickly weeds out devices where more testing would be a waste of time. The web development equivalent of the smoke test is opening a URL and seeing if it loads, and the smoke is the error page or, in ASP.NET, the yellow screen of death. Doing smoke testing by hand means opening a browser, typing a URL, and watching the page load.  But why do by hand what a computer can do more quickly? This post will walk though a simple way to make an automated smoke test for Sitecore. While this method won't replace unit tests, browser-based functional testing with Selenium, or your QA team, it will give you the ability to catch simple errors more quickly.

How It Works
The Sitecore smoke test first retrieves a given root item and all of its children, then filters out the items without presentation settings, and then gets the URLs of the remaining items. These URLs are used to generate a page containing a test method for each item. You can then paste those methods into an MSTest class and run it from within Visual Studio with unit tests or on its own.

Caveat Emptor
Before we get to the code there are a few things to note. The test only checks to see that the page loaded successfully—it does not care what the page is displaying. This method will not test Javascript or other page interaction, and it will not load pages requiring authentication –that kind of automation is best left to tools like Selenium.

For the sake of this post I am assuming we are using ASP.NET MVC, MSTest in Visual Studio, that all of the pages are public, and that we will cache the results of the page. If you need an additional reference on how to create testing projects in Visual Studio, read this unit testing article by Microsoft.

If you are using Web Forms you would need to put these methods in a code-behind and have the output set to a label. If the page of generated method tests should not be public it can be put it in the Sitecore application. Since this page will retrieve every item in the content tree, be careful that if it is publicly available that you cache the results because during this time period, since one person refreshing the page could easily tie up the server. URLs that are not generated by the code can be added by hand, or you can modify the script to list .aspx pages or MVC routes and also test pages that take query string parameters this way.

If you are using a testing framework other than MStest, you may need to change the test method template.

The Code
Here is the class to generate the test methods:

001.using System;
002.using System.Collections.Generic;
003.using System.Linq;
004.using System.Web;
005.using System.Web.Mvc;
006.using Sitecore.Data.Items;
007.using Sitecore.Diagnostics;
008.using System.Text;
010.namespace Web.Controllers
012.    public class PageListController : Controller
013.    {
014.        // need to escape the curly-braces due to the string format template
015.        // the TestCategory attribute allows you to run tests as a group: 
017.        private const string TestMethodTemplate =
018.            @"
019.            [TestMethod]
020.            [TestCategory(""SmokeTest"")]
021.            public void does{0}_load()
022.            {{
023.                var response = GetUrl(""{1}"");
024.                Assert.AreEqual(
025.                                response, 
026.                                System.Net.HttpStatusCode.OK, 
027.                                ""Expected "" + System.Net.HttpStatusCode.OK + "", got "" + response
028.                                ); 
029.            }}
030.            ";
032.        private const string rootUrl = "";
034.        /* This should generate a method that looks like this:
035.        [TestMethod]
036.        [TestCategory("SmokeTest")]
037.        public void does_test_web_page_load()
038.        {
039.            var response = GetUrl("");
040.            Assert.AreEqual(
041.                            response, 
042.                            System.Net.HttpStatusCode.OK, 
043.                            "Expected " + System.Net.HttpStatusCode.OK + ", got " + response
044.                            ); 
045.        }
046.        */
048.        // This MVC method could easily be placed in Page_Load WebForms method
049.        // and set string to a label instead
050.        public ActionResult Index()
051.        {
052.            // item root path "/sitecore/content/home"
054.            var items = GetItems("/sitecore/content/home");
056.            var stringBuilder = new StringBuilder();
058.            stringBuilder.AppendLine("<pre>");
060.            foreach (Item item in items)
061.            {
062.                stringBuilder.AppendLine(MakeTestMethod(item, rootUrl));
063.            }
065.            stringBuilder.AppendLine("</pre>");
067.            return Content(stringBuilder.ToString());
068.        }
070.        public List<Item> GetItems(string rootItemPath)
071.        {
072.            Debug.ArgumentNotNullOrEmpty(rootItemPath, "rootItemPath");
074.            var rootItem = Sitecore.Context.Database.GetItem(rootItemPath);
076.            Debug.ArgumentNotNull(rootItem, "rootItem");
078.            var descendants = rootItem.Axes.GetDescendants();
080.            List<Item> items = new List<Item>();
082.            items.Add(rootItem);
084.            foreach (Item item in descendants)
085.            {
086.                if (IsPresentationSet(item))
087.                {
088.                    items.Add(item);
089.                }
090.            }
092.            return items;
093.        }
095.        private string MakeTestMethod(Item item, string rootUrl)
097.            var itemInfo = GetItemLabels(item, rootUrl);
099.            return string.Format(TestMethodTemplate, itemInfo.Name, itemInfo.Url);
100.        }
102.        private bool IsPresentationSet(Item item)
103.        {
104.            return !string.IsNullOrWhiteSpace(item.Fields[Sitecore.FieldIDs.LayoutField].Value);
105.        }
107.        // Create a URl for a given item
108.        private string GetItemUrl(Item item, string rootUrl)
109.        {
110.            return rootUrl + Sitecore.Links.LinkManager.GetItemUrl(item);
111.        }
113.        // Get a valid mathod name and URl for a given item
114.        private ItemLabels GetItemLabels(Item item, string rootUrl)
115.        {
116.            return new ItemLabels {
117.                Name = ConvertPathToMethodName(item),
118.                Url = GetItemUrl(item, rootUrl)
119.            };
120.        }
122.        //Convert the item path into a valid method name 
123.        private string ConvertPathToMethodName(Item item)
124.        {
125.            return item.Paths.Path.Replace("/", "_").Replace(" ","_").Replace("-","_").Replace("*","");
126.        }
128.        // Helper class to store the item URL and Name
129.        private class ItemLabels
130.        {
131.            public string Name { get; set;}
132.            public string Url { get; set; } 
134.    }

Add this to your MVC project, or, if you are using WebForms, change the Index() method to Page_Load and visit the URL. If all goes well you will see a page full of web methods. Set this page aside—you’ll be pasting these into the class below.

Here is the class where you can insert the generated links. You can add the methods anywhere in the class (or in the "paste here" area if you are a paint-between-the-lines sort of person).

01.using System;
02.using System.Text;
03.using System.Collections.Generic;
04.using System.Linq;
05.using Microsoft.VisualStudio.TestTools.UnitTesting;
06.using System.Net;
08.namespace Web.IntegrationTest
11.    [TestClass]
12.    public class CheckLinksWithPresentation
13.    {
15.        // Here is an example test that loads
16.        // It succeeds if returns
17.        // a 200 status, and fails otherwise.
19.        [Ignore] // remove the [Ignore] attribute to run this test, 
20.                 // or delete the example altogether
21.        [TestMethod]
22.        [TestCategory("SmokeTest")]
23.        public void does_home_load()
24.        {
25.            var response = GetUrl("");
26.            Assert.AreEqual(
27.                            response, 
28.                            System.Net.HttpStatusCode.OK, 
29.                            "Expected " + System.Net.HttpStatusCode.OK + ", got " + response
30.                            ); 
31.        }
33.        //----- paste the generated unit tests here
37.        //-------------------------
39.        // Request the URL and only return its status code.
40.        private System.Net.HttpStatusCode GetUrl(string url)
41.        {
42.            var request = (HttpWebRequest)WebRequest.Create(url);
43.            var response = (HttpWebResponse)request.GetResponse();
45.            return response.StatusCode;
46.        }
47.    }

If your project contains unit tests and you run those tests frequently you will want to exclude these tests, since these tests have external dependencies and can take minutes to run. You can do this by excluding the “SmokeTest” category.

When the test is complete you will see either a PASS or FAIL in your unit test runner, and if they each pass you will know that your site has waved goodbye to the “Yellow Screen of Death” and you can move on to the more advanced Selenium tests or testing by hand. 

Extra credit
This code was written to show the idea of generating automated smoke tests for Sitecore sites and could be improved. One such improvement would be to generate the list of methods without adding a code generation page to the website. One way to do this is to make a plugin for Sitecore Rocks which would retrieve the list of items, visit each item’s URL, and display the result.

For tests that run immediately after the build, T4 templates could be used to generate the test class just like TDS does for custom items and glass items. Just as templates are used to generate each class for a given list of items, templates could generate the methods.

Sitecore testing, Sitecore development, Sitecore custom code


Add a Comment


Please confirm you are human by typing the text you see in this image: