Automated Tests: Don’t listen to Microsoft

I decided to write this article because of few things which are going on now. The first is that I will speak about AppSource on DynamicsCon.com (thank you all for so many votes), second, I needed to reread Luc’s book about automated tests (I only can recommend it to you if you still not writing tests or you want to do it better).

However, the most important reason is that Microsoft’s approach to automated tests for AppSource apps has changed. 

Automated tests – “only” strongly recommended

In previous years automated tests were mandatory if you want to publish an app on AppSource. Now if you would open Technical Validation Checklist (https://docs.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-checklist-submission) you can see that they are only strongly recommended:

 We strongly recommend you are using automated testing, using the AL Test Toolkit. You are not required to include the test package with your extension.

Microsoft splits the roles now. They are responsible for Basic and System apps quality and we (ISVs, VARs) are responsible for our code quality. We should make sure that we do not break our customers and partners. 

What is wrong with this approach? Nothing at all but…

Automate tests are something that many times saved my code and without it, my code would be worst. It is not only about the code, but I am sure that also the functionality is better when I write automated tests.

And do you know what is the hardest thing about writing automated tests? It is to start writing them.

First, you need to learn how to write tests (that is quite easy) then you need to convince your managers that they are needed. Not always (I see more and more people who see benefits right away) but I know for some developers it is harder than writing the first test- “We worked with them for years why we should change that?“. 

Writing automated tests takes time. However, still it takes less than fixing bugs and gaps which you or your customers will find after release. 

My concerns

How did I start? I was “forced” by Microsoft (“you want to have app on AppSource you need to have automated tests“). Would I stop now? Never! And I believe there are much more people who think the same way. 

We have tests from NAV 2009 but we (at least most of us) started to use  them around 10 years later. It took us many years and I am glad that we had some push from Microsoft. Now we lost it.  

I hope some people will not start thinning now “Tests are only recommended so we do not need to do them“. That would be bad for them but also for all others who can use their app from AppSource.

I fully understand the Microsoft approach. I think it is a good direction and a good split – we need to think about quality and performance.

However, I would be glad if from “mandatory” to “strongly recommended” Microsoft would wait at least 2 more years. 

I think then most developers would switch and start writing automated tests and also management would see how many benefits we gain by spending time on it.

So, don’t listen to Microsoft. If you already write automated tests don’t stop. If you didn’t start yet then think about them as mandatory! Trust me it is worth it.