How Much Time Takes Your Smoke Test?
The first sign of smoke is always an early indicator that fire is not far away. That is the idea behind the smoke test – quick test to get confirmation that after a change system works as usual.
Every tester tend to agree to this, but the fun starts when I ask: “how quick is “quick”?” or more specific: “how much time takes your smoke test?” In trainings usually I get answers between 10 minutes and several hours. In case of hours I ask what is the difference between smoke test and regression test. We wanted to check the smoke, not to retest all the features, right?
My smoke test consist of one main pathway, nothing fancy, just straight through. Tipically it takes 30-40 seconds for automated test and around 5 minutes for manual test. If I observe “smoke” than first step is to inform a team (during development phase) or a customer and operations team (if software is already on production). In case of smoke team needs to investigate why it is not working and also why unit and integration tests did not find it, what do we need to improve to get feedback earlier. If smoke in on production in most of cases first step is roll back and only when system is stable again, we can analyze what happened and why. If there is no smoke, than continue with regression tests or few selected checks of new features.