Testing strategy for mobile apps

The biggest challenge most of the mobile app development companies face is maintaining huge inventory of mobile devices which requires good amount of investment in procurement of the devices and then maintenance cost and operational cost like data usage etc. Also as new mobile devices are coming in the market every now and then (especially in case of android) so it’s practically not feasible to test app on each and every devices. Because of all these constraints mobile app testers should take balanced approach in testing.

Testing on emulators: Initial functional testing of the AUT should be done on emulators. Today most of emulators and simulators are capable of simulating the functionality of real device so first approach testers should follow is test AUT on emulator with target OS version to uncover most of the functional issues. AUT should be tested on each supported OS version.

Incase of android you can fully configure you emulator like SD card support, keyboard support etc.

Testing on real device: Once you feel that build is stable enough and most of the application specific functional and UI issues have been uncovered and fixed then you should start testing of AUT on real devices to uncover device specific issues. If you have all the devices available then you should perform testing on each device otherwise you should select required devices intelligently like one device of each target screen resolution and one device of each manufacturer, one device of each OS version. You should talk to developers to know if developers are using any OS version specific APIs etc.

Note: One important point to note here is you should select devices based on targeted mobile operator like AUT might work differently on two iPhones, one on Verizon and other on AT&T. This happens because of different technologies on both. And also sometimes because of customization done for mobile operator like one operator might block some APIs etc.

Testing on virtual devices: Today we have virtual environments like DeviceAnyWhere, PerfectoMobile etc. They have very good device inventory which you can use to test your AUT. They are very helpful as you need not to maintain huge device inventory in your organization. But as they charge on hourly basis, you need to make sure that you use them intelligently. When you have tested on app on different combination of screen resolution, os versions etc. then you could DAW etc. to sanity test your app on other devices with same configuration just to verify AUT works perfectly on them also. Before using them, make sure you have very good network connectivity otherwise you might observe some issues which might not occur on real devices.