How risky is the testing of mobile application only on simulator / emulator

Launching the application based on the testing on emulators / simulators could be very risky for companies / developers and may affect the brand value of the application and company. Developing a mobile application is very easy and quick these days and requires very minimum investment. Now a days with the existence of so many application stores, it is very easy to launch the application and reach to wide user base but if application has issues then it'll get negative feedback even if concept of the application is very good and has the potential of becoming top rated app. This happens mostly because of absence of proper testing of the application from the technical as well as user's point.

Many companies especially start-ups, do not invest much on testing of their mobile application that affects them adversely. Some companies do perform some testing but only on emulators / simulators instead of actual mobile handsets. It is very difficult to test the application on each and every available mobile handset but app should be tested on some candidate handsets. First few iterations of testing should be done on emulators / simulators but final testing should be done on a set of actual handsets. Following are some cons of testing application only on emulators / simulators:

1. Network issue: Emulators emulate the network speed but emulation is not 100% correct and most of the emulators use the network speed of the PC so performance of AUT on emulator / simulator may not be very reliable.
2. Resolution issue: Screen size and resolution of a PC also affects the app UI especially in case of gaming apps e.g. user experience on 4 inch screen having 1136x640 resolution is different from that on 4 inch screen having 800x480 resolution i.e user experience changes based on pixel density. Emulating exact screen size and screen resolution is very difficult on emulators as emulators use resolution of the PC.
3. PC resources: Emulator uses PC resources because of which actual user experience may be very different from that on emulators.
4. Apps using device camera or GPS cannot be tested properly on emulators.
5. Carrier specific testing cannot be done on emulators.
6. It is difficult to do interrupt testing (e.g. Call / SMS etc.) on emulators.

Testers are no more road blocker

Few days back I attended a conference on "Verification Futures". It was primarily for hardware verification engineers. There were speakers from major EDA vendors like Synopsys, Mentor and Cadence. They talked about current challenges faced by verification engineers and innovations happening in the industry.

A key trend which is observed, is change of focus from design oriented VIPs to verification oriented VIPs which means if anything cannot be tested and then it should not be designed which is very true in software industry also.

As in software industry, focus is moving from first develop and then test to first write test and then develop code for it i.e. TDD (Test Driven Development). As complexity of the softwares is increasing day by day so proper testing of the software is becoming very-very important and now both clients and senior management have realized the importance of testing and hence are more concerned about proper and thorough testing and testing artifacts. Now a days, testers are not just black box tester they dig into code to find probable issues and suggest solutions. Instead of performing only random testing, they are now more systematic and focused, they use both scientific methods as well as their experience. They are expected to have more technical as well as domain knowledge.

In essence now a tester is no more considered as someone who blocks the release of the product but as one who is integral part of the team and who provides confidence in the product.