Author: Chaitanya M Bhatt
Email: chaitanya.bhatt@performancecompetence.com
In my observation, persistently there has been a lack of awareness of the difference between “Simultaneously connected active users” and “Simultaneously connected passive users”. Commonly this problem is observed both from executive stakeholder side as well as the performance engineering/testing team side.
Impact of not understanding these concepts makes a lot of difference when it comes to meeting the actual performance test requirement and can make a huge difference on the legitimacy of your load test report.
For your information: Simultaneously connected active users can be also termed as “active concurrent application users” and simultaneously connected passive users can be termed as “connected users in passive mode”.
Connected users in passive mode:
I will quickly explain the differences and the nuances with an example: Imagine if “Facebook” was your AUT (Application Under Test); Many end users of Facebook.com often set www.facebook.com as their default browser page and hence when the browser is opened — a connection (may not be persistent) would be maintained between the end user and the Facebook.com server. The user may leave the Facebook.com session idle without firing any query like searching for a friend or adding a friend. This case is a typical example of connected user who is in a passive state.
Facebook.com might have 100’s of thousands of such users who are in this state and it is important to simulate this condition during load tests because even passive sessions or connections can hold certain resources of the server as well as can have impact on the overall thread pool size and can also affect various tunable configuration parameters of web/app/db servers.
Active concurrent application users:
Continuing with the same Facebook.com example, it is known that there are also users who would not just be connected to Facebook.com but would also be actively using facebook.com features like adding a friend, poking a friend, updating status, uploading photos etc. It is during this kind of user connection state that the actual payload gets affected heavily.Higher the rate of user activity higher would be the transactional throughput.
Care should be taken while setting a test execution window, because clients might say facebook .com has 200,000 concurrent active connections in a day with an average user base executing 9,600,000 transactions daily (this means 10 million transactions such as adding friend, poking friend, photo upload etc.), but when you break the transactions to fit your test window – you often forget to think about the persistence duration of users per login-logout sessions against your test window.
What I meant to say, is that, if you are executing a 60 minutes load test and for scenario configuration purpose if you’re breaking the daily stats to fit your 60 minutes test window by dividing total users per day by 24 (approx. 8300 virtual users). And, also you would have calculated total transactions fired per hour, which is equal to 2 (derived the result after dividing 9.6 million by 200,000 = 48 transactions/user/day; 48/24 = 2 transactions/hour), but ignored the persistence of transactions; you may face a situation where all virtual users executing those 2 transactions (like: adding a friend and poking a friend) would finish execution within seconds leaving the rest of the test window void without any transactions, which will surely cause over-stressing of the server leading to inaccurate result. In the other extreme case without thoughtfully setting virtual user transaction persistence can cause a situation, specifically with ‘login – Perform. a transaction – logout’ scenario type, where one user finishes a cycle quickly before the other enters; this would actually lead to lesser active user concurrency and resulting in inaccurate result.
The solution to maintain realistic transaction persistence is by making virtual users to stay back in the connected session with proper “pacing” between transactions and “realistic think time” , so that transactions are optimally(try following actual/live pattern) spaced within the test execution duration/window.
Emulating realistic concurrency of user at application usage level is the key to understand the true application behavior. during load test.
