Thursday, January 24, 2008

Performance Testing Challenges


Agenda


“We need Performance Testing, but what is it?”
“I know you want it to be fast, but how fast?”
“What do you mean, what will the 500 users be
doing?”
“Houston, we have a problem… but is it the
application or the tool?”
“What do I do with all this data?”
“That’s a cool chart, but is the performance good or
bad?”
“Does that mean we’re done?”


About Performance Testing:
Performance Testing Determines…
- Speed
- Scalability
- Stability
- Confidence
…while focusing on
- User Expectations
- System Constraints
- Costs
Specifically, it answers:
- How many…?
- How much…?
- What happens if…?

Performance Test Cycle

Evaluate System
Develop Test Assets
Baselines and Benchmarks
Analyze Results
Tune
Identify Exploratory Tests
Execute Scheduled Tests
Complete Engagement

How Fast is Fast Enough?
There are no industry standards!
Based on system context, determine:
- User Psychology
- System Considerations
- Usage Considerations
Then assess:
- User Expectations
- Resource Limitations
- Stakeholder Expectations
Finally determine:
- Speed, Scalability and Stability Requirements for
• User Experience measurements, not component metrics.
• Batch type requirements.
• Exception requirements.
• Component metrics (use sparingly!).
- Composite Requirements for your application!
Will be continue...