In order to analyze the effectiveness of a system, we look at it from several different viewpoints. What are those viewpoints? What questions do we ask in order to have a nice overall picture of a system?
These are some dimensions I look at when during analysis and some typical questions I like to ask.
Performance and Scalability
The ability to grow a system’s capacity in the future to support projected growth is scalability.
Typical questions include:
- How do we measure capacity?
- What capacity must the system support on opening day?
- How do we grow capacity over time?
- How big can the system grow before we throw it away and start over?
Security
Although there are many implementations of security in a software system, the basic idea is simple: Provide a standard way to ensure appropriate access by people and other software to the system and to system components.
Typical questions include:
- Does this system need to be secure? Why? Which parts in particular?
- Does the security model integrate with my pre-existing security framework? Active Directory, for instance?
- Is security role based, name based, both?
- How is security auditing capability provided. Is there a log of who signed in when and what they accessed?
- Is the data secured separately from system functionality? How? This is where your SOX and HIPAA audits will bite you later.
Reliability
This dimension speaks to the ability of a system to remain up and stable without memory leaks, unpredictable behavior, or other instability.
Typical questions include:
- How long has a system run in production between maintenance cycles?
- Are there any known resource leaks in the system?
- What is the most common cause of production downtime in other installations of the system?
Run Time Configuration and Support
Monitoring and instrumentation are significant issues of supportability. Will the developer who wrote it be getting a call in the middle of the night or will the operations technicians be able to troubleshoot the application? This dimension also looks at the ability of the system to change without a recompile/re-deployment.
Typical questions include:
- Must the system be taken offline during a typical update or maintenance cycle?
- How may the system be customized without writing code?
- When upgrading to future versions, does the architecture enable run time deployment without downtime?
- If constant uptime is a requirement, how will system sessions be redirected into a new version stack during an upgrade process? Will old sessions be bled off?
- Is there sufficient monitoring capability to troubleshoot the system?
- What is the logging model and how might it be extended?
Other Considerations
Obviously this is a non-exhaustive list and the dimensions of interest will vary per project and system under consideration. Other areas to consider may include portability, multi-tenant capability, hardware requirements, support for applicable standards, and a clear product roadmap. What things do you consider when looking at a buy option for software system?
Technorati tags: System Architecture, Software Architecture, Requirements Analysis