Email this article An Emerging Experience Factory to Support High-Quality Applications Based on Software Components and Services (Invited Paper)
Abstract
Software components and services (SCS) are playing an increasingly important role in software engineering, particularly as building blocks of systems that demand high quality and dependability. A major impediment to advances in developing such systems is the difficulty of providing objective evaluations and conducting rigorous experiments to determine the efficacy of selected SCS and the resulting systems. This paper presents a framework that facilitates such experimentation to measure SCS quality in the target usage environment, to provide unbiased quality assessment, and to support effective usage of SCS in high quality applications. We are building a comprehensive collection of 1) usage scenarios in a set of target operational profiles and 2) a flexible defect classification and analysis framework and related quality analyses. Our work will form the first step of an operational experience factory for SCS. Resulting repositories and supporting facilities from applying our approach to web-based applications are included to demonstrate its viability.
Keywords
References
[1] V. R. Basili, “The experience factory and its relationship to other quality approaches,” in Advances in Computers, Vol.41, M. V. Zelkowitz, Ed. San Diego, CA: Academic Press, 1995, pp. 65–82.
[2] J. D. Musa, Software Reliability Engineering. NewYork: McGraw-Hill, 1998.
[3] A. A. Aviˇzienis, J.-C. Laprie, B. Randell, and C. Landwehr, “Basic concepts and taxonomy of dependable and secure computing,” IEEE Trans. on Dependable and Secure Computing, vol. 1, no. 1, pp. 11–33, Jan. 2004.
doi:10.1109/TDSC.2004.2
[4] B. Beizer, Software Testing Techniques, 2nd Ed. Boston, MA: International Thomson Computer Press, 1990.
[5] N. G. Leveson and K. A. Weiss, “Making embedded software reuse practical and safe,” in Proc. 12th Foundations of Software Engineering Conference (ACM Sigsoft 2004/FSE- 12), 2004.
[6] S. L. Pfleeger, L. Hatton, and C. C. Howell, Solid Software. Upper Saddle River, New Jersey: Prentice Hall, 2002.
[7] V. R. Basili, M. V. Zelkowitz, F. E. McGarry, J. Page, S. Waligora, and R. Pajerski, “SEL’s software processimprovement program,” IEEE Software, vol. 12, no. 6, pp. 83–87, Nov. 1995.
doi:10.1109/52.469763
[8] N. Alaeddine, “Anomalies analysis and classification for web quality and reliability improvement,” Ph.D. dissertation, Southern Methodist University, Dallas, Texas, U.S.A., 2009.
[9] C. Kallepalli and J. Tian, “Measuring and modeling usage and reliability for statistical web testing,” IEEE Trans. on Software Engineering, vol. 27, no. 11, pp. 1023–1036, Nov. 2001.
doi:10.1109/32.965342
[10] Z. Li, N. Alaeddine, and J. Tian, “Multi-faceted quality and defect measurement for web software and source contents,” Journal of Systems and Software, vol. (to appear), 2009.
[11] L. Ma and J. Tian, “Web error classification and analysis for reliability improvement,” Journal of Systems and Software, vol. 80, no. 6, pp. 795–804, June 2007.
doi:10.1016/j.jss.2006.10.017
[12] J. D. Musa, “Operational profiles in software reliability engineering,” IEEE Software, vol. 10, no. 2, pp. 14–32, Mar. 1993.
doi:10.1109/52.199724
[13] H. D. Mills, “On the statistical validation of computer programs,” IBM Federal Syst. Div., Tech. Rep. 72-6015, 1972.
[14] J. A. Whittaker and M. G. Thomason, “A Markov chain model for statistical software testing,” IEEE Trans. on Software Engineering, vol. 20, no. 10, pp. 812–824, Oct. 1994.
doi:10.1109/32.328991
[15] J. Tian and L. Ma, “Web testing for reliability improvement,” in Advances in Computers, Vol.67, M. V. Zelkowitz, Ed. San Diego, CA: Academic Press, 2006, pp. 177–224.
[16] J. Tian, Software Quality Engineering: Testing, Quality Assurance, and Quantifi able Improvement. Hoboken, New Jersey: John Wiley & Sons, Inc. and IEEE CS Press, 2005.
[17] J. Voas, Software Fault Injection - Inoculating Programs Against Errors. New York: Wiley Computer Publishing, 1998.
[18] R. G. Hamlet, “Testing programs with the aid of a compiler,” IEEE Trans. on Software Engineering, vol. 3, pp. 279–290, 1977.
doi:10.1109/TSE.1977.231145
[19] R. Chillarege, I. Bhandari, J. Chaar, M. Halliday, D. Moebus, B. Ray, and M.-Y. Wong, “Orthogonal defect classi- fication — a concept for in-process measurements,” IEEE Trans. on Software Engineering, vol. 18, no. 11, pp. 943– 956, Nov. 1992.
doi:10.1109/32.177364
[20] R. L. de Oliveira Moraes and E. Martins, “JACA - a software fault injection tool,” in in Proc. Int. Conf. on Dependable Systems and Networks, 2003, p. 667.
doi:10.1109/DSN.2003.1209976
[21] J. Hao and E. Mendes, “Usage-based statistical testing of web applications,” in Proc. 6th International Conference on Web Engineering, Menlo Park, California, July 2006, pp. 17–24.
doi:10.1145/1145581.1145585
[22] N. Delgado, A. Q. Gates, and S. Roach, “A taxonomy and catalog of runtime software-fault monitoring tools,” IEEE Trans. on Software Engineering, vol. 30, no. 12, pp. 859– 872, Dec. 2004.
doi:10.1109/TSE.2004.91
[23] A. G. Koru and J. Tian, “Defect handling in medium and large open source software projects,” IEEE Software, vol. 21, no. 4, pp. 54–61, July 2004.
doi:10.1109/MS.2004.12
[24] IEEE Standard Glossary of Software Engineering Terminology, IEEE, 1990.
[25] D. Mackenzie, “Computer-related accidental death: An empirical exploration,” Science and Public Policy, pp. 233– 248, Aug. 1994.
[26] B. Boehm and V. R. Basili, “Software defect reduction top 10 list,” IEEE Computer, vol. 34, no. 1, pp. 135–137, Jan. 2001.
[27] N. Fenton and S. L. Pfleeger, Software Metrics: A Rigorous and Practical Approach, 2nd ed. Boston, MA: PWS Publishing, 1996.
[28] S. H. Kan, Metrics and Models in Software Quality Engineering, 2/e. Reading, MA: Addison-Wesley, 2002.
[29] A. L. Goel and K. Okumoto, “A time dependent error detection rate model for software reliability and other performance measures,” IEEE Trans. on Reliability, vol. 28, no. 3, pp. 206–211, 1979.
doi:10.1109/TR.1979.5220566
[30] J. Tian, L. Ma, Z. Li, N. Alaeddine, and R. Geng, “Accelerated quality improvement through risk-based testing,” in Raytheon Information Systems and Computing Symposium (ISaCTN), Dallas, Texas, Apr. 2010.
Full Text: PDF