Virtual Library Computing Software Engineering Hazards and Risk

Relevant newsgroups

Repositories and mailing lists

Courses

Clubs and organizations

On-line publications

Journals

Books

Companies

Meetings

Searchable bibliography

Relevant newsgroups

• The ACM Forum on Risks and the associated moderated newsgroup comp.risks is recommended reading. Also available on the Web from Deja News. See a count of and link to URLs mentioned in RISKS articles from People Helping One Another Know Stuff (PHOAKS). You can join the gatewayed RISKS mailing list (only if you cannot access the newsgroup for some reason please!) by sending an email request to risks-request@csl.sri.com. Submissions for possible inclusion may be sent to risks@csl.sri.com. RISKS back-issues and a searchable database via WAISGATE are available on-line. A RISKS archive in the UK is also available, including the latest issue and searching. Submissions are sometimes included in the ACM SIGSOFT Software Engineering Notes. See associated book.

• Discussion on safety-critical systems also crops up in comp.software-eng from time to time. Messages relating to formal methods and software safety in this newsgroup have been archived.

• The sci.aeronautics.airliners newsgroup often includes discussion and information on aircraft safety.

• The sci.engr.safety newsgroup on the Safety of Engineered Systems has been established by Sethu Rathinam rathinam@netins.net (see RISKS DIGEST 16.53, November 1994). This was passed by 334 votes to 11 (20 March 1995).

• The misc.industry.safety.personal for discussion on personal protective equipment (PPE) - any device worn or held by the user in order to provide protection whilst performing the given task. See also other misc.industry.safety newsgroups.

• Other safety newsgroups.

Repositories and mailing lists

• A BibTeX bibliographic database of publications of relevance to safety-critical systems (second sourced in the US). A revised version of the bibliography is searchable.

• NASA Langley formal methods program including many applications to safety-critical systems.

• At the Software Engineering Institute, Carnegie-Mellon University, there is an archive of information relevant to dependable software including a README file. There is an associated mailing list. Contact Charles Weinstock on weinstoc@sei.cmu.edu for further information. Messages for the list should be sent to depend-sw@sei.cmu.edu.

• Jonathan Moffett ( jdm@minster.york.ac.uk) of the High Integrity Systems Engineering Group (HISE), Department of Computer Science, University of York, UK, runs a safety-critical mailing list forum. To join the mailing list, please complete the on-line form.

  See a moderated discussion on C++ and safety, edited by Brian Wichmann, March 1998.

• An archive of aviation safety information especially with respect to fly-by-wire aircraft such as the Airbus A320 and the Boeing 777 is available. For example, see the plot of the aviation safety size of software packages of various avionics implementations over the years. This archive is maintained by Robert Dorsett, rdd@netcom.com.

• Since 1989, a moderated mailing list dedicated to all types of safety issues, both occupational and environmental, has been in existence. Subscribe by sending an email message to listserv@uvmvm.uvm.edu with "SUB SAFETY your name" in the body of the message). To post a message, email safety@uvmvm.uvm.edu.

• The Center for High Assurance Computing Systems at Information Technology Division of the Naval Research Laboratory conducts interdisciplinary research and development in techniques for processing and communicating data that preserve critical system properties such as safety.

• The ESPRIT ProCoS and PDCS Basic Research projects investigated techniques concerned with improvements for safety-critical systems.

• The Experimental Computation Laboratory is an organization in the Computer Science Department at the University of Missouri - Rolla dedicated to research in advanced methods in distributed and parallel computation. The current focus is on the use of rigorous mathematics through formal methods to create fault-tolerant and secure real-time distributed computing systems.

• The High Integrity Systems Engineering Group (University of York, UK) undertakes research into all aspects of high integrity computer-based systems, including the ASAM-II project dedicated to the production of a prototype Safety Argument Manager and method.

• Information on Software Risk Management (including the SEI Program on Software Risk Management) and Dependability from the Software Engineering Institute, Carnegie-Mellon University, Pittsburgh, USA.

• ESPRIT Basic Research: PDCS 2 (Predictably Dependable Computing Systems), ProCoS II and ProCoS-WG (Provably Correct Systems).

• The TTM/RTTL framework for real-time reactive programs and devices found in embedded, safety-critical, or concurrent systems.

• High Integrity Software System Assurance (HISSA), NIST, USA.

• NASA Information Services including Goddard Space Flight Center and Johnson Space Center.

• Search for "safety critical" by Lycos.

• Safety-related publications from the IEE, UK.

• Software Integrity using SPARK ADA from DRA, UK.

• Internet Disaster Information Network - latest news on disaster situations.

• Safety Critical Publications from the IEE, UK. See Safety-related systems: Guidance for engineers and Workshop report on Formal methods and their role in developing safe systems, 20 March 1995. See also other organisations and sources of information.

• Computer-Related Incidents with Commercial Aircraft by Prof. Peter Ladkin, with hyperlinks to RISKS articles. See also Talking to Newspapers: A Cautionary Tale with Moral.

• The Aviation Disaster/Crash Page.

• Nancy Leveson, Professor of Aeronautics and Astronautics, MIT, USA. A leading software safety expert.

• Air Accidents Investigation Branch (AAIB), Department of Transport, UK. See formal investigations.

• Health and Safety Executive (HSE), UK. See Paddington/Ladbroke Grove Train Crash, 5 October 1999. See also Health and Safety Laboratory (HSL), Sheffield / Buxton, UK.

• ARIANE 5 Flight 501 Failure Report, 19 July 1996, from the European Space Agency (ESA).

• Safety information from NASA Office of Flight Assurance, USA.

• MIL STD 882C, System Safety Programme Requirements, US military safety standard. (See also Appendix A.)

• FAA Office of Regulation and Certification, promoting safety standards, including the Office of Accident Investigation, from the US Federal Aviation Administration. See the Aviation Safety Program and recent Preliminary Accident/Incident Reports.

• Report to the Defence and Aerospace Foresight Panel, Executive Summary, High Integrity Real-Time Software Working Party, NPL, prepared under the auspices of the BCS.

• Programme for Highly Dependable Systems, Department of Computer Science, University of the Witwatersrand, South Africa.

• UK DTI/ EPSRC Safety and Mission Critical Systems Programme information (originally called "SafeIT"). See project descriptions. See also advances in safety-critical systems.

• Glasgow Accident Analysis Group, UK. Developing new means of generating accident reports using formal methods.

• Therac-25 incident. See also information from DLSF Systems Inc.

• DLSF Systems Inc., Ottawa, Canada. See rail system safety, software system safety course overview and related Web sites.

• National Transportation Safety Board (NTSB), USA. See aviation accident searchable database and statistics.

• Safety Critical Computing Group, School of Computer Studies, University of Leeds, UK.

• Hazop Home Page, School of Computing and Mathematics, University of Teesside, UK. Includes a UK Defence Standards guide.

• White House Commission on Aviation Safety and Security, Department of Transporation, USA. See Final Report in PDF and (long) HTML format. (Completed 12 February 1997.)

• North Texas System Safety Society, USA. Includes software safety links.

• IFIP WG10.4 on Dependable Computing and Fault Tolerance.

• GRS, Germany. A non-profit organization for the assessment and development of the safety of complex technical systems, such as nuclear plants. (In English and German.)

• Aviation Safety Web Pages: Accident Database (1945 onwards).

• Flight Safety Foundation, including publications.

• Transportation Safety Board of Canada (in and French), including aviation occurrence reports and useful links.

• Disaster series on BBC2, UK, starting 5 January 1998.

• System safety verification information, including many links, from the formalWARE collaborative project, Canada.

• ESPRESS project, Germany. Engineering of safety-critical embedded systems.

• WorldSafety.com. A global community center for safety and security professionals. See links.

• Dependable System Architecture Group, SRI Computer Science Laboratory, California, USA. See 00-56 Defence Standard information.

• Safety-Critical Sysytems class information, part of a Virginia Tech course.

• Safety Systems Research Centre (SSRC) and Safety Systems Group, University of Bristol, UK.

• Software Safety Planning Group, IEEE Software Engineering Standards Committee (SESC).

• Safety-related mailing lists from Mailbase, UK.

Courses

• Critical Systems Engineering course, Integrated Graduate Development Programme in Software Engineering, Oxford University, UK.

• Modular MSc in Safety Critical Systems Engineering, University of York, UK.

• Olos Summer School, European Summer School on Reliability and Safety of Human-Machine Systems, Crete, Greece, 6-13 September 1998.

• PES Hazard Study course, School of Computing and Mathematics, University of Teesside, UK.

• Safety-Critical Systems Short Course, Continuing Professional Development programme, Imperial College, London, UK.

• Short hazard analysis course, Adelard, UK.

• Software System Safety Course by DLSF Systems Inc., Ottawa, Canada.

• Safety-Critical Systems Development final year undergraduate course by Prof. Chris Johnson, University of Glasgow, UK. Includes on-line slides.

Clubs and organizations

• Safety-Critical Systems Club (SCSC), UK. Enquiries should be sent to Mrs. J. Atkinson on csr@ncl.ac.uk ( Centre for Software Reliability, University of Newcastle upon Tyne, Newcastle NE1 7RU, UK). The club organizes the annual Safety-critical Systems Symposium and produces a regular newsletter Safety Systems (Editor: Felix Redmill, email felix.redmill@ncl.ac.uk, 22 Onslow Gardens, London N10 3JU, UK; tel & fax: +1-181-883-0789). See also BCS information.

• European Network of Clubs for REliability and Safety of Software (ENCRESS), sponsored by the ESSI Programme of the EC (1994-1997).

• European Safety and Reliability Association (ESRA). General Secretariat: Apartado 1690, 1016 Lisboa Codex, Portugal. Chairman: David Harvey, WS Atkins, Woodcote Grove, Ashley Road, Epsom, Surrey KT18 5BW, UK (email dwharvey@wsatkins.co.uk, tel +44-1372-726140 x4604, fax +44-1372-740055). Associated with the journal Reliability Engineering & System Safety.

• EWICS TC7 (European Workshop on Industrial Computer Systems, Technical Committee 7) on Reliability, Safety and Security. An ESRA Technical Committee since 1995. Contact Robin Bloomfield on reb@adelard.co.uk for further information.

• Safety/Security Club, UK DTI-funded Safety Policy and Models (SPaM) project. Email John Elliott on jje@src1.demon.co.uk for further details.

• IEEE SESC (Software Engineering Standards Committee) Safety Study Group.

• IEEE Society on Social Implications of Technology (SSIT). Includes environment, health and safety issues. See IEEE Technology and Society Magazine.

• American Society of Safety Engineers (ASSE). See related sites including other professional organizations.

• National Safety Council, USA.

• International Safety Council.

• Canada Safety Council, Canada.

• National Safety Council, Ireland.

• Board of Certified Safety Professionals (BCSP), USA.

• System Safety Society, Unionville, VA, USA.

• EUROCONTROL, European organization for the safety of air navigation.

• MISRA, The Motor Industry Software Reliability Association, UK. See Guidelines for the Use of the C Language in Vehicle Based Software.

• Lloyd's Register, shipping safety, UK.

See also the French National Institute for Transport and Safety Research (INRETS) in English and French.

On-line publications

• Formal Methods in Safety-Critical Standards, Jonathan Bowen. In Proc. 1993 Software Engineering Standards Symposium (SESS'93), Brighton, UK, 30 August - 3 September 1993. IEEE Computer Society Press, pp 168-177, 1993.
• The Ethics of Safety-Critical Systems, Jonathan Bowen. Invited talk at ENCRESS'97, 3rd International Conference on Reliability, Quality & Safety of Software-Intensive Systems, Athens, Greece, 29-30 May 1997.
• Safety-Critical Systems, Formal Methods and Standards, Jonathan Bowen and Victoria Stavridou. IEE/BCS Software Engineering Journal, 8(4):189-209, July 1993. Previously issued as a Programming Research Group Technical Report PRG-TR-5-92. A revised version is available as Chapter 1 in Towards Verified Systems. Winner of the IEE Charles Babbage Premium award, 1994.
• The Industrial Take-up of Formal Methods in Safety-Critical and Other Areas: A Perspective, Jonathan Bowen and Victoria Stavridou. In J.C.P. Woodcock and P.G. Larsen (eds.), FME'93: Industrial-Strength Formal Methods, First International Symposium of Formal Methods Europe, Odense, Denmark, 19-23 April 1993, Springer-Verlag, LNCS 670, pp 183-195, 1993.
• Formal Methods and Software Safety, Jonathan Bowen and Victoria Stavridou. In Heinz Frey (ed.), Safety of Computer Control Systems 1992 (SAFECOMP'92), Proc. IFAC Symposium, Zürich, Switzerland, 28-30 October 1992, Pergamon Press, pp 93-98, 1992.
• Formal Methods for the Specification and Design of Real-Time Safety Critical Systems, J. Ostroff. Journal of Systems and Software, 18(1):33-60, April 1992.
• Safety-Critical Software: Status Report and Annotated Bibliography, P. Place and K. Kang. Technical Report CMU/SEI-93-TR-5, Software Engineering Institute, Carnegie Mellon University, USA, 1993.
• Formal Specification and Verification for Critical Systems: Tools, Achievements and Prospects, John Rushby (SRI). In Electric Power Research Institute TR-100294, pp 9-1 to 9-14, January 1991.
• Formal Verification of Algorithms for Critical Systems, John Rushby (SRI). IEEE Transactions on Software Engineering, 19(1):113-233, January 1993.
• Formal Methods and the Certification of Critical Systems, John Rushby (SRI). SRI Technical Report CSL-93-7, December 1993. (300 pages) See also CSL-95-1.
• Formal Methods and their Role in the Certification of Critical Systems, John Rushby. SRI Technical Report CSL-95-1, March 1995.
  This is a shorter (50 pages) and less technical treatment of the material in CSL-93-7. It will become a chapter in the FAA Digital Systems Validation Handbook (a guide to assist FAA Certification Specialists with advanced technology issues).
• New Directions in Software Safety: Causal Modelling as an Aid to Integration, Peter Fenelon and John A McDermid. High Integrity Systems Engineering Group, Department of Computer Science, University of York, Heslington, York YO1 5DD, UK. See also other papers and reports by Pete Fenelon.
• Risk management is for everyone, Felix Redmill. IText 1(2), British Computer Society and Oxford University Press, 1995.
• The list of publications held at NASA Langley produced under NASA sponsorship since 1989 by various organizations, together with some on-line publications may be of interest. An overview of the NASA Langley programme with a good selection of references is available.
• Papers on safety-critical systems in computer science journals (authors/titles only) and technical reports (abstracts).
• Software Risk publication abstracts (searchable) from Research Access.
• DOD-STD-2167A Military Standard: Defense system software development.
• How numbers are tricking you by Arnold Barnett in the MIT Technology Review magazine.
• Safety-Critical Systems Computer Language Survey Results by Robin Rowe, posted on comp.software-eng, etc., November 1994.
• Fault Detection, Supervision and Safety for Technical Processes 1994, edited by Ruokonen, Pergamon, 1994.
• Fault Detection, Supervision and Safety for Technical Processes 1997, edited by Ruokonen, Pergamon, 1997. In preparation.
• Safety, Reliability and Applications of Emerging Intelligent Control Technologies, edited by Ng. and Hung, Pergamon, 1995.
• Publications on safety from a searchable catalogue by Elsevier.
• Safety Related Computer Controlled Systems Market Study by Coopers & Lybrand, HMSO, 1992.
• 00-55, 00-56 and other Defence Standards from the UK Ministry of Defence (MOD) Directorate of Standardization. See also 00-56 and Computer Guidance.
• System and Software Safety in Critical Systems, Ulla Isaksen, Jonathan Bowen and Nimal Nissanke. Technical Report RUCS/97/TR/062/A, Department of Computer Science, The University of Reading, UK, 1997. Also in PDF format.

Search for publications on safety-critical systems and see the Dependability References Database in The Collection of Computer Science Bibliographies.

Journals

• Control and Instrumentation, covers hardware and software for process engineering (much of which is safety critical). Editor: Brian Tinham, 30 Calderwood Street, London SE18 6QH, UK (tel: +44-181-855-7777, fax: 0181 316 3422). Assistant Editor: Jeremy Jones, email cz72@cityscape.co.uk.

• High Integrity Systems, published by Oxford University Press, vol 1 nos 1-6, 1994-1996. An excellent forum for papers on safety-critical systems, now sadly ceased publication. Editor: Prof. A.D. McGettrick, email adm@cs.strath.ac.uk.

• Reliability Engineering & System Safety from Elsevier Applied Science, Oxford, is edited by G.E. Apostolakis, School of Engineering and Applied Science, University of California, 38-137 Engineering IV, Los Angeles, California 90024-1597, USA (fax: +1-310-206-2302, no known email address).

• International Journal of Reliability, Quality and Safety Engineering from World Scientific.

• International Journal of Risk & Safety in Medicine from Elsevier, Amsterdam.

• Safety-related journals in the Directory of Computing Science Journals.

• Journals relevant to formal methods.

Books

• Safeware: System Safety and Computers, by Nancy G. Leveson. Addison-Wesley, 1995. ISBN: 0-201-11972-2, $49.50.

• Computer Related Risks, by Peter G. Neumann, SRI, ACM Press Books ( ACM Press / Addison-Wesley), 1995. ISBN: 0-201-55805-X, $24.75. See also Risks Forum.

• Software in Safety Related Systems, by Brian A. Wichmann, NPL, Wiley, 1992. ISBN: 0471-93474-7.

• Safety-Critical Computer Systems, by Neil Storey, Addison-Wesley, 1996. ISBN: 0-201-42787-7.

• High Integrity Ada: The Spark Approach, by John Barnes with Praxis Critical Systems, Addison Wesley Longman, 1997. ISBN: 0-201-17517-7. (Includes CD-ROM.)

• Software Safety and Reliability: Techniques, Approaches, and Standards of Key Industrial Sectors, by Debra S. Herrmann. IEEE Computer Society Press, 2000. ISBN: 0-7695-0299-7.

• High-Integrity System Specification and Design, by Jonathan Bowen and Mike Hinchey. Springer-Verlag, 1999. ISBN: 3-540-76226-4.

• Safety-critical books listed by Amazon.com USA site.
• Safety-critical books listed by Amazon.co.uk UK site.

Companies

• Adelard, London, UK. Consultancy in the area of dependable systems, including safety cases. See a useful list of safety-related links, including standards.

• Aonix, San Francisco, California, USA. See safety-critical solutions using Ada.

• Critical Software Lda, Coimbra, Portugal. See links.

• CSE International Ltd (The Centre for Software Engineering), Flixborough, UK. Consultancy in computer-based high integrity and safety-critical systems and their software. Chairman and Managing Director: Phil Bennett.

• ERA Technology, UK. System safety engineering services, independent safety consultancy, safety design expertise for electronics and programmable electronic systems.

• Praxis Critical Systems, Bath, UK. Specialises in all aspects of high-integrity systems. See SPARK, an integrated approach to the design, development and verification of complex software.

• Programming Research Ltd, Surrey, UK. Software quality. See QA C and the MISRA Standard. See also MISRA.

• Redifon MEL, UK. See Air Traffic Control.

• Safeware Engineering Corporation, Everett, WA, USA. Chief Executive Officer: Nancy G. Leveson.

• TekSci, Seattle, WA, USA.

Meetings

• Australian Workshop on Safety Critical Systems and Software, organized by the Australian Computer Society's Technical Committee on Safety Critical Systems. SCS'99, 4th Workshop, Canberra, Australia, 26 November 1999.

• CHI'98 (Workshop on Designing User Interfaces for Safety Critical Systems) Los Angeles, USA, 18-19 April 1998.

• COMPASS (Annual IEEE Conference on COMPuter ASSurance).
• CSR events, Centre for Software Reliability, UK. Includes related events.
• DCCA-7 (7th IFIP International Working Conference on Dependable Computing for Critical Applications) San Jose, California, USA, 6-8 January 1999.

• ESREL'98 (European Safety and Reliability Conference) Trondheim, Norway, 17-19 June 1998. Supported by ESRA.

• High Assurance Systems Engineering Symposium (HASE).

• 16th International System Safety Conference, Seattle, Washington, USA, 14-19 September 1998.

• ISESS'97 Workshop on Computer Related Standards and Safety, ISESS'97 (International Symposium on Software Engineering Standards) Walnut Creek, California, USA, 1-6 June 1997. See also ISESS'99, Brazil, 17-21 May 1999.

• PRFTS (Pacific Rim International Symposium on Fault-Tolerant Systems).

• SAFECOMP (Annual IFAC Symposium on SAFEty of COMPuter Control Systems).
  • SAFECOMP'89 proceedings.
  • SAFECOMP'90 proceedings.
  • SAFECOMP'91 proceedings.
  • SAFECOMP'92 proceedings.
  • SAFECOMP'93 proceedings.
  • SAFECOMP'94, the 13th International Conference on Computer Safety, Reliability and Security, Anaheim, California, USA, 23-26 October 1994. No known on-line information available.
  • SAFECOMP'95, the 14th International Conference on Computer Safety, Reliability and Security, Villa Carlotta, Belgirate, Italy, 11-13 October 1995.
  • SAFECOMP'96, the 15th international Conference on Computer Safety, Reliability and Security, Vienna, Austria, 23-25 October 1996.
  • SAFECOMP'97, York, UK, 8-10 September 1997.
  • SAFECOMP'98, Heidelberg, Germany, 5-7 October 1998.
  • SAFECOMP'99, Toulouse, France, 27-29 September 1999.

• SAFEPROCESS'91 proceedings.

• SSS (Safety-critical Systems Symposium). Annual UK Safety-Critical Systems Club symposium. Contact Joan.Atkinson@newcastle.ac.uk for further information.

• 2nd Workshop on Human Error, Safety, and System Development, Seattle, Washington, USA, 1-2 April 1998.

Safety-critical systems and the World Wide Web, by Jonathan Bowen. Safety Systems: The Safety-Critical Systems Club Newsletter, 4(2):16, January 1995.

Final thought

"If the automobile had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside."

- Robert X. Cringely

Part of the OUCL archive.