Safety Analysis for System Development - Essay Example

ID 19714 Academia Research Order No. 239717 03 September 2008 Safety Analysis for System Development Table of Contents: Introduction: Process Control & Engineering are highly dependent on IT systems & software applications in the modern era. Automation of system processes has put a lot of demand on the underlying IT systems and software applications to the extent that there can be major hazards and accidents if there is an outage of one or more IT components that are responsible to control the critical states of the overall process flow. Criticalities of System uptime, System Performance, Human & Property Safety considerations and consistency in data captured & recorded from controlled devices are major result areas that are required across various automated systems. Such challenges put a lot of pressure on the software development functions which are tasked to develop and maintain critical software systems & applications. One of the biggest challenges faced by the development team is management of hazards in critical software implementations for control systems and automation. The objective of this paper is to present a detailed analysis of challenges of System Safety, analysis of hazards, techniques of implementing System safety and global best practices followed. Mueller in 1968 described System Safety Engineering as an "organized common sense" (Leveson, 2003). Quoting this comment in her paper on safety engineering, Nancy Leveson (2003) stressed on the need for a disciplined and systematic approach to identify, analyze and control the hazards throughout the life cycle of a system (Leveson, 2003). She proposed a systematic approach of safety engineering in this paper. The steps of her approach will be taken as benchmark and mapped with the modern approach to System Safety in developing Software for Critical Systems in this paper. Risk Management: Nancy Leveson emphasized the need for Risk Management as one of the major disciplines in Safety Engineering (Leveson, 2003). Risk Management itself is a very vast system requiring proactive analysis of threats to business, vulnerabilities within the system, impact to business, analyzing the overall risk and the mitigation strategies. There is a high probability of the Software deviating from the System Requirements or becoming vulnerable to hackers & unauthorized modifications in production environment if improper controls are practiced in the development environment. Example, If the software is supposed to control electro-mechanical devices then vulnerabilities and unauthorized modifications in the software system may lead to hazards, accidents, loss of property and loss of mission in the operating environment. There can be many approaches to Risk Management in developing Software for critical systems. The most appropriate Risk Management approach applicable in the modern Software Development environments is defined in the Risk Management guide by National Institute of Standards & Technology, US Department of commerce (Stoneburner, Guguen, et al, 2004) and the BS ISO/IEC 27005:2008 standard (www.bsi-global.com). The approach presented herewith (Figure 1) can be very easily mapped with a software development project. This process is an intelligent mix of qualitative as well as quantitative analytical processing. The first step is to collate a list of all assets planned to be used in the software environment and then carry out their characterization. Risk Assessment Workflow Figure 1 The assets used in a software controlled critical production environment are: Software Workflows, Software Components (Units, Modules, Connectors, etc.), Servers, Desktops/Laptops, RDBMS systems, Middleware, Interfacing devices, Control devices, High Availability components, Underlying Network Architecture, Alerts & Alarm systems, Network Integration components (example, TCP/IP to RS232 converters), etc. The characterization of these assets essentially requires proper identification (asset tagging), asset ownership, purpose of asset and location of asset. Post characterization, the asset value is calculated which is a function of Confidentiality, Integrity and Availability (BS ISO/IEC 27001:2005 standard). Following is a simple chart to calculate the Asset value: Confidentiality (C) Rule 1 Publicly accessible assets (e.g., anyone can gain access to the asset) 2 Assets accessible to limited people (e.g., all members of control group) 3 Assets accessible to restricted no. of people (e.g., only Supervisors) Integrity (I) Rule 1 System Integrity or Unauthorized modification causes minimal impact 2 System Integrity or Unauthorized modification causes manageable impact 3 System Integrity or Unauthorized modification causes serious impact Availability (A) Rule 1 Asset downtime causes minimal impact 2 Asset downtime causes manageable impact 3 Asset downtime causes Serious impact ASSET VALUE Highest value among C, I and A. The highest among C, I, and A is taken as asset value because all three parameters carry equal importance for an asset. The Threat value is a product of Impact Value and Probability Value. Calculation of Impact value is carried out against various threat scenarios in the software, like Systems, Environment, Technology, People, Processes, Security, Litigation, Politics, Sabotage, Terrorism, etc and all the corresponding impacts are taken into account, like functional specification deviation, failure of input/output components, Security impact, Environmental Impact, Health & Safety impact, Reputation impact, Financial impact, Regulatory Impact, etc. Every impact has a potential to cause a set of known and unknown hazards that can potentially affect the final deliverables of the Software in terms of Loss of Mission (e.g., loss of Services, Loss of market value, Litigation, Business disruption/shutdown), Loss of Human Life, Loss of Property, etc. There can be numeric levels defined in every impact -example, Failure of Low Critical Components - low impact (value 1) Failure of Medium Critical Components - medium impact (value 2) Functional High critical components - high impact (value 3) The overall impact is presented as a function of every impact taken into account restricted within the three values - 1, 2 and 3 to maintain consistency. Organizations can maintain more than three values but they need to be consistent in the number of values chosen. Probability Value calculation is done against an evaluation of existing controls and vulnerabilities. It again gets three values, 1, 2 & 3 against low, medium and high probabilities respectively. The vulnerability value is the values assigned to the evaluation against existing controls and vulnerabilities. The Risk Value finally is the function of Asset, Threat, Probability and Vulnerability Values. The calculation of Risk Value can be decided internally by the organization - it may be a sum or an average of all these values. For treatment of risks, an organization can have a color coding system. Example: Values 1 to 4: Risks in Green; action - risks accepted; Values 5 to 7: Risks in Orange; action - risks that need to be closely managed Values 8 and above: Risks in Red; action - risks needing immediate mitigation This analysis gives a clear cut idea on where the assets of the Software controlled environment stand in line with existing people, processes and technologies. Based on this analysis, assets needing maximum attention can be identified and risk mitigation implemented. There is no short-cut to this process because risk values are totally dependent upon the current scenarios in the Software environment and do not possess finite values which are industry standard. Moreover, effectiveness of controls may vary due to changing scenarios. Hence effectiveness of controls needs to be measured periodically to verify if they are meeting the System Safety requirements. Whenever the scenarios in the software environment changes, the Risk values and corresponding effectiveness of controls will also change, and hence the entire process needs revisiting at periodic intervals. Modern Practices of Software Development to cater to System Safety requirements: System safety considerations take into account Loss of Mission, Loss of Human Life, Loss of Property, and Loss to Environment. In this section, a detailed evaluation of the Software Development Life Cycle (SDLC) of a critical software system has been presented by enhancing the conventional SDLC to meet the requirements of the System Development part of the Model of Socio-Economic control presented by Nancy Leveson. According to the model, When a software development group is engaged for developing a Software to be used in highly Critical environments, the following demands (a partial list only) are put on the development group by the Stake Holders to ensure reduction probability of hazards in the production environment to be controlled by the software (Steele, 2001; BS ISO/IEC 27001:2005 standard; BS ISO/IEC 17799:2005 code of best practices; Leveson, 2004)): (a) Accurate documentation, version controlling, multi-level review and security of the following: i. System Requirement Specifications (SRS): ii. Technical Requirement Specifications (TRS) iii. Safety Requirement Specifications and Hazard Analysis reports iv. Compliance requirements to legislation and government reports v. Safety Policy standards in the company vi. Risk Assessment report vii. Separate documentation identifying system safety design constraints viii. High Level Design ix. Low Level Design x. World Class Coding Standards xi. Project & Resource Management plans xii. Project Environment description xiii. Module Release process xiv. Budget and Effort trackers xv. All operating processes, documents and logs xvi. Change Management Plan (includes testing plans, deployment plans, make-live plans and training plans) (b) Protection of Project Assets (c) Code Repository Management - secured storage of codes at various stages of Software development, protection against unauthorized modifications, check-outs of codes via Software Configuration Management only, and backup/recovery-testing of codes. (d) Managing compliance against the established coding standards; review coding standards periodically to meet global best practices. (e) Project reviews and third party auditing with corrective actions taken and preventive actions planned (f) Iterative method of enhancements (Plan - Do - Check - Act) to continuously learn from mistakes and take corrective/preventive actions (g) Strict adherence to functional specifications; any deviations to be governed by Software Change Management only. (h) Segregation of development and test environments, such that developers cannot carry out unauthorized modifications in the codes already gone into testing. (i) Input and Output Code validation - every line of code reviewed before entering a module or transitioning from component building to component testing stage. (j) Restricted usage of Shareware or Freeware - proper analysis and approval of freeware and shareware allowed to be used in development. (k) Technical Vulnerability Testing and Control, i.e., Vulnerability Analysis and Penetration Testing (l) Incident Management and Root Cause Analysis (m) Strong Access control to development & test environments and program source codes (n) All coding to be done in original; incorporating ready to use codes from Internet is strictly prohibited in critical software development. Such codes may consist of Trojan codes and covert channels which may lead to an exploiter taking control of a system after gaining access to the network. (o) Disaster Recovery and System Continuity of the Software development assets and environment Every Software development project follows a structured Software Development Life Cycle (SDLC), irrespective of the criticality (Steele, 2001). Managing an SDLC for critical applications supposed to deliver effective System safety requires lot of extra efforts in addition to the conventional software project management. Waterfall Model is preferred over Rapid Application Development (RAD) in critical software development (Steele, 2001) because Software Change management is very strictly governed and codes are frozen for further development in a component when transitioned into the testing stage. A Critical Software Development Life Cycle typically comprises of the following stages after the project is approved and development triggered: (a) Software Change Management: Change Management in software development is very strictly followed. Once the SRS & TRS are frozen by the Stake holders, any desired modifications will go through a change management board that in turn will carry out a thorough analysis to ensure that the new modifications are very essential for the system and will not cause any hazards in the bigger picture. Once such changes approved, the development team will incorporate them only in the next cycle of component coding. The current cycle is not disturbed unless it is decided that not deploying the changes itself will cause hazards. (b) Component Coding: Every developer carrying out coding in their respective development environments and saving the codes developed by end of the day in the code repository software. Common code repository software used in the Industry are IBM Rational Clearcase, Microsoft VSS, CVS (freeware), Subversion (freeware), etc. (c) Component testing: Once the finished codes are reviewed by the code reviewers (for a strict adherence to coding standards), they are transitioned into the Component Testing stage (a separate area in the code repository). Testing is carried out against pre-documented and approved test plans which are in accordance with the SRS and TRS. The codes which pass are allowed to transition into the Component Integration testing stage. Codes that fail are sent back to developers. (d) In the Component Integration testing (CIT) stage, integration aspects of all modules are tested as per a CIT plan. CIT adheres to very strict rules and guidelines. As a matter of fact, codes from this stage also can be sent back to developers. Codes passing this stage are allowed to transition into System Testing phase (a separate area in the code repository). (e) System testing is a very expensive affair and is carried out in multiple sub-stages - like, load simulation testing, data flow testing, control flow testing, use case testing, etc (Copeland, 2003). In non-critical software development normally only functional testing is provided. However, in critical software development, all the test cycles are followed comprehensively. In the entire SDLC, this is the biggest opportunity that the Project Manager gets to enhance system safety considerations and reduce probability of hazards as much as possible. The test environments simulated will possess test cases for Systems Safety in addition to the other attributes. Although rare, but codes from this stage can also be sent back to developers if deemed essential. All the testing exercises are carried out in presence of experienced Software Quality Assurance (SQA) professionals. SQAs finally sign-off the tests if they qualify to a pre-defined checklist which again is very stringent looking in accordance with the criticality level of the software. The criticality level parameters are an outcome of impact values calculated in the Risk Assessment exercises. (f) Once the system testing cycles are over, the codes are transitioned into the Operations Acceptance testing (OAT). This is a testing that is carried out in simulated environments that are exact replica of the targeted production environments. These tests are carried out in presence of the IT and Process Control Services function. These functions will later on be responsible to run the software in production environments. The test conducted verifies all operational, management, hazard management, safety rules, disaster recovery, system continuity and security aspects of the software as it would behave in the production environment. This stage is the last opportunity for the testers to take measures to reduce probability of hazards as much as possible before the systems are exposed to human beings in production environment. (g) Once OAT in simulation environment is carried out with satisfactory results, the software goes through a structured make-live process. In this process, step by step implementation of the software is carried out in the production environment. If essential, a scheduled outage is taken after approval from the Change Management board to run mock exercises on safety and other controls that have been built. All the essential OAT test cycles are repeated here. A failure will lead to roll-back of the codes returned to development. A pass will allow the production environment to be released to users. (h) The final step is to carry out post make-live activities, like, user training, administration training, safety training, checking real world problems, etc. These bugs are solved by the application production support team in consultation with the development team. Safety considerations at the design stage: If the safety aspects are considered at the design stage itself, the probability of occurrence of hazards will reduce considerably (Leveson, 2003). The analysis needs to be very comprehensive covering every planned asset for the Software. For critical software systems, the entire design should be discussed and prepared by a group of specialists pertaining to Hardware, Software Platform, Operating System, Networking and Application Workflow. The design considerations for the end to end system shall include functional efficiency, ease of operation, ease of maintenance, high availability architecture, risk of obsolescence, external support availability, skilled manpower availability, data consistency (while creation, transit, storage and destruction), disaster recovery and system continuity. A detailed analysis of System Safety considerations pertaining to the Software Development includes the following: (a) Requirements Validation: Incorrect or vaguely understood requirements form a major root cause of hazards in the production environment controlled by the software (Leveson, 2002). Flawed requirements normally occur due to wrong assumptions about operating of the control system and unhandled/ignored control system states. In some designs, system states pertaining to Safety may not have been included at all. The System architect needs to validate all the specifications, states and flows and ensure that adequate system safety criteria has been included at appropriate places. Example, if the software is supposed to control an automatic door, the sensor discovering a human being or another object between the doors should possess high availability and if it fails anyway, the door closing signals should stop immediately. (b) Software Architecture: The software architecture should have a very clearly documented to define the functionalities mapped with the requirement specifications. Deviations from functionalities are taken as serious impacts in the Risk Assessment presented earlier. The functionalities ensuring System Safety and identified separately and analyzed in detail separately involving the safety experts of the system that needs to be controlled. All the known hazards, known human errors, known malfunctions, etc. should be considered due diligently to ensure that no angle of safety consideration has been left out. It should be appreciated that the software need not be safe even if all components are working correctly. (c) Safety of chosen software platforms: Known vulnerabilities of the chosen operating system and development platforms are analyzed to evaluate the risk of exploit by known and emerging threats. Knowledge of the existing exploits, cases of malfunctions, failure events and unsolved problem areas (known bugs) pertaining to the software tools, OS, database, middleware etc. is very useful in assessing the corresponding risks and applying mitigation strategies in development environments. Moreover, detailed analysis is conducted to evaluate safe operation of the software platform on the various hardware and networking platforms. Analytical reports like Hardware Compatibility List, Software Inter-operability, Middleware compatibility, Interface compatibility, Database compatibility, Number of expected transactions, System load estimation, etc are prepared to support the decision of choosing an OS, development platform, integration components, process control devices, middleware, Relational Database Platform, etc. (d) Safety of chosen hardware platforms: The chosen hardware platform depends upon the approved Hardware Compatibility List. Special emphasis should be given to optimum configuration that shall support the software environment keeping the safety considerations intact. Example, if the risk assessment has necessitated the need for cryptography of .jar files from the software to the control device, then an additional load of cryptography is planned on the Processor, RAM and hard Disk Storage of the servers over and above the load that the software development platform will generate. (e) Safety in network infrastructure: Underlying network infrastructure is the most vulnerable source of failure and exploits. Failure of interfaces, Malware attacks, exploits, etc are threats which happen at the networking level. Hence, the best possible protection and safety strategies are planned in the network architecture. Redundancies, Automatic Fail-over, Access Control, Firewalls, Flow control, Intrusion Prevention, etc. are strategies that need to be established at the network architecture level. (f) High Availability: High availability is required to be designed in all the components where a failure can lead to an accident, loss of property or complete system shut down. The objective of High Availability is that a hot-standby system takes over the rest of operation in a flow of states in order to complete a control logic being executed. High Availability is implemented at control logic servers, control devices, network core, critical connections, and critical software components. Various high availability solutions are available from manufacturers in a software controlled environment. EMC, Veritas and IBM are some manufacturers who have created extensive solutions to ensure that the hardware and software framework of the system possesses adequate hot standby integrated via alternate routes of network connectivity. Effectiveness of a High Availability design depends upon the expected downtime. Following is a chart that measures availability (Han, 2005): Uptime Downtime/Year Downtime/week 98.00% 7.3 days 202 minutes 99.0% 3.65 days 101minutes 99.50% 43.8 hours 50 minutes 99.9% 8.76 hours 10 minutes 99.99% 52.6 minutes 1 minute This chart reveals that a system running 24 X 7 may actually require availability of 99.999% or above. The high availability design with such an expectation might need multiple levels of redundancy and not just a 1+1 redundancy. Example, an International jet plane can easily fly with one engine but is provided with four engines in a hot standby mode. Similarly, in the software system implementation for a process control engine having high probability of hazards may need 1+2 or 1+3 level of redundancy depending upon the known mean-time-between-failure (MTBF) statistics. Safety Analysis for Hazard Management in development of critical systems: The safety framework for managing hazards shall require the following steps (Leveson, 2002): (a) Prepare a comprehensive list of all known hazards, system malfunctions, threats and vulnerabilities (b) Translate this knowledge into detailed system safety design constraints (c) Assess the hazards in detail - evaluate the probability of emerging hazards that may not have happened earlier in the system in consideration but have happened somewhere in the world on a similar system (d) Prepare a Hazard log (e) Map the mitigation of risk of the logged hazards with the design constraints applied. The design of the flow of control logic is carried out using a state diagram in the process control system. The corresponding diagram in the software controlled flow is the UML (Unified Modeling Language) State diagram and Sequence diagram. The present state in the state diagram will change to a next state after the application of the control logic by the software. Such control signals are in the form low voltage or current inputs to the controlling devices. When looking at the state diagram carefully, the architect should think of instances when a human body or a foreign object comes in contact with a moving part of the machine and plan for reversing the state back to previous state after getting a sensor input. Example, a sensor deployed very close to a rotating wheel such that if a human body comes closer to the wheel beyond a threshold, the wheel automatically stops. This example presents a scenario when a hazard could potentially occur without a system failure. The stopped wheel would definitely stop the system but will save a human life. Another scenario can be that the wheel is stopped as soon as the vibrations sensed in the axle increases beyond a threshold. This design constraint can prevent a major accident if the axle breaks while the wheel is at its full speed. The message conveyed here is that hazards do not always happen due to system failures and hence the design constraints should be carefully applied much beyond the considerations for reducing system failures only. Reduction of occupational hazards due to system failures can be carried out using the Fault Tree Analysis (a graphical representation of a logical structure representing failures and their causes) methodology. However, reduction of occupational hazards in stable and reliable systems require an organized common sense keeping in view probability of every hazard that can occur irrespective of whether the system fails or not. Proactive monitoring of the dashboards on the software console presenting accurate statistics on the actual events happening in the system is an effective way of reducing probability of failures. The modern dashboard systems carry out proactive analysis of event statistics against pre-defined rules and raises alarms if the trend appears to increase the probability of a failure. Conflict management in System Safety: Sometimes, the design constraints for system safety may conflict with functional, operational and sometimes political goals of the desired system. Such situations should be managed by arriving at a two way compromise but definitely giving more emphasis to safety. Nancy Leveson and Joel Cutcher analyzed the shortcomings of System Safety in the Columbia accident. They stressed upon the fact that prioritizing safety is more of a cultural issue rather than an engineering issue. People tend to become complacent and over-confident towards the safety considerations as system matures and appear to be reliable in the business as usual mode. Focus tends to get aligned more towards reduction of failure than reduction of probability of hazards. As a result, the hazard analysis itself is not comprehensive. Change management is focused more towards operational efficiency whereby safety is more perceived than planned, implemented and managed. To assure the desired level of emphasis to system safety in the scenarios of multiple conflicts, following practices should be ensured: (a) System Safety should not be substituted by reliability engineering i.e., focus only on reducing failures of components and not of occupational hazards. (b) Established standards should not be allowed to dilute at all. It is a good idea to get the practices audited by independent third party auditors. (c) Implement global best practices. OHSAS 18001 standard is a good framework that can be implemented to ensure that the level of focus to System Safety is maintained at a desired level. This standard is internationally accepted and is certifiable. It covers a wide range of scenarios where probability of hazards are high - hot water systems, sprinkler systems, high noise zones, high temperature systems, general plant & machinery systems, compressed gas cylinders, high voltage/high current electrical equipment, cranes, hoists, platforms, etc. (d) Software based safety monitoring on the system is effectively implemented. (e) Special care should be taken to ensure that System Safety doesn't become cosmetic. Conclusions: System Safety practices take into account appropriate methodology and controls to protect against hazards, accidents, loss of property and loss of mission. Safety engineering in systems design needs an in-depth definitive approach with an intelligent mix of qualitative and quantitative analysis. A structured Risk Assessment will present threats and the corresponding impacts that can be evident pertaining to occurrence of hazards, non-compliance to regulations, loss of property, loss of mission, major incidents, loss of human life, loss of reputation, loss of business, etc. The threats, vulnerabilities and risk values are used to assess the appropriate controls in the system safety considerations when designing the software for the control of the overall engineering process. The conventional Software Development Life Cycle (SDLC) processes require a lot of additional considerations that Nancy Leveson has described in the System Development part of the socio-technical model. The SDLC processes need to take into account Risk Assessments, Hazard Analysis, Safety Regulations, known incidents & accidents, test cases related to safety, safety specialists included in the overall architecture & designing, tightly controlled & secured source code generation, much longer cycles of testing, and verification of hazards/failures before systems are allowed to go in production environment. Most of the safety issues get inducted during the design phase and hence special care needs to be taken to include safety in the overall design. System safety is a cultural aspect whereby people tend to become complacent and over-confident towards safety. In such a case, external quality standards like OHSAS 18001 can be adopted and the implementation audited by third party auditors. References: Harvard - A Guide to referencing, Victoria University 2002, A new school of thought, Australia Leveson, Nancy. White paper on approaches to Safety Engineering. 2003 Kakadia, Deepak and Halabi, Sam et al, Enterprise Network Design Patterns: High Availability, SUN Microsystems, CA, USA, 2003 Han, Yan. An Integrated High Availability Computing Platform. 2005 Olzak, Tom. Strengthen Data Protection with Network Access Controls. 2006 Copeland, Lee. A Practitioner's Guide to Software test design, Artech House, 2003. Secure Software development Life Cycle Processes, U.S. Department of Homeland Security, viewed 04 September 2008, Parkinson, Paul and Kinnan, Larry. Safety Critical Software Development for Integrated Modular Avionics. Wind River. 2007 Steele, John. The Software development Life Cycle. 2001 Leveson, Nancy. Software System safety. 2002 Leveson, Nancy. A Systems Theoretic Approach to safety in Software Intensive Systems. 2004 OHSAS 18001 Occupational Health and safety. 4 September 2008. Fault Tree Analysis. 4 September 2008. Stoneburner, Gary and Gogien, Alice et al. Risk Management Guide for Information Technology Systems. Recommendations of the National Institute of Standards and Technology, US Department of Commerce. 2002 Safety First - Avoiding Software Mishaps. 4 September 2008. < http://www.embedded.com/2000/0011/0011feat1.htm> Safety Critical Products. 4 September 2008. < http://www.ghs.com/products/safety_critical/index.html> Galloway, Andy and Iwu, Frantz, et al. On the formal development of Safety-Critical Software. Department of Computer Science. University of York, UK Leveson, Nancy and Gershenfeld, Joel. What System safety engineering can learn from Columbia Accident. In addition to the cited references, I would like to extend my special thanks to all those who extended to me knowledge and information that helped me to put together this paper. On their request, their names have not been published herewith. Bibliography: Sohn, S.D and Seong, Hyun. Reliability Engineering and System Safety. Elsevier Publications, Redmill, Felix. Components of System safety. Springer Publishing, 2002 Redmill, Felix and Anderson Tom. Development in Risk based approaches to safety. Springer Publisher, 2006 Redmill, Felix and Anderson Tom. Improvements in System safety. Springer Publisher, 2008 Redmill, Felix and Anderson Tom. Practical Elements of safety. Springer Publisher, 2004 Leveson, Nancy. Safeware: System safety and computers. Addion - Wesley, 1995 Wysocki, Robert. Effective Software Project Management, Wiley, 2006 Everett, Gerald. Software Testing - testing across the entire software development life cycle. Wiley IEEE Computer Society. 2007 Merkow, Mark and Breithaupt, James. Information Security - Principles and Practices. Prentice Hall, 2005 Peltier, Thomas. Information Security Risk Analysis. Auerbach publications, 2005 End of Document Read More