Multi-Criteria Decision Analysis for Conceptual Ship Design

TWI Ltd, Granta Park Great Abington, Cambridge, CB21 6AL, UK

A framework for multi-criteria decision analysis (MCDA) applied to conceptual stage of ship design

P.Zhou
University of Strathclyde, Glasgow, UK

Paper presented at IMAM 2017 (International Maritime Association of the Mediterranean); Lisbon, Portugal , 9 -11 October 2017.

Abstract

To support SME (small and medium enterprise) naval architects, shipbuilders and ship owners make the appropriate decision that reduces the time and cost while improving the production at the ship’s conceptual design stage, there is a need for rational decision support approaches that consider not only economic but also environmental and risk factors. This research proposes a Multi-criteria Decision Analysis (MCDA) framework to evaluate different decision criteria, which integrates the evaluation of life-cycle cost analysis, environmental impact and risk assessment. The MCDA framework will support decision making from a ship lifecycle or through life perspective. As part of a larger project –SHIPLYS, this approach will support the evaluation of different design options for the three scenarios predefined in the project. In this paper, the MCDA framework with particular focus on uncertainty treatment and sensitivity analysis will be introduced followed by a demonstration of its application to a predefined scenario.

1. Introduction

SHIPLYS (Ship Lifecycle Software Solutions) is a three-year project that started in September 2016 with funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 690770. The project is in response to needs of SME (small and medium enter-prise) naval architects, shipbuilders and ship owners, who, in order to survive in the world market, need to: improve their capability to reduce the time and costs of design and production; reliably produce better ship concepts through rapid virtual prototyping; and meet the increasing requirements for LCCA (Life Cycle Cost Analysis), environmental assessment, risk assessments and end-of-life considerations as differentiators.

SHIPLYS aims to build on existing experience in the shipping sector and transfer experience from the development of life cycle modelling and rapid virtual prototyping in other industry sectors. A key out-come of the project is the development of “SHIPLYS Life Cycle Tools (SHIPLYS LCT)”, which is the suite of tools providing LCCA, environmental assessment, risk assessment, and multi-criteria decision support functionalities. SHIPLYS is focused on the conceptual stage. The “conceptual stage” in the context of SHIPLYS means the very initial design that shipyards need to arrive at in response to a tender to estimate the scope and cost of work involved in a reliable way within the limited time that they have. 

In this paper, an MCDA framework that has been developed as a starting point for this research work is first presented. This is followed by a review of certain MCDA techniques that can potentially be applied to the framework. Then, the treatment of uncertainty in the data used as well as the sensitivity analysis in the context of MCDA is reviewed.

It is important to factor in uncertainty in the analysis to make decisions based on realistic inputs that are often best described as a range of values or in terms of a distribution. Sensitivity analysis is important to identify “high impact” inputs and such analysis can then help decision makers to establish the value of getting more precise inputs for high impact variables.

2. Background

2.1 Multi-criteria decision analysis (MCDA)

MCDA is the set of tools and methods providing the mathematical methodology that incorporates the value of decision makers and stakeholders as well as technical information to select the best solution for problems and to make more logical and scientifically defensible decision (Linkov, & Moberg, 2012). The employment of MCDA concepts will improve the flexibility of SHIPLYS LCT to satisfy the various requirements from different stakeholders or decision makers. Figure 1 shows the generic framework for an MCDA workflow that has been developed and the following paragraphs describe it.

Decision context means to understand who the stakeholders are, what their objectives are, what the decision alternatives and criteria are and so on. This clarifies the baseline for evaluation, the required level of detail and complexity, as well as the available data and resources. Then, based on the decision con-text, the collected data can be processed using data sensitivity and uncertainty analysis.

Based on the baseline and data collected from the previous steps, the performance of different decision alternatives with regard to the models for comparison can be determined. In SHIPLYS, these models for comparison are aiming at minimization of the LCC, environmental impact and risks depending on the objective of various stakeholders.

Finally, all this information is fed into the MCDA module.  In the first step, the most appropriate MCDA technique needs to be determined. Then the criteria weights and alternatives scoring system is developed subsequently to account for the importance of different models for comparison, i.e. minimisation of LCC, environmental impact and risks during the lifecycle or the importance of different life cycle stages. Finally, the result is calculated. Regarding generating the results, it is important to consider the level of details required and how to present the key findings. The results can be a ranked as a list of alternatives or a range of probabilities that the alternative will be well accepted (Reiss, 2016). In addition, interpretation of these results and providing recommendations should also be included as part of this framework.

The suggested framework builds on MCDA already employed in many different fields where the simultaneous consideration of several criteria is necessary to make the final decision. One example of its utilization in marine sector has been carried out by (Begovic, Bertorello, Stella, & Barone, 2006). In this case, a multi-attribute procedure suitable to ship de-sign was presented by assessing the key criteria of powering, strength, stability and seakeeping, etc.

As introduced by (Linkov, &Moberg, 2012), de-pending on the context of the decision problem and available data, there are many types of MCDA models. The three basic categories of these models are shown below and all applied to hypothetical examples in Section 4:

• Multi-Attribute Utility Theory (MAUT), where the disparate units will be resolved into utility or value so can be comparable.
  
  The advantage of MAUT is that it provides clear choice criteria, which is particularly helpful when a wide range of perspectives and decision alternatives have to be checked on and considered. However, it may be difficult and time consuming to achieve the consensus on the attributes in the model and on the rough range of weights to be used (Cresswell, et al, 2003).

• Analytical Hierarchy Process (AHP), where the pairwise comparisons are used rather than using direct weights or value functions. Every criterion is contrasted straightforwardly with another criterion and the decision makers can make a relative judgement between the two.
  
  Moreover, the consistency analysis may also be performed to make sure that the original preference ratings were consistent. The consistency indexes capturing the frequency and level of inconsistency occurrence is acceptable below 0.1 as permissible (Saaty, 1980). For this reason, AHP is more flexible than MAUT in the requirement of the decision maker to be rational, but the paired comparison among criteria or alternatives is relatively subjective and arbitrary (Han, Xie, & Duan, 2017).

• Outranking, where the alternatives will be ordered by finding ones that outperform or dominate. There are many different models included in the outranking methods family such as ELECTRE, PROMETHEE, and GAIA etc. Based on the decision context, an appropriate method is selected.
  
  Generally speaking, the outranking method is making comparisons to support decision makers to find a solution based on their preference. This makes it more judgement-based as opposed to an optimisation algorithm that presents an optimal solution.   However, similar to AHP method, out-ranking is also concerned that the method is based on some rather arbitrary definitions of what precisely constitutes outranking and the threshold parameters that are set and controlled by the decision maker (Department for Communities and Local Government, 2009).

2.2 Uncertainty treatment

Uncertainty is a state of having limited knowledge where it is impossible to exactly describe the existing state or future outcome. There are mainly two kinds of uncertainties in data, Aleatoric uncertainty and Epistemic uncertainty. Aleatoric uncertainty is a kind of random uncertainty caused by the nature of the data. Epistemic uncertainty is the uncertainty due to lack of knowledge. Epistemic uncertainty can be reduced by deep data mining or gaining more knowledge of the data. However, this may be costly so that such uncertainty is often just described and considered as the risk in the assessment. Generally speaking, aleatory uncertainty decreases the precision of the assessment while epistemic uncertainty decreases the accuracy of the assessment (Bevington, &Robinson, 2003).

In order to account for uncertainty, for quantitative data, it can be described by using the characteristic values of the data such as mean and standard deviation. The level of confidence for this kind of data can be also evaluated by providing the maxima, minima or most likely values. If the data is qualitative, the qualitative scoring system can be designed such as high, medium and low. This method based on engineering judgement will be subjective; therefore, each optional score should be defined precisely. Alternatively, the data can be transferred to distributions and perform further analysis on the distribution, the commonly used distribution types are uniform, triangular (PERT), trapezoidal and normal distribution, etc. (GAO, 2009) as shown in Figure 2. Another possibility is to provide confidence intervals, such as one has x% confidence that the value of a variable y is within the range z. Uncertainty treatment is very important for a decision support system and a number of well-established techniques can be selected depending on the property of the data and the purpose of the treatment. For example, in the case of a purely quantitative model, such as life cycle cost (LCC) model, it is common to transfer the initially deterministic model into a probabilistic one. This can be done by modelling inputs with distribution functions, applying Monte Carlo Simulation and evaluate the results statistically (Reiss, 2016)

Another example is to use Exponentially Weighted Moving Average (EWMA) to smoothen the uncertainty in long-term historical data. This method is particularly useful when the historical data is available, and certain data requires weighted treatment and more value needs to be awarded to the recent data rather than old ones.  Figure 3 is the example for EWMA graph.  It can be seen that the EWMA value for this example for year 1995 considering values for the past 30 years is predicted to be as 15.43 per year.

2.3 Sensitivity analysis

Sensitivity analysis is a technique used to investigate how changes in input values or assumptions affect the model outputs. Depending on the characterization of input variables and the purpose of the assessment, the variation in the assessment could represent variability, uncertainty or both. Sensitivity analysis is closely linked to the overall uncertainty analysis, which provides references when trying to determine where to focus additional resources. The techniques for sensitivity analysis can be graphical or statistical. 

Generally speaking, sensitivity analysis can be mainly used in three aspects. Firstly, it can support the decision making through quantitatively identifying the most effective variables in the assessment. Based on the type of information needed, (U.S. EPA 2001) suggests the sensitivity analysis to be carried out using the tiered approach: 

• Tier 1: Sensitivity analysis is carried out by comparing the changes in the output after changing one or more inputs or assumptions. Example methods used in Tier 1 approach are sensitivity ratio, point estimate, equation inspection, etc.

The sensitivity ratio (SR) as calculated by Equation [1], also known as elasticity equation, is the most commonly used method in Tier 1 approach. The assessment will be considered most sensitive to input variable that yields the highest absolute value for SR. The SR can be further defined as local SR where small changes (~5%) will result in changes in output and range SR where the input is varied across the entire range. Usually, local SR and range SR will suggest the same results but if it differs, and assessor can conclude “different variables are driving the assessment near high-end (extreme tails) than at the central tendency region”. (U.S. EPA, 2001).  

The Tier 1 approach is a good screening tool to highlight the variables that contribute the most to the assessment but it normally requires the variables under investigation to be independent. Otherwise, Tier 2 and Tier 3 approach should be considered which will vary multiple inputs simultaneously and account for correlations. 

• Tier 2: Example methods to be used in Tier 2 approach is 1-D Monte Carlo for correlation analysis using Pearson and /or Spearman Rank, multiple Lincar regression, etc. 
• Tier 3: Example methods to be used in Tier 3 approach is similar to Tier 2, but extends to 2-D Monte Carlo, including Bayesian analysis, etc.

Furthermore, sensitivity analysis is a key feature to determine the expected value of information (EVOI) (U.S. EPA, 2001), which is looking for the balance between the importance of the input factor and cost of gaining new information about this factor. According to the information provided from sensitivity analysis, the decision regarding allocation of further resources and data collection efforts to reduce the influence of lack of knowledge can be made. Besides, in some cases when it is not possible to obtain additional data to reduce uncertainty, identifying the most influential factors to risk is still helpful for the risk assessment.

Sensitivity and uncertainty analysis should be viewed alongside each other to identify the key factors in the model. The data that is uncertain and has a high contribution towards to the results (i.e. highly sensitive) are likely to be a key issue/factor in the model. This is depicted in Figure 4.

3. SHIPLYS scenarios and the application of MCDA, uncertainty treatment and sensitivity analysis

In SHIPLYS project, three early ship design scenarios will serve as a base for the development of soft-ware functionalities that include MCDA (Bharadwaj, et al, 2017):

• Scenario 1: Optimisation of a novel hybrid propulsion system used in a short-route ferry
• Scenario 2: Development of conceptual ship design with inputs from risk-based life cycle assessments
• Scenario 3: Development of software to support early planning and costing of ship retrofitting accounting for life cycle costs and risk assessments

Implementation of the MCDA framework for each scenario will be carried out based on the generic framework presented in Figure 1.The first step of the project is to identify the decision context for each scenario including consideration of decision maker, decision objectives, alternatives and evaluation criteria. 

Secondly, for each scenario, the models for comparison in terms of LCCA, environmental impact and risk assessment will be developed specifically and the required input data collected accordingly. This step will also include the consideration of compensability and trade-off between different criteria, which will guide the final decision that which MCDA approach will be used. 

The principle of the MCDA framework in SHIPLYS is shown in Figure 5. Different scenarios will cover different life cycle stages: assessment for Scenario 1 and 3 are from production to end of life, and for Scenario 2 is from the design stage to the end of life. The early design information will be in-put by the shipyard to SHIPLYS Then according to this information and using SHIPLYS database, the LCCA, environmental assessment and risk assessment will be carried out for each life cycle stage to be criteria for comparison. After this, MCDA will be performed using these results and provide decision makers with the most optimised option. 

For each input variable, different sensitivity and uncertainty analysis methods will be performed depending on the properties of the data and the context of the assessment. In order to describe or reduce the data uncertainty, the data will be collected by requiring information such as the historical record of the variable, characteristic values for distributed data, the level of confidence, used in which scenario (s) and source of the data and so on. For the variable which is highly uncertain due to lack of knowledge, the sensitivity analysis will be performed to decide whether it is worth further investment in gaining knowledge of such data. 

Sensitivity and uncertainty analysis will also be considered when developing the weighting and scoring system for the MCDA framework. Different weighting system will be created for different scenarios. It will weigh the importance of different life cycle stages (i.e. design, production, O&M and end of life) and the importance of LCC, environmental impact and risk aspects relative to each other. The system should be flexible and adaptive to the specific requirements from different stakeholders or decision makers. Depending on the level of uncertainty and contribution to the assessment, weighting factors might be also used on these variables to account for the information provided by uncertainty and sensitivity analysis.

4. Examples

4.1 Example applications of MCDA

In this section, hypothetical examples are shown to demonstrate the application of the common MCDA techniques discussed in Section 2.1 in the context of SHIPLYS:

4.1.1 Multi-Attribute Utility Theory (MAUT)

Suppose LCC, environmental impact and risk during the ship lifecycle can be considered as attributes, and different ship design options are alternatives. The value of each alternative is evaluated based on these attributes. The attributes are assessed quantitatively and converted to comparable units such as monetary value ($, £, €) and assigned a weight that reflects their importance to the decision. Then a score for each alternative for each attribute, such as a scale of 1 – 10, is multiplied by the weight of that attribute and the total is calculated. This total represents the value of this alternative. Decision makers can com-pare the values of different alternatives and make the final decision.

One key challenge in the assignment of weights is the handling of subjective information that is provided to the decision maker. This challenge is addressed in (Ma, Fan &Huang, 1998), where a two-objective programming model has been proposed to consider the effect of both subjective and objective information. An application is shown in by (Begovic, Bertorello, Stella, & Barone, 2006).

4.1.2 Analytical Hierarchy Process (AHP)

Figure 6 shows an example of the hierarchical structure for making the decision of optimised ship propulsion system. Here, pairwise comparison between each criterion and comparison between different alternatives with respect to each criterion is conduct-ed. Figure 7 is an example how the pairwise comparison is conducted between each criterion and similar practice is conducted for each alternative with respect to each criterion. The result from this step is a number of normalised paired comparison matrixes as shown in Figure 8. Then the overall composite score of each alternative choice is computed by normalisation of linear combination of multiplication between weight and priority vector. The priority factor shows which of the alternatives is the most suitable option for certain criteria. For example, in the matrix of comparison with respect to achieving lower LCC in Figure 8, the hybrid propulsion system has the highest score, which means the hybrid propulsion system will have relatively lower LCC comparing to DM and DE system. In this hypothetical example, the hybrid propulsion system has the highest overall composite weight as shown in Table 1 so that hybrid system will be the optimised ship propulsion system. It should be noted that this is just an example to demonstrate how AHP is performed. The result may not be true in reality. Expert judgement in conducting correct pairwise comparison is essential for this method.

Table 1: Overall composite weight for different ship propulsion system

The challenge of handling subjective information is applicable in this technique too, and this can ad-dressed as mentioned in Section 4.1.1.

4.1.3 Outranking

The optimized ship propulsion system as shown in Figure 6 can be also identified using outranking method. This example will show a simple outranking method, which is similar to AHP, and in a way can present the values from AHP in a format as depicted in Figure 9. To show the similarity between AHP and outranking the same example as what in AHP is used for outranking method. Furthermore, to simplify the example, here assume each criterion is equally important, i.e. the weight for each criterion is 1. 

A decision matrix can be created as shown in Figure 9, where the scores are the priority vectors obtained from the pairwise comparisons between each alternative with respect to each criterion.   It can be seen these values correspond to the values in Figure 8.

According to this decision matrix that hybrid system “outranks” DM system and it also “outranks” DE system because of the higher overall score of such system with respect to each criterion. Therefore, it can be concluded that the hybrid system is better than DM and DE system.

4.2 Example of applications of uncertainty treatment and sensitivity analysis

The example variables in this section are chosen to demonstrate sensitivity and uncertainty analysis discussed in this paper. It should be noted that in reality, the analysis should depend on the context where these variables are used. To simplify, in this section, these example variables are analysed only in the context of production stage in a shipyard.

4.2.1 Shipyard energy consumption

The shipyard energy consumption is an important factor to calculate the production cost, i.e. highly sensitive. However, this variable is unlikely to be a deterministic value. Most probably, this will be a range of values or distributed data from historical database.  Characteristic values of the dataset such as maxi-mum, minimum, most likely, 95% upper / lower bound or average value can be then used for assessment.   In the current instance, one of the ship-yards within the SHIPLYS project uses the average value from a range of values recorded in their data-base for shipyard energy consumption (kw/day) for the purpose of cost estimation.

As the variable has both, a high impact on the output of the model in which it is used and uncertainty, it occupies the “Critical input” area in Figure 4. Moreover, since the data uncertainty in this example can be reduced by gaining additional information, sensitivity analysis will help to decide whether it is worth investing more effort to get more specific and updated information. 

4.2.2 Crew number and position

Crew number and position are variable that each shipyard would know with a high degree of confidence. Also, these variables are not deemed important in the assessment. Therefore, according to Figure 4, the variables occupy the “low impact input” area.

4.2.3 Emissions

Information about emissions is very important to evaluate the environmental impact during production stage. However, emission information has high uncertainty and variability. Thus, this variable is a key input in the model and occupies the “critical input” area in Figure 4. Therefore, more effort should be allocated for more detailed investigation. 

In SHIPLYS, the emission during production is considered mainly as the conversion to CO₂ (kg). Such data can be found in shipyard’s database or other LCA database such as GaBi. To get maximum benefit of all available data, generic databases can be used to provide ‘prior’ data which can be conditioned by specific data using a Bayesian approach. One such application is shown in (ASME, 2003) where a generic database relating to plant and equipment is modified using specific information, thereby gaining benefit of a larger sample and locally available information.

4.2.4 Labour cost

Labour cost is important in order to calculate production cost. It is a variable that each shipyard will know with more certainty. As part of the SHIPLYS project, different shipyards have provided such information as a monetary value or cost database quoting “high level of confidence”. For this reason, the variable is categorised in “high impact input due to high sensitivity” area in Figure 4.

Moreover, this variable can be used not only for estimating the production cost but also post-MCDA analysis for the purpose of the optimisation. This is a good example to emphasise that data sensitivity and uncertainty analysis should be built on the basis of the context. Labour cost is important for production cost but does not seem to be important for LCC assessment considering different life cycle stages. This is because during the O&M stage, labour cost is not too high compared with other costs such as fuel, replacement component, inspections, and so on. This example shows the advantage of using MCDA approach, because MCDA will make the software flexible and adaptive to the different stakeholders’ expectation.

5. Future Work

At the moment, the project has made a good start and is on-going. With the progress of the project, the approach will need to adapt to the processing of the Scenarios. Other tools and techniques not shown here may need to be considered as well. 

6. Acknowledgement

This paper presents work done in the project "Ship Lifecycle Software Solutions" (SHIPLYS) that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 690770.

7. References

1. ASME, 2003. Risk Based Methods for Equipment Life Management, CRTD Vol 41. Obtaining and combining data. New York, USA.: ASME International; p. 115-46.
2. Begovic, E. Bertorello, C. Stella, M. & Barone, M. 2006. Preference Communication in Multiattribute Procedures for Optimised Ship Design, Maritime Transportation and Exploitation of Ocean and Coastal Resources, Two Volume Set. p1181 -1188
3. Bevington, P. R. &Robinson, D. K. 2003. Data Reduction and Error Analysis for the Physical Sciences, 3rd edition, McGraw-Hill, New York
4. Bharadwaj, U.R., et al. 2017. Ship Lifecycle Software Solutions (SHIPLYS) – an overview of the project, its first phase of development and challenges, Proceedings of IMAM 2017, International maritime association of the Mediterranean,  Lisbon, Portugal.
5. Cresswell, A. et al. 2003. And Justice for All: Designing Your Business Case for Integrating Justice Information 6, Appendix A. 4 Tools for identifying & evaluating options, Center for Technology in Government, University at Albany, SUNY
6. Department for Communities and Local Government. 2009. Multi-criteria analysis: a manual, Department for Communities and Local Government: London, ISBN: 978-1-4098-1023-0
7. Han, W. Xie, H. Duan, W. 2017. New method of risky multiple attribute decision making based on criteria dependence. Computer Engineering and Applications, 2017, 53(7): 9-14.
8. Linkov, I. & Moberg. E. 2012. Multicriterial decision analysis, environmental applications and case studies, CRC Press, Taylor & Francis Group, LLC
9. Ma, J. Fan, Z. P. & Huang, L. H. 1998. A Subjective And Objective Integrated Approach To Determine Attribute Weights, City University of Hong Kong, Hong Kong
10. Reiss, S. 2016. Multi-criteria Decision Support for Sustainable Material Choices with Applications in the Shipping Sector, Dissertation submitted for the degree of Doctor of Engineering, University of Surrey
11. Saaty, T. L. 1980. The analytic hierarchy process: planning, priority setting, resource allocation. New York, McGraw-Hill
12. U.S. EPA, 2001. Risk Assessment Guidance for Superfund: Volume III - Part A, Process for Conducting Probabilistic Risk Assessment, Office of Emergency and Remedial Response U.S. Environmental Protection Agency Washington, DC 20460
13. U.S. Government Accountability Office (GAO), 2009. GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP, Washington, D.C.