Evaluating risk with precision: How SERAT outperforms traditional models in project management

VALIDATION AND DISCUSSION

Sensitivity analysis of the model

To get a better level of confidence about the model that we designed, a simple sensitivity analysis has been made. In order to evaluate the value of sensitivity analysis first the model needs to be tested by validation against other results elsewhere. So we choose as a comparison, the probability model that has been developed by the project Team (data has been provided by a contact in the project team from the project report).

For all the major risk groups that we defined earlier in chapter 3, we compared our results (using the matlab model) with the ones already found in the following table. Figure 6.1 is a representation of both our results and the previous results on the same graph.

risks	SERAT	Project team
subcontractors	45.31	65.21
Technical	50	55
Financial	65	62.5
Client	47.23	40.1
Time delay	52.25	51.3
Economic	38.57	35.6
Political	50	48.5
Social	43.34	45
Project risk	65	50.52

Table 6.1: Sensitivity analysis

As we can see from the table the final result for the probabilistic analysis and the serat model is really different. As our model is assuming the project is somehow risky, the mean values model is a on some points in accordance with the problems that could be found on field, this might be a problem on the execution part, where the project team might need some qualified field agents to reduce all these uncertainties.

On the other side, as our design might be a little bit severe than the probabilistic model or maybe a little bit expensive, as the project will go on, the project team can set up a recovering plan, and with less experience needed all the time on field.

Figure 6.1: sensitivity analysis

The red line represents the project team’s probabilistic model results, whereas the blue line represents our serat model results for the project risks for every category. We can see from the analysis that instead of the subcontractors and technical risks, all the way long the serat design is over the mean values model. This is to say that in these cases, our model might be more secure, and less ‘risky ‘for the project team

Validation survey

To validate our designed model, we performed a small survey that we submitted to more than 25 companies over Africa and Asia. By the end of September, just nine of the companies responded to our suitability questionnaire that you can get a sample from the annex provided. As the sample is less than 30 and assuming that we got a normal distribution for our participants, we have the constraint of performing a hypothesis test with one sample. To make the study simple, we took as a Yes for a question that the answer was rated more than 3.

The validity of the model is presented in table 6.2 using one-sample t-test with the help of experts and field project managers and companies mostly from Africa. Due to the level of uncertainty and the focused region for the validation, the significant levels of the test are 5 %. With the help of experts’ knowledge and through a standard questionnaire we will show the validity of the model by testing the model’s variables. Respondent were asked to choose the appropriate number to indicate the level of your agreement or disagreement with the following statements about our methodology and user interface. (Strongly agree=5, Agree=4, neither agree or disagree=3, Disagree=2, strongly Disagree=1)

Questions	1	2	3	4	5	Mean frequency
The model can support project managers in assessing solar energy risks	0	0	2	2	5	0.77
The risk breakdown structure is clear for risk Identification	0	0	3	2	4	0.66
The use of fuzzy logic and linguistic variables in an improvement for risk assessment	0	0	1	2	6	0.88
We’ve already applied such a method in our projects	0	7	0	2	0	0.22
I find the User interface easy to use	0	1	3	1	4	0.55
I find User interface is flexible to interact with	0	1	3	0	5	0.55
It seems learning to operate the system would be easy for managers	0	2	3	0	4	0.44
My interaction with the model would be clear and understandable	0	0	3	0	6	0.66
We will be willing to apply this method in our organization I think that the method is innovative	0	0	2	1	6	0.77

Table 6.2: model validity results

The mean here represents the average frequency level of agreements, as we defined earlier, the companies that responded YES to any of these questions. The observed mean frequency is 0.67, with a standard deviation of 0.24.We need to prove that In this case:

Null hypothesis: The expected frequency mean of agreement with our designed assessment tool is less than five.

Alternative hypothesis: the expected frequency mean of agreement with our designed assessment tool is more than five (claim).

The t-score is made a relative score by dividing the difference between the sample mean and m by the standard error of the mean. The critical t-value (for n=9 and 95% level of significance) can be found directly from the t distribution table (t0=2.306). To compute the expected t value, we can use the basic formula of finding the t-value.

Is the expected mean

With n=9,,, s=0.24, we get t=0.236

Since the calculated value is less than the critical t-value, we have enough evidence to support the alternative hypothesis which claim that our expected mean is more than 0.5, which means that our mean is actually acceptable and we can say with 95% level of confidence that the designed model can be used in a real project.

As we can clearly show that our designed model is acceptable, the respondents also gave us some advices that we need to take in consideration to improve our future work.

We should add more discussion part before the risk assessment starts

Depending on the nature of the project, the assessment concept should be different, we should think about more membership functions

We should much more include the final costumer in our assessment, as he is an important shareholder

We should try to add more numbers and much more quantify the Policy risks

We should try to think about how the model will deals with much more perspectives.

Discussion

The goal of establishment of first integrated expert systems was about simulating the human behavior and reasoning way of inference system. The limitations of project management decision making systems do not result from their limited computational power, but lies in the lack of proper interface between project managers and computers. Till date, almost 100% of project managers are using computerized systems, but many of them still not prefer to use as an assessment tool.

The proposed model offers a tool for risk assessment in photovoltaic projects. The model lies on fuzzy inference. The knowledge base used by fuzzy rules is built on expert’s knowledge of risks. Although photovoltaic projects are known for their high level of risk, very few dedicated risk systems were developed especially for them. Therefore, the fuzzy model for risk evaluation in research projects is an innovative instrument which can be used to forecast project failure: stakeholders can save money, time, effort, without giving up the quality of predictions. The model was used to develop a software tool for assessing risk, such that the Project manager will have a clear understanding of the risks he is facing.

The results of our medialization are compatible with the project assessment troubleshot. It is safe, by the logic test that we have made to assume that the developed model is acceptable. These results show that it is easy to take care of using this method all the uncertainty problems that could be faced. This can never be eliminated but it can be fine, tuned by adding more fuzzy functions to the analysis. In terms of reasoning and hypothetical data, the model is also acceptable, but in order to test the integrity of the system, it should be applied to test the more projects past and present, failed and successful.

CONCLUSION

In response to the problems that led us to our present work problems, the research describes in this thesis introduces a new model called SERAT. This modest research work proposes a different way of taking project risk assessment to face risks associated with the PV projects in the various situations in which the information to analyze risks is uncertain, incomplete or non-obtainable.

The model allows members in the risk assessment team to give their advices by means of linguistic terms instead of mathematical computations. Since linguistic terms are not operable, to deal with complexity, each linguistic term is associated with a fuzzy function, which represents the meaning of each verbal word given by every team member.

In order to make easier the application of this model to problems with a large number of risks, a hierarchical analytic process method has been used to assess the weight of risks using the fuzzy functions that have been developed. The risk pair-wise comparative judgements are generally not consistent. To make all the calculation part easier for us, MATLAB and Simulink have been used to express all the logical fuzzy values and express them in a more cognitive way.

To show how the approach works, we implemented the model in a small software desktop tool, and a real problem on risk assessment of a solar energy power plant project has been implemented using serat.

SERAT provides a clear and effective algorithm for modelling risk assessment problems involving surrealistic evaluations of the members in the risk assessment group. The developed methodology is applicable to the general fuzzy risk assessment problem where a ranking of risks is required.

The difficulties faced while realizing this research, and the advices of experts from our model we can be used as the basis for next exciting research in this topic area of fuzzy logic risk assessment and risk management that may include:

Continuing the research with more fuzzy functions and inputs thus link the overall system with a neural network, and teach the system how to deal with any project.

Improving the SERAT desktop tool and refer it to a better data base and make it as a complete online application.

The validation of serat with more calibration and improvement can be done by validating the model using additional industrial and public data sets.

Linking the SERAT system with the other areas of project management.

REFERENCES

Ward SC. Requirements for an effective risk management process. Project Management Journal 1999(September):37–42.

Chia, S.E., 2006. Risk assessment framework for project management. IEEE, 376–379.

Jang J.-S. R., et al, “Neuro-Fuzzy and Soft Computing, A Computational Approach to Learning and Machine Intelligence”, Prentice-Hall, Inc., 1997, pp.72.

Shin Y. C. and Xu C., “Intelligent Systems: Modeling, Optimization, and Control”, CRC Press, Taylor & Francis Group, LLC, 2009, pp.22-26.

Sumathi S. and Surekha P., “Computational Intelligence Paradigms Theory &, Applications using MATLAB”, CRC Press, 2010, pp.261.

J.C. Bennett, G.A. Bohoris, E.M. Aspinwall, R.C. Hall, Risk analysis techniques and their application to software development, European Journal of Operational Research 95 (1996)

A guide to the project management Body of knowledge

Zadeh, L. (1965), “Fuzzy Sets,” Information and Control, pp. 338-353.

Applying Fuzzy Logic to Risk Assessment and Decision-Making – Sponsored by CAS/CIA/SOA Joint Risk Management Section Prepared by Kailan Shang1 Zakir Hossen2 November 2013

Soft Computing-Based Risk Management – Fuzzy, Hierarchical Structured Decision-Making System Márta Takács Óbuda University Budapest Hungary

Fuzzy decision support system for risk analysis in e-commerce development E.W.T. Ngai*, F.K.T. Wat Department of Management and Marketing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, People’s Republic of China

Risk Analysis in E-commerce via Fuzzy Logic ,1*M.H. Zirakja, 2R . Samizadeh

RISK ANALYSIS USING FUZZY LOGIC,1 Ms Manisha.Ingle, 2Dr.Mohommad Atique, 3 Prof. S. O. Dahad

Project Management Efficiency –A Fuzzy Logic Approach ,Vinay Kumar Nassa, Sri Krishan Yadav

The Application of Fuzzy Logic for Risk Evaluation of Corporate Clients,by Bc. LUBOŠ MLČOCH

Improving the Risk Identification Process for a Global Supply Chain by Amil Mody B.S. Operations Research, Columbia University, 2005

Project Risk Management Development of Risk Based Contingency Values, for a Baseline Project Budget Estimate, Prepared for the Autoridad del Canal de Panamá by the Expert Technical Committee

Application of Fuzzy Logic on Understanding of Risks in Supply Chain and Supplier Selection, by POORNA CHANDU KARUTURI

Modelling and assessment of critical risks in BOT road projects ,A. V. THOMAS1*, SATYANARAYANA N. KALIDINDI2 and L. S. GANESH2

^{A Fuzzy DSS for Contractual Risk Allocation in Bioenergy}Projects,Daniel Wright^1,*, Prasanta Dey¹, John Brammer¹

Fuzzy adaptive decision making model for selection balanced risk allocation,Garshasb Khazaeni a, Mostafa Khanzadi a, Abas Afshar

MODELING RESEARCH PROJECT RISKS WITH FUZZY MAPS Constanta Nicoleta BODEA1, Mariana Iuliana DASCALU2

Fuzzy Model for Optimizing Strategic Decisions using Matlab Amandeep Kaur1, Vinay Chopra2

Salas V, Olias E. Overview of the photovoltaic technology status and perspective in Spain. Renewable & Sustainable Energy Reviews 2009;13(5):1049–57.

MODELLING RISKS OF RENEWABLE ENERGY INVESTMENTS, Hans Cleijne, Walter Ruijgrok – KEMA (The Netherlands) JULY 2004

Komendantova N, Patt AG, Barras L, Battaglini A. Perception of risks in renewable energy projects: power in North Africa. Energy Policy (in press, published online December 2009).

Solar power investment in North Africa: Reducing perceived risks Nadejda Komendantova∗, Anthony Patt, Keith Williges International Institute for Applied Systems Analysis, Laxenburg, Austria

RISK AND DECISION MAKING PROCESS Katarína RIPLOVÁ University of Žilina, Faculty of Management Science and Informatics, Slovak Republic

Fuzzy Expert-COCOMO Risk Assessment and Effort Contingency Model in Software Project Management Ekananta Manalif The University of Western Ontario

DEVELOPING AN EXPERT SYSTEM FOR DIABETES MELLITUS PATIENTS Dina AbdulAziz M. Al-Hammadi

Using fuzzy risk assessment to rate overrun risk in international projects Dikmen,Talat,Sedat December 2006

ANNEX

INTERVIEW 1 QUESTIONS
Survey Questionnaire for project risk Identification

The purpose of this short questionnaire is to find and rate all possible hazards that could come up while undertaking business projects management. The questionnaire will take less than 5 minutes of your time. Thank you for taking part.

Q 1: Basic information

Name :

Nationality :

Age :

a)less than 30

b) between 30 and 40 years

c) between 40 and 50 years

d) more than 50 years

Experience

a)less than 1 year

b) between 1 and 5 years

c) between 6 and 10 years

d) more than 10 years

Studies level

a)BSC

b) Master

c) MBA d) PhD

Position or Area of specialization

a)CEO

b) project manager

c) researcher

d) other(specify)

Organization type : (Check only one box)

Government company

Government research institution

Independent research institution

Private energy company

Other, specify

Q2: Please choose the appropriate number to indicate the level of your agreement or disagreement with the following hazards. (High risk = 5, Medium risk =4, Low risk = 3, Not Risky=2,Not Sure=1)	1	2	3	4	5
1. Laws on obtaining government agreement
2. Level of paper work
3. Disputes between workers
4. Partner resignation
5. Different construction sites conditions
6. Unidentified work utilities
7. Quality control
8. Work conditions risks
9. Accidents
10. Workers Safety
11. Delayed payment
12. Change in scope of work
13. Errors in establishment of a project cost
14. Errors or omissions on duty
15. Inadequate labour reports
16. Inflation
17. Tax rate charge
18. Conformity of the company with international Normalizations
19. Changes on electricity prices
20. Land acquisition consequence
21. Inadequate procurement planning
22. Failures during the construction and operation
23.Risk on the final product reliability
24. Productivity rate changes
25. Performance risk
26. Unplanned maintenance
27. Lack of conformity in the organization of the project team
28. Accuracy of Market estimations
29. Expecting Project Profitability
30.Risk that the partners will not honour their contracts
31. Extreme weather conditions
32. Difficulties balancing the purchase and selling prices

Q3: If you have any other hazard that was not listed in question 2, please list the proposed risks below:

INTERVIEW 2 QUESTIONS
Survey Questionnaire for risk assessment methodology validation

The purpose of this short questionnaire is to help us for a validation of the model and software tool that we designed to assist managers on decision making in solar energy project risk assessment. The questionnaire will take less than 5 minutes of your time. Thank you for taking part.