A METHODOLOGICAL APPROACH

Dorien J. DeTombe & Cor van Dijkum (Editors)

CIP-DATA KONINKLIJKE BIBLIOTHEEK, DEN HAAG

DeTombe, Dorien J. & C. van Dijkum

Analyzing Complex Societal Problems: a methodological approach / Dorien J. DeTombe & C. van Dijkum (Editors) Munich; Mering; Hampp, 1996 With ref. - With authors description

ISBN 3-87988-189-8 NE: DeTombe, Dorien J.[Hrsg.]

Subject headings: methodology/ tools/ simulation/ complex societal problems

ã 1996 Dorien J. DeTombe, Amsterdam

P.O. Box. 3286, 1001 AB Amsterdam, The Netherlands, Europe

Tel: +31 20 6927526

E-Mail: DeTombe@lri.jur.uva.nl http://www.geocities.com/doriendetombe

Publisher: Rainer Hampp Verlag Munchen und Mering Meringzeller Str. 16 D-86415 Mering Germany Fax + 49 8233 307 55

A system dynamic simulation model expressing the relations between the aspects of a societal problem can be used to make scenarios. Scenarios can be defined as explorations of future development (Jager, 1990). Scenarios are useful for making policy decisions only if the model and the data in the model represent reality. Given the uncertainties in a model of a societal problem we raise the question in how far a system dynamic simulation model of a societal problem can be used for future exploration and future prediction of the development of a problem. In reflecting on this question we discuss criticism of the use of system dynamic modeling for scenarios from the theory of complex societal problems (DeTombe, 1994), from chaos theory (Gleick, 1987), and from system theory (Flood & Jackson, 1992). We conclude that models which express societal problems contain much uncertainty. Scenarios based on these models will also contain a large amount of uncertainty. This makes it hard to make reliable predictions based on these scenarios for future development of the problem.

Mankind has always been interested in the future. Predicting future developments of societal phenomena has been of much interest to scientist as well as non-scientists. The use of system dynamic modeling for creating scenarios for societal problems has increased greatly from the time it was initiated by Forrester and Meadows in the sixties and seventies (see Forrester, 1961; Meadows, Meadows, Randers & Behrens III, 1972). We discuss the use of system dynamic modeling for predicting the future development of societal problems, because system dynamic modeling seems to be a useful tool for creating scenarios for this kind of problems. However, there are limitations to the use of this specific tool for system dynamic modeling. These limitations are highlighted by the ideas from system theory (Flood & Jackson, 1991). There are also fundamental limitations to the correct prediction of future developments of societal problems, which are made clear by the criticism from the theory of complex societal problems (DeTombe, 1994) and chaos theory (Gleick, 1987). These restrictions will, in our view, not be removed by other tools.

Many societal problems, for instance, many of the complex technical policy problems, as the rapid growth of metropolises, the pollution of rivers and the problems this creates concerning the tension between ecology, economy and the living conditions of the people, can be categorized as complex interdisciplinary societal problems.

Some shared characteristics of complex interdisciplinary societal problem are (DeTombe, 1994, p. 9, 10):

- there is uncertainty about the starting point, the development and the end of the problem

- knowledge and data about the problem are incomplete or not directly available

- there are often many phenomena involved, phenomena (people,

institutes, countries) often with different interests.

 

A definition of a (complex interdisciplinary societal) problem is (DeTombe, 1994, p. 33)

"Something is called a problem when there is a discrepancy between the actual or (near) future situation and the desired future situation and/or there is a lack of knowledge and/or a lack of know-how, and/or a lack of relevant data; as for complex interdisciplinary societal problems, the problem is often undefined and the actual and the desired situation is not always clear."

 

 

A problem can be expressed with a model or with a combination of models. Our definition of a model is (DeTombe, 1994, p.77):

"A model is a goal related image of a problem in reality, consisting of phenomena and relations between phenomena that the subject(s), who formulate(s) the problem, consider(s) relevant."

 

A model can be based on theor(y)(ies), assumption(s), hypothese(s), experience(s), intuition(s) or any combination of these. Theoretical ideas differ with respect to their validity. Theories can be strictly founded or merely be a hypotheses, an assumption, or an idea based on experience and/or intuition. These theoretical ideas determine the way reality is viewed. In our opinion it is not possible to formulate an objective model of a complex interdisciplinary societal problem. It is only possible to formulate a model according to the subjective view of the modeling team. Whether a model is a correct or incorrect, a complete or incomplete representation of a problem is a matter of the inter- and/or intra-subjective opinion of the subject(s) that create(s) or use(s) the model.

A problem is defined by a description of the conceptual model of the problem, which can be expressed in several (sub-)models (see the seven-layer-model of DeTombe, 1994). Based on the conceptual model an empirical model can be created. This model can also consist of several sub-models. An empirical model is more detailed, and is more based on data, than the conceptual model. One of the sub-models for describing an empirical model of a problem can be a system dynamic simulation model. Figure one shows an example of a graphical representation of a (simplified) system dynamic model of the transportation of people and goods in relation to economy and environmental demands of fast growing metropolises.

In order to use a system dynamic empirical model as a representation of reality, the discrepancy between the model and reality should not be too large. Given complex interdisciplinary societal problems, it is not always possible to determine exactly when a conceptual or empirical model contains enough aspects captured in the way that they can be used as an adequate representation of reality. There can be many causes for a model not to be formulated according to reality. This can be caused by blind spots, by forgetting to include phenomena in the model, by not knowing that certain phenomena should be included, by deliberately excluding phenomena because of the limitations of the model, by a wrong interpretation or estimation of the relations between the phenomena, by missing relevant data (white spots), or by having data that are in contradiction with each other.

The system dynamic simulation part of the empirical model can be the start for making scenarios. Scenarios can be defined as explorations of future development (Jager, 1990). Exploring the future with scenarios is done in the expectation to 'predict' the future development of the problem, and/or to select an optimal strategy for changing the problem in the desired direction. Selecting an optimal strategy for changing the problem is done by comparing the assumed effect of interventions with the desired situation.

Several kinds of scenarios may be distinguished. There is the so-called null-option scenario. This is the scenario in which, at least by the members of this problem handing team, no deliberate interventions are included. One could translate the null-option scenario into: 'the development of the problem without our interventions'. The null-option scenario could also be called and considered the basic scenario. Because there may be several major possible changes in the future which might influence the problem, one can start creating scenarios with creating several basic scenarios. Based on these different basic scenarios one can reflect on the effect of different (combined) interventions. In a scenario, the future can be explored by simulating the expected changes in and between the aspects of the problem. The effect of the changes can be explored by simulating the changes in time steps.

The basic scenarios, the interventions and their assumed effects are based on theoretical ideas about the development of the future, the phenomena, their relations and their reaction to the suggested interventions.

Using a system dynamic simulation model as a start for a scenario for exploring future developments of societal problems and situations is done by many researchers and institutes. Examples are Bruckmann & Fleissner (1989), who made a prediction of the Austrian economy based on a system dynamic model, and Meadows, Meadows, Randers & Behrens III (1972), who use system dynamic modeling to predict the use of future resources in the Club of Rome book 'Limits of Growth' and in their book 'Beyond Limits of Growth' (Meadows, Meadows & Randers, 1992). In the Netherlands system dynamic models are used for future prediction at the National Institute for Public Health and Environmental Protection (RIVM) and at the Research Center of Public Mental Health Care (NcGv).

The outcomes of these studies sometimes have a significant impact on society, in the way of policy making and policy advises. Because of that it is interesting to reflect on the question: "Is it possible to predict the future of such a complex societal issue?"

One may wonder whether it is possible to create an empirical model and scenarios reliable enough to base policy decisions on.

This question will be discussed from the point of view of system theory (Flood & Jackson, 1991), the theoretical ideas of complex societal problems (DeTombe, 1994), and chaos theory (Gleick, 1987). In this discussion we first reflect on models, then on scenarios. Is it possible to make a complete and objective model of the situation which can serve as a basis of a scenario?

There is criticism on using system dynamic simulation models for future predictions from system theory. System thinking emerged as a response to the failure of mechanistic thinking to explain biological phenomena. This thinking was soon transferred to the study of other 'systems' such as organizations. However, since the system view was originally born in biology it tended to rely on biological analogies, introducing ideas such as survival, adaptability, development, growth, flexibility and stability. Flood & Jackson (1991, pp. 5, 6) write about systems:

"A system consists of a number of elements and the relationships between the elements. A richly interactive group of elements can be separated from those in which few and/or weak interactions occur. This can be achieved by drawing a boundary around the richly interactive group. The system identified by a boundary will have inputs and outputs, which may be physical or abstract. The system does the work of transforming the inputs into outputs. The processes in the system are characterised by feedback, whereby the behavior of one element may feed back, either directly from another element by way of their relationship, or indirectly via a series of connected elements, to influence the element that initiated the behavior."

 

The way an organization is viewed in system thinking to represent problems and to use a system thinking approach for analyzing them can be applied for complex interdisciplinary societal problems. For instance, the 'temporary' boundaries of an open system, including input and output and the communication between the elements may be compared with to boundaries of the model of a problem. There is also a hierarchy by which a system can be a part of a wider system and in return can consist of smaller sub-systems like Flood & Jackson write (1991, p. 6, 7). The system dynamic approach is created in order to be able to tackle greater complexity than is possible with other approaches. In doing so it loosens some of the characteristics of the scientific method.

In the field of system theory there are two standpoints, one viewpoint is held by those who call themselves hard system theoreticians and the other by those who call themselves soft system theoreticians. According to the hard system theoreticians the system dynamic method is too soft and too weak, and cannot quantify enough. According to the soft system theoreticians the system dynamic method is not soft enough, it wants to quantify too much. Using system dynamic modeling techniques has been criticized from these two viewpoints. These respective critiques refer to theory, methodology, ideology and utility. We will discuss some points described in Flood & Jackson (1991, p. 78-83).

Soft system thinkers question the underlying assumptions of system dynamic modelers that there is an external world made up of systems, whose structure can be grasped using models built upon feedback processes. To soft system thinkers, social systems are much more complex than this. Social systems cannot be structured 'objectively' from the outside. System dynamic failed to embrace the 'subjectivity', which is an essential part of the complex-pluralist situation. We agree, to a certain point, that it is not possible to grasp the complex reality into a model. However we think that although the reality of complex societal problems is far more complex than any model can show, models can be very useful in trying to comprehend at least some parts of the problem.

The soft system scientists state that the phenomena and their relations in a model cannot be quantified. We agree that it is very hard, and sometimes not very useful to quantify all aspects, all phenomena and their relations and that one often has to be satisfied with an estimation with an amount of uncertainty (see also de Tombe, 1992). However, we think that, where it is possible, some quantification can be very useful. Although we realize that one can easy step into the pitfall of giving numbers to not quantifiable phenomena.

More fundamental criticism coming from soft system thinkers, is that subjective intentions of human beings cannot be captured in such 'objective' models. In the opinion of soft system thinkers models are designed to increase mutual understanding, not to represent external reality. Social systems are in their point of view (Flood & Jackson, 1991, p. 79)

".. the creative construction of human beings whose intuitions, motivations and actions play a significant part in shaping 'system' behavior. ..System dynamics do not deal with the innate subjectivity of human beings and the consequences of this for the study of social systems."

 

Soft system thinkers say that societal systems cannot be captured, because in a model the natural subjectivity of the human beings cannot be captured in a model. Although we agree that societal systems are created by humans with all their hidden agenda's, motivation, subjectivity, personal and historical background, and we do know that these original drives and motivation to create a system cannot be shown in the model, however we still think that some aspects of a societal system can be captured in the model, at least enough aspects to justify making models. In our view one of the aims of a conceptual model is to increase the understanding of the problem, and when it is used and/or built by the same group of persons it can be used as a vehicle for communication (Sol, 1982). To this point we agree with soft system scientists. However, we have the opinion that for understanding the problem the model must be representative as well, and the deviation from reality should not be too large. Where it concerns the empirical model we, like many others, use the model or would like to use the model to represent reality.

Hard system scientists (Flood & Jackson, 1991, p. 78-83) emphasize that the group that did the modeling should be the same group that suggests interventions and policy. One of the reasons is that this group knows exactly the value of the knowledge and the data on which the model is based. We recognize that uncertain factors can be interpreted as objective factors when they are captured in a model. This is one of the reasons why we agree on this point and emphasize that the same group of modelers should interpret the subjectivity of a model (see DeTombe, 1994).

Ideology critique coming from hard system thinkers emphasizes that the people who build the model have the idea that they are elite technicians: the model is theirs. They do not allow any involvement of other 'stakeholders' (Flood & Jackson, 1991, p. 81). This is also our critique on making models by one to three people, as often happens. The model is made by a few modelers, and not by a small group of content experts or members of the different parties involved in the problem. Another theoretical critique from hard system thinkers is that conclusions are based on uncertain data and knowledge.

"It (system dynamic) apparently jumps to conclusions about whole system behavior before the data have been collected and the laws verified which would make such conclusions justifiable." (Flood & Jackson, 1991, p. 79)

 

This is also in agreement with another critique of the hard system scientists on the methodology. The information to make the empirical model is insufficient. Models should only be made when sufficient information is available on the issue.

The critique of the hard system scientists is also directed at the utility of the model. This also concerns the availability of data. The system dynamic model uses poor data. These last three theoretical critiques correspond with our critique coming from the theory of complex interdisciplinary societal problems.

Based on the critique above we could say, that modeling of societal problems according to the ideas of system dynamic modeling is subjective, the data that are used are incomplete, and the boundaries are relative and depending on one's subjective view. The utility of the model is limited. This means that the model based on system dynamic modeling efforts should not be given to others than the modelers themselves to base conclusions on, because the model is, depending on the complexity, more or less deviant from reality.

This criticism of the use of system dynamic models as a scenario for future prediction is based on the idea that it is impossible to make a model of a complex interdisciplinary societal problem that is complete and reliable enough to base policy making on. This critique not only concerns system dynamic modeling but modeling and making scenarios in general.

According to the theory of complex interdisciplinary societal problems (DeTombe, 1994) making a suitable model of a societal problem is very difficult. The problem is complex and imbedded in a dynamic environment, which in turn reacts to the problem. This prevents us from knowing all the aspects and their relations. Not all existing phenomena, data, and relations are imbedded in the model, which is in our view impossible. A model of a societal problem often comprise phenomena, relations between the phenomena that are certain and relations that might be possible. The model comprises data that are certain and that are uncertain, the modelers sometimes even have to chose between data in contradiction with each other. Concerning the empirical model, the data of the model will in many cases be uncertain, unreliable and incomplete (DeTombe, 1992). Not all data and phenomena we do know can be included in the model in detail as much as one could wish.

The model of a complex interdisciplinary societal problem will contain much uncertainty and scenarios based on those models will also contain much uncertainty, even more so because in scenarios the uncertainty will be enlarged. Developing scenario based on these models increases the amount of uncertainty. There is uncertainty about the main basic scenarios: are they well selected, what is the chance that they will be become reality, and concerning the interventions: are they well chosen, which effect will they have, what will be the cumulative effect etc.? This shows that one should be very careful to base policy decisions on using scenarios of societal problems.

The third critique on system dynamic modeling of societal problems comes from chaos theory, includes fundamental criticism with respect to the ability to create reliable scenarios in general. This criticism concerns two aspects, the unpredictability based on incomplete data and the unpredictability based on non-linear feedback loops.

It is hard to find a suitable definition of the chaos theory. Tennekens (1990b, p. 8) describes chaos as a feature of a non-linear system. He states (p. 9):

Chaos theory seems able to give a description and an explanation of certain issues which, until some decades ago, were not noticed, were neglected or seemed to be inexplicable. The knowledge of chaos theory enables to notice things not noticed in the same way before. The language of chaos theory enables to describe some of these views. Chaos theory, as far as it is developed now, describes many, and sometimes totally divergent matters. In reflecting this, the phenomenon 'chaos theory' seems more a collective noun to describe certain phenomena, then a thoroughly worked out theory. One of the general focuses of chaos theory is unpredictability and non-linear feedback loops.

Scientists like Newton (1642-1727) and Laplace (1749-1827) had the idea that everything can be known, predicted and may be even controlled at the moment all the data and the connections between the data are known (Laplace, 1796). In physics the common opinion is that unpredictability is a matter of incomplete data and limited capacity of computers. However, chaos theory shows that even in physics it is sometimes impossible to predict future developments (Broere & Verhulst, 1990). Tennekens (1990) pointed out that according to meteorological models, even if one triples the data collecting points and ten times doubles the computer capacity, the weather cannot be predicted for more than five to seven days in advance with reasonable certainty, and often not even that. Van Dijkum states in his article about unpredictability and chaos theory (1992, p. 26):

"A process that is unpredictable is undetermined. Time is a variable in a prediction which together with the values of other variables determines the state of the process. If there is no prediction possible then there is also no determination of all values of the chaotic function possible."

 

The opposite is that, when there is no definition of all the values of a function, there is no prediction possible.

The theory of complex societal problems (see DeTombe, 1994) shows that data of a complex interdisciplinary societal problem are never complete. Even when there are no blind and white spots, and there are, the data would never be complete because of the changing problem and the changing environment.

A part of chaos theory describes unpredictability of non-linear processes (Gleick, 1987; Prigogine & Stengers, 1984; Lorenz, 1979). The models of complex interdisciplinary problems contain many non-linear feedback-loops. There can be moments in which these non-linear feedback loops lead to unpredictability (see DeTombe, 1992b).

In this paper we tried to give some answers to the question: "Is it possible to use a model and scenarios of societal problems to base policy decisions on?" We first answered this question by indicating the limitations of using a system dynamic tool for predicting. We then approached the question in a more fundamental manner: "Is it possible to make models and use them for prediction of such complex things as societal problems?"

Even when making a model of a societal problem is carried out with as much knowledge, tools, methodological support and human effort available, these models will still contain a large amount of uncertainty (DeTombe, 1994). It will be clear that a reasonable matching empirical model cannot be made. Although the quality of interventions and scenarios will depend on the way the whole process of problem handling is performed, there will always be a large amount of uncertainty in the estimated effect of the interventions and thus in the scenarios. Models made of complex interdisciplinary societal problems contain much uncertainty, and scenarios based on these models will even contain a larger amount of uncertainty. Developing scenarios, that correspond to the future, will almost be impossible. The range of uncertainties can be that large that one can hardly expect to be able to make real policy decisions based on these scenarios. Policy making based on scenarios of complex interdisciplinary societal problems is a risky thing to do and as a consequence one should be very restrained using scenarios for this (DeTombe, 1992a). A scenario provides at most some directions in which the problem might develop, and it should only be used as a tool for discussing the future development of the problem.

Nevertheless, for the want of anything better, many scenarios are used for policy making. When we exclude intentional misuse, there is still the danger that, although the designers of the model are aware of the problematic data and the uncertainties of the relations in the model, when taken out of the hands of the creators, the scenario may live a life of its own. Other people may use the scenario as a correct representation of reality, instead of using the scenario as a tool to discuss the problem. Even the creators take the risk of interpreting their own subjective model as a fact.

In using scenarios we can make a comparison between predicting the future development of societal problems and the weather forecast. In both cases that some short term predictions can be made with a reasonable chance on success, however the longer the period the more difficult it will be to predict the future development. This might even be impossible.

References

Bahlmann, J.P. & B.A.C. Meesters (1988) Denken & Doen. Een studie naar ontwikkeling en strategische heroriëntatie van zes Nederlandse ondernemingen rond een crisis. Delft: Eburon.

Broer, H.W. & F. Verhulst (Red.) (1990) Dynamische systemen en chaos. Utrecht: Epsilon.

Bruckmann, G. & P. Fleissner (1989) Am Steuerrad der Wirtschaft. Ein ökonomisch kybernetisches Modell für Österreich. Wien: Springer-Verlag.

Checkland, P.B. (1981) Systems thinking, systems practice. Chichester: Wiley.

Checkland, P.B. (1982) Soft systems methodology as process. A reply to M.C. Jackson. Journal of Applied Systems Analysis, 9, 37-9.

Checkland, P.B. & J. Scholes (1990) Soft Systems Methodology in Action. Chichester: Hohn Wiley & Sons.

Dangerfield, B. & C. Roberts (1990a). OR Work in HIV/AIDS. London: ORS Publication Unit.

Dangerfield, B. (1990b). Listing of the variable infectivity model. Salford: University of Salford.

DeTombe, D.J. (1992a). Using a simulation tool for defining global problems. ISAGA'91; 22nd Annual International Conference of the International Simulation and Gaming Association. Kyoto, Japan (abstract).

DeTombe D.J. (1992b). Onzekerheden in de voorspelling van de Aids epidemie. In: Dijkum, C. van & D. DeTombe (Red.). Gamma Chaos. Onzekerheid en orde in de menswetenschappen. Bloemendaal: Aramith uitgevers, pp. 126-137.

DeTombe, D.J. (1994). Defining complex interdisciplinary societal problems. A theoretical study for constructing a co-operative problem analyzing method: the method COMPRAM. Amsterdam: Thesis publishers.

DeTombe, D.J. (1995) Co-operative Handling of Complex Interdisciplinary Societal Problems. In Guangzhong Liu, Kang-Hoh Phua, Jigang Ma, Juping Xu, Fuwen Gu & Changzheng He (Eds.) Optimization Techniques and application, ICOTA '95, vol. 2. Chendu University of Science and Technology, Chengdu, China. Singapore, New Jersey, London, Hong Kong: World Scientific, pp. 1408-1416.

Dijk, J.A.G.M. van (1989) Popularising Delphi Method, Developing an Instrument to Control Technological Change for Employees. Quality and Quality 23, pp. 109-203.

Dijk, J.A.G.M. van (1990). Delphi questionnaires versus individual and group interviews, A Comparison Case. Technological Forecasting and Social Change 37, pp. 399-407.

Dijk, J. van, M. de Goede, H. 't Hart & J. Teunissen (1991) Onderzoeken en veranderen. Methoden van praktijkonderzoek. Leiden: Stenfert Kroese.

Dijkum, C. van (1988) Spelen met paradoxen. Een methodologische studie naar vicieuze cirkels in denken en handelen. Utrecht: Rijks Universiteit Utrecht.

Dijkum, C. van & F. Wallner (1991) Constructive realism in discussion. Amsterdam: Sokrates Science Publisher.

Dijkum, van C. & DeTombe, D.J. (Red.) (1992). Gamma chaos. Onzekerheid en orde in de menswetenschappen. Bloemendaal: Aramith uitgevers.

Dörner, D. (1989) Die logik des Mißlinges. Strategisches Denken in komplexen Situationen. Hamburg: Rowohlt verlag.

Dörne, D., H.W. Kreuzig, F. Reither & Th. Stäudel (1983) Lohhausen: vom Umgang mit Unbestimmtheit und Komplexität. Bern: Huber.

Fleissner, P., W. Böhme, H.U. Brautzsch, J. Höhne, J. Siassi & K. Stark (1993) Input-Output-Analyse. Eine Einfuhrung in Theorie und Anwendungen. Wien: Springer-Verlag.

Flood, R.L. & M.C. Jackson (1991). Creative problem solving. Total systems intervention. Chicester: Wiley.

Flood, R.L. & E.R. Carson (1992). Dealing with complexity: an Introduction to the Theory and Application of Systems Science. New York: Plenum, 2nd edition.

Forrester, J.W. (1961) Industrial dynamics. Cambridge, Mass.: The MIT Press.

Forrester, J.W. (1969) Urban dynamics. Cambridge, Mass.: The MIT Press.

Forrester, J.W. (1973) World dynamics. Cambridge, Mass.: Wright-Allen Press , 2nd edition.

Forrester, J.W. (1969) Principles of systems. Cambridge, Mass.: Wright-Allen Press.

Forrester, J.W. (1987) Lessons from system dynamics modeling. System Dynamics Review, 3-2, pp. 136-149.

Forrester, J.W. (1990) System dynamics as a foundation for pre-college education. In Anderson, D.F., G.P. Richardson & J.D. Sterman (Eds.) System Dynamics '90. Boston, Mass.: The System Dynamics Society, pp. 367-381.

Gleick, J. (1987) Chaos: making a new science. New York: Viking.

Gleick, J. (1991) Chaos. De derde wetenschappelijke revolutie. Amsterdam: Uitgeverij Contact.

Hart, H. 't (1991) Onderzoek en praktijkfasen. In Dijk, J. van, M. de Goede, H.'t Hart & J. Teunissen Onderzoeken en veranderen. Methoden van praktijkonderzoek. Leiden: Stenfert Kroese, pp. 55-65.

Heisterkamp, S.H., B.J. de Haan, J.C. Jager, J.A.M. van Druten & J.C.M. Hendriks (1992) Short and medium term protections of the AIDS/HIV epidemic by a dynamic model with an application to the risk group of homo/bisexual men in Amsterdam. Statistics in Medicine, 11, pp. 1- 17.

Jager, J.C. & E.J. Ruitenberg (1987) Statistical analysis and mathematical modelling of AIDS. AIDS 1987, 1, pp. 129-130.

Jager, J.C. & E.J. Ruitenberg (Eds.) (1988). Statistical analysis and mathematical modelling of AIDS. Oxford: Oxford University Press.

Jager, J.C. & M.J. Postma (1989a). De AIDS-epidemie: prognoses en economische gevolgen. In Vuysje, H. & R.A. Coutinho (Eds.) Dilemma's rondom AIDS. Amsterdam: Swets-Zeitlinger, pp. 185-205.

Laplace, P.S. (1796). Exposition du systeme du monde. Paris: Courier.

Lorenz, E. (1979) Predictability: Does the flap of a butterfly's wings in brazil set off a tornado in Texas? Paper presented at the annual meeting of the American Association for the Advancement of Science. Washington, December 29th 1979.

Meadows, D.H., D.L. Meadows, R. Randers & W.W. Behrens III (1972) The limits to growth. London: Potomac Associates Book, Earth Island Limited.

Meadows, D. (1980) The unavoidable a priori. In Randers, J. (Ed.) Elements of the system dynamics method. Cambridge, Mass.: The MIT Press.

Meadows, D.H., D.L. Meadows & J. Randers (1992) Beyond the limits. Global collapse or a sustainable future. London: Earthscan Publications.

Poincaré, H. (1913) The foundations of science. New York: Science Press.

Prigogine, I. (1980) From being to becoming. New York: Freeman.

Prigogine, I. & I. Stengers (1984) Order out of Chaos. New York: Bantham Books.

Tennekens, H. (1990a) Dan leef ik liever in onzekerheid. Bloemendaal: Aramith.

Tennekens, H. (Ed.) (1990b) De vlinder van Lorenz. De verrassende dynamica van chaos. Bloemendaal: Aramith.

Verhulst, F. (1990) De historische route naar chaos. In Tennekens, H. De vlinder van Lorenz. De verrassende dynamica van chaos. Bloemendaal: Aramith Uitgevers.

See for more publications of Dorien J. DeTombe

See for lectures of Dorien J. DeTombe

home