3 Research Design

The purpose of this Part of the report is to discuss the major methodological issues pertinent to this research, present the general methods for data collection, matters concerning data analysis and ethical clearance. In order to address the areas of interest, the research was divided into two studies:

Study One: The comparative study – This study focussed on testing how Australian opinions on energy options differ from the Dutch when measured using the ICQ.

Study Two: The ICQ online study – This study focussed on investigating how an online design could enhance the quality of the original ICQ by making the questionnaire an interactive application.

The following sections detail each of the studies research designs.

3.1 Study One: The comparative study

Aim: This study investigated the choices the general Australian public would make concerning carbon dioxide emission reduction options after having received and evaluated expert information on the consequences pertaining to these options compared to the Dutch public’s responses to their country specific options. Therefore, this study addressed the following research question:

Research question 1: How do Australian opinions on energy options differ from the Dutch when measured using the ICQ?

Method and measures: The method used was the ICQ survey tool, which assesses not only informed evaluations of, in this case, carbon dioxide reduction options; it also investigates the evaluations of the consequences of these options after information has been processed. The results from the survey give insight into which consequences influence the overall evaluation, rejection and choice of an option.

Before respondents in the ICQ choose between the policy options, they receive information to make a more informed choice (see Appendix A for the full set of Australian and Dutch options and consequences). First, the choice is explicitly framed as a decision problem and respondents are informed about the background of the challenge. Second, respondents are provided with information about the consequences of the different policy options which address the decision problem. To stimulate information processing and to help respondents reach a decision, they are requested to give a quantitative evaluation of each consequence. On the basis of these quantitative evaluations, the subjective utility of each option may be determined.

The figures below depict the first stages of the online ICQ process, in which respondents were asked to select which country they are from. They are then lead to information about carbon dioxide emission issues and their country specific decision problem.

Figure 1. Selecting country specific information

Figure 2. Presenting the policy problem, options and consequences

It is essential to define a clearly specified and policy relevant decision problem that is not overly demanding for respondents. Furthermore, when informing people about the decision problem and about the consequences of the options that can solve this problem, it is vital that this information is valid and balanced. Therefore, the information for the ICQ was compiled by experts from different backgrounds and different organisations and checked by another similarly differentiated group of experts. Only after their approval was given was the information included in the ICQ. This rigorous process provides not only accurate and balanced information but also credibility. Based on discussions with and between experts, the following seven options to solve the policy problem of how best can Australian demand for energy be fulfilled in 2030 in such a way that emissions of carbon dioxide will be reduced by 33% compared to 2000 were defined as:

Australian options and their aims

Option 1: Energy efficiency in residential and commercial sectors

This package aims to reduce 50 million tonnes of carbon dioxide by 2030 by improving the energy efficiency in residential and commercial sectors focusing on changes to construction materials, technology and appliance use. Household electricity consumption has the potential to drop to just over 50% of today’s electricity consumption by replacing lighting, old refrigerators and freezers, electric hot water systems, shower heads and taps and avoiding stand-by power use. Electricity efficiency in the commercial sector (which includes services such as finance, property and health, not heavy industry such as manufacturing and mining) can lead to a 60% reduction and has the potential for further decreases of up to 75%. Such reductions are possible in several ways, including the use of glazing, natural daylight, well maintained and efficient equipment, use of monitoring systems and a reduction in air conditioning usage. Energy efficiency improvement resulting in a 20% reduction in residential and commercial electricity consumption will be sufficient to achieve the reduction of 50 million tonnes of carbon dioxide.

Option 2: Efficiency in the manufacturing and mining industries

This package aims to reduce 50 million tonnes of carbon dioxide by 2030 by increasing energy efficiency in manufacturing and mining industries by 11.4% and 10% respectively. This will require upgrade and replacement programs, improvements in heating systems, and the introduction of high efficiency technologies and controls, with opportunities for new technologies to provide additional potential for further energy efficiency.

Option 3: Replacing future planned coal-fired power stations with gas

This package aims to reduce 50 million tonnes of carbon dioxide by 2030 by building new gas-fired power plants instead of new coal-fired power plants. Australia has vast reserves of most energy resources (with the exception of oil). Where coal is located, primarily in the larger Eastern states, it is the preferred electricity generation fuel because of its lower cost. Currently, 77% of Australia’s electricity is produced by 38 coal-fired power stations and 15% is generated by 26 gas-fired power stations. Gas-fired power plants emit less carbon dioxide than coal-fired power stations, so there is an opportunity to reduce greenhouse gas emissions by building gas-fired instead of coal-fired power stations in the future. In order to achieve the 50 million tonne reduction by 2030 around 50% of all Australia’s electricity needs will have to be produced from gas-fired power generation. This will effectively mean all new power stations are gas fired and that existing coal plants retire at their expected end of life.

Gas reserves in Australia are sourced from underground natural gas reservoirs and coal seams. Australia’s Gas Industry has experienced substantial growth in the development of coal seam gas extraction technology over the past ten years and is set to continue to expand its use over the coming decades. With production of coal seam gas occurring already in Queensland and New South Wales, significant potential exists into the future.

Option 4: Carbon dioxide capture and storage with coal

This package aims to reduce 50 million tonnes of carbon dioxide by 2030 by capturing, transporting and storing geologically emissions from flue gas emitted by coal-fired power plants in Australia. Carbon dioxide emissions can be captured, compressed into a liquid, transported and permanently stored deep beneath the Earth’s surface. This process is called carbon dioxide capture and storage or CCS and allows for a continued use of fossils fuels in a low-carbon economy. In order to achieve the aim of this package Australia would require CCS to be installed at a quarter of the existing coal-fired power plants which make up 77% of current electricity generation; although it can be used in combination with gas-fired power plants as well. Presently, CCS is limited to a few industrial-scale projects related to the oil and gas industry. Australian CCS development for power sector applications is comparable to overseas progress and includes a range of pilot and demonstration projects across the nation.

Option 5: Deploying renewable sources

This option aims to reduce 50 million tonnes of carbon dioxide by 2030, by increasing the use of a range of renewable energy generation technologies (e.g. solar, wind, geothermal, biomass). Currently most of Australia’s energy is generated from fossil fuels (92%) with renewable sources contributing around 8%. To achieve the aim of this option, the share of renewable electricity in the total electricity production has to grow to approximately 23% in 2030.

The largest existing renewable sources are hydroelectricity (5%) and wind (2%). In the future, both solar and wind electricity generation resources have the potential to contribute over 100% of Australia’s electricity needs but their supply will be less secure with their generation output being driven by the climate (such generation is called “intermittent”). Sustainably produced biomass would not be intermittent but its resource is more limited such that it could only produce 30% of power needs each year. Geothermal energy is not intermittent and is available on the scale of solar and wind resources. Consequently, while the technology is less proven, it has a strong potential to increase the share of renewable sources in electricity generation.

Option 6: Participating in an international emissions trading scheme

This option aims to reduce 50 million tonnes of carbon dioxide by 2030 by buying international emission permits. This would typically be introduced as part of a national emissions trading scheme that would induce abatement in many sectors across the economy and such is the case in Australia from July 2015 (following on from a fixed price scheme starting in July 2012). A domestic carbon price scheme is one of many policy initiatives that could be used to encourage people to take up abatement options and can be targeted to achieve any level of abatement (the Australian Government’s target is 80% below 2000 levels by 2050). However in this abatement option we are strictly only interested in the part of the emissions trading scheme that allows the purchasing of emissions permits from overseas which can substitute for taking domestic abatement action.

When Australia introduces emissions trading there will be a limited number of permits available in Australia and the scheme will allow the option to buy permits internationally to cover emission liabilities. The purchasing of international permits will be limited to only 50% of a business’s annual emissions under Australian emissions trading scheme rules. In order for Australian businesses to participate in purchasing international permits, the Government would need to agree to link its carbon trading scheme with other countries trading schemes. Presently no such agreements exist but the Australian Government foresees international linking at the start of its flexible price cap-and-trade scheme from 1 July 2015 onward.

Option 7: Nuclear power

This package aims to reduce 50 million tonnes of carbon dioxide by 2030 through the introduction of nuclear powered electricity. In order to achieve this Australia would need to install five new nuclear power plants. The next generation of nuclear power plants would have a 60 year lifespan and use automated safety systems which are a significant improvement on some existing nuclear power plants. Currently Australia has no working nuclear power facilities; however, there is a research nuclear reactor in Sydney for research and medical purposes.

The 2012 Dutch ICQ options were developed by experts similar to a previous Dutch ICQ study conducted in 2007 by de Best-Waldhober, Daamen, Ramirez, Faaij, Hendriks and Visser (2008). The policy problem was to reduce the Netherlands emissions by 50% by 2030. The seven Dutch options were defined as:

Dutch options and their aims

Option 1: Improvement of energy efficiency

This package aims to reduce carbon dioxide emissions with 40 million tonnes in 2030 by making appliances, cars, houses and the production of goods more energy efficient. “Energy efficiency” is the decrease of energy that is necessary for an equal result. For instance, the energy that is needed to heat a medium-sized house. Or, the energy needed to produce a tonne of steel; or the energy needed to drive 1 kilometer with a car. For instance, by developing more efficient technologies or better isolated houses or more efficient cars, less energy will be needed to get the same result. Without extra measures the energy saving improves every year. To save 40 million tonnes of carbon dioxide emission, an additional energy efficiency increase of 1% per year needs to be realised for appliances, cars, houses and factories. To achieve this additional 1 % of energy saving per year, the government has to take mandatory measures. These measures are needed to ensure that companies and civilians make an effort to increase the energy efficiency of their appliances, cars and houses and to optimise the production of goods. Because this package requires less energy to get the same result, less fuel is needed to generate energy.

Option 2: Improvement of energy efficiency and decreased use of material and energy

This package aims to reduce the emission of carbon dioxide by40 million tonnes in 2030. This package is an addition to the first package “Improved energy efficiency”. This first package aims to reduce the emission of carbon dioxide with 40 million tonnes, by improving the efficiency of appliances, cars and houses with 1% per year. This second package is an addition to the first package and aims to reduce another 40 million tonnes of carbon dioxide by improving the efficiency with another 1% per year. The first and second package together lead to a reduction of carbon dioxide emission by 80 million tonnes in 2030. To implement this package the government has to take extremely tough and compulsory measures, even tougher measures than in the first package. These measures have to make sure that companies as well as individuals will do their absolute best to make their appliances, cars and houses more efficient. In addition, very strict government policies such as deposits, taxes and fines will have to force people to reduce the use of energy and materials.

Option 3: Electricity from wind turbines at sea

This package aims to reduce the emission of carbon dioxide with forty million tonnes by the year 2030 by generating electricity using approximately twenty clusters of wind turbines in the Dutch North sea. These clusters will be placed at several locations in the sea along the whole Dutch coast at least twenty kilometers from the coast.

Option 4: Conversion of biomass to car fuel and electricity

This package aims to reduce the emission of carbon dioxide by forty million tonnes by powering a share of the cars using fuel converted from biomass and by making power plant use biomass as a fuel for the generation of electricity. Biomass is a term used for a variety of organic materials such as wood, grass, organic waste, etc. Biomass can be used to generate electricity, but also to create fuel for cars. When plants grow they withdraw carbon dioxide from the air. This carbon dioxide is released again when biomass is being burned. By burning plants, the amount of carbon dioxide that is released is not lower that the amount of carbon dioxide that has been withdrawn by the plants during growth. Therefore biomass is carbon dioxide neutral. This package is not completely carbon dioxide neutral because of the need for transportation and handling of the biomass. To reduce forty million tonnes of carbon dioxide by the year 2030 by using biomass, approximately 80 percent of the biomass will have to be imported. Most of this biomass will be converted into modern biofuel for cars, partly abroad, partly in the Netherlands. Biofuel factories will have to be built where biomass can be converted into fuel. A share of the currently used oil refineries, where crude oil is converted to petrol and diesel oil, may gradually be converted into or replaced by biofuel factories. In that case a small portion of this biomass in the Netherlands will be converted into electricity by three or four large power plants in seaports like Rijnmond, Eemshaven or Terneuzen.

Option 5: Large plants where coal or gas is converted into electricity with CCS

This option aims to decrease carbon dioxide emissions with 40 million tonnes by capturing carbon dioxide that is produced by coal-fired and gas-fired power plants and storing it underground in the Netherlands or under the Dutch part of the North Sea. Carbon dioxide capture can take place at existing power plants or be integrated into new plants. It is expected that, by 2030, about half of the power plants with carbon dioxide capture and storage will be coal-fired and the other half will be gas-fired. This package can be implemented temporarily because the space available for carbon dioxide storage will get full and natural gas and coal will eventually run out. The current knowledge of the subsoil leads to the expectation that there will be storage space for about 100 to 300 years. More research on safety and availability will be needed to determine if all this storage space can be used. Research may, however, show that more space is available than currently expected.

Option 6: Large plants where gas is converted into electricity with CCS

This package aims to reduce carbon dioxide emissions with 40 million tonnes by producing hydrogen and capturing and storing the carbon dioxide that is produced in this process. Hydrogen is a gas that releases energy in the process of combustion. Hydrogen can be used to generate electricity. It can also be used as fuel for cars, or to replace natural gas in households. About 20 to 25 large hydrogen factories will be built for this package. The carbon dioxide that is produced during the conversion of natural gas into hydrogen will be captured and stored underground in the Netherlands and under the North Sea. The hydrogen from the 20 to 25 factories will be used in part to provide most of the cars in the Netherlands in 2030 with fuel. Current fuel stations will have to be adjusted in such a way that hydrogen can be stored and withdrawn there. The hydrogen will also be used in part to provide the majority of households and industry with hydrogen, where the hydrogen can be converted into electricity and heat in small installations. In households, such an installation is comparable to a central heating boiler. This package can be implemented temporarily because the space available for carbon dioxide storage will get full and natural gas and coal will eventually run out. The current knowledge of the subsoil leads to the expectation that there will be storage space for about 100 to 300 years. More research on the safety and availability will be needed to determine if all this storage space can be used. However, research could also show that there is more space available than currently expected. It is likely that alternative uses can be found for the infrastructure (such as installations, fuel stations and the pipeline grid) after this time, because by then other ways will have been developed to produce hydrogen without natural gas.

Option 7: Electricity from nuclear power plants

This package aims to reduce the emission of forty million tons of carbon dioxide by generating electricity in five large nuclear power plants by the year 2030. Nuclear power plants use uranium as fuel. Uranium is extracted from uranium mines. Generating electricity with uranium does not produce carbon dioxide. The amount of uranium required for this package will be available for at least one hundred years, even when more countries will start to use uranium and thus increase global use. It is very likely that new uranium sources will be discovered, in which case the nuclear power plants can be supplied for a long time.

The website’s front end information was heavy with ethical clearance related information in line with guidelines stipulated by the Australian National Statement on Ethical Conduct in Human Research (www.nhmrc.gov.au). This information pre-empted the commencement of the ICQ resulting in several introductory pop up six screens needing to be viewed prior to instructions for completion of the ICQ became visible. Participants were required to confirm that they understood the and agreed with the conditions of the website, e.g., they understood it to be anonymous, voluntarily, and that data sourced via the ICQ would be used in reporting on the research findings, including but not limited to publications in several journals, and that the findings would be made available to them should they require. A brief overview of the two organisations conducting the research was provided, including contact details for corresponding representatives prior to a participant commencing the ICQ. The figure below depicts how this information was provided to the respondents.

Figure 3. Project and organisation information and ethical consent

Once participants had received information on the organisations and consent was provided, they were invited to register by creating a log in name using a unique URL provided to them in an invitation to complete the ICQ; the URL to be used to assess each of the different components of the ICQ (11 in total). The impetus behind this requirement was an inability to integrate all aspects of the ICQ into one single website. As a result, respondents were directed to an online survey company’s website at predetermined points throughout the ICQ host website via visual links.

In order to ensure that data collection from the different components of the website were readily identifiable to one respondent, the log in name was mandatory and once completed triggered background programming to register data from the survey in a panel using the log in name to denote the respondent.

Each component hosted by the online survey company website included this requirement. This imposed some difficulty for respondents. Other implications and consequences of the use of this design will be discussed in the second study.

The respondents were given information on the current use of energy and ways in which energy is produced. Next, they were explained what the frequent use of oil, gas and coal means for the climate, by explaining the role of carbon dioxide in global warming. Respondents were given consequences to evaluate that are expected to occur when the Earth’s temperature rises as much as expected by scientists. They were also asked to state their overall evaluation on global warming. The series of figures below show how this formation was provided to the respondents.

Figure 4. First screen to link to information on energy and its uses

Figure 5. Pop up tabs were used to provide information on energy production and use

Preliminary to the information on global warming, respondents were given information on ways to reduce emissions of carbon dioxide. It was explained that this questionnaire focused on seven options that can help to reduce carbon dioxide emissions. It was made clear to the respondents that three of the seven options were necessary to reduce carbon dioxide emissions (by the required amount – Australia by 33% and the Dutch by 50%) needed in order to address the policy problem. The figures below depict how information was provided for the climate change and global warming topics.

Figure 6. Climate change and global warming information

After this information, an overview page of pictures with a reference to the seven options was provided. Care was taken to keep the pictures as neutral as possible. For example, the option ‘Electricity from wind turbines at sea’ was depicted by a picture of a wind turbine. In addition to a picture, the description of the option was provided, and a question regarding the content of the option to invoke curiosity to find out more about the specific option. To continue the example, for the option ‘Electricity from wind turbines at sea’ the question: ‘what height will the wind turbines have, that are necessary for this option?’ was added. The figures below demonstrate how respondents navigated to the each of the seven options.

Figure 7. Website pages that lead to the seven policy options

Once participants chose one of the seven options to evaluate, respondents received a description of the option. Descriptions of the options contained information on, for instance, the essence of the technologies, the amount and location of plants or fuel cells, conditions for implementation, or the kind of end use. After the general description, respondents were asked to evaluate all the consequences of the option in question.

Respondents were asked to state for every consequence if they thought the consequence to be an advantage, a disadvantage or not important. If the consequence was evaluated as an advantage or a disadvantage, respondents could state how much of an advantage or disadvantage they thought the consequence to be. This was asked on a 9-point scale, with one being a very small advantage or disadvantage, and nine being a very large advantage or disadvantage. This way, respondents could evaluate all the relevant consequences of an option one by one. After rating all the consequences separately respondents were then given the opportunity to provide their overall evaluation by providing a report mark for the option, on a scale of one to 10 (1 = not favourable to 10 = very favourable).

Once all seven options had been assessed, respondents were directed to a pop up screen containing introductory text relating to the three out of the seven options selection requirement. Following this, respondents received an overview of all the options, their overall evaluations and total disadvantage and advantage scores. Respondents were told that, at this point, having read all the information on the seven options, this was their opportunity to change their opinions if they desired. Once any changes were made, respondents were then asked which three of the options they preferred to be implemented on a large scale. A reminder was provided which stated that they could base their choice on their overall evaluations of the options and/or on the total disadvantage and advantage scores.

Following this process, respondents were subsequently asked if there were any options in the questionnaire they thought was absolutely unacceptable if implemented on a large scale. To be defined as ‘unacceptable’ the respondent would need to consider taking action if the Australian or Dutch society considered implementing this option on a large scale. The figures below show the selection of the three options process in order to address the policy problem.

Figure 8. Selecting the three options to address the policy problem

After respondents had provided their overall evaluations and selected their three option preferences, they were asked to give an opinion of the online ICQ, with questions posed focused on the amount, impartiality, clarity and completeness of the information provided. Furthermore, it was asked whether respondents thought the procedure of the ICQ had aided them in making a decision between the different options, whether it was comprehensible, or complicated. Respondents were asked if they had felt restricted in their choice for packages. Finally, respondents were asked what they thought of the website itself; whether they would recommend it, and what they liked or disliked.

For this research the service of a panel recruitment company was used. Participants were paid (AUD $15.13 or EUR €10) to complete the full survey. As the Australian ICQ was newly developed, the necessity of finding sufficient participants was important; the aim therefore was to collect 400 responses from a representative sample of the Australian population. Since the Dutch ICQ had been extensively tested previously, the aim was to gather a minimum of 100 responses in order to test the consistency of the previous Dutch findings.

Data analysis: For the first study, the evaluation of consequences in relation to overall option evaluation scores was analysed. Before respondents made a choice between the seven energy options, they evaluated, one by one, all the consequences of the seven energy options. Respondents stated whether they thought the consequence was an advantage, a disadvantage or not important. When the consequence was thought to be an advantage or disadvantage, they evaluated how much of an advantage or disadvantage the consequence was on a scale of one to nine. In the results section, the average evaluations of consequences and their relation to the overall evaluation of an option are presented and discussed per option. The evaluations of the consequences were recalculated in a way that each consequence was evaluated on a scale of -9 to 9, with -9 meaning a very big disadvantage, 0 meaning neutral and 9 meaning a very big advantage.

Results are provided in a table for each option which includes: the multiple correlation (R), the average evaluation of the specific consequence, the overall evaluation score and the correlations between the evaluation of the individual consequences and the overall evaluation of the option. The correlations give some insight to the relative influence of the different consequences on the overall evaluation score decision. A correlation can vary between -1 and 1, with 0 meaning no relationship between two variables. A correlation of 1 means a perfect linear relation between two variables, in the sense that the values of one variable are perfectly predictable from the value of the other variable. A correlation of -1 also means a perfect linear relation between two variables, however, a negative correlation means that as one variable increases, the other variable decreases, and vice versa. A positive correlation means that as one variable increases, the other variable also increases, and if one variable decreases, so does the other variable. As the correlation between the overall evaluation and the evaluation of a consequence rises, the consequence is likely to play a more important role in the determination of the overall evaluation.

The multiple correlations (R) represent how much the evaluations of the consequences of an option together are connected to the overall evaluation of an option. A multiple correlation can vary between 0 and 1. The closer the multiple correlation is to 1, the more the evaluations of the consequences explain the evaluation of the option, in other words, how much the evaluation of the option is based on the overall information provided. Linear regression analyses were conducted to investigate these relationships.

Although there were multiple consequences for each option, not all were significant influences on decision making with regards to the policy options. Therefore, to avoid long and irrelevant discussions, only the consequences with high correlations to the overall evaluation (or unexpectedly low correlations) are explained and discussed.

3.2 Study Two: The effectiveness of the ICQ online study

Aim: The current project’s secondary aim was to develop an interactive online ICQ tool in order to investigate various design and delivery aspects and its ability to attract and retain users for the duration of the process. Therefore, this study addressed the following research question:

Research question 2: Is it possible to enhance the quality of the original ICQ by making the questionnaire an interactive application?

Method and measures: In order to capture the data for the second study, a set of semi-structured interviews were conducted throughout an “out a loud” test of the online interaction with the Dutch ICQ. The following factors were the focus of the interview questions:

Education Focus

  • Is there interest in the topics of climate change and ways of addressing carbon dioxide emissions mitigation?
  • ICQ – is it education friendly and have the potential for discussion?
  • Is the information provided independent and multiple-angled?
  • Does the ICQ provide local, concrete and practical consequences?
  • How best is a tool like the ICQ disseminated or used in policy development?
  • Is there a link between the study’s website with their information purpose?
  • Is the website and survey user-friendly and a visually pleasing interface?
  • Did the website develop a narrative and storyline to support the content?
  • Did the website present a clear goal for the user with feedback about the progress?
  • Did the website provide the user with some control?
  • Did the website create a social presence?
  • Did the website make use of questions to instil curiosity and enhance deliberation after the policy advice was been given?
  • Did the website create rewards and competition?

3.3 Ethical Considerations

Ethical clearance was approved for all data collection (CSIRO Social Science and Human Research Clearance Number: 012/11). In this research the matching of survey components was essential and therefore, all attempts were made to preserve the confidentiality of the project respondents through the use of a coding system. In addition, all respondents signed consent forms and received information regarding the project’s focus, aims, data collection techniques and confidentiality.

Leave a Reply

Your email address will not be published. Required fields are marked *