The thematic literature review was based on an extensive search of electronic databases (Medline, CINAHL, PubMed) as well as web-based searches (Google, Google Scholar) for additional material, such as reports commissioned by government as well as non-governmental agencies. Due to the limited scope of the literature on palliative care, sources dealing with online cancer information were included in this literature review. Permutations of search terms (Palliative, Cancer, Care, Online, Internet, Information, Needs) were used to collect a total of 388 bibliographical entries in an EndNote file. A researcher scanned the abstracts of these articles for relevance, thus eliminating 314 articles. The remaining 74 articles were sourced for inclusion into the literature review. In addition a separate literature review was conducted focusing on the available clinical evidence in subcategories of palliative care such as pain, medication, and psychosocial needs. For the clinical subject areas systematic reviews submitted to the Cochrane database, clinical guidelines, as well as expert consensus panels were included. The research underpinning the clinical information included on the website was evaluated and ranked by clinical palliative care experts. Due to the dearth of research-based evidence in sub-categories such as psycho-social needs, the PRG decided to include also what is considered 'best practice' in those fields.

A short online survey was designed to provide a cursory overview of users and their palliative care information needs. The 10 question survey was posted on the existing PCV website. Participants were asked:

• why they accessed the website

• whether they were health professionals or patients/caregivers/members of the public,

• what kind of information they were looking for,

• whether they found the desired information,

• to indicate areas that should be considered for inclusion in the website,

• whether it was easy to locate the information on the existing site, and

• to comment information gaps in the current site.

Internet users who accessed the site were asked to complete the form online. The survey was posted on the internet for a total of five and a half months beginning from 8 October 2003. Response to the survey was anonymous. The webmaster collected the responses and coded and pre-compiled the data. Subsequently, the responses were screened, duplicates or void forms were eliminated, and the pre-compiled data was analysed.

Furthermore, an extensive search of existing health websites was undertaken to identify the key features of reputable or 'Health on the Net' (HON) accredited websites. The Economic and Social Council of the United Nations accredited HON Foundation is the leading organization promoting and guiding the deployment of useful and reliable online medical and health information, and its appropriate and efficient use.

Taking into account the preliminary findings from the literature review, the web search, and the results of the pre-development survey, both UWGs drafted a detailed gap analysis framework addressing the information needs of the stakeholders they represented. Moreover, the Project Reference Group as well as key stakeholders and the experts from the Centre for Online Multimedia and Educational Technologies (COMET) at La Trobe University, evaluated the original website of Palliative Care Victoria for content, relevance, accessibility, links, search potential, design features and language level. The original website catered predominantly for palliative care 'insiders' (mainly general practitioners and physicians) and posted a series of pamphlets and press releases that could be handed on to patients. The literature reviews in conjunction with the website search as well as guidelines provided an overall framework for content and design, whereas the online survey gave an initial indication of the kind of subject areas health professionals, patients, and caregivers might expect the future site to cover. In this sense, the 'gap analysis' provided very rough guidelines for the construction of a beta site. However, it is important to remember that, because of the formative, iterative process that underpinned the design of the website, needs and gap analyses were ongoing as stakeholder representatives co-designed and debated the content of the site through the numerous iteration cycles.

An effective Project Reference Group was formed. The Project Reference Group (PRG) brought together nine professionals active in a range of palliative care-related fields and who held senior positions in councils, associations, and academia. Moreover, the PRG included a Project Officer who had extensive knowledge of the palliative care field. The Project Officer moderated most group discussions. The sizeable networks of these individuals were instrumental during the implementation phase of the project as many of the User Working Group members were recruited through these networks. Moreover, these participants brought to the project expertise in clinical palliative care, project management, team leadership, online educational technology, and research/evaluation techniques that could be drawn upon over the course of the project. Several members of the PRG joined an editorial panel that took responsibility for drafting editorial guidelines and monitoring content. The PRG functioned as the central mediation vehicle and most editorial decisions were discussed within this forum.

Two User Working Groups (UWG) were formed to participate in the sourcing and evaluation of content. One UWG drew on the expertise of focus group participants brought together by the Cancer Information & Support Services (CISS) run by The Cancer Council Victoria, as part of their activities directed at identifying unmet needs for people affected by cancer. The working group included 11 cancer patients, caregivers and volunteers. The other UWG was made up of 12 health professionals and included four general practitioners, two palliative care physicians, and six palliative care nurses representing nine organisations. Again the networks associated with these members were very extensive and greatly facilitated the user-based evaluation of the process. It was envisaged that these groups would meet on a monthly basis. UWG participants as well as members of their professional networks were asked to provide feedback in the form of summary grids (see additional file 1) after each design cycle.

In addition to the PRG and UWGs, a substantial number of experts in fields such as palliative care, nursing, community support, or allied health made major contributions to the project. Also, a consumer focus group was conducted by a market research firm. This focus group dealt primarily with the look and feel (colour schemes, navigability) of the website. Because representatives of all major stakeholders were involved in the reference and working groups, it is reasonable to suggest that these stake holder groups had a direct input into the design of the web site. Moreover, bearing in mind that 197 and 166 participants respectively responded to the pre-development and evaluation online surveys, it is likely that over 300 individuals contributed to the design and evaluation of the PCV website.

An evaluation survey was attached to the new website. Not withstanding minor changes, the evaluation survey mirrored the content of the pre-development survey. Based on the experience with the pre-development survey, it was decided to separate the information for the public and health professionals. The pre-development survey did not make this separation. Moreover, the evaluation survey was more structured providing a list of possible responses that participants could tick in addition to an 'other' category followed by an open-ended prompt to specify this response.

The literature review revealed a growing recognition of the existence of unmet information needs regarding palliative care practice 6935363738394041424344. Recent evidence suggests that doctors, nurses, patients, and caregivers increasingly turn to the internet to meet some of these information needs 1213141618202245464748495051. One of the key concerns that surfaced in the literature focusing on online health is the issue of quality. A large number of studies express concerns with regard to the quality of information provided by web sites that post cancer health information on the Internet 52535455565758. Most of these studies alert to the fact that websites commonly display misleading, incomplete, and/or erroneous information and that the confidentiality of data is often not assured 55596061626364. Surveys of websites and the way users access them suggest that it is difficult and time consuming to identify good online resources and that the capacity of users to find and access valuable information is limited 6566. For most researchers the key problem faced by users is how to evaluate the quality of online information and how to establish the trustworthiness of sources.

In particular, researchers 467576466676869707172 identify the following key issues:

• Lack of editorial control,

• misleading or incomplete content,

• inappropriate design,

• inappropriate cultural bias,

• inappropriate commercial content,

• lack of readability (especially for cultural minorities or people with disabilities),

• inappropriate navigation tools,

• overall disorganisation of the site,

• accessibility of audio and visual resources, and

• problems regarding volume management and maintenance.

In response to these quality concerns, authors propose the use of tools to evaluate the quality of health information web sites 7374, and/or to introduce obligatory standards or evidence-based editorial criteria to improve the quality of health information sites 54667576. Yet, recent contributions to the field have been critical of evaluation tools, claiming that they are largely ineffective 77.

Other commentators have recommended the design of designated portal sites or gateway servers that organise and edit web content 78. To some degree, sites such as OncoLink, the Cancer Net, Cancer Help UK, and others fit into this category. A number of articles give an overview of 798081828384 and/or evaluate such sites 8586. Designated portal sites or gateway servers generally enjoy impressive approval ratings among clinicians as well as the general public. Nevertheless, recent articles have raised the issue of sustainability and costs, update, and maintenance work faced by online health information providers 87. More recent evidence questions the proposition that the design of web portals lastingly improves the quality of online health information [88].

In summary, the literature suggests a need for a more comprehensive information and education strategy focusing on palliative care. It makes it clear that an understanding of the audiences to be targeted as well as their online information needs is crucial for the development of reliable and effective health information websites 22. Moreover, it appears that for online health information resources to be efficient, four basic conditions have to be met: content needs to be clinically authoritative, accessible, sustainable, and has to respond to the information needs of potential user groups. Furthermore, the design process benefits from an ongoing involvement of stakeholder groups. However, there is a dearth of evidence regarding the information that should be included in a site providing educational information on palliative care to health professionals as well as to patients and associated caregivers as well as the methodology might be successfully employed to that end.

The survey attached to the existing website generated a total of 197 responses (see Table 1 below). All responses were considered valid, although not all questionnaires were completed in full. The majority of respondents who did not complete all ten questions left out the open ended questions. One hundred and thirty two (67.01%) respondents identified themselves as health professionals and 63 (31.98%) as non-health professionals.

The most important reasons respondents provided for accessing the existing palliative care website were to seek information in support of their role as health professionals (79.26%) and professional practice (13%), to find available resources (66.21%), and to find information that would assist them in their care-giving role (15%). Of the respondents 128 (64.97%) indicated that they had found the information they were looking for and 68 (34.52%) stated that they did not. Respondents to the existing website thought that in-depth information about topics such as pain, grief, volunteering and ethical issues should be included. Also noted was the absence of links for volunteers, a site map, a list of recommended publications, a listing of employment opportunities and short courses, comprehensive information for professionals, course information on professional practice, and contact details.

When asked if it was easy to find the information that they were seeking, 130 (65.99%) of the respondents to the existing site gave an affirmative response, whereas 57 (28.93%) responded negatively. Sixty two (31.47%) participants specified their answer. Of these, 26 (13.20%) gave positive feedback, whereas 20 (10.15%) provided constructive criticism. The issues raised by these respondents were:

• the need for better navigation tools,

• the site's limited content,

• the ineffectiveness of the site's search engine, and

• the fact that the site had not been updated.

The expert panel discovered limitations in the organisation, content, language, accessibility, design, navigability, and readability of the existing PCV website. To some degree this finding was expected, given the limited resources available for the ongoing support and development of a site mainly administered by volunteers. Among the most important shortcomings noted by the panel was the very limited scope of the website, the lack of effective search capacity, the complexity of the language used and accessibility issues for people with reading difficulties. It was felt that the website was largely providing information to 'insiders' in the palliative care community but not to general health professionals nor to a substantial segment of caregivers, patients, and the general public.

The two working parties considered material from the gap analysis, the website search, as well as a preliminary analysis of emerging survey responses. Also the working party members' opinions of unmet needs as identified from their own experience or from interaction with their peer networks. Discussion ensued about how much information should be available for a quick scan approach, how much should be printable information sheets or pamphlets, and what links should be included to provide greater depth of knowledge. Each working party developed a framework for the health professional or the consumer web portal respectively. An overview of these frameworks is provided in Tables 2 and 3.

Moreover, working parties in conjunction with the PRG defined the overall scope of the new website delimiting the topics that would be covered by the site. Excluded from the site were topic such as careers and financial entitlements.

Because this phase has been published in detail elsewhere only a short summary of its key elements of this phase is given. The formative user evaluation underpinning the project worked exceptionally well. Perhaps the most surprising element of this phase was the level of support that participants were willing to invest in the project. Whereas the UWGs provided crucial information during the initial stages of the project, the editorial panel in conjunction with key user groups associated with the representatives participating in the UWGs provided valuable input during the evaluation of the beta test site. As anticipated, feedback occurred continuously throughout the project and was circulated through the PRG and UWGs. As a result, the PRG and UWGs were able to respond quickly to user feedback and shape the content and overall design of the website accordingly. Expert evaluators became important mediators of conflicts. Referring to 'good practice' guidelines and by providing their 'expert opinion', they, supported by the PRG, managed to resolve diverging views as well as conflicts of interests that surfaced within UWGs.

The online survey posted on the new website resulted in a total of 166 responses (see Table 1 below). A total of 121 (73%) respondents identified themselves as health professionals and 45 (27%) as care givers, patients, or members of the public.

Care givers, patients, and members of the public responding to the new website sought primarily information regarding the health care system (96%), about palliative care services in Victoria (40%), as well as information in order to help others with an illness (33%), help the person they cared for (31%), and help them cope (27%). Health professionals responding to the new website (B) sought information about education (50%), symptom management (37%), advance care planning (31%), loss and grief (25%), talking with clients and families (23%), Palliative Care Victoria (22%), death (21%), services available to their clients (21%), and a resource website for clients (17%).

Of the respondents, 137 (82%) indicated that they had found all or some of the desired information and 18 (11%) stated that they did not. Respondents of the evaluation survey thought that topics such as further education, conferences, scholarships, as well as lists of places to go to as a family, signs of dying/clinical information, volunteering and financial entitlements should be considered.

A total of 122 (74%) respondents claimed that it was easy to access the desired information, whereas 38 (23%) claimed that it was not.

Comments focusing on the new website were positive regarding content and design.