Social welfare professionals willing to participate in client information system development – Results from a large cross-sectional survey

ABSTRACT Human-centered design methods should be implemented throughout the client information system (CIS) development process to understand social welfare professionals’ needs, tasks, and contexts of use. The aim of this study was to examine Finnish social welfare professionals’ experiences of participating in CIS development. A national cross-sectional web-based survey on the CIS experiences of social welfare professionals (1145 respondents) was conducted in Finland in spring 2019. This study focused on statements concerning the experiences of end users with CIS development and participation. The results are reported by professional and age groups. Half (50%) of the 1145 respondents had participated in CIS development. Half (56%) knew to whom and how to send feedback to software developers, but most (87%) indicated that changes and corrections were not made according to suggestions and quickly enough. The most preferred methods of participation were telling a person in charge of information systems development about usage problems (53%) and showing developers on site how professionals work (34%); 19% were not interested in participating. Social welfare professionals are willing to participate in CIS development, but vendors and social welfare provider organizations are underutilizing this resource. Social welfare informaticists are needed to interpret the needs of end users to software developers.


Introduction
A thorough understanding of users, their needs, tasks, and contexts of use is necessary for the successful development and implementation of information systems (IS) with good usability. 1 Human-centered design (HCD) approach contributes to the success of IS by improving software quality and increasing user satisfaction. [2][3][4][5][6][7] Early focus on users and end-user involvement are particularly important in the case of complex systems that are used for complicated contexts and tasks, 5 such as those in social welfare or healthcare. However, the demanding task of developing IS can be undermined by poor communication and a failure to identify clear responsibilities for various stakeholders, such as social welfare professionals, managers, IT staff, and IS vendors. 8 HCD has also been suggested as a potential approach to achieve organizational innovation and change. 9 Surveys, usability tests, and interviews are the most commonly used HCD methods for involving potential end-users in IS development. 1,10,11 Moreover, developers may observe and interview end users on site. 12 Larger numbers of end users can be engaged in focus groups. 13 End-user feedback, such as system error reporting, can also be collected with dedicated web-based systems. 14,15 In healthcare, many organizations train and employ physician or nurse informaticists who engage in both clinical work and health information system (HIS) development. [16][17][18][19][20] However, reports on such roles in social welfare are largely lacking, even though many authors have highlighted the importance of social welfare informaticists who actively participate in the design of technology that will shape, guide, and support their practice of social work. [21][22][23] In Finland, IS coverage in healthcare and social welfare has been regularly monitored since the beginning of the 2000s. 24 While practically all public and major private healthcare providers have already stored patient data electronically since 2007 and integrated to the national Patient Data Repository ("Kanta") services since 2015, [24][25][26] progress has been slower in social welfare. Only in 2017 did the availability of electronic client information systems (CIS) in public social welfare organizations exceed 90%. 24 However, of the 3,971 units registered as private sector social service providers and mainly providing institutional care for the elderly or the disabled, or in child welfare, only half reported electronically storing the majority of client data. 24 Perhaps as a consequence of the slower progress in social welfare in comparison with healthcare, the user experiences of physicians have been monitored nationally since 2010 and the experiences of nurses since 2017, [27][28][29] but the first national survey among social welfare professionals was only conducted in 2019. 30 Information technology is seen as a means to transform the social and healthcare system. 24 In Finland, the strategic aims are to ensure the access of social welfare and healthcare professionals to information systems that support their work, refinement of information and knowledge management, and interoperable and modular architecture. 31 This strategy has pushed many social welfare organizations to develop and update their CIS. Moreover, some IS transformation projects seek to combine social welfare and healthcare data into a shared information system. [32][33][34] In recent years, interest in end-user experiences of CIS has grown. [35][36][37][38] These studies have highlighted usability problems and experiences of managerial and reporting needs ruling over the needs of frontline professionals. 8,[35][36][37][38][39] However, research into the experiences of social welfare professionals while participating in CIS development is scarce; the few, mainly qualitative ethnographic investigations have been conducted in Australia, England, New Zealand, and Scotland 21,35,36,40-42. These studies have only comprised a small number of social welfare professionals and they mainly focus on their user experiences rather than seek to comprehend how the end users could be engaged in CIS development. In a study conducted among physicians and nurses in Finland, the end users were dissatisfied with their ability to influence HIS development. 43 While some studies suggest that younger healthcare professionals are more contended with the usability of HIS than the older ones., 44,45 younger age has not been associated with increased satisfaction among Finnish healthcare professionals. Similarly, younger clinicians have not been more contended with the vendors' responsiveness than the older; however, they are more willing to participate. 43 The aim of this study was to explore the experiences of end users participating in CIS development in a large Finnish national survey conducted among social welfare professionals.
The research questions were as follows: • RQ1. What experiences do social welfare professionals have with giving feedback to CIS developers? • RQ2. Have they participated in CIS development? • RQ3. How would they prefer to participate in CIS development?

Central concepts
Since occupational positions in the field of social work and social welfare vary between countries depending on socio-political contexts and other factors, we chose to apply a generic concept of a social welfare professional when describing the respondents, regardless of their occupational position (e.g., social workers, social counselors, or practical nurses). In Finland, licensed social workers have university-level master's degrees in either the social sciences or political sciences. Social counselors have bachelor's degrees in social services from universities of applied sciences. Many providers, particularly in the field of institutional care, have degrees in nursing (registered nurses/practical nurses).
HCD was used as a theoretical framework for this study. The international standard (ISO 9241-210:2019) describes HCD as "an approach to systems design and development that aims to make interactive systems more usable by focusing on the use of the system and applying human factors/ ergonomics and usability knowledge and techniques." 46 Most HCD techniques and methods require the participation of users.
Usability refers to the extent to which a system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. 46

Social welfare organization professionals in Finland
In Finland, local public authorities (municipalities or federations of municipalities) are responsible for organizing and funding general social services for the entire population and special services for certain population groups; the Social Insurance Institution of Finland (KELA) manages social income support. Local public authorities may purchase social services from private service providers, including non-governmental organizations (NGOs) (n = 3,971 in 2017). 24 Social welfare and healthcare data are stored in separate registries; client data documented by social welfare professionals of, for example, elderly services are stored in the social welfare registry, but the physician addressing the health problems documents data in the healthcare registry. In many contexts of use, sharing client and patient data across social welfare and healthcare is only allowed after specific consent is given by the client/patient.
In 2018, most (3,888; 95%) social workers were employed by municipalities, and fewer (196; 5%) worked in the private sector/NGOs. Fifty-six percent (5,638) of social counselors worked in municipalities, and 44% (4,384) worked in private sector/NGOs. The work tasks of social workers mainly focus on managing cases and making decisions about services and benefits, whereas social counselors usually provide services during scheduled appointments or at housing service units, most often in child welfare and disability services. Moreover, 32,227 and 18,284 practical nurses provide daily care mainly for the elderly and disabled persons in municipalities and the private sector/NGOs, respectively. 47,48

Design of the study
This study was part of a large national cross-sectional CIS user experience survey conducted among social welfare professionals. 39 The user participation methods in the questionnaire were (Table 1;  The survey questionnaire was adjusted from a validated National Usability-focused HIS Scale (NuHISS) instrument for physicians. 49 The statements were originally created for the national physician survey in 2010 and has been utilized for usability-focused surveys involving physicians and registered nurses. 27,28,50,51 The survey included a section on the experiences of users participating in IS development; the results for physicians and registered nurses have been previously reported. 43,52 The adjustments for the social welfare professionals' questionnaire focused on the viewpoints of social workers and counselors. Registered and practical nurses working in social welfare mainly use HIS instead of CIS, and therefore the statements were not modified to consider their tasks and information needs. The demographic variables included in the survey were gender, level of education, employment sector, social welfare service line, and age group.

Data collection
The data were collected in April-May 2019 via an online questionnaire. To reach professionals from the various social welfare service lines and organizations, the questionnaire was distributed through the largest trade unions and associations with members from the field of social welfare. An invitation to participate with a non-individualized link to the questionnaire was sent by e-mail to working-age social welfare professionals under 65 years of age who had provided e-mail addresses and were members of the Talentia trade union for social welfare professionals with higher education (approximately 26,000 members; 11,302 e-mails sent), the Trade Union for the Public and Welfare Sectors (approximately 200 000 members; 6744 e-mails sent), or the Finnish Society of Social Work Research (565 members; 510 e-mails sent). The link was also distributed on social media when the number of respondents was at risk to remain low. The study protocol was approved by the ethics committee of Aalto University. At the beginning of the questionnaire, there was a privacy statement, thereafter all voluntary respondents gave informed consent to participate in the study. No identifying information was collected as part of the survey and the researchers were not able to identify individual respondents.
The exact response rate cannot be calculated since the memberships to the above-mentioned unions/society may overlap and because of data protection regulations, the unions/society were not allowed to reveal to whom they had sent the e-mails. Additionally, some respondents may have received the link from colleagues or via social media.

Analysis
For the analysis of statements Q1.A-D (Table 1), of the five-point Likert scale assessments, "Fully agree" and "Somewhat agree" were combined and denoted by "Agree," and "Somewhat disagree" and "Fully disagree" were denoted by "Disagree." Statistical analyses were carried out using SPSS 22 (IBM Corp, Armonk, NY). A chi-squared test was used to compare categorical variables. Statistical significance was set at p < .05.

Results
A total of 1145 social welfare professionals responded to the questionnaire. Of these, 26% (n = 302) were social workers, 30% (n = 343) were social counselors, and 14% (n = 157) had leading or managerial positions. In the group "others," 30% (n = 338) comprised other professionals and experts working in the field of social welfare; of these, 66 (6% of all respondents) were nurses. A total of 93% of the respondents were female, and 53% were under 45 years of age. Most of the respondents (80%) worked in public sector. Child welfare was specialty of 35% of respondents.

Experiences of giving feedback
Approximately one-third did not know how to provide feedback to vendors and developers (Q1.A). In total, 22% thought that the vendor was interested in end-user feedback (Q1.B); 18% and 13% responded that corrections and development ideas were implemented according to suggestions (Q1. C) or quickly enough (Q1.D), respectively (Figure 1, supplementary Table 1). Those in managerial or leadership positions appeared to have somewhat more positive experiences than social workers (67% of managers vs. 55% of social workers agreed with statement about providing feedback to vendors, and 37% vs.16% and 26% vs.14% agreed with statements about vendors being interested in end-user feedback and implementing corrections according to suggestions, respectively).

Participating in CIS development
Half of the respondents had not been engaged at all in CIS development; 26% of social welfare professionals working in leading or managerial positions, but 44% of social workers, and 49-64% of others reported no participation (p < .001 managers vs. others) (Figure 2, supplementary  table 3).

Preferred Methods to Participate in CIS Development
The most preferred (53%) method of participating was communicating problems and development ideas to a person responsible for CIS development within the organization (Figure 3, supplementary Table 2). Showing developers on site how social welfare professionals work was endorsed by 34% of respondents. Sending feedback via e-mail (25%), participating in focus groups (24%), or documenting feedback on a dedicated webpage (15%) were less preferred. Of all the respondents, 19% were not interested in participation at all.

Relation of age and experiences of CIS development
When the age groups were compared, the youngest (under 35 years) and the oldest (over 55 years) appeared to have participated less than the other groups ( Figure 4; supplementary table 4). The youngest respondents did not have more positive experiences with vendor reactions to feedback than the older respondents (supplementary table 5). The most popular participation method for all age groups was communicating with a person responsible for CIS development within the organization (Q3.E.), but those over 55 years of age favored this method the most. Only 25% of respondents over the age of 55, compared with 39% in the two youngest age groups, were interested in showing software developers how they work (Q3.A). The oldest were also the least interested in participating in focus groups (11% vs. 23-30%). The youngest were the most willing to send development ideas to developers via e-mail (34% vs. 20% in the two oldest groups) and on a dedicated webpage (20% vs. 9-14% in the two oldest groups). In total, 17% of the two youngest age groups, but 24% of the oldest respondents, were not interested in participating at all (supplementary table 6).

Discussion
Good usability is essential for developing successful and high-quality IS. Applying a HCD approach is one of the central means for achieving good usability. 1 Moreover, user involvement in IS development should be active, by participating in design, acting as a source of relevant data, or evaluating solutions. 1,47,53 The main finding of this study was that social welfare professionals are willing to participate in CIS development, but vendors and social welfare provider organizations are not fully utilizing this resource.

Vendors considered unresponsive to CIS-related feedback
When compared with other social welfare professionals, those working in leadership and managerial positions reported more positive experiences regarding their feedback to CIS vendors and developers; however, only 26% and 17% of leaders/managers indicated that their suggestions were implemented according to their wishes and quickly enough, respectively. Interestingly, social workers provided the most negative assessments. There are several possible reasons behind this finding. In recent years, the development of CIS has been heavily directed by national guidelines. It is likely that frontline social workers have not been included in these discussions; therefore, they may not have fully understood why some changes were implemented in a particular manner. Moreover, they may not personally agree with the guidelines. In addition, nationally enforced CIS developments may emphasize data collection over practical informational needs and the objectives of social work. 54,55 Presumably the employers of social care workers have not allocated working time for development; this may have led to the underrepresentation of social workers in various focus groups or the overlooking of their viewpoints. Some researchers suggest that end-user tools may have been mainly developed between vendors and IT staff, 8 or end-user participation may have been used to gain buy-in for technology instead of seeking collaboration. 53 Since the developers do interact with end users, 56,57 the most probable explanation for end users experiencing unresponsiveness is that the vendors and  Table 2 developers fail to inform end users about developments and corrections in a timely and easily understandable manner. Simply fixing problems/issues is ineffective if users do not know they have been fixed and may lead to end users feeling that their ideas have not been considered at all.

Social welfare informaticists and on-site developers highly valued
Of the various HCD methods for user participation, 1 the respondents of this study favored the most explaining problems/development needs to a dedicated person in charge of CIS development in the organization. A social welfare informaticist could serve as an interpreter between the IT-focused  language of the developers and the field-specific uses of the language of the end users. 8 They could also explain to both groups what is possible and what is not, either from a technical or workflow perspective. Indeed, Lagsten and Andersson (see ref. 8) identified that a lack of common language and structures for describing social work to CIS developers hampered critical discussions of CIS improvements. In healthcare, many organizations have employed physician and nurse informaticists who also use the information system themselves in clinical work. [16][17][18][19][20] International requirements for such roles in the social welfare sector are lacking, but Gillingham (see ref., 58 ) for example, has suggested that social workers and their managers should take the lead in CIS development as they know their work best. Similar to health care professionals 9 frontline social welfare professionals need to be supported in their attempts to act as champions of innovation and change. Social welfare informaticists would enable the development of CIS that multilaterally and effectively support the practice of social work, data utilization, and service development for various stakeholders. 23 They are also needed when developing processes and workflows alongside information system development.
One third of respondents were willing to show their work to developers. On-site visits provide developers with a unique opportunity to understand how an information system is used and to hear immediate feedback on possible development plans. However, an individual end user does not necessarily represent a typical user or may not always be aware of the strategic goals of the organization. Therefore, site visits should always be complemented by collecting feedback from a diverse audience. 59,60 One fourth of respondents were interested in participating by attending focus group meetings. It may be difficult to arrange time for frontline social welfare professionals who also see clients and manage cases. Moreover, to work efficiently, the participants of these groups should have a thorough understanding of not only the use contexts and end-user needs but also the CIS itself. We suggest that resource-intensive focus group meetings should be used sparingly. Instead, social welfare informaticists could gather end-user feedback through everyday work contacts.
From a vendor/developer perspective, collecting feedback on dedicated webpages or via e-mail is often the most feasible method of obtaining detailed information on individual problems or issues in the CIS. However, providing screenshots and accurate descriptions on use contexts may require a considerable amount of work from the end users. Vendors should develop automated feedback collection tools built into the information systems. 61

Leaders and managers had participated more actively than the others
Those working in leadership and managerial positions had participated more in CIS development than other social welfare professionals. Accordingly, managerial needs may have become overrepresented in the development of CIS. 36 As most leaders/managers in social welfare do not participate in frontline social work, special attention should be paid to routine workflow design by also engaging nonmanagerial professionals; this applies particularly to usability testing. On the other hand, leaders and managers are likely to be more aware of the strategic and quality goals of the organization, as well as national CIS documentation requirements; indeed, collaboration between the various stakeholders is important. 8 Moreover, more attention should be given in social welfare organizations to communicating these goals to frontline professionals.

Younger social welfare professionals willing to participate in CIS development
The finding that the youngest age groups had participated less than the older groups was not unexpected since CIS development requires an understanding of the work processes and strategic goals of an organization. The more experienced workers have been described as being more able to make constructive suggestions for changes to CIS user interfaces and functionality to serve their purposes more efficiently than novice users. 40 However, since (at least in Finland) the majority of older social welfare professional generations have a relatively short history of electronic documentation and utilization of CIS in daily work, they may try to copy paper-based workflows into the CIS instead of being able to reevaluate the whole process. Moreover, novice users may identify usability problems that may not be noticed by experienced users. 40 Except for communicating via a social welfare informaticist, the youngest respondents were more interested than the oldest respondents in all other participatory methods included in our survey; while this has also been reported in studies involving physicians and nurses, 43 we found no studies that would explain this difference. However, vendors or social welfare organizations should not assume that, compared to older workers, the younger have more positive impressions of vendors being responsive to feedback and implementing CIS improvements.

Only a few not interested at all in participating
One of the central findings of this study was that a relatively small proportion (19%) of social welfare professionals were not interested in participating in CIS development. This is in accordance with the findings involving physicians and nurses. 43 Moreover, half of the respondents had participated to at least some extent. It is likely that those who responded to this survey have been more actively involved in CIS development than those end users who did not respond; this may have also resulted in overestimating the proportion of those interested. However, it is evident that a large proportion of social welfare professionals are willing to participate in CIS development. Since the majority were not satisfied with the end results of the interaction between end users and system developers and vendors (changes and corrections were not made according to suggestions and quickly enough), social welfare organizations, CIS vendors, and developers should seek to find more suitable means of improving this experience. The principles and methods of HCD provide tools for the involvement of end users for all stages of the IS design process; this may in turn improve the chances for successful IS development and outcomes. 1

Limitations
One of the central limitations of our study was that we could not assess the response rate; however, to our best knowledge, with 1145 respondents, this was the largest survey to date among social welfare professionals concerning their experiences with participating in CIS development. Moreover, the results can be considered generalizable since the respondents represented social welfare service lines in similar proportions as reported nationally. 48 It is evident that social welfare comprises different services and the social welfare professionals may have different educational levels in different countries; national surveys are needed to best implement HCD methods. Even though we did not ask the respondents open-ended questions about participation, the strength of our survey method was that we were able to also include those who do not usually participate in development; most previous observational studies have focused on those who are already involved, and the numbers of study subjects have been low. 21,35,36,[40][41][42] One limitation deserving attention is that our survey did not cover the experiences of the largest group of social welfare professionals in Finland i.e. practical nurses. However, they mainly use HIS instead of CIS, and their information needs differ considerably from those of social workers and social counselors. Moreover, in most international contexts, their tasks would be regarded as healthcare instead of social welfare. We suggest that their experiences should be covered in a separate survey.
We also did not ask how the participants had participated in CIS development; this should be addressed in future studies to assess whether, for example, those who have social welfare informaticists in their organizations or have had developers on site are more satisfied. Moreover, the questionnaire did not cover the phase of software development in which the respondents had participated. End users tend to be more active in reporting errors and development ideas during the implementation phase of a new IS; however, at the time of the survey, there had been no major changes in the CIS brands in use in Finland. 62

Future research
Although the usefulness of end-user participation in IS development is widely acknowledged, there are few studies on the experiences of social welfare professionals. Indeed, we encourage similar surveys in other countries as well. Moreover, large studies, both surveys and experimental studies, are needed to explore the usefulness of the various methods in the field of social welfare. Surveys should be complemented with interviews and usability tests, but the viewpoints of larger audiences are difficult to reach with qualitative ethnographic studies.

Conclusion
Our findings emphasize the importance of end-user involvement and the deployment of appropriate HCD methods for CIS development. One of the greatest challenges in the relationship between end users and developers is communication. End users must not only verbalize their needs to developers, but developers must also communicate when and how the needs of end users will be implemented; positions for social welfare informaticists should be established in organizations to alleviate this lack of communication. Social welfare professionals are willing to participate in CIS development, but vendors and social welfare provider organizations are not fully utilizing this resource.

Disclosure statement
Martikainen has been previously employed by one of the CIS software vendors included in the study. Lääveri, Salovaara and Tyllinen are/have been employed by a CIS software vendor not included in the study. The employer did not provide any support, financial or otherwise, for the study. Moreover, the vendor was not involved in the design of the study or in the collection, analysis, and interpretation of the data.

Funding
The author(s) reported there is no funding associated with the work featured in this article.