iStock/metamorworks We present here a conversation between Nishan Chelvachandran, founder of Iron Lakes and chair of IEEE SA’s Trustworthy Technological Implementations of Children’s Online and Offline Experiences Industry Connections Programme, and Katina Michael, director of the Society Policy Engineering Collective in the College of Global Futures at Arizona State University.
Nishan Chelvachandran: Katina, how are digital products and services designed with children in mind?
Katina Michael: Organizations are gaining awareness that digital products and services targeted at the children’s market segment need to go beyond adopting the “mindset” of a child [1]. Rather, it is necessary to actually invite children to participate in the design process [2]. Participation takes the form of direct stakeholder consultation from the outset, allowing decisions to be influenced, if not driven, by children and other stakeholder types [3], [4]. Nishan, given your experience, can you describe in more detail what this awareness means in practice?
Organizations are gaining awareness that digital products and services targeted at the children’s market segment need to go beyond adopting the “mindset” of a child.
Chelvachandran: Basically, key stakeholders, primarily children, are involved in the design process. By introducing co-design and co-creation elements, the design approach shifts from “Made for…” to “Made with/by…” [5]. When participation and engagement are integrated at key design and creation stages, it becomes less likely that systems will be designed in ways that are unsuitable for kids [6], [7]. Katina, how would you say that all of this relates to the ISTAS21 conference theme “responsible innovation?”
Michael: Incorporating a responsible innovation framework [8] provides a vehicle for raising questions related to the social impact of online services on children [7]. The four dimensions of responsible innovation are anticipation, reflexivity, inclusion, and responsiveness [9]. Including children as direct stakeholders in the design process allows designers to better anticipate potential risks and respond to these with reflexivity.
Chelvachandran: Would you call the goal of such products and services specifically designed with children “technology in the public interest?”
It is necessary to actually invite children to participate in the design process.
Michael: Absolutely. Public interest technology (PIT) is central to the philosophy of designing digital services with children [10]. PIT is also the outcome, in the form of a robust digital product or service that can benefit children [11]. In designing for children, PIT could entail several approaches [12], [13]. Intentional design spaces engage the whole child with creative play elements in an online space [14]. Interaction design applies fundamental design thinking principles [15], embedding diversity and accessibility within functional specifications, rather than treating them as an afterthought [13]. Nishan, how do these ideas relate to the creation of the IEEE 2089–2021 Standard [16]?
Chelvachandran: The IEEE 2089–2021 Standard for an Age Appropriate Digital Services Framework was based on and inspired by the 5Rights Principles for Children [17], which also fed into the Age Appropriate Design Code [18] created by the U.K.’s Information Commissioner’s Office. For me, this standard does a great job with specificity, establishing requirements for the inclusion of appropriate and accountable stakeholders across organizational transactions and clearly defining processes in specific stages [19]. These processes create what is essentially a set of fair terms to proactively address the expectations of children and parents, ensure that legal obligations are met, and incorporate the types of risk management/mitigation approaches that are considered best practices in wider cybersecurity circles [20]. Katina, can you tell us a little more about the standard?
Michael: Rather than describing hardware or protocols, IEEE 2089–2021 is a process—a set of activities organizations engage in during the design of digital services for children (Figure 1). Before the design lifecycle begins, there is a preparation phase where the organization considers the products and services it wants to develop strategically for the children’s market segment, identifying objectives, and more. Then the design team applies “5Rights Principles for Children” to implement the age-appropriate digital services framework as outlined in the standard itself, conducting risk-based, age-appropriate design and development, followed by age-appropriate deployment, operations upgrade, monitoring, and decommissioning.
Rather than describing hardware or protocols, IEEE 2089–2021 is a process—a set of activities organizations engage in during the design of digital services for children.
Chelvachandran: Katina, can you tell us more about the stakeholders engaged in these 2089–2021 processes? You have already mentioned children; who else plays a role?
Michael: A wide range of stakeholders are involved in the products and services that impact children. Internal stakeholders include the many roles required to commission, develop, build, and market products and services, such as: top management champions, systems experts, age-appropriate leads, child rights advocates, senior product managers, moderators, and transparency managers. The design team must have defined competencies that they focus on in the design process. Primary stakeholders, on the other hand, include both children and adults (parents/guardians, school teachers, social workers, medical practitioners, regulators, and so on). This standard offers a set of processes that engage these diverse groups with each other to develop a product or service that prioritizes the rights and needs of children. Nishan, how might the standard be perceived by the industry stakeholders involved?
Chelvachandran: Hopefully quite well. This standard incorporates best practices into the design thinking process, so it could serve as an inspiring benchmark or a model when developing further standardization work relating to responsible design and the development of human- and environment-centric technologies. Katina, it is not just all positive, though; what about the potential controversies?
Michael: Some software developers might argue that getting children to participate in the design of digital services slows the entire process down and is costly because they have to be accompanied by an adult, and designers working with them must have special governmental clearance (e.g., in NSW, Australia, individuals must go through a Working with Children Check). Designers also need to gain clear consent with regards to several matters from the parents or guardians of children, for the purposes of observing a child using a prototype, or even recording their opinions, if interviewed. You can just imagine a company needing to halt development as a result of risks and harms and particular what-if scenarios they had never thought of because they consulted with the stakeholder too late in the design process. This is exactly why the standard encourages early participation by stakeholders. If we say we “empathize” during the initial design phase, doesn’t that mean we should be speaking with children directly from the outset? Digital services organizations also might feel that they are being forced to comply with a standard they did not contribute to and that the standard might stifle their ability to innovate freely or add additional costs in the design process (e.g., the creation of new roles and responsibilities). The age-appropriate register (AAR) may also raise questions about whether or not risks have been adequately identified with corresponding treatments to minimize potential harms. The AAR is an information repository that focuses on the features and functions of products or services that are used by children. Its aim is to ensure that age-appropriate digital services for children are developed in ways that eliminate both hazards (conditions with a potential for causing harm) and harm (negative events or social developments in children). However, developers might see AAR intervention as a roadblock, even though it will enhance outcomes.
Chelvachandran: Your comments raise the important question, how does a standard such as IEEE 2089–2021 function?
Michael: Standards are not enforceable; they are voluntary documents that set out guidelines that have been established by consensus, with input from subject matter experts and different stakeholders. Accreditation often follows standards development to ensure appropriate employee training and certification toward quality processes. IEEE 2089–2021 aims to ensure that online services accessible by children are safe and that those online services allow children to thrive. Many organizations could, therefore, see implementing IEEE 2089–2021 as providing them an edge over their competitors. The standard may well be adopted by smaller players who are more agile, and by providers of niche digital products or services, for example, using gameplay in education. Nishan, what do you see as the opportunities before us?
Chelvachandran: You made the apt point about standards: that they are not enforceable legal instruments, but voluntary compliance documents that quantify best practices, principles, and guidelines. However, that is not to disregard standards work as toothless. IEEE 2089–2021 guidelines provide the opportunity to proactively create common and equitable frameworks that will allow for the responsible design of products and services for children [21]. Taking a multidisciplinary, multistakeholder approach can inspire further work that is amicably market-compatible [22]. It can also lead to legislative frameworks that are fit for purpose, with a holistic understanding of the emerging challenges that we face.
The standard encourages early participation by stakeholders.
Michael: So, does that mean that standards can concretely help mitigate the risk associated with digital services available to children?
Chelvachandran: Yes, I think so. Rather than wait for the wheels to come off the wagon, for actual harm, a standard like this can prompt developers to take the initiative to co-create, co-design, and co-build the mechanisms that will solve problems traditionally created by the few, but affecting the many. The standard sets the benchmark and provides guidance for parties to reshape their processes in age-appropriate ways; and we can look to further IEEE initiatives, such as the Trustworthy Technical Implementations for Children’s Online and Offline Experiences Industry Connections Programme (which is building a transdisciplinary, multistakeholder consortium) to take an even more proactive approach to develop tangible and deployable methodologies for trustworthy and safe child-centric technologies.
Michael: Nishan, these ideas sound closely aligned with the principle of technological stewardship that was central to ISTAS21.
Chelvachandran: Yes, technological development frameworks are playing catch up in regard to stewardship. Because algorithmic processes have so often fallen short through short-sighted, ill-informed, or biased construction, it is important that we conceptualize, design, and implement technology that reflects the people, communities, and cultures that use it [23]. To be truly human-centric, stewards when designing with and for kids, we must take a transdisciplinary approach to technological design and creation [24]. We need to integrate technical and nontechnical expertise and consider underlying values and intentional decisions. But as we pursue these opportunities, there are challenges to overcome.
Michael: The challenges are many. How to break with the way “we’ve always done things” (driven predominantly by new revenue-generating streams as opposed to need) and how to really value not only “children” but all stakeholder types relevant to the design of children’s digital services [25]. We have been allowing children to access adult digital services, such as social media, without any customization. Penalties have been handed down to organizations by various government agencies (e.g., for the unauthorized collection of children’s data) but how will we make changes to so many existing digital services to make them safe for children [26]? We need to take a good hard look at these digital service offerings, ask whether they are age-appropriate and if they are not, apply the IEEE 2089–2021 Standard in conjunction with local regulations. Age verification poses both security and privacy minefields but is the next step in the suite of possible 2089 Standards.
ACKNOWLEDGMENTS
The authors would like to acknowledge the contribution of their co-presenter at ISTAS21, Moira Patterson of the IEEE Standards Association, for her collaboration. Additionally, they would like to thank Terri Bookman and Heather Love for their editorial support.
JOIN SSIT