Online Products and Services Assessment
E-Governemt Program (Yesser)

A. Problem Analysis

 1. What was the problem before the implementation of the initiative?
A major aim of Yesser is to utilize information and communications technology (ICT) in Saudi Arabia for the promotion and support of transparency, accountability and integrity, and to accelerate work towards the Sustainable Development Agenda for 2030. Central to the e-Government concept is the role of websites as critical channels for providing government information and services to the population online. Yesser undertook analysis studies on government agency websites (190 in total), resulting in a number of important findings. These findings can be categorized as the situation that existed before the initiative began and as the issues that needed to be addressed: • There was a considerable lack of important content, such as no reference to agency policies, no information that could assist in recognizing mandates, and a lack of clarity with regard to what agency policies are, projects being undertaken, budget allocations and progress being made. • As a result, people still believed that they needed to visit physical government department offices (as they always have) to gain access to services and/or important information. While this was inconvenient and difficult for urban populations and the able, it was much more so for rural populations and for those less able, such as the poor, the sick and the elderly, to the extent that for some people government services were effectively not available to them. • Not only websites but associated areas of ICT were not being utilised, such as social network channels, which had been identified as important means for providing information, consulting, and as inclusive platforms that should make users part of decision making processes. • An important cornerstone of e-Governance, the ability for users to gain access to data that could be cross-referenced between agencies, was not being facilitated. • Little or no consideration had been given to those with special needs in the design and capabilities of websites. • No form of feedback was being provided by government agencies, including no means for complaints, no transparency of feedback loops for indicating that issues had been resolved, or lessons learned from mistakes made. • Satisfaction/dissatisfaction surveys were non-existent and therefore there was no measure of performance from a user perspective. • Website content was only available in Arabic, which disregarded the needs of approximately 13 million non-Saudi workers, most of whom are not conversant with this language. It is clear that the problems identified were creating significant barriers for at least most of the population of 31.724,308 people in gaining access to services that they are entitled to, with particular difficulties for groups such as rural populations, the disadvantaged (mentally and physically disabled, those with educational needs, the poor etc.) and non-Saudis. Indeed, the analysis indicated a disturbing reality, namely that government agency information, including policies and regulations, was only available for small groups of people working in the agencies and their relatives or friends. Those not so connected (most people) had to rely on what they were told in physical offices, often with no other source of information concerning the responsibilities of departments or contact information.

B. Strategic Approach

 2. What was the solution?
The solution was the development of an initiative named the “Online Products and Services Assessment “ and it evaluated many aspects, including transparency, by using quantitative and qualitative approaches to come up with numeric and non-numeric results. These results included detailed reports and recommendations provided to each website owner as well as overall scores. In this way, the assessments, periodically repeated, provide necessary encouragement and the means (instigating the recommendations) to gain higher scores in the future.

 3. How did the initiative solve the problem and improve people’s lives?
The core of the strategy developed was the identification of criteria that supported and enabled maximum levels of transparency, accountability and integrity across all of the 190 government agency websites. Meeting these criteria means ongoing assessment activities and evaluations against the standards set. It also means the production of periodic reports, including recommendations and overall scores, provided to government agencies, and subsequent support in the form of meetings and workshops to facilitate the implementation of measures that meet the recommendations made. It is an incremental and dynamic process, where ongoing assessment/reassessment enables the constant improvement of services. The overall aim is to improve and maintain trust between the government and all of its citizens and resident non-citizens by achieving the following objectives: • Make all government information, policies, legislation and access to services available online in ways that are user-friendly to all people, including those with special needs and those who do not speak or read in Arabic. • Ensure that all government agencies publish online accurate statistics, key performance indicators, performance reports and spending. • Ensure that the mandates, responsibilities and service responsibilities are transparently available online across all government agencies. • That all people, including the disadvantaged and non-Arabic speakers, have effective and transparent channels available for expressing their needs, giving feedback and making complaints. • To ensure that all open government data is available online. • Ensuring that those seeking government jobs have equal opportunities and sufficient and effective information to apply for jobs and track their applications online. • Making sure that small and medium enterprises are provided with information concerning all available opportunities and the online capability to make bids and track the subsequent status of those bids. It is clear that the target population is all people living in Saudi Arabia, but the further aim is to provide equality of access. This means providing specific facilities that will reduce disadvantages to the greatest extent. For rural populations, this means journeys to government offices that were extremely hard and expensive for some and impossible for others no longer need to be undertaken, so the disadvantage is eliminated. The same is true for poor people in urban areas, as well as the physically disabled in terms of having to travel to government departments, who have struggled or found it impossible to meet the transport costs and the loss of wages necessitated by the need to visit government agencies – these financial and time disadvantages have been eliminated. For those challenged by educational difficulties in terms of literacy and numeracy, or with hearing and sight challenges, assistive technologies have been developed, such as screen reading features, and there are others that are in the process of being developed and installed, such as speech recognition. For those people who are not nationals of Saudi Arabia and who are challenged by language asymmetries, government agencies have provided, and are continuously encouraged to further provide, translations of all policies, mandates, services etc., and the capability to interact through various channels with the agencies. This has considerably reduced disadvantages encountered by foreign nationals living and working in Saudi Arabia. Small and medium companies that have been disadvantaged in the past due to lack of transparency and information availability are now able to compete with larger entities that have traditionally had closer ties with government agencies. The transparency and provision of all bidding opportunities on government agency websites, as well as bid tracking capabilities, has at least considerably reduced disadvantages that SME’s have traditionally encountered. This aspect, furthermore, can be directly related to improving economic efficiency through increased competition, which is of benefit to the nation as a whole.

C. Execution and Implementation

 4. In which ways is the initiative creative and innovative?
One unique aspect of the initiative is the extent to which it utilizes and exploits human psychology in achieving its aims. Yasser has no direct authority over government agencies, so its ability to ‘make’ things happen is limited. However, agencies do have strong loyalties and pride, so by producing and publishing scores (including to ministers), there are strong psychological incentives to improve (and move up the ‘league table’). By focusing on equality of access, it has been possible to shift the mentality of government agencies not only towards user-focused service provision but also to finding ways of ensuring that disadvantaged groups are part of that user-focused provision. While it may not be appropriate to describe support (in the form of meetings, workshops, informal discussions) as unique, it is an aspect of government business that is sometimes lacking. In the case of this initiative, support mechanisms are seen as an important and necessary part of the ‘jigsaw’ – the reports, statistics and scores are produced, the incentives are created (as described), and then the means for positively responding to challenges set are made available. This means that the initiative becomes a part of the solution (not a problem created) for government agencies.

 5. Who implemented the initiative and what is the size of the population affected by this initiative?
Controlled by several ministries and as an integral part of the Saudi e-Government programme, Yasser has a primary role of ensuring that there is cooperation and continuity in the implementation of ICT across all areas of government. The department responsible for implementing the initiative is within the Yasser framework and, in line with equality and inclusion initiatives, is led and predominantly staffed by women. The initiative is potentially of benefit to all individuals living in Saudi Arabia, which is a population of 31,742,308 people.
 6. How was the strategy implemented and what resources were mobilized?
This initiative is funded by Yesser, which is financed from government funds. The technical resources used are already available at Yesser, meaning that the primary costs come from human resources. As has been previously noted, the initiative is led and mainly staffed by women. Some staff are on site and four others (3 female and one male) work remotely. The most common job function among staff is assessing, and training has been provided for those selected through an application and interviewing process. The first step in the development process was reviewing and analysing government websites. From this process it became clear that there were considerable and very worrying deficiencies that were undermining the e-Government programme to the extent of making it inoperable in any way that could be associated with the aims and objectives established by Yesser in fulfilling its primary role. The second step was the establishment of what should be available on government agency websites, including factors such as linkages to other sites, mandates, information about services, agency performance, key indicators of performance, access to interactive media, feedback loops, target achievement, special provisions for disadvantaged users, information in languages used in Saudi Arabia etc. From this came the development of a heuristic evaluation checklist that sought to deal with the areas identified in the previous step. The checklist included: • Accessibility: The evaluation of website compliance to W3C standards and testing content accessibility using assistive technologies for people with special needs. • Content: Ensuring all government information is available online, such as organization structure, contact information, services catalogue with detailed information about each service, annual reports, performance reports, key performance indicators, projects and initiatives, government spending etc.  Online services delivery: Checking content regarding government agency services, ensuring that all information required by users is available, and that users were able to provide their opinions on the services. Checking statistics concerning usage and that the impact for each online service was available (transparency and efficiency).  E-Participation: This aspect assesses many parameters, with the main goal of making sure that governments agencies have appropriate policies for e-Participation, and that they are utilising online channels in ways that support users in 3 key levels, which are e-Information, e-Consultation and e-Decision making.  Open data: To ensure government agencies are opening up all their data correctly and at the same time considering privacy and data protection standards. This checklist also enabled a further step, which was the production of guidelines for government agencies. The next step is assessment and the assessment service has the following steps: 1. Overview analysis conducted by the assessor on the government agency and its website. 2. Website evaluation using the heuristic evaluation checklist and, as a result, providing scores. 3. Peer review is carried out by another assessor on the heuristic evaluation results and scores given by the main assessor. 4. Expert review carried out by an expert assessor on the heuristic evaluation results and scores given after the peer review. 5. Analysis of the final results and scores, followed by the preparation of a comprehensive report, including results and recommendations with showcases and best practices examples. 6. Peer and Expert Reviews of the report. 7. Delivery of the final report to the head of the government agency. Following delivery of the report, a meeting is usually scheduled with government agency website teams to discuss results and recommendations and to offer support in achieving the goals set. Copies of all reports are kept in order that the team can produce national levels statistics, from which ongoing progress can be monitored and evaluated.

 7. Who were the stakeholders involved in the design of the initiative and in its implementation?
Internal stakeholders involved in the design and implementation of the initiative are: • The Yesser programme director general • The Yesser strategic and planning team • The Yesser government agencies relations team • The initiative team External stakeholders are: • Government agency leaders • Government agency website teams (considered as stakeholders because they undertook to implement the changes on their websites) When working on design, the initiative team involved some citizens in focus groups to elicit their needs and expectations of government agency websites in respect of transparency, accountability and integrity. The initiative team also involved some private sector experts when they were developing the assessment methodology and frame work, and also when designing the evaluations checklists. During implementation, qualified remote assessors were also involved.

 8. What were the most successful outputs and why was the initiative effective?
In terms of reducing inequality and empowering women, effectively signposting an important positive step with regard to Saudi society, this initiative has been very successful as it was led and predominantly staffed by women. In another sense with regard to sustainable development goals, the initiative promotes accountability, transparency and integrity in public institutions as well as side-lining unwarranted and unfair patronage by one small group (public servants, their friends and families) over the many. These key general benefits aside, five concrete outputs are: 1. In terms of measurable accessibility and functional user benefits, the initiative has changed a situation where no government agency websites passed W3C validation standards to one where 95% are complaint with them. Further evidence of accessibility can be seen in areas such as a screen reader facility now being available for the major (and most used) sectors of health and healthcare, education, employment, finance and the economy and social support. A position where no government agency websites could be completely browsed and used with a keyboard and where none provided special accessibility for people with motor disabilities, furthermore, has been changed to one where there is 100% accessibility via a keyboard and 100% have motor assistive technology. 2. In terms of website content, the situation has been changed from one where government agencies had no significant content that supported transparency, accountability and integrity, to one where 90% of them within the main sectors are publishing content that is empowering in areas such information about government agency organization structure, information about the people behind the agency, office locations and contact information, performance reports, financial reports, key performance indicators, initiatives and projects with allocated budgets for each, project statuses, as well as content concerned with the contribution of government agencies towards sustainable development goals . 3. In terms of measurable accountability, prior to the initiative this did not exist. Now 95% of agencies provide information concerning service mandates, expected waiting times for responses, costs, individual contact information for agency staff and their respective responsibilities and agency alignments with SDG’s 4. Prior to the initiative there were no user satisfaction surveys whatsoever, while now all provide them as well as feedback requests, and 50% of websites, predominantly in the main sectors, now show service usage rates and their impact. 5. Since the start of the initiative, e-Participation practices have risen from being available on zero to 95% of websites.

 9. What were the main obstacles encountered and how were they overcome?
One main obstacle can be categorised as attitudes and perceptions in two important areas. One is in the fact that the initiative was fully led and predominantly staffed by women. However, if any concerns existed within the minds of the women involved in terms of how they would feel in empowered positions and in how they would perform were rapidly overcome as the initiative evolved and became increasingly seen in terms of its success, dynamism, originality and sustainability. A second such obstacle was in the attitudes and perceptions of the government agencies and their staff. These may be described as bureaucratic organisations with staff that were set in their ways and likely to be reluctant to change. However, the approaches used, which included human psychology in terms of producing quantifiable and comparable scores and consistent help and support, meant that the changes were embraced and accepted. In fact, the momentum in terms of enthusiasm has continued to grow. A further identifiable obstacle was the nature of the task and the effective construction of standards, many of which had never existed before, and setting these out in measurable and assessable formats. This obstacle was overcome through a determination based in the knowledge that overcoming it would facilitate the success of the wider Yasser mission.

D. Impact and Sustainability

 10. What were the key benefits resulting from this initiative?
This initiative changes the way that government agencies think about their websites, representing a paradigm shift from looking on them as something cosmetic, to an understanding and belief that they can be essential and powerful channels for interacting with people and serving them online. If government websites and how they looked before this initiative and after its implementation are considered, many very important and significant differences can be seen across a range of aspects, for example in accessibility , usability , content , services delivery , e- Participation , and open data . This initiative means that individuals and households are able to access and find valuable content in a minimum of two languages (Arabic and English), which further supports accessibility and transparency for a potentially disadvantaged group (resident foreign nationals who do not speak Arabic). Accessing content makes non-nationals as well as nationals aware of what government agencies are currently doing, what their performance is according to published key performance indicators, what they are planning to do, what their current projects are and the status of each. It also makes them aware of how budgets are spent and whether their work is aligned with sustainable development goals that support better lives for people in the present and in the future. By being able to access such information people are empowered to make better choices and take better decisions. If they feel any dissatisfaction, they can complete a user friendly online complaint form, available on all government agency websites, and can subsequently check the status of the complaint. The initiative ensures that government agency website content available to users has value and that information is accurate and reflective of the range of services offered by the agency in question as well as all other agencies. Again, this promotes transparency, accountability and integrity. The extent to which the needs of people are being met is constantly assessed and whether the services are easy to use against a standard that represents equality of access for all users, including the old, the infirm, the disabled, the illiterate and other groups with special needs. The assessment service initiative also seeks to measure how public participation is encouraged, how reassurances are given in terms of negative consequences when complaints are made and how people are encouraged to be a part of decision making processes (e-Information, e-Consultation and e-Decision making). The approach and methods used, which include multiple annual visits and regular website scrutiny, means that there is an in-built and heuristic dynamic for ensuring constant improvement. As this is applied across all government agencies and relative positions are established through the scoring mechanism, it also means that standards, beginning from an admittedly low start point, are also constantly being upwardly adjusted. This initiative, led and predominantly staffed by women in positions of influence and responsibility, can be seen as an important breakthrough in terms of women in society and in work. This empowerment is enhanced and fed by the success of the initiative.

 11. Did the initiative improve integrity and/or accountability in public service? (If applicable)
The transparent ways in which content is now produced and published on government agency websites is in itself a strong indication of integrity and accountability. Everything is there, up front, and it relates to the experiences that people have had. Over time, this leads to trust and restored confidence in government agencies – it renews belief that they are working with and for the people. Because of transparency through, for example, the clear display of mandates, performance related to key indicators and through user feedback, the public is able to make its officials more accountable in delivering public services. If a user feels that clear and transparent principles and standards have not been met, they know that there is a meaningful complaints procedure that can be tracked and monitored. These aspects feed the system with positivity, which along with ongoing assessments, means that accountability, transparency and integrity remain on an upward curve. The system, supported as it is through e-Participation, social networks and other such channels, means that people make informed choices, they will not be wasting their time, energy and money in seeking answers in the wrong places. This in turn means that public servants are more efficient because no time is wasted redirecting people and dealing with disgruntled individuals. Corruption in this area of public service has, in the past, come from officials providing information to their friends and families, and in giving priority to their needs at the expense of the majority who do not have such ‘inside’ advantages (a factor that de facto made it even more difficult for disadvantaged groups to gain equal access). All this has now changed because such unfair priority and preferential treatment cannot be given online and everybody now has access to information. If such practices did occur, the inbuilt transparency that now exists would quickly expose it – people (all people) have, effectively, been empowered. The empowerment of the people through the results of the initiative and the use of modern technology in the ways it was designed for can be seen in two of many examples – some major decisions have been made as a result of public feedback through social network posts and two ministers were subsequently replaced in 2016. These examples could not have happened if people did not have the information to make such judgments and feel empowered to express them – information enabled as a result of this initiative.

 12. Were special measures put in place to ensure that the initiative benefits women and girls and improves the situation of the poorest and most vulnerable? (If applicable)
Even though the rules governing discrimination and the treatment of all citizens may have suggested that women and girls were treated equally by government agencies, traditional and culturally based attitudes and views may have meant this was not the case, even if these came from feelings and beliefs that existed within the minds of women and girls themselves – if a person feels or is made to feel disadvantaged, they de facto are disadvantaged. However, these feelings and perceptions are removed when the services are seamlessly and equally accessible online in a way that is gender neutral and has the transparency, accessibility, accountability and integrity to reinforce that neutrality and equality. The initiative strongly reinforces this equality and even leans it towards the empowerment of women because it has been designed, led and implemented by women. This is yet further reinforced when it is noted that the positions held by women in the initiative have involved a process of competitive recruitment, considerable training and, through these factors and the performances subsequent to them, positioned women in respected and influential positions within the Yasser organisation and in their dealings with all government agencies.

Contact Information

Institution Name:   E-Governemt Program (Yesser)
Institution Type:   Government Department  
Contact Person:   Hend Alhazzani
Title:   Engineer  
Telephone/ Fax:  
Institution's / Project's Website:  
Postal Code:  

          Go Back

Print friendly Page