Annex C Case Studies
The following is a collection of case studies which demonstrate the value that open data is bringing to individuals, companies and public authorities in Scotland.
If you are interested in case studies which cover a particular theme:
|Improving public services through the use of open data||
|Increasing transparency and accountability through the use of open data||
|Generating improved civic engagement through the use of open data||
|Engaging data users in the process||
|Building a business case around open data||
|Improving data governance through the use of open data||
|Generating innovation and economic growth through the use of open data||
If you have a case study you would like to share or you would like to be put in touch with the case study subjects, then get in touch - OpenDataPolicy@gov.scot.
1. Clackmannanshire Council: Open Data Scotland and Code for Europe
Many of the problems which Open Data is typically used to solve don't exist in a small Council. Mass transportation isn't an issue with only 3 bus routes. There isn't a developer community taking part in hack events and generating innovative applications. Why then should a small Council pursue Open Data?
Open Data affords opportunities to be more efficient, whether through being nimble by adopting freely available Civic Apps to improve service delivery or by reducing the time spent responding to information requests from the public or partners. In time, it is likely that we will be required by statute to share more data anyway.
Location based services will become increasingly important. In the near future citizens will expect to be able to use their personal device and using the tools of their choice, see and interact with services which are nearby. In order for Council services to be part of this world, data about those services must be published openly.
'Open Data Scotland' is a programme which has involved over the last year, four of Scotland's local authorities - Edinburgh, Aberdeen, East Lothian and Clackmannanshire. Aberdeen and Edinburgh City Councils have been at the leading edge of nascent open data work in Scotland and can be seen as 'mature' players, willing to share their knowledge and expertise with others. East Lothian and Clackmannanshire came to the programme with little or no experience of open data, but with an ambitious attitude and a willingness to experiment and embrace innovation.
Each local authority was appointed a 'Code Fellow in Residence' (a technologist) who has worked intensively with the local authority staff over 12 months to open up data sets, publish these on a portal so they can be re-used and created new digital public services - apps and web content to enhance both citizens and visitors experiences of the local authority. A 'Designer in Residence' also worked with the technologists and local authority staff across the four authorities.
We have been part of the wider 'Code for Europe' programme which has involved designers and technologists across Europe working with civic authorities to increase the use of open data sets to enhance civic transparency and improve decision making.
Clackmannanshire Council is Scotland's smallest mainland local authority and their learning from the programme below demonstrates that this is not beyond the reach of any government agency or public body in Scotland, with the right culture and access to skills.
With little prior knowledge of Open Data, our initial ambition for this project was to develop a mobile app which would provide personalised access to childcare resources as part of the early intervention strand in our 'Making Clackmannanshire Better' change programme.
As the project evolved we focused on three main areas: Knowledge Transfer, Developing a Portal and our Childcare Application.
Knowledge transfer provided Council officers with information about standards and systems used in Open Data, the ecosystem of agencies involved in Open Data and the sources of existing Open Data applications which were available for re-use.
We successfully built a CKAN Open Data portal and developed an app called Clacks Kids which is a location based service directory. Spin-off activities have lead us to develop an open GIS mapping portal which is likely to inform our future GIS Strategy and a reporting platform based on the Open311 standard.
- The biggest lesson is that even a very small Council can engage with Open Data. The key components are having access to people with the right skills and attitudes, easy access to servers and software with which to tinker, and permission to experiment.
- Developing an Open Data infrastructure is important if the project is to be sustainable. While the "app" may be the most high impact product, without the infrastructure there will be no data to use in the app. Apps are also transient, they will be replaced by other apps in future.
- Follow your nose! We have revamped our GIS infrastructure opening up the opportunity of significant future cost savings as a direct consequence of our need to provide mapping tools for this project.
- Civic Apps are not as easily transferrable from one Council to another as you might expect.
- In a shrinking Council, persuading others to prioritise your project can be difficult especially when there is nothing concrete to demonstrate. You need to have something to show people. Once we had a working app, we then found services coming on board as they could see how it could be used.
2. Crichton Institute: Regional Observatory
In promoting this project both within and beyond the region, both local and global issues have collided. It is clear that new technology has created an accelerating hunger for information and we have observed with interest the parallel dialogue around Open Government and the 'Smart City' agenda. It seems to us that there is something of gap in strategic thinking and policy and we have been asking the question: '…if there is such a thing as the Smart City, what would the Smart Countryside look like…?'. So there is a dialogue that needs to take place about rural-specific opportunities in the open sharing of data and service improvement and provision which we feel we should follow.
The above issue is compounded by the overall capacity constraints which rural agencies face. While an obvious plea would be for more resources for rural areas in this field, there are perhaps more immediate advances that can be made by better sharing of experience and existing resources currently being directed to urban areas/solutions.
Raising this issue has gained us some exposure. We have, in addition, been cited as an example of regional-level data innovation in the recently published SG Open Data Strategy. We have also been encouraged by the fact that others, including those much better resourced than us, have had to grapple with the same issues and that we are seeing emerge a community of like-minded people who are prepared to provide advice and support.
Crichton Institute is a Scottish Funding Council ( SFC) funded collaboration involving the academic partners based on the multi-institutional Crichton Campus in Dumfries, south-west Scotland (University of Glasgow, University of West of Scotland, SRUC, Open University, Dumfries & Galloway College). The Regional Observatory is one of the strategic arms of the Institute and has been in development since the start of CI in January 2013. CI's work has very much concentrated on Dumfries & Galloway. The SFC funding period comes to an end in December 2016.
The objective of RO is to provide an information and knowledge portal that acts as a one-stop open access service for open data, information and intelligence on a wide range of social, economic and environmental factors across Dumfries and Galloway and the South of Scotland.
Rural areas have in general been poorly resourced in terms of data gathering, access and usage and D&G and the South of Scotland are no exception. In many cases, public and 3rd sector agencies have had to resource external consultancy to assist with even the most basic of regional data gathering and interpretation (the exception being the local NHS Board which has a well - resourced public health intelligence unit). Effective data sharing has, as a consequence, been somewhat the exception.
With a lack of capacity and no consistent track record of high level collaboration, the benefits to be derived from sound data management and data sharing have not been fully understood or exploited. While individual agencies are striving to take advantage of new information and communications technologies, the absence of effective data management is inhibiting the genuine desire to move to a more 'open government/open data' culture. Change in recent developments in Community Planning, a move towards better understanding the needs of service users and service integration (within and between agencies) is however supporting the drive towards a culture of open government/open data in the region. In fact, over the last few months Dumfries & Galloway Council ( DGC) have auditing their records management and are working on an open data strategy. At the same time Third Sector Interface in D&G have been working with Think Data Scotland around the issues of data gathering and sharing within the Third Sector community http://www.thinkdata.org.uk.
The setting up of the RO was approached as much as an organisational development and trust-building issue as simply an exercise in data management/sharing. Hence the RO has been developed in close collaboration with the Dumfries and Galloway's Community Planning Partnership. The objective from the outset has been to ensure buy-in to the concepts of Open Government, collaboration, service improvement and data sharing across institutions, communities and businesses across the region.
The first step was to engage in discussions with the local Community Planning Partnership to ensure that there was a view that such a thing as a data observatory was needed, but also that there was high level cross-agency support for its development. The proposal was greeted with enthusiasm and two tranches of support funding for the early development stages of the Observatory.
With support secured, an initial Technical Group was established which included representatives from DGC, NHS Dumfries and Galloway ( NHSDG), Scottish Centre for Enabling Technologies ( SCET) and CI.
This group worked together to agree on the vision, look and feel of the online portal, the process of populating it, maintaining it and promoting it. As part of this some desk research was done to look into what other observatories and open data portals offer. Some of these were approached directly to inform us whether we were heading in the right direction. Armed with this background, a Project Initiation Document was agreed and specialist part-time consultancy engaged to convert the vision into reality.
One issue we struggled with was '…when do we go public…?' We were confident with the basic functionality/feel of the portal, but less assured on content issues. We have made life difficult with the notion that our customers would not just be the usual professional data-users. Our vision also included, for example, local P6 pupils using the portal for a project on "Jobs in our Region", complete with map-building and visualisation tools.
We decided to opt for a 'soft launch' of the RO website in June 2014. The Technical Group became the Data Suppliers Group. With the key technical issues addressed and the portal functioning it was felt that the 'harvesting' of data and documents should now widen to include others, for instance Third Sector Interface Dumfries & Galloway. Approaches have been made to Police Scotland and other local community organisations. These discussions have served a dual purpose. As well as serving to promote the RO and secure further data access, they have opened up dialogue about the promotion of data sharing, better understanding, and more effective service delivery. This was supported through a workshop in the autumn of 2014, which was also attended by Scottish Government providing us with confirmation that we are moving in the right direction and that we have the same issues as other open data projects.
However, even though feedback has been overwhelmingly positive and encouraging as people like our vision and the objective, non-partisan approach, since RO does not have any authority over the various stakeholders, progression around data 'harvesting' has been extremely slow.
With Phase 1 of the project complete, we have a well- functioning and attractive portal, though it is still fairly one-dimensional. However, local stakeholders and the discussions as part of the autumn workshop showed a great interest in RO providing access to a regional economic dashboard, providing instant, up-to-date access to key economic performance data for D&G, as well as access to interactive mapping and other visualisation tools. With not having technical expertise within RO and the Data Suppliers Group we were looking at how best to enhance the portal. This led us to the School of Computing Science at the University of Glasgow. After detailed conversations we ended up with 140 3 rd year Computing Science students who worked on a number of IT related projects with RO and other D&G based organisations: Crichton Carbon Centre, NHS Dumfries & Galloway, Third Sector Interface D&G and The Stove Network. A total of 11 projects presented a range of real-life challenges to the students helping to find IT solutions to questions such as:
- How can regional economic indicators be visualised and made more accessible to everybody?
- How can we sell waste products from our industrial process, rather than land-filling?
- How can individuals find out about local volunteering events and sign up to participate?
- How can I discover where people go and what they do in my visitor centre?
Prototypes were created and in some cases the solutions offered are of such quality that the intention is to have them go live. One of the projects also looked at a redesign of the RO website, using WordPress for the actual website, while bolting on CKAN open source application to deal with the data.
This collaboration provided a great platform to show what is possible, but it also left us again in a situation that, unless we find resources to secure technical support we are unable to migrate the projects, implement them, maintain and further develop the solutions given to us.
3. The City of Edinburgh Council: ARC-E App
The app enables the service area (Health & Social Care) to open up data that was not previously available to the public in this format. Previously service information for addition recovery support groups in Edinburgh had only been available through leaflet and PDF formats. An API was created with this data and can be shared with the app and other applications.
The app has been built so that it is scalable and more features can be added. The framework of the app can also be redeveloped to suit other groups with similar needs.
The application, the Addiction Recovery Companion - Edinburgh app ( ARC-E app), is aimed to support those in the process of recovery from an addiction. The app allows users to document and reflect on their progress, becoming a constant companion and supporting them as they help themselves. It will make it possible for people who already have some support from Council services to use their mobile device to help them in their recovery from addiction.
The approach taken to develop the app was an agile, co-creative approach. Through working this way the team have been able to develop the app with the Council service area and services users to ensure that the deliverables are being met and a worthwhile product is created.
Design & Build
The approach of the design was to put the needs, wants and limitations of the
users at the heart of the design process. From the start, the project ran focus groups with potential users of the app to inform on how to move forward. First the team made sure that the objectives reflected problems that impacted recovering addicts and then tested and iterated on potential solutions using low-cost prototypes before implementing them.To build the product, a version of the Scrum agile development approach was adopted and adapted to fit the small and distributed team. This approach recognises that requirements often change during a project and the team has to be in a position where it can quickly adapt to these changes.
The project was divided into objective themes. Each objective theme contained a collection of user stories and at the end of every iteration the team produced a build of the app. This build was tested against the user stories for the iteration and used as an artifact for user testing. This allowed the Council to assess the current build for milestone acceptance and potential users to test and feedback on its value to them. The outcomes of testing influenced the planning of future iterations e.g. new user stories maybe added to the backlog or remove ones that have been shown to be invalid. This ensures that a meaningful product is being built at every stage.
ARC-E App was developed in order to:
- improve access to appropriate local support services and information about the service.
- make it straightforward for users to reach out for immediate support in times of crisis.
- help users keep track their appointments and commitments, related to managing their recovery.
- keep users up-to-date on events organised by the council or by members of the recovery community that might be relevant to their recovery.
- allow users to look back at daily messages to support motivation to stay on track.
- allow users to access mindfulness activities, particular during a crisis/emergency.
The main lessons learned have been around working co-creatively. The client (Health and Social Care) and the end user (people in recovery from addiction) have been involved at every step of the process. Working this way has ensured that milestones have been hit on time and on budget whilst creating an app that meets the user's needs and achieves the objectives set out by Health and Social Care.
4. The City of Edinburgh Council: Edinburgh Apps
This programme is completely transferrable to any other organisation and sector. Edinburgh did not create something that was untried - civic challenge competitions take place all over the world, and are very successful. Supporting events, hack weekends, data days etc. are also happening widely, and are not expensive to produce. All of these events add to learning and increases awareness of the power of open data. It is a new way of working, but it is already the way many companies work, and something the public sector needs to do to find efficient and cost effective solutions.
Edinburgh's track record speaks for itself - its agile approach to development meant that quality products could be built quickly and were known to meet customer needs. Most of the products are shareable which means the public sector can use them right now.
Launched in 2013, EdinburghApps was the first event of its kind in the UK, a civic challenge programme that works with the Council and other partners, encouraging developers, designers, creatives and small businesses to take part, and offering winners business support and the potential opportunity to work with the Council to develop their concepts further. Participants choose from challenges set by the Council around a number of key themes.
At its core is a vision to change the city through encouraging innovation with technology, design and user-centric development. Edinburgh has exceptional design and tech communities and a large number of young companies in these areas whose fresh thinking mean that Edinburgh has great opportunities to produce original and cutting edge solutions to city challenges.
The programme of challenge events:
- supports growth of and partnership with new IT, design and other related businesses and partners in the city
- encourages a digital culture change internally, supporting skill development for council officers
- delivers innovative and efficient solutions for the Council's customers, in line with the Council's priorities and the ICT and Digital Strategy objectives.
Edinburgh Apps was developed to support the Council's Open Data strategy. For each challenge, data sets are shared, increasing the Council's delivery of open data and opportunities for innovation EdinburghApps wants to change the city by providing creative, customer driven solutions to city challenges. It aims to work with everyone interested in making this change happen.
EdinburghApps began as an annual once a year competition with the Council providing challenges and teams taking part over 6/7 weeks to develop strong concepts or/and prototypes which are then judged in a final event. The winners then have the opportunity to work with the Council to develop their ideas, and deliver products.
EdinburghApps now runs a range of events to encourage solution finding working with key partners, Council officers and customers
- Annual challenge competition
- Subject hackathons
- Service area mini events
All of these events aim to support partners in finding innovative solutions to business and city challenges. Data is a core requirement in all of this, and is published as open data whenever possible.
Winners of these events have the opportunity to take forward their proposal for development with the appropriate area.
The benefits of this approach are:
- Delivery of new digital products which meet a clearly defined need
- Delivery using an agile approach, and at a far reduced cost to working with larger companies
- Customised solutions, co-created, which are built directly to meet requirements
- Building longer term relationships with local IT & Digital companies
- Opportunities to support the growth of the city's business economy
The competition event is now in its third year and has been very successful encouraging a wide range of entries and the delivery of a number of products. These include:
- Tend - routing tool which optimises planning and deliveries for Health and Social Care's Equipment Delivery Service
- Recycling Edinburgh - a location app case study, sharing recycling facilities in the city
- Run The City - an app aimed at visitors looking to explore the city using running routes, offering a commentary on places on interest
- ARC-Edinburgh - a buddy app to support those in recovery programmes for addiction
The Council has helped winners to start their business from scratch, and also supported participants to find other business opportunities. Edinburgh Up Close was also recently launched, working with technology developed by a winner from EdinburghApps 2013.
The event runs in three events - a kick off weekend, a midway workshop and the judging final.
When EdinburghApps was first launched it was intended to bring about a number of benefits, including:
- new thinking to solve city challenges
- innovation in technology and design
- the sharing of civic data
- stimulate the city economy through working with SMEs
- social change for the city
The programme has achieved these outcomes, but it has also brought about much more:
- innovative and cheaper solutions
- improved sharing and publishing of open data
- ongoing relationships with new businesses
- change to ways of working
- awareness raising for open data
- new business thinking
- benefits to customers
EdinburghApps has now taken place three times, and hack weekends have also been run in between to generate wider awareness and explore specific subject based challenges. In 2016, Edinburgh expanded their approach to work with partners such as NHS Lothian and Sustrans Scotland which meant that challenges could be more strategic in nature, and provide service improvements that had a holistic impact. There are synergies for organisations both in terms of challenges and for product and data usage. Longer term, Edinburgh will continue to build relationships both with partners and businesses in the city and with the tech community to achieve sustainable outcomes.
Edinburgh Council will continue to expand the sharing of data as well as the sharing of ideas.
- Build support: it is important to have a suitable sponsor in your organisation (and some funding) to do this. Edinburgh could demonstrate it was achievable because it had been done elsewhere and this helped them find supporters. Align with relevant strategies in your organisation, this will also build support.
- Changing business thinking: when Edinburgh started this programme they didn't realise the impact of bringing business change into the Council. Inviting developers and designers to work with them brought fresh thinking and new ways of working. This has had an interesting internal ripple, and they now find service areas keen to see what can be achieved, not just with a product development, but for their service generally.
Data: this is the central component and takes
time to find, cleanse and publish. This can be challenging and
service areas may need help with this work. Ideally a data
resource should be available to do this.
For the competition itself: Edinburgh have
discovered that a mix of skills works better for teams and builds
better prototypes, so they now advertise across a number of
sectors. You need developers to support the whole event, provide
mentoring and knowledge sharing, so build relationships with your
local tech and design communities. Some teams have no idea how to
deliver their idea
- Edinburgh Council are now offering a midway workshop to help teams learn how to build a proposal, cost and plan their concept. They have developed their own processes around the event which include the use of design thinking, customer experience and business planning.
- Challenges: Whoever submits a challenge must now take part in the whole event, providing further information and advice for teams. This means that, whoever wins, the challenge owner is already engaged with them and it makes it easier to take the project forward.
- Funding: funding for the competition is not straightforward, and should be bundled into a larger business case for open data and innovation. For those taking part, there is an expectation that the Council will fund development. In 2016 Edinburgh put in place a co-production approach. Winners continue to work with challenge owners, who will work to identify funding once a concept has reached that stage of maturity. A lot of entrants have full-time occupations, so there would be a risk in going straight into a contractual arrangement with timelines that are difficult to meet. Both parties have time instead to reflect following the event, and can decide how they want to move forward. For funding, identifying appropriate sources is very useful; and for this involving funding officers is really worthwhile.
Sponsors: a range of sponsors and types of
sponsorship are required and this is time consuming to achieve.
It is never too early to start working on this.
- Communications: communication has to be regular and continue throughout the year, not just around events. This requires resource and should not be under- estimated. Engagement is essential to keep your audience interested and encourage them to come back each year. Use appropriate platforms such as Twitter and Instagram and set up a blog so that you keep everyone up to date on activities.
Finally, and most importantly, look for any opportunity to work with others in this area. This is one approach but there many other methods for engaging and changing thinking, and developing open data. Build partnerships to make it easier to accomplish more. Edinburgh work with individual developers and creatives as well as companies, and we do look for those who have the same goals for data and innovation and want to see change happen.
5. The City of Edinburgh Council: Run the City App
The Run the City App solves a challenge faced by running enthusiasts who are new to the city by providing routes around the city of different lengths. It increases visitors' engagement with the city by highlighting city sights and providing engaging anecdotes.
The app has been built so it is scalable and with the intention that other cities and routes will be added.
Run the City is a guided tour for runners and winner of the 'wild card' challenge for EdinburghApps 2014. Runners will always get their run in, even when away on business, but running in a strange city is difficult when you don't know where to go. Run the City solves this challenge as the app, through audio messages, not only gives runners directions but also highlights their attention to the city sights and makes their run in Edinburgh more engaging with anecdotes about the areas they are running through.
It utilises the Council's open data as content for the app and will also create data the Council can make open.
The project was undertaken over two main stages the Build Phase, and the Beta Phase. The initial build phase allowed us to deliver a minimum value product which could be tested to ensure that on the project was on track to deliver the objectives before full build was complete. This also ensures that a valuable product was being delivered that people wanted to use.
The team that developed the app worked co-creatively with the Council service area to ensure that the app met customer expectations and also aligned with Council outcomes.
The build phase developed the main components of the app (login, cities, routes, tracker, activity, settings, activity timer and location tracker pages.) In this phase,
before building the user-interface of the app Edinburgh Council created a route planning functionality, which allowed us to design and record routes to be uploaded into our app. The milestone for this phase will be the delivery of the MVP (Minimum Value Product)
This phase involves both production of the audio for the tour and device testing and user testing. The test app was shared with runners/walkers around the city and they were asked to test the runs and report back any ideas or issues they had. Edinburgh anticipated that user testing of that app would take three to four weeks but it actually took longer and was carried out in two phases due to the changes required after feedback was received.
An engaging running app, which considers routes which would appeal to walkers, has been created using Council open data. The app has been aligned to Edinburgh Outdoors and has created the additional benefit of creating data which can be shared.
The app is now live for Apple and Android users, and is gathering very positive feedback. This is encouraging the company who developed it to have conversations with a number of other cities for future development.
The idea behind the app was sound, and user research demonstrated that runners thought it would be a great way to experience the city. It turned out to be quite a challenging development, with issues around design and build. A major issue was calibrating the app for runners and walkers (due to differing speeds) and getting the geo-location to match. This problem was not easy to predict - the developer was very experienced; it was just an issue that had to be understood and solved.
Both members of the new company were in full-time employment which meant development was elongated. Ideally it would be better to work with an existing company who could commit specific time to the project. This would have increased costs and EdinburghApps is clear that it wants to support new business development.
This development has sparked a lot of thinking around digital tourism. Cities and towns will have a number of apps in the market place for their visitors. It is worth giving some consideration as to the audience you want to target. Our experience suggests that specific audiences may be more likely to download and use an app on their visit than general tourists.
6. Royal Commission on the Ancient and Historical Monuments of Scotland - SENESCHAL: Semantic ENrichment Enabling Sustainability of arCHAeological Links
Adopting Linked Open Data can benefit the wider heritage community through improving standards and introducing efficiencies. The benefits of publishing controlled vocabularies are starting to be realised. Simply by adding a SENESCHAL RESTful service into their Collections Management System, the Archaeology Data Service, University of York were able to access the authoritative controlled vocabularies remotely. This not only eliminates errors that inevitably creep in through free text typing but improves the consistency of indexing records.
Controlled vocabularies are key to both the storage of information in the database and its discovery online. In particular, we use thesauri to help classify the types of monument, object and maritime craft associated with each site record. We encourage the use of thesauri standards amongst local Historic Environment Records ( HERs), who maintain databases about the historic environment for local authority areas across Scotland, and more widely amongst the profession.
For cultural heritage, demand for Linked Open Data came from the research community. They saw the absence of controlled vocabularies as limiting opportunities for combining data from different providers through semantic links.
Major controlled vocabularies should act as hubs for the Web of Data, but publication as free text strings limits opportunities for connecting to data published elsewhere. Although we publish our controlled vocabularies online as thesauri, they are not particularly visible. The thesaurus for architecture, implemented in 2005, limits the potential of the terminology as the terms lack the persistent Uniform Resource Identifiers ( URIs) that would allow our resources to act as hubs for the Web of Data. Adopting a Simple Knowledge Organisation System, or SKOS, using the Resource Description Framework ( RDF) provides a more flexible approach enabling the vocabulary owner to define a concept rather than the term. Each concept is expressed as a URI. The concept may then be expressed in any number of ways including alternate labels, dialect terms or in different languages.
The development of Linked Open Data for cultural heritage is part of good practice, helping to deliver Government policy towards transparency and Open Data. Scotland's Open Data Strategy encourages Public Data to be published in reusable,machine readable form under an open licence which enables free reuse, including commercial reuse to open standards following relevant recommendations of the World Wide Web Consortium. Moreover, Public Data from different departments about the same subject will be published in the same, standard formats and with the same definitions. Defining the concepts used to index records about cultural heritage is a first step towards meeting that goal. It introduces the standards and machine- readable formats necessary for interoperability. However, before becoming operational, it requires acceptance of the standards, investment in research and development time beyond the day-to-day operations of many organisations.
The solution was to find partners who understood the Linked Open Data requirements and to secure funding to enable the research and publication of Linked Open Data. We were fortunate that colleagues at English Heritage already had an established relationship with the Hypermedia Research Unit at the University of South Wales and that there was a shared recognition of the need to publish our vocabularies as Linked Open Data.
The partnership approach between a university research department and public bodies enabled a successful application to the Arts and Humanities Research Council for a one year Knowledge Exchange project. This made it significantly easier for vocabulary providers, such as RCAHMS, to make their vocabularies available as Linked Data and for users to index their data with uniquely identified (machine readable) controlled terminology that is semantically enriched and compatible with Linked Data.
The resultant SENESCHAL project (Semantic ENrichment Enabling Sustainability of arCHAeological Links) brought together vocabulary providers from English Heritage, RCAHMS and RCAHMW, together with the Archaeology Data Service University of York with the domain experts, Doug Tudhope as Principal Investigator and Ceri Binding as Research Fellow, at the University of South Wales. Outcomes
|Intended Outcome||Actual Outcome|
|Freely accessible and reusable persistent vocabulary resources as linked data, the techniques to achieve this being made freely available.||Achieved:
established as the home for Cultural heritage reference
vocabularies and concept schemes published for
Each concept has its own unique reference indicator.
|Web Services to SKOS representations of the vocabularies and semantic enrichment services, along with web application components||Achieved: Downloads, Services and Widgets published. Users are able to download the vocabularies in various flavours of RDF (N- Triples, Turtle, JSON or XML). A series of REST URI calls have been developed for the vocabularies with results returned in a JSON structured string which permit AJAX callbacks for use in browser based applications.|
|Knowledge exchange tools to facilitate semantic enrichment (via URIs) within data entry. Development of downloads, Services and Widgets.||The project has also developed a suite of predefined visual user interface tools, or widgets.|
|Mechanism for feedback of supplementary terms to augment existing vocabularies||Not Achieved: candidate terms are still submitted through RCAHMS own thesaurus management system and data periodically uploaded to heritagedata.org website|
|Raising the profile of Linked Open Data with Historic Environment data curators in Scotland||Achieved: through a workshop was held in Edinburgh at the end of the project for stakeholders and presentations on Linked Open Data to stakeholder groups.|
Demonstrating application of approach to handle multi-lingual expressions of concepts: During the course of the project we were able to make use of Gaelic translations of the monument type vocabulary provided by Historic Scotland from a Bòrd na Gàidhlig funded project.
So a concept may now be expressed in English or Gaelic, with a preferred or alternate label.
Publishing the terminologies as Linked Open Data is the first tentative step toward delivering cultural heritage data as 5 star data. Maintenance and update of the terminologies is not seamless and requires periodic data uploads, so the vocabularies may not be up-to-date instantaneously.
Exposing controlled vocabularies is inevitably organisation-driven and there is a need, where appropriate, to align vocabularies by theme to deliver further efficiencies in maintaining and developing resources. Through our membership of MEDIN we are exploring opportunities to develop more marine and maritime- related Linked Data vocabularies with colleagues in Belfast and Dublin.
The benefits of Linked Open Data have still to be fully realised within the business and more widely across the heritage community. However, making the terminology more openly accessible as Linked Data should encourage wider adoption of standard terminology, develop interoperability with other related resources, and encourage community feedback on possible improvements to the vocabularies. Opportunities will continue as part of the new organisation Historic Environment Scotland when RCAHMS and Historic Scotland come together in October 2016 to form the new lead body for Scotland's historic environment.
7. Registers of Scotland (RoS): Cadastral data for the INSPIRE directive
Scotland has made significant progress in publishing spatial data in a prescribed format driven by the EU INSPIRE directive. The local government sector is also currently developing a project to support a more collective approach to the management and publication of spatial information, providing access to all spatial data created by local government in a consistent form.
RoS is proud to be at the forefront of the provision of spatial data. We would recommend any approach that supports collective management and publication of spatial information. The challenges faced internally are far outweighed by benefits realised in the short and longer term.
The experience we have gained over the last seven years with INSPIRE has led us to increasingly identify considerable benefits of a coherent, trusted, and consistent set of information on land and property in Scotland. Enabling access to core land and property information in one place where it can be made available to all is increasingly important for Scotland as a whole.
European Directive 2007/2/EC, better known as ' INSPIRE,' was transposed into UK law in December 2009. The aim of the directive is to establish a spatial data infrastructure ( SDI) for Europe. In general terms, this means providing an IT infrastructure which allows access to harmonised spatial data (data collected to the same standards and requirements) via the internet. In theory, an SDI should improve the access and use of data at local, regional, national and international levels, improve data sharing between public authorities, and improve public access to spatial data.
INSPIRE instructs EU member states to make spatial data available in a consistent format which come within the scope of the directive, as well as providing network services (mostly internet access) and metadata to support the data. You can read an informal consolidated text of the Scottish INSPIRE regulations here.
The Scottish government is responsible for the management of INSPIRE in Scotland. The management is coordinated by the Spatial Information Board and work has been broken down into five main areas. The two areas of interest to RoS are land, property and addressing; and service delivery and technical implementation. All EU member states are required to submit a monitoring report with details of available datasets to the European Commission every May. You can read the UK's most recent monitoring report here.
Our first step was to establish a project and project team to handle the legal, commercial and technical aspects of INSPIRE.
During the lifetime of the project we consulted widely with other European and UK organisations, both within and outside our domain. Colleagues within the Scottish government and UK geographic community provided an excellent and knowledgeable resource. This enabled us to overcome a wide variety of challenges and allowed us to improve our own expertise in a number of crucial areas.
As well as the technical requirements, we had to consider the wider implications of INSPIRE on our business and staff. These included the effect on our commercial activity and the types of services we offered, our IT infrastructure, and any legal impact on our day-to-day activity. Each of these requirements was processed by a small multi-disciplinary team reporting back to the project board who led the overall INSPIRE strategy.
The nature of the legislation naturally broke the project into a number of phases, each of which required an increasing level of resource and budget.
- Phase 1: metadata - in May 2011, we complied delivering GEMINI 2.1 metadata describing our land register data and the future web mapping service ( WMS).
- Phase 2: discovery and view services - in November 2011, RoS provided access to the metadata created in phase 1 to the Scottish Spatial Data Infrastructure. At the same time, RoS provided a view of its initial cadastral parcel data. For the deadline, RoS chose to use the services of a third party (ThinkWhere) to host the WMS element of the service.
- Phase 3: download - RoS delivered a service that will allow a customer to download all or part of our land register dataset. RoS again chose to use professional services of a Think Where to host the download service. Licensing considerations on the reuse of data were investigated and led to the creation of an INSPIRE download license.
- Phase 4: fully compliant - this phase will deliver full inspire compliance by supplying parcelled cadastral data by November 2017.Outcome
RoS has delivered the first three phases and is on course to fully comply by November 2017. The service is being increasingly used by customers and has sparked wider thinking about our data within RoS.
The majority of the challenges that RoS faced were based on technical and compliance issues as well as data re-engineering. Our recommendation for any organisation with an INSPIRE obligation would be to ensure your internal domain expertise is brought together to guarantee you have a firm grasp of the issues and technical requirements required. For RoS, this meant a multi-disciplinary team drawn from IT, Geographic Information Systems ( GIS), senior management, legal, commercial and core business. RoS consulted widely with fellow organisations and took part in a number of UK and European working groups to make sure we had an understanding of our responsibilities, as well as having an opportunity to influence those discussions. We would recommend that organisations seek advice, support and best practice from professional bodies, as well as learning and investigating best practice from examples throughout the world, including RoS and the Scottish Government.
Although the investment in INSPIRE can be onerous, there are considerable benefits that can be accrued if your organisation is committed to INSPIRE. For RoS, this meant spatial data has been brought to the forefront of the business, improved our expertise, developed staff, and led to several customer-focused initiatives. The core aspect of INSPIRE, data, and access to it, led us to re-evaluating data and data quality, as well as influencing a wider digital transformation project.
8. Scotland's Environment Web: EcoHack
Scotland's Environment Web wants to help people discover and understand more about the environment. Environmental data is really important - to provide context to reports on the state and quality of the environment, to improve our understanding of the challenges and opportunities our environment faces, and encourage communities, school children and individuals to investigate their own local environment further, observing what is happening around them, collect their own data and take action to protect and improve their local environment.
Putting our objectives into practice, a hackathon event was organised over the weekend of 30th and 31st May 2015. Students from universities throughout Scotland were invited to Edinburgh, to come up with fresh new innovative ideas to make better use of available data, and to collect new local environmental data that can help further our understanding, and encourage people to get interested and get involved in Scotland's Environment.
Interest was generated in the event via a number of routes:
- We had university lecturers and students on the steering group and who also helped out as mentors so were able to help spread the word to their students and peers.
- A leaflet was sent to all universities and posted on their Facebook pages.
- For students one of the most accessible forms of quickly sharing information is on social media, with a lot of co-ordinated information sharing posts on Facebook and twitter (#ScotEcoHack) in the run up to and during the event that were shared and retweeted to an extended audience, bringing lots of new twitter followers to @ScotEnvironment following #ScotEcoHack
Examples of the interest generated on twitter can be found here.
The EcoHack challenge
During the weekend event we challenged teams of students and mentors to explore data and develop ideas that could make a real difference in helping people observe, monitor, educate and take action in the environment. Ideas were encouraged around exploring new data relationships to help analyse the state of our environment and the impact it has on us, develop apps that use and visualise data to help explain and view the environment, and provide new ways of collecting and viewing data.
A wide range of open source data was available to the teams - dataset list - and they were allowed to choose any platform and programming language and spent the weekend collaborating and being creative, innovative and inventive.
In the run up to the event, we provided links to information about a range of environmental issues to inspire new Ecohack ideas, covering topics such as Air Pollution, Water, Soil, Young People and Citizen Science, Environmental data, Nature, data visualisation, EcoSchools, Climate Change and communities, mobile apps, infographics.
We couldn't have run the event without the help and support from our mentors. With a wide range of skills and experience, they were on hand to provide advice and guidance to the students throughout the development of their ideas from initial scoping and definition right through to the development and presentation of the prototypes. Some of the mentors saw some real opportunities in using some of their own data and tapping into the expertise of their mentor colleagues, and worked together to develop some of their own ideas to share with us at EcoHack.
Feedback from all who attended was overwhelmingly positive and we hope to keep in touch with many of those who supported the event - judges, mentors and students. The standard of ideas was very high and in the end the judges selected 2 winning ideas and 1 runner up. More information on the winning ideas and a video of soundbites from the event on the EcoHack webpage.
9. Perth and Kinross Council: Open Data Workshop
It is important for an organisation to engage data users as it begins to consider making more of its data open and accessible. When it comes to how best to engage, there is no one right way. However, garnering views early will allow particular datasets to be prioritised for publication. It will allow you to understand what goals stakeholders and re-users have and what datasets they identify that would help them to achieve these goals. It will also offer the opportunity to advertise the release of key datasets to partner organisations and wider potential re-users. This will help to create a network of open data advocates.
Perth and Kinross Council covers one of the largest areas of any council within Scotland and has been one of the fastest growing areas of Scotland in past decades. With a population of around 147,000 and predicted to grow by another 20% to 2035 they have a lot of challenges to face that a mature open data framework can help with. However, like most other public bodies the need to select the appropriate data sets which are related to city challenges was an issue. Therefore, Perth and Kinross Council embarked on a process to allow them to prioritise the datasets they can release by gaining feedback from key stakeholders.
Perth and Kinross Council ran an "open data identification" workshop with community planning partners, regional organisations and council officers from a range of services. This ensured that a wide spectrum of individuals were able to give insight into which data sets would be most useful. The workshop aimed to answer three main questions:
- What are the major challenges facing Perth and Kinross?
- What key datasets will help us understand these issues?
- How can this data be used to inform policy and service provision?
To answer these questions attendees were divided into four groups each with a facilitator. The first issue was to identify the major challenges facing Perth and Kinross under themes that included: Live, Work, Visit, Move, Learn and Inform. Attendees were then asked to come up with key datasets that were pertinent to these themes. A wide range of issues were identified including: costs of housing; GP waiting times; the services available to an ageing population; low wages; access to childcare; and access to broadband. If attendees felt that the pre-identified themes where too restrictive they were encouraged to come up with their own themes which were added to the set.
Using the answers from the first question the groups then moved onto identifying potential datasets which could help alleviate or tackle these challenges. Many datasets were identified as priority, reflecting a wide range of concerns and challenges facing the area. Among these were destination of graduates of the local University; footfall in Perth City Centre; road traffic collisions; crime data; and statistics for the number of visitors to local events and attractions.
Further notes from the day included:
- There was agreement on the need to involve community groups in the process of data selection and the crowd sourcing of community data was also identified as playing an important role in the platform.
- The need to aggregate to anonymise data was stressed. As was having a robust mechanism in place to ensure data quality.
- People wanted to know not only about spending and access to services, but how it can be broken down by area.
- The economic issues identified revolved around the fact that while unemployment is relatively low, there is a lack of high-skill, high-wage jobs.
The datasets identified by the stakeholder workshop gave Perth and Kinross' team a solid basis from which to start creating a Publication Plan, though the process also involved looking at Council priorities and the plans other local authorities have published. Along with specific datasets, there were several reoccurring issues. Perhaps the most prominent of these was the different challenges faced by urban and rural communities in Perth and Kinross in terms of issues such as transport, digital connectivity and access to services. Consideration was given to how the data platform can reflect these challenges.
In addition, it was the first real chance for the Council to "advertise" not only the benefits of open data, but also to announce the development of an open data platform to a wide range of organisations. This in particular has been vital as the team has moved towards gathering datasets; knowing that they are grounded in stakeholders' views of the challenges facing the area and bringing an understanding of the nature of the data being released.
Perth and Kinross are now considering running future workshops based around particular themes. While these are yet to be organised, they are looking to "integrate" open data in number of topics and major initiatives. For example, this includes areas like Health & Social Care integration or how the council reports and manages performance.
10. NHS National Services Scotland: Prescribing Activity Data
The Scottish Government Open Data Strategy sets out the aim that Public Sector organisations should aspire to reach at least 3-star standard open data by 2017. Information Services Division ( ISD)'s Transforming Information Programme therefore asks us to look differently at the way we publish and present data and intelligence. With England, Wales and Northern Ireland already releasing prescribing activity data in an open format, this presented an opportunity to learn from and build on the successes of those who had gone before.
A demonstrably high public interest in prescribing activity data provided us with a good starting point for publishing and developing a prescribing Open Data file. This being very much a pilot for the wider organisation, evaluation of the process and outcomes became a key component. We have seen some success in reducing the number of and amount of time spent on Freedom of Information ( FOI) Requests, and are now using this publication.
Prescribing was identified as a priority area for developing new ways of presenting NHS Scotland data and intelligence. It enjoys an already extremely high public interest in its raw data which singles it out for benefits that can be gained both from reducing the time and effort spent on, and increasing the value of, any individual data releases. The natural solution is to place all the data into one file for download by many.
A public consultation raised the possibility of open data in the autumn of 2015. It received a record number of responses (27), with roughly half from outwith the NHS. A clear divide emerged between respondents who wanted as much data as possible to be released (non- NHS), and respondents who recognised the value of open data but were to varying levels concerned about disclosure of patient and prescriber information ( NHS). A full summary of responses to this consultation is published on our website.
We decided on this basis to pilot the publication of a Monthly Prescribing Activity Data file, pending a disclosure risk assessment. An organisational aim, to have our publications compliant with UK Stats Authority requirements for Official Statistics, prompted an agreement to label them "experimental statistics" in recognition of their pilot status.
Scotland's island and rural geography uniquely shape the delivery of primary care, and so our risk assessment focussed on disclosures which might arise in this context. NHS Scotland's data landscape presents some challenges shared by other Scottish public bodies, particularly those that are community-based. Organisations may find our disclosure risk assessment helpful when contemplating their own.
The core approach of the project was straightforward: replicate the other UK nation's prescribing activity files with NHS Scotland data. Although we were fortunate that other nations had led the way, the responses to our consultation still recommended a cautious approach and we were proactive in making our plans and our protections open to scrutiny by national prescriber and pharmacy leads.
Each file comprising one month of prescribing activity in Scotland contains over a million rows of data. Given that we were publishing only a quarter of the volume of the HSCIC (England) data we chose to stick with their convention. Later on we were to encounter several parties who found these files unmanageable.
Endeavouring to reach out to new audiences we developed a communications plan centred on social media, in order to garner the interest, support and feedback of the active Open Data community. We also felt that the nature of open data meant that we had a responsibility to make the information accessible to all. Our latest development has been to produce a visualisation of the data which allows users to intuitively explore the information contained within.
ISD Prescribing aimed to achieve the following through this pilot project:
1. Complete the UK picture of prescribing trends
2. Reduce volume of and amount of time spent on FOI requests
3. Explore the practicalities of publishing open data and disseminate lessons learned to the wider organisation
4. Engage new audiences with prescribing data
Although we thought objective 1 would be completed with the publication of the data, a data quality issue, detailed on page 8 of the FAQ document, led to suppression of one of the fields. The feedback we received in relation to this one issue was extensive. Many stated that it is essential for UK-wide analysis which, with eventual publication of the field, actually validated the outcome. In national stats, acknowledgement that a data release is being used as intended is surprisingly rare, making this a genuine win.
Analysis of FOI workload so far has returned mixed results. By most measures, FOI workload has decreased. The exception to this is the workload from brief FOI requests, which would tend to comprise raw data requests subsumed by Open Data. Contrary to expectations workload on these appears to have increased as a proportion of the (lower) total. Qualitative investigation may shed light on this shift.
Practical issues that arose included how we deal with missing data, the sheer size of the file, and how the data and supplementary information is presented on the website. A lot of work has been done to institutionalise the learning from our venture. Feedback from Open Data Champions within and outwith the organisation have been invaluable in shaping this learning, as they can often confront us with the issues that organisationally we might be blind to.
As part of continuous improvement we have collected feedback and become aware of a subset of customers who wish to use the data but find the files unmanageable. With these in mind we have dovetailed this project with further pilot work in data visualisation. Users can now explore a high-level summary of each month's Open Data file at the click of a button.
- Open Data can reduce strain on services receiving relatively large numbers of FOI requests, particularly where these comprise releases of raw data.
- Organisations embarking on large data releases are likely to come across a wealth of locally-specific IT issues. These are rarely unassailable.
- Consider your audience: data and IT literacy may restrict engagement with new audiences.
- The Open Data community has established conventions by which it presents data and supplementary information. Although these are well-reasoned, largely stemming from ease of discoverability, they may not be immediately obvious to entrants in the arena. Fortunately they comprise an active and forthcoming community and their input can be invaluable when sought.
11. Marine Scotland: Open Data Network
An organisation's open data publication will likely be comprised of a number of different strands. Marine Scotland have embarked on a process to both align their open data approaches and expand the amount of data they were making open and accessible. This has culminated in the recent launch of a third open data portal to make their data more accessible.
Interactions with the new portal have been very positive and Marine Scotland are starting to promote engagement with it now it is live. With this open data network, Marine Scotland now have the right tools in place and are already making much data open and available.
Marine Scotland cover many aspects of managing Scotland's seas; from policy development, planning and licenses to enforcement and science. Marine Scotland had already made progress in terms of increasing the amount of data they are making open and accessible. However, they had a challenge in terms of converging and aligning their already available open data tools and approaches.
Marine Scotland now have three sites in place that make data more accessible, enabling the public and colleagues to more easily locate and download data.
A clear business case was presented to the Marine Scotland Senior Management Board, and the necessity to align and expand open data publication efforts was recognised. After an initial review of available publishing portals, DKAN was selected as the most appropriate platform to progress.
Marine Scotland have also mandated particular metadata standards into the process of dataset publication. Adopting the MEDIN metadata standard, no dataset will be made available on either of the three publication portals unless it has appropriately high quality metadata attached. While this occasionally slows dataset publication, it also ensures that consistent metadata collection will become a routine business practice.
Marine Scotland now have in place a 'trinity' of tools to make data more accessible:
- The National Marine Plan interactive ( http://maps.marine.gov.scot) - which delivers in depth access to GIS layers from Marine Scotland and a range of partners (Scottish Natural Heritage; Scottish Environment Protection Agency; Joint Nature Conservation Committee; Marine Alliance for Science and Technology for Scotland and many more). From here you get access to view the maps, download the ones available, and access more information about the layers.
- Marine Scotland Data Publication Portal ( http://data.marine.gov.scot). This is where Marine Scotland publish downloadable files and tabular data. The individual datasets published here are registered with Digital Object identifiers ( DOI) to allow better citation of the datasets. Currently, the Data Portal predominantly hosts more scientific data and reports. Within peer reviewed journals, there are increasing demand that data mentioned in papers is made available. There is also demand around the ability to mint DOI for datasets which allows scientists to make data available without having to place it in orphan repositories.
- Marine Scotland Information ( http://marine.gov.scot) - Marine Scotland's newest portal and where resources are tied together. On this site, Marine Scotland take in the service data from the other two platforms and provide information pages where the relevant maps, data sources and contextual information are put together. While both the mapping and data portal provide in-depth information and functionality for the specific jobs that they do, Marine Scotland Information is meant to be approachable for all. Users can search across all information types as well as more descriptive information that puts the maps and data sources into context. Marine Scotland Information merges content from several separate sections of Marine Scotland's web, which was previously used to supply datasets.
On the National Marine Plan Interactive, Marine Scotland now make approximately 800 spatial data layers/maps available to view. On the Data Publication Portal, they currently publish 100 data sets consisting of a mixture of data and reports. Finally, on the new Marine Scotland Information site, links to all of the above are available along with an additional 300 information pages crafted and maintained within Marine Scotland. It also includes hundreds of links to data sources and map layers from other providers.
Overall, the three sites take in between approximately 500 and 1,000 visitors per month each. While undoubtedly there will be a proportion of overlaps in these statistics, they are still serving up a considerable amount of data.
The work in Marine Scotland has taken a long time to get to the position where all of the appropriate tools are in place. Three sites were eventually chosen rather than one, due to a mixture of providing enough specialised tools for maps and data along with the gradual development of the approach.
Marine Scotland are now planning to do further stakeholder engagement and take in initial user feedback from visiting the sites. Now the new portal is live, they are also planning further promotion work. This is likely to be in the form of blog posts and tweets, along with an increased awareness within both Marine Scotland and the Open Data Community.
One of the key things will also be to integrate with other new exercises. So when new initiatives for data sharing in government and/or the marine data community pops up, Marine Scotland will seek to use the services now set up to be able to provide data to these exercises without additional work.
12. Aberdeen City Council: Leisure App
Many websites confuse the medium of delivery with the content that they seek to deliver. There is no separation of the information or data from the presentation, be it html, PDF, or print.
This lack of separation means that the data can only be used for the single reason for which it was published - and even that publishing is often poorly done.
By separating the data from delivery - and making it available as Open Data - allows users to consume it, and interact with it in the way that they need, potentially in much more sophisticated ways than were originally intended.
Code The City, who now run ODI Aberdeen (Scotland's first node of the Open Data Institute) have run hack weekends using Open Data in Aberdeen for the last four years. In 2014, Aberdeen City Council commissioned Code The City to run a weekend-long session on the theme of Sports Fitness and Wellbeing. The council, as a service provider, was keen to understand from a service user's point of view what got in the way of their using the city's sports facilities to their full potential.
The weekend involved sports staff, service users, coders, designers, UX specialists, data wranglers, bloggers and others.
Several ideas were put forward, refined, and then teams formed to work on these. Some rapid prototyping followed. By tea time Sunday the attendees had developed a number of prototypes which were demonstrated to the council sports staff.
One prototype sought to address a specific problem which had been identified in relation to access to sport centre timetables. Its aim was to provide not only a solution to that but to show what might be possible if a better approach was taken to timetable data.
At the hack weekend it was shown that someone coming to the city council website, in the expectation of finding information on what sports activities were provided at what time, would face two hurdles: redirection to the websites of eight arm's length service providers; then 31 different timetables in a variety of formats. Many of these were poorly presented and were designed for print, not web.
Over the weekend, the data from these timetables was extracted using a combination of techniques including manual transcription and machine scraping of the data. A database was created and all timetable data was imported, and the data enhanced, which included adding geographic co-ordinates for all venues.
Finally a search mechanism was created allowing citizens to quickly find what they wanted through a single interface.
Subsequent to the hack weekend, the Sports team commissioned the creation of a mobile app (which drew on the updated open data store) which allowed users to search, navigate and locate sports classes within geographic and time parameters; to store preferences, eliminate irrelevant classes; and for users rate classes for exertion levels. The app was designed after extensive user input and testing ensuring that specific, articulated needs were addressed.
Creating an Open Data store to hold timetable data allowed the data to be repurposed many times, for many purposes. While maintaining that data store raised issues which meant that the processes of multiple providers had to be changed to put the data (rather than the presentation) first and to ensure continuity of publication, the benefits to end users were clear and there was commitment to the continuity of data publication.
Email: Stuart Law, Stuart.Law@gov.scot
Kyle Malcolm, Kyle.Malcolm@gov.scot