RIP SDI-EA ver 1?

I am about to rant.

This blog has - let's face it - been pretty moribund this year and it's time to put it out of its misery, at least in its current form.

The good news is that the ad hoc SDI-East Africa effort, as it was, has been overtaken by events and - in a perfect world, rightfully - been made redundant. How so? Firstly, the work that started in 2007 was possible because certain individuals had enough institutional leeway to get away with it. Many of those individuals have moved to other organization, had their responsibilities shifted, or - as in my own case - found their institutions shifting around them. One key institution seems destined to non-existence, period.

Meanwhile, Kenya got off to a very rocky start to 2008 that focussed many origanizations' attention on life-and-death realities of addressing real humanitarian needs, not indulging in the abstract niceties of data sharing. A lot of institutional momentum was lost. November 2007 saw the SDI-EA showcase exercise for UNGIWG and then.... nothing.

So, now, to SDI-EA 2.0, or Son-of-SDI-EA, of SDI-EA - the Next Generation or whatever we seek to call it. Why will this be better, stronger and faster? Firstly, it seems that its natural institutional home is coming of age. A year ago, NASA and USAID opened negotiations around the notion of a SERVIR-Africa, building on the experiences building SERVIR at CATHALAC in Panama. SERVIR has been described before in this blog and elsewhere in ways that don't need re-hashing here. What is newsworthy is that, just over a week ago, Africa-SERVIR was inaugurated at the Regional Centre or Mapping for Resource Development (RCMRD) in Nairobi. The lights are on, the disks are spinning, the staff are trained. Now, let the apppications begin.

Why is RCMRD a 'natural home' for SDI-EA? Because it has a regional mandate to serve over 15 states. Because it is already a centre of technical excellence. Because it has a long track record - nearly 30 years - of international partnership and support from many OECD countries. because its management 'get' the notion of SDI as a necessary tool for national development in its constituent states, and because they are good neighbours to and collaborators in their host country's efforts to build a Kenyan national SDI. No agency of the UN system based in Nairobi has so many of the right attributes.

Secondly: the institutional context for the UN bodies that kick-started SDI-EA is also changing. Namely, the evolution of a UN SDI that stands to move these types of data and service sharing initiatives out of the realm of well-meaning amateurism into one that programme managers can understand and treat seriously. To whit: the meeting of UNGIWG in Vienna last month saw 31 UN agencies unanimously endorse a statement directed to the highest levels of UN management that SDI is a necessary capability in which the UN must invest.

The text of statement has not appeared on-line yet but the fact that such a variety of interests - headquarters, field operations, humanitarian, food security, disaster relief, environment, nuclear regulation, public health, security and peace keeping - sent such a clear message all the more powerful at a time when UN business is under heavy scrutiny and the demands for reform are strident. Yay. Back that statement up with a clear, deliverable work plan - also agreed at UNGIWG-9 - and it now almost becomes behoving upon UN management and donors as to why such a sensible, noble and widely-agreed recommendation ought not get the funding it needs.

So: SDI-EA can now legitimately be seen as the juxtaposition between delivery a UN-wide joined-up services and a regional-wide clientele leveraged by RCMRD, an example of what I think of as an 'apex network' - the type of hight value single point of contact through which the limited resources of the UN can reach the greatest number of Member States. Let UNEP work across eastern and southern Africa with RCMRD and with UNHCR and FAO and UNIDO and WHO to join up services to focus on issues of environment and health, or climate change and ecosystem management, or the social and environmental dimension of agri-business such biofuels.

Let the applications begin. Roll on SDI-EA 2.0

Benefits of Open Access Data Policies

Tom hammond at IUCN Canada posted the following to the Conservation Commons mailing list today. It describes the measurable impact felt when Canada opened up access to what had previously been fee-based access to standard framework national data sets, and you'll see that the result is impressive.

What I will be more curious to see is what effect this has on the quality of these data once they're being used more by more people in more different applications - more aggressively perhaps - and the Government starts getting specific feedback about inaccuracies, omissions and required updates. A key component of the "Better Data Sooner" mantra is that community-driven feedback is the best driver for custodians' continuous improvement of their data products and services.

Perhaps the Canadians are providing a real-world laboratory in which to test this theory.

" The HYPERLINK http://www.cits.rncan.gc.ca/cit/servlet/CIT/site_id=01&page_id=1-005-002-001.html National Topographic Data Base (NTDB) comprises digital vector data sets that cover the entire Canadian landmass. This product includes thirteen“layers” such as hydrography, hypsography and the road network.

" The NTDB is a complete and uniform product that can be highly useful in a broad range of activities– such as planning, research,conservation work,and private sector development. For example, it can be used for preparing thematic maps and makes it suitable for geographic information system (GIS) applications because the NTDB and its attributes make it possible to use a variety of spatial analysis techniques.

" Downloads of NTDB data during the government fiscal year ending in 2007, when a fee for use policy was still in place, numbered under100,000. A change of policy was enacted during the current fiscal year making the NTDB an open access resource– during which downloads jumpedby a magnitude of 54to well over 5 million."

HYPERLINK http://www.thinkwell.ca/cgdi-icdg/libraryDocs/FeevsFree.pdf

'Scraping'the web for geo-information

By now most of us will have seen the joint ESRI/Google announcements at Where 2.0 about the upcoming ability to use Google Earth and Google Maps to discover, locate and portray geo-data hosted in ESRI ver 9.3 geo-server stacks to be released later this year.

In the interest of balance (and with no desire to provoke any flame wars) I draw your attention to another announcement at Where 2.0 addressing Geoserver's upcoming 1.7.0 release with similar capability - see http://blog.geoserver.org/2008/05/13/geoserver-and-googles-geo-search/

The point here is not about commercial versus free versus proprietary versus open source software. It's about two other more important things.

First: the maturation and acceptance of standards. Google has embraced the OGC specifications process and KML is now an anointed OGC standard; ESRI has embraced the OGC specifications standard and is positioning its products appropriately (e.g. ver 9.3 will also provide a fully-compliant web feature service as well as full support for PostgreSQL/PostGIS back-ends). The more this momentum build the more of this joined-up capability we'll all benefit from.

Second: the maturation and acceptance of visualization tools for non-specialists. If the Google/ESRI announcement had happened 5 years ago there'd have been barely a ripple outside the hard-core GIS community. The fact that The Rest of Us can now contemplate being able to publish geo-information (whether social and community based, conservation-related or whatever) --and-- have it discovered, integrated, employed and appreciated by non-specialists audiences via attractive spinny-globe-type-products is a powerful combination.

What is crucial here, and to me will make or break this effort, is the ready availability of easy-to-use tools to take my existing geo-data and get it into a search-able, discover-able, use-able form without having to have a masters degree and without having to learn to speak techno-babble. And that is equally true whether 'I' am individual Mick Wilson or whether 'I' am a multi-national conglomerate.

We need to keep lowering the bar to the publishing of real-world data about real-world events by real-world-people.

A Little Ray of Sunshine

Well! Finally some light falling upon what has been a decidedly dark and gloomy SDI-EA landscape over the past few months, at least I hope so for the practitioners in this region.

I yesterday discovered that, after almost a year of hushed expectation, the UN in Nairobi has finally gotten operational with KIX, the local Kenyan internet exchange point. The upshot is that users of any of the local network providers in Kenya now have much better access to web services operated at UNEP Headquarters. And by 'much' I mean that I tested it from home last over my dial-up connection and response times to sites like http://dewa03.unep.org/geoserver/ or http://www.dephadata.org were about 1/20th what they used to be. The practical consequence is that there is now a chance for real operational interoperation with partners in Nairobi like KNSDI (who, by the way, have started operating a geo-portal at http://212.49.76.177:8080/Portal), the CG centres, FAO/SWALIM, KWS, ACC and the like. The long-awaited showcases recommended at the Better Data Faster workshop last October might start making better headway.

And for the tech-heads out there - why does the KIX link make so much difference? Because, up until last week, internet packets going from my house in Nairobi to servers at UNEP Nairobi had to go all the way to Canada, across to Italy then back to Nairobi via the UN satellite links; of course, responses from the servers then had to traverse that path the other way a total of 4 satellite hops to geostationary orbit and back and each introducing 750 ms latency because of the finite speed of light. It was impossible to get a response in less than 3 seconds, no matter how speedily the servers at UNEP run. Then there's the factor of congestion i.e. that the satellite links are expensive and therefore kept as narrow as possible, which means at peak times packets have to queue and so are further delayed. In short, the UNEP internet experience was less than dazzling for folk here in east Africa. The link to KIX eliminates all that fiddling around with satellites and enables a brisker user experience. Yay.

So, why have these past months been "dark and gloomy"? A number of factors...

First was the whole disturbed environment in Kenya at the start of the year, which rightly preoccupied most GIS workers either professionally or domestically, so there was little chance to maintain momentum. Secondly, I lost my loyal and faithful sidekick John Mugwe for lack of contract renewal. When it came to delivering hands-on training about geographic web services in local institutions, John was far better able to communicate the practicality of OWS setup and operations than I ever was with my tendency to talk over peoples' heads. So the whole capacity building effort has ground to a halt as well. Thirdly, UNEP as a whole and the section where I work (the Division of Early Warning and Assessment) has been going through a protracted period of programme re-alignment, so it's been back-to-back management retreats for the better part of a month, with all that that entails about re-crafting work plans, drafting strategy papers and the likes. Little time left to even think about SDI-EA.

Anyway, I hope that's all history and that we can get back on track. We've just gone past the first anniversary of SDI-EA (yay, again) and I guess we can point to some progress but still have a ways to go....

And a question for next time: we now have the global geospatial xperts social networking site, which has in a few days attracted an impressive amount of talent (over 620 players) from around the world. I have set up an SDI-EA group there and now wonder: should I migrate this blog that new home? Your opinions, please.

Life and Times in Nairobi

The post-election violence in various parts of Kenya was front-page news for at least two or three days, a remarkably long period. Kenya will be months or years recovering from the aftermath, one dimension of which is more than 200,000 people estimated by the UN to have been displaced. Setting up camps and safe-guarding the inhabitants, ensuring food and medical supplies, establishing and maintaining sanitation are task that will keep UN agencies, NGOs, IGOs, church groups and the Government engaged long after the story has ceased to be even page 10 newsworthy.

The crisis has - no surprise here - exposed a number of shortcomings in the management of the information needed to plan and manage responses. The story has been the same old confusion, duplication, crossed efforts and lost time that always seems to mark such times. Yes, of course, crises always occur at the most inconvenient time and the circumstances in Kenya were particularly confounding - key staff are out of town or out of the country because it's still holiday time here means that many ; the UN in Nairobi effectively closed down for the whole Christmas - New Year week and then stayed closed for the first week of January, with staff ostensibly 'working' from home so no access to servers or datasets that weren't visible to the internet; some key staff who happen to have data on their computers can't get to office safely. Mostly, though it exposes the fact that few agencies as yet manage and publish their data in ways that enable true use and re-use a la SDI.

Pertinent points include:

• UN bodies in New York seeking to upgrade their security plans for Nairobi, and looking for the data used to create the last set of maps, couldn't find them and contacted me (at home) by e-mail (on a Friday night) wondering if I could find the right people they think might have had the right data;
  
• the custodians of the data respond, but one's out of town on holiday, one is in town but cannot get to the office, and anyway the data are on the hard disk of a third staff member's PC and they don't have the password;
• the data were never published to the web, nor was their metadata, because they were prepared as a small contract job and no-one thought there'd be any re-use. Whoops.
• a colleague in another agency not only has the requested data (not just the PDFs) but even better built up for their own purposes since, but shouldn't distribute them as per agreement with their originator... but then again his agency has rectified the data and extended their attributes so, yes, okay, he sends the data to New York, who are appropriately appreciative...
• ...except that the data dispatched are in a proprietary format because that's what they use in their office. Does New York use the same software? I have no idea but would think it a damn shame if NY couldn't use the data for lack of a software license...
  
• ...and, meanwhile, a third colleague charged with coordinating information management for humanitarian response is sending out requests to the same people for the same type of data, admittedly for different application but still involving the same players in more work.
  

They foregoing highlights just how far we have to go with an exercise like SDI-EA before we can claim real legitimacy:

• where is the one-stop shop catalogue discovery system that would enable to New York to at least confirm the identity of the custodian of the original data? Such catalogues are alive and kicking in Nairobi (see FAO/SWALIM's GeoNetwork node) as resources for the community - why aren't they being used?
  
• where are the metadata being routinely published that describe the sort of interim data products that underlay the UN security plans, that in a very real sense have been produced using taxpayers money, but are allowed to languish in dark cupboards?
• where is the sense of planning for re-use, of seeking to maximize return-on-investment on these data products? Why is it not yet routine that all data end up in managed repositories from where maybe they could be published to the web? in vendor-neural formats that guarantee their re-usability?
  
• where are the instances data improvers being willing to return to the data originator the enhancements that they've made? Can we ever expect to do better than using the same old tired framework layers from DCW or whatever if no-one is prepared to feed back to authoritative custodians their improvements made to baseline products?
• where are the mechanisms by which data improvements can be proposed or lodged? How many data distributors have procedures in place for dealing with feedback from their users when it comes to quality improvement?
  

Sorry to start the New Year with a rant like this but these issues are to important to not grasp an opportunity to highlight them. Fine we can talk melodramatically about "life and death situations in Kenya" and not be entirely exaggerating, but even the prosaic tasks of getting food distributed to camps on time, or ensuring that water is tanked from the most accessible clean sources, can help alleviate an otherwise completely miserable situation for thousands and thousands of people.

Showing the World

Yesterday was one of the coolest and most satisfying in a long time. We finally got to show an international audience that SDI-East Africa is at least taking baby steps and could in the another year or so start walking confidently and even be running, skipping and jumping.

The venue was the 8th session of the UN Geographic Information Working Group (UNGIWG). Long-time readers might recall was that it was at the UNGIWG meeting last year that the first seeds of SDI-EA were planted, then heavily fertilized at the UNGIWG Global Partners' Meeting earlier this year. So it was somewhat satisfying to return to that forum and show that their inspiration had wrought.

The previous post post to this blog noted that, since the East African consultation back in October, there'd been a flurry of inspired activity to actually get data-sharing services on line. Yesterday we were able to show how flood-related data, originating from four different UN agencies (UNHCR, OCHA, FAO/SWALIM, UNEP and GEMS/Water) and hosted on three different servers in Nairobi and one in Canada, could be at least visually integrated to provide humanitarian relief managers with both synoptic and detailed views of potential impacts on refugees and displaced persons in the region. Across a variety of open-source and commercial services, all agreeing to 'speak' an OGC-standard interoperability dialect called Web Feature Services.

These views were previously separately available but never before brought together on demand. We were also able to showcase the crucial role to be played by facilities such as the inter-agency Data Exchange Platform for the Horn of Africa (DEPHA) as a broker publishing data on behalf of agencies that cannot afford or lack mandate to build the capacity to publish data on-line themselves. The KML needed for spinning the showcase up in Google Earth is <here>. Give me a month or so to get back from leave and I'll have equivalent packages for NASA WorldWind, MS Virtual Earth, uDIG and QGIS, all working off the same services

The amazing part is that it worked. Not just the technology, but the message - UN agencies field and regional offices can actually afford the luxury of starting to think about this sort of inter-operation. The technology hurdles are not the insurmountable barrier so often assumed.

Yes, the scenario shown was limited and somewhat contrived. Yes, there were many, many components of a true SDI missing, like the abilities to discover and integrate additional mdata sources, or to discover and display stuff using the correct UN-standard symbols, or even to know the most basic background information about where the data originate or how they can realistically be used. On the other hand, others here in the UNGIWG meeting do Get It and are keen to start plugging gaps in the next year - FAO Geonetwork will work with us to plug the discoverability gaps; OCHA will work with FAO to get symbologies hosted, discoverable and accessible; WFP with the UN Joint Logistics Centre and the ITHACA project will starte getting their transportation data model to integrate automatically to help drive the symbology and portrayal needs. All good, One-UN sort of stuff! I believe we have a viable kernel around which the emerging UN spatial data infrastructure will gain and document its own experiences and growing pains and lessons learned.

Now I just have to convince my bosses to let me keep up my involvement in all this as we move into next year's shiny new work programme!

I'm off on holiday for three weeks. I may or may not be inspired to follow up this post soon - I should: there were some interesting chats with the ESRI rep that bear telling.... If I don't, and happy end-of-year/ mid-winter/ mid-summer/ whatever season to you all

Sunshine and Happiness

A few months back I realized that, after six months of collective SDI-A effort, there was not actually any more data flowing in East Africa than before when we started the exercise. I got all despondent and sort of lost interest in maintaining this blog. Silly me. Events during the past three weeks have brought the sunshine back into the world of SDI-EA.

The first was the meeting at the end of October that brought together over 40 of the SDI-EA players in East Africa under the theme of Better Data Sooner to consider the question of how best a United Nations SDI would have to be run to be most useful to countries, organizations and societies in the region. Yes, there were all the recommendations about how the UN ought to help SDI proponents with getting policies and standards in place, with finding capacity building opportunities, with getting data flowing out of the UN system while providing opportunities for governments, NGOs and sectoral programmes to publish their data into the UN system. There's more about the SDI-Live effort at http://dewa03.unep.org/live-sdi/ and the report should be out Real Soon Now. I'll describe the motivation for the meeting in a later post but suffice to say it s recommendations seem needed as input to the upcoming UN Geographic Information Working Group UNGIWG meeting in Bangkok when we consider UNSDI implementation over the next year.

What was the real surprise was that, having recognized the crucial necessity of communicating clearly with senior policy types, the participants hit on the notion of building SDI showcases around realistic and solid scenarios that managers could associate with. What was even more surprising was they actually went off and started doing it. Within the space of 10 days we went from having only one real internet-accessible source of data on the network in East Africa (that being UNEP) to having nearly half a dozen - FAO/SWALIM, UNHCR, OCHA, UN-Habitat - and can start telling meaningful stories: a lot of the current data being served concerns the floods across Africa during 2007, their potential impacts on refugees and displaced persons. All of this can be spun up in Google Earth, WorldWind, desktop GIS and the likes. Have a look at this bit of KML ( http://dewa03.unep.org/sdi-ea/system/files/SDI-showcase.kmz ) or at least the screen caps here and here to get an idea where this might go.

It's actually becoming necessary to think about getting a services registry going for this part of world!

Now, how to maintain this momentum? How to use this profile to get more services running - RCMRD, are you reading this?