Welcome to the collaboration site for the IFM project
This is the project wiki for the MBIE project "Framework for Interoperable Freshwater Models", which ran for 3 years from 2010 to 2014. The aim of the project was to investigate software frameworks for interoperable freshwater models in New Zealand. This wiki served as a collaboration space, but also documents several of the outputs, including:
- User needs elicitation workshop, with an associated report.
- Database of freshwater models and environmental data sources used in New Zealand.
- Interactive graphical tool to help search for models and identify potential linkages between them.
- Compilation of information on interoperability frameworks related to freshwater models (as of mid 2012).
- Report summarising interoperabilty frameworks and screening them in relation to user needs identified earlier, and selecting a framwork, OMS3 for testing.
- Webinar recording presenting OMS3 and our testing of it, held with a range of end-users
- Results of our testing of OMS3, our consideration of its suitability for adoption in the light of this testing, and recommendations for future work, documented in the final report.
Please browse the wiki and check out the information above.
The executive summary of the final report is shown below.
For more information, please contact the project leader, firstname.lastname@example.org
Purpose and scope
This report summarises the three-year MBIE-funded project Framework for Interoperable Freshwater Models (FIFM) which investigated flexible computer-based frameworks for integrating freshwater models. Key results and findings are presented and recommendations for further work are made.
The project addressed the need for integrated freshwater modelling to support freshwater policy and management, as expressed by the Land and Water Forum, the Ministry for the Environment Water Research Strategy, and the Freshwater Reforms. The project focussed on frameworks for interoperability, that is, software infrastructure that enables various models of water quality and quantity to interoperate. An interoperable freshwater modelling framework would enable the co-ordinated, integrated, and flexible coupling and re-use of a diverse range of models and associated data sources, leading to improved integrated modelling and freshwater management. Such interoperability contrasts with the current situation in New Zealand, where there are disparate freshwater models with only occasional inflexible integration.
The intended end-point of this project was a plan for future development of an interoperability framework. That point was reached by conducting preliminary proof-of-concept testing (i.e., demonstrating that the framework could work in practice) to assess the difficulties and advantages of a linking framework in relation to end-user requirements, and for the range of data sources and models used in New Zealand. Subsequent stages of funding would be required to develop or modify the framework to achieve a fully functional or useful implementation.
We purposefully did not embark on design of a new framework. Rather, it was preferable select an existing framework and trial it in relation to end-user needs and in the context of New Zealand models and data sources. The scope was restricted to models of freshwater quality and quantity and associated data, to ensure that the project was tractable. This focus does not preclude extension to other domains such as ecology, economics or planning in the future.
The project involved the following components:
- Developing a comprehensive publicly-accessible wiki (https://teamwork.niwa.co.nz/display/IFM) to disseminate information and findings from this project.
- Holding an end-user workshop to identify user needs for interoperability, and documentation of the findings in a report (Snow et al. 2011).
- Compilation of an inventory of freshwater models and environmental data currently used in New Zealand, categorised in relation to a common set of attributes relevant to coupling. This information is available in a structured database on the project wiki.
- Development of a tool, ModelVis, to identify and display potential linkages between current freshwater models and data sources (see Elliott et al. 2012 and the project wiki).
- Reviewing a range of existing interoperability frameworks and screening them in relation to user needs, leading to the selection of a framework for more detailed hands-on assessment. This led to a second project report (Elliott et al. 2012).
- Testing the preferred framework, OMS3, for its ability to link freshwater models commonly used in New Zealand (summarised in Section 3). This involved setting up the hydrology model WATYIELD in OMS3, converting it to a spatio-temporal model, linking to climate time-series provided by standard web services, obtaining spatial data from standard geospatial data sources, manipulating data with geo-processing routines, and displaying the model results as a time series. We also set up the farm nutrient loss model OVERSEER as an OMS3 component (either a simplified version of the engine run as a dll, or as a web service) with a simple user interface to modify rainfall. We linked the OVERSEER component to the network aggregation and routing components of the catchment model CLUES. We also attempted to set up components of the agricultural point-scale simulation system APSIM in OMS3.
- Results of initial testing were presented at an end-user webinar, and comments from this webinar were incorporated into final recommendations.
- Developing recommendations for future framework implementation, taking into account results of testing, end-user feedback, and further consideration of alternatives and recent developments in the literature (Sections 4 and 5) (this report).
End-user needs assessment confirmed the strong perceived benefits of an interoperable modelling system, such as improving integration of models across freshwater domains, re-using model components, and making better use of the increased availability of environmental datasets and associated standards.
The review of models and datasets identified a large number of freshwater models and associated datasets in New Zealand. Documenting these models and data sources in a structured database enabled better understanding of the attributes of the models, especially the potential for linking them. The ModelVis tool enables interactive searching for models by a limited set of attributes, along with graphical display of the potential linkages between models.
Over 18 existing environmental modelling frameworks were identified from a literature review. Subsequent screening identified a shortlist of six existing frameworks that met many of the criteria identified by end-users and the project team. Considering that several frameworks had promise, it was considered best to conduct hands-on testing of one of the most promising frameworks, rather than developing a new framework. The report on this phase of the project (Elliott et al. 2012) also summarised key technologies and concepts in the interoperability arena, such as web services and associated standards for providing data over the web.
The framework selected for testing was OMS3 (David et al. 2013). That framework was developed by the US Department of Agriculture (USDA) for component-based model and simulation development on multiple computer platforms. It was developed mainly for simulations of effects of agricultural systems on water quantity and quality, although broader hydrological modelling is also done with this framework and the framework design if fairly general. A key aspect of OMS3 was the desire of the USDA to foster both new model development and re-use along with integration of existing legacy models.
From the standpoint of the project, OMS3 met most of the key and high importance user needs identified at the end-user workshop including:
- Is open source.
- Receives substantial investment from the USDA as part of the large Conservation Delivery Streamlining Initiative (CDSI) programme and therefore has a high likelihood for continuity and longevity.
- Is supported by integrated development environments (IDE) for developing testing and running interoperable models.
- Has its own integrated user interface for ease of use.
- Has a considerable and active international user community.
- The primary developer was open and responsive to questions and queries.
Our trialling has demonstrated the feasibility of using OMS3 to couple freshwater models together in a variety of ways, and also of coupling models to input environmental datasets. All of the tests were successful, except for setting up APSIM. For example, we were able to set up components from a variety of sources, link and run them, develop simple user interfaces, visualise the results, and access data and simple models provided as web services. In some cases, especially where the user interface was coupled closely to the calculations, it was best to re-write the model in Java rather than attempting to work with the existing code. It was necessary to have competent Java programmers involved in setting up OMS3 components, writing the scripts that link files, and writing user interfaces. Overall, we demonstrated that OMS3 has many of the desired building blocks for a modelling framework for New Zealand.
Despite these positive signals and successful testing, users are not yet in a position to fully embrace OMS3 for several reasons:
- Funding is needed to support transferring models to the framework, maintaining the models and components, and contributing to international development communities.
- There are some weaknesses in the framework, including difficulty in setting up models written using Microsoft .NET, there is no current publicly-available repository of components, documentation is patchy, user interfaces need to be constructed from scratch, there is no core support for many data structures commonly used in hydrology (such as networks), there is little core functionality for visualising results, and there is no core geospatial awareness. Many of these shortcomings could be addressed by building new components, especially ones using third-party libraries such as GDAL, and over time more components and utilities will become available. The extra work required to build such utility components presents a barrier to adoption by a wide user base.
- The ‘indirect users’ have called for demonstrating of a polished showcase demonstration project, to demonstrate complex arrangements of model components and a polished user interface and visualisation.
- We have not yet tested OMS3 for computationally-demanding applications.
- The user base is currently small, and there is reliance on continued funding from the USDA.
- Recent adoption of OpenMI as an OGC standard for interoperability has created some confusion about the best future pathway. We do not propose shifting to OpenMI at this stage, for reasons outlined below.
Given these reservations, we re-considered alternatives to OMS3, taking into account recent literature and development.
An important development since we undertook framework screening was the ratification of OpenMI as a standard by the Open Geospatial Consortium (OGC). Potentially this could lead to wider uptake of OpenMI, especially considering the rapidly-increasing uptake of OGC standards for time series and geospatial data (including within New Zealand). Also, there are several recent papers on integrating OpenMI with other frameworks or data sources (e.g., Castronova et al. 2013b). Despite these developments, we do not currently recommend adoption of OpenMI standards in New Zealand for several reasons: there is little uptake of the OpenMI 2.0 standard by other groups so far; there are currently no open-source software development environments to aid implementation of models in the OpenMI standard; and OMS3 could be made OpenMI compliant, which would mean that OMS3 would not need to be abandoned. We recommend undertaking a periodic review of progress with OpenMI standards as part of continued development of an interoperable modelling framework for New Zealand.
A further development in interoperability is the approach of running models as a web service, building on successful initiatives in the data provisioning area. Models as a service is a promising and active area of development, and several recent papers have pointed to this approach as an important future direction. We have already successfully tested some simple models provided as web services in OMS3, and we consider that web services could overcome institutional, IP, and technical barriers to interoperability. However, the technology for running linked models as web services is young, and there are some inherent pitfalls such as delays due to transferring data over the web, so we proposed a staged process to adoption of web services.
Recommendations for further work
As a result of this project, the following further steps are recommended:
1) Develop a showcase integrated model built with OMS3, scaling up from our experience to date. This showcase will demonstrate the capabilities of OMS3 to the range of end-users, as well as provide a deeper and more demanding level of testing (for more complex and computationally-demanding models and more elaborate user interfaces).
2) Further exploration of using models as a web service. This exploration would follow a staged process, initially adopting a set of standards for New Zealand data, then setting up OMS3 components to access these data through standard web services such as SOS and WPS, setting up web-based models as web services linked to OMS3, setting up computational components as web services using OMS3, and finally moving to more web-centric linking and co-ordination technologies as they become available. This will provide early benefits (enhanced data provision to models) with a longer-term pathway to frameworks that use models as a web service, with associated benefits such as institutions being able to maintain control of their models and being less bound by programming language and platform dependencies.
3) Establish baseline funding and institutional arrangements for an interoperability framework to co-ordinate efforts across and within institutions, build consensus on standards adoption, keep abreast of developments (such as OpenMI), remain part of international communities, assist with setting up models in the framework, and provide education and training. Ideally, the local activities would be funded from a dedicated MBIE or other central government funding stream, as other funding streams are largely committed, and individual programmes are usually more interested in developing data or models in their immediate domain of interest. An alternative is to build interoperability as a project within the National Science Challenge or Centre of Research Excellence programmes.