The DEVELOP Program is a dual capacity-building program in which teams of participants conduct rapid 10-week feasibility projects with partner organizations to assist in environmental and policy decision-making. Located within the Applied Sciences’ Capacity Building Program in NASA’s Earth Sciences Division, DEVELOP seeks to build capacity in its participants and partners to use NASA’s Earth observations for decision-making.
DEVELOP projects apply NASA Earth observations to a broad range of problems in diverse focus areas, including water resources, agriculture, disaster management, and ecological forecasting. As each project is designed to assist a particular partner organization in its decision-making process, projects vary in scope, design, and satellite data utilized.
As a result, there is no “fixed” set of indicators that encompasses all relevant impacts of all projects. Rather, a flexible toolkit of shared indicators and individualized approaches is needed to capture the diverse outcomes of these projects, while still allowing for comparability of the projects. This has been done through the creation of pre-and post-project partner assessments that capture partner needs, capabilities, and expectations. This provides both baseline data and an overview of project impacts on partners. Selected projects are then individually assessed in greater detail through partner follow-ups and research into the quantification of project impacts utilizing interdisciplinary approaches to fit each project.
In addition, to build broader public awareness of the applications of NASA Earth observing data, DEVELOP has partnered with Earthzine to host Virtual Poster Sessions (VPS) since 2011. Each VPS includes short videos created by each DEVELOP project that summarizes the community concerns, methodology, NASA data used, and highlights how the partner organization will apply the project results in its decision-making.
DEVELOP Partners
DEVELOP partners with a variety of organizations including government (local, state, and federal), nonprofits, academic, and international organizations. Besides addressing a variety of decisions, partners face challenges of using Earth observations for decision-making due to varying levels of remote sensing experience.
Through its projects, DEVELOP seeks to build capacity in its partners by highlighting what NASA space-borne and airborne remote sensing data exist, providing information on how to access the data, expanding their knowledge of how NASA Earth observations can be used in decision-making and demonstrating applications of its use. To help ascertain how well this is being accomplished, partners are given assessments before and after the project to assess needs, capabilities, and expectations.
Pre- and Post-Assessments in the Project Timeline
The pre- and post-assessments have been incorporated into the DEVELOP project timeline. Once a partnership is initiated, a project proposal is developed between the DEVELOP team that will be conducting the project and the partner to ensure that the project will address a particular need of that partner.
The proposal is then approved by the DEVELOP National Program Office and NASA Applied Sciences Program management, and the pre-assessment is given to the partner to gather additional information about the partner’s needs and capabilities that can be used to further tailor the project. The assessment also provides baseline data on the partner’s capacity that can be compared to the post-assessment, after the project is completed.
The team then conducts the project with continued engagement of the partner through regular communication (e.g., teleconferences, emails), and depending on the project, other activities such as fieldwork, to provide regular updates on the project and allow for partner feedback on ensuring the project meets the partner’s needs. For example, DEVELOP team in Mobile, Alabama, recently partnered with the Padre Island National Seashore of the National Park Service (NPS) to investigate the potential effects of increasing mesquite trees on estuary health.
Throughout the project, the DEVELOP team and the NPS partner communicated weekly during which the team discussed project progression and the partner provided his subject expertise and feedback. At the end of the project, the end products and tutorials are given to the partner for implementation and use. The partner is then given the post-assessment to evaluate their current capacity, project experience, and utilization of the end products. From the post-assessments, partners are selected to follow up with for more in-depth discussions of the impacts resulting from their use of the end products.
Selected Partner Follow-Ups
Each project is designed to focus on a partner’s particular decision-making needs and therefore has a specific application it is addressing. This creates a wide variety of impacts to quantify and makes comparing impacts across projects challenging. To address this, DEVELOP uses the pre- and post-assessments for all partners, and then follows up with individual partners to discuss one-on-one how they are using project tools in their decision-making. For example, the DEVELOP team at NASA’s Stennis Space Center partnered with local groups in southeastern Louisiana that were working to address the issues of storm surge on coastal parishes by planting trees, as greater vegetation lessens the storm surge effects. The team developed site suitability maps for planting trees, which helped the partners focus their resources during the decision process to determine where to plant about 700 baldcypress and 100 live oaks in 2013-2014.
Next Steps
DEVELOP is working to improve its systematic assessment of impacts for following up with partners, as well as increasing the overall response rates of partners. Future statistical analyses also will serve to improve the robustness of results to date.
DEVELOP is continuing to systematize its impact assessments through a Project Strengths Index, a rubric modeled after NASA’s Applied Readiness Levels (ARLs) for measuring a project’s strengths and progress. An additional challenge lies in determining the appropriate schedule for allowing partners the time to implement the end products, while maintaining the partner relationship in order to actualize impact from product implementation.
Considerations and Limitations
While DEVELOP has begun to see successes in this process, there are several limitations to consider. As “capacity building” is a broad concept, how this term is operationalized affects how and what impacts are assessed. Often the end products that DEVELOP projects produce are meant to be one tool in a decision-maker’s toolbox of resources to address a particular decision. As a result, distinguishing the level of effect of each of the many facets used in a decision is a complicated endeavor. Adding the timeframe needed to see the outcome of decisions makes this process even more complex. In addition, the reliance of self-reporting as the primary method of data collection needs to be considered. Furthermore, a low partner response rate has limited the response pool available for analysis. Low response rates are being improved by setting clearer expectations at the beginning of partnerships. In addition, the “newness” of the assessments means only a limited pool of partners has completed all steps.
Conclusions
Overall, the standardized pre- and post-assessments allow for a measurement of capacity built in partners through the course of the project, as well as comparison between partners. Challenges remain in assessing the impacts of implemented project end products, which are varied and require a longer timeline for evaluation. Combining these assessments with individualized follow-up interviews allows for better understanding of impacts on individual partners. Although more work needs to be done to improve this process, DEVELOP continues to seek and pursue efforts to measure the impact of using NASA Earth observations as a resource for decision-making by its partners.