diff --git a/_quarto.yml b/_quarto.yml index a8a10f0..b6b3e60 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -24,6 +24,8 @@ website: href: champions.qmd - text: "Cookbook" href: cookbook-navbar-page.qmd + - text: "News" + href: blog.qmd - text: "Events" href: events.qmd - icon: github diff --git a/about.qmd b/about.qmd index 5544677..f78f8a9 100644 --- a/about.qmd +++ b/about.qmd @@ -34,7 +34,7 @@ Through this work, some highlights of impact to date: (1 Engage): 7 DAACs partic ## Blog -**We write [blog posts](https://openscapes.org/blog#category=nasa-framework) about NASA Openscapes on our main openscapes.org blog.** +**We are cross-posting [blog posts](blog.qmd) about NASA Openscapes from our main openscapes.org blog.** **Additional blogs about our work:** diff --git a/blog.qmd b/blog.qmd new file mode 100644 index 0000000..14b3172 --- /dev/null +++ b/blog.qmd @@ -0,0 +1,15 @@ +--- +title: "NEWS" +listing: + contents: blog + sort: "date desc" + type: default #grid + image-align: right + max-description-length: 175 + date-format: medium + categories: true +page-layout: full +#title-block-banner: "#F8D98F" # use this for solid color banner +#aliases: +# - https://www.openscapes.org/blog/ +--- diff --git a/blog/2023-02-21-news-february-2023/index.qmd b/blog/2023-02-21-news-february-2023/index.qmd new file mode 100644 index 0000000..1022e6b --- /dev/null +++ b/blog/2023-02-21-news-february-2023/index.qmd @@ -0,0 +1,67 @@ +--- +title: "Openscapes Newsletter #6: Winter 2023" +author: Julie Lowndes and Stefanie Butland +date: 2023-02-21 +slug: news-winter-2023 +categories: + - blog + # - community + - newsletter + - nasa-framework + - noaa-fisheries +image: sb_paradise_road.png +aliases: + - https://www.openscapes.org/blog/2023/02/21/news-february-2023/ +--- + +## **Openscapes Newsletter #6: Winter 2023** + +*Welcome to Openscapes' sixth newsletter! If you're interested in seeing these infrequent updates in your inbox, please [sign up here](https://docs.google.com/forms/d/e/1FAIpQLSdgVXRp3V-w94GPWkR31RUfyBl37EphdQSlCOcnyeNlf8OLWw/viewform) (linked from our [get involved](https://openscapes.org/contact) page).* + +------------------------------------------------------------------------ + +Hello! As we continue into 2023, we at Openscapes continue to come back to the core of what we do: we engage, empower, and amplify. Whether it is with tech like Quarto and JupyterHubs or communities like R-Ladies, Ladies of Landsat, Black in Marine Science, and NASA Earthdata, it's about welcoming folks to better ways of working and open science. + +In 2023, we're committed to being braver in connecting the impact of our work to the environment and the climate movement; to culture and mindset change for individuals, for teams, and for institutions; and as always, to kinder science. + +**In the last year**, what emerged was a clear sense of movement building towards better science for future us. The Openscapes community grew as Champions and Mentors prioritized time for learning, sharing, teaching with each other, and also advocated for themselves and their colleagues to have time to do so. + +- [NASA Openscapes Mentors](https://nasa-openscapes.github.io/mentors.html) continued co-creating and reusing teaching resources to support colleagues and researchers using NASA Earthdata to migrate workflows to the Cloud. Mentors also taught workshops, gave talks, participated and created in public forums, and built software and conceptual infrastructure to support users and practice open science, and are also investing their knowledge and lessons back into the [Earthdata Cloud Cookbook](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/). Co-lead Erin Robinson led a publication now under review about The Openscapes Flywheel: A framework for managers to facilitate and scale inclusive Open science practices ([preprint](https://eartharxiv.org/repository/view/4560/)). + +- [NOAA Fisheries Mentors](https://nmfs-openscapes.github.io/mentors.html) supported their colleagues through six Champions Cohorts. Read about the experiences and impacts of Fall 2022 Openscapes training [nationwide](https://openscapes.org/blog/2023/01/24/2022-noaa-nmfs-fall/), with additional posts from NOAA Fisheries [Alaska](https://openscapes.org/blog/2023/02/16/2022-noaa-afsc-fall/), and [Southwest](https://openscapes.org/blog/2022/12/09/case-study-swfsc-sael/), Summer [Southeast](https://openscapes.org/blog/2022/09/30/2022-sefsc-summer/), and Winter [Alaska](https://openscapes.org/blog/2022/04/07/afsc-supportive-forum/) Fisheries Science Centers. Additional resources include the [NMFS Openscapes](https://nmfs-openscapes.github.io/) site and [NMFS Open Science Resource Book](https://nmfs-opensci.github.io/ResourceBook/). + +- [California Water Boards / CalEPA Mentors](https://cawaterboarddatacenter.github.io/swrcb-openscapes/contact.html) forked the Champions program and co-led the first Cohort for the California Water Boards ([blog post](https://www.openscapes.org/blog/2022/12/02/swrcb-2022/)) and fully documented it in a Quarto book, [Openscapes at the Water Boards](https://cawaterboarddatacenter.github.io/swrcb-openscapes/). + +- 240 Champions participated in Openscapes this year; their work and energy has been awesome! It's so wonderful to see teams of Champions come back to guest-teach their peers and share their ongoing journeys toward more open, kinder science. See their stories above with NASA, NOAA Fisheries, and CalEPA. + +- Ileana Fenwick and Stefanie Butland joined the Openscapes team ([tweet](https://twitter.com/openscapes/status/1534951248369025031))! They both do incredibly valuable work and we are so grateful to work together. + +- We hosted 3 Community Calls: A Qualitative Data Analysis Chat with Dr. Beth Duckles; Hello Quarto! A Quarto Chat with NASA Openscapes, co-hosted with R-Ladies Santa Barbara; Reimagining open science as part of the climate movement, a chat with Dr. Monica Granados ([summary posts](https://www.openscapes.org/tags/community-call/) of Community Calls). + +- Julie gave a co-keynote at rstudio::conf(2022) with Mine Cetinkaya-Rundel, titled Hello Quarto: share, collaborate, teach, reimagine. This was the official launch of Quarto ([blog post](https://www.openscapes.org/blog/2022/08/10/quarto-keynote/)) and it was such an honor to share about the Openscapes community on this big stage! + +- One of the things we hear often from researchers is that they feel like they can't take vacation - too much rests solely on their shoulders, solely on their laptops. In the shared-joy and sustainability-through-open-science departments, Julie wrote about how our open science process made it possible for her to unplug for a 2-week vacation ([blog post](https://www.openscapes.org/blog/2022/11/03/vacationing-with-open-science/)). + +- Shoutout to all the Champions and Mentors and community who have shared their successes - big and small - with us. You keep us energized! + +**This year**, we're continuing our work with NASA, NOAA, and the California Water Boards and have started new relationships and programs. We're excited to continue to grow the Open Science movement as part of the Year of Open Science ([Biden Harris administration announcement](https://www.whitehouse.gov/ostp/news-updates/2023/01/11/fact-sheet-biden-harris-administration-announces-new-actions-to-advance-open-and-equitable-research/)). + +- Already we've launched [Pathways to Open Science](https://openscapes.github.io/pathways-to-open-science/) for Black environmental & marine researchers, led by Ileana Fenwick in partnership with [Black in Marine Science (BIMS)](https://www.blackinmarinescience.org/) and [Black Women in Ecology, Evolution, and Marine Science (BWEEMS)](https://www.bweems.org/). 80 people attended the first session(!) and we are committing to offer this annually. + +- At the [ESIP Winter meeting](https://2023januaryesipmeeting.sched.com/event/1EwXx) in January we coordinated a panel discussion. Mentors from NASA, NOAA Fisheries, CalEPA, and Pathways to Open Science programs shared their stories of open science movement building through Openscapes. Look for a summary blog post in March! + +- February through May, five research teams from the U.S Environmental Protection Agency (EPA) Center for Environmental Measurement and Modeling (CEMM) are participating in an Openscapes Champions Cohort with help from new Mentors Gayle Hagler and Jeff Hollister. + +- Thirty Openscapes Mentors across all of our programs and the Openscapes core team are coming together in their own six-session Cohort to practice Mentoring with a Coach Approach, led by [Tara Robertson](https://tararobertson.ca/), professional coach and Openscapes advisor and collaborator. + +**Coming up** + +- Nominations open for our 2023 [**NASA Champions Cohort**](https://nasa-openscapes.github.io/champions) that will run April - June. Please spread the word for teams using NASA Earthdata and interested in migrating workflows to the Cloud. + +- **Community Calls** with Ileana Fenwick, Aneese Williams and Alex Davis about their experiences leading the Pathways to Open Science program, and with Sean Kross describing the workflow creating `kyber`. + +- In June we'll launch a brand new program, **Openscapes Reflections**! This will be a 3-week program, modeled after leadership coaching in a mostly self-paced format. It will offer structure and accountability for participants to reflect and plan on analytical and reporting workflows to improve daily habits and fuel institutional culture change. + +Thanks for reading, and here's to kinder science and the Year of Open Science 2023! + +

diff --git a/blog/2023-02-21-news-february-2023/sb_paradise_road.png b/blog/2023-02-21-news-february-2023/sb_paradise_road.png new file mode 100644 index 0000000..085625f Binary files /dev/null and b/blog/2023-02-21-news-february-2023/sb_paradise_road.png differ diff --git a/blog/2023-03-16-esip-winter-2023/esip-2023-speakers-square.png b/blog/2023-03-16-esip-winter-2023/esip-2023-speakers-square.png new file mode 100644 index 0000000..5da1bde Binary files /dev/null and b/blog/2023-03-16-esip-winter-2023/esip-2023-speakers-square.png differ diff --git a/blog/2023-03-16-esip-winter-2023/index.qmd b/blog/2023-03-16-esip-winter-2023/index.qmd new file mode 100644 index 0000000..0f673c4 --- /dev/null +++ b/blog/2023-03-16-esip-winter-2023/index.qmd @@ -0,0 +1,119 @@ +--- +title: "3 approaches for the year of open science" +author: Corey Clatterbuck, Ileana Fenwick, Josh London, Luis Lopez, Cassie Nickles, Adyan Rios, Stefanie Butland, Julie Lowndes, Erin Robinson +date: 2023-03-16 +slug: esip-winter-2023 +categories: + - blog + - conference + - noaa-fisheries + - nasa-framework +image: esip-2023-speakers-square.png +aliases: + - https://www.openscapes.org/blog/2023/03/16/esip-winter-2023/ +--- + +*At the 2023 ESIP Winter Meeting, "Opening Doors to Open Science", we held a session called "[Better Science for Future Us: Openscapes stories and approaches for the Year of Open Science](https://2023januaryesipmeeting.sched.com/event/1EwXx)" with speakers from University of North Carolina (UNC), National Oceanic and Atmospheric Administration (NOAA) Fisheries, California Water Boards, NASA's National Snow and Ice Data Center, and NASA's Physical Oceanography Distributed Active Archive Center. The goals of this session were to hear from and boost a diverse set of leaders from across the US government and academia to highlight open science in daily work, including peer-teaching, mentoring, and learning. Building from our session at ESIP Winter 2022, we hope to create more channels for inter- and cross-agency learning, and share open science stories across agencies, as part of the [2023 Year of Open Science](https://www.whitehouse.gov/ostp/news-updates/2023/01/11/fact-sheet-biden-harris-administration-announces-new-actions-to-advance-open-and-equitable-research/) as recently declared by the US Biden-Harris Administration. Speakers shared stories about open science in government and their experiences with Openscapes. Stories were shared in a "Fishbowl" format, where speakers each shared and then there was a broader discussion with the 50+ participants. This blog is co-authored with the speakers.* + +*Quick links:* + +- [*ESIP Session page*](https://2023januaryesipmeeting.sched.com/event/1EwXx)*- links to with [video recording](https://youtu.be/1IWFIyPzsKY) and [notes](https://docs.google.com/document/d/1hUnAwjyvK8rxsybPlhwgBrKZ3pCG_1YekRLfBWtRfdg/edit)* +- *Movement building through:* + - *A first welcome to open data science - Ileana Fenwick ([slides](https://docs.google.com/presentation/d/1j9TAXxCw1zjXBcYMe4UizOgo6LR8dRn3/edit?usp=share_link&ouid=117331415908654768512&rtpof=true&sd=true))* + - *Inside NOAA Fisheries - Josh London, Adyan Rios* + - *Forking Openscapes internally at the California EPA Water Boards - Corey Clatterbuck ([slides](https://docs.google.com/presentation/d/1Okil31Fe0fXEfGeAEr84GMdcqi4dFIiQtMCCFZwTljk/edit?usp=share_link))* + - *Contributing back to open science at the NASA Data Centers- Cassie Nickles, Luis Lopez ([slides](https://docs.google.com/presentation/d/1nCnIxfsTC2XazHK-JvbL4qcBEB77rxSBFZp59Q4G93c/))* + +------------------------------------------------------------------------ + +## 3 approaches for the year of open science + +This session brought together a diverse set of leaders from across the U.S. government and academia to highlight open science in daily work, including peer-teaching, mentoring, and learning. They shared stories, examples, and concrete tips for supporting each other and our colleagues with collaborative, inclusive open science approaches, with the aim of strengthening channels for inter- and cross-agency action in the Year of Open Science 2023. This session builds from our session at ESIP Winter 2022, and you can review the summary blog and videos: [3 takeaways for planning for the year of open science](https://www.openscapes.org/blog/2022/02/17/esip-winter-2022/). Those 3 takeaways were 1. Both top-down and grassroots efforts are necessary; 2. Dissolve silos by supporting early adopters; 3. Reuse and build from existing efforts to accelerate change. + +This year, presenters shared Openscapes stories and approaches following another year's work mentoring, coaching, and teaching colleagues as well as listening, advocating, and informing open science policy at different levels. Their stories come from the many phases of open science movement building they represent: a first welcome, inside government organizations, "forking" Openscapes internally, and contributing back to open science. What emerged was real lightbulb moments and real-time interplay between panelists who have not previously worked together as we recognized common themes and learned from each other. + +**Here are 3 overlapping takeaways from this session that all speakers embodied:** + +- Examine the cracks; invest in real relationships +- Break the hero mentality in science; invest in radical collaboration +- Build morale through learning; advocate for learning time + +We describe these more fully below. And as we advocate for open science in this Year of Open Science, let's connect open science with building morale and empowering individuals as well as with the higher quality of work created. + +## Examine the cracks; invest in real relationships + +"Examining the cracks'' was a concept brought up by Ileana Fenwick at the very beginning of our session, and it resonated with the other panelists throughout. Ileana designed and leads the [Pathways to Open Science](https://openscapes.github.io/pathways-to-open-science/) program, a remote event series of community calls and coworking for Black environmental & marine researchers to build community for the future of data intensive science. The first session, held on the evening of this ESIP panel, welcomed over [80 participants](https://twitter.com/curly_scientist/status/1618735555415805953)! Ileana shared that as she became empowered with open science, she thought exposure alone would be enough to attract other people to open science events. It wasn't; there are many reasons why folks do not feel included in science and open science. It caused her to examine the cracks - why wouldn't she participate in this program if she saw it in her inbox? This influenced the design of the Pathways program, with a deep investment in engagement. Ileana partnered Openscapes with [Black in Marine Science (BIMS)](https://www.blackinmarinescience.org/) and [Black Women in Ecology, Evolution, and Marine Science (BWEEMS)](https://www.bweems.org/), and met 1:1 with faculty at HBCUs, to learn about their students' needs and introduce her ideas for the program. Ileana shared these three pieces of advice, saying"inclusion is not an afterthought": + +1. Surface level efforts are easy to spot. Some questions to ask as you plan your program goals and efforts are: What is your motivation for this work? Is it genuine? Is the space you're inviting students to a safe space for marginalized students? (Do they know that?) It is clear when the broader impacts are written last. Inclusion is not an afterthought, your planning and process should demonstrate that. + +2. We exist. If your efforts are not yielding diverse participation, it's not because we aren't out there. It's because your efforts need to be tailored more specifically and intentionally. + +3. Center diverse voices. If there are no members of your team from your target audience or your team is not reflective, this is a moment to step back and ask yourself if YOU need to be the one doing this work. Amplify and empower diverse leadership, support them, compensate them, use privilege to push the project forward. + +
slide title Challenging my views on how to engage. Text says Exposure is not enough. Lead with listening ear, flexibility, open mind. To right of text is screenshot from website Pathways to Open Science with logos of Openscapes, BWEEMS and BIMS
Slide from Ileana Fenwick's talk on challenging her views on how to engage. Examining the cracks led Ileana to see that that exposure to information is not enough, and this influenced the design of the Pathways to Open Science program

+ +Later, Corey Clatterbuck described the momentum at the California Water Boards following an Openscapes Cohort she and Anna Holder co-led in 2022. They knew they wanted to ["fork" the Openscapes curriculum](https://www.openscapes.org/blog/2022/12/02/swrcb-2022/) to adapt it to the needs of their colleagues, so they examined the cracks using results of an internal 2019 [Data Literacy Survey](https://www.waterboards.ca.gov/resources/oima/cowi/docs/data_literacy_strategy_202011.pdf) with Water Boards employees. Corey and Anna were surprised by the results and, like Ileana, designed their programming to meet these needs. The results showed that respondents were motivated by interest in particular data tasks (image below, top half) and not in specific software or coding languages (image below, bottom half). In short, people were more interested in learning how to do things in their job (goal-oriented), than learning specific software (tool-oriented). From this, the [design of their Cohort](https://cawaterboarddatacenter.github.io/swrcb-openscapes/) emphasized the human over technical approach to open science to reach the majority of Water Boards colleagues who might be interested in open science. They increased the length of each call from 1.5 hours to 2 hours, and added a new module on documentation to replace the one on coding-strategies. + +
slide with 8 rows of bars filled to show percent of people not interested, somewhat, or very interested in a topic. Text says emphasize human over technical approach to open science
Results from the California Water Boards internal 2019 [Data Literacy Survey](https://www.waterboards.ca.gov/resources/oima/cowi/docs/data_literacy_strategy_202011.pdf) showed that respondents were motivated by interest in particular data tasks (top half) and not in specific software or coding languages (bottom half). Examining these cracks influenced how Clatterbuck and Holder forked and designed the Champions Program at the California Water Boards. (slide from Clatterbuck)

+ +Ileana's approach of "Examining the cracks" also caught Cassie Nickles' ear. She is a Mentor with NASA Openscapes, a community that supports researchers as they migrate analytical workflows to the Cloud. Cassie and her colleagues across other NASA Earthdata data centers are co-creating common tutorials, a review & reuse process, and as well as community of practice for teaching, mentoring, and facilitation for NASA Earthdata Cloud. As she was onboarding into her current role, Cassie became overwhelmed by the steps and processes required by end users to access and use NASA Earthdata. She figured if she was confused, others might be too and wanted to help simplify navigating the complicated systems. + +> The data analysis needed for the pressing problems we face should not be limited by the complexity of the underlying systems or a lack of computer engineering skills. - Luis Lopez, NSIDC + +Cassie and Openscapes Mentor-colleague Catalina Oaida Taglialatela made [Cheatsheets](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/cheatsheets.html#workflow-cheatsheet) with clickable icons that lead to tutorials for NASA Earthdata Cloud researchers! Having filled some gaps, Cassie is now recognizing more cracks moving forward: how do we get these resources in front of the folks they're intended for? + +## Break the hero mentality in science; invest in radical collaboration + +Erin Robinson opened the session saying that solutions to large-scale environmental and social challenges require radical collaboration, but have been limited by the hero model, where science is competitive, siloed, and rewards an individual hero, often a white man. She said we together are changing this story. We do this by welcoming people where they are and through creative approaches that blend methods from open source software development, mentoring, coaching, and art. + +Reflecting as a Mentor at NOAA Fisheries supporting [6 Openscapes cohorts (240 staff) in 2022](https://www.openscapes.org/tags/noaa-fisheries/), Josh London said that "collaboration is something we're never taught." Individuality is instilled in grad school and we never have a chance to change this through "business as usual." We need the space and place to relearn. Don't spend your time trying to solve things yourself. Learn together first! It's ok to ask for help early. Adyan Rios is also a NOAA Fisheries Mentor, who is helping develop a learning culture at the Southeast Fisheries Science Center through leading "Surf Sessions" where colleagues cowork remotely and share what they know. Adyan echoed this need for radical collaboration, saying that it's radical because it means being vulnerable. It means people sharing things they haven't figured out yet, while they're still messy. Or share a cool thing you learned, like a keyboard shortcut. If you just learned it, chances are someone in your community didn't know it yet either. Sharing makes their day a little better and make their work more efficient. This harkens back to Cassie creating cheatsheets to share what she figured out to help others. + +::: {.blockquote-blue} +> *"Team collaboration is not something that we've all learned from the beginning. In grad school it's instilled in us this idea that anything meaningful that we do in science is supposed to be done by us as individuals."* **-- Josh London** +::: + +
+ +::: {.blockquote-blue} +> *"This mindset and practice \[of learning **together** first\] has been revolutionary for us at NOAA."* **--- Adyan Rios** + +Breaking the hero model is something that Luis Lopez has been thinking about too. He shared that as a software engineer, something he has learned through the NASA Openscapes Mentors community is to empathize with researchers and not expect that software alone is the hero. The key is to really understand learners and researchers, get to know them and see how they work and where they are stuck. Working with his NASA Mentor colleagues, Luis identified that there were many steps involved before researchers could be hands-on with data in the cloud, and was intent on reducing that "time to science". Luis has led development of the [earthaccess](https://nsidc.github.io/earthaccess/) python library to search, download or stream NASA Earth science data, enabling a researcher to get to the science with four lines of Python code, instead of running command line code that saves a hidden .netrc file to the user's computer and requires reactivation after one hour. Collaborating with other Mentors has been critical here -- again breaking the hero model -- as Cassie's Cheatsheets helped visualize all the steps previously involved that were cumbersome for scientists. + +## Build morale through learning; advocate for learning time + +"How to make learning part of your job" and "When do you carve out time to learn" are questions that come up frequently, as they did here. Pointing to examples of how other individuals and agencies do this has been super powerful: it is negotiated with supervisors at NOAA to be included in Individual Work Plans, and the NASA Earthdata Mentors are a fantastic example of the power of high-level support for learning. The NASA Mentors shared how the Mentor community has been able to learn and collaborate together over the past two years. This was because their time was approved at a high level, by NASA policy and by their managers. This approved time has given space for them to first share and listen, find the common, and then build together, reuse, and amplify. + +::: {.blockquote-blue} +> *"Learning builds morale: it gives people an opportunity to change workflows, learn new skills, improve the culture, and take ownershipof that"* **--- Josh London** +::: + +We're seeing some of this trickle out beyond the Openscapes curriculum. Josh London talked about how we often focus on the shiny new tools (python, R, Quarto, GitHub) but it's the culture shifts that are the real meaningful change. He said it's so encouraging to see in a calendar invite for a meeting that someone's already initiated an agenda in a shared document and that there are multiple people contributing notes there at the same time. Additionally, learning to use tools like GitHub for shared "todo" lists and better project management for teams has increased morale for NOAA Fisheries teams. Teams are also using these tools for more inclusive communication to onboard other team members to their projects, passing forward this increased morale that comes from learning together. + +Loneliness during the Pandemic was mentioned several times - Adyan, Cassie, Corey, and Ileana all onboarded to their current roles during the pandemic. They noted that shared practices like coworking along with collaborating asynchronously with GitHub and Google Docs helped them feel less alone and build real connections with their colleagues. And, it helped them learn skills they needed to do their jobs. They saw that these skills would help other colleagues as well, so they have dedicated time to teaching and supporting colleagues, and advocating for learning time from supervisors. + +::: {.blockquote-blue} +> *"Not everyone needs the same learning curve; we can send down a rope and help others get to where they need to be sooner."* **--- Adyan Rios** + +
slide with 3 images of round fuzzy monsters with subheading psychological safety & culture, uniting data strategies, documentation thru Pathway & GitHub
The California Water Boards Openscapes Cohort emphasized psychological safety and culture, uniting data strategies, and documentation to create a welcoming environment, build community, and reduce loneliness in learning new skills. (slide from Clatterbuck; artwork from [GitHub Illustrated Series](https://www.openscapes.org/blog/2022/05/27/github-illustrated-series/))

+ +## Onward + +One of the most inspiring aspects of this panel was the mutual recognition of common themes, challenges, and potential approaches among speakers from government agencies and academia who had not previously worked together. ESIP is a great place to make those connections across agencies and convening these sessions is an effective way to move us all forward together. Open science isn't the ultimate goal. The vision is what's possible because of open science: climate solutions, social justice, and democracy. As we advocate in this Year of Open Science, let's connect it not just to the higher efficiency and quality of work that is produced, but especially with higher team morale and empowerment of individuals to change their institutional culture. + +
+ +## Speakers Bios + +**Corey Clatterbuck** is an environmental scientist at the California Water Boards in the Office of Information Management and Analysis and the Surface Water Ambient Monitoring Program. A trained ecologist, Corey's current work uses landscape and water data to identify healthy watersheds in California and determine the policies & practices to keep them that way. Having supported Openscapes trainings throughout 2021, "forked" Openscapes with her colleague Anna Holder and taught her colleagues internally within the Water Boards in 2022. + +**Ileana Fenwick** is a third-year marine fisheries PhD student at UNC and part of the Openscapes core team. She launched the Pathways to Open Science program to welcome more Black marine scientists to open science and to build skills and community! + +**Josh London** is a Wildlife Biologist at NOAA Fisheries, in the Marine Mammal Laboratory Alaska Fisheries Science Center (AFSC). Josh's research focuses on ecology and conservation of Arctic seals. Josh is a long-time open source developer and champion; mentor with Openscapes, co-leading Winter and 4 Fall Cohorts with NOAA Fisheries. + +**Luis Lopez** is a Research Software Engineer at the National Snow and Ice Data Center (NSIDC) in Boulder, Colorado. He has helped develop tools and services to facilitate data access and discoverability across different NASA Earth missions. He is part of the first cohort of NASA Openscapes Mentors, and lead developer for the earthaccess python library and corn jupyterhub environment. + +**Cassie Nickles** is an Applied Science Systems Engineer working for NASA's PO.DAAC, the Physical Oceanography Data Active Archive Center. She became a NASA-Openscapes mentor in 2022 and is consistently looking for better ways to make the complicated simple for data end users through mechanisms like data tutorials, workflow diagrams & cheatsheets! + +**Adyan Rios** is a research ecologist at NOAA Fisheries, in the Caribbean Fisheries Branch of the Sustainable Fisheries Division at the Southeast Fisheries Science Center (SEFSC). Adyan participated in Openscapes in Fall 2021 and became an organizer and mentor in 2022; co-leading Summer and 4 Fall Cohorts with NOAA Fisheries. + +
headshots of 6 speakers around Openscapes hex logo
ESIP 2023 panelists

diff --git a/blog/2023-05-17-mentor-coach/index.qmd b/blog/2023-05-17-mentor-coach/index.qmd new file mode 100644 index 0000000..da493f3 --- /dev/null +++ b/blog/2023-05-17-mentor-coach/index.qmd @@ -0,0 +1,155 @@ +--- +title: "How coaching skills have made us better open data science mentors" +author: Tara Robertson, Stefanie Butland, Julie Lowndes +date: 2023-05-17 +slug: mentor-coach +categories: + - blog + # - community + # - impact + # - impact + - noaa-fisheries + - nasa-framework +image: mrlowndes-bee-square.jpg +aliases: + - https://www.openscapes.org/blog/2023/05/17/mentor-coach/ +--- + +*February through May 2023, Openscapes Mentors from across governments and academia came together for some hands-on learning and practice that had a profound effect on the way we will teach and lead going forward. We learned coaching skills that can help us as professionals -- skills like listening rather than solutioneering, asking open-ended questions that empower people to find their own agency and meet their needs. This is a program designed and facilitated by Tara Robertson, a Certified Professional Co-Active and International Coaching Federation Associate Certified Coach. Openscapes has been developing a professional collaborative relationship with Tara since [2021](https://www.openscapes.org/blog/2021/05/27/tara-robertson/). It was also impactful because it brought together Openscapes Mentors from across government groups (NASA Earthdata, NOAA Fisheries, EPA, California Waterboard) so they can learn together and support each other in the 2023 Year of Open Science.* + +------------------------------------------------------------------------ + +::: {.blockquote-blue} +> *"This quietly subversive training showed me how to be a better mentor and coach by using new tools or old tools in better ways to hear what people are saying and help them find solutions within themselves...and by helping me get out of my own way in the process."* **---Amanda Bradford, Research Ecologist, PIFSC, NOAA Fisheries** +::: + + + +::: {.blockquote-blue} +> *"This was amazing! I love the courses that make you feel uncomfortable and safe at the same time because that's where change happens. It was such a positive growing experience. I was also able to connect with so many amazing people and that in itself was worth it. Learning from the others has expanded my views on coaching/mentoring/ etc so much!"* **---Alex Davis, The Diversity Project, UCLA** +::: + + + +::: {.blockquote-blue} +> *"This program provides a solid foundation for learning what goes into good mentoring and coaching conducted by an instructor who leads by example. When members of the group place their trust in the instructor and program, it becomes free group therapy with other scientists, where vulnerability is welcomed and mental breakthroughs are made. This bred an extremely comfortable and supportive cohort that I would like to keep in touch with!"* **--- Alexis Hunzinger, NASA GES DISC Data Center** +::: + + + +::: {.blockquote-blue} +> *"This has been unlike any leadership training offered to us through typical NOAA/government channels. I think it has been a real eye opener to the importance of getting training outside of our typical (and comfortable) space"* **---Josh London, Wildlife Biologist at NOAA Alaska Fisheries Science Center** +::: + +This program had a profound impact on the three of us too, as participants as well as program designers and facilitators. We are still trying to define what it was about the program, the process, and everyone involved that led to such a profound sense of trust among us so that we were able to be vulnerable and do courageous things. While these Openscapes Mentors have been collaborating as colleagues together within their organizations (e.g. as NASA Earthdata Mentors or Pathways to Open Science Mentors), this was the first time most of these folks met as a cross-Openscapes mentor community (besides brief encounters at virtual ESIP conference sessions in [2023](https://www.openscapes.org/blog/2023/03/16/esip-winter-2023/) and [2022](https://www.openscapes.org/blog/2022/02/17/esip-winter-2022/)). + +This truly was a cohort of [early adopter](https://en.wikipedia.org/wiki/Early_adopter) "[bright spots](https://www.shrm.org/resourcesandtools/hr-topics/benefits/pages/danheath.aspx)" - folks who wanted to be better mentors and grow as leaders, even if we didn't know what that meant yet. Not only were folks working across organizations with other mentors they didn't know, they were expert scientists being asked to step way outside their comfort zone and be novices. In this setting, they were enthusiastic and fearless learners. There was a really high rate of participation and very few people dropped off over the six sessions. We saw individual growth and impacts feeding back and forth: something introduced gently here that we can practice together makes us comfortable, and now we can take this to other spaces and introduce it gently to others too. + +## Designing a courageous space + +We connected these bright spots by creating space and place. We met every 2 weeks for 6 remote sessions for a total of 12 hours. Twenty-eight people participated as part of their paid jobs, supported by their organizations rather than doing it "off the side of their desk". We used a practice already familiar to all Openscapes Mentors: collaborative note-taking Agendas all in one Google doc where we had a shared responsibility to take notes, clarify explanations, add links to resources, and add shared joy and encouragement via emojis, +1's and comments. We co-designed and helped each other when facilitating, which was instrumental when Tara got sick and lost her voice and Julie stepped in to facilitate the session based on Tara's prep ahead of time. + +The 6 sessions focused on specific topics: powerful questions, 3 levels of listening, identifying values, leader within, saboteurs and then taking these coaching skills into mentoring, supervising and sponsoring. For each of the topics, Tara briefly described the topic or skill, did a live demo with one of the participants, and the group described what they observed. Then, Tara gave us a topic to coach on, or some guardrails, and participants went into breakout rooms in pairs to practice coaching and being coached. + +There was a lot of hands-on doing. Most of the participants said this was the best part of the course and also the most uncomfortable bit. In these pairs, the coaches were learning how to ask questions that help people open up...and this also challenged the coachees, who really dived into their needs and blockers for how to better support open science. It was also a testament to Tara's own willingness to be vulnerable, demonstrating as a coachee and giving us her own real-life examples. + +**Thoughts from Tara:** Something really special happened with this cohort. There were three main ingredients to this magic: + +1. The participants were enthusiastic and fearless learners who were willing to step outside their comfort zones and learn new things together. This group of expert scientists from different organizations were willing to be beginner learners with each other. This group embodied courage. + +2. We were intentional in the design to maximize psychological safety. At the start of Week 1 we designed an alliance together on how we wanted to be and learn together. I ordered the topics to start with the lower stakes topics and we were intentional about mixing people up across organizations. In the week 5 coaching demo one of the coaching demos the coachee chose to be quite vulnerable and bring a real challenge. This was a turning point where we saw that we can use these skills to surface challenging topics and have honest conversations about barriers in science. + +3. Openscapes investment in coaching skills was a unique way to collectively learn some new skills AND build connections between open science leaders across different organizations and Openscapes initiatives: Mentors at NOAA Fisheries, NASA and from the growing Mentor community from CalEPA and Pathways to Open Science. + +## Building skills for listening not solutioneering + +So called "soft skills" are anything but. This was hard. Learning to ask open ended questions was hard, and what was harder still was listening to be able to ask the next powerful question. How we listen differs whether we are in mentor-mode or coach-mode: + +- When we listen as mentors, we're trying to understand where our expertise can help someone solve a problem. + +- When we're listening as coaches, we're trying to help the other person define their problem, and tap into their own wisdom to find their solution, likely outside of our domain expertise. + +What do we mean by a "powerful question"? It's a short question that can open up a really powerful conversation. It's open-ended, usually starts with the word "What", ideally is 5 words or less, and can't be answered by yes or no. What feels like success to you? What's in the way? What, if anything, is going unsaid? Which of your core values is being messed with? + +Distinguishing when we're listening as mentors and when we're listening as coaches is important, because it determines how we will respond as early adopters who may not know the answer to the question -- think cloud computing, how to store increasingly large data, how to give hard feedback to a superior, and how to ask for help in a culture built on competition and the myth of the lone genius. Developing these coach listening skills, including what questions to ask, has helped us help others better (and notice and avoid solutioneering!). It's also helped us feel less overwhelmed as we learn that it's okay to not always have the answer! + +::: {.blockquote-blue} +> *"We need these coaching skills to do powerful work and inspire meaningful connections; this program is a safe space to learn, to challenge our preconceptions, to practice, and to grow. I found it to be so invaluable because it's a chance to acknowledge and learn about skills that are often unspoken in their ability to improve mentorship and collaboration."* **--- Ileana Fenwick** +::: + +**Thoughts from Stef.** During the first two sessions, in the breakout rooms to practice coaching in pairs, I defaulted to talking more about the process and how we felt about it because actually practicing coaching, even with specific prompts, felt really really hard. But somehow over time, it got easier. Now I find myself trying out "powerful questions" with my peers in one-to-one conversations or coworking, rather than my usual behavior of suggesting solutions to people's challenges. And people seem to respond! + +But what really made this work? + +1. People were primed for this based on their prior experiences as Openscapes Mentors. They came expecting a familiar set of norms and psychological safety that allowed them first to get comfortable and then to take risks. With this foundation, together we were able to build trust early on. This is key to developing healthy teams and communities. We all knew it would be worth it even if we didn't know what we were getting into. + +2. Often, to offer a program like this, an organization might contract an external professional to deliver it. We took it a step further. Openscapes contracted Tara, with whom we've collaborated before, and who is highly respected for her tell-it-like-it-is approach to diversity and inclusion. The core team of Tara, Julie, and me, worked to onboard ourselves and each other to this process using an open facilitation approach rather than working behind a curtain. We met frequently and defined our roles together, agreeing to role-model transparency and vulnerability and to name these things. Julie and I took turns managing breakout rooms each week so we could also take turns practicing coaching. + +3. The mentors were not our students; they're our peers and collaborators. Once people opted in, there was no judgment or "sorries" if someone was late or had to miss a session because of life's responsibilities. + +4. Practice! We moved away from analyzing and into the doing. It was uncomfortable and people still came and showed up as leaders + +## Building mindsets as leaders + +Developing coaching skills also helped us build new mindsets. There was a braveness that grew throughout our time together that showed up in different ways. One person shared that they are more brave with whom/when/where they might interact as a coach and mentor. They had recently found themselves in-person with someone senior and rather than making nervous small talk found themselves asking powerful questions and really connecting with the senior person more, and felt that they both learned something in this short moment together. + +We grew these new mindsets with our peers and built trust and community in a really profound way. The topics people surfaced and shared were so vulnerable -- navigating racism, personal and structural saboteurs, and the integrations of personal and work life. We had never experienced such vulnerability in our work, and knew we could not shy away from difficult topics but needed to dig further and out based on the courage and trust in the room. + +**Thoughts from Julie.** I've talked a lot about the impact of open data science on my career and my life. My science thinking was transformed when I realized I could do an analysis just as easily 100 times with a for-loop as doing it twice, and that I could publish my work via RMarkdown and GitHub to the open web and share via a single URL instead of sending followup emails with different versions of the same PDF or Word doc. Now, I am realizing that the impact of coaching on my thinking might be at this same level. I am still processing it all, but I am feeling more confident in places I have been struggling to show up as my full self. It is somehow helping me bring confidence into a room with me, knowing that I have new skills and a strengthened sense of shared braveness together with this cohort. When we first planned this cohort with Tara, this was new terrain. We asked ourselves, "**If this is wildly successful what are the outcomes? What's possible?"** Here are some ideas we laid out, and I think the Mentors are meeting each one: + +- People model inclusive behavior in work, life, Openscapes, and our home organizations, to influence culture change + +- People are braver, share stories and energy at work and other places + +- Someone speaks up in a meeting + +- Everyone learns something new! (me too!) + +- We listen better/differently + +- We scale participation - use coaching strategies to empower others sooner, so that existing mentors don't burn out + +- More people feel safe in Openscapes and open science, and in turn, inspire others to be braver + +## Fearless in service of something + +Working with Tara and bringing Openscapes Mentors across different organizations and initiatives is part of movement building with our [Flywheel](https://doi.org/10.31223/X5CQ02). This is a [concept developed by Jim Collins](https://www.jimcollins.com/concepts/the-flywheel.html) that we adapted for Openscapes with Erin Robinson (Erin first brought coaching into Openscapes strategy and practices too!). Transformations occur from relentlessly pushing a giant, heavy flywheel that builds momentum over time. Using this model, we welcomed folks to opt in, created space and place so we could invest in learning and trust, and then worked openly together as we practiced as coaches and coachees, leveraging common experiences and skills. This turn of the flywheel is part of building the momentum of kinder and open science, building from past, parallel, and collaborative work from many places. Mentors are already inspiring others with their leadership, pushing the next turn of the flywheel. + +Bravery has come up throughout this cohort and this post, and this braveness and fearlessness is in service of something. We're wanting to connect our daily work with the global moment at hand, so we can better address issues stemming from climate change and social justice. As one participant said, "I don't want to be scared to ask questions anymore". We want to ask questions with no ego, in rooms with our science peers. We want to break the silences, opening up the rooms so that the next person who has a question but was not sure they would voice it is empowered to speak up. + +In our final session we paired up to coach each other through developing our own statement of purpose in open science. + +::: {.blockquote-blue} +> *"I am an elevator to a better tomorrow"* +::: + + + +::: {.blockquote-blue} +> *"I am a pollinator who connects people to each other and ideas so everyone flourishes"* +::: + + + +::: {.blockquote-blue} +> *"I am the momentum that answers to my peers"* +::: + + + +**We are all the momentum that answers to our peers**. To make meaningful change, we must work across all levels and elevate those who will be the change along with us. + +
+ +```{r} +#| echo: false +#| fig-align: center +#| out-width: "85%" +#| fig-alt: "A bee standing on a pink flower with multiple florets, against a black background" +knitr::include_graphics("mrlowndes-bee.jpg") +``` + +::: {.caption-text .center-text} +Photo by Elliot Lowndes +::: + diff --git a/blog/2023-05-17-mentor-coach/mrlowndes-bee-square.jpg b/blog/2023-05-17-mentor-coach/mrlowndes-bee-square.jpg new file mode 100644 index 0000000..95b0430 Binary files /dev/null and b/blog/2023-05-17-mentor-coach/mrlowndes-bee-square.jpg differ diff --git a/blog/2023-05-17-mentor-coach/mrlowndes-bee.jpg b/blog/2023-05-17-mentor-coach/mrlowndes-bee.jpg new file mode 100644 index 0000000..37eb935 Binary files /dev/null and b/blog/2023-05-17-mentor-coach/mrlowndes-bee.jpg differ diff --git a/blog/2023-05-31-zenodo-community/index.qmd b/blog/2023-05-31-zenodo-community/index.qmd new file mode 100644 index 0000000..2c8656c --- /dev/null +++ b/blog/2023-05-31-zenodo-community/index.qmd @@ -0,0 +1,71 @@ +--- +title: "New Zenodo Openscapes Community helps you Find and Cite Openscapes things" +author: Stefanie Butland +date: 2023-05-31 +slug: zenodo-community +categories: + - blog + - how-we-work + - noaa-fisheries + - nasa-framework +# tags: +# - impact +# - noaa-fisheries +# - nasa-framework +image: zenodo-openscapes-community-square.png +--- + +# We created a Zenodo Openscapes Community! + +We want to share our work in a way that people can find it, use it, improve it, and cite it, or get credit for their contributions. For people who have participated in our programs like Openscapes Champions[^1], or Pathways to Open Science[^2], we want a robust way to add these to their CV as professional development. For contributors to our open educational resources, like the NASA Earthdata Cloud Cookbook[^3], we want them to get credit and visibility. For people who want to reuse or remix a slide from a presentation, we want them to feel great about it by providing an easy way to cite the presentation. To enable all of this, we created a [Zenodo Openscapes Community](https://zenodo.org/communities/openscapes/) as a semantically meaningful group of selected research products. NASA's Transform to Open Science ([TOPS](https://zenodo.org/communities/tops/)) and the Center for Scientific Collaboration and Community Engagement's ([CSCCE](https://zenodo.org/communities/cscce/)) Zenodo Communities were inspirations for ours. + +[^1]: Julia Stewart Lowndes & Erin Robinson. (2022). Openscapes Champions Lesson Series (2022.12). Zenodo. + +[^2]: Ileana Fenwick & Julia Stewart Lowndes. (2023). Pathways to Open Science (2023.02). Zenodo. + +[^3]: Andy Barrett, Chris Battisto, Brandon Bottomley, Aaron Friesz, Alexis Hunzinger, Mahsa Jami, Alex Lewandowski, Bri Lind, Luis López, Jack McNelis, Cassie Nickles, Catalina Oaida Taglialatela, Celia Ou, Brianna Pagán, Sargent Shriver, Amy Steiker, Michele Thornton, Makhan Virdi, Jessica Nicole Welch, Erin Robinson, Julia Stewart Lowndes. (2023). NASA EarthData Cloud Cookbook (2023.03). Zenodo. + +[Zenodo](https://zenodo.org/) is a general-purpose open repository that allows researchers to deposit research related digital artefacts like research papers, data sets, research software, reports, lesson materials, and presentations. For each submission, a persistent digital object identifier ([DOI](https://en.wikipedia.org/wiki/Digital_object_identifier)) is minted, which makes the stored items easily citable (adapted from [Wikipedia](https://en.wikipedia.org/wiki/Zenodo)). Zenodo allows for versioning and we can preserve GitHub repositories in Zenodo too (GitHub itself is *not* a repository!). + +
+ +```{r} +#| eval: true +#| echo: false +#| out-width: "70%" +#| fig-align: "center" +#| fig-alt: "screenshot of the Openscapes Community page on Zenodo" +knitr::include_graphics("zenodo-openscapes-community.png") +``` + +## Our use cases + +The [Openscapes Approach Guide](https://openscapes.github.io/approach-guide/approach/tooling.html#sec-zenodo) gives instructions for our use cases, with pointers for things that might not be obvious to a first-time user. + +- What do we curate in our Zenodo Openscapes Community? +- How to add your existing Zenodo record to the Zenodo Openscapes Community +- How to publish a new record in Zenodo to get a DOI +- How to get a DOI for materials on GitHub +- How to cite Openscapes publications + +## Bonus things I learned + +Of course I want credit for my contributions! When we add an author's ORCID ID to a Zenodo record, their ORCID profile is automatically updated. I learned of this bonus when I uploaded a post on which I'm a co-author, and then received email notification that this record had been added to my ORCID profile. Why is this so cool? An [ORCID ID](https://support.orcid.org/hc/en-us/articles/360006897334-What-is-my-ORCID-iD-and-how-should-I-use-it-) is a unique, open digital identifier that distinguishes you from every other researcher with the same or a similar name to you. My ORCID profile is a bit like a CV. I use it to collect my research publications (not limited to peer reviewed papers) along with things like education and service on boards. Having it automatically updated is great. + +
+ +```{r} +#| eval: true +#| echo: false +#| out-width: "50%" +#| fig-align: "center" +#| fig-alt: "screenshot showing zenodo record called 3 Approaches for the Year of Open Science with green ID symbol linked with an arrow to a second screenshot showing the ORCID record for that publication" +knitr::include_graphics("zenodo-orcid.png") +``` +::: {.caption-text .center-text} +*Associating a Zenodo record with an author's ORCID ID results in that record being automatically listed in their ORCID profile.* +::: + +
+ +Want to create a Zenodo Community? Play in the [Sandbox](https://sandbox.zenodo.org/) first, where anyone can create and refine a draft Community before publishing it in Zenodo. Creating a Sandbox version forced me to recognize decisions to make before creating the real thing, like: needing to create it from an account that looks professional like "curator", rather than my personal email username; or deciding what types of research products to include or exclude. This webinar section "[How to create a community](https://youtu.be/yj2r8RayIX8?t=734)" screenshares a walk-through that makes things crystal clear. diff --git a/blog/2023-05-31-zenodo-community/zenodo-openscapes-community-square.png b/blog/2023-05-31-zenodo-community/zenodo-openscapes-community-square.png new file mode 100644 index 0000000..8caa67c Binary files /dev/null and b/blog/2023-05-31-zenodo-community/zenodo-openscapes-community-square.png differ diff --git a/blog/2023-05-31-zenodo-community/zenodo-openscapes-community.png b/blog/2023-05-31-zenodo-community/zenodo-openscapes-community.png new file mode 100644 index 0000000..9254384 Binary files /dev/null and b/blog/2023-05-31-zenodo-community/zenodo-openscapes-community.png differ diff --git a/blog/2023-05-31-zenodo-community/zenodo-orcid.png b/blog/2023-05-31-zenodo-community/zenodo-orcid.png new file mode 100644 index 0000000..11892dc Binary files /dev/null and b/blog/2023-05-31-zenodo-community/zenodo-orcid.png differ diff --git a/blog/2023-08-01-nasa-champions/earthdata-clinic-zoom.png b/blog/2023-08-01-nasa-champions/earthdata-clinic-zoom.png new file mode 100644 index 0000000..e42c181 Binary files /dev/null and b/blog/2023-08-01-nasa-champions/earthdata-clinic-zoom.png differ diff --git a/blog/2023-08-01-nasa-champions/index.qmd b/blog/2023-08-01-nasa-champions/index.qmd new file mode 100644 index 0000000..0045d58 --- /dev/null +++ b/blog/2023-08-01-nasa-champions/index.qmd @@ -0,0 +1,149 @@ +--- +title: "Exciting Progress for Research Teams using NASA Earthdata in the Cloud: 2023 NASA Openscapes Champions Wrap-up" +author: Erin Robinson, Julie Lowndes, Amy Steiker, Alexis Hunzinger, Luis Lopez, Andy Barrett, Cassie Nickles, and NASA DAAC Mentors +date: 2023-08-01 +slug: 2023-08-01-nasa-champions +categories: + - blog + - champions + - nasa-framework +image: zoomie-2023-nasa-champions-square.png +--- + +*From April-June 2023, the NASA Openscapes project team co-led the second Champions Cohort with NASA Mentors who span seven Distributed Active Archive Centers (DAACs). The Cohort included seven research teams from academia and government that were curious about working with NASA Earthdata in the Cloud. Cloud migration takes time, so in the ten weeks we worked together, the focus was on planning the transition, identifying resources, and initial experiments using the Cloud through our 2i2c JupyterHub. All of this work is underpinned by Openscapes and NASA's commitment to Open science practices. This cohort is funded by NASA and is part of our [NASA Openscapes Framework project](https://nasa-openscapes.github.io/).* + +*Quick links:* + +- *Cohort webpage: *  + +------------------------------------------------------------------------ + +## **NASA Champions Cohort Overview** + +The [NASA Openscapes](https://nasa-openscapes.github.io/) Project is a multi-year project to develop a cohesive approach to building Cloud migration capacity across NASA Earthdata from NASA's Earth Observing System Data and Information System (EOSDIS) Distributed Active Archive Centers (DAACs) and the research teams that the DAACs support. We do this through supporting a community of NASA DAAC mentors, who are primarily dedicated to user support. This community has learned together how to use NASA Earthdata on the Cloud. They have translated that experience into a series of Hackathons, workshops, self-paced tutorial material in the NASA Earthdata Cloud Cookbook, and through an intensive, 10-week program: [NASA Openscapes Champions](https://nasa-openscapes.github.io/2023-nasa-champions/). + +Cloud migration can often have a steep learning curve and feel overwhelming. The NASA Openscapes Champions Cohort brings together **research teams** that are interested in migrating their existing NASA Earthdata workflows to the Cloud with **NASA DAAC Mentors** who are extremely knowledgeable about the data they serve and the initial pathways to using that data on the Cloud. This Cohort provides a common, welcoming place for teams to learn together, ask questions about using the Cloud, plan their transition, and do initial experimentation using the NASA Openscapes 2i2c JupyterHub. Because this is a more intensive experience, the teams build collaborative partnerships with DAAC mentors, and the mentors can more quickly identify and work on solving issues that will make Cloud migration easier for many more users. We led the [first NASA Openscapes Champions Cohort in 2022](https://openscapes.org/blog/2022-05-12-nasa-2022-champions/). + +The second NASA Openscapes Champions Cohort ran formally in April-June 2023 with seven research teams interested in a wide variety of NASA Earthdata and various stages of Cloud technology familiarity. You can learn more about their research below. + +```{r} +#| eval: true +#| echo: false +#| fig-align: "center" +#| out-width: "75%" +#| fig-alt: "screenshot of 23 zoom participant faces smiling and waving" +knitr::include_graphics("zoomie-2023-nasa-champions.png") +``` + +::: {.caption-text .center-text} +Zoomie (♥️) of some researchers in the 2023 NASA Openscapes Champions Cohort +::: + +Together as a Champions Cohort, these teams discussed what worked and didn't work as they migrated workflows to the Cloud, focusing on collaboration and open science. We met as a cohort five times over two months on alternating Wednesdays. Each cohort call included a welcome and code of conduct reminder, two teaching sessions with time for reflection in small groups or silent journaling and group discussion, before closing with suggestions for future team meeting topics ("Seaside Chats"), Efficiency Tips, and Inclusion Tips. All topics and the slides presented are shared on the [2023 Cohort page](https://nasa-openscapes.github.io/2023-nasa-champions/). Additional coworking sessions were scheduled on alternate weeks, where researchers could work quietly, screenshare to ask questions or meet with their team to discuss further. In addition, the teams have access to Openscapes' 2i2c Jupyter Hub, which will continue for the next year.  + +Thanks to the NASA Openscapes Mentors for supporting the Champions and for their contributions to the curriculum! In particular, the NASA Openscapes Champions Curriculum had significant additions:  + +1. Andy Barrett from the National Snow and Ice Data Center (NSIDC) created a version of **Data Strategies for Future Us** **that is applicable to gridded, remotely sensed data**. Slides are here:   + +2. Amy Steiker and Luis Lopez from NSIDC and Alexis Hunzinger from Goddard Earth Sciences Data and Information Services Center (GES DISC) refactored the **Coding Strategies for Future Us to be tailored to NASA Earthdata search**, **the earthaccess python library**, **and experiences** from teams at Goddard DAAC (GES DISC) that have learned to use the Cloud.  + +3. Amy Steiker from NSIDC designed and led the **Earthdata Cloud Clinic**! This was a hands-on 1-hour clinic that allowed teams to get familiar with the 2i2c JupyterHub, and practice finding and accessing NASA Earthdata via direct access through two methods from a Jupyter Notebook: earthaccess and Harmony-py services. This material can be found in the cookbook here -\>  \ + +```{r} +#| eval: true +#| echo: false +#| fig-align: "center" +#| out-width: "60%" +#| fig-alt: "screenshot of zoom window with left side showing screenshare of 2i2c JupyterHub and right side faces of participants, some of whom have a green checkmark with 'yes'" +knitr::include_graphics("earthdata-clinic-zoom.png") +``` + +::: {.caption-text .center-text} +Screenshot of Amy Steiker teaching the Earthdata Clinic. We used Zoom's "green check mark" feature during pauses for people to indicate that they had completed a step. +::: + +4. Cassie Nickles screenshared and introduced the Earthdata Cookbook and NASA Earthdata Cloud cheatsheets in the final cohort call to help teams understand available resources moving forward.  + +## **Shared Lessons Learned** + +The Openscapes Champions provides a space for teams to come together to learn from each other and across teams. It is a way to collaborate and distribute leadership across teams and reduce the burden on the team leader feeling the need to learn everything first to teach it to the rest of the team. Below are just a few things we've learned and are carrying forward.  + +**Understanding 'When to Cloud.'** The Cloud makes some things easier and some things harder. Over the series of five synchronous cohort calls, we considered when Cloud is effective and when the download model may still be more appropriate. Andy Barrett highlighted some considerations in Call 2 on Data Strategies for Future Us. + +```{r} +#| eval: true +#| echo: false +#| fig-align: "center" +#| out-width: "60%" +#| fig-alt: "bullet text says: When to cloud?; What is the data volume?; How long will it take to download?; Can you store all that data (cost and space)?; Do you have the computing power for processing?; Does your team need a common computing environment?; Do you need to share data at each step or just an end product?" +knitr::include_graphics("when-to-cloud-barrett.png") +``` + +::: {.caption-text .center-text} +Figure from [Andy's slides](https://nsidc.github.io/data_strategies_for_future_us/data_strategies_slides#/when-to-cloud). +::: + +[When to Cloud](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/when-to-cloud.html) is a recent addition as an early chapter of the NASA Earthdata Cloud Cookbook and will evolve as we continue to learn when to Cloud and when not to. Kudos to Alexis Hunzinger from GES DISC for leading this effort. + +**Undifferentiated Heavy Lifting - \`earthaccess\` for the win!** [Jeff Bezos coined this term in his 2006 keynote](https://aws.amazon.com/blogs/aws/we_build_muck_s/) describing AWS doing undifferentiated heavy lifting so that everyone doesn't have to do the mucky parts. The NASA DAAC Mentors' work with research teams allows them to identify places of friction that, if fixed once, would benefit many users. One of the places the NASA DAAC mentors, led by Luis Lopez, have done this is with [earthaccess](https://github.com/nsidc/earthaccess), a Python library to search, download or stream NASA Earth science data with just a few lines of code. Luis described earthaccess in a [video demo](https://github.com/nsidc/earthaccess) we shared in Call 3, and the Cohort used earthaccess in Call 4. We heard over and over how useful earthaccess was to the Cohort teams.  + +```{r} +#| eval: true +#| echo: false +#| fig-align: "center" +#| out-width: "40%" +#| fig-alt: "screenshot of zoom chat says 'Earthaccess is a really nice improvement over the way we were doing S3 access'" +knitr::include_graphics("zoom-chat-earthaccess.png") +``` + +::: {.caption-text .center-text} +Zoom chat from a researcher in the 2023 Champions Cohort, who also participated in the 2021 Cloud Hackathon which preceded \`earthaccess\` development. +::: + +However, **several challenges working with NASA Earthdata in the Cloud are still unresolved.** These include:  + +- We have made the first steps to migrating to the Cloud possible, but workflows are generally designed for serial processing. Refactoring code to take advantage of parallelization and scaling available in the Cloud isn't a skillset we have on our project team.  + +- Our Cloud tutorials have been Python-centric, and Champions research teams use a variety of languages, including R, Matlab, IDL, and Fortran. File formats - particularly HDF4 and HDFEOS - that some teams used aren't as straightforward to use in the Cloud.  + +- The 2i2c JuypterHub is a great resource for initial experimentation, but teams do not necessarily want to depend on the Hub. The hurdle to go from accessing the Cloud through a GitHub login in a browser to interacting with the AWS console interface is giant. It also requires a credit card, which leads to the most consistent challenge - it is still difficult to understand how much the Cloud will cost.  + +## **Gaining Cloud Momentum** + +Part of the Openscapes Champions approach is that teams decide what to work on. The teams devoted at least 8 hours a month to focus on their workflows, learning, and collaborating within and across their teams. During this time, they thought through and discussed their current NASA Earthdata workflows and planned and experimented with transitioning their workflows to the Cloud using Openscapes' 2i2c-hosted Jupyter Hub as a first step. As in [other Openscapes Champions cohorts](https://openscapes.org/blog.html#category=champions), teams also realized the power of Open science and that open is a spectrum that includes considering future us - potentially just you or your group in 3 months.  + +We saw teams and the NASA DAAC mentors make significant progress in migrating workflows to the Cloud. A few highlights from teams that participated in the cohort included:  + +- The Geoweaver team made a Python version of Geoweaver, pyGeoweaver, and contributed a [tutorial](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/fair-workflow-geoweaver-demo.html) to the EarthData Cloud Cookbook!  + +- The CLIMCAPS Team did initial experiments with earthaccess. They found it was a factor of 2 faster, but expected a more significant increase in speed. + +- The HEAT Team was able to access data through earthaccess and bring it into Geopandas. They have additional work to do to use the Cloud to process data to connect to other services like GIOVANNI.  + +- The S-MODE Team made initial progress using earthaccess with S-MODE data. They had challenges being able to only access one file at a time.  + +- The Hydrometeorology and GESTAR Teams connected with our MathWorks contacts and made progress accessing NASA Earthdata in the Cloud via MATLAB. We've submitted a talk at the AGU fall conference about this, stay tuned! + +- The LASERS Team had experience with Google Earth Engine and spent some time trying to use AWS for the same tasks. They also noted how their Cloud collaboration was a pathway toward more open, collaborative work across their team.  + +## **Onward!** + +If our NASA Openscapes Champions Cohort is any indication, the NASA Earthdata community is making significant strides in building capacity to use Cloud resources, and the transition is successfully happening. While the cohort is officially over, these teams are just at the beginning, and we are excited to follow their results. Several Champions will participate in Carpentries Instructor Training this summer; The Carpentries is a nonprofit that teaches introductory coding skills around the world, Instructor Training is not coding-specific, but it is a pedagogical approach to teaching technical topics. As part of our NASA grant, we partner with The Carpentries and are excited to extend this opportunity to Champions since many of them mentioned wanting to contribute more to open science efforts going forward. + +We are grateful to this Champion Cohort's early adopter spirit, their time and effort to make this migration and all of the feedback and input they provided. They all participated in this cohort, knowing that while this was the second Cohort, they were among the first research teams to use NASA Earthdata in the Cloud. What they learned and shared will make it easier for subsequent teams to make this same shift. Many teams articulated this spirit of Open leadership, explicitly asking how they could help other teams. We also learned so much, and we will refine the NASA Openscapes Champions as we plan for our next cohort and our work with the DAAC mentors in the future years of our project.\ + +## **About the NASA Openscapes Champions Teams** + +**CLIMCAPS Team** We use a number of NASA products in [CLIMCAPS](https://airs.jpl.nasa.gov/data/products/climcaps-L2-L3/) to stabilize and improve the retrieval of atmospheric profiles from hyperspectral infrared measurements. These include (i) MERRA-2 and (ii) the MEASURES CAMEL dataset. We distribute CLIMCAPS as Level 2 and Level 3 files to a range of different users. Our research focuses on maintaining/improving the CLIMCAPS algorithm and collaborating with users to improve and tailor CLIMCAPS data products. Since releasing the CLIMCAPS record via GES DISC in 2020 (2002--present), we have seen our user base grow. We have had the chance to collaborate with many groups. This gave us the opportunity to address their questions, clarify the product and help prepare custom Level 3 files that are tailored to target applications. The CLIMCAPS Level 2 product contains many different types of uncertainty metrics that can be used to filter and refine data usage. We would absolutely love the opportunity to make these workflows, that we've helped develop for users, more widely available. I have no doubt that other communities will find it useful also, and we want, in turn, to learn from others. Experience is teaching us that good science happens when developers and scientists collaborate together. And this is why I think Cloud-based workflows is one of the most exciting technology advances in recent years. + +**HEAT (HydroEnergy Analytics Team)** There is an increasing global demand for food, water and energy mainly driven by rapid urbanization, rising population, economic growth and regional conflicts. As water, energy and food are interconnected and are part of a system, we will adopt an integrated Water-Energy-Food Nexus (W-E-F) approach to examine the interdependence of water, energy and food and the impacts of climate change on water, energy, and food consumption in Nepal. Nepal, a country located in Southeast Asia, presents a unique case, where the datasets are either sparse, extremely challenging to obtain, or unavailable. Assessing W-E-F Nexus is particularly challenging due to gaps in datasets and lack of long-term observations. NASA's Earth Observation data can be used to overcome these obstacles by providing ongoing long-term observations of the planet at various spatial and temporal resolutions, effectively filling in the gaps in the data. Our approach involves utilizing data-driven techniques to analyze the W-E-F nexus at watershed level in Nepal. Specifically, we plan to combine NASA's Earth Observation data with hydropower generation, agricultural production and socioeconomic data in Nepal to gain a more comprehensive understanding of the W-E-F nexus in the region. We have identified several potential datasets, and will address two major questions in the cohort: 1) How has water availability, hydropower generation and consumption, demographics, crop yield changed in Nepal over the last 20 years? 2) What is the relationship between change in hydropower generation with respect to changes in hydrometeorology and demographics? Outcome: We will develop a Jupyter Notebook that outlines a comprehensive method for generating outputs from data acquisition using a Cloud environment. This end-to-end process will be clearly described in the notebook, providing an intuitive and efficient workflow that can be replicated by others adhering to the open science principles. Through this effort, we aim to facilitate the use of Cloud computing in data analysis and dissemination, streamlining the research process and enabling more efficient collaboration among researchers. The final Jupyter Notebook will be shared in the collaborative space (GitHub). The pain points encountered from an interdisciplinary user's perspective will be documented. + +**LASERS Team** Our team has expertise in assessing vegetation biophysical parameters using data from all platforms, terrestrial, airborne and spaceborne. We have experience in using the Google Earth Engine and [ICESat-2](https://icesat-2.gsfc.nasa.gov/) data to produce gridded-maps of canopy height and canopy cover and we have produced open science software tools for displaying and labeling ICESat-2 photons, e.g., PhotonLabeler, or Waveformlidar, both available on GitHub. Our research efforts of producing Landsat-resolution gridded maps of canopy height, canopy cover, and biomass, are limited in scale by using local computing and data download-intensive approaches. Educational accounts on Cloud-based platforms, such as GEE, have limitations for data availability, data volume and modeling options, and the learning curve and cost for migrating to platforms like AWS, have their own challenges. As such, we hope that by joining NASA's Openscapes we'll remove barriers to producing timely vegetation products at continental and global scales to better answer pressing questions in ecology while empowering students and young scientists to use Cloud-computing and open science tools. We use ICESat-2 data, Landsat, and derived data to characterize vegetation structure and estimate biophysical parameters, such as height, cover, biomass. For producing gridded maps of vegetation parameters at scales larger than regional, e.g., continental and global, the volume of data download and processing time are hampering the workflow, therefore we intend to migrate the process to the Cloud. + +**Geoweaver Team** Our group has a lot of interests and collaboration history with DAACs and the Earth science community, and we are dedicated to developing [Geoweaver](https://github.com/ESIPFed/Geoweaver) to support Earth science teams as well as DAAC staff (data pipeline, ingestion, migration, analytics) to be productive, and their work are tangible and FAIR for other scientists to reproduce. We use NASA data as input variables to AI models to train the models with the ability to discover Earth insights in time and be actionable. We are very interested in migrating the workflow into the Cloud because that is where the data is and there are data retrieval and I/O steps in our workflows which are slow if they are executed outside the Cloud. By migrating workflows into Cloud, we want to experiment and showcase that: (1) scientists can easily switch from their personal computing environments to Cloud environment seamlessly and effortlessly using Geoweaver; (2) the hybrid collaboration environment provided by NASA and researchers' home institution, including their laptops, can work together for one single workflow (/purpose) without under-using (wasting) resources; (3) make the building-testing-debugging iteration more quick, useful, transparent, and of course FAIR (Geoweaver records everything people did no matter where) and help scientists get serious about workflow run history sharing in a single zip file; (4) show that Geoweaver is a click-button solution if scientists want to deploy their AI workflows into operational services to run periodically like every day. + +**S-MODE Team** Our group has a diversity of professional experience, ranging from graduate students to PI's. Everyone in our group is interested in exploring open-source workflows on the Cloud and building machinery to analyze a variety of data from [S-MODE](https://espo.nasa.gov/s-mode/content/S-MODE) (Sub-Mesoscale Ocean Dynamics Experiment). Our collaboration will benefit the general S-MODE community as we plan on sharing our findings. Our overall goal is to foster a data analysis community for S-MODE by creating machinery that can be used by many research groups. From Mackenzie: "I work with NASA funded saildrone data to investigate submesoscale dynamics of the upper ocean and air-sea interactions. Specifically, I am working with saildrone datasets from the Atlantic Tradewind Ocean Atmosphere Interaction Campaign (ATOMIC) and S-MODE. I analyze the saildrone data using python and Jupyter notebooks. Additionally, I use satellite data from SMAP, Aquarius, and SWOT to provide environmental context on where the saildrone data was collected. I currently download most of the datasets and keep them on my university's HPC system. I use open-source software to analyze the data, which includes python packages such as Xarray, numpy, matplotlib, cartopy, and pandas." + +**Hydrometeorology Team** We are highly motivated to migrate our ground validation workflow for the [NASA GPM - Global Precipitation Measurement](https://gpm.nasa.gov/) product to the Cloud. With over a decade of experience working with ground weather radar products and developing comprehensive workflows for ground validation work, we are eager to streamline and optimize our process. We use NASA data in three areas. First, we conduct cross-validation between NASA GPM products and NOAA ground weather radar products. Second, we create synergy of multiple NASA remote sensing measurements to achieve better precipitation product. Third, we apply NASA remote sensing data to monitor and forecast natural hazards such as flash flooding, drought, tornado, etc. Our team recognizes the numerous benefits that Cloud computing resources can provide. Firstly, Cloud services are highly scalable and can easily accommodate changes in demand from other scientists. Secondly, by migrating our workflow to the Cloud, we can automate many tasks and improve efficiency while reducing the risk of errors or delays. Thirdly, Cloud computing resources are accessible from anywhere with an internet connection, making it easier to collaborate with other scientists. Lastly, Cloud services can be integrated with other tools and services, such as machine learning or data analytics platforms, enhancing the overall capabilities of the workflow. We strongly believe that ground validation efforts require a global approach, and migrating our ground validation workflow to the Cloud can greatly enhance our ability to collaborate with other scientists and improve our work on a global scale. + +**GESTAR Team** We conduct sensor calibration, mission support and support data applications. I work in the [GESTAR2](https://gestar2.umbc.edu/) group (Goddard Earth Sciences Technology and Research II) at UMBC (University of Maryland, Baltimore County). Our team, lead by Dr. Strow with us has been responsible for the development of the CHIRP (Climate Hyperspectral Radiance Product) now being hosted on GES DISC DAAC and intimately concerned with the Aqua AIRS, NOAA CrIS MetOP IASI sensors so that their data can be used for climate studies. We are heavily invested in Matlab with C and Fortran libraries. We deal with 10s of TB data almost daily using the HPC at UMBC - a fantastic resource. There is an interest and growing incentive to migrate to the Cloud, but for us the capital (of effort) to 're-locate' is very significant. We need to understand the cost/benefit in our situation. There is a very steep learning curve and very little time available and there isn't a one solution to fit all. diff --git a/blog/2023-08-01-nasa-champions/when-to-cloud-barrett.png b/blog/2023-08-01-nasa-champions/when-to-cloud-barrett.png new file mode 100644 index 0000000..4928cfc Binary files /dev/null and b/blog/2023-08-01-nasa-champions/when-to-cloud-barrett.png differ diff --git a/blog/2023-08-01-nasa-champions/zoom-chat-earthaccess.png b/blog/2023-08-01-nasa-champions/zoom-chat-earthaccess.png new file mode 100644 index 0000000..0ba1007 Binary files /dev/null and b/blog/2023-08-01-nasa-champions/zoom-chat-earthaccess.png differ diff --git a/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions-square.png b/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions-square.png new file mode 100644 index 0000000..c56909b Binary files /dev/null and b/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions-square.png differ diff --git a/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions.png b/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions.png new file mode 100644 index 0000000..4bf1bfb Binary files /dev/null and b/blog/2023-08-01-nasa-champions/zoomie-2023-nasa-champions.png differ diff --git a/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_2i2c.png b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_2i2c.png new file mode 100644 index 0000000..a068a32 Binary files /dev/null and b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_2i2c.png differ diff --git a/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_coiled.png b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_coiled.png new file mode 100644 index 0000000..7ffef7f Binary files /dev/null and b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_coiled.png differ diff --git a/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_playground.png b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_playground.png new file mode 100644 index 0000000..1879c51 Binary files /dev/null and b/blog/2023-10-13-nasa-jupyterhub-coiled/UWG2023_Cloud_Environment_Opportunities_playground.png differ diff --git a/blog/2023-10-13-nasa-jupyterhub-coiled/amy-steiker-uwg.png b/blog/2023-10-13-nasa-jupyterhub-coiled/amy-steiker-uwg.png new file mode 100644 index 0000000..1c40642 Binary files /dev/null and b/blog/2023-10-13-nasa-jupyterhub-coiled/amy-steiker-uwg.png differ diff --git a/blog/2023-10-13-nasa-jupyterhub-coiled/index.qmd b/blog/2023-10-13-nasa-jupyterhub-coiled/index.qmd new file mode 100644 index 0000000..cdc4f34 --- /dev/null +++ b/blog/2023-10-13-nasa-jupyterhub-coiled/index.qmd @@ -0,0 +1,118 @@ +--- +title: "Cloud Environment Opportunities" +description: "Managed JupyterHub options for Cryosphere and NASA Earthdata user communities" +author: + - name: Amy Steiker + orcid: 0000-0002-3039-0260 + - name: Luis Lopez + - name: Andy Barrett + orcid: 0000-0003-4394-5445 + - name: the NASA Openscapes Mentors + - name: James Bourbeau of Coiled +date: 2023-10-13 +citation: + url: https://openscapes.org/blog/2023-10-13-nasa-jupyterhub-coiled +categories: [nasa-framework, blog] +image: amy-steiker-uwg.png +--- + +*We support users at the NASA National Snow and Ice Data Center Distributed Active Archive Center ([NSIDC DAAC](https://nsidc.org/data/data-programs/nsidc-daac)). This week we had an in person meeting with our User Working Group (UWG), a group that consists of fourteen members representing our cryospheric user community by providing recommendations on the DAAC's data resources and overall objectives and priorities. We presented a slide deck with an overview of Cloud Environment Opportunities focused on managed JupyterHub options, and a live demo of Coiled, which is a company providing software and expertise for scalable Cloud computing built on Dask. This work currently builds from our Cloud infrastructure set up with NASA Openscapes. The purpose was to share the options currently available, and to invite UWG members to work and improve from these ideas.* + +*We're also part of the NASA Openscapes community; we support researchers using NASA Earthdata as they migrate their data analysis workflows to the Cloud. This blog post does not go into deep detail about how NASA Earthdata is migrating to the Cloud, but you can read more about our efforts with NASA Openscapes at .* + +*This blog post gives a brief summary of the slides and some thoughts going forward.* + +*Quick link: [slides](https://docs.google.com/presentation/d/1NEsE58SIMjQ_fjC_11HIXqd7Urvi5yy5/edit#slide=id.g23808f09f20_2_714)* + +------------------------------------------------------------------------ + +## When to Cloud? + +We started off with considerations of "When to Cloud?" This covered things to consider for you now and in the future: + +- What is the data volume? +- How long will it take to download? +- Can you store all that data (cost and space)? +- Do you have the computing power for processing? +- Does your team need a common computing environment? +- Do you need to share data at each step or just an end product? + +Andy Barrett created and presented more in-depth slides to the [2023 NASA Champions Cohort](https://openscapes.org/blog/2023-08-01-nasa-champions/) of Science teams: [Data strategies for Future Us, for Cloud](https://nsidc.github.io/data_strategies_for_future_us/data_strategies_slides). + +**Assuming you are "ready to Cloud" based on the considerations above, there are two main solutions for accessing NASA Earthdata Cloud: Do it yourself or using a managed Cloud service.** If you do it yourself, this involves creating an AWS Account, connecting to an EC2 instance, and using resources like the [Earthdata Cloud Primer](https://earthdata.nasa.gov/learn/user-resources/webinars-and-tutorials/cloud-primer) for more setup and cost management information. If you use a managed Cloud service, organizations like [2i2c](https://2i2c.org/) can provide Cloud-hosted JupyterHubs for research and education. Your institution may also support smaller or larger scale options. + +## Comparing/Overview of Managed Hubs + + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "slide 4 in https://docs.google.com/presentation/d/1NEsE58SIMjQ_fjC_11HIXqd7Urvi5yy5/edit#slide=id.g23808f09f20_2_714. Heading 'NASA Openscapes 2i2c JupyterHub', text box to left, screenshot to right" +knitr::include_graphics("UWG2023_Cloud_Environment_Opportunities_2i2c.png") +``` + +::: {.small-text .center-text .dark-gray-text} +The NASA Openscapes 2i2c JupyterHub, one of the six options presented, provides a valuable shared Cloud environment not only for our Science Champions and workshop learners, but also for our DAAC scientists, developers, and user support staff across NASA EOSDIS (Earth Observing System Data and Information System). +::: + + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "slide 6 in https://docs.google.com/presentation/d/1NEsE58SIMjQ_fjC_11HIXqd7Urvi5yy5/edit#slide=id.g23808f09f20_2_714. Heading 'Earthdata Cloud Playground', text box to left, screenshots to right show a GitHub repo and out put of a python notebook showing graphic of the Great Lakes" +knitr::include_graphics("UWG2023_Cloud_Environment_Opportunities_playground.png") +``` + +::: {.small-text .center-text .dark-gray-text} +The Earthdata Cloud Playground is in development as a long-term resource for users learning and testing their data workflows in the Cloud. +::: + + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "slide 8 in https://docs.google.com/presentation/d/1NEsE58SIMjQ_fjC_11HIXqd7Urvi5yy5/edit#slide=id.g23808f09f20_2_714. Heading 'Coiled', text box to left, screenshots to right" +knitr::include_graphics("UWG2023_Cloud_Environment_Opportunities_coiled.png") +``` + +::: {.small-text .center-text .dark-gray-text} +Coiled can be a resource for those who wish to offboard or scale from an existing Hub environment. +::: + +## Coiled live demo + +This Fall, Openscapes is partnering with [Coiled](https://www.coiled.io/) to support us experimenting with another approach to Cloud access, as well as refactoring workflows from serial processes (for-loops) to parallel in order to leverage the true power of Cloud. Amy Steiker and Luis Lopez lead a [live demo for UWG participants](https://docs.google.com/document/d/1tqV1eF15qqyZL8lMAa9a3PyXJ6CC3fvHe4oKzw_iaUs/edit#heading=h.c3zsga28bzu), leveraging the same Google doc approach used during the Science Champions [Earthdata Cloud Clinic](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/Earthdata-cloud-clinic.html) for this event, including more information on Coiled.  + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "Photo with foreground showing backs of 14 people seated at tables with labtops, watching a woman at podium and a projection of a presentation slide" +knitr::include_graphics("amy-steiker-uwg.png") +``` + +::: {.small-text .center-text .dark-gray-text} +Amy Steiker presenting to NSIDC User Working Group +::: + +The demo showcased a [Python script](https://github.com/nsidc/NSIDC-Data-Tutorials/blob/main/notebooks/ICESat-2_Cloud_Access/h5cloud/workflow.py) that processed large amounts of altimetry data from NASA's ICESat-2 mission. While the script was run from Amy's local computer, the data processing steps were run on the Cloud using [Coiled Functions](https://docs.coiled.io/user_guide/usage/functions/index.html) for running Python functions on Cloud virtual machines (VMs). This approach was particularly convenient as it allows existing Python functions to be run in the Cloud by lightly annotating them with a `@coiled.function` decorator. + +This workflow benefited from running on the Cloud because the ICESat-2 mission data was already stored on the Cloud in S3, so moving the data processing to be co-located next to the data avoided data transfers, which are both slow and expensive. + +## Recap + +We closed our presentation and demo with the following advice for the User Working Group scientists: + +::: {.blockquote-blue} +> *First ask yourself: When to Cloud? You may continue to download data, or work locally using Cloud-based service outputs, and optionally take advantage of Cloud.* +::: + + +If you "passed go", there are a growing number of options to easily onboard to a Cloud environment. The options we presented are not exhaustive! We want to hear from you on other options you are pursuing and how your Cloud transition is going. diff --git a/blog/2023-10-17-matlab-on-openscapes/index.qmd b/blog/2023-10-17-matlab-on-openscapes/index.qmd new file mode 100644 index 0000000..62990a4 --- /dev/null +++ b/blog/2023-10-17-matlab-on-openscapes/index.qmd @@ -0,0 +1,76 @@ +--- +title: "MATLAB on Openscapes" +description: "Accessing and Analyzing NASA Earthdata in the Cloud with MATLAB" +author: + - name: Lisa Kempler +date: 2023-10-17 +citation: + url: https://openscapes.org/blog/2023-10-17-matlab-on-openscapes +categories: [nasa-framework, blog] +image: matlab-launch.png +--- + +*Lisa Kempler works at MathWorks as a Research and Geoscience Strategist. She supports research and educator communities seeking to integrate their platforms with software tools and resources that enable effective data access, computing, and results sharing and publishing. She regularly meets with research communities, including site developers and users of data and compute services, developing programs and working with teams to provide implementation and user support. Lisa attended Brown University, Boston University, and Northeastern University.* + +***Quick links:*** + +- [*Recording*](https://drive.google.com/file/d/15kuVavXX2fVnLqiOlr9rHVmphZQ5NZNa/view) *of the presentation at the NASA Openscapes Mentors Call on Sept. 6, 2023 by Cedric Wannaz (MathWorks)* +- [*Tutorial*](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html)*: Using MATLAB on NASA Openscapes* + +------------------------------------------------------------------------ + +More and more data is being made available for users on [NASA's Earthdata Cloud](https://www.earthdata.nasa.gov/eosdis/cloud-evolution) platform. [NASA Openscapes](https://nasa-openscapes.github.io/) collaborates with a community of user-support staff across ten of the NASA Earth science Distributed Active Archives (DAACs), with the aim to help researchers transition their computational workflows to the Cloud using NASA Earthdata. + +Through the [NASA Openscapes Champions,](https://nasa-openscapes.github.io/champions.html) an annual program that supports cohorts of science teams, a number of researchers expressed interest in using the data hosted on NASA Earthdata with MATLAB. The initial NASA Openscapes' JupyterHub platform hosted by [2i2c](https://2i2c.org/), and tutorials, were Python-based. However, to make this transition, users need to be able to use software tools that are familiar to them that enable access to the data and can process it. The NASA Openscapes team reached out to MathWorks, developers of MATLAB, to support the effort to integrate MATLAB into NASA Openscapes JupyterHub and tutorials. The goal was to enable direct Cloud data access from MATLAB.  + +Together, our two teams have successfully installed MATLAB on NASA Openscapes JupyterHub, visible in the screenshot below. It is now available for researchers participating in NASA Openscapes affiliated learning events to try out with Earthdata data. Researchers will "bring their own license" (BYOL) and will be prompted to input that information to access MATLAB. + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "JupyterHub interface with folders listing on left and launcher window on right including 3 rows of icons. Top row inludes MATLAB launcher" +knitr::include_graphics("matlab-launch.png") +``` + +::: {.small-text .center-text .dark-gray-text} +Two MATLAB computing options on the Openscapes 2i2c JupyterHub: Full MATLAB desktop (with IDE) and via a Jupyter Notebook interface. +::: + +The MATLAB implementation on Openscapes JupyterHub consists of + +- MATLAB desktop: A full MATLAB, including the IDE (bring your own license[^1]) + +- 15 MATLAB toolboxes available to use with the MATLAB installation + + - Including: Mapping, Signal Processing, Statistics and Machine Learning toolboxes + - As users identify additional toolboxes they require, this list can grow. + +[^1]: Many universities and research institutes have site-wide licenses for MATLAB -- called "Campus-Wide Licenses" and "Institute-Wide Licenses", respectively. Most universities in the U.S. and Canada have CWLs. In those cases, all researchers, faculty and students have access to a MATLAB license via their institutions that work in this BYOL setup. Check with your university system administrators to find out if you have access to a MATLAB license at your institution. + +In addition, we've written a detailed tutorial to help users learn the system and process the data. The [MATLAB tutorial](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html) from NASA Openscapes includes code examples, that cover: + +- [Launching MATLAB in Openscapes 2i2c JupyterHub](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html#launch-matlab-in-2i2c-jupyterhub) +- Login and Authentication +- [Accessing data](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html#accessing-data) on Earthdata +- [Launching with AWS Console](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html#launch-with-aws-console) + +```{r} +#| eval: true +#| echo: false +#| fig-align: center +#| out-width: "75%" +#| fig-alt: "Map showing sea surface temperature in yellow, green, blue. Heading 'sea surface temperature [kelvin]'; x-axis from 24 to 44; y-axis from -80 to -50" +knitr::include_graphics("matlab-map.png") +``` + +::: {.small-text .center-text .dark-gray-text} +Map showing sea surface temperature created from HDF5 data with MATLAB contour() using the AWS console (screenshot from the MATLAB tutorial in the Earthdata Cloud Cookbook) +::: + +It's new to work with NASA Earthdata on the NASA Openscapes JupyterHub, and even newer with MATLAB! We are excited that teams participating in the 2022 and 2023 NASA Openscapes Champions program are already logging into Openscapes and using MATLAB on the platform and continuing to push forward NASA Earthdata Cloud access through MATLAB. We will share more results on this work at the upcoming [American Geophysical Union Fall Meeting on Tuesday, December 12.](https://agu.confex.com/agu/fm23/meetingapp.cgi/Paper/1281461) If you are interested in this work, please get in touch ([lisak\@mathworks.com](mailto:lisak@mathworks.com)). + +To learn more about using MATLAB with data on NASA Openscapes, watch the [video presentation to the NASA Openscapes Mentors](https://drive.google.com/file/d/15kuVavXX2fVnLqiOlr9rHVmphZQ5NZNa/view?pli=1) or read the [MATLAB Tutorial](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html). + +*Acknowledgements: A special thanks to Erin Robinson of NASA Openscapes and Luis Lopez Espinosa of the National Snow and Ice Data Center (NSIDC) for their collaboration on the NASA Openscapes MATLAB implementation and their contributions to this blog post.* diff --git a/blog/2023-10-17-matlab-on-openscapes/matlab-launch.png b/blog/2023-10-17-matlab-on-openscapes/matlab-launch.png new file mode 100644 index 0000000..d11a714 Binary files /dev/null and b/blog/2023-10-17-matlab-on-openscapes/matlab-launch.png differ diff --git a/blog/2023-10-17-matlab-on-openscapes/matlab-map.png b/blog/2023-10-17-matlab-on-openscapes/matlab-map.png new file mode 100644 index 0000000..d132d0b Binary files /dev/null and b/blog/2023-10-17-matlab-on-openscapes/matlab-map.png differ diff --git a/index.qmd b/index.qmd index 9774fe7..0e950ae 100644 --- a/index.qmd +++ b/index.qmd @@ -9,6 +9,6 @@ We are influenced and inspired by many leaders and community organizers, particu Project announcements: [NASA](https://earthdata.nasa.gov/learn/openscapes) • [Openscapes](https://www.openscapes.org/blog/2021/03/10/nasa-announcement/) -Learn about our recent work: [Blog Posts](/about.qmd#blog) • [Presentations](/about.qmd#slides) • [Annual Reports](https://github.com/NASA-Openscapes/how_we_work) • [Flywheel Preprint](https://nasa-openscapes.github.io/about.html#flywheel-preprint) • [White Paper: The Value of Hosted JupyterHubs](https://nasa-openscapes.github.io/about.html#white-paper-value-of-hosted-jupyterhubs) +Learn about our recent work: [Blog Posts](blog.qmd) • [Presentations](/about.qmd#slides) • [Annual Reports](https://github.com/NASA-Openscapes/how_we_work) • [Flywheel Preprint](https://nasa-openscapes.github.io/about.html#flywheel-preprint) • [White Paper: The Value of Hosted JupyterHubs](https://nasa-openscapes.github.io/about.html#white-paper-value-of-hosted-jupyterhubs) Please connect with us on Twitter [\@openscapes](https://twitter.com/openscapes) or join [our newsletter](https://docs.google.com/forms/d/e/1FAIpQLSdgVXRp3V-w94GPWkR31RUfyBl37EphdQSlCOcnyeNlf8OLWw/viewform)