Refactor / Overhaul Docs for 3.1

[jpwhitemn]

📚 Docs or Wiki Bug Report

This issue supersedes #1141

Per the Napa Release planning meeting, it was discussed and approved for incorporation into the next release to refactor and greatly improve the documentation. Specifically, there was agreement for “significant doc updates”.

@ejlee3 has provided some input to this task with issue #1146.

This topic was discussed in the Jun 21, 2023 architect’s meeting and feedback from that meeting is provided/annotated in the information below.

Having reviewed the documentation and all issues in the edgex-docs repository, the following concrete items are recommended to the community for comment, refinement and ultimately developer/writer actions for the upcoming 3.1 release:

Overall Organization

The top bar sections (Introduction, Getting Started, Security, …) seems haphazard and not helpful to adopters getting to the general category of information they need. Could we provide a more complete, detailed, side menu bar to layout the entire documentation structure for the user? Then use a tile approach on the main page to provide quick clicks into areas of the documentation? This approach is used by projects like Kubernetes, Grafana, or TensorFlow? The tiles take you to important topics directly in the side bar menu and could even someday be selected based on usage statistics.

The side bar menu would be organized per below (highlighted areas are main page tiles). It would be more detailed and contain duplicate references to topics that are common in multiple categories (for example, configuration information would be available through both the service category as well as configuration category). This should be accomplished with duplicate menu items but without duplication of content (i.e. link to the same content from different menu options). Major bullets being the displayed topics with the sub bullets being expanded menu selections.

Note: per the architect’s meeting, there is a desire to keep the menu bar across the top, but the menu bar would contain “bigger” items (but there is not clarity as to what would be in the menu bar).

• Fundamentals
  • Overview of the project (some of which is under Introductions today)
  • EdgeX Fundamentals (Concepts, Terminology and Definitions; some under reference today but needs expanding)
  • Overview of the architecture (some under intro and micro services today)
  • Platform Requirements (under reference today)
  • Releases and LTS
  • Getting Involved (guidance on resources to get involved with the project – mostly pointers to things like the wiki)
• Getting Started
  • Install EdgeX (the quick start guide today)
    • Using Docker (in the getting started area today)
    • Using Snaps (in the getting started area today)
    • Using Helm
  • Running EdgeX
    • Connect a device
    • Export data to the Cloud
  • Checking EdgeX Status and Troubleshooting
    • Ping Check
    • Logs
  • Walkthrough (an end-to-end walkthrough similar to the API Walkthrough)
• Upgrading
  • V1 to V2 Migration Guide (put security migration guides as sub topics?)
  • V2 to V3 Migration Guide (put security migration guides as sub topics?)
  • Using Nightly Builds (from the nexus repository docs)
• Services (see more below)
  • Core
  • Device
    • MQTT
    • Virtual
    • BACnet
    • …
  • Application
    • LLRP
    • Configurable
  • Supporting
  • Security
  • SDKs
    • Device Service Go
    • Device Service C
    • App Functions
  • GUI & Tools
  • Miscellaneous Components (3rd party)
  • Configuration (a lot of what is in Configuration & Registry today)
    • Configuration Guide: Common, local and overrides
    • Configuration structure
    • Writable vs Readable
    • Secrets configuration
    • Configuration reference
• Developer
  • Object Models (explanations and detailed information on EdgeX object models)
    • Devices and Device Services
    • Device Profiles and Device Resources
    • Events and Readings
    • Configuration organization
    • DTO Validation
  • Message Bus (from the Message Bus page)
    • Message structure
    • Topic Structure
    • Implementation options
  • Building and Running EdgeX Natively (in the getting started area today)
  • Working in a hybrid environment with containers and native services
  • Creating and building a device service (in Go)
  • Creating and building a device service (in C)
  • Creating and building an application service
  • Building your own docker compose file
• Tutorials (see more below)
• References
  • Default service ports
  • Container names
  • Security Concerns
    • Filing a security issue
    • CORS settings
    • Threat models
  • Examples
    • Explanation of code “Examples” in EdgeX
    • List of Edge Examples with links to repositories
  • Project Requirements, design decisions
    • UCR and ADR processes
    • Use Case Requirements
    • Design Records
  • Redis Database Performance
  • Is EdgeX Foundry cloud native

Note: from the architect’s meeting it was noted that the tiles would largely be the important “getting started guides” and “deployment guides” are things needed first. Eventually, the tiles may be populated dynamically based on doc site stats (unsure how that would be accomplished).

Reorganization/Standardization of documentation Sections

Each section of the microservice documentation sections is organized and written as it came from different authors with different objectives and considerations. This is, in actually, a point of fact versus criticism, but it also makes it difficult for adopters to negotiate the documentation and find the information they need. The entire documentation set should be overhauled such that each of the related sections should have the same categories of information and structure. Also, the micro services sections contained a lot of information (a hodge podge of model info, tutorials, configuration, etc) that made finding the needed information hard. This section should be greatly simplified to provide some general guidance on what the service does and then pointers out to api and configuration guides, and pointers to tutorials for the service. A small section (Details) can provide service specific detailed information (where there is no other place to put it).

The following structure is suggested for each service section (using core and core data as an example):

• Core Services
  • General Information (general description of the purpose of each service in this area; which are required, optional, and when)
  • Core Data
    • Purpose (how and when to use)
    • Configuration (specific to core data)
    • Tutorials
    • API Reference
    • Details
      • Streaming

Increased Focus on Getting Started Guides

Getting started guides need to focus on the most fundamental and critical needs of the platform (install run, get stuff connected, check that its working) in small, easily digestible guides. A final guide should help walk an adopter through the entire platform at a high level

Increased Focus on Practical Examples/Tutorials

We need more step-by-step guides on some of the most common needs in the system, probably organized by category. Below is a set of known tutorial needs

• Creating an app service using the app function SDK
• Creating a device service using the device service SDK in C
• Creating a device service using the device service SDK in Go
• Setting up the virtual device service to simulate sensor data
• Sending and using binary data in EdgeX
• Commanding devices using the rules engine
• Establishing rules in the rules engine
• Using provision watchers
• Setting up sensor data capture with auto events
• Scheduling data clean up
• Setting up and applying units of measure to sensor data
• Configuring and using service metrics
• Using EdgeX in Secure Mode
• Making authenticated calls of microservices
• How to secure the message bus
• How to secure add-on/custom services
• How to secure the configuration store (Consul)
• How to seed service secrets
• How to build a delayed-start service
• How to setup the API Gateway
• Writing and using device profile
• Working in a hybrid (Docker / Native) environment

Considerations, Open Questions, and Research Needed

• How do we do this (big sidebar menu with tiles) with mkdocs and environment we have today?
• New version of mkdocs provides for a better way to handle the version situation. How is this done and what would be the impact?
  • Version dropdown is always moving around. It should be at the top.
  • OpenAMT has this done well.
• What would go in a menu bar?
• We want to maintain/keep the page table of contents (showing currently on the right side of the page)
• Top level page should have short/concise description of the project
• What would all this look like on a mobile device? We need to make sure what we do is still looking good on smaller displays.
• Can we bring analytics into the picture – what topics are hot? Can that drive a list or the tile display.

Relevant edgex-docs issue input

• Update stale images and reference files for v3.x #1106 – stale images
• Device Grove snap link broken #1037, Better looking 404 page #1103 – get rid of 404s, broken links and provide a better page for reporting broken links
• more documentation around Device Profiles #28, Simplify Getting Started Device SDK examples #195, Extend the Go Device SDK API documentation #855 – improve documentation around how to use the device services SDK, device profiles, and API documents around the same
• Use Git LFS to manage images #714 – move to a better image tool
• Provide more explicit documentation on export of data to the clouds (Azure, AWS, Google) #642 – better documentation/examples on cloud exports
• Document how to consume and produce CBOR values #21, Update CBOR documentation to refer to new CBOR library #121, Update Sending and Consuming Binary Data example #489 – better documentation on using binary data/CBOR data
• Document how to create a Docker image for an Application Service using the SDK #33 – better documentation on how to create new app services and docker images for those

[jpwhitemn]

Next steps:

1. do some prototyping with mkdocs to test the art of the possible of some of these demands (especially around mkdocs new version)
2. identify open source documentation that the community thinks serves as a good template

[lenny-goodell]

Nice job @jpwhitemn
I think another next step is to create a project board containing the work break-down to get this completed. Draft cards are good enough for now and then convert to issues when ready.

The hard part is how we do this in one release cycle.
IMO, we should start with just the restructuring of existing content into what you have described above. Then work on cleaning up the existing content as mush as possible before Napa LTS release.

[bnevis-i]

Since Snap suppot is removed with Napa, we should also be removing the documentation as well. https://docs.edgexfoundry.org/3.1/getting-started/Ch-GettingStartedSnapUsers/

[lenny-goodell]

@bnevis-i , when I go to https://docs.edgexfoundry.org/3.1/getting-started/ the Snap documentation is no longer in the menu. Also the document your link references is not longer in the repo at: https://github.com/edgexfoundry/edgex-docs/tree/napa/docs_src/getting-started

Please verify that when you navigate to https://docs.edgexfoundry.org/3.1/getting-started/ the Snap doc is gone. THX!

[bnevis-i]

Ok. This is wierd.

If I start at https://docs.edgexfoundry.org/3.0/getting-started/Ch-GettingStartedSnapUsers/ I see it in the index.
If I then change the version drop down to 3.1, it sends me to https://docs.edgexfoundry.org/3.1/getting-started/Ch-GettingStartedSnapUsers/ which actually does have the page.

I guess a prerendered version of the page is at https://github.com/edgexfoundry/edgex-docs/blob/gh-pages/3.1/getting-started/Ch-GettingStartedSnapUsers/index.html, but not getting replaced since we removed the source.

[lenny-goodell]

Yep, and if you instead switch to 3.2 Odessa, you get the expected Not Found.

I will kick-off a rebuild of Napa to see if that cleans it up.