Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerated Publishing Initiative #6577

Open
27 of 40 tasks
mchelen-gov opened this issue Sep 29, 2021 · 11 comments
Open
27 of 40 tasks

Accelerated Publishing Initiative #6577

mchelen-gov opened this issue Sep 29, 2021 · 11 comments
Labels
Accelerated Publishing Initiative Initiatives are collections of epics that drive toward a common goal defined within Crew's Objective team-drupal-cms Drupal CMS Team Initiatives. Used on OCTO DMC board

Comments

@mchelen-gov
Copy link
Contributor

mchelen-gov commented Sep 29, 2021

Background

User Stories

As a content editor,
I need a higher limit on content volume capacity,
So that I can publish any necessary content on VA.gov.

As a content editor,
I need published content to appear on VA.gov as quickly as possible,
So that I can get information to users in a timely manner.

As a VFS engineer,
I need to easily build frontend applications using CMS content,
So that I can display it to VA.gov users.

As a VA.gov user,
I need content on VA.gov to load reliably,
So that I can depend on VA.gov as a source of information.

Affected users and stakeholders

  • CMS Content Editors
  • Developers of VFS teams
  • VA.gov users

Overall Initiative Goals

  • Faster time between publishing in CMS and availability on VA.gov
  • Increased content scaling capacity within the CMS
  • Content preview within CMS more closely matches production
  • All use cases of content-build are migrated to other systems and content-build is retired
  • Reduced difficulty of VFS FE development
  • Increased reuse of common FE libraries
  • VA.gov user experience is equivalent or improved

Work board

Accelerated Publishing Github Project for work management.

Project aims and associated work

Next Build and HTML generation

Next Build is the primary code output of the Accelerated Publishing (AP) Initiative. It is a front-end build system built on Next.js that generates HTML for VA.gov. This HTML usually includes content sourced from the VA.gov CMS. In the future, there is opportunity for it to also build the HTML that allows VA.gov's vets-website React applications to be delivered to users.

Next Build is a replacement for Content Build, VA.gov's legacy static HTML generation system. Much of the work of the AP Initiative is to replicate the templates and business logic that Content Build provides in a modern React-based application.

The following epics contain work towards completing Next Build as an HTML generation system for VA.gov:

HTML generation

Preview Give feedback
  1. 8 of 36
    Accelerated Publishing Epic Needs refining
  2. 32 of 63
    Accelerated Publishing Epic Migration Needs refining
  3. 0 of 5
    Accelerated Publishing Epic Needs refining
  4. 7 of 7
    Accelerated Publishing Epic Needs refining
  5. 7 of 7
    Accelerated Publishing Epic Needs refining
  6. 2 of 9
    Accelerated Publishing Epic Needs refining Post MVP
  7. 0 of 3
    Accelerated Publishing Epic Needs refining
  8. 1 of 10
    Accelerated Publishing Epic Needs refining

Delivery of Next Build content to VA.gov

At initial launch to production, Next Build will act as a static site generator. It will export all CMS HTML for all page types that it is aware of and deploy those to an S3 bucket. This will be done on a cycle continuously during VA business hours (8:00 am - 8:00 pm, M-F), as well as on-demand when certain content is created or updated in the CMS. This is the first phase in a two phase approach.

The end state of Next Build content delivery is to have a persistently running Next.js server that responds to requests for VA.gov content. Rather than building static HTML to deploy to an S3 bucket, Next.js will instead build content on-demand and deliver it to the VA.gov user. This will allow us to only build content that is requested, which will greatly increase the speed at which updated content can be delivered to VA.gov users. This is the key to accelerated publishing of VA.gov content.

Delivery of Next Build content to VA.gov

Preview Give feedback
  1. 16 of 18
    Accelerated Publishing Epic Needs refining
    nfpappas-oddball
  2. 12 of 13
    Accelerated Publishing Epic Needs refining
  3. 1 of 3
    Accelerated Publishing Epic Needs refining
  4. 3 of 8
    Accelerated Publishing Epic Needs refining
  5. 0 of 3
    Accelerated Publishing Epic Needs refining Post MVP
  6. 1 of 5
    Accelerated Publishing Epic Needs refining

Work in support of VA.gov CMS

One of Next Build's primary use cases is to act as a front-end for the VA.gov CMS. Content that is created, updated, and managed by VA staff is sourced by Next Build and used to generate the CMS 'page' that is delivered to a VA.gov visitor. These systems are separate from each other, but Next Build is how CMS content reaches the public. Some work within the CMS and coordinating the CMS and Next Build is required for these systems to work together.

Work in support of VA.gov CMS

Preview Give feedback
  1. 38 of 40
    Accelerated Publishing Epic Needs refining
  2. Accelerated Publishing Epic Needs refining Post MVP
  3. 8 of 8
    Accelerated Publishing Epic

Content and application integrity

Work is necessary to ensure that the Next Build application is functioning correctly, and that appropriate people are alerted when it isn't. This can include active monitoring of running application servers as well alerting of build failures. Writing of application tests and the framework for running those tests is gathered here.

Additionally, it is important to test the generated HTML itself to ensure that it is functioning correctly. This work includes checking for accessibility, checking for broken links and other site malfunctions, page weight and usability, and SEO for both search.gov and for private sector search engines.

Content and application integrity

Preview Give feedback
  1. 4 of 7
    Accelerated Publishing Epic Needs refining
  2. 7 of 8
    Accelerated Publishing Epic Needs refining

Developer experience and support

One of the prime motivations of the AP Initiative is to improve the front-end development experience at VA.gov, especially for engineers working with CMS content. Just moving to Next.js and React templates accomplishes this on its own. However, it is necessary to make sure that the system is properly documented for engineers, that tools are in place to guide code quality (linting, type checking), and that local or remote development environments are easy to set up and use.

Developer experience and support

Preview Give feedback
  1. 9 of 9
    Accelerated Publishing Epic Needs refining
  2. 1 of 5
    Accelerated Publishing Epic Needs refining

Historical Sprint documents

Completed tasks by sprint report

See all sprints

Historical Sprint documents

Preview Give feedback
  1. 18 of 25
    Accelerated Publishing Needs refining
  2. 14 of 15
    Accelerated Publishing Needs refining
  3. 21 of 22
    Accelerated Publishing Needs refining
  4. 25 of 26
    Accelerated Publishing Needs refining
  5. 15 of 16
    Accelerated Publishing Needs refining
  6. 24 of 24
    Accelerated Publishing Needs refining
  7. 12 of 12
    Accelerated Publishing Needs refining
  8. 11 of 11
    Accelerated Publishing Needs refining
  9. 17 of 17
    Accelerated Publishing
  10. 11 of 11
    Accelerated Publishing
  11. 9 of 9
    Accelerated Publishing
  12. 14 of 14
    Accelerated Publishing Needs refining
  13. 16 of 16
    Accelerated Publishing Needs refining
  14. 13 of 13
    Accelerated Publishing Needs refining
  15. 14 of 14
    Accelerated Publishing Needs refining
  16. 12 of 12
    Accelerated Publishing
  17. 11 of 11
    Accelerated Publishing
  18. 13 of 13
    Accelerated Publishing
  19. 13 of 13
    Accelerated Publishing
@mchelen-gov mchelen-gov added Epic Issue type Needs refining Issue status Initiative Initiatives are collections of epics that drive toward a common goal defined within Crew's Objective labels Sep 29, 2021
@mchelen-gov mchelen-gov changed the title Public Content API [CMS] Content Publishing Oct 4, 2021
@mchelen-gov mchelen-gov changed the title [CMS] Content Publishing [CMS] Accelerated Publishing Oct 13, 2021
@timcosgrove
Copy link
Contributor

Added 'VA.gov users' as a stakeholder and 'VA.gov user experience is equivalent or improved' as an AC/goal.

@cmaeng
Copy link
Contributor

cmaeng commented Oct 4, 2022

This effort has been put on pause as of 9/11/22 and will be shifted to the backlog.

@TheBoatyMcBoatFace
Copy link

Closing due to AP pause. Will re-open when path forward determined

@EWashb
Copy link
Contributor

EWashb commented May 9, 2023

This work will resume in July 2023.

@EWashb EWashb reopened this May 9, 2023
@EWashb EWashb assigned EWashb and unassigned mchelen-gov May 9, 2023
@timcosgrove timcosgrove changed the title [CMS] Accelerated Publishing ⏩ Initiative: Accelerated Publishing Jun 9, 2023
@EWashb
Copy link
Contributor

EWashb commented Jul 18, 2023

Is the Lagoon Milestone attached to this epic arbitrary?

@timcosgrove timcosgrove changed the title ⏩ Initiative: Accelerated Publishing Accelerated Publishing Initiative Jul 19, 2023
@EWashb EWashb removed this from the Lagoon MVP - On-Prem AWS Cloud milestone Jul 26, 2023
@timcosgrove timcosgrove removed the Epic Issue type label Aug 16, 2023
@timcosgrove
Copy link
Contributor

Is the Lagoon Milestone attached to this epic arbitrary?

It has been removed. We are still refining epics but we've organized our listing of epics to correspond to our current understanding of the milestones that will be delivered within the current PoP.

@timcosgrove timcosgrove removed the Needs refining Issue status label Aug 18, 2023
@EWashb EWashb added the team-drupal-cms Drupal CMS Team Initiatives. Used on OCTO DMC board label Oct 30, 2023
@EWashb EWashb added this to the AP: Milestone 4 milestone Oct 30, 2023
@EWashb
Copy link
Contributor

EWashb commented Oct 30, 2023

Added project to the OCTO DE board. Teams, POs, and VA leadership will look at this initiative-level ticket for updates when needed.

@EWashb EWashb removed this from the AP: Milestone 4 milestone Nov 2, 2023
@EWashb EWashb added roadmap-DMC Used for tracking DMC-level initiatives. Added by PO and removed roadmap-DMC Used for tracking DMC-level initiatives. Added by PO labels Nov 2, 2023
@EWashb
Copy link
Contributor

EWashb commented Nov 3, 2023

I tried to link this to the OCTO board, but I've decided to keep this specific initiative at the team level. Excuse the labeling and project noise.

@davidconlon
Copy link
Contributor

@EWashb can we take a look at the structure of these tickets. I think there is a different level of Fidelity that we want to be showing at the OCTO board level

@EWashb
Copy link
Contributor

EWashb commented Nov 3, 2023

@davidconlon I spoke with Erica yesterday and backed myself off of the previous attempt to link this. It should be removed now.

@timcosgrove
Copy link
Contributor

About to make significant changes to the structure of the initiative, so archiving the previous structure here.

Initiative state prior to changes April 1, 2024 ## Background

User Story or Problem Statement

North Stars

As a content editor,
I need a higher limit on content volume capacity,
So that I can publish any necessary content on VA.gov.

As a content editor,
I need published content to appear on VA.gov as quickly as possible,
So that I can get information to users in a timely manner.

As a VFS engineer,
I need to easily build frontend applications using CMS content,
So that I can display it to VA.gov users.

As a VA.gov user,
I need content on VA.gov to load reliably,
So that I can depend on VA.gov as a source of information.

Affected users and stakeholders

  • CMS Content Editors
  • Developers of VFS teams
  • VA.gov users
### Overall Initiative Goals
- [ ] Faster publishing time
- [ ] Increased content scaling capacity
- [ ] Reduced difficulty of VFS FE development
- [ ] Increased reuse of common FE libraries
- [ ] Fewer applications in the legacy content-build
- [ ] Content preview more closely matches production
- [ ] VA.gov user experience is equivalent or improved

Work board

Accelerated Publishing is managed via Github Projects. Our Github Project can be viewed here.

MVP Goals & Milestones

The current AP contract period of performance (PoP) is July 13, 2023 - April 30, 2024. Within the PoP, our aim is:

Build static pages for VA.gov with a React-based static generation system and deploy those to VA.gov production incrementally.

  • "Incrementally" here specifically means that we only build what is needed, based on what has content has been changed or added. The currently existing system builds and deploys all CMS content. This generates over 20,000 pages each time. This is the primary source of the slowness of the current system. We expect the biggest speed gains to come from only building and deploy content that has been added or updated.
  • "React-based" as a requirement is there to bring CMS content templating in line with the rest of VA.gov front-end and with modern practices.

The MVP for the current contract will be delivered in three milestones.

Milestone 1: Proof of complete HTML generation by the new system (Completed)

Completed: Nov 2023

The first milestone focuses on setting up our new build system for generating complete HTML from CMS content with full functional and design parity with the existing system. The first use case is Facility Stories and Story listings, demonstrated by these pages:
Story Listing: https://www.va.gov/northern-california-health-care/stories/
Story: https://www.va.gov/northern-california-health-care/stories/congressman-ami-bera-brought-valentine-cards-to-veterans/

During this period, work on Milestone 2 will also be proceeding; see below.

### Epics required Milestone 1 completion
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14465
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14739
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/10906

Milestone 2: HTML generated by the system is deployed to production, replacing the old system for that content

Estimated delivery: Jan 2023

The second milestone focuses on delivering the HTML content that is generated by the new system to production, and removing that content from being generated by the old system. This will include:

  • Setting up the build and deployment pipeline to move content to VA.gov
  • Ensuring that content produced by the old system and the new system coexist and interoperate without issue

It should be noted that at this stage, all content that is managed by the new system will be built with every build cycle, similar to how the current system does it. In other words, at this stage we will not just build content that has updated, but all content.

During this milestone period, work on Milestone 3 will also be proceeding; see below.

### Epics required Milestone 2 completion
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/10903
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14052
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14469
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15952
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14454
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14455
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14457

Milestone 3: HTML is built and deployed by the new system 'incrementally'; second product area is built and delivered by the new system

Estimated delivery: April 2023
The primary focus of the third Milestone is incremental publishing, as defined above. Content will be built in response to updates and additions in the CMS as needed, rather than all content being built every time. This is where we expect to see the most speed gains relative to publishing via the current system.

Additionally, we expect to deliver the second product area. Currently we are targeting Benefits Hub content, for example the collection of content re: Disability Compensation Benefits that is reached from this top-level page: https://www.va.gov/disability/ . We are targeting this product area due to its importance to Congressional mandates to provide important Federal content to people with limited English proficiency. It would be inefficient to implement multilingual support in the current system, and so we aim to move Benefits content to the new system and support multilingual publishing there.

### Epics for Milestone 3 completion
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14467
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14466
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14360
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15488
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15926
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16307
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16490
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16497

Post-MVP work

Finally, as we do not expect to migrate the full set of content supported by the CMS within the bounds of the current PoP, we will need to provide backlog for all remaining work and documentation about the system to support whichever team or teams do that migration. Additionally, there are some features of the publishing pipeline which we have identified as out-of-scope for the current PoP, but which would benefit the overall goal of providing content to Veterans as quickly as possible. Those will be backlogged as well.

### Post-MVP epics
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14468
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14712
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14456
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16308
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16306
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16515
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16516

Historical Sprint documents

Completed tasks by sprint report

### Historical Sprint documents
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17673
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17672
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17437
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17243
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17245
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16944
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16780
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16510
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16310
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16270
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16027
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15834
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15701
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15486
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15238
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15021
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14903
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14608
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14422

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accelerated Publishing Initiative Initiatives are collections of epics that drive toward a common goal defined within Crew's Objective team-drupal-cms Drupal CMS Team Initiatives. Used on OCTO DMC board
Projects
None yet
Development

No branches or pull requests

8 participants