Case Study: Using NEXT.js in Zendro

[davelsan]

Summary

The Zendro frontend, as part of our data warehouse platform, is a minimal implementation of a customizable user interface to send CRUD requests to our GraphQL endpoint. This discussion can be thought of as a development log, where I would like to describe the reasons why we chose NEXT.js and how it is working for us, in the context of this project.

Our original goal was to develop a versatile interface that could be adapted to the different use cases of our subsidiary projects. We wanted to use a framework or set of libraries capable of generating an application that was either:

• Static-site generated (SSG).
• Server-side rendered (SSR).
• Or a hybrid of both strategies on a per-page basis (on demand).

In the following comments, I will try to outline the key aspects that were relevant to us during the design phase, and how NEXT helps us address them.

Table of Contents

• Code Generation
• Routing
• Authentication (TODO)
• Customization
• Private Data Sources
• Performance: Bundle Size, SEO, Metrics

[davelsan]

Code Generation

Zendro is a project designed to bootstrap a fullstack application tailored to user-provided data models. Because certain parts of the frontend need to be generated at build time, a code-generator step is necessary. This allows us to create reusable code for queries, mutations, and more.

In addition, part of this generated code should offer the possibility of customization with minimal effort, and successive builds should be able to re-import and use the modified code. Where editing of the generated core is impractical, overrides should be supported, so a certain degree of flexibility was required.

Many of these aspects are already addressed by NEXT design philosophy, allowing us to focus on producing the application that we want to deliver.

A page can be generated for each model

Page generation was one of the first aspects we needed to focus on. We wanted to avoid generating repetitive components for each data model, potentially bloating the application with almost identical code and exponentially increasing the complexity of the default application.

Through the use of the getStaticPaths and getStaticProps APIs, each model could reuse a single template file to generate its own set of routes and static properties.

These APIs proved invaluable in the early stages of development. In our case, the biggest benefit was keeping the core of the application contained to a few page components that could easily be overridden in later projects when necessary.

Furthermore, because we had the ability to switch into a server-side rendering strategy by renaming the latter function as getServerSideProps, these APIs took care of most of the flexibility we want to achieve.

Static props are not always the solution

However, later on, we found that getStaticProps was not always the best solution to provide model data to our pages, and eventually this responsibility was shared with a more appropriate strategy for this use case.

Because the application we are building is a CRUD interface, it is often required that a certain amount of interoperability exists between pages (models). For example, in the associations' management interface, where we needed to fetch the schemas from other models at build time.

This scenario meant that building this data on a per page basis often led to code duplication by adding the same fetch logic in various pages or potentially having a burst of API calls during the build stage. We were concerned that this behavior would not be optimal in sites with a large count of generated pages.

To address this use case, and until getStaticProps is available in _app, we leveraged webpack loaders to pre-evaluate model data once at bundle time, and then make it available site-wide. At first, we used val-loader via a custom webpack configuration, and later we migrated to a library called next-plugin-preval. Though this is likely not yet a definitive solution.

[davelsan]

Routing

Our use case required support for standard router features, such as dynamic routes, query parameters, and redirects. At the same time, we wanted to keep in mind potential later customizations, with the idea that the user should not require to modify core files much or at all.

NEXT offered a very convenient file-base routing API, that for the most part simplified our job and laid out the groundwork for later customizations without resorting to extraneous code-generation strategies.

Early, a hybrid of statically generated and client-side site

In our CRUD interface frontend, dynamic routes (i.e. model pages) are known at build time and therefore can be easily generated via getStaticPaths.

Because routes automatically mirror the structure of the pages folder, we were able to reduce the core code by leveraging dynamic routes using the brackets notation (e.g. [route], or [...route] for variadic paths).

However, in the early stages of a project, the contents of each page (i.e. the actual model records) are added and updated frequently, so it would not be optimal to statically generate this data. For Zendro, this meant that by default we could only pre-generate a scaffold of the models' layout (along with any other static pages in the application), while the actual content components (tables, forms) were fully client-side and imported via dynamic.

An simplified example can be seen below:

# simplified for clarity
pages
├── [model]
│   ├── index.tsx
│   ├── details.tsx
│   ├── edit.tsx
│   └── new.tsx
└── index.tsx

Later, even when statically generated, we can keep it dynamic

When a data warehouse matures and is expected to receive less frequent or no updates, perhaps only additions, we can leverage the full power of static site generation.

If we were to implement Zendro in a mature project, we could then statically generate the full pages using a fallback strategy, with potentially some revalidation interval set. This would open up new features, such as accessing cached content (via CDN) even if the main server is down, or the ability to keep using the site while the user is offline (e.g. as a PWA).

For this, the site must be deployed using a Node.js server (e.g. by running next start) with fallback: true. Then, requesting a dynamic page (e.g. [model] or [record] below) for the first time will trigger its generation. If a validate interval is set, the first request after that interval will trigger regeneration.

Moreover, we could even build the application partially by generating only those most frequently requested models, then deploy via next start. New static pages would be generated at runtime as requested, and even as new models or records are added (Incremental Static Re-generation).

# simplified for clarity
pages
├── [model]
│   ├── index.tsx
│   └── details
│       └── [record].tsx
└── index.tsx

[davelsan]

Customization

As we briefly mentioned in the routing section, because Zendro implements a generic user interface for CRUD requests, it is expected that in subsidiary projects we will want to at least modify pages to add visualizations or tailor the user experience for specific models.

In these projects, feature requests and requirements may be quite different from what we initially conceived when designing the default Zendro experience. It was important for us to simplify customization as much as possible, to maintain a healthy developer experience.

Route overrides simplify customization

Here is where overriding a dynamic route by simply creating a new file in the pages folder with the desired name is excellent for us. This makes customization very easy to implement.

For example, by adding a books folder with a custom index.tsx file within, we are already customizing the model page. No other changes are required. All links in the application that point to /books will send the user to the new page, while keeping the default routes inside /book/... intact.

# simplified for clarity
# overriding `index.tsx` for the `books` model
pages
├── [model]
│   ├── index.tsx
│   ├── details.tsx
│   ├── edit.tsx
│   └── new.tsx
├── books
│   └── index.tsx
└── index.tsx

Non-developers can easily contribute content

In the projects that we implement Zendro, many contributors may not be comfortable with JavaScript. However, they can still contribute to the site by adding very useful and informative content.

This is often the case in scientific projects, where users are accustomed to analyze data and generate custom visualizations using R or python. These content creators can submit PRs with Markdown pages. Moreover, a workflow could be established where a developer creates beautiful components that can easily be added within MDX files.

[davelsan]

Private Data Sources

A reasonable use case for Zendro is to handle potentially sensible data, like patient history, or unpublished results. In these situations, this information may not be available via public APIs.

Server-side rendering can be used to request from private APIs.

Because NEXT supports server-side rendering out of the box, any page can be easily switched to query an internal resource not exposed to the public.

Within the getServerSideProps function the full Node.js APIs are available, so we could directly query a separate database using something like knex, and nothing will ever be exposed to the browser. This also applies to getStaticPaths and getStaticProps. The database can act as a source of static data, which is pulled during the initial build.

To further simplify the implementation, NEXT provides a special api folder within pages that can be used to store these server-side functions.

[davelsan]

Performance: Bundle Size, SEO, Metrics

Building a fully static application that can be deployed to any static hosting site (e.g. Github Pages, Netlify), then cached and served by a CDN was an important feature for us. In fact, one of the requirements for Zendro was that by default it could be served statically (using next export).

Zendro may be more than a CRUD application

Normally, projects that only use the CRUD interface provided with default Zendro should not necessarily benefit from pre-rendering pages. However, thanks to easy customization and extensibility, the normal use case of our platform could quickly evolve into a full project site that benefits from (and requires) improved SEO and accessibility.

For us, at least some of our projects required a home page with the project overview, a list of partners, and useful links. To put it bluntly, it is a market where we want our partners to succeed over their competitors. Having our static pages pre-rendered by default facilitates this task.

In the case of the content that is by default served as client-only (e.g. model data), we also realized that mature projects may switch to a different deployment strategy, as discussed in the Code Generator section, making this feature even more important.

Any project benefits from code splitting

Code-splitting is something we wanted to implement one way or another. Our concern was that, in our related projects, as feature requests grew, dependencies and bundle size could increase significantly, thus degrading the user experience.

Code splitting each route by default into separate chunks, literally without our intervention, and additionally using an improved code-splitting strategy that could potentially reduce bundle size, was a big selling point for us.

References

Pre-rendering is normally related to improved SEO and time to first contentful paint (FCP), while code-splitting usually improves time to interactive (TTI). More about these metrics at web.dev.

Both pre-rendering and code-splitting, when properly implemented, can potentially reduce bundle size, as less JavaScript is required to be sent and run in the browser.

• Code-splitting: each route (page) is separated into the minimal JavaScript chunks it is required to function. When a client requests a page, only the relevant chunks are sent to the browser.
• Pre-rendering: each page's HTML and CSS is rendered during the build step. When the page is requested, the pre-rendered data is sent with the initial packet, then the JavaScript bundle is hydrated in the browser and the page becomes interactive.