feat(repository): migrateSchema APIs

Introduce new APIs to simplify database migrations: - A new interface `MigrateableRepository` describing repositores that know how to migrate their database schema. - A new Application-level method `app.migrateSchema()` provided by `RepositoryMixin` and running schema migration provided by all registered repositories under the hood. Also: - Implement `MigrateableRepository` in `DefaultCrudRepository` using autoupdate/automigrate APIs provided by legacy-juggler's DataSource class. - Simplify the instructions shown in `Database-migrations.db` - Add an example `migrate.ts` script to our Todo example to verify that the code snippet shown in the docs works as intended.
loopbackio · Nov 20, 2018 · 223f5bd · 223f5bd
1 parent f89d6ae
commit 223f5bd
Show file tree

Hide file tree

Showing 9 changed files with 266 additions and 106 deletions.
diff --git a/docs/site/Database-migrations.md b/docs/site/Database-migrations.md
@@ -20,123 +20,75 @@ LoopBack offers two ways to do this:
 - **Auto-update**: Change database schema objects if there is a difference
   between the objects and model definitions. Existing data will be kept.
 
-## Implementation Example
+{% include warning.html content="Auto-update will attempt to preserve data while
+updating the schema in your target database, but this is not guaranteed to be
+safe.
 
-Below is an example of how to implement
-[automigrate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-automigrate)
-and
-[autoupdate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-autoupdate),
-shown with the
-[TodoList](https://loopback.io/doc/en/lb4/todo-list-tutorial.html) example.
+Please check the documentation for your specific connector(s) for a detailed
+breakdown of behaviors for automigrate! " %}
 
-Create a new file **src/migrate.ts** and add the following import statement:
+## Examples
 
-```ts
-import {DataSource, Repository} from '@loopback/repository';
-```
+LoopBack applications are typically using `RepositoryMixin` to enhance the core
+`Application` class with additional repository-related APIs. One of such methods
+is `migrateSchema`, which iterates over all registered repositories and asks
+them to migrate their schema. Repositories that do not support schema migrations
+are silently skipped.
 
-Import your application and your repositories:
+### Auto-update database at start
 
-```ts
-import {TodoListApplication} from './index';
-import {TodoRepository, TodoListRepository} from './repositories';
-```
+To automatically update the database schema whenever the application is started,
+modify your main script to execute `app.migrateSchema()` after the application
+was bootstrapped (all repositories were registered) but before it is actually
+started.
 
-Create a function called _dsMigrate()_:
+{% include code-caption.html content="src/index.ts" %}
 
 ```ts
-export async function dsMigrate(app: TodoListApplication) {}
-```
-
-In the _dsMigrate()_ function, get your datasource and instantiate your
-repositories by retrieving them, so that the models are attached to the
-corresponding datasource:
-
-```ts
-const ds = await app.get<DataSource>('datasources.db');
-const todoRepo = await app.getRepository(TodoRepository);
-const todoListRepo = await app.getRepository(TodoListRepository);
-```
-
-Then, in the same function, call _automigrate()_:
-
-```ts
-await ds.automigrate();
-```
-
-This call to automigrate will migrate all the models attached to the datasource
-db. However if you want to only migrate some of your models, add the names of
-the classes in the first parameter:
-
-```ts
-// Migrate a single model
-ds.automigrate('Todo');
-```
-
-```ts
-// Migrate multiple models
-ds.automigrate(['Todo', 'TodoList']);
-```
-
-The implementation for _autoupdate()_ is similar. Create a new function
-_dsUpdate()_:
+export async function main(options: ApplicationConfig = {}) {
+  const app = new TodoListApplication(options);
+  await app.boot();
+  await app.migrateSchema();
+  await app.start();
 
-```ts
-export async function dsUpdate(app: TodoListApplication) {
-  const ds = await app.get<DataSource>('datasources.db');
-  const todoRepo = await app.getRepository(TodoRepository);
-  const todoListRepo = await app.getRepository(TodoListRepository);
+  const url = app.restServer.url;
+  console.log(`Server is running at ${url}`);
 
-  await ds.autoupdate();
+  return app;
 }
 ```
 
-The completed **src/migrate.ts** should look similar to this:
-
-```ts
-import {DataSource, Repository} from '@loopback/repository';
-import {TodoListApplication} from './index';
-import {TodoRepository, TodoListRepository} from './repositories';
-
-export async function dsMigrate(app: TodoListApplication) {
-  const ds = await app.get<DataSource>('datasources.db');
-  const todoRepo = await app.getRepository(TodoRepository);
-  const todoListRepo = await app.getRepository(TodoListRepository);
-
-  await ds.automigrate();
-}
-
-export async function dsUpdate(app: TodoListApplication) {
-  const ds = await app.get<DataSource>('datasources.db');
-  const todoRepo = await app.getRepository(TodoRepository);
-  const todoListRepo = await app.getRepository(TodoListRepository);
+### Auto-update the database explicitly
 
-  await ds.autoupdate();
-}
-```
+It's usually better to have more control about the database migration and
+trigger the updates explicitly. To do so, you can implement a custom script as
+shown below.
 
-Finally, in **src/index.ts**, import and call the _dsMigrate()_ or _dsUpdate()_
-function:
+{% include code-caption.html content="src/migrate.ts" %}
 
 ```ts
 import {TodoListApplication} from './application';
-import {ApplicationConfig} from '@loopback/core';
 
-// Import the functions from src/migrate.ts
-import {dsMigrate, dsUpdate} from './migrate';
+export async function migrate(args: string[]) {
+  const rebuild = args.includes('--rebuild');
+  console.log('Migrating schemas (%s)', rebuild ? 'rebuild' : 'update');
 
-export {TodoListApplication};
-
-export async function main(options: ApplicationConfig = {}) {
-  const app = new TodoListApplication(options);
+  const app = new TodoListApplication();
   await app.boot();
-  await app.start();
-
-  const url = app.restServer.url;
-  console.log(`Server is running at ${url}`);
-
-  // The call to dsMigrate(), or replace with dsUpdate()
-  await dsMigrate(app);
-  return app;
+  await app.migrateSchema({rebuild});
 }
+
+migrate(process.argv).catch(err => {
+  console.error('Cannot migrate database schema', err);
+  process.exit(1);
+});
 ```
+
+After you have compiled your application via `npm run build`, you can update
+your database by running `node dist/src/migrate` and rebuild it from scratch by
+running `node dist/src/migrate --rebuild`. It is also possible to save this
+commands as `npm` scripts in your `package.json` file.
+
+In the future, we would like to provide finer-grained control of database schema
+updates, learn more in the GitHub issue
+[#487 Database Migration Management Framework](https://github.com/strongloop/loopback-next/issues/487)
diff --git a/examples/todo/src/index.ts b/examples/todo/src/index.ts
@@ -9,6 +9,7 @@ import {ApplicationConfig} from '@loopback/core';
 export async function main(options: ApplicationConfig = {}) {
   const app = new TodoListApplication(options);
   await app.boot();
+  await app.migrateSchema();
   await app.start();
 
   const url = app.restServer.url;

diff --git a/examples/todo/src/migrate.ts b/examples/todo/src/migrate.ts
@@ -0,0 +1,20 @@
+// Copyright IBM Corp. 2017,2018. All Rights Reserved.
+// Node module: @loopback/example-todo
+// This file is licensed under the MIT License.
+// License text available at https://opensource.org/licenses/MIT
+
+import {TodoListApplication} from './application';
+
+export async function migrate(args: string[]) {
+  const rebuild = args.includes('--rebuild');
+  console.log('Migrating schemas (%s)', rebuild ? 'rebuild' : 'update');
+
+  const app = new TodoListApplication();
+  await app.boot();
+  await app.migrateSchema({rebuild});
+}
+
+migrate(process.argv).catch(err => {
+  console.error('Cannot migrate database schema', err);
+  process.exit(1);
+});
diff --git a/packages/repository/src/mixins/repository.mixin.ts b/packages/repository/src/mixins/repository.mixin.ts
@@ -3,11 +3,18 @@
 // This file is licensed under the MIT License.
 // License text available at https://opensource.org/licenses/MIT
 
-import {Class} from '../common-types';
-import {Repository} from '../repositories/repository';
-import {juggler} from '../repositories/legacy-juggler-bridge';
-import {Application} from '@loopback/core';
 import {BindingScope} from '@loopback/context';
+import {Application} from '@loopback/core';
+import * as debugFactory from 'debug';
+import {Class} from '../common-types';
+import {
+  isMigrateableRepository,
+  juggler,
+  Repository,
+  SchemaMigrationOptions,
+} from '../repositories';
+
+const debug = debugFactory('loopback:repository:mixin');
 
 /**
  * A mixin class for Application that creates a .repository()
@@ -163,6 +170,34 @@ export function RepositoryMixin<T extends Class<any>>(superClass: T) {
         }
       }
     }
+
+    /**
+     * Update or recreate the database schema for all repositories.
+     *
+     * **WARNING**: By default, `migrateSchema()` will attempt to preserve data
+     * while updating the schema in your target database, but this is not
+     * guaranteed to be safe.
+     *
+     * Please check the documentation for your specific connector(s) for
+     * a detailed breakdown of behaviors for automigrate!
+     *
+     * @param options Migration options, e.g. whether to update tables
+     * preserving data or rebuild everything from scratch.
+     */
+    async migrateSchema(options?: SchemaMigrationOptions): Promise<void> {
+      const repoBindings = this.findByTag('repository');
+
+      for (const b of repoBindings) {
+        const repo = await this.get(b.key);
+
+        if (isMigrateableRepository(repo)) {
+          debug('Migrating repository %s', b.key);
+          await repo.migrateSchema(options);
+        } else {
+          debug('Skipping migration of repository %s', b.key);
+        }
+      }
+    }
   };
 }
 
@@ -180,6 +215,7 @@ export interface ApplicationWithRepositories extends Application {
   ): void;
   component(component: Class<{}>): void;
   mountComponentRepositories(component: Class<{}>): void;
+  migrateSchema(options?: SchemaMigrationOptions): Promise<void>;
 }
 
 /**
@@ -293,4 +329,19 @@ export class RepositoryMixinDoc {
    * @param component The component to mount repositories of
    */
   mountComponentRepository(component: Class<{}>) {}
+
+  /**
+   * Update or recreate the database schema for all repositories.
+   *
+   * **WARNING**: By default, `migrateSchema()` will attempt to preserve data
+   * while updating the schema in your target database, but this is not
+   * guaranteed to be safe.
+   *
+   * Please check the documentation for your specific connector(s) for
+   * a detailed breakdown of behaviors for automigrate!
+   *
+   * @param options Migration options, e.g. whether to update tables
+   * preserving data or rebuild everything from scratch.
+   */
+  async migrateSchema(options?: SchemaMigrationOptions): Promise<void> {}
 }
diff --git a/packages/repository/src/repositories/index.ts b/packages/repository/src/repositories/index.ts
@@ -8,3 +8,4 @@ export * from './legacy-juggler-bridge';
 export * from './kv.repository.bridge';
 export * from './repository';
 export * from './constraint-utils';
+export * from './migrateable.repository';
diff --git a/packages/repository/src/repositories/legacy-juggler-bridge.ts b/packages/repository/src/repositories/legacy-juggler-bridge.ts
@@ -28,6 +28,10 @@ import {
 } from '../relations';
 import {resolveType} from '../type-resolver';
 import {EntityCrudRepository} from './repository';
+import {
+  MigrateableRepository,
+  SchemaMigrationOptions,
+} from './migrateable.repository';
 
 export namespace juggler {
   export import DataSource = legacy.DataSource;
@@ -75,7 +79,7 @@ export function ensurePromise<T>(p: legacy.PromiseOrVoid<T>): Promise<T> {
  * and data source
  */
 export class DefaultCrudRepository<T extends Entity, ID>
-  implements EntityCrudRepository<T, ID> {
+  implements EntityCrudRepository<T, ID>, MigrateableRepository<T> {
   modelClass: juggler.PersistedModelClass;
 
   /**
@@ -332,4 +336,9 @@ export class DefaultCrudRepository<T extends Entity, ID>
   protected toEntities(models: juggler.PersistedModel[]): T[] {
     return models.map(m => this.toEntity(m));
   }
+
+  async migrateSchema(options?: SchemaMigrationOptions): Promise<void> {
+    const operation = options && options.rebuild ? 'automigrate' : 'autoupdate';
+    await this.dataSource[operation](this.modelClass.modelName);
+  }
 }
diff --git a/packages/repository/src/repositories/migrateable.repository.ts b/packages/repository/src/repositories/migrateable.repository.ts
@@ -0,0 +1,49 @@
+// Copyright IBM Corp. 2017,2018. All Rights Reserved.
+// Node module: @loopback/repository
+// This file is licensed under the MIT License.
+// License text available at https://opensource.org/licenses/MIT
+
+import {Options} from '../common-types';
+import {Model} from '../model';
+import {Repository} from './repository';
+
+export interface SchemaMigrationOptions extends Options {
+  /**
+   * When set to true, schema migration will drop existing tables and recreate
+   * them from scratch, removing any existing data along the way.
+   */
+  rebuild?: boolean;
+}
+
+/**
+ * A repository capable of database schema migration (auto-update/auto-migrate).
+ */
+export interface MigrateableRepository<T extends Model> extends Repository<T> {
+  /**
+   * Update or recreate the database schema.
+   *
+   * **WARNING**: By default, `migrateSchema()` will attempt to preserve data
+   * while updating the schema in your target database, but this is not
+   * guaranteed to be safe.
+   *
+   * Please check the documentation for your specific connector(s) for
+   * a detailed breakdown of behaviors for automigrate!
+   *
+   * @param options Migration options, e.g. whether to update tables
+   * preserving data or rebuild everything from scratch.
+   */
+  migrateSchema(options?: SchemaMigrationOptions): Promise<void>;
+}
+
+/**
+ * A type guard for detecting repositories implementing MigratableRepository
+ * interface.
+ *
+ * @param repo The repository instance to check.
+ */
+export function isMigrateableRepository<T extends Model = Model>(
+  // tslint:disable-next-line:no-any
+  repo: any,
+): repo is MigrateableRepository<T> {
+  return typeof repo.migrateSchema === 'function';
+}