Skip to content

Commit

Permalink
feat(repository): migrateSchema APIs
Browse files Browse the repository at this point in the history 
Introduce a new Application-level method `app.migrateSchema()` provided
by `RepositoryMixin`, this method executes schema migration as
implemented by datasources registered with the application.

Simplify the instructions shown in `Database-migrations.db`

Add an example `migrate.ts` script to our Todo example to verify
that the code snippet shown in the docs works as intended.
  • Loading branch information
@bajtos
bajtos committed Nov 29, 2018
1 parent 7be76a4 commit ad0229b
Show file tree
Hide file tree
Showing 5 changed files with 299 additions and 99 deletions.
168 changes: 78 additions & 90 deletions docs/site/Database-migrations.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,123 +20,111 @@ LoopBack offers two ways to do this:
- **Auto-update**: Change database schema objects if there is a difference
between the objects and model definitions. Existing data will be kept.

## Implementation Example
{% include warning.html content="Auto-update will attempt to preserve data while
updating the schema in your target database, but this is not guaranteed to be
safe.

Below is an example of how to implement
[automigrate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-automigrate)
and
[autoupdate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-autoupdate),
shown with the
[TodoList](https://loopback.io/doc/en/lb4/todo-list-tutorial.html) example.
Please check the documentation for your specific connector(s) for a detailed
breakdown of behaviors for automigrate! " %}

Create a new file **src/migrate.ts** and add the following import statement:
## Examples

```ts
import {DataSource, Repository} from '@loopback/repository';
```

Import your application and your repositories:
LoopBack applications are typically using `RepositoryMixin` to enhance the core
`Application` class with additional repository-related APIs. One of such methods
is `migrateSchema`, which iterates over all registered repositories and asks
them to migrate their schema. Repositories that do not support schema migrations
are silently skipped.

```ts
import {TodoListApplication} from './index';
import {TodoRepository, TodoListRepository} from './repositories';
```
In the future, we would like to provide finer-grained control of database schema
updates, learn more in the GitHub issue
[#487 Database Migration Management Framework](https://github.com/strongloop/loopback-next/issues/487)

Create a function called _dsMigrate()_:
### Auto-update database at start

```ts
export async function dsMigrate(app: TodoListApplication) {}
```

In the _dsMigrate()_ function, get your datasource and instantiate your
repositories by retrieving them, so that the models are attached to the
corresponding datasource:

```ts
const ds = await app.get<DataSource>('datasources.db');
const todoRepo = await app.getRepository(TodoRepository);
const todoListRepo = await app.getRepository(TodoListRepository);
```
To automatically update the database schema whenever the application is started,
modify your main script to execute `app.migrateSchema()` after the application
was bootstrapped (all repositories were registered) but before it is actually
started.

Then, in the same function, call _automigrate()_:
{% include code-caption.html content="src/index.ts" %}

```ts
await ds.automigrate();
```

This call to automigrate will migrate all the models attached to the datasource
db. However if you want to only migrate some of your models, add the names of
the classes in the first parameter:
export async function main(options: ApplicationConfig = {}) {
const app = new TodoListApplication(options);
await app.boot();
await app.migrateSchema();
await app.start();

```ts
// Migrate a single model
ds.automigrate('Todo');
```
const url = app.restServer.url;
console.log(`Server is running at ${url}`);

```ts
// Migrate multiple models
ds.automigrate(['Todo', 'TodoList']);
return app;
}
```

The implementation for _autoupdate()_ is similar. Create a new function
_dsUpdate()_:

```ts
export async function dsUpdate(app: TodoListApplication) {
const ds = await app.get<DataSource>('datasources.db');
const todoRepo = await app.getRepository(TodoRepository);
const todoListRepo = await app.getRepository(TodoListRepository);
### Auto-update the database explicitly

await ds.autoupdate();
}
```
It's usually better to have more control about the database migration and
trigger the updates explicitly. To do so, you can implement a custom script as
shown below.

The completed **src/migrate.ts** should look similar to this:
{% include code-caption.html content="src/migrate.ts" %}

```ts
import {DataSource, Repository} from '@loopback/repository';
import {TodoListApplication} from './index';
import {TodoRepository, TodoListRepository} from './repositories';
import {TodoListApplication} from './application';

export async function dsMigrate(app: TodoListApplication) {
const ds = await app.get<DataSource>('datasources.db');
const todoRepo = await app.getRepository(TodoRepository);
const todoListRepo = await app.getRepository(TodoListRepository);
export async function migrate(args: string[]) {
const dropExistingTables = args.includes('--rebuild');
console.log('Migrating schemas (%s)', rebuild ? 'rebuild' : 'update');

await ds.automigrate();
const app = new TodoListApplication();
await app.boot();
await app.migrateSchema({dropExistingTables});
}

export async function dsUpdate(app: TodoListApplication) {
const ds = await app.get<DataSource>('datasources.db');
const todoRepo = await app.getRepository(TodoRepository);
const todoListRepo = await app.getRepository(TodoListRepository);

await ds.autoupdate();
}
migrate(process.argv).catch(err => {
console.error('Cannot migrate database schema', err);
process.exit(1);
});
```

Finally, in **src/index.ts**, import and call the _dsMigrate()_ or _dsUpdate()_
function:
After you have compiled your application via `npm run build`, you can update
your database by running `node dist/src/migrate` and rebuild it from scratch by
running `node dist/src/migrate --rebuild`. It is also possible to save this
commands as `npm` scripts in your `package.json` file.

```ts
import {TodoListApplication} from './application';
import {ApplicationConfig} from '@loopback/core';

// Import the functions from src/migrate.ts
import {dsMigrate, dsUpdate} from './migrate';
### Implement additional migration steps

export {TodoListApplication};
In some scenarios, the application may need to define additional schema
constraints or seed the database with predefined model instances. This can be
achieved by overriding the `migrateSchema` method provided by the mixin.

export async function main(options: ApplicationConfig = {}) {
const app = new TodoListApplication(options);
await app.boot();
await app.start();
The example below shows how to do so in our Todo example application.

const url = app.restServer.url;
console.log(`Server is running at ${url}`);
{% include code-caption.html content="src/application.ts" %}

// The call to dsMigrate(), or replace with dsUpdate()
await dsMigrate(app);
return app;
```ts
import {TodoRepository} from './repositories';
// skipped: other imports

export class TodoListApplication extends BootMixin(
ServiceMixin(RepositoryMixin(RestApplication)),
) {
// skipped: the constructor, etc.

async migrateSchema(options?: SchemaMigrationOptions) {
// 1. Run migration scripts provided by connectors
await super.migrateSchema(options);

// 2. Make further changes. When creating predefined model instances,
// handle the case when these instances already exist.
const todoRepo = await this.getRepository(TodoRepository);
const found = await todoRepo.findOne({where: {title: 'welcome'}});
if (found) {
todoRepo.updateById(found.id, {isComplete: false});
} else {
await todoRepo.create({title: 'welcome', isComplete: false});
}
}
}
```
25 changes: 25 additions & 0 deletions examples/todo/src/migrate.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Copyright IBM Corp. 2017,2018. All Rights Reserved.
// Node module: @loopback/example-todo
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

import {TodoListApplication} from './application';

export async function migrate(args: string[]) {
const existingSchema = args.includes('--rebuild') ? 'drop' : 'alter';
console.log('Migrating schemas (%s existing schema)', existingSchema);

const app = new TodoListApplication();
await app.boot();
await app.migrateSchema({existingSchema});

// Connectors usually keep a pool of opened connections,
// this keeps the process running even after all work is done.
// We need to exit explicitly.
process.exit(0);
}

migrate(process.argv).catch(err => {
console.error('Cannot migrate database schema', err);
process.exit(1);
});
20 changes: 19 additions & 1 deletion packages/repository/src/datasource.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

import {AnyObject} from './common-types';
import {AnyObject, Options} from './common-types';
import {Connector} from './connectors';

/**
Expand All @@ -17,3 +17,21 @@ export interface DataSource {
// tslint:disable-next-line:no-any
[property: string]: any; // Other properties that vary by connectors
}

export interface SchemaMigrationOptions extends Options {
/**
* When set to 'drop', schema migration will drop existing tables and recreate
* them from scratch, removing any existing data along the way.
*
* When set to 'alter', schema migration will try to preserve current schema
* and data, and perform a non-destructive incremental update.
*/
existingSchema?: 'drop' | 'alter';

/**
* List of model names to migrate.
*
* By default, all models are migrated.
*/
models?: string[];
}
67 changes: 63 additions & 4 deletions packages/repository/src/mixins/repository.mixin.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

import {Class} from '../common-types';
import {Repository} from '../repositories/repository';
import {juggler} from '../repositories/legacy-juggler-bridge';
import {BindingScope, Binding} from '@loopback/context';
import {Application} from '@loopback/core';
import {BindingScope} from '@loopback/context';
import * as debugFactory from 'debug';
import {Class} from '../common-types';
import {juggler, Repository} from '../repositories';
import {SchemaMigrationOptions} from '../datasource';

const debug = debugFactory('loopback:repository:mixin');

/**
* A mixin class for Application that creates a .repository()
Expand Down Expand Up @@ -163,6 +166,46 @@ export function RepositoryMixin<T extends Class<any>>(superClass: T) {
}
}
}

/**
* Update or recreate the database schema for all repositories.
*
* **WARNING**: By default, `migrateSchema()` will attempt to preserve data
* while updating the schema in your target database, but this is not
* guaranteed to be safe.
*
* Please check the documentation for your specific connector(s) for
* a detailed breakdown of behaviors for automigrate!
*
* @param options Migration options, e.g. whether to update tables
* preserving data or rebuild everything from scratch.
*/
async migrateSchema(options: SchemaMigrationOptions = {}): Promise<void> {
const operation =
options.existingSchema === 'drop' ? 'automigrate' : 'autoupdate';

// Instantiate all repositories to ensure models are registered & attached
// to their datasources
const repoBindings: Readonly<Binding<unknown>>[] = this.findByTag(
'repository',
);
await Promise.all(repoBindings.map(b => this.get(b.key)));

// Look up all datasources and update/migrate schemas one by one
const dsBindings: Readonly<Binding<object>>[] = this.findByTag(
'datasource',
);
for (const b of dsBindings) {
const ds = await this.get(b.key);

if (operation in ds && typeof ds[operation] === 'function') {
debug('Migrating dataSource %s', b.key);
await ds[operation](options.models);
} else {
debug('Skipping migration of dataSource %s', b.key);
}
}
}
};
}

Expand All @@ -180,6 +223,7 @@ export interface ApplicationWithRepositories extends Application {
): void;
component(component: Class<{}>): void;
mountComponentRepositories(component: Class<{}>): void;
migrateSchema(options?: SchemaMigrationOptions): Promise<void>;
}

/**
Expand Down Expand Up @@ -293,4 +337,19 @@ export class RepositoryMixinDoc {
* @param component The component to mount repositories of
*/
mountComponentRepository(component: Class<{}>) {}

/**
* Update or recreate the database schema for all repositories.
*
* **WARNING**: By default, `migrateSchema()` will attempt to preserve data
* while updating the schema in your target database, but this is not
* guaranteed to be safe.
*
* Please check the documentation for your specific connector(s) for
* a detailed breakdown of behaviors for automigrate!
*
* @param options Migration options, e.g. whether to update tables
* preserving data or rebuild everything from scratch.
*/
async migrateSchema(options?: SchemaMigrationOptions): Promise<void> {}
}
Loading

0 comments on commit ad0229b

Please sign in to comment.