Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistency issue in LDA model during worker-side data migration #1222

Open
wynot12 opened this issue Jul 27, 2017 · 1 comment
Open

Consistency issue in LDA model during worker-side data migration #1222

wynot12 opened this issue Jul 27, 2017 · 1 comment

Comments

@wynot12
Copy link
Contributor

wynot12 commented Jul 27, 2017

When migrating worker-side data in LDA app, we have a problem that model is damaged during migration. It's because some of our apps have also partial models in the input table.

However, ET/EM assumes that worker-side table has only immutable input data. As a result, we design them not to care about worker-side table data consistency.

To resolve this problem, we need to provide a consistency mechanism for worker-side tables.
Or we may change LDA app to not to use local cache or cope with damage in local model.

@wynot12
Copy link
Contributor Author

wynot12 commented Nov 15, 2017

Need to fix it, since LDA is a primary workload in multi-job situation.

@wynot12 wynot12 added the bug label Nov 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant