Update via webUI from 9.1.6 to 10.0.2 via webUI daily channel with encryption enabled. The default encryption module gets disabled

[davitol]

Using this PR for updater-server: owncloud/administration#129

owncloud.log

{"reqId":"j678md2CQBsZeb99Fxyo","level":3,"time":"2017-06-01T10:37:42+00:00","remoteAddr":"82.159.139.58","user":"aaliyah_beer","app":"no app in context","method":"POST","url":"\/index.php\/login","message":"Exception: {\"Exception\":\"OC\\\\Encryption\\\\Exceptions\\\\ModuleDoesNotExistsException\",\"Message\":\"Module with id: OC_DEFAULT_MODULE does not exist.\",\"Code\":0,\"Trace\":\"#0 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(952): OC\\\\Encryption\\\\Manager->getEncryptionModule('OC_DEFAULT_MODU...')\\n#1 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(517): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->getEncryptionModule('files_versions\\\/...')\\n#2 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(492): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->fixUnencryptedSize('files_versions\\\/...', 216071, 216071)\\n#3 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(184): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->verifyUnencryptedSize('files_versions\\\/...', 216071)\\n#4 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Checksum.php(171): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->getMetaData('files_versions\\\/...')\\n#5 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(577): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Checksum->getMetaData('files_versions\\\/...')\\n#6 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(113): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->getMetaData('files_versions\\\/...')\\n#7 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(145): OC\\\\Files\\\\Cache\\\\Scanner->getData('files_versions\\\/...')\\n#8 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(409): OC\\\\Files\\\\Cache\\\\Scanner->scanFile('files_versions\\\/...', 1, '148', Object(OC\\\\Files\\\\Cache\\\\CacheEntry), true)\\n#9 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(377): OC\\\\Files\\\\Cache\\\\Scanner->handleChildren('files_versions\\\/...', true, 1, '148', true, 462485)\\n#10 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(380): OC\\\\Files\\\\Cache\\\\Scanner->scanChildren('files_versions\\\/...', true, 1, '148', true)\\n#11 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(380): OC\\\\Files\\\\Cache\\\\Scanner->scanChildren('files_versions', true, 1, '144', true)\\n#12 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Scanner.php(311): OC\\\\Files\\\\Cache\\\\Scanner->scanChildren('', true, 1, '28', true)\\n#13 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/util.php(404): OC\\\\Files\\\\Cache\\\\Scanner->scan('', true)\\n#14 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/User\\\/Session.php(412): OC_Util::copySkeleton('aaliyah_beer', Object(OC\\\\Files\\\\Node\\\\Folder))\\n#15 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/User\\\/Session.php(477): OC\\\\User\\\\Session->prepareUserLogin(true)\\n#16 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/User\\\/Session.php(299): OC\\\\User\\\\Session->loginWithPassword(*** sensitive parameters replaced ***)\\n#17 \\\/opt\\\/owncloud\\\/core\\\/Controller\\\/LoginController.php(205): OC\\\\User\\\\Session->login(*** sensitive parameters replaced ***)\\n#18 [internal function]: OC\\\\Core\\\\Controller\\\\LoginController->tryLogin(*** sensitive parameters replaced ***)\\n#19 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(159): call_user_func_array(Array, Array)\\n#20 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(89): OC\\\\AppFramework\\\\Http\\\\Dispatcher->executeController(Object(OC\\\\Core\\\\Controller\\\\LoginController), 'tryLogin')\\n#21 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/App.php(98): OC\\\\AppFramework\\\\Http\\\\Dispatcher->dispatch(Object(OC\\\\Core\\\\Controller\\\\LoginController), 'tryLogin')\\n#22 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Routing\\\/RouteActionHandler.php(46): OC\\\\AppFramework\\\\App::main('LoginController', 'tryLogin', Object(OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer), Array)\\n#23 [internal function]: OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler->__invoke(Array)\\n#24 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Route\\\/Router.php(299): call_user_func(Object(OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler), Array)\\n#25 \\\/opt\\\/owncloud\\\/lib\\\/base.php(918): OC\\\\Route\\\\Router->match('\\\/login')\\n#26 \\\/opt\\\/owncloud\\\/index.php(49): OC::handleRequest()\\n#27 {main}\",\"File\":\"\\\/opt\\\/owncloud\\\/lib\\\/private\\\/Encryption\\\/Manager.php\",\"Line\":198}”}

Enabled:
  - activity: 2.3.4
  - comments: 0.3.0
  - configreport: 0.1.1
  - dav: 0.2.9
  - federatedfilesharing: 0.3.0
  - federation: 0.1.0
  - files: 1.5.1
  - files_external: 0.7.0
  - files_pdfviewer: 0.8.2
  - files_sharing: 0.10.0
  - files_texteditor: 2.2
  - files_trashbin: 0.9.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 16.0.2
  - market: 0.1.0
  - notifications: 0.3.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.1
  - updatenotification: 0.2.1
  - user_ldap: 0.9.1
Disabled:
  - encryption
  - external
  - files_antivirus
  - theme-example
  - user_external

WorkAround: As soon as it is enabled manually again, everything works

@VicDeo @PVince81

[davitol]

Scenario 2. Test update from 9.1.6 to 10.0.2 without LDAP app enabled

Retested in order to check if it could be maybe related to LDAP users only, so this scenario has no LDAP app enable. Using regular users, the server stucks in maintenance mode trying to update from 9.1.6 to 10.0.2

{"reqId":"ixYlNzFL188gPDOUq3hl","level":3,"time":"2017-06-01T11:35:39+00:00","remoteAddr":"82.159.139.58","user":"--","app":"index","method":"GET","url":"\/index.php\/core\/js\/oc.js?v=36fe55ecd66e668f9d68770bbce3594d","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\TableNotFoundException\",\"Message\":\"An exception occurred while executing 'SELECT * FROM `accounts` WHERE `lower_user_id` = :dcValue1' with params [\\\"\\\"]:\\n\\nSQLSTATE[42S02]: Base table or view not found: 1146 Table 'owncloud.accounts' doesn't exist\",\"Code\":0,\"Trace\":\"#0 \\\/opt\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/opt\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Statement.php(177): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOMySql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT * FROM `...', Array)\\n#2 \\\/opt\\\/owncloud\\\/lib\\\/public\\\/AppFramework\\\/Db\\\/Mapper.php(252): Doctrine\\\\DBAL\\\\Statement->execute()\\n#3 \\\/opt\\\/owncloud\\\/lib\\\/public\\\/AppFramework\\\/Db\\\/Mapper.php(285): OCP\\\\AppFramework\\\\Db\\\\Mapper->execute('SELECT * FROM `...', Array, NULL, NULL)\\n#4 \\\/opt\\\/owncloud\\\/lib\\\/public\\\/AppFramework\\\/Db\\\/Mapper.php(379): OCP\\\\AppFramework\\\\Db\\\\Mapper->findOneQuery('SELECT * FROM `...', Array, NULL, NULL)\\n#5 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/User\\\/AccountMapper.php(128): OCP\\\\AppFramework\\\\Db\\\\Mapper->findEntity('SELECT * FROM `...', Array)\\n#6 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/User\\\/Manager.php(158): OC\\\\User\\\\AccountMapper->getByUid(false)\\n#7 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Group\\\/Manager.php(472): OC\\\\User\\\\Manager->get(false)\\n#8 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/user.php(333): OC\\\\Group\\\\Manager->inGroup(false, 'admin')\\n#9 \\\/opt\\\/owncloud\\\/core\\\/js\\\/config.php(75): OC_User::isAdminUser(false)\\n#10 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Route\\\/Route.php(155) : runtime-created function(1): require_once('\\\/opt\\\/owncloud\\\/c...')\\n#11 [internal function]: __lambda_func()\\n#12 \\\/opt\\\/owncloud\\\/lib\\\/private\\\/Route\\\/Router.php(299): call_user_func('\\\\x00lambda_1216', Array)\\n#13 \\\/opt\\\/owncloud\\\/lib\\\/base.php(918): OC\\\\Route\\\\Router->match('\\\/core\\\/js\\\/oc.js')\\n#14 \\\/opt\\\/owncloud\\\/index.php(49): OC::handleRequest()\\n#15 {main}\",\"File\":\"\\\/opt\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractMySQLDriver.php\",\"Line\":53}"}

So I run sudo -u www-data ./occ maintenance:mode --offvia CLI and refresh the browser:

Finally the update went fine with regular user and the Default Encryption Module app kept enabled after the update.

[PVince81]

ouch... Base table or view not found: 1146 Table 'owncloud.accounts' doesn't exist
what does the web updater do that occ upgrade doesn't ?

[PVince81]

@davitol were you logged in as an LDAP user who is also an admin ?
Maybe the updater is using the session where it shouldn't.

[davitol]

@PVince81 The admin user is a regular user. No admins between LDAP users

[VicDeo]

@PVince81 >Base table or view not found: 1146 Table 'owncloud.accounts' doesn't exist

are you updating to 10.0.2?
it should be fixed with owncloud/core#27965

[davitol]

@VicDeo Yes, updating to 10.0.2 I get these results

[VicDeo]

wow, owncloud/core#27965 fixes the issue for the user with
uid === 'null'

in your log I see another user with uid === 'false' (boolean)
I guess we will have users 'Array' or 'StdObject' in future...

[VicDeo]

this log entry seems to be unrelated to the failure btw.
it happened while requesting a js file: /index.php/core/js/oc.js?v=36fe55ecd66e668f9d68770bbce3594d

[VicDeo]

Can't reproduce for 9.1.6->10.0.2 via web.
The only difference is disabled (not installed) user_ldap:

916> php occ app:list
Enabled:
  - activity: 2.3.4
  - comments: 0.3.0
  - configreport: 0.1.1
  - dav: 0.2.9
  - encryption: 1.3.0
  - federatedfilesharing: 0.3.0
  - federation: 0.1.0
  - files: 1.5.1
  - files_external: 0.7.0
  - files_pdfviewer: 0.8.2
  - files_sharing: 0.10.0
  - files_texteditor: 2.2
  - files_trashbin: 0.9.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 16.0.2
  - market: 0.1.0
  - notifications: 0.3.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.1
  - updatenotification: 0.2.1
Disabled:
  - external
  - files_antivirus
  - theme-example
  - user_external

will check with enabled user_ldap

[VicDeo]

... and encryption app left enabled as expected...

[VicDeo]

@davitol I hijacked one of your instances.

The real reason seems to be related to the marketplace. Most likely it is not available at some point:

{"reqId":"WB4p3WqOLIacwbGakQ0G",
"level":3,
"time":"2017-06-01T11:34:16+00:00",
"remoteAddr":"::1",
"user":"--",
"app":"core",
"method":"POST",
"url":"\/index.php\/occ\/upgrade",
"message":"Exception: {\"Exception\":\"OC\RepairException\",\"Message\":\"Upgrade is not possible\",\"Code\":0,\"Trace\":\"
#0 /opt/owncloud/lib/private/Repair.php(88): OC\Repair\Apps->run(Object(OC\Repair))\
#1 /opt/owncloud/lib/private/Updater.php(221): OC\Repair->run()\
#2 /opt/owncloud/lib/private/Updater.php(108): OC\Updater->doUpgrade('9.1.6.2', '9.1.6.2')\
#3 /opt/owncloud/core/Command/Upgrade.php(249): OC\Updater->upgrade()\
#4 /opt/owncloud/lib/composer/symfony/console/Command/Command.php(262): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#5 /opt/owncloud/lib/composer/symfony/console/Application.php(826): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#6 /opt/owncloud/lib/composer/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#7 /opt/owncloud/lib/composer/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#8 /opt/owncloud/lib/private/Console/Application.php(160): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#9 /opt/owncloud/core/Controller/OccController.php(108): OC\Console\Application->run(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\BufferedOutput))\
#10 [internal function]: OC\Core\Controller\OccController->execute('upgrade', 'Zd2FC/pEqoeD4cV...', Array)\
#11 /opt/owncloud/lib/private/AppFramework/Http/Dispatcher.php(159): call_user_func_array(Array, Array)\
#12 /opt/owncloud/lib/private/AppFramework/Http/Dispatcher.php(89): OC\AppFramework\Http\Dispatcher->executeController(Object(OC\Core\Controller\OccController), 'execute')\
#13 /opt/owncloud/lib/private/AppFramework/App.php(98): OC\AppFramework\Http\Dispatcher->dispatch(Object(OC\Core\Controller\OccController), 'execute')\
#14 /opt/owncloud/lib/private/AppFramework/Routing/RouteActionHandler.php(46): OC\AppFramework\App::main('OC\\Core\\Control...', 'execute', Object(OC\AppFramework\DependencyInjection\DIContainer), Array)\
#15 [internal function]: OC\AppFramework\Routing\RouteActionHandler->__invoke(Array)\
#16 /opt/owncloud/lib/private/Route/Router.php(299): call_user_func(Object(OC\AppFramework\Routing\RouteActionHandler), Array)\
#17 /opt/owncloud/lib/base.php(922): OC\Route\Router->match('/occ/upgrade')\
#18 /opt/owncloud/index.php(54): OC::handleRequest()\
#19 {main}\",\"File\":\"/opt/owncloud/lib/private/Repair/Apps.php\",\"Line\":178}"}

[davitol]

Rechecked and I was not able to reproduce it now...