BK-2085 Add support for Django 1.11 LTS #887

ride90 · 2018-05-18T16:25:02Z

Helmy's Update:

Points to be considered for testing - just so I don't forget about things ;)

Export

mpdf
epub and brothers
pdfreactor

danielhjames · 2018-05-18T16:41:48Z

Hi @ride90, nice work! One thing that confuses me is that lib/booktype/apps/importer/management/commands/bookimport.py and lib/booktype/apps/loadsave/management/commands/bookload.py have the same functionality. Should we deprecate one of these? If so, I suggest we deprecate bookimport.py as we are getting rid of bookexport.py. This would leave us with bookload.py and booksave.py commands.

danielhjames · 2018-05-18T16:46:19Z

lib/booktype/apps/loadsave/management/commands/booksave.py

+                            action='store',
+                            dest='output_name',
+                            default=None,
+                            help='Output filename or -- for STDOUT, e.g. my_book.zip.')


Line 37 should be:

help='Output filename or -- for STDOUT, e.g. my_book.epub')

danielhjames · 2018-05-18T16:48:24Z

lib/booktype/apps/loadsave/management/commands/booksave.py

+                            action='store',
+                            dest='book_version',
+                            default=None,
+                            help='Book version, e.g.')


Need to give an example of book_version on line 32. Is the only possible version 1.0 since we hid the version feature?

ride90 · 2018-05-20T18:39:17Z

Hi @danielhjames ,
This PR marked with label "work in progress". I'm fine with depreciation for some commands. Let's discuss it next week.

ride90 · 2018-06-01T13:05:52Z

Hi @eos87 and @danielhjames , please test this pr.
It's quite huge 🐻

danielhjames · 2018-06-01T14:18:07Z

Good work @ride90 ! I'll test this when I'm back in the office.

ride90 · 2018-06-08T08:29:20Z

Hi @danielhjames, @eos87 any feedback? 😄

eos87 · 2018-06-12T17:15:11Z

Hi @ride90, I checked this but I still feel like a need to test it more deeply. I will try to do it in next days and will be adding any suggestion that I could make. Great work so far :)

danielhjames · 2018-06-15T17:03:28Z

Hi @ride90 I tested this branch on a Debian 9.x virtualbox server and the installation went fine, I can import EPUB and edit the book, but publishing does not work for me yet.

I think our celery instructions need updating to work with these changes. Please see Running celery with supervisor in the 2.4.0 manual.

This line in the (previously working) supervisord config no longer works, it results in unknown command: 'celery' in the supervisord error log:

command=/var/www/booktype/instance1/manage_dev.py celery worker --concurrency=10 -l info

Following the output of which celery I changed the command to:

command=/usr/local/bin/celery worker --concurrency=10 -l info

Now the error is:

Received unregistered task of type 'booktype.apps.edit.tasks.publish_book'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.

danielhjames · 2018-06-15T19:09:51Z

ride90 · 2018-06-16T09:07:48Z

Hi @danielhjames, @eos87 seems like not all changes I've added to the script which will generate skeleton with the settings. I will update pr.

danielhjames · 2018-06-16T09:38:55Z

Thanks @ride90, looks like we are nearly there!

danielhjames · 2018-08-14T14:56:55Z

requirements/_base.txt

@@ -13,6 +13,6 @@ python-ooxml
 django-rest-swagger==2.1.0
 djangorestframework==3.5.3
 Markdown==2.6.7
-django-filter==1.0.0
+django-filter==1.1.0
 Ebooklib==0.16


This needs bumping to:
Ebooklib==0.17
to merge with #895

ride90 · 2018-08-17T14:13:35Z

Hi @danielhjames @eos87 ,
I've updated PR, please, test it 🍻

danielhjames · 2018-08-17T14:47:21Z

lib/booktype/skeleton/dev_settings.py.original

@@ -3,10 +3,9 @@ import os
 from .base import *

 # WEB SITE URL
-THIS_BOOKTYPE_SERVER = os.environ.get('BOOKTYPE_SERVER', '')
+THIS_BOOKTYPE_SERVER = os.environ.get('BOOKTYPE_SERVER', '127.0.0.1:8000')


Do we need a similar change for prod_settings.py.original please?

Hi @danielhjames , no, we don't need it. I've made it in dev settings to make it easier to kickstart dev instance from scratch. For production instance developer will change settings by himself.

Sounds good!

danielhjames · 2018-08-17T16:54:25Z

scripts/createbooktype

@@ -664,7 +664,7 @@ def create_supervisor(destination, profile, virtual_env):

    s = '''[program:%(project_name)s-celery-worker]
 directory       = %(project_directory)s
-command         = %(virtualenv)s/bin/python %(project_directory)s/manage_%(profile)s.py celery worker --concurrency=20
+command         = %(virtualenv)s/bin/python -m celery worker --app=%(project_name)s_site --concurrency=20


This change fixed the celery issue for me, thanks @ride90!

danielhjames · 2018-08-17T17:14:00Z

EPUB3 works but not EPUB2, I am seeing this error again: https://dev.sourcefabric.org/browse/BK-2329

[2018-08-17 18:21:01,778: INFO/ForkPoolWorker-10] Starting new HTTP connection (1): booktype.64studio.com
[2018-08-17 18:21:01,786: ERROR/ForkPoolWorker-12] Task convert_one[epub2:2601d172-15ce-4bb3-a36b-5c83dfd32c80] raised unexpected: TypeError('_write_opf_file() takes exactly 1 argument (2 given)',)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 375, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 632, in protected_call
return self.run(*args, **kwargs)
File "/usr/local/src/booktype/lib/booktype/apps/convert/tasks.py", line 50, in decorated_func
return func(request, *args, **kwargs)
File "/usr/local/src/booktype/lib/booktype/apps/convert/tasks.py", line 66, in convert_one
result = run_conversion(*args, **kwargs)
File "/usr/local/src/booktype/lib/booktype/convert/runner.py", line 58, in run_conversion
conversion_result = converter.convert(book, output)
File "/usr/local/src/booktype/lib/booktype/convert/epub/converter.py", line 167, in convert
epub_writer.write()
File "/usr/local/lib/python2.7/dist-packages/ebooklib/epub.py", line 1300, in write
self._write_opf()
File "/usr/local/lib/python2.7/dist-packages/ebooklib/epub.py", line 1080, in _write_opf
self._write_opf_file(root)
TypeError: _write_opf_file() takes exactly 1 argument (2 given)

danielhjames · 2018-08-17T17:26:51Z

For the mPDF converter I got the error:

AttributeError: 'dict' object has no attribute 'render_context'

According to https://stackoverflow.com/questions/6333367/django-on-apache-web-server-dict-object-has-no-attribute-render-context it may be Django 1.11 related.

danielhjames · 2018-08-17T17:28:29Z

XHTML, MOBI, DOCX and ICML work though, we are on our way :-)

danielhjames · 2018-08-20T11:41:18Z

[2018-08-17 18:20:52,523: WARNING/ForkPoolWorker-3] /usr/local/lib/python2.7/dist-packages/unidecode/init.py:46: RuntimeWarning: Argument <type 'str'> is not an unicode object. Passing an encoded string will likely have unexpected results.
_warn_if_not_unicode(string)
[2018-08-17 18:20:52,608: INFO/ForkPoolWorker-10] Starting new HTTP connection (1): booktype.64studio.com
[2018-08-17 18:20:52,705: DEBUG/ForkPoolWorker-10] "GET /_convert/e3b9739e-a241-11e8-acce-0800276569ef HTTP/1.1" 200 20
[2018-08-17 18:20:53,214: INFO/ForkPoolWorker-10] Starting new HTTP connection (1): booktype.64studio.com
[2018-08-17 18:20:53,298: DEBUG/ForkPoolWorker-10] "GET /_convert/e3b9739e-a241-11e8-acce-0800276569ef HTTP/1.1" 200 20
[2018-08-17 18:20:53,406: DEBUG/ForkPoolWorker-12] [EPUB] Epub2Converter.convert
[17/Aug/2018 18:20:53] ERROR [booktype.convert.pdf:513] Writing styles failed for academic theme.
Traceback (most recent call last):
File "/usr/local/src/booktype/lib/booktype/convert/mpdf/converter.py", line 510, in _write_style
_style = tmpl.render(self.config)
File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 203, in render
with context.render_context.push_state(self):
AttributeError: 'dict' object has no attribute 'render_context'

danielhjames · 2018-08-20T11:44:48Z

This could be relevant to the mPDF issue: https://www.reddit.com/r/django/comments/64izv8/what_has_changed_with_django_111_in/

…ings.

danielhjames · 2018-08-24T16:35:05Z

requirements/_base.txt

@@ -14,5 +14,6 @@ Ebooklib==0.17
 django-rest-swagger==2.1.0
 djangorestframework==3.5.3
 Markdown==2.6.7
-django-filter==1.0.0
+django-filter==1.1.0
+Ebooklib==0.16


We have two lines for Ebooklib in this file now.

danielhjames · 2018-08-24T17:02:55Z

Seen when trying to run the compress step:

./manage_prod.py compress
Traceback (most recent call last):
File "./manage_prod.py", line 26, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 363, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 355, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 205, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 40, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
File "/usr/local/lib/python2.7/dist-packages/compressor/management/commands/compress.py", line 37, in
class Command(BaseCommand):
File "/usr/local/lib/python2.7/dist-packages/compressor/management/commands/compress.py", line 39, in Command
option_list = BaseCommand.option_list + (
AttributeError: type object 'BaseCommand' has no attribute 'option_list'

ride90 · 2018-10-01T15:46:13Z

@danielhjames , please test it.

ride90 · 2018-10-02T10:31:45Z

There is a PR for docker support booktype/booktype-docker#2

ride90 · 2018-10-08T07:38:39Z

Hi @eos87 , please, test this pr :)

eos87

I tested this and seems to be working good. I'm going to merge it and when can create later tickets for newly found bugs.

eos87 · 2018-10-22T21:47:56Z

Deployed and test on http://booktype-dev.sourcefabric.org

ride90 added the WIP label May 18, 2018

danielhjames reviewed May 18, 2018

View reviewed changes

BK-2085 Add support for Django 1.11 LTS

4e52d50

ride90 force-pushed the BK-2085 branch from 3f55f96 to 4e52d50 Compare June 1, 2018 13:03

ride90 removed the WIP label Jun 1, 2018

danielhjames reviewed Aug 14, 2018

View reviewed changes

ride90 added 3 commits August 17, 2018 15:29

BK-2085 Update dev settings.

84bdccd

BK-2085 Update supervisor settings to fit latest celery requirements.

a1e60ed

BK-2085 Bumped up an ebooklib version.

f459145

danielhjames reviewed Aug 17, 2018

View reviewed changes

BK-2085 Bumped down an ebooklib version. Proper dev static/media sett…

889a517

…ings.

ride90 force-pushed the BK-2085 branch from 8e817cc to 889a517 Compare August 24, 2018 15:30

Merge branch 'master' into BK-2085

c72ef6b

danielhjames reviewed Aug 24, 2018

View reviewed changes

BK-2085 Bumped up django-compressor.

b83818d

Remove duplication in requirements.

19b79b8

Fix mpdf theme style rendering.

fc11ee2

eos87 self-requested a review October 15, 2018 18:05

eos87 approved these changes Oct 22, 2018

View reviewed changes

eos87 merged commit ecb3673 into booktype:master Oct 22, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BK-2085 Add support for Django 1.11 LTS #887

BK-2085 Add support for Django 1.11 LTS #887

ride90 commented May 18, 2018 •

edited by eos87

Loading

danielhjames commented May 18, 2018

danielhjames May 18, 2018

danielhjames May 18, 2018

ride90 commented May 20, 2018

ride90 commented Jun 1, 2018

danielhjames commented Jun 1, 2018

ride90 commented Jun 8, 2018

eos87 commented Jun 12, 2018

danielhjames commented Jun 15, 2018

danielhjames commented Jun 15, 2018

ride90 commented Jun 16, 2018

danielhjames commented Jun 16, 2018

danielhjames Aug 14, 2018

ride90 commented Aug 17, 2018

danielhjames Aug 17, 2018

ride90 Aug 17, 2018

danielhjames Aug 17, 2018

danielhjames Aug 17, 2018

danielhjames commented Aug 17, 2018 •

edited

Loading

danielhjames commented Aug 17, 2018

danielhjames commented Aug 17, 2018

danielhjames commented Aug 20, 2018

danielhjames commented Aug 20, 2018

danielhjames Aug 24, 2018

danielhjames commented Aug 24, 2018

ride90 commented Oct 1, 2018

ride90 commented Oct 2, 2018

ride90 commented Oct 8, 2018

eos87 left a comment

eos87 commented Oct 22, 2018

BK-2085 Add support for Django 1.11 LTS #887

BK-2085 Add support for Django 1.11 LTS #887

Conversation

ride90 commented May 18, 2018 • edited by eos87 Loading

Points to be considered for testing - just so I don't forget about things ;)

Export

danielhjames commented May 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ride90 commented May 20, 2018

ride90 commented Jun 1, 2018

danielhjames commented Jun 1, 2018

ride90 commented Jun 8, 2018

eos87 commented Jun 12, 2018

danielhjames commented Jun 15, 2018

danielhjames commented Jun 15, 2018

ride90 commented Jun 16, 2018

danielhjames commented Jun 16, 2018

Choose a reason for hiding this comment

ride90 commented Aug 17, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielhjames commented Aug 17, 2018 • edited Loading

danielhjames commented Aug 17, 2018

danielhjames commented Aug 17, 2018

danielhjames commented Aug 20, 2018

danielhjames commented Aug 20, 2018

Choose a reason for hiding this comment

danielhjames commented Aug 24, 2018

ride90 commented Oct 1, 2018

ride90 commented Oct 2, 2018

ride90 commented Oct 8, 2018

eos87 left a comment

Choose a reason for hiding this comment

eos87 commented Oct 22, 2018

ride90 commented May 18, 2018 •

edited by eos87

Loading

danielhjames commented Aug 17, 2018 •

edited

Loading