code syntax highlighting does not work in LaTeX

[marufr]

Hi,

Thank you very much for the well thought out and very useful code.

I have python code examples within my document. The html renders the syntax highlighting perfectly, but when I build latex (and from that pdf), the code comes up all jumbled up.

As a simple example, the following rest code...

..  code-block:: python

    >>> python main.py

comes out like this in the pdf:

\PYG {g+gp }{\PYGZgt{} \ PYGZgt{} \ PYGZgt{} } \PYG{n}{python} \PYG{n}{main}\PYG{o}{.}\
PYG{n}{py}

Can you please advise if I am doing something wrong, or which files I should be looking into to fix this?

Thanks,
MR

[jterrace]

Hmm, I'm not sure I ever tested that. It looks like those are pygments styles that are supposed to be written by sphinx, but I must have disabled them at some point. Looking quickly at the sphinx source, I'm not sure how they actually get written into the document. Are you up for looking into it?

[marufr]

I have started looking into it. It is likely something in 'sphinx.sty' that has been commented out.

I should put out the disclaimer that I am not a proper software dev. But would definitely like to make this work.

MR

[jterrace]

For reference, here's the original version of the file:
https://bitbucket.org/birkenfeld/sphinx/src/ef3092d458cc00c4b74dd342ea05ba1059a5da70/sphinx/texinputs/sphinx.sty?at=default

I wasn't sure where those pygments styles are coming from in there

[marufr]

I have found a fix for the issue. But it introduces another (but lesser) issue -- which is aesthetic and far less crucial.
Basically, the method depart_literal_block in the script latex_mod.py had to be restored to its original form. This fixed the syntax highlighting, but took away the background shading and borders from the code and literal blocks.

    def depart_literal_block(self, node):
        code = self.verbatim.rstrip('\n')
        lang = self.hlsettingstack[-1][0]
        linenos = code.count('\n') >= self.hlsettingstack[-1][1] - 1
        highlight_args = node.get('highlight_args', {})
        if 'language' in node:
            # code-block directives
            lang = node['language']
            highlight_args['force'] = True
        if 'linenos' in node:
            linenos = node['linenos']
        def warner(msg):
            self.builder.warn(msg, (self.curfilestack[-1], node.line))
        hlcode = self.highlighter.highlight_block(code, lang, warn=warner,
                linenos=linenos, **highlight_args)
        # hlcode = hlcode.replace('\$', '$')
        # hlcode = hlcode.replace('\%', '%')
        # workaround for Unicode issue
        hlcode = hlcode.replace(u'€', u'@texteuro[]')
        # must use original Verbatim environment and "tabular" environment
        if self.table:
            hlcode = hlcode.replace('\\begin{Verbatim}',
                                    '\\begin{OriginalVerbatim}')
            self.table.has_problematic = True
            self.table.has_verbatim = True
        # get consistent trailer
        hlcode = hlcode.rstrip()[:-14] # strip \end{Verbatim}
        hlcode = hlcode.rstrip() + '\n'
        # hlcode = '\n' + hlcode + '\\end{%sVerbatim}\n' % (self.table and 'Original' or '')
        # hlcode = hlcode.replace('Verbatim', 'lstlisting')
        # begin_bracket = hlcode.find('[')
        # end_bracket = hlcode.find(']')
        # hlcode = hlcode[:begin_bracket] + '[]' + hlcode[end_bracket+1:]
        # self.body.append(hlcode)
        # self.verbatim = None
        self.body.append('\n' + hlcode + '\\end{%sVerbatim}\n' %
                         (self.table and 'Original' or ''))
        self.verbatim = None

[jterrace]

could you submit it as a pull request and add a code block to the sample?

[marufr]

I can do that. By 'sample' do mean the sample thesis?

[jterrace]

Yep, sorry I just meant the sample thesis. I was thinking ch-typography/text.rst would be a good place for it, next to the simple code block in there.

[erikbgithub]

I tried changing depart_literal_block() to the one proposed by @marufr and I tried to leave out the method completely (which then should simply default to Sphinx' depart_literal_block()). In both cases, the output loses the grey block around it, but the code is still not highlighted correctly.

[erikbgithub]

I don't like the idea of using listing instead of Verbatim for codeblocks. listing is a package that does code highlighting for you, and Pygments is a tool that does code highlighting for you. Only using one is okay I think. Using both might be part of the issue why it's not working. I also think there is not much of a decision to make between both, because Pygments also solves the highlighting for HTML. So we better stick with it. If we want to change colors, borders and so on, we can actually do that in the sphinx.sty just fine.

Until now I still don't have a solution to the \PYG extension, though.

[erikbgithub]

I think I uncommented all interesting parts in sphinx.py, still no change.

[erikbgithub]

I am currently working on proving or disproving the assumption, that the wrong environment is used. Reason for this is, that the Pygments documentation says, that it needs the fancyvrb package and generates it output in a Verbatim environment. Because the shown example is a parametrized Verbatim environment I guess it uses not the default, but an Environment changed by the fancyvrb package.

edit: Commenting out the fancyvrb library in my normal Sphinx devenv results in the same bad output as in my sphinxtr devenv.

edit: got it working. create pull request now.

[bnewbold]

I tried @erikb85's changed (hand merged, might have made some typos) and they seem to work for me. To get grey boxes I used:

\definecolor{VerbatimColor}{rgb}{0.95,0.95,0.95}
\definecolor{VerbatimBorderColor}{rgb}{0.75,0.75,0.75}

I discovered the problem when I found underline characters being escaped in my code blocks; python code was also broken.

@erikb85, can you submit a pull request?