pep8'ed Python source being generated

Also --keep-texts-layout option to mibdump.py implemented and set to False by default.

CHANGES.rst

@@ -1,9 +1,11 @@
-Revision 0.1.1, 29-03-2017
+Revision 0.1.1, 30-03-2017
 --------------------------
 
 * Generate REFERENCE and STATUS fields at various SMI objects
 * Generate DESCRIPTION field followed REVISION field at MODULE-IDENTITY objects
 * Generate PRODUCT-RELEASE field at AGENT-CAPABILITIES objects
+* Generated Python source aligned with PEP8
+* MIB texts cleaned up by default, --keep-texts-layout preserves original formatting
 * Fix to the `ordereddict` conditional dependency
 * Missing test module recovered
 * Failing tests fixed

README.md

@@ -50,6 +50,7 @@ Byte-compile Python modules: no (optimization level no)
 Ignore compilation errors: no
 Generate OID->MIB index: no
 Generate texts in MIBs: yes
+Keep original texts layout: no
 Try various filenames while searching for MIB module: yes
 Created/updated MIBs: IANAifType-MIB, IF-MIB, SNMPv2-MIB
 Pre-compiled MIBs borrowed: 

docs/source/user-perspective.rst

@@ -33,6 +33,7 @@ into various formats.
          [--dry-run]
          [--no-mib-writes]
          [--generate-mib-texts]
+         [--keep-texts-layout]
          [mibfile [mibfile [...]]]
    Where:
        url      - file, http, https, ftp, sftp schemes are supported.
@@ -276,6 +277,11 @@ To save space and CPU time, PySMI does not by default include those texts
 into transformed MIBs. However this can be reverted by adding
 --generate-mib-texts option.
 
+When MIB texts are generated, whitespaces and new lines are stripped by
+default. Sometimes that breaks down ASCII art should it occur in MIB texts.
+To preserve original text formatting, --keep-texts-layout option may
+be used.
+
 Building MIB indices
 --------------------
 
@@ -343,7 +349,6 @@ list top-level OIDs branches defined in MIB modules. Full index
 build over thousands of MIBs could be seen
 `here <http://mibs.snmplabs.com/json/index.json>`_.
 
-
 Minor speedups
 --------------
 

pysmi/codegen/base.py

@@ -26,7 +26,6 @@ def updateDict(d1, d2):
 
 
 class AbstractCodeGen(object):
-
     # never compile these, they either:
     # - define MACROs (implementation supplies them)
     # - or carry conflicting OIDs (so that all IMPORT's of them will be rewritten)

pysmi/codegen/jsondoc.py

@@ -584,11 +584,11 @@ def genConceptualTable(self, data):
     # noinspection PyMethodMayBeStatic,PyUnusedLocal
     def genContactInfo(self, data):
         text = data[0]
-        return re.sub('\s+', ' ', text)
+        return self.textFilter('contact-info', text)
 
     # noinspection PyUnusedLocal
     def genDisplayHint(self, data):
-        return re.sub('\s+', ' ', data[0])
+        return data[0]
 
     # noinspection PyUnusedLocal
     def genDefVal(self, data, objname=None):
@@ -667,18 +667,15 @@ def genDefVal(self, data, objname=None):
 
     # noinspection PyMethodMayBeStatic
     def genDescription(self, data):
-        text = data[0]
-        return re.sub('\s+', ' ', text)
+        return self.textFilter('description', data[0])
 
     # noinspection PyMethodMayBeStatic
     def genReference(self, data):
-        text = data[0]
-        return re.sub('\s+', ' ', text)
+        return self.textFilter('reference', data[0])
 
     # noinspection PyMethodMayBeStatic
     def genStatus(self, data):
-        text = data[0]
-        return re.sub('\s+', ' ', text)
+        return data[0]
 
     def genEnumSpec(self, data):
         items = data[0]
@@ -726,8 +723,7 @@ def genIntegerSubType(self, data):
 
     # noinspection PyMethodMayBeStatic,PyUnusedLocal
     def genMaxAccess(self, data):
-        access = data[0]
-        return access
+        return data[0]
 
     def genOctetStringSubType(self, data):
         sizes = []
@@ -792,17 +788,15 @@ def genTime(self, data):
 
     # noinspection PyMethodMayBeStatic,PyUnusedLocal
     def genLastUpdated(self, data):
-        text = data[0]
-        return re.sub('\s+', ' ', text)
+        return data[0]
 
     # noinspection PyMethodMayBeStatic,PyUnusedLocal
     def genOrganization(self, data):
-        text = data[0]
-        return re.sub('\s+', ' ', text)
+        return self.textFilter('organization', data[0])
 
     # noinspection PyUnusedLocal
     def genRevisions(self, data):
-        times = self.genTime(data[0])
+        times = self.genTime([x[0] for x in data[0]])
         return times
 
     def genRow(self, data):
@@ -866,7 +860,7 @@ def genTypeDeclarationRHS(self, data):
     # noinspection PyMethodMayBeStatic,PyUnusedLocal
     def genUnits(self, data):
         text = data[0]
-        return re.sub('\s+', ' ', text)
+        return self.textFilter('units', text)
 
     handlersTable = {
         'agentCapabilitiesClause': genAgentCapabilities,
@@ -914,6 +908,7 @@ def genUnits(self, data):
 
     def genCode(self, ast, symbolTable, **kwargs):
         self.genRules['text'] = kwargs.get('genTexts', False)
+        self.textFilter = kwargs.get('textFilter') or (lambda symbol, text: re.sub('\s+', ' ', text))
         self.symbolTable = symbolTable
         self._rows.clear()
         self._cols.clear()