saltclass master_top module fails to populate __salt__ in some cases (salt-ssh + saltutil)

[ptitdoc]

Description of Issue/Question

When using saltclass master_top module with a salt-ssh client, it fail to populate the salt object and crash with the following message:
[ERROR ] Top function saltclass failed with error 'dict object' has no attribute 'saltutil' for minion myminion

Setup

In my classes, I use a macro that populates the pillar with some certificates found in the mine. These macro are using salt.saltutil.runner:

cacert.macros:

{% macro root_cert(ca_minion, key_name="root_cert", pem_mine="ca_pem_entries", pem_path="/salt/ca/", tab_level=3) %}
{% set ca_mine = __salt__.saltutil.runner('mine.get', tgt=ca_minion, fun=pem_mine) %}
{% if ca_mine is defined and ca_mine %}
  {% if ca_mine.success is defined and ca_mine.success and ca_mine.return is defined %}
    {% set ca_mine = ca_mine.return %}
  {% endif %}
  {% if ca_minion in ca_mine %}
    {% for ca_path in ca_mine[ca_minion].keys() %}
      {# Search for the published ROOT CA path #}
      {% if ca_path.find(pem_path) > 0 %}
        {% if tab_level < 2 %}
  {{ key_name }}: {{ ca_mine[ca_minion][ca_path].replace('\n','') }}
        {% elif tab_level < 3 %}
    {{ key_name }}: {{ ca_mine[ca_minion][ca_path].replace('\n','') }}
        {% elif tab_level < 4 %}
      {{ key_name }}: {{ ca_mine[ca_minion][ca_path].replace('\n','') }}
        {% endif %}
      {% endif %}
    {% endfor %}
  {% endif %}
{% endif %}
{% endmacro %}

ca.yml:

{% import 'cacert.macros' as macros %}
states:
  - ca.cert

pillars:
  config:
    pki_path: /etc/pki/
    ca:
      {{ macros.root_cert("mycaminion", key_name="cert") }}

master config snippet:

enable_ssh_minions: True

master_tops:
  saltclass:
    path: /home/user/salt/cloud

ext_pillar:
  - saltclass:
    - path: /home/user/salt/cloud

Steps to Reproduce Issue

When running salt myminion pillar.item, the pillar is resolved "almost" properly (I have a specific bug opened to make it actually working: #36796)

When running salt myminion state.apply, the following message appears in master logs:
[ERROR ] Top function saltclass failed with error 'dict object' has no attribute 'saltutil' for minion myminion

And the output of the command is:

Summary for myminion
-----------
Succeeded: 0
Failed:   0
-----------
Total states run:    0
Total run time:  0.000 ms

Versions Report

Salt Version:
           Salt: 2018.3.3
 
Dependency Versions:
           cffi: 1.11.5
       cherrypy: Not Installed
       dateutil: 2.7.5
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.27.4
        libnacl: Not Installed
       M2Crypto: 0.30.1
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.7.0
   pycryptodome: Not Installed
         pygit2: 0.27.2
         Python: 2.7.15 (default, Jun 27 2018, 13:05:28)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 5.1.1
            ZMQ: 4.2.5
 
System Versions:
           dist:   
         locale: UTF-8
        machine: x86_64
        release: 4.14.74-1.pvops.qubes.x86_64
         system: Linux
        version: Not Installed

[garethgreenaway]

@saltstack/team-core Anyone more familiar with saltclass have any insight here?

[cheribral]

I am hitting this issue as well. When calling pillar.items, jinja calls to salt modules within class files works correctly. When trying to run a highstate, however, saltclass will fail entirely because salt has no modules so no states will make into the list for the highstate run.

It seems that the loading of salt modules needs to be moved to happen earlier in the process, or the list of states for highstate needs to be generated later.

saltclass is huge help, and has been a game changer for us. It would be nice to see this working properly!

[cheribral]

Perhaps @olivier-mauras might have some insight?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.