Skip to content
This repository has been archived by the owner on Aug 30, 2021. It is now read-only.

rabbitmqctl's use of su is blocked by SELinux on RHEL/CentOS 7 in the context of logrotate #32

Closed
gvde opened this issue Jul 17, 2017 · 24 comments
Assignees
Milestone

Comments

@gvde
Copy link

gvde commented Jul 17, 2017

See also: https://bugzilla.redhat.com/show_bug.cgi?id=1426600

During logrotate for rabbitmq-server the call to rabbitmqctl fails with an error on RHEL 7/CentOS 7

/etc/cron.daily/logrotate:

Password: su: Authentication information cannot be recovered
error: error running shared postrotate script for '/var/log/rabbitmq/*.log '

The server is still writing into the old log file which has been renamed by logrotate.

@michaelklishin
Copy link
Member

michaelklishin commented Jul 17, 2017

The issue in Red Hat's Bugzilla is reported for RabbitMQ 3.3.5, which is close to 3 years old. If there is a way to reproduce this with a modern version, please post some details to rabbitmq-users, our public mailing list.
It's not something CentOS/RHEL/Fedora users reported in the last couple of years IIRC, so there's a good chance it's something specific to a particular setup.

@michaelklishin
Copy link
Member

Also, our team started producing CentOS 7 releases in the 3.6.x series. 3 years ago there was no CentOS 7 RPM produced by this team, so chances are the package in question is custom-built.

@gvde
Copy link
Author

gvde commented Jul 17, 2017

The wrapper script is still the same in the master branch as is the replacement for SU_RABBITMQ_SH_C in the specfile to 'su rabbitmq -s /bin/sh -c'. Thus that's the same in the master and doesn't have anything to do with the old version in EPEL7.

One difference I have just noticed is the missing postrotate section in the logrotate configuration. How does the server know when to reopen the log file?

@michaelklishin
Copy link
Member

@gvde current releases are produced from the stable branch.

The server would create a log file when it does not exist.

It would be very helpful if you could explain why the use of su in rabbitmqctl is the problem and what is the recommended alternative. Note that rabbitmqctl is used in all kinds of environments, not just RHEL-derivatives and there's only so much room for changes to it. Anything that can be done with RPM package installation scripts would be considered by our team.

@gvde
Copy link
Author

gvde commented Jul 17, 2017

The problem seems to be that SeLinux on CentOS 7 doesn't allow to use su in the context of a logrotate script. I can't tell whether that is intentionally but it can't be worked around with a simple SeLinux policy either.

Using sudo instead of su works fine for me.

@michaelklishin michaelklishin changed the title /etc/logrotate.d/rabbitmq-server leads to "Password: su: Authentication failure" rabbitmqctl's use of su is blocked by SELinux on RHEL/CentOS 7 in the context of logrotate Jul 17, 2017
@michaelklishin
Copy link
Member

@gvde thanks for the details. Moved to rabbitmq/rabbitmq-server#1292 as currently the thinking is that we cannot work around this with packaging.

@gvde
Copy link
Author

gvde commented Jul 28, 2017

rabbitmq/rabbitmq-server#1292 was closed with wontfixed. Thus it comes back here. If sudo isn't an alternative it's necessary to fix the selinux policies to be installed with the rhel7/centos7 rpms...

@michaelklishin
Copy link
Member

@gvde we'd definitely welcome a PR with some ideas as to what the policies should be. Not sure how common it is for packages to install their own policies, by the way.

@mliker
Copy link

mliker commented Aug 13, 2017

I am also running into a similar issue on Centos 7 with RabbitMQ installed from the official repo (http://www.rabbitmq.com/releases/rabbitmq-server), installed version is 3.6.10.

Logrotate fails with the following message:

/etc/cron.daily/logrotate:

Error: Failed to initialize erlang distribution: {{shutdown,
                                                  {failed_to_start_child,
                                                   auth,
                                                   {{error,
                                                     "Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces"},
                                                    [{auth,init_cookie,0,
                                                      [{file,"auth.erl"},
                                                       {line,286}]},
                                                     {auth,init,1,
                                                      [{file,"auth.erl"},
                                                       {line,140}]},
                                                     {gen_server,init_it,2,
                                                      [{file,
                                                        "gen_server.erl"},
                                                       {line,365}]},
                                                     {gen_server,init_it,6,
                                                      [{file,
                                                        "gen_server.erl"},
                                                       {line,333}]},
                                                     {proc_lib,
                                                      init_p_do_apply,3,
                                                      [{file,"proc_lib.erl"},
                                                       {line,247}]}]}}},
                                                 {child,undefined,
                                                  net_sup_dynamic,
                                                  {erl_distribution,
                                                   start_link,
                                                   [['rabbitmq-cli-24',
                                                     shortnames],
                                                    false]},
                                                  permanent,1000,supervisor,
                                                  [erl_distribution]}}.
error: error running shared postrotate script for '/var/log/rabbitmq/*.log '

Digging through audit.log I found a denied access to /var/lib/rabbitmq/.erlang.cookie.

audit2allow suggests the following two selinux rules to be added in order for logrotate for rabbitmq to work:

root@n2 / # grep rabbitmq /var/log/audit/audit.log.1 | audit2allow 

#============= logrotate_t ==============
allow logrotate_t rabbitmq_var_lib_t:dir { read write };
allow logrotate_t rabbitmq_var_lib_t:file getattr;

Should the rabbitmq rpm ship with an updated SELinux policy or is this up to the user to update the policy?

@michaelklishin
Copy link
Member

michaelklishin commented Aug 13, 2017 via email

@gvde
Copy link
Author

gvde commented Aug 14, 2017

logrotate itself shouldn't access rabbitmq_var_lib_t. You should set up a domain transition, instead, so that only processes in the rabbitmq domain access rabbitmq files.

See the domain transition chapter on https://www.digitalocean.com/community/tutorials/an-introduction-to-selinux-on-centos-7-part-2-files-and-processes for an example...

@mliker
Copy link

mliker commented Aug 14, 2017 via email

@gvde
Copy link
Author

gvde commented Aug 14, 2017

Which is exactly why you implement domain transition so that rabbitmqctl (or whichever best suited) can operate in the rabbitmq_t domain instead of logrotate_t when called through logrotate.

@lukebakken lukebakken self-assigned this Aug 14, 2017
@lukebakken
Copy link
Contributor

@gvde as noted above in this comment, if you could provide assistance in crafting a fix it would be completed much sooner. Thanks!

@gvde
Copy link
Author

gvde commented Aug 14, 2017

@lukebakken Well, there is a mixup of issues in this one. I use the EPEL 7 version of rabbitmq and I still haven't found the definite solution for that, yet.

For the other issues raised here it seems like a transition problem. But I don't use that version of rabbitmq and wouldn't be able to test it. But so far it looks to me like a different problem except that both are SeLinux related...

@lukebakken
Copy link
Contributor

@mliker - please see #39 for discussion about your file permissions issue.

@lukebakken
Copy link
Contributor

@gvde - I have tried out the reproduction steps you list here and notice the same issue in my CentOS 7 virtual machine. However, if I use runuser rather than su, it appears to work correctly. Here are the contents of /root/test.sh that I am using:

#!/bin/sh
echo "id: `id`"
echo "id -r -u: `id -r -u`"
echo "id -r -g: `id -r -g`"
/sbin/runuser rabbitmq -s /bin/sh -c "id"
/bin/su rabbitmq -s /bin/sh -c "id"

Here is the output I get from cron:

id: uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:logrotate_t:s0-s0:c0.c1023
id -r -u: 0
id -r -g: 0
uid=996(rabbitmq) gid=994(rabbitmq) groups=994(rabbitmq) context=system_u:system_r:logrotate_t:s0-s0:c0.c1023
Password: su: Authentication failure

Here is the SELinux policy applied to /root/test.sh:

[root@localhost ~]# ls -Z /root/test.sh
-rwxr-xr-x. root root system_u:object_r:logrotate_exec_t:s0 /root/test.sh

If convenient, could you see if switching from su to runuser addresses the issue in your environment as well? Thanks.

@rabbitmq rabbitmq deleted a comment from michaelklishin Aug 14, 2017
@lukebakken
Copy link
Contributor

lukebakken commented Aug 14, 2017

SELinux notes:

After running emanage permissive -a rabbitmq_var_lib_t, the following is in /var/log/audit/audit.log as rotation executes:

type=AVC msg=audit(1502746862.765:584): avc:  denied  { getattr } for  pid=2972 comm="async_9" path="/var/lib/rabbitmq/.erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file
type=AVC msg=audit(1502746862.765:585): avc:  denied  { getattr } for  pid=2973 comm="async_10" path="/var/lib/rabbitmq/.erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file
type=AVC msg=audit(1502749801.825:645): avc:  denied  { read } for  pid=4336 comm="async_6" name=".erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file
type=AVC msg=audit(1502749801.825:646): avc:  denied  { open } for  pid=4338 comm="async_8" path="/var/lib/rabbitmq/.erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file
type=AVC msg=audit(1502750041.833:698): avc:  denied  { read } for  pid=4954 comm="async_6" name=".erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file
type=AVC msg=audit(1502750041.833:699): avc:  denied  { open } for  pid=4956 comm="async_8" path="/var/lib/rabbitmq/.erlang.cookie" dev="dm-0" ino=551112 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file

@lukebakken
Copy link
Contributor

These two lines would normally assign a rabbitmq specific domain to the erlang executables, but on 64-bit platforms, Erlang installs to /usr/lib64. I suspect this is the reason the domain transition used to work but no longer does.

@mliker
Copy link

mliker commented Aug 16, 2017

Thanks @lukebakken I will test it out and report back

@gvde
Copy link
Author

gvde commented Aug 16, 2017

On a 64 bit centos7 there should be an equivalence rule /usr/lib64 = /usr/lib which would assign the same labels to files in lib64 as lib.

It's best to verify the assigned labels with ls -Z to see if that's really the problem.

Check all fcontext rules with
semanage fcontext --list

Equivalence rules are at the bottom.

@lukebakken
Copy link
Contributor

Please see my followup comment. I am trying to keep the su and eacces issues separate. Thanks!

lukebakken added a commit that referenced this issue Aug 17, 2017
Use /sbin/runuser to execute script as the `rabbitmq` user as this command
does not cause issues with SELinux. See #32 for details.
gerhard pushed a commit that referenced this issue Aug 31, 2017
Because the rabbitmq user shell defaults to /bin/false, we must
explicitly define a shell when using the runuser command

Issue: #32
PR: #40

[#150221349]

Signed-off-by: Gerhard Lazu <[email protected]>
@michaelklishin
Copy link
Member

This is addressed by #45 after all. @gvde we are happy to provide a snapshot build for you to test if you'd be interested :)

@michaelklishin michaelklishin added this to the 3.7.0 milestone Sep 20, 2017
@gvde
Copy link
Author

gvde commented Sep 20, 2017

@michaelklishin I don't need a snapshot. I have modified the script already to use runuser and it works. Thus, as long as the patch implements runuser it's fine.

Of course, now I run into #39 :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants