-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
211 lines (167 loc) · 7.92 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
+-------------+
| Description |
+-------------+
Pdsh is a multithreaded remote shell client which executes commands on
multiple remote hosts in parallel. Pdsh can use several different
remote shell services, including standard "rsh", Kerberos IV, and ssh.
See the man page in the doc directory for usage information.
+---------------+
| Configuration |
+---------------+
Pdsh uses GNU autoconf for configuration. Dynamically loadable
modules of each shell service (as well as other features) will be
compiled based on configuration. By default, rsh, Kerberos IV,
and SDR (for IBM SPs) will be compiled if they exist on the system.
The README.modules file distributed with pdsh contains a description
of each module available, as well as its requirements and/or
conflicts.
If your system does not support dynamically loadable modules, you
may compile modules in statically using the --enable-static-modules
option.
To configure in additional feature modules:
./configure [options]
--without-rsh
Disable support for BSD rcmd(3) (standard rsh).
--with-ssh
Enable support of ssh(1) remote shell service.
--with-machines=/path/to/machines
Use a flat file list of machine names for -a instead of
genders, nodeattr, or SDRGetObjects.
--with-qshell
Enable support for running parallel jobs on the Quadrics Elan
interconnect via the qshell service option (-R qsh) and qshell daemon.
See README.QsNet for more information.
--with-genders
Enable support of a genders database through the genders(3)
library. For pdsh's -i option to function properly, the genders
database must have alternate node names listed as the value of
the "altname" attribute.
--with-dshgroups
Enable support of dsh-style group files in ~/.dsh/group/groupname
or /etc/dsh/group/groupname. Allows use of -g/-X to target
or exclude hosts in dsh group files.
--with-netgroup
Enable use of netgroups (via /etc/netgroup or NIS) to build lists
of target hosts using -g/-X to include/exclude hosts.
--with-nodeattr=/path/to/nodeattr
Enable support of a genders database through the nodeattr(1)
command. This is primarily for older systems that do not yet
have genders(3) library support. For pdsh's -i option to
function properly, the genders database must have alternate
node names listed as the value of the "altname" attribute and
the nodeattr command must have the -r option available.
--with-nodeupdown
Enable support of dynamic elimination of down nodes through
the nodeupdown(3) library.
--with-mrsh
Enable support of mrsh(1) remote shell service.
--with-mqshell
Enable support for runnig parallel jobs on the Quadrics Elan
interconnect via the mqshell service option (-R mqsh) and
mqshell dameon. Mqshell is identical to qshell but adds munge
authentication (authentication used by mrsh).
--with-rms
Support running pdsh under RMS allocation.
--with-slurm
Support running pdsh under SLURM allocation.
--with-fanout=N
Specify default fanout (default is 32).
--with-timeout=N
Set default connect timeout (default is 10 seconds).
--with-readline
Use the GNU readline library to parse input in interactive mode.
--without-pam
Disable PAM from the qshell and mqshell daemons. By default, they
are enabled.
Note that a number of the above configurations options may "conflict"
with each other because they perform identical operations. For
example, genders and nodeattr both support the -g option. If several
modules are installed that support identical options, the options will
default to one particular module. Static compilation of modules will
fail if conflicting modules are selected. See the man page in this
directory for details on which modules conflict.
+------------+
| INSTALLING |
+------------+
make
make install
By default, pdsh is now installed without setuid permissions. This
is because, for the majority of the rcmd connect protocols, root
permissions are not necessarily needed. If you are using either of
the "rcmd/rsh" or "rcmd/qsh" modules, you will need to change the
permissions of pdsh and pdcp to be setuid root after the install.
For example:
> chown root PREFIX/bin/pdsh PREFIX/bin/pdcp
> chmod 4755 PREFIX/bin/pdsh PREFIX/bin/pdcp
If you compile the qshell and/or mqshell with PAM support, remember to
update your PAM configuration files to support the "qshell" and/or
"mqshell" service names. There are sample xinetd(8) config files
for qshd and mqshd in the etc/ directory. Also be sure read the
README.QsNet file in this directory.
+---------+
| GOTCHAS |
+---------+
Watch out for the following gotchas:
1) When executing remote commands via rsh, krb4, qsh, or ssh, pdsh
uses one reserved socket for each active connection, two if it is
maintaining a separate connection for stderr. It obtains these
sockets by calling rresvport(), which normally draws from a pool of
256 sockets. You may exhaust these if multiple pdsh's are running
simultanously on a machine, or if the fanout is set too high. Mrsh
and mqsh do not use reserved ports, and therefore are not affected
this problem as severely.
2) When pdsh is using a remote shell service that is wrapped with TCP
wrappers, there are three areas where bottlenecks can be created:
IDENT, DNS, and SYSLOG. If your hosts.allow includes "user@", e.g.
"in.rshd : ALL@ALL : ALLOW" and TCP wrappers is configured to support
IDENT, each simultaneous remote shell connection will result in an
IDENT query back to the source. For large fanouts this can quickly
overwhelm the source. Similarly, if TCP wrappers is configured to
query the DNS on every connection, pdsh may overwhelm the DNS server.
Finally, if every remote shell connection results in a remote syslog
entry, syslogd on your loghost may be overwhelmed and logs may grow
excessively long.
If local security policy permits, consider configuring TCP wrappers to
avoid calling IDENT, DNS, or SYSLOG on every remote shell connection.
Configuring without the "PARANOID" option (which requires all
connections to be registered in the DNS), permitting a simple list of
IP addresses or a subnet (no names, and no user@ prefix), and setting
the SYSLOG severity for the remote shell service to a level that is
not remotely logged will avoid these pitfalls. If these actions are
not possible, you may wish to reduce pdsh's default fanout (configure
--with-fanout=N).
+---------------------+
| THEORY OF OPERATION |
+---------------------+
We will generalize for the common remote shell service rsh. The
following is similar for all other shell services (ssh, krb4, qsh,
etc.), but other shell services may include additional security or
features.
A thread is created for each rsh connection to a node. Each thread
opens a connection using an MT-safe rcmd-like function, returns
stdin and stderr streams, then terminates.
The mainline starts fanout number of rsh threads and waits on a
condition variable that is signalled by the rsh threads as they
terminate. When the condition variable is signalled, the main thread
starts a new rsh thread to maintain the fanout, until all remote
commands have been executed.
A timeout thread is created that monitors the state of the threads and
terminates any that take too much time connecting or, if requested on
the command line, take too long to complete.
Typing ^C causes pdsh to list threads that are in the connected state.
Another ^C immediately following the first one terminates the program.
+--------+
| AUTHOR |
+--------+
Jim Garlick <[email protected]>
Please send suggestions, bug reports, or just a note letting me know
that you are using pdsh (it would be interesting to hear how many
nodes are in your cluster).
+------+
| NOTE |
+------+
This product includes software developed by the University of
California, Berkeley and its contributors. Modifications have been
made and bugs are probably mine.
The PDSH software package has no affiliation with the Democratic Party
of Albania (www.pdsh.org).