Support for attaching to std in is missing.

[mzeo]

No description provided.

[rgrunber]

Just to clarify are you referring to the fact that one is not able to send data back through the stream sent back by attachContainer ?

I don't believe there's a clean way of doing this (although it is possible). As far as I can tell, the problem is docker-client can't provide this due to Apache HttpClient's lack of support for hijacking

[missedone]

similar case here, i'd like to send data via stdin when execStart

so as long as the docker client use apache httpComponents which does not support hijacking, does it mean docker-client won't support this feature neither?

thanks

[rgrunber]

In case anyone would be interested in just getting something "working", one can always use reflection to traverse the various fields, to the level below apache-httpclient and then cast that object to an interface that permits write operations. I was able to do this for Linux Tools Docker Plugin to support interacting with a container started as a shell. It's far from ideal, but even dockerpy traverses internal fields to do the same.

[Cydhra]

The workaround of the previous comment does not work with the latest version. Therefore, here is my fixed implementation, for those who search one. Note, that I removed much of the code, that I personally do not need for my project (like the eclipse-platform dependencies) - so if you need them, re-include them by merging the two work-arounds.
Gist: here

[ajmalrehman]

@Cydhra Hi I am new to the docker , I am facing similar Issue but in a different way. I am trying to run a java program on docker using execStart(execCreation) method of docker client. I tried to use your Hijacking class but i couldn't figure out how could i send the stdin. Below is my code for your reference .It could be of great help if you advise something.

package com.examenginedashboard.docker.utils;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.spotify.docker.client.DefaultDockerClient;
import com.spotify.docker.client.DockerClient;
import com.spotify.docker.client.LogStream;
import com.spotify.docker.client.messages.ContainerConfig;
import com.spotify.docker.client.messages.ContainerCreation;
import com.spotify.docker.client.messages.ExecState;
import com.spotify.docker.client.messages.HostConfig;
import com.spotify.docker.client.messages.PortBinding;

public class CodeCompiler {

String id=null;
String execOutput = null;
DockerClient docker= null;

public String compileAndExecuteCode(String dockerimage,String[] command,String mountedpath){
	 try {
		docker = DefaultDockerClient.fromEnv().build();
		docker.pull(dockerimage);
		final String[] ports = {"99", "9999"};
		final Map<String, List<PortBinding>> portBindings = new HashMap<String, List<PortBinding>>();
	
		for (String port : ports) {
		    List<PortBinding> hostPorts = new ArrayList<PortBinding>();
		    hostPorts.add(PortBinding.of("0.0.0.0", port));
		    portBindings.put(port, hostPorts);
		}
		
		   
		List<PortBinding> randomPort = new ArrayList<PortBinding>();
		randomPort.add(PortBinding.randomPort("0.0.0.0"));
		portBindings.put("443", randomPort);
		final HostConfig hostConfig = HostConfig.builder().binds(mountedpath).
				portBindings(portBindings).build();
		

		
		final ContainerConfig containerConfig = ContainerConfig.builder()
			    .hostConfig(hostConfig)
			    .image(dockerimage).exposedPorts(ports)
			    .cmd("sh", "-c", "while :; do sleep 1; done")
			    .build();
		
		final ContainerCreation creation = docker.createContainer(containerConfig);
		 id = creation.id();
		 String uri ="unix:///var/run/docker.sock";
		// Start container
		docker.startContainer(id);
		// Exec command inside running container with attached STDOUT and STDERR
		System.out.println(docker.inspectContainer(id));
		
		//final String[] command = {"bash", "-c", "cd mydockerbuild/ && javac "+filename+"  && java -cp . "+executablefilename+" exit"};
		System.out.println(command);
		final String execCreation = docker.execCreate(
		    id, command,DockerClient.ExecCreateParam.attachStdin(),DockerClient.ExecCreateParam.attachStdout(),
		    DockerClient.ExecCreateParam.attachStderr());
		/*----------------------------------execution line--------------------*/
		final LogStream output = docker.execStart(execCreation);

/* ---------------------------------------------------after this I need to pass the stdin-------------------------------*/
final ExecState execinfo =docker.execInspect(execCreation);
System.out.println(execinfo);
if(execinfo.running()&& execinfo.exitCode()==null){
System.out.println("Waiting for user input....................");

		}
				
		try{
			execOutput = output.readFully();
			
			}catch(RuntimeException e){
				e.printStackTrace();
				
				
			}
			System.out.println(execOutput);
	} catch (InterruptedException e) {
		// TODO Auto-generated catch block
		e.printStackTrace();
	} catch (com.spotify.docker.client.DockerCertificateException e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	} catch (com.spotify.docker.client.DockerException e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	} catch (Exception e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	}	finally{
		
		try {
			// Kill container
			docker.killContainer(id);
			// Remove container
			docker.removeContainer(id);
			// Close the docker client
			docker.close();
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (com.spotify.docker.client.DockerException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

}

	return execOutput;
	
}

}

I have mentioned my comments the program where exactly i need to pass the input. I am using the below command to run the java program.
javac HelloWorld.java && java -cp . HelloWorld exit

[Cydhra]

The Workaround provides you with a method, that takes a LogStream object (like your "output" variable). It then returns a WriteableByteChannel, that can either be transformed into an OutputStream or you can write directly into it (see the Java Channel API).

If you choose to get an OutputStream of it, you can then write anything into this OutputStream which will be forwarded into the standard input of the Docker-Container. So if anything inside the container waits for user input, it will receive, what you write into the OutputStream.

For further reference, see SmartLambda, where I used this workaround. The package docker contains the code, that communicates with the application inside the container.

[ajmalrehman]

@Cydhra Thank you for replyin, you really raised my hope. I have done the changes as per your comment. I was getting some casting error in the hijacking file i was able to resolve those though. Though my code does not throw any error, i am not sure whether it is writing to the writable channel or not. Below is both of my code . If you could tell what I am doing wrong it would be of great help!!. Please find my code below.

package com.examenginedashboard.docker.utils;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.WritableByteChannel;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.spotify.docker.client.DefaultDockerClient;
import com.spotify.docker.client.DockerClient;
import com.spotify.docker.client.LogStream;
import com.spotify.docker.client.messages.ContainerConfig;
import com.spotify.docker.client.messages.ContainerCreation;
import com.spotify.docker.client.messages.ExecState;
import com.spotify.docker.client.messages.HostConfig;
import com.spotify.docker.client.messages.PortBinding;
public class CodeCompiler {
String id=null;
String execOutput = null;
DockerClient docker= null;
public String compileAndExecuteCode(String dockerimage,String[] command,String mountedpath){
try {
docker = DefaultDockerClient.fromEnv().build();
docker.pull(dockerimage);
final String[] ports = {"99", "9999"};
final Map<String, List> portBindings = new HashMap<String, List>();
for (String port : ports) {
List hostPorts = new ArrayList();
hostPorts.add(PortBinding.of("0.0.0.0", port));
portBindings.put(port, hostPorts);
}
List randomPort = new ArrayList();
randomPort.add(PortBinding.randomPort("0.0.0.0"));
portBindings.put("443", randomPort);
final HostConfig hostConfig = HostConfig.builder().binds(mountedpath).
portBindings(portBindings).build();
final ContainerConfig containerConfig = ContainerConfig.builder()
.hostConfig(hostConfig)
.image(dockerimage).exposedPorts(ports)
.cmd("sh", "-c", "while :; do sleep 1; done")
.build();

		final ContainerCreation creation = docker.createContainer(containerConfig);
		 id = creation.id();
		 String uri ="unix:///var/run/docker.sock";
		// Start container
		docker.startContainer(id);
		// Exec command inside running container with attached STDOUT and STDERR
		System.out.println(docker.inspectContainer(id));
		
		//final String[] command = {"bash", "-c", "cd mydockerbuild/ && javac "+filename+"  && java -cp . "+executablefilename+" exit"};
		System.out.println(command);
		final String execCreation = docker.execCreate(
		    id, command,DockerClient.ExecCreateParam.attachStdin(),DockerClient.ExecCreateParam.attachStdout(),
		    DockerClient.ExecCreateParam.attachStderr());
		final ExecState execinfo =docker.execInspect(execCreation.toString());
		System.out.println("docker status before execution......... "+execinfo);
		final LogStream output = docker.execStart(execCreation);
			System.out.println("Waiting for user input....................");
			String in ="rock";
			WritableByteChannel out = HttpUserDynamicCodeInput.getOutputStream(output,uri);
			ByteBuffer b = ByteBuffer.wrap(in.getBytes());
			out.write(b);
		final ExecState execinfo3 = docker.execInspect(execCreation.toString());
		System.out.println("docker status after execution...."+execinfo3);
		try{
			execOutput = output.readFully();
			}catch(RuntimeException e){
				e.printStackTrace();
			}
			System.out.println("Server Response............."+ execOutput);
	} catch (InterruptedException e) {
		// TODO Auto-generated catch block
		e.printStackTrace();
	} catch (com.spotify.docker.client.DockerCertificateException e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	} catch (com.spotify.docker.client.DockerException e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	} catch (Exception e1) {
		// TODO Auto-generated catch block
		e1.printStackTrace();
	}	finally{
		try {
			// Kill container
			docker.killContainer(id);
			// Remove container
			docker.removeContainer(id);
			// Close the docker client
			docker.close();
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (com.spotify.docker.client.DockerException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}		

}
return execOutput;
}
}
------------------------------------------below is your that i changed a little bit--------------------------------------

package com.examenginedashboard.docker.utils;

/*******************************************************************************
 * Copyright (c) 2015 Red Hat.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * Contributors:
 *     Red Hat - Initial Contribution
 *******************************************************************************/

import java.io.FilterInputStream;
import java.lang.reflect.Field;
import java.net.Socket;
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import java.util.LinkedList;
import java.util.List;

import org.apache.http.conn.EofSensorInputStream;
import org.apache.http.entity.BasicHttpEntity;
import org.apache.http.entity.HttpEntityWrapper;
import org.apache.http.impl.io.IdentityInputStream;
import org.apache.http.impl.io.SessionInputBufferImpl;
import org.glassfish.jersey.message.internal.EntityInputStream;

import sun.nio.ch.ChannelInputStream;

import com.spotify.docker.client.LogReader;
import com.spotify.docker.client.LogStream;

/**
 * This is a workaround for lack of HTTP Hijacking support in Apache
 * HTTPClient. The assumptions made in Apache HTTPClient are that a
 * response is an InputStream and so we have no sane way to access the
 * underlying OutputStream (which exists at the socket level)
 * <p>
 * References :
 * https://docs.docker.com/reference/api/docker_remote_api_v1.16/#32-hijacking
 * https://github.com/docker/docker/issues/5933
 * <p>
 * This document was altered to work with the latest versions of spotify.docker, apache.httpclient and all their dependencies
 */
public class HttpUserDynamicCodeInput {
	
	/**
	 * This is a utility class and shall not be instantiated
	 */
	private HttpUserDynamicCodeInput() {
		
	}
	
	/**
	 * Get a output stream that can be used to write into the standard input stream of  docker container's running process
	 *
	 * @param stream the docker container's log stream
	 * @param uri    the URI to the docker socket
	 *
	 * @return a writable byte channel that can be used to write into the http web-socket output stream
	 *
	 * @throws Exception on any docker or reflection exception
	 */
	static WritableByteChannel getOutputStream(final LogStream stream, final String uri) throws Exception {
		// @formatter:off
		final String[] fields =
				new String[] {"reader",
				              "stream",
				              "original",
				              "input",
				              "in",
				              "in",
				              //"in",
				              "eofWatcher",
				              "wrappedEntity",
				              "content",
				              "in",
				              "instream"};
		
		final String[] containingClasses =
				new String[] {"com.spotify.docker.client.LogStream",
				              LogReader.class.getName(),
				              "org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream",
				              EntityInputStream.class.getName(),
				              FilterInputStream.class.getName(),
		                      FilterInputStream.class.getName(),
		                     // FilterInputStream.class.getName(),
		                      EofSensorInputStream.class.getName(),
		                      HttpEntityWrapper.class.getName(),
		                      BasicHttpEntity.class.getName(),
		                      IdentityInputStream.class.getName(),
		                      SessionInputBufferImpl.class.getName()};
		// @formatter:on
		
		final List<String[]> fieldClassTuples = new LinkedList<>();
		for (int i = 0; i < fields.length; i++) {
			fieldClassTuples.add(new String[] {containingClasses[i], fields[i]});
		}
		
		if (uri.startsWith("unix:")) {
			fieldClassTuples.add(new String[] {ChannelInputStream.class.getName(), "ch"});
		} else if (uri.startsWith("https:")) {
			final float jvmVersion = Float.parseFloat(System.getProperty("java.specification.version"));
			fieldClassTuples.add(new String[] {"sun.security.ssl.AppInputStream", jvmVersion < 1.9f ? "c" : "socket"});
		} else {
			fieldClassTuples.add(new String[] {"java.net.SocketInputStream", "socket"});
		}
		
		final Object res = getInternalField(stream, fieldClassTuples);
		if (res instanceof WritableByteChannel) {
			return (WritableByteChannel) res;
		} else if (res instanceof Socket) {
			return Channels.newChannel(((Socket) res).getOutputStream());
		} else {
			throw new AssertionError("Expected " + WritableByteChannel.class.getName() + " or " + Socket.class.getName() + " but found: " +
					res.getClass().getName());
		}
	}
	
	/**
	 * Recursively traverse a hierarchy of fields in classes, obtain their value and continue the traversing on the optained object
	 *
	 * @param fieldContent     current object to operate on
	 * @param classFieldTupels the class/field hierarchy
	 *
	 * @return the content of the leaf in the traversed hierarchy path
	 */
	private static Object getInternalField(final Object fieldContent, final List<String[]> classFieldTupels) {
		Object curr = fieldContent;
		for (final String[] classFieldTuple : classFieldTupels) {
			//noinspection ConstantConditions
			final Field field;
			try {
				field = Class.forName(classFieldTuple[0]).getDeclaredField(classFieldTuple[1]);
				field.setAccessible(true);
				curr = field.get(curr);
			} catch (NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
				e.printStackTrace();
			}
		}
		return curr;
	}
}

[ajmalrehman]

@Cydhra Here I am again, May be i didn't appreciate, I apologize for that. Actually your code helped alot and very close to what I am trying to achieve and your help can lead me upto there.If you could suggest me it would be very helpful.

Following is the way i am wriiting into the channel.

	String in ="rock";
			InputStream inputstream = new ByteArrayInputStream(in.getBytes(StandardCharsets.UTF_8));
			WritableByteChannel channel = HttpUserDynamicCodeInput.getOutputStream(output,uri);
			/*ByteBuffer b = ByteBuffer.wrap(in.getBytes());
			out.write(b);
			out.close();*/
			 // Create a direct ByteBuffer;
		    ByteBuffer buffer = ByteBuffer.allocateDirect(10);		 
		    byte[] bytes = new byte[1024];
		    int count = 0;
		    int index = 0;
		    // Continue writing bytes until there are no more
		    while (count == 0) {
		        if (index == count) {
		            count = inputstream.read(bytes);
		            index = 0;
		        }
		        // Fill ByteBuffer
		        while (index < count && buffer.hasRemaining()) {
		            buffer.put(bytes[index++]);
		        }
		        buffer.flip();
		        // Write the bytes to the channel
		        int numWritten = channel.write(buffer);
		        System.out.println("Number of bytes wrutten ..."+numWritten);

So basically I am wrinting string "rock" into the channel which is an input to the program. Once the input is passed the program will execute and I am sure it must be getting executed. But there is no way i could find how to get the final executed program output. Please suggest something it will be of great help.

Thank you very much for everything you provided so far.

Thanks in advance.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dmandalidis]

Hi all,

Since this project went on mature status, please re-open this issue (if it still stands) to https://github.com/dmandalidis/docker-client. Thanks