CopyFile EOF error

[GopherJ]

No description provided.

[GopherJ]

I trited https://github.com/viant/afs/tree/master/scp, it doesn't have this issue

[bramvdbogaerde]

Thanks for reporting, however I need more information
to be able to reproduce this locally,
could you provide me with a file, and code you tried this with?

[GopherJ]

hi @bramvdbogaerde the code is the same with your example. The file is a 4MB's gzipped package, sorry I cannot provide you the file

this error is random, but it happened 4 times over 5, so I think there must be something wrong

[mchepukov]

Hi, I can reproduce this, how we can talk, I'll show you.

[bramvdbogaerde]

@mchepukov Great, could you try to write down the steps to reproduce it as a comment on this Github issue?
Thanks!

[mchepukov]

package main

import (
	"flag"
	"fmt"
	"io"
	"log"
	"os"
	"time"
	scp "github.com/bramvdbogaerde/go-scp"
	"golang.org/x/crypto/ssh"
)

func main() {
	start := time.Now()
	log.Printf("Start execution")

	hostname := flag.String("hostname", "192.168.0.1", "hostname or ip address of device")
	user := flag.String("user", "root", "user for ssh connection")
	port := flag.String("port", "22", "ssh port")
	password := flag.String("password", "password", "ssh password")
	flag.Parse()

	connString := fmt.Sprintf("%s:%s", *hostname, *port)

	config := &ssh.ClientConfig{
		User: *user,
		Auth: []ssh.AuthMethod{
			ssh.Password(*password),
		},
		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
	}

	conn, err := ssh.Dial("tcp", connString, config)
	fmt.Printf("Connect to host %s to port %s \n", *hostname, *port)
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
	fmt.Printf("Connection to host %s established \n", *hostname)

	runCommand("mkdir -p /root/test/python/", conn)

	copyFileToRemote("Python-3.9.0.tgz", "/root/test/python/", "0644", conn)

	//copy ca-certificates.crt
	fmt.Println("Create dir /root/test/etc/ssl/certs/")
	runCommand("mkdir -p /root/test/etc/ssl/certs/", conn)

	copyFileToRemote("ca-certificates.crt", "/root/test/ssl/certs/", "0755", conn)

	log.Printf("Execution took: %s", time.Since(start))
	defer conn.Close()
}

func runCommand(cmd string, conn *ssh.Client) {

	fmt.Println("Execute command: " + cmd)

	sess, err := conn.NewSession()
	if err != nil {
		panic(err)
	}
	defer sess.Close()
	sessStdOut, err := sess.StdoutPipe()
	if err != nil {
		panic(err)
	}

	go io.Copy(os.Stdout, sessStdOut)
	sessStderr, err := sess.StderrPipe()
	if err != nil {
		panic(err)
	}
	go io.Copy(os.Stderr, sessStderr)
	err = sess.Run(cmd) // eg., /usr/bin/whoami
	if err != nil {
		panic(err)
	}
}

func copyFileToRemote(fileName string, remotePath string, permissions string, conn *ssh.Client) {
	client, err := scp.NewClientBySSH(conn)
	if err != nil {
		fmt.Println("Error creating new SSH session from existing connection", err)
	}

	f, err := os.Open(fileName)

	if err != nil {
		fmt.Println(err)
		os.Exit(1)

	}

	fi, err := f.Stat()
	if err != nil {
		// Could not obtain stat, handle error
	}
	fmt.Println("Copy " + fileName + " to " + remotePath)
	fmt.Printf("The file is %d bytes long \n", fi.Size())

	err = client.Copy(f, remotePath+fileName, "0644", fi.Size())

	//err = client.CopyFile(f, remotePath+fileName, permissions)

	if err != nil {
		fmt.Println("Error while copying file ", err)
		os.Exit(1)
	}

	defer f.Close()
	defer client.Close()
}

[bramvdbogaerde]

Great, thanks for the code snippet. I will let you know if I was able to reproduce the bug locally.

[mchepukov]

Hello, may be you have good news for us about this issue?

[kolapapa]

any progress?

[sfc-gh-ssudakovich]

Had similar issue. My scp client did not support -q. Removing from the line below fixes the issue for me

err := a.Session.Run(fmt.Sprintf("%s -qt %q", a.RemoteBinary, remotePath))

[dshemin]

I experience the same issue with Hetzner Storage Box.

[MyMarvel]

I assume it is an issue when you copy a file to a destination of another already existent file. Is there a way to tell the library to override the file?

[hell0w0orld]

I encountered the same problem

[lysShub]

It seems to be a flaw in the function checkResponse, my code

 scp.Copy(context.Background(), fh, dst, "744", info.Size()) // incorrect permissions

will return EOF immediately, but checkResponse passed；when fix to "0744" it work expected。

maybe stdout read race between checkResponse and https://github.com/golang/crypto/blob/v0.27.0/ssh/mux.go#L215