Wrong date? #6
Comments
|
Interesting... Unfortunately, I have no clue about what it could be. I just tried arm32v7/debian on Windows 10 (Docker Desktop includes QEMU for ARM targets), and results are correct: # docker run --rm -it debian date
Mon Nov 2 15:58:42 UTC 2020
# docker run --rm -it i386/debian date
Mon Nov 2 15:58:50 UTC 2020
# docker run --rm -it arm32v7/debian date
Mon Nov 2 15:58:56 UTC 2020
# docker run --rm -it arm64v8/debian date
Mon Nov 2 15:59:11 UTC 2020Is the mismatch you found consistent using different images? |
|
Yes, Alpine shows the same problem: Here is a thing that I noticed that might help find the issue? |
|
I have the same issue. I've noticed that the date inside the container is different according to the distribution launched, and it doesn't change over time. All sorts of things, from ping to apt, fail to run properly because of this problem. |
|
@HimbeersaftLP , @struanb , did you both use Raspbian or some other host OS? |
|
@umarcor Raspbian 32bit on a Pi 4 |
Background (maybe off-topic)I'm trying to run the x86_64 deno docker container. On Raspbian I was compiling QEMU with ForegroundWith the use of this repository, I end up with: I have no idea what's going on, but maybe it will help someone: I was trying to do something with Maybe the problem is with Raspberry Pi kernel? Or the |
I'm not sure to understand what are you trying to do here. Did you build QEMU, start a QEMU VM (qemu-system) and try building, installing and running Docker in there? Although that might theoretically work, that is very far from the expected use case in this repo/project. See https://dbhi.github.io/qus/#context-docker-and-qemu. The concept of qus is that you a start with some host where you can run regular native containers without issues. Hence, on Rpi you would install docker through Installing QEMU otherhow, and/or building/installing Docker with custom options is out of the scope of this project, because that is precisely what we are trying to avoid. Calling the aptman/qus container works on Windows or Linux, with Podman or Docker, "regardless" of the host architecture. Introducing custom build procedures would conflict with that approach. Naturally, constraining qus to that scope does involve limitations regarding usability in non-trivial applications. Don't take me wrong, I'm ok with you sharing your experiments in this regard, as it is always interesting to be aware of which other approaches do work or which don't. However, I think it'd be better located in a separated issue.
I don't think this is the same issue. Other users could succesfully run foreign images and execute commands inside. The problem is that commands/tools relying on time had issues. Honestly, I did not see that error before. Where is that Rust code from? AFAIK, neither Docker nor QEMU are written in Rust. Is it possible that the tool inside the container is written in Rust and that one is crashing instantly? If that is the case, I would suggest running the tool on the host first. That is, get an x86_61/amd64 build/binary/tarball and try running it on the RPi after executing aptman/qus. As a matter of fact, the interpreters registered by qus are useful for either containers or host applications.
My perception is that Raspbian is partially broken in several ways. On the one hand, it's a 32 bit OS, so that it can be executed on any RPi (the earliest versions don't support 64 bit). However, arm64v8 containers can be successfully executed on Raspbian, using the default docker installation. I would expect that to have corner cases, and handling time might be one of those. On the other hand, AFAIK raspbian is not free; some software components are included as binary blobs. In these embedded devices, interacting with peripherals is typically non-standard and finding bugs in those blobs can be really hard. I guess that's why other distributions (Fedora, Ubuntu, Arch, etc.) did not (announce) support for RPi until several years later. Nevertheless, other users have reported successfully running x86/amd64 containers on RPi: #2 (comment). @jrcichra, would you kindly let us know whether you used Raspbian or some other OS? |
Yes, I know, I've written it here:
Also, I was trying to do it with the qus project, and I failed. So I began to try it on my own with full QEMU emulation. But it is so slow that the Docker can't start. And it's all being my background. I wanted to say what I want to achieve and how I'm trying to do it now. It's all off-topic, but for now, I can tell that the problem with no-ticking time does not occur on RPi with full-emulated QEMU system. I don't know if it's useful, but I think it's worth mentioning that the problem is not in QEMU itself. But maybe it's in
Do you really want me to open the issue with "the another approach"? 😄
The deno is written i. e. in Rust. And it's an issue with time, see So it's container-dependent and will only occur when someone run Rust, but it's related to no-ticking time in a qus container on RPi.
It is not possible due to denoland/deno#2295. There is no support for my host CPU, and I think there is no hope for it for the next two months. But on my PC and other servers running x86_64 natively, it's working fine. I tried running different commands on RPi, and I thought it could help someone with no direct access to RPi when I share some of my guessed commands (and its outputs) here. I would be glad if I could help somehow with a running RPi. Sorry if I misled some concepts with userspace emulation (it's out of my mind). And thanks for the outstanding answer, I wouldn't have expected that. |
I would expect that because qemu-system and qemu-user are, AFAIAA, different codebases with different features. In a full-emulated system, the host kernel is not used. It's a complete VM, isolated from the host. However, with qemu-user, the kernel is the one on the host, only instructions are translated (dynamically modified). Therefore, issues/bugs on one system are rarely reproducible on the other.
Honestly, there is no much activity in the issues of this repo. I think it is harmless if you want to use one issue for that or for discussing experiments related to Docker and QEMU. For instance, it should theoretically be possible to convert containers to qemu-system images and vice versa. Not straightforward, probably ugly, but possible. Naturally, if you want to use a gist, that's awesome too.
Now I got it, and it makes sense to be here. Thanks for clarifying.
That's why you are setting up a qemu-user interpreter with qus first 😉 When the kernel calls the qemu-user interpreter for translating (in this case) amd64 instructions to armhf/aarch64, it is independent from the instructions coming from inside a container or from the host. Therefore, you can: docker run --rm --privileged aptman/qus -s -- -p amd64
curl -o app URL_TO_YOUR_AMD64_EXECUTABLE
./appYou don't get the isolation that containers provide, but you can run foreign tools. Trying this will let us know whether the problem has anything to do with docker, or it's an issue about qemu-user and raspbian only. BTW, you could run the same test after installing qemu-user through
No worries at all, and you are welcome! |
It ends up with TestingI tried to get deno executable: When I provided the file, I kept getting errors like: There are commands to provide all libs (which were mentioned in errors): and finally I got: I tried installing libs from debian repos, but I couldn't: I also found something like and I found: so it's specific for RPi. I have no chance to find it on amd64 platform, do I? I also tried with more trivial program and I built it on another machine: and on RPi: So it works, but there is this strange error. I don't know why I can't run Deno. Maybe I don't have one lib? I've got the same result with qemu-user from |
You are correct. I have "architecture nomalisation" functions in CI scripts, but the QEMU naming must be used when running the container. Sorry about that.
Ok. This is partially good. I forgot to specify that the x86_64/amd64 binary must be statically compiled/linked. The errors you are getting are because the app is dynamically linked. Therefore, as soon as you run it, it tries to first load the shared libraries it depends on, but those don't exist because it is referring the locations/names on a different system. Therefore, you are potentially missing multiple You might try finding all the missing libs in the repos, or in your workstation and copying all of them manually. However, that's kind of stupid because chroot's and containers are precisely meant for that purpose. We do not want to reinvent that wheel! Can you try building
This is nice! Let's start with this example. I believe that just adding
This is good, as expected. I don't think it's worth building from sources, unless you find a bug report which is fixed on the development branch. At the moment, the context is that qus, qemu-user through I think that the main path we should follow now is the minimal C program you wrote. If you are good, let's continue investigating that. |
Positive, it just works. |
|
Hmmm... this gets interesting. It seems to be a problem with docker and qemu-user, indeed. Can you please try running that same binary inside a container? docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu:18 ./dateLet's see if we can get a MWE to fail, instead of "regular" tools with many lines of code. |
|
|
Awesome! Thank you very much for getting here! I would propose first confirming that the issue is independent from qus by checking it with qemu-user installed through
|
|
Oh, it's a surprise. I ran and: Probably I forgot to unregister qus when I was installing qemu-user-static through apt before. |
|
Wait, where is my Raspbian sources... I just double-checked with original |
|
It's interesting. I tried with Deno, it ends up with mentioned Logs (qemu from apt)isolated Logs (qus) |
|
Can you please confirm that you are using the latest version of qus? # docker run --rm -it --entrypoint=sh aptman/qus -c '/qus/bin/qemu-x86_64-static --version'
qemu-x86_64 version 5.2.0 (Debian 1:5.2+dfsg-3)
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers |
|
Positive, the same output. |
|
I guess some testing script needs to be setup for testing it several times with each approach. The only lead I have at the moment is that all failures seem to provide either It's surprising both that the years are two only and that it's not one only. It's not different users reporting different years. You got both of those on the same device too! |
I tried building qemu from sources (the same result like from apt) |
|
The binaries that qus uses are the ones from http://ftp.debian.org/debian/pool/main/q/qemu/. Currently, these might be the ones being used:
According to your logs ( I would expect these binaries to be slightly different from the ones you got when building QEMU from sources, since you used
EDIT The tarball you can download from https://github.com/dbhi/qus/releases/tag/v0.0.6-v5.2%2Bdfsg-3 column arm32v7, row x86_64, should be the same as the one |
I have written the following script to check which dates I can get: test-distro() {
echo -en " |\n| $1 | ";
sudo docker run --rm -it $1 /bin/sh -c "date | tr -d '\r' | tr -d '\n'; sleep 10; echo -n ' | '; date | tr -d '\r' | tr -d '\n';";
}
test() {
sudo docker run --rm --privileged aptman/qus -s -- -p $1 > /dev/null
for distro in ubuntu:20.04 ubuntu:18.04 ubuntu:14.04 alpine centos centos:6 debian debian:8 amazonlinux fedora cirros clearlinux; do
test-distro $2/$distro
done;
# sudo docker system prune -af; # CAUTION! it will delete all docker-related stuff
}
( echo -en "| distro | date | date after 10s |\n| - | - | -";
test x86_64 amd64;
test i386 i386;
test aarch64 arm64v8;
echo " |"; ) | tee out; echo -e '\n\n\n'; cat out;and I left RPi working through the night, but today I can't connect to it, although it seems to be a connection issue (I don't have it physically, I have borrowed it from @Marwyk2003, thanks).
If I have done everything correctly, the result of using my built QEMU is the same as the one of
It's a nice idea! I will try when I have access.
Why? I don't know, I just found it here. I used it when was compiling QEMU for full system emulation, and it worked. But I don't know if this would work without this option.
As far as I know, I can't. Raspberry Pi 3 Model B Rev 1.2 provides me |
|
Unless you actually downloaded the DEB package from that URL, I wouldn't be sure about what you get through
It seems that you copied the options corresponding to
The RPi3 has a 64 bit CPU, which can handle armv7 and armv8 architectures (as shown in the gentoo wiki). Therefore, it can run 64 bits. You get ARMv7 from the system because Raspbian is built for 32 bits only, so that the same SD works on any RPi. However, I could successfully run arm64v8 containers on Raspbian. Just try |
|
Nope. My |
|
Okay... and and and If someone wants to reproduce it, I sometimes have |
|
I have observed that all versions mentioned above fail with |
I tried building QEMU with |
|
Experiencing the exact same issue on another Raspberry Pi 4, date stuck in 2004 and containers crashing Edit: I am also able to confirm that installing the qemu-user-static from buster backports somehow sidesteps this issue |
Using Raspbian 32bit on a Pi 4, the date inside the docker does not match the system date.
The text was updated successfully, but these errors were encountered: