What is the recommended upgrade procedure?

[dolphinkite]

TLDR: What is the recommended upgrade procedure?

Bonjour Philippe,

I've been using this docker container for many years now (inside a Synology NAS), but recently, I've been unable to upgrade to the latest version without completely redoing the configuration from scratch.

I did view your comment: "starting with Unifi Video 3.10.x, in-place upgrades are not fully supported. It's best to generate and save a backup file, and remove your data folder. Then restore the backup from the web setup page." but can you share more information on how to achieve this?

I used to map 2 folders, one for my data (-v /volume1/docker/unifi-video/data:/var/lib/unifi-video) and another one for my recordings (-v /volume1/docker/unifi-video/recordings:/var/lib/unifi-video/videos). Before 3.10, I would simply delete the container, create a new one and everything would be fine.

Since 3.10 (I assume), it doesn't work. I did try to backup my config (Settings --> Download configuration). Then delete my data folder (/volume1/docker/unifi-video/data) and from now on, only map my recorded videos folder when I run the container.

At the welcome page, I chose restore config and this brings me a working installation of the controller. But my camera cannot connect to the controller. At times, I even can't connect directly to my camera (my old password doesn't work, nor ubnt/ubnt). I do a master reset on the camera, and then manage (sometimes) to have it connect to the controller (tried the adoption token, with mixed results). But when I do, none of the settings (motion detection zone, etc.) are kept and I must configure the camera again. Also, none of my previous recordings are visible (although I can see they are on disk). Even weirder, when I go to System --> System configuration, it states that the recording path is /usr/lib/unifi-video/data/videos but I think it used to be var/lib/unifi-video/videos. Changing the path there doesn't fix the situation - my old recordings are still not visible within the controller.

Can you explain in more details the recommended steps to upgrade the controller?

As a side note, for the past few years, I've been windering why it is impossible to use the update mechanism that is built in unifi-video? I've tried it many times (as others I'm sure) when I got a message that a new version of Unifi Video is available inside the controller, but it inevitaly fails, I'm sure there is a logical explanation but I don't understand why..?

Thanks for your help.

[fryfrog]

Docker containers are meant to be disposable, if you update one and then it goes away and you re-run the old one, your internal settings will be fore the new version, but your software will be old.

I've been using the same database and config through all the versions of the image, so I honestly don't understand the issues others have been having. :/

[dolphinkite]

I'm not sure I follow you. How would it just "go away"? I agree with you that they are disposable, and the usual procedure is to delete one and then run the new version.

But my side question was: Why can't I keep one container (and not make it "go away") and simply upgrade the firmware inside the container... Why isn't that working.

But, more importantly, can you tell me what procedure you are using to upgrade to a new version of the container once it is release.. while keeping your old camera settings and recordings. Maybe a bullet point list would help me very much.

Thanks

[thoschworks]

But, more importantly, can you tell me what procedure you are using to upgrade to a new version of the container once it is release.. while keeping your old camera settings and recordings. Maybe a bullet point list would help me very much.

I think this question is not specific on this image but it's about Dockers basics.

By using bind mounts (as in the example for docker runin the REAMDME.md) or volumes (both specified with --volumeor -v) you can define a method (and location) for storing data outside of the container.

If the parameters are set properly, the data will be save, when the container is destroyed. And the new container will find the data and use them.

If this doesn't work with your setup, it would be useful if you post your complete run command with all parameters.

[fryfrog]

Yeah, you need to be saving your config data outside the container which is what the example docker run shows. My updates are just docker pull the new one and then restart my container, because I use docker run --rm which deletes the container every time it stops.

Show us your docker run.

[dolphinkite]

Well, just to clarify, I've been using this image for the past 3,5 years on my Synogy NAS and it worked relatively well (including preserving videos and configuration) up until (I suppose?) version 3.10.X. Up until then, I was using 2 mounted drives, one for my data and another one for the videos.

To quote the documentation: "Starting with Unifi Video 3.10.x, in-place upgrades are not fully supported. It's best to generate and save a backup file, and remove your data folder. Then restore the backup from the web setup page."

From then on, it hasn't been smooth for me. At all. Including exporting the config (which is a pain because I would sometimes forget to do the export before deleting the container...) and then trying to restore it in the new image.

Here is my CURRENT docker run command:

docker run -tdi --restart=always --name unifi-video --security-opt apparmor:unconfined --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH -p 1935:1935 -p 6666:6666 -p 7080:7080 -p 7442:7442 -p 7443:7443 -p 7444:7444 -p 7445:7445 -p 7446:7446 -p 7447:7447 -v /volume1/docker/unifi-video/recordings:/var/lib/unifi-video/videos -e "TZ=EST" -e PUID=99 -e PGID=100 -e DEBUG=1 pducharme/unifi-video-controller

The way I USED to do it was with this command (preserving my settings):
docker run -tdi --restart=always --name unifi-video --security-opt apparmor:unconfined --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH -p 1935:1935 -p 6666:6666 -p 7080:7080 -p 7442:7442 -p 7443:7443 -p 7444:7444 -p 7445:7445 -p 7446:7446 -p 7447:7447 -v /volume1/docker/unifi-video/data:/var/lib/unifi-video -v /volume1/docker/unifi-video/recordings:/var/lib/unifi-video/videos -e "TZ=EST" -e PUID=99 -e PGID=100 -e DEBUG=1 pducharme/unifi-video-controller

[fryfrog]

Your current docker run isn't preserving your config folder, so every time it gets removed you'd lose all your config. Could it be as simple as that? And doesn't your -ti mean it runs in console mode?

                   docker run --rm \
                              --name unifi-video \
                              --cap-add DAC_READ_SEARCH \
                              --cap-add NET_BIND_SERVICE \
                              --cap-add SYS_PTRACE \
                              --cap-add SETUID \
                              --cap-add SETGID \
                              -p 10001:10001 \
                              -p 1935:1935 \
                              -p 6666:6666 \
                              -p 7080:7080 \
                              -p 7442:7442 \
                              -p 7443:7443 \
                              -p 7444:7444 \
                              -p 7445:7445 \
                              -p 7446:7446 \
                              -p 7447:7447 \
                              -v /data/unifi-video/data:/var/lib/unifi-video \
                              -v /data/unifi-video/videos:/var/lib/unifi-video/videos \
                              --tmpfs /var/cache/unifi-video \
                              -e JVM_MX=2048M \
                              -e UMASK=002 \
                              -e PUID=957 \
                              -e PGID=957 \
                              -e CREATE_TMPFS=no \
                              -e TZ=America/Los_Angeles \
                              -e DEBUG=1 \
                              pducharme/unifi-video-controller:testing

^ This is mine, of note is that the config/db inside at /var/lib/unifi-video is being saved somewhere and I'm using the new --tmpfs /var/cache/unifi-video and -e CREATE_TMPFS=no options. I also use the --rm option because it really drives home the point that containers are disposable.

[dolphinkite]

Thanks for your response.

I USED to preserve the config... but a while ago I started having issues (don't remember what they were) so I went back to the documentation and this is when I saw the paragraph I quoted. The author specifically stated that I needed to EXPORT the config and IMPORT it again...

Now, thinking about it again, perhaps he meant that I only need to do this procedure ONCE (to go from 3.9 to 3.10)?!? I understood from that text that I needed to export and import my config EACH TIME a new image is available. that was a pain. Now, with your comments.. I'm thinking that I can go back to how I was doing things (2 mounted drives preserving config and recordings).

That would certainly be good news. I'll try it tomorrow.

As for the -ti, if I remember correctly, I think that it is because I run the contained from a SSH shell, and then I need to close that shell. If I don't use those commands, the shell needs to remain connected.

Thanks for helping me on this. Hopefully that settles it.

[fryfrog]

Yeah, I think that was for the 3.9 -> 3.10 transition where they went from whatever the old mongodb format was to the new wired tiger format.

Why are you running it from a shell at all? I'm weird and like systemd .service files, so that is how I run mine. Looks like...

# /etc/systemd/system/unifi-video.service
[Unit]
Description=Ubiquiti UniFi Video Server
After=network.target zfs.target docker.service
Requires=docker.service zfs.target

[Service]
Restart=on-failure
Type=simple
ExecStart=/usr/bin/docker run --rm \
                              --name unifi-video \
                              --cap-add DAC_READ_SEARCH \
                              --cap-add NET_BIND_SERVICE \
                              --cap-add SYS_PTRACE \
                              --cap-add SETUID \
                              --cap-add SETGID \
                              -p 10001:10001 \
                              -p 1935:1935 \
                              -p 6666:6666 \
                              -p 7080:7080 \
                              -p 7442:7442 \
                              -p 7443:7443 \
                              -p 7444:7444 \
                              -p 7445:7445 \
                              -p 7446:7446 \
                              -p 7447:7447 \
                              -v /data/unifi-video/data:/var/lib/unifi-video \
                              -v /data/unifi-video/videos:/var/lib/unifi-video/videos \
                              --tmpfs /var/cache/unifi-video \
                              -e JVM_MX=2048M \
                              -e UMASK=002 \
                              -e PUID=957 \
                              -e PGID=957 \
                              -e CREATE_TMPFS=no \
                              -e TZ=America/Los_Angeles \
                              -e DEBUG=1 \
                              pducharme/unifi-video-controller:testing
                              #fryfrog/unifi-video-controller:testing

ExecStop=/usr/bin/docker stop -t 30 unifi-video

[Install]
WantedBy=default.target

But obviously, you'd need to tune that to your own setup.

[dolphinkite]

Why are you running it from a shell at all?

Mostly because I didn't know any better. I would SSH to my NAS and from there I would create and delete containers.

Also, because it isn't running "standard linux" but Synology's DSM (their OS, based from Linux), I have been wary of changing things in system directories that can eventually get modified during an update. I've had bad experiences when I "reinforced" my openVPN (trying to harned the basic settings that comes in the Synology package). My custom settings would get deleted.

Are you also running your container on a Synology NAS?

PS: I can't believe this whole time I though I needed to export/import the config. Perhaps the readme note should be clarified. I'm probably not the only one who got confused.

Thanks.

[fryfrog]

Oh, I see you're using Synology. Anything you do via CLI should get reflected in their GUI, you shouldn't need the -dit I don't think. And ignore my systemd service suggestion, that'd be for a plain old Linux. Why not just use their GUI to get the image going? You could then use a neat Docker image to get the run command it uses! :)

https://old.reddit.com/r/usenet/wiki/docker#wiki_get_docker_run_command

I just run this on my Arch Linux server, not Synology. :(

[thoschworks]

Why are you running it from a shell at all? I'm weird and like systemd .service files, so that is how I run mine. Looks like...

I think running containers as a service isn't usual (anymore?). Configured as a service Docker will take care that the containers are running, if you use a restart policy and @dolphinkite has already --restart=always in his run command to use this feature.

If this is not cool enough, you can use docker-compose. 😎

@dolphinkite I also start my containers from the shell and simply use -d. Why do you use the -iand ´-t` options?.

[fryfrog]

@thoschworks: Yeah, running containers from a systemd .service file is super weird. I actually only use Docker for a couple things, most of my stuff is native. So for me using a .service makes sense because I can control and set dependencies all the same.

I would totally not recommend it for most, docker-compose or maybe Portainer or even the system a Synology uses would make way more sense in virtually all cases. :)

[dolphinkite]

Hi,

Thanks all for the input. A few comments and answers to the various questions:

@fryfrog

I'm using the new --tmpfs /var/cache/unifi-video and -e CREATE_TMPFS=no

Can you explain the advantage of those options?

Anything you do via CLI should get reflected in their GUI

When I first started dabbling with Docker, over 4 years ago, the Synology GUI was very immature. Since this was new to me, I read a lot and it became clear that in my setup I needed to use the option --restart=always but that specific options (and others I think?) was not available in the GUI. Basically, most people suggested at the time to create and manipulate the containers from a SSH shell and disregard the Synology GUI. This may have evolved quite a bit since (and I do see an "enable auto-restart" option in the GUI now) so I may revisit... but I'm now used to doing it command line style.

@thoschworks

Why do you use the -iand ´-t` options?

Again, it goes back 4 years ago. If I remember correctly, those options allowed me to "connect to the container" later and do things like edit files. For example, rereading my own documentation that I wrote while building those docker files, at some point (when I first created the container? I don't even remember) I needed to edit one of the files from within the container. The code in my notes is the following:
docker exec -ti unifi-video bash echo "is_default=true" > data/system.properties exit
But perhaps you are totally right and it is not required anymore (?).

Thanks all for your input.