failed test for snazzer-send-wrapper #33

florianjacob · 2016-10-17T09:51:23Z

When executing the new make test, the following test fails:

 ✗ sudo -n snazzer --list-snapshots '--all' 'foo=" some stuff "' 'hel'\\'' squot '\\''lo' 'asd " dquot " fgh' 'ap ple' ' bon'\\''squot'\\''jour' 'there'
   (in test file tests/snazzer-send-wrapper.bats, line 45)                                                                                                                                                          
     `[ "$status" = "0" ]' failed

I'm not sure what the problem is there, though.

Related: Did you try to run all tests and not only snazzer-prune-candidates via travis, maybe the beta image with Ubuntu 14.04 contains a new-enough btrfs for the tests to work?

The text was updated successfully, but these errors were encountered:

csirac2 · 2016-10-17T10:14:26Z

Unfortunately, sstephenson/bats is great for running tests but not debugging them. My strategy originally was to add echo "$output" >>/tmp/something to the testcases. Perhaps we should switch to one of the better maintained forks of bats. Could you retry the tests with https://github.com/harschware/bats ?

PATH=.:$PATH bats -r -o /tmp/log tests/ seems to emit some useful stdout, and the it seems some stderr might be going to /tmp/log in my quick tests of the harschware fork.

Regarding travis, the real reason it isn't running the bats tests is because it requires root privileges to create and mount manipulate filesystems. I'm quite sure the containers in travis don't permit this sort of thing, at least they didn't last time I tried, admittedly a long time ago now... happy to be proven wrong. If I forked snazzer into a community group I help run then I could get a CI build VM running over there if that's interesting.

florianjacob · 2016-10-17T12:23:39Z

Managed to get travis to work, see #34, hurray. Can't reproduce the test failure there, though.

On the contrary, harschware/bats didn't work at all, it only produces the following output and then stalls forever for some reason:

[florian@izumo snazzer]$ PATH=.:$PATH bats -r -o /tmp/log tests/
[sudo] Passwort für florian: 
usage: btrfs [--help] [--version] <group> [<group>...] <command> [<args>]

    btrfs subvolume create [-i <qgroupid>] [<dest>/]<name>
        Create a subvolume
    btrfs subvolume delete [options] <subvolume> [<subvolume>...]
        Delete subvolume(s)
    btrfs subvolume list [options] [-G [+|-]value] [-C [+|-]value] [--sort=gen,ogen,rootid,path] <path>
        List subvolumes (and snapshots)
    btrfs subvolume snapshot [-r] [-i <qgroupid>] <source> <dest>|[<dest>/]<name>
        Create a snapshot of the subvolume
    btrfs subvolume get-default <path>
        Get the default subvolume of a filesystem
    btrfs subvolume set-default <subvolid> <path>
        Set the default subvolume of a filesystem
    btrfs subvolume find-new <path> <lastgen>
        List the recently modified files in a filesystem
    btrfs subvolume show <subvol-path>
        Show more information of the subvolume
    btrfs subvolume sync <path> [<subvol-id>...]
        Wait until given subvolume(s) are completely removed from the filesystem.

    btrfs filesystem df [options] <path>
        Show space usage information for a mount point
    btrfs filesystem du [options] <path> [<path>..]
        Summarize disk usage of each file.
    btrfs filesystem show [options] [<path>|<uuid>|<device>|label]
        Show the structure of a filesystem
    btrfs filesystem sync <path>
        Force a sync on a filesystem
    btrfs filesystem defragment [options] <file>|<dir> [<file>|<dir>...]
        Defragment a file or a directory
    btrfs filesystem resize [devid:][+/-]<newsize>[kKmMgGtTpPeE]|[devid:]max <path>
        Resize a filesystem
    btrfs filesystem label [<device>|<mount_point>] [<newlabel>]
        Get or change the label of a filesystem
    btrfs filesystem usage [options] <path> [<path>..]
        Show detailed information about internal filesystem usage .

    btrfs balance start [options] <path>
        Balance chunks across the devices
    btrfs balance pause <path>
        Pause running balance
    btrfs balance cancel <path>
        Cancel running or paused balance
    btrfs balance resume <path>
        Resume interrupted balance
    btrfs balance status [-v] <path>
        Show status of running or paused balance

    btrfs device add [options] <device> [<device>...] <path>
        Add a device to a filesystem
    btrfs device delete <device>|<devid> [<device>|<devid>...] <path>    btrfs device remove <device>|<devid> [<device>|<devid>...] <path>
        Remove a device from a filesystem
    btrfs device scan [(-d|--all-devices)|<device> [<device>...]]
        Scan devices for a btrfs filesystem
    btrfs device ready <device>
        Check device to see if it has all of its devices in cache for mounting
    btrfs device stats [-z] <path>|<device>
        Show current device IO stats.
    btrfs device usage [options] <path> [<path>..]
        Show detailed information about internal allocations in devices.

    btrfs scrub start [-BdqrRf] [-c ioprio_class -n ioprio_classdata] <path>|<device>
        Start a new scrub. If a scrub is already running, the new one fails.
    btrfs scrub cancel <path>|<device>
        Cancel a running scrub
    btrfs scrub resume [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
        Resume previously canceled or interrupted scrub
    btrfs scrub status [-dR] <path>|<device>
        Show status of running or finished scrub

    btrfs check [options] <device>
        Check structural integrity of a filesystem (unmounted).

    btrfs rescue chunk-recover [options] <device>
        Recover the chunk tree by scanning the devices one by one.
    btrfs rescue super-recover [options] <device>
        Recover bad superblocks from good copies
    btrfs rescue zero-log <device>
        Clear the tree log. Usable if it's corrupted and prevents mount.

    btrfs restore [options] <device> <path> | -l <device>
        Try to restore files from a damaged filesystem (unmounted)

    btrfs inspect-internal inode-resolve [-v] <inode> <path>
        Get file system paths for the given inode
    btrfs inspect-internal logical-resolve [-Pv] [-s bufsize] <logical> <path>
        Get file system paths for the given logical address
    btrfs inspect-internal subvolid-resolve <subvolid> <path>
        Get file system paths for the given subvolume ID.
    btrfs inspect-internal rootid <path>
        Get tree ID of the containing subvolume of path.
    btrfs inspect-internal min-dev-size [options] <path>
        Get the minimum size the device can be shrunk to. The
    btrfs inspect-internal dump-tree [options] device
        Dump tree structures from a given device
    btrfs inspect-internal dump-super [options] device [device...]
        Dump superblock from a device in a textual form
    btrfs inspect-internal tree-stats [options] <device>
        Print various stats for trees

    btrfs property get [-t <type>] <object> [<name>]
        Gets a property from a btrfs object.
    btrfs property set [-t <type>] <object> <name> <value>
        Sets a property on a btrfs object.
    btrfs property list [-t <type>] <object>
        Lists available properties with their descriptions for the given object.

    btrfs send [-ve] [-p <parent>] [-c <clone-src>] [-f <outfile>] <subvol> [<subvol>...]
        Send the subvolume(s) to stdout.
    btrfs receive [-ve] [-f <infile>] [--max-errors <N>] <mount>
        Receive subvolumes from stdin.

    btrfs quota enable <path>
        Enable subvolume quota support for a filesystem.
    btrfs quota disable <path>
        Disable subvolume quota support for a filesystem.
    btrfs quota rescan [-sw] <path>
        Trash all qgroup numbers and scan the metadata again with the current config.

    btrfs qgroup assign [options] <src> <dst> <path>
        Assign SRC as the child qgroup of DST
    btrfs qgroup remove <src> <dst> <path>
        Remove a child qgroup SRC from DST.
    btrfs qgroup create <qgroupid> <path>
        Create a subvolume quota group.
    btrfs qgroup destroy <qgroupid> <path>
        Destroy a quota group.
    btrfs qgroup show -pcreFf [--sort=qgroupid,rfer,excl,max_rfer,max_excl] <path>
        Show subvolume quota groups.
    btrfs qgroup limit [options] <size>|none [<qgroupid>] <path>
        Set the limits a subvolume quota group.

    btrfs replace start [-Bfr] <srcdev>|<devid> <targetdev> <mount_point>
        Replace device of a btrfs filesystem.
    btrfs replace status [-1] <mount_point>
        Print status and progress information of a running device replace
    btrfs replace cancel <mount_point>
        Cancel a running device replace operation.

    btrfs help [--full]
        Display help information
    btrfs version
        Display btrfs-progs version

Use --help as an argument for information on a specific group or command.
usage: mkfs.btrfs [options] dev [ dev ... ]
options:
        -A|--alloc-start START  the offset to start the FS
        -b|--byte-count SIZE    total number of bytes in the FS
        -d|--data PROFILE       data profile, raid0, raid1, raid5, raid6, raid10, dup or single
        -f|--force              force overwrite of existing filesystem
        -l|--leafsize SIZE      deprecated, alias for nodesize
        -L|--label LABEL        set a label
        -m|--metadata PROFILE   metadata profile, values like data profile
        -M|--mixed              mix metadata and data together
        -n|--nodesize SIZE      size of btree nodes
        -s|--sectorsize SIZE    min block allocation (may not mountable by current kernel)
        -r|--rootdir DIR        the source directory
        -K|--nodiscard          do not perform whole device TRIM
        -O|--features LIST      comma separated list of filesystem features, use '-O list-all' to list features
        -U|--uuid UUID          specify the filesystem UUID
        -q|--quiet              no messages except errors
        -V|--version            print the mkfs.btrfs version and exit
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/.snapshotz/2016-10-17T130249+0200'
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/.snapshotz/2016-10-17T130249+0200'
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e/.snapshotz/2016-10-17T130249+0200'
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home/.snapshotz/2016-10-17T130249+0200'
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='[]'{}|:<>,./?' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='[]'{}|:<>,./?/.snapshotz/2016-10-17T130249+0200'
Create a readonly snapshot of '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing' in '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing/.snapshotz/2016-10-17T130249+0200'

2 subvolumes excluded in /tmp/snazzer-tests/btrfs-snapshots.working.mnt by /home/florian/Programming/systemautomation/snazzer/tests/data/exclude.patterns.
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/.snapshot_measurements.exclude' >/dev/null
srv
home
var/cache
var/lib/docker/btrfs
echo \`ls "/"; ls /;\`; ~!@#\$(ls)%^&*()_+-='[]'{}|:<>,./?
tmp_thing
EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/.snapshot_measurements.exclude'
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/.snapshot_measurements.exclude' >/dev/null
srv/s p a c e
EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/.snapshot_measurements.exclude'
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e/.snapshot_measurements.exclude' >/dev/null

EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/srv/s p a c e/.snapshot_measurements.exclude'
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home/.snapshot_measurements.exclude' >/dev/null

EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/home/.snapshot_measurements.exclude'
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='\''[]'\''{}|:<>,./?/.snapshot_measurements.exclude' >/dev/null

EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='\''[]'\''{}|:<>,./?' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='\''[]'\''{}|:<>,./?/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/echo `ls "/"; ls /;`; ~!@#$(ls)%^&*()_+-='\''[]'\''{}|:<>,./?/.snapshot_measurements.exclude'
    cat <<EXCL | sudo tee '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing/.snapshot_measurements.exclude' >/dev/null

EXCL
    sudo btrfs subvolume snapshot -r '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing' '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing/.snapshotz/2016-10-17T130252+0200'
    sudo rm '/tmp/snazzer-tests/btrfs-snapshots.working.mnt/tmp_thing/.snapshot_measurements.exclude'

#2 subvolumes excluded in /tmp/snazzer-tests/btrfs-snapshots.working.mnt by /home/florian/Programming/systemautomation/snazzer/tests/data/exclude.patterns.

No idea what's happening there, though, and what they could have done in comprasion to sstephenson/bats to cause this. 😞

csirac2 · 2016-10-17T12:46:54Z

Hrmm. Could you attach your /tmp/log here?

csirac2 · 2016-10-17T12:51:03Z

Does your mkfs.btrfs not support mkfs.btrfs --help ? Perhaps mkfs.btrfs --version would be a better check for command presence. Or which mkfs.btrfs...

florianjacob · 2016-10-17T15:24:00Z

Forgot to mention, /tmp/log does not get created.

Just analysed, my version of mkfs.btrfs --help / btrfs --help return with exit code 0 and print their usage on stdout, the usage information is like it's printed above in the output. Do yours behave differently? This is btrfs-progs 4.7.3, btw.

Regarding the existence test, I'm not sure what's the most clean / compatible / robust way, other alternatives that came to my mind would be bash's integrated hash mkfs.btrfs or also integrated but POSIX-compliant command -v mkfs.btrfs >/dev/null…

(„There should be one-- and preferably only one --obvious way to do it.“ 😆 )

For testing, I replaced mkfs.btrfs --help with hash mkfs.btrfs. This only cleans up the output as the usage information vanishes, but does not change the result of no /tmp/log and endless waiting after the “#2 subvolumes excluded in …” line.

florianjacob · 2016-10-27T14:06:01Z

@Semoar can you reproduce this?

Semoar · 2016-10-27T14:59:14Z

Yup, I'm also running Arch Linux and get the same error (although on btrfs-progs 4.8.1)

Additionally I have a failing test for snazzer-send-wrapper, don't know if this is related.

✗ snazzer-send-wrapper
   (in test file tests/snazzer-send-wrapper.bats, line 18)
     `[ "$status" -eq "1" ]' failed

florianjacob · 2016-11-05T23:43:44Z

With the work from #43, I managed to get some output!

 ✗ sudo -n snazzer --list-snapshots '--all' 'foo=" some stuff "' 'hel'\\'' squot '\\''lo' 'asd " dquot " fgh' 'ap ple' ' bon'\\''squot'\\''jour' 'there'
   (in test file tests/snazzer-send-wrapper.bats, line 65)                                                                                  
     `diff -u $(expected_file) $(actual_file)' failed                                                                                       
   --- /tmp/snazzer-tests/snazzer-send-wrapper.bats_7.expected  2016-11-06 00:40:51.458084026 +0100                                         
   +++ /tmp/snazzer-tests/snazzer-send-wrapper.bats_7.actual    2016-11-06 00:40:51.478084354 +0100                                         
   @@ -1 +1 @@                                                                                                                              
   -10                                                                                                                                      
   +/tmp/snazzer-tests/bin/snazzer-send-wrapper (for ) REJECTED: sudo -n snazzer --list-snapshots '--all' 'foo=" some stuff "' 'hel'\\'' squot '\\''lo' 'asd " dquot " fgh' 'ap ple' ' bon'\\''squot'\\''jour' 'there' ### REASON: This should never happen, stopped: '--all' 'foo=" some stuff "' 'hel'\\'' squot '\\''lo' 'asd " dquot " fgh' 'ap ple' ' bon'\\''squot

csirac2 · 2016-11-06T10:26:22Z

Okay, according to #43 (comment) @florianjacob's system uses bash for /bin/sh. So it would appear that I've accidentally introduced a dash-ism (the default /bin/sh on debian), or I've done something else that isn't properly emulated by bash in /bin/sh personality.

florianjacob · 2016-11-06T10:29:32Z

@Semoar should have bash as /bin/sh, too, as he's on Arch Linux as well.

florianjacob · 2016-11-06T10:33:38Z

A dashism seems quite legit, this would explain why it's running fine on travis as well. 😆

csirac2 · 2016-11-06T10:54:25Z

Ugh, I'm seeing bash interpolating escape \ slightly differently to dash :( What on earth was I thinking when I wrote snazzer-send-wrapper...

EDIT: SSH_ORIGINAL_COMMAND is getting extra slashes... or rather, dash was eating them... from the bats tests?! This is making me insane :)

csirac2 · 2016-11-06T12:28:43Z

This is now a stack overflow question :) https://unix.stackexchange.com/questions/321422/why-does-dash-expand-differently-to-bash

csirac2 · 2016-11-06T13:16:09Z

Alright, the fix will be to replace much of my current usage of echo with printf, which is more portable in its expansion of escape characters... I'm afraid I've burnt all my snazzer time this weekend on this stupid issue, I will progress the two merge requests next weekend, thanks for your patience.

Created koalaman/shellcheck#771 for shellcheck.net once we have a make test target for that

florianjacob · 2016-11-06T13:39:13Z

What on earth was I thinking when I wrote snazzer-send-wrapper...

Probably:

Why do all those mature backup solutions do not leverage btrfs features, particularly copy-on-write snapshots or send/receive transport?!?!??

so exactly what we all were thinking. 😉

Jokes aside, wasting time on issues that turn out to be simple happens to all of us. 😆
Until next weekend! Thanks for all your time!

florianjacob assigned csirac2 Oct 17, 2016

florianjacob changed the title ~~failed test~~ failed test for snazzer-send-wrapper Oct 17, 2016

florianjacob mentioned this issue Nov 5, 2016

tests: make failure message output from bats useful #43

Merged

csirac2 mentioned this issue Nov 6, 2016

Failing tests when /bin/sh is provided by bash #44

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

failed test for snazzer-send-wrapper #33

failed test for snazzer-send-wrapper #33

florianjacob commented Oct 17, 2016 •

edited

Loading

csirac2 commented Oct 17, 2016

florianjacob commented Oct 17, 2016

csirac2 commented Oct 17, 2016

csirac2 commented Oct 17, 2016

florianjacob commented Oct 17, 2016

florianjacob commented Oct 27, 2016 •

edited

Loading

Semoar commented Oct 27, 2016 •

edited

Loading

florianjacob commented Nov 5, 2016

csirac2 commented Nov 6, 2016

florianjacob commented Nov 6, 2016

florianjacob commented Nov 6, 2016

csirac2 commented Nov 6, 2016 •

edited

Loading

csirac2 commented Nov 6, 2016

csirac2 commented Nov 6, 2016 •

edited

Loading

florianjacob commented Nov 6, 2016

failed test for snazzer-send-wrapper #33

failed test for snazzer-send-wrapper #33

Comments

florianjacob commented Oct 17, 2016 • edited Loading

csirac2 commented Oct 17, 2016

florianjacob commented Oct 17, 2016

csirac2 commented Oct 17, 2016

csirac2 commented Oct 17, 2016

florianjacob commented Oct 17, 2016

florianjacob commented Oct 27, 2016 • edited Loading

Semoar commented Oct 27, 2016 • edited Loading

florianjacob commented Nov 5, 2016

csirac2 commented Nov 6, 2016

florianjacob commented Nov 6, 2016

florianjacob commented Nov 6, 2016

csirac2 commented Nov 6, 2016 • edited Loading

csirac2 commented Nov 6, 2016

csirac2 commented Nov 6, 2016 • edited Loading

florianjacob commented Nov 6, 2016

florianjacob commented Oct 17, 2016 •

edited

Loading

florianjacob commented Oct 27, 2016 •

edited

Loading

Semoar commented Oct 27, 2016 •

edited

Loading

csirac2 commented Nov 6, 2016 •

edited

Loading

csirac2 commented Nov 6, 2016 •

edited

Loading