Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

My file is not found after a while, after a restart it works again. #7905

Closed
JanisPlayer opened this issue Feb 7, 2021 · 6 comments
Closed

My file is not found after a while, after a restart it works again. #7905

JanisPlayer opened this issue Feb 7, 2021 · 6 comments
Assignees
@Stebalien
Labels
kind/bug A bug in existing code (including security flaws) need/analysis Needs further analysis before proceeding

Comments

@JanisPlayer
Copy link

JanisPlayer commented Feb 7, 2021 •
edited by lidel
Loading

My file is not found after a while, after a restart it works again for a while and then the problem is the same again.
The file is pinned.
Here is my config:
config.txt

{
  "Identity": {
    "PeerID": "12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
    "PrivKey": ""
  },
  "Datastore": {
    "StorageMax": "50GB",
    "StorageGCWatermark": 90,
    "GCPeriod": "1h",
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "HashOnRead": false,
    "BloomFilterSize": 0
  },
  "Addresses": {
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic",
      "/ip6/::/udp/4001/quic"
    ],
    "Announce": [],
    "NoAnnounce": [
      "/ip4/10.0.0.0/ipcidr/8",
      "/ip4/100.64.0.0/ipcidr/10",
      "/ip4/169.254.0.0/ipcidr/16",
      "/ip4/172.16.0.0/ipcidr/12",
      "/ip4/192.0.0.0/ipcidr/24",
      "/ip4/192.0.0.0/ipcidr/29",
      "/ip4/192.0.0.8/ipcidr/32",
      "/ip4/192.0.0.170/ipcidr/32",
      "/ip4/192.0.0.171/ipcidr/32",
      "/ip4/192.0.2.0/ipcidr/24",
      "/ip4/192.168.0.0/ipcidr/16",
      "/ip4/198.18.0.0/ipcidr/15",
      "/ip4/198.51.100.0/ipcidr/24",
      "/ip4/203.0.113.0/ipcidr/24",
      "/ip4/240.0.0.0/ipcidr/4",
      "/ip6/100::/ipcidr/64",
      "/ip6/2001:2::/ipcidr/48",
      "/ip6/2001:db8::/ipcidr/32",
      "/ip6/fc00::/ipcidr/7",
      "/ip6/fe80::/ipcidr/10"
    ],
    "API": "/ip4/127.0.0.1/tcp/5001",
    "Gateway": "/ip4/127.0.0.1/tcp/8080"
  },
  "Mounts": {
    "IPFS": "/ipfs",
    "IPNS": "/ipns",
    "FuseAllowOther": false
  },
  "Discovery": {
    "MDNS": {
      "Enabled": false,
      "Interval": 10
    }
  },
  "Routing": {
    "Type": "dhtclient"
  },
  "Ipns": {
    "RepublishPeriod": "",
    "RecordLifetime": "",
    "ResolveCacheSize": 128
  },
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
  ],
  "Gateway": {
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range",
        "User-Agent"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "RootRedirect": "",
    "Writable": false,
    "PathPrefixes": [],
    "APICommands": [],
    "NoFetch": false,
    "NoDNSLink": false,
    "PublicGateways": null
  },
  "API": {
    "HTTPHeaders": {}
  },
  "Swarm": {
    "AddrFilters": [
      "/ip4/10.0.0.0/ipcidr/8",
      "/ip4/100.64.0.0/ipcidr/10",
      "/ip4/169.254.0.0/ipcidr/16",
      "/ip4/172.16.0.0/ipcidr/12",
      "/ip4/192.0.0.0/ipcidr/24",
      "/ip4/192.0.0.0/ipcidr/29",
      "/ip4/192.0.0.8/ipcidr/32",
      "/ip4/192.0.0.170/ipcidr/32",
      "/ip4/192.0.0.171/ipcidr/32",
      "/ip4/192.0.2.0/ipcidr/24",
      "/ip4/192.168.0.0/ipcidr/16",
      "/ip4/198.18.0.0/ipcidr/15",
      "/ip4/198.51.100.0/ipcidr/24",
      "/ip4/203.0.113.0/ipcidr/24",
      "/ip4/240.0.0.0/ipcidr/4",
      "/ip6/100::/ipcidr/64",
      "/ip6/2001:2::/ipcidr/48",
      "/ip6/2001:db8::/ipcidr/32",
      "/ip6/fc00::/ipcidr/7",
      "/ip6/fe80::/ipcidr/10"
    ],
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": true,
    "EnableRelayHop": false,
    "EnableAutoRelay": false,
    "Transports": {
      "Network": {},
      "Security": {},
      "Multiplexers": {}
    },
    "ConnMgr": {
      "Type": "basic",
      "LowWater": 50,
      "HighWater": 100,
      "GracePeriod": "20s"
    }
  },
  "AutoNAT": {},
  "Pubsub": {
    "Router": "",
    "DisableSigning": false
  },
  "Peering": {
    "Peers": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "all"
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "UrlstoreEnabled": false,
    "ShardingEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "StrategicProviding": false
  },
  "Plugins": {
    "Plugins": null
  }
}

It just doesn't work that the provider can be found permanently without restarting, which makes the whole thing quite unstable.

Files:
https://ipfs.io/ipfs/QmUKAPG1MNwoJ79kXbZrLRSNtPZkivS9hFg86tgREgoW53?filename=svpihole2010.zip
https://heldendesbildschirms.dynv6.net/download/software/betriebssysteme/svpihole2010/

Nobody in the forum knew what it was:
https://discuss.ipfs.io/t/data-can-only-be-found-in-ipfs-after-the-server-has-been-restarted/9417

Thanks in advance for the great help, the problem is really annoying and I have not yet been able to find the cause.
@JanisPlayer JanisPlayer added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Feb 7, 2021
@welcome
Copy link

welcome bot commented Feb 7, 2021

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@Stebalien Stebalien added need/analysis Needs further analysis before proceeding and removed need/triage Needs initial labeling and prioritization labels Mar 22, 2021
@Stebalien Stebalien self-assigned this Mar 22, 2021
@Stebalien
Copy link
Member

  • What's "after a while"? A day, a few hours?
  • Could you post the output of ipfs id when this happens?
  • Could you try reducing Reprovider.Interval to "6h"? Or maybe even "4h"? It may be that the DHT is simply forgetting your data.

@JanisPlayer
Copy link
Author

JanisPlayer commented Mar 30, 2021 •
edited
Loading

Thanks for the help, I set it to 4h. Since then the file has been found after a few minutes, slowly but at least downloading.

ipfs@vmd47190:~$ ./ipfs id {
"ID": "12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"PublicKey": "CAESILCrHbn83JWHov/EgidYGbM6Nq3GFnP605YcN2QGzOhJ",
"Addresses": [
"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip4/127.0.0.1/udp/4001/quic/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip4/164.68.125.80/tcp/4001/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip4/164.68.125.80/udp/4001/quic/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip6/2a02:c207:3004:7190::1/tcp/4001/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip6/2a02:c207:3004:7190::1/udp/4001/quic/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip6/::1/tcp/4001/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96",
"/ip6/::1/udp/4001/quic/p2p/12D3KooWMi1G1UkaFCVtFeDADHjphU8jxXr9cquGNfUHytLrAY96"
],
"AgentVersion": "go-ipfs/0.7.0/",
"ProtocolVersion": "ipfs/0.1.0",
"Protocols": [
"/ipfs/bitswap",
"/ipfs/bitswap/1.0.0",
"/ipfs/bitswap/1.1.0",
"/ipfs/bitswap/1.2.0",
"/ipfs/id/1.0.0",
"/ipfs/id/push/1.0.0",
"/ipfs/ping/1.0.0",
"/libp2p/autonat/1.0.0",
"/libp2p/circuit/relay/0.1.0",
"/p2p/id/delta/1.0.0",
"/x/"
]
}

@Stebalien
Copy link
Member

Ok, so it looks like the main issue was #6383. In terms of downloading, does downloading slow down over time or is it just always slow. Also, what is "slow"? How big is the file and how long does it take to download.

@JanisPlayer
Copy link
Author

The server can have up to 200 Mbits, but only 0.1-10 Mbits when distributing in the network.
After the file is distributed, it works much faster, of course.
Can you also ensure that my peer provides the file directly or is that not possible due to the network, which always takes the closest provider.

https://heldendesbildschirms.dynv6.net/download/software/betriebssysteme/svpihole2010/
File size is 1.99 GiB
It still takes up to 5 minutes until the file is found, but at least they are found.

@JanisPlayer
Copy link
Author

For all those who have the problem, simply delete ipfs config, make a backup of the keys beforehand and reinitialize it as a server. Then it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Stebalien Stebalien

Labels
kind/bug A bug in existing code (including security flaws) need/analysis Needs further analysis before proceeding
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Stebalien @JanisPlayer