Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ancestor property contains incorrect data #352

Open
ClimbFlyCamp opened this issue Aug 25, 2023 · 9 comments
Open

Ancestor property contains incorrect data #352

ClimbFlyCamp opened this issue Aug 25, 2023 · 9 comments
Labels
bug Something isn't working

Comments

@ClimbFlyCamp
Copy link

Looking at the ancestor property on an area I see the following:

"ancestors":"1db1e8ba-a40e-587c-88a4-64f5ea814b8e,60bc2823-f54f-59fa-9443-f61a87765009,8293b197-46ad-5f05-8f6d-b72387f2f191"

These ids do not correlate to any ids inside of the data. Think these should be converted to a similar model that the children field follows:

"children":["63689d10e80bff5a994f15ad"]

The children field correctly points to _id of the sub area, while the ancestors field is incorrect. There is no way to currently match an ancestor to a sub area

@musoke
Copy link
Contributor

musoke commented Aug 25, 2023

Could you clarify this issue? As far as I can tell, each of the three returned ids corresponds to an area that exists in the production database:

It is a bit odd that the three are in a single comma separated string.

@ClimbFlyCamp
Copy link
Author

Could you clarify this issue? As far as I can tell, each of the three returned areas exists in the production database:

* https://openbeta.io/crag/1db1e8ba-a40e-587c-88a4-64f5ea814b8e

* https://openbeta.io/crag/60bc2823-f54f-59fa-9443-f61a87765009

* https://openbeta.io/crag/8293b197-46ad-5f05-8f6d-b72387f2f191

If you look at the exported data, here is an example:

https://github.com/OpenBeta/openbeta-export/tree/production/usa/alabama/bucks_pocket

Searching on any of those ancestor ids brings up the following:

https://github.com/search?q=repo%3AOpenBeta%2Fopenbeta-export%201db1e8ba-a40e-587c-88a4-64f5ea814b8e&type=code

You can see there are not actual hits, besides in the ancestor field. These do not point to the _id field of an area like the children do. With this data there is no way for me to link an ancestor to a sub area

@ClimbFlyCamp
Copy link
Author

There was a discussion here:

https://discord.com/channels/815145484003967026/815490505143484427/1144641711209971756

@musoke
Copy link
Contributor

musoke commented Aug 25, 2023

Ah, I see! It's just a bug in exported data.

I suspect that the ancestor property already has the ID one would actually want and it's the other IDs that have a bug.

The export script is here: https://github.com/OpenBeta/openbeta-graphql/blob/develop/src/db/export/json/index.ts and can be invoked with something like yarn export-prod.

@musoke musoke added the bug Something isn't working label Aug 25, 2023
@ClimbFlyCamp
Copy link
Author

yeah that could be the case, but the _id and child fields in the export data match, whereas the ancestor does not. So either ancestor is wrong or both _id and children are wrong

@ClimbFlyCamp
Copy link
Author

Is there a workaround for converting the id seen in ancestor (1db1e8ba-a40e-587c-88a4-64f5ea814b8e) to the model in _id (63689d10e80bff5a994f15ad)?

@ClimbFlyCamp
Copy link
Author

@musoke do you happen to know of a workaround to convert UUID to ObjectId? Not super familiar with uuid-mongodb, but wondering if I can convert acestors to an ObejctId when I import the data into my db for the time being

@vnugent
Copy link
Contributor

vnugent commented Sep 5, 2023

As mentioned in the Discord chat, they are two different fields in the DB, therefore we can't simply convert _id to areaId which is used in the ancestors field and openbeta.io URL slug.

query GetUSA {
  area(uuid: "1db1e8ba-a40e-587c-88a4-64f5ea814b8e") {
    areaName
    children {
      id
      areaName
      metadata {
        areaId
      }
    }
  }
}

result (truncated to save space)

{
  "data": {
    "area": {
      "areaName": "USA",
      "children": [
        {
          "id": "63689d10e80bff5a994f159a", // We only store this id field in the db.  The rest is populated during the query.
          "areaName": "Alabama",
          "metadata": {
            "areaId": "60bc2823-f54f-59fa-9443-f61a87765009"
          }
        },
        {
          "id": "63689d10e80bff5a994f1667",
          "areaName": "Arkansas",
          "metadata": {
            "areaId": "b33d5a8f-4494-55c0-a863-049dce4065d9"
          }
        }  
   ]}}
}

When inserting new areas, the parent's children[] only contains foreign keys of child area IDs (autogenerated by Mongo). During a query, we populate children[] with the actual area object.

  • The database export script simply converts the area document to JSON.
  • metadata.areaId was added to the DB sometime later. It's used in the ancestor field and page slug on openbeta.io

The proper fix is described in #109.

@ClimbFlyCamp can you insert area data as is and do a join in your db queries?

@musoke It is a bit odd that the three are in a single comma separated string.

It allows us to use regular expressions to test relationships such as is this area under Colorado? or does this area share the ancestor as crag Y? (Model tree structures with materialized paths)

@ClimbFlyCamp
Copy link
Author

@vnugent: Thanks for the helpful info, but still had one question. When looking at the area_id in the metadata field in the generated JSON files, I am not seeing the data stored in the format described above. Here is an example:

https://github.com/OpenBeta/openbeta-export/blob/production/usa/alabama/bankhead_forest_thompson_creek_trail/journey_area_the/journey_area_the.json

"area_id":"l9gdD2NyXWSmLfkwhEsZNg=="

I was going to join on the area_id found in the metadata object, but this also does not seem to match the expected values. I thought this must of been an encoded string, but couldn't find the proper decoding mechanism.

Any insight here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants