Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directory Error #11

Open
Spruces opened this issue Oct 9, 2020 · 7 comments
Open

Directory Error #11

Spruces opened this issue Oct 9, 2020 · 7 comments

Comments

@Spruces
Copy link

Spruces commented Oct 9, 2020

Hello, I tried to extract the features with extract_features.py

image
and the result is just like this.

I don't get where the open_data_type_78_header_valid.csv's location is.

  • Can I just use this model as easier as Sato Web Demo,
    just upload and get return value?

Too complicate to use, and not much instructions

@jason022085
Copy link

Same error occurs to me.
I think the solution is to understand the usage of Viznet, because there is similar code in it.
https://github.com/mitmedialab/viznet

@Spruces
Copy link
Author

Spruces commented Mar 17, 2021

Same error occurs to me.
I think the solution is to understand the usage of Viznet, because there is similar code in it.
https://github.com/mitmedialab/viznet

Thanks bro, it's weird.
To execute it, you need certain .csv files(like open_data_type_78_header_valid.csv) that doesn't provided by sato.

and I had examined the codes, you need certain type of NLTKs to generate open_data_type_78_header_valid.csv.

Also, after I found certain types of NLTKs, I couldn't figure out where exactly the NLTK should be located in certain directory.

Poor guidelines.

@Spruces
Copy link
Author

Spruces commented Mar 17, 2021

And how they can detect semantic columns with NLTKs?

Like my ID, and yours.
They don't have any semantic things as they are just nouns.

I think that for detecting 'ID' columns based on our id(Spruces,jason022085) they need something else like the lengths of inputs or how many blanks they had, and the number of separator comma(address).

cus there's no meaning in our names of addresses as themselves.

@jason022085
Copy link

jason022085 commented Mar 19, 2021

p.s. Remember to download VizNet data first(produced by retrieve_corpora.sh) and set RAW_DIR to the path of raw data form Viznet
In my case, it is os.environ['RAW_DIR'] = r"D:\viznet-master\raw"

Since other dataset are too large, I am working on "manyeyes" dataset.
image

And I found there is a extract_header.py. It may help
image

I got it ! You have to extract header first, and then extract features
image

@RichaMax
Copy link

RichaMax commented Mar 24, 2021

Hello, from what I understand you can download open_data_type_78_header_valid.csv and the sherlocks features they used using their script ./download_data.sh

@horseno
Copy link
Collaborator

horseno commented Mar 24, 2021

Sorry about the confusion. If you wish to extract the features, you'll first need to run extract_header.py to get the valid headers (in the 78 types after canonicalization) from the dataset.

@iganand
Copy link

iganand commented Sep 9, 2021

I have empty header files generated when I try extract_headers.py for manyeyes. Can you please help? What it might be. Please let me know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants