Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingesting files with delimiting character in a quoted column #8

Open
kristinriebe opened this issue Dec 19, 2016 · 0 comments
Open

Comments

@kristinriebe
Copy link
Member

kristinriebe commented Dec 19, 2016

E.g. columns like "[0,1,2]" for csv-file or "This is an example" in a space-separated file.

AsciiIngest does not respect quotes in columns, i.e. if there is a column "[0,1,2]" in a csv-file, then it will be read as 3 columns, not 1, if the delimiter is specified as comma ',' in the structure file.
Luckily it is possible to define a different delimiter for each column, so the correct delimiter could be '",' (double quotes+comma) for finding the end of the quoted columns, but AsciiIngest seems to enter an infinite loop if there is more than one delimiting character.

So currently, the work-around is:
Use delimiter ',' everywhere, then just before the quoted column: use double quote '"' as delimiter and skip the column with SKIP_THIS_COL, the part until the next double quote is the stuff that can be mapped to a field in the database, then one needs to skip again until the next comma and then one can continue as usual.
E.g. for the following data row:

1,54.6,"[0,1,2]",2.8,5

the delimiter should be:

','
','
'"' SKIP_THIS_COL
'"'
',' SKIP_THIS_COL
','
'\n'

An example is also given in this post.
Of course, it would be possible to edit the data file and use e.g. semi-colons to be on the save side. Anyway, this is a feature request, not a bug report, to:

  • allow usage of multiple different characters as delimiter
  • introduce a quote-character in addition to the delimiter character for each column
@kristinriebe kristinriebe changed the title Ingesting files with delimiting character in a quoted column, e.g. "[0,1,2]" for csv-file or "This is an example" in a space-separated file. Ingesting files with delimiting character in a quoted column, e.g. "[0,1,2]" for csv-file or "This is an example" in a space-separated file Dec 19, 2016
@kristinriebe kristinriebe changed the title Ingesting files with delimiting character in a quoted column, e.g. "[0,1,2]" for csv-file or "This is an example" in a space-separated file Ingesting files with delimiting character in a quoted column Dec 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant