Cannot customize the array separator

[xiaomaiyun]

How to define the array separator as square brackets'[]'?E.g:1,[1,2,3],['name01','name02','name03'],'pen',1
The output of the following code is:1,'1,2,3','name01,name02,name03',pen,1
The expected output is:1,[1,2,3],['name01','name02','name03'],'pen',1

public class CsvTest {
    public static void main(String[] args) throws JsonProcessingException {

        OrderDto orderDto = new OrderDto(1L, "pen", 1, Arrays.asList("name01", "name02", "name03"), Arrays.asList(1, 2, 3));
        CsvMapper mapper = new CsvMapper();
        // schema from 'Pojo' definition
        CsvSchema schema = mapper.schemaFor(OrderDto.class).withoutHeader().withAllowComments(true);
        String csv = mapper.writer(schema).writeValueAsString(orderDto);

        // 结果：1,'1,2,3','name01,name02,name03',pen,1
        // 期待的结果：1,[1,2,3],['name01','name02','name03'],'pen',1
        System.out.println(csv);
    }

    public static class OrderDto {

        private Long user;
        private String product;
        private Integer amount;

        private List<String> name_list;
        private List<Integer> id_list;

        public OrderDto() {
        }

        public OrderDto(Long user, String product, Integer amount, List<String> name_list, List<Integer> id_list) {
            this.user = user;
            this.product = product;
            this.amount = amount;
            this.name_list = name_list;
            this.id_list = id_list;
        }

        public Long getUser() {
            return user;
        }

        public void setUser(Long user) {
            this.user = user;
        }

        public String getProduct() {
            return product;
        }

        public void setProduct(String product) {
            this.product = product;
        }

        public Integer getAmount() {
            return amount;
        }

        public void setAmount(Integer amount) {
            this.amount = amount;
        }

        public List<String> getName_list() {
            return name_list;
        }

        public void setName_list(List<String> name_list) {
            this.name_list = name_list;
        }

        public List<Integer> getId_list() {
            return id_list;
        }

        public void setId_list(List<Integer> id_list) {
            this.id_list = id_list;
        }

        @Override
        public String toString() {
            return "OrderDto{" +
                    "user=" + user +
                    ", product='" + product + '\'' +
                    ", amount=" + amount +
                    ", name_list=" + name_list +
                    ", id_list=" + id_list +
                    '}';
        }
    }
}

[cowtowncoder]

There is no functionality currently for "embedded" arrays like this: only array item separator (defaults to ;) can be specified. There is also no way to add quoting within array.

I will leave this open in case someone would like to add support for optional start/end markers (brackets here); support would be needed for both serialization and deserialization.

[xiaomaiyun]

OK, thanks!

[mrpiggi]

I recently stumbled across this issue as I have to deal with arrays of nested objects within a CSV string. I am not quite sure if I can find some time to work on this issue. But if so, than adapting CsvParser#_handleArrayValue() would be needed, am I right?

The question arises, if only something like foo,[1,2,3],bar should be supported as a slightly different format other than foo,1;2;3,bar or if it would be desirable to get support for nested objects as well. If the latter is the case, what would be a meaningful syntax? Something like foo,{anobject,1},bar and foo,[{anobject,1},{someobject,2}],bar?

[cowtowncoder]

I would limit functionality to arrays of scalar, and not having to invent new format/notation for full set of types. In theory, supporting full set could be done by allowing nesting of JSON within cells, but that'd require whole new level of functionality (although could be useful for embedding within other formats too).

And without looking deeper into code, I think you are right that _handleArrayValue() would be the place.

[mrpiggi]

For my specific application, nesting types are exactly what is required. I need to transmit data via an ancient radio data link. We are talking about bytes, not kilobytes, which is why the payload has to be kept as minimal as possible. Essentially, the data looks like this:

123,foo,[abc,456,0;xyz,789,1],stuff,42

where ; acts as a separator for array elements and translates to something like this:

record Payload(
	int id,
	String label,
	List<Nested> nested,
	String description,
	int number
) { }

record Nested(
	String text,
	int code,
	boolean state
) { }

Since this is currently not feasible with Jackson, I have built a module in which I register customized AsArraySerializerBase and CollectionDeserializer via BeanSerializerModifier.modifyCollectionSerializer() and BeanDeserializerModifier.modifyCollectionDeserializer() together with an upstream parser for escaping column separators within arrays during deserialization.

So for me, this is not urgent (anymore). Unfortunately, I won't be able to work on it in the near future and therefore won't be able to provide a PR in the short term. Should I unexpectedly find some time for this in the future, I will report here.

[cowtowncoder]

@mrpiggi Ok, I understand. But maybe your work can help future developments.

And just to make it clear: I think that additional module(s), to support "embedded" structured data, would make lots of sense. It's hard to come with "one and only" notation to make it default for CSV, but ability to plug in alternate extensions would be awesome. Doing that may need improved configurability/pluggability, of course.

[mrpiggi]

I would like to contribute to an expansion, otherwise I wouldn't have given my input here. At the moment, however, I unfortunately have absolutely no time to do anything on the side. The list of things I want to do after work is getting longer rather than shorter. And at the moment my working day stretches deep into the night...

[cowtowncoder]

@mrpiggi Understood. Sorry, did not mean to sound like I was expecting you to do more here; I know how precious time is. Take care!

[mrpiggi]

That's not how I understood it either ;) Many thanks.