-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCSV_XS.pm
1709 lines (1255 loc) · 48.8 KB
/
CSV_XS.pm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
package Text::CSV_XS;
# Copyright (c) 2007-2009 H.Merijn Brand. All rights reserved.
# Copyright (c) 1998-2001 Jochen Wiedmann. All rights reserved.
# Portions Copyright (c) 1997 Alan Citterman. All rights reserved.
#
# This program is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
################################################################################
# HISTORY
#
# Written by:
# Jochen Wiedmann <[email protected]>
#
# Based on Text::CSV by:
# Alan Citterman <[email protected]>
#
# Extended and Remodelled by:
# H.Merijn Brand ([email protected])
#
############################################################################
require 5.005;
use strict;
use warnings;
use DynaLoader ();
use Carp;
use vars qw( $VERSION @ISA );
$VERSION = "0.70";
@ISA = qw( DynaLoader );
bootstrap Text::CSV_XS $VERSION;
sub PV { 0 }
sub IV { 1 }
sub NV { 2 }
# version
#
# class/object method expecting no arguments and returning the version
# number of Text::CSV. there are no side-effects.
sub version
{
return $VERSION;
} # version
# new
#
# class/object method expecting no arguments and returning a reference to
# a newly created Text::CSV object.
my %def_attr = (
quote_char => '"',
escape_char => '"',
sep_char => ',',
eol => '',
always_quote => 0,
quote_space => 1,
binary => 0,
keep_meta_info => 0,
allow_loose_quotes => 0,
allow_loose_escapes => 0,
allow_whitespace => 0,
blank_is_undef => 0,
empty_is_undef => 0,
verbatim => 0,
auto_diag => 0,
types => undef,
_EOF => 0,
_STATUS => undef,
_FIELDS => undef,
_FFLAGS => undef,
_STRING => undef,
_ERROR_INPUT => undef,
_COLUMN_NAMES => undef,
_BOUND_COLUMNS => undef,
_AHEAD => undef,
);
my $last_new_err = Text::CSV_XS->SetDiag (0);
sub _check_sanity
{
my $attr = shift;
for (qw( sep_char quote_char escape_char )) {
exists $attr->{$_} && defined $attr->{$_} && $attr->{$_} =~ m/[\r\n]/ and
return 1003;
}
$attr->{allow_whitespace} and
(defined $attr->{quote_char} && $attr->{quote_char} =~ m/^[ \t]$/) ||
(defined $attr->{escape_char} && $attr->{escape_char} =~ m/^[ \t]$/) and
return 1002;
return 0;
} # _check_sanity
sub new
{
$last_new_err = SetDiag (undef, 1000,
"usage: my \$csv = Text::CSV_XS->new ([{ option => value, ... }]);");
my $proto = shift;
my $class = ref ($proto) || $proto or return;
@_ > 0 && ref $_[0] ne "HASH" and return;
my $attr = shift || {};
for (keys %{$attr}) {
if (m/^[a-z]/ && exists $def_attr{$_}) {
$] >= 5.008002 && m/_char$/ and utf8::decode ($attr->{$_});
next;
}
# croak?
$last_new_err = SetDiag (undef, 1000, "INI - Unknown attribute '$_'");
return;
}
my $self = {%def_attr, %{$attr}};
if (my $ec = _check_sanity ($self)) {
$last_new_err = SetDiag (undef, $ec);
return;
}
$last_new_err = SetDiag (undef, 0);
defined $\ && !exists $attr->{eol} and $self->{eol} = $\;
bless $self, $class;
defined $self->{types} and $self->types ($self->{types});
$self;
} # new
# Keep in sync with XS!
my %_cache_id = ( # Only expose what is accessed from within PM
quote_char => 0,
escape_char => 1,
sep_char => 2,
binary => 3,
keep_meta_info => 4,
always_quote => 5,
allow_loose_quotes => 6,
allow_loose_escapes => 7,
allow_double_quoted => 8,
allow_whitespace => 9,
blank_is_undef => 10,
eol => 11, # 11 .. 18
verbatim => 22,
empty_is_undef => 23,
auto_diag => 24,
quote_space => 25,
_is_bound => 26, # 26 .. 29
);
# A `character'
sub _set_attr_C
{
my ($self, $name, $val, $ec) = @_;
defined $val or $val = 0;
$] >= 5.008002 and utf8::decode ($val);
$self->{$name} = $val;
$ec = _check_sanity ($self) and
croak ($self->SetDiag ($ec));
$self->_cache_set ($_cache_id{$name}, $val);
} # _set_attr_C
# A flag
sub _set_attr_X
{
my ($self, $name, $val) = @_;
defined $val or $val = 0;
$self->{$name} = $val;
$self->_cache_set ($_cache_id{$name}, 0 + $val);
} # _set_attr_X
# A number
sub _set_attr_N
{
my ($self, $name, $val) = @_;
$self->{$name} = $val;
$self->_cache_set ($_cache_id{$name}, 0 + $val);
} # _set_attr_N
# Accessor methods.
# It is unwise to change them halfway through a single file!
sub quote_char
{
my $self = shift;
if (@_) {
my $qc = shift;
$self->_set_attr_C ("quote_char", $qc);
}
$self->{quote_char};
} # quote_char
sub escape_char
{
my $self = shift;
if (@_) {
my $ec = shift;
$self->_set_attr_C ("escape_char", $ec);
}
$self->{escape_char};
} # escape_char
sub sep_char
{
my $self = shift;
@_ and $self->_set_attr_C ("sep_char", shift);
$self->{sep_char};
} # sep_char
sub eol
{
my $self = shift;
if (@_) {
my $eol = shift;
defined $eol or $eol = "";
$self->{eol} = $eol;
$self->_cache_set ($_cache_id{eol}, $eol);
}
$self->{eol};
} # eol
sub always_quote
{
my $self = shift;
@_ and $self->_set_attr_X ("always_quote", shift);
$self->{always_quote};
} # always_quote
sub quote_space
{
my $self = shift;
@_ and $self->_set_attr_X ("quote_space", shift);
$self->{quote_space};
} # quote_space
sub binary
{
my $self = shift;
@_ and $self->_set_attr_X ("binary", shift);
$self->{binary};
} # binary
sub keep_meta_info
{
my $self = shift;
@_ and $self->_set_attr_X ("keep_meta_info", shift);
$self->{keep_meta_info};
} # keep_meta_info
sub allow_loose_quotes
{
my $self = shift;
@_ and $self->_set_attr_X ("allow_loose_quotes", shift);
$self->{allow_loose_quotes};
} # allow_loose_quotes
sub allow_loose_escapes
{
my $self = shift;
@_ and $self->_set_attr_X ("allow_loose_escapes", shift);
$self->{allow_loose_escapes};
} # allow_loose_escapes
sub allow_whitespace
{
my $self = shift;
if (@_) {
my $aw = shift;
$aw and
(defined $self->{quote_char} && $self->{quote_char} =~ m/^[ \t]$/) ||
(defined $self->{escape_char} && $self->{escape_char} =~ m/^[ \t]$/) and
croak ($self->SetDiag (1002));
$self->_set_attr_X ("allow_whitespace", $aw);
}
$self->{allow_whitespace};
} # allow_whitespace
sub blank_is_undef
{
my $self = shift;
@_ and $self->_set_attr_X ("blank_is_undef", shift);
$self->{blank_is_undef};
} # blank_is_undef
sub empty_is_undef
{
my $self = shift;
@_ and $self->_set_attr_X ("empty_is_undef", shift);
$self->{empty_is_undef};
} # empty_is_undef
sub verbatim
{
my $self = shift;
@_ and $self->_set_attr_X ("verbatim", shift);
$self->{verbatim};
} # verbatim
sub auto_diag
{
my $self = shift;
@_ and $self->_set_attr_X ("auto_diag", shift);
$self->{auto_diag};
} # auto_diag
# status
#
# object method returning the success or failure of the most recent
# combine () or parse (). there are no side-effects.
sub status
{
my $self = shift;
return $self->{_STATUS};
} # status
sub eof
{
my $self = shift;
return $self->{_EOF};
} # status
# error_input
#
# object method returning the first invalid argument to the most recent
# combine () or parse (). there are no side-effects.
sub error_input
{
my $self = shift;
return $self->{_ERROR_INPUT};
} # error_input
# erro_diag
#
# If (and only if) an error occured, this function returns a code that
# indicates the reason of failure
sub error_diag
{
my $self = shift;
my @diag = (0 + $last_new_err, $last_new_err, 0);
if ($self && ref $self && # Not a class method or direct call
$self->isa (__PACKAGE__) && exists $self->{_ERROR_DIAG}) {
@diag = (0 + $self->{_ERROR_DIAG}, $self->{_ERROR_DIAG});
exists $self->{_ERROR_POS} and $diag[2] = 1 + $self->{_ERROR_POS};
}
my $context = wantarray;
unless (defined $context) { # Void context, auto-diag
if ($diag[0] && $diag[0] != 2012 && $self && ref $self) {
my $msg = "# CSV_XS ERROR: $diag[0] - $diag[1]\n";
my $lvl = $self->{auto_diag};
if ($lvl < 2) {
my @c = caller (2);
if (@c >= 11 && $c[10] && ref $c[10] eq "HASH") {
my $hints = $c[10];
(exists $hints->{autodie} && $hints->{autodie} or
exists $hints->{"guard Fatal"} &&
!exists $hints->{"no Fatal"}) and
$lvl++;
# Future releases of autodie will probably set $^H{autodie}
# to "autodie @args", like "autodie :all" or "autodie open"
# so we can/should check for "open" or "new"
}
}
$lvl > 1 ? die $msg : warn $msg;
}
return;
}
return $context ? @diag : $diag[1];
} # error_diag
# string
#
# object method returning the result of the most recent combine () or the
# input to the most recent parse (), whichever is more recent. there are
# no side-effects.
sub string
{
my $self = shift;
return ref $self->{_STRING} ? ${$self->{_STRING}} : undef;
} # string
# fields
#
# object method returning the result of the most recent parse () or the
# input to the most recent combine (), whichever is more recent. there
# are no side-effects.
sub fields
{
my $self = shift;
return ref $self->{_FIELDS} ? @{$self->{_FIELDS}} : undef;
} # fields
# meta_info
#
# object method returning the result of the most recent parse () or the
# input to the most recent combine (), whichever is more recent. there
# are no side-effects. meta_info () returns (if available) some of the
# field's properties
sub meta_info
{
my $self = shift;
return ref $self->{_FFLAGS} ? @{$self->{_FFLAGS}} : undef;
} # meta_info
sub is_quoted
{
my ($self, $idx, $val) = @_;
ref $self->{_FFLAGS} &&
$idx >= 0 && $idx < @{$self->{_FFLAGS}} or return;
$self->{_FFLAGS}[$idx] & 0x0001 ? 1 : 0;
} # is_quoted
sub is_binary
{
my ($self, $idx, $val) = @_;
ref $self->{_FFLAGS} &&
$idx >= 0 && $idx < @{$self->{_FFLAGS}} or return;
$self->{_FFLAGS}[$idx] & 0x0002 ? 1 : 0;
} # is_binary
# combine
#
# object method returning success or failure. the given arguments are
# combined into a single comma-separated value. failure can be the
# result of no arguments or an argument containing an invalid character.
# side-effects include:
# setting status ()
# setting fields ()
# setting string ()
# setting error_input ()
sub combine
{
my $self = shift;
my $str = "";
$self->{_FIELDS} = \@_;
$self->{_FFLAGS} = undef;
$self->{_STATUS} = (@_ > 0) && $self->Combine (\$str, \@_, 0);
$self->{_STRING} = \$str;
$self->{_STATUS};
} # combine
# parse
#
# object method returning success or failure. the given argument is
# expected to be a valid comma-separated value. failure can be the
# result of no arguments or an argument containing an invalid sequence
# of characters. side-effects include:
# setting status ()
# setting fields ()
# setting meta_info ()
# setting string ()
# setting error_input ()
sub parse
{
my ($self, $str) = @_;
my $fields = [];
my $fflags = [];
$self->{_STRING} = \$str;
if (defined $str && $self->Parse ($str, $fields, $fflags)) {
$self->{_ERROR_INPUT} = undef;
$self->{_FIELDS} = $fields;
$self->{_FFLAGS} = $fflags;
$self->{_STATUS} = 1;
}
else {
$self->{_FIELDS} = undef;
$self->{_FFLAGS} = undef;
$self->{_STATUS} = 0;
}
$self->{_STATUS};
} # parse
sub column_names
{
my ($self, @keys) = @_;
@keys or
return defined $self->{_COLUMN_NAMES} ? @{$self->{_COLUMN_NAMES}} : undef;
@keys == 1 && ! defined $keys[0] and
return $self->{_COLUMN_NAMES} = undef;
if (@keys == 1 && ref $keys[0] eq "ARRAY") {
@keys = @{$keys[0]};
}
elsif (join "", map { defined $_ ? ref $_ : "" } @keys) {
croak ($self->SetDiag (3001));
}
$self->{_BOUND_COLUMNS} && @keys != @{$self->{_BOUND_COLUMNS}} and
croak ($self->SetDiag (3003));
$self->{_COLUMN_NAMES} = [ map { defined $_ ? $_ : "\cAUNDEF\cA" } @keys ];
@{$self->{_COLUMN_NAMES}};
} # column_names
sub bind_columns
{
my ($self, @refs) = @_;
@refs or
return defined $self->{_BOUND_COLUMNS} ? @{$self->{_BOUND_COLUMNS}} : undef;
@refs == 1 && ! defined $refs[0] and
return $self->{_BOUND_COLUMNS} = undef;
$self->{_COLUMN_NAMES} && @refs != @{$self->{_COLUMN_NAMES}} and
croak ($self->SetDiag (3003));
join "", map { ref $_ eq "SCALAR" ? "" : "*" } @refs and
croak ($self->SetDiag (3004));
$self->_set_attr_N ("_is_bound", scalar @refs);
$self->{_BOUND_COLUMNS} = [ @refs ];
@refs;
} # bind_columns
sub getline_hr
{
my ($self, @args, %hr) = @_;
$self->{_COLUMN_NAMES} or croak ($self->SetDiag (3002));
my $fr = $self->getline (@args) or return;
@hr{@{$self->{_COLUMN_NAMES}}} = @$fr;
\%hr;
} # getline_hr
sub types
{
my $self = shift;
if (@_) {
if (my $types = shift) {
$self->{_types} = join "", map { chr $_ } @{$types};
$self->{types} = $types;
}
else {
delete $self->{types};
delete $self->{_types};
undef;
}
}
else {
$self->{types};
}
} # types
1;
__END__
=head1 NAME
Text::CSV_XS - comma-separated values manipulation routines
=head1 SYNOPSIS
use Text::CSV_XS;
my @rows;
my $csv = Text::CSV_XS->new ({ binary => 1 }) or
die "Cannot use CSV: ".Text::CSV->error_diag ();
open my $fh, "<:encoding(utf8)", "test.csv" or die "test.csv: $!";
while (my $row = $csv->getline ($fh)) {
$row->[2] =~ m/pattern/ or next; # 3rd field should match
push @rows, $row;
}
$csv->eof or $csv->error_diag ();
close $fh;
$csv->eol ("\r\n");
open $fh, ">:encoding(utf8)", "new.csv" or die "new.csv: $!";
$csv->print ($fh, $_) for @rows;
close $fh or die "new.csv: $!";
=head1 DESCRIPTION
Text::CSV_XS provides facilities for the composition and decomposition of
comma-separated values. An instance of the Text::CSV_XS class can combine
fields into a CSV string and parse a CSV string into fields.
The module accepts either strings or files as input and can utilize any
user-specified characters as delimiters, separators, and escapes so it is
perhaps better called ASV (anything separated values) rather than just CSV.
=head2 Embedded newlines
B<Important Note>: The default behavior is to only accept ascii characters.
This means that fields can not contain newlines. If your data contains
newlines embedded in fields, or characters above 0x7e (tilde), or binary data,
you *must* set C<< binary => 1 >> in the call to C<new ()>. To cover the widest
range of parsing options, you will always want to set binary.
But you still have the problem that you have to pass a correct line to the
C<parse ()> method, which is more complicated from the usual point of
usage:
my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
while (<>) { # WRONG!
$csv->parse ($_);
my @fields = $csv->fields ();
will break, as the while might read broken lines, as that doesn't care
about the quoting. If you need to support embedded newlines, the way to go
is either
my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
while (my $row = $csv->getline (*ARGV)) {
my @fields = @$row;
or, more safely in perl 5.6 and up
my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
open my $io, "<", $file or die "$file: $!";
while (my $row = $csv->getline ($io)) {
my @fields = @$row;
=head2 Unicode (UTF8)
On parsing (both for C<getline ()> and C<parse ()>), if the source is
marked being UTF8, then all fields that are marked binary will also be
be marked UTF8.
On combining (C<print ()> and C<combine ()>), if any of the combining
fields was marked UTF8, the resulting string will be marked UTF8.
For complete control over encoding, please use Text::CSV::Encoded:
use Text::CSV::Encoded;
my $csv = Text::CSV::Encoded->new ({
encoding_in => "iso-8859-1", # the encoding comes into Perl
encoding_out => "cp1252", # the encoding comes out of Perl
});
$csv = Text::CSV::Encoded->new ({ encoding => "utf8" });
# combine () and print () accept *literally* utf8 encoded data
# parse () and getline () return *literally* utf8 encoded data
$csv = Text::CSV::Encoded->new ({ encoding => undef }); # default
# combine () and print () accept UTF8 marked data
# parse () and getline () return UTF8 marked data
=head1 SPECIFICATION
While no formal specification for CSV exists, RFC 4180 1) describes a common
format and establishes "text/csv" as the MIME type registered with the IANA.
Many informal documents exist that describe the CSV format. How To: The Comma
Separated Value (CSV) File Format 2) provides an overview of the CSV format in
the most widely used applications and explains how it can best be used and
supported.
1) http://tools.ietf.org/html/rfc4180
2) http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm
The basic rules are as follows:
B<CSV> is a delimited data format that has fields/columns separated by the comma
character and records/rows separated by newlines. Fields that contain a special
character (comma, newline, or double quote), must be enclosed in double quotes.
However, if a line contains a single entry which is the empty string, it may be
enclosed in double quotes. If a field's value contains a double quote character
it is escaped by placing another double quote character next to it. The CSV file
format does not require a specific character encoding, byte order, or line
terminator format.
=over 2
=item *
Each record is one line terminated by a line feed (ASCII/LF=0x0A) or a
carriage return and line feed pair (ASCII/CRLF=0x0D 0x0A), however,
line-breaks can be embedded.
=item *
Fields are separated by commas.
=item *
Allowable characters within a CSV field include 0x09 (tab) and the inclusive
range of 0x20 (space) through 0x7E (tilde). In binary mode all characters
are accepted, at least in quoted fields.
=item *
A field within CSV must be surrounded by double-quotes to contain a
the separator character (comma).
=back
Though this is the most clear and restrictive definition, Text::CSV_XS is way
more liberal than this, and allows extension:
=over 2
=item *
Line termination by a single carriage return is accepted by default
=item *
The separation-, escape-, and escape- characters can be any ASCII character
in the range from 0x20 (space) to 0x7E (tilde). Characters outside this range
may or may not work as expected. Multibyte characters, like U+060c (ARABIC
COMMA), U+FF0C (FULLWIDTH COMMA), U+241B (SYMBOL FOR ESCAPE), U+2424 (SYMBOL
FOR NEWLINE), U+FF02 (FULLWIDTH QUOTATION MARK), and U+201C (LEFT DOUBLE
QUOTATION MARK) (to give some examples of what might look promising) are
therefor not allowed.
If you use perl-5.8.2 or higher, these three attributes are utf8-decoded, to
increase the likelihood of success. This way U+00FE will be allowed as a
quote character.
=item *
A field within CSV must be surrounded by double-quotes to contain an embedded
double-quote, represented by a pair of consecutive double-quotes. In binary
mode you may additionally use the sequence C<"0> for representation of a
NULL byte.
=item *
Several violations of the above specification may be allowed by passing
options to the object creator.
=back
=head1 FUNCTIONS
=head2 version ()
(Class method) Returns the current module version.
=head2 new (\%attr)
(Class method) Returns a new instance of Text::CSV_XS. The objects
attributes are described by the (optional) hash ref C<\%attr>.
Currently the following attributes are available:
=over 4
=item eol
An end-of-line string to add to rows. C<undef> is replaced with an
empty string. The default is C<$\>. Common values for C<eol> are
C<"\012"> (Line Feed) or C<"\015\012"> (Carriage Return, Line Feed).
Cannot be longer than 7 (ASCII) characters.
If both C<$/> and C<eol> equal C<"\015">, parsing lines that end on
only a Carriage Return without Line Feed, will be C<parse>d correct.
Line endings, whether in C<$/> or C<eol>, other than C<undef>,
C<"\n">, C<"\r\n">, or C<"\r"> are not (yet) supported for parsing.
=item sep_char
The char used for separating fields, by default a comma. (C<,>).
Limited to a single-byte character, usually in the range from 0x20
(space) to 0x7e (tilde).
The separation character can not be equal to the quote character.
The separation character can not be equal to the escape character.
See also CAVEATS
=item allow_whitespace
When this option is set to true, whitespace (TAB's and SPACE's)
surrounding the separation character is removed when parsing. If
either TAB or SPACE is one of the three major characters C<sep_char>,
C<quote_char>, or C<escape_char> it will not be considered whitespace.
So lines like:
1 , "foo" , bar , 3 , zapp
are now correctly parsed, even though it violates the CSV specs.
Note that B<all> whitespace is stripped from start and end of each
field. That would make it more a I<feature> than a way to be able
to parse bad CSV lines, as
1, 2.0, 3, ape , monkey
will now be parsed as
("1", "2.0", "3", "ape", "monkey")
even if the original line was perfectly sane CSV.
=item blank_is_undef
Under normal circumstances, CSV data makes no distinction between
quoted- and unquoted empty fields. They both end up in an empty
string field once read, so
1,"",," ",2
is read as
("1", "", "", " ", "2")
When I<writing> CSV files with C<always_quote> set, the unquoted empty
field is the result of an undefined value. To make it possible to also
make this distinction when reading CSV data, the C<blank_is_undef> option
will cause unquoted empty fields to be set to undef, causing the above to
be parsed as
("1", "", undef, " ", "2")
=item empty_is_undef
Going one step further than C<blank_is_undef>, this attribute converts
all empty fields to undef, so
1,"",," ",2
is read as
(1, undef, undef, " ", 2)
Note that this only effects fields that are I<realy> empty, not fields
that are empty after stripping allowed whitespace. YMMV.
=item quote_char
The char used for quoting fields containing blanks, by default the
double quote character (C<">). A value of undef suppresses
quote chars. (For simple cases only).
Limited to a single-byte character, usually in the range from 0x20
(space) to 0x7e (tilde).
The quote character can not be equal to the separation character.
=item allow_loose_quotes
By default, parsing fields that have C<quote_char> characters inside
an unquoted field, like
1,foo "bar" baz,42
would result in a parse error. Though it is still bad practice to
allow this format, we cannot help there are some vendors that make
their applications spit out lines styled like this.
In case there is B<really> bad CSV data, like
1,"foo "bar" baz",42
or
1,""foo bar baz"",42
there is a way to get that parsed, and leave the quotes inside the quoted
field as-is. This can be achieved by setting C<allow_loose_quotes> B<AND>
making sure that the C<escape_char> is I<not> equal to C<quote_char>.
=item escape_char
The character used for escaping certain characters inside quoted fields.
Limited to a single-byte character, usually in the range from 0x20
(space) to 0x7e (tilde).
The C<escape_char> defaults to being the literal double-quote mark (C<">)
in other words, the same as the default C<quote_char>. This means that
doubling the quote mark in a field escapes it:
"foo","bar","Escape ""quote mark"" with two ""quote marks""","baz"
If you change the default quote_char without changing the default
escape_char, the escape_char will still be the quote mark. If instead
you want to escape the quote_char by doubling it, you will need to change
the escape_char to be the same as what you changed the quote_char to.
The escape character can not be equal to the separation character.
=item allow_loose_escapes
By default, parsing fields that have C<escape_char> characters that
escape characters that do not need to be escaped, like:
my $csv = Text::CSV_XS->new ({ escape_char => "\\" });
$csv->parse (qq{1,"my bar\'s",baz,42});
would result in a parse error. Though it is still bad practice to
allow this format, this option enables you to treat all escape character
sequences equal.
=item binary
If this attribute is TRUE, you may use binary characters in quoted fields,
including line feeds, carriage returns and NULL bytes. (The latter must
be escaped as C<"0>.) By default this feature is off.
If a string is marked UTF8, binary will be turned on automatically when
binary characters other than CR or NL are encountered. Note that a simple
string like C<"\x{00a0}"> might still be binary, but not marked UTF8, so
setting C<{ binary => 1 }> is still a wise option.
=item types
A set of column types; this attribute is immediately passed to the
I<types> method below. You must not set this attribute otherwise,
except for using the I<types> method. For details see the description
of the I<types> method below.
=item always_quote
By default the generated fields are quoted only, if they need to, for
example, if they contain the separator. If you set this attribute to
a TRUE value, then all fields will be quoted. This is typically easier
to handle in external applications. (Poor creatures who aren't using
Text::CSV_XS. :-)
=item quote_space
By default, a space in a field would trigger quotation. As no rule
exists this to be forced in CSV, nor any for the opposite, the default
is true for safety. You can exclude the space from this trigger by
setting this option to 0.
=item keep_meta_info
By default, the parsing of input lines is as simple and fast as
possible. However, some parsing information - like quotation of
the original field - is lost in that process. Set this flag to
true to be able to retrieve that information after parsing with
the methods C<meta_info ()>, C<is_quoted ()>, and C<is_binary ()>
described below. Default is false.
=item verbatim
This is a quite controversial attribute to set, but it makes hard
things possible.
The basic thought behind this is to tell the parser that the normally
special characters newline (NL) and Carriage Return (CR) will not be
special when this flag is set, and be dealt with as being ordinary
binary characters. This will ease working with data with embedded
newlines.
When C<verbatim> is used with C<getline ()>, C<getline ()>
auto-chomp's every line.
Imagine a file format like
M^^Hans^Janssen^Klas 2\n2A^Ja^11-06-2007#\r\n
where, the line ending is a very specific "#\r\n", and the sep_char
is a ^ (caret). None of the fields is quoted, but embedded binary
data is likely to be present. With the specific line ending, that
shouldn't be too hard to detect.
By default, Text::CSV_XS' parse function however is instructed to only
know about "\n" and "\r" to be legal line endings, and so has to deal
with the embedded newline as a real end-of-line, so it can scan the next
line if binary is true, and the newline is inside a quoted field.
With this attribute however, we can tell parse () to parse the line
as if \n is just nothing more than a binary character.
For parse () this means that the parser has no idea about line ending
anymore, and getline () chomps line endings on reading.
=item auto_diag
Set to true will cause C<error_diag ()> to be automatically be called
in void context upon errors.
In case of error C<2012 - EOF>), this call will be void.
If set to a value greater than 1, it will die on errors instead of
warn.
Future extensions to this feature will include more reliable auto-detection
of the C<autodie> module being enabled, which will raise the value of
C<auto_diag> with C<1> on the moment the error is detected.
=back
To sum it up,
$csv = Text::CSV_XS->new ();
is equivalent to
$csv = Text::CSV_XS->new ({
quote_char => '"',
escape_char => '"',
sep_char => ',',
eol => $\,
always_quote => 0,