Export ElasticSearch objects to CSV with Perl - perl

Question

I need to export some objects from an ElasticSearch db in the form of CSV "tables".
I just need to retrieve all records from a specified index.
I've tried this found from clintongormley, but I'm facing inssues. The perl code is:
#!/usr/bin/perl
use ElasticSearch;
use Text::CSV_XS;
my $csv_file = 'output.csv';
open my $fh, '>:encoding(utf8)', $csv_file or die $!;
my $csv = Text::CSV_XS->new;
my $e = ElasticSearch->new(servers => '127.0.0.1:9200');
my $s = $e->scrolled_search(
index => 'myindex',
type => 'mytype',
query => { match_all => '' }
);
my #field_names = qw(title name foo bar);
while (my $doc = $s->next) {
my #cols = map {$doc->$_} #field_names;
$csv->print($fh, \#cols);
}
close $fh or die $!;
I get the following:
[_na] query malformed, no field after start_object];
I think the problem is in the es query.
Any suggestions?

Answer 1

Mea culpa. I wrote that code very quickly without testing it :)
Also, the code is very old and refers to the now deprecated ElasticSearch.pm modulde. The new module is Elasticsearch.pm (note the small s).
Here is the code rewritten to use the new module, and so it actually works:
#!/usr/bin/perl
use Elasticsearch;
use Elasticsearch::Scroll;
use Text::CSV_XS;
my $csv_file = 'output.csv';
open my $fh, '>:encoding(utf8)', $csv_file or die $!;
my $csv = Text::CSV_XS->new;
$csv->eol("\r\n");
my $es = Elasticsearch->new( nodes => '127.0.0.1:9200' );
my $s = Elasticsearch::Scroll->new(
es => $es,
index => 'myindex',
type => 'mytype',
body => { query => { match_all => {} } }
);
my #field_names = qw(title count);
while ( my $doc = $s->next ) {
my #cols = map { $doc->{_source}{$_} } #field_names;
$csv->print( $fh, \#cols );
}
close $fh or die $!;
To test it, you can run these curl commands to setup an index with some data:
# delete the test index in case it already exists
curl -XDELETE localhost:9200/myindex
# create some sample docs
curl -XPOST localhost:9200/myindex/mytype/_bulk -d'
{"index": {}}
{"title": "Doc one", "count": 1}
{"index": {}}
{"title": "Doc two", "count": 2}
{"index": {}}
{"title": "Doc three", "count": 3}
'
If you then run the Perl code, the file output.csv will look like this:
"Doc two",2
"Doc three",3
"Doc one",1
Apologies for the bad original example code