I would like to import a CSV into Org-mode. Others have already asked about importing CSV to Org-mode tables. That's not what I am trying to do. I need to import CSV to Org-mode properties.
For example, a CSV like this:
Name,Tel,Mobile,Fax
John,11111,22222,33333
should become:
:PROPERTIES:
:Name: John
:Tel: 11111
:Mobile: 22222
:Fax: 33333
:END:
Do you happen to know a painless way to do it?
The easiest approach I can see is to mark the data rows as the region, and then use a regexp search and replace:
M-x replace-regexp RET \(.*\),\(.*\),\(.*\),\(.*\) RET :PROPERTIES: C-q C-j :Name: \1 C-q C-j :Tel: \2 C-q C-j :Mobile: \3 C-q C-j :Fax: \4 C-q C-j :END: RET
If you needed to do this for many variable CSV files with different headers and numbers of columns, then I would probably approach it with keyboard macros.
user310031's answer would make a good basis for that. The macro could narrow the buffer to each row, insert the header row above it, perform the csv-transpose (which appears to require CSV mode) do the search+replace, add in the :PROPERTIES: and :END: lines, widen the buffer again, and leave point on the line before the next data row. Then just mark the remaining data rows as the region, and type C-x C-k r.
use csv-mode, transpose rows and columns by csv-transpose and format with replace-regexp:
search \(.*\),\(.*\)
replace for: :\1: \2
You can do something like this. Your example is not too clear. If there is more than one line, it will just set the properties in the same header over and over. you may want to use the Name to create a new heading, and then set properties on the heading. the code below works for pretty well formatted csv files.
(let ((lines (with-temp-buffer
(insert-file-contents "data.csv")
(split-string (buffer-string) "\n")))
(properties)
(values))
(setq properties (split-string (car lines) ","))
(loop for line in (cdr lines)
do
(setq values (split-string line ","))
(loop for property in properties
for value in values
do
(org-entry-put (point) property value))))
Related
I am trying to create an org-table based on the content of a CSV file that uses ";" as a delimiter.
I thought it would be easy to have a source code block to "cat" the content of the file and then pass the result to a function that creates the table, but I got stuck: I can't find a way to use the "results" of the first source code block. The function I know (org-table-convert-region) expects a region to work on and I don't know how to pass the "cat"-ed text as a region.
#+NAME: csvraw
#+BEGIN_SRC sh :results raw
cat afile.csv
#+END_SRC
I'd appreciate your help to generate a code block that produces an org-table out of my csv file, which contains lines like the following:
ID;Region;SubRegion;Area;No
1234;Asia;India;45;2
24251;Europe;Romania;456;67
There is org-table-convert-region (bound to C-c |) which can do the transformation fairly simply. The only trick is to specify ; as the separator. You can do that by invoking it with the proper prefix argument - the doc string says:
(org-table-convert-region BEG0 END0 &optional SEPARATOR)
Convert region to a table.
The region goes from BEG0 to END0, but these borders will be moved
slightly, to make sure a beginning of line in the first line is included.
SEPARATOR specifies the field separator in the lines. It can have the
following values:
(4) Use the comma as a field separator
(16) Use a TAB as field separator
(64) Prompt for a regular expression as field separator
integer When a number, use that many spaces, or a TAB, as field separator
regexp When a regular expression, use it to match the separator
nil When nil, the command tries to be smart and figure out the
separator in the following way:
- when each line contains a TAB, assume TAB-separated material
- when each line contains a comma, assume CSV material
- else, assume one or more SPACE characters as separator.
The (64) value is just three C-u in a row, so the process is as follows:
insert the CSV file with C-x i.
C-x C-x to mark the inserted contents as the active region.
C-u C-u C-u C-c | ; RET
What's even cooler, leaving an empty line in the CSV file between the first line and the rest of the lines, will make the first line a header in the table automatically.
And you can wrap it up in a code block as well:
#+begin_src elisp :var file="/tmp/foo.csv" :results raw
(defun csv-to-table (file)
(with-temp-buffer
(erase-buffer)
(insert-file file)
(org-table-convert-region (point-min) (point-max) ";")
(buffer-string)))
(csv-to-table file)
#+end_src
#+RESULTS:
| a | b | c |
|---+---+---|
| d | e | f |
| g | h | i |
(defun jea-convert-csv-to-org-table (fname)
(interactive "fCSV to convert: ")
(let ((result '("|-\n")))
(with-temp-buffer
(save-excursion (insert-file-contents-literally fname))
(while (and (not (eobp)) (re-search-forward "^\\(.+\\)$" nil t nil))
(push (concat "|" (replace-regexp-in-string ";" "|" (match-string 1)) "|\n")
result))
(push '"|-\n" result))
(concat (seq-mapcat #'identity (reverse result)))))
install the elisp code in your ~/.emacs file and restart emacs. Or, better yet, eval it into existence (CTRL+x, CTRL+e or ALT+x eval-last-sexp).
#+NAME: csvraw
#+BEGIN_SRC elisp :results raw
(jea-convert-csv-to-org-table "/Users/jamesanderson/Downloads/test1.csv")
#+END_SRC
note the change to elisp from sh in the above. here is a gif of it in action:
emacs converting csv to org table
The code is not optimized for large files, and, to be frank, quite quickly thrown together. But, after a tiny bit of testing, seems to work.
I often paste items separated by newlines or line feeds into an Emacs buffer resulting in each item residing on a different line like this:
one
two
three
four
Very often I actually want a list of comma-separated values like this:
"one", "two", "three", "four"
It would be great to be able to do a one-touch conversion from lines to list. I imagine I can convert this using a regex, but it seems like the kind of commonly used operation that might already have a built-in Emacs function. Can anybody suggest one?
M-q would replace linebreaks with spaces (in reasonably short lists of short words) but won't add quotes and commas. Or, maybe M-^ many times, until you have them all on the same line. Other than that - nothing built in comes to mind.
Obviously, a keyboard macro is a good candidate for this.
But a faster way, that doesn't create many undo steps would be something along these lines:
(defun lines-to-cslist (start end &optional arg)
(interactive "r\nP")
(let ((insertion
(mapconcat
(lambda (x) (format "\"%s\"" x))
(split-string (buffer-substring start end)) ", ")))
(delete-region start end)
(insert insertion)
(when arg (forward-char (length insertion)))))
Edit: I do see you're looking for a function... but since the only answer is simply to write your own (i.e. no built-in one exists), I figured I'd chime in with what the regex would be, since others may stumble on this and appreciate an alternative to writing the function and putting it in .emacs.
This is two steps, but only because you wanted your text quoted:
As pasted in Emacs *scratch* buffer (added five six to show it works with multiple words per line, if that's of interest):
one
two
three
four
five six
First, replace individual word with "word":
M-x replace-regexp RET \(.*\) RET "\1" RET produces:
"one"
"two"
"three"
"four"
"five six"
Now, replace each carriage return (in Emacs, C-q C-j) with a ,:
M-x replace-regexp RET C-q C-j RET , RET produces:
"one", "two", "three", "four", "five six"
I wrote a solution for this at work today. Below are functions to convert from lines to csv, and from csv to lines, with a user-specify-able separator. This function operates on the currently-highlighted region.
(defun lines-to-csv (separator)
"Converts the current region lines to a single line, CSV value, separated by the provided separator string."
(interactive "sEnter separator character: ")
(setq current-region-string (buffer-substring-no-properties (region-beginning) (region-end)))
(insert
(mapconcat 'identity
(split-string current-region-string "\n")
separator)))
(defun csv-to-lines (separator)
"Converts the current region line, as a csv string, to a set of independent lines, splitting the string based on the provided separator."
(interactive "sEnter separator character: ")
(setq current-region-string (buffer-substring-no-properties (region-beginning) (region-end)))
(insert
(mapconcat 'identity
(split-string current-region-string separator)
"\n")))
To use this, highlight the region you want to edit, then do M-x and specify the separator to use.
I usually accomplish these kinds of tasks using macros. M-X kmacro-start-macro and M-x kmacro-end-or-call-macro, which you can then do repeatedly then.
I have a single line text file of csv values
I would like to able to 'pretty-print' the file to span multiple lines to make it more readable
The 1st no. represents the no. of csv values in the next section and so on
e.g.
3,1,2,3,3,4,5,6
would be converted to:
3,1,2,3
3,4,5,6
I know a little about making macros, e.g.
C-x (
C-s RET ,
C-x )
using this I can do:
C-u 3 C-x e to move 3 csv values along
My sticking point is how to use the value from file to paste into the arg to C-u
maybe I should be using an e-lisp function instead as its a function I would like to 'save' for continual use across emacs sessions. Is it possible to save macros as such?
any ideas gratefully received
I find elisp easier to think about than keyboard macros. How about this:
(defun csv-line-breaks ()
(interactive)
(while (search-forward "," nil t
(1+ (string-to-number (thing-at-point 'word))))
(delete-char -1)
(insert "\n")))
(global-set-key (kbd "C-c b") 'csv-line-breaks)
With this in your .emacs (or just evaluate the code in your scratch buffer), you put point at the beginning of the line, then hit C-c b to break the line up into the chunks you want.
What this does:
Looping over the buffer until it runs out of values, and for each loop:
Read the first value. (thing-at-point 'word) grabs anything it finds between whitespace of punctuation (more or less).
Convert the value, which is actually a string, into a number
Add one to that number, and move forward that many commas
Delete the previous comma
Insert a new line
You might want to take a look at csv-mode for Emacs: http://emacswiki.org/emacs/CsvMode
Although it might not do exactly what you're looking for, it has a feature for formatting for readability, as well as other features for munging csv files in a variety of ways.
Emacs has a very nice extension by the name of org-mode.
I would like to be able to easily load CSV files into org-mode without significant grief. All I've been able to find is table-import or table-capture, which, simply put, don't work even approximately well.
Note that part of my issue is text strings with a comma within them. 1,2,3,4 is different than 1,2,"3,4".
Is there a function out there or a perl script that one could run to transform a csv file into org-mode format?
Thanks!
From the org-mode manual:
C-c | Convert the active region to
table. If every line contains at least
one TAB character, the function
assumes that the material is tab
separated. If every line contains a
comma, comma-separated values (CSV)
are assumed. If not, lines are split
at whitespace into fields. You can use
a prefix argument to force a specific
separator: C-u forces CSV, C-u C-u
forces TAB, and a numeric argument N
indicates that at least N consecutive
spaces, or alternatively a TAB will be
the separator. If there is no active
region, this command creates an empty
Org table.
So just paste the data into an org file, select it, and do C-u C-c | .
Have a look at:
C-h f org-table-convert-region
I'm always converting csv so I added this to my .emacs.
(defun org-convert-csv-table (beg end)
(interactive (list (mark) (point)))
(org-table-convert-region beg end ",")
)
(add-hook 'org-mode-hook
(lambda ()
(define-key org-mode-map (kbd "<f6>") 'org-convert-csv-table)))
update
Here is another function that considers the quoted commas as well :
a,"12,12",b --> a | 12,12 |b
feel free to improve it :-).
(defun org-convert-csv-table (beg end)
; convert csv to org-table considering "12,12"
(interactive (list (point) (mark)))
(replace-regexp "\\(^\\)\\|\\(\".*?\"\\)\\|," (quote (replace-eval-replacement
replace-quote (cond ((equal "^" (match-string 1)) "|")
((equal "," (match-string 0)) "|")
((match-string 2))) )) nil beg end)
I'm assuming you want to convert your CSV specifically into org-mode tables. If that's not the case, you may want to be more explicit about output format in your question.
Something like this should do it, or at least get you a starting point you can hack on:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
my $csv = Text::CSV->new();
while ( my $line = <DATA> ) {
if ( $csv->parse( $line )) {
my $str = join '|' , $csv->fields();
print "|$str|\n";
}
}
__DATA__
1,2,3,4
1,2,"3,4"
Try this:
;; Insert a file and convert it to an org table
(defun aleblanc/insert-file-as-org-table (filename)
"Insert a file into the current buffer at point, and convert it to an org table."
(interactive (list (ido-read-file-name "csv file: ")))
(let* ((start (point))
(end (+ start (nth 1 (insert-file-contents filename)))))
(org-table-convert-region start end)
))
Here is a bit of a hack-job, but it works.
when you export the CSV file, force quotes around each entry, then replace all "," as a vertical bar.
Finally, I make a macro that does something like this:
C-a ; Beginning-of-line
| ; Self-insert-char
C-e ; end-of-line
| ; Self-insert-char
<down> ; Down one line
(I am not 100% sure if | is a self-insert-char or not, so its best to record your own macro)
Hit tab somewhere in the middle of the table, and org-mode formats it properly. I find this easier to do in a narrowed region.
Caveat: If you have a vertical bar in your input.. it probably won't work quite right. Situations like that should be easy to spot, and relatively easy to fix.
Generally, when importing something text-like into org-mode, I have found a combination of some smart macro writing, and search-replace to be the easiest way
I hope that helps.
Say I have a line in an emacs buffer that looks like this:
foo -option1 value1 -option2 value2 -option3 value3 \
-option4 value4 ...
I want it to look like this:
foo -option1 value1 \
-option2 value2 \
-option3 value3 \
-option4 value4 \
...
I want each option/value pair on a separate line. I also want those subsequent lines indented appropriately according to mode rather than to add a fixed amount of whitespace. I would prefer that the code work on the current block, stopping at the first non-blank line or line that does not contain an option/value pair though I could settle for it working on a selected region.
Anybody know of an elisp function to do this?
Nobody had what I was looking for so I decided to dust off my elisp manual and do it myself. This seems to work well enough, though the output isn't precisely what I asked for. In this version the first option goes on a line by itself instead of staying on the first line like in my original question.
(defun tcl-multiline-options ()
"spread option/value pairs across multiple lines with continuation characters"
(interactive)
(save-excursion
(tcl-join-continuations)
(beginning-of-line)
(while (re-search-forward " -[^ ]+ +" (line-end-position) t)
(goto-char (match-beginning 0))
(insert " \\\n")
(goto-char (+(match-end 0) 3))
(indent-according-to-mode)
(forward-sexp))))
(defun tcl-join-continuations ()
"join multiple continuation lines into a single physical line"
(interactive)
(while (progn (end-of-line) (char-equal (char-before) ?\\))
(forward-line 1))
(while (save-excursion (end-of-line 0) (char-equal (char-before) ?\\))
(end-of-line 0)
(delete-char -1)
(delete-char 1)
(fixup-whitespace)))
In this case I would use a macro. You can start recording a macro with C-x (, and stop recording it with C-x ). When you want to replay the macro type C-x e.
In this case, I would type, C-a C-x ( C-s v a l u e C-f C-f \ RET SPC SPC SPC SPC C-x )
That would record a macro that searches for "value", moves forward 2, inserts a slash and newline, and finally spaces the new line over to line up. Then you could repeat this macro a few times.
EDIT: I just realized, your literal text may not be as easy to search as "value1". You could also search for spaces and cycle through the hits. For example, hitting, C-s a few times after the first match to skip over some of the matches.
Note: Since your example is "ad-hoc" this solution will be too. Often you use macros when you need an ad-hoc solution. One way to make the macro apply more consistently is to put the original statement all on one line (can also be done by a macro or manually).
EDIT: Thanks for the comment about ( versus C-(, you were right my mistake!
Personally, I do stuff like this all the time.
But I don't write a function to do it unless I'll be doing it
every day for a year.
You can easily do it with query-replace, like this:
m-x (query-replace " -option" "^Q^J -option")
I say ^Q^J as that is what you'll type to quote a newline and put it in
the string.
Then just press 'y' for the strings to replace, and 'n' to skip the wierd
corner cases you'd find.
Another workhorse function is query-replace-regexp that can do
replacements of regular expressions.
and also grep-query-replace, which will perform query-replace by parsing
the output of a grep command. This is useful because you can search
for "foo" in 100 files, then do the query-replace on each occurrence
skipping from file to file.
Your mode may support this already. In C mode and Makefile mode, at least, M-q (fill-paragraph) will insert line continuations in the fill-column and wrap your lines.
What mode are you editing this in?