I am trying to create an org-table based on the content of a CSV file that uses ";" as a delimiter.
I thought it would be easy to have a source code block to "cat" the content of the file and then pass the result to a function that creates the table, but I got stuck: I can't find a way to use the "results" of the first source code block. The function I know (org-table-convert-region) expects a region to work on and I don't know how to pass the "cat"-ed text as a region.
#+NAME: csvraw
#+BEGIN_SRC sh :results raw
cat afile.csv
#+END_SRC
I'd appreciate your help to generate a code block that produces an org-table out of my csv file, which contains lines like the following:
ID;Region;SubRegion;Area;No
1234;Asia;India;45;2
24251;Europe;Romania;456;67
There is org-table-convert-region (bound to C-c |) which can do the transformation fairly simply. The only trick is to specify ; as the separator. You can do that by invoking it with the proper prefix argument - the doc string says:
(org-table-convert-region BEG0 END0 &optional SEPARATOR)
Convert region to a table.
The region goes from BEG0 to END0, but these borders will be moved
slightly, to make sure a beginning of line in the first line is included.
SEPARATOR specifies the field separator in the lines. It can have the
following values:
(4) Use the comma as a field separator
(16) Use a TAB as field separator
(64) Prompt for a regular expression as field separator
integer When a number, use that many spaces, or a TAB, as field separator
regexp When a regular expression, use it to match the separator
nil When nil, the command tries to be smart and figure out the
separator in the following way:
- when each line contains a TAB, assume TAB-separated material
- when each line contains a comma, assume CSV material
- else, assume one or more SPACE characters as separator.
The (64) value is just three C-u in a row, so the process is as follows:
insert the CSV file with C-x i.
C-x C-x to mark the inserted contents as the active region.
C-u C-u C-u C-c | ; RET
What's even cooler, leaving an empty line in the CSV file between the first line and the rest of the lines, will make the first line a header in the table automatically.
And you can wrap it up in a code block as well:
#+begin_src elisp :var file="/tmp/foo.csv" :results raw
(defun csv-to-table (file)
(with-temp-buffer
(erase-buffer)
(insert-file file)
(org-table-convert-region (point-min) (point-max) ";")
(buffer-string)))
(csv-to-table file)
#+end_src
#+RESULTS:
| a | b | c |
|---+---+---|
| d | e | f |
| g | h | i |
(defun jea-convert-csv-to-org-table (fname)
(interactive "fCSV to convert: ")
(let ((result '("|-\n")))
(with-temp-buffer
(save-excursion (insert-file-contents-literally fname))
(while (and (not (eobp)) (re-search-forward "^\\(.+\\)$" nil t nil))
(push (concat "|" (replace-regexp-in-string ";" "|" (match-string 1)) "|\n")
result))
(push '"|-\n" result))
(concat (seq-mapcat #'identity (reverse result)))))
install the elisp code in your ~/.emacs file and restart emacs. Or, better yet, eval it into existence (CTRL+x, CTRL+e or ALT+x eval-last-sexp).
#+NAME: csvraw
#+BEGIN_SRC elisp :results raw
(jea-convert-csv-to-org-table "/Users/jamesanderson/Downloads/test1.csv")
#+END_SRC
note the change to elisp from sh in the above. here is a gif of it in action:
emacs converting csv to org table
The code is not optimized for large files, and, to be frank, quite quickly thrown together. But, after a tiny bit of testing, seems to work.
Related
I would like to count characters in a subtree (heading) in org mode. Right now I have figured out how to count characters in a single paragraph, but not over multiple paragraphs. I first define a source block:
#+NAME: countChars
#+BEGIN_SRC sh :var X="" :results output
echo "$X" | wc --chars
#+END_SRC
And then I use it on named paragraphs:
#+NAME: paragraph
This is the paragraph
#+CALL: countChars(paragraph)
This works well but the #+NAME: only covers one paragraph. I have tried to use a heading as argument but I could not make it work.
EDIT: Based on comments, I came up with:
#+NAME: countChars
#+BEGIN_SRC emacs-lisp :results output :eval no-export :exports results
(interactive)
(save-excursion
(org-mark-subtree)
(setq a (- (mark) (point)))
(deactivate-mark)
(prin1 'Count= )
(prin1 a))
#+END_SRC
which does almost what I want when invoked as
#+CALL: countChars()
but has the problem of counting source code blocks (including itself) as well as text. I would like to count text only (excluding the headline).
You can only use #+NAME in front of a source block, not a subtree.
It's easier to write this in emacs lisp.
This code block will count the number of characters in the current subtree, not including the header or the last line of the content:
If you want to count the number of characters in the current subtree using emacs lisp, try this:
(save-excursion
(org-mark-subtree) ;mark the whole subtre
(forward-line 1) ;move past header
(exchange-point-and-mark) ;swap point and mark (ends of region)
(forward-line -1) ;move backwards past the last line
(let ((nchars (- (point) (mark))))
(deactivate-mark) ;clear the region
(message "%d" nchars))))
I have a file formatted as
abc|<hoge>
a|<foo> b|<foo> c|<foo>
family|<bar> guy|<bar>
a|<foo> comedy|<bar> show|<foo>
action|<hoge>
and want to search search strings by raw (Like "a comedy show" instead of a|<foo> comedy|<bar> show|<foo>) on emacs.
I believe using grep on lisp would be the easiest answer but I have not yet figured out how. Would someone enlighten me?
Well, grep is a separate program (which you could also use). In Emacs, you'd use the function search-forward-regexp, which you can run using either M-x (hold Meta, usually Alt key, and press x) and then type search-forward-regexp and press Return.
You'll then need to key in the regexp to search. Put simply, it seems like you want to ignore |< something >, which in Emacs's variety of regexes is:
|<[a-z]+>
so you might search for e.g.
a|<[a-z]+> comedy|<[a-z]+> show|<[a-z]+>
You can create a Lisp function to convert a string this way, by splitting it on spaces and adding the regex sequences:
(defun find-string-in-funny-file (s) ; Define a function
"Find a string in the file with the |<foo> things in it." ; Document its purpose
(interactive "sString to find: ") ; Accept input if invoked interactively with M-x
(push-mark) ; Save the current location, so `pop-global-mark' can return here
; (usually C-u C-SPC)
(goto-char 0) ; Start at the top of the file
(let ((re (apply #'concat ; join into one string…
(cl-loop
for word in (split-string s " ") ; for each word in `s'
collect (regexp-quote word) ; collect that word, plus
collect "|<[a-z]+> ")))) ; also the regex bits to skip
(search-forward-regexp ; search for the next occurrence
(substring re 0 (- (length re) 2))))) ; after removing the final space from `re'
You can explore what those functions each do in the (online) Emacs Lisp manual; for example, pick from the menu "Help→Describe→Function" or press C-h f (Control+h, then f) and type interactive (RET) for the manual's documentation of that special form.
If you paste the above (defun) into the *scratch* buffer, and position the cursor after the final ) at the end, you can press C-j to evaluate it, and the function will remain with you until you close Emacs.
If you save it in a file named something .el, you can use M-x load-file to load it again in future.
If you then load your "funny" file, and type M-x find-string-in-funny-file, it'll search your file for your string, and leave the cursor on the string. If it's not found, you'll see a message to that effect.
BUGS: The function is less than spectacular style
I would like to import a CSV into Org-mode. Others have already asked about importing CSV to Org-mode tables. That's not what I am trying to do. I need to import CSV to Org-mode properties.
For example, a CSV like this:
Name,Tel,Mobile,Fax
John,11111,22222,33333
should become:
:PROPERTIES:
:Name: John
:Tel: 11111
:Mobile: 22222
:Fax: 33333
:END:
Do you happen to know a painless way to do it?
The easiest approach I can see is to mark the data rows as the region, and then use a regexp search and replace:
M-x replace-regexp RET \(.*\),\(.*\),\(.*\),\(.*\) RET :PROPERTIES: C-q C-j :Name: \1 C-q C-j :Tel: \2 C-q C-j :Mobile: \3 C-q C-j :Fax: \4 C-q C-j :END: RET
If you needed to do this for many variable CSV files with different headers and numbers of columns, then I would probably approach it with keyboard macros.
user310031's answer would make a good basis for that. The macro could narrow the buffer to each row, insert the header row above it, perform the csv-transpose (which appears to require CSV mode) do the search+replace, add in the :PROPERTIES: and :END: lines, widen the buffer again, and leave point on the line before the next data row. Then just mark the remaining data rows as the region, and type C-x C-k r.
use csv-mode, transpose rows and columns by csv-transpose and format with replace-regexp:
search \(.*\),\(.*\)
replace for: :\1: \2
You can do something like this. Your example is not too clear. If there is more than one line, it will just set the properties in the same header over and over. you may want to use the Name to create a new heading, and then set properties on the heading. the code below works for pretty well formatted csv files.
(let ((lines (with-temp-buffer
(insert-file-contents "data.csv")
(split-string (buffer-string) "\n")))
(properties)
(values))
(setq properties (split-string (car lines) ","))
(loop for line in (cdr lines)
do
(setq values (split-string line ","))
(loop for property in properties
for value in values
do
(org-entry-put (point) property value))))
Emacs has a very nice extension by the name of org-mode.
I would like to be able to easily load CSV files into org-mode without significant grief. All I've been able to find is table-import or table-capture, which, simply put, don't work even approximately well.
Note that part of my issue is text strings with a comma within them. 1,2,3,4 is different than 1,2,"3,4".
Is there a function out there or a perl script that one could run to transform a csv file into org-mode format?
Thanks!
From the org-mode manual:
C-c | Convert the active region to
table. If every line contains at least
one TAB character, the function
assumes that the material is tab
separated. If every line contains a
comma, comma-separated values (CSV)
are assumed. If not, lines are split
at whitespace into fields. You can use
a prefix argument to force a specific
separator: C-u forces CSV, C-u C-u
forces TAB, and a numeric argument N
indicates that at least N consecutive
spaces, or alternatively a TAB will be
the separator. If there is no active
region, this command creates an empty
Org table.
So just paste the data into an org file, select it, and do C-u C-c | .
Have a look at:
C-h f org-table-convert-region
I'm always converting csv so I added this to my .emacs.
(defun org-convert-csv-table (beg end)
(interactive (list (mark) (point)))
(org-table-convert-region beg end ",")
)
(add-hook 'org-mode-hook
(lambda ()
(define-key org-mode-map (kbd "<f6>") 'org-convert-csv-table)))
update
Here is another function that considers the quoted commas as well :
a,"12,12",b --> a | 12,12 |b
feel free to improve it :-).
(defun org-convert-csv-table (beg end)
; convert csv to org-table considering "12,12"
(interactive (list (point) (mark)))
(replace-regexp "\\(^\\)\\|\\(\".*?\"\\)\\|," (quote (replace-eval-replacement
replace-quote (cond ((equal "^" (match-string 1)) "|")
((equal "," (match-string 0)) "|")
((match-string 2))) )) nil beg end)
I'm assuming you want to convert your CSV specifically into org-mode tables. If that's not the case, you may want to be more explicit about output format in your question.
Something like this should do it, or at least get you a starting point you can hack on:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
my $csv = Text::CSV->new();
while ( my $line = <DATA> ) {
if ( $csv->parse( $line )) {
my $str = join '|' , $csv->fields();
print "|$str|\n";
}
}
__DATA__
1,2,3,4
1,2,"3,4"
Try this:
;; Insert a file and convert it to an org table
(defun aleblanc/insert-file-as-org-table (filename)
"Insert a file into the current buffer at point, and convert it to an org table."
(interactive (list (ido-read-file-name "csv file: ")))
(let* ((start (point))
(end (+ start (nth 1 (insert-file-contents filename)))))
(org-table-convert-region start end)
))
Here is a bit of a hack-job, but it works.
when you export the CSV file, force quotes around each entry, then replace all "," as a vertical bar.
Finally, I make a macro that does something like this:
C-a ; Beginning-of-line
| ; Self-insert-char
C-e ; end-of-line
| ; Self-insert-char
<down> ; Down one line
(I am not 100% sure if | is a self-insert-char or not, so its best to record your own macro)
Hit tab somewhere in the middle of the table, and org-mode formats it properly. I find this easier to do in a narrowed region.
Caveat: If you have a vertical bar in your input.. it probably won't work quite right. Situations like that should be easy to spot, and relatively easy to fix.
Generally, when importing something text-like into org-mode, I have found a combination of some smart macro writing, and search-replace to be the easiest way
I hope that helps.
Say I have a line in an emacs buffer that looks like this:
foo -option1 value1 -option2 value2 -option3 value3 \
-option4 value4 ...
I want it to look like this:
foo -option1 value1 \
-option2 value2 \
-option3 value3 \
-option4 value4 \
...
I want each option/value pair on a separate line. I also want those subsequent lines indented appropriately according to mode rather than to add a fixed amount of whitespace. I would prefer that the code work on the current block, stopping at the first non-blank line or line that does not contain an option/value pair though I could settle for it working on a selected region.
Anybody know of an elisp function to do this?
Nobody had what I was looking for so I decided to dust off my elisp manual and do it myself. This seems to work well enough, though the output isn't precisely what I asked for. In this version the first option goes on a line by itself instead of staying on the first line like in my original question.
(defun tcl-multiline-options ()
"spread option/value pairs across multiple lines with continuation characters"
(interactive)
(save-excursion
(tcl-join-continuations)
(beginning-of-line)
(while (re-search-forward " -[^ ]+ +" (line-end-position) t)
(goto-char (match-beginning 0))
(insert " \\\n")
(goto-char (+(match-end 0) 3))
(indent-according-to-mode)
(forward-sexp))))
(defun tcl-join-continuations ()
"join multiple continuation lines into a single physical line"
(interactive)
(while (progn (end-of-line) (char-equal (char-before) ?\\))
(forward-line 1))
(while (save-excursion (end-of-line 0) (char-equal (char-before) ?\\))
(end-of-line 0)
(delete-char -1)
(delete-char 1)
(fixup-whitespace)))
In this case I would use a macro. You can start recording a macro with C-x (, and stop recording it with C-x ). When you want to replay the macro type C-x e.
In this case, I would type, C-a C-x ( C-s v a l u e C-f C-f \ RET SPC SPC SPC SPC C-x )
That would record a macro that searches for "value", moves forward 2, inserts a slash and newline, and finally spaces the new line over to line up. Then you could repeat this macro a few times.
EDIT: I just realized, your literal text may not be as easy to search as "value1". You could also search for spaces and cycle through the hits. For example, hitting, C-s a few times after the first match to skip over some of the matches.
Note: Since your example is "ad-hoc" this solution will be too. Often you use macros when you need an ad-hoc solution. One way to make the macro apply more consistently is to put the original statement all on one line (can also be done by a macro or manually).
EDIT: Thanks for the comment about ( versus C-(, you were right my mistake!
Personally, I do stuff like this all the time.
But I don't write a function to do it unless I'll be doing it
every day for a year.
You can easily do it with query-replace, like this:
m-x (query-replace " -option" "^Q^J -option")
I say ^Q^J as that is what you'll type to quote a newline and put it in
the string.
Then just press 'y' for the strings to replace, and 'n' to skip the wierd
corner cases you'd find.
Another workhorse function is query-replace-regexp that can do
replacements of regular expressions.
and also grep-query-replace, which will perform query-replace by parsing
the output of a grep command. This is useful because you can search
for "foo" in 100 files, then do the query-replace on each occurrence
skipping from file to file.
Your mode may support this already. In C mode and Makefile mode, at least, M-q (fill-paragraph) will insert line continuations in the fill-column and wrap your lines.
What mode are you editing this in?