I have a .dat file that is space/tab delimited with 1 line of headers. There are about 60 columns of data in this file. There are others with other numbers of columns.
How can I read in the headers (as a vector, perhaps?) such that I can index into the appropriate column of the data-matrix without having to count my way manually to the correct column?
I seem to recall Matlab being able to create cell-arrays with headers as indexes. Is anything like that remotely possible in Octave?
So far, all I can get is the actual data according to this:
a = dlmread('Core.dat'," ",r0=1,c0=0);
Any and all help is much appreciated! Thanks!
I've been looking for an easy way to do this using just the standard packages, also, but there doesn't seem to be one.
However, it does look like the dataframe package, might let you do this sort of thing.
It does seem like something simpler should be built into the language, though.
Related
At the moment I have a lookup table in a Simulink model which is reading the first and second columns from the workspace; in other words, I put the names of my vector variables in these fields. Then, I generated the code using rsim.tlc, run the batch and got the expected results. Nevertheless, when I try to run the batch again changing the vectors (they have different length in comparison with the ones I used when compiled the executable), I always get a message which says that the checksum number mismatches. I already verified if the rtP structure correctly reads the new values I´m using in my lookup table, thus I have no idea how to solve this.
Could someone help me?
As additional information, my target is to output the number if a second column of the lookuptable based on a clock number which is seek in the first column of the table. I would not mind using a .mat in the table, but I don´t have any idea how to do so.
I would appreacite any hint to solve this.
Don´t hesitate to ask me for more info! thanks in advance
In the newest version of MATLAB there are two new data types: Tables and Categorical Arrays.
Table is a new data type suitable for holding data and metadata, and can be used with mixed-type tabular data that are often stored as columns in a text file or in a spreadsheet. It consists of rows and column-oriented variables.
Categorical arrays are useful for holding categorical data - which have values from a finite list of discrete categories.
In previous versions I would have handled these use cases using cell and struct arrays. What are the differences between these and the new data types?
I haven't upgraded yet so I can't play around but based on this video and this article I can already see some advantages. They're not necessarily adding functionality that you couldn't do before, but rather just taking the hassle out of it. Using readtable over xlsread is immediately appealing to me. Being able to access columns by name rather than just by index is great, I do it in other languages often. In a table where column order doesn't really matter (unlike a matrix) it's really convenient to be able to address a column by it's name instead of having to know the column order. Also you can merge table using the join function which wasn't that easy to do with cell arrays before. I see that you can name the rows too, I didn't see what advantage that gives you and I can't play around but I know in some languages (like PANDAS in Python and I think in R as well) naming rows means you can work with time series data with different series that are not completely overlapping and not have to worry about alignment. I hope this is the case in Matlab too! Categorical arrays also look like just an extra layer of convenience, kind of like an enum. You never actually need a enum but it just makes development more pleasant.
Anyway that's just my two cents, I probably won't get an opportunity to play around with them any time soon but I look forward to using them when I do need them.
I use the table format to organize different input/output cases in my data, where the result may come from different tables. Main advantages compared to struct or cell array:
convenient table functions such as join, innerjoin, outerjoin
the use of fields <> more robust programming than arrays
data format is easy to export/import (e.g. delimited .txt file) <> no fprintf()
the data file can be opened in excel/Calc (libreoffice) <> no .mat
Lets say I have the following spreadsheet that I can parse in perl which looks like this:
I want to insert a column between Column1 and Column2. So end result looks like this:
It doesn't look like there is a set method for this in Spreadsheet-WriteExcel.
Does anyone know an easy way of doing this in Perl?
Many Thanks in advance!
The only(*) thing that Spreadsheet::WriteExcel can do is write spreadsheets. It doesn't have any facilities for reading an existing spreadsheet. Using it to modify an existing spreadsheet would involve first reading it with some other method (like Spreadsheet::ParseExcel) and then writing a brand new spreadsheet with the data the way you want it. Note that if you try this, you will lose macros, graphs, and any other feature that Spreadsheet::WriteExcel doesn't support.
The documentation for Spreadsheet::WriteExcel goes over a lot of the alternatives. See the WRITING EXCEL FILES and MODIFYING AND REWRITING EXCEL FILES sections. Win32::OLE for instance, gives you full access to Excel's internals with all the power and ease-of-use you'd expect from a Microsoft API. I'll leave it to you to judge whether any of these approaches qualify as "easy".
(*) - I don't mean that in a bad way. A Perl module that can write spreadsheets is pretty freaking cool.
I'm trying to store a lightly filtered copy of a database for offline reference, using ADO.NET DataSets. There are some columns I need not to take with me. So far, it looks like my options are:
Put up with the columns
Get unmaintainably clever about the way I SELECT rows for the DataSet
Hack at the XML output to delete the columns
I've deleted the columns' entries in the DataSet designer. WriteXMl still outputs them, to my dismay. If there's a way to limit WriteXml's output to typed rows, I'd love to hear it.
I tried to filter the columns out with careful SELECT statements, but ended up with a ConstraintException I couldn't solve. Replacing one table's query with SELECT * did the trick. I suspect I could solve the exception given enough time. I also suspect it could come back again as we evolve the schema. I'd prefer not to hand such a maintenance problem to my successors.
All told, I think it'll be easiest to filter the XML output. I need to compress it, store it, and (later) load, decompress, and read it back into a DataSet later. Filtering the XML is only one more step — and, better yet, will only need to happen once a week or so.
Can I change DataSet's behaviour? Should I filter the XML? Is there some fiendishly simple way I can query pretty much, but not quite, everything without running into ConstraintException? Or is my approach entirely wrong? I'd much appreciate your suggestions.
UPDATE: It turns out I copped ConstraintException for a simple reason: I'd forgotten to delete a strongly typed column from one DataTable. It wasn't allowed to be NULL. When I selected all the columns except that column, the value was NULL, and… and, yes, that's profoundly embarrassing, thank you so much for asking.
It's as easy as Table.Columns.Remove("UnwantedColumnName"). I got the lead from
Mehrdad's wonderfully terse answer to another question. I was delighted when Table.Columns turned out to be malleable.
I'm guessing this won't apply to 99.99% of anyone that sees this. I've been doing some Sawtooth survey programming at work and I've been needing to create a webpage that shows some aggregate data from the completed surveys. I was just wondering if anyone else has done this using the flat files that Sawtooth generates and how you went about doing it. I only know very basic Perl and the server I use does not have PHP so I'm somewhat at a loss for solutions. Anything you've got would be helpful.
Edit: The problem with offering example files is that it's more complicated. It's not a single file and it occasionally gets moved to a different file with a different format. The complexities added in there are why I ask this question.
Doesn't Sawtooth export into CSV format? There are many Perl parsers for CSV files. Just about every language has a CSV parser or two (or twelve), and MS Excel can open them directly, and they're still plaintext so you can look at them in any text editor.
I know our version of Sawtooth at work (which is admittedly very old) exports Sawtooth data into SPSS format, which can then be exported into various spreadsheet formats including CSV, if all else fails.
If you have a flat (fixed-width field) file, you can easily parse it in Perl using regular expressions or just taking substrings of each line one at a time, assuming you know the width of the fields. Your question is too general to give much better advice, sorry.
Matching the values up from a plaintext file with meta-data (variable names and labels, value labels etc.) is more complicated unless you already have the meta-data in some script-readable format. Making all of that stuff available on a web page is more complicated still. I've done it and it can be a bit of a lengthy project to roll your own. There are packages you can buy, like SDA, which will help you build a website where people can browse and download your survey data and view your codebooks.
Honestly though the easiest thing to do if you're posting statistical data on a website is get the data into SPSS or SAS or another statistics package format and post those files for download directly. Then you don't have to worry about it.