What information is available about these file formats? What tools are available for parsing these files?
Very little information is publicly available. Here's the little I've found:
MDL and SLX are MathWorks proprietary file formats for storing Simulink models. SLX was introduced in Simulink R2012a and made the default file format in R2012b. Besides the file structure, the content of SLX and MDL files are very similar. For example, key-value pairs appear to be the same between the two formats. People often say that parsing these files is a bad idea because they can change between Simulink versions (see e.g. am304 and my comments above), but I have not seen much evidence of this.
The MDL format seems to have been developed in-house at MathWorks. There seems to have been an MDL parser for python, but it was of limited functionality, and the website is down as of May 2014.
An SLX file is a zip file containing a collection of XML files, with most of the model specification stored in simulink/blockdiagram.xml. #am304 pointed out this information from the MathWorks website:
SLX is a compressed package that conforms to the Open Packaging
Conventions (OPC) interoperability standard. SLX stores model
information using Unicode® UTF-8 in XML and other international
formats. Saving Simulink models in the SLX format:
Typically reduces file size compared to MDL. The file size reduction between MDL and SLX varies depending on the model.
Solves some problems in previous releases with loading and saving MDL files containing Korean and Chinese characters.
Enables incremental loading and saving. Simulink optimizes performance and memory usage by loading only required parts of the
model and saving only modified parts of the model.
Here are a few more references besides the ones in the text above:
How convert simulink files to XML
http://www.scootersoftware.com/vbulletin/showthread.php?t=11568
http://blog.xogeny.com/blog/dont-zip/
http://blog.developpez.com/matlab/p11469/simulink-2/nouveau-format-slx-pour-les-modeles-simulink
Update (2015/04/02)
The new version of the Simulink Library for Java has full SLX format support. The documentation is not explicit, but the source code contains all details for parsing it.
Old answer
As answered by rob, the Simulink Library for Java supports Simulink's MDL file format and also can parse the Stateflow content. The library is Open Source, but the only documentation is the source code.
We are currently (as of September 2014) working on SLX support and expect to release this in the next 1 or 2 months. If you need the code before this time, feel free to contact me.
It is true that when using the library, your code may possibly break with a new Simulink release, as the file format is not documented and we had to reverse engineer most of it. However, we are currently actively updating the library in case of problems and with the source code you might be able to fix it even if we are not around.
PS: I would have posted this as a comment to rob's answer, but it seems I do not have sufficient reputation to do so :(
Disclosure: I am one of the developers of the mentioned library.
What information is available about these file formats?
MathWorks does have some documentation for the MDL file format in R2007b.
SLX files are zipfile containers whose internal structure is based on OOXML's OPC format. The SLX files contain one or more XML files whose internal structure is similar to that of an MDL file, but in XML format. In addition, binary resources such as graphics may be stored in separate JPG files rather than being text-encoded and directly embedded as they are in an MDL file.
Both formats change as new features are added to Simulink, but you can expect SLX to be less stable as MathWorks refactors SLX's internal file structure. For example, in R2014b, MathWorks has started breaking sections of the traditionally monolithic blockdiagram.xml out into separate files such as stateflow.xml and graphicalInterface.xml.
What tools are available for parsing these files?
There are a few publicly-available libraries/APIs for parsing Simulink, but I haven't used any of them so I'm not sure how well they work.
Simulink Library for Java (formerly called ConQAT) (Java) - MDL, SLX
TSMP - Tiny Simulink Model Parser (.NET) - MDL
Simulink-Model-Parsing-Tools (Python) - MDL
You may also be able to find others by searching for Simulink parser.
If none of those do the trick, some commercial tools parse MDL and SLX directly rather than relying on the MATLAB API. You could possibly inquire about licensing the parser used in some commercially-available Simulink tool.
Related
What are people using in vfp 9 for a replacement for the built-in scctext.prg that translates binary files in vfp to a textual representation?
We’ve moving an existing project that’s in vfp 9 sp1 into tfs source control, but we need a way to make sure that the non-textual files are able to get the benefits of comparison that only non-binary text files allow. We plan to check both the textual representation and the binary file into source control (the binary is more for the “just in case” scenario)
According to the document at
http://www.ita-software.com/papers/Borup_Mercurial_Published.pdf
there are at least three options for converting .scx, .frx, .lbx, .prj and other non-prg dbf files in visual foxpro (vfp) to a textual representation. Only some of them allow for converting the textual information back to binary - not sure how often we’d really use that or not.
ALTERNATE SCCTEXT
This one seems older with latest version in 2009 - not sure if it’s still the preferred tool - and it seems to have no way to take the textual representation and convert it back to a binary file.
http://vfpx.codeplex.com/releases/view/12955
TWOFOX
This one seems similar to the foxbin2prg except it creates xml files - seems like only one dev is working on it unlike the others that are open to contributions from others so not sure how current it is and how much it’s being used by other developers - it does have two way conversion like fox2binprg has.
http://www.foxpert.com/downloads.htm
FOXBIN2PRG
This one is fairly recent - but not sure if it’s production ready enough to use for prod coding working - it does have two way conversion
http://vfpx.codeplex.com/releases/view/116407
TRIGGER INVOKE ONE OF THE ABOVE ON CHANGE OF BINARY FILES IN VFP IDE
What are people using to invoke these textual representation options?
I’ve seen this class that was created to run one of the programs listed above for all files in the project. Apparently it does it when the date time of the last generate is older that the date time on the textual version of the file. One detriment I’ve read is that it generates for foundation classes and other things that really are not items that a dev is working on (code that is referenced by but not included in your project).
http://codepaste.net/9yy1gm
Thanks for any advice from those that are using vfp 9 with source control out there!
You should check out the scX library written by Paul McNett which is published on Ed Leafe's web site. I haven't used it in a mission-critical software project yet, but I have tested it out. It seemed to catch all the potential problems I've encountered with other scctext replacements.
The reason I haven't used it in a big project for a couple of reasons.
It is a breaking change for source control history. So, comparing source code in your current SCA or VCA files with the new files generated by scX isn't going to be simple.
It isn't a drop in replacement for scctext. Instead of checking files into and out of source control directly from the IDE, you'll have an intermediary folder.
You'll check your files out of source control into one folder, convert them to FoxPro format, and then edit them in the FoxPro IDE.
Then, you'll save your changes in the FoxPro IDE, convert them to scX format, and then check them into source control.
I'm sure much of #2 can be automated; but combined with #1, making the change to scX wasn't worth it for me.
FoxBin2Prg is Production ready, and AFAIK, it's the only tool that allow Diff and Merge of the generated text (tx2) files, and can regenerate the binaries from them.
The generated files are PRG style, so developers can see them as modifying a PRG (with PROc/ENDPROC structures and such), but they aren't mean to compile. Primary use is for SCM tools, but can be used seperately.
I'm actually using on production code with a 10 member team using concurrent modifications on forms and classes.
Some documentation is available on VFPx in English and Spanish, Internal messages are vailable on both languages and from version v1.19.24 a new translation to German is available too.
More info on VFPx site,
Best regards!
Firstly, i'm very poor in data pre-processing. I was looking for WebKB data in libsvm format. Later after searching a lot over the internet, i came across this data obtained after stemming and stop-word removal. The format is as follows,
Each line represents a vector and the first word in each file contains the class name followed by some list of words which forms the feature delimited by spaces.
How do i convert such a text file to lib-svm format? Is there any Weka or Matlab tool to construct it?
libshorttext1.1 is a python module having utilities for this purpose with so many extra features. try it, or i think scikit learn packages also have this functionality
I am using a MATLAB script to tune the control system on a machine. When the tuning is complete, I would like a report containing text (especially serial number, date/time and the values determined during tuning) and plots, especially transfer functions.
What do to you recommend?
Whatever solution I use should be compatible with the MATLAB compiler so I can distribute my solution to a team of field engineers.
Ideally the report will be a PDF document.
The MATLAB report generator does not seem to be the right product as it appears that I have to break up my script into little pieces and embed them in the report template. My script contains opportunities for the user to intervene and change values or reject the tune if plots don't look right and my hunch is that this will be difficult if the code runs from the report generator. Also, I fear code structure and maintainability will be lost if the code structure is determined by the requirements of the report template.
Please comment if my assumptions are wrong.
UPDATE
I have now switched to use the MATLAB Report Generator with release r2016b and it is working very well for my compiled code users. Unfortunately it means that colleagues who have a MATLAB licence need to buy the Report Generator too, to use my tools scripted.
As the MATLAB Report Generator's development manager, I am concerned that this question may leave the wrong impression about the Report Generator's capabilities.
For one thing, the Report Generator does not require you to break a script up into little pieces and run them inside a template. You can do this if you choose and in some circumstances, it makes sense, but it is not a requirement. In fact, many Report Generator applications use a MATLAB script or program to interact with a user, generate data in the MATLAB workspace, and as a final step, generate a report from the workspace data.
Moreover, as of the R2014b version, the MATLAB Report Generator comes with a document generation API, called the DOM API, that allows you to embed document generation statements in a MATLAB program. For example, you can programmatically create a document object, add and format text, paragraphs, tables, images, lists, and subdocuments, and output Microsoft Word, HTML, or PDF output, depending on the output type you select. You can even programmatically fill in the blanks in forms that you create, using Word or an HTML editor.
The API runs on Windows, Linux, and Mac platforms and generates Word and HTML output on all three, without the use of Word. On Windows, it uses Word under the hood to produce PDF output from the Word documents that it generates.
The latest release of the MATLAB Report Generator introduces a PowerPoint API with capabilities similar to the DOM API. If you need to include report generation in your MATLAB application, please don't rule out the MATLAB Report Generator based on past impressions. You may be surprised at just how powerful it has become.
I've done this quite a bit. You're right that MATLAB Report Generator is typically not a great solution. #Max suggests the right approach (automating Word through its COM interface), but I'd add a few extra comments and tips, based on my experiences.
Remember that if you're going with this solution, you are depending that your end-users will be running Windows, and have a copy of Office on their machine. If you want to ultimately produce a PDF report, that will need to be Office 2010 or above.
I would bet that you'll find it easier to automate the report generation in Excel rather than Word. Given that you're producing a report from MATLAB, you'll likely be wanting quite a lot of things in tables of numbers, which are easier to lay out in Excel.
If you are going to do it in Word, the easiest way is to first (without MATLAB) create a template .doc/.docx file, which contains any generic text that will be the same for all reports and blank tables for any information. Turn on track changes, and insert empty comments at each point that you will be filling in information. Then within your report creation routine in MATLAB, connect to Word and iterate through each comment, replacing it with whatever data you wish.
If you are learning to automate Excel from MATLAB, this page from the Excel Interop documentation is really helpful. There's an equivalent one for Word.
Unlike #Max, I've never had good results by saving figures to an .emf file and then inserting them. In theory that does preserve editability, but I've never found that valuable. Instead, get the figure looking right (and the right size) in MATLAB, then copy it to the clipboard with print(figHandle, 'dbitmap') and paste to Excel with Worksheet.Range('A1').PasteSpecial.
To save as a PDF, use Workbook.ExportAsFixedFormat('xlTypePDF', pathToOutputFile).
Hope that helps!
I think you are right about the report generator.
In my opinion the fastest/easiest approach would be to generate the report in a html document. For that you just need the figures and write a text file, conversion should be trivial.
Quite similar approach would be to create a Latex file. And then create a pdf from it - though for this you'd need to install latex on your deployed machines.
Lastly you could use the good integration of Java in Matlab. There are several libraries you could use - like this. But I wonder if all the complication will be worth it.
Have you considered driving Microsoft Word through its ActiveX interface? I've done this in compiled Matlab programs and it works well. Look at the Matlab help for actxserver(): The object you want to create is of type Word.Application.
Edit to add: To get figures into the document, save them as .emf files using the -dmeta argument to print(), then add them to the document like this:
WordServer.Selection.InlineShapes.AddPicture(fileName);
Has anyone ever heard of something that might facilitate the work with XMP metadata in Matlab?
For instance, EXIF metadata can be read simply by using the exifread command -
output = exifread(filename);
I've found this thread, but it seems to be dead.
Currently I am thinking about the following options:
Writing MEX file using C++ XMP SDK
Calling Java routines using JAVA XMP SDK
To summarize, the question is:
Do you have any idea on how XMP can be read/written in Matlab?
XMP is just XML, so you can use any MATLAB XML toolbox. My personal favourite is xml_io_tools.
If you want to use the SDK to avoid having to manually interpret what bits of the XML means, then of your two options the Java one sounds preferable. Calling Java from MATLAB is straightforward, and you avoid the hassle of building things that MEX entails.
I have found the answer. The best way is to download ExifTool and any Matlab JSON parser. It is possible to extract it from any file format, including .DNG, .XMP, .JPEG, .TIFF.
Step 1: Extract the info into temporary JSON file by using
system(['exiftool -struct -j ' fileName '>' tempFile]);
Step 2: Call the JSON parser on the tempFile
Step 3: You have the data in Matlab struct.
I've downloaded a data set that I am interested in. However, it is in .mat format and I do not have access to Matlab.
I've done some googling and it says I can open it in SciLab.
I tried a few things, but I haven't found any good tutorials on this.
I did
fd = matfile_open("file.mat")
matfile_listvar(fd)
and that prints out the filename without the extension. I tried
var1 = matfile_varreadnext(fd)
and that just gives me "var1 = "
I don't really know how the data is organized. The repository described the data it contains, but not how it is organized.
So, my question is, what am I doing wrong in extracting/viewing this data? I'm not committed to SciLab, if there is a better tool for this I am open to that.
One options is to use Octave, which can read .mat files and run most Matlab .m files. Octave is open source with binaries available for Linux, Mac, and Windows. Inside of Octave you can load the file using:
load file
See Octave's manual section 14.1.3 Simple File I/O for more details.
In Scilab:
loadmatfile('file.mat');
(Source)
I had this same interest a few years back. I used this question as a guide. It uses Python and SciPy. There are options for NumPy and hd5f as well. Another option is to write your own reader for the .mat format in whatever language you need. Here is the link to the mat file format definition.