Problem: helpers fail when reading an .xls into R using gdata
I have some .xls that i'd like to read into R. I am able to do so using read.xls in the gdata package, however the helper functions sheetNames and sheetCount fail - i'd like to understand what i'm doing wrong so that i can use them as they would be very useful.
require(gdata)
fp <- file.path('~/data/first.xls')
When i know the structure of the sheets, and am able to point it at my intended data sheet, the perl script runs just fine:
firstdata <- read.xls(fp, sheet=2)
and i have my data... in firstdata.
However, in the very same sheets, the helpers fail.
I find myself opening the .xls in excel, figuring them out, and then loading into R using read.xls(fp, sheet=N) -- not a disaster, but neither is it ideal.
In particular, when the sheets are not my own, and I need information about them before i can set sheet=N in read.xls(), the helper functions sheetNames and sheetCount would be very useful, however they fail -- why?
sheetCount(fp)
> sheetNames(fp)
Error in read.table(tc, as.is = TRUE, header = FALSE) :
no lines available in input
In addition: Warning message:
running command ''/usr/bin/perl' '~/R/wd/raRpackages/gdata/perl/sheetNames.pl' '~/data/first.xls'' had status 2
Unable to open file '~/data/first.xls'.
and:
> sheetCount(fp)
Error in read.table(tc, as.is = TRUE, header = FALSE) :
no lines available in input
In addition: Warning message:
running command ''/usr/bin/perl' '~/R/wd/raRpackages/gdata/perl/sheetCount.pl' '~/data/first.xls'' had status 2
Unable to open file '~/data/first.xls'.
After a bit of fiddling, i (quite by accident) found that using the full path solves this problem:
fp2 <- file.path("/Users/ricardo/data/first.xls")
sheetcount(fp2)
[1] 13
It looks like ~ isn't being expanded out to your home directory. This expansion is usually done by the shell so R probably won't do it and perl definitely won't.
Instead, use the explicit path or $HOME or $ENV{HOME} from within the Perl program.
Related
I am trying a proof of concept here:
Using Chaquopy 4.0.0 (I use python 2.7.15), I am trying to write a string to file in a specific folder (getFilesDir()) using Python, then reading in via Android.
To check whether the file was written, I am checking for the file's length (see code below).
I am expecting to get any length latger than 0 (to verify that the file indeed has been written to the specific location), but I keep getting 0.
Any help would be greatly appreciated!!
main.py:
import os.path
save_path = "/data/user/0/$packageName/files/"
name_of_file = raw_input("test")
completeName = os.path.join(save_path, name_of_file+".txt")
file1 = open(completeName, "w")
toFile = raw_input("testAsWell")
file1.write(toFile)
file1.close()
OnCreate:
if (! Python.isStarted()) {
Python.start(new AndroidPlatform(this));
File file = new File(getFilesDir(), "test.txt");
Log.e("TEST", String.valueOf(file.length()));
}```
It's not clear whether you've based your app on the console example, so I'll give an answer for both cases.
If you have based your app on the console example, then the code in onCreate will run before the code in main.py, and the file won't exist the first time you start the activity. It should exist the second time: if it still doesn't, try using the Android Studio file explorer to see what's in the files directory.
If you haven't based your app on the console example, then you'll need to execute main.py manually, like this:
Python.getInstance().getModule("main");
Also, without the input UI which the console example provides, you won't be able to read anything from stdin. So you'll need to do one of the following:
Base your app on the console example; or
Replace the raw_input calls with a hard-coded file name and content; or
Create a normal Android UI with a text box or something, and get input from the user that way.
I have a Modelica file that references c code during simulation through an external library *.a file.
For example:
model CallAdd
input Real FirstInput(start=0);
input Real SecondInput(start=0);
output Real FMUOutput(start=0);
function CAdd
input Real x(start=0);
input Real y(start=0);
output Real z(start=0);
external "C" annotation(Library = "CAdd", LibraryDirectory = "modelica://CallAdd");
end CAdd;
equation
FMUOutput = CAdd(FirstInput,SecondInput);
annotation(uses(Modelica(version = "3.2.1")));
end CallAdd;
When opening the Modelica model in OpenModelica the required files appear to be automatically loaded because it simulates and gives appropriate results.
However, when I try to compile the Modelica file with JModelica-SDK-1.12 I receive an error that the library *.a file could not be found.
So my question is: What is the proper way to reference additional files when using compile_fmu in JModelica?
With no success, I've tried:
# Import the compiler function
from pymodelica import compile_fmu
model_name = "CallAdd"
mo_file = "CallAdd.mo"
# Compile the model and save the return argument, for use later if wanted
my_fmu = compile_fmu(model_name, mo_file, target="cs",compiler_options = {'extra_lib_dirs':'C:/ToFolderContainingLib/'})
The strange thing is that when I was using JModelica-1.17 (non-SDK) the file compiled fine but the results didn't make sense. I was recommended to try the SDK version to see if it fixed my errors in my previous post here.
Try positioning the external library in sub-folder named as the platform your currently on. So in your example, I'd position the library (libCAdd.a) in sub-folder named linux64, as I'm on a 64bit Linux machine and then run the code.
If is a small piece of C code, as a last alternative you could try to include the C file directly in the Modelica code:
external "C" annotation(Include="
// the entire C code here
");
Hopefully the JModelica people will give you a better answer soon.
You could try to ask this on their website also:
http://www.jmodelica.org/forum
we use a scripting environment to automatically build, run and verify a stand-alone executable. We are working with Matlab R2010a x64. The Matlab compiler mcc is called from the Windows command line building a stand-alone application:
mcc -m -v -w enable -I source_folder -I common_folder -a specific_files_we_need our_program
The program consists of about 25 modules (.m files) and is using about 5 toolboxes. This is working fine as long as the correct license is available. mcc checks for a compiler license available, resolves dependencies, packs everything in the executable.
However if the license does not include the required toolboxes, mcc does not issue any warning or error. It builds the executable without the toolboxes. So the executable starts, seems to run at a first glance but crashes if a line of code requiring a toolbox is reached.
I am expecting from a compiler that it informs me about missing components. What can I do to get informed about missing components? How can I make sure that mcc does not put together incomplete executables? Am I missing something in the call to mcc?
Preferably I would like to setup the compiling in a way that it stops if things are missing.
\Zweikeks
The simplest way is in your compilation script you can checkout the licenses required, i.e.
license('checkout','Compiler')
license('checkout','control_toolbox')
You just add the 5 toolboxes that need to be checked out -> if the license function cant checkout the license it returns false which you can then use to abort the compilation.
This is what I finally came up with:
I can check before compilation which toolboxes are needed and call license() accordingly. Or I can implement a built-in check into the executable itself. (Called with a special parameter after compilation triggers a self-check for available toolboxes.) In either case I need the names of the required toolboxes.
I tried several ways to generate a list of toolboxes. In short: Running the program in Matlab and then entering license('inuse') is not very reliable. depfun() descends way to much. mydepfun() and fdep() do not descend enough.
I think the problem with mydepfun() and fdep() is that they do not descend into the \toolbox\shared folder. So I took mydepfun() from Tobias Kienzler (link to the original sources) and modified it:
function [list,callers,tboxes_found] = i_scan(f)
func = i_function_name(f);
[list,~,~,~,~,~,callers,~] = depfun(func,'-toponly','-quiet');
toolboxroot = fullfile(matlabroot,'toolbox');
sharedroot = strcat(toolboxroot, filesep, 'shared');
intoolbox = strncmpi(list,toolboxroot,numel(toolboxroot));
inshared = strncmpi(list,sharedroot, numel(sharedroot));
tboxes_found = list(intoolbox & ~inshared);
tboxes_found = regexpi(tboxes_found, '[\\/]toolbox[\\/](.+?)[\\/]', 'tokens');
tboxes_found = cellfun(#(cfun) cfun{1}, tboxes_found);
list = list(~intoolbox | inshared);
callers = callers(~intoolbox | inshared);
for jj = 1:numel(list)
c = callers{jj};
cs = cell(numel(c),1);
for kk = 1:numel(c)
cs{kk} = list{c(kk)};
end;
callers{jj} = cs;
end;
This way i_scan(f) is returning the toolboxes and also descends into \toolbox\shared. The main function of mydepfun() just collects the toolboxes:
function [filelist,callers,toolboxes] = mydepfun(fn,recursive)
.
.
toolboxes = {};
[filelist,callers,tboxes_found] = i_scan(foundfile);
toolboxes = [toolboxes; tboxes_found];
.
.
[newlist,newcallers,tboxes_found] = i_scan(toscan{1});
toolboxes = [toolboxes; tboxes_found];
.
.
toolboxes = unique(toolboxes);
The toolboxes listed are the ones our source code uses. The modified mydepfun() seems to work fine. (Apart from the typical problems caused by elements only resolved during run time like eval(), function handles, callbacks etc.)
And: The dependeny walkers I have seen - like mydepfun() - are using depfun() inside. depfun() is not reliable since it silently ignores all source code not on the path (also its return prob_files is empty in this case). So care has to be taken that the Matlab path is set correctly. (Also any additional pathes are problematic since Matlab may take unexpectedly functions with the same name from other locations.)
After all, I think, this is a good way to make my build process more reliable.
/Zweikeks
I just got another hint from the Mathworks forum. The compiler writes out mccExludedFiles.log. This is listing missing toolboxes.
For example
mccExludedFiles.log:
C:\Program Files\MATLAB\R2010a\toolbox\shared\optimlib\fmincon.m
called by ...c:\temp\whatever\source\code.m
(because the required licenses are not available.)
(Other missing files in the source code do not get listed, though.)
/Zweikeks
I've been trying to use raster calculation in ipython for a tif file I have uploaded, but I'm unable to find the whole code for the function. I keep finding examples such as below, but am unsure how to use this.
gdal_calc.py -A input.tif --outfile=result.tif --calc="A*(A>0)" --NoDataValue=0
I then tried another process by assigning sections, however this still doesn't work (code below)
a = '/iPythonData/cstone/prec_7.tif'
outfile = '/iPythonData/cstone/prec_result.tif'
expr = 'A<125'
gdal_calc.py -A=a --outfile=outfile --calc='expr' --NoDataValue=0
It keeps coming up with can't assign to operator. Can someone please help with the whole code.
Looking at the source code for gdal_calc.py, the file is only about 300 lines. Here is a link to that file.
https://raw.githubusercontent.com/OSGeo/gdal/trunk/gdal/swig/python/scripts/gdal_calc.py
The punchline is that they just create an OptionParser object in main and pass it to the doit() method (Line 63). You could generate the same OptionParser instance based on the same arguments you pass to it via the command-line and call their doit method directly.
That said, a system call is perfectly valid per #thomas-k. This is only if you really want to stay in the Python environment.
I have a separate application printing logs in every 10 seconds. I need to create RRD files from the log files. I need some Perl code to read the log files and create the RRD only without the graphs.
I have also gone through the available Perl module in CPAN, i.e. RRD::Simple and RRD::Simple::Examples, but I still need help.
I'd start with RRD::Simple. There's some example code in the documentation. Since you don't need to create a graph, simply skip that section of the example.
Some of the examples read a single sample of data, call the update function once, and then exit. Those scripts are meant to be run periodically to collect data in real time. The example that's probably more pertinent to your needs is ApacheAccessLogActivity.pl, which reads an Apache log file, parses each line with a regular expression, does a bit of analysis to figure out what it just read, and then calls update, all in a loop. Note that that example uses the standalone functions rather than the object-oriented versions.
If you've already read the documentation for that module and need more information about how to use it, or if you've tried it and found that it has shortcomings that prevent you from using it, then please be more specific about what you need to do.
RRDTool::OO also looks promising.
I'd recommend RRDTool::OO.
Exerpt from the perldoc:
$rrd->create( ... )
Creates a new round robin database (RRD). A RRD consists of one or
more data sources and one or more archives:
$rrd->create(
step => 60,
data_source => { name => "mydatasource",
type => "GAUGE" },
archive => { rows => 5 });