In Matlab's Database toolbox, I used the Data Explorer to set up a data source corresponding to my accdb file. Is there a way to do perform Data Explorer functions using code rather than the GUI?
This has been posted on:
Stack Overflow
comp.soft-sys.matlab
The following response from TMW solved my problem:
'...the File DSN example on the database function page? Example is titled: “Connect to Microsoft Access Using a File DSN”. This allows you to build a path that you can use in the database function as the last input argument. In this case, no data source setup is required. Hence, everything can be done by writing code on the command line or a MATLAB script.'
Here is some sample code that uses the above solution to connect to the *.accdb file, pull in data, and use the data:
wbe3accdb_path='C:\cygwin64\home\My.User.Name\Projects\SomeProject\WBEs\wbe3\wbe3.accdb';
wbe3accdb_url= [ 'jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='';DBQ=' wbe3accdb_path ];
conn = database('','','','sun.jdbc.odbc.JdbcOdbcDriver',wbe3accdb_url);
curs=exec(conn,'SELECT * FROM SomeTableInAccdbDatabase')
curs=fetch(curs)
columnnames(curs)
close(conn)
scatter( ...
curs.Data.SomeNumericalFieldName, ...
curs.Data.AnotherNumericalFieldName, ...
's' ...
)
set(gca,'xscale','log','yscale','log')
corr( ...
curs.Data.SomeNumericalFieldName, ...
curs.Data.AnotherNumericalFieldName, ...
'rows', ...
'pairwise' ...
)
Related
Actually I want to get user input from user whoever runs the script. I do not want to hardcode the testdata path in the script. for example when I run a script to test angularjs using protractor and javascript. User should be able to give path of the testdata, so that I can use that variable inside the script.
You can do this by passing in a params.testData value from the command line.
protractor conf.js --params.testData=D:\path\to\testdata.xlsx
Then in your test you will reference it using the global browser.params object. You will also need to use fs to read the file and process the data. Honestly, it would probably be easier if you created a .json file for the test data instead of an .xlsx but it looks there are libraries out there to help you parse an xlsx document already if you have to stick with that. Check this answer for some examples.
This code will not work as is but the basic idea will be something like this:
before(() => {
const testDataPath = browser.params.testData;
fs.readFile(testDataPath, (err, data) => {
if(err) { // fail? };
const testData = data;
// do some other stuff with test data ...
});
}
You are going to need to do some additional processing of the data from the .xlsx file to get it in the correct format but this should hopefully help get you on the right path.
I need to customize other's code,
so I found they used
Mage::getSingleton('core/session')->getMyCustomBlockInfo();
in Order.php file for custom order email
so I can't find this function getMyCustomBlockInfo();
Can anyone tell me where this function reside?
Thanks
those are magic functions get() and set() and you are asking a session variable there that is set as
Mage::getSingleton('core/session')->setMyCustomBlockInfo();
somewhere in your code. If you use terminal you can easily find by making a following grep:
grep '>setMyCustomBlockInfo(' . -rsni
and it will list the files where your variable is set to session.
example :
Mage::getModel('catalog/product'); //or
Mage::getSingleton('catalog/product');
the code must be in '../app/core/Mage/Catalog/Model/Product.php' file
then
Mage::getSingleton('core/session');
the code must be in '../app/core/Mage/Core/Model/Session.php' file
because the class Mage_Core_Model_Session's parent::parent is Varien_Object, then you can do all magic functions and you can ->getData() to see the Data inside.
Mage::getSingleton('core/session')->getData();
on your problem when go call ->getData() you can see data : [my_custom_block_info]
you can set it with call
Mage::getSingleton('core/session')->setMyCustomBlockInfo('what');
Mage::getSingleton('core/session')->getMyCustomBlockInfo();
// will return 'what'
Ok, so I am trying to extract the information from a cache.dat database sent from another business. I am trying to get at the data using the ODBC. I am able to see the globals from the samples namespace when trying to export to Access, but I can't get the data from this new database to show up.
I've tried to tackle this problem two ways. First, I simply shut down Cache, replaced the
existing database in InterSystems\TryCache\mgr\samples and restart cache. Once I restart I can see all the globals in the Management Portal from the new database. If I test the ODBC connection from the Windows ODBC administrator it connects. However, when I try to pull them into an access database using ODBC there are no tables showing up to import.
I've also tried to add the database to my Cache but it gave me the error:
ERROR #5805: ID key not unique for extent 'Config.Databases'
I tried to fool around with the values in there but to no avail. This is my first time messing with anything like this and any, ANY help would be awesome.
If you access the Management Portal do you see any table definitions defined for your namespace. If not, the application was written in CacheObjectScript with no Classes created to provide Object/SQL access. If this is the case then it could be a fair amount of work to create the classes that describe the data(global structures.)
Matt,
Did the business that provided the CACHE.DAT file indicate that you should have ODBC access to the data?
Did they provide some document describing the data/globals? If they provided a document that describes the globals you could create the classes that map the data. Depending on what you want to do this could either be a resource intensive process or not.
If you want to directly access globals you can create a stored procedure that will do so. You should consider the security implications before you do this - it will expose all data in the global to anyone with ODBC access.
Here is an example of a stored procedure that returns the values of up to 9 global subscripts, plus the value at that node. You can modify it pretty easily if you need to.
Query OneGlobal(GlobalName As %String) As %Query(ROWSPEC = "NodeValue:%String,Sub1:%String,Sub2:%String,Sub3:%String,Sub4:%String,Sub5:%String,Sub6:%String,Sub7:%String,Sub8:%String,Sub9:%String") [SqlProc]
{
}
ClassMethod OneGlobalExecute(ByRef qHandle As %Binary, GlobalName As %String) As %Status
{
S qHandle="^"_GlobalName
Quit $$$OK
}
ClassMethod OneGlobalClose(ByRef qHandle As %Binary) As %Status [ PlaceAfter = OneGlobalExecute ]
{
Quit $$$OK
}
ClassMethod OneGlobalFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ PlaceAfter = OneGlobalExecute ]
{
S Q=qHandle
S Q=$Q(#Q) b
I Q="" S Row="",AtEnd=1 Q $$$OK
S Depth=$QL(Q)
S $LI(Row,1)=$G(#Q)
F I=1:1:Depth S $LI(Row,I+1)=$QS(Q,I)
F I=Depth+1:1:9 S $LI(Row,I+1)=""
S AtEnd=0
S qHandle=Q
Quit $$$OK
}
I don't have code for you to get this from access, but for reference, to access this from python you might use (with pyodbc):
import pyodbc
import win32com.client
import urllib2
class CacheOdbcClient:
connectionString="DSN=MYCACHEDSN"
def __init__(self):
pass
def getGlobalAsOverlyLargeList(self):
connection=pyodbc.connect(self.connectionString)
cursor=connection.cursor()
cursor.execute("call MyPackageName.MyClassName_OneGlobal ?","MYGLOBAL")
list=[]
for row in cursor :
list.append((row.NodeValue,row.Sub1,row.Sub2,row.Sub3,row.Sub4,row.Sub5,row.Sub6,row.Sub7,row.Sub8,row.Sub9))
return list
Following on from this question, regarding accessing a PDF on a web page using Matlab which is originally buried behind a Javascript function. I now have a URL which allows me to access the page directly, this works okay using the Matlab webrowser object (the PDF appears on screen), but to save the PDF for subsequent processing I appear to need to use the Matlab urlread/urlwrite functions. However, these functions provide no method for offering authentication credentials.
How do I provide username/password for Matlab's urlread/urlwrite functions?
Matlab's urlread() function has a "params" argument, but these are CGI-style parameters that get encoded in the URL. Authentication is done with lower-level HTTP Request parameters. Urlread doesn't support these, but you can code directly against the Java URL class to use them.
You can also use a Sun's sun.misc.BASE64Encoder class to do the Base 64 encoding programmatically. This is a nonstandard class, not part of the standard Java library, but you know that the JVM shipping with Matlab will have it, so you can get away with coding to it.
Here's a quick hack showing it in action.
function [s,info] = urlread_auth(url, user, password)
%URLREAD_AUTH Like URLREAD, with basic authentication
%
% [s,info] = urlread_auth(url, user, password)
%
% Returns bytes. Convert to char if you're retrieving text.
%
% Examples:
% sampleUrl = 'http://browserspy.dk/password-ok.php';
% [s,info] = urlread_auth(sampleUrl, 'test', 'test');
% txt = char(s)
% Matlab's urlread() doesn't do HTTP Request params, so work directly with Java
jUrl = java.net.URL(url);
conn = jUrl.openConnection();
conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]);
conn.connect();
info.status = conn.getResponseCode();
info.errMsg = char(readstream(conn.getErrorStream()));
s = readstream(conn.getInputStream());
function out = base64encode(str)
% Uses Sun-specific class, but we know that is the JVM Matlab ships with
encoder = sun.misc.BASE64Encoder();
out = char(encoder.encode(java.lang.String(str).getBytes()));
%%
function out = readstream(inStream)
%READSTREAM Read all bytes from stream to uint8
try
import com.mathworks.mlwidgets.io.InterruptibleStreamCopier;
byteStream = java.io.ByteArrayOutputStream();
isc = InterruptibleStreamCopier.getInterruptibleStreamCopier();
isc.copyStream(inStream, byteStream);
inStream.close();
byteStream.close();
out = typecast(byteStream.toByteArray', 'uint8'); %'
catch err
out = []; %HACK: quash
end
As an update to this: another option is the new function webread which you can explicitely give the username and password for basic authenticaition.
options = weboptions('Username','user','Password','your password');
data = webread(url, options);
This can also be used for websave or webwrite. More info on weboptions here
urlwrite_auth is the next step, so here it is...
function [output,status]=urlwrite_auth(url, user, password,location,wanted)
%URLWRITE_AUTH Like URLWRITE, with basic authentication
%
% location is where you want the file saved
% wanted is the name of the file you want
% Returns the output file which is now saved to location.
%
% Examples:
% sampleUrl = 'http://browserspy.dk/password-ok.php';
% [output,status] = urlwrite_auth(sampleUrl, 'user', 'password', location, wanted);
% Matlab's urlread() doesn't do HTTP Request params, so work directly with Java
jUrl = java.net.URL(url);
conn = jUrl.openConnection();
conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]);
conn.connect()
%note this calls the function below
% Specify the full path to the file so that getAbsolutePath will work when the
% current directory is not the startup directory and urlwrite is given a
% relative path.
file = java.io.File(location);
% the path.
try
file = file.getCanonicalFile;
catch
error('MATLAB:urlwrite:InvalidOutputLocation','Could not resolve file "%s".',char(file.getAbsolutePath));
end
% Open the output file.
pathy=strcat(location,'\',wanted);
try
fileOutputStream = java.io.FileOutputStream(pathy);
catch
error('MATLAB:urlwrite:InvalidOutputLocation','Could not open output file "%s".',char(file.getAbsolutePath));
end
% Read the data from the connection.
try
inputStream = conn.getInputStream;
import com.mathworks.mlwidgets.io.InterruptibleStreamCopier;
% This StreamCopier is unsupported and may change at any time.
isc = InterruptibleStreamCopier.getInterruptibleStreamCopier;
isc.copyStream(inputStream,fileOutputStream);
inputStream.close;
fileOutputStream.close;
output = char(file.getAbsolutePath);
catch
fileOutputStream.close;
delete(file);
if catchErrors, return
else error('MATLAB:urlwrite:ConnectionFailed','Error downloading URL. Your network connection may be down or your proxy settings improperly configured.');
end
end
status = 1;
function out = base64encode(str)
% Uses Sun-specific class, but we know that is the JVM Matlab ships with
encoder = sun.misc.BASE64Encoder();
out = char(encoder.encode(java.lang.String(str).getBytes()));
%this is the bit of code that makes it connect!!!!
Note this is a development of the answer by Andrew to download files from a http username and password site.
It turns out the intranet site is using basic authentication, which isn't supported by Matlab out-of-the-box but there is a workaround solution described on the Mathworks site here which works fine. In the first instance I used Firebug to get me the Base64 encoded string I needed for access, but I also did a direct calculation using the tool here. I have now saved my PDF report file to disk - so job done. For my next trick I will be converting it into text...
My understanding is that the get and post methods are distinct from the basic authentication method, but that basic authentication is not often used on the open net.
I don't know matlab, this is just an educated guess.
The function documentation here lists the options as so:
s = urlread('url','method','params')
Depending on what kind of authentication they use this may or may not work, you are going to want to use a post method.
// Params is supposed to be a "cell array of name/value pairs, I don't know matlab...
s = urlread('http://whatever.com','post', {'username' 'ian'; 'password' 'awesomepass'})
You will have to look at the actual request HTML form or view the net tab in firebug to see what the actual name's/values of he user name and password parameters are.
Is there a way to make stack trace to display the whole generated SQL statement when there is an error instead just the first few characters of it?
This is what it currently displays
...\Zend\Db\Adapter\Pdo\Abstract.php(220): Zend_Db_Adapter_Abstract->query('UPDATE "diction...', Array)
..and I would like to see the whole update statement before sent to the db to track what is wrong with it.
Thanks for the help.
SWK
If you want to view the complete sql statement you can use Zend_Debug. For example if your sql statement is in the variable $select and you want to view the complete sql statement you can use the following line of code:
Zend_Debug::Dump($select);
exit;
Or if your code is created withe the Zend_Db_Table class you can use:
$select = new Zend_Db_Select(Zend_Registry::get('db'));
$select->from('string');
Zend_Debug::Dump($select->assemble());
exit;
I think the best way to view the sql statement is by using the profiling function on the database connection. This is combination withe the logging function and the firePHP add-on for Firefox is my favorite setup.
If you use the MVC configuration of Zend Framework this is done white this lines of code:
// setup the database connection
$db = Zend_Db::factory(Zend_Registry::get('config')->database->adapter,Zend_Registry::get('config')->database->params);
// create a new profiler
profiler = new Zend_Db_Profiler_Firebug('All DB Queries');
// enable profiling (this is only recommended in development mode, disable this in production mode)
$profiler->setEnabled(true);
// add the profiler to the database object
$db->setProfiler($profiler);
// setup the default adapter to use for database communication
Zend_Db_Table_Abstract::setDefaultAdapter($db);
// register the database object to access it in other parts of the project
Zend_Registry::set('db',$db);
/**
*
* This part is optional
*
* You can use this logger to log debug information to the firephp add-on for Firefox
* This is handy for debugging but must be disabled in production mode
*
*/
// create logger
$logger = new Zend_Log();
// create firebug writer
$firebug_writer = new Zend_Log_Writer_Firebug();
// add writer to logger
$logger->addWriter($firebug_writer);
// register the logger object to access it in other parts of the project
Zend_Registry::set('log',$logger);
The firebug add-on (requirement for firephp) can be found on this website:
Firebug
The FirePHP add-on can be found on this website:
FirePHP
Ivo Trompert
whilst the profiler is V cool - it doesn't help debug when the system throws an exception..
check out this post on giving a more detailed stack trace inc full SQL
ONLY TO BE USED IN DEV ENVIRONMENTS for obvious reasons
http://www.edmondscommerce.co.uk/blog/zend-framework/zend-framework-more-detailed-stack-trace/