How can I access a resource when running an SBT runTask? - scala

I've got an XML file that I need to read from the classpath in order to load some test data for my project with DBUnit when running a custom runTask in SBT.
The XML file is located in /src/main/resources and is copied properly to the /target/scala_2.8.1/classes during the build, but I get a MalformedURLException when trying to access it.
The weird thing is, I can access the file when this data loading functionality was part of my Scala specs unit tests.
Any ideas?

In my case the problem was that I used getClass.getResourceAsStream() in early initialiser. Had to specify the class explicitly with Class.forName() to solve it: Class.forName(<class name>).getResourceAsStream("/data.xml")

If the error is saying that the URL is malformed, it's probably true.
Here's a code I use to grab file from resource during test:
def copyFileFromResource(source: String, dest: File) {
val in = getClass.getResourceAsStream(source)
val reader = new java.io.BufferedReader(new java.io.InputStreamReader(in))
val out = new java.io.PrintWriter(new java.io.FileWriter(dest))
var line: String = null
line = reader.readLine
while (line != null) {
out.println(line)
line = reader.readLine
}
in.close
out.flush
}

Related

Intellij not able to identify root folder when SBT build is used

Below is my project structure. I'm trying to read my properties file pgmproperties.properties in my program below but getting error File Not found. Please tell me where should i place my file. I'm using SBT build tool.
My code
def main(args: Array[String]) {
// This is used to read Properties files and get the values
val is: InputStream = ClassLoader.getSystemResourceAsStream("pgmproperties.properties")
val prop: Properties = new Properties()
if (is != null) {
prop.load(is)
}
else {
throw new FileNotFoundException("Properties file cannot be loaded")
}
val sndrmailid = prop.getProperty("SndrMailId") //Sender Mail id
val psswd = prop.getProperty("SndrMailPsswd") //Sender gmail Mailbox id password
val tomailid = prop.getProperty("RcvrMailId") // Receiver email id.
println( sndrmailid )
}
Error getting
The right place where to put resources is: src/main/resources (as said in the official SBT documentation).
Try to move your file there and it should work :)

fixture 'pylon_config_missing_usageplan' not found while running pytest-bdd

#scenario('../features/config.feature', 'Loading a valid config')
def test_config():
pass
#given("I have provided a valid pylon config",target_fixture="pylon_config")
def pylon_config():
input_data = {
}
return input_data
#when("The configuration is loaded")
def excute_pylon_config(pylon_config):
createFilterPattern(pylon_config)
#then("It should be enriched with the expected FilterPatterns")
def no_error_message(pylon_config):
test_data1= {
}
test_data_2 = {
}
result = pylon_config
#scenario('../features/config.feature', 'Missing usagePlan section')
def test_missing_usageplan():
pass
#given("I have provided a pylon config with a missing key",target_fixture="pylon_config_missing_usageplan")
def pylon_config_missing_usageplan():
input_data = {
'metricFilters': {
'defaults': {
'xyz': []
}
}
}
return input_data
#when("The configuration is loaded")
def excute_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass
#then("I should receive an exception")
def error_message_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
print(pylon_config_missing_usageplan)
I have written multiple test case with specifying target_fixture in both #given scenario.
While running the test case it's throwing an error with
fixture 'pylon_config_missing_usageplan' not found
available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pylon_config, pytestbdd_given_I have provided a pylon config with a missing key, pytestbdd_given_I have provided a valid pylon config, pytestbdd_given_trace, pytestbdd_then_I should receive an exception, pytestbdd_then_It should be enriched with the expected FilterPatterns, pytestbdd_then_trace, pytestbdd_when_The configuration is loaded, pytestbdd_when_trace, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
use 'pytest --fixtures [testpath]' for help on them.
Can anyone help me over here?
The issue is, that the step When The configuration is loaded has two different implementations in code:
#when("The configuration is loaded")
def excute_pylon_config(pylon_config):
createFilterPattern(pylon_config)
#when("The configuration is loaded")
def excute_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass
The function excute_pylon_config_missing_usageplan overrides the step implementation of excute_pylon_config and therefore if you try to load the pylon config in scenario Loading a valid config, pytest-bdd actually tries to execute the function excute_pylon_config_missing_usageplan, which is expecting the fixture pylon_config_missing_usageplan (which is not available in this scenario ...)
Solutions
Have two distinct steps for loading a valid/invalid config, e.g. When The configuration is loaded and When The invalid configuration is loaded (I would recommend this approach, since it is simpler and easier to read than solution 2)
Add a variable to the step for loading the configuration which contains the type of the configuration
Example of variable in configuration loading step:
#when(parsers.parse("The {config_type} configuration is loaded"))
def excute_pylon_config(request, config_type):
if config_type == 'valid':
# Retrieve fixture dynamically by name
pylon_config = request.getfixturevalue('pylon_config')
createFilterPattern(pylon_config)
else:
# Retrieve fixture dynamically by name
pylon_config_missing_usageplan = request.getfixturevalue('pylon_config_missing_usageplan')
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass

Unable to use WURFL with Scala

When I run WURFL demo app for scala:
object Demo {
def main(args: Array[String]) {
// Create WURFL passing a GeneralWURFLEngine object with a wurfl xml
val wurflWrapper = new Wurfl(new GeneralWURFLEngine("classpath:/resources/wurfl.zip"))
// Set cache provider
wurflWrapper.setCacheProvider(new LRUMapCacheProvider)
// Set Performance/Accuracy Mode
wurflWrapper.setTargetAccuracy
// Set Capability Filter
wurflWrapper.setFilter(
"can_assign_phone_number",
"marketing_name",
"brand_name",
"model_name",
"is_smarttv",
"is_wireless_device",
"device_os",
"device_os_version",
"is_tablet",
"ux_full_desktop",
"pointing_method",
"preferred_markup",
"resolution_height",
"resolution_width",
"xhtml_support_level")
// User-Agent here
var userAgent = ""
// Defining headers
var headers = Map("Accept-Datetime"->"Thu, 31 May 2007 20:35:00 GMT")
headers += ("Content-Type"-> "application/x-www-form-urlencoded")
var device = wurflWrapper.deviceForHeaders(userAgent, headers)
val matchType = device.matchType
if (matchType == MatchType.conclusive)
{
println("Match Type is conclusive")
}
val wireless = device.capability("is_wireless_device")
println("Is wireless: " + wireless)
}
}
I get this exception:
[main] ERROR net.sourceforge.wurfl.core.GeneralWURFLEngine - cannot initialize: java.lang.NullPointerException: in is null
java.lang.NullPointerException: in is null
at java.util.zip.ZipInputStream.<init>(ZipInputStream.java:101)
at java.util.zip.ZipInputStream.<init>(ZipInputStream.java:80)
at net.sourceforge.wurfl.core.resource.FileLoader.fromZipFile(FileLoader.java:248)
at net.sourceforge.wurfl.core.resource.FileLoader.openInputStream(FileLoader.java:230)
at net.sourceforge.wurfl.core.resource.FileLoader.getStream(FileLoader.java:288)
at net.sourceforge.wurfl.core.resource.XMLResource.getData(XMLResource.java:163)
at net.sourceforge.wurfl.core.resource.DefaultWURFLModel.init(DefaultWURFLModel.java:115)
at net.sourceforge.wurfl.core.resource.DefaultWURFLModel.<init>(DefaultWURFLModel.java:107)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.init(GeneralWURFLEngine.java:340)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.initIfNeeded(GeneralWURFLEngine.java:319)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.getDeviceForRequest(GeneralWURFLEngine.java:451)
at com.scientiamobile.wurfl.Wurfl.deviceForHeaders(Wurfl.scala:77)
at com.Demo$.main(Demo.scala:49)
at com.Demo.main(Demo.scala)
Exception in thread "main" net.sourceforge.wurfl.core.exc.WURFLRuntimeException: WURFL unexpected exception
at net.sourceforge.wurfl.core.GeneralWURFLEngine.initIfNeeded(GeneralWURFLEngine.java:322)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.getDeviceForRequest(GeneralWURFLEngine.java:451)
at com.scientiamobile.wurfl.Wurfl.deviceForHeaders(Wurfl.scala:77)
at com.Demo$.main(Demo.scala:49)
at com.Demo.main(Demo.scala)
Caused by: java.lang.NullPointerException: in is null
at java.util.zip.ZipInputStream.<init>(ZipInputStream.java:101)
at java.util.zip.ZipInputStream.<init>(ZipInputStream.java:80)
at net.sourceforge.wurfl.core.resource.FileLoader.fromZipFile(FileLoader.java:248)
at net.sourceforge.wurfl.core.resource.FileLoader.openInputStream(FileLoader.java:230)
at net.sourceforge.wurfl.core.resource.FileLoader.getStream(FileLoader.java:288)
at net.sourceforge.wurfl.core.resource.XMLResource.getData(XMLResource.java:163)
at net.sourceforge.wurfl.core.resource.DefaultWURFLModel.init(DefaultWURFLModel.java:115)
at net.sourceforge.wurfl.core.resource.DefaultWURFLModel.<init>(DefaultWURFLModel.java:107)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.init(GeneralWURFLEngine.java:340)
at net.sourceforge.wurfl.core.GeneralWURFLEngine.initIfNeeded(GeneralWURFLEngine.java:319)
... 4 more
The "wurfl.zip" is well located under "resources".
I also tried adding it to main Scala classes path, but still not luck.
From a code perspective
val wurflWrapper = new Wurfl(new GeneralWURFLEngine("classpath:/resources/wurfl.zip"))
is a proper way to initialize your WURFL engine.
You may want to provide information about how you're running the demo, if you are running it inside an IDE (IDEA, Eclipse or Netbeans), or using command line, or other ways. It can also be useful to tell whether you're using Maven or not.
In case you are running it using command line, please provide a sample of how you launch the Scala app and how you set the classpath.
Assuming a scenario where you are compiling with maven and executing the project directly into the target dir using -cp classes, execution will result in your classpath error because resource files are not included in the classes directory.
Make sure that wurfl-scala-example-.jar is included your classpath.
If you are using the Demo project inside IntelliJ IDEA, please make sure that the resource directory is marked as "resource", otherwise IDEA run tool will not include the wurfl.zip file as a resource.
Hope this helps.

Run Scales XML's pullXml in weblogic server

I used pullXml due to the file size is quite large, > 100M, I wrote a sample program as follows:
object TestXml {
val mc = new java.math.MathContext(1024)
val zero = BigDecimal(0, mc)
def calculate(infile: String, encoding: String): BigDecimal = {
val inStream = new FileInputStream(infile)
val pull = pullXml(new InputSource(new InputStreamReader(inStream, encoding)))
val ns = Namespace("urn:abcaus.onair.sintecmedia.com")
val qnames = List(ns("ePGResp"), "EPGResponse"l, "Event"l)
def eventStream = iterate(qnames, pull).toStream
var count = zero
eventStream foreach { event => count += eventId(event) }
inStream.close
count
}
def eventId(event: XmlPath): Long =
text(event.\*("EventID")).toLong
def main(args: Array[String]): Unit = args.toList match {
case infile :: encoding :: Nil => println(calculate(infile, encoding))
case _ => println("usage scala -cp classpath au.net.abc.epg.TestLoadXml infile encoding")
}
}
I run the program on a command line as follows:
$JAVA_HOME/bin/java -cp JarContainsSampleProgram.jar:scala-library-2.10.2.jar:scala-reflect-2.10.2.jar:scalalogging-slf4j_2.10-1.0.1.jar:scalaz-core_2.10-7.0.0.jar:scalaz-effect_2.10-7.0.0.jar:scalaz-iterv_2.10-7.0.0.jar:scales-xml_2.10-0.6.0-M1.jar:slf4j-api-1.6.4.jar TestLoadXml /home/wonga4d/EPG/Huge.xml utf-16
It runs successfully returning a value, say, 879452677392.
However, when I deploy it as a oracle service bus Java callout (which is ok because Scala is JVM lang) to be used by an OSB proxy, still use the same input file and encoding, I got the following error
Callout to java method "public static scala.math.BigDecimal au.net.abc.epg.TestLoadXml.calculate(java.lang.String,java.lang.String)" resulted in exception: Got an event (Text()) that should not be in the prolog java.lang.RuntimeException: Got an event (Text()) that should not be in the prolog
at scala.sys.package$.error(package.scala:27)
at scala.Predef$.error(Predef.scala:142)
at scales.utils.package$.error(package.scala:19)
at scales.xml.parser.pull.PullUtils$$anonfun$getMisc$1.apply(PullIterator.scala:144)
at scales.xml.parser.pull.PullUtils$$anonfun$getMisc$1.apply(PullIterator.scala:141)
at scala.util.Either.fold(Either.scala:97)
at scales.xml.parser.pull.PullUtils$.getMisc(PullIterator.scala:141)
at scales.xml.parser.pull.XmlPull$class.start(PullIterator.scala:89)
at scales.xml.parser.pull.XmlPulls$$anon$1.start(XmlPull.scala:134)
at scales.xml.parser.pull.XmlPulls$$anon$1.<init>(XmlPull.scala:156)
at scales.xml.parser.pull.XmlPulls$class.pullXml(XmlPull.scala:134)
at scales.xml.package$.pullXml(package.scala:7)
at TestXml$.calculate(TestLoadXml.scala:23)
at TestXml.calculate(TestLoadXml.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at stages.transform.runtime.JavaCalloutRuntimeStep$1.run(JavaCalloutRuntimeStep.java:173)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.security.Security.runAs(Security.java:61)
at stages.transform.runtime.JavaCalloutRuntimeStep.processMessage(JavaCalloutRuntimeStep.java:195)
at com.bea.wli.sb.pipeline.debug.DebuggerRuntimeStep.processMessage(DebuggerRuntimeStep.java:74)
at com.bea.wli.sb.stages.StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:346)
at com.bea.wli.sb.pipeline.PipelineStage.processMessage(PipelineStage.java:84)
It fails at pullXml. But it always succeeds when running on a commandline which I showed before. If I don't use pullXml but loadXml, it will always succeed even when running in a weblogic server. But loadXml will get a problem if loading a huge xml file. Both pullXml & loadXml methods are located in the same jar, scales-xml_2.10-0.6.0-M1.jar.
Just wonder if anyone ever used scales xml in weblogic server. Sounds like I got to give up using scales xml if weblogic server is the execution environment.
Thanks
that your code runs outside of weblogic shows its a javax.xml implementation issue. Its possible that this only needs you to supply an alternate implementation (e.g. aalto-xml) with a "child first" or app first classloader setting.
If you could let me know (answering here is cool) if that sorts things out that would be great, I'll add it to the docs.
Outside of this issue I hope Scales is working out for you :)
Cheers,
Chris

FOP/ikvm: error "Provider com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl not found"

I have produced a fop.dll from fop-1.0 with ikvm:
ikvmc -target:library -reference:IKVM.OpenJDK.Core.dll -recurse:{myPathToJars}\*.jar -version:1.0 -out:{myPathToJars}\fop.dll
If I use my fop.dll in a Windows Application, everything works perfect.
If I use it in a Class Library, I get the following error:
"Provider com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl not found" at javax.xml.transform.TransformerFactory.newInstance()
The code line is: TransformerFactory factory = TransformerFactory.newInstance();
Here is the code of method:
public static void xmlToPDF(String xmlPath, String xslPath, SortedList arguments, String destPdfPath)
{
java.io.File xmlfile = new java.io.File(xmlPath);
java.io.File pdffile = new java.io.File(destPdfPath);
try
{
// configure fopFactory as desired
FopFactory fopFactory = FopFactory.newInstance();
FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
// configure foUserAgent as desired
// Setup output
OutputStream outputStream = new java.io.FileOutputStream(pdffile);
outputStream = new java.io.BufferedOutputStream(outputStream);
try
{
// Construct fop with desired output format
Fop fop = fopFactory.newFop("application/pdf" /*MimeConstants.MIME_PDF*/, foUserAgent, outputStream);
// Setup XSLT
TransformerFactory factory = TransformerFactory.newInstance();
java.io.File xsltfile = new java.io.File(xslPath);
Transformer transformer = factory.newTransformer(new StreamSource(xsltfile.getAbsoluteFile()));
// Set the value of a in the stylesheet
if (arguments != null)
{
IList keys = arguments.GetKeyList();
foreach (var key in keys)
{
Object value = arguments[key];
transformer.setParameter(key.ToString(), value);
}
}
// Setup input for XSLT transformation
Source src = new StreamSource(xmlfile);
// Resulting SAX events (the generated FO) must be piped through to FOP
Result res = new SAXResult(fop.getDefaultHandler());
// Start XSLT transformation and FOP processing
transformer.transform(src, res);
}
catch (Exception e1)
{
System.Console.WriteLine(e1.Message);
}
finally
{
outputStream.close();
}
}
catch (Exception ex)
{
System.Console.WriteLine(ex.Message);
}
}
I used ikvm-0.46.0.1 to make my fop.dll (based on fop 1.0). I included the following jars:
avalon-framework-4.2.0.jar
batik-all-1.7.jar
commons-io-1.3.1.jar
commons-logging-1.0.4.jar
fop.jar
serializer-2.7.0.jar
xalan-2.7.0.jar
xercesImpl-2.7.1.jar
xml-apis-1.3.04.jar
xml-apis-ext-1.3.04.jar
xmlgraphics-commons-1.4.jar
Any idea why this error occurs? Why is the behaviour different between Windows Application and Class Library?
Addition 10/19/11:
I managed to get working the following:
MyMainPrg (a Windows Forms Application)
MyFopWrapper (a Class Library that calls fop.dll)
But for my case this is not the solution, because in my target project, I have the following structure:
MainCmdLinePrg (a Console Application; calls DLL_1)
DLL_1 (calls DLLsharedFop) {there are several DLLs that can call DLLsharedFop}
DLLsharedFop (calls directly fop.dll; or - I don't care - might call MyFopWrapper)
Unfortunately this construct results in the error.
You can shorten to a pair (ACmdLinePrg,MyFopWrapper): already this does not work! But (MyMainPrg,MyFopWrapper) does...
Here is how I got that error and how I resolved:
My solultion looks like this:
ClientApp (references)--> ClassLibrary1
My ClassLibrary1 public functions are using, but not exposing any IKVM related objects, therefore the caller (ClientApp) did not have to add IKVM references. All is good in compile time.
However in runtime, the situation is different. I got the same exception and realized that ClientApp also needed to reference the correct IKVM dll (IKVM.OpenJDK.XML.Transform.dll) that contains "com.sun.org.apache.xalan.#internal.xsltc.trax" namespace.
I resolved a similar problem by adding the following before the problematic line:
var s = new com.sun.org.apache.xerces.#internal.jaxp.SAXParserFactoryImpl();
var t = new com.sun.org.apache.xalan.#internal.xsltc.trax.TransformerFactoryImpl();
As described here
Do you have the dll with the missing class in your working directory?
If you have the dll then it is a classloader problem. Look in the IKVM wiki. Often the BootClassPathAssemby help.
I was using NuGet Packages of FOP.dll v1.1.0 and IKVM pacakges of v7.1.45 in C#.NET app. I got this issue on Windows 2016 x64 server with error messages like:
------------------------------ Fop.cs (111): Provider com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl
not found - at javax.xml.transform.TransformerFactory.newInstance()
Fop.cs (125): Provider
com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl not found
- at javax.xml.parsers.SAXParserFactory.newInstance()\r\n at org.apache.avalon.framework.configuration.DefaultConfigurationBuilder..ctor(Boolean
enableNamespaces)\r\n at
org.apache.avalon.framework.configuration.DefaultConfigurationBuilder..ctor()\r\n
I resolved the problem by adding those two lines at begins of procedure
com.sun.org.apache.xerces.#internal.jaxp.SAXParserFactoryImpl s = new com.sun.org.apache.xerces.#internal.jaxp.SAXParserFactoryImpl();
com.sun.org.apache.xalan.#internal.xsltc.trax.TransformerFactoryImpl t = new com.sun.org.apache.xalan.#internal.xsltc.trax.TransformerFactoryImpl();
helpful link:
https://github.com/KevM/tikaondotnet/issues/21