I have just spent hours on this. I'm trying to make a pretty complicated query in PostgreSQL through Slick in Scala, but Slick will NEVER take any of my parameters into account. If I try something as simple as:
def get(location: String) = {
val query = sql"select * from cities_v where name = $location limit 10"
println(query.toString)
}
The output will be:
SQLActionBuilder(Vector(select * from cities_v where name = ? limit 10),<function2>)
If instead, I try the literal insert:
def get(location: String) = {
val query = sql"select * from cities_v where name = #$location limit 10"
println(query.toString)
}
The output will be:
SQLActionBuilder(Vector(select * from cities_v where name = , ville, limit 10),<function2>
Slick will ALWAYS add commas around any literal argument, no matter where it is placed, even if it's the only argument in the query, as in:
sql"#$q"
Now, what I'd like is make more complex queries, with calculations and function calls within Postgres. But I can't get anywhere given that Slick won't let me use any of my variables.
I tried setting all sorts of implicits, some that extend GetResult some that extend StatementParameters, Slick seems to ignore all them and either replaces my arguments with ? or surrounds them with commas, thereby rendering all of my queries invalid. I would like to add that the #$ is not great because it does not provide sanitization. I'd like to stick with Slick because it's asynchronous, but I'm not sure where to go from here.
These are my version numbers, according to build.sbt:
"postgresql" % "postgresql" % "9.1-901-1.jdbc4",
"com.typesafe.slick" % "slick_2.12" % "3.2.3",
What am I doing wrong?
There is nothing wrong with that ? appears in you result statements. It is just parametrized query, and each ? represents placeholder for future value.
Just run you queries against any database and check results.
Related
Motivation
Modelica does store units of measurement (e.g. SI units and Non-SI units) as an attribute with regard to a variable. Here is an example for a Non-SI-unit:
type Time_months = Real( quantity = "Time", unit = "mo", displayUnit = "months" )
Since for models in economics it will be rather akward to give rates in seconds, I would like to write a rather general unit conversion function that will allow to convert units of time. So ideally a function to convert to another time base should work with three inputs and one output:
input Real timeValue "the value of time to be converted";
input String timeBaseA "the time base for timeValue, e.g. \"mo\" ";
input String timeBaseB "the time base to convert to, e.g. \"yr\" ";
output Real convertedTimeValue "the result of the conversion";
Questions
If we assume that a variable for some time value already has a specific unit attribute (e.g. "mo") it would make sense to use that meta information within a model.
Question 1: How can meta information like unit be accessed within a model?
Ideally something like the following would be great:
String timeBaseA := timeValue.unit;
or
String timeBaseA := getUnit( timeValue ) "some function to read unit information";
Question 2: How can meta information like unit be assigned within a function?
In the example we would of course like to return the output value with the correct unit of time. So ideally we would like to have:
output Real convertedTime( quantity = "Time", unit = strTimeBaseB )
Unfortunately, using an input will give rise to an error as the variability is different: The unit attribute should have constant variability but the input variable has parameter variability. (Using a function - which would be nice - also fails for the same reason.)
Regarding Question 1:
I have never used Wolfram SystemModeler, but the Modelica Language Specification 3.4 says in chapter 4.8 (Predefined Types and Classes):
The attributes of the predefined variable types (Real, Integer, Boolean, String) ... cannot be accessed using dot notation, and are not constrained by equations and algorithm sections.
Regarding Question 2:
I think it is only possible to define the unit of a variable on declaration from a literal or from a final parameter - at least this is what I observed in Dymola.
Alternative - use operator records
You could use operator records for your task. This will allow you to store the time in seconds and convert it to what ever needed when the value comes to use.
Operator records allow you to define several function to create them, compare or add them, convert to String, etc.
See the brief example below, where a operator record Time is defined, which can be created with two different constructor functions from seconds or days and can be converted to Strings with day or seconds
operator record Time
Integer s "Second";
encapsulated operator 'constructor'
import Time;
function from_s
input Integer s "Seconds";
output Time t(s=s);
algorithm
end from_s;
function from_d
input Integer d "Days";
output Time t(s=d*24*3600);
algorithm
end from_d;
end 'constructor';
encapsulated operator 'String' "Convert Time to string"
import Time;
function formated
input Time t;
input String format = "s" annotation(choices(choice="s" "seconds", choice="d" "days"));
output String str;
algorithm
if format == "d" then
str :=String(t.s/24/3600);
else
str :=String(t.s);
end if;
end formated;
end 'String';
encapsulated operator function '==' "Compare time records"
import Time;
input Time t1;
input Time t2;
output Boolean result "= t1 == t2";
algorithm
result := t1.s == t2.s;
end '==';
end Time;
Usage:
import Modelica.Utilities.Streams.print
t1 = Time(d=12) // create record using day constructor
t2 = Time(s=3600*24*2) // create record using second constructor
print(String(t1, format="s")) // prints 1036800
print(String(t1, format="d")) // prints 12
print(String(t2, format="s")) // prints 172800
print(String(t2, format="d")) // prints 2
See Modelica Spec 3.4 Chapter 14 "Overloaded Operators" for details.
Note: This was tested with Dymola 2019, not with Wolfram SystemModeler
In Modelica usually every variable is computed based on SI units. Then you have displayUnits to plot them in a different unit (not affecting the actual computation).
I don't know about SystemModeler, but in Dymola the conversion between the unit (of computation) and the displayUnit (only for plotting) is handled by a pre-defined script (displayUnit.mos). It can be extended by the user to contain custom displayUnits. The code for the display units related to time is shown below. I extended it to have week (w) additionally to the predefined ones.
// Syntax:
// defineUnitConversion(<unit>, <derived unit>, <scale>, <opt. offset>);
// Time
defineUnitConversion("s", "ms", 1000);
defineUnitConversion("s", "min", 1/60);
defineUnitConversion("s", "h", 1/3600);
defineUnitConversion("s", "d", 1/86400);
defineUnitConversion("s", "w", 1/604800);
This can then be selected in plots manually or as the default ´displayUnit´ via Modelica.SIunits.Time t(displayUnit = "w") = ...;
The disadvantage is, that this extension has to be done in a file in the install directory. So it has to be changed again after re-installing the tool or when using a different computer.
If there are numerical reasons to not compute solutions in seconds (e.g. because values would get to big), the solution would be the nominal attribute, which enables a scaling of the variables.
BTW: I think months are not a very good unit of time as they can have 28 to 31 days. That's why I chose weeks in my example.
You could use conversion like is done in the MSL, for example the function Modelica.SIunits.Conversions.to_degC which has the signature:
function to_degC
input Temperature Kelvin "Kelvin value";
output NonSIunits.Temperature_degC Celsius "Celsius value";
end to_degC;
This works, but you need one such function for each unit you want to convert between (which is why most calculations are done using SI-units).
I'd like to get values from map like that
user_TopicResponses.put(
"3"+"_"+topicid,
Access.useradVectorMap.getOrElse(
"3"+"_"+topicid,
Access.useradVectorMap.getOrElse("3"+"_"+"0"),
Array(0.0)
)
)
What means if key in map value will be get, of else key is set to "3+"0" and value will also be get.
but it will be reported that:
too many arguments for method getOrElse: (key: String, default: => B1)B
You mixed up parentheses a bit :
Access.useradVectorMap.getOrElse("3"+"_"+"0"),Array(0.0) shoud in fact be
Access.useradVectorMap.getOrElse("3"+"_"+"0",Array(0.0))
It should be ok after that !
First off, I would suggest, at least for debugging purposes, to break your one statement into multiple statements. Your problem stems from a missing/misplaced parentheses. This would be much easier to see if you split your code up.
Secondly, it's good practice to use a variable or function for any repeated code, it makes it far easier to change and maintain (I like to use them for any hard coded value that might change later as well).
In order to only calculate the secondaryValue only if the primaryValue.getOrElse(...) goes to the "else" value, you can use a lazy val, which only evaluates if needed:
val primaryKey = "3_" + topicid
val secondaryKey = "3_0"
val secondaryDefaultValue = Array(0.0)
lazy val secondaryValue = Access.useradVectorMap.getOrElse(secondaryKey, secondaryDefaultValue )
val primaryValue = Access.useradVectorMap.getOrElse(primaryKey, secondaryValue)
user_TopicResponses.put(primaryKey, primaryValue)
This code is far easier to read and, more importantly, far easier to debug
I'm trying to solve two ScalaCheck (+ specs2) problems:
Is there any way to change the number of cases that ScalaCheck generates?
How can I generate strings that contain some Unicode characters?
For example, I'd like to generate about 10 random strings that include both alphanumeric and Unicode characters. This code, however, always generates 100 random strings, and they are strictly alpha character based:
"make a random string" in {
def stringGenerator = Gen.alphaStr.suchThat(_.length < 40)
implicit def randomString: Arbitrary[String] = Arbitrary(stringGenerator)
"the string" ! prop { (s: String) => (s.length > 20 && s.length < 40) ==> { println(s); success; } }.setArbitrary(randomString)
}
Edit
I just realized there's another problem:
Frequently, ScalaCheck gives up without generating 100 test cases
Granted I don't want 100, but apparently my code is trying to generate an overly complex set of rules. The last time it ran, I saw "gave up after 47 tests."
The "gave up after 47 tests" error means that your conditions (which include both the suchThat predicate and the ==> part) are too restrictive. Fortunately it's often not too hard to bake these into your generator, and in your case you can write something like this (which also addresses the issue of picking arbitrary characters, not just alphanumeric ones):
val stringGen: Gen[String] = Gen.chooseNum(21, 40).flatMap { n =>
Gen.buildableOfN[String, Char](n, arbitrary[Char])
}
Here we pick an arbitrary length in the desired range and then pick that number of arbitrary characters and concatenate them into a string.
You could also increase the maxDiscardRatio parameter:
import org.specs2.scalacheck.Parameters
implicit val params: Parameters = Parameters(maxDiscardRatio = 1024)
But that's typically not a good idea—if you're throwing away most of your generated values your tests will take longer, and refactoring your generator is generally a lot cleaner and faster.
You can also decrease the number of test cases by setting the appropriate parameter:
implicit val params: Parameters = Parameters(minTestsOk = 10)
But again, unless you have a good reason to do this, I'd suggest trusting the defaults.
I am following the example here
http://www.postgresql.org/docs/current/interactive/sql-fetch.html
And then looping the following command manually
FETCH FORWARD 5 FROM liahona;
in java. I have the above in an infinite loop and would like to know how I detect I'm at the end of the data set so I can break the loop
You get an empty result set when running FETCH FORWARD and you are at the end of the cursor's total result set. (This is described in slightly different words in the documentation.)
In use JDBC for this kind of SQL - it's for postgres command-line only.
For JDBC, you need something like this:
ResultSet rs = connection.createStatement().executeQuery("select * from mytable");
while(rs.next()) { // next() returns false if there are no more rows
int col1 = rs.getInt(1);
String col2 = rs.getString(2);
}
Is it possible to get the text of an OleDbCommand with all parameters replaced with their values? E.g. in the code below I'm looking for a way to get the query text
SELECT * FROM my_table WHERE c1 = 'hello' and c2 = 'world'
after I finished assigning the parameters.
var query = "SELECT * FROM my_table WHERE c1 = ? and c2 = ?";
var cmd = new OleDbCommand(query, connection);
cmd.Parameters.Add("#p1", OleDbType.WChar).Value = "hello";
cmd.Parameters.Add("#p2", OleDbType.WChar).Value = "world";
No: you have to iterate through the parameters collection yourself, doing a string.Replace() to get the equivalent. It's particularly painful when you have to use the ? syntax rather than the #parametername syntax.
The reason for this is that the full string is never assembled. The parameters and sent to the server and treated as data, and are never included in the string.
All the same, I for one understand your pain. It would have been nice if they included some kind of .ComposeSQL() method you could call for debugging purposes, that perhaps also produces a compiler warning to help avoid use in production.
If you just need to see what query was executed and dont need to work with it programmatically, you can use SQL Profiler.