Cypher: find path using Scala AnormCypher? - scala

AnormCypher doc provides an example how to retriev data using the Stream API:
http://anormcypher.org/
"The first way to access the results of a return query is to use the Stream API.
When you call apply() on any Cypher statement, you will receive a lazy Stream of CypherRow instances, where each row can be seen as a dictionary:
// Create Cypher query
val allCountries = Cypher("start n=node(*) where n.type = 'Country' return n.code as code, n.name as name")
// Transform the resulting Stream[CypherRow] to a List[(String,String)]
val countries = allCountries.apply().map(row =>
row[String]("code") -> row[String]("name")
).toList
I am trying to use the same aproach to get path with the following Cypher query:
MATCH p = (n {id: 'n5'})-[*]-(m) RETURN p;
Yet, when running this code:
Cypher("MATCH p = (n {id: 'n5'})-[*]-(m) RETURN p;")().map {row =>
println(row[Option[org.anormcypher.NeoRelationship]]("p"))
}
I get exception (see below). How to get path info from CypherRow in this case?
Exception in thread "main" java.lang.RuntimeException: TypeDoesNotMatch(Unexpected type while building a relationship)
at org.anormcypher.MayErr$$anonfun$get$1.apply(Utils.scala:21)
at org.anormcypher.MayErr$$anonfun$get$1.apply(Utils.scala:21)
at scala.util.Either.fold(Either.scala:97)
at org.anormcypher.MayErr.get(Utils.scala:21)
at org.anormcypher.CypherRow$class.apply(AnormCypher.scala:303)
at org.anormcypher.CypherResultRow.apply(AnormCypher.scala:309)
at bigdata.test.n4j.Simple$$anonfun$main$1.apply(Simple.scala:31)
at bigdata.test.n4j.Simple$$anonfun$main$1.apply(Simple.scala:29)
at scala.collection.immutable.Stream.map(Stream.scala:376)
at bigdata.test.n4j.Simple$.main(Simple.scala:29)

Paths in Cypher were changed as of 2.0, so you can't work with them easily directly, as they're not collections. There probably should be a new Path type of some sort in AnormCypher, but for now you can use paths along with relationships() or nodes().
For example, you could do this to extract the relationships:
Cypher("MATCH p = (n {id: 'n5'})-[*]-(m) RETURN relationships(p);")().map {row =>
println(row[Seq[NeoRelationship]]("relationships(p)"))
}

Related

SLICK: To insert value in 3 different tables using one slick api call

The question is related to slick:
I have three tables:
1) Users
2)Team2members
3)Team2Owners
In my post request to users, I am passing values of memberOf and managerOf, these values will be inserted in the tables Team2members and Team2Owners respectively and not in Users table. Though other values of the post request will be inserted in 'Users' table.
My Post request looks like below:
{"kind": "via#user",
"userReference":{"userId":"priya16"},
"user":"preferredNameSpecialChar#domain1.com","memberOf":{"teamReference":{"organizationId":"airtel","teamId":"supportteam"}},
"managerOf":{"teamReference":{"organizationId":"airtel","teamId":"supportteam"}},
"firstName":"Special_fn1",
"lastName":"specialChar_ln1",
"preferredName":[{"locale":"employee1","value":"##$%^&*(Z0FH"}],
"description":" preferredNameSpecialChar test "}
I am forming the query which is shown below:
The query seems to work fine when only memberInsert is defined, when I try to define both the values i.e.memberInsert and managerInsert then insertion happens only for second value.
val query = config.api.customerTableDBIO(apiRequest.parameters.organizationId).flatMap { tables =>
val userInsert = tables.Users returning tables.Users += empRow
val memberInsert = inputObject.memberOf.map(m => m.copy(teamReference = m.teamReference.copy(organizationId = apiRequest.parameters.organizationId))).map { r =>
for {
team2MemberRow <- tables.Team2members returning tables.Team2members += Teams2MembersEntity.fromEmtToTeams2Members(r, empRow.id)
team <- tables.Teams.filter(_.id === r.teamReference.teamId.toLowerCase).map(_.friendlyName).result.headOption
} yield (team2MemberRow, team)
}
val managerInsert = inputObject.managerOf.map(m => m.copy(teamReference = m.teamReference.copy(organizationId = apiRequest.parameters.organizationId))).map { r =>
for {
team2OwnerRow <- tables.Team2owners returning tables.Team2owners += Teams2OwnersEntity.fromEmtToTeam2owners(r, empRow.id)
team <- tables.Teams.filter(_.id === r.teamReference.teamId.toLowerCase).map(_.friendlyName).result.headOption
} yield (team2OwnerRow, team)
}
userInsert.flatMap { userRow =>
val user = UserEntity.fromDbEntity(userRow)
if (memberInsert.isDefined) memberInsert.get
.map(r => user.copy(memberOf = Some(Teams2MembersEntity.fromEmtToMemberRef(r._1, r._2.map(TeamEntity.toApiFriendlyName).getOrElse(List.empty)))))
else DBIO.successful(user)
if (managerInsert.isDefined) managerInsert.get
.map(r => user.copy(managerOf = Some(Teams2OwnersEntity.fromEmtToManagerRef(r._1, r._2.map(TeamEntity.toApiFriendlyName).getOrElse(List.empty)))))
else DBIO.successful(user)
}
}
The query seems to work fine when only memberInsert is defined, when I try to define both the values i.e.memberInsert and managerInsert then insertion happens only for second value.
The problem looks to be with the final call to flatMap.
That should return a DBIO[T]. However, your expression generates a DBIO[T] in various branches, but only one value will be returned from flatMap. That would explain why you don't see all the actions being run.
Instead, what you could do is assign each step to a value and sequence them. There are lots of ways you could do that, such as using DBIO.seq or andThen.
Here's a sketch of one approach that might work for you....
val maybeInsertMemeber: Option[DBIO[User]] =
member.map( your code for constructing an action here )
val maybeInsertManager Option[DBIO[User]] =
manager.map( your code for constructing an action here )
DBIO.sequenceOption(maybeInsertMember) andThen
DBIO.sequenceOption(maybeInsertManager) andThen
DBIO.successful(user)
The result of that expression is a DBIO[User] which combines three queries together.

Scala fasterxml jackson. Reading in a Yaml file, but the return data structure is difficult to work with

I am Writing a scala method that reads in a Yaml file and returns a Map of the content of the Yaml file. I can do this successfully, but working with the data structure is very cumbersome as I will demonstrate below.
Note I can and have used jackson in scala to take a yaml file and constitute it into a case class. That works great and is not cumbersome to use. In this problem the yaml is dynamic so we need to put it into a dynamic data structure i.e Map or List of Map
In Java there is no issue in solving the problem. The data structure that is returned back is easy to work with.
Java example:
public Map readMapYml(String fullFileName) {
ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
try {
return mapper.readValue(new File(fullFileName), Map.class);
} catch (Exception e) {
throw new RuntimeException("JavaParser->writeYml:
Unable to write yaml file: " + e.getMessage());
}
}
My equivalent scala code. (I have also tried many variation of the code below)
def readMapYml(fullFileName: String): Map[String,Any] = {
val mapper = new ObjectMapper(new YAMLFactory())
mapper.registerModule(DefaultScalaModule)
try {
mapper.readValue(new File(fullFileName), classOf[Map[String,Any]])
}
catch {
case e: Exception =>
throw new RuntimeException("Parser->readMapYml: Unable to read yaml
file to map. filename: " + fullFileName + " Message: " + e.getMessage)
}
}
So this works and I can parse through the data, but it is really cumbersome.
Example of how cumbersome:
result.get("groups").get.asInstanceOf[List[Map[String,Any]]](0).get("group").get.asInstanceOf[Map[String,Any]].get("colors").get.asInstanceOf[List[Map[String,Any]]](0).get("color").get
Btw interop works just fine I can write this in Java and call it from scala. However, in this case we need to get our scala code working
My question: I would like fasterxml Jackson to return a data structure that is much easier to use similar to the data structure I get back when done through Java. How do I do that?
One of the difficulties with the following approach is that it requires the datatype to be defined every time we extract the data it takes up datatype Any - thereby forcing us to define the data type for values.
mapper.readValue(new File(fullFileName), classOf[Map[String,Any]])
Since the YAML file is expected to be dynamic, it is better to use the much more developed JsonNode data type from Jackson, leveraging following approach:
val yaml = mapper.readValue(new File(fullFileName), classOf[Any])
val jsonString = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(yaml)
val jsonObj = mapper.readTree(jsonString)
The resulting jsonObj is of JsonNode data type, which will have a dynamic schema & supports data conversion/typecasting needs with its in-built methods
The following code does a good job navigating a kv map returned from Jackson.
/**
* Purpose is to parse through a generic kv map of data returned from Jackson.
* #param structure data return from Jackson or a sub-structure of data returned from
Jackson
* #param path A path to the data we want to return. A stack so order is leaf to
branch to branch .... to root
* #return the section requested. The first item added to your stack. In other words
the last item pop.
*/
def getStructure(structure: Any, path: mutable.Stack[String]): Any = {
var retVal: Any = structure
if (path.nonEmpty) {
structure match {
case map: Map[String, Any] =>
retVal = map.get(path.pop())
case some: Some[Any] =>
retVal = some.get
case list: List[Any] =>
retVal = list(path.pop().toInt)
case None =>
throw new IllegalStateException("DataHelper->getStructure: Bad path keyword does not exist in section of path. remaining stack: " + path)
case _ =>
throw new IllegalStateException("DataHelper->getStructure: Structure type is unexpected. Type: " + structure.getClass.getName)
}
if (path.nonEmpty) {
retVal = getStructure(retVal, path)
}
}
retVal match {
case some: Some[Any] =>
retVal = some.get //If the last item is a some get the content of the some.
case _ =>
}
retVal
}
Test code:
test("testMyDataHelper") {
val mapParser = new MapParser
val result = mapParser.readMapYml("test.yaml")
var path = mutable.Stack[String]()
path.push("name","0","groups")
println(DataHelper.getStructure(result, path))//Joe
path.push("name","1","groups")
println(DataHelper.getStructure(result, path))//Bill
path.push("part2","0","items","0","groups")
println(DataHelper.getStructure(result,path))//dash
path.push("part2","2","items","0","groups")
println(DataHelper.getStructure(result,path))//line
//Example of getting a subsection of yaml
path.push("items","0","groups")
val subsection = DataHelper.getStructure(result,path)
//use the subsection
path.push("part1","2")
println(DataHelper.getStructure(subsection,path))//green
path.push("part2","0")
println(DataHelper.getStructure(subsection,path))//dash
}
yaml file
document: "0.0.1"
groups:
- version: "0.0.0"
type: "A"
name: "Joe"
agency: "skjenco"
number: 8
items:
- part1: "red"
part2: "dash"
- part1: "brown"
part2: "underline"
- part1: "green"
part2: "line"
- version: "0.0.1"
type: "B"
name: "Bill"
agency: "billco"
number: 3
items:
- part1: "orange"
part2: "line"
- part1: "pink"
part2: "dash"
- part1: "blue"
part2: "line"

Play Scala Anorm dynamic SQL for UPDATE query

My Google-fu is letting me down, so I'm hoping you can help
I'm building some webservices is the play framework using scala and anorm for database access
One of my calls is to update an existing row in a database - i.e run a query like
UPDATE [Clerks]
SET [firstName] = {firstName}
,[lastName] = {lastName}
,[login] = {login}
,[password] = {password}
WHERE [id] = {id}
My method receives a clerk object BUT all the parameters are optional (except the id of course) as they may only wish to update a single column of the row like so
UPDATE [Clerks]
SET [firstName] = {firstName}
WHERE [id] = {id}
So I want the method to check which clerk params are defined and build the 'SET' part of the update statement accordingly
It seems like there should be a better way than to go through each param of the clerk object, check if it is defined and build the query string - but I've been unable to find anything on the topic so far.
Does anyone have any suggestions how this is best handled
As the commenters mentioned it appears to not be possible - you must build the query string yourself.
I found that examples around this lacking and it took more time to resolve this than it should have (I'm new to scala and the play framework, so this has been a common theme)
in the end this is what I implemented:
override def updateClerk(clerk: Clerk) = {
var setString: String = "[modified] = {modified}"
var params: collection.mutable.Seq[NamedParameter] =
collection.mutable.Seq(
NamedParameter("modified", toParameterValue(System.currentTimeMillis / 1000)),
NamedParameter("id", toParameterValue(clerk.id.get)))
if (clerk.firstName.isDefined) {
setString += ", [firstName] = {firstName}"
params = params :+ NamedParameter("firstName", toParameterValue(clerk.firstName.getOrElse("")))
}
if (clerk.lastName.isDefined) {
setString += ", [lastName] = {lastName}"
params = params :+ NamedParameter("lastName", toParameterValue(clerk.lastName.getOrElse("")))
}
if (clerk.login.isDefined) {
setString += ", [login] = {login}"
params = params :+ NamedParameter("login", toParameterValue(clerk.login.getOrElse("")))
}
if (clerk.password.isDefined) {
setString += ", [password] = {password}"
params = params :+ NamedParameter("password", toParameterValue(clerk.password.getOrElse("")))
}
if (clerk.supervisor.isDefined) {
setString += ", [isSupervisor] = {supervisor}"
params = params :+ NamedParameter("supervisor", toParameterValue(clerk.supervisor.getOrElse(false)))
}
val result = DB.withConnection { implicit c =>
SQL("UPDATE [Clerks] SET " + setString + " WHERE [id] = {id}").on(params:_*).executeUpdate()
}
}
it likely isn't the best way to do this, however I found it quite readable and the parameters are properly handled in the prepared statement.
Hopefully this can benefit someone running into a similar issue
If anyone wants to offer up improvements, they'd be gratefully received
Since roughly 2.6.0 this is possible directly with anorm using their macros, http://playframework.github.io/anorm/#generated-parameter-conversions
Here is my example:
case class UpdateLeagueFormInput(transferLimit: Option[Int], transferWildcard: Option[Boolean], transferOpen: Option[Boolean])
val input = UpdateLeagueFormInput(None, None, Some(true))
val toParams: ToParameterList[UpdateLeagueFormInput] = Macro.toParameters[UpdateLeagueFormInput]
val params = toParams(input)
val dynamicUpdates = params.map(p => {
val snakeName = camelToSnake(p.name)
s"$snakeName = CASE WHEN {${p.name}} IS NULL THEN l.$snakeName ELSE {${p.name}} END"
})
val generatedStmt = s"""UPDATE league l set ${dynamicUpdates.mkString(", ")} where league_id = ${league.leagueId}"""
SQL(generatedStmt).on(params: _*).executeUpdate()
producing:
UPDATE league l set transfer_limit = CASE WHEN null IS NULL THEN l.transfer_limit ELSE null END, transfer_wildcard = CASE WHEN null IS NULL THEN l.transfer_wildcard ELSE null END, transfer_open = CASE WHEN true IS NULL THEN l.transfer_open ELSE true END where league_id = 26;
Notes:
The camelToSnake function is just my own (There is an obvious ColumnNaming.SnakeCase available for parser rows, but I couldn't find something similar for parameter parsing)
My example string interpolates {league.leagueId}, when it could treat this as a parameter as well
Would be nice to avoid the redundant sets for null fields, however I don't think it's possible, and in my opinion clean code/messy auto-generated sql > messy code/clean auto-generated sql

Apache-Spark: method in foreach doesn't work

I read file from HDFS, which contains x1,x2,y1,y2 representing a envelope in JTS.
I would like to use those data to build STRtree in foreach.
val inputData = sc.textFile(inputDataPath).cache()
val strtree = new STRtree
inputData.foreach(line => {val array = line.split(",").map(_.toDouble);val e = new Envelope(array(0),array(1),array(2),array(3)) ;
println("envelope is " + e);
strtree.insert(e,
new Rectangle(array(0),array(1),array(2),array(3)))})
As you can see, I also print the e object.
To my surprise, when I log the size of strtree, it is zero! It seems that insert method make no senses here.
By the way, if I write hard code some test data line by line, the strtree can be built well.
One more thing, those project is packed into jar and submitted in the spark-shell.
So, why does the method in foreach not work ?
You will have to collect() to do this:
inputData.collect().foreach(line => {
... // your code
})
You can do this (for avoiding collecting all data):
val pairs = inputData.map(line => {
val array = line.split(",").map(_.toDouble);
val e = new Envelope(array(0),array(1),array(2),array(3)) ;
println("envelope is " + e);
(e, new Rectangle(array(0),array(1),array(2),array(3)))
}
pairs.collect().foreach(pair => {
strtree.insert(pair._1, pair._2)
}
Use .map() instead of .foreach() and reassign the outcome.
Foreach does not return the outcome of applyied function. It can be used for sending data somewhere, storing to db, printing, and so on.

Entity Framework check against a local list

I have a local list of values that I need to have entity framework check against the database and return them.
If the list was already in the database, the following would work:
var list = /* some ef query */;
var myList = context.Logs.Where(l => list.Any(li => l.LogNumber == li.LogNumber));
But if the list is local, it would throw an error:
var list = new List<Log>();
var myList = context.Logs.Where(l => list.Any(li => l.LogNumber == li.LogNumber));
Exception: Unable to process the type 'Data.Log[]', because it has no known mapping to the value layer.
So how can I match a local list against a database list using EF?
I got a different error than you with the code sample, but I believe it's the same idea. EF doesn't know how to translate List<Log> into a SQL store expression. It works when you're still in a query because it hasn't been serialized yet.
I realize this is less than ideal, but I was able to make this query work by extracting the scalar values of LogNumber and then using that in the query.
var list = new List<Log>();
list.Add(new Log()
{
LogNumber = 1
});
var numbers = list.Select(l => l.LogNumber);
var myList = m.Logs.Where(l => numbers.Contains(l.LogNumber));