I have 1 problem and 1 question when using BSONDocument getAs.
Whenever I try to access the value l in the format below by calling this:
docFound.getAs[Int]("v.1.0.2013.9.9.l")
it returns None. However if I do:
docFound.getAs[BSONDocument]("v")
it returtns a valid BSONDocument for the whole v section. What is wrong in my first call? Does reactivemongo support path traversal?
BSONDocument: {
v: {
1.0: {
2013: {
9: {
9: {
l: BSONInteger(0),
s: BSONInteger(8)
}
}
}
}
}
}
The second question is:
I find a document in DB with the following filter:
BSONDocument(
"_id" -> 0,
"v.1.0.2013.9.9.l" -> 1)
But it seems like instead of extracting just these values "_id" & "l" it extracts the whole document. When I do BSONDocument.pretty(foundDoc) I see the whole document, not just "l" value that I have requested. Please clarify if it is even worth specifying fields I am interested in if it always downloads the whole document.
Thanks.
It seems like according to the sources it is not supported in reactivemongo. So I have created a quick helper:
def getAsByPath[T](path: String, doc: BSONDocument)
(implicit reader: BSONReader[_ <: BSONValue, T]): Option[T] = {
val pathChunks = path.split('.')
var pathIndex = 0
var curDoc: Option[BSONDocument] = Some(doc)
var currentChunk = ""
while(pathIndex != pathChunks.size) {
currentChunk += pathChunks(pathIndex)
// If this is the last chunk it must be a value
// and if previous Doc is valid let's get it
if (pathIndex == pathChunks.size - 1 && curDoc != None)
return curDoc.get.getAs[T](currentChunk)
val tmpDoc = curDoc.get.getAs[BSONDocument](currentChunk)
if (tmpDoc != None) {
currentChunk = ""
curDoc = tmpDoc
} else {
// We need to support this as sometimes doc ID
// contain dots, for example "1.0"
currentChunk += "."
}
pathIndex += 1
}
None
}
However my second question is still valid. If someone knows please let me know.
Related
So I am working on a project where we are making a small compiler program but before I can move on to the other parts I am having troubles with getting the lexical analyzer to output anything after '\BEGIN' afterwards I debugged it and it seems the value is stuck in a loop where the condition is saying the next character is always a newline. Is it because I haven't added the pattern matching yet to the defined tokens?
Here is the code
import java.util
//import com.sun.javafx.fxml.expression.Expression.Parser.Token
/*Lexical analyzer will be responsible for the following:
- finds the lexemes
- Checks each given character determining the tokens
* */
class MyLexicalAnalyzer extends LexicalAnalyzer {
//Array full of the keywords
//val SpecialCharacters = List(']', '#', '*', '+', '\\', '[', '(',')', "![", '=')
val TEXT = "[a-z] | _ | 0-9 | [A-Z]:"
private var sourceLine: String = null
private val lexeme: Array[Char] = new Array[Char](999)
private var nextChar: Char = 0
private var lexLength: Int = 0
private var position: Int = 0
private val lexems: util.List[String] = new util.ArrayList[String]
def start(line: String): Unit = {
initializeLexems()
sourceLine = line
position = 0
getChar()
getNextToken()
}
// A helper method to determine if the current character is a space.
private def isSpace(c: Char) = c == ' '
//Defined and intialized tokens
def initializeLexems(): Any = {
lexems.add("\\BEGIN")
lexems.add("\\END")
lexems.add("\\PARAB")
lexems.add("\\DEF[")
lexems.add("\\USE[")
lexems.add("\\PARAE")
lexems.add("\\TITLE[")
lexems.add("]")
lexems.add("[")
lexems.add("\\")
lexems.add("(")
lexems.add(")")
lexems.add("![")
lexems.add("=")
lexems.add("+")
lexems.add("#")
}
//val pattern = new regex("''").r
def getNextToken() ={
lexLength = 0
// Ignore spaces and add the first character to the token
getNonBlank()
addChar()
getChar()
// Continue gathering characters for the token
while ( {
(nextChar != '\n') && (nextChar != ' ')
}) {
addChar()
getChar()
}
// Convert the gathered character array token into a String
val newToken: String = new String(lexeme)
if (lookup(newToken.substring(0, lexLength)))
MyCompiler.setCurrentToken(newToken.substring(0,lexLength))
}
// A helper method to get the next non-blank character.
private def getNonBlank(): Unit = {
while ( {
isSpace(nextChar)
}) getChar()
}
/*
Method of function that adds the current character to the token
after checking to make sure that length of the token isn't too
long, a lexical error in this case.
*/
def addChar(){
if (lexLength <= 998) {
lexeme({
lexLength += 1; lexLength - 1
}) = nextChar
lexeme(lexLength) = 0
}
else
System.out.println("LEXICAL ERROR - The found lexeme is too long!")
if (!isSpace(nextChar))
while ( {
!isSpace(nextChar)
})
getChar()
lexLength = 0
getNonBlank()
addChar()
}
//Reading from the file its obtaining the tokens
def getChar() {
if (position < sourceLine.length)
nextChar = sourceLine.charAt ( {
position += 1;
position - 1
})
else nextChar = '\n'
def lookup(candidateToken: String): Boolean ={
if (!(lexems.contains(candidateToken))) {
System.out.println("LEXICAL ERROR - '" + candidateToken + "' is not recognized.")
return false
}
return true
}
}
else nextChar = '\n'<- this is where the condition goes after rendering the first character '\BEGIN' then just keeps outputting in the debug console as listed below.
This is what the debug console it outputting after '\BEGIN' is read through
Can anyone please let me know why that is? This happens after I keep stepping into it many times as well.
Here is the driver class that uses the lexical analyzer
import scala.io.Source
object MyCompiler {
//check the arguments
//check file extensions
//initialization
//get first token
//call start state
var currentToken : String = ""
def main(args: Array[String]): Unit = {
val filename = args(0)
//check if an input file provided
if(args.length == 0) {
//usage error
println("USAGE ERROR: Must provide an input file. ")
System.exit(0)
}
if(!checkFileExtension(args(0))) {
println("USAGE ERROR: Extension name is invalid make sure its .gtx ")
System.exit(0)
}
val Scanner = new MyLexicalAnalyzer
val Parser = new MySyntaxAnalyzer
//getCurrentToken(Scanner.getNextToken())
//Parser.gittex()
for (line <- Source.fromFile(filename).getLines()){
Scanner.start(line)
println()
}
//.......
//If it gets here, it is compiled
//post processing
}
//checks the file extension if valid and ends with .gtx
def checkFileExtension(filename : String) : Boolean = filename.endsWith(".gtx")
def getCurrentToken() : String = this.currentToken
def setCurrentToken(t : String ) : Unit = this.currentToken = t
}
The code is operating as it is supposed to. The first line contains only the string \BEGIN so the lexical analyser is treating the end of the first line as an '\n' as shown in this method:
def getChar() {
if (position < sourceLine.length)
nextChar = sourceLine.charAt ( {
position += 1;
position - 1
})
else nextChar = '\n'
However, the comment directly above that method does not describe what the method actually does. This could be a hint as to where your confusion lies. If the comment says it should read from the file, but it is not reading from the file, maybe that's what you've forgotten to implement.
I have created a function in scala equivalant to ORACLE DECODE function. I want to use the function with SPARK dataframes columns. I have tried it but getting multiple issues with Datatype mismatches.
I do not want to create UDF for each program. I want to create something generic and reuse it multiple times.
Function:
def ODECODE(column: Any, Param: Any*) : Any = {
var index = 0
while (index < Param.length) {
var P = Param(index)
var Q = column
if (P.equals(Q))
return Param(index + 1)
else index = index + 1
}
return Param (Param.length - 1)
}
I want to use it some thing like this:
Assuming "Emp" is a dataframe containing data from employee table with columns(first name, Last Name, Grade).
Emp.select(ODECODE("grade", "A", 1, "B", 2, "C", 3, "FAIL")).show()
This is one example. The datatype in the grade column can be String or Integer. So I have taken Datatypes in the decode function (Above) as ANY but with Dataframes it does not perform the Transformation. It gives datatype mismatches.
I want to create individual functions/Methods for some of the unsupported Oracle functions and reuse them where ever required in my transformations. So any suggestion to make this work is appreciated.
I know this is late, but I actually needed this and found your example. I was able to implement it with a few changes. I am no expert though, there may be a better way of doing this.
import util.control.Breaks._;
def ODECODE[T](column: String, params: Seq[T]) : String = {
try {
var index = 0;
breakable {
while (index < params.length) {
var P = params(index);
var Q = column;
if(P.equals(Q)) {
break;
}
index += 1;
}
}
params(index - 1).toString;
}catch {
case ife: Exception =>
ife.printStackTrace();
"0";
}
}
println(ODECODE("TEST", 0, "TEgST", 8, "***", 0))
For exercise I'm trying to implement a LinkedList in Scala.
Main problem is about Null reference.
But first some code:
class Node(xkey: String, xnext: Option[Node], xinfo: Int) {
val key: String = xkey;
var next = xnext.getOrElse(None);
var info: Int = xinfo;
def this(xkey: String, xinfo: Int) {
this(xkey, None, xinfo);
}
def this(xkey: String) {
this(xkey, None, -1);
}
#Override
override def toString: String = key + ":" + info
}
At this point, I'm already concerned about things.
I declare xnext in construct as a Option[Node], because the tail in this linkedList does not have a next.
In my first try, it was just a Node, but had problem with null object because compilator just told me that "null can't cast to Node" (or something like that, I do not remember now) - And so I switch to this Option.
But, is it ok? Because, you know, for me next should be a Node, not a Option, otherwise, I don't know, in the linkedList how to reference to next Node.
Whatever, second class (i.e. my Linked List)
class LinkedNode {
private var first: Option[Node] = None;
private var last: Option[Node] = None;
def addNode(newNode: Node) = {
if (first == null) {
first = Some(newNode);
last = Some(newNode);
first.next = last;
}
else {
last.next = newNode;
newNode.next = null;
last = newNode
}
}
def size(): Long = {
var currentNode : = first;
var size = 0L;
while (currentNode != null) {
size+=1;
currentNode = currentNode.next;
}
size
}
def findNodeByKey(key: String) : Node = {
var currentNode = first;
while(currentNode != null) {
if (currentNode.key.equals(key))
currentNode
else {
currentNode = currentNode.next;
}
}
currentNode;
}
def delNodeByKey(key : String) : Boolean = {
var currentNode = first;
var previousNode = first;
while(currentNode != null) {
if (currentNode.key.equals(key)) {
previousNode = currentNode.next;
return true;
}
previousNode = currentNode;
currentNode = currentNode.next;
}
return false;
}
}
And nothing. I'm already block to my constructor because first and last.
How should I declare them? Node? Or Option[Node]?
Problems are also in Add method.
When I add a node, I want to add a Node object, not an Option[Node].
And I don't get how to achieve things I want with all Option, Some and None classes.
I know I should not be so vague with my request, but any help?
P.S. I've already read this Q/A and it didn't help me
At this point, I'm already concerned about things. I declare xnext in construct as a Option[Node], because the tail in this linkedList does not have a next.
[...]
But, is ok? because, you know, for me next should be a Node, not a Option, otherwise, I don't know, in the linkedList how to reference to next Node.
This is a good solution to replacing null, which you definitely want to do to prevent null-pointer exceptions and the like. An Option[Node] is simply a Node wrapped in a container (or None). You can check whether or not it has a value with isEmpty or get its value with get (which will throw an exception if the Option is empty).
The only difference to null, as you'd use it in Java, is that you need to check if it isEmpty instead of checking for null, and that you need to unwrap (option.get) it explicitly when you're sure that it is not None.
A more paradigmatic (scala-typical) way of retrieving the value from an option is pattern matching:
option match {
case Some(x) => println(x)
case None => println("Whoops, no value :(")
}
Regarding your other questions, they are indeed a little vague.
How should I declere them? Node? or Option[Node]?
Use Option[Node] if the possibility exists that there's no value for the variable (i.e., if you'd set it to null sometimes in Java).
When I add a node, I want to add a Node object, not a Option[Node].
No, you want to add an Option[Node] internally, because you will need to check later on if a node is set or not. In Scala it is preferrable to do this via Option[Node].isEmpty compared to setting things to null. You're already doing this in some parts of your code (e.g., addNode), where you do Some(newNode) (I'd call this "wrapping the node in an Option", but I'm not entirely sure if that's the correct terminology).
And I don't get how to achieve things I want with all Option, Some and None class.
What you're doing in your addNode does seem correct to a degree, but you somehow try to use null again in the else branch. What about:
// We don't need Option[Node] for the parameter, because there
// _must_ be a Node value to be added
def addNode(newNode: Node) = {
if (first.isEmpty) {
first = Some(newNode)
last = Some(newNode)
first.next = last
} else {
newNode.next = None
last.next = Some(newNode)
last = Some(newNode)
}
}
(I didn't run that code, nor did I do an thorough check of your logic)
New to scala and scalafx and having an issue with a tableview in a simple stock quote app [note: no prior javafx experience except in ways it is similar to things Android]
Problem: (see image) erroneous data displayed in change column where there should be none.
To create: Multiple scenarios, shown here is entry of new ticker symbol. It seems unpredictable how many cells are in error. Changing window size (say shorter then taller) generally creates more bad cells. Never happens if no change to windowsize and/or symbol list are made.
Code for the 'ticker' and 'change' columns below, though I suspect it is something in how I implemented the change column to do green/red text coloring.
(Note: data is updated in a single batch periodically which is why the new symbol does not immediately display quote data)
val colTicker = new TableColumn[Quote, String] {
editable = true
text = "Ticker"
prefWidth = 80
alignmentInParent = scalafx.geometry.Pos.Center
cellValueFactory = {
_.value.ticker
}
cellFactory = _ => new TextFieldTableCell[Quote, String](new DefaultStringConverter())
onEditCommit = (evt: CellEditEvent[Quote, String]) => {
val quote: Quote = evt.rowValue
val newTickerVal: String = evt.newValue.toUpperCase()
val oldTickerVal: String = evt.oldValue
// is it a valid ticker and not a dupe or is it blank (erase old ticker)?
if ((isValidTicker(newTickerVal) || newTickerVal.length == 0) && !symbolList.contains(newTickerVal)) {
// lock in the new value on the screen
quote.ticker.set(newTickerVal)
// if the new value is not empty add it to symbol list
if (newTickerVal.length > 0) {
symbolList.append(newTickerVal)
}
// now delete the old value
symbolList -= oldTickerVal
// sort and add another blank line
characters.sortWith(_.ticker.getValueSafe < _.ticker.getValueSafe)
if (oldTickerVal.length < 1) characters += Quote()
// now need to update the data file
putListToFile(dataFile, symbolList.sorted)
} else {
// bad ticker so keep the old one and don't update file
quote.ticker.set(oldTickerVal)
evt.getTableView.getColumns.get(0).setVisible(false)
evt.getTableView.getColumns.get(0).setVisible(true)
println("bad ticker, exiting symbol list: " + symbolList)
}
}
}
val colLast = new TableColumn[Quote, String] {
editable = false
text = "Last"
cellValueFactory = {
_.value.last
}
prefWidth = 80
alignmentInParent = scalafx.geometry.Pos.Center
}
val colChange = new TableColumn[Quote, String] {
editable = false
text = "Change"
cellFactory = {
_ =>
new TableCell[Quote, String] {
item.onChange { (_, _, newChange) =>
if (newChange != null) {
if (newChange.toString.contains("+")) textFill = Color.Green
else textFill = Color.Red
text = newChange
}
}
}
}
cellValueFactory = {
_.value.change
}
prefWidth = 80
alignmentInParent = scalafx.geometry.Pos.Center
}
JavaFX is reusing cells when rendering. This is especially noticeable when dynamically updating TableView content. Your cellFactory has to clear cell content when receiving and empty or null item: text and graphic need to be set to null. It may be sufficient to simply check for newChange == null
cellFactory = { _ =>
new TableCell[Quote, String] {
item.onChange { (_, _, newChange) =>
if (newChange == null) {
text = null
graphic = null
else {
if (newChange.toString.contains("+")) textFill = Color.Green
else textFill = Color.Red
text = newChange
}
}
}
}
It that is nor reliable you will have to implement the cellFactory the JavaFX way by implementing javafx.scene.control.TableCell and overwriting method updateItem that is passing in the empty flag:
cellFactory = {_ =>
new javafx.scene.control.TableCell[Quote, String] {
override def updateItem(item: String, empty: Boolean): Unit = {
super.updateItem(item, empty)
if (item == null || empty) {
text = null
graphic = null
}
else {
...
}
}
}
}
I'm getting frustrated trying to convert a small part of the Golang templating language to Scala.
Below are the key parts of the lex.go source code: https://github.com/golang/go/blob/master/src/text/template/parse/lex.go
The tests are here: https://github.com/golang/go/blob/master/src/text/template/parse/lex_test.go
Basically this "class" takes a string and returns an Array of "itemType". In the template string, the start and end of special tokens is using curly braces {{ and }}.
For for example:
"{{for}}"
returns an array of 4 items:
item{itemLeftDelim, 0, "{{" } // scala case class would be Item(ItemLeftDelim, 0, "")
item{itemIdentifier, 0, "for"}
item{itemRightDelim, 0, "}}"}
item{itemEOF, 0, ""}
The actual call would look like:
l := lex("for", `{{for}}`, "{{", "}}") // you pass in the start and end delimeters {{ and }}
for {
item := l.nextItem()
items = append(items, item)
if item.typ == itemEOF || item.typ == itemError {
break
}
}
return
The key parts of the source code are below:
// itemType identifies the type of lex items.
type itemType int
const (
itemError itemType = iota // error occurred; value is text of error
itemEOF
itemLeftDelim // left action delimiter
// .............. skipped
)
const (
leftDelim = "{{"
rightDelim = "}}"
leftComment = "/*"
rightComment = "*/"
)
// item represents a token or text string returned from the scanner.
type item struct {
typ itemType // The type of this item.
pos Pos // The starting position, in bytes, of this item in the input string.
val string // The value of this item.
}
// stateFn represents the state of the scanner as a function that returns the next state.
type stateFn func(*lexer) stateFn
// lexer holds the state of the scanner.
type lexer struct {
name string // the name of the input; used only for error reports
input string // the string being scanned
leftDelim string // start of action
rightDelim string // end of action
state stateFn // the next lexing function to enter
pos Pos // current position in the input
start Pos // start position of this item
width Pos // width of last rune read from input
lastPos Pos // position of most recent item returned by nextItem
items chan item // channel of scanned items
parenDepth int // nesting depth of ( ) exprs
}
// lex creates a new scanner for the input string.
func lex(name, input, left, right string) *lexer {
if left == "" {
left = leftDelim
}
if right == "" {
right = rightDelim
}
l := &lexer{
name: name,
input: input,
leftDelim: left,
rightDelim: right,
items: make(chan item),
}
go l.run()
return l
}
// run runs the state machine for the lexer.
func (l *lexer) run() {
for l.state = lexText; l.state != nil; {
l.state = l.state(l)
}
}
// nextItem returns the next item from the input.
func (l *lexer) nextItem() item {
item := <-l.items
l.lastPos = item.pos
return item
}
// emit passes an item back to the client.
func (l *lexer) emit(t itemType) {
l.items <- item{t, l.start, l.input[l.start:l.pos]}
l.start = l.pos
}
// lexText scans until an opening action delimiter, "{{".
func lexText(l *lexer) stateFn {
for {
if strings.HasPrefix(l.input[l.pos:], l.leftDelim) {
if l.pos > l.start {
l.emit(itemText)
}
return lexLeftDelim
}
if l.next() == eof {
break
}
}
// Correctly reached EOF.
if l.pos > l.start {
l.emit(itemText)
}
l.emit(itemEOF)
return nil
}
// next returns the next rune in the input.
func (l *lexer) next() rune {
if int(l.pos) >= len(l.input) {
l.width = 0
return eof
}
r, w := utf8.DecodeRuneInString(l.input[l.pos:])
l.width = Pos(w)
l.pos += l.width
return r
}
// lexLeftDelim scans the left delimiter, which is known to be present.
func lexLeftDelim(l *lexer) stateFn {
l.pos += Pos(len(l.leftDelim))
if strings.HasPrefix(l.input[l.pos:], leftComment) {
return lexComment
}
l.emit(itemLeftDelim)
l.parenDepth = 0
return lexInsideAction
}
// lexRightDelim scans the right delimiter, which is known to be present.
func lexRightDelim(l *lexer) stateFn {
l.pos += Pos(len(l.rightDelim))
l.emit(itemRightDelim)
return lexText
}
// there are more stateFn
So I was able to write the item and itemType:
case class Item(typ: ItemType, pos: Int, v: String)
sealed trait ItemType
case object ItemError extends ItemType
case object ItemEOF extends ItemType
case object ItemLeftDelim extends ItemType
...
..
.
The stateFn and Lex definitions:
trait StateFn extends (Lexer => StateFn) {
}
I'm basically really stuck on the main parts here. So things seem to be kicked of like this:
A Lex is created, then "go l.run()" is called.
Run is a loop, which keeps looping until EOF or an error is found.
The loop initializes with lexText, which scans until it finds an {{, and then it sends a message to a channel with all the preceding text of type 'itemText', passing it an 'item'. It then returns the function lexLeftDelim. lexLeftDelim does the same sort of thing, it sends a message 'item' of type itemLeftDelim.
It keeps parsing the string until it reaches EOF basically.
I can't think in scala that well, but I know I can use an Actor here to pass it a message 'Item'.
The part of returning a function, I asked I got some good ideas here: How to model recursive function types?
Even after this, I am really frustrated and I can seem to glue these concepts together.
I'm not looking for someone to implement the entire thing for me, but if someone could write just enough code to parse a simple string like "{{}}" that would be awesome. And if they could explain why they did a certain design that would be great.
I created a case class for Lex:
case class Lex(
name: String,
input: String,
leftDelim: String,
rightDelim: String,
state: StateFn,
var pos: Int = 0,
var start: Int = 0,
var width: Int = 0,
var lastPos: Int = 0,
var parenDepth: Int = 0
) {
def next(): Option[String] = {
if (this.pos >= this.input.length) {
this.width = 0
return None
}
this.width = 1
val nextChar = this.input.drop(this.pos).take(1)
this.pos += 1
Some(nextChar)
}
}
The first stateFn is LexText and so far I have:
object LexText extends StateFn {
def apply(l: Lexer) = {
while {
if (l.input.startsWith(l.leftDelim)) {
if (l.pos > l.start) {
// ????????? emit itemText using an actor?
}
return LexLeftDelim
}
if (l.next() == None) {
break
}
}
if(l.pos > l.start) {
// emit itemText
}
// emit EOF
return None // ?? nil? how can I support an Option[StateFn]
}
}
I need guidance on getting the Actor's setup, along with the main run loop:
func (l *lexer) run() {
for l.state = lexText; l.state != nil; {
l.state = l.state(l)
}
}
This is an interesting problem domain that I tried to tackle using Scala, and so far I am a bit confused hoping some else finds it interesting and can work with what little I have so far and provide some code and critique if I am doing it correctly or not.
I know deep down I shouldn't be mutating, but I'm still on the first few pages of the functional book :)
If you translate the go code literally into Scala, you'll get very unidiomatic piece of code. You'll probably get much more maintainable (and shorter!) Scala version by using parser combinators. There are plenty of resources about them on the internet.
import scala.util.parsing.combinator._
sealed trait ItemType
case object LeftDelim extends ItemType
case object RightDelim extends ItemType
case object Identifier extends ItemType
case class Item(ty: ItemType, token: String)
object ItemParser extends RegexParsers {
def left: Parser[Item] = """\{\{""".r ^^ { _ => Item(LeftDelim, "{{") }
def right: Parser[Item] = """\}\}""".r ^^ { _ => Item(RightDelim, "}}") }
def ident: Parser[Item] = """[a-z]+""".r ^^ { x => Item(Identifier, x) }
def item: Parser[Item] = left | right | ident
def items: Parser[List[Item]] = rep(item)
}
// ItemParser.parse(ItemParser.items, "{{foo}}")
// res5: ItemParser.ParseResult[List[Item]] =
// [1.8] parsed: List(Item(LeftDelim,{{), Item(Identifier,foo), Item(RightDelim,}}))
Adding whitespace skipping, or configurable left and right delimiters is trivial.