I have a dataframe column table_name which is having below string value:
tradingpartner.parent_supplier,lookup.store,lab_promo_invoice.tl_cc_mbr_prc_wkly_inv,lab_promo_invoice.mpp_club_card_promotion_funding_view,lab_promo_invoice.supplier_sale_apportionment_cc,tradingpartner.supplier,stores.rpm_zone_location_mapping,lookup.calendar
How to extract DB name and table name from the above string and store it as DB name in one column and tablename in another column.
I want the output as below
One possible solution is to define two different UDF to achieve the goal.
Starting from this input DataFrame, called dfInput:
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|table_name |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|tradingpartner.parent_supplier,lookup.store,lab_promo_invoice.tl_cc_mbr_prc_wkly_inv,lab_promo_invoice.mpp_club_card_promotion_funding_view,lab_promo_invoice.supplier_sale_apportionment_cc,tradingpartner.supplier,stores.rpm_zone_location_mapping,lookup.calendar,sauces,plant|
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The first UDF, called dbName, is responsible to take from the input String column all the database names from the String:
def dbNames(k: String): String = {
// this String is the returning value
// containing all the databases from the input string
var dbNames=""
// split the input String by comma
val arrays = k.split(",")
for (str <- arrays){
// if in the input String there is a value like
// database.table take just the database value
if(str.contains(".")) {
val indexOfPoint = str.indexOf(".")
dbNames += str.substring(0, indexOfPoint) + ", "
}
}
// delete last occurence of char ", "
return dbNames.dropRight(2)
}
val dbName = udf[String, String](dbNames)
The second UDF, called tableName, is responsible to take from the input String column all the table names from the String:
def tableNames(k: String): String = {
// this String is the returning value
// containing all the tables from the input string
var tableNames=""
// split the input String by comma
val arrays = k.split(",")
for (str <- arrays){
// if in the input String there is a value like
// database.table take just the table value
// else is intended to be just the table name
if(str.contains(".")) {
val indexOfPoint = str.indexOf(".")
tableNames += str.substring(indexOfPoint+1) + ", "
}
else tableNames += str + ", "
}
// delete last occurence of char ", "
return tableNames.dropRight(2)
}
val tableName = udf[String, String](tableNames)
Then, to obtain the expected output we need to call the UDFs like the following:
val dfOutput = dfInput.withColumn("DBName", dbName(col("table_name")))
.withColumn("Table", tableName(col("table_name")))
dfOutput.show(true)
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|table_name |DBName |Table |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|tradingpartner.parent_supplier,lookup.store,lab_promo_invoice.tl_cc_mbr_prc_wkly_inv,lab_promo_invoice.mpp_club_card_promotion_funding_view,lab_promo_invoice.supplier_sale_apportionment_cc,tradingpartner.supplier,stores.rpm_zone_location_mapping,lookup.calendar,sauces,plant|tradingpartner, lookup, lab_promo_invoice, lab_promo_invoice, lab_promo_invoice, tradingpartner, stores, lookup|parent_supplier, store, tl_cc_mbr_prc_wkly_inv, mpp_club_card_promotion_funding_view, supplier_sale_apportionment_cc, supplier, rpm_zone_location_mapping, calendar, sauces, plant|
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Related
I have a text field and this text field gives me a String and I want to convert this String to a Function. for example >>>
"{
print("a")
}"
this is the String and I want this String to be a function like this>>>
{
print("a")
}
There is a way in my mind.
using textField with text controller to get data after user type
to use that string to make sure that string which user typed will meet two different regex patterns which shown on the pictures.
and then using if..else to invoke print().
RegExp regExp1 = new RegExp("your algorium"); <- to find print
RegExp regExp2 = new RegExp("your algorium"); <- to find "xxxx"
var a = regExp1.hasMatch("{print("a")}")
var b = regExp2.hasMatch("{print("a")}")
if ( a && b)
{
String test = regExp2.stringMatch("{print("a")}").toString()
print(test);
}
i have to queries which I want to merge.
these are two sample columns i want to compare.
i need to find the string from the second list, in the first one. If there is a match, append a list with the matching string to the first list.
Those lists are from two different queries
Those lists are from two different queries
That's fine, you can reference another query using its name. That could be Source1 and Source2 below.
need to find the string from the second list, in the first on
append a list with the matching string to the first list.
I wasn't sure if you meant
forall B in A =>
Text.Combine( {_} & {A}, "_")
I thought you meant to accumulate a list, that's what I did.
let
json1 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wysw1NjIxNTMz0isoKlGK1QGJmJqhiiQnlsRD1SGpAvLj0TSiG1Wc9mH+hI1pWUBxHCowLcNqEUF7oBJA67YZG5sCqQ7jrDSIfCwA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [File = _t]),
Source1 = Table.TransformColumnTypes(json1,{{"File", type text}}),
json2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSkzLyjEyNskyVYrViVYyNjKBMMzMjMD0h/kTtsEYHWBGZi6YgqlTio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Substring = _t]),
Source2 = Table.TransformColumnTypes(json2,{{"Substring", type text}}),
/*
find all matching values in the source text
then merge the matching values with separator
optional params with defaults:
separator: string to join on, "_" by default
comparer: case sensitivity, off by default
*/
SummarizeMatchingText = (source as text, patterns as list, optional options as nullable record) as any =>
let
separator = options[separator]? ?? "_",
comparer = options[comparer]? ?? Comparer.OrdinalIgnoreCase,
select_matching = List.Select(
patterns,
(item) =>
Text.Contains( source, item, comparer)
),
merged = Text.Combine( select_matching, separator)
in
merged,
col_matches = Table.AddColumn(
Source1,
"Matches",
(row) =>
SummarizeMatchingText(
row[File], Source2[Substring],
[ separator = "—" ]
),
Text.Type
),
// summarize multiple steps
Summary = [
Source1 = Source1,
Source2 = Source2,
col_matches = col_matches,
expectEmpty = SummarizeMatchingText(
"sf🐱fj324", Source2[Substring] ),
expectMatch = SummarizeMatchingText(
"cat_im32456.prt", Source2[Substring] )
],
col_matches1 = Summary[col_matches]
in
col_matches1
My Original Code
func getUserByEmpNum(_ id: Int) -> String {
let nameQuery: String = "SELECT fld_str_firstname, fld_str_lastName FROM userView WHERE fld_int_id = \(id);"
var returnStr = ""
do {
let dbQueue = try DatabaseQueue(path: MYCDatabase.pathToDatabase)
try dbQueue.inTransaction(.none) { (db) -> Database.TransactionCompletion in
let returnStrs = try String.fetchAll(db, sql: nameQuery)
// Placing a breakpoint here, returnStrs only has one element?
return .commit
}
} catch {
print (error)
}
return returnStr
}
My Question
In this code if I do a query like select fld_str_firstname from myOwnUserView where fld_int_id = 2; I get one element in my returnStrs array, which is as expected. Then selecting the two fields, as in nameQuery, I still only ever get one string in the returnStrs array.
Why is this, and how do I fit it to get all the selected columns in the response?
String.fetchAll returns an array of Strings extracted from the leftmost selected column, as documented. One string for each fetched row. Not one string for each selected column.
If you want to grab strings from several columns, use Row.fetchAll, which returns an array of database rows. From those rows, you can extract each column you are interested into:
let rows = try Row.fetchAll(db, sql: "SELECT fld_str_firstname, fld_str_lastName FROM ...")
for row in rows {
let firstName: String = row["fld_str_firstname"]
let lastName: String = row["fld_str_lastName"]
}
See this chapter of the documentation for more information about extracting values from database rows.
Since you are reading the name from a single row identified with its id, you may prefer the fetchOne method, which consumes a single database row (see Fetching Methods):
if let row = try Row.fetchOne(db, sql: "SELECT ... WHERE fld_int_id = ?", arguments: [id]) {
let firstName: String = row["fld_str_firstname"]
let lastName: String = row["fld_str_lastName"]
// Use first and last name
} else {
// ID does not exist in the database: do what is appropriate.
}
I have created a function in scala equivalant to ORACLE DECODE function. I want to use the function with SPARK dataframes columns. I have tried it but getting multiple issues with Datatype mismatches.
I do not want to create UDF for each program. I want to create something generic and reuse it multiple times.
Function:
def ODECODE(column: Any, Param: Any*) : Any = {
var index = 0
while (index < Param.length) {
var P = Param(index)
var Q = column
if (P.equals(Q))
return Param(index + 1)
else index = index + 1
}
return Param (Param.length - 1)
}
I want to use it some thing like this:
Assuming "Emp" is a dataframe containing data from employee table with columns(first name, Last Name, Grade).
Emp.select(ODECODE("grade", "A", 1, "B", 2, "C", 3, "FAIL")).show()
This is one example. The datatype in the grade column can be String or Integer. So I have taken Datatypes in the decode function (Above) as ANY but with Dataframes it does not perform the Transformation. It gives datatype mismatches.
I want to create individual functions/Methods for some of the unsupported Oracle functions and reuse them where ever required in my transformations. So any suggestion to make this work is appreciated.
I know this is late, but I actually needed this and found your example. I was able to implement it with a few changes. I am no expert though, there may be a better way of doing this.
import util.control.Breaks._;
def ODECODE[T](column: String, params: Seq[T]) : String = {
try {
var index = 0;
breakable {
while (index < params.length) {
var P = params(index);
var Q = column;
if(P.equals(Q)) {
break;
}
index += 1;
}
}
params(index - 1).toString;
}catch {
case ife: Exception =>
ife.printStackTrace();
"0";
}
}
println(ODECODE("TEST", 0, "TEgST", 8, "***", 0))
I implemented Sqlite in my project by adding #import <sqlite3.h> in my header.h file and libsqlite3.dylib.
How can I pass an array as parameter to my query, here is what I thought:
var arrayId = [1,2] // array with interested Id
var query:NSString = "Select id from Product where id IN \(arrayId)" // I want to select Products with id that are in the array
Edit:
Does it change if arrayId is NSArray ? Because I also need arrayId as NSArray.
then I proceed with open sqlite database, prepare query and so on.
Thank you in advance.
You can easily combine the array into a string with join function.
var arrayId = [1,2] // array with interested Id
var inExpression = ",".join(map(arrayId) { "\($0)"})
// inExpression = "1,2"
var query = "Select id from Product where id IN (\(inExpression))"
Update for Swift3:
var arrayId = [1,2] // array with interested Id
var inExpression = arrayId.flatMap{ String($0) }.joined(separator: ",")
// inExpression = "1,2"
var query = "SELECT id FROM Product WHERE id IN (\(inExpression))"
You need to accomplish two things: convert your array of Ints to Strings and then implode the array into a string, by joining them with commas (as you would want to do with an IN SQL statement).
Here's a rudimentary function that does just that:
func implode(ints: [Int]) -> String {
// Convert to Strings
let strs = ints.map { String($0) }
// Join Strings with commas
return ",".join(strs)
}
And then in use:
"WHERE id IN (\(implode(arrayId)))"
I'd probably use something like:
var arrayId = [1,2] // array with interested Id
var str = ",".join(arrayId.map { return "\($0)" })
var query = "SELECT id FROM Product WHERE id IN [\(str)]"
Using Swift's own string interpolation to create SQL statements can be risky (as with any language). The sqlite3 library provides parameter binding for this purpose:
if (statement.prepare("SELECT name FROM products WHERE id = ?") != .Ok) {
// Deal with error here
}
// Bind the question mark to your value
statement.bindInt(1, value: 8766)
if (statement.step() == .Row) {
let name = statement.getStringAt(1)
// ...do something with your data from the statement
}
// Done.
statement.finalizeStatement()
EDIT:
For the comment below, you need () brackets, not []:
select id, body from test where id in (1,2);
not
select id, body from test where id in [1,2];