I have a table with String columns "key1", "col1", "col2", "col3". I have a snippet below to test the bulk insert. When I run it, I don't get any error, yet I don't see any changes to the "test" table.
Did I miss something?
Object[][] data = new Object[4][];
ArrayList<String>[] rec = new ArrayList[4];
rec[0] = new ArrayList<String>();
rec[1] = new ArrayList<String>();
rec[2] = new ArrayList<String>();
rec[3] = new ArrayList<String>();
for (Integer i = 0; i < 10; i++) {
rec[0].add(i.toString() + i.toString() + i.toString());
rec[1].add(i.toString() + i.toString() + i.toString());
rec[2].add(i.toString() + i.toString() + i.toString());
rec[3].add(i.toString() + i.toString() + i.toString());
}
for (int i = 0; i < 4; i++) {
data[i] = rec[i].toArray(new Object[rec[i].size()]);
}
c.Dict dict = new c.Dict(Arrays.asList("key1", "col1", "col2", "col3").toArray(new String[4]), data);
c.Flip flip = new c.Flip(dict);
Object[] updStatement = new Object[] { ".u.upd", "test", flip };
conn.ks(updStatement);
Adding onto what you said, you want to be using the ks() method with 2 arguments.
From the javadoc given in the c.java class:
Use this to
* invoke a function in kdb+ which takes 2 arguments and does not return a value. e.g. to invoke f[x;y] use ks("f",x,y);
* to invoke a lambda, use ks("{x+y}",x,y);
The function .u.upd takes in 2 arguments and, by default, has the same signature as insert:
First argument is a symbol for the table name, which has the type String in Java
Second argument is the records which has the type Object[] in Java
The length of the Object[] in the second argument should be equal to the number of columns. Each Object in the Object[] should itself be an array that has a length equal to the number of records. The order of the inner arrays should be the same order as the columns, and the values of each inner array should have the matching type to the column type in kdb and have the same order as the records.
Your Object[] should look like:
new Object[]{
new Object[]{row1col1, row2col1, /*..., */ rowNcol1},
new Object[]{row1col2, row2col2, /*..., */ rowNcol2},
/* column 3 to column N-1 values */
new Object[]{row1colN, row2colN, /*..., */ rowNcolN}
}
And your ks() method call should look like:
conn.ks(".u.upd", "test", new Object[]{ /*.... */});
Related
Need to sort/order a list of data based on an undetermined number of columns (1 or more).
What i'm trying to do is loop through the desired columns and add an OrderBy or ThenBy based on their number to the query'd list, but i'm unsuccessful...
Done this, but it doesn't compile:
var query = GetAllItems(); //returns a IQueriable list of items
//for each selected column
for (int i = 0; i < param.Columns.Length; i++)
{
if (i == 0)
{
query = query.OrderBy(x => x.GetType().GetProperty(param.Columns[i].Name));
}
else
{
//ERROR: IQueriable does not contain a definition for "ThenBy" and no extension method "ThenBy"...
query = query.ThenBy(x => x.GetType().GetProperty(param.Columns[i].Data));
}
}
How can i resolve this issue? Or any alternative to accomplish this requirement?
SOLUTION: #Dave-Kidder's solution is well thought and resolves the compile errors i had. Just one problem, OrderBy only executes (actually sorts the results) after a ToList() cast. This is an issue because i can't convert a ToList back to an IOrderedQueryable.
So, after some research i came across a solution that resolve all my issues.
Microsoft assembly for the .Net 4.0 Dynamic language functionality: https://github.com/kahanu/System.Linq.Dynamic
using System.Linq.Dynamic; //need to install this package
Updated Code:
var query = GetAllItems(); //returns a IQueriable list of items
List<string> orderByColumnList = new List<string>(); //list of columns to sort
for (int i = 0; i < param.Columns.Length; i++)
{
string column = param.Columns[i].Name;
string direction = param.Columns[i].Dir;
//ex.: "columnA ASC"
string orderByColumn = column + " " + direction;
//add column to list
orderByColumnList.Add(orderBy);
}
//convert list to comma delimited string
string orderBy = String.Join(",", orderByColumnList.ToArray());
//sort by all columns, yay! :-D
query.OrderBy(orderBy).ToList();
The problem is that ThenBy is not defined on IQueryable, but on the IOrderedQueryable interface (which is what IQueryable.OrderBy returns). So you need to define a new variable for the IOrderedQueryable in order to do subsequent ThenBy calls. I changed the original code a bit to use System.Data.DataTable (to get a similar structure to your "param" object). The code also assumes that there is at least one column in the DataTable.
// using System.Data.DataTable to provide similar object structure as OP
DataTable param = new DataTable();
IQueryable<DataTable> query = new List<DataTable>().AsQueryable();
// OrderBy returns IOrderedQueryable<TSource>, which is the interface that defines
// "ThenBy" so we need to assign it to a different variable if we wish to make subsequent
// calls to ThenBy
var orderedQuery = query.OrderBy(x => x.GetType().GetProperty(param.Columns[0].ColumnName));
//for each other selected column
for (int i = 1; i < param.Columns.Count; i++)
{
orderedQuery = orderedQuery.ThenBy(x => x.GetType().GetProperty(param.Columns[i].ColumnName));
}
you should write ThenBy after OrderBy like this:
query = query
.OrderBy(t=> // your condition)
.ThenBy(t=> // next condition);
I am trying to insert multiple records to MongoDB at once , so for this i created a javaBean for each record to be inserted and added them to ArrayList .
And finally from the ArrayList , i am trying to perform a insert operation as shown below
public void insert(ArrayList<QuoteReportBean> quotelist) {
BasicDBList totalrecords = new BasicDBList();
StringBuffer sb = new StringBuffer();
int valuecount=0;
for (QuoteReportBean reportbean: quotelist) {
valuecount++;
BasicDBObject dbrecord = new BasicDBObject();
dbrecord.append("cust_id", reportbean.getCustomerId());
dbrecord.append("unique_symbol", reportbean.getUniqueSymbol());
sb.append(reportbean.getUniqueSymbol()+",");
dbrecord.append("exch", reportbean.getExchange());
dbrecord.append("access_time", reportbean.getDate());
totalrecords.add(dbrecord);
}
WriteResult result = coll.insert(totalrecords,WriteConcern.NORMAL);
}
But i am the follwoing error
Exception in thread "taskExecutor-1" java.lang.IllegalArgumentException: BasicBSONList can only work with numeric keys, not: [_id]
at org.bson.types.BasicBSONList._getInt(BasicBSONList.java:159)
at org.bson.types.BasicBSONList._getInt(BasicBSONList.java:150)
at org.bson.types.BasicBSONList.get(BasicBSONList.java:104)
at com.mongodb.DBCollection.apply(DBCollection.java:501)
at com.mongodb.DBCollection.apply(DBCollection.java:491)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:195)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:180)
at com.mongodb.DBCollection.insert(DBCollection.java:58)
Could anybody please help me as how to resolve this ??
BasicDBList can't be used to do inserts of multiple documents, it's only used for arrays inside a single document. To do a bulk insert, you need to pass an array of DBObjects into the insert method instead.
I changed your code to do this, and it worked without error:
StringBuffer sb = new StringBuffer();
int valuecount = 0;
final QuoteReportBean[] quotelist = {new QuoteReportBean()};
DBObject[] totalrecords = new BasicDBObject[quotelist.length];
for (int i = 0; i < quotelist.length; i++) {
QuoteReportBean reportbean = quotelist[i];
valuecount++;
BasicDBObject dbrecord = new BasicDBObject();
dbrecord.append("cust_id", reportbean.getCustomerId());
dbrecord.append("unique_symbol", reportbean.getUniqueSymbol());
sb.append(reportbean.getUniqueSymbol() + ",");
dbrecord.append("exch", reportbean.getExchange());
dbrecord.append("access_time", reportbean.getDate());
totalrecords[i] = dbrecord;
}
WriteResult result = coll.insert(totalrecords, WriteConcern.NORMAL);
I just want to merge two ArrayLists and have the contents in one ArrayList. Both lists contain the object which is an instance of the same class.
The object reference themselves are different though. However I am getting this unexpected size for the combined arraylist. I use JAVA 1.4
ArrayList a1 = new ArrayList();
ArrayList b1 = new ArrayList();
ClassA cls1A = new ClassA();
ClassA cls1B = new ClassA();
a1.add(cls1A);
b1.add(cls1B);
a1.size() = 100;
b1.size() = 50;
//merge the two arraylist contents into one
//1st method and its result
a1.addAll(b1);
//Expected Result
a1.size = 150
//but
//Obtained result
a1.size = 6789
//2nd method and its result
Collections.copy(a1, b1)
//Expected result
a1.size() = 150
//but
//Obtained result
a1.size = 6789
How can I have an ArrayList which displays the combined size??
I came up with the following solution. First get the size of both the arraylists, then increase the capacity(by using ensureCapacity method of ArrayList class) of the arraylist to which the two needs to be merged. Then add the objects of the 2nd arraylist from the last index of the first arraylist.
a1.add(cls1A);
b1.add(cls1B);
int p = a1.size();
int w = b1.size();
int j = (p + w);
a1.ensureCapacity(j);
for(int r = 0; r <w; r++)
{
ClassA cls1B = new ClassA();
Object obj = b1.get(r);
cls1B = (ClassA)obj;
a1.add(p, cls1B);
p++;
}
The System.Author Windows property is a multiple value string. Windows Search returns this value as an array of strings in a DataColumn. (The column's data-type is string[] or String().) When I call the WriteXML method on the resulting data-table, I get the following InvalidOperationException exception.
Is there a way to specify the data-table's xml-serializer to use for specific columns or specific data-types?
Basically, how can I make WriteXML work with this data-table?
System.InvalidOperationException:
Type System.String[] does not
implement IXmlSerializable interface
therefore can not proceed with
serialization.
You could easily copy your DataTable changing the offending Authors column to a String and joing the string[] data with a proper delimiter like "|" or "; ".
DataTable xmlFriendlyTable = oldTable.Clone();
xmlFriendlyTable.Columns["Author"].DataType = typeof(String);
xmlFriendlyTable.Columns["Author"].ColumnMapping = MappingType.Element;
foreach(var row in oldTable.Rows) {
object[] rowData = row.ItemArray;
object[] cpyRowData = new object[rowData.Length];
for(int i = 0; i<rowData.Length; i++) {
if(rowData[i] != null && rowData[i].GetType() == typeof(String[])) {
cpyRowData[i] = String.Join("; ", (rowData[i] as String[]));
} else {
cpyRowData[i] = rowData[i];
}
xmlFriendlyTable.Rows.Add(cpyRowData);
}
}
xmlFriendlyTable.WriteXml( ... );
NOTE Wrote the above in the web browser, so there may be syntax errors.
I'm attempting to parse SQL using the TSql100Parser provided by microsoft. Right now I'm having a little trouble using it the way it seems to be intended to be used. Also, the lack of documentation doesn't help. (example: http://msdn.microsoft.com/en-us/library/microsoft.data.schema.scriptdom.sql.tsql100parser.aspx )
When I run a simple SELECT statement through the parser it returns a collection of TSqlStatements which contains a SELECT statement.
Trouble is, the TSqlSelect statement doesn't contain attributes such as a WHERE clause, even though the clause is implemented as a class. http://msdn.microsoft.com/en-us/library/microsoft.data.schema.scriptdom.sql.whereclause.aspx
The parser does recognise the WHERE clause as such, looking at the token stream.
So, my question is, am I using the parser correctly? Right now the token stream seems to be the most useful feature of the parser...
My Test project:
public static void Main(string[] args)
{
var parser = new TSql100Parser(false);
IList<ParseError> Errors;
IScriptFragment result = parser.Parse(
new StringReader("Select col from T1 where 1 = 1 group by 1;" +
"select col2 from T2;" +
"select col1 from tbl1 where id in (select id from tbl);"),
out Errors);
var Script = result as TSqlScript;
foreach (var ts in Script.Batches)
{
Console.WriteLine("new batch");
foreach (var st in ts.Statements)
{
IterateStatement(st);
}
}
}
static void IterateStatement(TSqlStatement statement)
{
Console.WriteLine("New Statement");
if (statement is SelectStatement)
{
PrintStatement(sstmnt);
}
}
Yes, you are using the parser correctly.
As Damien_The_Unbeliever points out, within the SelectStatement there is a QueryExpression property which will be a QuerySpecification object for your third select statement (with the WHERE clause).
This represents the 'real' SELECT bit of the query (whereas the outer SelectStatement object you are looking at has just got the 'WITH' clause (for CTEs), 'FOR' clause (for XML), 'ORDER BY' and other bits)
The QuerySpecification object is the object with the FromClauses, WhereClause, GroupByClause etc.
So you can get to your WHERE Clause by using:
((QuerySpecification)((SelectStatement)statement).QueryExpression).WhereClause
which has a SearchCondition property etc. etc.
Quick glance around would indicate that it contains a QueryExpression, which could be a QuerySpecification, which does have the Where clause attached to it.
if someone lands here and wants to know how to get the whole elements of a select statement the following code explain that:
QuerySpecification spec = (QuerySpecification)(((SelectStatement)st).QueryExpression);
StringBuilder sb = new StringBuilder();
sb.AppendLine("Select Elements");
foreach (var elm in spec.SelectElements)
sb.Append(((Identifier)((Column)((SelectColumn)elm).Expression).Identifiers[0]).Value);
sb.AppendLine();
sb.AppendLine("From Elements");
foreach (var elm in spec.FromClauses)
sb.Append(((SchemaObjectTableSource)elm).SchemaObject.BaseIdentifier.Value);
sb.AppendLine();
sb.AppendLine("Where Elements");
BinaryExpression binaryexp = (BinaryExpression)spec.WhereClause.SearchCondition;
sb.Append("operator is " + binaryexp.BinaryExpressionType);
if (binaryexp.FirstExpression is Column)
sb.Append(" First exp is " + ((Identifier)((Column)binaryexp.FirstExpression).Identifiers[0]).Value);
if (binaryexp.SecondExpression is Literal)
sb.Append(" Second exp is " + ((Literal)binaryexp.SecondExpression).Value);
I had to split a SELECT statement into pieces. My goal was to COUNT how many record a query will return. My first solution was to build a sub query such as
SELECT COUNT(*) FROM (select id, name from T where cat='A' order by id) as QUERY
The problem was that in this case the order clause raises the error "The ORDER BY clause is not valid in views, inline functions, derived tables, sub-queries, and common table expressions, unless TOP or FOR XML is also specified"
So I built a parser that split a SELECT statment into fragments using the TSql100Parser class.
using Microsoft.Data.Schema.ScriptDom.Sql;
using Microsoft.Data.Schema.ScriptDom;
using System.IO;
...
public class SelectParser
{
public string Parse(string sqlSelect, out string fields, out string from, out string groupby, out string where, out string having, out string orderby)
{
TSql100Parser parser = new TSql100Parser(false);
TextReader rd = new StringReader(sqlSelect);
IList<ParseError> errors;
var fragments = parser.Parse(rd, out errors);
fields = string.Empty;
from = string.Empty;
groupby = string.Empty;
where = string.Empty;
orderby = string.Empty;
having = string.Empty;
if (errors.Count > 0)
{
var retMessage = string.Empty;
foreach (var error in errors)
{
retMessage += error.Identifier + " - " + error.Message + " - position: " + error.Offset + "; ";
}
return retMessage;
}
try
{
// Extract the query assuming it is a SelectStatement
var query = ((fragments as TSqlScript).Batches[0].Statements[0] as SelectStatement).QueryExpression;
// Constructs the From clause with the optional joins
from = (query as QuerySpecification).FromClauses[0].GetString();
// Extract the where clause
where = (query as QuerySpecification).WhereClause.GetString();
// Get the field list
var fieldList = new List<string>();
foreach (var f in (query as QuerySpecification).SelectElements)
fieldList.Add((f as SelectColumn).GetString());
fields = string.Join(", ", fieldList.ToArray());
// Get The group by clause
groupby = (query as QuerySpecification).GroupByClause.GetString();
// Get the having clause of the query
having = (query as QuerySpecification).HavingClause.GetString();
// Get the order by clause
orderby = ((fragments as TSqlScript).Batches[0].Statements[0] as SelectStatement).OrderByClause.GetString();
}
catch (Exception ex)
{
return ex.ToString();
}
return string.Empty;
}
}
public static class Extension
{
/// <summary>
/// Get a string representing the SQL source fragment
/// </summary>
/// <param name="statement">The SQL Statement to get the string from, can be any derived class</param>
/// <returns>The SQL that represents the object</returns>
public static string GetString(this TSqlFragment statement)
{
string s = string.Empty;
if (statement == null) return string.Empty;
for (int i = statement.FirstTokenIndex; i <= statement.LastTokenIndex; i++)
{
s += statement.ScriptTokenStream[i].Text;
}
return s;
}
}
And to use this class simply:
string fields, from, groupby, where, having, orderby;
SelectParser selectParser = new SelectParser();
var retMessage = selectParser.Parse("SELECT * FROM T where cat='A' Order by Id desc",
out fields, out from, out groupby, out where, out having, out orderby);