ADO.NET: How can we update a DataTable column values as fast as calculated columns does? - ado.net

I have a small test program that creates a DataTable with 1M rows.
Then I add a calculated column (Expression column), it's super fast like a second.
Then I just try to add a new column and set values to it, it takes more than 10 seconds.
DataTable update is slow. How can I make it as fast as calculated column?
10 seconds does seem acceptable but that is just a test program to show the difference.
This can become worst depending on how many rows, columns, etc.
Anyway if calculated column is able to do it that fast, hopefully we can get same performance with normal updates.
It seems calculated columns is able to turn off some notifications or other things. I'm not sure.
Any idea? Advice? Maybe I'm missing something when updating the DataTable.
Thanks in advance
Below is the code sample I'm using:
using System;
using System.Data;
namespace Test1
{
class Program
{
static void Main(string[] args)
{
try
{
System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
watch.Start();
Console.WriteLine("DataTable creation started ...");
DataTable data = new DataTable();
int nbrCols = 50;
for (int i = 0; i < nbrCols; i++)
{
data.Columns.Add(string.Concat("COL_", (i + 1)), typeof(double));
}
System.Random r = new System.Random();
data.BeginLoadData();
int nbrows = 1000000;
for (int i = 0; i < nbrows; i++)
{
DataRow newRow = data.NewRow();
for (int j = 0; j < nbrCols; j++)
{
newRow[j] = r.NextDouble();
}
data.Rows.Add(newRow);
}
data.EndLoadData();
watch.Stop();
Console.WriteLine("DataTable creation = " + watch.ElapsedMilliseconds / 1000.0);
int colIndexFlow1 = data.Columns.IndexOf("COL_1");
int colIndexFlow2 = data.Columns.IndexOf("COL_2");
/* Add a calculated columns */
watch.Restart();
data.Columns.Add("CALC_1", typeof(double), "COL_1 + COL_2");
data.AcceptChanges();
watch.Stop();
Console.WriteLine("Calculated Column creation = " + watch.ElapsedMilliseconds / 1000.0);
/* Add a new column */
watch.Restart();
data.BeginLoadData();
data.Columns.Add("NEW_1", typeof(double));
int colIdx = data.Columns.IndexOf("NEW_1");
foreach (DataRow row in data.Rows)
{
//double val = row.Field<double>("COL_1") + row.Field<double>("COL_2");
//row.SetField<double>(colIdx, val);
// Most of the time is spent to actually set the data
// Question is how can we make the update happening as fast as the update done when using a calculated column.
row.SetField<double>(colIdx, 2.0);
}
data.AcceptChanges();
data.EndLoadData();
watch.Stop();
Console.WriteLine("New Column creation = " + watch.ElapsedMilliseconds / 1000.0);
}
catch (Exception ex)
{
Console.WriteLine("Error=" + ex.Message);
}
finally
{
Console.ReadLine();
Console.WriteLine("Press any key to exit");
}
}
}
}

Related

How to use selection sort in objects and classes

I'm creating two classes called stop watch and random numbers, which I have already done, but I needed to create a test program that would measure the execution time of sorting 100,000 numbers using selection sort. I know how to create a selection sort, I just don't know how to take the random numbers class and put it together with the selection sort, I get the error message "incompatible types random numbers cannot be converted to int" I hope someone can help me.
My random numbers class
import java.util.Random;
public class randomnumbers {
Random ran1 = new Random();
private int size;
public randomnumbers(){
size = 100000;
}
public int getSize(){
return size;
}
public void setSize(int newSize){
size = newSize;
}
public int [] createArray(int [] size){
for (int i = 0; i < size.length; i++){
size[i] = ran1.nextInt();
}
return size;
}
public static void printArray (int [] array){
for (int i = 0; i < array.length; i++){
if (i < 0){
System.out.println(array[i] + " ");
}
}
}
}
My test Program
public static void main (String [] args){
// Create a StopWatch object
StopWatch timer = new StopWatch();
//create random numbers
randomnumbers numbers = new randomnumbers();
//Create the size of the array
numbers.getSize();
// Invoke the start method in StopWatch class
timer.start();
//sort random numbers
selectionSort();
// Invoke the stop method in StopWatch class
timer.stop();
// Display the execution time
System.out.println("The execution time for sorting 100,000 " +
"numbers using selection sort: " + timer.getElapsedTime() +
" milliseconds");
}
// selectionSort performs a selection sort on an array
public static void selectionSort(int[] array) {
for (int i = 0; i < array.length - 1; i++) {
int min = array[i];
int minIndex = i;
for (int j = i + 1; j < array.length; j++) {
if (array[j] < min) {
min = array[j];
minIndex = j;
}
}
if (i != minIndex) {
array[minIndex] = array[i];
array[i] = min;
}
}
}
}
Where exactly are you getting "incompatible types random numbers cannot be converted to int" error?
There are multiple issues with the code:
Unconventional naming
size field is in randomnumbers class is used as actual array size in constructor but in createArray it's overshadowed with a parameter of the same name but different type and meaning.
You are not passing any array to selectionSort in Main. This is where I get compile error on your code.
printArray has if (i < 0) condition that is false for all ran1.nextInt() numbers so it will not print anything.
Feeding selectionSort with numbers.createArray(new int[numbers.getSize()]) compiles and ends up sorting the array.

Attempts to call a method in the same class not working (java)

I'm creating a random number generator which then sorts the digits from largest to smallest. Initially it worked but then I changed a few things. As far as I'm aware I undid all the changes (ctrl + z) but now I have errors at the points where i try to call the methods. This is probably a very amateur problem but I haven't found an answer. The error i'm met with is "method in class cannot be applied to given types"
Here's my code:
public class RandomMath {
public static void main(String[] args) {
String bigger = bigger(); /*ERROR HERE*/
System.out.println(bigger);
}
//create method for generating random numbers
public static int generator(int n){
Random randomGen = new Random();
//set max int to 10000 as generator works between 0 and n-1
for(int i=0; i<1; i++){
n = randomGen.nextInt(10000);
// exclude 1111, 2222, 3333, 4444, 5555, 6666, 7777, 8888, 9999, 0000
if((n==1111 || n==2222 || n==3333 || n ==4444 || n==5555)
||(n==6666 || n==7777 || n==8888 || n==9999 || n==0000)){
i--;
}
}
return n;
}
//create method for denoting the bigger number
public static String bigger(int generated){
generated = generator(); /*ERROR HERE*/
System.out.println(generated);
int[] times = new int[10];
while (generated != 0) {
int val = generated % 10;
times[val]++;
generated /= 10;
}
String bigger = "";
for (int i = 9; i >= 0; i--) {
for (int j = 0; j < times[i]; j++) {
bigger += i;
}
}
return bigger;
}
}
You have not defined a method bigger(), only bigger(int generated). Therefore, you must call your bigger method with an integer parameter.

User input with linear and binary searching in java

I'm trying to write a program that asks the user to input the size of the array the user wants to create, and then asks the user to fill the array with elements, and then it should display the array with its elements and, ask the user to conduct a search for an integer. It should conduct a linear and binary search, while displaying how many probes it took to determine is the element is in the array. So far the only output i have gotten is that the element has not been found. If you could look at my code and see what the problem is, because i have tried for hours and i have changed everything i can think of. Any help would be greatly appreciated.
import java.util.Scanner;
public class Searching
{
public static int[] anArray = new int[100];
private int numberOfElements;
public int arraySize = numberOfElements;
public String linearSearch(int value)
{
int count = 0;
boolean valueInArray = false;
String indexOfValue = "";
System.out.print("The Value was Found in: ");
for(int i = 0; i < arraySize; i++)
{
if(anArray[i] == value)
{
valueInArray = true;
System.out.print(i + " ");
indexOfValue += i + " ";
}
count ++;
}
if(!valueInArray)
{
indexOfValue = " None found";
System.out.print(indexOfValue);
}
System.out.println("\nIt took " + count + " probes with a linear search to find");
return indexOfValue;
}
public void binarySearch(int value)
{
int min = 0;
int max = arraySize - 1;
int count = 0;
while(min <= max)
{
int mid = (max + min) / 2;
if(anArray[mid] < value) min = mid + 1;
else if(anArray[mid] > value) max = mid - 1;
else
{
System.out.println("\nFound a Match for " + value + " at Index " + mid);
min = max + 1;
}
count ++;
}
System.out.println("It took " + count + " probes with a binary search to find");
}
public static void main(String[] args)
{
#SuppressWarnings("resource")
Scanner scan = new Scanner(System.in);
System.out.println("Input the number of elements in your Array");
int numberOfElements = scan.nextInt();
if(numberOfElements <= 0)
{
System.exit(0);
}
int[] anArray = new int[numberOfElements];
System.out.println("\nEnter " + numberOfElements + " Integers");
for(int i = 0; i < anArray.length; i ++)
{
System.out.println("Int # " + (i + 1) + ": ");
anArray[i] = scan.nextInt();
}
System.out.println("\nThe integers you entered are: ");
for(int i = 0; i < anArray.length; i ++) // for loop used to print out each element on a different line
{
System.out.println(anArray[i]);
}
System.out.println("Which element would you like to find?");
int value = scan.nextInt();
Wee3Q2JOSHBALBOA newArray = new Wee3Q2JOSHBALBOA();
newArray.linearSearch(3);
newArray.binarySearch(value);
}
}
hmm I am not sure you are using an array the correct way, you see an Array is a set size and I dont think you can expand like they way you are doing...
Instead try setting the size of the array to certain length.
Or use vectors

Repeating command in gwt

I use flex table to show the data fetched from db. When table contains a large amount of columns it takes much time to render the flextable(I've tested it on table with 80 columns and 38 rows) and while it is not completely rendered I can not do anything with a page. So i use Schedule.get().scheduleIncremental(ReapitingCommand command) as follows:
final int WORK_CHUNK = 2;
Scheduler.get().scheduleIncremental(new RepeatingCommand() {
int rowCounter = 0;
#Override
public boolean execute() {
for (int i = rowCounter; i < rowCount; i++, rowCounter++) {
for (int j = 0; j < columnCount; j++) {
table.setText(i, j, data.get(i)[j]);
}
if (rowCounter % WORK_CHUNK == 0)
return true;
}
return false;
}
});
But if i have 9 rows and 3 columns in data object it only render 2 rows.
How I can improve its performance, because as I said earlier if have 38 rows and 80 columns it takes too much time to render all the data without scheduleIncremental. Even browser popups the window that script may have stoped responing.
Why don't use a CellTable instead of a FlexTable to render your data?
CellTable is supposed to be much more efficient in rendering large amounts of data compared to FlexTable.
For above code I would have done it this way. Get rid of the first for loop because execute() is recursive.
Scheduler.get().scheduleIncremental(new RepeatingCommand() {
int rowCounter = 0;
int rowCount = 10;
#Override
public boolean execute() {
// do some work
for (int j = 0; j < columnCount; j++) {
table.setText(i, j, data.get(i)[j]);
}
rowCounter++;
if (rowCounter >= rowCount)
return false;
return true;
}
});

Specified cast is not valid

SqlDataReader reader;
string r="C:\\Users\\Shivam\\Documents\\";
if ((FileUpload1.PostedFile != null)&&(FileUpload1.PostedFile.ContentLength > 0))
{
r += System.IO.Path.GetFileName(FileUpload1.PostedFile.FileName);
}
OleDbConnection oconn =
new OleDbConnection
(#"Provider=Microsoft.Jet.OLEDB.4.0;"
+ "Data Source="+r+";"
+ #"Extended Properties=""Excel 8.0;HDR=Yes;""");
oconn.Open();
conn.Open();
OleDbCommand dbcom = new OleDbCommand("SELECT * FROM [Sheet1$]", oconn);
OleDbDataReader dbreader = dbcom.ExecuteReader();
//dbread = dbreader;
int rni = dbreader.GetOrdinal ("RollNo");
int mki = dbreader.GetOrdinal ("marks");
int rowcount =0;
while(dbreader.Read())
{ rowcount++; }
//dbreader.Close();
//OleDbDataReader dbread = dbcom.ExecuteReader();
int[] rn = new int[rowcount];
int[] gr = new int[rowcount];
while (dbreader.Read())
{
int o = 0;
for(int i=0;i<rowcount;i++)
{
int q = (int)dbreader.GetValue(rni);
int p = (int)dbreader.GetValue(mki);
rn[i] = q;
gr[i] = p;
//roll[i] = valid(odr, 0);//Here we are calling the valid method
//marks[i] = valid(odr, 1);
//i++;
if (gr[i] >= 11)
{ o=i; }
}
if(o!=0)
{ break; }
TextBox4.Text += rn + "\t" + gr;
//Here using this method we are inserting the data into the database
x = TextBox2.Text.Substring(0, 1);
y = TextBox2.Text.Substring(1, 1);
//for (int s = 0; s < roll.Length; s++)
//{
//SqlDataAdapter sda = new SqlDataAdapter("select StudentID from Student where APID=" + int.Parse(y)+ "and Semester=" + int.Parse(z) + "and Roll_No=" + int.Parse(RollNo), conn);
//DataSet ds = new DataSet();
//sda.Fill(ds);
//GridView1.DataSource = ds;
//GridView1.DataBind();
SqlCommand command = new SqlCommand();
command.Connection = conn;
//command.Connection.Open();
command.CommandType = CommandType.Text;
int c = rn.Length;
for (int n = 0; n<rn.Length; n++)
{
command.CommandText = "Select StudentID from Student where APID=" + int.Parse(x) + "and Semester=" + int.Parse(y) + "and Roll_No=" + rn[n];
}
reader = command.ExecuteReader();
while (reader.Read())
{
TextBox4.Text = reader.GetInt32(0).ToString();
a = (int)reader["StudentID"];
for (int v = 0; v < rn.Length; v++)
{
insertdataintosql(rn[v], gr[v]);
}
}
//}
}
conn.Close();
oconn.Close();
The problem here is that the statements in the while(dbreader.read()) are not executed, rather directly conn.Close() is performed. And if I take another datareader with same command after closing the previous datareader, the error "Specified cast not valid" is thrown at "int q = (int)dbreader.GetValue(rni);". Please help me out...thanks in advance
DataReaders are forward-only. You can't "reuse" it.
All you are doing is allocating an array based on the number of records. Either run a COUNT query first to get the number of records, or re-allocate your array on the fly in the one loop.
OleDbCommand dbcom = new OleDbCommand("SELECT COUNT(*) as RowCount FROM [Sheet1$]", oconn);
dbreader = dbcom.ExecuteReader();
dbreader.Read();
rowcount = dbreader.GetOrdinal("RowCount");
dbcom.Close();
dbcom = new OleDbCommand("SELECT * FROM [Sheet1$]", oconn);
dbreader = dbcom.ExecuteReader();
... carry on with your 2nd loop
There is more to say about releasing resources properly. Best practice is to add parameter CommandBehavior.CloseConnection to ExecuteReader, create your data reader inside a using() construct and issue cmd.Dispose() at the end to ensure SQL connection resources are released properly.
.. though actually since it's a local file you're using it may not matter so much but generally speaking you should do that. Otherwise its very easy to find that an orphaned DataReader has not released its connection.