I'm getting an error when trying to insert a new row into a SQL Server table using ADO.NET, that column CITY does not accept nulls.
I am invoking the DataAdapter's Update method, supplying the Table associated with the current Row of the DataRowView as the argument:
DA.Update(drv.Row.Table);
The DataAdapter's InsertCommand includes this parameter declaration:
c.Parameters.Add(new SqlParameter("#city", SqlDbType.VarChar, 25, "city"));
When I examine the data objects in the debugger, the CITY value is present in position [4] (see below).
Is there anything else which could be causing this error?
?drv.Row.Table.Columns.List
Count = 20
[0]: {id}
[1]: {practiceid}
[2]: {addr1}
[3]: {addr2}
[4]: {city}
[5]: {state}
[6]: {zipcode}
[7]: {geozip}
[8]: {wheelchair}
[9]: {transportation}
[10]: {braille}
[11]: {fax}
[12]: {phone}
[13]: {remarks}
[14]: {createdby}
[15]: {datecreated}
[16]: {lastupdatedby}
[17]: {lastupdated}
[18]: {rowversion}
[19]: {dirty}
?drv.Row.ItemArray
{object[20]}
[0]: -1
[1]: 7819
[2]: "123 Main St"
[3]: {}
[4]: "Anywhere"
[5]: "KS"
[6]: "35577"
[7]: {}
[8]: true
[9]: true
[10]: true
[11]: {}
[12]: {}
[13]: "test remark"
[14]: "SRVR1\\Tim"
[15]: {}
[16]: {}
[17]: {}
[18]: {}
[19]: false
Related
I have two LazyFrames, df1 and df2.
After filtering df2 according to df1 max value, I want to concatenate them.
But combination of with_context() and filter() on LazyFrames will raise NotFoundError.
What's the best way to do this?
import polars as pl
df1 = pl.DataFrame({'foo': [0, 1], 'bar': ['a', 'a']}).lazy()
df2 = pl.DataFrame({'foo': [1, 2, 3], 'bar': ['b', 'b', 'b']}).lazy()
df = pl.concat(
[
df1,
df2.with_context(df1.select(pl.col('foo').alias('foo_')))
.filter(pl.col('foo') > pl.col('foo_').max())
]).collect()
# ---------------------------------------------------------------------------
# NotFoundError Traceback (most recent call last)
# [<ipython-input-2-cf44deab2d4b>](https://localhost:8080/#) in <module>
# 4 df2 = pl.DataFrame({'foo': [1, 2, 3], 'bar': ['b', 'b', 'b']}).lazy()
# 5
# ----> 6 df = pl.concat(
# 7 [
# 8 df1,
#
# 1 frames
# [/usr/local/lib/python3.8/dist-packages/polars/utils.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
# 327 def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
# 328 _rename_kwargs(fn.__name__, kwargs, aliases)
# --> 329 return fn(*args, **kwargs)
# 330
# 331 return wrapper
#
# [/usr/local/lib/python3.8/dist-packages/polars/internals/lazyframe/frame.py](https://localhost:8080/#) in collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, no_optimization, slice_pushdown, common_subplan_elimination, streaming)
# 1166 streaming,
# 1167 )
# -> 1168 return pli.wrap_df(ldf.collect())
# 1169
# 1170 def sink_parquet(
#
# NotFoundError: foo_
When I assign comparison result as a column, error not raised,
(df2.with_context(df1.select(pl.col('foo').alias('foo_')))
.with_column((pl.col('foo') > pl.col('foo_').max()).alias('x'))
.filter(pl.col('x'))).collect()
# OK
But I drop that column after filter(), error again.
(df2.with_context(df1.select(pl.col('foo').alias('foo_')))
.with_column((pl.col('foo') > pl.col('foo_').max()).alias('x'))
.filter(pl.col('x'))
.drop('x')).collect()
# NotFoundError: foo_
Finally I find this works. But what's the difference between the previous?
(it seems verbose. other good solution exists?)
(df2.with_context(df1.select(pl.col('foo').alias('foo_')))
.with_column(pl.col('foo_').max())
.filter(pl.col('foo') > pl.col('foo_'))
.drop('foo_')).collect()
# OK
related?
https://stackoverflow.com/a/71108312/7402018
I have a YAML file which has documents of this sort
%YAML 1.2
---
!some_tag
name: xyz
constants:
state: abc
After reading in the documents, before dumping using ruamel.yaml.YAML().dump, I want to remove this part of the document
%YAML 1.2
---
My output file should have just these sections of the document
!some_tag
name: xyz
constants:
state: abc
How can this be done?
If you have an explicit tag with version the .version attribute on the
YAML() instance gets set. You can either instantiate a new instance for dumping
or just set that attribute to None:
import sys
import pathlib
import ruamel.yaml
inp = pathlib.Path('input.yaml')
yaml = ruamel.yaml.YAML()
data = yaml.load(inp)
print(f'>>> version info {yaml.version}\n')
yaml.version = None
yaml.dump(data, sys.stdout)
which gives:
>>> version info (1, 2)
!some_tag
name: xyz
constants:
state: abc
I can use poi in windbg scripting to traverse the fields and then print the field i am interested in.
For e.g. if i have all objects of type X which has field X.y.z where z is an array, where y is at offset 0x10 and z is at offset 0x8. I can write
.foreach ( dSM { !dumpheap -short -type X})
{
.printf "%p\n", poi(poi(${dSM}+0x10)+0x8);
!dumparray poi(poi(${dSM}+0x10)+0x8)
}
Now i want to print first/second element of all these arrays, how can i do that ?
using !do poi(poi(poi(${dSM}+0x10)+0x8)) does not work.
0:045> !DumpArray /d 000001d3b96787a8
Name: ABC[]
MethodTable: 00007ffc951e76e0
EEClass: 00007ffcf22f4480
Size: 56(0x38) bytes
Array: Rank 1, Number of elements 4, Type CLASS
Element Methodtable: 00007ffc951e6cc0
[0] 000001d3b9678788
[1] null
[2] null
[3] null
0:045> !dumpobj /d poi(000001d3b96787a8)
<Note: this object has an invalid CLASS field>
Invalid object
Array class is:
:045> !DumpClass /d 00007ffcf22f4480
Class Name: System.Object[]
mdToken: 0000000002000000
File: C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Parent Class: 00007ffcf22f5018
Module: 00007ffcf2221000
Method Table: 00007ffcf2949e80
Vtable Slots: 18
Total Method Slots: 1c
Class Attributes: 2101
Transparency: Transparent
NumInstanceFields: 0
NumStaticFields: 0
Given this program:
using System;
namespace WinDbgArrayAccess
{
class Program
{
static void Main()
{
Program[] justAnArray = new Program[20];
for (int i =0; i<justAnArray.Length;i++) justAnArray[i] = new Program();
Console.WriteLine("Access the elements of the array in WinDbg now!");
Console.ReadLine();
}
}
}
You can see
0:006> !DumpArray /d 0336243c
Name: WinDbgArrayAccess.Program[]
MethodTable: 01914db0
EEClass: 71967820
Size: 92(0x5c) bytes
Array: Rank 1, Number of elements 20, Type CLASS
Element Methodtable: 01914d60
[0] 03362498
[1] 033624a4
[2] 033624b0
[3] 033624bc
[4] 033624c8
[5] 033624d4
...
Now you need to find those numbers somewhere in the memory. Since we hardly have a different starting point, let's start at the array's address:
0:006> dp 0336243c L10
0336243c 01914db0 00000014 03362498 033624a4
0336244c 033624b0 033624bc 033624c8 033624d4
0336245c 033624e0 033624ec 033624f8 03362504
0336246c 03362510 0336251c 03362528 03362534
Now, 01914db0 is the type of the object (called Method Table, MT). 0x14 is 0n20, which is the size of the array. And after that, it seems we have the elements, 03362498, 033624a4, 033624b0 etc.
How do we access that programmatically? Well, it's easy now:
0:006> .printf "%p",poi(0336243c+(2+3)*$ptrsize)
033624bc
Where 2 is to skip MT and Length and 3 is the index of the array, giving you the 4th element.
I was able to get first 2 elements by going forward poi(poi(array+10)+8) and poi(poi(array+10)+10) offsets from array
0:298> .foreach (obj { !DumpHeap -short -live -mt 00007ffc951e7ab8}) { .printf "%p\n", ${obj}; !mex.do2 poi(poi(poi(poi(${obj}+0x10)+0x8)+0x10)+0x8); !mex.do2 poi(poi(poi(poi(${obj}+0x10)+0x8)+0x10)+0x10); }
000001d379ac94a0
[raw] 000001d379ac8bc8 "XYZ_String"
[raw] 000001d379ac8c88 "XYZ_String_2"
000001d379e87168
[raw] 000001d379e86888 "ABCD_String"
[raw] 000001d379e86948 "ABCD_String_2"
000001d3b972a218
I am attempting to write a simple code to simply plot a bar graph of some fruit names in the x-axis vs corresponding sales units. The aim of this code is just to understand how to query postgres results from heroku hosted database through a dash app.
Below is the code,
from dash import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.graph_objs as go
import psycopg2
import os
DATABASE_URL = os.environ['DATABASE_URL']
conn = psycopg2.connect(DATABASE_URL, sslmode='require')
cur = conn.cursor()
cur.execute("SELECT fruits FROM pgrt_table")
fruits1=cur.fetchall()
#print(fruits1)
cur.execute("SELECT sales FROM pgrt_table")
sales1=cur.fetchall()
app = dash.Dash()
app.layout = html.Div(children=[
html.H1(
children='Hello Dash'
),
html.Div(
children='''Dash: A web application framework for Python.'''
),
dcc.Graph(
id='example-graph',
figure=go.Figure(
data=[
go.Bar(
x=fruits1, y=sales1, name='SF'),
#{'x': [1, 2, 3], 'y': [2, 4, 5], 'type': 'bar', 'name': u'Montréal'},
],
#'layout':{
# 'title': 'Dash Data Visualization'
#}
)
)
])
if __name__ == '__main__':
app.run_server(debug=True)
The output is below,
Output to the above code
The corresponding output is just the axes with no bar graphs. The connection with the db is working since printing fruits1 or sales1 gives me the values from the columns in postgres. The only issue is the plotting.
NOTE: This question has been heavily modified since the previous draft was extremely vague without any code to show for.
Example:
fruits1 = [('apple',), ('banana',),
('mango',), ('pineapple',),
('peach',), ('watermelon',)]
The output of your database cannot be used directly:
xData = [fruit[0] for fruit in fruits1]
# gives ['apple', 'banana', 'mango', 'pineapple', 'peach', 'watermelon']
yData = [sales[0] for sales in sales1]
You have to assign your data to the go.Bar object:
go.Bar(x=xData, y=yData, name='SF')
I use Pandas dataframe heavily. And need to attach some data to the dataframe, for example to record the birth time of the dataframe, the additional description of the dataframe etc.
I just can't find reserved fields of dataframe class to keep the data.
So I change the core\frame.py file to add a line _reserved_slot = {} to solve my issue. I post the question here is just want to know is it OK to do so ? Or is there better way to attach meta-data to dataframe/column/row etc?
#----------------------------------------------------------------------
# DataFrame class
class DataFrame(NDFrame):
_auto_consolidate = True
_verbose_info = True
_het_axis = 1
_col_klass = Series
_AXIS_NUMBERS = {
'index': 0,
'columns': 1
}
_reserved_slot = {} # Add by bigbug to keep extra data for dataframe
_AXIS_NAMES = dict((v, k) for k, v in _AXIS_NUMBERS.iteritems())
EDIT : (Add demo msg for witingkuo's way)
>>> df = pd.DataFrame(np.random.randn(10,5), columns=list('ABCDEFGHIJKLMN')[0:5])
>>> df
A B C D E
0 0.5890 -0.7683 -1.9752 0.7745 0.8019
1 1.1835 0.0873 0.3492 0.7749 1.1318
2 0.7476 0.4116 0.3427 -0.1355 1.8557
3 1.2738 0.7225 -0.8639 -0.7190 -0.2598
4 -0.3644 -0.4676 0.0837 0.1685 0.8199
5 0.4621 -0.2965 0.7061 -1.3920 0.6838
6 -0.4135 -0.4991 0.7277 -0.6099 1.8606
7 -1.0804 -0.3456 0.8979 0.3319 -1.1907
8 -0.3892 1.2319 -0.4735 0.8516 1.2431
9 -1.0527 0.9307 0.2740 -0.6909 0.4924
>>> df._test = 'hello'
>>> df2 = df.shift(1)
>>> print df2._test
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Python\lib\site-packages\pandas\core\frame.py", line 2051, in __getattr__
(type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute '_test'
>>>
This is not supported right now. See https://github.com/pydata/pandas/issues/2485. The reason is the propogation of these attributes is non-trivial. You can certainly assign data, but almost all pandas operations return a new object, where the assigned data will be lost.
Your _reserved_slot will become a class variable. That might not work if you want to assign different value to different DataFrame. Probably you can assign what you want to the instance directly.
In [6]: import pandas as pd
In [7]: df = pd.DataFrame()
In [8]: df._test = 'hello'
In [9]: df._test
Out[9]: 'hello'
I think a decent workaround is putting your datafame into a dictionary with your metadata as other keys. So if you have a dataframe with cashflows, like:
df = pd.DataFrame({'Amount': [-20, 15, 25, 30, 100]},index=pd.date_range(start='1/1/2018', periods=5))
You can create your dictionary with additional metadata and put the dataframe there
out = {'metadata': {'Name': 'Whatever', 'Account': 'Something else'}, 'df': df}
and then use it as out[df]