I have created http get service that is returning collection data as json. But I don't want to show column data type
This is my function code.
const result = doc.find({},{id:1,desc:1,label:1,LocationTree:1,path:1,coordinates:1,_id: 0}).toArray();
return result;
I contacted MongoDB support about this issue when I first started with Stitch. They indicated there is no way to suppress the type returned with numeric and datetime value types.
Your client application must handle the type when processing the JSON response. This is done with dot notation.
Assume your id field is of type Int32.
A typical API call might return an integer or string at result.id. The Stitch HTTP service, however, appends the type to the value, returning it at result.id.$numberInt.
As noted above, type data is included with numeric and datetime values. Booleans, Strings, Objects, and Arrays do not include type data. They can be accessed at result.someBoolean, result.someString, result.someObject, or result.someArray.
I haven't found a full list of types returned by the Stitch HTTP service. For some reason, they do NOT match the BSON Type aliases.
Related
I'm currently improving a library client for Postgresql, the library already has working communication protocol including DataRow and RowDescription.
The problem I'm facing right now is how to deal with values.
Returning plain string with array of integers for example is kind of pointless.
By my research I found that some other libraries (like for Python) either return is as unmodified string or convert primitive types including arrays.
What I mean by conversion is making Postgres DataRow raw data as Python-type value: Postgres integer is parsed as python number, Postgres booleans as python booleans, etc.
Should I make second query to get information column type and use its converters or should I leave it plain?
You could opt to get the array values in the internal format by setting the corresponding "result-column format code" in the Bind message to 1, but that is typically a bad choice, since the internal format varies from type to type and may even depend on the server's architecture.
So your best option is probably to parse the string representation of the array on the client side, including all the escape characters.
When it comes to finding the base type for an array type, there is no other option than querying pg_type like
SELECT typelem::regtype FROM pg_type WHERE oid = 1007;
typelem
---------
integer
(1 row)
You could cache these values on the client side so that you don't have to query more than once per type and database session.
I've a MongoDB Kafka Connector v1.3 configured with output.json.formatter=com.mongodb.kafka.connect.source.json.formatter.SimplifiedJson, so my UUID field is outputed like that: "_id":"MSRJCs07SFy4sMpopdRvEA==".
But, one of clients of that Kafka message need to convert that SimplifiedJson in a valid ExtendedJson, and from that ExtendedJson instantiate a org.bson.BsonDocument (mongodb Java SDK), so the _id should be recongnized as a BsonBinary (UUID) with value 3124490a-cd3b-485c-b8b0-ca68a5d46f10.
Trying BsonDocument.parse directly from SimplifiedJson, that value "_id":"MSRJCs07SFy4sMpopdRvEA==" is treated as a BsonString.
The client can't do that manually because there is not standard in that field, may be a object id, a uuid or any other thing which make sense to the application that generate that data.
So, there is any built-in way in MongoDB library to convert a SimplifiedJson to a Bson with the same original field types (object id, binary, dates, numbers, etc)?
When I tried to get some values in a DataFrame, like:
df.select("date").head().get(0) // type: Any
The result type is Any, which is not expected.
Since a dataframe contains the schema of the data, it should know the DataType for each column, so when i try to get a value using get(0), it should return the value with the correct type. However, it does not.
Instead, I need to specify which DataType i want using getDate(0), which seems weird, inconvenient, and makes me mad.
When I have specified the schema with the correct DataTypes for each column when i created the Dataframe, I don't want to use different getXXX()' for differentcolumn`s.
Are there some convenient ways that I can get the values with their own correct types? That is to say, how can I get the values with the correct DataType specified in the schema?
Thank you!
Scala is a statically typed language. so the get method defined on the Row can only return values with a single type because the return type of the get method is Any. It cannot return Int for one call and a String for another.
you should be calling the getInt, getDate and other get methods provided for each type. Or the getAs method in which you can pass the type as a parameter (for example row.getAs[Int](0)).
As mentioned in the comments other options are
use Dataset instead of a DataFrame.
use Spark SQL
You can call the generic getAs method as getAs[Int](columnIndex), getAs[String](columnIndex) or use specific methods like getInt(columnIndex), getString(columnIndex).
Link to the Scaladoc for org.apache.spark.sql.Row.
I have a postgres query with one input parameter of type varchar.
value of that parameter is used in where clause.
Till now only single value was sent to query but now we need to send multiple values such that they can be used with IN clause.
Earlier
value='abc'.
where data=value.//current usage
now
value='abc,def,ghk'.
where data in (value)//intended usage
I tried many ways i.e. providing value as
value='abc','def','ghk'
Or
value="abc","def","ghk" etc.
But none is working and query is not returning any result though there are some matching data available. If I provide the values directly in IN clause, I am seeing the data.
I think I should somehow split the parameter which is comma separated string into multiple values, but I am not sure how I can do that.
Please note its Postgres DB.
You can try to split input string into an array. Something like that:
where data = ANY(string_to_array('abc,def,ghk',','))
I have been implementing user defined types in Postgresql 9.2 and got confused.
In the PostgreSQL 9.2 documentation, there is a section (35.11) on user defined types. In the third paragraph of that section, the documentation refers to input and output functions that are used to construct a type. I am confused about the purpose of these functions. Are they concerned with on-disk representation or only in-memory representation? In the section referred to above, after defining the input and output functions, it states that:
If we want to do anything more with the type than merely store it,
we must provide additional functions to implement whatever operations
we'd like to have for the type.
Do the input and output functions deal with serialization?
As I understand it, the input function is the one which will be used to perform INSERT INTO and the output function to perform SELECT on the type so basically if we want to perform an INSERT INTO then we need a serialization function embedded or invoked in the input or output function. Can anyone help explain this to me?
Types must have a text representation, so that values of this type can be expressed as literals in a SQL query, and returned as results in output columns.
For example, '2013-20-01' is a text representation of a date. It's possible to write VALUES('2013-20-01'::date) in a SQL statement, because the input function of the date type recognizes this string as a date and transforms it into an internal representation (for both using it in memory and storing to disk).
Conversely, when client code issues SELECT date_field FROM table, the values inside date_field are returned in their text representation, which is produced by the type's output function from the internal representation (unless the client requested a binary format for this column).