I am trying to insert two columns using PQputCopyData with following code. But once it checks for the final result it shows error invalid byte sequence for encoding UTF8 and data is not getting inserted into the database.
Both columns type is character varying. What am I doing wrong here ?
const char *buffer = "john,doe";
PGresult *res;
res=PQexec(conn,"COPY john FROM STDIN DELIMITER ',';");
cout<<buffer;
if(PQresultStatus(res) != PGRES_COPY_IN)
{
cout<<"copy in not ok";
}
else
{
if(PQputCopyData(conn,buffer,400) == 1)
{
if(PQputCopyEnd(conn,NULL) == 1)
{
PGresult *res = PQgetResult(conn);
if(PQresultStatus(res) == PGRES_COMMAND_OK)
{
cout<<"done";
}
else
{
cout<<PQerrorMessage(conn); Here I get the error invalid byte sequence for encoding "UTF8"
}
}
else
{
cout<<PQerrorMessage(conn);
}
}
}
if(PQputCopyData(conn,buffer,400) == 1)
What's wrong is passing 400 instead of the actual size of the contents in buffer, making it send unallocated garbage after the real data. Use strlen(buffer) instead.
Also you want each line to finish with a newline, so buffer should be :
const char *buffer = "john,doe\n";
Related
I'm using the flutter_bluetooth_serial 0.4.0 package, it has a listen function that receives a function that returns the reading of the sending of a string (List Uint8), but for my case I need to carry out the communication at a very high speed, and when that it happens it does not understand where the end of a string is and it joins it until it overflows with 230 bytes and shows it as if it were a single string, I tried to solve this in several ways, but I can only receive the complete string (18 bytes) when I reduce the transmission speed, I've tried using some characters like '\n', to see if it understands where a string ends and another begins, but I wasn't successful either. If I could read character by character for me it would also solve it, because the messages have a sending pattern.
Do you have an idea how I could be solving this? Some package that works better than this one for this purpose or some way to determine where the end of the string is. I thank!
Here is the code snippet I get the data:
_dataSubscription = connection.input!.listen(_onDataReceived);
void _onDataReceived(Uint8List data) {
print('Data incoming: ${ascii.decode(data)}');
// Allocate buffer for parsed data
var backspacesCounter = 0;
for (var byte in data) {
if (byte == 8 || byte == 127) backspacesCounter++;
}
var buffer = Uint8List(data.length - backspacesCounter);
var bufferIndex = buffer.length;
// Apply backspace control character
backspacesCounter = 0;
for (int i = data.length - 1; i >= 0; i--) {
if (data[i] == 8 || data[i] == 127) {
backspacesCounter++;
} else {
if (backspacesCounter > 0) {
backspacesCounter--;
} else {
buffer[--bufferIndex] = data[i];
}
}
// print(backspacesCounter);
// print(buffer);
// print(bufferIndex);
}
I've tried using some characters like '\n', to see if it understands where a string ends and another begins, read character per character, but doesn't have function to do this.
I have the following text file structure (the text file is pretty big, around 100,000 lines):
A|a1|111|111|111
B|111|111|111|111
A|a2|222|222|222
B|222|222|222|222
B|222|222|222|222
A|a3|333|333|333
B|333|333|333|333
...
I need to extract a piece of text related to a given key. For example, if my key is A|a2, I need to save the following as a string:
A|a2|222|222|222
B|222|222|222|222
B|222|222|222|222
For my C++ and Objective C projects, I used the C++ getline function as follows:
std::ifstream ifs(dataPathStr.c_str());
NSString* searchKey = #"A|a2";
std::string search_string ([searchKey cStringUsingEncoding:NSUTF8StringEncoding]);
// read and discard lines from the stream till we get to a line starting with the search_string
std::string line;
while( getline( ifs, line ) && line.find(search_string) != 0 );
// check if we have found such a line, if not report an error
if( line.find(search_string) != 0 )
{
data = DATA_DEFAULT ;
}
else{
// we need to form a string that would include the whole set of data based on the selection
dataStr = line + '\n' ; // result initially contains the first line
// now keep reading line by line till we get an empty line or eof
while(getline( ifs, line ) && !line.empty() )
{
dataStr += line + '\n'; // append this line to the result
}
data = [NSString stringWithUTF8String:navDataStr.c_str()];
}
As I am doing a project in Swift, I am trying to get rid of getline and replace it with something "Cocoaish". But I cannot find a good Swift solution to address the above problem. If you have an idea, I would really appreciate it. Thanks!
Using the StreamReader class from Read a file/URL line-by-line in Swift, you could do that it Swift like this:
let searchKey = "A|a2"
let bundle = NSBundle.mainBundle()
let pathNav = bundle.pathForResource("data_apt", ofType: "txt")
if let aStreamReader = StreamReader(path: pathNav!) {
var dataStr = ""
while let line = aStreamReader.nextLine() {
if line.rangeOfString(searchKey, options: nil, range: nil, locale: nil) != nil {
dataStr = line + "\n"
break
}
}
if dataStr == "" {
dataStr = "DATA_DEFAULT"
} else {
while let line = aStreamReader.nextLine() {
if countElements(line) == 0 {
break
}
dataStr += line + "\n"
}
}
aStreamReader.close()
println(dataStr)
} else {
println("cannot open file")
}
I read from a file to a byte array:
auto text = cast(immutable(ubyte)[]) read("test.txt");
I can get the type of character encoding using the following function:
enum EncodingType {ANSI, UTF8, UTF16LE, UTF16BE, UTF32LE, UTF32BE}
EncodingType DetectEncoding(immutable(ubyte)[] data){
switch (data[0]){
case 0xEF:
if (data[1] == 0xBB && data[2] == 0xBF){
return EncodingType.UTF8;
} break;
case 0xFE:
if (data[1] == 0xFF){
return EncodingType.UTF16BE;
} break;
case 0xFF:
if (data[1] == 0xFE){
if (data[2] == 0x00 && data[3] == 0x00){
return EncodingType.UTF32LE;
}else{
return EncodingType.UTF16LE;
}
}
case 0x00:
if (data[1] == 0x00 && data[2] == 0xFE && data[3] == 0xFF){
return EncodingType.UTF32BE;
}
default:
break;
}
return EncodingType.ANSI;
}
I need a function that takes a byte array and returns the text string (utf-8).
If the text is encoded in UTF-8, then the transformation is trivial. Similarly, if the encoding is UTF-16 or UTF-32 native byte order for the system.
string TextDataToString(immutable(ubyte)[] data){
import std.utf;
final switch (DetectEncoding(data[0..4])){
case EncodingType.ANSI:
return null;/*???*/
case EncodingType.UTF8:
return cast(string) data[3..$];
case EncodingType.UTF16LE:
wstring result;
version(LittleEndian) { result = cast(wstring) data[2..$]; }
version(BigEndian) { result = "";/*???*/ }
return toUTF8(result);
case EncodingType.UTF16BE:
return null;/*???*/
case EncodingType.UTF32LE:
dstring result;
version(LittleEndian) { result = cast(dstring) data[4..$]; }
version(BigEndian) { result = "";/*???*/ }
return toUTF8(result);
case EncodingType.UTF32BE:
return null;/*???*/
}
}
But I could not figure out how to convert byte array with ANSI encoded text (for example, windows-1251) or UTF-16/32 with NOT native byte order.
I ticked the appropriate places in the code with /*???*/.
As a result, the following code should work, with any encoding of a text file:
string s = TextDataToString(text);
writeln(s);
Please help!
BOMs are optional. You cannot use them to reliably detect the encoding. Even if there is a BOM, using it to distinguish UTF from code page encodings is problematic, because the byte sequences are usually valid (if nonsensical) in those, too. E.g. 0xFE 0xFF is "юя" in Windows-1251.
Even if you could tell UTF from code page encodings, you couldn't tell the different code pages from another. You could analyze the whole text and make guesses, but that's super error prone and not very practical.
So, I'd advise you to not try to detect the encoding. Instead, require a specific encoding, or add a mechanism to specify it.
As for trandscoding from a different byte order, example for UTF16BE:
import std.algorithm: map;
import std.bitmanip: bigEndianToNative;
import std.conv: to;
import std.exception: enforce;
import std.range: chunks;
alias C = wchar;
enforce(data.length % C.sizeof == 0);
auto result = data
.chunks(C.sizeof)
.map!(x => bigEndianToNative!C(x[0 .. C.sizeof]))
.to!string;
I have a NSData object coming back from my server, it varies in its content but sticks to a particular structure.
I would like to know (hopfully with some example code) how to work though this object to get the data I need out of it.
the structure of the data objects inside the objects are like this
leading value (UInt16) - (tells me what section of the response it is)
Size of string (UInt32) or number - (UInt32)
String (not null terminated) i.e. followed by the next leading value.
I have been reading through the Binary Data Programming Guide however that's only really showing me how to put my data into new NSData objects and accessing and compairing the bytes.
The thing I am stuck on is how do I say grab the info dynamically. Check the NSdata objects first leading value figure out if its string or int then get the string or int and move onto the next leading value..
any suggestions or example code would be really helpfull.. just stuck in abit of a mind block as I have never attempted anything like this in objective C.
Some of this depends on how your server is written to encode the data into what it is sending you. Assuming it is encoding the numeric values using standard network byte ordering (big-endian) you will want it converted to the correct byte-ordering for iOS (I believe that is always little-endian).
I would approach it something like this:
uint16_t typeWithNetworkOrdering, typeWithLocalOrdering;
uint32_t sizeWithNetworkOrdering, sizeWithLocalOrdering;
char *cstring = NULL;
uint32_t numberWithNetworkOrdering, numberWithLocalOrdering;
const void *bytes = [myData bytes];
NSUInteger length = [myData length];
while (length > 0) {
memcpy(&typeWithNetworkOrdering, bytes, sizeof(uint16_t));
bytes += sizeof(uint16_t);
length -= sizeof(uint16_t);
memcpy(&sizeWithNetworkOrdering, bytes, sizeof(uint32_t));
bytes += sizeof(uint32_t);
length -= sizeof(uint32_t);
typeWithLocalOrdering = CFSwapInt16BigToHost(typeWithNetworkOrdering);
sizeWithLocalOrdering = CFSwapInt32BigToHost(sizeWithNetworkOrdering);
if (typeWithLocalOrdering == STRING_TYPE) { // STRING_TYPE is whatever type value corresponds to a string
cstring = (char *) malloc(sizeWithLocalOrdering + 1);
strncpy(cstring, bytes, sizeWithLocalOrdering);
cstring[sizeWithLocalOrdering] = '\0';
NSString *resultString = [NSString stringWithCString:cstring encoding:NSUTF8StringEncoding];
NSLog(#"String = %#", resultString);
free(cstring);
bytes += sizeWithLocalOrdering;
length -= sizeWithLocalOrdering;
// Do whatever you need to with the string
}
else if (typeWithLocalOrdering == NUMBER_TYPE) { // NUMBER_TYPE is whatever type value corresponds to a number
memcpy(&numberWithNetworkOrdering, bytes, sizeof(uint32_t));
numberWithLocalOrdering = CFSwapInt32BigToHost(numberWithNetworkOrdering);
NSLog(#"Number = %u", numberWithLocalOrdering);
bytes += sizeof(uint32_t);
length -= sizeof(uint32_t);
// Do whatever you need to with the number
}
}
Define your own internal structs and cast the pointer to it:
NSData* data;
struct headerType
{
uint16_t type;
uint32_t length;
};
const struct headerType* header=(const struct headerType*)[data bytes]; // get the header of the response
if (header->type==1)
{
const char* text=((const char*)header)+6; // skip the header (16bits+32bits=6 bytes offset)
}
EDIT:
If you need to read them in a loop:
NSData* data;
const uint8_t* cursor=(const uint8_t*)[data bytes];
while (true)
{
uint16_t type=*((uint16_t*)cursor);
cursor+=2;
if (cursor==1)
{
// string
uint32_t length=*((uint32_t*)cursor);
cursor+=4;
const char* str=(const char*)cursor;
cursor+=length;
}
else if (cursor==2)
{
// another type
}
else
break;
}
Test Case '-[TestParse testParsing]' started.
/Developer/Tools/RunPlatformUnitTests.include: line 415: 3256 Segmentation fault "${THIN_TEST_RIG}" "${OTHER_TEST_FLAGS}" "${TEST_BUNDLE_PATH}"
/Developer/Tools/RunPlatformUnitTests.include:451: error: Test rig '/Developer/Platforms /iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.2.sdk/Developer/usr/bin/otest'
exited abnormally with code 139 (it may have crashed).
I got this seg fault message while I built test case randomly (sometime it built successfully, sometimes it throws seg fault). I'm not sure how I could fix this error.
Only thing I test here is I wrote one class name Parse with class level method. And in test case I just call it like
var = [Parse methodName:filepath];
method is like this
NSMutableDictionary *tempBox = [[NSMutableDictionary alloc] init];
FILE *fd = fopen([filePath UTF8String], "r");
if(!fd){
NSLog(#"fail to open file\n");
}
char buf[4096], *ptr;
char name[512], description[4096];
int isNewInfo = 2, description_continue = 0;
// for (line = 0; line < [args objectAtIndex:1]; line++) {
// fgets(buf, 4096, fd);
// }
while(fgets(buf, sizeof(buf), fd) != NULL){
if(strcmp(buf, "\n") == 0){
isNewInfo -= 1;
if(isNewInfo == 0){
isNewInfo = 2;
description_continue = 0;
description[strlen(description)-1] = '\0';
[self saveDrinkandResetBuf:name
detail:description box:tempBox];
if(name[0] != 0 || description[0] != 0){
NSLog(#"fail to reset...");
}
}
}
if(description_continue){
strcat(description, buf);
continue;
}
if((ptr = strstr(buf, "Drink Name: "))){
memcpy(name, buf+12, strlen(buf));
name[strlen(name)] = '\0';
continue;
}
if((ptr = strstr(buf, "Description: "))){
memcpy(description, buf+13, strlen(buf));
description_continue = 1;
continue;
}
}
fclose(fd);
NSLog(#"finish parsing section\n");
//[tempBox release];
return tempBox;
Not sure what is going on here..
I suppose, the problem is in array management.
In C if the array is declared in a function (and is not declared as a global or static one), then value of its elements is undefined. So your char description[4096] is filled with any values. And nobody said that '\0' will be there.
And the result of strlen(...) for non-null-terminated char string is not defined. It may result in a memory access violation, as it will keep counting until it reaches the first memory byte whose value is 0.
Moreover, when you call description[strlen(description)-1], strlen can return 0 (imagine that the first value, stored there initially was '\0' and your file was started with two empty lines [to reach this line of code]) - so array index will be -1...