Read data with ILNumerics.IO.HDF5 - ilnumerics

I try ILNumerics.IO.HDF5 and can not read the following data:
Variable length strings in Datasets and Attributes.
Datasets with variable length arrays. Each cell contain a array of numbers, which are histograms.
Compound data, ie. Datasets with structs containing some numbers.
In HDFView 2.10.1 I can read this data:
https://anonfiles.com/file/13756916026cafc4e4ec7c333f235bda
How can I use ILNumerics.IO.HDF5 with this data?
I found an other post with suggestion to read string as char.
But with the variable length string an exception is thrown: "Error reading data from the attribute!"
var file = new H5File("test.h5");
H5Dataset ds1 = file.First<H5Dataset>("Wind");
var att = ds1.Attributes["Aggregator"];
var value = att.Get<char>();

Could you provide more info on how you write the string attributes and what exactly is the issue. When you say 'can not read',Do you get a null return value or do you get an exception.
I write strings as attributes in my application and it works fine. I am guessing there could be a problem in the way you write the string. As per Haymo's suggestion, I convert the string into char array and write as attribute. Here is the sample code
private ILRetArray<Char> ConvertStringToArray(string str)
{
using (ILScope.Enter())
{
ILArray<Char> A = ILMath.array<Char>(' ', 1, str.Length);
for (int i = 0; i < str.Length; i++)
{
A.SetValue(str[i], 0, i);
}
return A;
}
}
Test Case :
using (var file = new H5File("testwrite.h5"))
{
var ds = new H5Dataset("data", ILMath.rand(10,10));
file.Add(ds);
string teststr = "Test string";
ILArray<char> charStr = ConvertStringToArray(mystr);
ds.Attributes.Add(new H5Attribute("mystring",charStr));
//Read back the dataset and its attributes
var group = file.Find<H5Dataset>("data").First();
ILArray<Char> storedData = group.Attributes["mystring"].Get<Char>();
}

Related

Complex Dictionary Get Values as Strings With Conditions

I have built a survey using laanlabs/SwiftSurvey and I am able to get the results from the surveyComplete method as a dictionary.
The results return in a complex structure and I need to get the values for each response by the key tag of the questions array -> question object. This object contains an array of choices and within each choice object there is a key of selected. If the selected key's value is true (1) I need to get the text key's value that is in the same object. Some of these choices will have multiple selected keys with a value of true (1), if this is the case I'd like to concatenate the text key values with a comma in between the values.
The intention is then to insert the keys in to a SQLite database.
I am new to decoding dictionaries and traversing them in the correct way, I can access the dictionary print(dictionary) and also get into the the correct NSArray - print(dictionary["questions"] but from there I am stumped, could someone show me how please.
The results are below unfortunately its a large block apologies.
[
"version": 001,
"metadata":
{
"app_version" = "1.1";
build = 22;
debug = true;
},
"questions": <__NSArrayI 0x600000614d20>(
{
question = {
allowsMultipleSelection = 0;
choices = (
{
allowsCustomTextEntry = 0;
selected = 1;
text = Physician;
uuid = "224E1B76-D220-4068-AA22-6861E5F836CB";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = Dietitian;
uuid = "2DB2B6FB-E344-4BBF-A551-2FABE0DFF6AA";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = "Genetic Counsellor";
uuid = "A9BE7093-B95C-4BF4-B629-12FDA3154ABE";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = "Nurse/Nurse Practitioner/Physician Assistant";
uuid = "8E75A41B-0D8C-4ADA-A31C-2BC408F8269D";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = "Pharmacist / Pharmaceutical Industry";
uuid = "C943430D-EA48-4BCB-8ADF-011A223BDF36";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = "Academic/Researcher";
uuid = "E28377A4-37FC-4351-A857-88383A3D5A3B";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = "Patient/Patient Advocacy Group";
uuid = "E5836187-6C08-4272-A88E-40578F4FCF44";
},
{
allowsCustomTextEntry = 1;
selected = 0;
text = "Other (please specify)";
uuid = "EFF22342-48A9-4B8E-81A0-BB44D0E86EBC";
}
);
required = 1;
tag = "hcp-specialty";
title = "Please select the option that best describes your specialty:";
uuid = "7F77E248-8429-463E-9291-241B94BEE4F8";
};
type = 0;
},
{
question = {
autoAdvanceOnChoice = 1;
choices = (
{
allowsCustomTextEntry = 0;
selected = 1;
text = Yes;
uuid = "3C7A330D-F16B-4F3E-8ABC-6767A1A6332A";
},
{
allowsCustomTextEntry = 0;
selected = 0;
text = No;
uuid = "0E4F5360-FCCD-4860-9971-86E23BB8F6C1";
}
);
required = 1;
tag = "newborn-screening";
title = "Is newborn screening for classical homocystinuria available in your region/country?";
uuid = "F7C1A9D5-43AB-420D-80CF-F6644B95C73E";
};
type = 1;
},
{
question = {
feedback = "This is a free text response";
required = 1;
tag = "biggest-unmet-need";
title = "What do you believe is the biggest unmet need for patients living with classical HCU?";
uuid = "133E2EDC-8FF4-48D1-8BFA-3A20E5DA0052";
};
type = 4;
}
)
]
Based on their result example, you are getting JSON. Converting JSON to Dictionary is the easiest, of course, but also is the dirtiest. You are basically getting "whatever", so when it's time to use the data, you have to do a lot of - as you said - "decoding", validation, etc.
Instead of that, create a few Decodable structures that match your response. In this case you just need 3 structures:
struct Survey: Codable {
let questions: [Question]
}
struct Question: Codable {
let allowsMultipleSelection: Int?
let choices: [Choice]?
let required: Int
let tag: String
let title: String
let uuid: String
let feedback: String?
}
struct Choice: Codable {
let allowsCustomTextEntry: Int
let selected: Int
let text: String
let uuid: String
}
(I didn't verify every fields, you can adjust as needed. And you can omit any properties you don't need / don't care about.)
And then you decode it like this:
// Assume jsonData is your original JSON, which you currently decode as dictionary. So instead you do this:
let decoder = JSONDecoder()
let product = try decoder.decode(Survey.self, from: jsonData)
This approach allows you to
Most importantly, having a well-defined data makes working with database easier. You can even implement database encoder, based on your codable structures, which means you don't need to manually walk through columns of the database. Also when. you read from database, you get the same structures, no need to have 2 sets of rules / validations for database and dictionary you decoded.
This also allows you to be confident about data you decoded: it has proper names and types. You don't need to validate it (it was validated for you on decoding). You are in control which properties are required, which defaults to set, and so on. In more complicated cases, you may need to add manual decoding to your structures, but even then this manual decoding is inside the structure itself, easy to change / work with / test. All this instead of giant messy dictionary with "some stuff" in it.

Swift: Create Array of Pointers For Calling a C Function

Context
From Swift, I am trying to call a specific function of the libxlsxwriter C library. The documentation is here: https://libxlsxwriter.github.io/worksheet_8h.html#a62bf44845ce9dcc505bf228999db5afa
The function assembles a "rich string" (the equivalent of an AttributedString, where different formats/styles apply to different ranges) and writes it to a specific cell in the Excel worksheet. In C, the function works like this:
lxw_format *bold = workbook_add_format(workbook);
format_set_bold(bold);
lxw_format *italic = workbook_add_format(workbook);
format_set_italic(italic);
lxw_rich_string_tuple fragment11 = {.format = NULL, .string = "This is " };
lxw_rich_string_tuple fragment12 = {.format = bold, .string = "bold" };
lxw_rich_string_tuple fragment13 = {.format = NULL, .string = " and this is "};
lxw_rich_string_tuple fragment14 = {.format = italic, .string = "italic" };
lxw_rich_string_tuple *rich_string1[] = {&fragment11, &fragment12,
&fragment13, &fragment14, NULL};
worksheet_write_rich_string(worksheet, CELL("A1"), rich_string1, NULL);
The Problem
I have an array of lxw_rich_string_tuple structs, but I'm unclear how to convert this into the array of pointers that worksheet_write_rich_string() accepts:
// The worksheet object and `lxw_format` objects are already existent.
// This array contains multiple well-formed `lxw_rich_string_tuple` structs, which I can see in the debugger.
//
var rawTuples: [lxw_rich_string_tuple] = ...
// Parameters:
// - the worksheet on which to write this string
// - the row of the cell in which to write
// - the column of the cell in which to write
// - a null-terminated array of pointers to `lxw_rich_string_tuple` structs
// - an optional format object to use, null in this case.
//
worksheet_write_rich_string(worksheet, 0, 1, ?, NULL);
The trouble is the ?. I've tried all sorts of withUnsafeBytes and withUnsafeMutableBytes and UnsafeMutableRawPointer().bindMemory(to:capacity:) and I cannot figure out the magic Swift gibberish to do what is such a SIMPLE thing in C. Thanks.
My Attempt
This gives no compiler errors, but crashes with a Bad Access exception:
let argsSize: Int = rawTuples.count
rawTuples.withUnsafeMutableBufferPointer { rawTuplesPointer in
let ptr = UnsafeMutableRawPointer(rawTuplesPointer.baseAddress!).bindMemory(to: lxw_rich_string_tuple.self, capacity: argsSize)
var tuplePointers: [UnsafeMutablePointer<lxw_rich_string_tuple>?] = []
for i in 0 ..< argsSize
{
let tp: UnsafeMutablePointer<lxw_rich_string_tuple>? = ptr + (i * MemoryLayout<lxw_rich_string_tuple>.stride)
tuplePointers.append(tp)
}
tuplePointers.append(nil)
tuplePointers.withUnsafeBufferPointer { tpPointer in
let m = UnsafeMutablePointer(mutating: tpPointer.baseAddress)
worksheet_write_rich_string(lxw_worksheet, cell.row, cell.col, m, nil)
}
}
Try this:
rawTuples.withUnsafeMutableBufferPointer { p in
guard let arrBaseAddress = p.baseAddress else { return }
var pointersToEachArrayElement: [UnsafeMutablePointer<_>?] =
Array(arrBaseAddress ..< arrBaseAddress.advanced(by: p.count))
pointersToEachArrayElement.append(nil)
pointersToEachArrayElement.withUnsafeMutableBufferPointer { q in
// use q.baseAddress in the call to worksheet_write_rich_string
}
}
The idea is similar to your attempt - to create an array of pointers to each of the array elements. But unlike your attempt, I avoided calling the initialisers of the pointer types (which I don't think you are supposed to do), and instead tried to use the withXXX functions as much as I could.
You should also consider just writing Objective-C wrappers around the C function and lxw_rich_string_tuple. Sometimes C functions are just not bridged into Swift in a very convenient way, and this is not the first time I've experienced something like this, unfortunately :(

How to identify the type of an object?

Here is my JSON response for a particular API.
Case 1
ChallengeConfiguration = {
AnswerAttemptsAllowed = 0;
ApplicantChallengeId = 872934636;
ApplicantId = 30320480;
CorrectAnswersNeeded = 0;
MultiChoiceQuestion = (
{
FullQuestionText = "From the following list, select one of your current or previous employers.";
QuestionId = 35666244;
SequenceNumber = 1;
},
{
FullQuestionText = "What color is/was your 2010 Pontiac Grand Prix?";
QuestionId = 35666246;
SequenceNumber = 2;
}
)
}
The key "MultiChoiceQuestion" returns an array with two questions. So here is my code.
let QuestionArray:NSArray = dict1.objectForKey("ChallengeConfiguration")?.objectForKey("MultiChoiceQuestion") as! NSArray
Case 2
ChallengeConfiguration =
{
AnswerAttemptsAllowed = 0;
ApplicantChallengeId = 872934636;
ApplicantId = 30320480;
CorrectAnswersNeeded = 0;
MultiChoiceQuestion = {
FullQuestionText = "From the following list, select one of your
current or previous employers.";
QuestionId = 35666244;
SequenceNumber = 1;
}
}
For Case 2 my code does not work and app crashes because it returns a dictionary for that specific Key. So how could I write a generic code that would work for all objects?
It looks like the key can contain either an array of dictionary values or a dictionary, so you just need to try casting to see which one you have.
so I would likely do it like this:
if let arr = dict1.objectForKey("ChallengeConfiguration")?.objectForKey("MultiChoiceQuestion") as? Array {
// parse multiple items as an array
} else if let arr = dict1.objectForKey("ChallengeConfiguration")?.objectForKey("MultiChoiceQuestion") as? [String:AnyObject] {
// parse single item from dictionary
}
You should never really use ! to force unwrap something unless you are completely certain that the value exists and is of the type you are expecting.
Use conditional logic here to test the response and parse it safely so that your app doesn't crash, even in failure.

How to use a function or method on a Spark data frame column for transformation using Scala

I have created a function in scala equivalant to ORACLE DECODE function. I want to use the function with SPARK dataframes columns. I have tried it but getting multiple issues with Datatype mismatches.
I do not want to create UDF for each program. I want to create something generic and reuse it multiple times.
Function:
def ODECODE(column: Any, Param: Any*) : Any = {
var index = 0
while (index < Param.length) {
var P = Param(index)
var Q = column
if (P.equals(Q))
return Param(index + 1)
else index = index + 1
}
return Param (Param.length - 1)
}
I want to use it some thing like this:
Assuming "Emp" is a dataframe containing data from employee table with columns(first name, Last Name, Grade).
Emp.select(ODECODE("grade", "A", 1, "B", 2, "C", 3, "FAIL")).show()
This is one example. The datatype in the grade column can be String or Integer. So I have taken Datatypes in the decode function (Above) as ANY but with Dataframes it does not perform the Transformation. It gives datatype mismatches.
I want to create individual functions/Methods for some of the unsupported Oracle functions and reuse them where ever required in my transformations. So any suggestion to make this work is appreciated.
I know this is late, but I actually needed this and found your example. I was able to implement it with a few changes. I am no expert though, there may be a better way of doing this.
import util.control.Breaks._;
def ODECODE[T](column: String, params: Seq[T]) : String = {
try {
var index = 0;
breakable {
while (index < params.length) {
var P = params(index);
var Q = column;
if(P.equals(Q)) {
break;
}
index += 1;
}
}
params(index - 1).toString;
}catch {
case ife: Exception =>
ife.printStackTrace();
"0";
}
}
println(ODECODE("TEST", 0, "TEgST", 8, "***", 0))

xpages typeahead autocomplete

i couldnt do aautocopmlete edit box. i want to take names from another database. i wrote my code to typeahead's value list. but it dont work. i am using same server but different database.anybody help me ? here is my code:
//Getting the view containing a document for each of the employees
var searchView:NotesView = session.getDatabase("servername","test/application name.nsf")
.getView("viewname");
// Creating a Lotus Notes search query. Notice the reference to lupkey!
var query = "(FIELD Ad Soyad CONTAINS *" + lupkey +"*)";
// Creating an array to store hits in
var searchOutput:Array = ["å","åå"];
// Doing the actual search
var hits = searchView.FTSearch(query);
var entries = searchView.getAllEntries();
var entry = entries.getFirstEntry();
//Sort the array manually, since Notes doesn't want to sort them alphabetically
for (i=0; i<hits; i++) {
searchOutput.push(entry.getColumnValues()[0]);
entry = entries.getNextEntry();
}
searchOutput.sort();
// Build the resulting output HTML code
var result = "<ul><li><span class='informal'>Suggestions:</span></li></ul>";
var limit = Math.min(hits,20);
for (j=0; j<limit; j++) {
var name = searchOutput[j].toString();
var start = name.indexOfIgnoreCase(lupkey)
var stop = start + lupkey.length;
//Make the matching part of the name bold
name = name.insert("</b>",stop).insert("<b>",start);
result += "<li>" + name + "</li>";
}
result += "</ul>";
return result;
There are plenty of issues with your code:
the query can't return any result since your field has a space in it
Do you really need an FTSearch to return values and not a sorted view?
the typeahead -as the name suggest- presents values that match left to right and not somewhere substring. If you need that you need to roll your own typeahead function using Ajax
The typeahead function doesn't take a parameter, so your lupkey doesn't go anywhere. The function needs to return all values and XPages will do the matching
Instead of copying one by one into an array for sorting, copy the returning Vector() into a TreeSet(). This is one line, sorts it and removes duplicates
To get it working check this example based on dojo, previously asked here. You will need the REST control
i do it like that
var directoryTypeahead = function (searchValue:string) {
// update the following line to point to your real directory
//var directory:NotesDatabase = session.getDatabase(database.getServer(), "names.nsf");
var directory:NotesDatabase = session.getDatabase(database.getServer(), "org/test.nsf");
var allUsers:NotesView = directory.getView("SVFHP2");
var matches = {};
var includeForm = {
Person: true,
Group: true
}
searchValue = searchValue.replace("I","i")
var matchingEntries:NotesViewEntryCollection = allUsers.getAllEntriesByKey(searchValue, false);
var entry:NotesViewEntry = matchingEntries.getFirstEntry();
var resultCount:int = 0;
while (entry != null) {
var matchDoc:NotesDocument = entry.getDocument();
var matchType:string = matchDoc.getItemValueString("Form");
//if (includeForm[matchType]) { // ignore if not person or group
var fullName:string = matchDoc.getItemValue("Name").elementAt(0) + " " + matchDoc.getItemValue("Title").elementAt(0);
if (!(matches[fullName])) { // skip if already stored
resultCount++;
var matchName:NotesName = session.createName(fullName);
matches[fullName] = {
cn: matchName.getCommon(),
photo: matchDoc.getItemValueString("Photo"),
job: matchDoc.getItemValueString("sum"),
email: matchDoc.getItemValueString("email"),
}
}
// }
/*if (resultCount > 15) {
entry = null; // limit the results to first 10 found
}
else {*/
entry = matchingEntries.getNextEntry(entry);
//}
};
}