I am currently trying to read the files of an RDF4J triplestore from the universAAL platform and put them into an InfluxDB to merge the data from different smart living systems.
However, I have noticed that the individual index files of the Native repository are encrypted/unreadable (See image below).
Is there any experience from the community on how to get human readable content out of the RDF4J files (namespace, triples.prop, triples-cosp, triples-posc, triples-spoc, values.hash, values.dat, values.id) and merge them into another database?
The documentation of RDF4J did not help me here, so I could not create a decent export.
Encrypted File from Triplestore
The files are not encrypted, they're simply a binary format, optimized for efficient storage and retrieval, used by RDF4J's Native Store database implementation. They're not meant for direct manipulation.
The easiest way to convert them to readable RDF is to spin up a Native Store on top of them and then use the RDF4J API to query/export its data. Assuming you have a complete set of data files it should be as simple as something like this:
Repository rep = new SailRepository(new NativeStore(new File("/path/to/datafiles/");
try(RepositoryConnection conn = rep.getConnection()) {
conn.export(Rio.createWriter(RDFFormat.TURTLE, System.out));
}
finally {
rep.shutDown();
}
Obviously, replace System.out with a FileOutputstream if you want to write the data to file rather than the console. And change RDFFormat.TURTLE to something else if you want a different syntax format.
Related
I want to replicate the functionality of the IBM i Access Client Solutions "Printer Output" tool that is used to retrieve PDF's of spooled files from our IBM Db2 environment. Instead of a user interface, I want to replicate the functionality as an API.
I want to construct an API which takes inputs such as the filter parameters pictured below:
The output of the API would be PDF(s) of the printer output spooled files that match the parameters specified.
I figure that if I am able to access the i Access Printer Output tool, then I should be able to use my credentials to access the spool files using an API or something like that.
Where would I start in constructing something like this?
Also, are there any IBM guides that contain relevant information? I have looked but been unsuccessful. The Programmer's Toolkit is, also, not available with my version of i Access.
Also, I don't have developer roles, so if this is possible, it would need to be something that I can do with little authority within the IBM i servers and the Access client.
First off, IBM ACS is Java based. Thus everything it does can be found in the IBM Toolbox for Java, aka JTOpen aka JT400.
http://jt400.sourceforge.net/
Documentation https://www.ibm.com/docs/en/i/7.4?topic=java-toolbox
You're going to want to look at the reading a transformed spool file example
The transformation actually happens on the IBM i side, by specifying the appropriate workstation customization object, QCTXPDF in this case rather than the examples original QWPTIFFG4
// The following examples demonstrate how to set up a PrintParameterList to
// obtain different transformations when reading spooled file data. In the code
// segments that follow, assume a spooled file already exists on a server, and
// the createSpooledFile() method creates an instance of the SpooledFile class
// representing the spooled file.
// Create a spooled file
SpooledFile splF = createSpooledFile();
// Set up print parameter list
PrintParameterList printParms = new PrintParameterList();
printParms.setParameter(PrintObject.ATTR_WORKSTATION_CUST_OBJECT, "/QSYS.LIB/QCTXPDF.WSCST");
printParms.setParameter(PrintObject.ATTR_MFGTYPE, "*WSCST");
// Create a transformed input stream from the spooled file
PrintObjectTransformedInputStream is = splF.getTransformedInputStream(printParms);
I'm writing a spring-batch application with spring-boot support and I'm looking for a way to know which files were generated by MultiResourceItemWriter. The first solution I have in mind is to have a folder for only the files generated and check the content, but if there is something already implemented on spring-batch would be great!
The intention is to encrypt and then upload each file to an sftp server.
The file names generated by the MultiResourceItemWriter are the combination of the resource name + the suffix created by the ResourceSuffixCreator. For example, if you create the writer like the following:
MultiResourceItemWriter<String> writer = new MultiResourceItemWriter<>();
writer.setResource(new FileSystemResource(new File("data.txt")));
writer.setResourceSuffixCreator(index -> "part" + index);
Then the generated files will be data.txt.part1, data.txt.part2, etc.
MultiResourceItemWriter doesn't perform write directly but delegate this job to other components.
All those components are ResourceAwareItemWriterItemStream implementors so you may write a ResourceAwareItemWriterItemStreamDelegate, intercept setResource() method and store resource into current step execution-context as a collection.
If you want to pass this list of resources to next steps you may use an ExecutionContextPromotionListener.
I'm trying to create a web service in PHP that can deliver an SVG with reference to a PNG raster image. Both the data for the SVG as well as the binary PNG image come from a MySQL database on the server.
Option A: Encode the PNG data in base-64 and embed it directly in the SVG, such as:
<image xlink:href="data:image/png;base64,..."/>
Concerns: 30% heavier load than loading it as pure binary and noticeable delay when loading it with Postman (or is this just because of Postman).
Option B: Call the PNG data as binary and save it as a file on the file system, then call the SVG file, which would then reference the physical PNG file.
Concerns: Involvement of the file system (which implies I need to start managing physical files, expiration dates etc).
Is there perhaps another way that an SVG can reference the binary data on the fly without it having to be on the file system?
To accomplish something similar (in my case sending data for SVGs with additional data about each file as binary files, which are much smaller than sending xml, text, or json) - I use CBOR. In my case, I compress the SVG using LZString compression first, and add this along with additional data attributes to a JSON object. Then I convert the JSON object to CBOR. I think CBOR can handle your base 64 data without any need for conversion - more information about it is here: cbor.io
I found a PHP library for CBOR here: https://github.com/2tvenom/CBOREncode
This may not be the way to go at all for you, but I thought I'd throw it out there just in case.
how can I create file (PDF file for example) from binary stream I have stored in global? I have stream stored in caché global and I need to create and save the file created by the stream using ObjectScript.
Thanks :)
It is not so easy. There is only one official way to create pdf in Cache, and it is ZEN reports. With ZEN reports you could create not only pdf, also possible to make html, xlsx. ZEN Reports used Apache FOP for generating it, any other ways also possible, but you should do it only by yourself.
Or maybe I misunderstood you, and you mean that your binary stream already contains PDF, and you just want to save it to some file. If so, you just have to copy your globalstream to filestream, with code like this:
set fs=##class(%Stream.FileBinary).%New()
set fs.Filename="c:\temp.pdf"
set tSC=fs.CopyFrom(yourStream)
set tSC=fs.%Save()
I am looking for the best solution for custom file parsing for our enterprise import routines. I want to basically change one file format into a standard file format and have one routine that imports that data into the database. I need to be able to create custom scripts for each client since its difficult to get the customer to comply with a standard or template format. I have looked at PowerShell and Iron Python to do this so far but I am not sure this is the route I want to go. I have also looked at some tools such as Talend which is a drag and drop style tool which may or may not give me what I want as far as flexibility. We are a .NET shop and have created custom code to do this in the past but I need something that is quicker to create then coding custom parsing functions each time we get a new file format in.
Depending on the complexity and variability of your work, you should consider an ETL tool like SSIS (SQL Server Integration Services).
Python is wonderful for this kind of thing. That's why we use. Each new customer transfer is a new adventure and Python gives us the flexibility to respond quickly.
Edit. All python scripts that read files are "custom file parsers". Without an actual example, it's not sensible to provide a detailed example.
with open( "some file", "r" ) as source:
for line in source:
process( line )
That's about all there is to a "custom file parser". If you're parsing .csv or .xml files, then Python has modules for that. If you're parsing fixed-format files, you'd use string slicing operations. If you're parsing other files (X12? JSON? YAML?) you'll need appropriate parsers.
Tab-Delim.
from collections import namedtuple
RecordLayout = namedtuple('RecordLayout',['field1','field2','field3',...])
def process( aLine ):
record = RecordLayout( aLine.split('\t') )
...
Fixed Layout.
from collections import namedtuple
RecordLayout = namedtuple('RecordLayout',['field1','field2','field3',...])
def process( aLine ):
fields = ( aLine[:10], aLine[10:20], aLine[20:30], ... )
record = RecordLayout( fields )
...