HAPI v2 after terser: get entire changed message - hapi

I have an HL7 message whose content I'm manipulating slightly with the terser.set() method. Once I've done that, I see in the debugger that it's been changed just how I want it, but I can't seem to get the whole message back intact. I've tried (for example):
HapiContext context = new DefaultHapiContext();
Parser parser = context.getGenericParser();
Message message = parser.parse( MESSAGE );
Terser terser = new Terser( message );
terser.set( "/PID-2", "XXX XX XXXX" );
String fixedMessage = message.encode();
...which gets me close, however, lines (segment lines) that ended in just vertical bars (pipes) with no values in their fields come back trimmed (the vertical bars are simply dropped). I want the message to remain identical to what I put in (if also modified where I did it on purpose).

I think you need to use addForcedEncode in the ParserConfiguration.
Like
#Test
public void testSetManualRepetitions() {
try {
String m = "MSH|^~\\&|hl7Integration|hl7Integration|||||ADT^A01|||2.3|\r" +
"EVN|A01|20130617154644\r" +
"PID|1|465 306 5961||407623|Wood^Patrick^^^MR||19700101|1||||||||||\r" +
"PV1|1||Location||||||||||||||||261938_6_201306171546|||||||||||||||||||||||||20130617134644|||||||||";
HapiContext hc = new DefaultHapiContext();
ExecutorService es = hc.getExecutorService(); // to avoid npe when closing context should be fixed https://sourceforge.net/p/hl7api/bugs/223/
ParserConfiguration pc = hc.getParserConfiguration();
PipeParser pipeParser = hc.getPipeParser();
Message message = pipeParser.parse(m);
Terser terser = new Terser(message);
//Add first Address
terser.set("/.PID-11(0)-1", "13 Oxford Road");
terser.set("/.PID-11(0)-3", "Oxford");
//Add second Address
terser.set("/.PID-11(1)-1", "16 London Road");
terser.set("/.PID-11(1)-3", "London");
pc.addForcedEncode("PID-26-1"); // make sure PID has 26 fields
System.out.println(message.encode().replaceAll("\r", "\r\n"));
hc.close();
} catch (HL7Exception e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
yields:
/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/bin/java -ea -Didea.launcher.port=7540 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar:/Applications/IntelliJ IDEA CE.app/Contents/plugins/junit/lib/junit-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/lib/tools.jar:/Users/thomas/git/Hapi-HL7-Terser/target/test-classes:/Users/thomas/.m2/repository/junit/junit/4.4/junit-4.4.jar:/Users/thomas/.m2/repository/ca/uhn/hapi/hapi-base/2.2/hapi-base-2.2.jar:/Users/thomas/.m2/repository/ca/uhn/hapi/hapi-structures-v21/2.2/hapi-structures-v21-2.2.jar:/Users/thomas/.m2/repository/ca/uhn/hapi/hapi-structures-v23/2.2/hapi-structures-v23-2.2.jar:/Users/thomas/.m2/repository/ca/uhn/hapi/hapi-structures-v24/2.2/hapi-structures-v24-2.2.jar:/Users/thomas/.m2/repository/ca/uhn/hapi/hapi-structures-v25/2.2/hapi-structures-v25-2.2.jar:/Users/thomas/.m2/repository/org/slf4j/slf4j-simple/1.6.0/slf4j-simple-1.6.0.jar:/Users/thomas/.m2/repository/org/slf4j/slf4j-api/1.6.0/slf4j-api-1.6.0.jar:/Users/thomas/.m2/repository/org/slf4j/log4j-over-slf4j/1.6.2/log4j-over-slf4j-1.6.2.jar" com.intellij.rt.execution.application.AppMain com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 com.hl7integration.hapi.tests.SetRepetitionsTerserTest,testSetManualRepetitions
68 [main] INFO ca.uhn.hl7v2.util.Home - hapi.home is set to /Users/thomas/git/Hapi-HL7-Terser/.
170 [main] INFO ca.uhn.hl7v2.VersionLogger - HAPI version is: 2.2
197 [main] INFO ca.uhn.hl7v2.VersionLogger - Default Structure libraries found for HL7 versions 2.1, 2.3, 2.4, 2.5,
MSH|^~\&|hl7Integration|hl7Integration|||||ADT^A01|||2.3
EVN|A01|20130617154644
PID|1|465 306 5961||407623|Wood^Patrick^^^MR||19700101|1|||13 Oxford Road^^Oxford~16 London Road^^London|||||||||||||||
PV1|1||Location||||||||||||||||261938_6_201306171546|||||||||||||||||||||||||20130617134644
Technically you are not changing the message, you parse it to a Java object and encode it back to string. So that your output message looks like your input message. There could still be subtle differences afterwards (e.g. if your input message is dynamic).
From the docs (http://hl7api.sourceforge.net/configuring_hapi.html):
Forced Encoding
By default, when encoding a message HAPI will not encode any segments or fields that have no content and therefore have no semantic meaning in the message.
This can cause problems if you need to transmit a message to a system that expects certain empty content to be present in order to get "hints" about where in the message it is.
The addForcedEncode method may be used to add Terser paths which should be forced to be encoded:
parser.getParserConfiguration().addForcedEncode("PATIENT_RESULT/ORDER_OBSERVATION/ORC-4");
// ORC-4 will still exist (but be empty) even if ORC has no content
String encoded = parser.encode(message);
See the JavaDoc for examples.

Related

Error reading from an IP-Camera

I capture the image from an IP-Camera and I work with the frames. My programm reads when there is a movement, and then, it makes a photo and save it on the computer.
It works perfectly at first, but when it is running like 2-3 hours, it usually get an error, and I do not find a explanation for this. Because, if it is an error on getting the image or the processing, it should happens since first, shouldn't it?
The error I get is the next:
Exception in thread "main" java.lang.NullPointerException
at com.googlecode.javacv.IPCameraFrameGrabber.grab(IPCameraFrameGrabber.java:105)
at Llamada.main(Llamada.java:34)
I have looked for the error nº105 but I have not found anything.
The program is the next:
public class Llamada {
public static void main(String[] args) throws Exception {
IPCameraFrameGrabber grabber = new IPCameraFrameGrabber("http://192.168.2.102:80/mjpg/video.mjpg");
//OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);
grabber.start();
IplImage frame = grabber.grab();
IplImage image = null;
IplImage prevImage = null;
IplImage diff = null;
Date data = new Date();
String output = "";
int i=0, j=0;
CanvasFrame canvasFrame = new CanvasFrame("IP Camera");
canvasFrame.setCanvasSize(frame.width(), frame.height());
CvMemStorage storage = CvMemStorage.create();
while (canvasFrame.isVisible() && (frame = grabber.grab()) != null) {
cvSmooth(frame, frame, CV_GAUSSIAN, 9, 9, 2, 2);
if (image == null) {
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
} else {
prevImage = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
prevImage = image;
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
}
if (diff == null) {
diff = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
}
if (prevImage != null) {
// perform ABS difference
cvAbsDiff(image, prevImage, diff);
// do some threshold for wipe away useless details
cvThreshold(diff, diff, 64, 255, CV_THRESH_BINARY);
canvasFrame.showImage(diff);
// recognize contours
CvSeq contour = new CvSeq(null);
cvFindContours(diff, storage, contour, Loader.sizeof(CvContour.class), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
while (contour != null && !contour.isNull()) {
if (contour.elem_size() > 0) {
output = data.toString();
if (data != null)
output = output.substring(0,10);
if(i%300 == 0)
cvSaveImage((j++)+" "+ output +"-capture.jpg", frame);
CvBox2D box = cvMinAreaRect2(contour, storage);
// test intersection
if (box != null) {
CvPoint2D32f center = box.center();
CvSize2D32f size = box.size();
}
i++;
}
contour = contour.h_next();
}
}
}
grabber.stop();
canvasFrame.dispose();
}
}
Thank you for everything!
Have you tried using a debugger and setting a break point? I understand that waiting around for 2-3 hours isn't fun, but maybe it'd help you get a handle on what's going on.
That seems to be in your while loop's second conditional part. Something inside the method grab on the grabber object is throwing a NullPointerException.
Probably the way you've initialized the grabber has led it to do this.
And it would be useful to know which version of the IPCameraFrameGrabber class you're using and what the author of that class really expected. Namely it's initialized to respond to a particular camera's url. In reading the class, it would appear this makes no claim to work with all IP cameras' MJPEG streams.
Let's look at one example comment in there:
foscam url http://host/videostream.cgi?user=username&pwd=password
http://192.168.0.59:60/videostream.cgi?user=admin&pwd=password
android ipcam http://192.168.0.57:8080/videofeed
And compare that to your url:
http://192.168.2.102:80/mjpg/video.mjpg
I gather it is not a foscam videostream.cgi url nor an android ipcam videofeed url, which would appear to be the only tested urls. It reminds me of an Axis camera url. More on that later.
In a recent version of that class (also in the older one actually), there seems to be some hackish attempt at reading only to the end of a subheader that is always delimited by crlfcrlf which could have been done just as well with a buffered input reader reading lines until it gets an empty line. What I do see here that seems likely to cause an npe is:
When your url's http server's response does not contain the content-length header, which is quite possible, the returned readImage() byte[] is null.
Since javax.imageio.ImageIO specifies that it will throw an IllegalArgumentException when it gets a null input, I'm guessing it's the ByteArrayInputStream constructor in the grabBufferedImage method that's throwing this, the IplImage.createFrom(null) in the old version, or the b.length in the newer version that is.
None of the line numbers of these versions line up with the error message you've shown that you're getting, so maybe your version of the library is yet again different, and broken differently. Try using the debugger, edit and patch the source of the IPCameraFrameGrabber to better support your mjpeg over http "device" based on what you find out is really in the input stream of the http response.
Since the url format reminds me of an Axis camera, I tried this with one running firmware v5.50 with the boa server built in:
$ curl -I http://user:pass#10.10.10.10:8080/mjpg/video.mjpg
HTTP/1.0 200 OK
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Connection: close
Content-Type: multipart/x-mixed-replace; boundary=myboundary
So you can see the content length is missing there. However, you do say you're getting frames initially for hours, then then, so I'm kind of at a loss with that part. I mean it sounds as though EITHER the input stream is getting closed, or the java implementation wrapping the stream, implemented in the http protocol handler, runs out of some kind of total space or open connection timer for some reason. I know this seems vague.
Another thing that seems odd is that from what I read in the two example classes of IPCameraFrameGrabber linked, every call to grab reads the input stream looking for headers first, which doesn't make sense to me right now, and I feel as though I must be misreading that.

How do I avoid truncation of message subjects at 255 chars with the EWS managed API?

I have an email messge on an Exchange server (2010 SP1) with a Subject header that is 272 characters long. Both Outlook and OWA show it truncated to the first 252 characters followed by "...". EWSEditor shows it the same way. I know, however, that the full Subject is stored somewhere, because when I look at the headers in Message Options dialog Outlook or in the Message Details in OWA, all 272 characters are there.
My code is only gettting the truncated Subject, and I need a way to get the full string.
My code is using SyncFolderItems to get a ChangeCollection of ItemChange objects. I have two code branches for this. One retrieves FirstClassProperties, and one retrieves IdOnly. I have a function called getItemStringProp(), and depending on the branch, I either call it directly with the Item that I get from the ItemChange, or with the Item that I get by binding to the ItemChange.Item.Id. In both cases, my getItemStringProp() uses Item.TryGetProperty() and returns a max of 255 characters for the Subject. If the actual subject is longer, then I get 252 chars followed by "...".
Here's my code from the branch doing SyncFolderItems with FirstClassProperties:
useIdOnly = false;
icc = exchange.SyncFolderItems(folderId, PropertySet.FirstClassProperties, null, syncFolderItemsBatchSize, SyncFolderItemsScope.NormalItems, result.getSyncState());
and from the other branch:
useIdOnly = true;
icc = exchange.SyncFolderItems(folderId, PropertySet.IdOnly, null, syncFolderItemsBatchSize, SyncFolderItemsScope.NormalItems, result.getSyncState());
Following this, I drill down to get the Subject:
foreach (ItemChange ic in icc)
{
if (!useIdOnly)
{
icSubject = getItemStringProp(ic.Item, EmailMessageSchema.Subject,"Subject", folderName,"");
}
else
{
PropertySet itemProps = new PropertySet(BasePropertySet.IdOnly);
itemProps.Add(EmailMessageSchema.Subject);
itemProps.Add(EmailMessageSchema.DateTimeSent);
itemProps.Add(EmailMessageSchema.ItemClass);
Item item = Item.Bind(exchange, ic.Item.Id, itemProps);
icSubject = getItemStringProp(item, EmailMessageSchema.Subject, "Subject", folderName, "");
}
}
And here's the function that gets the Subject:
private String getItemStringProp(Item item, PropertyDefinition propDef, String propName, String fName, String defaultValue)
{
// some debug logging code and error checks omitted
object prop = null;
String value = "";
try
{
if (item.TryGetProperty(propDef, out prop) && prop != null)
{
value = prop.ToString();
}
if (prop == null || value == null)
{
value = defaultValue;
}
}
return value;
}
By the way, I'm aware that neither Outlook (at least the 2007 version) nor OWA allows creation of a message with a Subject longer than 255 characters. The message in question came into Exchange via SMTP, and a Subject far longer than 255 characters is legal according to the RFCs.
Don't rely on Item.Bind(), sync, search, or any other operation in EWS to load up all of the properties you're looking for. Have you tried getting the item, then doing a .load(PropertySet) or ExchangeService.loadPropertiesForItems()? Some properties won't come through in various retrieval actions even if you specifically request them. Some may come through, but get truncated. What makes it more fun is that I don't think there's any documentation telling you exactly which operations will return which properties, so you get to guess and check. You have to load the property set after you retrieve the Item(s), so it's usually best to get the Item with the ID only, then load the property set.

Why code shows "Error 354 (net::ERR_CONTENT_LENGTH_MISMATCH): The server unexpectedly closed the connection."

I am building my HTTP WEB SERVER in JAVA.
If client request any file and that file is on that place in server, then server gives that file to client. I also made this code, and it works fine.
The part of code, that shows above functionality,
File targ = [CONTAINS ONE FILE]
PrintStream ps;
InputStream is = new FileInputStream(targ.getAbsolutePath());
while ((n = is.read(buf)) > 0) {
System.out.println(n);
ps.write(buf, 0, n);
}
But now to make my code optimized, I replace this code with below code,
InputStream is = null;
BufferedReader reader = null;
String output = null;
is = new FileInputStream(targ.getAbsolutePath());
reader = new BufferedReader(new InputStreamReader(is));
while( (output = reader.readLine()) != null) {
System.out.println("new line");
//System.out.println(output);
ps.print(output);
}
But it sometimes shows one error Why code shows "Error 354 (net::ERR_CONTENT_LENGTH_MISMATCH): The server unexpectedly closed the connection.". I didn't understand, why it shows this error. This error is very weird, because server shows 200 code, that means, that file is there.
Help me please.
Edit no. 1
char[] buffer = new char[1024*16];
int k = reader.read(buffer);
System.out.println("size : " + k);
do {
System.out.println("\tsize is : " + k);
//System.out.println(output);
ps.println(buffer);
}while( (k = reader.read(buffer)) != -1 );
This prints all the file, but for bigger files, it shows unreadable characters.
It shows below output (Snapshot of client browser)
You do output = reader.readLine() to get the data, which omits the newline characters. Then you ps.print(output), so the newline characters are not sent to the client.
Say you read this
Hello\r\n
World\r\n
Then you send this:
Content-length: 14
HelloWorld
And then close the connection, confusing the browser as it still was waiting for the other 4 bytes.
I guess you'll have to use ps.println(output).
You would have seen this if you were monitoring the network traffic, which can prove quite useful when writing or debugging a server that is supposed to communicate using the network.
Anyway this will cause trouble if the newlines of the file and the system have a mismatch (\n vs \r\n). Say you have this file:
Hello\r\n
World\r\n
Its length is 14 bytes. However when your system treats a newline when printing as \n, your code with println() will print this:
Hello\n
World\n
Which is 12 bytes, not 14. You better just print what you read.

NodeJS: What is the proper way to handling TCP socket streams ? Which delimiter should I use?

From what I understood here, "V8 has a generational garbage collector. Moves objects aound randomly. Node can’t get a pointer to raw string data to write to socket." so I shouldn't store data that comes from a TCP stream in a string, specially if that string becomes bigger than Math.pow(2,16) bytes. (hope I'm right till now..)
What is then the best way to handle all the data that's comming from a TCP socket ? So far I've been trying to use _:_:_ as a delimiter because I think it's somehow unique and won't mess around other things.
A sample of the data that would come would be something_:_:_maybe a large text_:_:_ maybe tons of lines_:_:_more and more data
This is what I tried to do:
net = require('net');
var server = net.createServer(function (socket) {
socket.on('connect',function() {
console.log('someone connected');
buf = new Buffer(Math.pow(2,16)); //new buffer with size 2^16
socket.on('data',function(data) {
if (data.toString().search('_:_:_') === -1) { // If there's no separator in the data that just arrived...
buf.write(data.toString()); // ... write it on the buffer. it's part of another message that will come.
} else { // if there is a separator in the data that arrived
parts = data.toString().split('_:_:_'); // the first part is the end of a previous message, the last part is the start of a message to be completed in the future. Parts between separators are independent messages
if (parts.length == 2) {
msg = buf.toString('utf-8',0,4) + parts[0];
console.log('MSG: '+ msg);
buf = (new Buffer(Math.pow(2,16))).write(parts[1]);
} else {
msg = buf.toString() + parts[0];
for (var i = 1; i <= parts.length -1; i++) {
if (i !== parts.length-1) {
msg = parts[i];
console.log('MSG: '+msg);
} else {
buf.write(parts[i]);
}
}
}
}
});
});
});
server.listen(9999);
Whenever I try to console.log('MSG' + msg), it will print out the whole buffer, so it's useless to see if something worked.
How can I handle this data the proper way ? Would the lazy module work, even if this data is not line oriented ? Is there some other module to handle streams that are not line oriented ?
It has indeed been said that there's extra work going on because Node has to take that buffer and then push it into v8/cast it to a string. However, doing a toString() on the buffer isn't any better. There's no good solution to this right now, as far as I know, especially if your end goal is to get a string and fool around with it. Its one of the things Ryan mentioned # nodeconf as an area where work needs to be done.
As for delimiter, you can choose whatever you want. A lot of binary protocols choose to include a fixed header, such that you can put things in a normal structure, which a lot of times includes a length. In this way, you slice apart a known header and get information about the rest of the data without having to iterate over the entire buffer. With a scheme like that, one can use a tool like:
node-buffer - https://github.com/substack/node-binary
node-ctype - https://github.com/rmustacc/node-ctype
As an aside, buffers can be accessed via array syntax, and they can also be sliced apart with .slice().
Lastly, check here: https://github.com/joyent/node/wiki/modules -- find a module that parses a simple tcp protocol and seems to do it well, and read some code.
You should use the new stream2 api. http://nodejs.org/api/stream.html
Here are some very useful examples: https://github.com/substack/stream-handbook
https://github.com/lvgithub/stick

Protovis - dealing with a text source

lets say I have a text file with lines as such:
[4/20/11 17:07:12:875 CEST] 00000059 FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:/Prgs/testing/WebSphere/AppServer/profiles/ProcCtr01/logs/ffdc/server1_3d203d20_11.04.20_17.07.12.8755227341908890183253.txt com.test.testserver.management.cmdframework.CmdNotificationListener 134
[4/20/11 17:07:27:609 CEST] 0000005d wle E CWLLG2229E: An exception occurred in an EJB call. Error: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
com.lombardisoftware.core.TeamWorksException: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
at com.lombardisoftware.server.ejb.persistence.CommonDAO.assertNotNull(CommonDAO.java:70)
Is there anyway to easily import a data source such as this into protovis, if not what would the easiest way to parse this into a JSON format. For example for the first entry might be parsed like so:
[
{
"Date": "4/20/11 17:07:12:875 CEST",
"Status": "00000059",
"Msg": "FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I",
},
]
Thanks, David
Protovis itself doesn't offer any utilities for parsing text files, so your options are:
Use Javascript to parse the text into an object, most likely using regex.
Pre-process the text using the text-parsing language or utility of your choice, exporting a JSON file.
Which you choose depends on several factors:
Is the data somewhat static, or are you going to be running this on a new or dynamic file each time you look at it? With static data, it might be easiest to pre-process; with dynamic data, this may add an annoying extra step.
How much data do you have? Parsing a 20K text file in Javascript is totally fine; parsing a 2MB file will be really slow, and will cause the browser to hang while it's working (unless you use Workers).
If there's a lot of processing involved, would you rather put that load on the server (by using a server-side script for pre-processing) or on the client (by doing it in the browser)?
If you wanted to do this in Javascript, based on the sample you provided, you might do something like this:
// Assumes var text = 'your text';
// use the utility of your choice to load your text file into the
// variable (e.g. jQuery.get()), or just paste it in.
var lines = text.split(/[\r\n\f]+/),
// regex to match your log entry beginning
patt = /^\[(\d\d?\/\d\d?\/\d\d? \d\d:\d\d:\d\d:\d{3} [A-Z]+)\] (\d{8})/,
items = [],
currentItem;
// loop through the lines in the file
lines.forEach(function(line) {
// look for the beginning of a log entry
var initialData = line.match(patt);
if (initialData) {
// start a new item, using the captured matches
currentItem = {
Date: initialData[1],
Status: initialData[2],
Msg: line.substr(initialData[0].length + 1)
}
items.push(currentItem);
} else {
// this is a continuation of the last item
currentItem.Msg += "\n" + line;
}
});
// items now contains an array of objects with your data