Manipulate Date/Time in RDF4J for Debugging - date

I'm using RDF4J 2.2.1 on Windows 10 Professional 64-bit. I will have some SPIN constructor rules which are sensitive to date/time. For example, I may want to compare a triple containing an xsd:dateTime datatype property to the output of SPARQL's built-in now() function. To debug this functionality, it would be convenient to manipulate RDF4J's perception of date/time somehow rather than manipulating the system clock. I'm aware that there is general commercial software (e.g. Solution Soft's "Time Machine") that can generally manipulate the perception of time for any Windows process. However, this software appears to be far too expensive for our little proof-of-concept project.
What I'd like to be able to do:
Set RDF4J's date/time to some arbitrary date/time value.
Have RDF4J's date/time proceed at real time speed or at some programmable faster speed during debugging.
Does anyone have suggestions for how to manipulate in this manner date/time for RDF4J? It would make my debugging of time-sensitive SPIN rules much more efficient. I'd prefer not to fight my PC's system clock since many other things depend on it. I suppose that running an entire virtual PC and debugging on the virtual PC is another option, but it seems there should be a simpler way.
Thanks.

You could accomplish this by implementing a custom SPARQL function and using that instead of the actual now() function. Call it mock_now() for example. Since you implement it, you have full control over its behavior.

I'm posting my solution to my question in hopes it might help others as a further example of a custom SPARQL function under RDF4J. I don't hold this out as en elegant solution (due to how I set test conditions), but it does work and meets my requirements. This solution extends the answer from #jeen_broekstra based on http://docs.rdf4j.org/custom-sparql-functions/...
I now have a custom implemented in the namespace defined by PREFIX soo: <http://www.disa.mil/dso/a2i/ontologies/PBSM/Sharing/SpectrumOperationsOntology#>as a function called soo:spectrumOpsDateTime() which can take either three or no arguments. The three arguments case allows setting the scaled date time as follows.
First argument: xsd:boolean... use system clock if true or use scaled clock if false
Second argument: xsd:dateTime (ignored if first argument is true)... the starting date/time for scaled clock operation
Third argument: xsd:double (ignored if first argument is true)... the scaled clock rate (e.g. 2.0 means the scaled clock runs faster, at twice real time)
If there are no arguments, soo:spectrumOpsDateTime() returns the scaled date/time or the system date/time depending on what the initial values in the Java code specify or what the last three-argument call specified. The SPARQL and SPIN code under test will use only the no-argument version. Test setup queries will set up the time conditions for particular tests.
Here's an example SPARQL setup query to set up a 2x speed starting this morning:
PREFIX soo: <http://www.disa.mil/dso/a2i/ontologies/PBSM/Sharing/SpectrumOperationsOntology#>
SELECT DISTINCT *
WHERE {
BIND(soo:spectrumOpsDateTime("false"^^xsd:boolean, "2017-08-22T10:49:21.019-05:00"^^xsd:dateTime, "2.0"^^xsd:double) AS ?testDateTime) .
}
Here's an example SPARQL query to get the scaled date/time:
PREFIX soo: <http://www.disa.mil/dso/a2i/ontologies/PBSM/Sharing/SpectrumOperationsOntology#>
SELECT DISTINCT *
WHERE {
BIND(soo:spectrumOpsDateTime() AS ?testDateTime) .
}
The single class used to implement this custom function is:
/**
*
*/
package mil.disa.dso.spo.a2i.nsc.sharing2025.scaledDateTime;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import org.eclipse.rdf4j.model.IRI;
import org.eclipse.rdf4j.model.Literal;
import org.eclipse.rdf4j.model.Value;
import org.eclipse.rdf4j.model.ValueFactory;
import org.eclipse.rdf4j.model.impl.SimpleValueFactory;
import org.eclipse.rdf4j.query.algebra.evaluation.ValueExprEvaluationException;
import org.eclipse.rdf4j.query.algebra.evaluation.function.Function;
/**
* Class for generating a configurable date/time clock that can either be a pass-through of the
* system clock or a scaled clock starting at a specified date/time running at a specified
* rate from that specified time (first call).
* #author Greg Cox of Roberson and Associates &copy Copyright 2017 Roberson and Associates, All Right Reserved
*
*/
public class DateTimeGenerator implements Function {
private static final String thisClassName = "RDF4JCustomSPARQLFunction." + DateTimeGenerator.class.getSimpleName();
private static final String thisClassFullName = DateTimeGenerator.class.getName();
private static final boolean errorMessages = true;
private static final boolean verboseMessages = true;
private double clockPace = 2.0; // the speed of the clock, 1.0 is real time, 2.0 is 2x real time (double speed)
private boolean useSystemClock = false; // flag to indicate whether to use scaled clock or pass through the system clock
private ZonedDateTime startingRealDateTime = null; // the real time stamp at the first call to the evaluate function
private ZonedDateTime startingScaledDateTime = // the scaled time stamp (starting scaled time) at the first call to the evaluate function
ZonedDateTime.parse("2016-08-21T17:29:37.568-05:00");
// define a constant for the namespace of custom function
private static String NAMESPACE = "http://www.disa.mil/dso/a2i/ontologies/PBSM/Sharing/SpectrumOperationsOntology#"; // defined as soo: elsewhere
// this is the evaluate function needed to implement the RDF4J Function interface
// it can take 0 or 3 arguments
// 0 - get the current scaled time (starting by first call)
// 3 - useSystemClock flag (true/false), starting date/time (xsd:dateTime), clock pace (non-negative real w/ 1.0 meaning 1sec = 1sec)
#SuppressWarnings("unused")
#Override
public Value evaluate(ValueFactory valueFactory, Value... args) throws ValueExprEvaluationException {
String thisMethodMessagePrefix = "";
if (errorMessages || verboseMessages ) {
String thisMethodName = ".evaluate: ";
thisMethodMessagePrefix = thisClassName + thisMethodName;
}
if (args.length == 3) {
// Three arguments --> attempting to set mode/parameters, so attempt to parse/check them
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "attempting to set scaled clock mode/parameters");
boolean argErrFlag = false;
boolean newUseSystemClock = false;
String argErrMessage = "";
// first argument should be true/false on whether to use system clock (true) or scaled clock (false)
if (!(args[0] instanceof Literal)) {
argErrFlag = true;
argErrMessage += "first argument must be a literal true/false value... ";
} else {
String useSystemClockString = args[0].stringValue();
if (useSystemClockString.equalsIgnoreCase("true")) {
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "use system clock specified");
newUseSystemClock = true;
} else if (useSystemClockString.equalsIgnoreCase("false")) {
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "use scaled clock specified");
newUseSystemClock = false;
}
else {
argErrFlag = true;
argErrMessage += "first argument must be a literal true/false value... ";
}
}
// second argument should be starting date/time for scaled clock (ignore if using system clock)
ZonedDateTime startTime = null;
if (!newUseSystemClock) {
if (!(args[1] instanceof Literal)) {
argErrFlag = true;
argErrMessage += "second argument must be literal xsd:dateTime value for start of scaled date/time... ";
} else {
String startDateTimeString = args[1].stringValue();
try {
startTime = ZonedDateTime.parse(startDateTimeString);
} catch (Exception e) {
argErrFlag = true;
argErrMessage += "could not parse starting date/time... " + e.getMessage() + "... ";
}
}
}
// third argument should be clock pace for scaled clock (ignore if using system clock)
Double newClockPace = null;
if (!newUseSystemClock) {
if (!(args[2] instanceof Literal)) {
argErrFlag = true;
argErrMessage += "third argument must be literal xsd:double value for clock pace... ";
} else {
String clockPaceString = args[2].stringValue();
try {
newClockPace = Double.parseDouble(clockPaceString);
} catch (Exception e) {
argErrFlag = true;
argErrMessage += "could not parse clock pace which should be a positive xsd:double... ";
}
if ((newClockPace != null) && (newClockPace <= 0.0)) {
argErrFlag = true;
argErrMessage += "clock pace must be positive, got " + newClockPace + "... ";
}
}
}
// check for errors and set up the generator if no errors...
if (argErrFlag) {
if (errorMessages) System.err.println(thisMethodMessagePrefix + "ERROR - " + argErrMessage);
if (errorMessages) System.err.println(thisMethodMessagePrefix + "throwing exception...");
throw new ValueExprEvaluationException(
"spectrum operations time function soo:spectrumOpsDateTime() encountered errors in function arguments... " +
argErrMessage);
} else if (newUseSystemClock) {
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "using unscaled system clock");
useSystemClock = newUseSystemClock;
} else if (!newUseSystemClock) {
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "using scaled time");
useSystemClock = newUseSystemClock;
startingRealDateTime = ZonedDateTime.now();
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "setting starting real time to " + startingRealDateTime.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME));
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "setting start time to " + startTime.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME));
startingScaledDateTime = startTime;
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "setting clock pace to " + String.format("%5.2f", newClockPace * 100.0) + "%");
clockPace = newClockPace;
}
} else if (args.length != 0) { // can only have no arguments or three arguments...
throw new ValueExprEvaluationException(
"spectrum operations time function soo:spectrumOpsDateTime() requires "
+ "zero arguments or three arguments, got "
+ args.length + " arguments");
}
// now run the generator and return the result...
IRI xsdDateTimeIRI = valueFactory.createIRI("http://www.w3.org/2001/XMLSchema#dateTime"); // long-form equivalent to xsd:dateTime
if (useSystemClock) {
String unscaledTimeString = millisTrailingZeroes(ZonedDateTime.now().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME));
return valueFactory.createLiteral(unscaledTimeString, xsdDateTimeIRI);
} else {
errString = null;
String scaledTimeString = millisTrailingZeroes(getScaledDateTime().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME));
if (scaledTimeString == null) {
if (errorMessages) System.err.println(thisMethodMessagePrefix + "ERROR - scaled time returned null");
if (errorMessages) System.err.println(thisMethodMessagePrefix + "thowing exception...");
throw new ValueExprEvaluationException("could not generate valid scaled time string" + ((errString == null) ? "" : "... " + errString));
}
return valueFactory.createLiteral(scaledTimeString, xsdDateTimeIRI);
}
}
private static String errString = null;
/**
* Utility method to make all the millisecond fields of an <tt>ISO_OFFSET_DATE_TIME</tt> three digits by
* adding trailing zeroes as needed. Why? Because of trouble with various implementations interpreting
* 1 and 2 digit milliseconds differently. Should be standard decimal, but sometimes interpreted
* as number of milliseconds (e.g. .39T interpreted as 39 millieconds inststead of 390 milliseconds)
* #param <tt>ISO_OFFSET_DATE_TIME</tt> string to check for millisecond field length
* #return <tt>ISO_OFFSET_DATE_TIME</tt> strnig with trailing zeroes in milliseconds field
* as require to make the field three digits or <tt>null</tt> on error
*/
private static String millisTrailingZeroes(String isoDateTimeString) {
if (isoDateTimeString == null) {
errString = "DateTimeGenerator.millisTrailingZeroes: got null isoDateTimeString argument, returning null...";
return null;
}
String[] ss_l1 = isoDateTimeString.split("\\."); // Example: 2017-08-18T13:01:05.39-05:00 --> 2017-08-18T13:01:05 AND 39-05:00
if (ss_l1.length != 2) {
errString = "DateTImeGenerator.millisTrailingZeros: first parsing split of isoDateTimeString=" + isoDateTimeString + " by '.' got unexpected number of parts=" + ss_l1.length;
return null;
}
String[] ss_l2 = ss_l1[1].split("-"); // 39-05:00 --> 39 AND 05:00
if (ss_l2.length != 2) {
errString = "DateTImeGenerator.millisTrailingZeros: second parsing split of " + ss_l1[1] + " by '-' got unexpected number of parts=" + ss_l2.length;
return null;
}
if (ss_l2[0].length() == 1) {
ss_l2[0] = ss_l2[0] + "00";
} else if (ss_l2[0].length() == 2)
ss_l2[0] = ss_l2[0] + "0"; // 39 --> 390
return ss_l1[0] + "." + ss_l2[0] + "-" + ss_l2[1]; // 2017-08-18T13:01:05.390-05:00
}
/**
* Method to get the current scaled date time according to the state of this DateTimeGenerator.
* If <tt>useSystemClock</tt> is <tt>true</tt>, then time is not
* scaled and system time is returned instead of scaled time.
* #return scaled date time if <tt>useSystemClock</tt> is <tt>true</tt> or
* system date time if <tt>useSystemClock</tt> is <tt>false</tt>
*/
private ZonedDateTime getScaledDateTime() {
ZonedDateTime scaledDateTime = null;
if (useSystemClock) {
scaledDateTime = ZonedDateTime.now();
} else {
if (startingRealDateTime == null)
startingRealDateTime = ZonedDateTime.now();
long realMillisFromFirstCall = ChronoUnit.MILLIS.between(startingRealDateTime, ZonedDateTime.now());
long scaledMillisFromFirstCall = (long) ((double) realMillisFromFirstCall * clockPace);
scaledDateTime = ChronoUnit.MILLIS.addTo(startingScaledDateTime, scaledMillisFromFirstCall);
}
return scaledDateTime;
}
#Override
public String getURI() {
return NAMESPACE + "spectrumOpsDateTime";
}
/**
* Test main method
* #param args command line arguments (ignored)
*/
#SuppressWarnings("unused")
public static void main(String[] args) {
String thisMethodMessagePrefix = "";
if (errorMessages || verboseMessages ) {
String thisMethodName = ".main: ";
thisMethodMessagePrefix = thisClassName + thisMethodName;
}
DateTimeGenerator testGen = new DateTimeGenerator();
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "custom SPARQL method URI: " + testGen.getURI());
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "fully-qualified class name: " + thisClassFullName);
ValueFactory testVF = SimpleValueFactory.getInstance();
Value testValues[] = new Value[0];
while (true) {
if (verboseMessages) System.out.println(thisMethodMessagePrefix + "scaled: " + testGen.evaluate(testVF, testValues).stringValue() +
" current real: " + millisTrailingZeroes(ZonedDateTime.now().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME)));
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
In my case, the jar file exported from Eclipse executes under my installation of Apache and resides at C:\Apache\apache-tomcat-8.5.15\webapps\rdf4j-server\WEB-INF\lib\ScaledDateTime.jar I restart the Apache server after replacing this jar file when I do mofifications.

Related

Plc4x library Modbus serial (RTU) get is not retrieving data

I am trying to write a sample program to retrieve temperature data from SHT20 temperature sensor using serial port with apache plc4x library.
private void plcRtuReader() {
String connectionString =
"modbus:serial://COM5?unit-identifier=1&baudRate=19200&stopBits=" + SerialPort.ONE_STOP_BIT + "&parityBits=" + SerialPort.NO_PARITY + "&dataBits=8";
try (PlcConnection plcConnection = new PlcDriverManager().getConnection(connectionString)) {
if (!plcConnection.getMetadata().canRead()) {
System.out.println("This connection doesn't support reading.");
return;
}
PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
builder.addItem("value-1", "holding-register:258[2]");
PlcReadRequest readRequest = builder.build();
PlcReadResponse response = readRequest.execute().get();
for (String fieldName : response.getFieldNames()) {
if (response.getResponseCode(fieldName) == PlcResponseCode.OK) {
int numValues = response.getNumberOfValues(fieldName);
// If it's just one element, output just one single line.
if (numValues == 1) {
System.out.println("Value[" + fieldName + "]: " + response.getObject(fieldName));
}
// If it's more than one element, output each in a single row.
else {
System.out.println("Value[" + fieldName + "]:");
for (int i = 0; i < numValues; i++) {
System.out.println(" - " + response.getObject(fieldName, i));
}
}
}
// Something went wrong, to output an error message instead.
else {
System.out.println(
"Error[" + fieldName + "]: " + response.getResponseCode(fieldName).name());
}
}
System.exit(0);
} catch (PlcConnectionException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
Connection is established with the device using serial communication. But it fails to get data and instead prints the below warning messages continously.
debugger hangs at below line:
PlcReadResponse response = readRequest.execute().get();
with below logs printing continuously.
2021-06-03-17:41:48.425 [nioEventLoopGroup-2-1] WARN io.netty.channel.nio.NioEventLoop - Selector.select() returned prematurely 512 times in a row; rebuilding Selector org.apache.plc4x.java.transport.serial.SerialPollingSelector#131f8986.
2021-06-03-17:41:55.080 [nioEventLoopGroup-2-1] WARN io.netty.channel.nio.NioEventLoop - Selector.select() returned prematurely 512 times in a row; rebuilding Selector org.apache.plc4x.java.transport.serial.SerialPollingSelector#48c328c5.
With same URL data (i.e baudrate,stopBits etc..) using modpoll.exe it works and returns the data over RTU. I am not sure what is missing here.
Kindly shed some light here.

STM32 Read event from remoteControlEvent_t and parse data. Values are wrong with passing

I have small problem with adding and reading values.
Define variables
#define ADC_BIT_MASK 0x0FFF
static TaskHandle_t remoteControlTaskHandle = NULL;
typedef enum
{
...
rcEvent_FreshADC = 0x80000000,
...
}
Code for notify task. adc12bitVal_pedal value is for example 100 and adc12bitVal_lr value for example 1000. I am shifting adc12bitVal_lr to the left that I can pass params ...
static void remoteControl_FreshADC(uint16_t adc12bitVal_pedal, uint16_t adc12bitVal_lr)
{
if(remoteControlTaskHandle != NULL)
{
xTaskNotify(remoteControlTaskHandle,
(rcEvent_FreshADC | (adc12bitVal_pedal & ADC_BIT_MASK) | (adc12bitVal_lr & ADC_BIT_MASK)<<12),
eSetValueWithOverwrite);
}
}
and then remoteControlHandleEvent which handle an event. Here I have problem with adcVal_lr which should be 1000 but is for example 52123. I need to shift 12 to the left that I get correct value. Or this is wrong?
returnCode_t remoteControlHandleEvent(remoteControlEvent_t event)
{
if(event & rcEvent_motorControlACK)
{
uint8_t ackNum = (uint8_t) ( ((event >> MOTOR_CONTROL_ACK_BIT_POS) & MOTOR_CONTROL_ACK_BIT_MASK));
printf("CONF: %u\n", (unsigned int)ackNum);
}
if(event & rcEvent_FreshADC)
{
...
// Value is 100
uint16_t adcVal_pedal = (uint16_t)(event & ADC_BIT_MASK);
// DOESN'T WORK VALUE IS 52123 instead of 1000
uint16_t adcVal_lr = (uint16_t)((event & ADC_BIT_MASK)<<12);
...
}
}
I don't understand and know why wrong value for
uint16_t adcVal_lr = (uint16_t)((event & ADC_BIT_MASK)<<12);
Thnak you for all comments and help.
You are shifting left (up) twice. To extract a value that you shifted left you need to shift right (down). You also apply the mask before shifting, when it should be after.
uint16_t adcVal_lr = (uint16_t)((event >> 12) & ADC_BIT_MASK);

How to use simulation time to triger action in modules?

I want to create simulation with server and 2 nodes. Nodes have defined vector that contain times for turn on/off.(example timersOnOff = 5,10,13,25 … nod will turn on in 5th second of beginning simulation, and then will be shutdown in 10th seconds etc). How to trigger action at certain time to send msg to node to "turn on" or "turn off".?
Let's assume that these times are written in timersOnOff declared as:
std::vector<simtime_t> timersOnOff;
In initialize() add the following code:
for (int i = 0; i < timersOnOff.size(); i = i + 2) {
simtime_t timeOn = timersOnOff[i];
simtime_t timeOff = timersOnOff[i+1];
cMessage * msgOn = new cMessage("nodeOn"); // (1)
cMessage * msgOff = new cMessage("nodeOff"); // (2)
scheduleAt (timeOn, msgOn);
scheduleAt (timeOff, msgOff);
}
The above code schedules all ON and OFF events.
Then, in handleMessage() add:
if (msg->isSelfMessage()) {
if (msg->isName("nodeOn")) { // the same name as in (1)
delete msg;
// turning on the node
} else if (msg->isName("nodeOff")) { // the same name as in (2)
delete msg;
// turning off the node
}
} else {
// ...
}

Blank results while using Tokens Regex rules to identify Named Entities

I am struggling with writing the correct rule which involves macros to identify organizations in a text.
To Identify Matrix Inc. in:
With it's rising share prices Matrix Inc. has come out a winner this quarter.
I am trying to check for words like Inc within the entity and thus defined a macros and rule as below:
$ORGANIZATION_TITLES = "/pharmaceuticals?|group|corp|corporation|international|co.?|inc.?|incorporated|holdings|motors|ventures|parters|llc|limited liability corporation|pvt.? ltd.?/"
ENV.defaults["stage"] = 1
{
ruleType: "tokens",
pattern: ([$ORGANIZATION_TITLES]),
action: ( Annotate($0, ner, "ORGANIZATION") )
}
ENV.defaults["stage"] = 2
{ ( [{tag:NNP}]+? ($ORGANIZATION_TITLES)) => ORGANIZATION }
I tried using bindings also and then applying the rule.
env.bind("$ORGANIZATION_TITLES", TokenSequencePattern.compile(env,"/pharmaceuticals?|group|corp|corporation|international|co.?|inc.?|incorporated|holdings|motors|ventures|parters|llc|limited liability corporation|pvt.? ltd.?/"));
Nothing seems to be working. I need to define more complex pattern rules involving macros like:
pattern: ( [ { ner:PERSON } ]+ /,/*? ($TITLES_CORPORATE_PREFIXES)*? $TITLES_CORPORATE+? /,/*? /of|for/? /,/*? [ { ner:ORGANIZATION } ]+ )
where $TITLES_CORPORATE_PREFIXES and $TITLES_CORPORATE are macros similar to $ORGANIZATION_TITLES.
What am I doing wrong?
EDIT
Here's my code:
public static void main(String[] args)
{
String rulesFile = "D:\\Workspace\\resource\\NERRulesFile.txt";
String dataFile = "D:\\Workspace\\resource\\GoldSetSentences.txt";
Properties props = new Properties();
props.put("annotators", "tokenize, ssplit, pos, lemma");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
// pipeline.addAnnotator(new TokensRegexAnnotator(rulesFile));
String inputText = "Bill Edelman , CEO and Chairman , for Paragonix commented on the Supply Agreement with Essential Pharmaceuticals .";
Annotation document = new Annotation(inputText.toLowerCase());
pipeline.annotate(document);
List<CoreMap> sentences = document.get(SentencesAnnotation.class);
CoreMapExpressionExtractor extractor = CoreMapExpressionExtractor.createExtractorFromFiles(TokenSequencePattern.getNewEnv(), rulesFile);
/* Next we can go over the annotated sentences and extract the annotated words,
Using the CoreLabel Object */
for (CoreMap sentence : sentences)
{
List<MatchedExpression> matched = extractor.extractExpressions(sentence);
for(MatchedExpression phrase : matched){
// Print out matched text and value
System.out.println("matched: " + phrase.getText() + " with value " + phrase.getValue());
// Print out token information
CoreMap cm = phrase.getAnnotation();
for (CoreLabel token : cm.get(TokensAnnotation.class))
{
String word = token.get(TextAnnotation.class);
String lemma = token.get(LemmaAnnotation.class);
String pos = token.get(PartOfSpeechAnnotation.class);
String ne = token.get(NamedEntityTagAnnotation.class);
System.out.println("matched token: " + "word="+word + ", lemma="+lemma + ", pos=" + pos + "ne=" + ne);
}
}
}
}
Here is a rules file that should work:
ner = { type: "CLASS", value: "edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation" }
$ORGANIZATION_TITLES = "/inc\.|corp\./"
{ pattern: ([{pos: NNP}]+ $ORGANIZATION_TITLES), action: ( Annotate($0, ner, "RULE_FOUND_ORG") ) }
I have made some changes to our code base to make the TokensRegexAnnotator more easily accessible. You will need to get the latest version from GitHub: https://github.com/stanfordnlp/CoreNLP
java -Xmx8g edu.stanford.nlp.pipeline.StanfordCoreNLP -annotators tokenize,ssplit,pos,lemma,ner,tokensregex -tokensregex.rules organization.rules -file samples.txt -outputFormat text -tokensregex.caseInsensitive
If you run this command or the equivalent Java API call it should work:

How do I replicate %Dictionary.ClassDefintionQuery's Summary() in SQL?

There is a procedure in %Dictionary.ClassDefinitionQuery which lists a class summary; in Java I call it like this:
public void readClasses(final Path dir)
throws SQLException
{
final String call
= "{ call %Dictionary.ClassDefinitionQuery_Summary() }";
try (
final CallableStatement statement = connection.prepareCall(call);
final ResultSet rs = statement.executeQuery();
) {
String className;
int count = 0;
while (rs.next()) {
// Skip if System is not 0
if (rs.getInt(5) != 0)
continue;
className = rs.getString(1);
// Also skip if the name starts with a %
if (className.charAt(0) == '%')
continue;
//System.out.println(className);
count++;
}
System.out.println("read: " + count);
}
}
In namespace SAMPLES this returns 491 rows.
I try and replicate it with a pure SQL query like this:
private void listClasses(final Path dir)
throws SQLException
{
final String query = "select id, super"
+ " from %Dictionary.ClassDefinition"
+ " where System = '0' and name not like '\\%%' escape '\\'";
try (
final PreparedStatement statement
= connection.prepareStatement(query);
final ResultSet rs = statement.executeQuery();
) {
int count = 0;
while (rs.next()) {
//System.out.println(rs.getString(1) + ';' + rs.getString(2));
count++;
}
System.out.println("list: " + count);
}
}
Yet when running the program I get this:
list: 555
read: 491
Why are the results different?
I have looked at the code of %Dictionary.ClassDefinitionQuery but I don't understand why it gives different results... All I know, if I store the names in sets and compare, is that:
nothing is missing from list that is in read;
most, but not all, classes returned by list which are not in read are CSP pages.
But that's it.
How I can I replicate the behaviour of the summary procedure in SQL?
Different is in one property. %Dictionary.ClassDefinitionQuery_Summary shows only classes with Deployed<>2. So, sql must be such.
select id,super from %Dictionary.ClassDefinition where deployed <> 2
But one more things is, why count may be different is, such sql requests may be compilled to temporary class, for example "%sqlcq.SAMPLES.cls22"