Marklogic REST API wildcard search - rest

I need to create a paginated search for json documents in one collection.
Document structure:
{
"Id": "OBJ-0000",
"Title": "sample text",
"Visible": true
}
I created element range indexes on fields Id, Title and Visible, and search options xml configuration:
<?xml version="1.0" encoding="UTF-8"?>
<search:options xmlns:search="http://marklogic.com/appservices/search">
<search:constraint name="id">
<search:range facet="false" type="xs:string">
<search:json-property>Id</search:json-property>
</search:range>
</search:constraint>
<search:constraint name="title">
<search:range facet="false" type="xs:string">
<search:json-property>Title</search:json-property>
</search:range>
</search:constraint>
<search:constraint name="visible">
<search:value type="boolean">
<search:json-property>Visible</search:json-property>
</search:value>
</search:constraint>
<search:operator name="sort">
<search:state name="idAsc">
<search:sort-order direction="ascending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
<search:state name="idDesc">
<search:sort-order direction="descending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
<search:state name="titleAsc">
<search:sort-order direction="ascending">
<search:json-property>Title</search:json-property>
</search:sort-order>
<search:sort-order direction="descending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
<search:state name="titleDesc">
<search:sort-order direction="descending">
<search:json-property>Title</search:json-property>
</search:sort-order>
<search:sort-order direction="descending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
<search:state name="isvisibleAsc">
<search:sort-order direction="ascending">
<search:json-property>Visible</search:json-property>
</search:sort-order>
<search:sort-order direction="descending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
<search:state name="isvisibleDesc">
<search:sort-order direction="descending">
<search:json-property>Visible</search:json-property>
</search:sort-order>
<search:sort-order direction="descending">
<search:json-property>Id</search:json-property>
</search:sort-order>
</search:state>
</search:operator>
<search:return-results>true</search:return-results>
<search:return-metrics>false</search:return-metrics>
<search:transform-results apply="raw">
</search:transform-results>
<search:debug>false</search:debug>
</search:options>
After, I created POST request with reference to options file:
LATEST/search?format=json&pageLength=20&start=1&options=objectSearch
with structured query:
{
"query": {
"operator-state": {
"operator-name": "sort",
"state-name": "TitleAsc"
},
"and-query": {
"term-query": { "text": "*exam*" }
}
}
}
and it works. But, however, when I make a query with the parameter start=1, in "total" I get more results than it really does, but if I specified start=20 for example, "total" is calculated correctly. This happens only when I use the search parameters with the wildcard (*) in the first place in the search term and term contains more than 3 characters.
How I can fix this issue?

From the documentation :
https://docs.marklogic.com/search:search
The output of search:search returns a element, which in turn contains a total attribute. The value of the total attribute is an estimate, based on the index resolution of the query, and it is not filtered for accuracy. The accuracy of the index resolution depends on the index configuration of the database, on the query, and on the data being searched.
This means that you will only receive an accurate total when all of the necessary indexes are turned on for your query.
Using the recommended Wildcard Index settings here should help you get accurate totals : https://docs.marklogic.com/guide/search-dev/wildcard#id_14163
From your comment above, you appear to be missing a word lexicon in the codepoint collation.

Related

AutocompleteInput suggestions are not working

I'm working on a project in react-admin and am trying to use an AutocompleteInputs within ReferenceInputs for Filter in a List. The list is a product that has a relationship with two different users, a consumer and provider. I'm currently able to obtain the choices for consumers and providers and filter my product list by either.
However, my issues arises with the autosuggestion. It's simply not working. Typing in the AutocompleteInput will not filter the choices to select from for either consumers or providers. I have another AutocompleteArrayInput within an ArrayReferenceInput for providers that's setup similarly and the autosuggestion is working perfectly fine.
Any thoughts?
Here's the broken AutocompleteInput:
export const ProductFilter = props => (
<Filter {...props}>
<ReferenceInput label="Consumer" reference="consumers"
source="consumer_user_id" allowEmpty>
<AutocompleteInput source="id" optionText={FullNameRenderer} />
</ReferenceInput>
<ReferenceInput label="Provider" reference="providers"
source="provider_user_id" alwaysOn allowEmpty>
<AutocompleteInput optionText={FullNameRenderer} />
</ReferenceInput>
<DateRangeInput alwaysOn />
</Filter>
)
And this is the working AutocompleteArrayInput:
export const ConsumerEdit = props => (
<Edit {...props}>
<TabbedForm redirect="show">
<FormTab label="Providers" path="provider_assignmenmts">
<ReferenceArrayInput label='Assigned Providers'
reference='providers' defaultValue={[]}
source="provider_ids" allowEmpty>
<AutocompleteArrayInput optionText={FullNameRenderer}/>
</ReferenceArrayInput>
</FormTab>
</TabbedForm>
</Edit>
)
In the component <AutocompleteInput source = "id" optionText = {FullNameRenderer} /> the attribute source = "id" is superfluous.
Well, it looks like this one is a larger issue within react-admin.
https://github.com/marmelab/react-admin/issues/3098

Azure Media Services v3 create asset filter returns BadRequest

Azure Media Services v3 create asset filter returns BadRequest.
Both code and manifest are included below. Asset duration is 00:00:26.4330000
Have tried adjusting the tracks, no tracks.
Have tried adjusting the time parameters.
Any ideas on what is wrong?
Is it possible for the API to report more information on what is wrong?
Can a sample be created that shows how to use AssetFilters.CreateOrUpdate with and without track filters?
Code
var timingData = GetManifestTimingData(new Uri(asset.ManifestUrl));
Log.Info($"Timescale: {timingData.TimeScale} Offset: {timingData.TimestampOffset} clip: ( {TimeSpan.FromSeconds(startTimeSec)} - {TimeSpan.FromSeconds(endTimeSec)} ) as ( {TimeSpan.FromSeconds(startTimeSec)} to {TimeSpan.FromSeconds(endTimeSec)} ) {name} ----- {asset.ToAssetInfo()?.hashKey} {asset.Name} -- Manifest: {asset.PlaybackUrl}");
if (startTimeSec >= timingData.AssetDuration.TotalSeconds)
{
Log.Error($"Invalid start time: {startTimeSec} > {timingData.AssetDuration}");
return null;
}
if (endTimeSec >= timingData.AssetDuration.TotalSeconds)
{
endTimeSec = (int)Math.Floor(timingData.AssetDuration.TotalSeconds);
}
var startTimeSpan = timingData.OffSetAsTimeSpan + TimeSpan.FromSeconds(startTimeSec);
var endTimeSpan = timingData.OffSetAsTimeSpan + TimeSpan.FromSeconds(endTimeSec);
double timeScale = timingData.TimeScale ?? TimeSpan.TicksPerSecond;
var scaledStartTime = Convert.ToInt64(Math.Truncate(startTimeSpan.Ticks * (timeScale / TimeSpan.TicksPerSecond)));
var scaledEndTime = Convert.ToInt64(Math.Truncate(endTimeSpan.Ticks * (timeScale / TimeSpan.TicksPerSecond)));
var range = new PresentationTimeRange(scaledStartTime, scaledEndTime, 9223372036854775807, 0, (long)timeScale, false);
var tracks = new List<FilterTrackSelection>
{
new FilterTrackSelection(new List<FilterTrackPropertyCondition>
{
new FilterTrackPropertyCondition(FilterTrackPropertyType.Type, "Audio", FilterTrackPropertyCompareOperation.Equal),
}),
new FilterTrackSelection(new List<FilterTrackPropertyCondition>
{
new FilterTrackPropertyCondition(FilterTrackPropertyType.Type, "Video", FilterTrackPropertyCompareOperation.Equal),
}),
};
var firstQuality = new FirstQuality(128000);
var filterParams = new AssetFilter(null, name, FilterType, range, firstQuality, tracks);
var filter = _client.AssetFilters.CreateOrUpdate(ResourceGroup, AccountName, asset.Id, name, filterParams);
if (filter == null)
{
Log.Warn($"Failed to create filter {name} in {asset}");
return null;
}
return filter;
Manifest
<SmoothStreamingMedia MajorVersion="2" MinorVersion="2" Duration="264333333" TimeScale="10000000">
<StreamIndex Chunks="13" Type="video" Url="QualityLevels({bitrate})/Fragments(video={start time})" QualityLevels="1">
<QualityLevel Index="0" Bitrate="770760" FourCC="H264" MaxWidth="1280" MaxHeight="720" CodecPrivateData="000000016764001FACD9405005BA10000003001000000303C0F18319600000000168EBECB22C" />
<c t="0" d="20333333" r="2" />
<c d="20333334" />
<c d="20333333" r="2" />
<c d="20333334" />
<c d="20333333" r="2" />
<c d="20333334" />
<c d="20333333" r="2" />
<c d="20333334" />
<c d="20333333" />
</StreamIndex>
<StreamIndex Chunks="14" Type="audio" Url="QualityLevels({bitrate})/Fragments(aac_UND_2_127999={start time})" QualityLevels="1" Name="aac_UND_2_127999">
<QualityLevel AudioTag="255" Index="0" BitsPerSample="16" Bitrate="127999" FourCC="AACL" CodecPrivateData="1190" Channels="2" PacketSize="4" SamplingRate="48000" />
<c t="0" d="20053333" r="2" />
<c d="20053334" />
<c d="20053333" r="2" />
<c d="20053334" />
<c d="20053333" r="2" />
<c d="20053334" />
<c d="20053333" r="2" />
<c d="20053334" />
<c d="20053333" />
<c d="3626667" />
</StreamIndex>
<Protection>
<ProtectionHeader SystemID="B47B251A-2409-4B42-958E-08DBAE7B4EE9">
<ContentProtection schemeIdUri="urn:mpeg:dash:sea:2012" xmlns:sea="urn:mpeg:dash:schema:sea:2012">
<sea:SegmentEncryption schemeIdUri="urn:mpeg:dash:sea:aes128-cbc:2013" />
<sea:KeySystem keySystemUri="urn:mpeg:dash:sea:keysys:http:2013" />
<sea:CryptoPeriod IV="0xF6BCAD06C97D0FEC81B702C86183355B" keyUriTemplate="https://testurstream.keydelivery.westus.media.azure.net?kid=d6c1f008-d43f-4c60-926f-76ba613b7b4b" />
</ContentProtection>
</ProtectionHeader>
</Protection>
</SmoothStreamingMedia>
There was a known bug that we hit with Asset Filters that was fixed and rolled out to production now. Looks like you may have hit the same bug. The team is working on also fixing this issue in the .NET SDK and rolling it out.
For now, can you first delete any and all old filters created in v3 API. To fix the issue, you need to remove all old asset and global filters first from this account. We had a versioning mis-match on the filters when we moved from preview to GA that caused a mismatch (and thus the error message you are seeing.) You should be able to get a more detailed error message also if you look at the stacktrace. It typically would show a version conflict issue. Let us know what you see in the details of the error.
Carlos,
Looking at this today for you. Sorry for delay.
I'm using the Postman collection just to see what is going on with the .NET SDK exactly.
Looks like the following filer works fine. I think that the issue you are getting is related to the line of code that creates the new PresentationTimeRange. The values for presentaitonWindowDuration and liveBackoffDuration are not needed for a VOD filter. They are only used for live filters. Try using the following.
new PresentationTimeRange(scaledStartTime, scaledEndTime, null, null, (long)timeScale, null);
{
"properties": {
"presentationTimeRange": {
"startTimestamp": 0,
"endTimestamp": 264333333,
"presentationWindowDuration": null,
"liveBackoffDuration": null,
"timescale": 10000000,
"forceEndTimestamp": null
},
"tracks": [{
"trackSelections": [{
"property": "Type",
"operation": "Equal",
"value": "Video"
},
{
"property": "Type",
"operation": "Equal",
"value": "Audio"
}
]
}]
}
}
I'll double check to see if this is an issue when I run it through the .NET SDK.
I think that our issue here may be that the values for presentationTimeRange are all marked as "REQUIRED" in the current Swagger file here:
https://github.com/Azure/azure-rest-api-specs/blob/dec75495352902ebb6393d42c50465b6195f239d/specification/mediaservices/resource-manager/Microsoft.Media/stable/2018-07-01/AccountFilters.json#L63

Which Attribute Of The Queues is Referenced In Diverts?

The examples of diverts have the same name for the queue as the entry so I am not clear which I use for the divert reference. e.g.
<jms-destinations>
<jms-queue name="VendorQueue">
<entry name="queue/vendor"/>
<entry name="java:jboss/exported/queue/vendor"/>
<durable>false</durable>
</jms-queue>
</jms-destinations>
Is this correct?
jms-queue add --queue-address=monitoring --entries=queue/monitoring
/subsystem=messaging/hornetq-server=default/divert=my-divert:add(divert-address=jms.queue.VendorQueue,forwarding-address=jms.queue.monitoring,exclusive=false)
i.e. is the divert-address supposed to be jms.queue.VendorQueue or jms.queue.vendor?
To get the correct divert-address you need to run this
/subsystem=messaging/hornetq-server=default:read-children-names(child-type="jms-queue")
Then prefix jms.queue
e.g.
{
"outcome" => "success",
"result" => [
"VendorQueue",
"monitoring"
]
}
becomes
jms.queue.VendorQueue
In reference to the xml, the jms-queue name is the attribute you reference not the entry name.

Image formats of theme not recognized

I want to add Thumbnail-Settings for my theme.
I'm using the default theme for the moment.
There is already a image-formats.xml in the themes config.
<?xml version="1.0" encoding="UTF-8"?>
<formats xmlns="http://schemas.sulu.io/media/formats"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://schemas.sulu.io/media/formats http://schemas.sulu.io/media/formats-1.0.xsd">
<format>
<name>640x480</name>
<commands>
<command>
<action>resize</action>
<parameters>
<parameter name="x">640</parameter>
<parameter name="y">480</parameter>
</parameters>
</command>
</commands>
</format>
<format>
<name>1200x</name>
<commands>
<command>
<action>scale</action>
<parameters>
<parameter name="x">1200</parameter>
</parameters>
</command>
</commands>
</format>
</formats>
The corresponding config looks like that:
# LIIP Theme Configuration
liip_theme:
themes: ["default"]
active_theme: "default"
load_controllers: false
If i dump the images in my assets the new format (the 1200x) is not shown.
I tried to reload the config and clear the cache. I also cleared the web/uploads/media folder.
What else can it be?
Here is the dump of the image in the template.
As you see, my own format is not appearing.
Media {#2167 ▼
#url: "/media/12/download/1.png?v=1"
#formats: array:10 [▼
"400x400" => "/uploads/media/400x400/02/12-1.png?v=1"
"400x400-inset" => "/uploads/media/400x400-inset/02/12-1.png?v=1"
"200x200" => "/uploads/media/200x200/02/12-1.png?v=1"
"190x" => "/uploads/media/190x/02/12-1.png?v=1"
"170x170" => "/uploads/media/170x170/02/12-1.png?v=1"
"150x100" => "/uploads/media/150x100/02/12-1.png?v=1"
"100x100" => "/uploads/media/100x100/02/12-1.png?v=1"
"100x100-inset" => "/uploads/media/100x100-inset/02/12-1.png?v=1"
"50x50" => "/uploads/media/50x50/02/12-1.png?v=1"
"640x480" => "/uploads/media/640x480/02/12-1.png?v=1"
]
#locale: "de"
#version: null
#additionalVersionData: array:1 [▶]
#fileVersion: FileVersion {#2754 ▶}
#file: File {#2751 ▶}
#entity: Media {#2748 ▶}
}
FileVersionMeta {#2759 ▼
-id: 12
-title: "1"
-description: null
-copyright: null
-locale: "de"
-fileVersion: FileVersion {#2754 ▼
-name: "1.png"
-version: 1
-size: 105564
-mimeType: "image/png"
-storageOptions: "{"segment":"04","fileName":"1.png"}"
-storageType: null
-downloadCounter: 0
-created: DateTime {#2742 ▶}
-changed: DateTime {#2743 ▶}
-id: 12
-contentLanguages: PersistentCollection {#2761 ▶}
-publishLanguages: PersistentCollection {#2763 ▶}
-meta: PersistentCollection {#2758 ▶}
-file: File {#2751 ▶}
-tags: PersistentCollection {#2756 ▶}
-changer: User {#2430 ▶ …2}
-creator: User {#2430 ▶ …2}
-defaultMeta: FileVersionMeta {#2759}
-properties: "[]"
}
}
Image Formats are stored in the container by compiler pass in the sulu_media.image.formats parameter. When you add a new image format a cache delete for BOTH Kernels is needed.
app/console cache:clear -e yourenvironment
app/webconsole cache:clear -e yourenvironment
Seems that there was a Caching problem. I already deleted the cache-folder, but it didn't work. I restarted my VM and now it works. Semms to be a problem on my side :-)

Drools camel multiple ksession not firing

I am using drools 5.6. In camel-server.xml I have my route defined as:
<route>
<from uri="cxfrs://bean://rsServer"/>
<policy ref="droolsPolicy">
<unmarshal ref="xstream-json" />
<to uri="drools:node1" />
<marshal ref="xstream-json" />
</policy>
</route>
I am not defining a ksession. Then in my knowledge-service.xml I have 2 sessions defined (by user) that point to different DRL files. These import the same common DRL file
<drools:kbase id="kbase1" node="node1">
<drools:resources>
<drools:resource ref="ruletemplate" />
<drools:resource type="DRL" source="classpath:7.drl"/>
</drools:resources>
</drools:kbase>
<drools:ksession id="ksession-7" type="stateless" kbase="kbase1" node="node1"/>
<drools:kbase id="kbase2" node="node1">
<drools:resources>
<drools:resource ref="ruletemplate" />
<drools:resource type="DRL" source="classpath:12.drl"/>
</drools:resources>
</drools:kbase>
<drools:ksession id="ksession-12" type="stateless" kbase="kbase2" node="node1"/>
The idea is the ksession will be defined in the JSON request passed in:
{"batch-execution": { "lookup":"ksession-12", "commands": [ { "insert": { "out-identifier": ...
The server starts up and changing the lookup between 7 and 12 works. HOWEVER whatever lookup I run first is the only rule that actually fires. So if I start the server and run ksession-7 it will run successfully, then change to ksession-12 nothing happens (does not hit debugging in 12.drl)
If I then restart the server and run 12 first it will run successfully, then change to ksession-7 and nothing happens
I am confused by this and believe I am missing something. My goal is to have a common rule template with specific rules for each user. Can anyone help guide me why this is not being directed correctly (or not processing)
EDIT: Adding example
t1.drl:
package org.company.t1.rules
import org.test.*
rule "check patients age" dialect "mvel"
when
$p : Patient( age >= 10)
then
System.out.println("t1 - age greater then 10")
$ca = new Message("age greater then 10")
$p.alerts.add($ca)
end
t2.drl:
package org.company.t2.rules
import org.test.*
rule "check patients age" dialect "mvel"
when
$p : Patient( age >= 15)
then
System.out.println("t2 - age greater then 15")
$ca = new Message("age greater then 15")
$p.alerts.add($ca)
end
ruletemplate.drl:
package org.test
declare Message
text : String
end
declare Patient
id:String
age : Integer
alerts : java.util.ArrayList
end
rule "setup rule for alerts" dialect "mvel" salience 100
when
$p : Patient()
then
System.out.println("initialize array")
$p.alerts = new java.util.ArrayList();
end
knowledge-server.xml:
<drools:resource id="ruletemplate" type="DRL" source="classpath:ruletemplate.drl" />
<drools:kbase id="kbase5" node="node1">
<drools:resources>
<drools:resource ref="ruletemplate" />
<drools:resource type="DRL" source="classpath:t1.drl"/>
</drools:resources>
</drools:kbase>
<drools:kbase id="kbase6" node="node1">
<drools:resources>
<drools:resource ref="ruletemplate" />
<drools:resource type="DRL" source="classpath:t2.drl"/>
</drools:resources>
</drools:kbase>
<drools:ksession id="ksession5" type="stateless" kbase="kbase5" node="node1"/>
<drools:ksession id="ksession6" type="stateless" kbase="kbase6" node="node1"/>
First request that works:
{"batch-execution": { "lookup":"ksession5", "commands": [ { "insert": { "out-identifier": "patient", "object": { "org.test.Patient":[{"age":16}]}}},{"fire-all-rules": ""}]}}
{"execution-results":{"results":{"result":{"identifier":"patient","value":{"org.test.Patient":{"age":16,"alerts":{"org.test.Message":{"text":"age greater then 10"}}}}},"fact-handle":{"identifier":"patient","external-form":"0:2:537549803:537549803:2:DEFAULT"}}}}
Request 2: (doesnt work unless run first after server restart:
{"batch-execution": { "lookup":"ksession6", "commands": [ { "insert": { "out-identifier": "patient", "object": { "org.test.Patient":[{"age":16}]}}},{"fire-all-rules": ""}]}}
{"execution-results":{"results":{"result":{"identifier":"patient","value":{"org.test.Patient":{"age":16}}},"fact-handle":{"identifier":"patient","external-form":"0:2:552327108:552327108:2:DEFAULT"}}}}
Tailing catalina.out shows the debugger output for t1 as it was run first, but nothing on t2
What are you inserting? what kind of rules do you have in there?
Could be that the first run everything works well and the second is crashing in the back?