Altering a Datomic schema using Datomisca - scala

How do I add isComponent to a Datomic attribute using the Datomisca library?
In Datomic, I would do the following:
{:db/id :person/favorite-food
:db/isComponent true
:db.alter/_attribute :db.part/db}

Unfortunately, I haven’t had time to add full support for schema alteration in Datomisca.
However, schema alteration is no different to any other transactions, so there should be no issue with building the transaction data that you describe above.
Entity.add(Namespace("person") / "favorite-food") (
Attribute.isComponent -> true,
Namespace("db.alter") / "_attribute" -> Partition.DB
)
What Datomisca is lacking is
http://docs.datomic.com/javadoc/datomic/Connection.html#syncSchema(long)
But a datomisca Connection is just datomic Connection, so you can still access that underlying API. I will endeavor to add the new sync APIs in the near future.
For future reference, the google group is a good place to ask questions like these, as I’m more likely to notice them (a colleague noticed your question).
https://groups.google.com/forum/?fromgroups#!forum/datomisca

Related

Read-Only Postgraphile-CLI API

I am currently implementing a public API for a Open Data Platform with Postgraphile creating the needed API for me. The API should be completly public, with no authentification whatsoever and because of that the API should only implement read-only queries. Has anyone found a possibility to use Postgraphile-CLI to only create read-only functionality?
So far I have sucessfully setup a Postgraphile-CLI API for my postgres databases, with a user that only has the "GRANT SELECT" for the schemas in Postgres. However, this doesn't seem to work for my use case, since I still can use the mutation in graphql and insert or delete data from my schemas.
Since I don't know too much about postgres database administration, I therefor wonder If it is possible to just not provide mutations with Postgraphile-CLI.
Kind regards
Grigorios
EDIT0: I have found the mistake with my Postgres database rights. That may solve that read-only problem, but If anybody knows an answer to the initial question, I would be curious to know anyway.
You have a number of options:
Use permissions, as you suggest, along with the --no-ignore-rbac option - you will have to ensure your database permissions are and remain correct (no default grants to the public role, for example) for this to work
Use PostGraphile's --disable-default-mutations (-M) option; this will stop the CRUD mutations being generated but won't prevent custom mutation functions from being exposed, if you have any
Skip the MutationPlugin via --skip-plugins graphile-build:MutationPlugin - this will prevent the Mutation type from being added to the schema in the first place, so no mutations can be added.
For a real belt-and-braces approach, why not all three?
postgraphile \
--no-ignore-rbac \
--disable-default-mutations \
--skip-plugins graphile-build:MutationPlugin

Is it possible to obtain the ServicePath when querying entities in Orion?

the question I'd like to ask was raised some time ago (FIWARE Orion: How to retrieve the servicePath of an entity?) but as far as I've seen, there is no final answer.
In short, I'd like to retrieve the service path of entities when I exec a GET query to /v2/entitites which returns multiple results.
In our FIWARE instance, we strongly rely on the servicePath element to differentiate between entities with the same id. It is not a good design choice but, unfortunately, we cannot change it as many applications use that id convention at the moment.
There was an attempt three years ago to add a virtual field 'servicePath' to the query result (https://github.com/telefonicaid/fiware-orion/pull/2880) but the pull request was discarded because it didn't include test coverage for that feature and the final NGSIv2 spec didn't include that field.
Is there any plan to implement such feature in the future? I guess the answer is no, what brings me to the next question: is there any other way to do it which does not involve creating subscriptions (we found that the initial notification of a subscription does give you that info, but the notification is limited to 1000 results, what is too low for the number of entities we want to retrieve, and it does not allow pagination either)?
Thanks in advance for your responses.
A possible workaround is to use an attribute (provided by the context producer application) to keep the service path. Somehow, this is the same idea of the builtin attribute proposed in PR #2880.

Microsoft Graph: Is it safe to assume that plannerTask.id and plannerTaskDetails.id are always the same?

In reading the Microsoft Graph documentation, we come across two related resource types:
plannerTask
plannerTaskDetails
Both plannerTask and plannerTaskDetails have an id attribute, which in both cases is 28 characters long and case-sensitive, with validation taken care of Microsoft.
In working with Microsoft Graph, I have noticed that for a given task, plannerTask.id === plannerTaskDetails.id. This makes sense, as there would be no need to define a new id for the details resource type, since this is a one-one relationship and MS may simply re-use the plannerTask id.
However in the documentation, the id's seem to refer to different things. I quote:
For plannerTask: "ID of the task"
For plannerTaskDetails: "ID of the task details"
This seems to leave the possibility of plannerTask.id !== plannerTaskDetails.id.
I was simply wondering if, in people's experience, if it is safe to assume what I noticed above, i.e. that the two id's are always the same, as it is unclear from working with MS Graph and reading the documentation. Perhaps someone working on MS Planner itself could elucidate.
You should not make this assumption. From the API perspective, they are different keys. Using Graph API, you don't really need to use the Task details id anywhere, and you can always refer to it as task abc's details.
The id values currently are coincidentally the same, but we are actively investigating some features which would require them to be different. If you take dependency on this, your app will break.
Yes, but if you are doing a patch call then the If-Match header which is Etag is going to be different. Because although task and task details have the same id they are different objects.

Marklogic REST API search for latest document version

We need to restrict a MarkLogic search to the latest version of managed documents, using Marklogic's REST api. We're using MarkLogic 6.
Using straight xquery, you can use dls:documents-query() as an additional-query option (see
Is there any way to restrict marklogic search on specific version of the document).
But the REST api requires XML, not arbitrary xquery. You can turn ordinary cts queries into XML easily enough (execute <some-element>{cts:word-query("hello world")}</some-element> in QConsole).
If I try that with dls:documents-query() I get this:
<cts:properties-query xmlns:cts="http://marklogic.com/cts">
<cts:registered-query>
<cts:id>17524193535823153377</cts:id>
</cts:registered-query>
</cts:properties-query>
Apart from being less than totally transparent... how safe is that number? We'll need to put it in our query options, so it's not something we can regenerate every time we need it. I've looked on two different installations here and the the number's the same, but is it guaranteed to be the same, and will it ever change? On, for example, a MarkLogic upgrade?
Also, assuming the number is safe, will the registered-query always be there? The documentation says that registered queries may be cleared by the system at various times, but it's talking about user-defined registered queries, and I'm not sure how much of that applies to internal queries.
Is this even the right approach? If we can't do this we can always set up collections and restrict the search that way, but we'd rather use dls:documents-query if possible.
The number is a registered query id, and is deterministic. That is, it will be the same every time the query is registered. That behavior has been invariant across a couple of major releases, but is not guaranteed. And as you already know, the server can unregister a query at any time. If that happens, any query using that id will throw an XDMP-UNREGISTERED error. So it's best to regenerate the query when you need it, perhaps by calling dls:documents-query again. It's safest to do this in the same request as the subsequent search.
So I'd suggest extending the REST API with your own version of the search endpoint. Your new endpoint could add dls:documents-query to the input query. That way the registered query would be generated in the same request with the subsequent search. For ML6, http://docs.marklogic.com/6.0/guide/rest-dev/extensions explains how to do this.
The call to dls:documents-query() makes sure the query is actually registered (on the fly if necessary), but that won't work from REST api. You could extend the REST api with a custom extension as suggested by Mike, but you could also use the following:
cts:properties-query(
cts:and-not-query(
cts:element-value-query(
xs:QName("dls:latest"),
"true",
(),
0
),
cts:element-query(
xs:QName("dls:version-id"),
cts:and-query(())
)
)
)
That is the query that is registered by dls:documents-query(). Might not be future proof though, so check at each upgrade. You can find the definition of the function in /Modules/MarkLogic/dls.xqy
HTH!

stubbing data in REST apis for large system/integration tests

The Problem
Say I've got a cool REST resource /account.
I can create new accounts
POST /account
{accountName:"matt"}
which might produce some json response like:
{account:"/account/matt", accountName:"matt", created:"November 5, 2013"}
and I can look up accounts created within a date range by calling:
GET /account?created-range-start="June 01, 2013"&created-range-end="December 25, 2013"
which might also produce something like:
{accounts: {account:"/account/matt", accountName:"matt", created:"November 5, 2013"}, {...}, ...}
Now, let's say I want to set up some sample data and write some tests against the GET /account resource within some specified creation date range.
For example I want to somehow insert the following accounts into the system
name=account1, created=January 1, 2010
name=account2, created=January 2, 2010
name=account3, created=December 29, 2010
name=account4, created=December 30, 2010
then call
GET /account?created-range-start="January 2, 2010"&created=range-end="December 29,2010"
and verify that only accounts 2 and 3 are returned.
How should I insert these sample accounts to write my tests?
Possible Solutions
1) I could use inversion of control and allow the user to specify the creation date for new accounts.
POST /account
{account:"matt", created="June 01, 2013"}
However, even if the created field were optional, I don't like this approach because I may not want to allow my users the ability to set the creation date of their account. I surely need to be able to do it for testing but having that functionality as part of the public api seems wrong to me. Maybe I want to give a $5 credit to anyone who joined prior to some particular day. If they can specify their create date users can game the system. Not good.
2) I could add one or more testing configuration resources
PUT /account/creationDateTimestampProvider
{provider="DefaultProvider"}
or
PUT /account/creationDateTimestampProvider
{provider="FixedDateProvider", date="June 01, 2013"}
This approach affords me the ability to lock down these resources with security constraints so that only my test context can call them, but it also necessarily has side effects on the system that may become a pain to manage, especially if I have a bunch of backdoor configuration resources.
3) I could interact directly with the database circumventing the REST api altogether to set my sample data.
INSERT INTO ACCOUNTS ...
GET /account?...
However this can allow me to get into states that using the REST api may not allow me to get into and as the db model evolves maintaining these sql scripts might also be a pain.
So... how do i test my GET /account resource? Is there another way I'm not thinking of that is more elegant?
There are a lot of ways to do this, and you've come up with some solid (though maybe not perfect for your situation) solutions.
In the setup for the test, I would spin up an in-memory database like HSQLDB (there are others) and do the inserts. The test configuration will inject the appropriate database configuration into your service provider class. Run the tests, and then shut the database down on teardown.
This post provides a good example at least for the persistence side of things.
Incidentally, do not change the API of your service just to help facilitate a test. Maybe I misunderstood and you aren't anyway, but I thought I would mention just in case.
Hope that helps.
For what it's worth, these days I'm primarily using the second approach for most of my system level (black box) tests.
I create backdoor admin / test apis that have security requirements which only my system tests can access. These superpower apis allow me to seed data. I try to limit the scope of these apis as much as possible so they are not overly coupled to the specific implementation details but are flexible enough to allow specifying whatever is needed for the desired seed data.
The reason I prefer this approach to the database solution that Vidya provided, is so that my tests aren't coupled to the specific data storage technology. If I decide to switch from mongo to dynamo or something like that; using an admin api frees me from having to update all of my tests--instead I only need to update the admin api/impl.