Google Storage documentation tells about Object Versioning. There are two kinds of the object versions: live and noncurrent.
gsutil allow listing both noncurrent and live versions using -a switch: https://cloud.google.com/storage/docs/using-object-versioning#list.
Also, I can list all the versions programmatically by supplying versions: true option to the Bucket.getFiles method.
However I have not found any way to programmatically find out whether a particular object version is live or noncurrent. There seems to be no property or method in the File object for this.
What is the proper way of finding this out given a File instance?
By looking at the REST API, there isn't a state for the live/noncurrent version of the objects. You have a generation number per object resource representation.
I assume that you have to apply yourselves an algorithm for this
Use List API (getFiles) on a single object with the version option to true
The highest generation is the live version, others are noncurrent
Except is timeDeleted is populated on the highest generation (timestamp of the live version deletion). Therefore all the version are noncurrent.
Related
I have added GTM and GA4 to some website apps to produce tables of detailed stats on click-throughs of ads per advertiser for a date range. I now have suitable reports working successfully using Data Studio, but my attempts to do the same using the PHP implementation of Analytics Data API V1 Beta (in order to do batch runs covering many date ranges) repeatedly hit a brick wall: the methods needed to analyse the response from instantiating BetaAnalyticsDataClient and then invoking runPivotReport or batchRunReports or batchRunPivotReports (and so on) appear not be specified.
The only example that I could work from is the ‘quickstart’ one that does a basic dimension and metric retrieval, and even this employs:
getRows()
getDimensionValues()
getValue()
getMetricValues
that do not appear in the API documentation, at least that I can find.
The JSON response format for each report is of course documented: for example the output from running runPivotReport is documented as an instantiation of runPivotReportResponse.
But nowhere can I find a specification of the methods to be used to traverse the JSON tree (vide getDimensionValues() above) and extract some output data.
Guesswork has taken me part way, but purely for example, when retrieving pivot data, should a
getPivotDimensionHeaders()[0]
be followed by a
getDimensionValues()
or a
getPivotDimensionValues()
I am obviously approaching this all wrong, but what should I do, please?
Kuberentes has a mechanism for supporting versioning of CRDs. See https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/. What is not clear to me is how you actually support an evolution of CRD v1 to CRD v2 when you cannot always convert from v1 <-> v2. Suppose we introduce a new field in v2 that can not be populated by a web hook conversion, then perhaps all we can do is leave the field null? Furthermore when you request api version N you always get back an object as version N even if it was not written as version N so how can you controller know how to treat the object?
As you can read in Writing, reading, and updating versioned CustomResourceDefinition objects
If you update an existing object, it is rewritten at the version that is currently the storage version. This is the only way that objects can change from one version to another.
Kubernetes returns the object to you at the version you requested, but the persisted object is neither changed on disk, nor converted in any way (other than changing the apiVersion string) while serving the request.
If you update an existing object, it is rewritten at the version that is currently the storage version. This is the only way that objects can change from one version to another.
You read your object at version v1beta1, then you read the object again at version v1. Both returned objects are identical except for the apiVersion field, Upgrade existing objects to a new stored version
The API server also supports webhook conversions that call an external service in case a conversion is required.
The webhook handles the ConversionReview requests sent by the API servers, and sends back conversion results wrapped in ConversionResponse. You can read about Webbooks here.
Webhook conversion was introduced in Kubernetes v1.13 as an alpha feature.
When the webhook server is deployed into the Kubernetes cluster as a service, it has to be exposed via a service on port 443.
When deprecating versions and dropping support, devise a storage upgrade procedure.
In the case you describe I think you are forced to define some kind of 'default' value for that new mandatory field being introduced, so custom objects created before that change can be converted to the updated spec.
But then it becomes a backward compatible change, so no need to define that v2, you can remain in v1.
If It Is not possible to define a default value for it nor to derive any value at all (for that mandatory introduced field) from other fields existing in a custom object compliant with current v1 (i.e. the schema to follow before introducing this update) then you should assure v2 to be the new stored versione and then manually update all existing v1 compliant custom objects to properly set that mandatory parameter.
Or even to define a new custom object ...
Let say you have CRD specification like below.
// API v6.
type Frobber struct {
Height int `json:"height"`
Param string `json:"param"`
}
Then you added some new spec called Width.
// Still API v6.
type Frobber struct {
Height int `json:"height"`
Width int `json:"width"`
Param string `json:"param"`
}
So as long as you can handle this change using controller logic you do not need any version change. Which means this addition would be backward compatible.
But let say now you are changing the previous spec as below.
// Internal, soon to be v7beta1.
type Frobber struct {
Height int
Width int
Params []string
}
Here you change an existing spec which is not backward compatible. Also, you might not handle this change using controller logic. In these kinds of changes, it is better to use a version upgrade.
For more details about kubernetes API version changes, refer the following links.
[1] https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md
[2] https://groups.google.com/forum/#!topic/operator-framework/jswZUe1rlho
1)
The documentation states:
A universally unique identifier (UUID): This ID is assigned to every object in an OBS system. This UUID allows the object storage system to differentiate objects from one another and is used to find the data without needing to know the exact physical drive, array, or site where the data is.
However, I cannot find info in the API about how to retrieve objects via their UUID, or how to retrieve an object's UUID. Can I do this?
2) Does COS support automatic versioning of stored objects, like aws does? For example, if I store data in a version-enabled bucket, I receive a header x-amz-version-id (see here) with the version ID that was assigned. Does COS support versioning, and if so, how do I retrieve the versions of an object?
The object's UUID is actually just the {bucket-name}/{object-key} combination. Because each bucket name must be globally unique, this allows for the object key to define the unique identifier for the object. Thanks for asking about this, I should rewrite that part of the docs to not use UUID as that implies there is an additional underlying identifier.
At the moment, no, COS does not support AWS S3-style versioning of objects in the public cloud (although this is possible in private cloud implementations). This is on the roadmap for later this year.
Regaring version, I do not know, but If you wnat to have the UUID the way AWS present it, then, you might also talk about the Etag. Etag, for a given object, is something like this :
52816d090462f946a7a6273ea3d3896c
To get this Etag, BOTO3 is awesome :
use this call to get it :
s3ressource = client(
service_name='s3',
endpoint_url= "endpoint_url",
aws_access_key_id= "aws_access_key_id",
aws_secret_access_key="aws_secret_access_key",
use_ssl=True,
)
OBJE = s3ressource.head_object(Bucket = TheNameOfYourBucket, Key = theNameOfYourObject)
ETAGOBJE=OBJE['ETag']
Object Versioning is now supported btw.
https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning
I am a beginner in Softlayer and we need to implement block storage functionality provided by softlayer in an Application. We just need to make REST call to fetch the locations on the basis of storage type selected. I need to know relationship between Storage Type and locations. What method I need to call in REST API and object mask required for same.. Thanks in Advance.
If you want to order a new “Network storage” and know what “locations” are available for this item, see these steps:
1. The first that we need to know is the “package id” to use for this order.
The “package id” is very important to valid and verifies what kind of items, what locations are available at the moment to order network storage.
For example:
Storage Type: Endurance, the package to use is “240”
Storage Type: Performance, the package to use is “222”
To get all active packages, please review:
http://sldn.softlayer.com/reference/services/SoftLayer_Product_Package/getAllObjects
2. Once we know what package to use, we need to get the valid “item price ids” according to LOCATION. The following request can help us:
https://[username]:[apikey]#api.softlayer.com/rest/v3/SoftLayer_Product_Package/[package_id]/getItemPrices?objectMask=mask[id,item[keyName,description],pricingLocationGroup[locations[id, name, longName]]]
Method: GET
Where:
A price id with a locationGroupId = null is considered "A standard price" and the API will internally switch the prices for the customer. But we recommend to execute first the verifyOrder in order to see if the wanted order is ok (the fee can vary).
Reference: http://sldn.softlayer.com/blog/cmporter/Location-based-Pricing-and-You
Also, this method can help you to get available locations for a specific package:
http://sldn.softlayer.com/reference/services/SoftLayer_Product_Package/getRegions
3. Then you will able to order a new Network storage, please see:
API for Performance and Endurance storage(Block storage)
Now, if you want to list the network storage of your account, please see:
http://sldn.softlayer.com/reference/services/SoftLayer_Account/getNetworkStorage
This is an example, where the result displays properties like: “location” and “network storage type”.
https://[username]:[apikey]#api.softlayer.com/rest/v3/SoftLayer_Account/getNetworkStorage?objectMask=mask[storageType, billingItem[description,location[id,longName]]]
Using filters:
Filtering by network Storage Type: “Endurance Storage” or “Block Storage (Performance)”
https://[username]:[apikey]#api.softlayer.com/rest/v3/SoftLayer_Account/getNetworkStorage?objectMask=mask[id,username,nasType,storageType, billingItem[description,location[id,longName]]]&objectFilter={"networkStorage":{"nasType":{"operation":"ISCSI"},"billingItem":{"description":{"operation":"Endurance Storage"}}}}
Regards.
You need to use the http://sldn.softlayer.com/reference/services/SoftLayer_Product_Package/getRegions method
It returns the valid locations for a package, each storage type belongs to an specific package (that is the relation you are looking for) to get the packages use http://sldn.softlayer.com/reference/services/SoftLayer_Product_Package/getAllObjects method
see this post for more information
Filter parameters to POST verify and place order request for Performance storage
We need to restrict a MarkLogic search to the latest version of managed documents, using Marklogic's REST api. We're using MarkLogic 6.
Using straight xquery, you can use dls:documents-query() as an additional-query option (see
Is there any way to restrict marklogic search on specific version of the document).
But the REST api requires XML, not arbitrary xquery. You can turn ordinary cts queries into XML easily enough (execute <some-element>{cts:word-query("hello world")}</some-element> in QConsole).
If I try that with dls:documents-query() I get this:
<cts:properties-query xmlns:cts="http://marklogic.com/cts">
<cts:registered-query>
<cts:id>17524193535823153377</cts:id>
</cts:registered-query>
</cts:properties-query>
Apart from being less than totally transparent... how safe is that number? We'll need to put it in our query options, so it's not something we can regenerate every time we need it. I've looked on two different installations here and the the number's the same, but is it guaranteed to be the same, and will it ever change? On, for example, a MarkLogic upgrade?
Also, assuming the number is safe, will the registered-query always be there? The documentation says that registered queries may be cleared by the system at various times, but it's talking about user-defined registered queries, and I'm not sure how much of that applies to internal queries.
Is this even the right approach? If we can't do this we can always set up collections and restrict the search that way, but we'd rather use dls:documents-query if possible.
The number is a registered query id, and is deterministic. That is, it will be the same every time the query is registered. That behavior has been invariant across a couple of major releases, but is not guaranteed. And as you already know, the server can unregister a query at any time. If that happens, any query using that id will throw an XDMP-UNREGISTERED error. So it's best to regenerate the query when you need it, perhaps by calling dls:documents-query again. It's safest to do this in the same request as the subsequent search.
So I'd suggest extending the REST API with your own version of the search endpoint. Your new endpoint could add dls:documents-query to the input query. That way the registered query would be generated in the same request with the subsequent search. For ML6, http://docs.marklogic.com/6.0/guide/rest-dev/extensions explains how to do this.
The call to dls:documents-query() makes sure the query is actually registered (on the fly if necessary), but that won't work from REST api. You could extend the REST api with a custom extension as suggested by Mike, but you could also use the following:
cts:properties-query(
cts:and-not-query(
cts:element-value-query(
xs:QName("dls:latest"),
"true",
(),
0
),
cts:element-query(
xs:QName("dls:version-id"),
cts:and-query(())
)
)
)
That is the query that is registered by dls:documents-query(). Might not be future proof though, so check at each upgrade. You can find the definition of the function in /Modules/MarkLogic/dls.xqy
HTH!