There is a similar looking question asked at :
How can I shorten the URL query paramters?
But here the query parameter is single and its value is comma separated list of long Ids.
Eg. http://example.com/page?q='111100000123,111100000234,11134423213,238418249,823481293,841298472384,89234798124,981248923,24982134983'
Encode this value to something like
htp://example.com/page?q='cdw,erw,ere...'
or
htp://example.com/page?q=asfjeoren
and then decode it back at server side to original value.
Assuming you want the encoded versions of your ids to use alphanumeric characters, and if your ids are all 12 digits (or less) then you would need up to 6 encoded characters per id - you won't get it down to 3 characters as in your example, there simply aren't enough combinations of alphanumeric characters to be able to uniquely represent each id.
I would question if it is actually worth doing what you propose - how many ids are going to be in your urls? Have you proved that the long urls actually cause a problem or are you pre-optimising?
Related
How do we use numbers as tags for dollar quoted strings?
INSERT INTO table(user_id,user_data)
values (22176,to_jsonb($123${"name": "Helo. $ what is this $"}$123$::jsonb))
The above query fails, however if I replace numeric tags with alphabetic then it works. I didn't find anything in the documentation against using numbers for tags.
I need to make my tags as unique as possible, since I'm trying to avoid a situation where user content inside the jsonb matches my tags, for example
$abc${"name": "hello $abc$"}$abc$
I was trying to use UUIDs but it's not accepting numbers as tags.
Note: It's an example query, I have a lot of ' in my json values.
You cannot use $123$, because PostgreSQL uses $1, $2 etc. as placeholders in prepared statements. $a1$ would be ok.
To get a rare string to avoid collisions, drop on the keyboard a few times and make sure not to hit a digit first.
I have a table with a varchar column named key, which is supposed to hold a unique, 8-char random string, which is going to be used as an unique identifier by users. This field should be generated and saved on creation of objects, I have a question about how to create it:
Most of recommendations point to UUID field, but it's not applicable for me because it's too long, and if just get a subset of it then there's no guarantee of uniqueness.
Currently I've just implemented a loop in my backend (not DB), which generates a random string and tries to insert it to DB, and retries if the string turns out to be not unique. But I feel that this is just a really bad practice.
What's the best way to do this?
I'm using Postgresql 9.6
UPDATE:
My main concern is to remove the loop that retries to find a random, short string (or number, doesn't matter) that is unique in that table. AFAIK the solution should be a way to generate the string in DB itself. The only thing that I can find for Postgresql is uuid and uuid-ossp that does something like this, but uuid is way too long for my application, and I don't know of any way to have a shorter representation of uuid without compromising it's uniqueness (and I don't think it's possible theoretically).
So, how can I remove the loop and it's back-and-forth to DB?
Encryption is guaranteed unique, it has to be otherwise decryption would not work. Provided you encrypt unique inputs, such as 0, 1, 2, 3, ... then you are guaranteed unique outputs.
You want 8 characters. You have 62 characters to play with: A-Z, a-z, 0-9 so convert your binary output from the encryption to a base 62 number.
You may need to use the cycle walking technique from Format-preserving encryption to handle a few cases.
I am working on a database that (hopefully) will end up using a primary key with both numbers and letters in the values to track lots of agricultural product. Due to the way in which the weighing of product takes place at more than one facility, I have no other option but to maintain the same base number but use letters in addition to this base number to denote split portions of each lot of product. The problem is, after I create record number 99, the number 100 suddenly floats up and underneath 10. This makes it difficult to maintain consistency and forces me to replace this alphanumeric lot ID with a strictly numeric value in order to keep it sorted (which I use "autonumber" as the data type). Either way, I need the alphanumeric lot ID, and so having 2 ID's for the same lot can be confusing for anyone inputting values into the form. Is there a way around this that I am just not seeing?
If you're using query as a data source then you may try to sort it by string converted to number, something like
SELECT id, field1, field2, ..
ORDER BY CLng(YourAlphaNumericField)
Edit: you may also try Val function instead of CLng - it should not fail on non-numeric input
Why not properly format your key before saving ? e.g: "0000099". You will avoid a costly conversion later.
Alternatively, you could use 2 fields as the composite PK. One with the Number (as Long) and one with the Location (as String).
So here is the problem: I'm using MongoDB in my project so there are 24-characters ObjectId, using only hexadecimal alphabet. I'm make http request in my project to a provider, in this request I need to put a unique Id for callbacks purpose, but the provider allows only 20 characters for this id, and I don't know why.
So, my question is, with a 16 characters alphabet (hexa), there are : 16^24 possible mongo Ids, right ?
Supposing I use in the HTTP request an Id based on 64 different characters ([0-9][a-z][A-Z]-_),
correct me if I'm wrong but I think there are 64^20 possible Ids.
So technically, it is possible to encode every possible MongoDB ObjectId with a corresponding Id, isn't it ?
It seems to be a classic Base64 encoding but mysteriously this does not work as I expected, I think I didn't understand how Base64 encoding works because the generated strings are bigger than original strings...
Do you think all of this is even possible or did I totally miss something ?
Thanks in advance!
EDIT:
One of my colleague tried something which seems to work.
Here is the Java code :
byte[] decodedHex = Hex.decodeHex("53884594e4b0695f366f8128".toCharArray());
byte[] encodedHexB64 = Base64.encodeBase64(decodedHex);
System.out.println(new String(encodedHexB64)); // --> U4hFlOSwaV82b4Eo
For a reason that I ignore, doing this is not the same:
String anotherB64 = Base64.encodeBase64String("53884594e4b0695f366f8128".getBytes());
System.out.println(anotherB64);
And it prints : NTM4ODQ1OTRlNGIwNjk1ZjM2NmY4MTI4
MongoDB is using ObjectId as a default primary key for the documents because it's fast to generate and very likely to be unique.
But you are not forced to use it as a primary key. You can use any BSON data type in the _id field as long is not an array. That being said, you can use your 20-char Id in _id field.
EDIT:
From your original question I didn't know that you're using an existing DB. The _id field is immutable and it cannot be changed in an existing document.
If you only wanted to convert the existing ObjectId to something else that's 20 chars long the method you posted will work.
The second method produces a long string because you're basically base64 encoding a string which will produce an even longer string.
I have an email token collection in my mongodb database for a meteor app and I stick these email tokens in the reply address of my email (eg. #example.com) so that when I parse it I know what it's relating to.
The problem I have is that the email token uses the default _id algorithm to generate a unique id and that algorithm generates a string that is a mixture of upper case and lower case characters.
However, I've discovered that some email clients, lowercases the entire reply address, which means that I can only identify the addresses case-insensitvely.
I guess now I have two options.
1) The easiest option would be to match the email tokens with the reply address case insensitively. What would be the chance of clashes in that respect?
2) Make the email token some sort of guid and generate this guid independent of the mongodb ID creation.
Yes, you would get issues. Meteor uses both upper and lower case values in its 17 character id values. You can have a look at the code in the Random package: https://github.com/meteor/meteor/tree/devel/packages/random.
So it would be possible to get two distinct values of which the differences could only be casing. This could cause mixups if your client's email applications convert the address to lowercase characters.
In your case it is best not to use Random.id(), rather to make up your own Random character generator. Something like this might work:
var lowerCaseId = function() {
var digits = [],
self = this;
for (var i = 0; i < 17; i++) {
digits[i] = Random.choice("23456789abcdefghijkmnopqrstuvwxyz");
}
return digits.join("");
};
Also of note is the meteor _id value is built up of 'unmistakeable characters' - There are no characters that can cause confusion such as 0 vs O, 1 vs I, etc.
If you don't use it in your _id field, you would have to generate a value with this and check it does not exist in your database before inserting it, or using a unique index for it.
Additionally also be aware there will be a significant decrease in entropy since the number of possible combinations will have dropped with the loss of the uppercased characters. If this is of significance to you, you could increase the number of digits from 17 in the code above.
Meteor is generating it's own Id's which are different from the MongoDB ObjectId's. As noted, these would be subject to clash when converting case or checking case insensitively. This is kind of interesting and I'm not sure of the project's reasons for this.
Under the hood however the mongodb node native driver. So the ObjectId creation functions should be available if you want to use them.
https://github.com/mongodb/js-bson/blob/master/lib/bson/objectid.js#L68-L74
The important part is in these calls:
value.toString(16)
So the radix here is set to 16 for hex or all the characters 0-9a-f.
You can also note in drivers that they will Regex check like this:
^[0-9a-fA-F]{24}$
So it would seem that case sensitivity is not an issue.
Still if you want to use something alternate there is a section in the documentation that might serve as a useful guide.
http://docs.mongodb.org/manual/core/document/#the-id-field