Get BinData UUID from Mongo as string - mongodb

I currently have some ids stored in Mongo as UUIDs (necessary for processing). They get returned like this:
"_id" : new BinData(3, "JliB6gIMRuSphAD2KmhzgQ==")
What would be an easy way to turn this value into a string for debugging?
Just to be clear - the application can handle the data fine. I just need a way to get the actual UUID from Mongo quickly.

The answer to your question is more complicated that you would expect! The main reason it's complicated is that for historical reasons (unfortunately) different drivers have written UUIDs to the database using different byte orders. You don't mention which driver you are using, but I'll use the C# driver as an example.
Suppose I use the following code to insert a document:
var guid = new Guid("00112233-4455-6677-8899-aabbccddeeff");
collection.Insert(new BsonDocument {
{ "_id", guid },
{ "x", 1 }
});
If I then examine the document using the Mongo shell, it looks like this:
> db.test.findOne()
{ "_id" : BinData(3,"MyIRAFVEd2aImaq7zN3u/w=="), "x" : 1 }
>
The Mongo shell has a built-in function called hex that you can use to display the binary value as a hex string:
> var doc = db.test.findOne()
> doc._id.hex()
33221100554477668899aabbccddeeff
>
Look carefully: the byte order of the hex string doesn't match the original UUID value used in the C# program. That's because the C# driver uses the byte order returned by Microsoft's ToByteArray method of the Guid class (which sadly returns the bytes in a bizarre order, which fact was not discovered for many months). Other drivers have their own idiosyncracies.
To help out with this we have some helper functions written in Javascript that can be loaded into the Mongo shell. They are defined in this file:
https://github.com/mongodb/mongo-csharp-driver/blob/master/uuidhelpers.js
The Mongo shell can be told to process a file as it starts up by providing the name of the file on the command line (along with the --shell argument). Having loaded this file we have access to a number of helper functions to create and display BinData values that are UUIDs. For example:
C:\mongodb\mongodb-win32-x86_64-2.0.1\bin>mongo --shell uuidhelpers.js
MongoDB shell version: 2.0.1
connecting to: test
type "help" for help
> var doc = db.test.findOne()
> doc._id.toCSUUID()
CSUUID("00112233-4455-6677-8899-aabbccddeeff")
> db.test.find({_id : CSUUID("00112233-4455-6677-8899-aabbccddeeff")})
{ "_id" : BinData(3,"MyIRAFVEd2aImaq7zN3u/w=="), "x" : 1 }
>
In this example the toCSUUID function is used to display a BinData value as a CSUUID and the CSUUID function is used to create a BinData value for a UUID using the C# driver's byte ordering conventions so that we can query on a UUID. There are similar functions for the other drivers (toJUUID, toPYUUID, JUUID, PYUUID).
Some day in the future all drivers will standardize on a new binary subtype 4 with a standard byte order. In the meantime you have to use the appropriate helper function that matches whatever driver you are using.

Use this function before your query:
function ToGUID(hex) {
var a = hex.substr(6, 2) + hex.substr(4, 2) + hex.substr(2, 2) + hex.substr(0, 2);
var b = hex.substr(10, 2) + hex.substr(8, 2);
var c = hex.substr(14, 2) + hex.substr(12, 2);
var d = hex.substr(16, 16);
hex = a + b + c + d;
var uuid = hex.substr(0, 8) + '-' + hex.substr(8, 4) + '-' + hex.substr(12, 4) + '-' + hex.substr(16, 4) + '-' + hex.substr(20, 12);
return '"' + uuid + '"';
}
var id = new BinData(3, "JliB6gIMRuSphAD2KmhzgQ==");
ToGUID(id.hex());
Result:
"ea815826-0c02-e446-a984-00f62a687381"

If you are using Java spring-data, you can use this algorithm:
function ToUUID(hex) {
var msb = hex.substr(0, 16);
var lsb = hex.substr(16, 16);
msb = msb.substr(14, 2) + msb.substr(12, 2) + msb.substr(10, 2) + msb.substr(8, 2) + msb.substr(6, 2) + msb.substr(4, 2) + msb.substr(2, 2) + msb.substr(0, 2);
lsb = lsb.substr(14, 2) + lsb.substr(12, 2) + lsb.substr(10, 2) + lsb.substr(8, 2) + lsb.substr(6, 2) + lsb.substr(4, 2) + lsb.substr(2, 2) + lsb.substr(0, 2);
hex = msb + lsb;
var uuid = hex.substr(0, 8) + '-' + hex.substr(8, 4) + '-' + hex.substr(12, 4) + '-' + hex.substr(16, 4) + '-' + hex.substr(20, 12);
return uuid;
}

def binaryToUUID(byte: Array[Byte]): String = {
if (byte == null) null
else {
val bb = ByteBuffer.wrap(byte)
new UUID(bb.getLong, bb.getLong()).toString
}
}

If your mongodb driver uses javascript, then you could try this to get hex:
BinData(3,"ABEiM0RVZneImaq7zN3u/w==").toString('hex');

Related

How to convert MongoDB UUID to string in MongoDB query? [duplicate]

I currently have some ids stored in Mongo as UUIDs (necessary for processing). They get returned like this:
"_id" : new BinData(3, "JliB6gIMRuSphAD2KmhzgQ==")
What would be an easy way to turn this value into a string for debugging?
Just to be clear - the application can handle the data fine. I just need a way to get the actual UUID from Mongo quickly.
The answer to your question is more complicated that you would expect! The main reason it's complicated is that for historical reasons (unfortunately) different drivers have written UUIDs to the database using different byte orders. You don't mention which driver you are using, but I'll use the C# driver as an example.
Suppose I use the following code to insert a document:
var guid = new Guid("00112233-4455-6677-8899-aabbccddeeff");
collection.Insert(new BsonDocument {
{ "_id", guid },
{ "x", 1 }
});
If I then examine the document using the Mongo shell, it looks like this:
> db.test.findOne()
{ "_id" : BinData(3,"MyIRAFVEd2aImaq7zN3u/w=="), "x" : 1 }
>
The Mongo shell has a built-in function called hex that you can use to display the binary value as a hex string:
> var doc = db.test.findOne()
> doc._id.hex()
33221100554477668899aabbccddeeff
>
Look carefully: the byte order of the hex string doesn't match the original UUID value used in the C# program. That's because the C# driver uses the byte order returned by Microsoft's ToByteArray method of the Guid class (which sadly returns the bytes in a bizarre order, which fact was not discovered for many months). Other drivers have their own idiosyncracies.
To help out with this we have some helper functions written in Javascript that can be loaded into the Mongo shell. They are defined in this file:
https://github.com/mongodb/mongo-csharp-driver/blob/master/uuidhelpers.js
The Mongo shell can be told to process a file as it starts up by providing the name of the file on the command line (along with the --shell argument). Having loaded this file we have access to a number of helper functions to create and display BinData values that are UUIDs. For example:
C:\mongodb\mongodb-win32-x86_64-2.0.1\bin>mongo --shell uuidhelpers.js
MongoDB shell version: 2.0.1
connecting to: test
type "help" for help
> var doc = db.test.findOne()
> doc._id.toCSUUID()
CSUUID("00112233-4455-6677-8899-aabbccddeeff")
> db.test.find({_id : CSUUID("00112233-4455-6677-8899-aabbccddeeff")})
{ "_id" : BinData(3,"MyIRAFVEd2aImaq7zN3u/w=="), "x" : 1 }
>
In this example the toCSUUID function is used to display a BinData value as a CSUUID and the CSUUID function is used to create a BinData value for a UUID using the C# driver's byte ordering conventions so that we can query on a UUID. There are similar functions for the other drivers (toJUUID, toPYUUID, JUUID, PYUUID).
Some day in the future all drivers will standardize on a new binary subtype 4 with a standard byte order. In the meantime you have to use the appropriate helper function that matches whatever driver you are using.
Use this function before your query:
function ToGUID(hex) {
var a = hex.substr(6, 2) + hex.substr(4, 2) + hex.substr(2, 2) + hex.substr(0, 2);
var b = hex.substr(10, 2) + hex.substr(8, 2);
var c = hex.substr(14, 2) + hex.substr(12, 2);
var d = hex.substr(16, 16);
hex = a + b + c + d;
var uuid = hex.substr(0, 8) + '-' + hex.substr(8, 4) + '-' + hex.substr(12, 4) + '-' + hex.substr(16, 4) + '-' + hex.substr(20, 12);
return '"' + uuid + '"';
}
var id = new BinData(3, "JliB6gIMRuSphAD2KmhzgQ==");
ToGUID(id.hex());
Result:
"ea815826-0c02-e446-a984-00f62a687381"
If you are using Java spring-data, you can use this algorithm:
function ToUUID(hex) {
var msb = hex.substr(0, 16);
var lsb = hex.substr(16, 16);
msb = msb.substr(14, 2) + msb.substr(12, 2) + msb.substr(10, 2) + msb.substr(8, 2) + msb.substr(6, 2) + msb.substr(4, 2) + msb.substr(2, 2) + msb.substr(0, 2);
lsb = lsb.substr(14, 2) + lsb.substr(12, 2) + lsb.substr(10, 2) + lsb.substr(8, 2) + lsb.substr(6, 2) + lsb.substr(4, 2) + lsb.substr(2, 2) + lsb.substr(0, 2);
hex = msb + lsb;
var uuid = hex.substr(0, 8) + '-' + hex.substr(8, 4) + '-' + hex.substr(12, 4) + '-' + hex.substr(16, 4) + '-' + hex.substr(20, 12);
return uuid;
}
def binaryToUUID(byte: Array[Byte]): String = {
if (byte == null) null
else {
val bb = ByteBuffer.wrap(byte)
new UUID(bb.getLong, bb.getLong()).toString
}
}
If your mongodb driver uses javascript, then you could try this to get hex:
BinData(3,"ABEiM0RVZneImaq7zN3u/w==").toString('hex');

Hash Alphanumeric string to a 16 Byte Hex String

I need a hash algorithm that takes a 28 character alphanumeric [a-zA-Z0-9] string and outputs a 16 Byte Hex UUID.
Example: hash("cVoFfGI0vhfvWD61Hh1QTsmUJRT2") = 2f234454-cf6d-4a0f-adf2-f4911ba9ffa6
Is there a way to do so using md5 or SHA1?
Thanks.
EDIT 1 :
The alphanumeric string has a fixed length of 28 characters.
here a solution in javascript (nodeJS)
var crypto = require('crypto');
function hash(input) {
var s = crypto.createHash('md5').update(input).digest("hex");
return s.substr(0, 8) + '-' + s.substr(8, 4) + '-' + s.substr(12, 4) + '-' + s.substr(16, 4) + '-' + s.substr(20, 12);
}
hash('cVoFfGI0vhfvWD61Hh1QTsmUJRT2');

Issues with naming ranges for charts within the Google Spreadsheet Script

I've been trying for days to create charts with an intelligent range, that differs when the data in the google spreadsheet is updated. However i succeeded doing so, i can't get the .setOption aspect to work. I want for example, a title, description etc with the chart. But this is not the main issue since i can insert there by hand.
More important however is the range name, because there isn't when i use the script. So, within the chart it is not possible to see what each column represents, and i really want to fix that. I tried to use the .setNamedRange() aspects, but that is not working.
Someone who can help me with that?
function check() {
var sheet = SpreadsheetApp.getActiveSheet();
var end = sheet.getLastRow();
var start = (end - 5);
var endnew = (end - 4);
var startnew = (end - 6);
if(sheet.getCharts().length == 0){
Logger.log("Er is geen grafiek");
var chartBuilder = sheet.newChart()
.asColumnChart().setStacked()
.addRange(sheet.getRange("A" + startnew + ":" + "A" + endnew)) // should have a name
.addRange(sheet.getRange("B" + startnew + ":" + "B" + endnew)) // should have a name
.addRange(sheet.getRange("E" + startnew + ":" + "E" + endnew)) //should have a name
.setOption('title', 'Effectief gebruik kantoorruimte') //not working
.setPosition(10, 10, 0, 0)
var chart = chartBuilder.build();
sheet.insertChart(chart);
}
else{
Logger.log("Er is wel een grafiek");
var charts = sheet.getCharts();
for (var i in charts) {
var chart = charts[i];
var ranges = chart.getRanges();
var builder = chart.modify();
for (var j in ranges) {
var range = ranges[j];
builder.removeRange(range);
builder
.addRange(sheet.getRange("A" + (start) + ":" + "A" + end)) //should have a name
.addRange(sheet.getRange("B" + (start) + ":" + "B" + end)) //should have a name
.addRange(sheet.getRange("E" + (start) + ":" + "E" + end)) // should have a name
.setOption('title', 'Effectief gebruik kantoorruimte')
.build();
sheet.updateChart(builder.build());
}
}
}
}
I'm assuming that this code is the issue?
builder
.addRange(sheet.getRange("A" + (start) + ":" + "A" + end))
Maybe try using the JavaScript toString() method to make sure that your text formula is working.
.addRange(sheet.getRange("A" + start.toString() + ":" + "A" + end.toString()))
There is a different format that you can use:
getRange(row, column, numRows, numColumns)
So, it would be:
getRange(start, 1, 1, numColumns)
That starts on row "start" in column A. It gets one row of data, and how ever many number of columns.

Mapping binary data in perl

I have the following predefined codes that represent an index in a binary bitmap:
0 = standard
1 = special
2 = regular
3 = late
4 = early
5 = on time
6 = generic
7 = rfu
An example value I would take as an input would be 213, which becomes 11010101 in binary. Index 0, 2, 4, 6, and 7 have their bit flipped indicating that this record is:
standard + regular + early + generic + rfu.
I am trying to figure out in perl how to take that binary data and build a string, like mentioned above with code + code + code, etc.
Any help would be greatly appreciated. Thanks.
Edit: My thoughts on how I might approach this are:
Convert decimal to binary
Find length of binary string
Using substr get the value (0 or 1) index by index
If index value = 1 then add relevant code to string
Is there a better way to go about this?
You can test bits on input from 0 to 7, and take only these that are set,
my $in = 213;
my #r = ("standard","special","regular","late","early","on time","generic","rfu");
print join " + ", #r[ grep { $in & (1 << $_) } 0 .. $#r ];
# or
# print join " + ", map { $in & (1<<$_) ? $r[$_] : () } 0 .. $#r;
output
standard + regular + early + generic + rfu

How to use Netbeans Variable Formatters?

By default when viewing watches/variables of objects in Netbeans, it shows its address instead of its value. This is quite tiresome since I have to expand the variable to see its real value (e.g. for Double, Integer, Date, etc). As it turns out, Netbeans has "Variable formatters" but there is hardly any documentation that i can find for it.
How would I go about displaying e.g. a simple Date variable in a human readable format in the Watches/Variables window? I don't fully understand the "Edit Variable Formatter" dialog.
I was able to properly do it for Double and Integer by using the following code snippet:
toString()
So the code seems to run in the context of the Double/Integer class. How would I refer to the actual variable if I need to do something more advanced such as:
return DateHelpers.formatDate(dateVariableName??, "yyyy-MM-dd");
In the variables view, you have a small $ icon (at the top left) which tooltip says :"Show variable value as toString() or formatted value".
Just click that, it will show you the "value" of those variables.
EDIT: If you want to add a variable formatter, it's very simple. On the variable formatter view, just click the "Add ..." button then:
In "Formatter Name" put the name of you formatter eg. "My Date formatter"
"Class Types" put your complete class name eg. java.util.Date
Select "Value formatted as a result of code snippet" and type the code to apply. For instance:
toString()
but if you want to manipulate the data or display some other thing you can. For instance:
toString() + " (" + getTime() + ")"
Which will display the time in human readable format plus the time as a long.
Don't forget to select the $ icon on the view to apply your formatter.
My answer won't solve the question. It will rather echo the previous answers. First, I haven't seen the $ sign in the variables-panel in NetBeans. Seems it was replaced with a context menu in current versions.
I haven't found the answer to the actual question, as of how you would reference the variable to debug, within the "Variable Formatters" Dialog. Something like "this" or "$1" isn't working definitely. Also the facility does not seem to know about Standard Java JRE classes like SimpleDateFormatter.
So when debugging Java JRE classes, I guess you have to live with what they're offering in terms of public methods.
Here's a workaround for the especially user friendly Date class, if you're stuck with a JDK below version 8 (as me). Just create a new Variable Formatter in NetBeans via
Tools > Options > Java > Variable Formatters > Add
Then in the "Class Types" editfield enter:
java.util.Date
Under "Value formatted as a result of code snippet" use one of the next snippets.
// German format - "dd.MM.yyyy hh:mm"
((getDate() < 10) ? ("0" + getDate()) : getDate()) + "." + ((getMonth() < 9) ? ("0" + (getMonth() + 1)) : (getMonth() + 1) ) + "." + (getYear() + 1900) + " " + ((getHours() < 10) ? "0" + getHours() : getHours()) + ":" + ((getMinutes() < 10) ? "0" + getMinutes() : getMinutes()) + ":" + ((getSeconds() < 10) ? "0" + getSeconds() : getSeconds())
// US format - "MM/dd/yyyy hh:mm"
((getMonth() < 9) ? ("0" + (getMonth() + 1)) : (getMonth() + 1) ) + "/" + ((getDate() < 10) ? ("0" + getDate()) : getDate()) + "/" + (getYear() + 1900) + " " + ((getHours() < 10) ? "0" + getHours() : getHours()) + ":" + ((getMinutes() < 10) ? "0" + getMinutes() : getMinutes()) + ":" + ((getSeconds() < 10) ? "0" + getSeconds() : getSeconds())
// ISO-8601 - "yyyy-MM-dd hh:mm"
(getYear() + 1900) + "-" + ((getMonth() < 9) ? ("0" + (getMonth() + 1)) : (getMonth() + 1) ) + "-" + ((getDate() < 10) ? ("0" + getDate()) : getDate()) + " " + ((getHours() < 10) ? ("0" + getHours()) : getHours()) + ":" + ((getMinutes() < 10) ? ("0" + getMinutes()) : getMinutes()) + ":" + ((getSeconds() < 10) ? ("0" + getSeconds()) : getSeconds())
The next snippet could also come in handy, when your lost in the the debug-output overkill of an instance of java.util.Calendar:
// German format - "dd.MM.yyyy hh:mm"
((get(5) < 10) ? ("0" + get(5)) : get(5)) + "." + ((get(2) < 9) ? ("0" + (get(2) + 1)) : (get(2) + 1) ) + "." + (get(1)) + " " + ((get(10) < 10) ? "0" + get(10) : get(10)) + ":" + ((get(12) < 10) ? "0" + get(12) : get(12)) + ":" + ((get(13) < 10) ? "0" + get(13) : get(13))
You can even put a much more complex code in the variable formatter, as long as you return a string. For example, if I have a class with two string members, name and surname I can paste the follwing code in the "Value formatted.." box:
String result;
if (name != null) {
result = name + " " + surname;
} else {
result = "<null>";
}
return result;
.. and for displaying as children the name and surname separately you can paste a code in the "Children displayed as.." box to return a String[], for example:
String[] results = new String[2];
results[0] = "name: " + name;
results[1] = "surname: " + surname;
return results;
This will display two nodes as children in the variables debug window with the name and surname
The variable itself can be referenced by this (at least it works in Netbeans 8.1).
So, let's say we want the identityHashCode of our Collection after its size:
"size = " +size() + " #" + System.identityHashCode(this)