I am watching a course about using MongoDB in NodeJS applications, this is one line of its codes:
const url = 'mongodb://localhost:27017/conFusion';
Then it uses it like this:
MongoClient.connect(url).then((db) => {
...
}
I like to know what is the reason it uses the above address instead of the following?
const url = 'localhost:27017/conFusion';
Is there any difference? What is the reason for adding mongodb:// in front of the URL address and what does it do?
"mongodb://localhost:27017/conFusion" points to your local MongoDB database created in MyMongoDB folder. The connect() method returns the database reference if the specified database is already exists, otherwise it creates a new database.
mongodb:// is a required prefix to identify that this is a string in the standard connection format. It is as per the official documentation - Standard Connection String.
Standard connection schema is
mongodb://[username:password#]host1[:port1][,...hostN[:portN]][/[database][?options]]
Hence, you can't use localhost:27017/conFusion as mongodb connection string.
I have build coturn and run it successfully. ip:192.168.1.111. Now the question I faced is to get the Turn credential through REST API.
https://datatracker.ietf.org/doc/html/draft-uberti-behave-turn-rest-00 According to the passage the request format should be
GET /?service=turn&username=mbzrxpgjys
and response should be JSON. Now my question is:
a) How to configure and command TURN SERVER to make it run in REST API mode?
b) How to write a http request in the right format so TURN SERVER can reply correctly? could you give me an example?
Few things to be clarified here are:
GET /?service=turn&username=mbzrxpgjys which returns a JSON, is just a suggested uri for retrieving time-limited TURN credentials from the server, you do not have to follow that, your uri can be just /?giveMeCredentials. In fact, I use my socket connection to retrieve this data, not direct http call with json response. End of day, it does not matter how you( the client that uses said TURN) get those credentials as long as they are valid.
You do not make any requests to the TURN server directly, no rest api call to TURN server is under your control.
you allocate a secret key when you are starting the TURN server, this can be taken from a db(thus dynamically changable), but lazy that I am, just hard-coded, and gave it in the turn config file, also remember to enable REST API. As part of turn command, turnserver ... --use-auth-secret --static-auth-secret=MySecretKey
Now, in your application server, you would use the same secret key to generate credentials, for username, it is UNIX timestamp and some string( can be random or user id or something) seperated by : and the password would be HMAC of the username with your secret key.
about the UNIX timestamp, this has be the time in TURN server till which your credentials has to be valid, so which calculating this make sure you take into account of the clock time difference between your application server and your turn server.
Now some sample code taken from my answer to another question
command for stating TURN server:
turnserver -v --syslog -a -L xx.xxx.xx.xx -X yy.yyy.yyy.yy -E zz.zzz.zz.zzz --max-bps=3000000 -f -m 3 --min-port=32355 --max-port=65535 --use-auth-secret --static-auth-secret=my_secret --realm=north.gov --cert=turn_server_cert.pem --pkey=turn_server_pkey.pem --log-file=stdout -q 100 -Q 300 --cipher-list=ALL
node.js code for creating TURN credentials in application server:
var crypto = require('crypto');
function getTURNCredentials(name, secret){
var unixTimeStamp = parseInt(Date.now()/1000) + 24*3600, // this credential would be valid for the next 24 hours
username = [unixTimeStamp, name].join(':'),
password,
hmac = crypto.createHmac('sha1', secret);
hmac.setEncoding('base64');
hmac.write(username);
hmac.end();
password = hmac.read();
return {
username: username,
password: password
};
}
Browser code for using this:
...
iceServers:[
{
urls: "turn:turn_server_ip",
username: username,
credential:password
}
...
After (many) hours of frustration, #Mido's excellent answer here was the only thing that actually got CoTurn's REST API working for me.
My credential server is PHP and I use CoTurn's config file 'turnserver.conf' so here's a tested and working translation of Mido's work for that situation:
Assuming a 'shared secret' of '3575819665154b268af59efedee8826e', here are the relevant turnserver.conf entries:
lt-cred-mech
use-auth-secret
static-auth-secret=3575819665154b268af59efedee8826e
...and the PHP (which misled me for ages):
$ttl = 24 * 3600; // Time to live
$time = time() + $ttl;
$username = $time . ':' . $user;
$password = base64_encode(hash_hmac('sha1', $username, '3575819665154b268af59efedee8826e', true));
Building upon #Mido and #HeyHeyJC answers, here is the Python implementation to build credentials for coturn.
import hashlib
import hmac
import base64
from time import time
user = 'your-arbitrary-username'
secret = 'this-is-the-secret-configured-for-coturn-server'
ttl = 24 * 3600 # Time to live
timestamp = int(time()) + ttl
username = str(timestamp) + ':' + user
dig = hmac.new(secret.encode(), username.encode(), hashlib.sha1).digest()
password = base64.b64encode(dig).decode()
print('username: %s' % username)
print('password: %s' % password)
Here is a web application to test the login to your coturn server. Use turn:host.example.com as the server name.
I came across similar issue (getting REST API working with TURN server) recently and learned that TURN server doesn't support REST API calls at all and just provides support for an authentication format with shared secret when we enable REST API support in TURN config. The draft only provides info on things that we need to consider while implementing such REST API and WE need to create the API on our own or use something like turnhttp to generate the temporary username password combo.
As #mido detailed, you can implement the username/password generation part in the application itself. But if you have reasons to separate this from the application and want to implement it as an entirely different API service, instead of implementing a complete API as per the draft, I came across another post in which the OP provided a PHP script to generate temp username & password and this one works pretty well once you modify the hash_hmac() function to the following,
$turn_password = hash_hmac('sha1', $turn_user, $secret_key, true);
We need to base64 encode the RAW output of hash_hmac to get it working and I believe this is why it was not working for the OP in that link.
You should be able to test authentication using turnutils_uclient command to verify that the temp username/password combo is working as expected.
turnutils_uclient -y -u GENERATED_USERNAME -w GENERATED_PASSWORD yourturnserver.com
Once you have verified authentication and confirmed that it's working, you can setup webserver for the PHP script to make it available to your application and fetch the temporary username/password combo. Also, you would need to implement other security setup (authentication) to protect the API from unauthorized access.
I know this is an old post, just sharing my findings here hoping that it will be useful for someone someday.
Here is my c# implementation with TTL
public string[] GenerateTurnPassword(string username)
{
long ttl = 3600 * 6;
var time = DateTimeOffset.Now.ToUnixTimeSeconds() + ttl;
var newuser = time + ":" + username;
byte[] key = Encoding.UTF8.GetBytes("YOURSECRET");
HMACSHA1 hmacsha1 = new HMACSHA1(key);
byte[] buffer = Encoding.UTF8.GetBytes(newuser);
MemoryStream stream = new MemoryStream(buffer);
var hashValue = hmacsha1.ComputeHash(stream);
string[] arr = new string[2];
arr[0] = Convert.ToBase64String(hashValue);
arr[1] = newuser;
return arr;
}
Well #Augusto Destrero provided implementation will cause TypeError: key: expected bytes or bytearray, but got 'str' on Python 3.7.6, for anyone looking for another Python implementation, here is an example:
import time
import hmac
import hashlib
import base64
secret = b'abcdefghijkmln'
def generateTurnUsernamePwd():
username = "arbitry username here"
password = hmac.new(secret, bytes(username, 'UTF-8'), hashlib.sha1).digest()
passwordStr = base64.b64encode(password).decode("utf-8")
return username,passwordStr
print(generateTurnUsernamePwd())
The main difference is key and message keyword arguments in hmac lib has to be bytes in newer version , while in older versions, it requires str.
I thought it worthwhile to add to the answer the actual text of the documentation of coturn regardingg this topic and a link to it for those interested:
--auth-secret TURN REST API flag. Flag that sets a special WebRTC authorization option that is based upon authentication secret. The
feature purpose is to support "TURN Server REST API" as described
in the TURN REST API section below. This option uses timestamp
as part of combined username: usercombo -> "timestamp:username",
turn user -> usercombo, turn password ->
base64(hmac(input_buffer = usercombo, key = shared-secret)). This
allows TURN credentials to be accounted for a specific user id. If
you don't have a suitable id, the timestamp alone can be used. This
option is just turns on secret-based authentication. The actual
value of the secret is defined either by option static-auth-secret,
or can be found in the turn_secret table in the database.
Here is an example for go with ttl:
import (
"crypto/hmac"
"crypto/sha1"
"encoding/base64"
"fmt"
"time"
)
const turnTokenTtl = time.Hour * 24
const turnSecret = "your secret"
func getTurnCredentials(name string) (string, string) {
timestamp := time.Now().Add(turnTokenTtl).Unix()
username := fmt.Sprintf("%d:%s", timestamp, name)
h := hmac.New(sha1.New, []byte(turnSecret))
h.Write([]byte(username))
credential := base64.StdEncoding.EncodeToString(h.Sum(nil))
return username, credential
}
In elastic search java api, suppose I am building the indexed data using this client
Node node = nodeBuilder().clusterName("es").node();
Client client = node.client();
IndexResponse response = client.prepareIndex("twitter", "tweet", "1")
.setSource(jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elasticsearch")
.endObject()
)
.execute()
.actionGet();
This indexed data is stored in folder XYZ/elasticsearch/data.
My question is how to retrieve this indexed data in java from some other client or some other code. Is there any way in which I can give the path and the already indexed data can be imported, then I can perform queries on it?
Edit :
The code for client on other computer
Node node = nodeBuilder().clusterName("es").node();
Client client = node.client();
MatchQueryBuilder qb = QueryBuilders.matchQuery("user", "kimchy");
SearchRequestBuilder srb = client.prepareSearch("twitter").setTypes("tweet");
SearchResponse big = srb.setQuery(qb).execute().actionGet();
SearchHit[] results = big.getHits().getHits();
This shows
search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed
Thanks
Arya
I'm not sure I understand the question, but if it's just querying in java here's an example :
MatchQueryBuilder qb = QueryBuilders.matchQuery("user", "kimchy);
SearchRequestBuilder srb = client.prepareSearch("twitter").setTypes("tweet");
srb.setQuery(qb);
SearchResponse response = srb.execute().actionGet();
//Here goes your code where you use response.getHits() that contains your items
The message you get by using my snippet usually indicates that your client doesn't manage to connect to your server. Check your server status (i.e. that you launched your server properly) and try using this.
Node node = NodeBuilder.nodeBuilder().client(true).node();
client = node.client();
I don't think you need to specify the clustername unless you have some custom configuration i'm not aware of. Also check your index name.
The examples of connecting to a K2 server in the K2 developer reference like here and here all involve setting a username and password in the connection parameters. Using this approach would mean I'd need to store a password in either plaintext or at best using two-way encryption, which is obviously not good practice.
Is there an alternative way, perhaps using a token, to establish these connections? It's hard to believe that every app using this functionality just stores a password somewhere, there are obvious security implications to that.
Note - I am not a K2 API expert, but I've come across it as a product, and do know for sure that it natively supports Windows Authentication. The same page, for which you provided the link, states the following -
connectionSetup.ConnectionParameters["Authenticate"] = "true";
connectionSetup.ConnectionParameters["Host"] = "LOCALHOST";
connectionSetup.ConnectionParameters["Integrated"] = "true";
connectionSetup.ConnectionParameters["IsPrimaryLogin"] = "true";
//connectionSetup.ConnectionParameters["Originator"] = "false";
connectionSetup.ConnectionParameters["Password"] = "{YourPassword}";
connectionSetup.ConnectionParameters["Port"] = "5252";
connectionSetup.ConnectionParameters["SecurityLabelName"] = "K2";
//connectionSetup.ConnectionParameters["SecurityPackage"] = "Kerberos,NTLM";
connectionSetup.ConnectionParameters["UserID"] = "{YourUserName}";
connectionSetup.ConnectionParameters["WindowsDomain"] = "{YourDomain}";
The third parameter here is Integrated=true and while the documentation is not explicit about it, I'd recommend to use this parameter, and try connect while ignoring the user id and password. Assuming that K2 is configured for Windows Authentication, it should work without having to provide explicit user name or password.. and certainly without need to store them yourself as a client of K2.
You may use Integrated Security. K2 will then use the same user than the one currently logged in and you do not need to pass any credential.
I had to migrate a legacy database with clear text password to a PostgresSQL database. I've looked up what's the best way to encrypt password in a database and found the pgcrypto extension with slow algorithm. (see pgcrypto documentation for 8.4)
The migration is done for data and everything is working well.
Now I have to write a CRUD application to handle this data.
I'm wondering what's the best way to use this strong encryption with grails ?
In my model, I've used the afterInsert event to handle this :
def afterInsert() {
Compte.executeUpdate("update Compte set hashpass=crypt(hashpass, gen_salt('bf', 8)) where id = (:compteId)", [compteId: this.id])
}
I guess that I should also check if the hashpass field is modified whenever the model is saved. But before that, is there another (best) way to achieve my goal ?
Edit : I cannot use the Spring Security bcrypt plugin here. The CRUD application that I'm writing use SSO CAS so I don't need such a plugin. The CRUD application manages accounts for another application that I don't own. I just need to create a new account, modify or delete an existing one. This is very simple. The tricky part is to hack grails so that it takes into account the password field and use a specific sql to store it to a postgreSQL database.
Edit 2 :
I've come up with the following code but it doesn't work
def beforeInsert() {
hashpass = encodePassword(hashpass);
}
def encodePassword(cleartextpwd) {
// create a key generator based upon the Blowfish cipher
KeyGenerator keygenerator = KeyGenerator.getInstance("Blowfish");
// create a key
SecretKey secretkey = keygenerator.generateKey();
// create a cipher based upon Blowfish
Cipher cipher = Cipher.getInstance(ALGORITHM);
// initialise cipher to with secret key
cipher.init(Cipher.ENCRYPT_MODE, secretkey);
// get the text to encrypt
String inputText = cleartextpwd;
// encrypt message
byte[] encrypted = cipher.doFinal(inputText.getBytes("UTF-8"));
return Base64.encodeBase64String(encrypted);
}
I get a hash that is not a blowfish hash (beginning with $2a$08$ )
Edit 3 :
I've finally came up with a cleaner grails solution after reading this wiki page : grails.org/Simple+Dynamic+Password+Codec (not enough reputation to put more than 2 links so add http:// before) and the bug report jira.grails.org/browse/GRAILS-3620
Following advice from #lukelazarovic, I've also used the algorithm from the spring security plugin.
Here is my password encoder to put into grails/utils :
import grails.plugin.springsecurity.authentication.encoding.BCryptPasswordEncoder;
class BlowfishCodec {
static encode(target) {
// TODO need to put the logcount = 8 in configuration file
return new BCryptPasswordEncoder(8).encodePassword(
target, null)
}
}
I've updated my Compte model to call my password encoder before saving / updating the data :
def beforeInsert() {
hashpass = hashpass.encodeAsBlowfish();
}
def beforeUpdate() {
if(isDirty('hashpass')) {
hashpass = hashpass.encodeAsBlowfish();
}
}
The tricky part is to hack grails so that it takes into account the
password field and use a specific sql to store it to a postgreSQL
database.
Is there any particular reason to do the hashing in database?
IMHO it's better to hash the password in Grails, therefore have a code that is not database-specific and easier to read.
For hashing passwords using Blowfish algorithm using Java or Groovy see Encryption with BlowFish in Java
The resulting hash begins with algorithm specification, iteration count and salt, separated with dollar sign - '$'. So the hash may look like "$2a$08$saltCharacters" where 2a is a algorithm, 08 is iteration count, then follows salt and after salt is the hash itself.
For broader explanation see http://www.techrepublic.com/blog/australian-technology/securing-passwords-with-blowfish. Don't mind that it concerns to Blowfish in PHP, the principles applies for Java or Groovy as well.