Python (Flask) MongoDB Speed Issue - mongodb

I have a big speed problem on my website using Flask/MongoDB as backend. A basic request (get 1 user for example) takes about 4 sec to respond.
Here is the python code :
#users_apis.route('/profile/<string:user_id>',methods= ['GET','PUT','DELETE'])
#auth_token_required
def profile(user_id):
if request.method == "GET":
avatar = ''
if user_id == str(current_user.id):
if(current_user.birthday):
age = (date.today().year - current_user.birthday.year)
else:
age = ''
return make_response(jsonify({
"id" : str(current_user.id),
"username" : current_user.username,
"email" : current_user.email,
"first_name": current_user.first_name,
"last_name" : current_user.last_name,
"age" : age,
"birthday" : current_user.birthday,
"gender" : current_user.gender,
"city" : current_user.city,
"country" : current_user.country,
"languages" : current_user.languages,
"description" : current_user.description,
"phone_number" : current_user.phone_number,
"countries_visited" : current_user.countries_visited,
"countries_to_visit" : current_user.countries_to_visit,
"zip_code" : str(current_user.zip_code),
"address" : current_user.address,
"pictures" : current_user.pictures,
"avatar" : "",
"interests" : current_user.interests,
"messages" : current_user.messages,
"invitations" : current_user.invitations,
"events" : current_user.events
}), 200)
And my mongodb database is build like this :
The selected user is nearly empty (has no friends, no events, no pictures...).
class BaseUser(db.Document, UserMixin):
username = db.StringField(max_length=64, unique=True, required=True)
email = db.EmailField(unique=True, required=True)
password = db.StringField(max_length=255, required=True)
active = db.BooleanField(default=True)
joined_on = db.DateTimeField(default=datetime.now())
roles = db.ListField(db.ReferenceField(Role), default=[])
class User(BaseUser)
# Identity
first_name = db.StringField(max_length=255)
last_name = db.StringField(max_length=255)
birthday = db.DateTimeField()
gender = db.StringField(max_length=1,choices=GENDER,default='N')
# Coordinates
address = db.StringField(max_length=255)
zip_code = db.IntField()
city = db.StringField(max_length=64)
region = db.StringField(max_length=64)
country = db.StringField(max_length=32)
phone_number = db.StringField(max_length=18)
# Community
description = db.StringField(max_length=1000)
activities = db.StringField(max_length=1000)
languages = db.ListField(db.StringField(max_length=32))
countries_visited = db.ListField(db.StringField(max_length=32))
countries_to_visit = db.ListField(db.StringField(max_length=32))
interests = db.ListField(db.ReferenceField('Tags'))
friends = db.ListField(db.ReferenceField('User'))
friend_requests = db.ListField(db.ReferenceField('User'))
pictures = db.ListField(db.ReferenceField('Picture'))
events = db.ListField(db.ReferenceField('Event'))
messages = db.ListField(db.ReferenceField('PrivateMessage'))
invitations = db.ListField(db.ReferenceField('Invitation'))
email_validated = db.BooleanField(default=False)
validation_date = db.DateTimeField()
I have a debian serveur with 6Go Ram and 1 vcore, 2,4GHz.
When I check the log for the mongoDB I don't see request that takes more then 378ms (for a search request)
If I use TOP during a request on my server:
I see for 1 sec a 97% CPU use for Python during the request.
When I check the python server output :
I see 4 second between the Option request and the Get Request.

I finally managed to "fix" my issue.
It seems all the problem was due to the #auth_token_required.
Each request done by the front end to the back end with the "headers.append('Authentication-Token',currentUser.token);" created a huge delay.
I replaced #auth_token_required by #login_required.
I m now using cookies.
Hope it helps someone.

Related

Extract first part of URL

I have an URL like so:
https://www.example.com/oauth/connect/token
I want to get only https://www.example.com and I have tried a bunch of ways to get this but all the ways I have tried require me to hardcode multiple /s.
Example:
$Url.Split('/')[0] + "//" + $url.split('/')[2]
Is there a way to do this without using the harcoding?
You may use [System.Uri], to 'cast' the string into an System.Uri class.
$url = [System.Uri]"https://www.example.com/oauth/connect/token"
You can then use the host and scheme to get the part of the URL you want.
$result = $url.Scheme, $url.Host -join "://"
You could also remove the AbsolutePath from the entire URL.
$url = [System.Uri]"https://www.example.com/oauth/connect/token"
$result = $url.AbsoluteUri.Replace($url.AbsolutePath, "")
This is the complete list of attributes that the System.Uri instance will have:
AbsolutePath : /oauth/connect/token
AbsoluteUri : https://www.example.com/oauth/connect/token
LocalPath : /oauth/connect/token
Authority : www.example.com
HostNameType : Dns
IsDefaultPort : True
IsFile : False
IsLoopback : False
PathAndQuery : /oauth/connect/token
Segments : {/, oauth/, connect/, token}
IsUnc : False
Host : www.example.com
Port : 443
Query :
Fragment :
Scheme : https
OriginalString : https://www.example.com/oauth/connect/token
DnsSafeHost : www.example.com
IdnHost : www.example.com
IsAbsoluteUri : True
UserEscaped : False
UserInfo :

Values for an attribute not assigned to the right variable

Sorry for the vague title I wasn't quite sure how to word it:
Let's say we have a class and it contains dictionaries embedded in a larger dictionary.
class AnObject(object):
grail = {'Grail' : '', 'Quest' : ''}
spam = {'More spam' : '', 'Less spam' : ''}
parrot = {'More parrot' : '', 'Less parrot' : '', 'Grail' : grail}
egg = {'Spam' : spam, 'Parrot' : parrot }
Then we want to call these attributes by other names
self.egg = egg
self.parrot = egg['Parrot']
self.moreparrot = self.parrot['More parrot']
This will bring up the right locations but for some reason I'm finding...
>>>knight = AnObject()
>>>knight.moreparrot = x
>>>knight.moreparrot
x
>>>knight.egg
{'Parrot' : {'More parrot' : '', 'Less parrot' : ''}...}
But this works fine:
>>>knight.egg['Parrot']['More parrot'] = x
>>>knight.egg['Parrot']['More parrot']
x
>>>knight.egg
{'Parrot' : {'More parrot' : '', 'Less parrot' : ''}...}
These should point to the same variable, but I am getting different results. How come?
Edit:
This might be totally accidental but for some reason Spam is consistent.
self.egg = egg
self.spam = egg['Spam']
self.morespam = self.spam['More spam']
>>>knight = AnObject()
>>>knight.morespam = x
>>>knight.morespam
x
>>>knight.egg['Spam']['More spam']
x
>>>knight.egg
{'Spam' : {'More spam' : 'x', 'Less spam' : ''}...}

How to get email address from the emails inside an Oulook folder via PowerShell?

I have an Outlook folder, let's call it LoremIpsum, where I have more than 1000+ email drafts that I want to enumarate and do some filtering via PowerShell. I can access the folder and see the emails already, using this script:
Function HandleRemaining {
[CmdletBinding()]
Param()
BEGIN {
Clear-Host
}
PROCESS {
$outlook = New-Object -ComObject outlook.application
$mapi = $outlook.getnamespace("MAPI");
$email = $mapi.Folders.Item(1).Folders.Item('LoremIpsum').Items(1)
foreach ($recip in $email.Recipients) {
$recip
}
$email.To
$email.CC
}
END {
}
}
HandleRemaining
The problem is that neither $recip nor $email.To return the email address of the To or CC of that email, instead I get the person's resolved name, example:
Application : Microsoft.Office.Interop.Outlook.ApplicationClass
Class : 4
Session : Microsoft.Office.Interop.Outlook.NameSpaceClass
Parent : System.__ComObject
Address : /o=ExchangeLabs/ou=Exchange Administrative Group (ALPHA-NUMERIC)/cn=Recipients/cn=LONG-ALPHANUMERIC-HERE
AddressEntry : System.__ComObject
AutoResponse :
DisplayType : 0
EntryID : <snip>
Index : 1
MeetingResponseStatus : 0
Name : John Walker
Resolved : True
TrackingStatus : 0
TrackingStatusTime : 01-Jan-01 00:00:00
Type : 1
PropertyAccessor : System.__ComObject
Sendable : True
John Walker
I changed the numbers and codes to preserve privacy, but that's the return I get. So, how can I get the proper emaill address of the recipients of a given email draft?
I think you need to use the PropertyAccessor.
$PR_SMTP_ADDRESS = "http://schemas.microsoft.com/mapi/proptag/0x39FE001E"
$smtpAddress = $recip.PropertyAccessor.GetProperty($PR_SMTP_ADDRESS)
See here (Warning! VBA): https://msdn.microsoft.com/en-us/VBA/Outlook-VBA/articles/obtain-the-e-mail-address-of-a-recipient

storage.objects().compose() not working

I am using the GCS JSON API via Java. My code to insert objects, delete objects, and copy objects all works great. But for some reason I cannot get storage.objects().compose() to work. No matter what I get a 400 or 500 error. Even when I go to use the "Try it now" feature for compose on the Google website I get the same error. So there must be something basic I am missing.
Here is my code:
StorageObject metadata = new StorageObject()
.setMetadata( ImmutableMap.of("OriginalFileName", originalFileName) )
.setContentType(contentType)
.setAcl( ImmutableList.of( new ObjectAccessControl().setEntity("allUsers").setRole("READER") ) );
// list of files to concatenate
List<SourceObjects> sourceObjects = new ArrayList<SourceObjects>();
for (int i = 0; i <= chunkNumber; i++) {
sourceObjects.add( new SourceObjects().setName(objectName + ".chunk" + i) );
}
ComposeRequest composeReq = new ComposeRequest()
.setSourceObjects(sourceObjects)
.setDestination(metadata);
storage.objects().compose(bucketName, objectName, composeReq).execute();
And here is the error I am getting:
500 { "code" : 500, "errors" : [
{ "domain" : "global", "message" : "Backend Error", "reason" : "backendError" }
], "message" : "Backend Error" }

Populate a nested ObjectRef

So here's my model:
CompetitionSchema = new Mongoose.Schema
name :
type : String
required : true
required_teams :
type : Number
teams : [
_team :
type : Mongoose.Schema.ObjectId
ref : 'teams'
weight :
type : Number
min : 0
]
I would like to be able to populate a competition object in order to access competition.teams[0]._team._id for which I have tried the following:
models('Competition')
.findById id
.populate('teams._team')
.exec (error, competition) ->
if error || !competition
error_callback error
else
success_callback competition
However this has no effect. I've also tried:
models('Competition')
.findById id
.exec (error, competition) ->
if error || !competition
error_callback error
else
options = [
path : 'teams._team'
model : 'teams'
]
models('Competition')
.populate(
competition
, options
, (error, competition) ->
if error || !competition
error_callback competition
else
success_callback competition
)
Also to no effect. I find the API documentation for Model.populate to be quite confusing, so please excuse me if it's plainly obvious!
OK after soaking myself in the topic a bit more I finally came to the below:
models('Competition')
.findById id
.exec (error, competition) ->
if error || !competition
error_callback error
else
options = [
path : '_team'
model : 'teams'
]
models('Team')
.populate(
competition.teams
, '_team'
, (error, competition_teams) ->
if error || !competition_teams
error_callback competition
else
success_callback competition
)
It boils down to: Instead of calling
Competition.populate(competition, { path : 'teams._team' } , callback);
I am now calling:
Team.populate(competition.teams, '_team', callback);
This has the benefit of modifying the competition.teams array rather than returning a copy, so the original competition object receives the changes too!