My Meteor app runs fine on localhost, but when I deploy it to myApp.meteor.com, I get the error below. I struggle to understand these error logs. Any ideas?
[Thu Jun 25 2015 06:35:23 GMT+0000 (UTC)]
WARNING /meteor/dev_bundles/0.4.18/lib/node_modules/fibers/future.js:278
throw(ex);
^
[Thu Jun 25 2015 06:35:23 GMT+0000 (UTC)]
WARNING MongoError: ns name too long, max size is 128
at Object.Future.wait
(/meteor/dev_bundles/0.4.18/lib/node_modules/fibers/future.js:398:15)
at [object Object].MongoConnection._ensureIndex
(packages/mongo/mongo_driver.js:733:1)
at FileCollection.Mongo.Collection._ensureIndex
(packages/mongo/collection.js:620:1)
at new FileCollection (packages/vsivsi:file-
collection/gridFS_server.coffee:65:15)
at FileCollection (packages/vsivsi:file-
collection/gridFS_server.coffee:21:24)
at app/lib/collections.js:15:12
at /meteor/containers/55bd5013-6244-5f59-9f74-
c6798eb58003/bundle/programs/server/boot.js:229:5
- - - - -
And the code in my collections.js file is listed below with line 15 noted in a comment. It is based on the Meteor File Collection Sample App line 10. Note that the MD5 is actually quite a long identifier, but I'm not sure how/if it's used by Mongo. Regardless, it works on localhost.
Meteor.startup(function () {
console.log('collections are loading');
myData = FileCollection('myFiles', { // This is line 15 that triggers error
resumable: true,
http: [
{
method: 'get',
// This GET routes path like /gridfs/myFiles/md5/9f4cd4e1d9e7cb1273ad2860fa909398.jpeg
path: '/md5/:md5',
// note that my :md5 also contains suffix with media type. Suffix was appended in link helper
// I added suffix so that other programs can determine MIME type from URL's suffix, without reading file header
lookup: function (params, query) {
// params.md5 contains more than the MD5 and hence needs to be split to get true MD5 portion
var tempArray = params.md5.split('.');
var firstHalf = tempArray[0]; // heres the MD5 part to ensure that Vaughn's code isn't affected
return {
md5: firstHalf // a query mapping url to myFiles
};
}
}
]
});
});
Looks like your problem is related to the closed issue here: https://github.com/vsivsi/meteor-file-collection/issues/55
Your local db name is probably much shorter than the one assigned to you in meteor.com environment.
Try adding something like
resumableIndexName: 'fci',
after line
resumable: true,
could you show me app/lib/collections.js:15 ?
Do you have a collection with really long name defined?
Related
MongoDB shell version v4.4.6
Build Info: {
"version": "4.4.6",
"gitVersion": "72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7",
"openSSLVersion": "OpenSSL 1.1.1f 31 Mar 2020",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distmod": "ubuntu2004",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}
I had mongodb crash during indexing:
{"t":{"$date":"2021-07-09T15:02:31.139+02:00"},"s":"I", "c":"STORAGE", "id":20649, "ctx":"IndexBuildsCoordinatorMongod-0","msg":"Index build failed","attr":{"buildUUID":{"uuid":{"$uuid":"5ed4dd6b-420b-46e3-8611-3fba75d9739e"}},"namespace":"testdb.articles","uuid":{"uuid":{"$uuid":"cdf839b1-a9b4-4dfb-a500-c1f80f2dea2a"}},"error":{"code":28755,"codeName":"Location28755","errmsg":"text contains invalid UTF-8"}}}
I wonder how can I actually locate this document and remove it? I am using pymongo
Is there any log file related to the buildUUID can tell me what the id of the document is? Or maybe there is some mongo command can let me detect the invalid UTF-8?
strange bson decoding error
I use following code to try to detect the problem
def check_doc(skip: int):
it_cursor = col.find().skip(skip)
count = 0
while True:
try:
doc = it_cursor.next()
print(f"{count}, {doc['_id']}")
except Exception as e:
print(f"i={count}, last doc {doc}", file=sys.stderr)
raise e
count += 1
Then I detect a problem after '_id':ObjectId('60e7a4e2a3a21f66d1895c46')
File "/home/user/.local/lib/python3.8/site-packages/pymongo/message.py", line 1615, in unpack_response
return bson._decode_all_selective(
File "/home/user/.local/lib/python3.8/site-packages/bson/__init__.py", line 1089, in _decode_all_selective
return decode_all(data, codec_options)
bson.errors.InvalidBSON: 'utf-8' codec can't decode byte 0xc2 in position 0: unexpected end of data
However, if I use following code I can actually get document and I cannot see anything abnormal there:
col.find({"_id":bson.ObjectId('60e7a4e2a3a21f66d1895c47')}).next()
Does this means the bug is actually in the driver and the document is actually ok?
I just found out change the skip value, the invalid utf8 error jumps around a lot. The documents are ok in some runs but reporting error in other runs. For example check_doc(23447800) and check_doc(23447850) both start to raise error at round 101, which cannot be true: if the document 23447901 really has problem then the second check_doc will raise error at round 51. So I guess this is a driver bug.
I try to reproduce this in mongo shell, but I cannot, following command does not cause any problem:
db.articles.find().skip(23447800).forEach(function(doc){printjson(doc);})
Another funny finding, if I impose a limit() then the error is gone. So this error only happens in pymongo iteration without limit and the text indexing. check_doc(23447800, 205) won't raise any error.
def check_doc2(skip: int, limit: int):
it_cursor = col.find().skip(skip).limit(limit)
count = 0
while True:
try:
doc = it_cursor.next()
print(f"{count}, {doc['_id']}, {doc['pmid']}")
except StopIteration:
break
except Exception as e:
print(f"i={count}, last doc {doc}", file=sys.stderr)
raise e
count += 1
after dig into the code. I found out that pymongo read document in bulky way when iterate through the cursor. Thus the check_doc() is not a good way to locate the problematic documents. When yield the current cursor, pymongo continue to read the later documents and raise error if the next bulk read has problem. The better way is to use following function with exact match to retrieve documents starting from the point where check_doc() start to give out error.
def check_doc3(start_id, count: int):
ret = []
st = int(str(start_id), 16)
for i in range(st, st+count):
try:
_id = bson.ObjectId("%x"%i)
col.find({"_id": _id}).next()
except StopIteration:
break
except Exception as e:
ret.append(i)
print(e)
return ret
I am trying MongoDB for first time but I am stuck on the following syntax error: unexpected token illegal. I checked it and it looks OK. I found similar a problem on StackOverflow but that is for a different error. What am I doing wrong?
Here is my script:
db.student.save({"_id":ObjectId(5983548781331adf45ec7),"name":"replaced","age":55})
The problem is here : ObjectId(5983548781331adf45ec7)
ObjectId accepts a string of 24 hex digits.
ObjectId("<24 hex digits here>")
E.g. ObjectId("0123456789abcdef01234567")
If you are using a backend source code such as groovy (with Grails GORM), you can try the following script which is very clean and readable:
def studentFromDB = db.student.findById("5983548781331adf45ec7")
studentFromDB.name = "replaced"
studentFromDB.age = 55
studentFromDB.save(flush: true, failOnError: true)
If you use it directly onto the mongo you should use the MongoDB update, for example:
db.student.update(
{"_id":ObjectId(5983548781331adf45ec7) },
{
name: "replaced",
age: 55
},
{ upsert: true }
)
I fixed it...the error is i missed quotes inside 'ObjectId'
the corrected script is `db.student.save({"_id":ObjectId("57fcf46763ecce707f071884"),"name":"rep_dsave","age":37}).
Thanks 4J41 and rotemy
`
I'm trying to use typeahead.js to get a list of authors from my own server (flask endpoint). When I use local source for Bloodhound everything works smoothly, only when I have to fetch the data from a server not all the results seem to be shown.
So this is the code:
var authors = new Bloodhound({
datumTokenizer: Bloodhound.tokenizers.whitespace,
queryTokenizer: Bloodhound.tokenizers.whitespace,
// local: ["Aad, Georges", "Abajyan, Tatevik", "Abbott, Brad"],
remote: {
url: '/search/authors?q=%QUERY',
transform: function (json) {
var names = $.map(json.results, function (author) {
return author.full_name;
});
console.log(names);
return names;
}
}
});
$('#author-suggest').typeahead(null, {
source: authors
});
I make sure to print the names that come back from the url and they seem to be identical to what I might put into the local variable:
$ Array [ "Aad, Georges", "Abajyan, Tatevik", "Abbott, Brad" ]
$ Array [ "Aad, Georges", "Abajyan, Tatevik", "Abbott, Brad" ]
…
Yet something strange happens when I test it. When I try to type in the input box - I get suggestions, however it's only a subset of those printed in the console. They also don't dynamically change when I type (Aad, Georges is still the first suggestion even when I'm already at "Abbott").
The other peculiar thing is that not all of the names are being suggested, even though all of them have clearly came back from the server. When the server sends 3 names, only the first 2 are in the suggestions. When it sends 4 names, 1 suggestion appears. The general pattern is:
[(1 -> 1), (2 -> 2), (3 -> 2), (4 -> 1), (5 -> 0), (6 -> 5), (7 -> 5)… ]
Where the first number in a pair represents the number of authors received and the second is the number of suggestions. What the heck is going on here?!
Thanks in advance!
Trying to learn the Meteor framework as well as coffeescript/node all at once. Been working on a simple file upload program that utilizes onloadend. When the FileReader onloadend event function is called I try to determine if the file already exists and if so I update with the new file data and version.
The code works for insert but not update. Can someone help? I've posted to meteor-talk w/o an answer as I suspect its the weekend (when I do most of my experimentation).
Code snippet...
file_reader.onloadend = ((file_event) ->
(event) ->
f_filename = escape file_event.name
version = 0
f_record = null
f_record = doc_repo.findOne { name: f_filename }
if f_record && f_record.name
doc_repo.update
name: f_filename
,
$set:
version: 10
else
doc_repo.insert
name: f_filename
data: event.target.result
version: 0
)(file_obj)
Error
Exception while invoking method '/documents/update' TypeError: Cannot read property 'toBSON' of undefined
at Function.calculateObjectSize (/usr/local/meteor/lib/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:210:12)
at BSON.calculateObjectSize (/usr/local/meteor/lib/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1463:15)
at UpdateCommand.toBinary (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/commands/update_command.js:67:20)
at Connection.write (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/connection/connection.js:138:40)
at __executeInsertCommand (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/db.js:1837:14)
at Db._executeInsertCommand (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/db.js:1912:7)
at Collection.update (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/collection.js:445:13)
at app/packages/mongo-livedata/mongo_driver.js:178:16
at Db.collection (/usr/local/meteor/lib/node_modules/mongodb/lib/mongodb/db.js:507:44)
at _Mongo._withCollection (app/packages/mongo-livedata/mongo_driver.js:51:13)
It looks like Mongo is not getting the second parameter which it needs to do the update. So in regular JavaScript it's expecting this:
collection.update({..selector..}, { .. modifier });
So I'd try putting some curly brackets around the modifier object, like this:
doc_repo.update
name: f_filename,
{
$set:
version: 10
}
lets say I have a text file with lines as such:
[4/20/11 17:07:12:875 CEST] 00000059 FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:/Prgs/testing/WebSphere/AppServer/profiles/ProcCtr01/logs/ffdc/server1_3d203d20_11.04.20_17.07.12.8755227341908890183253.txt com.test.testserver.management.cmdframework.CmdNotificationListener 134
[4/20/11 17:07:27:609 CEST] 0000005d wle E CWLLG2229E: An exception occurred in an EJB call. Error: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
com.lombardisoftware.core.TeamWorksException: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
at com.lombardisoftware.server.ejb.persistence.CommonDAO.assertNotNull(CommonDAO.java:70)
Is there anyway to easily import a data source such as this into protovis, if not what would the easiest way to parse this into a JSON format. For example for the first entry might be parsed like so:
[
{
"Date": "4/20/11 17:07:12:875 CEST",
"Status": "00000059",
"Msg": "FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I",
},
]
Thanks, David
Protovis itself doesn't offer any utilities for parsing text files, so your options are:
Use Javascript to parse the text into an object, most likely using regex.
Pre-process the text using the text-parsing language or utility of your choice, exporting a JSON file.
Which you choose depends on several factors:
Is the data somewhat static, or are you going to be running this on a new or dynamic file each time you look at it? With static data, it might be easiest to pre-process; with dynamic data, this may add an annoying extra step.
How much data do you have? Parsing a 20K text file in Javascript is totally fine; parsing a 2MB file will be really slow, and will cause the browser to hang while it's working (unless you use Workers).
If there's a lot of processing involved, would you rather put that load on the server (by using a server-side script for pre-processing) or on the client (by doing it in the browser)?
If you wanted to do this in Javascript, based on the sample you provided, you might do something like this:
// Assumes var text = 'your text';
// use the utility of your choice to load your text file into the
// variable (e.g. jQuery.get()), or just paste it in.
var lines = text.split(/[\r\n\f]+/),
// regex to match your log entry beginning
patt = /^\[(\d\d?\/\d\d?\/\d\d? \d\d:\d\d:\d\d:\d{3} [A-Z]+)\] (\d{8})/,
items = [],
currentItem;
// loop through the lines in the file
lines.forEach(function(line) {
// look for the beginning of a log entry
var initialData = line.match(patt);
if (initialData) {
// start a new item, using the captured matches
currentItem = {
Date: initialData[1],
Status: initialData[2],
Msg: line.substr(initialData[0].length + 1)
}
items.push(currentItem);
} else {
// this is a continuation of the last item
currentItem.Msg += "\n" + line;
}
});
// items now contains an array of objects with your data