Any ideas on how to upload picture files into Mongodb using Golang. I have done research and I can find lots of examples with many diiferent languages, but none for Go. Any links for blogs or tutorials would be great help. Thanks.
You may want to read some of the documentation on the mgo driver. Specifically under the gridfs section which covers how to store files in MongoDB. Examples are also available on the mgo website.
Also similar questions have been asked on stack already such as: Store Uploaded File in MongoDB GridFS Using mgo without Saving to Memory
Related
How can I store images in a MongoDB through Flutter/Dart?
We are developing an android application using MongoDB and Flutter. We want to store images during the registration of users in our app. I already referred the mongo-dart documentation but still, I can't find any solution about this.
I would suggest encoding your images to Base64 format. Then you can store them technically as plain text. There is a base64Encode function in dart:convert package.
You can also check this discussion: how to convert an image to base64 image in flutter?
MongoDB has GridFS for storing files:
A convention for storing large files in a MongoDB database. All of the official MongoDB drivers support this convention, as does the mongofiles program.
It looks like mongo-dart supports it as well although it's missing the documentation.
I've done quite a bit of searching, but haven't been able to find anything within this community that fits my problem.
I have a MongoDB collection that I would like to normalize and upload to Google Big Query. Unfortunately, I don't even know where to start with this project.
What would be the best approach to normalize the data? From there, what is recommended when it comes to loading that data to BQ?
I realize I'm not giving much detail here... but any help would be appreciated. Please let me know if I can provide any additional information.
If you're using python, easy way is to read collection chunky and use pandas' to_gbq method. Easy and quite fast to implement. But better to get more details.
Additionally to the answer provided by SirJ, you have multiple options to load data to BigQuery, including loading the data to Cloud Storage, local machine, Dataflow any more as mentioned here. Cloud Storage supports data in multiple formats such as CSV, JSON, Avro, Parquet and more. You also have various options to load data using Web UI, Command Line, API or using the Client Libraries which support C#, GO, Java, Node.JS, PHP, Python and Ruby.
i have been working on a monitoring platform using logstash elasticsearch and kibana, the idea is to stream tweets and have a real time dashboard, everything here works just fine, now i want to store the tweets on mongodb and index them with elasticsearch, i've already done the first step of storing tweets, configuring mongodb as the output of logstash, but i can't seem to find a way to index this data with elasticsearch, i've looked for so many tutorials but i could't find a solution, if someone can help i would be really gratefull
Basically you need to:
Create an elastic search index
Read your tweets from mongodb
Index the tweets in elastic search
I'm guessing you already know how to do no. 2. For no. 1 you can either use the REST API, or any number of APIs available, depending on what the rest of your app is written in or what you are comfortable with.
For no. 3 is the same thing.
I'm using the Javascript API with this package.
Say, if I want to store PDFs or ePub files using MongoDB's GridFS, is it possible to perform full-text searching on the data files?
You can't currently do real full text search within mongo: http://www.mongodb.org/display/DOCS/Full+Text+Search+in+Mongo
Feel free to vote for it here:
https://jira.mongodb.org/browse/SERVER-380
Mongo is more of a general purpose scalable data store, and as of yet it doesn't have any full text search support. Depending on your use case, you could use the standard b-tree indexes with an array of all of the words in the text, but it won't do stemming or fuzzy matches, etc.
However, I would recommend combining mongodb with a lucene-based application (elastic search is popular). You can store all of your data in mongodb (binary data, metadata, etc.), and then index the plain text of your documents in lucene. Or, if your use case is pure full text search, you might consider just using elastic search instead of mongodb.
Update (April 2013):
MongoDB 2.4 now supports a basic full-text index! Some useful resources below.
http://docs.mongodb.org/manual/applications/text-search/
http://docs.mongodb.org/manual/reference/command/text/#dbcmd.text
http://blog.mongohq.com/blog/2013/01/22/first-week-with-mongodb-2-dot-4-development-release/
Not using MongoDB APIs, not that I know of. GridFS seems to be designed to be more like a simplified file system with APIs that provides a straightforward key-value semantic. On their project ideas page they list two things that would help you if existed in production-ready state:
GridFS FUSE that would allow you to mount GridFS as a local file system and then index it like you would index stuff on your disk
Real-Time Full Text search integration with tools like Lucene and Solr. There are some projects on github and bitbucket that you might want to check out.
Also look at ElasticSearch. I have seen some integration with Mongo but I am not sure how much has been done to tap into the GridFS (GridFS attachment support is mentioned but I haven't worked with it to know for sure). Maybe you will be the one to build it and then opensource it? should be a fun adventure
I have a lots of documents stored on Amazon S3. My questions are:
Does Amazon provide any services/APIs using which I can index the contents of the document and search them (full text indexing and searching)?
If it does could someone please point me to any link in the documentation.
If it does not then could this be achieved with Lucene and Zend Framework? Have any one of you implemented this? Can I get some pointers?
UPDATE: I do not intend to save my index on Amazon S3 rather I am looking forward to indexing the contents of the documents on S3 and serving them based on a search.
You can see this question, or this blog post if you want to do pure lucene, or you can use Solr, which is probably easier. See also this post.
Zend has a PHP port of Lucene, which ties in very well. You can look at the Zend documentation for how to use it.