Currently I am new to Moses and have trained a few sample data set provided on websites.
I am looking for more data sets to train the system.
Are these available online?
What should I be looking at while searching on google?
You can find several corpora at: http://opus.lingfil.uu.se
Also, some open-source applications include their bilingual PO files, but you have to check the license.
My advice is to build a vertical (i.e. domain-specific) MT system, rather than a generic one, to get better results. So this decision will affect which corpora you choose.
I hope this helps!
Related
I am very new in this field and I would like to create a Neural Network to classify a dataset that I have in MongoDB. I would like some advise about where should I start, what technology should I use or any tutorial that you think it can help.
If you know about any open source code that already does this, I would love to take a look at it.
Thank you !!
Pick a platform
In essence, you should pick a platform or framework that does much of the dirty work for you and read up on some tutorials for that.
The big choice is between natural language processing frameworks such as NLTK or spaCy or Stanford NLP tools; or a generic machine learning framework such as Tensorflow or PyTorch.
Text classification is a popular task that's reasonably entry-level, is well supported by pretty much everything (so it's not much to say there in a shopping question, pick whatever you like) and would have a bunch of tutorials available online for any major platform.
Are there any tools to monitor network's training in PyTorch? Like tensorboard in tensorflow.
PyTorch 1.1.0 supports TensorBoard natively with torch.utils.tensorboard. The API is very similar to tensorboardX. See the documentation for more details.
I am using tensorboardX. It supports most (if not all) of the features of TensorBoard. I am using the Scalar, Images, Distributions, Histograms and Text. I haven't tried the rest, like audio and graph, but the repo also contains examples for those use cases. The installation can be done easily with pip. It's all explained in the README file of the repo.
There are also other github repos which implement a wrapper for PyTorch (and other languages/frameworks) to tensorboard. As far as I know they support fewer functionalities. But have a look at:
Crayon
Tensorboard-Logger
I have asked this question before in the forums. Tensorboard seems very convenient for Tensorflow and it is also made part of the library/framework itself. However, PyTorch wouldn't take the same approach. But there is a library called visdom here that is released by Facebook, that helps you log the training information. This gives you the flexibility of logging information the way you want. While this means a lot of flexibility, it also means you need to write some extra code to make things work.
Following up on blckbird's answer, I'm also a big fan of Tensorboard-PyTorch. However I also found that its API is relatively low level and I was writing a lot of similar code over and over to do the logging. So (shameless plug) I've written a small package on top of it to automate monitoring network training experiments with minimal code. Hopefully someone else finds it helpful. pytorch-monitor
Minetorch helps me a lot at the past 2 Kaggle competitions. I think it's ready for others to use. It has built-in tensorboard or matplotlib supported. And many other features which make the work easy, includes:
Logger
Tensorboard supported
Matplotlib (to generate png to file)
Auto resume training
Auto best model saving
Hook points for customize
...
It's still in developing so any issues or PRs are very welcomed : )
Are there any tools or algorithm in Matlab or OpenCv, which will take multiple images of any object as input (from different location around the object) and produce the 3D coordinate of the object in the world.
Like Naveh said, in OpenCV the building blocks are there, but putting it together is something you would have to do.
That being said, people have generated a number of SfM tools in both C++ and Matlab. Depending on your goals there are a number of prepackaged things you can look at:
-There is a SfM Matlab Toolbox here, I have not personally used it but I've seen it a number of times.
-If you are just looking for a black-box solution, check out Visual SfM, it is a GUI-fied version of a common SfM workflow.
-A while ago I put together a guide for installing the Visual SfM components individually on Fedora, if you wanted to dig into them. I'm not sure how relevant it is now but it might help.
Regardless, you should certainly educate yourself on the processes involved in creating 3D structure from imagery. It is a complicated process with many details which need to be understood.
What you are asking for is a fully fledged structure from motion algorithm. I don't think such a thing exists in MATLAB or OpenCV right off the shelf. However, the building blocks required for such an algorithm are there.
I suggest you do some background reading to better understand what specific algorithm will suit your needs. A good place to start is in Richard Szeliski's textbook, chapter 7. A free draft is available here. This book is recommended both in general as a good computer vision textbook, and specifically as well for your question, in which Szeliski himself is quite an expert.
I have experience with Simulink and CANbus interfaces for both simulation and code generation... but I really like open source. For quite awhile Octave has qualified as a MATLAB replacement (at my usage level) but I just recently found out about Modelica. I have yet to find any information about any blocksets (what term does Modelica tend to use?) for CANbus other than the broken link for Exite from Extessy.
Can anyone provide personal experience or a reference to information on using Modelica with CANbus? I know that I could write my own blockset, but it seems like the sort of thing someone else would already have done.
The best reference I could find on this topic was this paper. It was apparently developed as part of the EuroSysLib project. I do not know if it is publicly available anywhere. I would suggest you contact the authors.
another option for simulation of entire ECUs, including CAN is described here:
http://qtronic.de/en/index_news_12_6_ATZ.html
See paper "Building Virtual ECUs Quickly and Economically" in the June 2012 issue of ATZ electronic. Use Modelica to build vehicle simulation models for export as FMU,
and the Silver Basic Software (SBS) to configure CAN emulation based on DBC files,
and run both parts closed-loop in Silver.
We need to visualize BP (business process) into BPMN, but NOT by hands using modeler. We need to do it automatically in crm-web-based system written on PHP. I have input data (etc. array, xml, not care...(but not BPEL)), then I need to process it into nice BPMN graph (using SVG).
We have first nice-looking realization of it. We use matrix to draw: several times goes through matrix and optimize graph each time, no no, it working fast, but it not agile, hard to rebuilt, upgrade, add new features... We made this algorithm by ourselves (I mean we didn't find it in google or books). Problem is that we couldn't find any algorithms in the internet. I suppose we don't know correct keywords to do it. Every try returned us to BPEL vis. from BPMN, "Data flow vis." returned modelers...
Please help us to find some algorithms, or give correct keywords to find out information.
Think you're probably looking for "graph layout algorithms". The only library I'm aware of that can (I think) generate BPMN directly is the yFiles library from yWorks. It's not free. They do however offer a free application using the library that does auto-layout. Perhaps you could do some prototyping with that.
If that's not applicable, there are several other options. I'm not aware any of these can generate BPMN symbols directly; you'd have to construct the symbols. However all will auto-layout graphs according to various algorithms. Also all open source/free.
graphviz. Written in C. Quite old now but well used, stable and scalable.
tulip. Newer than graphviz. Haven't used it but heard good things about flexibility and scalability.
see also this post for javascript based options.
There are many more, just google for graph layout algorithms / libraries.
hth.