The supported list of transformations in IBM's ETL service DataConnect in Bluemix Cloud are these ones here: https://console.ng.bluemix.net/docs/services/dataworks1/using_operations.html#concept_h4k_5tf_xw
I have looked and looked but with no luck, what if I want to transform some of my data with an operation that is not included here? For example run custom code in a column and get some specific output?
Data Connect does not currently support refine operations outside of those provided with the service. We are adding new features and functionality weekly, but if you have a specific operation in mind, please let us know.
I will find out for you if we have the ability to execute custom code on our roadmap.
Regards,
Wesley - IBM Bluemix Data Connect Engineering
As Wes mentions above in the short term we will continue to add new data preparation and transformation capabilities to the service. Currently there is no extensibility that allows you to code new transformations.
In the longer term we are considering allowing users to edit/extend pipelines using languages like Scala and Python. We don't have a defined date for these new capabilities.
Regards,
Hernando Borda
IBM Bluemix Data Connect Product Manager
Related
I am a new bee to IBM Watson. I went through videos to create virtual assistant/chatbot where we could define intents/entities and answer accordingly. This seems fine when I have limited number of intents/entities. But say, I have a eBook and I want to train Watson to answer from this eBook. How do I achieve this. Anyone high level approach or direction will be really helpful.
There are different approaches.
You could use the integrated search skill which provides a link to Watson Discovery. You would upload your eBook to Watson Discovery and kind of index it.
Another approach is to use a database or something else as backend. Based on the input which identifies the search term and scopes which eBook to search, the answer would be retrieved from the backend database. This tutorial features a Db2 database and Watson Assistant retrieves the answer from the database. A similar approach is taken in this sample which shows how to retrieve excerpts from Wikipedia.
Reading the documentation at https://github.com/watson-developer-cloud/conversation-enhanced#local
it states "Ensure that you have a Bluemix account. While you can do part of this deployment locally, you must still use Bluemix.". If I use a custom dataset will any of this data be sent to IBM's servers ?
Specifically I think im referring to step circled red ('Create or import training data') :
I plan to build a similar app to conversation-enhanced (https://github.com/watson-developer-cloud/conversation-enhanced) but want to ensure local data will not be sent to IBM.
Yes, you will need to send data to IBM.
If you are planning on using the IBM BlueMix services, then I recommend you read the terms of use, found here, to understand your full agreement:
http://www-03.ibm.com/software/sla/sladb.nsf/pdf/6606-08/$file/i126-6606-08_05-2016_en_US.pdf
I have just started to learn IBM Watson services. I need to use Question and answer API of bluemix in java using REST API. But I couldn't find any service like Question and answer. Please can anybody tell me is the name is changed or where can I find the documentation for this service. I have tried with existing answers in SO. But those links which are in answers are not working removed.
Regards
The QA service has been discontinued since the end of last year.
Instead what has happened is that the components that made up QA have been broken down into their own services.
NL Classifier
Dialog
Retrieve and Rank
Document Conversion
The reason for this change is that not all types of applications required all these services. Also it allows you to pull in other services to act on data, for example Sentiment Analysis of a question can impact the answer/action given. Which wasn't part of QA.
More details here:
https://developer.ibm.com/watson/blog/2015/11/11/watson-question-and-answer-service-to-be-withdrawn/
I would like to use Bluemix to create an IBM Watson search engine (i.e. similar to a Google Search Engine interface) that will query either the internet (websites I specify) or online database and provide summaries of unstructured data, identify concepts, etc.
Are there any existing apps like this available or does anyone know how this can be setup with Bluemix or another platform?
You should take a look at the Alchemy API service on Bluemix.
It allows you to do things like extract entities and keywords.
Most of the APIs allow you to feed them html, text or web-based content. Stringing a bunch of these together and tagging content in a database such as Elasticsearch should allow you to achieve what you're after.
Hard to be too specific given the fairly broad nature of your question.
I am a h/w engineer interested in using Bluemix for an IOT application. Other than C, I do not know any programming language but I am willing to learn whatever necessary. My application is as follows:
My sensor nodes would upload data to an existing h/w server that has the capability to upload the data to an external SQL server. I want to analyze this data on the SQL server on a periodic basis and generate reports that I can publish to a mobile application or even a web-page to begin with.
Questions:
Is it possible to implement the "SQL server --> Data analysis --> Report generation + data visualization --> HTML(?) Publish" flow on Bluemix?
What modern/efficient languages can I learn in order to do this with the least effort?
Is there a standard implementation/example that I can use as reference for the flow described above?
This question actually has little to do with IoT--that just happens to be the source of the data--and focuses on how to process data for analysis, report generation, and publishing. You can do this mostly using services in Bluemix such that there's little if any code to write and so the programming language of the runtime may not matter.
First, to store the data, you could use SQL Database or dashDB. The former is "just" a database, whereas the latter includes R and R-Studio for data analysis. Second, for report generation, you can use Embeddable Reporting, which has Cognos (e.g. IBM Cognos Business Intelligence reports) built in.
The way Cloud Foundry in Bluemix works, you'll need to create a runtime with some language, then bind the service instances to it so you can use them. But you may not have any code to write, in which case the language doesn't matter. In case you do need to write some code, choose whichever language you think you can learn most easily. Java programmers prefer that, but it requires compiling; they may also prefer Go. You'll probably have an easier time with Node.js and PHP, which are popular interpreted languages.
A couple of resources for further info:
"Embed rich reports in your applications" shows how to use Embeddable Reporting with dashDB.
"Leverage IBM Cognos on IBM Bluemix using the Embeddable Reporting service" shows how to use Embeddable Reporting with SQL Database.
"Embed Reports and visualize Data in your Bluemix Applications" gives an overview of both approaches.
BTW, Bluemix also has a neat service called Internet of Things, which helps connect your Bluemix app to lots of things all over the Internet. Sounds like you already have this handled for this example, but as you continue to use Bluemix for IoT applications, you might want to look into this service too. The Internet of Things Foundation Starter helps you get started using Node.js, Cloudant, and Node-RED.