I am trying to convert documents using the Bluemix Document Conversion service with a Node.js application. I am getting nothing but errors in my app, but the test document I'm using converts fine using the demo page. Below is a minimal app that demonstrates the problem (Note that, while this app is converting a PDF from disk, the "real" app can't do that, hence the Buffer object).
'use strict';
var fs = require('fs');
var DocumentConversionV1 = require('watson-developer-cloud/document-conversion/v1');
var bluemix=require('./my_bluemix');
var extend=require('util')._extend; //Node.js' built-in object extend function
var dcCredentials = extend({
url: '<url>',
version: 'v1',
username: '<username>',
password: '<password>'
}, bluemix.getServiceCreds('document_conversion')); // VCAP_SERVICES
var document_conversion = new DocumentConversionV1(dcCredentials);
var contents = fs.readFileSync('./testdoc.pdf', 'utf8');
var parms={
file: new Buffer(contents,'utf8'),
conversion_target: 'ANSWER_UNITS', // (JSON) ANSWER_UNITS, NORMALIZED_HTML, or NORMALIZED_TEXT
content_type:'application/pdf',
contentType:'application/pdf', //don't know which of these two works, seems to be inconsistent so I include both
html_to_answer_units: {selectors: [ 'h1', 'h2','h3', 'h4']},
};
console.log('First 100 chars of file:\n******************\n'+contents.substr(0,100)+'\n******************\n');
document_conversion.convert(parms, function(err,answerUnits)
{
if (!err)
console.log('Returned '+answerUnits.length);
else
console.log('Error: '+JSON.stringify(err));
});
The results from running this program against the test PDF (782K) is:
$ node test.js
[DocumentConversion] WARNING: No version_date specified. Using a (possibly old) default. e.g. watson.document_conversion({ version_date: "2015-12-15" })
[DocumentConversion] WARNING: No version_date specified. Using a (possibly old) default. e.g. watson.document_conversion({ version_date: "2015-12-15" })
First 100 chars of file:
******************
%PDF-1.5
%����
1 0 obj
<</Type/Catalog/Pages 2 0 R/Lang(en-US) /StructTreeRoot 105 0 R/MarkInfo<<
******************
Error: {"code":400,"error":"Could not push back 82801 bytes in order to reparse stream. Try increasing push back buffer using system property org.apache.pdfbox.baseParser.pushBackSize"}
$
Can someone tell me
How to get rid of the warning messages
Why the document is not getting converted
How do I "increase the push back buffer"
Other documents give different errors, but I'm hoping if I can make this one work then the other errors will go away too.
You can get rid of the warning message by specifying a version date in your configuration. See the tests for an example. 1
If the document converted through the demo but failed to convert when using your application, it is likely an error with how the binary data is passed to the service. (For example, it's getting corrupted or truncated.) You can see the Node.js source code for the demo here 2. It may help you figure out the mistake or give you a different approach to loading/sending the file.
That is an error from one of the underlying libraries used by the service. Unfortunately, it's not something that a caller can adjust at this point.
Related
I love testing-library, have used it a lot in a React project, and I'm trying to use it in an Angular project now - but I've always struggled with the enormous error output, including the HTML text of the render. Not only is this not usually helpful (I couldn't find an element, here's the HTML where it isn't); but it gets truncated, often before the interesting line if you're running in debug mode.
I simply added it as a library alongside the standard Angular Karma+Jasmine setup.
I'm sure you could say the components I'm testing are too large if the HTML output causes my console window to spool for ages, but I have a lot of integration tests in Protractor, and they are SO SLOW :(.
I would say the best solution would be to use the configure method and pass a custom function for getElementError which does what you want.
You can read about configuration here: https://testing-library.com/docs/dom-testing-library/api-configuration
An example of this might look like:
configure({
getElementError: (message: string, container) => {
const error = new Error(message);
error.name = 'TestingLibraryElementError';
error.stack = null;
return error;
},
});
You can then put this in any single test file or use Jest's setupFiles or setupFilesAfterEnv config options to have it run globally.
I am assuming you running jest with rtl in your project.
I personally wouldn't turn it off as it's there to help us, but everyone has a way so if you have your reasons, then fair enough.
1. If you want to disable errors for a specific test, you can mock the console.error.
it('disable error example', () => {
const errorObject = console.error; //store the state of the object
console.error = jest.fn(); // mock the object
// code
//assertion (expect)
console.error = errorObject; // assign it back so you can use it in the next test
});
2. If you want to silence it for all the test, you could use the jest --silent CLI option. Check the docs
The above might even disable the DOM printing that is done by rtl, I am not sure as I haven't tried this, but if you look at the docs I linked, it says
"Prevent tests from printing messages through the console."
Now you almost certainly have everything disabled except the DOM recommendations if the above doesn't work. On that case you might look into react-testing-library's source code and find out what is used for those print statements. Is it a console.log? is it a console.warn? When you got that, just mock it out like option 1 above.
UPDATE
After some digging, I found out that all testing-library DOM printing is built on prettyDOM();
While prettyDOM() can't be disabled you can limit the number of lines to 0, and that would just give you the error message and three dots ... below the message.
Here is an example printout, I messed around with:
TestingLibraryElementError: Unable to find an element with the text: Hello ther. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.
...
All you need to do is to pass in an environment variable before executing your test suite, so for example with an npm script it would look like:
DEBUG_PRINT_LIMIT=0 npm run test
Here is the doc
UPDATE 2:
As per the OP's FR on github this can also be achieved without injecting in a global variable to limit the PrettyDOM line output (in case if it's used elsewhere). The getElementError config option need to be changed:
dom-testing-library/src/config.js
// called when getBy* queries fail. (message, container) => Error
getElementError(message, container) {
const error = new Error(
[message, prettyDOM(container)].filter(Boolean).join('\n\n'),
)
error.name = 'TestingLibraryElementError'
return error
},
The callstack can also be removed
You can change how the message is built by setting the DOM testing library message building function with config. In my Angular project I added this to test.js:
configure({
getElementError: (message: string, container) => {
const error = new Error(message);
error.name = 'TestingLibraryElementError';
error.stack = null;
return error;
},
});
This was answered here: https://github.com/testing-library/dom-testing-library/issues/773 by https://github.com/wyze.
Actually I want to get user input from user whoever runs the script. I do not want to hardcode the testdata path in the script. for example when I run a script to test angularjs using protractor and javascript. User should be able to give path of the testdata, so that I can use that variable inside the script.
You can do this by passing in a params.testData value from the command line.
protractor conf.js --params.testData=D:\path\to\testdata.xlsx
Then in your test you will reference it using the global browser.params object. You will also need to use fs to read the file and process the data. Honestly, it would probably be easier if you created a .json file for the test data instead of an .xlsx but it looks there are libraries out there to help you parse an xlsx document already if you have to stick with that. Check this answer for some examples.
This code will not work as is but the basic idea will be something like this:
before(() => {
const testDataPath = browser.params.testData;
fs.readFile(testDataPath, (err, data) => {
if(err) { // fail? };
const testData = data;
// do some other stuff with test data ...
});
}
You are going to need to do some additional processing of the data from the .xlsx file to get it in the correct format but this should hopefully help get you on the right path.
I get this code in my Meteor project, in a client/main.js file
Template.panel.onCreated(function loginOnCreated() {
var profile = Session.get('profile');
this.myvar = new ReactiveVar(User.find({}).fetch());
});
And the result of User.find({}) is empty. If I Query this anywhere else (including meteor mongo) I get an Array of users.
So I wonder if it is a problem with the fact that this code is running in client side. In this same file I get this query working in other places, but probably in the server context.
How can I populate this ReactiveVar with the Mongo result as soon as the Template/page is loaded?
If I do something like in Meteor.startup() at Server side:
console.log(User.find({}).count());
It gives me the correct number of Users. Immediately.
#edit
If I just add a setTimeout of a few seconds (it can't be jsut 1 second, it needs a longet time), it works in this very same place.
Template.panel.onCreated(function loginOnCreated() {
//...
setTimeout(function(){
template.timeline.set(User.find({}).fetch());
console.log(timeline)
},3000);
});
So, anyone knows why it takes so long to allow me to do this operation? Any workaround?
User.find({}).fetch() will give list of users on server side only.
You can probably write a meteor method for fetching the user list on server side and give it call using meteor.call.
In the callback function to this call you can assign the result to desired variable.
I want to open a new tab with a gloda conversation from inside calendar code.
I receive an error from error console:
window not defined (or document not defined), depending on which of the two I use to Access tabmail:
let tabmail = window.document.getElementById("tabmail");
let tabmail = document.getElementById("tabmail");
The code works fine if the js file is included in an overlay xul-file.
But I want to use it outside of xul in my code.
Somewhere in my calendar code (in my 'addevent'), the same code throws the error.
This code is originally called from a rightclick on an email, but several layers deep into calendar code.
In MDN, I read that window is global? So what do I Need to do to add an tab?
This part works if tabmail is properly referenced:
tabmail.openTab("glodaList", {
collection: queryCollection,
message: aCollection.items[0],
title: tabTitle,
background: false
});
So how do I get a reference for tabmail?
Any help is appreciated.
after trying and looking through code for really some time before posting, it took only ca. 20 minutes to accidentally find the solution after submitting the question..
While browsing mailutils on mxr for something else, I found the solution in some function:
mail3PaneWindow = Services.wm.getMostRecentWindow("mail:3pane");
if (mail3PaneWindow) var tabmail = mail3PaneWindow.document.getElementById("tabmail");
I am mimicking the code from John Papa's outstanding Pluralsight course on Gulp.
When I use the code as shown in John's course:
.pipe(jsFilter)
.pipe($.uglify())
.pipe(jsFilter.restore())
I get an error on the 3rd line of code:
TypeError: Object #<StreamFilter> has no method 'restore'
When I use the code as shown in the readme from gulp-filter
.pipe(jsFilter)
.pipe($.uglify())
.pipe(jsFilter.restore)
I get an error that it can't pipe to undefined.
Based on what I can find online, both of these patterns are working for others. Any clues as to why this might be happening?
Here is the whole task, if that helps and the console logging indicates that everything if fine until the filter restore call.
Here is the entire task if that helps:
gulp.task('build-dist', ['inject', 'templatecache'], function() {
log('Building the distribution files in the /dist folder');
var assets = $.useref.assets({searchPath: './'});
var templateCache = config.temp + config.templateCache.file;
var jsFilter = $.filter('**/*.js');
return gulp
.src(config.index)
.pipe($.plumber({errorHandler: onError}))
.pipe($.inject(gulp.src(templateCache, {read: false}), {
starttag: '<!-- inject:templates:js -->'
}))
.pipe(assets)
.pipe(jsFilter)
.pipe($.uglify())
.pipe(jsFilter.restore())
.pipe(assets.restore())
.pipe($.useref())
.pipe(gulp.dest(config.dist));
});
The way restore works has changed between the 2.x and 3.x release of gulp-filter.
It seems you're using the 3.x branch, so in order to use restore you'll have to set the restore option to true when defining the filter:
var jsFilter = $.filter('**/*.js', {restore: true});
Then you'll be able to do
.pipe(jsFilter.restore)
For more information, check out this section of the documentation for the latest version of gulp-filter:
https://github.com/sindresorhus/gulp-filter/tree/v3.0.1#restoring-filtered-files